sidekiq-unique-jobs 8.0.6 → 8.0.8

Sign up to get free protection for your applications and to get access to all the features.
Files changed (31) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +52 -4
  3. data/README.md +1 -3
  4. data/lib/sidekiq_unique_jobs/batch_delete.rb +1 -1
  5. data/lib/sidekiq_unique_jobs/config.rb +4 -4
  6. data/lib/sidekiq_unique_jobs/constants.rb +1 -0
  7. data/lib/sidekiq_unique_jobs/job.rb +4 -4
  8. data/lib/sidekiq_unique_jobs/lock/base_lock.rb +1 -1
  9. data/lib/sidekiq_unique_jobs/lock/while_executing.rb +2 -0
  10. data/lib/sidekiq_unique_jobs/lock.rb +13 -10
  11. data/lib/sidekiq_unique_jobs/locksmith.rb +3 -3
  12. data/lib/sidekiq_unique_jobs/lua/unlock.lua +16 -9
  13. data/lib/sidekiq_unique_jobs/middleware/client.rb +1 -1
  14. data/lib/sidekiq_unique_jobs/middleware/server.rb +1 -1
  15. data/lib/sidekiq_unique_jobs/middleware.rb +5 -5
  16. data/lib/sidekiq_unique_jobs/orphans/reaper.rb +2 -1
  17. data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +26 -7
  18. data/lib/sidekiq_unique_jobs/redis/sorted_set.rb +8 -0
  19. data/lib/sidekiq_unique_jobs/script/script.rb +1 -6
  20. data/lib/sidekiq_unique_jobs/server.rb +2 -0
  21. data/lib/sidekiq_unique_jobs/sidekiq_unique_ext.rb +13 -35
  22. data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +3 -3
  23. data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb +3 -3
  24. data/lib/sidekiq_unique_jobs/testing.rb +4 -4
  25. data/lib/sidekiq_unique_jobs/version.rb +1 -1
  26. data/lib/sidekiq_unique_jobs/web/helpers.rb +3 -3
  27. data/lib/sidekiq_unique_jobs/web/views/changelogs.erb +1 -1
  28. data/lib/sidekiq_unique_jobs/web/views/lock.erb +2 -2
  29. data/lib/sidekiq_unique_jobs/web/views/locks.erb +1 -1
  30. data/lib/sidekiq_unique_jobs/web.rb +16 -16
  31. metadata +3 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0f2baa17d73480d2a3af58b1202e5145935944abebfddbd94a199248df87e729
4
- data.tar.gz: 11037fd5c6ed9a61452026541c874928ec0d465def211c7f34e91d7d1bb23bb2
3
+ metadata.gz: 3e0587400a796faf2d96ad2882bb2da1d39378d188ce9b257f845f1234acc4bf
4
+ data.tar.gz: 7bbd398cc60c7b99bd491b352804f1b3d57e6c02318ad82d40855739dcb30582
5
5
  SHA512:
6
- metadata.gz: 927072c34b1b7131352dbfe5ccd269bace168d601977dfcb63199183033202b6ec04824612620375b336de9e06501a2b39237370357494af66a376385260faa3
7
- data.tar.gz: cb615cce8db1b353f0b12790ee35019c709586f94b497d5543ef43a50a782b1e09c47371ddd28b810243b15e7d983030c5a56999ef7535745f4596890fc983dd
6
+ metadata.gz: 2efce8d2064bd4b572c4a1db8533514eff3ada446f939b68d437d3d72c7244169594a84c3ecd400a5aa90f9e0666ee88ef849590aea37c3471bb9b32631816dc
7
+ data.tar.gz: 01af8820641cf873f01379da41279db5a81b90d23465dde4a1eff81b21ce60b70688e6e28c59deadfba3088e1ad148d3987c4fbb8f9fdaf11cbb3e4da6331a25
data/CHANGELOG.md CHANGED
@@ -1,5 +1,46 @@
1
1
  # Changelog
2
2
 
3
+ ## [v8.0.7](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.7) (2024-02-05)
4
+
5
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.6...v8.0.7)
6
+
7
+ **Implemented enhancements:**
8
+
9
+ - chore\(ci\): add test coverage from bropoplpush [\#828](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/828) ([mhenrixon](https://github.com/mhenrixon))
10
+
11
+ **Fixed bugs:**
12
+
13
+ - fix\(xss\): sanitize parameters [\#829](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/829) ([mhenrixon](https://github.com/mhenrixon))
14
+
15
+ **Closed issues:**
16
+
17
+ - No 'Changelog' link is being displayed on https://rubygems.org/gems/sidekiq-unique-jobs [\#825](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/825)
18
+ - Using rspec matcher and getting: `NameError: uninitialized constant SidekiqUniqueJobs::Lock::BaseLock::Validator` [\#741](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/741)
19
+
20
+ **Merged pull requests:**
21
+
22
+ - Fix Testing Instructions [\#827](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/827) ([jherdman](https://github.com/jherdman))
23
+
24
+ ## [v8.0.6](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.6) (2024-01-24)
25
+
26
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.5...v8.0.6)
27
+
28
+ **Implemented enhancements:**
29
+
30
+ - Returning same job id [\#814](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/814)
31
+ - fix: various minor issues [\#826](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/826) ([mhenrixon](https://github.com/mhenrixon))
32
+
33
+ **Closed issues:**
34
+
35
+ - should respond to `has_valid_sidekiq_options?` [\#822](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/822)
36
+ - Reaper manager registration is subject to race conditions [\#801](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/801)
37
+ - `while_executing` with `on_conflict: :reschedule` Reschedule job after job execution [\#800](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/800)
38
+ - Large retry queue causes reaper to run too slow [\#759](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/759)
39
+
40
+ **Merged pull requests:**
41
+
42
+ - fix: skip unless reaper was registered [\#820](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/820) ([mhenrixon](https://github.com/mhenrixon))
43
+
3
44
  ## [v8.0.5](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.5) (2023-11-11)
4
45
 
5
46
  [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.4...v8.0.5)
@@ -15,7 +56,7 @@
15
56
 
16
57
  ## [v8.0.4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.4) (2023-11-11)
17
58
 
18
- [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.30...v8.0.4)
59
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.31...v8.0.4)
19
60
 
20
61
  **Implemented enhancements:**
21
62
 
@@ -31,9 +72,6 @@
31
72
  **Closed issues:**
32
73
 
33
74
  - register\_reaper\_process nx: true crash Sidekiq on startup [\#817](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/817)
34
- - Should client middleware also be added to the Sidekiq server config? [\#803](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/803)
35
- - 8.0.1 Time on locks & changelog UI is incorrect/wrong [\#761](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/761)
36
- - Job executed twice when reaper runs [\#738](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/738)
37
75
 
38
76
  **Merged pull requests:**
39
77
 
@@ -42,6 +80,16 @@
42
80
  - Handle strategy fallbacks properly [\#809](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/809) ([matejrisek](https://github.com/matejrisek))
43
81
  - Fix CI status badge [\#802](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/802) ([tagliala](https://github.com/tagliala))
44
82
 
83
+ ## [v7.1.31](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.31) (2023-08-28)
84
+
85
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.30...v7.1.31)
86
+
87
+ **Closed issues:**
88
+
89
+ - Should client middleware also be added to the Sidekiq server config? [\#803](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/803)
90
+ - 8.0.1 Time on locks & changelog UI is incorrect/wrong [\#761](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/761)
91
+ - Job executed twice when reaper runs [\#738](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/738)
92
+
45
93
  ## [v7.1.30](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.30) (2023-07-17)
46
94
 
47
95
  [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.3...v7.1.30)
data/README.md CHANGED
@@ -582,8 +582,6 @@ end
582
582
  #spec/workers/bad_worker_spec.rb
583
583
 
584
584
  require "sidekiq_unique_jobs/testing"
585
- #OR
586
- require "sidekiq_unique_jobs/rspec/matchers"
587
585
 
588
586
  RSpec.describe BadWorker do
589
587
  specify { expect(described_class).to have_valid_sidekiq_options }
@@ -816,7 +814,7 @@ In my benchmarks deleting 1000 orphaned locks with lua performs around 65% faste
816
814
 
817
815
  On the other hand if I increase it to 10 000 orphaned locks per cleanup (`reaper_count: 10_0000`) then redis starts throwing:
818
816
 
819
- > BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. (Redis::CommandError)
817
+ > BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. (RedisClient::CommandError)
820
818
 
821
819
  If you want to disable the reaper set it to `:none`, `nil` or `false`. Actually, any value that isn't `:ruby` or `:lua` will disable the reaping.
822
820
 
@@ -9,7 +9,7 @@ module SidekiqUniqueJobs
9
9
  class BatchDelete
10
10
  #
11
11
  # @return [Integer] the default batch size
12
- BATCH_SIZE = 100
12
+ BATCH_SIZE = 500
13
13
 
14
14
  #
15
15
  # @return [Array<String>] Supported key suffixes
@@ -67,10 +67,10 @@ module SidekiqUniqueJobs
67
67
  # @return [Hash<Symbol, SidekiqUniqueJobs::Lock::BaseLock] all available default locks
68
68
  LOCKS =
69
69
  LOCKS_WHEN_BUSY.dup
70
- .merge(LOCKS_WHILE_ENQUEUED.dup)
71
- .merge(LOCKS_WITHOUT_UNLOCK.dup)
72
- .merge(LOCKS_FROM_PUSH_TO_PROCESSED.dup)
73
- .freeze
70
+ .merge(LOCKS_WHILE_ENQUEUED.dup)
71
+ .merge(LOCKS_WITHOUT_UNLOCK.dup)
72
+ .merge(LOCKS_FROM_PUSH_TO_PROCESSED.dup)
73
+ .freeze
74
74
 
75
75
  #
76
76
  # @return [Hash<Symbol, SidekiqUniqueJobs::OnConflict::Strategy] all available default strategies
@@ -15,6 +15,7 @@ module SidekiqUniqueJobs
15
15
  DEAD_VERSION = "uniquejobs:dead"
16
16
  DIGESTS = "uniquejobs:digests"
17
17
  EXPIRING_DIGESTS = "uniquejobs:expiring_digests"
18
+ ORPHANED_DIGESTS = "uniquejobs:orphaned_digests"
18
19
  ERRORS = "errors"
19
20
  JID = "jid"
20
21
  LIMIT = "limit"
@@ -41,19 +41,19 @@ module SidekiqUniqueJobs
41
41
  end
42
42
 
43
43
  def add_lock_timeout(item)
44
- item[LOCK_TIMEOUT] ||= SidekiqUniqueJobs::LockTimeout.calculate(item)
44
+ item[LOCK_TIMEOUT] = SidekiqUniqueJobs::LockTimeout.calculate(item)
45
45
  end
46
46
 
47
47
  def add_lock_args(item)
48
- item[LOCK_ARGS] ||= SidekiqUniqueJobs::LockArgs.call(item)
48
+ item[LOCK_ARGS] = SidekiqUniqueJobs::LockArgs.call(item)
49
49
  end
50
50
 
51
51
  def add_lock_digest(item)
52
- item[LOCK_DIGEST] ||= SidekiqUniqueJobs::LockDigest.call(item)
52
+ item[LOCK_DIGEST] = SidekiqUniqueJobs::LockDigest.call(item)
53
53
  end
54
54
 
55
55
  def add_lock_prefix(item)
56
- item[LOCK_PREFIX] ||= SidekiqUniqueJobs.config.lock_prefix
56
+ item[LOCK_PREFIX] = SidekiqUniqueJobs.config.lock_prefix
57
57
  end
58
58
 
59
59
  def add_lock_type(item)
@@ -109,7 +109,7 @@ module SidekiqUniqueJobs
109
109
  #
110
110
  # Call whatever strategry that has been configured
111
111
  #
112
- # @param [Symbol] origin: the origin `:client` or `:server`
112
+ # @param [Symbol] origin the origin `:client` or `:server`
113
113
  #
114
114
  # @return [void] the return value is irrelevant
115
115
  #
@@ -11,6 +11,8 @@ module SidekiqUniqueJobs
11
11
  #
12
12
  # @author Mikael Henriksson <mikael@mhenrixon.com>
13
13
  class WhileExecuting < BaseLock
14
+ #
15
+ # @return [String] returns :RUN
14
16
  RUN_SUFFIX = ":RUN"
15
17
 
16
18
  include SidekiqUniqueJobs::OptionsWithFallback
@@ -33,9 +33,9 @@ module SidekiqUniqueJobs
33
33
  #
34
34
  # @return [Lock] a newly lock that has been locked
35
35
  #
36
- def self.create(digest, job_id, lock_info = {})
37
- lock = new(digest, time: Timing.now_f)
38
- lock.lock(job_id, lock_info)
36
+ def self.create(digest, job_id, lock_info: {}, time: Timing.now_f, score: nil)
37
+ lock = new(digest, time: time)
38
+ lock.lock(job_id, lock_info, score)
39
39
  lock
40
40
  end
41
41
 
@@ -63,15 +63,16 @@ module SidekiqUniqueJobs
63
63
  #
64
64
  # @return [void]
65
65
  #
66
- def lock(job_id, lock_info = {})
66
+ def lock(job_id, lock_info = {}, score = nil)
67
+ score ||= now_f
67
68
  redis do |conn|
68
69
  conn.multi do |pipeline|
69
70
  pipeline.set(key.digest, job_id)
70
71
  pipeline.hset(key.locked, job_id, now_f)
71
72
  info.set(lock_info, pipeline)
72
- add_digest_to_set(pipeline, lock_info)
73
- pipeline.zadd(key.changelog, now_f, changelog_json(job_id, "queue.lua", "Queued"))
74
- pipeline.zadd(key.changelog, now_f, changelog_json(job_id, "lock.lua", "Locked"))
73
+ add_digest_to_set(pipeline, lock_info, score)
74
+ pipeline.zadd(key.changelog, score, changelog_json(job_id, "queue.lua", "Queued"))
75
+ pipeline.zadd(key.changelog, score, changelog_json(job_id, "lock.lua", "Locked"))
75
76
  end
76
77
  end
77
78
  end
@@ -333,12 +334,14 @@ module SidekiqUniqueJobs
333
334
  #
334
335
  # @return [nil]
335
336
  #
336
- def add_digest_to_set(pipeline, lock_info)
337
+ def add_digest_to_set(pipeline, lock_info, score = nil)
338
+ score ||= now_f
337
339
  digest_string = key.digest
340
+
338
341
  if lock_info["lock"] == :until_expired
339
- pipeline.zadd(key.expiring_digests, now_f + lock_info["ttl"], digest_string)
342
+ pipeline.zadd(key.expiring_digests, score + lock_info["ttl"], digest_string)
340
343
  else
341
- pipeline.zadd(key.digests, now_f, digest_string)
344
+ pipeline.zadd(key.digests, score, digest_string)
342
345
  end
343
346
  end
344
347
  end
@@ -188,7 +188,7 @@ module SidekiqUniqueJobs
188
188
  #
189
189
  # @param [Sidekiq::RedisConnection, ConnectionPool] conn the redis connection
190
190
  # @param [Method] primed_method reference to the method to use for getting a primed token
191
- # @param [nil, Integer, Float] time to wait before timeout
191
+ # @param [nil, Integer, Float] wait time to wait before timeout
192
192
  #
193
193
  # @yieldparam [string] job_id the sidekiq JID
194
194
  # @yieldreturn [void] whatever the calling block returns
@@ -258,8 +258,8 @@ module SidekiqUniqueJobs
258
258
 
259
259
  # NOTE: When debugging, change .value to .value!
260
260
  primed_jid = Concurrent::Promises
261
- .future(conn) { |red_con| pop_queued(red_con, timeout) }
262
- .value
261
+ .future(conn) { |red_con| pop_queued(red_con, timeout) }
262
+ .value
263
263
 
264
264
  handle_primed(primed_jid, &block)
265
265
  end
@@ -75,21 +75,28 @@ if lock_type ~= "until_expired" then
75
75
  redis.call("HDEL", locked, job_id)
76
76
  end
77
77
 
78
+ if redis.call("LLEN", primed) == 0 then
79
+ log_debug("UNLINK", primed)
80
+ redis.call("UNLINK", primed)
81
+ end
82
+
78
83
  local locked_count = redis.call("HLEN", locked)
79
84
 
80
- if locked_count and locked_count < 1 then
85
+ if locked_count < 1 then
81
86
  log_debug("UNLINK", locked)
82
87
  redis.call("UNLINK", locked)
83
88
  end
84
89
 
85
- if redis.call("LLEN", primed) == 0 then
86
- log_debug("UNLINK", primed)
87
- redis.call("UNLINK", primed)
88
- end
89
-
90
- if limit and limit <= 1 and locked_count and locked_count <= 1 then
91
- log_debug("ZREM", digests, digest)
92
- redis.call("ZREM", digests, digest)
90
+ if limit then
91
+ if limit <= 1 and locked_count <= 1 then
92
+ log_debug("ZREM", digests, digest)
93
+ redis.call("ZREM", digests, digest)
94
+ end
95
+ else
96
+ if locked_count <= 1 then
97
+ log_debug("ZREM", digests, digest)
98
+ redis.call("ZREM", digests, digest)
99
+ end
93
100
  end
94
101
 
95
102
  log_debug("LPUSH", queued, "1")
@@ -6,7 +6,7 @@ module SidekiqUniqueJobs
6
6
  #
7
7
  # @author Mikael Henriksson <mikael@mhenrixon.com>
8
8
  class Client
9
- include Sidekiq::ClientMiddleware if defined?(Sidekiq::ClientMiddleware)
9
+ include Sidekiq::ClientMiddleware
10
10
 
11
11
  # prepend "SidekiqUniqueJobs::Middleware"
12
12
  # @!parse prepends SidekiqUniqueJobs::Middleware
@@ -6,7 +6,7 @@ module SidekiqUniqueJobs
6
6
  #
7
7
  # @author Mikael Henriksson <mikael@mhenrixon.com>
8
8
  class Server
9
- include Sidekiq::ServerMiddleware if defined?(Sidekiq::ServerMiddleware)
9
+ include Sidekiq::ServerMiddleware
10
10
 
11
11
  # prepend "SidekiqUniqueJobs::Middleware"
12
12
  # @!parse prepends SidekiqUniqueJobs::Middleware
@@ -19,15 +19,15 @@ module SidekiqUniqueJobs
19
19
  # This method runs before (prepended) the actual middleware implementation.
20
20
  # This is done to reduce duplication
21
21
  #
22
- # @param [Sidekiq::Worker] worker_class
22
+ # @param [Sidekiq::Job] worker_class
23
23
  # @param [Hash] item a sidekiq job hash
24
24
  # @param [String] queue name of the queue
25
25
  # @param [ConnectionPool] redis_pool only used for compatility reasons
26
26
  #
27
- # @return [yield<super>] <description>
27
+ # @return [yield<super>] call the rest of the middleware stack
28
28
  #
29
- # @yieldparam [<type>] if <description>
30
- # @yieldreturn [<type>] <describe what yield should return>
29
+ # @yieldparam [void] if uniquejobs is disable
30
+ # @yieldreturn [void] delegate back to other sidekiq middleware
31
31
  def call(worker_class, item, queue, redis_pool = nil)
32
32
  @item = item
33
33
  @queue = queue
@@ -35,7 +35,7 @@ module SidekiqUniqueJobs
35
35
  self.job_class = worker_class
36
36
  return yield if unique_disabled?
37
37
 
38
- SidekiqUniqueJobs::Job.prepare(item) unless item[LOCK_DIGEST]
38
+ SidekiqUniqueJobs::Job.prepare(item)
39
39
 
40
40
  with_logging_context do
41
41
  super
@@ -22,11 +22,12 @@ module SidekiqUniqueJobs
22
22
  #
23
23
  # @return [Hash<Symbol, SidekiqUniqueJobs::Orphans::Reaper] the current implementation of reapers
24
24
  REAPERS = {
25
- lua: SidekiqUniqueJobs::Orphans::LuaReaper,
25
+ lua: SidekiqUniqueJobs::Orphans::RubyReaper,
26
26
  ruby: SidekiqUniqueJobs::Orphans::RubyReaper,
27
27
  none: SidekiqUniqueJobs::Orphans::NullReaper,
28
28
  nil => SidekiqUniqueJobs::Orphans::NullReaper,
29
29
  false => SidekiqUniqueJobs::Orphans::NullReaper,
30
+ true => SidekiqUniqueJobs::Orphans::RubyReaper,
30
31
  }.freeze
31
32
 
32
33
  #
@@ -13,6 +13,9 @@ module SidekiqUniqueJobs
13
13
  class RubyReaper < Reaper
14
14
  include SidekiqUniqueJobs::Timing
15
15
 
16
+ #
17
+ # @return [Integer] a best guess of Sidekiq::Launcher::BEAT_PAUSE
18
+ SIDEKIQ_BEAT_PAUSE = 10
16
19
  #
17
20
  # @return [String] the suffix for :RUN locks
18
21
  RUN_SUFFIX = ":RUN"
@@ -74,14 +77,30 @@ module SidekiqUniqueJobs
74
77
 
75
78
  BatchDelete.call(expired_digests, conn)
76
79
  BatchDelete.call(orphans, conn)
80
+
81
+ # orphans.each_slice(500) do |chunk|
82
+ # conn.pipelined do |pipeline|
83
+ # chunk.each do |digest|
84
+ # next if belongs_to_job?(digest)
85
+
86
+ # pipeline.zadd(ORPHANED_DIGESTS, now_f, digest)
87
+ # end
88
+ # end
89
+ # end
77
90
  end
78
91
 
79
92
  def expired_digests
80
- max_score = (start_time - reaper_timeout).to_f
81
-
82
93
  conn.zrange(EXPIRING_DIGESTS, 0, max_score, "byscore")
83
94
  end
84
95
 
96
+ def orphaned_digests
97
+ conn.zrange(ORPHANED_DIGESTS, 0, max_score, "byscore")
98
+ end
99
+
100
+ def max_score
101
+ (start_time - reaper_timeout - SIDEKIQ_BEAT_PAUSE).to_f
102
+ end
103
+
85
104
  #
86
105
  # Find orphaned digests
87
106
  #
@@ -89,10 +108,10 @@ module SidekiqUniqueJobs
89
108
  # @return [Array<String>] an array of orphaned digests
90
109
  #
91
110
  def orphans # rubocop:disable Metrics/MethodLength, Metrics/CyclomaticComplexity
92
- page = 0
93
- per = reaper_count * 2
94
111
  orphans = []
95
- results = conn.zrange(digests.key, page * per, (page + 1) * per)
112
+ page = 0
113
+ per = reaper_count * 2
114
+ results = digests.byscore(0, max_score, offset: page * per, count: (page + 1) * per)
96
115
 
97
116
  while results.size.positive?
98
117
  results.each do |digest|
@@ -107,7 +126,7 @@ module SidekiqUniqueJobs
107
126
  break if orphans.size >= reaper_count
108
127
 
109
128
  page += 1
110
- results = conn.zrange(digests.key, page * per, (page + 1) * per)
129
+ results = digests.byscore(0, max_score, offset: page * per, count: (page + 1) * per)
111
130
  end
112
131
 
113
132
  orphans
@@ -218,7 +237,7 @@ module SidekiqUniqueJobs
218
237
  end
219
238
 
220
239
  def considered_active?(time_f)
221
- (Time.now - reaper_timeout).to_f < time_f
240
+ max_score < time_f
222
241
  end
223
242
 
224
243
  #
@@ -48,6 +48,14 @@ module SidekiqUniqueJobs
48
48
  end
49
49
  end
50
50
 
51
+ def byscore(min, max, offset: nil, count: nil)
52
+ redis do |conn|
53
+ return conn.zrange(key, min, max, "byscore") unless offset && count
54
+
55
+ conn.zrange(key, min, max, "byscore", "limit", offset, count)
56
+ end
57
+ end
58
+
51
59
  #
52
60
  # Return the zrak of the member
53
61
  #
@@ -66,12 +66,7 @@ module SidekiqUniqueJobs
66
66
  end
67
67
 
68
68
  def load(conn)
69
- @sha =
70
- if conn.respond_to?(:namespace)
71
- conn.redis.script(:load, source)
72
- else
73
- conn.script(:load, source)
74
- end
69
+ @sha = conn.script(:load, source)
75
70
 
76
71
  self
77
72
  end
@@ -5,6 +5,8 @@ module SidekiqUniqueJobs
5
5
  #
6
6
  # @author Mikael Henriksson <mikael@mhenrixon.com>
7
7
  class Server
8
+ #
9
+ # @return [Proc] returns a default death handler for the gem to cleanup dead locks
8
10
  DEATH_HANDLER = (lambda do |job, _ex|
9
11
  return unless (digest = job["lock_digest"])
10
12
 
@@ -68,46 +68,24 @@ module Sidekiq
68
68
  prepend UniqueExtension
69
69
  end
70
70
 
71
- if Sidekiq.const_defined?(:JobRecord)
72
- # See Sidekiq::Api
73
- class JobRecord
74
- #
75
- # Provides extensions for unlocking jobs that are removed and deleted
71
+ # See Sidekiq::Api
72
+ class JobRecord
73
+ #
74
+ # Provides extensions for unlocking jobs that are removed and deleted
75
+ #
76
+ # @author Mikael Henriksson <mikael@mhenrixon.com>
77
+ #
78
+ module UniqueExtension
76
79
  #
77
- # @author Mikael Henriksson <mikael@mhenrixon.com>
80
+ # Wraps the original method to ensure locks for the job are deleted
78
81
  #
79
- module UniqueExtension
80
- #
81
- # Wraps the original method to ensure locks for the job are deleted
82
- #
83
- def delete
84
- SidekiqUniqueJobs::Unlockable.delete!(item)
85
- super
86
- end
82
+ def delete
83
+ SidekiqUniqueJobs::Unlockable.delete!(item)
84
+ super
87
85
  end
88
-
89
- prepend UniqueExtension
90
86
  end
91
- else
92
- # See Sidekiq::Api
93
- class Job
94
- #
95
- # Provides extensions for unlocking jobs that are removed and deleted
96
- #
97
- # @author Mikael Henriksson <mikael@mhenrixon.com>
98
- #
99
- module UniqueExtension
100
- #
101
- # Wraps the original method to ensure locks for the job are deleted
102
- #
103
- def delete
104
- SidekiqUniqueJobs::Unlockable.delete!(item)
105
- super
106
- end
107
- end
108
87
 
109
- prepend UniqueExtension
110
- end
88
+ prepend UniqueExtension
111
89
  end
112
90
 
113
91
  # See Sidekiq::Api
@@ -250,9 +250,9 @@ module SidekiqUniqueJobs # rubocop:disable Metrics/ModuleLength
250
250
  # Attempt to constantize a string worker_class argument, always
251
251
  # failing back to the original argument when the constant can't be found
252
252
  #
253
- # @return [Sidekiq::Worker]
253
+ # @return [Sidekiq::Job]
254
254
  def constantize(str)
255
- return str.class if str.is_a?(Sidekiq::Worker) # sidekiq v6.x
255
+ return str.class if str.is_a?(Sidekiq::Job) # sidekiq v6.x
256
256
  return str unless str.is_a?(String)
257
257
  return Object.const_get(str) unless str.include?("::")
258
258
 
@@ -269,7 +269,7 @@ module SidekiqUniqueJobs # rubocop:disable Metrics/ModuleLength
269
269
  # Attempt to constantize a string worker_class argument, always
270
270
  # failing back to the original argument when the constant can't be found
271
271
  #
272
- # @return [Sidekiq::Worker, String]
272
+ # @return [Sidekiq::Job, String]
273
273
  def safe_constantize(str)
274
274
  constantize(str)
275
275
  rescue NameError => ex
@@ -1,13 +1,13 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SidekiqUniqueJobs
4
- # Module with convenience methods for the Sidekiq::Worker class
4
+ # Module with convenience methods for the Sidekiq::Job class
5
5
  #
6
6
  # @author Mikael Henriksson <mikael@mhenrixon.com>
7
7
  module SidekiqWorkerMethods
8
8
  #
9
9
  # @!attribute [r] job_class
10
- # @return [Sidekiq::Worker] The Sidekiq::Worker implementation
10
+ # @return [Sidekiq::Job] The Sidekiq::Job implementation
11
11
  attr_reader :job_class
12
12
 
13
13
  # Avoids duplicating worker_class.respond_to? in multiple places
@@ -62,7 +62,7 @@ module SidekiqUniqueJobs
62
62
  # Attempt to constantize a string worker_class argument, always
63
63
  # failing back to the original argument when the constant can't be found
64
64
  #
65
- # @return [Sidekiq::Worker]
65
+ # @return [Sidekiq::Job]
66
66
  def job_class_constantize(klazz = @job_class)
67
67
  SidekiqUniqueJobs.safe_constantize(klazz)
68
68
  end
@@ -42,11 +42,11 @@ module Sidekiq
42
42
  end
43
43
 
44
44
  #
45
- # See Sidekiq::Worker in Sidekiq gem for more details
45
+ # See Sidekiq::Job in Sidekiq gem for more details
46
46
  #
47
47
  module Worker
48
48
  #
49
- # Adds class methods to Sidekiq::Worker
49
+ # Adds class methods to Sidekiq::Job
50
50
  #
51
51
  module ClassMethods
52
52
  #
@@ -110,14 +110,14 @@ module Sidekiq
110
110
  prepend Overrides
111
111
 
112
112
  #
113
- # Prepends methods to Sidekiq::Worker
113
+ # Prepends methods to Sidekiq::Job
114
114
  #
115
115
  module ClassMethods
116
116
  prepend Overrides::ClassMethods
117
117
  end
118
118
 
119
119
  #
120
- # Prepends singleton methods to Sidekiq::Worker
120
+ # Prepends singleton methods to Sidekiq::Job
121
121
  #
122
122
  module SignletonOverrides
123
123
  #
@@ -3,5 +3,5 @@
3
3
  module SidekiqUniqueJobs
4
4
  #
5
5
  # @return [String] the current SidekiqUniqueJobs version
6
- VERSION = "8.0.6"
6
+ VERSION = "8.0.8"
7
7
  end
@@ -133,7 +133,7 @@ module SidekiqUniqueJobs
133
133
  #
134
134
  # @return [String] a html safe string with relative time information
135
135
  #
136
- def relative_time(time)
136
+ def _relative_time(time)
137
137
  stamp = time.getutc.iso8601
138
138
  %(<time class="ltr" dir="ltr" title="#{stamp}" datetime="#{stamp}">#{time}</time>)
139
139
  end
@@ -145,12 +145,12 @@ module SidekiqUniqueJobs
145
145
  #
146
146
  # @return [String] a html safe string with relative time information
147
147
  #
148
- def safe_relative_time(time)
148
+ def _safe_relative_time(time)
149
149
  return unless time
150
150
 
151
151
  time = parse_time(time)
152
152
 
153
- relative_time(time)
153
+ _relative_time(time)
154
154
  end
155
155
 
156
156
  #
@@ -42,7 +42,7 @@
42
42
  <tbody>
43
43
  <% @changelogs.each do |changelog| %>
44
44
  <tr class="changelog-row">
45
- <td><%= safe_relative_time(changelog['time']) || "bogus" %></td>
45
+ <td><%= _safe_relative_time(changelog['time']) || "bogus" %></td>
46
46
  <td><%= changelog["digest"] %></td>
47
47
  <td><%= changelog["script"] %></td>
48
48
  <td><%= changelog["job_id"] %></td>
@@ -65,7 +65,7 @@
65
65
  <% @lock.locked_jids(with_values: true).each do |job_id, time| %>
66
66
  <tr>
67
67
  <td><%= job_id %></td>
68
- <td><%= safe_relative_time(time.to_f) %></td>
68
+ <td><%= _safe_relative_time(time.to_f) %></td>
69
69
  <td>
70
70
  <form action="<%= root_path %>locks/<%= @lock.key %>/jobs/<%= job_id %>/delete" method="get">
71
71
  <%= csrf_tag %>
@@ -92,7 +92,7 @@
92
92
  <tbody>
93
93
  <% @lock.changelogs.each do |entry| %>
94
94
  <tr>
95
- <td scope="row"><%= safe_relative_time(entry["time"].to_f) %></td>
95
+ <td scope="row"><%= _safe_relative_time(entry["time"].to_f) %></td>
96
96
  <td><%= entry["job_id"] %></td>
97
97
  <td><%= entry["message"] %></td>
98
98
  <td><%= entry["script"] %></td>
@@ -46,7 +46,7 @@
46
46
  <td><a href="<%= root_path %>locks/<%= lock.key %>"><%= lock.key %></a></td>
47
47
  <td><%= lock.info["lock"] %></td>
48
48
  <td><%= lock.locked.count %></td>
49
- <td><%= safe_relative_time(lock.created_at) %></td>
49
+ <td><%= _safe_relative_time(lock.created_at) %></td>
50
50
  </tr>
51
51
  </tbody>
52
52
  <% end %>
@@ -13,11 +13,11 @@ module SidekiqUniqueJobs
13
13
  end
14
14
 
15
15
  app.get "/changelogs" do
16
- @filter = params[:filter] || "*"
16
+ @filter = h(params[:filter] || "*")
17
17
  @filter = "*" if @filter == ""
18
- @count = (params[:count] || 100).to_i
19
- @current_cursor = params[:cursor].to_i
20
- @prev_cursor = params[:prev_cursor].to_i
18
+ @count = h(params[:count] || 100).to_i
19
+ @current_cursor = h(params[:cursor]).to_i
20
+ @prev_cursor = h(params[:prev_cursor]).to_i
21
21
  @total_size, @next_cursor, @changelogs = changelog.page(
22
22
  cursor: @current_cursor,
23
23
  pattern: @filter,
@@ -33,11 +33,11 @@ module SidekiqUniqueJobs
33
33
  end
34
34
 
35
35
  app.get "/locks" do
36
- @filter = params[:filter] || "*"
36
+ @filter = h(params[:filter]) || "*"
37
37
  @filter = "*" if @filter == ""
38
- @count = (params[:count] || 100).to_i
39
- @current_cursor = params[:cursor].to_i
40
- @prev_cursor = params[:prev_cursor].to_i
38
+ @count = h(params[:count] || 100).to_i
39
+ @current_cursor = h(params[:cursor]).to_i
40
+ @prev_cursor = h(params[:prev_cursor]).to_i
41
41
 
42
42
  @total_size, @next_cursor, @locks = digests.page(
43
43
  cursor: @current_cursor,
@@ -49,11 +49,11 @@ module SidekiqUniqueJobs
49
49
  end
50
50
 
51
51
  app.get "/expiring_locks" do
52
- @filter = params[:filter] || "*"
52
+ @filter = h(params[:filter]) || "*"
53
53
  @filter = "*" if @filter == ""
54
- @count = (params[:count] || 100).to_i
55
- @current_cursor = params[:cursor].to_i
56
- @prev_cursor = params[:prev_cursor].to_i
54
+ @count = h(params[:count] || 100).to_i
55
+ @current_cursor = h(params[:cursor]).to_i
56
+ @prev_cursor = h(params[:prev_cursor]).to_i
57
57
 
58
58
  @total_size, @next_cursor, @locks = expiring_digests.page(
59
59
  cursor: @current_cursor,
@@ -71,20 +71,20 @@ module SidekiqUniqueJobs
71
71
  end
72
72
 
73
73
  app.get "/locks/:digest" do
74
- @digest = params[:digest]
74
+ @digest = h(params[:digest])
75
75
  @lock = SidekiqUniqueJobs::Lock.new(@digest)
76
76
 
77
77
  erb(unique_template(:lock))
78
78
  end
79
79
 
80
80
  app.get "/locks/:digest/delete" do
81
- digests.delete_by_digest(params[:digest])
82
- expiring_digests.delete_by_digest(params[:digest])
81
+ digests.delete_by_digest(h(params[:digest]))
82
+ expiring_digests.delete_by_digest(h(params[:digest]))
83
83
  redirect_to :locks
84
84
  end
85
85
 
86
86
  app.get "/locks/:digest/jobs/:job_id/delete" do
87
- @digest = params[:digest]
87
+ @digest = h(params[:digest])
88
88
  @lock = SidekiqUniqueJobs::Lock.new(@digest)
89
89
  @lock.unlock(params[:job_id])
90
90
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sidekiq-unique-jobs
3
3
  version: !ruby/object:Gem::Version
4
- version: 8.0.6
4
+ version: 8.0.8
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2024-01-24 00:00:00.000000000 Z
11
+ date: 2024-02-12 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: concurrent-ruby
@@ -223,7 +223,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
223
223
  - !ruby/object:Gem::Version
224
224
  version: '0'
225
225
  requirements: []
226
- rubygems_version: 3.5.5
226
+ rubygems_version: 3.5.6
227
227
  signing_key:
228
228
  specification_version: 4
229
229
  summary: Sidekiq middleware that prevents duplicates jobs