sidekiq-unique-jobs 7.1.8 → 7.1.13

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f0525a9235a69f86ef177c23a54afea5977a4a99fe4c09867d88c88d29eeb280
-  data.tar.gz: f105453b5082cb2a37ff2484e8c92213ec578a488b5f97f894b7f06e901aca23
+  metadata.gz: bd2a8c713ad1593ff340793f2cbd1d07427bbb239d3efd4cc810489f99801e8b
+  data.tar.gz: e33006dc4b63292e1e8cf7576d7c80d59ee077dfea852d6dbb7230afa52ed116
 SHA512:
-  metadata.gz: 24eeed23d326f74ebf0c06d34913c8429d1ce7848bdb384606176bf8362ae74f8e1a3dbd0cffad10fbdac1344fe7a1170804e332f5974713c890f6fb330fae91
-  data.tar.gz: 61ce06ebeb3b49f25dbb3ffb12cfe90f164e9c668756e47b81d527181f66e27a07dc836f5230bb1949ca94429c59bdfe2fa7bc13c3dbac3a53ebe1feb2257080
+  metadata.gz: e0a410cd505b3ccf292ded829328fa78f7250f42a350709e88d220f2ce5ec73e7f93d7c6c6adaab7fc0ffb0c28314e3ba3ce6b6e96c02918c9a6f2d908cf31d0
+  data.tar.gz: 6fbbb12c4db8a91a29c5baf6cc96b208576074744d417649191b26bccc7b8069ceeddd178ecf864c5028fc35744700c7ffdf621f0a2cc8669d79c4f675a1ac54

data/CHANGELOG.md

@@ -1,5 +1,60 @@
 # Changelog
 
+## [v7.1.12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.12) (2021-12-01)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.11...v7.1.12)
+
+**Implemented enhancements:**
+
+- Improve Ruby Reaper performance under heavy load [\#663](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/663)
+- Improve reaper performance under heavy load [\#666](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/666) ([mhenrixon](https://github.com/mhenrixon))
+
+## [v7.1.11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.11) (2021-11-30)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.10...v7.1.11)
+
+**Fixed bugs:**
+
+- Fix ruby reaper edge case [\#661](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/661) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- Question: Wait instead of cancelling if it is executing? [\#655](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/655)
+- Expired Locks remain in zset of digests \[using "until\_expired" lock\] [\#653](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/653)
+- \[Q&A\] Performance & Dead Locks [\#652](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/652)
+- Sidekiq 6.3.0 includes Job module that clashes with sidekiq\_unique\_ext.rb class Job [\#651](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/651)
+
+## [v7.1.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.10) (2021-10-18)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.8...v7.1.10)
+
+**Fixed bugs:**
+
+- "IndexError: string not matched" when job is replaced on client [\#635](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/635)
+
+**Merged pull requests:**
+
+- Update URL for Sidekiq's Enterprise unique jobs [\#648](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/648) ([jsantos](https://github.com/jsantos))
+
+## [v7.1.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.8) (2021-10-08)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.13...v7.1.8)
+
+**Fixed bugs:**
+
+- undefined method `delete' for class `Sidekiq::Job' [\#634](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/634)
+- INFO keys not persisted when job is enqueued [\#602](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/602)
+- lock\_info set to true but no lock info showing up in web ui [\#589](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/589)
+- Prevent too eager cleanup of lock info [\#645](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/645) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- Compatibility with unreleased Sidekiq 6.3.0 [\#636](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/636)
+
+**Merged pull requests:**
+
+- Update docs [\#644](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/644) ([andypple](https://github.com/andypple))
+
 ## [v7.0.13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.13) (2021-09-27)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.7...v7.0.13)

data/README.md

@@ -599,7 +599,7 @@ Expected BadWorker to have valid sidekiq options but found the following problem
 If you are not using RSpec (a lot of people prefer minitest or test unit) you can do something like:
 
 ```ruby
-assert SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options)
+assert_raise(InvalidWorker){ SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options) }
 ```
 
 ### Uniqueness
@@ -1043,7 +1043,7 @@ There is a [![Join the chat at https://gitter.im/mhenrixon/sidekiq-unique-jobs](
 
 You can find a list of contributors over on [Contributors][]
 
-[Enterprise unique jobs]: https://www.dailydrip.com/topics/sidekiq/drips/sidekiq-enterprise-unique-jobs
+[Enterprise unique jobs]: https://github.com/mperham/sidekiq/wiki/Ent-Unique-Jobs
 [Contributors]: https://github.com/mhenrixon/sidekiq-unique-jobs/graphs/contributors
 [v4.0.18]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18
 [v5.0.10]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10.

data/lib/sidekiq_unique_jobs/changelog.rb

@@ -62,9 +62,9 @@ module SidekiqUniqueJobs
     #
     def page(cursor: 0, pattern: "*", page_size: 100)
       redis do |conn|
-        total_size, result = conn.multi do
-          conn.zcard(key)
-          conn.zscan(key, cursor, match: pattern, count: page_size)
+        total_size, result = conn.multi do |pipeline|
+          pipeline.zcard(key)
+          pipeline.zscan(key, cursor, match: pattern, count: page_size)
         end
 
         [

data/lib/sidekiq_unique_jobs/digests.rb

@@ -95,9 +95,9 @@ module SidekiqUniqueJobs
     #
     def page(cursor: 0, pattern: SCAN_PATTERN, page_size: 100)
       redis do |conn|
-        total_size, digests = conn.multi do
-          conn.zcard(key)
-          conn.zscan(key, cursor, match: pattern, count: page_size)
+        total_size, digests = conn.multi do |pipeline|
+          pipeline.zcard(key)
+          pipeline.zscan(key, cursor, match: pattern, count: page_size)
         end
 
         [

data/lib/sidekiq_unique_jobs/lock.rb

@@ -62,13 +62,13 @@ module SidekiqUniqueJobs
     #
     def lock(job_id, lock_info = {})
       redis do |conn|
-        conn.multi do
-          conn.set(key.digest, job_id)
-          conn.hset(key.locked, job_id, now_f)
+        conn.multi do |pipeline|
+          pipeline.set(key.digest, job_id)
+          pipeline.hset(key.locked, job_id, now_f)
           info.set(lock_info)
-          conn.zadd(key.digests, now_f, key.digest)
-          conn.zadd(key.changelog, now_f, changelog_json(job_id, "queue.lua", "Queued"))
-          conn.zadd(key.changelog, now_f, changelog_json(job_id, "lock.lua", "Locked"))
+          pipeline.zadd(key.digests, now_f, key.digest)
+          pipeline.zadd(key.changelog, now_f, changelog_json(job_id, "queue.lua", "Queued"))
+          pipeline.zadd(key.changelog, now_f, changelog_json(job_id, "lock.lua", "Locked"))
         end
       end
     end

data/lib/sidekiq_unique_jobs/lua/unlock.lua

@@ -65,18 +65,12 @@ redis.call("LREM", queued, -1, job_id)
 log_debug("LREM", primed, -1, job_id)
 redis.call("LREM", primed, -1, job_id)
 
-if limit and limit <= 1 and locked_count and locked_count <= 1 then
-  log_debug("ZREM", digests, digest)
-  redis.call("ZREM", digests, digest)
-end
-
 local redis_version = toversion(redisversion)
 local del_cmd       = "DEL"
 
 if tonumber(redis_version["major"]) >= 4 then del_cmd =  "UNLINK"; end
 
 if lock_type ~= "until_expired" then
-
   log_debug(del_cmd, digest, info)
   redis.call(del_cmd, digest, info)
 
@@ -86,11 +80,16 @@ end
 
 local locked_count = redis.call("HLEN", locked)
 
-if tonumber(locked_count) < 1 then
+if locked_count and locked_count < 1 then
   log_debug(del_cmd, locked)
   redis.call(del_cmd, locked)
 end
 
+if limit and limit <= 1 and locked_count and locked_count <= 1 then
+  log_debug("ZREM", digests, digest)
+  redis.call("ZREM", digests, digest)
+end
+
 log_debug("LPUSH", queued, "1")
 redis.call("LPUSH", queued, "1")
 

data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb

@@ -9,6 +9,7 @@ module SidekiqUniqueJobs
     #
     # @author Mikael Henriksson <mikael@mhenrixon.com>
     #
+    # rubocop:disable Metrics/ClassLength
     class RubyReaper < Reaper
       #
       # @return [String] the suffix for :RUN locks
@@ -54,13 +55,27 @@ module SidekiqUniqueJobs
       #
       # @return [Array<String>] an array of orphaned digests
       #
-      def orphans
-        conn.zrevrange(digests.key, 0, -1).each_with_object([]) do |digest, memo|
-          next if belongs_to_job?(digest)
+      def orphans # rubocop:disable Metrics/MethodLength
+        page = 0
+        per = reaper_count * 2
+        orphans = []
+        results = conn.zrange(digests.key, page * per, (page + 1) * per)
 
-          memo << digest
-          break if memo.size >= reaper_count
+        while results.size.positive?
+          results.each do |digest|
+            next if belongs_to_job?(digest)
+
+            orphans << digest
+            break if orphans.size >= reaper_count
+          end
+
+          break if orphans.size >= reaper_count
+
+          page += 1
+          results = conn.zrange(digests.key, page * per, (page + 1) * per)
         end
+
+        orphans
       end
 
       #
@@ -211,5 +226,6 @@ module SidekiqUniqueJobs
         conn.zscan_each(key, match: "*#{digest}*", count: 1).to_a.any?
       end
     end
+    # rubocop:enable Metrics/ClassLength
   end
 end

data/lib/sidekiq_unique_jobs/sidekiq_unique_ext.rb

@@ -68,7 +68,7 @@ module Sidekiq
     prepend UniqueExtension
   end
 
-  if Sidekiq.const_defined?("JobRecord")
+  if Sidekiq.const_defined?(:JobRecord)
     # See Sidekiq::Api
     class JobRecord
       #

data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb

@@ -34,12 +34,22 @@ module SidekiqUniqueJobs
 
     # The hook to call after a successful unlock
     # @return [Proc]
-    def after_unlock_hook
+    def after_unlock_hook # rubocop:disable Metrics/MethodLength
       lambda do
         if @worker_class.respond_to?(:after_unlock)
-          @worker_class.after_unlock # instance method in sidekiq v6
+          # instance method in sidekiq v6
+          if @worker_class.method(:after_unlock).arity.positive? # arity check to maintain backwards compatibility
+            @worker_class.after_unlock(item)
+          else
+            @worker_class.after_unlock
+          end
         elsif worker_class.respond_to?(:after_unlock)
-          worker_class.after_unlock # class method regardless of sidekiq version
+          # class method regardless of sidekiq version
+          if worker_class.method(:after_unlock).arity.positive? # arity check to maintain backwards compatibility
+            worker_class.after_unlock(item)
+          else
+            worker_class.after_unlock
+          end
         end
       end
     end

data/lib/sidekiq_unique_jobs/timing.rb

@@ -48,7 +48,7 @@ module SidekiqUniqueJobs
     # @return [Float]
     #
     def clock_stamp
-      if Process.const_defined?("CLOCK_MONOTONIC")
+      if Process.const_defined?(:CLOCK_MONOTONIC)
         Process.clock_gettime(Process::CLOCK_MONOTONIC)
       else
         now_f

data/lib/sidekiq_unique_jobs/version.rb

@@ -3,5 +3,5 @@
 module SidekiqUniqueJobs
   #
   # @return [String] the current SidekiqUniqueJobs version
-  VERSION = "7.1.8"
+  VERSION = "7.1.13"
 end

data/lib/sidekiq_unique_jobs/web/helpers.rb

@@ -27,7 +27,7 @@ module SidekiqUniqueJobs
       # @return [String] the file contents of the template
       #
       def unique_template(name)
-        File.open(unique_filename(name)).read
+        File.read(unique_filename(name))
       end
 
       #

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq-unique-jobs
 version: !ruby/object:Gem::Version
-  version: 7.1.8
+  version: 7.1.13
 platform: ruby
 authors:
 - Mikael Henriksson
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-10-08 00:00:00.000000000 Z
+date: 2022-02-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: brpoplpush-redis_script
@@ -211,15 +211,11 @@ files:
 - lib/sidekiq_unique_jobs/web/views/lock.erb
 - lib/sidekiq_unique_jobs/web/views/locks.erb
 - lib/tasks/changelog.rake
-homepage: https://mhenrixon.github.io/sidekiq-unique-jobs
+homepage: https://github.com/mhenrixon/sidekiq-unique-jobs
 licenses:
 - MIT
 metadata:
-  homepage_uri: https://mhenrixon.github.io/sidekiq-unique-jobs
-  bug_tracker_uri: https://github.com/mhenrixon/sidekiq-unique-jobs/issues
-  documentation_uri: https://mhenrixon.github.io/sidekiq-unique-jobs
-  source_code_uri: https://github.com/mhenrixon/sidekiq-unique-jobs
-  changelog_uri: https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/CHANGELOG.md
+  rubygems_mfa_required: 'true'
 post_install_message: |
   IMPORTANT!
 
@@ -261,7 +257,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.28
+rubygems_version: 3.3.4
 signing_key:
 specification_version: 4
 summary: Sidekiq middleware that prevents duplicates jobs