sidekiq-unique-jobs 7.1.11 → 7.1.15

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b360fae82f2428f0a6c2d390b6320291bc34da86cb6670e858d902ea111751cc
-  data.tar.gz: c9668d930a39b39635317451a21bd560bb971aa2eecfad6f6aeabbc5559cda1d
+  metadata.gz: 05fc5a66be1f5117707cf5409bfbd37c7b22677a7035f8a9377fc89a949b85f1
+  data.tar.gz: 5e7f255449d07dbc0758ffc315b7b22442132bb053cedbdf25a0253ea4c5a59f
 SHA512:
-  metadata.gz: 60540d8412247eae7dc99328eb77a79ad18dccc4fe83345b4abfb1bf4e3aa4e9fe713a49b6c2adaa5b081122471d95e88898cf19f5c97914a84f04fdfa259a11
-  data.tar.gz: c5c5c92b36675ce1567cbbba76839ba9466ac03fdd53000947d611f9f8a3a68df6cafca6909d947c3c367839014cdc61e5fd8a45c7f5c3e5ac114a811aa69828
+  metadata.gz: 3554b3ca12ea3dd79d6c6fced007e6cb8fd40a0314a7edeecae397f3da4bb1fd276f48bbb84c16f9f99c1d915b2780595547232a6fdc20d6b0cf559c82acc6c3
+  data.tar.gz: 169ee0a9a4809904e9a74957479a0a4f08e2f5d9f464d3e137a4eb2a2796eda3ea64a298f7e9b7a3c49d83d2b86a5df8c10848b4cfe74745828fd7166d5f9d79

data/CHANGELOG.md

@@ -1,5 +1,74 @@
 # Changelog
 
+## [v7.1.14](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.14) (2022-02-04)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.13...v7.1.14)
+
+**Implemented enhancements:**
+
+- Fix the remaining deprecation warnings [\#681](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/681) ([mhenrixon](https://github.com/mhenrixon))
+
+## [v7.1.13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.13) (2022-02-03)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.12...v7.1.13)
+
+**Implemented enhancements:**
+
+- Prepare for redis 5.0.0 [\#680](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/680) ([mhenrixon](https://github.com/mhenrixon))
+
+**Fixed bugs:**
+
+- Fix homepage url [\#667](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/667) ([dal-ioki](https://github.com/dal-ioki))
+
+**Closed issues:**
+
+- Job finished, but lock is not cleared [\#677](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/677)
+- sidekiq\_options lock conflicts with sidekiq-lock gem lock option [\#669](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/669)
+- Slow evalsha causing timeouts [\#668](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/668)
+- Inconsistent documentation for config validation [\#647](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/647)
+
+**Merged pull requests:**
+
+- Bump bundler and friends [\#674](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/674) ([mhenrixon](https://github.com/mhenrixon))
+- readme: fix minitest assertion. [\#672](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/672) ([crondaemon](https://github.com/crondaemon))
+- Pass `item` in `after_unlock` callback [\#665](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/665) ([piloos](https://github.com/piloos))
+
+## [v7.1.12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.12) (2021-12-01)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.11...v7.1.12)
+
+**Implemented enhancements:**
+
+- Improve Ruby Reaper performance under heavy load [\#663](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/663)
+- Improve reaper performance under heavy load [\#666](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/666) ([mhenrixon](https://github.com/mhenrixon))
+
+## [v7.1.11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.11) (2021-11-30)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.10...v7.1.11)
+
+**Fixed bugs:**
+
+- Fix ruby reaper edge case [\#661](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/661) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- Question: Wait instead of cancelling if it is executing? [\#655](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/655)
+- Expired Locks remain in zset of digests \[using "until\_expired" lock\] [\#653](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/653)
+- \[Q&A\] Performance & Dead Locks [\#652](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/652)
+- Sidekiq 6.3.0 includes Job module that clashes with sidekiq\_unique\_ext.rb class Job [\#651](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/651)
+
+## [v7.1.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.10) (2021-10-18)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.8...v7.1.10)
+
+**Fixed bugs:**
+
+- "IndexError: string not matched" when job is replaced on client [\#635](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/635)
+
+**Merged pull requests:**
+
+- Update URL for Sidekiq's Enterprise unique jobs [\#648](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/648) ([jsantos](https://github.com/jsantos))
+
 ## [v7.1.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.8) (2021-10-08)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.13...v7.1.8)
@@ -109,7 +178,7 @@
 **Implemented enhancements:**
 
 - Reflections [\#611](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/611) ([mhenrixon](https://github.com/mhenrixon))
-- Start new orphan reaper process if orphan reaper is not running [\#604](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/604) ([Assa1121](https://github.com/Assa1121))
+- Start new orphan reaper process if orphan reaper is not running [\#604](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/604) ([AlexFlint73](https://github.com/AlexFlint73))
 
 **Fixed bugs:**
 

data/README.md

@@ -599,7 +599,7 @@ Expected BadWorker to have valid sidekiq options but found the following problem
 If you are not using RSpec (a lot of people prefer minitest or test unit) you can do something like:
 
 ```ruby
-assert SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options)
+assert_raise(InvalidWorker){ SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options) }
 ```
 
 ### Uniqueness

data/lib/sidekiq_unique_jobs/batch_delete.rb

@@ -87,10 +87,10 @@ module SidekiqUniqueJobs
     #
     def batch_delete(conn)
       digests.each_slice(BATCH_SIZE) do |chunk|
-        conn.pipelined do
+        conn.pipelined do |pipeline|
           chunk.each do |digest|
-            del_digest(conn, digest)
-            conn.zrem(SidekiqUniqueJobs::DIGESTS, digest)
+            del_digest(pipeline, digest)
+            pipeline.zrem(SidekiqUniqueJobs::DIGESTS, digest)
             @count += 1
           end
         end
@@ -99,13 +99,13 @@ module SidekiqUniqueJobs
       @count
     end
 
-    def del_digest(conn, digest)
+    def del_digest(pipeline, digest)
       removable_keys = keys_for_digest(digest)
 
       if VersionCheck.satisfied?(redis_version, ">= 4.0.0")
-        conn.unlink(*removable_keys)
+        pipeline.unlink(*removable_keys)
       else
-        conn.del(*removable_keys)
+        pipeline.del(*removable_keys)
       end
     end
 

data/lib/sidekiq_unique_jobs/changelog.rb

@@ -62,9 +62,9 @@ module SidekiqUniqueJobs
     #
     def page(cursor: 0, pattern: "*", page_size: 100)
       redis do |conn|
-        total_size, result = conn.multi do
-          conn.zcard(key)
-          conn.zscan(key, cursor, match: pattern, count: page_size)
+        total_size, result = conn.multi do |pipeline|
+          pipeline.zcard(key)
+          pipeline.zscan(key, cursor, match: pattern, count: page_size)
         end
 
         [

data/lib/sidekiq_unique_jobs/digests.rb

@@ -95,9 +95,9 @@ module SidekiqUniqueJobs
     #
     def page(cursor: 0, pattern: SCAN_PATTERN, page_size: 100)
       redis do |conn|
-        total_size, digests = conn.multi do
-          conn.zcard(key)
-          conn.zscan(key, cursor, match: pattern, count: page_size)
+        total_size, digests = conn.multi do |pipeline|
+          pipeline.zcard(key)
+          pipeline.zscan(key, cursor, match: pattern, count: page_size)
         end
 
         [

data/lib/sidekiq_unique_jobs/lock.rb

@@ -62,13 +62,13 @@ module SidekiqUniqueJobs
     #
     def lock(job_id, lock_info = {})
       redis do |conn|
-        conn.multi do
-          conn.set(key.digest, job_id)
-          conn.hset(key.locked, job_id, now_f)
-          info.set(lock_info)
-          conn.zadd(key.digests, now_f, key.digest)
-          conn.zadd(key.changelog, now_f, changelog_json(job_id, "queue.lua", "Queued"))
-          conn.zadd(key.changelog, now_f, changelog_json(job_id, "lock.lua", "Locked"))
+        conn.multi do |pipeline|
+          pipeline.set(key.digest, job_id)
+          pipeline.hset(key.locked, job_id, now_f)
+          info.set(lock_info, pipeline)
+          pipeline.zadd(key.digests, now_f, key.digest)
+          pipeline.zadd(key.changelog, now_f, changelog_json(job_id, "queue.lua", "Queued"))
+          pipeline.zadd(key.changelog, now_f, changelog_json(job_id, "lock.lua", "Locked"))
         end
       end
     end
@@ -123,9 +123,9 @@ module SidekiqUniqueJobs
     #
     def del
       redis do |conn|
-        conn.multi do
-          conn.zrem(DIGESTS, key.digest)
-          conn.del(key.digest, key.queued, key.primed, key.locked, key.info)
+        conn.multi do |pipeline|
+          pipeline.zrem(DIGESTS, key.digest)
+          pipeline.del(key.digest, key.queued, key.primed, key.locked, key.info)
         end
       end
     end

data/lib/sidekiq_unique_jobs/lock_info.rb

@@ -55,13 +55,13 @@ module SidekiqUniqueJobs
     #
     # @return [Hash]
     #
-    def set(obj)
+    def set(obj, pipeline = nil)
       return unless SidekiqUniqueJobs.config.lock_info
       raise InvalidArgument, "argument `obj` (#{obj}) needs to be a hash" unless obj.is_a?(Hash)
 
       json = dump_json(obj)
       @value = load_json(json)
-      super(json)
+      super(json, pipeline)
       value
     end
   end

data/lib/sidekiq_unique_jobs/on_conflict/reject.rb

@@ -96,10 +96,10 @@ module SidekiqUniqueJobs
       #
       def push_to_deadset
         redis do |conn|
-          conn.multi do
-            conn.zadd("dead", now_f, payload)
-            conn.zremrangebyscore("dead", "-inf", now_f - Sidekiq::DeadSet.timeout)
-            conn.zremrangebyrank("dead", 0, -Sidekiq::DeadSet.max_jobs)
+          conn.multi do |pipeline|
+            pipeline.zadd("dead", now_f, payload)
+            pipeline.zremrangebyscore("dead", "-inf", now_f - Sidekiq::DeadSet.timeout)
+            pipeline.zremrangebyrank("dead", 0, -Sidekiq::DeadSet.max_jobs)
           end
         end
       end

data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb

@@ -21,7 +21,7 @@ module SidekiqUniqueJobs
       #   This will mess up sidekiq stats because a new job is created
       def call
         if sidekiq_worker_class?
-          if worker_class.perform_in(5, *item[ARGS])
+          if worker_class.set(queue: item["queue"].to_sym).perform_in(5, *item[ARGS])
             reflect(:rescheduled, item)
           else
             reflect(:reschedule_failed, item)

data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb

@@ -9,6 +9,7 @@ module SidekiqUniqueJobs
     #
     # @author Mikael Henriksson <mikael@mhenrixon.com>
     #
+    # rubocop:disable Metrics/ClassLength
     class RubyReaper < Reaper
       #
       # @return [String] the suffix for :RUN locks
@@ -54,13 +55,27 @@ module SidekiqUniqueJobs
       #
       # @return [Array<String>] an array of orphaned digests
       #
-      def orphans
-        conn.zrevrange(digests.key, 0, -1).each_with_object([]) do |digest, memo|
-          next if belongs_to_job?(digest)
+      def orphans # rubocop:disable Metrics/MethodLength
+        page = 0
+        per = reaper_count * 2
+        orphans = []
+        results = conn.zrange(digests.key, page * per, (page + 1) * per)
 
-          memo << digest
-          break memo if memo.size >= reaper_count
+        while results.size.positive?
+          results.each do |digest|
+            next if belongs_to_job?(digest)
+
+            orphans << digest
+            break if orphans.size >= reaper_count
+          end
+
+          break if orphans.size >= reaper_count
+
+          page += 1
+          results = conn.zrange(digests.key, page * per, (page + 1) * per)
         end
+
+        orphans
       end
 
       #
@@ -126,14 +141,14 @@ module SidekiqUniqueJobs
           return false if procs.empty?
 
           procs.sort.each do |key|
-            valid, workers = conn.pipelined do
+            valid, workers = conn.pipelined do |pipeline|
               # TODO: Remove the if statement in the future
-              if conn.respond_to?(:exists?)
-                conn.exists?(key)
+              if pipeline.respond_to?(:exists?)
+                pipeline.exists?(key)
               else
-                conn.exists(key)
+                pipeline.exists(key)
               end
-              conn.hgetall("#{key}:workers")
+              pipeline.hgetall("#{key}:workers")
             end
 
             next unless valid
@@ -211,5 +226,6 @@ module SidekiqUniqueJobs
         conn.zscan_each(key, match: "*#{digest}*", count: 1).to_a.any?
       end
     end
+    # rubocop:enable Metrics/ClassLength
   end
 end

data/lib/sidekiq_unique_jobs/redis/string.rb

@@ -25,7 +25,9 @@ module SidekiqUniqueJobs
       #
       # @return [true, false]
       #
-      def set(obj)
+      def set(obj, pipeline = nil)
+        return pipeline.set(key, obj) if pipeline
+
         redis { |conn| conn.set(key, obj) }
       end
 

data/lib/sidekiq_unique_jobs/sidekiq_unique_ext.rb

@@ -68,7 +68,7 @@ module Sidekiq
     prepend UniqueExtension
   end
 
-  if Sidekiq.const_defined?("JobRecord")
+  if Sidekiq.const_defined?(:JobRecord)
     # See Sidekiq::Api
     class JobRecord
       #

data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb

@@ -34,12 +34,22 @@ module SidekiqUniqueJobs
 
     # The hook to call after a successful unlock
     # @return [Proc]
-    def after_unlock_hook
+    def after_unlock_hook # rubocop:disable Metrics/MethodLength
       lambda do
         if @worker_class.respond_to?(:after_unlock)
-          @worker_class.after_unlock # instance method in sidekiq v6
+          # instance method in sidekiq v6
+          if @worker_class.method(:after_unlock).arity.positive? # arity check to maintain backwards compatibility
+            @worker_class.after_unlock(item)
+          else
+            @worker_class.after_unlock
+          end
         elsif worker_class.respond_to?(:after_unlock)
-          worker_class.after_unlock # class method regardless of sidekiq version
+          # class method regardless of sidekiq version
+          if worker_class.method(:after_unlock).arity.positive? # arity check to maintain backwards compatibility
+            worker_class.after_unlock(item)
+          else
+            worker_class.after_unlock
+          end
         end
       end
     end

data/lib/sidekiq_unique_jobs/timing.rb

@@ -48,7 +48,7 @@ module SidekiqUniqueJobs
     # @return [Float]
     #
     def clock_stamp
-      if Process.const_defined?("CLOCK_MONOTONIC")
+      if Process.const_defined?(:CLOCK_MONOTONIC)
         Process.clock_gettime(Process::CLOCK_MONOTONIC)
       else
         now_f

data/lib/sidekiq_unique_jobs/upgrade_locks.rb

@@ -87,9 +87,9 @@ module SidekiqUniqueJobs
       digest     = grabbed_key.gsub(":GRABBED", "")
       locks      = conn.hgetall(grabbed_key)
 
-      conn.pipelined do
-        conn.hmset(locked_key, *locks.to_a)
-        conn.zadd(DIGESTS, locks.values.first, digest)
+      conn.pipelined do |pipeline|
+        pipeline.hmset(locked_key, *locks.to_a)
+        pipeline.zadd(DIGESTS, locks.values.first, digest)
       end
     end
 
@@ -114,11 +114,11 @@ module SidekiqUniqueJobs
     def batch_delete(*keys)
       return if keys.empty?
 
-      conn.pipelined do
+      conn.pipelined do |pipeline|
         if VersionCheck.satisfied?(redis_version, ">= 4.0.0")
-          conn.unlink(*keys)
+          pipeline.unlink(*keys)
         else
-          conn.del(*keys)
+          pipeline.del(*keys)
         end
       end
     end

data/lib/sidekiq_unique_jobs/version.rb

@@ -3,5 +3,5 @@
 module SidekiqUniqueJobs
   #
   # @return [String] the current SidekiqUniqueJobs version
-  VERSION = "7.1.11"
+  VERSION = "7.1.15"
 end

data/lib/sidekiq_unique_jobs/web/helpers.rb

@@ -27,7 +27,7 @@ module SidekiqUniqueJobs
       # @return [String] the file contents of the template
       #
       def unique_template(name)
-        File.open(unique_filename(name)).read
+        File.read(unique_filename(name))
       end
 
       #

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq-unique-jobs
 version: !ruby/object:Gem::Version
-  version: 7.1.11
+  version: 7.1.15
 platform: ruby
 authors:
 - Mikael Henriksson
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-11-30 00:00:00.000000000 Z
+date: 2022-02-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: brpoplpush-redis_script
@@ -211,7 +211,7 @@ files:
 - lib/sidekiq_unique_jobs/web/views/lock.erb
 - lib/sidekiq_unique_jobs/web/views/locks.erb
 - lib/tasks/changelog.rake
-homepage: https://mhenrixon.github.io/sidekiq-unique-jobs
+homepage: https://github.com/mhenrixon/sidekiq-unique-jobs
 licenses:
 - MIT
 metadata:
@@ -257,7 +257,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.32
+rubygems_version: 3.3.6
 signing_key:
 specification_version: 4
 summary: Sidekiq middleware that prevents duplicates jobs