sidekiq-unique-jobs 7.0.0.beta21 → 7.0.0.beta22

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: aa60dc35cfa347f88d50dd3b115e0313c88f150368a5052eec648f1e136c4e4e
-  data.tar.gz: 708a1f4191b7ec2f0a07129ce6abc8528908c2289c66cf55e9142c93b661e603
+  metadata.gz: '09650d2cec14e6bfeaf6b74b938286eca9ef889275cd3e885b196cca2ffb1f2f'
+  data.tar.gz: 613901dec7e012a67d6e6e1289f1d2b0d9f47e6f6d2764eb1618d2ec389879e7
 SHA512:
-  metadata.gz: a097632a78d20a8ad3aaaf98819b18274a962965be59e3a0b3d0bd0208c47410dfc98bfd370007b35454d7a2ac3699374244d63fad3f11f549dde16998e542fc
-  data.tar.gz: beaa24b58dc8844aa689e108d2529f29b1390745df99687ab074e6822dad4cabb72402ae132b32307c4753551b70a5d44d1f0f475a6f64cb39a2405ecb4e9ff2
+  metadata.gz: 9a12b9d88451356effaeeebfe24e4f69f964aa53d7c85958fa55da58ae2a6a26482a5865dd78b0fbd6389bceb2936449dfde64a6a1238242ed097c24818b26fe
+  data.tar.gz: c4284c65010101466e6e3521a4358633c76337f7a5304040f4d188fae8a56f6af2e6f53350da283c5d45f90c4cb28f60d9768a201e3ddf7f19e7ee0cbfa44ffc

data/CHANGELOG.md

@@ -1,5 +1,31 @@
 # Changelog
 
+## [Unreleased](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/HEAD)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta21...HEAD)
+
+**Fixed bugs:**
+
+- Infinite loop in ruby reaper [\#515](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/515)
+
+## [v7.0.0.beta21](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta21) (2020-06-12)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta20...v7.0.0.beta21)
+
+**Implemented enhancements:**
+
+- Move gems to gemfile [\#513](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/513) ([mhenrixon](https://github.com/mhenrixon))
+- Move dev-gems from gemspec to gemfile [\#512](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/512) ([mhenrixon](https://github.com/mhenrixon))
+
+**Fixed bugs:**
+
+- Prevent indefinitely looping entries [\#516](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/516) ([mhenrixon](https://github.com/mhenrixon))
+- Expire reaper when not checking in [\#508](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/508) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- Missing web interface [\#514](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/514)
+
 ## [v7.0.0.beta20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta20) (2020-06-02)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta19...v7.0.0.beta20)
@@ -16,18 +42,10 @@
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta18...v7.0.0.beta19)
 
-**Fixed bugs:**
-
-- Expire reaper when not checking in [\#508](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/508) ([mhenrixon](https://github.com/mhenrixon))
-
 ## [v7.0.0.beta18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta18) (2020-05-21)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta17...v7.0.0.beta18)
 
-**Fixed bugs:**
-
-- Stringify on\_conflict hash in Job prepare method [\#507](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/507) ([jasonbekolay](https://github.com/jasonbekolay))
-
 ## [v7.0.0.beta17](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta17) (2020-05-20)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta16...v7.0.0.beta17)
@@ -36,13 +54,16 @@
 
 - Try GitHub actions [\#505](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/505) ([mhenrixon](https://github.com/mhenrixon))
 
+**Fixed bugs:**
+
+- Deep stringify worker options to account for hash in on\_conflict [\#506](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/506) ([jasonbekolay](https://github.com/jasonbekolay))
+
 ## [v7.0.0.beta16](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta16) (2020-05-19)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.22...v7.0.0.beta16)
 
 **Fixed bugs:**
 
-- Deep stringify worker options to account for hash in on\_conflict [\#506](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/506) ([jasonbekolay](https://github.com/jasonbekolay))
 - Deprecate configuration options with `default\_` [\#504](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/504) ([mhenrixon](https://github.com/mhenrixon))
 - Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon))
 
@@ -73,6 +94,7 @@
 - Brpoplpush::RedisScript::LuaError: WRONGTYPE Operation against a key holding the wrong kind of value [\#491](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/491)
 - Lua script bug [\#489](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/489)
 - Reaper will delete locks for running jobs [\#488](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/488)
+- Stringify on\_conflict hash in Job prepare method [\#507](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/507) ([jasonbekolay](https://github.com/jasonbekolay))
 - Fix access to hash members [\#496](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/496) ([mhenrixon](https://github.com/mhenrixon))
 - Fix cursor assignment [\#494](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/494) ([mhenrixon](https://github.com/mhenrixon))
 - Prevent reaping of active jobs [\#493](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/493) ([mhenrixon](https://github.com/mhenrixon))
@@ -154,7 +176,6 @@
 **Merged pull requests:**
 
 - README: Use SVG badges ✨ [\#470](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/470) ([olleolleolle](https://github.com/olleolleolle))
-- remove deprecated/broken OptionsWithFallback\#unique\_type [\#435](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/435) ([zvkemp](https://github.com/zvkemp))
 
 ## [v6.0.19](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.19) (2020-03-21)
 
@@ -297,6 +318,7 @@
 **Merged pull requests:**
 
 - Fix typo and some formatting issues in README [\#442](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/442) ([ajkerr](https://github.com/ajkerr))
+- remove deprecated/broken OptionsWithFallback\#unique\_type [\#435](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/435) ([zvkemp](https://github.com/zvkemp))
 
 ## [v7.0.0.beta2](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta2) (2019-10-08)
 
@@ -683,7 +705,6 @@
 - Duplicate jobs getting created [\#257](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/257)
 - Multiple non-unique jobs with until\_executed? [\#255](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/255)
 - :until\_executing not unlocking when starting to run [\#245](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/245)
-- Drop jobs hash [\#282](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/282) ([mhenrixon](https://github.com/mhenrixon))
 
 **Closed issues:**
 
@@ -726,6 +747,10 @@
 - Reject while scheduling [\#273](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/273) ([mhenrixon](https://github.com/mhenrixon))
 - Improve testing [\#272](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/272) ([mhenrixon](https://github.com/mhenrixon))
 
+**Fixed bugs:**
+
+- Drop jobs hash [\#282](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/282) ([mhenrixon](https://github.com/mhenrixon))
+
 ## [v6.0.0.beta](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.beta) (2018-06-17)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v5.0.10...v6.0.0.beta)
@@ -1134,7 +1159,6 @@
 - mock\_redis and the mess [\#62](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/62)
 - What is the exact behavior? [\#47](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/47)
 - Throttling jobs [\#39](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/39)
-- undefined method `get\_sidekiq\_options' for "MyScheduledWorker":String  [\#27](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/27)
 - Crash handling [\#14](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/14)
 - Missing info from README [\#6](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/6)
 
@@ -1231,6 +1255,7 @@
 - Short jobs are not unique for the given time window [\#33](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/33)
 - Not all sidekiq:sidekiq\_unique keys are removed from Redis [\#31](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/31)
 - What does uniqueness mean in case of this gem? [\#30](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/30)
+- undefined method `get\_sidekiq\_options' for "MyScheduledWorker":String  [\#27](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/27)
 - Server middleware removes payload hash key before expiration [\#26](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/26)
 - Lock remains when running with Sidekiq::Testing.inline! [\#23](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/23)
 - What is the use case for the uniqueness window? [\#22](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/22)
@@ -1243,7 +1268,6 @@
 - Clarify README about unique expiration [\#36](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/36) ([spacemunkay](https://github.com/spacemunkay))
 - Add option to make jobs unique on all queues [\#32](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/32) ([robinmessage](https://github.com/robinmessage))
 - Fix homepage in gemspec [\#29](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/29) ([tmaier](https://github.com/tmaier))
-- Compatibility with Sidekiq 2.12.1 Scheduled Jobs [\#16](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/16) ([lsimoneau](https://github.com/lsimoneau))
 
 ## [v2.7.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v2.7.0) (2013-11-24)
 
@@ -1262,6 +1286,7 @@
 - Make unlock/yield order configurable. [\#21](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/21) ([endofunky](https://github.com/endofunky))
 - Rely on Sidekiq's String\#constantize extension instead of rolling our own [\#19](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/19) ([disbelief](https://github.com/disbelief))
 - Attempt to constantize String `worker\_class` arguments passed to client middleware [\#17](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/17) ([disbelief](https://github.com/disbelief))
+- Compatibility with Sidekiq 2.12.1 Scheduled Jobs [\#16](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/16) ([lsimoneau](https://github.com/lsimoneau))
 - Allow worker to specify which arguments to include in uniquing hash [\#12](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/12) ([sax](https://github.com/sax))
 - Add support for unique when using Sidekiq's delay function [\#11](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/11) ([eduardosasso](https://github.com/eduardosasso))
 - Adding the unique prefix option [\#8](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/8) ([KensoDev](https://github.com/KensoDev))

data/lib/sidekiq_unique_jobs/lua/reap_orphans.lua

@@ -65,11 +65,7 @@ repeat
     -- TODO: Add check for jobs checked out by process
     if found ~= true then
       log_debug("Searching for digest:", digest, "in process sets")
-      local queue = find_digest_in_process_set(digest)
-      if queue then
-        log_debug("found digest:", digest, "in queue:", queue)
-        found = true
-      end
+      found = find_digest_in_process_set(digest)
     end
 
     if found ~= true then

data/lib/sidekiq_unique_jobs/lua/shared/_find_digest_in_process_set.lua

@@ -4,7 +4,7 @@ local function find_digest_in_process_set(digest)
   local pattern        = "*" .. digest .. "*"
   local found          = false
 
-  log_debug("searching in list processes:",
+  log_debug("Searching in process list",
             "for digest:", digest,
             "cursor:", process_cursor)
 
@@ -15,15 +15,26 @@ local function find_digest_in_process_set(digest)
     log_debug("Found number of processes:", #processes, "next cursor:", next_process_cursor)
 
     for _, process in ipairs(processes) do
+      local workers_key = process .. ":workers"
       log_debug("searching in process set:", process,
                 "for digest:", digest,
                 "cursor:", process_cursor)
 
-      local job = redis.call("HGET", process, "info")
+      local jobs = redis.call("HGETALL", workers_key)
 
-      if string.find(job, digest) then
-        log_debug("Found digest", digest, "in process:", process)
-        found = true
+      if #jobs == 0 then
+        log_debug("No entries in:", workers_key)
+      else
+        for i = 1, #jobs, 2 do
+          if string.find(jobs[i +1], digest) then
+            log_debug("Found digest", digest, "in:", workers_key)
+            found = true
+            break
+          end
+        end
+      end
+
+      if found == true then
         break
       end
     end

data/lib/sidekiq_unique_jobs/on_conflict/reject.rb

@@ -111,7 +111,7 @@ module SidekiqUniqueJobs
       # @return [String] a JSON formatted string
       #
       def payload
-        @payload ||= Sidekiq.dump_json(item)
+        @payload ||= dump_json(item)
       end
     end
   end

data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb

@@ -117,17 +117,24 @@ module SidekiqUniqueJobs
         end
       end
 
-      def active?(digest)
+      def active?(digest) # rubocop:disable Metrics/MethodLength
         Sidekiq.redis do |conn|
-          procs = conn.sscan_each("processes").to_a.sort
+          procs = conn.sscan_each("processes").to_a
+          return false if procs.empty?
 
-          result = conn.pipelined do
-            procs.map do |key|
-              conn.hget(key, "info")
+          procs.sort.each do |key|
+            valid, workers = conn.pipelined do
+              conn.exists(key)
+              conn.hgetall("#{key}:workers")
             end
-          end
 
-          result.flatten.compact.any? { |job| load_json(job)[LOCK_DIGEST] == digest }
+            next unless valid
+            next unless workers.any?
+
+            workers.each_pair do |_tid, job|
+              return true if load_json(job)[LOCK_DIGEST] == digest
+            end
+          end
         end
       end
 

data/lib/sidekiq_unique_jobs/version.rb

@@ -3,5 +3,5 @@
 module SidekiqUniqueJobs
   #
   # @return [String] the current SidekiqUniqueJobs version
-  VERSION = "7.0.0.beta21"
+  VERSION = "7.0.0.beta22"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq-unique-jobs
 version: !ruby/object:Gem::Version
-  version: 7.0.0.beta21
+  version: 7.0.0.beta22
 platform: ruby
 authors:
 - Mikael Henriksson