sidekiq-unique-jobs 7.0.0.beta16 → 7.0.0.beta21

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f290b8f99626c55f5152ecf2c15dc6776dda2090f8b1f848aefad2a08fc693b2
-  data.tar.gz: '08e56acded9b53d7653ea4af16dd8bd767096532c72f0f1c085c14e4ffabb4b6'
+  metadata.gz: aa60dc35cfa347f88d50dd3b115e0313c88f150368a5052eec648f1e136c4e4e
+  data.tar.gz: 708a1f4191b7ec2f0a07129ce6abc8528908c2289c66cf55e9142c93b661e603
 SHA512:
-  metadata.gz: 648e4bf89e3e4170cba81fe7bcdb68f7d15b8359dff1b0843dd5c38b0426628a4d5472f389d4f3df227a32f1cbfc8c32bc599eadfc8beca9ad4bbba27a72cab6
-  data.tar.gz: 19a9189c6e31d870528a2995a622cf1b41c681ec12315754fa704aee1f38f1b28c8ba9a31999598c9dfa0ae531f9866ab089b3d4708c850f0ed4341c0773a957
+  metadata.gz: a097632a78d20a8ad3aaaf98819b18274a962965be59e3a0b3d0bd0208c47410dfc98bfd370007b35454d7a2ac3699374244d63fad3f11f549dde16998e542fc
+  data.tar.gz: beaa24b58dc8844aa689e108d2529f29b1390745df99687ab074e6822dad4cabb72402ae132b32307c4753551b70a5d44d1f0f475a6f64cb39a2405ecb4e9ff2

data/CHANGELOG.md

@@ -1,5 +1,63 @@
 # Changelog
 
+## [v7.0.0.beta20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta20) (2020-06-02)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta19...v7.0.0.beta20)
+
+**Fixed bugs:**
+
+- Reaper can't be registered again if sidekiq gets killed by SIGKILL [\#490](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/490)
+
+**Closed issues:**
+
+- How do I turn this on for only one job class? [\#510](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/510)
+
+## [v7.0.0.beta19](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta19) (2020-05-21)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta18...v7.0.0.beta19)
+
+**Fixed bugs:**
+
+- Expire reaper when not checking in [\#508](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/508) ([mhenrixon](https://github.com/mhenrixon))
+
+## [v7.0.0.beta18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta18) (2020-05-21)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta17...v7.0.0.beta18)
+
+**Fixed bugs:**
+
+- Stringify on\_conflict hash in Job prepare method [\#507](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/507) ([jasonbekolay](https://github.com/jasonbekolay))
+
+## [v7.0.0.beta17](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta17) (2020-05-20)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta16...v7.0.0.beta17)
+
+**Implemented enhancements:**
+
+- Try GitHub actions [\#505](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/505) ([mhenrixon](https://github.com/mhenrixon))
+
+## [v7.0.0.beta16](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta16) (2020-05-19)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.22...v7.0.0.beta16)
+
+**Fixed bugs:**
+
+- Deep stringify worker options to account for hash in on\_conflict [\#506](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/506) ([jasonbekolay](https://github.com/jasonbekolay))
+- Deprecate configuration options with `default\_` [\#504](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/504) ([mhenrixon](https://github.com/mhenrixon))
+- Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- V7 Beta 15 `on\_conflict:` with Hash does not work on server [\#499](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/499)
+
+## [v6.0.22](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.22) (2020-04-13)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta15...v6.0.22)
+
+**Fixed bugs:**
+
+- \[LockType\] Allow overwriting default lock type at item level [\#497](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/497) ([EmmN](https://github.com/EmmN))
+
 ## [v7.0.0.beta15](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta15) (2020-04-10)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta14...v7.0.0.beta15)
@@ -155,7 +213,6 @@
 **Implemented enhancements:**
 
 - Clarify usage with global\_id and sidekiq-status [\#455](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/455) ([mhenrixon](https://github.com/mhenrixon))
-- Bump rails [\#450](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/450) ([mhenrixon](https://github.com/mhenrixon))
 
 **Merged pull requests:**
 
@@ -181,6 +238,7 @@
 
 **Implemented enhancements:**
 
+- Bump rails [\#450](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/450) ([mhenrixon](https://github.com/mhenrixon))
 - Rename myapp [\#449](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/449) ([mhenrixon](https://github.com/mhenrixon))
 - Just to keep track of this [\#445](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/445) ([mhenrixon](https://github.com/mhenrixon))
 
@@ -492,7 +550,6 @@
 **Fixed bugs:**
 
 - Enable replace strategy [\#315](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/315) ([mhenrixon](https://github.com/mhenrixon))
-- Bug fixes [\#310](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/310) ([mhenrixon](https://github.com/mhenrixon))
 
 **Closed issues:**
 
@@ -509,6 +566,7 @@
 **Fixed bugs:**
 
 - Not unlocking automatically \(version 6.0.0rc5\) [\#293](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/293)
+- Bug fixes [\#310](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/310) ([mhenrixon](https://github.com/mhenrixon))
 
 ## [v6.0.1](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.1) (2018-07-31)
 
@@ -1185,6 +1243,7 @@
 - Clarify README about unique expiration [\#36](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/36) ([spacemunkay](https://github.com/spacemunkay))
 - Add option to make jobs unique on all queues [\#32](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/32) ([robinmessage](https://github.com/robinmessage))
 - Fix homepage in gemspec [\#29](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/29) ([tmaier](https://github.com/tmaier))
+- Compatibility with Sidekiq 2.12.1 Scheduled Jobs [\#16](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/16) ([lsimoneau](https://github.com/lsimoneau))
 
 ## [v2.7.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v2.7.0) (2013-11-24)
 
@@ -1203,7 +1262,6 @@
 - Make unlock/yield order configurable. [\#21](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/21) ([endofunky](https://github.com/endofunky))
 - Rely on Sidekiq's String\#constantize extension instead of rolling our own [\#19](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/19) ([disbelief](https://github.com/disbelief))
 - Attempt to constantize String `worker\_class` arguments passed to client middleware [\#17](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/17) ([disbelief](https://github.com/disbelief))
-- Compatibility with Sidekiq 2.12.1 Scheduled Jobs [\#16](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/16) ([lsimoneau](https://github.com/lsimoneau))
 - Allow worker to specify which arguments to include in uniquing hash [\#12](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/12) ([sax](https://github.com/sax))
 - Add support for unique when using Sidekiq's delay function [\#11](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/11) ([eduardosasso](https://github.com/eduardosasso))
 - Adding the unique prefix option [\#8](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/8) ([KensoDev](https://github.com/KensoDev))

data/lib/sidekiq_unique_jobs/changelog.rb

@@ -40,7 +40,7 @@ module SidekiqUniqueJobs
       options[:count] = count if count
 
       redis do |conn|
-        conn.zscan_each(key, options).to_a.map { |entry| load_json(entry[0]) }
+        conn.zscan_each(key, **options).to_a.map { |entry| load_json(entry[0]) }
       end
     end
 

data/lib/sidekiq_unique_jobs/config.rb

@@ -185,25 +185,29 @@ module SidekiqUniqueJobs
     end
 
     def default_lock_ttl=(obj)
-      warn "[DEPRECATION] `#{self.class}##{__method__}` is deprecated. Please use `#{self.class}#lock_ttl=` instead."
+      warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_ttl=` instead."
       self.lock_ttl = obj
     end
 
     def default_lock_timeout=(obj)
-      warn "[DEPRECATION] `#{self.class}##{__method__}` is deprecated. Please use `#{self.class}#lock_timeout=` instead."
+      warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_timeout=` instead."
       self.lock_timeout = obj
     end
 
     def default_lock_ttl
-      warn "[DEPRECATION] `#{self.class}##{__method__}` is deprecated. Please use `#{self.class}#lock_ttl` instead."
+      warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_ttl` instead."
       lock_ttl
     end
 
     def default_lock_timeout
-      warn "[DEPRECATION] `#{self.class}##{__method__}` is deprecated. Please use `#{self.class}#lock_timeout` instead."
+      warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated. Please use `#{class_name}#lock_timeout` instead."
       lock_timeout
     end
 
+    def class_name
+      @class_name ||= self.class.name
+    end
+
     #
     # Adds a lock type to the configuration. It will raise if the lock exists already
     #

data/lib/sidekiq_unique_jobs/core_ext.rb

@@ -102,7 +102,7 @@ class Hash
           result[yield(key)] = _deep_transform_keys_in_object(value, &block)
         end
       when Array
-        object.map { |e| _deep_transform_keys_in_object(e, &block) }
+        object.map { |element| _deep_transform_keys_in_object(element, &block) }
       else
         object
       end

data/lib/sidekiq_unique_jobs/digests.rb

@@ -80,7 +80,7 @@ module SidekiqUniqueJobs
       options[:match] = pattern
       options[:count] = count if count
 
-      result = redis { |conn| conn.zscan_each(key, options).to_a }
+      result = redis { |conn| conn.zscan_each(key, **options).to_a }
 
       result.each_with_object({}) do |entry, hash|
         hash[entry[0]] = entry[1]

data/lib/sidekiq_unique_jobs/job.rb

@@ -10,6 +10,7 @@ module SidekiqUniqueJobs
     # Adds timeout, expiration, lock_args, lock_prefix and lock_digest to the sidekiq job hash
     # @return [Hash] the job hash
     def prepare(item)
+      stringify_on_conflict_hash(item)
       add_lock_timeout(item)
       add_lock_ttl(item)
       add_digest(item)
@@ -27,6 +28,13 @@ module SidekiqUniqueJobs
 
     private
 
+    def stringify_on_conflict_hash(item)
+      on_conflict = item[ON_CONFLICT]
+      return unless on_conflict.is_a?(Hash)
+
+      item[ON_CONFLICT] = on_conflict.deep_stringify_keys
+    end
+
     def add_lock_ttl(item)
       item[LOCK_TTL] = SidekiqUniqueJobs::LockTTL.calculate(item)
     end

data/lib/sidekiq_unique_jobs/lock_args.rb

@@ -63,7 +63,6 @@ module SidekiqUniqueJobs
     # @return [Array] args unfiltered when neither of the above
     def filtered_args
       return args if lock_args_disabled?
-      return args if args.empty?
 
       json_args = Normalizer.jsonify(args)
 

data/lib/sidekiq_unique_jobs/orphans/manager.rb

@@ -10,6 +10,8 @@ module SidekiqUniqueJobs
     module Manager
       module_function
 
+      DRIFT_FACTOR = 0.02
+
       include SidekiqUniqueJobs::Connection
       include SidekiqUniqueJobs::Logging
 
@@ -56,6 +58,7 @@ module SidekiqUniqueJobs
         @task ||= Concurrent::TimerTask.new(timer_task_options) do
           with_logging_context do
             redis do |conn|
+              refresh_reaper_mutex
               Orphans::Reaper.call(conn)
             end
           end
@@ -117,7 +120,9 @@ module SidekiqUniqueJobs
       # @return [true, false]
       #
       def registered?
-        redis { |conn| conn.get(UNIQUE_REAPER) }.to_i == 1
+        redis do |conn|
+          conn.get(UNIQUE_REAPER).to_i + drift_reaper_interval > current_timestamp
+        end
       end
 
       def disabled?
@@ -131,7 +136,17 @@ module SidekiqUniqueJobs
       # @return [void]
       #
       def register_reaper_process
-        redis { |conn| conn.set(UNIQUE_REAPER, 1) }
+        redis { |conn| conn.set(UNIQUE_REAPER, current_timestamp, nx: true, ex: drift_reaper_interval) }
+      end
+
+      #
+      # Updates mutex key
+      #
+      #
+      # @return [void]
+      #
+      def refresh_reaper_mutex
+        redis { |conn| conn.set(UNIQUE_REAPER, current_timestamp, ex: drift_reaper_interval) }
       end
 
       #
@@ -143,6 +158,14 @@ module SidekiqUniqueJobs
       def unregister_reaper_process
         redis { |conn| conn.del(UNIQUE_REAPER) }
       end
+
+      def drift_reaper_interval
+        reaper_interval + (reaper_interval * DRIFT_FACTOR).to_i
+      end
+
+      def current_timestamp
+        Time.now.to_i
+      end
     end
   end
 end

data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb

@@ -157,6 +157,8 @@ module SidekiqUniqueJobs
           entries     = conn.lrange(queue_key, range_start, range_end)
           page       += 1
 
+          break if entries.empty?
+
           entries.each do |entry|
             yield entry
           end

data/lib/sidekiq_unique_jobs/redis/entity.rb

@@ -46,7 +46,13 @@ module SidekiqUniqueJobs
       # @return [false] when not exists
       #
       def exist?
-        redis { |conn| conn.exists(key) }
+        redis do |conn|
+          value = conn.exists(key)
+          return true if value.is_a?(TrueClass)
+          return false if value.is_a?(FalseClass)
+
+          value.positive?
+        end
       end
 
       #

data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb

@@ -16,7 +16,7 @@ module SidekiqUniqueJobs
     def worker_options
       return {} unless sidekiq_worker_class?
 
-      worker_class.get_sidekiq_options.stringify_keys
+      worker_class.get_sidekiq_options.deep_stringify_keys
     end
 
     # Tests that the

data/lib/sidekiq_unique_jobs/upgrade_locks.rb

@@ -126,7 +126,7 @@ module SidekiqUniqueJobs
     def batch_scan(match:, count:)
       cursor = "0"
       loop do
-        cursor, values = conn.scan(cursor, match: match, per: count)
+        cursor, values = conn.scan(cursor, match: match, count: count)
         yield values
         break if cursor == "0"
       end

data/lib/sidekiq_unique_jobs/version.rb

@@ -3,5 +3,5 @@
 module SidekiqUniqueJobs
   #
   # @return [String] the current SidekiqUniqueJobs version
-  VERSION = "7.0.0.beta16"
+  VERSION = "7.0.0.beta21"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq-unique-jobs
 version: !ruby/object:Gem::Version
-  version: 7.0.0.beta16
+  version: 7.0.0.beta21
 platform: ruby
 authors:
 - Mikael Henriksson
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-05-19 00:00:00.000000000 Z
+date: 2020-06-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: brpoplpush-redis_script
@@ -90,158 +90,6 @@ dependencies:
     - - "<"
       - !ruby/object:Gem::Version
         version: '2.0'
-- !ruby/object:Gem::Dependency
-  name: bundler
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '2.1'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '2.1'
-- !ruby/object:Gem::Dependency
-  name: rack-test
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '1.0'
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '2.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '1.0'
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '2.0'
-- !ruby/object:Gem::Dependency
-  name: rake
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '13.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '13.0'
-- !ruby/object:Gem::Dependency
-  name: rspec
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.9'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.9'
-- !ruby/object:Gem::Dependency
-  name: sinatra
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '2.0'
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '3.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '2.0'
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '3.0'
-- !ruby/object:Gem::Dependency
-  name: timecop
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.9'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.9'
-- !ruby/object:Gem::Dependency
-  name: github-markup
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.0'
-- !ruby/object:Gem::Dependency
-  name: github_changelog_generator
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.14'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.14'
-- !ruby/object:Gem::Dependency
-  name: yard
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: 0.9.18
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: 0.9.18
-- !ruby/object:Gem::Dependency
-  name: gem-release
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '2.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '2.0'
 description: |
   Prevents simultaneous Sidekiq jobs with the same unique arguments to run.
   Highly configurable to suite your specific needs.
@@ -391,8 +239,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: 1.3.1
 requirements: []
-rubygems_version: 3.1.2
-signing_key: 
+rubygems_version: 3.1.4
+signing_key:
 specification_version: 4
 summary: Sidekiq middleware that prevents duplicates jobs
 test_files: []