sidekiq-unique-jobs 7.1.7 → 7.1.12

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: caf49af950fdcdac84c9c4d4567cfb8697aa93c21640b7e32368e42e20808708
-  data.tar.gz: aec81add0a9d86582d03c104600e0aee90429cd437928761f91d7350a415dc6b
+  metadata.gz: 636522bb794fcb282d02d8f0978f25764dea0c0eb07a80886d88be4212094529
+  data.tar.gz: 875231fe2ef546537ab6d152366c2555d664ca79951a4a1af563a78b7cd6efe8
 SHA512:
-  metadata.gz: 6f1a2c59d5facc875f25f1436fb04ef2ef36023212476b2ce036694094df8c00e11cdf16f936c67ba649ef2705e6d915bce6f93ac13c3ae50c4bdbba3f83f9f7
-  data.tar.gz: 0110e9e2ea06c40c39f083315254510de87d3c411d494047f0af9bc97586403edf88cf09d553a98046d750546dd283acc247a3627ecf623a67516004d6493eea
+  metadata.gz: db7884313d19f561be0ee68327f1068647123919450d72eee8ddba504d8105d16fa72b4930f990cc5904cea742b4307731ebdec6baeb00dc15e4271c9d8ab7d4
+  data.tar.gz: 923268a0919c0787ee835399c1a1b01e5a98b0c103cf59faf9361bc218fa7f19a5a2f13ed84b751588f25a7ea682bae67c36c0f0e8e5a7a90b3598b4bcac82dc

data/CHANGELOG.md

@@ -1,5 +1,67 @@
 # Changelog
 
+## [v7.1.11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.11) (2021-11-30)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.10...v7.1.11)
+
+**Fixed bugs:**
+
+- Fix ruby reaper edge case [\#661](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/661) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- Question: Wait instead of cancelling if it is executing? [\#655](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/655)
+- Expired Locks remain in zset of digests \[using "until\_expired" lock\] [\#653](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/653)
+- \[Q&A\] Performance & Dead Locks [\#652](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/652)
+- Sidekiq 6.3.0 includes Job module that clashes with sidekiq\_unique\_ext.rb class Job [\#651](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/651)
+
+## [v7.1.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.10) (2021-10-18)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.8...v7.1.10)
+
+**Fixed bugs:**
+
+- "IndexError: string not matched" when job is replaced on client [\#635](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/635)
+
+**Merged pull requests:**
+
+- Update URL for Sidekiq's Enterprise unique jobs [\#648](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/648) ([jsantos](https://github.com/jsantos))
+
+## [v7.1.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.8) (2021-10-08)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.13...v7.1.8)
+
+**Fixed bugs:**
+
+- undefined method `delete' for class `Sidekiq::Job' [\#634](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/634)
+- INFO keys not persisted when job is enqueued [\#602](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/602)
+- lock\_info set to true but no lock info showing up in web ui [\#589](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/589)
+- Prevent too eager cleanup of lock info [\#645](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/645) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- Compatibility with unreleased Sidekiq 6.3.0 [\#636](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/636)
+
+**Merged pull requests:**
+
+- Update docs [\#644](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/644) ([andypple](https://github.com/andypple))
+
+## [v7.0.13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.13) (2021-09-27)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.7...v7.0.13)
+
+## [v7.1.7](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.7) (2021-09-27)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.6...v7.1.7)
+
+**Implemented enhancements:**
+
+- Styles [\#642](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/642) ([mhenrixon](https://github.com/mhenrixon))
+
+**Fixed bugs:**
+
+- OnConflict::Replace: yield when lock was achieved [\#640](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/640) ([mhenrixon](https://github.com/mhenrixon))
+
 ## [v7.1.6](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.6) (2021-09-21)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.5...v7.1.6)
@@ -125,7 +187,6 @@
 
 **Fixed bugs:**
 
-- lock\_info set to true but no lock info showing up in web ui [\#589](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/589)
 - Fix recording lock\_info [\#599](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/599) ([mhenrixon](https://github.com/mhenrixon))
 
 ## [v7.0.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.8) (2021-04-14)

data/README.md

@@ -1043,7 +1043,7 @@ There is a [![Join the chat at https://gitter.im/mhenrixon/sidekiq-unique-jobs](
 
 You can find a list of contributors over on [Contributors][]
 
-[Enterprise unique jobs]: https://www.dailydrip.com/topics/sidekiq/drips/sidekiq-enterprise-unique-jobs
+[Enterprise unique jobs]: https://github.com/mperham/sidekiq/wiki/Ent-Unique-Jobs
 [Contributors]: https://github.com/mhenrixon/sidekiq-unique-jobs/graphs/contributors
 [v4.0.18]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18
 [v5.0.10]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10.

data/lib/sidekiq_unique_jobs/digests.rb

@@ -31,7 +31,8 @@ module SidekiqUniqueJobs
     #
     # @param [String] pattern a key pattern to match with
     # @param [Integer] count the maximum number
-    # @return [Array<String>] with unique digests
+    # @return [Hash<String,Float>] Hash mapping of digest matching the given pattern and score
+
     def delete_by_pattern(pattern, count: DEFAULT_COUNT)
       result, elapsed = timed do
         digests = entries(pattern: pattern, count: count).keys
@@ -80,11 +81,7 @@ module SidekiqUniqueJobs
       options[:match] = pattern
       options[:count] = count
 
-      result = redis { |conn| conn.zscan_each(key, **options).to_a }
-
-      result.each_with_object({}) do |entry, hash|
-        hash[entry[0]] = entry[1]
-      end
+      redis { |conn| conn.zscan_each(key, **options).to_a }.to_h
     end
 
     #

data/lib/sidekiq_unique_jobs/lua/lock.lua

@@ -76,6 +76,9 @@ if pttl and pttl > 0 then
 
   log_debug("PEXPIRE", locked, pttl)
   redis.call("PEXPIRE", locked, pttl)
+
+  log_debug("PEXPIRE", info, pttl)
+  redis.call("PEXPIRE", info, pttl)
 end
 
 log_debug("PEXPIRE", queued, 1000)
@@ -84,9 +87,6 @@ redis.call("PEXPIRE", queued, 1000)
 log_debug("PEXPIRE", primed, 1000)
 redis.call("PEXPIRE", primed, 1000)
 
-log_debug("PEXPIRE", info, 1000)
-redis.call("PEXPIRE", info, 1000)
-
 log("Locked")
 log_debug("END lock digest:", digest, "job_id:", job_id)
 return job_id

data/lib/sidekiq_unique_jobs/lua/unlock.lua

@@ -65,11 +65,6 @@ redis.call("LREM", queued, -1, job_id)
 log_debug("LREM", primed, -1, job_id)
 redis.call("LREM", primed, -1, job_id)
 
-if limit and limit <= 1 and locked_count and locked_count <= 1 then
-  log_debug("ZREM", digests, digest)
-  redis.call("ZREM", digests, digest)
-end
-
 local redis_version = toversion(redisversion)
 local del_cmd       = "DEL"
 
@@ -83,6 +78,18 @@ if lock_type ~= "until_expired" then
   redis.call("HDEL", locked, job_id)
 end
 
+local locked_count = redis.call("HLEN", locked)
+
+if locked_count and locked_count < 1 then
+  log_debug(del_cmd, locked)
+  redis.call(del_cmd, locked)
+end
+
+if limit and limit <= 1 and locked_count and locked_count <= 1 then
+  log_debug("ZREM", digests, digest)
+  redis.call("ZREM", digests, digest)
+end
+
 log_debug("LPUSH", queued, "1")
 redis.call("LPUSH", queued, "1")
 

data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb

@@ -9,6 +9,7 @@ module SidekiqUniqueJobs
     #
     # @author Mikael Henriksson <mikael@mhenrixon.com>
     #
+    # rubocop:disable Metrics/ClassLength
     class RubyReaper < Reaper
       #
       # @return [String] the suffix for :RUN locks
@@ -54,13 +55,27 @@ module SidekiqUniqueJobs
       #
       # @return [Array<String>] an array of orphaned digests
       #
-      def orphans
-        conn.zrevrange(digests.key, 0, -1).each_with_object([]) do |digest, memo|
-          next if belongs_to_job?(digest)
+      def orphans # rubocop:disable Metrics/MethodLength
+        page = 0
+        per = reaper_count * 2
+        orphans = []
+        results = conn.zrange(digests.key, page * per, (page + 1) * per)
 
-          memo << digest
-          break if memo.size >= reaper_count
+        while results.size.positive?
+          results.each do |digest|
+            next if belongs_to_job?(digest)
+
+            orphans << digest
+            break if orphans.size >= reaper_count
+          end
+
+          break if orphans.size >= reaper_count
+
+          page += 1
+          results = conn.zrange(digests.key, page * per, (page + 1) * per)
         end
+
+        orphans
       end
 
       #
@@ -211,5 +226,6 @@ module SidekiqUniqueJobs
         conn.zscan_each(key, match: "*#{digest}*", count: 1).to_a.any?
       end
     end
+    # rubocop:enable Metrics/ClassLength
   end
 end

data/lib/sidekiq_unique_jobs/version.rb

@@ -3,5 +3,5 @@
 module SidekiqUniqueJobs
   #
   # @return [String] the current SidekiqUniqueJobs version
-  VERSION = "7.1.7"
+  VERSION = "7.1.12"
 end

data/lib/tasks/changelog.rake

@@ -16,7 +16,7 @@ COMMIT_CHANGELOG_CMD = "git commit -a -m 'Update changelog'"
 
 desc "Generate a Changelog"
 task :changelog do
-  sh("git checkout master")
+  sh("git checkout main")
   sh(*CHANGELOG_CMD.push(ENV["CHANGELOG_GITHUB_TOKEN"]))
   sh(ADD_CHANGELOG_CMD)
   sh(COMMIT_CHANGELOG_CMD)

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq-unique-jobs
 version: !ruby/object:Gem::Version
-  version: 7.1.7
+  version: 7.1.12
 platform: ruby
 authors:
 - Mikael Henriksson
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-09-27 00:00:00.000000000 Z
+date: 2021-12-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: brpoplpush-redis_script
@@ -215,15 +215,11 @@ homepage: https://mhenrixon.github.io/sidekiq-unique-jobs
 licenses:
 - MIT
 metadata:
-  homepage_uri: https://mhenrixon.github.io/sidekiq-unique-jobs
-  bug_tracker_uri: https://github.com/mhenrixon/sidekiq-unique-jobs/issues
-  documentation_uri: https://mhenrixon.github.io/sidekiq-unique-jobs
-  source_code_uri: https://github.com/mhenrixon/sidekiq-unique-jobs
-  changelog_uri: https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/CHANGELOG.md
+  rubygems_mfa_required: 'true'
 post_install_message: |
   IMPORTANT!
 
-  Automatic configuration of the sidekiq middelware is no longer done.
+  Automatic configuration of the sidekiq middleware is no longer done.
   Please see: https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/README.md#add-the-middleware
 
   This version deprecated the following sidekiq_options
@@ -261,7 +257,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.28
+rubygems_version: 3.2.32
 signing_key:
 specification_version: 4
 summary: Sidekiq middleware that prevents duplicates jobs