sidekiq-unique-jobs 8.0.5 → 8.0.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1573748a7de5be552d85b9e74143425fd99b2e85e4e6be1ccdbbd8ba9ecc0f37
-  data.tar.gz: 520fe1d492c9e088df5b6dc138d8bc4024516486c216e8223fe0f55462cfe691
+  metadata.gz: de442ce815fcfc00992295bd4160cfcf122815282e652cccca8c73ce1e62b351
+  data.tar.gz: f01d32c217f81f41abf34666c43acdddcb8148e6f7ca0df8146d408f9ed14f8e
 SHA512:
-  metadata.gz: 23542e9860f30b16f14c1907790e1bd491e98524dad73b726b88592b8e0cbcf06fa4b535837fc534e4ccf4a80d619840bdf7ec01c3104e3053f1ec5661ea92d5
-  data.tar.gz: d61c18630ab31064bac8ed9fad52fc84a08ad9bc384ac39d9ed8a44e6e4b741aafa0ebd2845a9f465da8e811e634dab0a8f92ee768c9bb19f9ba8cdb64b10190
+  metadata.gz: aec1ab15a3a3c1959bc137aaad674fa9e0854ca354b2226c352901f4d7d7c9cc6c8ef8d805b3b1ad680672eebca52a332de916a5e10d5f4d8b7b439bb677c7c0
+  data.tar.gz: 9af5d9ce6dda38757fb33aa33d86c337cd40751fc55bad37ad5c3f98cb04225209953843b7ec137e9295b8da2c2a40a640c7735f1d623c17851621b577b7a03e

data/CHANGELOG.md

@@ -1,8 +1,80 @@
 # Changelog
 
+## [v8.0.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.8) (2024-02-12)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.7...v8.0.8)
+
+**Implemented enhancements:**
+
+- fix: ensure a new lock isn't conflicting with itself [\#830](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/830) ([mhenrixon](https://github.com/mhenrixon))
+
+**Fixed bugs:**
+
+- until\_and\_while\_executing not entering perform method on initial run [\#824](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/824)
+- fix\(digest\): write digest on middleware call [\#774](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/774) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- incompatibility with sidekiq-failures [\#790](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/790)
+- Jobs queued during existing job inherit lock digest [\#766](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/766)
+
+## [v8.0.7](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.7) (2024-02-05)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.6...v8.0.7)
+
+**Implemented enhancements:**
+
+- chore\(ci\): add test coverage from bropoplpush [\#828](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/828) ([mhenrixon](https://github.com/mhenrixon))
+
+**Fixed bugs:**
+
+- fix\(xss\): sanitize parameters [\#829](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/829) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- No 'Changelog' link is being displayed on https://rubygems.org/gems/sidekiq-unique-jobs [\#825](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/825)
+- Using rspec matcher and getting: `NameError:        uninitialized constant SidekiqUniqueJobs::Lock::BaseLock::Validator` [\#741](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/741)
+
+**Merged pull requests:**
+
+- Fix Testing Instructions [\#827](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/827) ([jherdman](https://github.com/jherdman))
+
+## [v8.0.6](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.6) (2024-01-24)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.5...v8.0.6)
+
+**Implemented enhancements:**
+
+- Returning same job id [\#814](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/814)
+- fix: various minor issues [\#826](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/826) ([mhenrixon](https://github.com/mhenrixon))
+
+**Closed issues:**
+
+- should respond to `has_valid_sidekiq_options?` [\#822](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/822)
+- Reaper manager registration is subject to race conditions [\#801](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/801)
+- `while_executing` with `on_conflict: :reschedule` Reschedule job after job execution  [\#800](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/800)
+- Large retry queue causes reaper to run too slow [\#759](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/759)
+
+**Merged pull requests:**
+
+- fix: skip unless reaper was registered [\#820](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/820) ([mhenrixon](https://github.com/mhenrixon))
+
+## [v8.0.5](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.5) (2023-11-11)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.4...v8.0.5)
+
+**Merged pull requests:**
+
+- Bump @babel/traverse from 7.22.8 to 7.23.3 in /myapp [\#819](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/819) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump postcss from 8.4.21 to 8.4.31 in /myapp [\#811](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/811) ([dependabot[bot]](https://github.com/apps/dependabot))
+- fix: `while_executing` should not invoke conflict strategy when the job was successfully executed \[v8\] [\#810](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/810) ([cuzik](https://github.com/cuzik))
+- Bump actions/checkout from 3 to 4 [\#808](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/808) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump semver from 6.3.0 to 6.3.1 in /myapp [\#798](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/798) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Because `replace` is a client strategy, it should only remove client locks aka queue locks. [\#778](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/778) ([bigzed](https://github.com/bigzed))
+
 ## [v8.0.4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v8.0.4) (2023-11-11)
 
-[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.30...v8.0.4)
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.31...v8.0.4)
 
 **Implemented enhancements:**
 
@@ -18,9 +90,6 @@
 **Closed issues:**
 
 - register\_reaper\_process nx: true crash Sidekiq on startup [\#817](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/817)
-- Should client middleware also be added to the Sidekiq server config? [\#803](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/803)
-- 8.0.1 Time on locks & changelog UI is incorrect/wrong [\#761](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/761)
-- Job executed twice when reaper runs [\#738](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/738)
 
 **Merged pull requests:**
 
@@ -29,6 +98,16 @@
 - Handle strategy fallbacks properly [\#809](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/809) ([matejrisek](https://github.com/matejrisek))
 - Fix CI status badge [\#802](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/802) ([tagliala](https://github.com/tagliala))
 
+## [v7.1.31](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.31) (2023-08-28)
+
+[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.30...v7.1.31)
+
+**Closed issues:**
+
+- Should client middleware also be added to the Sidekiq server config? [\#803](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/803)
+- 8.0.1 Time on locks & changelog UI is incorrect/wrong [\#761](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/761)
+- Job executed twice when reaper runs [\#738](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/738)
+
 ## [v7.1.30](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.30) (2023-07-17)
 
 [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v8.0.3...v7.1.30)
@@ -335,7 +414,6 @@
 
 - Job finished, but lock is not cleared [\#677](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/677)
 - sidekiq\_options lock conflicts with sidekiq-lock gem lock option [\#669](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/669)
-- Slow evalsha causing timeouts [\#668](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/668)
 - Inconsistent documentation for config validation [\#647](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/647)
 
 **Merged pull requests:**

data/README.md

@@ -582,8 +582,6 @@ end
 #spec/workers/bad_worker_spec.rb
 
 require "sidekiq_unique_jobs/testing"
-#OR
-require "sidekiq_unique_jobs/rspec/matchers"
 
 RSpec.describe BadWorker do
   specify { expect(described_class).to have_valid_sidekiq_options }
@@ -816,7 +814,7 @@ In my benchmarks deleting 1000 orphaned locks with lua performs around 65% faste
 
 On the other hand if I increase it to 10 000 orphaned locks per cleanup (`reaper_count: 10_0000`) then redis starts throwing:
 
-> BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. (Redis::CommandError)
+> BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. (RedisClient::CommandError)
 
 If you want to disable the reaper set it to `:none`, `nil` or `false`. Actually, any value that isn't `:ruby` or `:lua` will disable the reaping.
 

data/lib/sidekiq_unique_jobs/batch_delete.rb

@@ -9,7 +9,7 @@ module SidekiqUniqueJobs
   class BatchDelete
     #
     # @return [Integer] the default batch size
-    BATCH_SIZE = 100
+    BATCH_SIZE = 500
 
     #
     # @return [Array<String>] Supported key suffixes

data/lib/sidekiq_unique_jobs/changelog.rb

@@ -59,7 +59,7 @@ module SidekiqUniqueJobs
         # NOTE: When debugging, check the last item in the returned array.
         [
           total_size.to_i,
-          result[0].to_i,  # next_cursor
+          result[0].to_i, # next_cursor
           result[1].map { |entry| load_json(entry) }.select { |entry| entry.is_a?(Hash) },
         ]
       end

data/lib/sidekiq_unique_jobs/config.rb

@@ -67,10 +67,10 @@ module SidekiqUniqueJobs
     # @return [Hash<Symbol, SidekiqUniqueJobs::Lock::BaseLock] all available default locks
     LOCKS =
       LOCKS_WHEN_BUSY.dup
-                     .merge(LOCKS_WHILE_ENQUEUED.dup)
-                     .merge(LOCKS_WITHOUT_UNLOCK.dup)
-                     .merge(LOCKS_FROM_PUSH_TO_PROCESSED.dup)
-                     .freeze
+        .merge(LOCKS_WHILE_ENQUEUED.dup)
+        .merge(LOCKS_WITHOUT_UNLOCK.dup)
+        .merge(LOCKS_FROM_PUSH_TO_PROCESSED.dup)
+        .freeze
 
     #
     # @return [Hash<Symbol, SidekiqUniqueJobs::OnConflict::Strategy] all available default strategies

data/lib/sidekiq_unique_jobs/constants.rb

@@ -15,6 +15,7 @@ module SidekiqUniqueJobs
   DEAD_VERSION          = "uniquejobs:dead"
   DIGESTS               = "uniquejobs:digests"
   EXPIRING_DIGESTS      = "uniquejobs:expiring_digests"
+  ORPHANED_DIGESTS      = "uniquejobs:orphaned_digests"
   ERRORS                = "errors"
   JID                   = "jid"
   LIMIT                 = "limit"

data/lib/sidekiq_unique_jobs/job.rb

@@ -41,19 +41,19 @@ module SidekiqUniqueJobs
     end
 
     def add_lock_timeout(item)
-      item[LOCK_TIMEOUT] ||= SidekiqUniqueJobs::LockTimeout.calculate(item)
+      item[LOCK_TIMEOUT] = SidekiqUniqueJobs::LockTimeout.calculate(item)
     end
 
     def add_lock_args(item)
-      item[LOCK_ARGS] ||= SidekiqUniqueJobs::LockArgs.call(item)
+      item[LOCK_ARGS] = SidekiqUniqueJobs::LockArgs.call(item)
     end
 
     def add_lock_digest(item)
-      item[LOCK_DIGEST] ||= SidekiqUniqueJobs::LockDigest.call(item)
+      item[LOCK_DIGEST] = SidekiqUniqueJobs::LockDigest.call(item)
     end
 
     def add_lock_prefix(item)
-      item[LOCK_PREFIX] ||= SidekiqUniqueJobs.config.lock_prefix
+      item[LOCK_PREFIX] = SidekiqUniqueJobs.config.lock_prefix
     end
 
     def add_lock_type(item)

data/lib/sidekiq_unique_jobs/lock/base_lock.rb

@@ -109,7 +109,7 @@ module SidekiqUniqueJobs
       #
       # Call whatever strategry that has been configured
       #
-      # @param [Symbol] origin: the origin `:client` or `:server`
+      # @param [Symbol] origin the origin `:client` or `:server`
       #
       # @return [void] the return value is irrelevant
       #

data/lib/sidekiq_unique_jobs/lock/while_executing.rb

@@ -11,6 +11,8 @@ module SidekiqUniqueJobs
     #
     # @author Mikael Henriksson <mikael@mhenrixon.com>
     class WhileExecuting < BaseLock
+      #
+      # @return [String] returns :RUN
       RUN_SUFFIX = ":RUN"
 
       include SidekiqUniqueJobs::OptionsWithFallback

data/lib/sidekiq_unique_jobs/lock.rb

@@ -33,9 +33,9 @@ module SidekiqUniqueJobs
     #
     # @return [Lock] a newly lock that has been locked
     #
-    def self.create(digest, job_id, lock_info = {})
-      lock = new(digest, time: Timing.now_f)
-      lock.lock(job_id, lock_info)
+    def self.create(digest, job_id, lock_info: {}, time: Timing.now_f, score: nil)
+      lock = new(digest, time: time)
+      lock.lock(job_id, lock_info, score)
       lock
     end
 
@@ -63,15 +63,16 @@ module SidekiqUniqueJobs
     #
     # @return [void]
     #
-    def lock(job_id, lock_info = {})
+    def lock(job_id, lock_info = {}, score = nil)
+      score ||= now_f
       redis do |conn|
         conn.multi do |pipeline|
           pipeline.set(key.digest, job_id)
           pipeline.hset(key.locked, job_id, now_f)
           info.set(lock_info, pipeline)
-          add_digest_to_set(pipeline, lock_info)
-          pipeline.zadd(key.changelog, now_f, changelog_json(job_id, "queue.lua", "Queued"))
-          pipeline.zadd(key.changelog, now_f, changelog_json(job_id, "lock.lua", "Locked"))
+          add_digest_to_set(pipeline, lock_info, score)
+          pipeline.zadd(key.changelog, score, changelog_json(job_id, "queue.lua", "Queued"))
+          pipeline.zadd(key.changelog, score, changelog_json(job_id, "lock.lua", "Locked"))
         end
       end
     end
@@ -333,12 +334,14 @@ module SidekiqUniqueJobs
     #
     # @return [nil]
     #
-    def add_digest_to_set(pipeline, lock_info)
+    def add_digest_to_set(pipeline, lock_info, score = nil)
+      score ||= now_f
       digest_string = key.digest
+
       if lock_info["lock"] == :until_expired
-        pipeline.zadd(key.expiring_digests, now_f + lock_info["ttl"], digest_string)
+        pipeline.zadd(key.expiring_digests, score + lock_info["ttl"], digest_string)
       else
-        pipeline.zadd(key.digests, now_f, digest_string)
+        pipeline.zadd(key.digests, score, digest_string)
       end
     end
   end

data/lib/sidekiq_unique_jobs/locksmith.rb

@@ -188,7 +188,7 @@ module SidekiqUniqueJobs
     #
     # @param [Sidekiq::RedisConnection, ConnectionPool] conn the redis connection
     # @param [Method] primed_method reference to the method to use for getting a primed token
-    # @param [nil, Integer, Float] time to wait before timeout
+    # @param [nil, Integer, Float] wait time to wait before timeout
     #
     # @yieldparam [string] job_id the sidekiq JID
     # @yieldreturn [void] whatever the calling block returns
@@ -258,8 +258,8 @@ module SidekiqUniqueJobs
 
       # NOTE: When debugging, change .value to .value!
       primed_jid = Concurrent::Promises
-                   .future(conn) { |red_con| pop_queued(red_con, timeout) }
-                   .value
+        .future(conn) { |red_con| pop_queued(red_con, timeout) }
+        .value
 
       handle_primed(primed_jid, &block)
     end

data/lib/sidekiq_unique_jobs/lua/unlock.lua

@@ -75,21 +75,28 @@ if lock_type ~= "until_expired" then
   redis.call("HDEL", locked, job_id)
 end
 
+if redis.call("LLEN", primed) == 0 then
+  log_debug("UNLINK", primed)
+  redis.call("UNLINK", primed)
+end
+
 local locked_count = redis.call("HLEN", locked)
 
-if locked_count and locked_count < 1 then
+if locked_count < 1 then
   log_debug("UNLINK", locked)
   redis.call("UNLINK", locked)
 end
 
-if redis.call("LLEN", primed) == 0 then
-  log_debug("UNLINK", primed)
-  redis.call("UNLINK", primed)
-end
-
-if limit and limit <= 1 and locked_count and locked_count <= 1 then
-  log_debug("ZREM", digests, digest)
-  redis.call("ZREM", digests, digest)
+if limit then
+  if limit <= 1 and locked_count <= 1 then
+    log_debug("ZREM", digests, digest)
+    redis.call("ZREM", digests, digest)
+  end
+else
+  if locked_count <= 1 then
+    log_debug("ZREM", digests, digest)
+    redis.call("ZREM", digests, digest)
+  end
 end
 
 log_debug("LPUSH", queued, "1")

data/lib/sidekiq_unique_jobs/middleware/client.rb

@@ -6,7 +6,7 @@ module SidekiqUniqueJobs
     #
     # @author Mikael Henriksson <mikael@mhenrixon.com>
     class Client
-      include Sidekiq::ClientMiddleware if defined?(Sidekiq::ClientMiddleware)
+      include Sidekiq::ClientMiddleware
 
       # prepend "SidekiqUniqueJobs::Middleware"
       # @!parse prepends SidekiqUniqueJobs::Middleware

data/lib/sidekiq_unique_jobs/middleware/server.rb

@@ -6,7 +6,7 @@ module SidekiqUniqueJobs
     #
     # @author Mikael Henriksson <mikael@mhenrixon.com>
     class Server
-      include Sidekiq::ServerMiddleware if defined?(Sidekiq::ServerMiddleware)
+      include Sidekiq::ServerMiddleware
 
       # prepend "SidekiqUniqueJobs::Middleware"
       # @!parse prepends SidekiqUniqueJobs::Middleware

data/lib/sidekiq_unique_jobs/middleware.rb

@@ -19,15 +19,15 @@ module SidekiqUniqueJobs
     # This method runs before (prepended) the actual middleware implementation.
     #   This is done to reduce duplication
     #
-    # @param [Sidekiq::Worker] worker_class
+    # @param [Sidekiq::Job] worker_class
     # @param [Hash] item a sidekiq job hash
     # @param [String] queue name of the queue
     # @param [ConnectionPool] redis_pool only used for compatility reasons
     #
-    # @return [yield<super>] <description>
+    # @return [yield<super>] call the rest of the middleware stack
     #
-    # @yieldparam [<type>] if <description>
-    # @yieldreturn [<type>] <describe what yield should return>
+    # @yieldparam [void] if uniquejobs is disable
+    # @yieldreturn [void] delegate back to other sidekiq middleware
     def call(worker_class, item, queue, redis_pool = nil)
       @item       = item
       @queue      = queue
@@ -35,7 +35,7 @@ module SidekiqUniqueJobs
       self.job_class = worker_class
       return yield if unique_disabled?
 
-      SidekiqUniqueJobs::Job.prepare(item) unless item[LOCK_DIGEST]
+      SidekiqUniqueJobs::Job.prepare(item)
 
       with_logging_context do
         super

data/lib/sidekiq_unique_jobs/orphans/manager.rb

@@ -39,12 +39,13 @@ module SidekiqUniqueJobs
         self.task = test_task || default_task
 
         with_logging_context do
-          register_reaper_process
-          log_info("Starting Reaper")
+          if register_reaper_process
+            log_info("Starting Reaper")
 
-          task.add_observer(Observer.new)
-          task.execute
-          task
+            task.add_observer(Observer.new)
+            task.execute
+            task
+          end
         end
       end
 

data/lib/sidekiq_unique_jobs/orphans/reaper.rb

@@ -22,11 +22,12 @@ module SidekiqUniqueJobs
       #
       # @return [Hash<Symbol, SidekiqUniqueJobs::Orphans::Reaper] the current implementation of reapers
       REAPERS = {
-        lua: SidekiqUniqueJobs::Orphans::LuaReaper,
+        lua: SidekiqUniqueJobs::Orphans::RubyReaper,
         ruby: SidekiqUniqueJobs::Orphans::RubyReaper,
         none: SidekiqUniqueJobs::Orphans::NullReaper,
         nil => SidekiqUniqueJobs::Orphans::NullReaper,
         false => SidekiqUniqueJobs::Orphans::NullReaper,
+        true => SidekiqUniqueJobs::Orphans::RubyReaper,
       }.freeze
 
       #

data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb

@@ -13,6 +13,9 @@ module SidekiqUniqueJobs
     class RubyReaper < Reaper
       include SidekiqUniqueJobs::Timing
 
+      #
+      # @return [Integer] a best guess of Sidekiq::Launcher::BEAT_PAUSE
+      SIDEKIQ_BEAT_PAUSE = 10
       #
       # @return [String] the suffix for :RUN locks
       RUN_SUFFIX = ":RUN"
@@ -74,14 +77,30 @@ module SidekiqUniqueJobs
 
         BatchDelete.call(expired_digests, conn)
         BatchDelete.call(orphans, conn)
+
+        # orphans.each_slice(500) do |chunk|
+        #   conn.pipelined do |pipeline|
+        #     chunk.each do |digest|
+        #       next if belongs_to_job?(digest)
+
+        #       pipeline.zadd(ORPHANED_DIGESTS, now_f, digest)
+        #     end
+        #   end
+        # end
       end
 
       def expired_digests
-        max_score = (start_time - reaper_timeout).to_f
-
         conn.zrange(EXPIRING_DIGESTS, 0, max_score, "byscore")
       end
 
+      def orphaned_digests
+        conn.zrange(ORPHANED_DIGESTS, 0, max_score, "byscore")
+      end
+
+      def max_score
+        (start_time - reaper_timeout - SIDEKIQ_BEAT_PAUSE).to_f
+      end
+
       #
       # Find orphaned digests
       #
@@ -89,10 +108,10 @@ module SidekiqUniqueJobs
       # @return [Array<String>] an array of orphaned digests
       #
       def orphans # rubocop:disable Metrics/MethodLength, Metrics/CyclomaticComplexity
-        page = 0
-        per = reaper_count * 2
         orphans = []
-        results = conn.zrange(digests.key, page * per, (page + 1) * per)
+        page    = 0
+        per     = reaper_count * 2
+        results = digests.byscore(0, max_score, offset: page * per, count: (page + 1) * per)
 
         while results.size.positive?
           results.each do |digest|
@@ -107,7 +126,7 @@ module SidekiqUniqueJobs
           break if orphans.size >= reaper_count
 
           page += 1
-          results = conn.zrange(digests.key, page * per, (page + 1) * per)
+          results = digests.byscore(0, max_score, offset: page * per, count: (page + 1) * per)
         end
 
         orphans
@@ -218,7 +237,7 @@ module SidekiqUniqueJobs
       end
 
       def considered_active?(time_f)
-        (Time.now - reaper_timeout).to_f < time_f
+        max_score < time_f
       end
 
       #

data/lib/sidekiq_unique_jobs/redis/sorted_set.rb

@@ -48,6 +48,14 @@ module SidekiqUniqueJobs
         end
       end
 
+      def byscore(min, max, offset: nil, count: nil)
+        redis do |conn|
+          return conn.zrange(key, min, max, "byscore") unless offset && count
+
+          conn.zrange(key, min, max, "byscore", "limit", offset, count)
+        end
+      end
+
       #
       # Return the zrak of the member
       #

data/lib/sidekiq_unique_jobs/script/caller.rb

@@ -56,7 +56,7 @@ module SidekiqUniqueJobs
       def do_call(file_name, conn, keys, argv)
         argv = argv.dup.push(now_f, debug_lua, max_history, file_name, redis_version)
 
-        Script.execute(file_name, conn, keys: keys, argv: normalize_argv(argv))
+        SidekiqUniqueJobs::Script.execute(file_name, conn, keys: keys, argv: normalize_argv(argv))
       end
 
       #

data/lib/sidekiq_unique_jobs/script/client.rb

@@ -0,0 +1,94 @@
+# frozen_string_literal: true
+
+module SidekiqUniqueJobs
+  module Script
+    # Interface to dealing with .lua files
+    #
+    # @author Mikael Henriksson <mikael@mhenrixon.com>
+    class Client
+      include SidekiqUniqueJobs::Script::Timing
+
+      #
+      # @!attribute [r] logger
+      #   @return [Logger] an instance of a logger
+      attr_reader :logger
+      #
+      # @!attribute [r] file_name
+      #   @return [String] The name of the file to execute
+      attr_reader :config
+      #
+      # @!attribute [r] scripts
+      #   @return [Scripts] the collection with loaded scripts
+      attr_reader :scripts
+
+      def initialize(config)
+        @config  = config
+        @logger  = config.logger
+        @scripts = Scripts.fetch(config.scripts_path)
+      end
+
+      #
+      # Execute a lua script with the provided script_name
+      #
+      # @note this method is recursive if we need to load a lua script
+      #   that wasn't previously loaded.
+      #
+      # @param [Symbol] script_name the name of the script to execute
+      # @param [Redis] conn the redis connection to use for execution
+      # @param [Array<String>] keys script keys
+      # @param [Array<Object>] argv script arguments
+      #
+      # @return value from script
+      #
+      def execute(script_name, conn, keys: [], argv: [])
+        result, elapsed = timed do
+          scripts.execute(script_name, conn, keys: keys, argv: argv)
+        end
+
+        logger.debug("Executed #{script_name}.lua in #{elapsed}ms")
+        result
+      rescue ::RedisClient::CommandError => ex
+        handle_error(script_name, conn, ex) do
+          execute(script_name, conn, keys: keys, argv: argv)
+        end
+      end
+
+      private
+
+      #
+      # Handle errors to allow retrying errors that need retrying
+      #
+      # @param [RedisClient::CommandError] ex exception to handle
+      #
+      # @return [void]
+      #
+      # @yieldreturn [void] yields back to the caller when NOSCRIPT is raised
+      def handle_error(script_name, conn, ex)
+        case ex.message
+        when /NOSCRIPT/
+          handle_noscript(script_name) { return yield }
+        when /BUSY/
+          handle_busy(conn) { return yield }
+        end
+
+        raise unless LuaError.intercepts?(ex)
+
+        script = scripts.fetch(script_name, conn)
+        raise LuaError.new(ex, script)
+      end
+
+      def handle_noscript(script_name)
+        scripts.delete(script_name)
+        yield
+      end
+
+      def handle_busy(conn)
+        scripts.kill(conn)
+      rescue ::RedisClient::CommandError => ex
+        logger.warn(ex)
+      ensure
+        yield
+      end
+    end
+  end
+end