sidekiq-locking 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 920668894edb18696853798031f13f8192595b6f7c82f1e2902a6f7219bef21e
+  data.tar.gz: f7173b85a3f6b4d9524c312244f2dc90eb91869203a3b0049029e5d919d37262
+SHA512:
+  metadata.gz: a89b55c2dac277700ec9e59b9fd90faba733117ebad38a63ee1fd00f3030c449756bb226c11483196dcf1d557fb5c26eabc7cbf56036e893dd4a25987fc2693c
+  data.tar.gz: 86be7fb1fe072579f1682eec4b11d9d17d27e106755de55a2b583f752b3b516b810282997c73405c47315e47707c328248e8f669407c3a28fdfb43cbe8310534

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 BigBinary Technologies Pvt. Ltd.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,225 @@
+# sidekiq-locking
+
+Best-effort, **TTL-bounded enqueue deduplication** for Sidekiq jobs — drop
+duplicate enqueues of the same job while a short-lived lock is held, **without a
+database or a runtime mutex**.
+
+If application code enqueues the same work repeatedly — a webhook storm, a
+fan-out that re-triggers the same sync, a retry loop upstream — only the first
+copy is kept in Redis while the lock is active. A duplicate is skipped when it
+matches the same lock context: `(class, queue, args)` by default.
+
+This is intentionally **best-effort deduplication, not a correctness primitive.**
+It is not a distributed lock and not a runtime mutex: two copies can still run
+concurrently if the TTL expires before the first finishes. Jobs must stay
+idempotent, and business-critical uniqueness should be protected with database
+constraints, row locks, or advisory locks.
+
+> **Naming.** The gem is `sidekiq-locking` and the namespace is `Sidekiq::Locking`,
+> with a `lock_for` DSL. The word "lock" somewhat overclaims — this is
+> best-effort, TTL-bounded enqueue *deduplication*, not a true mutex. The name
+> and DSL are kept stable for now and may be revisited.
+
+## Installation
+
+```ruby
+gem "sidekiq-locking"
+```
+
+```sh
+bundle install
+```
+
+Requires Ruby >= 3.1 and Sidekiq >= 7.0. The gem depends only on `sidekiq` — no
+Rails or ActiveSupport required.
+
+## Quick start
+
+Install the middleware once, in the host application's Sidekiq initializer:
+
+```ruby
+# config/initializers/sidekiq.rb
+require "sidekiq-locking"
+
+Sidekiq::Locking.install! unless Rails.env.test?
+```
+
+`install!` registers the client middleware (acquires the lock and skips
+duplicate pushes), the server middleware (releases the lock around `perform`),
+and a death handler (releases the lock when a job is discarded). Disabling it in
+test keeps tests deterministic and avoids locks leaking between examples.
+
+Then opt a job in with a lock window:
+
+```ruby
+class SyncAddressJob
+  include Sidekiq::Job
+
+  sidekiq_options lock_for: 5.minutes
+
+  def perform(user_id)
+    # sync user address
+  end
+end
+```
+
+A second `SyncAddressJob` with the same queue and args cannot be enqueued while
+the first lock is active. The lock is released when the job succeeds, or when
+`lock_for` expires — whichever happens first. Keep `lock_for` short; a window
+longer than a few minutes usually means the job wants stronger application-level
+concurrency control.
+
+## Lock context
+
+By default jobs are deduplicated by `[class, queue, args]`, so the same job
+class with the same args can still be enqueued on different queues:
+
+```ruby
+SyncAddressJob.set(queue: "default").perform_async(1)
+SyncAddressJob.set(queue: "low").perform_async(1) # different lock — queue differs
+```
+
+### Custom lock args
+
+Use a `lock_args(job)` class method when only part of the args should
+participate in the lock key. It must return an `Array`; if it is absent, the
+full args array is used.
+
+```ruby
+class RefreshAccountCacheJob
+  include Sidekiq::Job
+
+  sidekiq_options lock_for: 5.minutes
+
+  # Only account_id participates in the lock; the reason arg is ignored.
+  def self.lock_args(job)
+    [job["args"].first]
+  end
+
+  def perform(account_id, reason = nil)
+    # refresh account cache
+  end
+end
+
+# These two share one lock:
+RefreshAccountCacheJob.perform_async(42, "user_updated")
+RefreshAccountCacheJob.perform_async(42, "manual_refresh")
+```
+
+### ActiveJob
+
+`Sidekiq::Locking` unwraps Sidekiq's ActiveJob wrapper and keys on the wrapped job
+class plus the real ActiveJob arguments. A wrapped job that defines
+`lock_args(job)` receives the unwrapped args.
+
+## Scheduled jobs
+
+Scheduled jobs include the delay in the lock TTL. With `lock_for: 10.minutes`:
+
+```ruby
+SyncAddressJob.set(wait: 1.hour).perform_async(1)
+```
+
+the lock lasts ~70 minutes total (the one-hour delay plus the ten-minute
+window), so a duplicate cannot be enqueued for that full period unless the
+original runs and releases the lock earlier.
+
+## Retries and the unlock policy
+
+`release_lock` controls when the lock is released.
+
+- **`:after_perform` (default)** — release only after `perform` returns
+  successfully. A failed job keeps its lock while it waits to retry, so a
+  transient failure does not let a duplicate slip in. The lock clears on
+  success, on death (via the death handler), or on TTL expiry.
+
+  ```ruby
+  sidekiq_options lock_for: 5.minutes, release_lock: :after_perform
+  ```
+
+- **`:before_perform`** — release immediately before the job starts running.
+  Use this only when it is acceptable for another copy to be enqueued while the
+  current job is still executing.
+
+  ```ruby
+  sidekiq_options lock_for: 5.minutes, release_lock: :before_perform
+  ```
+
+If the TTL expires while a job is still retrying, a duplicate can be enqueued.
+Design jobs with that best-effort behavior in mind.
+
+## Per-enqueue overrides
+
+Use Sidekiq's `set` API to disable or override the lock for one enqueue:
+
+```ruby
+SyncAddressJob.set(lock_for: false).perform_async(1)      # no lock this time
+SyncAddressJob.set(lock_for: 30.seconds).perform_async(1) # shorter window
+SyncAddressJob.set(lock_for: 10.minutes).perform_async(1) # longer window
+```
+
+## Return value on a duplicate enqueue
+
+A duplicate enqueue does not return a JID because the job was never pushed.
+Through Sidekiq's public `perform_async` API this is observed as `nil`:
+
+```ruby
+SyncAddressJob.perform_async(1) # => "abc123..."
+SyncAddressJob.perform_async(1) # => nil   (duplicate, lock held)
+```
+
+Do not rely on `perform_async` always returning a JID for lockable jobs. Each
+skipped duplicate is logged at info level —
+`Skipping enqueue for <Class>, lock held by JID <jid>`.
+
+## Redis key prefix
+
+Each active lock is one Redis string at `sidekiq:locking:<digest>`, whose value is
+the owning job's JID (releases are JID-guarded so a late finisher cannot delete
+a newer lock). The prefix is configurable:
+
+```ruby
+Sidekiq::Locking.redis_key_prefix = "my-app:sidekiq:locking:"
+```
+
+## How it works
+
+```text
+perform_async
+  → client middleware builds a lock key from (class, queue, args)
+  → Redis: SET sidekiq:locking:<digest> <jid> GET NX PX <ttl_ms>
+
+  Acquired (key was new):
+    → stamp job["lock_token"] = digest, push the job
+    → server middleware releases the lock on success (per release_lock)
+    → death handler releases the lock if the job is discarded
+
+  Already held:
+    → do not push; perform_async returns nil; log the skip
+```
+
+If a *downstream* client middleware aborts the push by returning `false` (for
+example a blackhole route from
+[sidekiq-routing](https://github.com/neetozone/sidekiq-routing)), the lock
+middleware releases the lock it just acquired so the next identical enqueue is
+not blocked until the TTL. When composing with other client middleware, the
+lock middleware should run **before** any middleware that may abort the push.
+
+## Caveats
+
+- Locks are best-effort and TTL-bounded; jobs must remain idempotent.
+- This is enqueue deduplication, not execution-time mutual exclusion — if the
+  TTL expires while a job is still running or retrying, another copy can be
+  enqueued.
+- Duplicate jobs are dropped, not rescheduled or replaced.
+- There is no Web UI, lock browser, or stale-lock reaper — the TTL is the safety
+  mechanism.
+- `release_lock: :before_perform` allows duplicates while the job is running.
+- Manually deleting jobs from Sidekiq queues may leave the lock in Redis until
+  the TTL expires.
+- The lock adds one Redis round-trip before each push; if Redis is unavailable,
+  enqueue can fail with a Redis/network error.
+
+## License
+
+Released under the [MIT License](LICENSE).

data/lib/sidekiq/locking/acquirer.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module Locking
+    # Claims a lock with `SET key <jid> NX PX <ttl>` plus the GET option — the
+    # check-and-set lock recipe from the Redis SET docs
+    # (https://redis.io/commands/set/). The acquiring job's JID is stored as the
+    # value so the release can later be guarded by ownership.
+    module Acquirer
+      Held = Struct.new(:jid) do
+        def held?
+          true
+        end
+      end
+
+      class << self
+        def try_acquire(key, jid, ttl_ms, redis_pool = nil)
+          setter = ->(conn) { conn.set(key.redis_key, jid, "get", "nx", "px", ttl_ms) }
+          holder = redis_pool ? redis_pool.with(&setter) : Sidekiq.redis(&setter)
+
+          holder ? Held.new(holder) : :acquired
+        end
+      end
+    end
+  end
+end

data/lib/sidekiq/locking/key.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+require "digest/sha2"
+require "json"
+
+module Sidekiq
+  module Locking
+    # Value object for a lock key. Builds the uniqueness tuple (job class, queue,
+    # args) from the Sidekiq payload and hashes it into the Redis key shared by
+    # the acquirer and releaser. The hash encoding (SHA-256 hex over a JSON dump
+    # of the tuple) is an internal implementation detail — it only has to be
+    # stable and collision-resistant for a given tuple.
+    class Key
+      ACTIVE_JOB_WRAPPERS = [
+        "Sidekiq::ActiveJob::Wrapper",
+        "ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper"
+      ].freeze
+
+      attr_reader :job, :klass, :queue, :args, :context, :digest
+
+      def initialize(job)
+        @job = job
+        @klass = job["wrapped"] || job["class"]
+        @queue = job["queue"]
+        @args = resolved_args
+        @context = [klass, queue, args]
+        @digest = Digest::SHA256.hexdigest(JSON.generate(context))
+      end
+
+      def redis_key
+        "#{Locking.redis_key_prefix}#{digest}"
+      end
+
+      private
+
+        def resolved_args
+          job_class = constantized_job_class
+          return default_args unless job_class&.respond_to?(:lock_args)
+
+          scratch = job.merge("class" => klass, "args" => copy_args(default_args))
+          result = job_class.lock_args(scratch)
+          unless result.is_a?(Array)
+            raise TypeError, "#{job_class.name}.lock_args must return an Array (got #{result.class})"
+          end
+
+          result
+        end
+
+        def default_args
+          if ACTIVE_JOB_WRAPPERS.include?(job["class"])
+            job.dig("args", 0, "arguments") || []
+          else
+            job["args"] || []
+          end
+        end
+
+        def constantized_job_class
+          Object.const_get(klass)
+        rescue NameError
+          nil
+        end
+
+        def copy_args(value)
+          value.respond_to?(:dup) ? value.dup : value
+        end
+    end
+  end
+end

data/lib/sidekiq/locking/middleware/client.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module Locking
+    module Middleware
+      # Acquires the lock before push. Skips the push (returns false) if a lock
+      # is already held for the same (class, queue, lock_args) tuple.
+      #
+      # Scheduled jobs (`perform_in`) include their delay in the TTL: a job
+      # scheduled an hour out with `lock_for: 10.minutes` holds the lock for 70
+      # minutes total.
+      class Client
+        def call(_worker_class, job, _queue, redis_pool = nil)
+          return yield unless lockable?(job)
+
+          expiry_ms = expiry_for(job)
+          if expiry_ms <= 0
+            Sidekiq.logger.info("Skipping lock for #{job["class"]}/#{job["jid"]}: lock window ends in the past")
+            return yield
+          end
+
+          key = Key.new(job)
+          result = Acquirer.try_acquire(key, job["jid"], expiry_ms, redis_pool)
+
+          if result == :acquired
+            job[TOKEN_KEY] = key.digest
+            pushed = yield
+            # If a downstream middleware aborted the push (e.g. Sidekiq::Routing's
+            # blackhole returns false) the job never reaches Redis but our lock
+            # is held. Release it so the next identical enqueue isn't blocked
+            # until the TTL expires.
+            Releaser.release(key.digest, job["jid"]) if pushed == false
+            pushed
+          else
+            klass = job["wrapped"] || job["class"]
+            Sidekiq.logger.info { "Skipping enqueue for #{klass}, lock held by JID #{result.jid}" }
+            false
+          end
+        end
+
+        private
+
+          def lockable?(job)
+            job[LOCK_FOR_KEY] && !job.key?(TOKEN_KEY) && job[LOCK_FOR_KEY].to_i >= 0
+          end
+
+          # How long the lock should live, in milliseconds. The base window is
+          # `lock_for` seconds; a scheduled job ("at" in the future) stays locked
+          # until it actually runs, so the remaining delay is added on top.
+          # Immediate jobs carry no delay.
+          def expiry_for(job)
+            window_seconds = job[LOCK_FOR_KEY].to_i
+            run_at = job["at"]
+            delay_seconds = run_at ? run_at - Time.now.to_f : 0.0
+
+            ((window_seconds + delay_seconds) * 1000).to_i
+          end
+      end
+    end
+  end
+end

data/lib/sidekiq/locking/middleware/server.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module Locking
+    module Middleware
+      # Releases the lock around perform.
+      #
+      #   release_lock: :after_perform (default) — release after successful
+      #     perform. Errored jobs keep the lock through retries until success or
+      #     death.
+      #   release_lock: :before_perform — release just before perform begins.
+      class Server
+        def call(_worker_class, job, _queue)
+          return yield unless locked?(job)
+
+          if release_before_perform?(job)
+            release(job)
+            yield
+          else
+            yield
+            release(job)
+          end
+        end
+
+        private
+
+          def locked?(job)
+            job[LOCK_FOR_KEY] && job[LOCK_FOR_KEY].to_i >= 0 && job[TOKEN_KEY]
+          end
+
+          def release_before_perform?(job)
+            job.fetch(RELEASE_LOCK_KEY, RELEASE_AFTER_PERFORM).to_s == RELEASE_BEFORE_PERFORM
+          end
+
+          def release(job)
+            Releaser.release(job[TOKEN_KEY], job["jid"])
+          end
+      end
+    end
+  end
+end

data/lib/sidekiq/locking/releaser.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module Locking
+    # Deletes a lock only when the Redis value still equals the releasing job's
+    # JID, so a late finisher cannot delete a newer lock acquired after the
+    # original TTL expired. This compare-and-delete is the standard safe-release
+    # pattern from the Redis SET docs (https://redis.io/commands/set/).
+    module Releaser
+      UNLOCK_SCRIPT = <<~LUA
+        if redis.call('get', KEYS[1]) == ARGV[1] then
+          redis.call('del', KEYS[1])
+        end
+      LUA
+
+      class << self
+        def release(digest, jid)
+          return unless digest && jid
+
+          Sidekiq.redis do |conn|
+            conn.call("EVAL", UNLOCK_SCRIPT, 1, "#{Locking.redis_key_prefix}#{digest}", jid)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/sidekiq/locking/version.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module Locking
+    VERSION = "0.1.0"
+  end
+end

data/lib/sidekiq/locking.rb

@@ -0,0 +1,80 @@
+# frozen_string_literal: true
+
+require "sidekiq"
+
+# Sidekiq::Locking — best-effort, TTL-bounded deduplication of Sidekiq enqueues.
+#
+# While a lock is held for a job, a second enqueue of the same job is dropped.
+# Treat it as a pruning convenience, never a guarantee — the lock expires on a
+# TTL and the check is inherently racy, so keep jobs idempotent and enforce real
+# uniqueness in your datastore.
+#
+# The Redis building blocks used here — a token-valued `SET ... NX PX` to claim
+# the lock and a compare-and-delete to release it — are the standard recipes from
+# the Redis SET documentation (https://redis.io/commands/set/).
+#
+# Opt a job in with a lock window, and optionally narrow the dedup scope:
+#
+#   # config/initializers/sidekiq.rb
+#   Sidekiq::Locking.install! unless Rails.env.test?
+#
+#   class MyJob
+#     include Sidekiq::Job
+#     sidekiq_options lock_for: 5.minutes
+#
+#     def self.lock_args(job)
+#       [job["args"].first]
+#     end
+#   end
+module Sidekiq
+  module Locking
+    LOCK_FOR_KEY = "lock_for"
+    RELEASE_LOCK_KEY = "release_lock"
+    TOKEN_KEY = "lock_token"
+
+    RELEASE_AFTER_PERFORM = "after_perform"
+    RELEASE_BEFORE_PERFORM = "before_perform"
+
+    DEFAULT_REDIS_KEY_PREFIX = "sidekiq:locking:"
+
+    class << self
+      attr_writer :redis_key_prefix
+
+      def redis_key_prefix
+        @redis_key_prefix || DEFAULT_REDIS_KEY_PREFIX
+      end
+
+      # Registers the client + server middleware on both client and server
+      # configurations, plus a death handler so dead jobs release their lock.
+      def install!
+        Sidekiq.configure_client do |config|
+          config.client_middleware do |chain|
+            chain.add Middleware::Client
+          end
+          # ensure `perform_inline` releases locks too
+          config.server_middleware do |chain|
+            chain.add Middleware::Server
+          end
+        end
+
+        Sidekiq.configure_server do |config|
+          config.client_middleware do |chain|
+            chain.add Middleware::Client
+          end
+          config.server_middleware do |chain|
+            chain.add Middleware::Server
+          end
+          config.death_handlers << ->(job, _ex) do
+            Releaser.release(job[TOKEN_KEY], job["jid"]) if job[TOKEN_KEY]
+          end
+        end
+      end
+    end
+  end
+end
+
+require "sidekiq/locking/key"
+require "sidekiq/locking/acquirer"
+require "sidekiq/locking/releaser"
+require "sidekiq/locking/middleware/client"
+require "sidekiq/locking/middleware/server"

data/lib/sidekiq-locking.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+require "sidekiq/locking/version"
+
+# Sidekiq::Locking: TTL-bounded, best-effort enqueue deduplication for Sidekiq jobs.
+#
+# Defines Sidekiq::Locking and its public API (install!, redis_key_prefix). The
+# module file requires its own collaborators (Key, Acquirer, Releaser, and the
+# client/server middleware), so requiring it here loads the whole subsystem.
+#
+# The per-job DSL (`lock_for`, `release_lock`, `lock_args`) is unchanged. "Lock"
+# somewhat overclaims: this is best-effort dedup, not a runtime mutex (see README).
+require "sidekiq/locking"

metadata

@@ -0,0 +1,79 @@
+--- !ruby/object:Gem::Specification
+name: sidekiq-locking
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Vishnu M
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2026-09-17 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: sidekiq
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+description: |
+  sidekiq-locking keeps only the first copy of a job in Redis while a
+  short-lived lock is active. A duplicate enqueue for the same lock context
+  (class, queue, args by default) is skipped until the original job succeeds
+  or the lock_for TTL expires — whichever comes first. It is deliberately
+  best-effort enqueue coalescing, not a correctness primitive or a runtime
+  mutex: jobs must still be idempotent and protect true uniqueness with
+  database constraints/locks where required. Opt a job in with
+  `sidekiq_options lock_for: 5.minutes`; customize the dedup scope with a
+  `lock_args` class method. No Rails or ActiveSupport required.
+email:
+- vishnu.m@bigbinary.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- LICENSE
+- README.md
+- lib/sidekiq-locking.rb
+- lib/sidekiq/locking.rb
+- lib/sidekiq/locking/acquirer.rb
+- lib/sidekiq/locking/key.rb
+- lib/sidekiq/locking/middleware/client.rb
+- lib/sidekiq/locking/middleware/server.rb
+- lib/sidekiq/locking/releaser.rb
+- lib/sidekiq/locking/version.rb
+homepage: https://github.com/neetozone/sidekiq-locking
+licenses:
+- MIT
+metadata:
+  source_code_uri: https://github.com/neetozone/sidekiq-locking
+  rubygems_mfa_required: 'true'
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.1'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.5.22
+signing_key:
+specification_version: 4
+summary: 'Best-effort, TTL-bounded enqueue deduplication for Sidekiq jobs: drop duplicate
+  enqueues of the same job while a lock is held.'
+test_files: []