activejob-uniqueness 0.1.3 → 0.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 881d2ffd39454784d884cc4049639e2e4455d854deec87217c55a95c4f4a5869
-  data.tar.gz: 469900d8be29af691e4efa9d883ba3d037dbbd06ac9ca7a6cf38b8cd0b2fb17d
+  metadata.gz: 3892ccb031a8fe8b0bc47154ed236dda2a700b404dbcc9063b83479fbe264dc7
+  data.tar.gz: 73a8f4866017e2310bc0c966a49d2f25578921ad64b2e23a2c28931923b34b04
 SHA512:
-  metadata.gz: c9907d17c12ff54e319d75253c7190a97b8114f7492e728d98b19ea94a14219722365f35e98422a3b2cd474d6aa14cb8c45bd5b7facfc68febdb541e14110c3a
-  data.tar.gz: a4f99ea028e1290f7742d67025b04069c0bec2912bdf4b455f0e422b91c411e8adb6e0ad2193c3b7a0067f2921cdb3b7fe7bf2ff3f251258e87b931956f96112
+  metadata.gz: a47bdbedbc98c95c24983ae4f46652f6c5f10584cfbe65312b4ee336a2d3fae21d03d6ea4c6b3fd09760cd5e9cfd3dde2fe02a89423e0918a6de0353a6f90c9b
+  data.tar.gz: ebfcf1e76b74662f5fb60008789b1858061c8685783a87317dff5818637064519894780f941c6a543687e02952747db2105e872efb95079e5b321a4f149e38bd

data/CHANGELOG.md

@@ -3,7 +3,34 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
-## [Unreleased](https://github.com/veeqo/activejob-uniqueness/compare/v0.1.3...HEAD)
+## [Unreleased](https://github.com/veeqo/activejob-uniqueness/compare/v0.2.2...HEAD)
+
+## [0.2.2](https://github.com/veeqo/activejob-uniqueness/compare/v0.2.1...v0.2.2) - 2021-10-22
+
+### Added
+- [#32](https://github.com/veeqo/activejob-uniqueness/pull/32) Add ability to set a custom runtime lock key for `:until_and_while_executing` strategy
+
+## [0.2.1](https://github.com/veeqo/activejob-uniqueness/compare/v0.2.0...v0.2.1) - 2021-08-24
+
+### Added
+- [#30](https://github.com/veeqo/activejob-uniqueness/pull/30) Add Sidekiq::JobRecord support (reported by [@dwightwatson](https://github.com/dwightwatson))
+
+## [0.2.0](https://github.com/veeqo/activejob-uniqueness/compare/v0.1.4...v0.2.0) - 2021-05-09
+
+### Added
+- [#22](https://github.com/veeqo/activejob-uniqueness/pull/22) Test with ruby 3.0.1
+
+### Changed
+- [#20](https://github.com/veeqo/activejob-uniqueness/pull/20) **Breaking** Sidekiq patch is not applied automatically anymore
+- [#21](https://github.com/veeqo/activejob-uniqueness/pull/21) Migrate from Travis to Github Actions
+- [#24](https://github.com/veeqo/activejob-uniqueness/pull/24) The default value for `retry_count` of redlock is now 0
+- Require ruby 2.5+
+
+## [0.1.4](https://github.com/veeqo/activejob-uniqueness/compare/v0.1.3...v0.1.4) - 2020-09-22
+
+### Fixed
+- [#11](https://github.com/veeqo/activejob-uniqueness/pull/11) Fix deprecation warnings for ruby 2.7 by [@DanAndreasson](https://github.com/DanAndreasson)
+- [#13](https://github.com/veeqo/activejob-uniqueness/pull/13) Fix deprecation warnings for ruby 2.7
 
 ## [0.1.3](https://github.com/veeqo/activejob-uniqueness/compare/v0.1.2...v0.1.3) - 2020-08-17
 

data/README.md

@@ -1,5 +1,5 @@
 # Job uniqueness for ActiveJob
-[![Build Status](https://travis-ci.com/veeqo/activejob-uniqueness.svg?branch=master)](https://travis-ci.com/veeqo/activejob-uniqueness) [![Gem Version](https://badge.fury.io/rb/activejob-uniqueness.svg)](https://badge.fury.io/rb/activejob-uniqueness)
+[![Build Status](https://github.com/veeqo/activejob-uniqueness/actions/workflows/main.yml/badge.svg?branch=main)](https://github.com/veeqo/activejob-uniqueness/actions/workflows/main.yml) [![Gem Version](https://badge.fury.io/rb/activejob-uniqueness.svg)](https://badge.fury.io/rb/activejob-uniqueness)
 
 The gem allows to protect job uniqueness with next strategies:
 
@@ -8,10 +8,16 @@ The gem allows to protect job uniqueness with next strategies:
 | `until_executing` | when **pushed** to the queue | when **processing starts** |
 | `until_executed` | when **pushed** to the queue | when the job is **processed successfully** |
 | `until_expired` | when **pushed** to the queue | when the lock is **expired** |
-| `until_and_while_executing` | when **pushed** to the queue | when **processing starts**<br>a runtime lock is acquired to **prevent simultaneous jobs** |
+| `until_and_while_executing` | when **pushed** to the queue | when **processing starts**<br>a runtime lock is acquired to **prevent simultaneous jobs**<br>*has extra options: `runtime_lock_ttl`, `on_runtime_conflict`* |
 | `while_executing` | when **processing starts** | when the job is **processed**<br>with any result including an error |
 
-Inspired by [SidekiqUniqueJobs](https://github.com/mhenrixon/sidekiq-unique-jobs), uses [Redlock](https://github.com/leandromoreira/redlock-rb) under the hood, sponsored by [Veeqo](https://www.veeqo.com/).
+Inspired by [SidekiqUniqueJobs](https://github.com/mhenrixon/sidekiq-unique-jobs), uses [Redlock](https://github.com/leandromoreira/redlock-rb) under the hood.
+
+<p align="center">
+  <a href="https://www.veeqo.com/" title="Sponsored by Veeqo">
+    <img src="https://static.veeqo.com/assets/sponsored_by_veeqo.png" width="360" />
+  </a>
+</p>
 
 ## Installation
 
@@ -21,6 +27,11 @@ Add the `activejob-uniqueness` gem to your Gemfile.
 gem 'activejob-uniqueness'
 ```
 
+If you want jobs unlocking for Sidekiq Web UI, require the patch explicitly. [**Queues cleanup becomes slower!**](#sidekiq-api-support)
+```ruby
+gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
+```
+
 And run `bundle install` command.
 
 ## Configuration
@@ -34,27 +45,87 @@ rails generate active_job:uniqueness:install
 
 ## Usage
 
-Define uniqueness strategy for your job via `unique` class method:
+
+### Make the job to be unique
+
+```ruby
+class MyJob < ActiveJob::Base
+  # new jobs with the same args will raise error until existing one is executed
+  unique :until_executed
+
+  def perform(args)
+    # work
+  end
+end
+```
+
+### Tune uniqueness settings per job
+
+```ruby
+class MyJob < ActiveJob::Base
+  # new jobs with the same args will be logged within 3 hours or until existing one is being executing
+  unique :until_executing, lock_ttl: 3.hours, on_conflict: :log
+
+  def perform(args)
+    # work
+  end
+end
+```
+
+You can set defaults globally with [the configuration](#configuration)
+
+### Control lock conflicts
+
+```ruby
+class MyJob < ActiveJob::Base
+  # Proc gets the job instance including its arguments
+  unique :until_executing, on_conflict: ->(job) { job.logger.info "Oops: #{job.arguments}" }
+
+  def perform(args)
+    # work
+  end
+end
+```
+
+### Control lock key arguments
 
 ```ruby
 class MyJob < ActiveJob::Base
   unique :until_executed
 
-  # Custom expiration:
-  # unique :until_executed, lock_ttl: 3.hours
+  def perform(foo, bar, baz)
+    # work
+  end
+
+  def lock_key_arguments
+    arguments.first(2) # baz is ignored
+  end
+end
+```
+
+### Control the lock key
+
+```ruby
+class MyJob < ActiveJob::Base
+  unique :until_executed
 
-  # Do not raise error on non unique jobs enqueuing:
-  # unique :until_executed, on_conflict: :log
+  def perform(foo, bar, baz)
+    # work
+  end
 
-  # Handle conflict by custom Proc:
-  # unique :until_executed, on_conflict: ->(job) { job.logger.info 'Oops' }
+  def lock_key
+    'qux' # completely custom lock key
+  end
 
-  # The :until_and_while_executing strategy supports extra attributes for a runtime lock:
-  # unique :until_and_while_executing runtime_lock_ttl: 10.minutes, on_runtime_conflict: :log
+  def runtime_lock_key
+    'quux' # completely custom runtime lock key for :until_and_while_executing
+  end
 end
 ```
 
-ActiveJob::Uniqueness allows to manually unlock jobs:
+### Unlock jobs manually
+
+The selected strategy automatically unlocks jobs, but in some cases (e.g. the queue is purged) it is handy to unlock jobs manually.
 
 ```ruby
 # Remove the lock for particular arguments:
@@ -91,7 +162,7 @@ ActiveJob::Uniqueness instruments `ActiveSupport::Notifications` with next event
 
 And then writes to `ActiveJob::Base.logger`.
 
-### ActiveJob prior to version `6.1` will always log `Enqueued MyJob (Job ID) ...` even if the callback chain was halted. [Details](https://github.com/rails/rails/pull/37830)
+**ActiveJob prior to version `6.1` will always log `Enqueued MyJob (Job ID) ...` even if the callback chain is halted. [Details](https://github.com/rails/rails/pull/37830)**
 
 ## Testing
 
@@ -107,15 +178,14 @@ bundle
 rake
 ```
 
-Use [wwtd](https://github.com/grosser/wwtd) to run test matrix:
-```sh
-gem install wwtd
-wwtd
-```
+## Sidekiq API support
 
-## Sidekiq adapter support
+ActiveJob::Uniqueness supports Sidekiq API to unset job locks on queues cleanup (e.g. via Sidekiq Web UI). Starting Sidekiq 5.1 job death also triggers locks cleanup.
+Take into account that **[big queues clanup becomes much slower](https://github.com/veeqo/activejob-uniqueness/issues/16)** because each job is being unlocked individually. In order to activate Sidekiq API patch require it explicitly in your Gemfile:
 
-ActiveJob::Uniqueness patches Sidekiq API to unset locks on jobs cleanup. Starting Sidekiq 5.1 job death also triggers locks cleanup.
+```ruby
+gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
+```
 
 ## Contributing
 
@@ -124,3 +194,9 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/veeqo/
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+## About [Veeqo](https://www.veeqo.com)
+
+At Veeqo, our team of Engineers is on a mission to create a world-class Inventory and Shipping platform, built to the highest standards in best coding practices. We are a growing team, looking for other passionate developers to [join us](https://veeqo-ltd.breezy.hr/) on our journey. If you're looking for a career working for one of the most exciting tech companies in ecommerce, we want to hear from you.
+
+[Veeqo developers blog](https://devs.veeqo.com)

data/lib/active_job/uniqueness/active_job_patch.rb

@@ -55,7 +55,7 @@ module ActiveJob
       end
 
       def lock_strategy
-        @lock_strategy ||= lock_strategy_class.new(**lock_options.merge(lock_key: lock_key, job: self))
+        @lock_strategy ||= lock_strategy_class.new(job: self)
       end
 
       # Override in your job class if you want to customize arguments set for a digest.
@@ -64,11 +64,11 @@ module ActiveJob
       end
 
       # Override lock_key method in your job class if you want to build completely custom lock key.
-      delegate :lock_key, to: :lock_key_generator
+      delegate :lock_key, :runtime_lock_key, to: :lock_key_generator
 
       def lock_key_generator
-        ActiveJob::Uniqueness::LockKey.new job_class_name: self.class.name,
-                                           arguments: lock_key_arguments
+        @lock_key_generator ||= ActiveJob::Uniqueness::LockKey.new job_class_name: self.class.name,
+                                                                   arguments: lock_key_arguments
       end
     end
 

data/lib/active_job/uniqueness/configuration.rb

@@ -11,14 +11,18 @@ module ActiveJob
     class Configuration
       include ActiveSupport::Configurable
 
-      config_accessor(:lock_ttl) { 1.day }
+      config_accessor(:lock_ttl) { 86_400 } # 1.day
       config_accessor(:lock_prefix) { 'activejob_uniqueness' }
       config_accessor(:on_conflict) { :raise }
-      config_accessor(:digest_method) { OpenSSL::Digest::MD5 }
       config_accessor(:redlock_servers) { [ENV.fetch('REDIS_URL', 'redis://localhost:6379')] }
-      config_accessor(:redlock_options) { {} }
+      config_accessor(:redlock_options) { { retry_count: 0 } }
       config_accessor(:lock_strategies) { {} }
 
+      config_accessor(:digest_method) do
+        require 'openssl'
+        OpenSSL::Digest::MD5
+      end
+
       def on_conflict=(action)
         validate_on_conflict_action!(action)
 

data/lib/active_job/uniqueness/lock_key.rb

@@ -28,6 +28,14 @@ module ActiveJob
         ].join(':')
       end
 
+      # used only by :until_and_while_executing strategy
+      def runtime_lock_key
+        [
+          lock_key,
+          'runtime'
+        ].join(':')
+      end
+
       def wildcard_key
         [
           lock_prefix,

data/lib/active_job/uniqueness/sidekiq_patch.rb

@@ -1,5 +1,8 @@
 # frozen_string_literal: true
 
+require 'activejob/uniqueness'
+require 'sidekiq/api'
+
 module ActiveJob
   module Uniqueness
     def self.unlock_sidekiq_job!(job_data)
@@ -73,12 +76,21 @@ end
 
 Sidekiq::SortedEntry.prepend ActiveJob::Uniqueness::SidekiqPatch::SortedEntry
 Sidekiq::ScheduledSet.prepend ActiveJob::Uniqueness::SidekiqPatch::ScheduledSet
-Sidekiq::Job.prepend ActiveJob::Uniqueness::SidekiqPatch::Job
 Sidekiq::Queue.prepend ActiveJob::Uniqueness::SidekiqPatch::Queue
 Sidekiq::JobSet.prepend ActiveJob::Uniqueness::SidekiqPatch::JobSet
 
+sidekiq_version = Gem::Version.new(Sidekiq::VERSION)
+
+# Sidekiq 6.2.2 renames Sidekiq::Job to Sidekiq::JobRecord
+# https://github.com/mperham/sidekiq/issues/4955
+if sidekiq_version >= Gem::Version.new('6.2.2')
+  Sidekiq::JobRecord.prepend ActiveJob::Uniqueness::SidekiqPatch::Job
+else
+  Sidekiq::Job.prepend ActiveJob::Uniqueness::SidekiqPatch::Job
+end
+
 # Global death handlers are introduced in Sidekiq 5.1
 # https://github.com/mperham/sidekiq/blob/e7acb124fbeb0bece0a7c3d657c39a9cc18d72c6/Changes.md#510
-if Gem::Version.new(Sidekiq::VERSION) >= Gem::Version.new('5.1')
+if sidekiq_version >= Gem::Version.new('5.1')
   Sidekiq.death_handlers << ->(job, _ex) { ActiveJob::Uniqueness.unlock_sidekiq_job!(job) }
 end

data/lib/active_job/uniqueness/strategies/base.rb

@@ -13,10 +13,10 @@ module ActiveJob
 
         attr_reader :lock_key, :lock_ttl, :on_conflict, :job
 
-        def initialize(lock_key:, lock_ttl: nil, on_conflict: nil, job: nil)
-          @lock_key = lock_key
-          @lock_ttl = (lock_ttl || config.lock_ttl).to_i * 1000 # ms
-          @on_conflict = on_conflict || config.on_conflict
+        def initialize(job:)
+          @lock_key = job.lock_key
+          @lock_ttl = (job.lock_options[:lock_ttl] || config.lock_ttl).to_i * 1000 # ms
+          @on_conflict = job.lock_options[:on_conflict] || config.on_conflict
           @job = job
         end
 

data/lib/active_job/uniqueness/strategies/until_and_while_executing.rb

@@ -9,12 +9,16 @@ module ActiveJob
       class UntilAndWhileExecuting < Base
         include LockingOnEnqueue
 
-        attr_reader :runtime_lock_ttl, :on_runtime_conflict
+        attr_reader :runtime_lock_key, :runtime_lock_ttl, :on_runtime_conflict
 
-        def initialize(runtime_lock_ttl: nil, on_runtime_conflict: nil, **params)
-          super(params)
-          @runtime_lock_ttl = runtime_lock_ttl.present? ? runtime_lock_ttl.to_i * 1000 : lock_ttl
-          @on_runtime_conflict = on_runtime_conflict || on_conflict
+        def initialize(job:)
+          super
+          @runtime_lock_key = job.runtime_lock_key
+
+          runtime_lock_ttl_option = job.lock_options[:runtime_lock_ttl]
+          @runtime_lock_ttl = runtime_lock_ttl_option.present? ? runtime_lock_ttl_option.to_i * 1000 : lock_ttl
+
+          @on_runtime_conflict = job.lock_options[:on_runtime_conflict] || on_conflict
         end
 
         def before_perform
@@ -33,10 +37,6 @@ module ActiveJob
         ensure
           unlock(resource: runtime_lock_key, event: :runtime_unlock) unless @job_aborted
         end
-
-        def runtime_lock_key
-          [lock_key, 'runtime'].join(':')
-        end
       end
     end
   end

data/lib/active_job/uniqueness/version.rb

@@ -2,6 +2,6 @@
 
 module ActiveJob
   module Uniqueness
-    VERSION = '0.1.3'
+    VERSION = '0.2.2'
   end
 end

data/lib/active_job/uniqueness.rb

@@ -1,22 +1,13 @@
 # frozen_string_literal: true
 
-require 'active_support'
-require 'active_support/core_ext'
+require 'active_job'
 require 'redlock'
-require 'openssl'
+
 require 'active_job/uniqueness/version'
 require 'active_job/uniqueness/errors'
 require 'active_job/uniqueness/log_subscriber'
 require 'active_job/uniqueness/active_job_patch'
 
-begin
-  require 'sidekiq/api'
-rescue LoadError
-  # ¯\_(ツ)_/¯
-else
-  require 'active_job/uniqueness/sidekiq_patch'
-end
-
 module ActiveJob
   module Uniqueness
     extend ActiveSupport::Autoload
@@ -40,8 +31,8 @@ module ActiveJob
         @lock_manager ||= ActiveJob::Uniqueness::LockManager.new(config.redlock_servers, config.redlock_options)
       end
 
-      def unlock!(*args)
-        lock_manager.delete_locks(ActiveJob::Uniqueness::LockKey.new(*args).wildcard_key)
+      def unlock!(**args)
+        lock_manager.delete_locks(ActiveJob::Uniqueness::LockKey.new(**args).wildcard_key)
       end
 
       def test_mode!

data/lib/generators/active_job/uniqueness/templates/config/initializers/active_job_uniqueness.rb

@@ -31,7 +31,7 @@ ActiveJob::Uniqueness.configure do |config|
   # Custom options for Redlock.
   # Read more at https://github.com/leandromoreira/redlock-rb#redlock-configuration
   #
-  # config.redlock_options = {}
+  # config.redlock_options = { retry_count: 0 }
 
   # Custom strategies.
   # config.lock_strategies = { my_strategy: MyStrategy }

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: activejob-uniqueness
 version: !ruby/object:Gem::Version
-  version: 0.1.3
+  version: 0.2.2
 platform: ruby
 authors:
 - Rustam Sharshenov
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-08-17 00:00:00.000000000 Z
+date: 2021-10-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -142,7 +142,7 @@ licenses:
 metadata:
   homepage_uri: https://github.com/veeqo/activejob-uniqueness
   source_code_uri: https://github.com/veeqo/activejob-uniqueness
-  changelog_uri: https://github.com/veeqo/activejob-uniqueness/blob/master/CHANGELOG.md
+  changelog_uri: https://github.com/veeqo/activejob-uniqueness/blob/main/CHANGELOG.md
 post_install_message: 
 rdoc_options: []
 require_paths:
@@ -151,14 +151,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: '2.5'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.6
+rubygems_version: 3.0.9
 signing_key: 
 specification_version: 4
 summary: Ensure uniqueness of your ActiveJob jobs