activejob-locking 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: cd8286a266d71bc225867b645c8e6839d492d1a3
-  data.tar.gz: ea9464a6fcd179539542168ab5f40f6a86e76d1d
+  metadata.gz: c1b1ea7063b5406f553e6d4282892299b6733cf5
+  data.tar.gz: 4fc8b9e358ec0583dc02eb6e6c069ef079653ad4
 SHA512:
-  metadata.gz: a20e9295791ae6e571afda6c7d243003cd601c1bf1f403bb757ce5c2022cbe34ecb0b6f7d64ea243021078b9cfa9e783bd4b1b0f6ec374e109d3a7d88a83415d
-  data.tar.gz: 25ea633bf9c6386a7d8e98b3008cb8cc4a85a865637165acfa0f0ea12b283269257d04b185114609d55f616ce7473b5fdbe21989ff222cef6a193342483f9a92
+  metadata.gz: e84f37fc29ad783441943835f2157e530522dc52ff2e6953c047d029e2d19ad654c6c2f300ae1df277013c7f9e2cca8cd048002e234fb81f518f4572aff667e8
+  data.tar.gz: 9f05089c530aa8bcb0db266eda5210126d7d25634462495c448861b9057aa7d9ff037301081b74c1ca90de574fc62b9358bca76d88059c035383903e88a523a2

data/HISTORY.md

@@ -1,3 +1,8 @@
+## 0.2.0 (2017-06-20)
+
+- Bug fixes
+- Improved tests
+
 ## 0.1.0 (2017-01-16)
 
 - Initial release

data/README.md

@@ -6,8 +6,8 @@ ActiveJob Locking
 
 activejob-locking lets you control how ActiveJobs are enqueued and performed:
 
-* Allow only one job to be enqueued at a time  (based on a lock_id)
-* Allow only one job to be peformed at a time (also based on a lock_id)
+* Allow only one job to be enqueued at a time - thus a "unique" job
+* Allow only one job to be performed at a time - thus a "serialized" job
 
 There are many other similar gems including [resque-lock-timeout](https://github.com/lantins/resque-lock-timeout),
 [activejob-traffic-control](https://github.com/nickelser/activejob-traffic_control), [activejob-lock](https://github.com/idolweb/activejob-lock),
@@ -24,14 +24,14 @@ Add this line to your application's Gemfile:
 gem 'activejob-locking'
 ```
 
-Enqueueing
+Unique Jobs
 ------------
 Sometime you only want to enqueue one instance of a job.  No other similar job should be enqueued until the first one 
 is completed. 
 
 ```ruby
-class EnqueueDropJob < ActiveJob::Base
-  include ActiveJob::Locking::Enqueue
+class UniqueJob < ActiveJob::Base
+  include ActiveJob::Locking::Unique
 
   # Make sure the lock_key is always the same
   def lock_key
@@ -48,14 +48,14 @@ never be enqueued or it will wait to the first job is performed.  That is contro
 [options](##options) described below.
 
 
-Performing
+Serialized Jobs
 ------------
 Sometime you only want to perform one instance of a job at a time.  No other similar job should be performed until the first one 
 is completed. 
 
 ```ruby
-class EnqueueDropJob < ActiveJob::Base
-  include ActiveJob::Locking::Perform
+class SerializedJob < ActiveJob::Base
+  include ActiveJob::Locking::Serialized
 
   # Make sure the lock_key is always the same
   def lock_key
@@ -136,7 +136,7 @@ available at:
 ```ruby
 ActiveJob::Locking.options
 ```
-This should be updated using a Rails initializer.  Each job class can override invidual options as it sees fit.
+This should be updated using a Rails initializer.  Each job class can override individual options as it sees fit.
 
 ### Adapter
 
@@ -151,7 +151,7 @@ Locally update:
 
 ```ruby
 class ExampleJob < ActiveJob::Base
-  include ActiveJob::Locking::Perform
+  include ActiveJob::Locking::Serialized
 
   self.adapter = ActiveJob::Locking::Adapters::SuoRedis
 end
@@ -171,58 +171,59 @@ Locally update:
 
 ```ruby
 class ExampleJob < ActiveJob::Base
-  include ActiveJob::Locking::Perform
+  include ActiveJob::Locking::Serialized
 
   self.hosts = 'localhost'
 end
 ```
 
-### Time
+### lock_acquire_time
 
-The is the time to live for any acquired locks.  For most locking gems this is mapped to their concept of "stale" locks.
-That means that if an attempt is made to access the lock after it is expired, it will be considered unlocked.  That is in
-contrast to aggressively removing locks for running jobs even if no other job has requested them.
-
-The value is specified in seconds and defaults to 100.
+The is the timeout for acquiring a lock.  The value is specified in seconds and defaults to 1. It must
+be greater than zero and cannot be nil.
 
 Globally update:
 
 ```ruby
-ActiveJob::Locking.options.time = 100
+ActiveJob::Locking.options.lock_acquire_time = 1
 ```
-Locally update (notice the different method name to avoid potential conflicts):
+Locally update:
 
 ```ruby
 class ExampleJob < ActiveJob::Base
-  include ActiveJob::Locking::Perform
+  include ActiveJob::Locking::Unique
 
-  self.lock_time = 100
+  self.lock_acquire_time = 1
 end
 ```
+This greatly influences how enqueuing behavior works.  If the timeout is short, then jobs that are waiting to
+be enqueued are dropped and the before_enqueue callback will fail. If the timeout is infinite, then jobs will wait 
+in turn to get enqueued.  If the timeout is somewhere in between then it will depend on how long the jobs
+take to execute.
 
-### Timeout
+### lock_time
 
-The is the timeout for acquiring a lock.  The value is specified in seconds and defaults to 1. It must
-be greater than zero and cannot be nil.
+The is the time to live for any acquired locks.  For most locking gems this is mapped to their concept of "stale" locks.
+That means that if an attempt is made to access the lock after it is expired, it will be considered unlocked.  That is in
+contrast to aggressively removing locks for running jobs even if no other job has requested them.
+
+The value is specified in seconds and defaults to 100.
 
 Globally update:
 
 ```ruby
-ActiveJob::Locking.options.timeout = 1
+ActiveJob::Locking.options.lock_time = 100
 ```
-Locally update (notice the different method name to avoid potential conflicts):
+Locally update:
 
 ```ruby
 class ExampleJob < ActiveJob::Base
-  include ActiveJob::Locking::Enqueue
+  include ActiveJob::Locking::Serialized
 
-  self.lock_acquire_timeout= = 1
+  self.lock_time = 100
 end
 ```
-This greatly influences how enqueuing behavior works.  If the timeout is short, then jobs that are waiting to
-be enqueued are dropped and the before_enqueue callback will fail. If the timeout is infinite, then jobs will wait 
-in turn to get enqueued.  If the timeout is somewhere in between then it will depend on how long the jobs
-take to execute.
+
 
 ### AdapterOptions
 
@@ -238,7 +239,7 @@ Locally update (notice the different method name to avoid potential conflicts):
 
 ```ruby
 class ExampleJob < ActiveJob::Base
-  include ActiveJob::Locking::Enqueue
+  include ActiveJob::Locking::Unique
 
   self.adapter_options = {}
 end

data/lib/activejob/locking/adapters/memory.rb

@@ -2,8 +2,6 @@ module ActiveJob
   module Locking
     module Adapters
       class Memory < Base
-        attr_reader :timeout
-
         @hash = Hash.new
         @mutex = Mutex.new
 
@@ -39,8 +37,8 @@ module ActiveJob
         end
 
         def lock
-          finish = Time.now + self.options.timeout
-          sleep_time = [5, self.options.timeout / 5].min
+          finish = Time.now + self.options.lock_acquire_time
+          sleep_time = [5, self.options.lock_acquire_time / 5].min
 
           begin
             lock = self.class.lock(key)
@@ -48,7 +46,7 @@ module ActiveJob
             sleep(sleep_time)
           end while Time.now < finish
 
-          return false
+          false
         end
 
         def unlock

data/lib/activejob/locking/adapters/redis-semaphore.rb

@@ -6,13 +6,14 @@ module ActiveJob
       class RedisSemaphore < Base
         def create_lock_manager
           mapped_options = {host: self.options.hosts,
-                            stale_client_timeout: self.options.time}.merge(self.options.adapter_options)
+                            resources: 1,
+                            stale_client_timeout: self.options.lock_time}.merge(self.options.adapter_options)
 
           Redis::Semaphore.new(self.key, mapped_options)
         end
 
         def lock
-          self.lock_token = self.lock_manager.lock(self.options.timeout)
+          self.lock_token = self.lock_manager.lock(self.options.lock_acquire_time)
         end
 
         def unlock

data/lib/activejob/locking/adapters/redlock.rb

@@ -6,14 +6,14 @@ module ActiveJob
       class Redlock < Base
         def create_lock_manager
           mapped_options = self.options.adapter_options
-          mapped_options[:retry_count] = ::Redlock::Client::DEFAULT_RETRY_COUNT
-          mapped_options[:retry_delay] = 2000 * ((self.options.timeout || 2**32) / (::Redlock::Client::DEFAULT_RETRY_COUNT * 1.0))
+          mapped_options[:retry_count] = 2 # Try to get the lock and then try again when timeout is expiring--
+          mapped_options[:retry_delay] = self.options.lock_acquire_time * 1000 # convert from seconds to milliseconds
 
           ::Redlock::Client.new(Array(self.options.hosts), mapped_options)
         end
 
         def lock
-          self.lock_token = self.lock_manager.lock(self.key, self.options.time * 1000)
+          self.lock_token = self.lock_manager.lock(self.key, self.options.lock_time * 1000)
         end
 
         def unlock

data/lib/activejob/locking/adapters/suo-redis.rb

@@ -6,8 +6,8 @@ module ActiveJob
       class SuoRedis < Base
         def create_lock_manager
           mapped_options = {connection: {host: self.options.hosts},
-                            stale_lock_expiration: self.options.time,
-                            acquisition_timeout: self.options.timeout}
+                            stale_lock_expiration: self.options.lock_time,
+                            acquisition_timeout: self.options.lock_acquire_time}
 
           Suo::Client::Redis.new(self.key, mapped_options)
         end

data/lib/activejob/locking/base.rb

@@ -7,15 +7,15 @@ module ActiveJob
         def lock_options
           @lock_options ||= ActiveJob::Locking::Options.new
         end
-        delegate :adapter, :hosts, :lock_time, :lock_acquire_timeout, :adapter_options, to: :lock_options
-        delegate :adapter=, :hosts=, :lock_time=, :lock_acquire_timeout=, :adapter_options=, to: :lock_options
+        delegate :adapter, :hosts, :lock_time, :lock_acquire_time, :adapter_options, to: :lock_options
+        delegate :adapter=, :hosts=, :lock_time=, :lock_acquire_time=, :adapter_options=, to: :lock_options
       end
 
       included do
-        # We need to serialize the lock token because it could be released in a different process
+        # We need to serialize the lock token that some gems create because it could be released in a different process
         def serialize
           result = super
-          result = result.merge('lock_token' => self.adapter.lock_token) if self.adapter.lock_token
+          result['lock_token'] = self.adapter.lock_token
           result
         end
 
@@ -29,11 +29,17 @@ module ActiveJob
         end
 
         def adapter
-          # Merge local and global options
-          merged_options = ActiveJob::Locking.options.dup.merge(self.class.lock_options)
+          @adapter ||= begin
+            # Make sure arguments are deserialized so calling lock key is safe
+            deserialize_arguments_if_needed
 
-          # Remember the lock might be acquired in one process and released in another
-          @adapter ||= merged_options.adapter.new(self.lock_key, merged_options)
+            # Merge local and global options
+            merged_options = ActiveJob::Locking.options.dup.merge(self.class.lock_options)
+
+            # Remember the lock might be acquired in one process and released in another
+            merged_options.adapter.new(self.lock_key, merged_options)
+          end
+          @adapter
         end
       end
     end

data/lib/activejob/locking/options.rb

@@ -5,30 +5,41 @@ module ActiveJob
     class Options
       attr_accessor :adapter
       attr_accessor :hosts
-      attr_accessor :time
-      attr_accessor :timeout
+      attr_accessor :lock_time
+      attr_accessor :lock_acquire_time
       attr_accessor :adapter_options
 
-      alias :lock_time :time
-      alias :lock_time= :time=
-      alias :lock_acquire_timeout :timeout
-      alias :lock_acquire_timeout= :timeout=
-
       def initialize(options = {})
         @adapter = options[:adapter]
         @hosts = options[:hosts]
-        @time = options[:time]
-        @timeout = options[:timeout]
+        @lock_time = options[:lock_time]
+        @lock_acquire_time = options[:lock_acquire_time]
         @adapter_options = options[:adapter_options]
       end
 
-      def timeout=(value)
-        if value.nil?
-          raise(ArgumentError, 'Lock timeout must be set')
-        elsif value == 0
-          raise(ArgumentError, 'Lock timeout must be greater than zero')
-        else
-          @timeout = value
+      def lock_time=(value)
+        case value
+          when NilClass
+            raise(ArgumentError, 'Lock time must be set')
+          when ActiveSupport::Duration
+            @lock_time = value.value
+          when 0
+            raise(ArgumentError, 'Lock time must be greater than zero')
+          else
+            @lock_time = value
+        end
+      end
+
+      def lock_acquire_time=(value)
+        case value
+          when NilClass
+            raise(ArgumentError, 'Lock acquire time must be set')
+          when ActiveSupport::Duration
+            @lock_acquire_time = value.value
+          when 0
+            raise(ArgumentError, 'Lock acquire time must be greater than zero')
+          else
+            @lock_acquire_time = value
         end
       end
 
@@ -36,8 +47,8 @@ module ActiveJob
         result = self.dup
         result.adapter = other.adapter if other.adapter
         result.hosts = other.hosts if other.hosts
-        result.time = other.time if other.time
-        result.timeout = other.timeout if other.timeout
+        result.lock_time = other.lock_time if other.lock_time
+        result.lock_acquire_time = other.lock_acquire_time if other.lock_acquire_time
         result.adapter_options = other.adapter_options if other.adapter_options
         result
       end

data/lib/activejob/locking/{perform.rb → serialized.rb}

@@ -1,20 +1,20 @@
 module ActiveJob
   module Locking
-    module Perform
+    module Serialized
       extend ::ActiveSupport::Concern
 
       included do
         include ::ActiveJob::Locking::Base
 
         around_perform do |job, block|
-          if self.adapter.lock
+          if job.adapter.lock
             begin
               block.call
             ensure
-              self.adapter.unlock
+              job.adapter.unlock
             end
           else
-            self.class.set(wait: 5.seconds).perform_later(job)
+            job.class.set(wait: job.class.lock_acquire_time).perform_later(*job.arguments)
           end
         end
       end

data/lib/activejob/locking/{enqueue.rb → unique.rb}

@@ -1,18 +1,23 @@
 module ActiveJob
   module Locking
-    module Enqueue
+    module Unique
       extend ::ActiveSupport::Concern
 
       included do
         include ::ActiveJob::Locking::Base
 
         before_enqueue do |job|
-          lock = self.adapter.lock
+          lock = job.adapter.lock
           throw :abort unless lock
         end
 
-        after_perform do |job|
+        rescue_from(Exception) do |exception|
           self.adapter.unlock
+          raise
+        end
+
+        after_perform do |job|
+          job.adapter.unlock
         end
       end
     end

data/lib/activejob-locking.rb

@@ -2,8 +2,8 @@ require 'activejob/locking/adapters/base'
 require 'activejob/locking/adapters/memory'
 
 require 'activejob/locking/base'
-require 'activejob/locking/enqueue'
-require 'activejob/locking/perform'
+require 'activejob/locking/unique'
+require 'activejob/locking/serialized'
 
 require 'activejob/locking/options'
 
@@ -11,8 +11,8 @@ module ActiveJob
   module Locking
     @options = ActiveJob::Locking::Options.new(adapter: ActiveJob::Locking::Adapters::Memory,
                                                hosts: 'localhost',
-                                               time: 100,
-                                               timeout: 1,
+                                               lock_time: 100,
+                                               lock_acquire_time: 1,
                                                adapter_options: {})
 
     def self.options

data/test/jobs/{perform_serially_job.rb → fail_job.rb}

@@ -1,7 +1,7 @@
-class PerformSeriallyJob < ActiveJob::Base
-  include ActiveJob::Locking::Perform
+class FailJob < ActiveJob::Base
+  include ActiveJob::Locking::Unique
 
-  self.lock_acquire_timeout = 1.hour
+  self.lock_acquire_time = 2
 
   # We want the job ids to be all the same for testing
   def lock_key
@@ -10,6 +10,6 @@ class PerformSeriallyJob < ActiveJob::Base
 
   # Pass in index so we can distinguish different jobs
   def perform(index, sleep_time)
-    sleep(sleep_time)
+    raise(ArgumentError, 'Job failed')
   end
 end

data/test/jobs/{enqueue_drop_job.rb → serial_job.rb}

@@ -1,7 +1,7 @@
-class EnqueueDropJob < ActiveJob::Base
-  include ActiveJob::Locking::Enqueue
+class SerialJob < ActiveJob::Base
+  include ActiveJob::Locking::Serialized
 
-  self.lock_acquire_timeout = 0.1
+  self.lock_acquire_time = 2
 
   # We want the job ids to be all the same for testing
   def lock_key

data/test/jobs/{enqueue_wait_job.rb → unique_job.rb}

@@ -1,7 +1,7 @@
-class EnqueueWaitJob < ActiveJob::Base
-  include ActiveJob::Locking::Enqueue
+class UniqueJob < ActiveJob::Base
+  include ActiveJob::Locking::Unique
 
-  self.lock_acquire_timeout = 1.hour
+  self.lock_acquire_time = 2
 
   # We want the job ids to be all the same for testing
   def lock_key

data/test/serialized_tests.rb

@@ -0,0 +1,63 @@
+require File.expand_path('../test_helper', __FILE__)
+
+module SerializedTests
+  def test_one_completed
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    start_time = Time.now
+    sleep_time = SerialJob.lock_acquire_time / 0.9
+    threads = 3.times.map do |i|
+      Thread.new do
+        SerialJob.perform_later(i, sleep_time)
+      end
+    end
+
+    # All the threads will complete after the sleep time has expired - since two jobs get requeued
+    threads.each {|thread| thread.join}
+    assert_equal(2, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(3, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    assert(Time.now - start_time > (1 * sleep_time))
+  end
+
+  def test_some_completed
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    start_time = Time.now
+    sleep_time = SerialJob.lock_acquire_time / 1.9
+    threads = 3.times.map do |i|
+      Thread.new do
+        SerialJob.perform_later(i, sleep_time)
+      end
+    end
+
+    # All the threads will complete after the sleep time has expired - since two jobs get requeued
+    threads.each {|thread| thread.join}
+    assert_equal(1, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(3, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    assert(Time.now - start_time > (1 * sleep_time))
+  end
+
+  def test_all_completed
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    start_time = Time.now
+    sleep_time = SerialJob.lock_acquire_time / 4
+    threads = 3.times.map do |i|
+      Thread.new do
+        SerialJob.perform_later(i, sleep_time)
+      end
+    end
+
+    # All the threads will complete after the sleep time has expired - since two jobs get requeued
+    threads.each {|thread| thread.join}
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(3, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    assert(Time.now - start_time > (1 * sleep_time))
+  end
+end

data/test/test_helper.rb

@@ -11,11 +11,9 @@ require 'activejob/locking/adapters/redis-semaphore'
 require 'activejob/locking/adapters/redlock'
 require 'activejob/locking/adapters/suo-redis'
 
-require_relative './jobs/enqueue_drop_job'
-require_relative './jobs/enqueue_wait_job'
-require_relative './jobs/enqueue_wait_timeout_job'
-require_relative './jobs/enqueue_wait_large_timeout_job'
-require_relative './jobs/perform_serially_job'
+require_relative './jobs/unique_job'
+require_relative './jobs/fail_job'
+require_relative './jobs/serial_job'
 
 def redis_reset
   Kernel.system('redis-cli FLUSHALL')

data/test/test_serialized_memory.rb

@@ -0,0 +1,11 @@
+require_relative('./serialized_tests')
+
+class SerializedMemory < MiniTest::Test
+  include SerializedTests
+
+  def setup
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
+  end
+end

data/test/{test_perform_redis_semaphore.rb → test_serialized_redis_semaphore.rb}

@@ -1,7 +1,7 @@
-require_relative('./perform_tests')
+require_relative('./serialized_tests')
 
-class EnqueueRedisSemaphoreTest < MiniTest::Test
-  include PerformTests
+class UniqueRedisSemaphoreTest < MiniTest::Test
+  include SerializedTests
 
   def setup
     redis_reset

data/test/{test_enqueue_redlock.rb → test_serialized_redlock.rb}

@@ -1,7 +1,7 @@
-require_relative('./enqueue_tests')
+require_relative('./serialized_tests')
 
-class EnqueueRedlockTest < MiniTest::Test
-  include EnqueueTests
+class SerializedRedlockTest < MiniTest::Test
+  include SerializedTests
 
   def setup
     redis_reset

data/test/{test_perform_suo_redis.rb → test_serialized_suo_redis.rb}

@@ -1,7 +1,7 @@
-require_relative('./perform_tests')
+require_relative('./serialized_tests')
 
-class PerformSuoRedisTest < MiniTest::Test
-  include PerformTests
+class SerializedSuoRedisTest < MiniTest::Test
+  include SerializedTests
 
   def setup
     redis_reset

data/test/test_suite.rb

@@ -0,0 +1,12 @@
+%w(
+  test_serialized_memory
+  test_serialized_redis_semaphore
+  test_serialized_redlock
+  test_serialized_suo_redis
+  test_unique_memory
+  test_unique_redis_semaphore
+  test_unique_redlock
+  test_unique_suo_redis
+).each do |test|
+  require File.expand_path("../#{test}", __FILE__)
+end

data/test/test_unique_memory.rb

@@ -0,0 +1,12 @@
+require_relative('./unique_tests')
+
+class UniqueMemoryTest < MiniTest::Test
+  include UniqueTests
+
+  def setup
+    ActiveJob::Locking::Adapters::Memory.reset
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
+  end
+end

data/test/{test_enqueue_redis_semaphore.rb → test_unique_redis_semaphore.rb}

@@ -1,7 +1,7 @@
-require_relative('./enqueue_tests')
+require_relative('./unique_tests')
 
-class EnqueueRedisSemaphoreTest < MiniTest::Test
-  include EnqueueTests
+class UniqueRedisSemaphoreTest < MiniTest::Test
+  include UniqueTests
 
   def setup
     redis_reset

data/test/{test_perform_redlock.rb → test_unique_redlock.rb}

@@ -1,7 +1,7 @@
-require_relative('./perform_tests')
+require_relative('./unique_tests')
 
-class PerformRedlockTest < MiniTest::Test
-  include PerformTests
+class UniqueRedlockTest < MiniTest::Test
+  include UniqueTests
 
   def setup
     redis_reset

data/test/{test_enqueue_suo_redis.rb → test_unique_suo_redis.rb}

@@ -1,7 +1,7 @@
-require_relative('./enqueue_tests')
+require_relative('./unique_tests')
 
-class EnqueueSuoRedisTest < MiniTest::Test
-  include EnqueueTests
+class UniqueSuoRedisTest < MiniTest::Test
+  include UniqueTests
 
   def setup
     redis_reset

data/test/unique_tests.rb

@@ -0,0 +1,100 @@
+require File.expand_path('../test_helper', __FILE__)
+
+module UniqueTests
+  def test_none_performed
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = false
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    sleep_time = UniqueJob.lock_acquire_time
+    threads = 3.times.map do |i|
+      Thread.new do
+        UniqueJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+    assert_equal(1, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+  ensure
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Base.queue_adapter.enqueued_jobs.clear
+  end
+
+  def test_one_performed
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    sleep_time = UniqueJob.lock_acquire_time * 2
+    threads = 3.times.map do |i|
+      Thread.new do
+        UniqueJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
+  end
+
+  def test_all_performed
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    start_time = Time.now
+    sleep_time = UniqueJob.lock_acquire_time / 4.0
+    threads = 3.times.map do |i|
+      Thread.new do
+        UniqueJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(threads.count, ActiveJob::Base.queue_adapter.performed_jobs.count)
+    assert(Time.now - start_time > (threads.count * sleep_time))
+  end
+
+  def test_some_performed
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    start_time = Time.now
+    sleep_time = UniqueJob.lock_acquire_time / 2.0
+    threads = 3.times.map do |i|
+      Thread.new do
+        UniqueJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(threads.count - 1, ActiveJob::Base.queue_adapter.performed_jobs.count)
+    assert(Time.now - start_time > ((threads.count - 1) * sleep_time))
+  end
+
+  def test_fail
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    start_time = Time.now
+    sleep_time = UniqueJob.lock_acquire_time
+    threads = 3.times.map do |i|
+      Thread.new do
+        begin
+          FailJob.perform_later(i, sleep_time)
+        rescue => e
+          # do nothing
+        end
+      end
+    end
+
+    threads.each {|thread| thread.join}
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(threads.count, ActiveJob::Base.queue_adapter.performed_jobs.count)
+  end
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: activejob-locking
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Charlie Savage
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-01-16 00:00:00.000000000 Z
+date: 2017-06-21 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -38,6 +38,20 @@ dependencies:
     - - ">"
       - !ruby/object:Gem::Version
         version: 5.10.0
+- !ruby/object:Gem::Dependency
+  name: redis-mutex
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: redis-semaphore
   requirement: !ruby/object:Gem::Requirement
@@ -84,7 +98,7 @@ description: |
   activejob-locking lets you control how ActiveJobs are enqueued and performed:
 
   Allow only one job to be enqueued at a time (based on a lock_id)
-  Allow only one job to be peformed at a time (also based on a lock_id)
+  Allow only one job to be performed at a time (also based on a lock_id)
 email: 
 executables: []
 extensions: []
@@ -102,26 +116,24 @@ files:
 - lib/activejob/locking/adapters/redlock.rb
 - lib/activejob/locking/adapters/suo-redis.rb
 - lib/activejob/locking/base.rb
-- lib/activejob/locking/enqueue.rb
 - lib/activejob/locking/options.rb
-- lib/activejob/locking/perform.rb
-- test/enqueue_tests.rb
-- test/jobs/enqueue_drop_job.rb
-- test/jobs/enqueue_wait_job.rb
-- test/jobs/enqueue_wait_large_timeout_job.rb
-- test/jobs/enqueue_wait_timeout_job.rb
-- test/jobs/perform_serially_job.rb
-- test/jobs/perform_serially_large_timeout_job.rb
-- test/perform_tests.rb
-- test/test_enqueue_memory.rb
-- test/test_enqueue_redis_semaphore.rb
-- test/test_enqueue_redlock.rb
-- test/test_enqueue_suo_redis.rb
+- lib/activejob/locking/serialized.rb
+- lib/activejob/locking/unique.rb
+- test/jobs/fail_job.rb
+- test/jobs/serial_job.rb
+- test/jobs/unique_job.rb
+- test/serialized_tests.rb
 - test/test_helper.rb
-- test/test_perform_memory.rb
-- test/test_perform_redis_semaphore.rb
-- test/test_perform_redlock.rb
-- test/test_perform_suo_redis.rb
+- test/test_serialized_memory.rb
+- test/test_serialized_redis_semaphore.rb
+- test/test_serialized_redlock.rb
+- test/test_serialized_suo_redis.rb
+- test/test_suite.rb
+- test/test_unique_memory.rb
+- test/test_unique_redis_semaphore.rb
+- test/test_unique_redlock.rb
+- test/test_unique_suo_redis.rb
+- test/unique_tests.rb
 homepage: http://github.com/cfis/activejob-locking
 licenses:
 - MIT
@@ -142,7 +154,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.8
+rubygems_version: 2.6.11
 signing_key: 
 specification_version: 4
 summary: ActiveJob locking to control how jobs are enqueued and performed.

data/test/enqueue_tests.rb

@@ -1,74 +0,0 @@
-require File.expand_path('../test_helper', __FILE__)
-
-module EnqueueTests
-  def test_drop
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-
-    start_time = Time.now
-    sleep_time = 2
-    threads = 2.times.map do |i|
-      Thread.new do
-        EnqueueDropJob.perform_later(i, sleep_time)
-      end
-    end
-
-    threads.each {|thread| thread.join}
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
-    assert(Time.now - start_time > (1 * sleep_time))
-  end
-
-  def test_wait
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-
-    start_time = Time.now
-    sleep_time = 2
-    threads = 3.times.map do |i|
-      Thread.new do
-        EnqueueWaitJob.perform_later(i, sleep_time)
-      end
-    end
-
-    threads.each {|thread| thread.join}
-
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-    assert_equal(threads.count, ActiveJob::Base.queue_adapter.performed_jobs.count)
-    assert(Time.now - start_time > (threads.count * sleep_time))
-  end
-
-  def test_wait_large_timeout
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-
-    start_time = Time.now
-    sleep_time = 2 * EnqueueWaitTimeoutJob.lock_acquire_timeout
-    threads = 3.times.map do |i|
-      Thread.new do
-        EnqueueWaitTimeoutJob.perform_later(i, sleep_time)
-      end
-    end
-
-    threads.each {|thread| thread.join}
-
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
-    assert(Time.now - start_time > (1 * sleep_time))
-  end
-
-  def test_wait_timeout
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-
-    start_time = Time.now
-    sleep_time = 0.2 * EnqueueWaitLargeTimeoutJob.lock_acquire_timeout
-    threads = 3.times.map do |i|
-      Thread.new do
-        EnqueueWaitLargeTimeoutJob.perform_later(i, sleep_time)
-      end
-    end
-
-    threads.each {|thread| thread.join}
-
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-    assert_equal(threads.count, ActiveJob::Base.queue_adapter.performed_jobs.count)
-    assert(Time.now - start_time > (threads.count * sleep_time))
-  end
-end

data/test/jobs/enqueue_wait_large_timeout_job.rb

@@ -1,16 +0,0 @@
-class EnqueueWaitLargeTimeoutJob < ActiveJob::Base
-  include ActiveJob::Locking::Enqueue
-
-  # Wait for 10 seconds to get a lock
-  self.lock_acquire_timeout = 10
-
-  # We want the job ids to be all the same for testing
-  def lock_key
-    self.class.name
-  end
-
-  # Pass in index so we can distinguish different jobs
-  def perform(index, sleep_time)
-    sleep(sleep_time)
-  end
-end

data/test/jobs/enqueue_wait_timeout_job.rb

@@ -1,16 +0,0 @@
-class EnqueueWaitTimeoutJob < ActiveJob::Base
-  include ActiveJob::Locking::Enqueue
-
-  # Wait for 1 second to get a lock
-  self.lock_acquire_timeout = 1
-
-  # We want the job ids to be all the same for testing
-  def lock_key
-    self.class.name
-  end
-
-  # Pass in index so we can distinguish different jobs
-  def perform(index, sleep_time)
-    sleep(sleep_time)
-  end
-end

data/test/jobs/perform_serially_large_timeout_job.rb

@@ -1,16 +0,0 @@
-class PerformSeriallyLargeTimeoutJob < ActiveJob::Base
-  include ActiveJob::Locking::Enqueue
-
-  # Wait for 10 seconds to get a lock
-  self.lock_acquire_timeout = 10
-
-  # We want the job ids to be all the same for testing
-  def lock_key
-    self.class.name
-  end
-
-  # Pass in index so we can distinguish different jobs
-  def perform(index, sleep_time)
-    sleep(sleep_time)
-  end
-end

data/test/perform_tests.rb

@@ -1,37 +0,0 @@
-require File.expand_path('../test_helper', __FILE__)
-
-module PerformTests
-  def test_serialize
-    start_time = Time.now
-    sleep_time = 2
-    threads = 3.times.map do |i|
-      Thread.new do
-        PerformSeriallyJob.perform_later(i, sleep_time)
-      end
-    end
-
-    threads.each {|thread| thread.join}
-
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-    assert_equal(threads.count, ActiveJob::Base.queue_adapter.performed_jobs.count)
-    assert(Time.now - start_time > (threads.count * sleep_time))
-  end
-
-  def test_wait_large_timeout
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-
-    start_time = Time.now
-    sleep_time = 2 * EnqueueWaitTimeoutJob.lock_acquire_timeout
-    threads = 3.times.map do |i|
-      Thread.new do
-        EnqueueWaitTimeoutJob.perform_later(i, sleep_time)
-      end
-    end
-
-    threads.each {|thread| thread.join}
-
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
-    assert(Time.now - start_time > (1 * sleep_time))
-  end
-end

data/test/test_enqueue_memory.rb

@@ -1,45 +0,0 @@
-require_relative('./enqueue_tests')
-
-class EnqueueMemoryTest < MiniTest::Test
-  include EnqueueTests
-
-  def setup
-    ActiveJob::Base.queue_adapter = :test
-    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
-    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
-  end
-
-  def test_enqueue_one
-    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = false
-    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
-
-    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = false
-    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-
-    sleep_time = 2
-    threads = 3.times.map do |i|
-      Thread.new do
-        EnqueueDropJob.perform_later(i, sleep_time)
-      end
-    end
-
-    threads.each {|thread| thread.join}
-    assert_equal(1, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
-  end
-
-  def test_perform_one
-    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
-
-    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
-
-    sleep_time = 1
-    threads = 3.times.map do |i|
-      Thread.new do
-        EnqueueDropJob.perform_later(i, sleep_time)
-      end
-    end
-
-    threads.each {|thread| thread.join}
-    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
-  end
-end

data/test/test_perform_memory.rb

@@ -1,14 +0,0 @@
-require_relative('./perform_tests')
-
-class PerformMemory < MiniTest::Test
-  include PerformTests
-
-  def setup
-    redis_reset
-
-    ActiveJob::Base.queue_adapter = :test
-    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
-    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
-    ActiveJob::Locking.options.hosts = Redlock::Client::DEFAULT_REDIS_URLS
-  end
-end