activejob-locking 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: cd8286a266d71bc225867b645c8e6839d492d1a3
+  data.tar.gz: ea9464a6fcd179539542168ab5f40f6a86e76d1d
+SHA512:
+  metadata.gz: a20e9295791ae6e571afda6c7d243003cd601c1bf1f403bb757ce5c2022cbe34ecb0b6f7d64ea243021078b9cfa9e783bd4b1b0f6ec374e109d3a7d88a83415d
+  data.tar.gz: 25ea633bf9c6386a7d8e98b3008cb8cc4a85a865637165acfa0f0ea12b283269257d04b185114609d55f616ce7473b5fdbe21989ff222cef6a193342483f9a92

data/Gemfile

@@ -0,0 +1,10 @@
+source 'https://rubygems.org'
+
+gem 'activejob', :require => 'active_job'
+
+group :test do
+  gem 'minitest'
+  gem 'redis-semaphore'
+  gem 'redlock'
+  gem 'suo'
+end

data/HISTORY.md

@@ -0,0 +1,3 @@
+## 0.1.0 (2017-01-16)
+
+- Initial release

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2017 Charlie Savage
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+Software), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,245 @@
+ActiveJob Locking
+===================
+
+[![Build Status](https://secure.travis-ci.org/lantins/activejob-locking.png?branch=master)](http://travis-ci.org/cfis/activejob-locking)
+[![Gem Version](https://badge.fury.io/rb/activejob-locking.png)](http://badge.fury.io/rb/activejob-locking)
+
+activejob-locking lets you control how ActiveJobs are enqueued and performed:
+
+* Allow only one job to be enqueued at a time  (based on a lock_id)
+* Allow only one job to be peformed at a time (also based on a lock_id)
+
+There are many other similar gems including [resque-lock-timeout](https://github.com/lantins/resque-lock-timeout),
+[activejob-traffic-control](https://github.com/nickelser/activejob-traffic_control), [activejob-lock](https://github.com/idolweb/activejob-lock),
+[activejob-locks](https://github.com/erickrause/activejob-locks).  What is different about this gem is that it
+is agnostic on the locking mechanism.  In the same way that ActiveJob works with many apapters, ActiveJob Locking 
+works with a variety of locking gems.
+ 
+Installation
+------------
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'activejob-locking'
+```
+
+Enqueueing
+------------
+Sometime you only want to enqueue one instance of a job.  No other similar job should be enqueued until the first one 
+is completed. 
+
+```ruby
+class EnqueueDropJob < ActiveJob::Base
+  include ActiveJob::Locking::Enqueue
+
+  # Make sure the lock_key is always the same
+  def lock_key
+    self.class.name
+  end
+ 
+  def perform
+    # do some work
+  end
+end
+```
+Only one instance of this job will ever be enqueued.  If an additional job is enqueued, it will either be dropped and
+never be enqueued or it will wait to the first job is performed.  That is controlled by the job
+[options](##options) described below.
+
+
+Performing
+------------
+Sometime you only want to perform one instance of a job at a time.  No other similar job should be performed until the first one 
+is completed. 
+
+```ruby
+class EnqueueDropJob < ActiveJob::Base
+  include ActiveJob::Locking::Perform
+
+  # Make sure the lock_key is always the same
+  def lock_key
+    self.class.name
+  end
+
+  def perform
+    # do some work
+  end
+end
+```
+Only one instance of this job will ever be performed.  If an additional job is enqueued, it will wait in its que until
+to the first job is performed. 
+
+Locking
+------------
+Locks are used to control how jobs are enqueued and performed. The idea is that locks are stored in a distributed
+system such as [Redis](https://redis.io/) or [Memcached](https://memcached.org/) so they can be used by
+multiple servers to coordinate the enqueueing and performing of jobs.
+
+The ActiveJob Locking gem does not include a locking implementation. Instead it provides adapters for
+distributed locking gems. 
+
+Currently three gems are supported:
+
+* [redis-semaphore](https://github.com/dv/redis-semaphore) 
+
+* [suo](https://github.com/nickelser/suo)
+
+* [redlock-rb](https://github.com/leandromoreira/redlock-rb)
+
+If you would like to have an additional locking mechanism supported, please feel free to send in a pull request.
+
+Please see the [options](##options) section below on how to specify a locking adapter.
+
+
+Lock Key
+---------
+
+Notice that the code samples above include a `lock_key` method. The return value of this method is used by the
+gem to create locks behind the scenes.  Thus it holds the key (pun intended) to controlling how jobs are enqueued
+and performed.
+
+By default the key is defined as:
+
+```ruby
+ def lock_key
+   [self.class.name, serialize_arguments(self.arguments)].join('/')
+ end 
+```
+Thus it has the format `<job class name>/<serialized_job_arguments>`
+
+To use this gem, you will want to override this method per job.
+  
+### Examples
+
+Allow only one job per queue to be enqueued or performed: 
+  
+```ruby
+ def lock_key
+   self.queue
+ end 
+```
+
+Allow only one instance of a job class to be enqueued of performed:  
+  
+```ruby
+ def lock_key
+   self.class.name
+ end 
+```
+
+Options
+-------
+The locking behavior can be dramatically changed by tweaking various options. There is a global set of options
+available at:
+
+```ruby
+ActiveJob::Locking.options
+```
+This should be updated using a Rails initializer.  Each job class can override invidual options as it sees fit.
+
+### Adapter
+
+Use the adapter option to specify which locking gem to use.  
+
+Globally update:
+
+```ruby
+ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::SuoRedis
+```
+Locally update:
+
+```ruby
+class ExampleJob < ActiveJob::Base
+  include ActiveJob::Locking::Perform
+
+  self.adapter = ActiveJob::Locking::Adapters::SuoRedis
+end
+```
+
+### Host
+
+The host(s) of the distributed system. This format will be dependent on the locking gem, so please read its 
+documentation.  
+
+Globally update:
+
+```ruby
+ActiveJob::Locking.options.hosts = 'localhost'
+```
+Locally update:
+
+```ruby
+class ExampleJob < ActiveJob::Base
+  include ActiveJob::Locking::Perform
+
+  self.hosts = 'localhost'
+end
+```
+
+### Time
+
+The is the time to live for any acquired locks.  For most locking gems this is mapped to their concept of "stale" locks.
+That means that if an attempt is made to access the lock after it is expired, it will be considered unlocked.  That is in
+contrast to aggressively removing locks for running jobs even if no other job has requested them.
+
+The value is specified in seconds and defaults to 100.
+
+Globally update:
+
+```ruby
+ActiveJob::Locking.options.time = 100
+```
+Locally update (notice the different method name to avoid potential conflicts):
+
+```ruby
+class ExampleJob < ActiveJob::Base
+  include ActiveJob::Locking::Perform
+
+  self.lock_time = 100
+end
+```
+
+### Timeout
+
+The is the timeout for acquiring a lock.  The value is specified in seconds and defaults to 1. It must
+be greater than zero and cannot be nil.
+
+Globally update:
+
+```ruby
+ActiveJob::Locking.options.timeout = 1
+```
+Locally update (notice the different method name to avoid potential conflicts):
+
+```ruby
+class ExampleJob < ActiveJob::Base
+  include ActiveJob::Locking::Enqueue
+
+  self.lock_acquire_timeout= = 1
+end
+```
+This greatly influences how enqueuing behavior works.  If the timeout is short, then jobs that are waiting to
+be enqueued are dropped and the before_enqueue callback will fail. If the timeout is infinite, then jobs will wait 
+in turn to get enqueued.  If the timeout is somewhere in between then it will depend on how long the jobs
+take to execute.
+
+### AdapterOptions
+
+This is a hash table of options that should be sent to the lock gem when it is instantiated. Read the lock 
+gems documentation to find appropriate values.
+
+Globally update:
+
+```ruby
+ActiveJob::Locking.options.adapter_options = {}
+```
+Locally update (notice the different method name to avoid potential conflicts):
+
+```ruby
+class ExampleJob < ActiveJob::Base
+  include ActiveJob::Locking::Enqueue
+
+  self.adapter_options = {}
+end
+```

data/Rakefile

@@ -0,0 +1,21 @@
+require 'rubygems'
+require 'rake'
+require 'rake/clean'
+require 'rake/testtask'
+require 'rubygems/package_task'
+
+# Set global variable so other tasks can access them
+::PROJECT_ROOT = File.expand_path(".")
+::GEM_NAME = 'activejob-locking'
+
+# Read the spec file
+spec = Gem::Specification.load("#{GEM_NAME}.gemspec")
+
+# Setup Rake tasks for managing the gem
+Gem::PackageTask.new(spec).define
+
+desc 'Run unit tests.'
+Rake::TestTask.new(:test) do |task|
+  task.test_files = FileList['test/test_*.rb']
+  task.verbose = true
+end

data/lib/activejob/locking/adapters/base.rb

@@ -0,0 +1,32 @@
+module ActiveJob
+  module Locking
+    module Adapters
+      class Base
+        attr_reader :key, :options, :lock_manager
+        attr_accessor :lock_token
+
+        def initialize(key, options)
+          @key = key
+          @options = options
+          @lock_manager = self.create_lock_manager
+        end
+
+        def create_lock_manager
+          raise('Subclass must implement')
+        end
+
+        def lock
+          raise('Subclass must implement')
+        end
+
+        def unlock
+          raise('Subclass must implement')
+        end
+
+        def refresh_lock!(refresh)
+          raise('Subclass must implement')
+        end
+      end
+    end
+  end
+end

data/lib/activejob/locking/adapters/memory.rb

@@ -0,0 +1,60 @@
+module ActiveJob
+  module Locking
+    module Adapters
+      class Memory < Base
+        attr_reader :timeout
+
+        @hash = Hash.new
+        @mutex = Mutex.new
+
+        def self.lock(key)
+          @mutex.synchronize do
+            if @hash[key]
+              false
+            else
+              @hash[key] = Time.now
+            end
+          end
+        end
+
+        def self.unlock(key)
+          @mutex.synchronize do
+            @hash.delete(key)
+          end
+        end
+
+        def self.locked?(key)
+          @mutex.synchronize do
+            @hash.include?(key)
+          end
+        end
+
+        def self.reset
+          @mutex.synchronize do
+            @hash = Hash.new
+          end
+        end
+
+        def create_lock_manager
+        end
+
+        def lock
+          finish = Time.now + self.options.timeout
+          sleep_time = [5, self.options.timeout / 5].min
+
+          begin
+            lock = self.class.lock(key)
+            return lock if lock
+            sleep(sleep_time)
+          end while Time.now < finish
+
+          return false
+        end
+
+        def unlock
+          self.class.unlock(self.key)
+        end
+      end
+    end
+  end
+end

data/lib/activejob/locking/adapters/redis-semaphore.rb

@@ -0,0 +1,25 @@
+require 'redis-semaphore'
+
+module ActiveJob
+  module Locking
+    module Adapters
+      class RedisSemaphore < Base
+        def create_lock_manager
+          mapped_options = {host: self.options.hosts,
+                            stale_client_timeout: self.options.time}.merge(self.options.adapter_options)
+
+          Redis::Semaphore.new(self.key, mapped_options)
+        end
+
+        def lock
+          self.lock_token = self.lock_manager.lock(self.options.timeout)
+        end
+
+        def unlock
+          self.lock_manager.signal(self.lock_token)
+          self.lock_token = nil
+        end
+      end
+    end
+  end
+end

data/lib/activejob/locking/adapters/redlock.rb

@@ -0,0 +1,26 @@
+require 'redlock'
+
+module ActiveJob
+  module Locking
+    module Adapters
+      class Redlock < Base
+        def create_lock_manager
+          mapped_options = self.options.adapter_options
+          mapped_options[:retry_count] = ::Redlock::Client::DEFAULT_RETRY_COUNT
+          mapped_options[:retry_delay] = 2000 * ((self.options.timeout || 2**32) / (::Redlock::Client::DEFAULT_RETRY_COUNT * 1.0))
+
+          ::Redlock::Client.new(Array(self.options.hosts), mapped_options)
+        end
+
+        def lock
+          self.lock_token = self.lock_manager.lock(self.key, self.options.time * 1000)
+        end
+
+        def unlock
+          self.lock_manager.unlock(self.lock_token)
+          self.lock_token = nil
+        end
+      end
+    end
+  end
+end

data/lib/activejob/locking/adapters/suo-redis.rb

@@ -0,0 +1,25 @@
+require 'suo'
+
+module ActiveJob
+  module Locking
+    module Adapters
+      class SuoRedis < Base
+        def create_lock_manager
+          mapped_options = {connection: {host: self.options.hosts},
+                            stale_lock_expiration: self.options.time,
+                            acquisition_timeout: self.options.timeout}
+
+          Suo::Client::Redis.new(self.key, mapped_options)
+        end
+
+        def lock
+          self.lock_token = self.lock_manager.lock
+        end
+
+        def unlock
+          self.lock_manager.unlock(self.lock_token)
+        end
+      end
+    end
+  end
+end

data/lib/activejob/locking/base.rb

@@ -0,0 +1,41 @@
+module ActiveJob
+  module Locking
+    module Base
+      extend ::ActiveSupport::Concern
+
+      module ClassMethods
+        def lock_options
+          @lock_options ||= ActiveJob::Locking::Options.new
+        end
+        delegate :adapter, :hosts, :lock_time, :lock_acquire_timeout, :adapter_options, to: :lock_options
+        delegate :adapter=, :hosts=, :lock_time=, :lock_acquire_timeout=, :adapter_options=, to: :lock_options
+      end
+
+      included do
+        # We need to serialize the lock token because it could be released in a different process
+        def serialize
+          result = super
+          result = result.merge('lock_token' => self.adapter.lock_token) if self.adapter.lock_token
+          result
+        end
+
+        def deserialize(job_data)
+          super
+          self.adapter.lock_token = job_data['lock_token']
+        end
+
+        def lock_key
+          [self.class.name, serialize_arguments(self.arguments)].join('/')
+        end
+
+        def adapter
+          # Merge local and global options
+          merged_options = ActiveJob::Locking.options.dup.merge(self.class.lock_options)
+
+          # Remember the lock might be acquired in one process and released in another
+          @adapter ||= merged_options.adapter.new(self.lock_key, merged_options)
+        end
+      end
+    end
+  end
+end

data/lib/activejob/locking/enqueue.rb

@@ -0,0 +1,20 @@
+module ActiveJob
+  module Locking
+    module Enqueue
+      extend ::ActiveSupport::Concern
+
+      included do
+        include ::ActiveJob::Locking::Base
+
+        before_enqueue do |job|
+          lock = self.adapter.lock
+          throw :abort unless lock
+        end
+
+        after_perform do |job|
+          self.adapter.unlock
+        end
+      end
+    end
+  end
+end

data/lib/activejob/locking/options.rb

@@ -0,0 +1,46 @@
+require 'ostruct'
+
+module ActiveJob
+  module Locking
+    class Options
+      attr_accessor :adapter
+      attr_accessor :hosts
+      attr_accessor :time
+      attr_accessor :timeout
+      attr_accessor :adapter_options
+
+      alias :lock_time :time
+      alias :lock_time= :time=
+      alias :lock_acquire_timeout :timeout
+      alias :lock_acquire_timeout= :timeout=
+
+      def initialize(options = {})
+        @adapter = options[:adapter]
+        @hosts = options[:hosts]
+        @time = options[:time]
+        @timeout = options[:timeout]
+        @adapter_options = options[:adapter_options]
+      end
+
+      def timeout=(value)
+        if value.nil?
+          raise(ArgumentError, 'Lock timeout must be set')
+        elsif value == 0
+          raise(ArgumentError, 'Lock timeout must be greater than zero')
+        else
+          @timeout = value
+        end
+      end
+
+      def merge(other)
+        result = self.dup
+        result.adapter = other.adapter if other.adapter
+        result.hosts = other.hosts if other.hosts
+        result.time = other.time if other.time
+        result.timeout = other.timeout if other.timeout
+        result.adapter_options = other.adapter_options if other.adapter_options
+        result
+      end
+    end
+  end
+end

data/lib/activejob/locking/perform.rb

@@ -0,0 +1,23 @@
+module ActiveJob
+  module Locking
+    module Perform
+      extend ::ActiveSupport::Concern
+
+      included do
+        include ::ActiveJob::Locking::Base
+
+        around_perform do |job, block|
+          if self.adapter.lock
+            begin
+              block.call
+            ensure
+              self.adapter.unlock
+            end
+          else
+            self.class.set(wait: 5.seconds).perform_later(job)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/activejob-locking.rb

@@ -0,0 +1,23 @@
+require 'activejob/locking/adapters/base'
+require 'activejob/locking/adapters/memory'
+
+require 'activejob/locking/base'
+require 'activejob/locking/enqueue'
+require 'activejob/locking/perform'
+
+require 'activejob/locking/options'
+
+module ActiveJob
+  module Locking
+    @options = ActiveJob::Locking::Options.new(adapter: ActiveJob::Locking::Adapters::Memory,
+                                               hosts: 'localhost',
+                                               time: 100,
+                                               timeout: 1,
+                                               adapter_options: {})
+
+    def self.options
+      @options
+    end
+  end
+end
+

data/test/enqueue_tests.rb

@@ -0,0 +1,74 @@
+require File.expand_path('../test_helper', __FILE__)
+
+module EnqueueTests
+  def test_drop
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+
+    start_time = Time.now
+    sleep_time = 2
+    threads = 2.times.map do |i|
+      Thread.new do
+        EnqueueDropJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
+    assert(Time.now - start_time > (1 * sleep_time))
+  end
+
+  def test_wait
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+
+    start_time = Time.now
+    sleep_time = 2
+    threads = 3.times.map do |i|
+      Thread.new do
+        EnqueueWaitJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(threads.count, ActiveJob::Base.queue_adapter.performed_jobs.count)
+    assert(Time.now - start_time > (threads.count * sleep_time))
+  end
+
+  def test_wait_large_timeout
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+
+    start_time = Time.now
+    sleep_time = 2 * EnqueueWaitTimeoutJob.lock_acquire_timeout
+    threads = 3.times.map do |i|
+      Thread.new do
+        EnqueueWaitTimeoutJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
+    assert(Time.now - start_time > (1 * sleep_time))
+  end
+
+  def test_wait_timeout
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+
+    start_time = Time.now
+    sleep_time = 0.2 * EnqueueWaitLargeTimeoutJob.lock_acquire_timeout
+    threads = 3.times.map do |i|
+      Thread.new do
+        EnqueueWaitLargeTimeoutJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(threads.count, ActiveJob::Base.queue_adapter.performed_jobs.count)
+    assert(Time.now - start_time > (threads.count * sleep_time))
+  end
+end

data/test/jobs/enqueue_drop_job.rb

@@ -0,0 +1,15 @@
+class EnqueueDropJob < ActiveJob::Base
+  include ActiveJob::Locking::Enqueue
+
+  self.lock_acquire_timeout = 0.1
+
+  # We want the job ids to be all the same for testing
+  def lock_key
+    self.class.name
+  end
+
+  # Pass in index so we can distinguish different jobs
+  def perform(index, sleep_time)
+    sleep(sleep_time)
+  end
+end

data/test/jobs/enqueue_wait_job.rb

@@ -0,0 +1,15 @@
+class EnqueueWaitJob < ActiveJob::Base
+  include ActiveJob::Locking::Enqueue
+
+  self.lock_acquire_timeout = 1.hour
+
+  # We want the job ids to be all the same for testing
+  def lock_key
+    self.class.name
+  end
+
+  # Pass in index so we can distinguish different jobs
+  def perform(index, sleep_time)
+    sleep(sleep_time)
+  end
+end

data/test/jobs/enqueue_wait_large_timeout_job.rb

@@ -0,0 +1,16 @@
+class EnqueueWaitLargeTimeoutJob < ActiveJob::Base
+  include ActiveJob::Locking::Enqueue
+
+  # Wait for 10 seconds to get a lock
+  self.lock_acquire_timeout = 10
+
+  # We want the job ids to be all the same for testing
+  def lock_key
+    self.class.name
+  end
+
+  # Pass in index so we can distinguish different jobs
+  def perform(index, sleep_time)
+    sleep(sleep_time)
+  end
+end

data/test/jobs/enqueue_wait_timeout_job.rb

@@ -0,0 +1,16 @@
+class EnqueueWaitTimeoutJob < ActiveJob::Base
+  include ActiveJob::Locking::Enqueue
+
+  # Wait for 1 second to get a lock
+  self.lock_acquire_timeout = 1
+
+  # We want the job ids to be all the same for testing
+  def lock_key
+    self.class.name
+  end
+
+  # Pass in index so we can distinguish different jobs
+  def perform(index, sleep_time)
+    sleep(sleep_time)
+  end
+end

data/test/jobs/perform_serially_job.rb

@@ -0,0 +1,15 @@
+class PerformSeriallyJob < ActiveJob::Base
+  include ActiveJob::Locking::Perform
+
+  self.lock_acquire_timeout = 1.hour
+
+  # We want the job ids to be all the same for testing
+  def lock_key
+    self.class.name
+  end
+
+  # Pass in index so we can distinguish different jobs
+  def perform(index, sleep_time)
+    sleep(sleep_time)
+  end
+end

data/test/jobs/perform_serially_large_timeout_job.rb

@@ -0,0 +1,16 @@
+class PerformSeriallyLargeTimeoutJob < ActiveJob::Base
+  include ActiveJob::Locking::Enqueue
+
+  # Wait for 10 seconds to get a lock
+  self.lock_acquire_timeout = 10
+
+  # We want the job ids to be all the same for testing
+  def lock_key
+    self.class.name
+  end
+
+  # Pass in index so we can distinguish different jobs
+  def perform(index, sleep_time)
+    sleep(sleep_time)
+  end
+end

data/test/perform_tests.rb

@@ -0,0 +1,37 @@
+require File.expand_path('../test_helper', __FILE__)
+
+module PerformTests
+  def test_serialize
+    start_time = Time.now
+    sleep_time = 2
+    threads = 3.times.map do |i|
+      Thread.new do
+        PerformSeriallyJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(threads.count, ActiveJob::Base.queue_adapter.performed_jobs.count)
+    assert(Time.now - start_time > (threads.count * sleep_time))
+  end
+
+  def test_wait_large_timeout
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+
+    start_time = Time.now
+    sleep_time = 2 * EnqueueWaitTimeoutJob.lock_acquire_timeout
+    threads = 3.times.map do |i|
+      Thread.new do
+        EnqueueWaitTimeoutJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
+    assert(Time.now - start_time > (1 * sleep_time))
+  end
+end

data/test/test_enqueue_memory.rb

@@ -0,0 +1,45 @@
+require_relative('./enqueue_tests')
+
+class EnqueueMemoryTest < MiniTest::Test
+  include EnqueueTests
+
+  def setup
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
+  end
+
+  def test_enqueue_one
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = false
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
+
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = false
+    assert_equal(0, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+
+    sleep_time = 2
+    threads = 3.times.map do |i|
+      Thread.new do
+        EnqueueDropJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+    assert_equal(1, ActiveJob::Base.queue_adapter.enqueued_jobs.count)
+  end
+
+  def test_perform_one
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
+
+    assert_equal(0, ActiveJob::Base.queue_adapter.performed_jobs.count)
+
+    sleep_time = 1
+    threads = 3.times.map do |i|
+      Thread.new do
+        EnqueueDropJob.perform_later(i, sleep_time)
+      end
+    end
+
+    threads.each {|thread| thread.join}
+    assert_equal(1, ActiveJob::Base.queue_adapter.performed_jobs.count)
+  end
+end

data/test/test_enqueue_redis_semaphore.rb

@@ -0,0 +1,12 @@
+require_relative('./enqueue_tests')
+
+class EnqueueRedisSemaphoreTest < MiniTest::Test
+  include EnqueueTests
+
+  def setup
+    redis_reset
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::RedisSemaphore
+  end
+end

data/test/test_enqueue_redlock.rb

@@ -0,0 +1,14 @@
+require_relative('./enqueue_tests')
+
+class EnqueueRedlockTest < MiniTest::Test
+  include EnqueueTests
+
+  def setup
+    redis_reset
+
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Redlock
+    ActiveJob::Locking.options.hosts = Redlock::Client::DEFAULT_REDIS_URLS
+  end
+end

data/test/test_enqueue_suo_redis.rb

@@ -0,0 +1,12 @@
+require_relative('./enqueue_tests')
+
+class EnqueueSuoRedisTest < MiniTest::Test
+  include EnqueueTests
+
+  def setup
+    redis_reset
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::SuoRedis
+  end
+end

data/test/test_helper.rb

@@ -0,0 +1,22 @@
+require 'bundler'
+Bundler.require(:default, :test)
+
+require 'minitest/autorun'
+
+# To make debugging easier, test within this source tree versus an installed gem
+$LOAD_PATH.unshift(File.expand_path('../../lib', __FILE__))
+require 'activejob-locking'
+
+require 'activejob/locking/adapters/redis-semaphore'
+require 'activejob/locking/adapters/redlock'
+require 'activejob/locking/adapters/suo-redis'
+
+require_relative './jobs/enqueue_drop_job'
+require_relative './jobs/enqueue_wait_job'
+require_relative './jobs/enqueue_wait_timeout_job'
+require_relative './jobs/enqueue_wait_large_timeout_job'
+require_relative './jobs/perform_serially_job'
+
+def redis_reset
+  Kernel.system('redis-cli FLUSHALL')
+end

data/test/test_perform_memory.rb

@@ -0,0 +1,14 @@
+require_relative('./perform_tests')
+
+class PerformMemory < MiniTest::Test
+  include PerformTests
+
+  def setup
+    redis_reset
+
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Memory
+    ActiveJob::Locking.options.hosts = Redlock::Client::DEFAULT_REDIS_URLS
+  end
+end

data/test/test_perform_redis_semaphore.rb

@@ -0,0 +1,12 @@
+require_relative('./perform_tests')
+
+class EnqueueRedisSemaphoreTest < MiniTest::Test
+  include PerformTests
+
+  def setup
+    redis_reset
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::RedisSemaphore
+  end
+end

data/test/test_perform_redlock.rb

@@ -0,0 +1,14 @@
+require_relative('./perform_tests')
+
+class PerformRedlockTest < MiniTest::Test
+  include PerformTests
+
+  def setup
+    redis_reset
+
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::Redlock
+    ActiveJob::Locking.options.hosts = Redlock::Client::DEFAULT_REDIS_URLS
+  end
+end

data/test/test_perform_suo_redis.rb

@@ -0,0 +1,12 @@
+require_relative('./perform_tests')
+
+class PerformSuoRedisTest < MiniTest::Test
+  include PerformTests
+
+  def setup
+    redis_reset
+    ActiveJob::Base.queue_adapter = :test
+    ActiveJob::Base.queue_adapter.perform_enqueued_jobs = true
+    ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::SuoRedis
+  end
+end

metadata

@@ -0,0 +1,149 @@
+--- !ruby/object:Gem::Specification
+name: activejob-locking
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Charlie Savage
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2017-01-16 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activejob
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 5.0.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 5.0.1
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">"
+      - !ruby/object:Gem::Version
+        version: 5.10.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">"
+      - !ruby/object:Gem::Version
+        version: 5.10.0
+- !ruby/object:Gem::Dependency
+  name: redis-semaphore
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: redlock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: suo
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: |
+  activejob-locking lets you control how ActiveJobs are enqueued and performed:
+
+  Allow only one job to be enqueued at a time (based on a lock_id)
+  Allow only one job to be peformed at a time (also based on a lock_id)
+email: 
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- Gemfile
+- HISTORY.md
+- LICENSE
+- README.md
+- Rakefile
+- lib/activejob-locking.rb
+- lib/activejob/locking/adapters/base.rb
+- lib/activejob/locking/adapters/memory.rb
+- lib/activejob/locking/adapters/redis-semaphore.rb
+- lib/activejob/locking/adapters/redlock.rb
+- lib/activejob/locking/adapters/suo-redis.rb
+- lib/activejob/locking/base.rb
+- lib/activejob/locking/enqueue.rb
+- lib/activejob/locking/options.rb
+- lib/activejob/locking/perform.rb
+- test/enqueue_tests.rb
+- test/jobs/enqueue_drop_job.rb
+- test/jobs/enqueue_wait_job.rb
+- test/jobs/enqueue_wait_large_timeout_job.rb
+- test/jobs/enqueue_wait_timeout_job.rb
+- test/jobs/perform_serially_job.rb
+- test/jobs/perform_serially_large_timeout_job.rb
+- test/perform_tests.rb
+- test/test_enqueue_memory.rb
+- test/test_enqueue_redis_semaphore.rb
+- test/test_enqueue_redlock.rb
+- test/test_enqueue_suo_redis.rb
+- test/test_helper.rb
+- test/test_perform_memory.rb
+- test/test_perform_redis_semaphore.rb
+- test/test_perform_redlock.rb
+- test/test_perform_suo_redis.rb
+homepage: http://github.com/cfis/activejob-locking
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.8
+signing_key: 
+specification_version: 4
+summary: ActiveJob locking to control how jobs are enqueued and performed.
+test_files: []