resque-uniqueue 0.2.2 → 0.3.0

data/Gemfile

@@ -5,13 +5,13 @@ source "http://rubygems.org"
 
 # Add dependencies to develop your gem here.
 # Include everything needed to run rake, tests, features, etc.
-gem 'resque', '~> 1.24.1'
+gem 'resque', '~> 1.25.0'
 
 group :development do
   gem 'mocha'
   gem "shoulda", ">= 0"
   gem "rdoc", "~> 3.12"
-  gem "bundler", "~> 1.3.0"
+  gem "bundler", "~> 1.5.0"
   gem "jeweler", "~> 1.8.4"
   gem "pry"
 end

data/Gemfile.lock

@@ -24,18 +24,18 @@ GEM
       method_source (~> 0.8)
       slop (~> 3.4)
     rack (1.5.2)
-    rack-protection (1.5.0)
+    rack-protection (1.5.3)
       rack
     rake (10.0.4)
     rdoc (3.12.2)
       json (~> 1.4)
-    redis (3.0.4)
-    redis-namespace (1.3.0)
-      redis (~> 3.0.0)
-    resque (1.24.1)
+    redis (3.0.7)
+    redis-namespace (1.4.1)
+      redis (~> 3.0.4)
+    resque (1.25.2)
       mono_logger (~> 1.0)
       multi_json (~> 1.0)
-      redis-namespace (~> 1.2)
+      redis-namespace (~> 1.3)
       sinatra (>= 0.9.2)
       vegas (~> 0.1.2)
     shoulda (3.5.0)
@@ -44,8 +44,8 @@ GEM
     shoulda-context (1.1.2)
     shoulda-matchers (2.1.0)
       activesupport (>= 3.0.0)
-    sinatra (1.4.2)
-      rack (~> 1.5, >= 1.5.2)
+    sinatra (1.4.5)
+      rack (~> 1.4)
       rack-protection (~> 1.4)
       tilt (~> 1.3, >= 1.3.4)
     slop (3.4.3)
@@ -57,10 +57,10 @@ PLATFORMS
   ruby
 
 DEPENDENCIES
-  bundler (~> 1.3.0)
+  bundler (~> 1.5.0)
   jeweler (~> 1.8.4)
   mocha
   pry
   rdoc (~> 3.12)
-  resque (~> 1.24.1)
+  resque (~> 1.25.0)
   shoulda

data/README.md

@@ -0,0 +1,70 @@
+# resque-uniqueue
+
+### Why use Uniqueue?
+
+Multiple identical jobs are wasteful. Uniqueue ensures that *for a given queue*, identical jobs are never enqueued, and you can stress a little less about your application code causing duplicate work.
+
+### Prerequisites
+Uniqueue uses Lua scripting, which requires **Redis 2.6 or greater**, so make sure your Redis installation is up to date.
+
+Before deploying an application with Uniqueue enabled for the first time, it's important that you **ensure your queues are empty**.
+
+### Installation
+Add Uniqueue to your Gemfile.
+
+    gem 'resque-uniqueue'
+    
+Then run bundle install within your app's directory.
+
+### Configuration
+You'll need to configure Uniqueue somewhere in your application's initialization process. If you are running a Rails application it's recommended you use your Resque initializer:
+
+    # config/initializers/resque.rb
+    Resque.unique_queues!
+
+#### Specifying queues 
+By default Uniqueue defaults to preventing identical jobs on all queues. However, if you need to scope Uniqueue to specific queues, then your intializer code should look like this:
+
+    # config/initializers/resque.rb
+    Resque.unique_queues = ["emails", "orders"] 
+    Resque.unique_queues!
+
+### How It Works
+
+Uniqueue overrides 3 resque commands: `push`, `pop`, and `remove_queue` in order to enforce *queue-level uniqueness* of jobs. And for each queue two additional Redis keys are created:
+
+1. `queue:[queue_name]:uniqueue` - A **set** containing MultiJSON dumps of the payload of all items on the queue
+2. `queue:[queue_name]:start_at` - A **list** containing the Unix timestamp of each job on the queue's start time, ordered identically to the actual job queue
+
+Now when Resque pushes a job the following happens:
+
+1. The length/cardinality of the queue, uniqueue set, and start_at list are verified to be equal. If they aren't, stuff has gone bad, and you'll get an exception.
+2. A Lua script is evaluated that executes `sadd` on the payload (well, a MultiJSON dump of it), which will add it to the uniqueue set if it is not already a member.
+3. If the payload's dump was not previously stored in the set, we `rpush` the start time of the job to the start_at list and `rpush` the job to the queue (following the lead of Resque's default `push` command).
+
+Because the three operations happen in the context of a Lua script, atomicity is guaranteed (See "Atomicity of Scripts" [here][eval]), and race conditions can never cause the uniqueue set, start_at list, and original queue to get out of sync. Each unique job is now successfully queued, its payload is present in our uniqueue set, and its start_at timestamp is on a list that corresponds exactly to the order of the queue list.
+
+Popping a job is very similar:
+
+1. Lengths and cardinalities are verified.
+2. MultiJSON load the payload.
+3. Return the payload with an additional key of 'start_at', which is the Unix timestamp that the job began.
+
+And this is how a unique job is born.
+
+### Contributing
+
+* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
+* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
+* Fork the project.
+* Start a feature/bugfix branch.
+* Commit and push until you are happy with your contribution.
+* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
+* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
+
+### Copyright
+
+Copyright (c) 2013 AcademicWorks, inc. See LICENSE.txt for
+further details.
+
+[eval]: http://redis.io/commands/eval

data/VERSION

@@ -1 +1 @@
-0.2.2
+0.3.0

data/lib/resque/uniqueue.rb

@@ -1,12 +1,12 @@
-require 'resque'
+require "resque"
 
 module Resque
   module Uniqueue
-    
+
     def push(queue, item)
       unique_queue?(queue) ? push_unique(queue, item) : super
     end
-    
+
     def pop(queue)
       unique_queue?(queue) ? pop_unique(queue)        : super
     end
@@ -30,7 +30,7 @@ module Resque
       results = redis.evalsha pop_unique_eval_sha, [queue]
       return nil unless results[0]
       job = decode results[0]
-      job['start_at'] ||= results[1].to_i
+      job["start_at"] ||= results[1].to_i
       return job
     end
 
@@ -39,8 +39,8 @@ module Resque
         local queue_name = KEYS[1]
         local uniqueue_name = queue_name..':uniqueue'
         local start_at_name = queue_name..':start_at'
-        local in_set = redis.call('sadd', uniqueue_name , ARGV[1])
-        if in_set == 1 then
+        local not_in_set = redis.call('sadd', uniqueue_name , ARGV[1])
+        if not_in_set == 1 then
           redis.call('rpush', start_at_name, ARGV[2])
           return redis.call('rpush', queue_name, ARGV[1])
         end
@@ -84,7 +84,7 @@ module Resque
 
     #if the queue and set sizes differ, something is very wrong and we should fail loudly
     def confirm_unique_queue_validity(queue)
-      response =  redis.evalsha queue_and_set_length_equal_eval_sha, [queue]
+      response = redis.evalsha queue_and_set_length_equal_eval_sha, [queue]
       return true if response == 1
       #TODO raise specific exception
       raise "Make sure your queues are empty before you start using uniqueue"
@@ -120,7 +120,7 @@ module Resque
 
     def confirm_compatible_redis_version
       redis_version = redis.info["redis_version"]
-      major, minor, patch = redis_version.split('.').map(&:to_i)
+      major, minor, patch = redis_version.split(".").map(&:to_i)
       if major < 2 || minor < 6
         #TODO raise specific exception
         raise "Redis version must be at least 2.6.0 you are running #{redis_version}"

data/resque-uniqueue.gemspec

@@ -5,23 +5,23 @@
 
 Gem::Specification.new do |s|
   s.name = "resque-uniqueue"
-  s.version = "0.2.2"
+  s.version = "0.3.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Aaron Scruggs"]
-  s.date = "2013-08-19"
+  s.date = "2014-04-20"
   s.description = "Unique Resque queues using redis 1.6.0 scripting, sets and not much else"
   s.email = "aaron@scrug.gs"
   s.extra_rdoc_files = [
     "LICENSE.txt",
-    "README.rdoc"
+    "README.md"
   ]
   s.files = [
     ".document",
     "Gemfile",
     "Gemfile.lock",
     "LICENSE.txt",
-    "README.rdoc",
+    "README.md",
     "Rakefile",
     "VERSION",
     "lib/resque-uniqueue.rb",
@@ -40,28 +40,28 @@ Gem::Specification.new do |s|
     s.specification_version = 3
 
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
-      s.add_runtime_dependency(%q<resque>, ["~> 1.24.1"])
+      s.add_runtime_dependency(%q<resque>, ["~> 1.25.0"])
       s.add_development_dependency(%q<mocha>, [">= 0"])
       s.add_development_dependency(%q<shoulda>, [">= 0"])
       s.add_development_dependency(%q<rdoc>, ["~> 3.12"])
-      s.add_development_dependency(%q<bundler>, ["~> 1.3.0"])
+      s.add_development_dependency(%q<bundler>, ["~> 1.5.0"])
       s.add_development_dependency(%q<jeweler>, ["~> 1.8.4"])
       s.add_development_dependency(%q<pry>, [">= 0"])
     else
-      s.add_dependency(%q<resque>, ["~> 1.24.1"])
+      s.add_dependency(%q<resque>, ["~> 1.25.0"])
       s.add_dependency(%q<mocha>, [">= 0"])
       s.add_dependency(%q<shoulda>, [">= 0"])
       s.add_dependency(%q<rdoc>, ["~> 3.12"])
-      s.add_dependency(%q<bundler>, ["~> 1.3.0"])
+      s.add_dependency(%q<bundler>, ["~> 1.5.0"])
       s.add_dependency(%q<jeweler>, ["~> 1.8.4"])
       s.add_dependency(%q<pry>, [">= 0"])
     end
   else
-    s.add_dependency(%q<resque>, ["~> 1.24.1"])
+    s.add_dependency(%q<resque>, ["~> 1.25.0"])
     s.add_dependency(%q<mocha>, [">= 0"])
     s.add_dependency(%q<shoulda>, [">= 0"])
     s.add_dependency(%q<rdoc>, ["~> 3.12"])
-    s.add_dependency(%q<bundler>, ["~> 1.3.0"])
+    s.add_dependency(%q<bundler>, ["~> 1.5.0"])
     s.add_dependency(%q<jeweler>, ["~> 1.8.4"])
     s.add_dependency(%q<pry>, [">= 0"])
   end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: resque-uniqueue
 version: !ruby/object:Gem::Version
-  version: 0.2.2
+  version: 0.3.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-08-19 00:00:00.000000000 Z
+date: 2014-04-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: resque
@@ -18,7 +18,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 1.24.1
+        version: 1.25.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -26,7 +26,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 1.24.1
+        version: 1.25.0
 - !ruby/object:Gem::Dependency
   name: mocha
   requirement: !ruby/object:Gem::Requirement
@@ -82,7 +82,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 1.3.0
+        version: 1.5.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -90,7 +90,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 1.3.0
+        version: 1.5.0
 - !ruby/object:Gem::Dependency
   name: jeweler
   requirement: !ruby/object:Gem::Requirement
@@ -129,13 +129,13 @@ executables: []
 extensions: []
 extra_rdoc_files:
 - LICENSE.txt
-- README.rdoc
+- README.md
 files:
 - .document
 - Gemfile
 - Gemfile.lock
 - LICENSE.txt
-- README.rdoc
+- README.md
 - Rakefile
 - VERSION
 - lib/resque-uniqueue.rb
@@ -158,7 +158,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -1795845455617588781
+      hash: -2465876499125034208
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:

data/README.rdoc

@@ -1,19 +0,0 @@
-= resque-uniqueue
-
-Description goes here.
-
-== Contributing to resque-uniqueue
- 
-* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
-* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
-* Fork the project.
-* Start a feature/bugfix branch.
-* Commit and push until you are happy with your contribution.
-* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
-* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
-
-== Copyright
-
-Copyright (c) 2013 AcademicWorks, inc. See LICENSE.txt for
-further details.
-