resque 1.19.0 → 1.20.0

data/HISTORY.md

@@ -1,3 +1,25 @@
+## 1.20.0 (2012-02-17)
+
+* Fixed demos for ruby 1.9 (@BMorearty, #445)
+* Fixed `#requeue` tests (@hone, #500)
+* Web UI: optional trailing slashes of URLs (@elisehuard, #449)
+* Allow * to appear anywhere in queue list (@tapajos, #405, #407)
+* Wait for child with specific PID (@jacobkg)
+* #decode raise takes a string when re-raising as a different exception class (Trevor Hart)
+* Use Sinatra's `pubilc_folder` if it exists (@defunkt, #420, #421)
+* Assign the job's worker before calling `before_fork` (@quirkey)
+* Fix Resque::Helpers#constantize to work correctly on 1.9.2 (@rtlong)
+* Added before & after hooks for dequeue (@humancopy, #398)
+* daemonize support using `ENV["BACKGROUND"]` (@chrisleishman)
+* requeue and remove failed jobs by queue name (@evanwhalen)
+* `-r` flag for resque-web for redis connection (@gjastrab)
+* Added `Resque.enqueue_to`: allows you to specif the queue and still run hooks (@dan-g)
+* Web UI: Set the default encoding to UTF-8 (@elubow)
+* fix finding worker pids on JRuby (John Andrews + Andrew Grieser)
+* Added distributed redis support (@stipple)
+* Added better failure hooks (@raykrueger)
+* Added before & after dequeue hooks (@humancopy)
+
 ## 1.19.0 (2011-09-01)
 
 * Added Airbrake (formerly Hoptoad) support.

data/README.markdown

@@ -210,7 +210,7 @@ loop do
   if job = reserve
     job.process
   else
-    sleep 5
+    sleep 5 # Polling frequency = 5 
   end
 end
 shutdown
@@ -265,6 +265,22 @@ worker process.  Use the PIDFILE option for easy access to the PID:
 
     $ PIDFILE=./resque.pid QUEUE=file_serve rake environment resque:work
 
+### Running in the background
+
+(Only supported with ruby >= 1.9). There are scenarios where it's helpful for
+the resque worker to run itself in the background (usually in combination with
+PIDFILE).  Use the BACKGROUND option so that rake will return as soon as the
+worker is started.
+
+    $ PIDFILE=./resque.pid BACKGROUND=yes QUEUE=file_serve \
+        rake environment resque:work
+
+### Polling frequency
+
+You can pass an INTERVAL option which is a float representing the polling frequency. 
+The default is 5 seconds, but for a semi-active app you may want to use a smaller value.
+
+    $ INTERVAL=0.1 QUEUE=file_serve rake environment resque:work
 
 ### Priorities and Queue Lists
 
@@ -445,6 +461,10 @@ You can also set the namespace directly using `resque-web`:
 
     $ resque-web -p 8282 -N myapp
 
+or set the Redis connection string if you need to do something like select a different database:
+
+    $ resque-web -p 8282 -r localhost:6379:2
+
 ### Passenger
 
 Using Passenger? Resque ships with a `config.ru` you can use. See
@@ -471,7 +491,7 @@ HTTP basic auth).
 
 ### Rails 3
 
-You can also easily mount Resque on a subpath in your existing Rails 3 app by adding this to your `routes.rb`:
+You can also mount Resque on a subpath in your existing Rails 3 app by adding `require 'resque/server'` to the top of your routes file or in an initializer then adding this to `routes.rb`:
 
 ``` ruby
 mount Resque::Server.new, :at => "/resque"
@@ -840,6 +860,8 @@ sort it out.
 Contributing
 ------------
 
+Read the [Contributing][cb] wiki page first. 
+
 Once you've made your great commits:
 
 1. [Fork][1] Resque
@@ -848,9 +870,6 @@ Once you've made your great commits:
 4. Create a [Pull Request](http://help.github.com/pull-requests/) from your branch
 5. That's it!
 
-You might want to checkout our [Contributing][cb] wiki page for information
-on coding standards, new features, etc.
-
 
 Mailing List
 ------------

data/bin/resque-web

@@ -20,4 +20,8 @@ Vegas::Runner.new(Resque::Server, 'resque-web', {
     runner.logger.info "Using Redis namespace '#{namespace}'"
     Resque.redis.namespace = namespace
   }
+  opts.on('-r redis-connection', "--redis redis-connection", "set the Redis connection string") {|redis_conf|
+    runner.logger.info "Using Redis connection '#{redis_conf}'"
+    Resque.redis = redis_conf
+  }
 end

data/lib/resque.rb

@@ -222,19 +222,37 @@ module Resque
   #
   # If no queue can be inferred this method will raise a `Resque::NoQueueError`
   #
+  # Returns true if the job was queued, nil if the job was rejected by a
+  # before_enqueue hook.
+  #
   # This method is considered part of the `stable` API.
   def enqueue(klass, *args)
+    enqueue_to(queue_from_class(klass), klass, *args)
+  end
+
+  # Just like `enqueue` but allows you to specify the queue you want to
+  # use. Runs hooks.
+  #
+  # `queue` should be the String name of the queue you're targeting.
+  #
+  # Returns true if the job was queued, nil if the job was rejected by a
+  # before_enqueue hook.
+  #
+  # This method is considered part of the `stable` API.
+  def enqueue_to(queue, klass, *args)
     # Perform before_enqueue hooks. Don't perform enqueue if any hook returns false
     before_hooks = Plugin.before_enqueue_hooks(klass).collect do |hook|
       klass.send(hook, *args)
     end
-    return if before_hooks.any? { |result| result == false }
+    return nil if before_hooks.any? { |result| result == false }
 
-    Job.create(queue_from_class(klass), klass, *args)
+    Job.create(queue, klass, *args)
 
     Plugin.after_enqueue_hooks(klass).each do |hook|
       klass.send(hook, *args)
     end
+
+    return true
   end
 
   # This method can be used to conveniently remove a job from a queue.
@@ -265,7 +283,17 @@ module Resque
   #
   # This method is considered part of the `stable` API.
   def dequeue(klass, *args)
+    # Perform before_dequeue hooks. Don't perform dequeue if any hook returns false
+    before_hooks = Plugin.before_dequeue_hooks(klass).collect do |hook|
+      klass.send(hook, *args)
+    end
+    return if before_hooks.any? { |result| result == false }
+
     Job.destroy(queue_from_class(klass), klass, *args)
+
+    Plugin.after_dequeue_hooks(klass).each do |hook|
+      klass.send(hook, *args)
+    end
   end
 
   # Given a class, try to extrapolate an appropriate queue based on a

data/lib/resque/failure.rb

@@ -66,5 +66,31 @@ module Resque
     def self.remove(index)
       backend.remove(index)
     end
+    
+    # Requeues all failed jobs in a specific queue.
+    # Queue name should be a string.
+    def self.requeue_queue(queue)
+      i=0
+      while job = Resque::Failure.all(i)
+        if job['queue'] == queue
+          Resque::Failure.requeue(i)
+        end  
+        i+=1
+      end
+    end
+
+    # Removes all failed jobs in a specific queue.
+    # Queue name should be a string.
+    def self.remove_queue(queue)
+      i=0
+      while job = Resque::Failure.all(i)
+        if job['queue'] == queue
+          # This will remove the failure from the array so do not increment the index.
+          Resque::Failure.remove(i)
+        else
+          i+=1    
+        end
+      end
+    end
   end
 end

data/lib/resque/helpers.rb

@@ -29,7 +29,7 @@ module Resque
       begin
         ::MultiJson.decode(object)
       rescue ::MultiJson::DecodeError => e
-        raise DecodeException, e
+        raise DecodeException, e.message, e.backtrace
       end
     end
 
@@ -40,9 +40,23 @@ module Resque
       dashed_word.split('-').each { |part| part[0] = part[0].chr.upcase }.join
     end
 
-    # Given a camel cased word, returns the constant it represents
+    # Tries to find a constant with the name specified in the argument string:
     #
-    # constantize('JobName') # => JobName
+    # constantize("Module") # => Module
+    # constantize("Test::Unit") # => Test::Unit
+    #
+    # The name is assumed to be the one of a top-level constant, no matter
+    # whether it starts with "::" or not. No lexical context is taken into
+    # account:
+    #
+    # C = 'outside'
+    # module M
+    #   C = 'inside'
+    #   C # => 'inside'
+    #   constantize("C") # => 'outside', same as ::C
+    # end
+    #
+    # NameError is raised when the constant is unknown.
     def constantize(camel_cased_word)
       camel_cased_word = camel_cased_word.to_s
 
@@ -55,7 +69,13 @@ module Resque
 
       constant = Object
       names.each do |name|
-        constant = constant.const_get(name) || constant.const_missing(name)
+        args = Module.method(:const_get).arity != 1 ? [false] : []
+
+        if constant.const_defined?(name, *args)
+          constant = constant.const_get(name)
+        else
+          constant = constant.const_missing(name)
+        end
       end
       constant
     end

data/lib/resque/job.rb

@@ -106,11 +106,6 @@ module Resque
       job_args = args || []
       job_was_performed = false
 
-      before_hooks  = Plugin.before_hooks(job)
-      around_hooks  = Plugin.around_hooks(job)
-      after_hooks   = Plugin.after_hooks(job)
-      failure_hooks = Plugin.failure_hooks(job)
-
       begin
         # Execute before_perform hook. Abort the job gracefully if
         # Resque::DontPerform is raised.
@@ -158,7 +153,7 @@ module Resque
       # If an exception occurs during the job execution, look for an
       # on_failure hook then re-raise.
       rescue Object => e
-        failure_hooks.each { |hook| job.send(hook, e, *job_args) }
+        run_failure_hooks(e)
         raise e
       end
     end
@@ -176,6 +171,7 @@ module Resque
     # Given an exception object, hands off the needed parameters to
     # the Failure module.
     def fail(exception)
+      run_failure_hooks(exception)
       Failure.create \
         :payload   => payload,
         :exception => exception,
@@ -201,5 +197,27 @@ module Resque
         payload_class == other.payload_class &&
         args == other.args
     end
+
+    def before_hooks
+      @before_hooks ||= Plugin.before_hooks(payload_class)
+    end
+
+    def around_hooks
+      @around_hooks ||= Plugin.around_hooks(payload_class)
+    end
+
+    def after_hooks
+      @after_hooks ||= Plugin.after_hooks(payload_class)
+    end
+
+    def failure_hooks 
+      @failure_hooks ||= Plugin.failure_hooks(payload_class)
+    end
+    
+    def run_failure_hooks(exception)
+      job_args = args || []
+      failure_hooks.each { |hook| payload_class.send(hook, exception, *job_args) }
+    end
+
   end
 end

data/lib/resque/plugin.rb

@@ -52,5 +52,15 @@ module Resque
     def before_enqueue_hooks(job)
       job.methods.grep(/^before_enqueue/).sort
     end
+
+    # Given an object, returns a list `after_dequeue` hook names.
+    def after_dequeue_hooks(job)
+      job.methods.grep(/^after_dequeue/).sort
+    end
+
+    # Given an object, returns a list `before_dequeue` hook names.
+    def before_dequeue_hooks(job)
+      job.methods.grep(/^before_dequeue/).sort
+    end
   end
 end

data/lib/resque/server.rb

@@ -4,12 +4,22 @@ require 'resque'
 require 'resque/version'
 require 'time'
 
+if defined? Encoding
+  Encoding.default_external = Encoding::UTF_8
+end
+
 module Resque
   class Server < Sinatra::Base
     dir = File.dirname(File.expand_path(__FILE__))
 
     set :views,  "#{dir}/server/views"
-    set :public, "#{dir}/server/public"
+
+    if respond_to? :public_folder
+      set :public_folder, "#{dir}/server/public"
+    else
+      set :public, "#{dir}/server/public"
+    end
+
     set :static, true
 
     helpers do
@@ -139,21 +149,21 @@ module Resque
     end
 
     %w( overview workers ).each do |page|
-      get "/#{page}.poll" do
+      get "/#{page}.poll/?" do
         show_for_polling(page)
       end
 
-      get "/#{page}/:id.poll" do
+      get "/#{page}/:id.poll/?" do
         show_for_polling(page)
       end
     end
 
     %w( overview queues working workers key ).each do |page|
-      get "/#{page}" do
+      get "/#{page}/?" do
         show page
       end
 
-      get "/#{page}/:id" do
+      get "/#{page}/:id/?" do
         show page
       end
     end
@@ -163,7 +173,7 @@ module Resque
       redirect u('queues')
     end
 
-    get "/failed" do
+    get "/failed/?" do
       if Resque::Failure.url
         redirect Resque::Failure.url
       else
@@ -183,7 +193,7 @@ module Resque
       redirect u('failed')
     end
 
-    get "/failed/requeue/:index" do
+    get "/failed/requeue/:index/?" do
       Resque::Failure.requeue(params[:index])
       if request.xhr?
         return Resque::Failure.all(params[:index])['retried_at']
@@ -192,24 +202,24 @@ module Resque
       end
     end
 
-    get "/failed/remove/:index" do
+    get "/failed/remove/:index/?" do
       Resque::Failure.remove(params[:index])
       redirect u('failed')
     end
 
-    get "/stats" do
+    get "/stats/?" do
       redirect url_path("/stats/resque")
     end
 
-    get "/stats/:id" do
+    get "/stats/:id/?" do
       show :stats
     end
 
-    get "/stats/keys/:key" do
+    get "/stats/keys/:key/?" do
       show :stats
     end
 
-    get "/stats.txt" do
+    get "/stats.txt/?" do
       info = Resque.info
 
       stats = []

data/lib/resque/server/public/style.css

@@ -41,6 +41,7 @@ body { padding:0; margin:0; }
 #main table.jobs td.args{ width:50%;}
 
 #main table.workers td.icon {width:1%; background:#efefef;text-align:center;}
+#main table.workers td.icon img { height: 16px; width: 16px; }
 #main table.workers td.where { width:25%;}
 #main table.workers td.queues { width:35%;}
 #main .queue-tag { background:#b1d2e9; padding:2px; margin:0 3px; font-size:80%; text-decoration:none; text-transform:uppercase; font-weight:bold; color:#3274a2; -webkit-border-radius:4px; -moz-border-radius:4px;}

data/lib/resque/tasks.rb

@@ -18,6 +18,13 @@ namespace :resque do
       abort "set QUEUE env var, e.g. $ QUEUE=critical,high rake resque:work"
     end
 
+    if ENV['BACKGROUND']
+      unless Process.respond_to?('daemon')
+          abort "env var BACKGROUND is set, which requires ruby >= 1.9"
+      end
+      Process.daemon(true)
+    end
+
     if ENV['PIDFILE']
       File.open(ENV['PIDFILE'], 'w') { |f| f << worker.pid }
     end

data/lib/resque/version.rb

@@ -1,3 +1,3 @@
 module Resque
-  Version = VERSION = '1.19.0'
+  Version = VERSION = '1.20.0'
 end

data/lib/resque/worker.rb

@@ -35,9 +35,19 @@ module Resque
 
       names.map! { |name| "worker:#{name}" }
 
-      reportedly_working = redis.mapped_mget(*names).reject do |key, value|
-        value.nil? || value.empty?
+      reportedly_working = {}
+
+      begin
+        reportedly_working = redis.mapped_mget(*names).reject do |key, value|
+          value.nil? || value.empty?
+        end
+      rescue Redis::Distributed::CannotDistribute
+        names.each do |name|
+          value = redis.get name
+          reportedly_working[name] = value unless value.nil? || value.empty?
+        end
       end
+
       reportedly_working.keys.map do |key|
         find key.sub("worker:", '')
       end.compact
@@ -118,13 +128,14 @@ module Resque
 
         if not paused? and job = reserve
           log "got: #{job.inspect}"
+          job.worker = self
           run_hook :before_fork, job
           working_on job
 
           if @child = fork
             srand # Reseeding
             procline "Forked #{@child} at #{Time.now.to_i}"
-            Process.wait
+            Process.wait(@child)
           else
             procline "Processing #{job.queue} since #{Time.now.to_i}"
             perform(job, &block)
@@ -150,6 +161,7 @@ module Resque
     def process(job = nil, &block)
       return unless job ||= reserve
 
+      job.worker = self
       working_on job
       perform(job, &block)
     ensure
@@ -181,7 +193,7 @@ module Resque
     def reserve
       queues.each do |queue|
         log! "Checking #{queue}"
-        if job = Resque::Job.reserve(queue)
+        if job = Resque.reserve(queue)
           log! "Found job on #{queue}"
           return job
         end
@@ -198,7 +210,7 @@ module Resque
     # A splat ("*") means you want every queue (in alpha order) - this
     # can be useful for dynamically adding new queues.
     def queues
-      @queues[0] == "*" ? Resque.queues.sort : @queues
+      @queues.map {|queue| queue == "*" ? Resque.queues.sort : queue }.flatten.uniq
     end
 
     # Not every platform supports fork. Here we do our magic to
@@ -378,7 +390,6 @@ module Resque
     # Given a job, tells Redis we're working on it. Useful for seeing
     # what workers are doing and when.
     def working_on(job)
-      job.worker = self
       data = encode \
         :queue   => job.queue,
         :run_at  => Time.now.strftime("%Y/%m/%d %H:%M:%S %Z"),
@@ -470,7 +481,7 @@ module Resque
 
     # Returns Integer PID of running worker
     def pid
-      @pid ||= to_s.split(":")[1].to_i
+      Process.pid
     end
 
     # Returns an Array of string pids of all the other workers on this
@@ -488,7 +499,7 @@ module Resque
     # Returns an Array of string pids of all the other workers on this
     # machine. Useful when pruning dead workers on startup.
     def linux_worker_pids
-      `ps -A -o pid,command | grep [r]esque | grep -v "resque-web"`.split("\n").map do |line|
+      `ps -A -o pid,command | grep "[r]esque" | grep -v "resque-web"`.split("\n").map do |line|
         line.split(' ')[0]
       end
     end
@@ -498,7 +509,7 @@ module Resque
     # Returns an Array of string pids of all the other workers on this
     # machine. Useful when pruning dead workers on startup.
     def solaris_worker_pids
-      `ps -A -o pid,comm | grep [r]uby | grep -v "resque-web"`.split("\n").map do |line|
+      `ps -A -o pid,comm | grep "[r]uby" | grep -v "resque-web"`.split("\n").map do |line|
         real_pid = line.split(' ')[0]
         pargs_command = `pargs -a #{real_pid} 2>/dev/null | grep [r]esque | grep -v "resque-web"`
         if pargs_command.split(':')[1] == " resque-#{Resque::Version}"

data/test/job_hooks_test.rb

@@ -277,7 +277,7 @@ context "Resque::Job before_enqueue" do
   test "the before enqueue hook should run" do
     history = []
     @worker = Resque::Worker.new(:jobs)
-    Resque.enqueue(BeforeEnqueueJob, history)
+    assert Resque.enqueue(BeforeEnqueueJob, history)
     @worker.work(0)
     assert_equal history, [:before_enqueue], "before_enqueue was not run"
   end
@@ -285,11 +285,71 @@ context "Resque::Job before_enqueue" do
   test "a before enqueue hook that returns false should prevent the job from getting queued" do
     history = []
     @worker = Resque::Worker.new(:jobs)
-    Resque.enqueue(BeforeEnqueueJobAbort, history)
+    assert_nil Resque.enqueue(BeforeEnqueueJobAbort, history)
     assert_equal 0, Resque.size(:jobs)
   end
 end
 
+context "Resque::Job after_dequeue" do
+  include PerformJob
+
+  class ::AfterDequeueJob
+    @queue = :jobs
+    def self.after_dequeue_record_history(history)
+      history << :after_dequeue
+    end
+
+    def self.perform(history)
+    end
+  end
+
+  test "the after dequeue hook should run" do
+    history = []
+    @worker = Resque::Worker.new(:jobs)
+    Resque.dequeue(AfterDequeueJob, history)
+    @worker.work(0)
+    assert_equal history, [:after_dequeue], "after_dequeue was not run"
+  end
+end
+
+
+context "Resque::Job before_dequeue" do
+  include PerformJob
+
+  class ::BeforeDequeueJob
+    @queue = :jobs
+    def self.before_dequeue_record_history(history)
+      history << :before_dequeue
+    end
+
+    def self.perform(history)
+    end
+  end
+
+  class ::BeforeDequeueJobAbort
+    @queue = :jobs
+    def self.before_dequeue_abort(history)
+      false
+    end
+
+    def self.perform(history)
+    end
+  end
+
+  test "the before dequeue hook should run" do
+    history = []
+    @worker = Resque::Worker.new(:jobs)
+    Resque.dequeue(BeforeDequeueJob, history)
+    @worker.work(0)
+    assert_equal history, [:before_dequeue], "before_dequeue was not run"
+  end
+
+  test "a before dequeue hook that returns false should prevent the job from getting dequeued" do
+    history = []
+    assert_equal nil, Resque.dequeue(BeforeDequeueJobAbort, history)
+  end
+end
+
 context "Resque::Job all hooks" do
   include PerformJob
 

data/test/redis-test-cluster.conf

@@ -0,0 +1,115 @@
+# Redis configuration file example
+
+# By default Redis does not run as a daemon. Use 'yes' if you need it.
+# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
+daemonize yes
+
+# When run as a daemon, Redis write a pid file in /var/run/redis.pid by default.
+# You can specify a custom pid file location here.
+pidfile ./test/redis-test-cluster.pid
+
+# Accept connections on the specified port, default is 6379
+port 9737
+
+# If you want you can bind a single interface, if the bind option is not
+# specified all the interfaces will listen for connections.
+#
+# bind 127.0.0.1
+
+# Close the connection after a client is idle for N seconds (0 to disable)
+timeout 300
+
+# Save the DB on disk:
+#
+#   save <seconds> <changes>
+#
+#   Will save the DB if both the given number of seconds and the given
+#   number of write operations against the DB occurred.
+#
+#   In the example below the behaviour will be to save:
+#   after 900 sec (15 min) if at least 1 key changed
+#   after 300 sec (5 min) if at least 10 keys changed
+#   after 60 sec if at least 10000 keys changed
+save 900 1
+save 300 10
+save 60 10000
+
+# The filename where to dump the DB
+dbfilename dump-cluster.rdb
+
+# For default save/load DB in/from the working directory
+# Note that you must specify a directory not a file name.
+dir ./test/
+
+# Set server verbosity to 'debug'
+# it can be one of:
+# debug (a lot of information, useful for development/testing)
+# notice (moderately verbose, what you want in production probably)
+# warning (only very important / critical messages are logged)
+loglevel debug
+
+# Specify the log file name. Also 'stdout' can be used to force
+# the demon to log on the standard output. Note that if you use standard
+# output for logging but daemonize, logs will be sent to /dev/null
+logfile stdout
+
+# Set the number of databases. The default database is DB 0, you can select
+# a different one on a per-connection basis using SELECT <dbid> where
+# dbid is a number between 0 and 'databases'-1
+databases 16
+
+################################# REPLICATION #################################
+
+# Master-Slave replication. Use slaveof to make a Redis instance a copy of
+# another Redis server. Note that the configuration is local to the slave
+# so for example it is possible to configure the slave to save the DB with a
+# different interval, or to listen to another port, and so on.
+
+# slaveof <masterip> <masterport>
+
+################################## SECURITY ###################################
+
+# Require clients to issue AUTH <PASSWORD> before processing any other
+# commands.  This might be useful in environments in which you do not trust
+# others with access to the host running redis-server.
+#
+# This should stay commented out for backward compatibility and because most
+# people do not need auth (e.g. they run their own servers).
+
+# requirepass foobared
+
+################################### LIMITS ####################################
+
+# Set the max number of connected clients at the same time. By default there
+# is no limit, and it's up to the number of file descriptors the Redis process
+# is able to open. The special value '0' means no limts.
+# Once the limit is reached Redis will close all the new connections sending
+# an error 'max number of clients reached'.
+
+# maxclients 128
+
+# Don't use more memory than the specified amount of bytes.
+# When the memory limit is reached Redis will try to remove keys with an
+# EXPIRE set. It will try to start freeing keys that are going to expire
+# in little time and preserve keys with a longer time to live.
+# Redis will also try to remove objects from free lists if possible.
+#
+# If all this fails, Redis will start to reply with errors to commands
+# that will use more memory, like SET, LPUSH, and so on, and will continue
+# to reply to most read-only commands like GET.
+#
+# WARNING: maxmemory can be a good idea mainly if you want to use Redis as a
+# 'state' server or cache, not as a real DB. When Redis is used as a real
+# database the memory usage will grow over the weeks, it will be obvious if
+# it is going to use too much memory in the long run, and you'll have the time
+# to upgrade. With maxmemory after the limit is reached you'll start to get
+# errors for write operations, and this may even lead to DB inconsistency.
+
+# maxmemory <bytes>
+
+############################### ADVANCED CONFIG ###############################
+
+# Glue small output buffers together in order to send small replies in a
+# single TCP packet. Uses a bit more CPU but most of the times it is a win
+# in terms of number of queries per second. Use 'yes' if unsure.
+glueoutputbuf yes

data/test/resque-web_test.rb

@@ -51,3 +51,9 @@ context "on GET to /stats/resque" do
 
   should_respond_with_success
 end
+
+context "also works with slash at the end" do
+  setup { get "/working/" }
+
+  should_respond_with_success
+end

data/test/resque_test.rb

@@ -7,6 +7,11 @@ context "Resque" do
     Resque.push(:people, { 'name' => 'chris' })
     Resque.push(:people, { 'name' => 'bob' })
     Resque.push(:people, { 'name' => 'mark' })
+    @original_redis = Resque.redis
+  end
+
+  teardown do
+    Resque.redis = @original_redis
   end
 
   test "can set a namespace through a url-like string" do
@@ -131,6 +136,16 @@ context "Resque" do
     assert_equal nil, Resque.reserve(:method)
   end
 
+  test "can define a queue for jobs by way of a method" do
+    assert_equal 0, Resque.size(:method)
+    assert Resque.enqueue_to(:new_queue, SomeMethodJob, 20, '/tmp')
+
+    job = Resque.reserve(:new_queue)
+    assert_equal SomeMethodJob, job.payload_class
+    assert_equal 20, job.args[0]
+    assert_equal '/tmp', job.args[1]
+  end
+
   test "needs to infer a queue with enqueue" do
     assert_raises Resque::NoQueueError do
       Resque.enqueue(SomeJob, 20, '/tmp')
@@ -201,7 +216,7 @@ context "Resque" do
   end
 
   test "keeps track of resque keys" do
-    assert_equal ["queue:people", "queues"], Resque.keys
+    assert_equal ["queue:people", "queues"].sort, Resque.keys.sort
   end
 
   test "badly wants a class name, too" do
@@ -238,7 +253,11 @@ context "Resque" do
     assert_equal 3, stats[:queues]
     assert_equal 3, stats[:processed]
     assert_equal 1, stats[:failed]
-    assert_equal [Resque.redis.respond_to?(:server) ? 'localhost:9736' : 'redis://localhost:9736/0'], stats[:servers]
+    if ENV.key? 'RESQUE_DISTRIBUTED'
+      assert_equal [Resque.redis.respond_to?(:server) ? 'localhost:9736, localhost:9737' : 'redis://localhost:9736/0, redis://localhost:9737/0'], stats[:servers]
+    else
+      assert_equal [Resque.redis.respond_to?(:server) ? 'localhost:9736' : 'redis://localhost:9736/0'], stats[:servers]
+    end
   end
 
   test "decode bad json" do

data/test/test_helper.rb

@@ -39,16 +39,26 @@ at_exit do
     exit_code = Test::Unit::AutoRunner.run
   end
 
-  pid = `ps -A -o pid,command | grep [r]edis-test`.split(" ")[0]
+  processes = `ps -A -o pid,command | grep [r]edis-test`.split("\n")
+  pids = processes.map { |process| process.split(" ")[0] }
   puts "Killing test redis server..."
-  `rm -f #{dir}/dump.rdb`
-  Process.kill("KILL", pid.to_i)
+  `rm -f #{dir}/dump.rdb #{dir}/dump-cluster.rdb`
+  pids.each { |pid| Process.kill("KILL", pid.to_i) }
   exit exit_code
 end
 
-puts "Starting redis for testing at localhost:9736..."
-`redis-server #{dir}/redis-test.conf`
-Resque.redis = 'localhost:9736'
+if ENV.key? 'RESQUE_DISTRIBUTED'
+  require 'redis/distributed'
+  puts "Starting redis for testing at localhost:9736 and localhost:9737..."
+  `redis-server #{dir}/redis-test.conf`
+  `redis-server #{dir}/redis-test-cluster.conf`
+  r = Redis::Distributed.new(['redis://localhost:9736', 'redis://localhost:9737'])
+  Resque.redis = Redis::Namespace.new :resque, :redis => r
+else
+  puts "Starting redis for testing at localhost:9736..."
+  `redis-server #{dir}/redis-test.conf`
+  Resque.redis = 'localhost:9736'
+end
 
 
 ##

data/test/worker_test.rb

@@ -33,12 +33,30 @@ context "Resque::Worker" do
   end
 
   test "fails uncompleted jobs on exit" do
-    job = Resque::Job.new(:jobs, [GoodJob, "blah"])
+    job = Resque::Job.new(:jobs, {'class' => 'GoodJob', 'args' => "blah"})
     @worker.working_on(job)
     @worker.unregister_worker
     assert_equal 1, Resque::Failure.count
   end
 
+  class ::SimpleJobWithFailureHandling
+    def self.on_failure_record_failure(exception, *job_args)
+      @@exception = exception
+    end
+    
+    def self.exception
+      @@exception
+    end
+  end
+
+  test "fails uncompleted jobs on exit, and calls failure hook" do
+    job = Resque::Job.new(:jobs, {'class' => 'SimpleJobWithFailureHandling', 'args' => ""})
+    @worker.working_on(job)
+    @worker.unregister_worker
+    assert_equal 1, Resque::Failure.count
+    assert(SimpleJobWithFailureHandling.exception.kind_of?(Resque::DirtyExit))
+  end
+
   test "can peek at failed jobs" do
     10.times { Resque::Job.create(:jobs, BadJob) }
     @worker.work(0)
@@ -97,6 +115,36 @@ context "Resque::Worker" do
     assert_equal 0, Resque.size(:blahblah)
   end
 
+  test "can work with wildcard at the end of the list" do
+    Resque::Job.create(:high, GoodJob)
+    Resque::Job.create(:critical, GoodJob)
+    Resque::Job.create(:blahblah, GoodJob)
+    Resque::Job.create(:beer, GoodJob)
+
+    worker = Resque::Worker.new(:critical, :high, "*")
+
+    worker.work(0)
+    assert_equal 0, Resque.size(:high)
+    assert_equal 0, Resque.size(:critical)
+    assert_equal 0, Resque.size(:blahblah)
+    assert_equal 0, Resque.size(:beer)
+  end
+
+  test "can work with wildcard at the middle of the list" do
+    Resque::Job.create(:high, GoodJob)
+    Resque::Job.create(:critical, GoodJob)
+    Resque::Job.create(:blahblah, GoodJob)
+    Resque::Job.create(:beer, GoodJob)
+
+    worker = Resque::Worker.new(:critical, "*", :high)
+
+    worker.work(0)
+    assert_equal 0, Resque.size(:high)
+    assert_equal 0, Resque.size(:critical)
+    assert_equal 0, Resque.size(:blahblah)
+    assert_equal 0, Resque.size(:beer)
+  end
+
   test "processes * queues in alphabetical order" do
     Resque::Job.create(:high, GoodJob)
     Resque::Job.create(:critical, GoodJob)
@@ -267,6 +315,11 @@ context "Resque::Worker" do
     end
   end
 
+  test "worker_pids returns pids" do
+    known_workers = @worker.worker_pids
+    assert !known_workers.empty?
+  end
+
   test "Processed jobs count" do
     @worker.work(0)
     assert_equal 1, Resque.info[:processed]
@@ -329,4 +382,24 @@ context "Resque::Worker" do
   test "returns PID of running process" do
     assert_equal @worker.to_s.split(":")[1].to_i, @worker.pid
   end
+  
+  test "requeue failed queue" do
+    queue = 'good_job'
+    Resque::Failure.create(:exception => Exception.new, :worker => Resque::Worker.new(queue), :queue => queue, :payload => {'class' => 'GoodJob'})
+    Resque::Failure.create(:exception => Exception.new, :worker => Resque::Worker.new(queue), :queue => 'some_job', :payload => {'class' => 'SomeJob'})
+    Resque::Failure.requeue_queue(queue)
+    assert Resque::Failure.all(0).has_key?('retried_at')
+    assert !Resque::Failure.all(1).has_key?('retried_at')
+  end
+
+  test "remove failed queue" do
+    queue = 'good_job'
+    queue2 = 'some_job'
+    Resque::Failure.create(:exception => Exception.new, :worker => Resque::Worker.new(queue), :queue => queue, :payload => {'class' => 'GoodJob'})
+    Resque::Failure.create(:exception => Exception.new, :worker => Resque::Worker.new(queue2), :queue => queue2, :payload => {'class' => 'SomeJob'})
+    Resque::Failure.create(:exception => Exception.new, :worker => Resque::Worker.new(queue), :queue => queue, :payload => {'class' => 'GoodJob'})
+    Resque::Failure.remove_queue(queue)
+    assert_equal queue2, Resque::Failure.all(0)['queue']
+    assert_equal 1, Resque::Failure.count
+  end
 end

metadata

@@ -1,21 +1,22 @@
 --- !ruby/object:Gem::Specification 
 name: resque
 version: !ruby/object:Gem::Version 
-  prerelease: false
+  hash: 71
+  prerelease: 
   segments: 
   - 1
-  - 19
+  - 20
   - 0
-  version: 1.19.0
+  version: 1.20.0
 platform: ruby
 authors: 
 - Chris Wanstrath
+- Terence Lee
 autorequire: 
 bindir: bin
 cert_chain: []
 
-date: 2011-09-02 00:00:00 -07:00
-default_executable: 
+date: 2012-02-17 00:00:00 Z
 dependencies: 
 - !ruby/object:Gem::Dependency 
   name: redis-namespace
@@ -25,6 +26,7 @@ dependencies:
     requirements: 
     - - ~>
       - !ruby/object:Gem::Version 
+        hash: 19
         segments: 
         - 1
         - 0
@@ -40,6 +42,7 @@ dependencies:
     requirements: 
     - - ~>
       - !ruby/object:Gem::Version 
+        hash: 31
         segments: 
         - 0
         - 1
@@ -55,6 +58,7 @@ dependencies:
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
+        hash: 63
         segments: 
         - 0
         - 9
@@ -70,6 +74,7 @@ dependencies:
     requirements: 
     - - ~>
       - !ruby/object:Gem::Version 
+        hash: 15
         segments: 
         - 1
         - 0
@@ -91,59 +96,59 @@ files:
 - Rakefile
 - LICENSE
 - HISTORY.md
-- lib/resque/errors.rb
-- lib/resque/failure/airbrake.rb
-- lib/resque/failure/base.rb
-- lib/resque/failure/hoptoad.rb
-- lib/resque/failure/multiple.rb
-- lib/resque/failure/redis.rb
-- lib/resque/failure/thoughtbot.rb
-- lib/resque/failure.rb
-- lib/resque/helpers.rb
+- lib/tasks/redis.rake
+- lib/tasks/resque.rake
 - lib/resque/job.rb
-- lib/resque/plugin.rb
+- lib/resque/server/views/workers.erb
+- lib/resque/server/views/next_more.erb
+- lib/resque/server/views/key_string.erb
+- lib/resque/server/views/error.erb
+- lib/resque/server/views/layout.erb
+- lib/resque/server/views/overview.erb
+- lib/resque/server/views/stats.erb
+- lib/resque/server/views/failed.erb
+- lib/resque/server/views/key_sets.erb
+- lib/resque/server/views/working.erb
+- lib/resque/server/views/queues.erb
+- lib/resque/server/public/jquery-1.3.2.min.js
+- lib/resque/server/public/working.png
 - lib/resque/server/public/favicon.ico
 - lib/resque/server/public/idle.png
-- lib/resque/server/public/jquery-1.3.2.min.js
-- lib/resque/server/public/jquery.relatize_date.js
-- lib/resque/server/public/poll.png
 - lib/resque/server/public/ranger.js
 - lib/resque/server/public/reset.css
 - lib/resque/server/public/style.css
-- lib/resque/server/public/working.png
+- lib/resque/server/public/jquery.relatize_date.js
+- lib/resque/server/public/poll.png
 - lib/resque/server/test_helper.rb
-- lib/resque/server/views/error.erb
-- lib/resque/server/views/failed.erb
-- lib/resque/server/views/key_sets.erb
-- lib/resque/server/views/key_string.erb
-- lib/resque/server/views/layout.erb
-- lib/resque/server/views/next_more.erb
-- lib/resque/server/views/overview.erb
-- lib/resque/server/views/queues.erb
-- lib/resque/server/views/stats.erb
-- lib/resque/server/views/workers.erb
-- lib/resque/server/views/working.erb
-- lib/resque/server.rb
-- lib/resque/stat.rb
-- lib/resque/tasks.rb
 - lib/resque/version.rb
+- lib/resque/failure.rb
 - lib/resque/worker.rb
+- lib/resque/errors.rb
+- lib/resque/plugin.rb
+- lib/resque/server.rb
+- lib/resque/helpers.rb
+- lib/resque/tasks.rb
+- lib/resque/stat.rb
+- lib/resque/failure/hoptoad.rb
+- lib/resque/failure/thoughtbot.rb
+- lib/resque/failure/multiple.rb
+- lib/resque/failure/redis.rb
+- lib/resque/failure/base.rb
+- lib/resque/failure/airbrake.rb
 - lib/resque.rb
-- lib/tasks/redis.rake
-- lib/tasks/resque.rake
-- bin/resque
 - bin/resque-web
+- bin/resque
 - test/airbrake_test.rb
-- test/hoptoad_test.rb
 - test/job_hooks_test.rb
-- test/job_plugins_test.rb
 - test/plugin_test.rb
-- test/redis-test.conf
+- test/redis-test-cluster.conf
 - test/resque-web_test.rb
-- test/resque_test.rb
-- test/test_helper.rb
+- test/redis-test.conf
 - test/worker_test.rb
-has_rdoc: true
+- test/test_helper.rb
+- test/hoptoad_test.rb
+- test/resque_test.rb
+- test/job_plugins_test.rb
 homepage: http://github.com/defunkt/resque
 licenses: []
 
@@ -157,6 +162,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
+      hash: 3
       segments: 
       - 0
       version: "0"
@@ -165,13 +171,14 @@ required_rubygems_version: !ruby/object:Gem::Requirement
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
+      hash: 3
       segments: 
       - 0
       version: "0"
 requirements: []
 
 rubyforge_project: 
-rubygems_version: 1.3.7
+rubygems_version: 1.8.11
 signing_key: 
 specification_version: 3
 summary: Resque is a Redis-backed queueing system.