resque 1.24.1 → 1.25.0.pre

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 38e90902e4b978a24840fc539e704cb215f5dd2a
-  data.tar.gz: 26a7b965a985bc1081301509ec15d2ef73212638
+  metadata.gz: e28312bcc43dec49cd4dae52b64416413436da85
+  data.tar.gz: 489f3640b552d6d9ea6987732b86427a42aec182
 SHA512:
-  metadata.gz: 22860bfff69fb573382a48b1a77d5bf7b8f180f4d3ac03096e5a70272cc467c9d9c5f4b764ebf5d22b91c38933f63d1afd2cf98fd3cdf029b7d30c43e97130eb
-  data.tar.gz: abf28fbbc53bd5f02ee78aa954b87b8c00f32f3c75bdd7e79d977120071975125630038274ea166f2ab0304de94caa8d454f68c3d294a5d932a44c9e59021bc9
+  metadata.gz: 3c9d30888178238ed2117f3991d83d204a7cbec53943631637353bc62103b0cd0b63fd2fe93ce60657ed883f1f4664e947745f419f1398671dee5c034fbc3aca
+  data.tar.gz: 9c7fda8c5677603ccde7ef5c2c35b19d6b7106d6309e1d3c032afc03730e7db8d80c50400924785d46376784cfc87f2a28797399f4385381d56b667ea10a6f0e

data/HISTORY.md

@@ -1,3 +1,28 @@
+## 1.25.0 (TBD)
+* Updates fork method so [resque-multi-job-forks](https://github.com/stulentsev/resque-multi-job-forks)
+  monkey patching works again. See discussion at https://github.com/defunkt/resque/pull/895 for more
+  context (@jonhyman)
+* Use Redis.pipelined to group batches of redis commands.
+  https://github.com/resque/resque/pull/902 (@jonhyman)
+* Fixed uninitialize constant for the module/class that contains the perform 
+  method causing job failures to no be reported, #792 (@sideshowcoder)
+* Fix Resque::Failure::Base.all to have the correct signature.
+  (@rentalutions)
+* Don't close stdio pipes when daemonizing so as to not hide errors. #967
+  (@sideshowcoder)
+* Fix for worker_pids on Windows. #980 (@kzgs)
+* Only prune workers for queues the current worker knows about. #1000
+  (!) (@dsabanin)
+* Handle duplicate TERM signals. #988 (@softwaregravy)
+* Fix issue with exiting workers and unintentionally deregistering the
+  parent when the forked child exits. #1017 (@magec)
+* Fix encoding errors with local date formatting. #1065 (@katafrakt)
+* Fix CI for 1.8.7 and 1.9.2 modes due to dependencies. #1090
+  (@adelcambre)
+* Allow using globs for queue names to listen on, allowing things like
+  listening on `staging_*`. #1085 (@adelcambre)
+
+
 ## 1.24.1 (2013-3-23)
 
 * Adds a default value for `per_page` on resque-server for plugins which add tabs (@jonhyman)
@@ -37,7 +62,6 @@
   typing purposes (jonhyman)
 * Fix log formatters not appending a new line (flavorpill)
 * redirect unauthorized resque-web polling requests to root url (trliner)
-* Disable forking with FORK_PER_JOB=false (@tarcieri)
 * Various resque-web fixes (@tarcieri)
 * Optional RedisMultiQueue failure backend, can be enabled with
   FAILURE_BACKEND=redis_multi_queue env var (@tarcieri)

data/README.markdown

@@ -193,7 +193,7 @@ If a job raises an exception, it is logged and handed off to the
 `Resque::Failure` module. Failures are logged either locally in Redis
 or using some different backend.
 
-For example, Resque ships with Hoptoad support.
+For example, Resque ships with Airbrake support.
 
 Keep this in mind when writing your jobs: you may want to throw
 exceptions you would not normally throw in order to assist debugging.
@@ -338,7 +338,7 @@ so using the `resque:workers` rake task:
 
     $ COUNT=5 QUEUE=* rake resque:workers
 
-This will spawn five Resque workers, each in its own thread. Hitting
+This will spawn five Resque workers, each in its own process. Hitting
 ctrl-c should be sufficient to stop them all.
 
 
@@ -543,48 +543,6 @@ Choose DelayedJob if:
 In no way is Resque a "better" DelayedJob, so make sure you pick the
 tool that's best for your app.
 
-
-Installing Redis
-----------------
-
-Resque requires Redis 0.900 or higher.
-
-Resque uses Redis' lists for its queues. It also stores worker state
-data in Redis.
-
-#### Homebrew
-
-If you're on OS X, Homebrew is the simplest way to install Redis:
-
-    $ brew install redis
-    $ redis-server /usr/local/etc/redis.conf
-
-You now have a Redis daemon running on 6379.
-
-#### Via Resque
-
-Resque includes Rake tasks (thanks to Ezra's redis-rb) that will
-install and run Redis for you:
-
-    $ git clone git://github.com/defunkt/resque.git
-    $ cd resque
-    $ rake redis:install dtach:install
-    $ rake redis:start
-
-Or, if you don't have admin access on your machine:
-
-    $ git clone git://github.com/defunkt/resque.git
-    $ cd resque
-    $ PREFIX=<your_prefix> rake redis:install dtach:install
-    $ rake redis:start
-
-You now have Redis running on 6379. Wait a second then hit ctrl-\ to
-detach and keep it running in the background.
-
-The demo is probably the best way to figure out how to put the parts
-together. But, it's not that hard.
-
-
 Resque Dependencies
 -------------------
 
@@ -662,7 +620,7 @@ Don't forget you can define a `resque:setup` hook in
 
 ### In a Rails 2.x app, as a plugin
 
-    $ ./script/plugin install git://github.com/defunkt/resque
+    $ ./script/plugin install git://github.com/resque/resque
 
 That's it! Resque will automatically be available when your Rails app
 loads.
@@ -764,11 +722,11 @@ Plugins and Hooks
 -----------------
 
 For a list of available plugins see
-<http://wiki.github.com/defunkt/resque/plugins>.
+<http://wiki.github.com/resque/resque/plugins>.
 
 If you'd like to write your own plugin, or want to customize Resque
 using hooks (such as `Resque.after_fork`), see
-[docs/HOOKS.md](http://github.com/defunkt/resque/blob/master/docs/HOOKS.md).
+[docs/HOOKS.md](http://github.com/resque/resque/blob/master/docs/HOOKS.md).
 
 
 Namespaces
@@ -820,7 +778,7 @@ send patches for any tweaks or improvements you can make to it.
 Questions
 ---------
 
-Please add them to the [FAQ](https://github.com/defunkt/resque/wiki/FAQ) or
+Please add them to the [FAQ](https://github.com/resque/resque/wiki/FAQ) or
 ask on the Mailing List. The Mailing List is explained further below
 
 
@@ -831,7 +789,7 @@ Want to hack on Resque?
 
 First clone the repo and run the tests:
 
-    git clone git://github.com/defunkt/resque.git
+    git clone git://github.com/resque/resque.git
     cd resque
     rake test
 
@@ -884,10 +842,10 @@ The archive can be found at <http://librelist.com/browser/resque/>.
 Meta
 ----
 
-* Code: `git clone git://github.com/defunkt/resque.git`
-* Home: <http://github.com/defunkt/resque>
-* Docs: <http://defunkt.github.com/resque/>
-* Bugs: <http://github.com/defunkt/resque/issues>
+* Code: `git clone git://github.com/resque/resque.git`
+* Home: <http://github.com/resque/resque>
+* Docs: <http://resque.github.com/resque/>
+* Bugs: <http://github.com/resque/resque/issues>
 * List: <resque@librelist.com>
 * Chat: <irc://irc.freenode.net/resque>
 * Gems: <http://gemcutter.org/gems/resque>
@@ -902,7 +860,7 @@ Chris Wanstrath :: chris@ozmm.org :: @defunkt
 
 [0]: http://github.com/blog/542-introducing-resque
 [1]: http://help.github.com/forking/
-[2]: http://github.com/defunkt/resque/issues
+[2]: http://github.com/resque/resque/issues
 [sv]: http://semver.org/
-[rs]: http://github.com/defunkt/redis-namespace
-[cb]: http://wiki.github.com/defunkt/resque/contributing
+[rs]: http://github.com/resque/redis-namespace
+[cb]: http://wiki.github.com/resque/resque/contributing

data/lib/resque.rb

@@ -21,9 +21,47 @@ require 'resque/plugin'
 require 'resque/vendor/utf8_util'
 
 module Resque
-  include Helpers
   extend self
 
+  # Given a Ruby object, returns a string suitable for storage in a
+  # queue.
+  def encode(object)
+    if MultiJson.respond_to?(:dump) && MultiJson.respond_to?(:load)
+      MultiJson.dump object
+    else
+      MultiJson.encode object
+    end
+  end
+
+  # Given a string, returns a Ruby object.
+  def decode(object)
+    return unless object
+
+    begin
+      if MultiJson.respond_to?(:dump) && MultiJson.respond_to?(:load)
+        MultiJson.load object
+      else
+        MultiJson.decode object
+      end
+    rescue ::MultiJson::DecodeError => e
+      raise Helpers::DecodeException, e.message, e.backtrace
+    end
+  end
+
+  extend Forwardable
+
+  def self.config=(options = {})
+    @config = Config.new(options)
+  end
+
+  def self.config
+    @config ||= Config.new
+  end
+
+  def self.configure
+    yield config
+  end
+
   # Accepts:
   #   1. A 'hostname:port' String
   #   2. A 'hostname:port:db' String (to select the Redis db)
@@ -169,8 +207,10 @@ module Resque
   #
   # Returns nothing
   def push(queue, item)
-    watch_queue(queue)
-    redis.rpush "queue:#{queue}", encode(item)
+    redis.pipelined do
+      watch_queue(queue)
+      redis.rpush "queue:#{queue}", encode(item)
+    end
   end
 
   # Pops a job off a queue. Queue name should be a string.
@@ -217,8 +257,10 @@ module Resque
 
   # Given a queue name, completely deletes the queue.
   def remove_queue(queue)
-    redis.srem(:queues, queue.to_s)
-    redis.del("queue:#{queue}")
+    redis.pipelined do
+      redis.srem(:queues, queue.to_s)
+      redis.del("queue:#{queue}")
+    end
   end
 
   # Used internally to keep track of which queues we've created.

data/lib/resque/failure.rb

@@ -22,8 +22,8 @@ module Resque
     # `Resque::Failure::Base`.
     #
     # Example use:
-    #   require 'resque/failure/hoptoad'
-    #   Resque::Failure.backend = Resque::Failure::Hoptoad
+    #   require 'resque/failure/airbrake'
+    #   Resque::Failure.backend = Resque::Failure::Airbrake
     def self.backend=(backend)
       @backend = backend
     end
@@ -76,8 +76,8 @@ module Resque
     end
 
     # Iterate across all failures with the given options
-    def self.each(offset = 0, limit = self.count, queue = nil, class_name = nil, &block)
-      backend.each(offset, limit, queue, class_name, &block)
+    def self.each(offset = 0, limit = self.count, queue = nil, class_name = nil, order = 'desc', &block)
+      backend.each(offset, limit, queue, class_name, order, &block)
     end
 
     # The string url of the backend's web interface, if any.

data/lib/resque/failure/airbrake.rb

@@ -4,14 +4,30 @@ rescue LoadError
   raise "Can't find 'airbrake' gem. Please add it to your Gemfile or install it."
 end
 
-require 'resque/failure/thoughtbot'
-
 module Resque
   module Failure
     class Airbrake < Base
-      include Resque::Failure::Thoughtbot
+      def self.configure(&block)
+        Resque.logger.warn "This actually sets global Airbrake configuration, " \
+          "which is probably not what you want. This will be gone in 2.0."
+        Resque::Failure.backend = self
+        ::Airbrake.configure(&block)
+      end
+
+      def self.count(queue = nil, class_name = nil)
+        # We can't get the total # of errors from Airbrake so we fake it
+        # by asking Resque how many errors it has seen.
+        Stat[:failed]
+      end
 
-      @klass = ::Airbrake
+      def save
+        ::Airbrake.notify_or_ignore(exception,
+            :parameters => {
+            :payload_class => payload['class'].to_s,
+            :payload_args => payload['args'].inspect
+            }
+          )
+      end
     end
   end
 end

data/lib/resque/failure/base.rb

@@ -42,7 +42,7 @@ module Resque
       end
 
       # Returns a paginated array of failure objects.
-      def self.all(offset = 0, limit = 1)
+      def self.all(offset = 0, limit = 1, queue = nil)
         []
       end
 

data/lib/resque/failure/redis.rb

@@ -5,7 +5,7 @@ module Resque
     class Redis < Base
       def save
         data = {
-          :failed_at => Time.now.strftime("%Y/%m/%d %H:%M:%S %Z"),
+          :failed_at => UTF8Util.clean(Time.now.strftime("%Y/%m/%d %H:%M:%S %Z")),
           :payload   => payload,
           :exception => exception.class.to_s,
           :error     => UTF8Util.clean(exception.to_s),
@@ -38,10 +38,21 @@ module Resque
         Resque.list_range(:failed, offset, limit)
       end
 
-      def self.each(offset = 0, limit = self.count, queue = :failed, class_name = nil)
-        Array(all(offset, limit, queue)).each_with_index do |item, i|
+      def self.each(offset = 0, limit = self.count, queue = :failed, class_name = nil, order = 'desc')
+        all_items = Array(all(offset, limit, queue))
+        reversed = false
+        if order.eql? 'desc'
+          all_items.reverse!
+          reversed = true
+        end
+        all_items.each_with_index do |item, i|
           if !class_name || (item['payload'] && item['payload']['class'] == class_name)
-            yield offset + i, item
+            if reversed
+              id = (all_items.length - 1) - (offset + i)
+            else
+              id = offset + i
+            end
+            yield id, item
           end
         end
       end

data/lib/resque/failure/redis_multi_queue.rb

@@ -33,17 +33,20 @@ module Resque
         end
       end
 
-      def self.all(offset = 0, limit = 1, queue = :failed)
-        Resque.list_range(queue, offset, limit)
+      def self.all(offset = 0, limit = 1, queue = :failed, order = 'desc')
+        Resque.list_range(queue, offset, limit, order)
       end
 
       def self.queues
         Array(Resque.redis.smembers(:failed_queues))
       end
 
-      def self.each(offset = 0, limit = self.count, queue = :failed, class_name = nil)
-        items = all(offset, limit, queue)
+      def self.each(offset = 0, limit = self.count, queue = :failed, class_name = nil, order = 'desc')
+        items = all(offset, limit, queue, order)
         items = [items] unless items.is_a? Array
+        if order.eql? 'desc'
+          items.reverse!
+        end
         items.each_with_index do |item, i|
           if !class_name || (item['payload'] && item['payload']['class'] == class_name)
             yield offset + i, item

data/lib/resque/helpers.rb

@@ -11,6 +11,14 @@ end
 module Resque
   # Methods used by various classes in Resque.
   module Helpers
+    def self.extended(parent_class)
+      warn("Resque::Helpers will be gone with no replacement in Resque 2.0.0.")
+    end
+
+    def self.included(parent_class)
+      warn("Resque::Helpers will be gone with no replacement in Resque 2.0.0.")
+    end
+
     class DecodeException < StandardError; end
 
     # Direct access to the Redis instance.
@@ -26,7 +34,6 @@ module Resque
       else
         MultiJson.encode object
       end
-
     end
 
     # Given a string, returns a Ruby object.

data/lib/resque/job.rb

@@ -12,8 +12,110 @@ module Resque
   #   klass = Resque::Job.constantize(job.payload['class'])
   #   klass.perform(*job.payload['args'])
   class Job
-    include Helpers
-    extend Helpers
+    def redis
+      Resque.redis
+    end
+
+    def self.redis
+      Resque.redis
+    end
+
+    # Given a Ruby object, returns a string suitable for storage in a
+    # queue.
+    def encode(object)
+      if MultiJson.respond_to?(:dump) && MultiJson.respond_to?(:load)
+        MultiJson.dump object
+      else
+        MultiJson.encode object
+      end
+    end
+
+    # Given a string, returns a Ruby object.
+    def decode(object)
+      return unless object
+
+      begin
+        if MultiJson.respond_to?(:dump) && MultiJson.respond_to?(:load)
+          MultiJson.load object
+        else
+          MultiJson.decode object
+        end
+      rescue ::MultiJson::DecodeError => e
+        raise DecodeException, e.message, e.backtrace
+      end
+    end
+
+    # Given a Ruby object, returns a string suitable for storage in a
+    # queue.
+    def self.encode(object)
+      if MultiJson.respond_to?(:dump) && MultiJson.respond_to?(:load)
+        MultiJson.dump object
+      else
+        MultiJson.encode object
+      end
+    end
+
+    # Given a string, returns a Ruby object.
+    def self.decode(object)
+      return unless object
+
+      begin
+        if MultiJson.respond_to?(:dump) && MultiJson.respond_to?(:load)
+          MultiJson.load object
+        else
+          MultiJson.decode object
+        end
+      rescue ::MultiJson::DecodeError => e
+        raise DecodeException, e.message, e.backtrace
+      end
+    end
+    
+    # Given a word with dashes, returns a camel cased version of it.
+    #
+    # classify('job-name') # => 'JobName'
+    def classify(dashed_word)
+      dashed_word.split('-').each { |part| part[0] = part[0].chr.upcase }.join
+    end
+    
+    # Tries to find a constant with the name specified in the argument string:
+    #
+    # constantize("Module") # => Module
+    # constantize("Test::Unit") # => Test::Unit
+    #
+    # The name is assumed to be the one of a top-level constant, no matter
+    # whether it starts with "::" or not. No lexical context is taken into
+    # account:
+    #
+    # C = 'outside'
+    # module M
+    #   C = 'inside'
+    #   C # => 'inside'
+    #   constantize("C") # => 'outside', same as ::C
+    # end
+    #
+    # NameError is raised when the constant is unknown.
+    def constantize(camel_cased_word)
+      camel_cased_word = camel_cased_word.to_s
+
+      if camel_cased_word.include?('-')
+        camel_cased_word = classify(camel_cased_word)
+      end
+
+      names = camel_cased_word.split('::')
+      names.shift if names.empty? || names.first.empty?
+
+      constant = Object
+      names.each do |name|
+        args = Module.method(:const_get).arity != 1 ? [false] : []
+
+        if constant.const_defined?(name, *args)
+          constant = constant.const_get(name)
+        else
+          constant = constant.const_missing(name)
+        end
+      end
+      constant
+    end
 
     # Raise Resque::Job::DontPerform from a before_perform hook to
     # abort the job.
@@ -166,6 +268,19 @@ module Resque
       @payload_class ||= constantize(@payload['class'])
     end
 
+    # Returns the payload class as a string without raising NameError
+    def payload_class_name
+      payload_class.to_s
+    rescue NameError
+      'No Name'
+    end
+
+    def has_payload_class?
+      payload_class != Object
+    rescue NameError
+      false
+    end
+
     # Returns an array of args represented in this job's payload.
     def args
       @payload['args']
@@ -220,7 +335,9 @@ module Resque
     def run_failure_hooks(exception)
       begin
         job_args = args || []
-        failure_hooks.each { |hook| payload_class.send(hook, exception, *job_args) } unless @failure_hooks_ran
+        if has_payload_class?
+          failure_hooks.each { |hook| payload_class.send(hook, exception, *job_args) } unless @failure_hooks_ran
+        end
       ensure
         @failure_hooks_ran = true
       end