resque 1.17.1 → 1.18.0

data/HISTORY.md

@@ -1,3 +1,13 @@
+## 1.18.0 (2011-08-18)
+
+* Added before_enqueue hook.
+* Resque workers now preload files under app/ in Rails
+* Switch to MultiJSON
+* Bugfix: Finding worker pids on Solaris
+* Web UI: Fix NaN days ago for worker screens
+* Web UI: Add Cache-Control header to prevent proxy caching
+* Web UI: Update Resque.redis_id so it can be used in a distributed ring.
+
 ## 1.17.1 (2011-05-27)
 
 * Reverted `exit` change. Back to `exit!`.

data/README.markdown

@@ -424,7 +424,7 @@ The Front End
 Resque comes with a Sinatra-based front end for seeing what's up with
 your queue.
 
-![The Front End](http://img.skitch.com/20091104-tqh5pgkwgbskjbk7qbtmpesnyw.jpg)
+![The Front End](https://img.skitch.com/20110528-pc67a8qsfapgjxf5gagxd92fcu.png)
 
 ### Standalone
 
@@ -560,13 +560,8 @@ together. But, it's not that hard.
 Resque Dependencies
 -------------------
 
-    gem install redis redis-namespace yajl-ruby vegas sinatra
-
-If you cannot install `yajl-ruby` (JRuby?), you can install the `json`
-gem and Resque will use it instead.
-
-When problems arise, make sure you have the newest versions of the
-`redis` and `redis-namespace` gems.
+    $ gem install bundler
+    $ bundle install
 
 
 Installing Resque
@@ -606,7 +601,7 @@ Alternately you can define a `resque:setup` hook in your Rakefile if you
 don't want to load your app every time rake runs.
 
 
-### In a Rails app, as a gem
+### In a Rails 2.x app, as a gem
 
 First install the gem.
 
@@ -637,7 +632,7 @@ Don't forget you can define a `resque:setup` hook in
 `lib/tasks/whatever.rake` that loads the `environment` task every time.
 
 
-### In a Rails app, as a plugin
+### In a Rails 2.x app, as a plugin
 
     $ ./script/plugin install git://github.com/defunkt/resque
 
@@ -652,6 +647,40 @@ Don't forget you can define a `resque:setup` hook in
 `lib/tasks/whatever.rake` that loads the `environment` task every time.
 
 
+### In a Rails 3 app, as a gem
+
+First include it in your Gemfile.
+
+    $ cat Gemfile
+    ...
+    gem 'resque'
+    ...
+
+Next install it with Bundler.
+
+    $ bundle install
+
+Now start your application:
+
+    $ rails server
+
+That's it! You can now create Resque jobs from within your app.
+
+To start a worker, add this to a file in `lib/tasks` (ex:
+`lib/tasks/resque.rake`):
+
+``` ruby
+require 'resque/tasks'
+```
+
+Now:
+
+    $ QUEUE=* rake environment resque:work
+
+Don't forget you can define a `resque:setup` hook in
+`lib/tasks/whatever.rake` that loads the `environment` task every time.
+
+
 Configuration
 -------------
 

data/lib/resque.rb

@@ -1,11 +1,5 @@
 require 'redis/namespace'
 
-begin
-  require 'yajl'
-rescue LoadError
-  require 'json'
-end
-
 require 'resque/version'
 
 require 'resque/errors'
@@ -63,6 +57,8 @@ module Resque
     # support 1.x versions of redis-rb
     if redis.respond_to?(:server)
       redis.server
+    elsif redis.respond_to?(:nodes) # distributed
+      redis.nodes.map { |n| n.id }.join(', ')
     else
       redis.client.id
     end
@@ -228,6 +224,12 @@ module Resque
   #
   # This method is considered part of the `stable` API.
   def enqueue(klass, *args)
+    # Perform before_enqueue hooks. Don't perform enqueue if any hook returns false
+    before_hooks = Plugin.before_enqueue_hooks(klass).collect do |hook|
+      klass.send(hook, *args)
+    end
+    return if before_hooks.any? { |result| result == false }
+
     Job.create(queue_from_class(klass), klass, *args)
 
     Plugin.after_enqueue_hooks(klass).each do |hook|

data/lib/resque/failure/redis.rb

@@ -5,11 +5,11 @@ module Resque
     class Redis < Base
       def save
         data = {
-          :failed_at => Time.now.strftime("%Y/%m/%d %H:%M:%S"),
+          :failed_at => Time.now.strftime("%Y/%m/%d %H:%M:%S %Z"),
           :payload   => payload,
           :exception => exception.class.to_s,
           :error     => exception.to_s,
-          :backtrace => Array(exception.backtrace),
+          :backtrace => filter_backtrace(Array(exception.backtrace)),
           :worker    => worker.to_s,
           :queue     => queue
         }
@@ -41,6 +41,11 @@ module Resque
         Resque.redis.lset(:failed, index, id)
         Resque.redis.lrem(:failed, 1, id)
       end
+
+      def filter_backtrace(backtrace)
+        index = backtrace.index { |item| item.include?('/lib/resque/job.rb') }
+        backtrace.first(index.to_i)
+      end
     end
   end
 end

data/lib/resque/helpers.rb

@@ -1,3 +1,11 @@
+require 'multi_json'
+
+# OkJson won't work because it doesn't serialize symbols
+# in the same way yajl and json do.
+if MultiJson.engine.to_s == 'MultiJson::Engines::OkJson'
+  raise "Please install the yajl-ruby or json gem"
+end
+
 module Resque
   # Methods used by various classes in Resque.
   module Helpers
@@ -11,29 +19,17 @@ module Resque
     # Given a Ruby object, returns a string suitable for storage in a
     # queue.
     def encode(object)
-      if defined? Yajl
-        Yajl::Encoder.encode(object)
-      else
-        object.to_json
-      end
+      ::MultiJson.encode(object)
     end
 
     # Given a string, returns a Ruby object.
     def decode(object)
       return unless object
 
-      if defined? Yajl
-        begin
-          Yajl::Parser.parse(object, :check_utf8 => false)
-        rescue Yajl::ParseError => e
-          raise DecodeException, e
-        end
-      else
-        begin
-          JSON.parse(object)
-        rescue JSON::ParserError => e
-          raise DecodeException, e
-        end
+      begin
+        ::MultiJson.decode(object)
+      rescue ::MultiJson::DecodeError => e
+        raise DecodeException, e
       end
     end
 

data/lib/resque/plugin.rb

@@ -47,5 +47,10 @@ module Resque
     def after_enqueue_hooks(job)
       job.methods.grep(/^after_enqueue/).sort
     end
+
+    # Given an object, returns a list `before_enqueue` hook names.
+    def before_enqueue_hooks(job)
+      job.methods.grep(/^before_enqueue/).sort
+    end
   end
 end

data/lib/resque/server.rb

@@ -119,13 +119,14 @@ module Resque
     end
 
     def show(page, layout = true)
+      response["Cache-Control"] = "max-age=0, private, must-revalidate"
       begin
         erb page.to_sym, {:layout => layout}, :resque => Resque
       rescue Errno::ECONNREFUSED
         erb :error, {:layout => false}, :error => "Can't connect to Redis! (#{Resque.redis_id})"
       end
     end
-    
+
     def show_for_polling(page)
       content_type "text/html"
       @polling = true
@@ -136,12 +137,12 @@ module Resque
     get "/?" do
       redirect url_path(:overview)
     end
-    
+
     %w( overview workers ).each do |page|
       get "/#{page}.poll" do
         show_for_polling(page)
       end
-      
+
       get "/#{page}/:id.poll" do
         show_for_polling(page)
       end

data/lib/resque/tasks.rb

@@ -40,3 +40,10 @@ namespace :resque do
     threads.each { |thread| thread.join }
   end
 end
+
+# Preload app files
+task :environment do
+  Dir['app/**/*.rb'].each do |file|
+    require file
+  end
+end

data/lib/resque/version.rb

@@ -1,3 +1,3 @@
 module Resque
-  Version = VERSION = '1.17.1'
+  Version = VERSION = '1.18.0'
 end

data/lib/resque/worker.rb

@@ -36,7 +36,7 @@ module Resque
       names.map! { |name| "worker:#{name}" }
 
       reportedly_working = redis.mapped_mget(*names).reject do |key, value|
-        value.nil?
+        value.nil? || value.empty?
       end
       reportedly_working.keys.map do |key|
         find key.sub("worker:", '')
@@ -302,7 +302,7 @@ module Resque
     def paused?
       @paused
     end
-    
+
     # Stop processing jobs after the current one has completed (if we're
     # currently running one).
     def pause_processing
@@ -381,7 +381,7 @@ module Resque
       job.worker = self
       data = encode \
         :queue   => job.queue,
-        :run_at  => Time.now.to_s,
+        :run_at  => Time.now.strftime("%Y/%m/%d %H:%M:%S %Z"),
         :payload => job.payload
       redis.set("worker:#{self}", data)
     end
@@ -473,14 +473,40 @@ module Resque
       @pid ||= to_s.split(":")[1].to_i
     end
 
-    # Returns an array of string pids of all the other workers on this
+    # Returns an Array of string pids of all the other workers on this
     # machine. Useful when pruning dead workers on startup.
     def worker_pids
+      if RUBY_PLATFORM =~ /solaris/
+        solaris_worker_pids
+      else
+        linux_worker_pids
+      end
+    end
+
+    # Find Resque worker pids on Linux and OS X.
+    #
+    # Returns an Array of string pids of all the other workers on this
+    # machine. Useful when pruning dead workers on startup.
+    def linux_worker_pids
       `ps -A -o pid,command | grep [r]esque | grep -v "resque-web"`.split("\n").map do |line|
         line.split(' ')[0]
       end
     end
 
+    # Find Resque worker pids on Solaris.
+    #
+    # Returns an Array of string pids of all the other workers on this
+    # machine. Useful when pruning dead workers on startup.
+    def solaris_worker_pids
+      `ps -A -o pid,comm | grep ruby | grep -v grep | grep -v "resque-web"`.split("\n").map do |line|
+        real_pid = line.split(' ')[0]
+        pargs_command = `pargs -a #{real_pid} 2>/dev/null | grep [r]esque | grep -v "resque-web"`
+        if pargs_command.split(':')[1] == " resque-#{Resque::Version}"
+          real_pid
+        end
+      end.compact
+    end
+
     # Given a string, sets the procline ($0) and logs.
     # Procline is always in the format of:
     #   resque-VERSION: STRING

data/test/job_hooks_test.rb

@@ -250,6 +250,46 @@ context "Resque::Job after_enqueue" do
   end
 end
 
+
+context "Resque::Job before_enqueue" do
+  include PerformJob
+
+  class ::BeforeEnqueueJob
+    @queue = :jobs
+    def self.before_enqueue_record_history(history)
+      history << :before_enqueue
+    end
+
+    def self.perform(history)
+    end
+  end
+
+  class ::BeforeEnqueueJobAbort
+    @queue = :jobs
+    def self.before_enqueue_abort(history)
+      false
+    end
+
+    def self.perform(history)
+    end
+  end
+
+  test "the before enqueue hook should run" do
+    history = []
+    @worker = Resque::Worker.new(:jobs)
+    Resque.enqueue(BeforeEnqueueJob, history)
+    @worker.work(0)
+    assert_equal history, [:before_enqueue], "before_enqueue was not run"
+  end
+
+  test "a before enqueue hook that returns false should prevent the job from getting queued" do
+    history = []
+    @worker = Resque::Worker.new(:jobs)
+    Resque.enqueue(BeforeEnqueueJobAbort, history)
+    assert_equal 0, Resque.size(:jobs)
+  end
+end
+
 context "Resque::Job all hooks" do
   include PerformJob
 

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: resque
 version: !ruby/object:Gem::Version 
-  hash: 81
+  hash: 95
   prerelease: 
   segments: 
   - 1
-  - 17
-  - 1
-  version: 1.17.1
+  - 18
+  - 0
+  version: 1.18.0
 platform: ruby
 authors: 
 - Chris Wanstrath
@@ -15,7 +15,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-05-27 00:00:00 -07:00
+date: 2011-08-18 00:00:00 -07:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -67,26 +67,18 @@ dependencies:
   type: :runtime
   version_requirements: *id003
 - !ruby/object:Gem::Dependency 
-  name: json
+  name: multi_json
   prerelease: false
   requirement: &id004 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
-    - - ">="
-      - !ruby/object:Gem::Version 
-        hash: 11
-        segments: 
-        - 1
-        - 4
-        - 6
-        version: 1.4.6
-    - - <
+    - - ~>
       - !ruby/object:Gem::Version 
-        hash: 3
+        hash: 15
         segments: 
         - 1
-        - 6
-        version: "1.6"
+        - 0
+        version: "1.0"
   type: :runtime
   version_requirements: *id004
 description: "    Resque is a Redis-backed Ruby library for creating background jobs,\n    placing those jobs on multiple queues, and processing them later.\n\n    Background jobs can be any Ruby class or module that responds to\n    perform. Your existing classes can easily be converted to background\n    jobs or you can create new classes specifically to do work. Or, you\n    can do both.\n\n    Resque is heavily inspired by DelayedJob (which rocks) and is\n    comprised of three parts:\n\n    * A Ruby library for creating, querying, and processing jobs\n    * A Rake task for starting a worker which processes jobs\n    * A Sinatra app for monitoring queues, jobs, and workers.\n"