backburner 0.0.1 → 0.0.2

data/README.md

@@ -1,17 +1,69 @@
 # Backburner
 
-Backburner is a beanstalkd-powered job queue designed to be as simple and easy to use as possible.
-You can create background jobs, place those on specialized queues and process them later.
+Backburner is a [beanstalkd](http://kr.github.com/beanstalkd/)-powered job queue which can handle a very high volume of jobs.
+You create background jobs and place those on multiple work queues to be processed later.
 
-Processing background jobs reliably has never been easier. Backburner works with any ruby-based
-web framework but is well-suited for use with [Sinatra](http://sinatrarb.com) and [Padrino](http://padrinorb.com).
+Processing background jobs reliably has never been easier then with beanstalkd and Backburner. This gem works with any ruby-based
+web framework but is especially suited for use with [Sinatra](http://sinatrarb.com), [Padrino](http://padrinorb.com) and Rails.
 
-If you want to use beanstalk for job processing, consider using Backburner. Backburner is heavily inspired by Resque and DelayedJob.
-Backburner can be a persistent queue if the beanstalk persistence mode is enabled, supports priority, delays, and timeouts.
-Backburner stores jobs as simple JSON payloads.
+If you want to use beanstalk for your job processing, consider using Backburner.
+Backburner is heavily inspired by Resque and DelayedJob. Backburner stores all jobs as simple JSON message payloads.
+Backburner can be a persistent queue if the beanstalk persistence mode is enabled, supports multiple queues, priorities, delays, and timeouts.
+
+## Why Backburner?
+
+Backburner is well tested and has a familiar, no-nonsense approach to job processing but that is of secondary importance.
+Let's face it; there are a lot of options for background job processing. [DelayedJob](https://github.com/collectiveidea/delayed_job),
+and [Resque](https://github.com/defunkt/resque) are the first that come to mind immediately. So, how do we make sense
+of which one to use? And why use Backburner over other alternatives?
+
+The key to understanding the differences lies in understanding the different projects and protocols that power these popular queue
+libraries under the hood. Every job queue requires a queue store that jobs are put into and pulled out of.
+In the case of Resque, jobs are processed through **Redis**, a persistent key-value store. In the case of DelayedJob, jobs are processed through
+**ActiveRecord** and a database such as PostgreSQL.
+
+The work queue underlying these gems tells you infinitely more about the differences then anything else.
+Beanstalk is probably the best solution for job queues available today for many reasons.
+The real question then is... "Why Beanstalk?".
+
+## Why Beanstalk?
+
+Illya has an excellent blog post
+[Scalable Work Queues with Beanstalk](http://www.igvita.com/2010/05/20/scalable-work-queues-with-beanstalk/) and
+Adam Wiggins posted [an excellent comparison](http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend/).
+
+You will quickly see that **beanstalkd** is an underrated but incredible project that is extremely well-suited as a job queue.
+Significantly better suited for this task then Redis or a database. Beanstalk is a simple,
+and a very fast work queue service rolled into a single binary - it is the memcached of work queues.
+Originally built to power the backend for the 'Causes' Facebook app, it is a mature and production ready open source project.
+[PostRank](http://www.postrank.com) uses beanstalk to reliably process millions of jobs a day.
+
+A single instance of Beanstalk is perfectly capable of handling thousands of jobs a second (or more, depending on your job size)
+because it is an in-memory, event-driven system. Powered by libevent under the hood,
+it requires zero setup (launch and forget, ala memcached), optional log based persistence, an easily parsed ASCII protocol,
+and a rich set of tools for job management that go well beyond a simple FIFO work queue.
+
+Beanstalk supports the following features natively, out of the box, without any questions asked:
+
+ * **Parallel Queues** - Supports multiple work queues, which are created and deleted on demand.
+ * **Reliable** - Beanstalk’s reserve, work, delete cycle, with a timeout on a job, means bad clients basically can't lose a job.
+ * **Scheduling** - Delay enqueuing jobs by a specified interval to schedule processing later.
+ * **Fast** - Beanstalkd is **significantly** [faster then alternatives](http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend). Easily processes thousands of jobs a second.
+ * **Priorities** - Specify a higher priority and those jobs will jump ahead to be processed first accordingly.
+ * **Persistence** - Jobs are stored in memory for speed (ala memcached), but also logged to disk for safe keeping.
+ * **Federation** - Fault-tolerance and horizontal scalability is provided the same way as Memcache - through federation by the client.
+ * **Buried jobs** - When a job causes an error, you can bury it which keeps it around for later debugging and inspection.
+
+Keep in mind that these features are supported out of the box with beanstalk and require no special code within this gem to support.
+In the end, **beanstalk is the ideal job queue** while also being ridiculously easy to install and setup.
 
 ## Installation
 
+First, you probably want to [install beanstalkd](http://kr.github.com/beanstalkd/download.html), which powers the job queues.
+Depending on your platform, this should be as simple as (for Ubuntu):
+
+    $ sudo apt-get install beanstalkd
+
 Add this line to your application's Gemfile:
 
     gem 'backburner'
@@ -61,11 +113,11 @@ class NewsletterJob
 end
 ```
 
-Notice that you must include the `Backburner::Queue` module and that you can set a `queue` name within the job automatically.
-Jobs can then be enqueued using:
+Notice that you can include the optional `Backburner::Queue` module so you can specify a `queue` name for this job.
+Jobs can be enqueued with:
 
 ```ruby
-Backburner.enqueue NewsletterJob, 'lorem ipsum...', 5
+Backburner.enqueue NewsletterJob, 'foo@admin.com', 'lorem ipsum...'
 ```
 
 `Backburner.enqueue` accepts first a ruby object that supports `perform` and then a series of parameters
@@ -75,7 +127,7 @@ if not otherwise specified.
 ### Simple Async Jobs ###
 
 In addition to defining custom jobs, a job can also be enqueued by invoking the `async` method on any object which
-includes `Backburner::Performable`.
+includes `Backburner::Performable`. Async enqueuing works for both instance and class methods on any _performable_ object.
 
 ```ruby
 class User
@@ -85,13 +137,20 @@ class User
     @device = Device.find(device_id)
     # ...
   end
+
+  def self.reset_password(user_id)
+    # ...
+  end
 end
 
+# Async works for instance methods on a persisted model
 @user = User.first
 @user.async(:pri => 1000, :ttr => 100, :queue => "user.activate").activate(@device.id)
+# ..as well as for class methods
+User.async(:pri => 100, :delay => 10.seconds).reset_password(@user.id)
 ```
 
-This will automatically enqueue a job that will run `activate` with the specified argument for that user record.
+This will automatically enqueue a job for that user record that will run `activate` with the specified argument.
 The queue name used by default is the normalized class name (i.e `{namespace}.user`) if not otherwise specified.
 Note you are able to pass `pri`, `ttr`, `delay` and `queue` directly as options into `async`.
 
@@ -174,17 +233,30 @@ If a job fails in beanstalk, the job is automatically buried and must be 'kicked
 
 Right now, all logging happens to standard out and can be piped to a file or any other output manually. More on logging coming later.
 
-### Front-end Monitoring
+### Front-end
 
 To be completed is an admin dashboard that provides insight into beanstalk jobs via a simple Sinatra front-end. Coming soon.
 
-## Why Backburner?
+### Workers in Production
+
+Once you have Backburner setup in your application, starting workers is really easy. Once [beanstalkd](http://kr.github.com/beanstalkd/download.html)
+is installed, your best bet is to use the built-in rake task that comes with Backburner. Simply add the task to your Rakefile:
 
-To be filled in. DelayedJob, Resque, Stalker, et al.
+    # Rakefile
+    require 'backburner/tasks'
+
+and then you can start the rake task with:
+
+    $ rake backburner:work
+    $ QUEUES=newsletter-sender,push-message rake backburner:work
+
+The best way to deploy these rake tasks is using a monitoring library. We suggest [God](https://github.com/mojombo/god/)
+which watches processes and ensures their stability. A simple God recipe for Backburner can be found in
+[examples/god](https://github.com/nesquena/backburner/blob/master/examples/god.rb).
 
 ## Acknowledgements
 
- * Nathan Esquenazi - Project maintainer
+ * [Nathan Esquenazi](https://github.com/nesquena) - Project maintainer
  * Kristen Tucker - Coming up with the gem name
  * [Tim Lee](https://github.com/timothy1ee), [Josh Hull](https://github.com/joshbuddy), [Nico Taing](https://github.com/Nico-Taing) - Helping me work through the idea
  * [Miso](http://gomiso.com) - Open-source friendly place to work
@@ -199,7 +271,18 @@ To be filled in. DelayedJob, Resque, Stalker, et al.
 
 ## References
 
-The code in this project has been adapted from a few excellent projects:
+The code in this project has been made in light of a few excellent projects:
 
  * [DelayedJob](https://github.com/collectiveidea/delayed_job)
- * [Stalker](https://github.com/han/stalker)
+ * [Resque](https://github.com/defunkt/resque)
+ * [Stalker](https://github.com/han/stalker)
+
+Thanks to these projects for inspiration and certain design and implementation decisions.
+
+## Links
+
+ * Code: `git clone git://github.com/nesquena/backburner.git`
+ * Home: <http://github.com/nesquena/backburner>
+ * Docs: <http://rdoc.info/github/nesquena/backburner/master/frames>
+ * Bugs: <http://github.com/nesquena/backburner/issues>
+ * Gems: <http://gemcutter.org/gems/backburner>

data/Rakefile

@@ -11,5 +11,7 @@ task :test do
   end
 end
 
+task :default => :test
+
 # task :doc do
 #  YARD::CLI::Yardoc.new.run

data/examples/god.rb

@@ -0,0 +1,46 @@
+God.watch do |w|
+  w.name   = "backburner-worker-1"
+  w.dir    = '/path/to/app/dir'
+  w.env = { 'PADRINO_ENV' => 'production', 'QUEUES' => 'newsletter-sender,push-message' }
+  w.group    = 'backburner-workers'
+  w.interval = 30.seconds
+  w.start = "bundle exec rake -f Rakefile backburner:start"
+  w.log   = "/var/log/god/backburner-worker-1.log"
+
+  # restart if memory gets too high
+  w.transition(:up, :restart) do |on|
+    on.condition(:memory_usage) do |c|
+      c.above = 50.megabytes
+      c.times = 3
+    end
+  end
+
+  # determine the state on startup
+  w.transition(:init, { true => :up, false => :start }) do |on|
+    on.condition(:process_running) do |c|
+      c.running = true
+    end
+  end
+
+  # determine when process has finished starting
+  w.transition([:start, :restart], :up) do |on|
+    on.condition(:process_running) do |c|
+      c.running = true
+      c.interval = 5.seconds
+    end
+
+    # failsafe
+    on.condition(:tries) do |c|
+      c.times = 5
+      c.transition = :start
+      c.interval = 5.seconds
+    end
+  end
+
+  # start if process is not running
+  w.transition(:up, :start) do |on|
+    on.condition(:process_running) do |c|
+      c.running = false
+    end
+  end
+end

data/lib/backburner.rb

@@ -14,36 +14,51 @@ require 'backburner/queue'
 module Backburner
   class << self
 
-    # Enqueues a job to be performed with arguments
-    # Backburner.enqueue NewsletterSender, self.id, user.id
+    # Enqueues a job to be performed with given arguments.
+    #
+    # @example
+    #   Backburner.enqueue NewsletterSender, self.id, user.id
+    #
     def enqueue(job_class, *args)
       Backburner::Worker.enqueue(job_class, args, {})
     end
 
     # Begins working on jobs enqueued with optional tubes specified
-    # Backburner.work('newsletter_sender', 'test_job')
+    #
+    # @example
+    #   Backburner.work('newsletter_sender', 'test_job')
+    #
     def work(*tubes)
       Backburner::Worker.start(tubes)
     end
 
     # Yields a configuration block
-    # Backburner.configure do |config|
-    #  config.beanstalk_url = "beanstalk://..."
-    # end
+    #
+    # @example
+    #   Backburner.configure do |config|
+    #     config.beanstalk_url = "beanstalk://..."
+    #   end
+    #
     def configure(&block)
       yield(configuration)
       configuration
     end
 
     # Returns the configuration options set for Backburner
-    # Backburner.configuration.beanstalk_url => false
+    #
+    # @example
+    #   Backburner.configuration.beanstalk_url => false
+    #
     def configuration
       @_configuration ||= Configuration.new
     end
 
     # Returns the queues that are processed by default if none are specified
-    # default_queues << "foo"
-    # default_queues => ["foo", "bar"]
+    #
+    # @example
+    #   Backburner.default_queues << "foo"
+    #   Backburner.default_queues => ["foo", "bar"]
+    #
     def default_queues
       configuration.default_queues
     end

data/lib/backburner/async_proxy.rb

@@ -8,8 +8,11 @@ module Backburner
 
   # Class allows async task to be proxied
   class AsyncProxy < BasicObject
-    # AsyncProxy(User, 10, :pri => 1000, :ttr => 1000)
     # Options include `pri` (priority), `delay` (delay in secs), `ttr` (time to respond)
+    #
+    # @example
+    #   AsyncProxy(User, 10, :pri => 1000, :ttr => 1000)
+    #
     def initialize(klazz, id=nil, opts={})
       @klazz, @id, @opts = klazz, id, opts
     end

data/lib/backburner/connection.rb

@@ -26,14 +26,20 @@ module Backburner
     end
 
     # Returns the beanstalk queue addresses
-    # beanstalk_addresses => ["localhost:11300"]
+    #
+    # @example
+    #   beanstalk_addresses => ["localhost:11300"]
+    #
     def beanstalk_addresses
       uris = self.url.split(/[\s,]+/)
       uris.map {|uri| beanstalk_host_and_port(uri)}
     end
 
     # Returns a host and port based on the uri_string given
-    # beanstalk_host_and_port("beanstalk://localhost") => "localhost:11300"
+    #
+    # @example
+    #   beanstalk_host_and_port("beanstalk://localhost") => "localhost:11300"
+    #
     def beanstalk_host_and_port(uri_string)
       uri = URI.parse(uri_string)
       raise(BadURL, uri_string) if uri.scheme != 'beanstalk'

data/lib/backburner/helpers.rb

@@ -18,13 +18,19 @@ module Backburner
     end
 
     # Given a word with dashes, returns a camel cased version of it.
-    # classify('job-name') # => 'JobName'
+    #
+    # @example
+    #   classify('job-name') # => 'JobName'
+    #
     def classify(dashed_word)
       dashed_word.to_s.split('-').each { |part| part[0] = part[0].chr.upcase }.join
     end
 
     # Given a class, dasherizes the name, used for getting tube names
-    # dasherize('JobName') => "job-name"
+    #
+    # @example
+    #   dasherize('JobName') # => "job-name"
+    #
     def dasherize(word)
       classify(word).to_s.gsub(/::/, '/').
             gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
@@ -34,19 +40,9 @@ module Backburner
 
     # Tries to find a constant with the name specified in the argument string:
     #
-    # constantize("Module") # => Module
-    # constantize("Test::Unit") # => Test::Unit
-    #
-    # The name is assumed to be the one of a top-level constant, no matter
-    # whether it starts with "::" or not. No lexical context is taken into
-    # account:
-    #
-    # C = 'outside'
-    # module M
-    #   C = 'inside'
-    #   C # => 'inside'
-    #   constantize("C") # => 'outside', same as ::C
-    # end
+    # @example
+    #   constantize("Module") # => Module
+    #   constantize("Test::Unit") # => Test::Unit
     #
     # NameError is raised when the constant is unknown.
     def constantize(camel_cased_word)
@@ -73,14 +69,20 @@ module Backburner
     end
 
     # Returns tube_namespace for backburner
-    # tube_namespace => "some.namespace"
+    #
+    # @example
+    #   tube_namespace => "some.namespace"
+    #
     def tube_namespace
       Backburner.configuration.tube_namespace
     end
 
     # Expands a tube to include the prefix
-    # expand_tube_name("foo") => <prefix>.foo
-    # expand_tube_name(FooJob) => <prefix>.foo-job
+    #
+    # @example
+    #   expand_tube_name("foo") # => <prefix>.foo
+    #   expand_tube_name(FooJob) # => <prefix>.foo-job
+    #
     def expand_tube_name(tube)
       prefix = tube_namespace
       queue_name = if tube.is_a?(String)

data/lib/backburner/job.rb

@@ -0,0 +1,68 @@
+module Backburner
+  # A single backburner job which can be processed and removed by the worker
+  class Job
+    include Backburner::Helpers
+
+    # Raises when a job times out
+    class JobTimeout < RuntimeError; end
+    class JobNotFound < RuntimeError; end
+
+    attr_accessor :task, :body, :name, :args
+
+    # Construct a job to be parsed and processed
+    #
+    # task is a reserved object containing the json body in the form of
+    #   { :class => "NewsletterSender", :args => ["foo@bar.com"] }
+    #
+    # @example
+    #  Backburner::Job.new(payload)
+    #
+    def initialize(task)
+      @task = task
+      @body = JSON.parse(task.body)
+      @name, @args = body["class"], body["args"]
+    end
+
+    # Processes a job and handles any failure, deleting the job once complete
+    #
+    # @example
+    #   @task.process
+    #
+    def process
+      timeout_job_after(task.ttr - 1) { job_class.perform(*args) }
+      task.delete
+    end
+
+    # Bury a job out of the active queue if that job fails
+    def bury
+      task.bury
+    end
+
+    protected
+
+    # Returns the class for the job handler
+    #
+    # @example
+    #   job_class # => NewsletterSender
+    #
+    def job_class
+      handler = constantize(name) rescue nil
+      raise(JobNotFound, name) unless handler
+      handler
+    end
+
+    # Timeout job after given time
+    #
+    # @example
+    #   timeout_job_after(3) { do_something! }
+    #
+    def timeout_job_after(secs, &block)
+      begin
+        Timeout::timeout(secs) { yield }
+      rescue Timeout::Error
+        raise JobTimeout, "#{name} hit #{secs}s timeout"
+      end
+    end
+
+  end # Job
+end # Backburner

data/lib/backburner/logger.rb

@@ -20,7 +20,9 @@ module Backburner
     end
 
     # Prints message about failure when beastalk cannot be connected
-    # failed_connection(ex)
+    #
+    # @example
+    #   failed_connection(ex)
     def failed_connection(e)
       log_error exception_message(e)
       log_error "*** Failed connection to #{connection.url}"
@@ -29,13 +31,17 @@ module Backburner
     end
 
     # Print a message to stdout
-    # log("Working on task")
+    #
+    # @example
+    #   log("Working on task")
     def log(msg)
       puts msg
     end
 
     # Print an error to stderr
-    # log_error("Task failed!")
+    #
+    # @example
+    #   log_error("Task failed!")
     def log_error(msg)
       $stderr.puts msg
     end

data/lib/backburner/performable.rb

@@ -11,7 +11,9 @@ module Backburner
     module InstanceMethods
       # Return proxy object to enqueue jobs for object
       # Options: `pri` (priority), `delay` (delay in secs), `ttr` (time to respond), `queue` (queue name)
-      # @model.async(:pri => 1000).do_something("foo")
+      # @example
+      #   @model.async(:pri => 1000).do_something("foo")
+      #
       def async(opts={})
         Backburner::AsyncProxy.new(self.class, self.id, opts)
       end
@@ -20,13 +22,15 @@ module Backburner
     module ClassMethods
       # Return proxy object to enqueue jobs for object
       # Options: `pri` (priority), `delay` (delay in secs), `ttr` (time to respond), `queue` (queue name)
-      # Model.async(:ttr => 300).do_something("foo")
+      # @example
+      #   Model.async(:ttr => 300).do_something("foo")
       def async(opts={})
         Backburner::AsyncProxy.new(self, nil, opts)
       end
 
       # Defines perform method for job processing
-      # perform(55, :do_something, "foo", "bar")
+      # @example
+      #   perform(55, :do_something, "foo", "bar")
       def perform(id, method, *args)
         if id # instance
           find(id).send(method, *args)

data/lib/backburner/queue.rb

@@ -7,9 +7,11 @@ module Backburner
     end
 
     module ClassMethods
-      # Returns or assigns queue name for this job
-      # queue "some.task.name"
-      # queue => "some.task.name"
+      # Returns or assigns queue name for this job.
+      #
+      # @example
+      #   queue "some.task.name"
+      #   queue => "some.task.name"
       def queue(name=nil)
         if name
           @queue_name = name

data/lib/backburner/version.rb

@@ -1,3 +1,3 @@
 module Backburner
-  VERSION = "0.0.1"
+  VERSION = "0.0.2"
 end

data/lib/backburner/worker.rb

@@ -1,12 +1,10 @@
+require 'backburner/job'
+
 module Backburner
   class Worker
     include Backburner::Helpers
     include Backburner::Logger
 
-    class JobNotFound < RuntimeError; end
-    class JobTimeout < RuntimeError; end
-    class JobQueueNotSet < RuntimeError; end
-
     # Backburner::Worker.known_queue_classes
     # List of known_queue_classes
     class << self
@@ -16,7 +14,10 @@ module Backburner
 
     # Enqueues a job to be processed later by a worker
     # Options: `pri` (priority), `delay` (delay in secs), `ttr` (time to respond), `queue` (queue name)
-    # Backburner::Worker.enqueue NewsletterSender, [self.id, user.id], :ttr => 1000
+    #
+    # @example
+    #   Backburner::Worker.enqueue NewsletterSender, [self.id, user.id], :ttr => 1000
+    #
     def self.enqueue(job_class, args=[], opts={})
       pri   = opts[:pri] || Backburner.configuration.default_priority
       delay = [0, opts[:delay].to_i].max
@@ -29,13 +30,15 @@ module Backburner
     end
 
     # Starts processing jobs in the specified tube_names
-    # Backburner::Worker.start(["foo.tube.name"])
+    # @example
+    #   Backburner::Worker.start(["foo.tube.name"])
     def self.start(tube_names=nil)
       self.new(tube_names).start
     end
 
     # Returns the worker connection
-    # Backburner::Worker.connection => <Beanstalk::Pool>
+    # @example
+    #   Backburner::Worker.connection # => <Beanstalk::Pool>
     def self.connection
       @connection ||= Connection.new(Backburner.configuration.beanstalk_url)
     end
@@ -43,7 +46,8 @@ module Backburner
     # List of tube names to be watched and processed
     attr_accessor :tube_names
 
-    # Worker.new(['test.job'])
+    # @example
+    #   Worker.new(['test.job'])
     def initialize(tube_names=nil)
       @tube_names = begin
         tube_names = tube_names.first if tube_names && tube_names.size == 1 && tube_names.first.is_a?(Array)
@@ -55,7 +59,8 @@ module Backburner
 
     # Starts processing new jobs indefinitely
     # Primary way to consume and process jobs in specified tubes
-    # @worker.start
+    # @example
+    #   @worker.start
     def start
       prepare
       loop { work_one_job }
@@ -63,7 +68,8 @@ module Backburner
 
     # Setup beanstalk tube_names and watch all specified tubes for jobs.
     # Used to prepare job queues before processing jobs.
-    # @worker.prepare
+    # @example
+    #   @worker.prepare
     def prepare
       self.tube_names ||= Backburner.default_queues.any? ? Backburner.default_queues : all_existing_queues
       self.tube_names = Array(self.tube_names)
@@ -80,25 +86,13 @@ module Backburner
     # Reserves one job within the specified queues
     # Pops the job off and serializes the job to JSON
     # Each job is performed by invoking `perform` on the job class.
-    # @worker.work_one_job
+    # @example
+    #   @worker.work_one_job
     def work_one_job
-      job = self.connection.reserve
-      body = JSON.parse job.body
-      name, args = body["class"], body["args"]
-      self.class.log_job_begin(body)
-      handler = constantize(name)
-      raise(JobNotFound, name) unless handler
-
-      begin
-        Timeout::timeout(job.ttr - 1) do
-          handler.perform(*args)
-        end
-      rescue Timeout::Error
-        raise JobTimeout, "#{name} hit #{job.ttr-1}s timeout"
-      end
-
-      job.delete
-      self.class.log_job_end(name)
+      job = Backburner::Job.new(self.connection.reserve)
+      self.class.log_job_begin(job.body)
+      job.process
+      self.class.log_job_end(job.name)
     rescue Beanstalk::NotConnected => e
       failed_connection(e)
     rescue SystemExit
@@ -106,8 +100,8 @@ module Backburner
     rescue => e
       job.bury
       self.class.log_error self.class.exception_message(e)
-      self.class.log_job_end(name, 'failed') if @job_begun
-      handle_error(e, name, args)
+      self.class.log_job_end(job.name, 'failed') if @job_begun
+      handle_error(e, job.name, job.args)
     end
 
     protected

data/test/job_test.rb

@@ -0,0 +1,85 @@
+require File.expand_path('../test_helper', __FILE__)
+
+module NestedDemo
+  class TestJobC
+    include Backburner::Queue
+    def self.perform(x); puts "Performed #{x} in #{self}"; end
+  end
+
+  class TestJobD
+    include Backburner::Queue
+    def self.perform(x); raise RuntimeError; end
+  end
+end
+
+describe "Backburner::Job module" do
+  describe "for initialize method" do
+    before do
+      @task_body =  { :class => "NewsletterSender", :args => ["foo@bar.com", "bar@foo.com"] }
+      @task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
+    end
+
+    it "should create job with correct task data" do
+      @job = Backburner::Job.new(@task)
+      assert_equal @task, @job.task
+      assert_equal ["class", "args"], @job.body.keys
+      assert_equal @task_body[:class], @job.name
+      assert_equal @task_body[:args], @job.args
+    end
+  end # initialize
+
+  describe "for process method" do
+    describe "with valid task" do
+      before do
+        @task_body =  { :class => "NestedDemo::TestJobC", :args => [56] }
+        @task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
+        @task.expects(:delete).once
+      end
+
+      it "should process task" do
+        @job = Backburner::Job.new(@task)
+        out = silenced(1) { @job.process }
+        assert_match /Performed 56 in NestedDemo::TestJobC/, out
+      end # process
+    end # valid
+
+    describe "with invalid task" do
+      before do
+        @task_body =  { :class => "NestedDemo::TestJobD", :args => [56] }
+        @task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
+        @task.expects(:delete).never
+      end
+
+      it "should raise an exception" do
+        @job = Backburner::Job.new(@task)
+        assert_raises(RuntimeError) { @job.process }
+      end # error invalid
+    end # invalid
+
+    describe "with invalid class" do
+      before do
+        @task_body =  { :class => "NestedDemo::TestJobY", :args => [56] }
+        @task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
+        @task.expects(:delete).never
+      end
+
+      it "should raise an exception" do
+        @job = Backburner::Job.new(@task)
+        assert_raises(Backburner::Job::JobNotFound) { @job.process }
+      end # error class
+    end # invalid
+  end # process
+
+  describe "for bury method" do
+    before do
+      @task_body =  { :class => "NestedDemo::TestJobC", :args => [56] }
+      @task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
+      @task.expects(:bury).once
+    end
+
+    it "should call bury for task" do
+      @job = Backburner::Job.new(@task)
+      @job.bury
+    end # bury
+  end # bury
+end

data/test/test_helper.rb

@@ -18,7 +18,10 @@ module Kernel
   # Redirect standard out, standard error and the buffered logger for sprinkle to StringIO
   # capture_stdout { any_commands; you_want } => "all output from the commands"
   def capture_stdout
-    return yield if ENV['DEBUG'] # Skip if debug mode
+    if ENV['DEBUG'] # Skip if debug mode
+      yield
+      ""
+    end
 
     out = StringIO.new
     $stdout = out

data/test/worker_test.rb

@@ -7,6 +7,11 @@ class TestJob
   def self.perform(x, y); $worker_test_count += x + y; end
 end
 
+class TestFailJob
+  include Backburner::Queue
+  def self.perform(x, y); raise RuntimeError; end
+end
+
 class TestAsyncJob
   include Backburner::Performable
   def self.foo(x, y); $worker_test_count = x * y; end
@@ -147,7 +152,20 @@ describe "Backburner::Worker module" do
         worker.work_one_job
       end
       assert_equal 3, $worker_test_count
-    end
+    end # enqueue
+
+    it "should work an enqueued failing job" do
+      $worker_test_count = 0
+      Backburner::Worker.enqueue TestFailJob, [1, 2], :queue => "foo.bar.fail"
+      Backburner::Job.any_instance.expects(:bury).once
+      out = silenced(2) do
+        worker = Backburner::Worker.new('foo.bar.fail')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_match(/Exception RuntimeError/, out)
+      assert_equal 0, $worker_test_count
+    end # fail
 
     it "should work for an async job" do
       $worker_test_count = 0
@@ -158,6 +176,6 @@ describe "Backburner::Worker module" do
         worker.work_one_job
       end
       assert_equal 15, $worker_test_count
-    end
+    end # async
   end # work_one_job
 end # Backburner::Worker

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: backburner
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.0.2
   prerelease: 
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-07-26 00:00:00.000000000 Z
+date: 2012-07-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: beanstalk-client
-  requirement: !ruby/object:Gem::Requirement
+  requirement: &2152606340 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,15 +21,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
+  version_requirements: *2152606340
 - !ruby/object:Gem::Dependency
   name: json_pure
-  requirement: !ruby/object:Gem::Requirement
+  requirement: &2152605640 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -37,15 +32,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
+  version_requirements: *2152605640
 - !ruby/object:Gem::Dependency
   name: dante
-  requirement: !ruby/object:Gem::Requirement
+  requirement: &2152605220 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -53,15 +43,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
+  version_requirements: *2152605220
 - !ruby/object:Gem::Dependency
   name: rake
-  requirement: !ruby/object:Gem::Requirement
+  requirement: &2152604580 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -69,15 +54,10 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
+  version_requirements: *2152604580
 - !ruby/object:Gem::Dependency
   name: minitest
-  requirement: !ruby/object:Gem::Requirement
+  requirement: &2152604000 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -85,15 +65,10 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
+  version_requirements: *2152604000
 - !ruby/object:Gem::Dependency
   name: mocha
-  requirement: !ruby/object:Gem::Requirement
+  requirement: &2152603460 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -101,12 +76,7 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
+  version_requirements: *2152603460
 description: Beanstalk background job processing made easy
 email:
 - nesquena@gmail.com
@@ -126,12 +96,14 @@ files:
 - bin/backburner
 - examples/custom.rb
 - examples/demo.rb
+- examples/god.rb
 - examples/simple.rb
 - lib/backburner.rb
 - lib/backburner/async_proxy.rb
 - lib/backburner/configuration.rb
 - lib/backburner/connection.rb
 - lib/backburner/helpers.rb
+- lib/backburner/job.rb
 - lib/backburner/logger.rb
 - lib/backburner/performable.rb
 - lib/backburner/queue.rb
@@ -141,6 +113,7 @@ files:
 - test/back_burner_test.rb
 - test/connection_test.rb
 - test/helpers_test.rb
+- test/job_test.rb
 - test/logger_test.rb
 - test/performable_test.rb
 - test/queue_test.rb
@@ -166,7 +139,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.21
+rubygems_version: 1.8.15
 signing_key: 
 specification_version: 3
 summary: Reliable beanstalk background job processing made easy for Ruby and Sinatra
@@ -174,6 +147,7 @@ test_files:
 - test/back_burner_test.rb
 - test/connection_test.rb
 - test/helpers_test.rb
+- test/job_test.rb
 - test/logger_test.rb
 - test/performable_test.rb
 - test/queue_test.rb