backburner 0.1.1 → 0.1.2

data/CHANGELOG.md

@@ -1,10 +1,17 @@
 # CHANGELOG
 
-## Released 0.1.1 (Nov 6th 2012)
+## Version 0.1.3 (Unreleased)
+
+## Version 0.1.2 (Nov 7th 2012)
+
+ * Adds ability to specify a custom logger.
+ * Adds job retry configuration and worker support.
+
+## Version 0.1.1 (Nov 6th 2012)
 
  * Fix issue with timed out reserves
 
-## Released 0.1.0 (Nov 4th 2012)
+## Version 0.1.0 (Nov 4th 2012)
 
  * Switch to beaneater as new ruby beanstalkd client
  * Add support for array of connections in `beanstalk_url`

data/README.md

@@ -86,14 +86,20 @@ Backburner.configure do |config|
   config.beanstalk_url = ["beanstalk://127.0.0.1", "beanstalk://127.0.0.1:11301"]
   config.tube_namespace = "some.app.production"
   config.on_error = lambda { |e| puts e }
+  config.max_job_retries  = 3 # default 0 retries
+  config.retry_delay      = 2 # default 5 seconds
   config.default_priority = 65536
-  config.respond_timeout = 120
+  config.respond_timeout  = 120
+  config.logger = Logger.new(STDOUT)
 end
 ```
 
-The `beanstalk_url` supports a string such as 'beanstalk://127.0.0.1' or an array of addresses.
-The `tube_namespace` is the prefix used for all tubes related to this backburner queue.
-The `on_error` is a callback that gets invoked with the error whenever a job fails
+ * The `beanstalk_url` supports a string such as 'beanstalk://127.0.0.1' or an array of addresses.
+ * The `tube_namespace` is the prefix used for all tubes related to this backburner queue.
+ * The `on_error` is a callback that gets invoked with the error whenever a job fails.
+ * The `max_job_retries` determines how many times to retry a job before burying
+ * The `retry_delay` determines the base time to wait (in secs) between retries
+ * The `logger` is the logger object written to when backburner wants to report info or errors.
 
 ## Usage
 
@@ -227,7 +233,20 @@ The `default_queues` stores the specific list of queues that should be processed
 
 ### Failures
 
-You can setup the error handler for jobs using configure:
+When a job fails in backburner (usually because an exception was raised), the job will be released 
+and retried again (with progressive delays in between) until the `max_job_retries` configuration is reached.
+
+```ruby
+Backburner.configure do |config|
+  config.max_job_retries  = 3 # retry jobs 3 times
+  config.retry_delay      = 2 # wait 2 seconds in between retries
+end
+```
+
+Note the default `max_job_retries` is 0, meaning that by default **jobs are not retried**.
+If continued retry attempts fail, the job will be buried and can be 'kicked' later for inspection.
+
+You can also setup a custom error handler for jobs using configure:
 
 ```ruby
 Backburner.configure do |config|
@@ -235,12 +254,23 @@ Backburner.configure do |config|
 end
 ```
 
-Now all beanstalk queue errors will show up on airbrake.
-If a job fails in beanstalk, the job is automatically buried and must be 'kicked' later.
+Now all backburner queue errors will appear on airbrake for deeper inspection.
 
 ### Logging
 
-Right now, all logging happens to standard out and can be piped to a file or any other output manually. More on logging coming later.
+Logging in backburner is rather simple. When a job is run, the log records that. When a job
+fails, the log records that. When any exceptions occur during processing, the log records that.
+
+By default, the log will print to standard out. You can customize the log to output to any
+standard logger by controlling the configuration option:
+
+```ruby
+Backburner.configure do |config|
+  config.logger = Logger.new(STDOUT)
+end
+```
+
+Be sure to check logs whenever things do not seem to be processing.
 
 ### Web Front-end
 

data/TODO

@@ -1,9 +1,6 @@
-- Front-end in sinatra for viewing beanstalk jobs
-  - https://github.com/denniskuczynski/beanstalkd_view
-- Better logging
-  - https://github.com/dpree/stalker/commit/70e94b7b445e08166bf2288dd1b7853396b26a86
-- Better error handling (failures, retries)
-  - https://github.com/michaeldwan/stalker/commit/afd7e94d20991bcd96b65eda7c979d92d8746c3b
-  - https://github.com/michaeldwan/stalker/blob/master/lib/stalker/worker.rb
+- Custom front-end in sinatra for viewing beanstalk jobs
+  - Refer to https://github.com/denniskuczynski/beanstalkd_view
 - Fork jobs to control memory
+  - https://github.com/michaeldwan/stalker/commit/386267690a7c03e11d1a8b7b6f08b7c9c7cd2c0d
+- Add hooks
   - https://github.com/michaeldwan/stalker/commit/386267690a7c03e11d1a8b7b6f08b7c9c7cd2c0d

data/backburner.gemspec

@@ -15,7 +15,7 @@ Gem::Specification.new do |s|
   s.require_paths = ["lib"]
   s.version       = Backburner::VERSION
 
-  s.add_runtime_dependency 'beaneater', '~> 0.1.1'
+  s.add_runtime_dependency 'beaneater', '~> 0.1.2'
   s.add_runtime_dependency 'dante', '~> 0.1.5'
 
   s.add_development_dependency 'rake'

data/examples/retried.rb

@@ -0,0 +1,31 @@
+$:.unshift "lib"
+require 'backburner'
+
+$error = 0
+
+class User
+  include Backburner::Performable
+  attr_accessor :id, :name
+
+  def self.foo(x, y)
+    $error += 1
+    raise "fail #{$error}" unless $error > 3
+    puts "User #foo args [#{x}, #{y}] Success!!"
+  end
+end
+
+# Configure Backburner
+Backburner.configure do |config|
+  config.beanstalk_url = "beanstalk://127.0.0.1"
+  config.tube_namespace = "demo.production"
+  config.on_error = lambda { |e| puts "HEY!!! #{e.class}" }
+  config.max_job_retries = 3
+  config.retry_delay     = 0
+end
+
+# Enqueue tasks
+User.async(:queue => "retried").foo("bar", "baz")
+
+# Run work
+# Backburner.default_queues << "user"
+Backburner.work

data/lib/backburner/configuration.rb

@@ -5,7 +5,10 @@ module Backburner
     attr_accessor :default_priority   # default job priority
     attr_accessor :respond_timeout    # default job timeout
     attr_accessor :on_error           # error handler
+    attr_accessor :max_job_retries    # max job retries
+    attr_accessor :retry_delay        # retry delay in seconds
     attr_accessor :default_queues     # default queues
+    attr_accessor :logger             # logger
 
     def initialize
       @beanstalk_url     = "beanstalk://localhost"
@@ -13,7 +16,10 @@ module Backburner
       @default_priority  = 65536
       @respond_timeout   = 120
       @on_error          = nil
+      @max_job_retries   = 0
+      @retry_delay       = 5
       @default_queues    = []
+      @logger            = nil
     end
   end # Configuration
 end # Backburner

data/lib/backburner/connection.rb

@@ -13,7 +13,7 @@ module Backburner
       connect!
     end
 
-    # Sets the delegator object to the underlying beanstalk connection
+    # Sets the delegator object to the underlying beaneater pool
     # self.put(...)
     def __getobj__
       __setobj__(@beanstalk)

data/lib/backburner/helpers.rb

@@ -68,13 +68,13 @@ module Backburner
       constant
     end
 
-    # Returns tube_namespace for backburner
+    # Returns configuration options for backburner
     #
     # @example
-    #   tube_namespace => "some.namespace"
+    #   config.max_job_retries => 3
     #
-    def tube_namespace
-      Backburner.configuration.tube_namespace
+    def config
+      Backburner.configuration
     end
 
     # Expands a tube to include the prefix
@@ -84,7 +84,7 @@ module Backburner
     #   expand_tube_name(FooJob) # => <prefix>.foo-job
     #
     def expand_tube_name(tube)
-      prefix = tube_namespace
+      prefix = config.tube_namespace
       queue_name = if tube.is_a?(String)
         tube
       elsif tube.respond_to?(:queue) # use queue name
@@ -94,7 +94,7 @@ module Backburner
       else # turn into a string
         tube.to_s
       end
-      [prefix.gsub(/\.$/, ''), dasherize(queue_name).gsub(/^#{prefix}/, '')].join(".")
+      [prefix.gsub(/\.$/, ''), dasherize(queue_name).gsub(/^#{prefix}/, '')].join(".").gsub(/\.+/, '.')
     end
 
   end

data/lib/backburner/job.rb

@@ -1,11 +1,12 @@
 module Backburner
   # A single backburner job which can be processed and removed by the worker
-  class Job
+  class Job < SimpleDelegator
     include Backburner::Helpers
 
     # Raises when a job times out
     class JobTimeout < RuntimeError; end
     class JobNotFound < RuntimeError; end
+    class JobFormatInvalid < RuntimeError; end
 
     attr_accessor :task, :body, :name, :args
 
@@ -21,6 +22,16 @@ module Backburner
       @task = task
       @body = task.body.is_a?(Hash) ? task.body : JSON.parse(task.body)
       @name, @args = body["class"], body["args"]
+    rescue => ex # Job was not valid format
+      self.bury
+      raise JobFormatInvalid, "Job body could not be parsed: #{ex.inspect}"
+    end
+
+    # Sets the delegator object to the underlying beaneater job
+    # self.bury
+    def __getobj__
+      __setobj__(@task)
+      super
     end
 
     # Processes a job and handles any failure, deleting the job once complete
@@ -33,11 +44,6 @@ module Backburner
       task.delete
     end
 
-    # Bury a job out of the active queue if that job fails
-    def bury
-      task.bury
-    end
-
     protected
 
     # Returns the class for the job handler

data/lib/backburner/logger.rb

@@ -1,3 +1,5 @@
+require 'logger'
+
 module Backburner
   module Logger
     # Loads in instance and class levels
@@ -6,16 +8,21 @@ module Backburner
     end
 
     # Print out when a job is about to begin
-    def log_job_begin(body)
-      log_info [ "Working", body ].join(' ')
-      @job_begun = Time.now
+    def log_job_begin(name, args)
+      log_info "Work job #{name} with #{args.inspect}"
+      @job_started_at = Time.now
     end
 
     # Print out when a job completed
-    def log_job_end(name, failed=false)
-      ellapsed = Time.now - @job_begun
+    def log_job_end(name, message = nil)
+      ellapsed = Time.now - job_started_at
       ms = (ellapsed.to_f * 1000).to_i
-      log_info "Finished #{name} in #{ms}ms #{failed ? ' (failed)' : ''}"
+      log_info("Finished #{name} in #{ms}ms #{message}")
+    end
+
+    # Returns true if the job logging started
+    def job_started_at
+      @job_started_at
     end
 
     # Print a message to stdout
@@ -24,7 +31,7 @@ module Backburner
     #   log_info("Working on task")
     #
     def log_info(msg)
-      puts msg
+      logger ? logger.info(msg) : puts(msg)
     end
 
     # Print an error to stderr
@@ -33,8 +40,12 @@ module Backburner
     #   log_error("Task failed!")
     #
     def log_error(msg)
-      $stderr.puts msg
+      logger ? logger.error(msg) : $stderr.puts(msg)
     end
 
+    # Return logger if specified
+    def logger
+      Backburner.configuration.logger
+    end
   end
 end

data/lib/backburner/version.rb

@@ -1,3 +1,3 @@
 module Backburner
-  VERSION = "0.1.1"
+  VERSION = "0.1.2"
 end

data/lib/backburner/worker.rb

@@ -93,16 +93,24 @@ module Backburner
     #
     def work_one_job
       job = Backburner::Job.new(self.connection.tubes.reserve)
-      self.class.log_job_begin(job.body)
+      self.log_job_begin(job.name, job.args)
       job.process
-      self.class.log_job_end(job.name)
-    rescue => e
-      self.class.log_error self.class.exception_message(e)
-      if job # bury failed job
+      self.log_job_end(job.name)
+    rescue Backburner::Job::JobFormatInvalid => e
+      self.log_error self.exception_message(e)
+    rescue => e # Error occurred processing job
+      self.log_error self.exception_message(e)
+      num_retries = job.stats.releases
+      retry_status = "failed: attempt #{num_retries+1} of #{config.max_job_retries+1}"
+      if num_retries < config.max_job_retries # retry again
+        delay = config.retry_delay + num_retries ** 3
+        job.release(:delay => delay)
+        self.log_job_end(job.name, "#{retry_status}, retrying in #{delay}s") if job_started_at
+      else # retries failed, bury
         job.bury
-        self.class.log_job_end(job.name, 'failed') if @job_begun
-        handle_error(e, job.name, job.args)
+        self.log_job_end(job.name, "#{retry_status}, burying") if job_started_at
       end
+      handle_error(e, job.name, job.args)
     end
 
     protected
@@ -111,7 +119,7 @@ module Backburner
     # Filtered for tubes that match the known prefix
     def all_existing_queues
       known_queues    = Backburner::Worker.known_queue_classes.map(&:queue)
-      existing_tubes  = self.connection.tubes.all.map(&:name).select { |tube| tube =~ /^#{tube_namespace}/ }
+      existing_tubes  = self.connection.tubes.all.map(&:name).select { |tube| tube =~ /^#{config.tube_namespace}/ }
       known_queues + existing_tubes
     end
 

data/test/helpers_test.rb

@@ -44,13 +44,13 @@ describe "Backburner::Helpers module" do
     end
   end # exception_message
 
-  describe "for tube_namespace" do
+  describe "for config" do
     before { Backburner.expects(:configuration).returns(stub(:tube_namespace => "test.foo.job")) }
 
-    it "accesses correct value" do
-      assert_equal "test.foo.job", tube_namespace
+    it "accesses correct value for namespace" do
+      assert_equal "test.foo.job", config.tube_namespace
     end
-  end # tube_namespace
+  end # config
 
   describe "for expand_tube_name method" do
     before { Backburner.expects(:configuration).returns(stub(:tube_namespace => "test.foo.job.")) }

data/test/job_test.rb

@@ -13,35 +13,50 @@ module NestedDemo
 end
 
 describe "Backburner::Job module" do
-  describe "for initialize method with hash" do
-    before do
-      @task_body =  { "class" => "NewsletterSender", "args" => ["foo@bar.com", "bar@foo.com"] }
-      @task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
-    end
+  describe "for initialize" do
+    describe "with hash" do
+      before do
+        @task_body =  { "class" => "NewsletterSender", "args" => ["foo@bar.com", "bar@foo.com"] }
+        @task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
+      end
 
-    it "should create job with correct task data" do
-      @job = Backburner::Job.new(@task)
-      assert_equal @task, @job.task
-      assert_equal ["class", "args"], @job.body.keys
-      assert_equal @task_body["class"], @job.name
-      assert_equal @task_body["args"], @job.args
-    end
-  end # initialize with json
+      it "should create job with correct task data" do
+        @job = Backburner::Job.new(@task)
+        assert_equal @task, @job.task
+        assert_equal ["class", "args"], @job.body.keys
+        assert_equal @task_body["class"], @job.name
+        assert_equal @task_body["args"], @job.args
+      end
+    end # with hash
 
-  describe "for initialize method with json string" do
-    before do
-      @task_body =  { "class" => "NewsletterSender", "args" => ["foo@bar.com", "bar@foo.com"] }
-      @task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
-    end
+    describe "with json string" do
+      before do
+        @task_body =  { "class" => "NewsletterSender", "args" => ["foo@bar.com", "bar@foo.com"] }
+        @task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
+      end
 
-    it "should create job with correct task data" do
-      @job = Backburner::Job.new(@task)
-      assert_equal @task, @job.task
-      assert_equal ["class", "args"], @job.body.keys
-      assert_equal @task_body["class"], @job.name
-      assert_equal @task_body["args"], @job.args
-    end
-  end # initialize with json
+      it "should create job with correct task data" do
+        @job = Backburner::Job.new(@task)
+        assert_equal @task, @job.task
+        assert_equal ["class", "args"], @job.body.keys
+        assert_equal @task_body["class"], @job.name
+        assert_equal @task_body["args"], @job.args
+      end
+    end # with json
+
+    describe "with invalid string" do
+      before do
+        @task_body =  "^%$*&^*"
+        @task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
+      end
+
+      it "should raise a job format exception" do
+        assert_raises(Backburner::Job::JobFormatInvalid) {
+          @job = Backburner::Job.new(@task)
+        }
+      end
+    end # invalid
+  end # initialize
 
   describe "for process method" do
     describe "with valid task" do
@@ -85,7 +100,7 @@ describe "Backburner::Job module" do
     end # invalid
   end # process
 
-  describe "for bury method" do
+  describe "for simple delegation method" do
     before do
       @task_body =  { "class" => "NestedDemo::TestJobC", "args" => [56] }
       @task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
@@ -96,5 +111,5 @@ describe "Backburner::Job module" do
       @job = Backburner::Job.new(@task)
       @job.bury
     end # bury
-  end # bury
+  end # simple delegation
 end

data/test/logger_test.rb

@@ -3,11 +3,26 @@ require File.expand_path('../test_helper', __FILE__)
 describe "Backburner::Logger module" do
   include Backburner::Logger
 
+  before do
+    @strio = StringIO.new
+    @logger = Logger.new(@strio)
+  end
+
   describe "for log_info method" do
     it "prints out to std out" do
       output = capture_stdout { log_info("foo") }
       assert_equal "foo\n", output
     end
+
+    it "can be configured to log to logger" do
+      Backburner.configure { |config| config.logger = @logger }
+      log_info("foo")
+      assert_match /I,.*?foo/, @strio.string
+    end
+
+    after do
+      Backburner.configure { |config| config.logger = nil }
+    end
   end # log_info
 
   describe "for log_error method" do
@@ -15,5 +30,15 @@ describe "Backburner::Logger module" do
       output = capture_stdout { log_error("bar") }
       assert_equal "bar\n", output
     end
+
+    it "can be configured to log to logger" do
+      Backburner.configure { |config| config.logger = @logger }
+      log_error("bar")
+      assert_match /E,.*?bar/, @strio.string
+    end
+
+    after do
+      Backburner.configure { |config| config.logger = nil }
+    end
   end # log_error
 end

data/test/test_helper.rb

@@ -90,4 +90,12 @@ class MiniTest::Spec
     silenced(3) { @res = connection.tubes.reserve }
     return @res, @res.body
   end
+
+  # clear_jobs!('foo')
+  def clear_jobs!(*tube_names)
+    tube_names.each do |tube_name|
+      expanded_name = [Backburner.configuration.tube_namespace, tube_name].join(".")
+      Backburner::Worker.connection.tubes.find(expanded_name).clear
+    end
+  end
 end # MiniTest::Spec

data/test/worker_test.rb

@@ -1,6 +1,7 @@
 require File.expand_path('../test_helper', __FILE__)
 
 $worker_test_count = 0
+$worker_success = false
 
 class TestJob
   include Backburner::Queue
@@ -13,13 +14,24 @@ class TestFailJob
   def self.perform(x, y); raise RuntimeError; end
 end
 
+class TestRetryJob
+  include Backburner::Queue
+  def self.perform(x, y)
+    $worker_test_count += 1
+    raise RuntimeError unless $worker_test_count > 2
+    $worker_success = true
+  end
+end
+
 class TestAsyncJob
   include Backburner::Performable
   def self.foo(x, y); $worker_test_count = x * y; end
 end
 
 describe "Backburner::Worker module" do
-  before { Backburner.default_queues.clear }
+  before do
+    Backburner.default_queues.clear
+  end
 
   describe "for enqueue class method" do
     it "should support enqueuing job" do
@@ -144,8 +156,13 @@ describe "Backburner::Worker module" do
   end # prepare
 
   describe "for work_one_job method" do
-    it "should work an enqueued job" do
+    before do
       $worker_test_count = 0
+      $worker_success = false
+    end
+
+    it "should work an enqueued job" do
+      clear_jobs!("foo.bar")
       Backburner::Worker.enqueue TestJob, [1, 2], :queue => "foo.bar"
       silenced(2) do
         worker = Backburner::Worker.new('foo.bar')
@@ -155,28 +172,94 @@ describe "Backburner::Worker module" do
       assert_equal 3, $worker_test_count
     end # enqueue
 
+    it "should fail quietly if there's an argument error" do
+      clear_jobs!("foo.bar")
+      Backburner::Worker.enqueue TestJob, ["bam", "foo", "bar"], :queue => "foo.bar"
+      out = silenced(2) do
+        worker = Backburner::Worker.new('foo.bar')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_match(/Exception ArgumentError/, out)
+      assert_equal 0, $worker_test_count
+    end # fail, argument
+
     it "should work an enqueued failing job" do
-      $worker_test_count = 0
-      Backburner::Worker.enqueue TestFailJob, [1, 2], :queue => "foo.bar.fail"
+      clear_jobs!("foo.bar")
+      Backburner::Worker.enqueue TestFailJob, [1, 2], :queue => "foo.bar"
       Backburner::Job.any_instance.expects(:bury).once
       out = silenced(2) do
-        worker = Backburner::Worker.new('foo.bar.fail')
+        worker = Backburner::Worker.new('foo.bar')
         worker.prepare
         worker.work_one_job
       end
       assert_match(/Exception RuntimeError/, out)
       assert_equal 0, $worker_test_count
-    end # fail
+    end # fail, runtime error
+
+    it "should work an invalid job parsed" do
+      Beaneater::Tubes.any_instance.expects(:reserve).returns(stub(:body => "{%$^}", :bury => true))
+      out = silenced(2) do
+        worker = Backburner::Worker.new('foo.bar')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_match(/Exception Backburner::Job::JobFormatInvalid/, out)
+      assert_equal 0, $worker_test_count
+    end # fail, runtime error
 
     it "should work for an async job" do
-      $worker_test_count = 0
-      TestAsyncJob.async(:queue => "bar.baz").foo(3, 5)
+      clear_jobs!('foo.bar')
+      TestAsyncJob.async(:queue => 'foo.bar').foo(3, 5)
       silenced(2) do
-        worker = Backburner::Worker.new('bar.baz')
+        worker = Backburner::Worker.new('foo.bar')
         worker.prepare
         worker.work_one_job
       end
       assert_equal 15, $worker_test_count
     end # async
+
+    it "should support retrying jobs and burying" do
+      clear_jobs!('foo.bar')
+      Backburner.configure { |config| config.max_job_retries = 1; config.retry_delay = 0 }
+      Backburner::Worker.enqueue TestRetryJob, ["bam", "foo"], :queue => 'foo.bar'
+      out = []
+      2.times do
+        out << silenced(2) do
+          worker = Backburner::Worker.new('foo.bar')
+          worker.prepare
+          worker.work_one_job
+        end
+      end
+      assert_match /attempt 1 of 2, retrying/, out.first
+      assert_match /Finished TestRetryJob/m, out.last
+      assert_match /attempt 2 of 2, burying/m, out.last
+      assert_equal 2, $worker_test_count
+      assert_equal false, $worker_success
+    end # retry, bury
+
+    it "should support retrying jobs and succeeds" do
+      clear_jobs!('foo.bar')
+      Backburner.configure { |config| config.max_job_retries = 2; config.retry_delay = 0 }
+      Backburner::Worker.enqueue TestRetryJob, ["bam", "foo"], :queue => 'foo.bar'
+      out = []
+      3.times do
+        out << silenced(2) do
+          worker = Backburner::Worker.new('foo.bar')
+          worker.prepare
+          worker.work_one_job
+        end
+      end
+      assert_match /attempt 1 of 3, retrying/, out.first
+      assert_match /attempt 2 of 3, retrying/, out[1]
+      assert_match /Finished TestRetryJob/m, out.last
+      refute_match(/failed/, out.last)
+      assert_equal 3, $worker_test_count
+      assert_equal true, $worker_success
+    end # retrying, succeeds
+
+    after do
+      Backburner.configure { |config| config.max_job_retries = 0; config.retry_delay = 5 }
+    end
   end # work_one_job
 end # Backburner::Worker

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: backburner
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.1.2
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-11-06 00:00:00.000000000 Z
+date: 2012-11-07 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: beaneater
@@ -18,7 +18,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 0.1.1
+        version: 0.1.2
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -26,7 +26,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 0.1.1
+        version: 0.1.2
 - !ruby/object:Gem::Dependency
   name: dante
   requirement: !ruby/object:Gem::Requirement
@@ -112,6 +112,7 @@ files:
 - examples/custom.rb
 - examples/demo.rb
 - examples/god.rb
+- examples/retried.rb
 - examples/simple.rb
 - lib/backburner.rb
 - lib/backburner/async_proxy.rb