backburner 0.2.0 → 0.2.5

data/CHANGELOG.md

@@ -1,21 +1,25 @@
 # CHANGELOG
 
-## Version 0.2.1 (Unreleased)
+## Version 0.2.6 (Unreleased)
 
-## Version 0.2.0 (Nov 7th 2012)
+## Version 0.2.5 (Nov 9 2012)
+
+ * Add support for multiple worker processing strategies through subclassing.
+
+## Version 0.2.0 (Nov 7 2012)
 
  * Add new plugin hooks feature (see HOOKS.md)
 
-## Version 0.1.2 (Nov 7th 2012)
+## Version 0.1.2 (Nov 7 2012)
 
  * Adds ability to specify a custom logger.
  * Adds job retry configuration and worker support.
 
-## Version 0.1.1 (Nov 6th 2012)
+## Version 0.1.1 (Nov 6 2012)
 
  * Fix issue with timed out reserves
 
-## Version 0.1.0 (Nov 4th 2012)
+## Version 0.1.0 (Nov 4 2012)
 
  * Switch to beaneater as new ruby beanstalkd client
  * Add support for array of connections in `beanstalk_url`

data/CONTRIBUTING.md

@@ -0,0 +1,37 @@
+We love pull requests. Here's a quick guide:
+
+1. Fork the repo.
+
+2. Run the tests. We only take pull requests with passing tests, and it's great
+to know that you have a clean slate: `bundle && rake test`
+
+3. Add a test for your change. Only refactoring and documentation changes
+require no new tests. If you are adding functionality or fixing a bug, we need
+a test!
+
+4. Make the test pass.
+
+5. Push to your fork and submit a pull request.
+
+At this point you're waiting on us. We like to at least comment on, if not
+accept, pull requests within three business days (and, typically, one business
+day). We may suggest some changes or improvements or alternatives.
+
+Some things that will increase the chance that your pull request is accepted:
+
+* Use Rails idioms and helpers
+* Include tests that fail without your code, and pass with it
+* Update the documentation and README for anything affected by your contribution
+
+Syntax:
+
+* Two spaces, no tabs.
+* No trailing whitespace. Blank lines should not have any space.
+* Prefer &&/|| over and/or.
+* MyClass.my_method(my_arg) not my_method( my_arg ) or my_method my_arg.
+* a = b and not a=b.
+* Follow the conventions you see used in the source already.
+
+And in case we didn't emphasize it enough: we love tests!
+
+NOTE: Adapted from https://raw.github.com/thoughtbot/factory_girl_rails/master/CONTRIBUTING.md

data/README.md

@@ -8,8 +8,11 @@ web framework, but is especially suited for use with [Sinatra](http://sinatrarb.
 
 If you want to use beanstalk for your job processing, consider using Backburner.
 Backburner is heavily inspired by Resque and DelayedJob. Backburner stores all jobs as simple JSON message payloads.
-Backburner can be a persistent queue when the beanstalk persistence mode is enabled.
-It supports multiple queues, priorities, delays, and timeouts.
+Persistent queues are supported when beanstalkd persistence mode is enabled.
+
+Backburner supports multiple queues, job priorities, delays, and timeouts. In addition, 
+Backburner has robust support for retrying failed jobs, handling error cases,
+custom logging, and extensible plugin hooks.
 
 ## Why Backburner?
 
@@ -83,20 +86,24 @@ Backburner is extremely simple to setup. Just configure basic settings for backb
 
 ```ruby
 Backburner.configure do |config|
-  config.beanstalk_url    = ["beanstalk://127.0.0.1", "beanstalk://127.0.0.1:11301"]
+  config.beanstalk_url    = ["beanstalk://127.0.0.1", "..."]
   config.tube_namespace   = "some.app.production"
   config.on_error         = lambda { |e| puts e }
   config.max_job_retries  = 3 # default 0 retries
   config.retry_delay      = 2 # default 5 seconds
   config.default_priority = 65536
   config.respond_timeout  = 120
+  config.default_worker   = Backburner::Workers::Simple
   config.logger           = Logger.new(STDOUT)
 end
 ```
 
+The key options available are:
+
  * The `beanstalk_url` supports a string such as 'beanstalk://127.0.0.1' or an array of addresses.
  * The `tube_namespace` is the prefix used for all tubes related to this backburner queue.
  * The `on_error` is a callback that gets invoked with the error whenever any job in the system fails.
+ * The `default_worker` is the processing worker that will be used if no other worker is specified.
  * The `max_job_retries` determines how many times to retry a job before burying
  * The `retry_delay` determines the base time to wait (in secs) between retries
  * The `logger` is the logger object written to when backburner wants to report info or errors.
@@ -203,6 +210,35 @@ bundle exec backburner newsletter-sender,push-message -d -P /var/run/backburner.
 
 This will daemonize the worker and store the pid and logs automatically.
 
+### Processing Strategies
+
+In Backburner, there are actually multiple different strategies for processing jobs
+which are reflected by multiple workers.
+Custom workers can be [defined fairly easily](https://github.com/nesquena/backburner/wiki/Defining-Workers).
+By default, Backburner comes with the following workers built-in:
+
+| Worker | Description                                                                 |
+| ------- | -------------------------------                                            |
+| `Backburner::Workers::Simple` | Single threaded, no forking worker. Simplest option. |
+
+You can select the default worker for processing with:
+
+```ruby
+Backburner.configure do |config|
+  config.default_worker = Backburner::Workers::Simple
+end
+```
+
+or determine the worker on the fly when invoking `work`:
+
+```ruby
+Backburner.work('newsletter_sender', :worker => Backburner::Workers::Threaded)
+```
+
+or when more official workers are supported, through alternate rake tasks.
+Additional workers such as `threaded`, `forking` and `threads_on_fork` will hopefully be
+developed in the future. If you are interested in helping, please let us know.
+
 ### Default Queues
 
 Workers can be easily restricted to processing only a specific set of queues as shown above. However, if you want a worker to
@@ -237,7 +273,7 @@ Backburner is highly extensible and can be tailored to your needs by using vario
 can be triggered across the job processing lifecycle. 
 Often using hooks is much easier then trying to monkey patch the externals. 
 
-Check out [HOOKS.md](HOOKS.md) for a detailed overview on using hooks.
+Check out [HOOKS.md](https://github.com/nesquena/backburner/blob/master/HOOKS.md) for a detailed overview on using hooks.
 
 ### Failures
 

data/lib/backburner.rb

@@ -10,6 +10,7 @@ require 'backburner/connection'
 require 'backburner/hooks'
 require 'backburner/performable'
 require 'backburner/worker'
+require 'backburner/workers/simple'
 require 'backburner/queue'
 
 module Backburner
@@ -28,9 +29,12 @@ module Backburner
     #
     # @example
     #   Backburner.work('newsletter_sender', 'test_job')
+    #   Backburner.work('newsletter_sender', 'test_job', :worker => NotSimpleWorker)
     #
     def work(*tubes)
-      Backburner::Worker.start(tubes)
+      options = tubes.last.is_a?(Hash) ? tubes.pop : {}
+      worker_class = options[:worker] || configuration.default_worker
+      worker_class.start(tubes)
     end
 
     # Yields a configuration block

data/lib/backburner/configuration.rb

@@ -9,6 +9,7 @@ module Backburner
     attr_accessor :retry_delay        # retry delay in seconds
     attr_accessor :default_queues     # default queues
     attr_accessor :logger             # logger
+    attr_accessor :default_worker     # default worker class
 
     def initialize
       @beanstalk_url     = "beanstalk://localhost"
@@ -20,6 +21,7 @@ module Backburner
       @retry_delay       = 5
       @default_queues    = []
       @logger            = nil
+      @default_worker    = Backburner::Workers::Simple
     end
   end # Configuration
 end # Backburner

data/lib/backburner/version.rb

@@ -1,3 +1,3 @@
 module Backburner
-  VERSION = "0.2.0"
+  VERSION = "0.2.5"
 end

data/lib/backburner/worker.rb

@@ -1,6 +1,10 @@
 require 'backburner/job'
 
 module Backburner
+  #
+  # @abstract Subclass and override {#process_tube_names}, {#prepare} and {#start} to implement
+  #   a custom Worker class.
+  #
   class Worker
     include Backburner::Helpers
     include Backburner::Logger
@@ -12,7 +16,7 @@ module Backburner
       def known_queue_classes; @known_queue_classes ||= []; end
     end
 
-    # Enqueues a job to be processed later by a worker
+    # Enqueues a job to be processed later by a worker.
     # Options: `pri` (priority), `delay` (delay in secs), `ttr` (time to respond), `queue` (queue name)
     #
     # @raise [Beaneater::NotConnected] If beanstalk fails to connect.
@@ -32,7 +36,7 @@ module Backburner
       return true
     end
 
-    # Starts processing jobs in the specified tube_names
+    # Starts processing jobs with the specified tube_names.
     #
     # @example
     #   Backburner::Worker.start(["foo.tube.name"])
@@ -41,7 +45,7 @@ module Backburner
       self.new(tube_names).start
     end
 
-    # Returns the worker connection
+    # Returns the worker connection.
     # @example
     #   Backburner::Worker.connection # => <Beaneater::Pool>
     def self.connection
@@ -51,45 +55,53 @@ module Backburner
     # List of tube names to be watched and processed
     attr_accessor :tube_names
 
+    # Constructs a new worker for processing jobs within specified tubes.
+    #
     # @example
     #   Worker.new(['test.job'])
     def initialize(tube_names=nil)
-      @tube_names = begin
-        tube_names = tube_names.first if tube_names && tube_names.size == 1 && tube_names.first.is_a?(Array)
-        tube_names = Array(tube_names).compact if tube_names && Array(tube_names).compact.size > 0
-        tube_names = nil if tube_names && tube_names.compact.empty?
-        tube_names
-      end
+      @tube_names = self.process_tube_names(tube_names)
     end
 
-    # Starts processing new jobs indefinitely
-    # Primary way to consume and process jobs in specified tubes
+    # Starts processing ready jobs indefinitely.
+    # Primary way to consume and process jobs in specified tubes.
     #
     # @example
     #   @worker.start
     #
     def start
-      prepare
-      loop { work_one_job }
+      raise NotImplementedError
     end
 
-    # Setup beanstalk tube_names and watch all specified tubes for jobs.
-    # Used to prepare job queues before processing jobs.
+    # Used to prepare the job queues before job processing is initiated.
     #
     # @raise [Beaneater::NotConnected] If beanstalk fails to connect.
     # @example
     #   @worker.prepare
     #
+    # @abstract Define this in your worker subclass
+    # to be run once before processing. Recommended to watch tubes
+    # or print a message to the logs with 'log_info'
+    #
     def prepare
-      self.tube_names ||= Backburner.default_queues.any? ? Backburner.default_queues : all_existing_queues
-      self.tube_names = Array(self.tube_names)
-      self.tube_names.map! { |name| expand_tube_name(name)  }
-      log_info "Working #{tube_names.size} queues: [ #{tube_names.join(', ')} ]"
-      self.connection.tubes.watch!(*self.tube_names)
+      raise NotImplementedError
+    end
+
+    # Processes tube_names given tube_names array.
+    # Should return normalized tube_names as an array of strings.
+    #
+    # @example
+    #   process_tube_names([['foo'], ['bar']])
+    #   => ['foo', 'bar', 'baz']
+    #
+    # @note This method can be overridden in inherited workers
+    # to add more complex tube name processing.
+    def process_tube_names(tube_names)
+      compact_tube_names(tube_names)
     end
 
-    # Reserves one job within the specified queues
-    # Pops the job off and serializes the job to JSON
+    # Reserves one job within the specified queues.
+    # Pops the job off and serializes the job to JSON.
     # Each job is performed by invoking `perform` on the job class.
     #
     # @example
@@ -143,5 +155,16 @@ module Backburner
         end
       end
     end
+
+    # Normalizes tube names given array of tube_names
+    # Compacts nil items, flattens arrays, sets tubes to nil if no valid names
+    # Loads default tubes when no tubes given.
+    def compact_tube_names(tube_names)
+      tube_names = tube_names.first if tube_names && tube_names.size == 1 && tube_names.first.is_a?(Array)
+      tube_names = Array(tube_names).compact if tube_names && Array(tube_names).compact.size > 0
+      tube_names = nil if tube_names && tube_names.compact.empty?
+      tube_names ||= Backburner.default_queues.any? ? Backburner.default_queues : all_existing_queues
+      Array(tube_names)
+    end
   end # Worker
 end # Backburner

data/lib/backburner/workers/simple.rb

@@ -0,0 +1,29 @@
+module Backburner
+  module Workers
+    class Simple < Worker
+      # Used to prepare job queues before processing jobs.
+      # Setup beanstalk tube_names and watch all specified tubes for jobs.
+      #
+      # @raise [Beaneater::NotConnected] If beanstalk fails to connect.
+      # @example
+      #   @worker.prepare
+      #
+      def prepare
+        self.tube_names.map! { |name| expand_tube_name(name)  }
+        log_info "Working #{tube_names.size} queues: [ #{tube_names.join(', ')} ]"
+        self.connection.tubes.watch!(*self.tube_names)
+      end
+
+      # Starts processing new jobs indefinitely.
+      # Primary way to consume and process jobs in specified tubes.
+      #
+      # @example
+      #   @worker.start
+      #
+      def start
+        prepare
+        loop { work_one_job }
+      end
+    end # Basic
+  end # Workers
+end # Backburner

data/test/back_burner_test.rb

@@ -13,6 +13,8 @@ class TestBackburnerJob
   end
 end
 
+class TestWorker < Backburner::Worker; end
+
 describe "Backburner module" do
   before { Backburner.default_queues.clear }
 
@@ -21,7 +23,7 @@ describe "Backburner module" do
       Backburner.enqueue TestBackburnerJob, 5, 6
       Backburner.enqueue TestBackburnerJob, 15, 10
       silenced(2) do
-        worker = Backburner::Worker.new('test.jobber')
+        worker = Backburner::Workers::Simple.new('test.jobber')
         worker.prepare
         2.times { worker.work_one_job }
       end
@@ -34,13 +36,24 @@ describe "Backburner module" do
   end # enqueue
 
   describe "for work method" do
-    it "invokes worker start" do
-      Backburner::Worker.expects(:start).with(["foo", "bar"])
+    it "invokes worker simple start" do
+      Backburner::Workers::Simple.expects(:start).with(["foo", "bar"])
+      Backburner.work("foo", "bar")
+    end
+
+    it "invokes other worker if specified in configuration" do
+      Backburner.configure { |config| config.default_worker = TestWorker }
+      TestWorker.expects(:start).with(["foo", "bar"])
       Backburner.work("foo", "bar")
     end
 
+    it "invokes other worker if specified in work method as options" do
+      TestWorker.expects(:start).with(["foo", "bar"])
+      Backburner.work("foo", "bar", :worker => TestWorker)
+    end
+
     it "invokes worker start with no args" do
-      Backburner::Worker.expects(:start).with([])
+      Backburner::Workers::Simple.expects(:start).with([])
       Backburner.work
     end
   end # work!
@@ -58,4 +71,8 @@ describe "Backburner module" do
       assert_same_elements ["foo", "bar"], Backburner.default_queues
     end
   end
+
+  after do
+    Backburner.configure { |config| config.default_worker = Backburner::Workers::Simple }
+  end
 end # Backburner

data/test/simple_worker_test.rb

@@ -0,0 +1,235 @@
+require File.expand_path('../test_helper', __FILE__)
+require File.expand_path('../fixtures/test_jobs', __FILE__)
+require File.expand_path('../fixtures/hooked', __FILE__)
+
+describe "Backburner::Workers::Basic module" do
+  before do
+    Backburner.default_queues.clear
+    @worker_class = Backburner::Workers::Simple
+  end
+
+  describe "for prepare method" do
+    it "should watch specified tubes" do
+      worker = @worker_class.new(["foo", "bar"])
+      out = capture_stdout { worker.prepare }
+      assert_equal ["demo.test.foo", "demo.test.bar"], worker.tube_names
+      assert_same_elements ["demo.test.foo", "demo.test.bar"], @worker_class.connection.tubes.watched.map(&:name)
+      assert_match /demo\.test\.foo/, out
+    end # multiple
+
+    it "should watch single tube" do
+      worker = @worker_class.new("foo")
+      out = capture_stdout { worker.prepare }
+      assert_equal ["demo.test.foo"], worker.tube_names
+      assert_same_elements ["demo.test.foo"], @worker_class.connection.tubes.watched.map(&:name)
+      assert_match /demo\.test\.foo/, out
+    end # single
+
+    it "should respect default_queues settings" do
+      Backburner.default_queues.concat(["foo", "bar"])
+      worker = @worker_class.new
+      out = capture_stdout { worker.prepare }
+      assert_equal ["demo.test.foo", "demo.test.bar"], worker.tube_names
+      assert_same_elements ["demo.test.foo", "demo.test.bar"], @worker_class.connection.tubes.watched.map(&:name)
+      assert_match /demo\.test\.foo/, out
+    end
+
+    it "should assign based on all tubes" do
+      @worker_class.any_instance.expects(:all_existing_queues).once.returns("bar")
+      worker = @worker_class.new
+      out = capture_stdout { worker.prepare }
+      assert_equal ["demo.test.bar"], worker.tube_names
+      assert_same_elements ["demo.test.bar"], @worker_class.connection.tubes.watched.map(&:name)
+      assert_match /demo\.test\.bar/, out
+    end # all assign
+
+    it "should properly retrieve all tubes" do
+      worker = @worker_class.new
+      out = capture_stdout { worker.prepare }
+      assert_contains worker.tube_names, "demo.test.test-job"
+      assert_contains @worker_class.connection.tubes.watched.map(&:name), "demo.test.test-job"
+      assert_match /demo\.test\.test-job/, out
+    end # all read
+  end # prepare
+
+  describe "for work_one_job method" do
+    before do
+      $worker_test_count = 0
+      $worker_success = false
+    end
+
+    it "should work an enqueued job" do
+      clear_jobs!("foo.bar")
+      @worker_class.enqueue TestJob, [1, 2], :queue => "foo.bar"
+      silenced(2) do
+        worker = @worker_class.new('foo.bar')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_equal 3, $worker_test_count
+    end # enqueue
+
+    it "should fail quietly if there's an argument error" do
+      clear_jobs!("foo.bar")
+      @worker_class.enqueue TestJob, ["bam", "foo", "bar"], :queue => "foo.bar"
+      out = silenced(2) do
+        worker = @worker_class.new('foo.bar')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_match(/Exception ArgumentError/, out)
+      assert_equal 0, $worker_test_count
+    end # fail, argument
+
+    it "should work an enqueued failing job" do
+      clear_jobs!("foo.bar")
+      @worker_class.enqueue TestFailJob, [1, 2], :queue => "foo.bar"
+      Backburner::Job.any_instance.expects(:bury).once
+      out = silenced(2) do
+        worker = @worker_class.new('foo.bar')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_match(/Exception RuntimeError/, out)
+      assert_equal 0, $worker_test_count
+    end # fail, runtime error
+
+    it "should work an invalid job parsed" do
+      Beaneater::Tubes.any_instance.expects(:reserve).returns(stub(:body => "{%$^}", :bury => true))
+      out = silenced(2) do
+        worker = @worker_class.new('foo.bar')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_match(/Exception Backburner::Job::JobFormatInvalid/, out)
+      assert_equal 0, $worker_test_count
+    end # fail, runtime error
+
+    it "should work for an async job" do
+      clear_jobs!('foo.bar')
+      TestAsyncJob.async(:queue => 'foo.bar').foo(3, 5)
+      silenced(2) do
+        worker = @worker_class.new('foo.bar')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_equal 15, $worker_test_count
+    end # async
+
+    it "should support retrying jobs and burying" do
+      clear_jobs!('foo.bar')
+      Backburner.configure { |config| config.max_job_retries = 1; config.retry_delay = 0 }
+      @worker_class.enqueue TestRetryJob, ["bam", "foo"], :queue => 'foo.bar'
+      out = []
+      2.times do
+        out << silenced(2) do
+          worker = @worker_class.new('foo.bar')
+          worker.prepare
+          worker.work_one_job
+        end
+      end
+      assert_match /attempt 1 of 2, retrying/, out.first
+      assert_match /Finished TestRetryJob/m, out.last
+      assert_match /attempt 2 of 2, burying/m, out.last
+      assert_equal 2, $worker_test_count
+      assert_equal false, $worker_success
+    end # retry, bury
+
+    it "should support retrying jobs and succeeds" do
+      clear_jobs!('foo.bar')
+      Backburner.configure { |config| config.max_job_retries = 2; config.retry_delay = 0 }
+      @worker_class.enqueue TestRetryJob, ["bam", "foo"], :queue => 'foo.bar'
+      out = []
+      3.times do
+        out << silenced(2) do
+          worker = @worker_class.new('foo.bar')
+          worker.prepare
+          worker.work_one_job
+        end
+      end
+      assert_match /attempt 1 of 3, retrying/, out.first
+      assert_match /attempt 2 of 3, retrying/, out[1]
+      assert_match /Finished TestRetryJob/m, out.last
+      refute_match(/failed/, out.last)
+      assert_equal 3, $worker_test_count
+      assert_equal true, $worker_success
+    end # retrying, succeeds
+
+    it "should support event hooks without retry" do
+      $hooked_fail_count = 0
+      clear_jobs!('foo.bar.events')
+      out = silenced(2) do
+        HookedObjectSuccess.async(:queue => 'foo.bar.events').foo(5)
+        worker = @worker_class.new('foo.bar.events')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_match /before_enqueue.*after_enqueue.*Working 1 queues/m, out
+      assert_match /!!before_enqueue_bar!! \[nil, :foo, 5\]/, out
+      assert_match /!!after_enqueue_bar!! \[nil, :foo, 5\]/, out
+      assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
+      assert_match /!!BEGIN around_perform_bar!! \[nil, "foo", 5\]/, out
+      assert_match /!!BEGIN around_perform_cat!! \[nil, "foo", 5\]/, out
+      assert_match /!!on_failure_foo!!.*HookFailError/, out
+      assert_match /attempt 1 of 1, burying/, out
+    end # event hooks, no retry
+
+    it "should support event hooks with retry" do
+      $hooked_fail_count = 0
+      clear_jobs!('foo.bar.events.retry')
+      Backburner.configure { |config| config.max_job_retries = 1; config.retry_delay = 0 }
+      out = silenced(2) do
+        HookedObjectSuccess.async(:queue => 'foo.bar.events.retry').foo(5)
+        worker = @worker_class.new('foo.bar.events.retry')
+        worker.prepare
+        2.times do
+          worker.work_one_job
+        end
+      end
+      assert_match /before_enqueue.*after_enqueue.*Working 1 queues/m, out
+      assert_match /!!before_enqueue_bar!! \[nil, :foo, 5\]/, out
+      assert_match /!!after_enqueue_bar!! \[nil, :foo, 5\]/, out
+      assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
+      assert_match /!!BEGIN around_perform_bar!! \[nil, "foo", 5\]/, out
+      assert_match /!!BEGIN around_perform_cat!! \[nil, "foo", 5\]/, out
+      assert_match /!!on_failure_foo!!.*HookFailError/, out
+      assert_match /!!on_failure_foo!!.*retrying.*around_perform_bar.*around_perform_cat/m, out
+      assert_match /attempt 1 of 2, retrying/, out
+      assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
+      assert_match /!!END around_perform_bar!! \[nil, "foo", 5\]/, out
+      assert_match /!!END around_perform_cat!! \[nil, "foo", 5\]/, out
+      assert_match /!!after_perform_foo!! \[nil, "foo", 5\]/, out
+      assert_match /Finished HookedObjectSuccess/, out
+    end # event hooks, with retry
+
+    it "should support event hooks with stopping enqueue" do
+      $hooked_fail_count = 0
+      clear_jobs!('foo.bar.events.retry2')
+      out = silenced(2) do
+        HookedObjectBeforeEnqueueFail.async(:queue => 'foo.bar.events.retry2').foo(5)
+      end
+      expanded_tube = [Backburner.configuration.tube_namespace, 'foo.bar.events.retry2'].join(".")
+      assert_nil @worker_class.connection.tubes[expanded_tube].peek(:ready)
+    end # stopping enqueue
+
+    it "should support event hooks with stopping perform" do
+      $hooked_fail_count = 0
+      clear_jobs!('foo.bar.events.retry3')
+      expanded_tube = [Backburner.configuration.tube_namespace, 'foo.bar.events.retry3'].join(".")
+      out = silenced(2) do
+        HookedObjectBeforePerformFail.async(:queue => 'foo.bar.events.retry3').foo(10)
+        worker = @worker_class.new('foo.bar.events.retry3')
+        worker.prepare
+        worker.work_one_job
+      end
+      assert_match /!!before_perform_foo!! \[nil, "foo", 10\]/, out
+      assert_match /before_perform_foo.*Finished/m, out
+      refute_match(/Fail ran!!/, out)
+      refute_match(/HookFailError/, out)
+    end # stopping perform
+
+    after do
+      Backburner.configure { |config| config.max_job_retries = 0; config.retry_delay = 5 }
+    end
+  end # work_one_job
+end # Worker

data/test/worker_test.rb

@@ -54,6 +54,11 @@ describe "Backburner::Worker module" do
   end # connection
 
   describe "for tube_names accessor" do
+    before do
+      Backburner.default_queues << "baz"
+      Backburner.default_queues << "bam"
+    end
+
     it "supports retrieving tubes" do
       worker = Backburner::Worker.new(["foo", "bar"])
       assert_equal ["foo", "bar"], worker.tube_names
@@ -64,9 +69,9 @@ describe "Backburner::Worker module" do
       assert_equal ["foo", "bar"], worker.tube_names
     end
 
-    it "supports empty nil array arg" do
+    it "supports empty nil array arg with default values" do
       worker = Backburner::Worker.new([nil])
-      assert_equal nil, worker.tube_names
+      assert_equal ['baz', 'bam'], worker.tube_names
     end
 
     it "supports single tube arg" do
@@ -74,239 +79,14 @@ describe "Backburner::Worker module" do
       assert_equal ["foo"], worker.tube_names
     end
 
-    it "supports empty array arg" do
+    it "supports empty array arg with default values" do
       worker = Backburner::Worker.new([])
-      assert_equal nil, worker.tube_names
+      assert_equal ['baz', 'bam'], worker.tube_names
     end
 
-    it "supports nil arg" do
+    it "supports nil arg with default values" do
       worker = Backburner::Worker.new(nil)
-      assert_equal nil, worker.tube_names
+      assert_equal ['baz', 'bam'], worker.tube_names
     end
   end # tube_names
-
-  describe "for prepare method" do
-    it "should watch specified tubes" do
-      worker = Backburner::Worker.new(["foo", "bar"])
-      out = capture_stdout { worker.prepare }
-      assert_equal ["demo.test.foo", "demo.test.bar"], worker.tube_names
-      assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
-      assert_match /demo\.test\.foo/, out
-    end # multiple
-
-    it "should watch single tube" do
-      worker = Backburner::Worker.new("foo")
-      out = capture_stdout { worker.prepare }
-      assert_equal ["demo.test.foo"], worker.tube_names
-      assert_same_elements ["demo.test.foo"], Backburner::Worker.connection.tubes.watched.map(&:name)
-      assert_match /demo\.test\.foo/, out
-    end # single
-
-    it "should respect default_queues settings" do
-      Backburner.default_queues.concat(["foo", "bar"])
-      worker = Backburner::Worker.new
-      out = capture_stdout { worker.prepare }
-      assert_equal ["demo.test.foo", "demo.test.bar"], worker.tube_names
-      assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
-      assert_match /demo\.test\.foo/, out
-    end
-
-    it "should assign based on all tubes" do
-      Backburner::Worker.any_instance.expects(:all_existing_queues).once.returns("bar")
-      worker = Backburner::Worker.new
-      out = capture_stdout { worker.prepare }
-      assert_equal ["demo.test.bar"], worker.tube_names
-      assert_same_elements ["demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
-      assert_match /demo\.test\.bar/, out
-    end # all assign
-
-    it "should properly retrieve all tubes" do
-      worker = Backburner::Worker.new
-      out = capture_stdout { worker.prepare }
-      assert_contains worker.tube_names, "demo.test.test-job"
-      assert_contains Backburner::Worker.connection.tubes.watched.map(&:name), "demo.test.test-job"
-      assert_match /demo\.test\.test-job/, out
-    end # all read
-  end # prepare
-
-  describe "for work_one_job method" do
-    before do
-      $worker_test_count = 0
-      $worker_success = false
-    end
-
-    it "should work an enqueued job" do
-      clear_jobs!("foo.bar")
-      Backburner::Worker.enqueue TestJob, [1, 2], :queue => "foo.bar"
-      silenced(2) do
-        worker = Backburner::Worker.new('foo.bar')
-        worker.prepare
-        worker.work_one_job
-      end
-      assert_equal 3, $worker_test_count
-    end # enqueue
-
-    it "should fail quietly if there's an argument error" do
-      clear_jobs!("foo.bar")
-      Backburner::Worker.enqueue TestJob, ["bam", "foo", "bar"], :queue => "foo.bar"
-      out = silenced(2) do
-        worker = Backburner::Worker.new('foo.bar')
-        worker.prepare
-        worker.work_one_job
-      end
-      assert_match(/Exception ArgumentError/, out)
-      assert_equal 0, $worker_test_count
-    end # fail, argument
-
-    it "should work an enqueued failing job" do
-      clear_jobs!("foo.bar")
-      Backburner::Worker.enqueue TestFailJob, [1, 2], :queue => "foo.bar"
-      Backburner::Job.any_instance.expects(:bury).once
-      out = silenced(2) do
-        worker = Backburner::Worker.new('foo.bar')
-        worker.prepare
-        worker.work_one_job
-      end
-      assert_match(/Exception RuntimeError/, out)
-      assert_equal 0, $worker_test_count
-    end # fail, runtime error
-
-    it "should work an invalid job parsed" do
-      Beaneater::Tubes.any_instance.expects(:reserve).returns(stub(:body => "{%$^}", :bury => true))
-      out = silenced(2) do
-        worker = Backburner::Worker.new('foo.bar')
-        worker.prepare
-        worker.work_one_job
-      end
-      assert_match(/Exception Backburner::Job::JobFormatInvalid/, out)
-      assert_equal 0, $worker_test_count
-    end # fail, runtime error
-
-    it "should work for an async job" do
-      clear_jobs!('foo.bar')
-      TestAsyncJob.async(:queue => 'foo.bar').foo(3, 5)
-      silenced(2) do
-        worker = Backburner::Worker.new('foo.bar')
-        worker.prepare
-        worker.work_one_job
-      end
-      assert_equal 15, $worker_test_count
-    end # async
-
-    it "should support retrying jobs and burying" do
-      clear_jobs!('foo.bar')
-      Backburner.configure { |config| config.max_job_retries = 1; config.retry_delay = 0 }
-      Backburner::Worker.enqueue TestRetryJob, ["bam", "foo"], :queue => 'foo.bar'
-      out = []
-      2.times do
-        out << silenced(2) do
-          worker = Backburner::Worker.new('foo.bar')
-          worker.prepare
-          worker.work_one_job
-        end
-      end
-      assert_match /attempt 1 of 2, retrying/, out.first
-      assert_match /Finished TestRetryJob/m, out.last
-      assert_match /attempt 2 of 2, burying/m, out.last
-      assert_equal 2, $worker_test_count
-      assert_equal false, $worker_success
-    end # retry, bury
-
-    it "should support retrying jobs and succeeds" do
-      clear_jobs!('foo.bar')
-      Backburner.configure { |config| config.max_job_retries = 2; config.retry_delay = 0 }
-      Backburner::Worker.enqueue TestRetryJob, ["bam", "foo"], :queue => 'foo.bar'
-      out = []
-      3.times do
-        out << silenced(2) do
-          worker = Backburner::Worker.new('foo.bar')
-          worker.prepare
-          worker.work_one_job
-        end
-      end
-      assert_match /attempt 1 of 3, retrying/, out.first
-      assert_match /attempt 2 of 3, retrying/, out[1]
-      assert_match /Finished TestRetryJob/m, out.last
-      refute_match(/failed/, out.last)
-      assert_equal 3, $worker_test_count
-      assert_equal true, $worker_success
-    end # retrying, succeeds
-
-    it "should support event hooks without retry" do
-      $hooked_fail_count = 0
-      clear_jobs!('foo.bar.events')
-      out = silenced(2) do
-        HookedObjectSuccess.async(:queue => 'foo.bar.events').foo(5)
-        worker = Backburner::Worker.new('foo.bar.events')
-        worker.prepare
-        worker.work_one_job
-      end
-      assert_match /before_enqueue.*after_enqueue.*Working 1 queues/m, out
-      assert_match /!!before_enqueue_bar!! \[nil, :foo, 5\]/, out
-      assert_match /!!after_enqueue_bar!! \[nil, :foo, 5\]/, out
-      assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
-      assert_match /!!BEGIN around_perform_bar!! \[nil, "foo", 5\]/, out
-      assert_match /!!BEGIN around_perform_cat!! \[nil, "foo", 5\]/, out
-      assert_match /!!on_failure_foo!!.*HookFailError/, out
-      assert_match /attempt 1 of 1, burying/, out
-    end # event hooks, no retry
-
-    it "should support event hooks with retry" do
-      $hooked_fail_count = 0
-      clear_jobs!('foo.bar.events.retry')
-      Backburner.configure { |config| config.max_job_retries = 1; config.retry_delay = 0 }
-      out = silenced(2) do
-        HookedObjectSuccess.async(:queue => 'foo.bar.events.retry').foo(5)
-        worker = Backburner::Worker.new('foo.bar.events.retry')
-        worker.prepare
-        2.times do
-          worker.work_one_job
-        end
-      end
-      assert_match /before_enqueue.*after_enqueue.*Working 1 queues/m, out
-      assert_match /!!before_enqueue_bar!! \[nil, :foo, 5\]/, out
-      assert_match /!!after_enqueue_bar!! \[nil, :foo, 5\]/, out
-      assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
-      assert_match /!!BEGIN around_perform_bar!! \[nil, "foo", 5\]/, out
-      assert_match /!!BEGIN around_perform_cat!! \[nil, "foo", 5\]/, out
-      assert_match /!!on_failure_foo!!.*HookFailError/, out
-      assert_match /!!on_failure_foo!!.*retrying.*around_perform_bar.*around_perform_cat/m, out
-      assert_match /attempt 1 of 2, retrying/, out
-      assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
-      assert_match /!!END around_perform_bar!! \[nil, "foo", 5\]/, out
-      assert_match /!!END around_perform_cat!! \[nil, "foo", 5\]/, out
-      assert_match /!!after_perform_foo!! \[nil, "foo", 5\]/, out
-      assert_match /Finished HookedObjectSuccess/, out
-    end # event hooks, with retry
-
-    it "should support event hooks with stopping enqueue" do
-      $hooked_fail_count = 0
-      clear_jobs!('foo.bar.events.retry2')
-      out = silenced(2) do
-        HookedObjectBeforeEnqueueFail.async(:queue => 'foo.bar.events.retry2').foo(5)
-      end
-      expanded_tube = [Backburner.configuration.tube_namespace, 'foo.bar.events.retry2'].join(".")
-      assert_nil Backburner::Worker.connection.tubes[expanded_tube].peek(:ready)
-    end # stopping enqueue
-
-    it "should support event hooks with stopping perform" do
-      $hooked_fail_count = 0
-      clear_jobs!('foo.bar.events.retry3')
-      expanded_tube = [Backburner.configuration.tube_namespace, 'foo.bar.events.retry3'].join(".")
-      out = silenced(2) do
-        HookedObjectBeforePerformFail.async(:queue => 'foo.bar.events.retry3').foo(10)
-        worker = Backburner::Worker.new('foo.bar.events.retry3')
-        worker.prepare
-        worker.work_one_job
-      end
-      assert_match /!!before_perform_foo!! \[nil, "foo", 10\]/, out
-      assert_match /before_perform_foo.*Finished/m, out
-      refute_match(/Fail ran!!/, out)
-      refute_match(/HookFailError/, out)
-    end # stopping perform
-
-    after do
-      Backburner.configure { |config| config.max_job_retries = 0; config.retry_delay = 5 }
-    end
-  end # work_one_job
 end # Backburner::Worker

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: backburner
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.2.5
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-11-07 00:00:00.000000000 Z
+date: 2012-11-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: beaneater
@@ -102,6 +102,7 @@ files:
 - .gitignore
 - .travis.yml
 - CHANGELOG.md
+- CONTRIBUTING.md
 - Gemfile
 - HOOKS.md
 - LICENSE
@@ -129,6 +130,7 @@ files:
 - lib/backburner/tasks.rb
 - lib/backburner/version.rb
 - lib/backburner/worker.rb
+- lib/backburner/workers/simple.rb
 - test/back_burner_test.rb
 - test/connection_test.rb
 - test/fixtures/hooked.rb
@@ -139,6 +141,7 @@ files:
 - test/logger_test.rb
 - test/performable_test.rb
 - test/queue_test.rb
+- test/simple_worker_test.rb
 - test/test_helper.rb
 - test/worker_test.rb
 homepage: http://github.com/nesquena/backburner
@@ -176,5 +179,6 @@ test_files:
 - test/logger_test.rb
 - test/performable_test.rb
 - test/queue_test.rb
+- test/simple_worker_test.rb
 - test/test_helper.rb
 - test/worker_test.rb