timberline 0.6.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 29b010166f8c116bf4469cf6c09d3d65b0f769ee
-  data.tar.gz: a06cc3487b8acaef76c85c87b7b56d32bd11785d
+  metadata.gz: 1e8abd9f7a068aed49df25b77c38489047e89ac1
+  data.tar.gz: ab2306c2f9ae72ee849e2fef5a4aa632b284b67a
 SHA512:
-  metadata.gz: 6d5ced0644e73be8bd3bce8905e46d99ddeb36c61ae7e78890a8bbd9fa727fd8969301ee4e074e5d4518dda7aa77046238e7f068e48dc4da5968fc899398f682
-  data.tar.gz: 75f4e3acc70d27d9021e8c0d4816f31ac40854a7a30f8bba1d625fb6a3498de8de12a3b89c99fe7102a4bea1a21c7903d4c69232557d87541a7a721f8174991f
+  metadata.gz: 5947f1e61eb13ccdf5663f1a678abc175683dbefcfd0e500d4844d14d287cd947a55120c807f5848a43ee7a9672457474ce123e64ed0923e5bf130f4185b0115
+  data.tar.gz: 4394761078279b91e7ccf86a2bd5c1b29eeb5755a05458b51d03339a4b6944d699071bc039c653f54fe6cf8f1a4beced5abfb89d8b1a1e5d9efc925531df649f

data/.yardoc/checksums

@@ -0,0 +1,8 @@
+lib/timberline.rb bde38a760f2c89941eb1db0dd9bfe16824eacb90
+lib/timberline/queue.rb f4463942e074f53cc10b14adaa9d00ce6a9fe606
+lib/timberline/config.rb b53bd318344a956a8efa1058319c88411c982874
+lib/timberline/worker.rb 3245a5b986391d28c1a0ea745fa09df5df375cf7
+lib/timberline/version.rb 926b9f154a084b54b1d8636c711cbaa7d8166998
+lib/timberline/envelope.rb fd34ad408e519b1655bf8163b4d81fb0600f7ca4
+lib/timberline/exceptions.rb 3dcda3d2e61e920696eba97d2b84e1a1df642a7a
+lib/timberline/anonymous_worker.rb 2f8fb80e35f3702ce61384d2b18ba334387046ef

data/CHANGELOG

@@ -1,3 +1,6 @@
+0.7.0
+  - Add Worker objects
+  - Add YARD documentation, publish docs to rubydoc.info
 0.6.0
   - Remove Rails-specific configuration logic in preparation for release of timberline-rails
 0.5.0

data/README.markdown

@@ -1,6 +1,7 @@
 # Timberline
 
-![](https://travis-ci.org/treehouse/timberline.svg)
+![](https://travis-ci.org/treehouse/timberline.svg) 
+[![Gem Version](https://badge.fury.io/rb/timberline.svg)](http://badge.fury.io/rb/timberline)
 
 ## Purpose
 
@@ -21,8 +22,28 @@ kind of issues you might be dealing with:
    jobs as fast as you possibly can. To that end, Timberline uses blocking reads
    in Redis to pull jobs off of the queue as soon as they're available.
 
+## Documentation
+
+Documentation for Timberline is available on rubydoc.info [here](http://rubydoc.info/github/treehouse/timberline/frames). The code itself is documented with YARD.
+
 ## Concepts
 
+### The Envelope
+
+Sounds SOAPy, I know. The envelope is a simple object that wraps the data you
+want to put on the queue - it's responsible for tracking things like the job ID,
+the queue it was put on, how many times it's been retried, etc., etc. It's also
+accessible to both the queue processor and whatever is putting jobs on the
+queue, so if you want to be able to check in on the administrative details (or
+add some of your own) this is a great place to do it instead of muddying up the
+meat of your message.
+
+### Workers
+
+Processing items on a Timberline queue is handled by Workers. A Worker can be
+created as simply as providing a block that executes on job items, or you can
+write your own Workers that perform special functionality.
+
 ### Retries
 
 Sometimes jobs just fail because of something that was outside of your control.
@@ -41,16 +62,6 @@ explicitly marked as bad jobs, or when they've been retried the maximum number
 of times. You can then check the jobs out and resubmit them to their original
 queue after you fix the issue.
 
-### The Envelope
-
-Sounds SOAPy, I know. The envelope is a simple object that wraps the data you
-want to put on the queue - it's responsible for tracking things like the job ID,
-the queue it was put on, how many times it's been retried, etc., etc. It's also
-accessible to both the queue processor and whatever is putting jobs on the
-queue, so if you want to be able to check in on the administrative details (or
-add some of your own) this is a great place to do it instead of muddying up the
-meat of your message.
-
 ## Usage
 
 Timberline is designed to be as easy to work with as possible, and operates almost
@@ -170,11 +181,6 @@ Still to be done:
 
 - **Monitor** - A simple Sinatra interface for monitoring the statuses of queues and
   observing/resubmitting errored-out jobs.
-- **Documentation** - need to get Tomdoc added so that the API is more completely
-  documented. For the time being, though, there are some fairly comprehensive
-  test suites.
-- **Refactor** - the singleton model made sense at some point for Timberline but now it's
-  cumbersome. Need to rewrite some of the basic stuff to be more OO-appropriate.
 - **Forking** - Timberline should support forking in its watcher model so that jobs can
   be run in parallel and independently of each other.
 

data/lib/timberline/anonymous_worker.rb

@@ -0,0 +1,48 @@
+class Timberline
+  # The AnonymousWorker is exactly what it says on the tin - a way to process a queue
+  # without defining a new class, by instead just passing in a block that will be
+  # executed for each item on the queue as it's popped.
+  #
+  class AnonymousWorker < Worker
+
+    # Creates a new AnonymousWorker.
+    # The block's binding will be updated to give it access to retry_item
+    # and error_item so that the block can easily control the processing
+    # flow for queued items.
+    #
+    # @param [String] queue_name the name of the queue to watch 
+    # @param [Block] block the block to run against each item that gets popped 
+    #   off the queue.
+    #
+    # @example Creating a simple AnonymousWorker
+    #   AnonymousWorker.new "test_queue" { |item| puts item.contents }
+    #
+    # @return [AnonymousWorker]
+    #
+    def initialize(queue_name, &block)
+      super(queue_name)
+      @block = block
+      fix_block_binding
+    end
+
+    # @see Timberline::Worker#watch
+    #
+    def process_item(item)
+      @block.call(item, self)
+    end
+
+  private
+    def fix_block_binding
+      binding = @block.binding
+      binding.eval <<-HERE
+        def retry_item(item)
+          Worker.retry_item(item)
+        end
+
+        def error_item(item)
+          Worker.error_item(item)
+        end
+      HERE
+    end
+  end
+end

data/lib/timberline/config.rb

@@ -1,8 +1,34 @@
 class Timberline
+  # Object that manages Timberline configuration. Responsible for Redis configs
+  # as well as Timberline-specific configuration values, like how many times an
+  # item should be retried in a queue.
+  #
+  # @attr [Integer] database part of the redis configuration - index of the 
+  #   redis database to use
+  # @attr [String] host part of the redis configuration - the hostname of the 
+  #   redis server
+  # @attr [Integer] port part of the redis configuration - the port of the
+  #   redis server
+  # @attr [Integer] timeout part of the redis configuration - the timeout for
+  #   the redis server
+  # @attr [String] password part of the redis configuration - the password for
+  #   the redis server
+  # @attr [Logger] logger part of the redis configuration - the logger to use
+  #   for the redis connection
+  # @attr [String] namespace the redis namespace for the keys that timberline
+  #   will create and manage
+  # @attr [Integer] max_retries the number of times that an item on the queue
+  #   should be allowed to retry itself before it is placed on the error queue
+  # @attr [Integer] stat_timeout the number of minutes that stats will stay live
+  #   in redis before they are expired
+  #
   class Config
-    attr_accessor :database, :host, :port, :timeout, :password, :logger, :namespace, :max_retries, :stat_timeout
+    attr_accessor :database, :host, :port, :timeout, :password, 
+                  :logger, :namespace, :max_retries, :stat_timeout
 
-    def initialize
+    # Attemps to load configuration from TIMBERLINE_YAML, if it exists.
+    # Otherwise creates a default Config object.
+    def initialize 
       if defined? TIMBERLINE_YAML
         if File.exists?(TIMBERLINE_YAML)
           yaml = YAML.load_file(TIMBERLINE_YAML)
@@ -13,18 +39,22 @@ class Timberline
       end
     end
 
+    # @return [String] the configured redis namespace, with a default of 'timberline'
     def namespace
       @namespace ||= 'timberline'
     end
 
+    # @return [Integer] the configured maximum number of retries, with a default of 5
     def max_retries
       @max_retries ||= 5
     end
 
+    # @return [Integer] the configured lifetime of stats (in minutes), with a default of 60 
     def stat_timeout
       @stat_timeout ||= 60
     end
 
+    # @return [Hash] a Redis-ready hash for use in instantiating a new redis object.
     def redis_config
       config = {}
 

data/lib/timberline/envelope.rb

@@ -1,6 +1,22 @@
 class Timberline
+  # An Envelope in Timberline is what gets passed along on the queue.
+  # The message itself - the part that the workers should intend to operate on - 
+  # is stored in the `contents` field of the Envelope. Any other data on the
+  # envelope is considered metadata. Metadata is mostly used by Timberline itself,
+  # but is also exposed to the end user in case they have any need for it.
+  #
+  # @attr [#to_json] contents the contents of the envelope; the message to be 
+  #   passed on the queue
+  # @attr [Hash] metadata the metadata information associated with the envelope
+  #
   class Envelope
 
+    # Given a JSON string representing an envelope, build an Envelope object
+    # with the appropriate data.
+    #
+    # @param [String] json_string the JSON string to parse
+    # @return [Envelope]
+    #
     def self.from_json(json_string)
       envelope = Envelope.new
       envelope.instance_variable_set("@metadata", JSON.parse(json_string))
@@ -11,16 +27,30 @@ class Timberline
     attr_accessor :contents
     attr_reader   :metadata
 
+    # Instantiates an Envelope with no metadata and nil contents.
+    # @return [Envelope]
+    #
     def initialize
       @metadata = {}
     end
 
+    # Builds a JSON string version of the envelope.
+    #
+    # @raise [MissingContentException] if the envelope is empty (has no contents)
+    # @return [String] a JSON representation of the envelope
+    #
     def to_s
       raise MissingContentException if contents.nil? || contents.empty?
 
       JSON.unparse(build_envelope_hash)
     end
 
+    # Passes any missing methods on to the metadata hash to provide better access.
+    # @example Easily read from metadata
+    #   some_envelope.origin_queue # returns metadata["origin_queue"]
+    # @example Easily write to metadata
+    #   some_envelope.origin_queue = "test_queue" # sets metadata["origin_queue"] to "test_queue"
+    #
     def method_missing(method_name, *args)
       method_name = method_name.to_s
       if method_name[-1] == "="
@@ -41,7 +71,4 @@ class Timberline
     end
 
   end
-
-  class MissingContentException < Exception
-  end
 end

data/lib/timberline/exceptions.rb

@@ -0,0 +1,17 @@
+class Timberline
+
+  # Used to indicate that an Envelope does not yet have contents, but is being
+  # operated on as though it should.
+  #
+  class MissingContentException < Exception; end
+
+  # Raised to indicate that the item currently being processed was retried.
+  # Prevents Workers from treating the item as a success.
+  #
+  class ItemRetried < Exception; end
+
+  # Raised to indicate that the item currently being processed experienced a
+  # fatal error. Prevents Workers from treating the item as a success.
+  #
+  class ItemErrored < Exception; end
+end

data/lib/timberline/queue.rb

@@ -1,7 +1,29 @@
 class Timberline
+  # Queue is the heart and soul of Timberline, which makes sense
+  # considering that it's a queueing library. This object represents
+  # a queue in redis (really just a list of strings) and is responsible
+  # for reading to the queue, writing from the queue, maintaining queue
+  # statistics, and managing other queue actions (like pausing and deleting).
+  #
+  # @attr_reader [String] queue_name the name of this queue
+  # @attr_reader [Integer] read_timeout how long this queue should wait, in
+  #   seconds, before determining that there isn't anything to read off of the
+  #   queue.
+  #
   class Queue
     attr_reader :queue_name, :read_timeout
 
+    # Build a new Queue object.
+    #
+    # @param [String] queue_name the redis queue that this object should represent
+    # @param [Hash] opts the options for creating this queue
+    # @option opts [Integer] :read_timeout the read_timeout for this queue.
+    #   defaults to 0 (which effectively disables the timeout).
+    # @option opts [boolean] :hidden whether this queue should be hidden from
+    #   Timberline's #all_queues list. Defaults to false.
+    # @raise [ArgumentError] if queue_name is not provided
+    # @return [Queue]
+    #
     def initialize(queue_name, opts = {})
       read_timeout = opts.fetch(:read_timeout, 0)
       hidden = opts.fetch(:hidden, false)
@@ -16,6 +38,11 @@ class Timberline
       end
     end
 
+    # Delete this queue, removing it from redis and all other references to it
+    # from Timberline.
+    #
+    # @return as Redis#srem
+    #
     def delete
       @redis.del @queue_name
       @redis.keys("#{@queue_name}:*").each do |key|
@@ -24,10 +51,21 @@ class Timberline
       @redis.srem "timberline_queue_names", @queue_name
     end
 
+    # The current number of items on the queue waiting to be processed.
+    #
+    # @return [Integer]
+    #
     def length
       @redis.llen @queue_name
     end
 
+    # Uses a blocking read from redis to pull the next item off the queue. If
+    # the queue is paused, this method will block until the queue is unpaused,
+    # at which point it will move on to the blocking read.
+    #
+    # @return [Timberline::Envelope] the Envelope representation of the item
+    #   that was pulled off the queue, or nil if the read timed out.
+    #
     def pop
       block_while_paused
 
@@ -40,6 +78,14 @@ class Timberline
       end
     end
 
+    # Pushes the specified data onto the queue.
+    #
+    # @param [#to_json, Timberline::Envelope] contents either contents that can
+    #   be converted to JSON and stuffed in an Envelope, or an Envelope itself
+    #   that needs to be put on the queue.  
+    # @param [Hash] metadata metadata that will be attached to the envelope for
+    # contents.
+    #
     def push(contents, metadata = {})
       case contents
       when Envelope
@@ -49,34 +95,70 @@ class Timberline
       end
     end
 
+    # Puts this queue into paused mode.
+    # @see Timberline::Queue#pop
+    #
     def pause
       @redis[attr("paused")] = "true"
     end
 
+    # Takes this queue back out of paused mode.
+    # @see Timberline::Queue#pop
+    #
     def unpause
       @redis[attr("paused")] = "false"
     end
 
+    # Indicates whether or not this queue is currently in paused mode.
+    # @return [boolean]
+    #
     def paused?
       @redis[attr("paused")] == "true"
     end
 
+    # Given a key, create a string namespaced to this queue name.
+    # This method is used to keep redis keys tidy.
+    #
+    # @return [String]
+    #
     def attr(key)
       "#{@queue_name}:#{key}"
     end
 
+    # The number of items that have encountered fatal errors on the queue
+    # during the last [stat_timeout] minutes.
+    #
+    # @return [Integer]
+    #
     def number_errors
       Timberline.redis.xcard attr("error_stats")
     end
 
+    # The number of items that have been retried on the queue
+    # during the last [stat_timeout] minutes.
+    #
+    # @return [Integer]
+    #
     def number_retries
       Timberline.redis.xcard attr("retry_stats")
     end
 
+    # The number of items that were processed successfully for this queue
+    # during the last [stat_timeout] minutes.
+    #
+    # @return [Integer]
+    #
     def number_successes
       Timberline.redis.xcard attr("success_stats")
     end
 
+    # Given all of the successful jobs that were executed in the last
+    # [stat_timeout] minutes, determine how long on average those jobs
+    # took to execute.
+    #
+    # @return [Float] the average execution time for successful jobs in the last
+    #   [stat_timeout] minutes.
+    #
     def average_execution_time
       successes = Timberline.redis.xmembers(attr("success_stats")).map { |item| Envelope.from_json(item)}
       times = successes.map do |item|
@@ -96,6 +178,15 @@ class Timberline
       end
     end
 
+    # Given an item that needs to be retried, increment the retry count,
+    # add any appropriate metadata about the retry, and push it back onto
+    # the queue. If the item has already been retried the maximum number of
+    # times, pass it on to error_item instead.
+    #
+    # @see Timberline::Queue#error_item
+    # 
+    # @param [Envelope] item an item that needs to be retried
+    #
     def retry_item(item)
       if (item.retries < Timberline.max_retries)
         item.retries += 1
@@ -107,26 +198,50 @@ class Timberline
       end
     end
 
+    # Given an item that errored out in processing, add any appropriate metadata
+    # about the error, track it as a statistic, and push it onto the error queue.
+    #
+    # @param [Envelope] item an item that has fatally errored
+    #
     def error_item(item)
       item.fatal_error_at = Time.now.to_f
       add_error_stat(item)
       self.error_queue.push(item)
     end
 
+    # Stores an item from the queue that was retried so we can keep track
+    # of things like how many retries have been attempted on this queue, etc.
+    #
+    # @param [Envelope] item an item that fatally errored on this queue
+    #
     def add_retry_stat(item)
       add_stat_for_key(attr("retry_stats"), item)
     end
 
+    # Stores an item from the queue that fatally errored so we can keep track
+    # of things like how many errors have occurred on this queue, etc.
+    #
+    # @param [Envelope] item an item that fatally errored on this queue
+    #
     def add_error_stat(item)
       add_stat_for_key(attr("error_stats"), item)
     end
 
+    # Stores a successfully processed queue item as a statistic so we can keep
+    # track of things like average execution time, number of successes, etc.
+    #
+    # @param [Envelope] item an item that was processed successfully for this
+    #   queue
+    #
     def add_success_stat(item)
       add_stat_for_key(attr("success_stats"), item)
     rescue Exception => e
       $stderr.puts "Success Stat Error: #{e.inspect}, Item: #{item.inspect}"
     end
 
+    # @return [Timberline::Queue] a (hidden) Queue object where this queue's
+    #  errors are pushed.
+    #
     def error_queue
       @error_queue ||= Timberline.queue(attr("errors"), hidden: true)
     end

data/lib/timberline/version.rb

@@ -1,3 +1,4 @@
 class Timberline
-  VERSION = "0.6.0"
+  # The current canonical version for Timberline.
+  VERSION = "0.7.0"
 end

data/lib/timberline/worker.rb

@@ -0,0 +1,74 @@
+class Timberline
+  # Worker is the base class for Timberline Workers. It defines the basics for
+  # processing items off of a queue; the idea is that creating your own worker
+  # is as easy as extending Worker and implementing #process_item. You can also
+  # override #keep_watching? and #initialize to provide your own custom behavior
+  # easily, although this is not necessary.
+  #
+  class Worker
+    # Creates a new worker that will watch a specific queue.
+    # @param [String] queue_name the name of the queue to watch.
+    #
+    def initialize(queue_name)
+      @queue = Queue.new(queue_name)
+    end
+
+    # Run the watch loop for this worker. As long as #keep_watching?
+    # returns true, this will pop items off the queue and process them
+    # with #process_item. This method is also responsible for managing
+    # some extra timberline metadata (tracking when processing starts and
+    # stops, for example) and shouldn't typically be overridden when you
+    # define your own worker.
+    #
+    def watch
+      while(keep_watching?)
+        item = @queue.pop
+        item.started_processing_at = Time.now.to_f
+
+        begin
+          process_item(item)
+        rescue ItemRetried, ItemErrored
+          next
+        end
+
+        item.finished_processing_at = Time.now.to_f
+        @queue.add_success_stat(item)
+      end
+    end
+
+    # Given an item off of the queue, process it appropriately.
+    # Not implemented in Worker, as Worker is just a base class.
+    # 
+    def process_item(item)
+      raise NotImplementedError
+    end
+
+    # Determine whether or not the worker loop in #watch should continue
+    # executing. By default this is always true.
+    #
+    # @return [boolean]
+    def keep_watching?
+      true
+    end
+
+    # Given an item this worker is processing, have the queue mark it
+    # as fatally errored and raise an ItemErrored exception so that the
+    # watch loop can process it correctly
+    #
+    # @raise [Timberline::ItemErrored]
+    def error_item(item)
+      @queue.error_item(item)
+      raise Timberline::ItemErrored
+    end
+
+    # Given an item this worker is processing, have the queue mark it
+    # attempt to retry it and raise an ItemRetried exception so that the
+    # watch loop can process it correctly
+    #
+    # @raise [Timberline::ItemRetried]
+    def retry_item(item)
+      @queue.retry_item(item)
+      raise Timberline::ItemRetried
+    end
+  end
+end

data/lib/timberline.rb

@@ -7,16 +7,35 @@ require 'redis-namespace'
 require 'redis-expiring-set/monkeypatch'
 
 require_relative "timberline/version"
+require_relative "timberline/exceptions"
 require_relative "timberline/config"
 require_relative "timberline/queue"
 require_relative "timberline/envelope"
 
+require_relative "timberline/worker"
+require_relative "timberline/anonymous_worker"
+
+# The Timberline class serves as a base namespace for Timberline libraries, but
+# also provides some convenience methods for accessing queues and quickly and
+# easily processing items.
+#
 class Timberline
   class << self
     attr_reader :config
     attr_accessor :watch_proc
   end
 
+  # Update the redis server that Timberline uses for its connections.
+  #
+  # If Timberline has not already been configured, this method will initialize
+  # a new Timberline::Config first.
+  #
+  # @param [Redis, Redis::Namespace, nil] server if Redis, wraps it in a namespace.
+  #   if Redis::Namespace, uses that namespace directly. If nil, clears out any reference
+  #   to the existing redis server.
+  #
+  # @raise [StandardError] if server is not an instance of Redis, Redis::Namespace, or nil.
+  #
   def self.redis=(server)
     initialize_if_necessary
     if server.is_a? Redis
@@ -30,6 +49,15 @@ class Timberline
     end
   end
 
+  # Obtain a reference to the redis connection that Timberline is using.
+  #
+  # If Timberline has not already been configured, this method will initialize
+  # a new Timberline::Config first.
+  #
+  # If a Redis connection has not yet been established, this method will establish one.
+  #
+  # @return [Redis::Namespace]
+  # 
   def self.redis
     initialize_if_necessary
     if @redis.nil?
@@ -39,107 +67,93 @@ class Timberline
     @redis
   end
 
+  # @return [Array<Timberline::Queue>] a list of all non-hidden queues for this
+  #   instance of Timberline
   def self.all_queues
     Timberline.redis.smembers("timberline_queue_names").map { |name| queue(name) }
   end
 
+  # Convenience method to create a new Queue object
+  # @see Timberline::Queue#initialize
   def self.queue(queue_name, opts = {})
     Queue.new(queue_name, opts)
   end
 
+  # Convenience method to push an item onto a queue
+  # @see Timberline::Queue#push
   def self.push(queue_name, data, metadata={})
     queue(queue_name).push(data, metadata)
   end
 
+  # Convenience method to retry a queue item
+  # @see Timberline::Queue#retry_item
   def self.retry_item(item)
     origin_queue = queue(item.origin_queue)
     origin_queue.retry_item(item)
   end
 
+  # Convenience method to error out a queue item
+  # @see Timberline::Queue#error_item
   def self.error_item(item)
     origin_queue = queue(item.origin_queue)
     origin_queue.error_item(item)
   end
 
+  # Convenience method to pause a Queue by name.
+  # @see Timberline::Queue#pause
   def self.pause(queue_name)
     queue(queue_name).pause
   end
 
+  # Convenience method to unpause a Queue by name.
+  # @see Timberline::Queue#unpause
   def self.unpause(queue_name)
     queue(queue_name).unpause
   end
 
+  # Method for providing custom configuration by yielding the config object.
+  # Lazy-loads the Timberline configuration.
+  # @param [Block] block a block that accepts and manipulates a Timberline::Config
+  #
   def self.configure(&block)
     initialize_if_necessary
     yield @config
   end
 
+  # Lazy-loads the Timberline configuration.
+  # @return [Integer] the maximum number of retries
   def self.max_retries
     initialize_if_necessary
     @config.max_retries
   end
 
+  # Lazy-loads the Timberline configuration.
+  # @return [Integer] the stat_timeout expressed in minutes
   def self.stat_timeout
     initialize_if_necessary
     @config.stat_timeout
   end
 
+  # Lazy-loads the Timberline configuration.
+  # @return [Integer] the stat_timeout expressed in seconds
   def self.stat_timeout_seconds
     initialize_if_necessary
     @config.stat_timeout * 60
   end
 
+  # Create and start a new AnonymousWorker with the given
+  # queue_name and block. Convenience method.
+  #
+  # @param [String] queue_name the name of the queue to watch.
+  # @param [Block] block the block to execute for each queue item
+  # @see Timberline::AnonymousWorker#watch
+  #
   def self.watch(queue_name, &block)
-    queue = queue(queue_name)
-    while(self.watch?)
-      item = queue.pop
-      fix_binding(block)
-      item.started_processing_at = Time.now.to_f
-
-      begin
-        block.call(item, self)
-      rescue ItemRetried
-        queue.add_retry_stat(item)
-      rescue ItemErrored
-        queue.add_error_stat(item)
-      else
-        item.finished_processing_at = Time.now.to_f
-        queue.add_success_stat(item)
-      end
-    end
+    Timberline::AnonymousWorker.new(queue_name, &block).watch
   end
 
-  private
+private
   def self.initialize_if_necessary
     @config ||= Config.new
   end
-
-  # Hacky-hacky. I like the idea of calling retry_item(item) and
-  # error_item(item)
-  # directly from the watch block, but this seems ugly. There may be a better
-  # way to do this.
-  def self.fix_binding(block)
-    binding = block.binding
-    binding.eval <<-HERE
-      def retry_item(item)
-        Timberline.retry_item(item)
-        raise Timberline::ItemRetried
-      end
-
-      def error_item(item)
-        Timberline.error_item(item)
-        raise Timberline::ItemErrored
-      end
-    HERE
-  end
-
-  def self.watch?
-    watch_proc.nil? ? true : watch_proc.call
-  end
-
-  class ItemRetried < Exception
-  end
-
-  class ItemErrored < Exception
-  end
 end

data/spec/lib/timberline_spec.rb

@@ -316,45 +316,4 @@ describe Timberline do
       end
     end
   end
-
-  describe ".watch" do
-    let(:queue) { Timberline.queue("test_queue") }
-    let(:queue_name) { queue.queue_name }
-    let(:success_item) { Timberline::Envelope.from_json(Timberline.redis.xmembers(queue.attr("success_stats")).first) }
-    let(:retried_item) { Timberline::Envelope.from_json(Timberline.redis.xmembers(queue.attr("retry_stats")).first) }
-    let(:errored_item) { Timberline::Envelope.from_json(Timberline.redis.xmembers(queue.attr("error_stats")).first) }
-
-    before do
-      # Make sure that the watch doesn't run forever.
-      Timberline.watch_proc = lambda { queue.length > 0 }
-      Timberline.push(queue_name, "Hey There!")
-    end
-
-    context "If the item can be processed successfully" do
-      it "logs the success of the item" do
-        expect_any_instance_of(Timberline::Queue).to receive(:add_success_stat)
-        Timberline.watch(queue_name) do |item|
-          # don't do anything
-        end
-      end
-    end
-
-    context "If the item is retried" do
-      it "logs that the item was retried" do
-        expect_any_instance_of(Timberline::Queue).to receive(:add_retry_stat)
-        Timberline.watch(queue_name) do |item|
-          raise Timberline::ItemRetried
-        end
-      end
-    end
-
-    context "If the item can be processed successfully" do
-      it "logs that the item was retried" do
-        expect_any_instance_of(Timberline::Queue).to receive(:add_error_stat)
-        Timberline.watch(queue_name) do |item|
-          raise Timberline::ItemErrored
-        end
-      end
-    end
-  end
 end

data/timberline.gemspec

@@ -24,6 +24,7 @@ Gem::Specification.new do |s|
   s.add_runtime_dependency "trollop"
   s.add_runtime_dependency "daemons"
 
+  s.add_development_dependency "yard"
   s.add_development_dependency "rake"
   s.add_development_dependency "rspec", '~> 3.0.0.rc1'
   s.add_development_dependency "pry"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: timberline
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Tommy Morgan
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-06-06 00:00:00.000000000 Z
+date: 2014-06-07 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: redis
@@ -80,6 +80,20 @@ dependencies:
     - - '>='
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -137,16 +151,23 @@ files:
 - .rspec
 - .ruby-gemset
 - .travis.yml
+- .yardoc/checksums
+- .yardoc/object_types
+- .yardoc/objects/root.dat
+- .yardoc/proxy_types
 - CHANGELOG
 - Gemfile
 - README.markdown
 - Rakefile
 - bin/timberline
 - lib/timberline.rb
+- lib/timberline/anonymous_worker.rb
 - lib/timberline/config.rb
 - lib/timberline/envelope.rb
+- lib/timberline/exceptions.rb
 - lib/timberline/queue.rb
 - lib/timberline/version.rb
+- lib/timberline/worker.rb
 - spec/config/test_config.yaml
 - spec/lib/timberline/config_spec.rb
 - spec/lib/timberline/envelope_spec.rb
@@ -181,3 +202,4 @@ specification_version: 4
 summary: Timberline is a simple and extensible queuing system built in Ruby and backed
   by Redis.
 test_files: []
+has_rdoc: