redis-scheduler 0.2 → 0.3

data/README

@@ -1,13 +1,14 @@
 This is a basic chronological scheduler for Redis. It allows you to schedule
 items to be processed at arbitrary points in time (via #schedule!), and then
-to easily retrieve only those items that are due to be processed (via #each!).
+to easily retrieve only those items that are due to be processed (via #each).
+
 Items are represented as strings.
 
 It does everything you'd want from a production scheduler:
-* You can schedule arbitrary items at arbitrary times.
+* You can schedule items at arbitrary times.
 * It supports multiple simultaneous readers and writers.
 * An exception causes the in-process item to be rescheduled at the original time.
-* A crash leaves the item in an error queue, from which it can later be recovered.
+* A crash leaves the item in a separate error queue, from which it can later be recovered.
 * You can iterate over ready items in either blocking or non-blocking mode.
 
 In non-blocking mode (the default), #each will iterate only over those work items
@@ -32,6 +33,10 @@ until items are available. In blocking mode, #each will never return.
   20.06058172: c
   ... waits forever ...
 
+== Implementation
+
+See http://masanjin.net/blog/using-redis-for-scheduling for some details.
+
 == Bug reports
 
 Please file bugs here: https://github.com/wmorgan/redis-scheduler/issues

data/lib/redis-scheduler.rb

@@ -5,9 +5,8 @@ class RedisScheduler
   CAS_DELAY  = 0.5 # seconds
 
   ## options:
-  ##   namespace: prefix for redis data, e.g. "scheduler/"
-  ##   blocking: whether #each should block or return immediately if
-  ##     there are items to be processed immediately.
+  ## * +namespace+: prefix for redis data, e.g. "scheduler/"
+  ## * +blocking+: whether #each should block or return immediately if there are items to be processed immediately.
   ##
   ## Note that nonblocking mode may still actually block as part of the
   ## check-and-set semantics, i.e. block during contention from multiple
@@ -35,6 +34,9 @@ class RedisScheduler
     [@queue, @error_queue, @counter].each { |k| @redis.del k }
   end
 
+  def size; @redis.zcard @queue end
+  def error_queue_size; @redis.llen @error_queue end
+
   ## yields items along with their scheduled times. only returns items
   ## on or after their scheduled times. items returned as strings. if
   ## @blocking is false, will stop once there are no more items that can
@@ -42,18 +44,27 @@ class RedisScheduler
   ## become available (and never terminate).
   def each
     while(x = get)
-      item, at = x
+      item, erritem, at = x
       begin
         yield item, at
       rescue Exception # back in the hole!
         schedule! item, at
         raise
       ensure
-        cleanup! item
+        cleanup! erritem
       end
     end
   end
 
+  ## returns an Enumerable of [item, schedule time] pairs, which can be used to
+  ## easily iterate over all the items in the queue, in order of earliest- to
+  ## latest-scheduled. note that this view is not coordinated with write
+  ## operations, and may be inconsistent (e.g. return duplicates, miss items,
+  ## etc).
+  ##
+  ## for these reasons, this operation is mainly useful for debugging purposes.
+  def items; ItemEnumerator.new(@redis, @queue) end
+
 private
 
   def get; @blocking ? blocking_get : nonblocking_get end
@@ -78,8 +89,8 @@ private
           throw :cas_retry
         end
         item =~ /^\d+:(\S+)$/ or raise InvalidEntryException, item
-        item = $1
-        [item, Time.at(at.to_f)]
+        original = $1
+        [original, item, Time.at(at.to_f)]
       end
     end
   end
@@ -87,4 +98,31 @@ private
   def cleanup! item
     @redis.lrem @error_queue, 1, item
   end
+
+  ## enumerable for just iterating over everything in the queue
+  class ItemEnumerator
+    include Enumerable
+    def initialize redis, q
+      @redis = redis
+      @q = q
+    end
+
+    BLOCK_SIZE = 10
+    def each
+      start = 0
+      while start < size
+        elements = @redis.zrange @q, start, start + BLOCK_SIZE,
+          :withscores => true
+        elements.each_slice(2) do |item, at| # isgh
+          item =~ /^\d+:(\S+)$/ or raise InvalidEntryException, item
+          item = $1
+          yield item, Time.at(at.to_f)
+        end
+        start += elements.size
+      end
+    end
+
+    def size; @redis.zcard @q end
+  end
+
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: redis-scheduler
 version: !ruby/object:Gem::Version
-  version: '0.2'
+  version: '0.3'
   prerelease: 
 platform: ruby
 authors:
@@ -9,8 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-04-18 00:00:00.000000000 -07:00
-default_executable: 
+date: 2012-04-29 00:00:00.000000000 Z
 dependencies: []
 description: A basic chronological scheduler for Redis. Add work items to be processed
   at specific times in the future, and easily retrieve all items that are ready for
@@ -23,7 +22,6 @@ files:
 - README
 - COPYING
 - lib/redis-scheduler.rb
-has_rdoc: true
 homepage: http://gitub.com/wmorgan/redis-scheduler
 licenses: []
 post_install_message: 
@@ -48,7 +46,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.6.0
+rubygems_version: 1.8.11
 signing_key: 
 specification_version: 3
 summary: A basic chronological scheduler for Redis.