redis-scheduler 0.1

data/COPYING

@@ -0,0 +1,14 @@
+Whistlepig is copyright (c) 2011-2012 William Morgan <wmorgan@masanjin.net>
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+    * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+    * Neither the name of Whistlepig nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+khash.h copyright (c) 2008, by Attractive Chaos <attractivechaos@aol.co.uk>
+See khash.h for terms.

data/README

@@ -0,0 +1,38 @@
+This is a basic chronological scheduler for Redis. It allows you to schedule
+items to be processed at arbitrary points in time (via #schedule!), and then
+to easily retrieve only those items that are due to be processed (via #each!).
+Items are represented as strings.
+
+It does everything you'd want from a production scheduler:
+* You can schedule arbitrary items at arbitrary times.
+* It supports multiple simultaneous readers and writers.
+* An exception causes the in-process item to be rescheduled at the original time.
+* A crash leaves the item in an error queue, from which it can later be recovered.
+* You can iterate over ready items in either blocking or non-blocking mode.
+
+In non-blocking mode (the default), #each will iterate only over those work items
+whose scheduled time is less than or equal to the current time, and then stop.
+In blocking mode, #each will iterate over the same items, but will also block
+until items are available. In blocking mode, #each will never return.
+
+== Synopsis
+
+  r = Redis.new
+  s = RedisScheduler.new r, blocking: true
+  startt = Time.now
+  s.schedule! "a", startt + 10
+  s.schedule! "b", startt + 15
+  s.schedule! "c", startt + 20
+  s.each { |item, at| puts "#{Time.now - startt}: #{item}" }
+
+  prints:
+
+  10.03481788: a
+  15.05255288: b
+  20.06058172: c
+  ... waits forever ...
+
+== Bug reports
+
+Please file bugs here: https://github.com/wmorgan/redis-scheduler/issues
+Please send comments to: wmorgan-redis-scheduler-readme@masanjin.net.

data/lib/redis-scheduler.rb

@@ -0,0 +1,90 @@
+class RedisScheduler
+  include Enumerable
+
+  POLL_DELAY = 1.0 # seconds
+  CAS_DELAY  = 0.5 # seconds
+
+  ## options:
+  ##   namespace: prefix for redis data, e.g. "scheduler/"
+  ##   blocking: whether #each should block or return immediately if
+  ##     there are items to be processed immediately.
+  ##
+  ## Note that nonblocking mode may still actually block as part of the
+  ## check-and-set semantics, i.e. block during contention from multiple
+  ## clients. "Nonblocking" mode just refers to whether the scheduler
+  ## should wait until events in the schedule are ready, or only return
+  ## those items that are ready currently.
+  def initialize redis, opts={}
+    @redis = redis
+    @namespace = opts[:namespace]
+    @blocking = opts[:blocking]
+
+    @queue = [@namespace, "q"].join
+    @error_queue = [@namespace, "errorq"].join
+    @counter = [@namespace, "counter"].join
+  end
+
+  ## schedule an item at a specific time. item will be converted to a
+  ## string.
+  def schedule! item, time
+    id = @redis.incr @counter
+    @redis.zadd @queue, time.to_f, "#{id}:#{item}"
+  end
+
+  def reset!
+    [@queue, @error_queue, @counter].each { |k| @redis.del k }
+  end
+
+  ## yields items along with their scheduled times. only returns items
+  ## on or after their scheduled times. items returned as strings. if
+  ## @blocking is false, will stop once there are no more items that can
+  ## be processed immediately; if it's true, will wait until items
+  ## become available (and never terminate).
+  def each
+    while(x = get)
+      item, at = x
+      begin
+        yield item, at
+      rescue Exception # back in the hole!
+        schedule! item, at
+        raise
+      ensure
+        cleanup! item
+      end
+    end
+  end
+
+private
+
+  def get; @blocking ? blocking_get : nonblocking_get end
+
+  def blocking_get
+    sleep POLL_DELAY until(x = nonblocking_get)
+    x
+  end
+
+  class InvalidEntryException < StandardError; end
+  def nonblocking_get
+    catch :cas_retry do
+      @redis.watch @queue
+      item, at = @redis.zrangebyscore @queue, 0, Time.now.to_f,
+        :withscores => true, :limit => [0, 1]
+      if item
+        @redis.multi do # try and grab it
+          @redis.zrem @queue, item
+          @redis.lpush @error_queue, item
+        end or begin
+          sleep CAS_DELAY
+          throw :cas_retry
+        end
+        item =~ /^\d+:(\S+)$/ or raise InvalidEntryException, item
+        item = $1
+        [item, Time.at(at.to_f)]
+      end
+    end
+  end
+
+  def cleanup! item
+    @redis.lrem @error_queue, 1, item
+  end
+end

metadata

@@ -0,0 +1,51 @@
+--- !ruby/object:Gem::Specification
+name: redis-scheduler
+version: !ruby/object:Gem::Version
+  version: '0.1'
+  prerelease: 
+platform: ruby
+authors:
+- William Morgan
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2012-04-18 00:00:00.000000000 -07:00
+default_executable: 
+dependencies: []
+description: A basic chronological scheduler for Redis. Add work items to be processed
+  at specific times in the future, and easily retrieve all items that are ready for
+  processing.
+email: wmorgan-redis-scheduler@masanjin.net
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README
+- COPYING
+- lib/redis-scheduler.rb
+has_rdoc: true
+homepage: http://gitub.com/wmorgan/redis-scheduler
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.6.0
+signing_key: 
+specification_version: 3
+summary: A basic chronological scheduler for Redis.
+test_files: []