thehenster-resque-scheduler 1.0.6

data/.gitignore

@@ -0,0 +1,3 @@
+*.gemspec
+pkg
+nbproject

data/HISTORY.md

@@ -0,0 +1,30 @@
+## 1.0.5 (2010-03-01)
+
+* Fixed support for overriding queue from schedule config.
+* Removed resque-web dependency on loading the job classes for "Queue Now",
+  provided "queue" is specified in the schedule.
+* The queue is now stored with the job and arguments in the delayed queue so
+  there is no longer a need for the scheduler to load job classes to introspect
+  the queue.
+
+## 1.0.4 (2010-02-26)
+
+* Added support for specifying the queue to put the job onto. This allows for 
+  you to have one job that can go onto multiple queues and be able to schedule
+  jobs without having to load the job classes.
+
+## 1.0.3 (2010-02-11)
+
+* Added support for scheduled jobs with empty crons. This is helpful to have
+  jobs that you don't want on a schedule, but do want to be able to queue by
+  clicking a button.
+
+## 1.0.2 (2010-02-?)
+
+* Change Delayed Job tab to display job details if only 1 job exists
+  for a given timestamp
+
+## 1.0.1 (2010-01-?)
+
+* Bugfix: delayed jobs close together resulted in a 5 second sleep
+

data/LICENSE

@@ -0,0 +1,21 @@
+Copyright (c) 2010 Ben VandenBos
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+

data/README.markdown

@@ -0,0 +1,142 @@
+resque-scheduler
+===============
+
+Resque-scheduler is an extension to [Resque](http://github.com/defunkt/resque)
+that adds support for queueing items in the future.
+
+Requires redis >=1.1.
+
+
+Job scheduling is supported in two different way:
+
+### Recurring (scheduled)
+
+Recurring (or scheduled) jobs are logically no different than a standard cron
+job.  They are jobs that run based on a fixed schedule which is set at startup.
+
+The schedule is a list of Resque worker classes with arguments and a
+schedule frequency (in crontab syntax).  The schedule is just a hash, but
+is most likely stored in a YAML like so:
+
+    queue_documents_for_indexing:
+      cron: "0 0 * * *"
+      class: QueueDocuments
+      args: 
+      description: "This job queues all content for indexing in solr"
+
+    clear_leaderboards_contributors:
+      cron: "30 6 * * 1"
+      class: ClearLeaderboards
+      args: contributors
+      description: "This job resets the weekly leaderboard for contributions"
+
+A queue option can also be specified. When job will go onto the specified queue
+if it is available (Even if @queue is specified in the job class). When the
+queue is given it is not necessary for the scheduler to load the class.
+
+    clear_leaderboards_moderator:
+      cron: "30 6 * * 1"
+      class: ClearLeaderboards
+      queue: scoring
+      args: moderators
+      description: "This job resets the weekly leaderboard for moderators"
+
+And then set the schedule wherever you configure Resque, like so:
+
+    require 'resque_scheduler'
+    Resque.schedule = YAML.load_file(File.join(File.dirname(__FILE__), '../resque_schedule.yml'))
+
+Keep in mind, scheduled jobs behave like crons: if your scheduler process (more
+on that later) is not running when a particular job is supposed to be queued,
+it will NOT be ran later when the scheduler process is started back up.  In that
+sense, you can sort of think of the scheduler process as crond.  Delayed jobs,
+however, are different.
+
+A big shout out to [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler)
+for handling the heavy lifting of the actual scheduling engine.
+
+### Delayed jobs
+
+Delayed jobs are one-off jobs that you want to be put into a queue at some point
+in the future.  The classic example is sending email:
+
+    Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :user_id => current_user.id)
+
+This will store the job for 5 days in the resque delayed queue at which time the
+scheduler process will pull it from the delayed queue and put it in the
+appropriate work queue for the given job and it will be processed as soon as
+a worker is available.
+
+NOTE: The job does not fire **exactly** at the time supplied.  Rather, once that
+time is in the past, the job moves from the delayed queue to the actual resque
+work queue and will be completed as workers as free to process it.
+
+Also supported is `Resque.enqueue_in` which takes an amount of time in seconds
+in which to queue the job.
+
+The delayed queue is stored in redis and is persisted in the same way the
+standard resque jobs are persisted (redis writing to disk). Delayed jobs differ
+from scheduled jobs in that if your scheduler process is down or workers are
+down when a particular job is supposed to be queue, they will simply "catch up"
+once they are started again.  Jobs are guaranteed to run (provided they make it
+into the delayed queue) after their given queue_at time has passed.
+
+One other thing to note is that insertion into the delayed queue is O(log(n))
+since the jobs are stored in a redis sorted set (zset).  I can't imagine this
+being an issue for someone since redis is stupidly fast even at log(n), but full
+disclosure is always best.
+
+
+Resque-web additions
+--------------------
+
+Resque-scheduler also adds to tabs to the resque-web UI.  One is for viewing
+(and manually queueing) the schedule and one is for viewing pending jobs in
+the delayed queue.
+
+The Schedule tab:
+
+![The Schedule Tab](http://img.skitch.com/20100111-km2f5gmtpbq23enpujbruj6mgk.png)
+
+The Delayed tab:
+
+![The Delayed Tab](http://img.skitch.com/20100111-ne4fcqtc5emkcuwc5qtais2kwx.jpg)
+
+
+Installation and the Scheduler process
+--------------------------------------
+
+To install:
+
+    gem install resque-scheduler
+
+You'll need to add this to your rakefile:
+
+    require 'resque_scheduler/tasks'
+    task "resque:setup" => :environment
+
+The scheduler process is just a rake task which is responsible for both queueing
+items from the schedule and polling the delayed queue for items ready to be
+pushed on to the work queues.  For obvious reasons, this process never exits.
+
+    $ rake resque:scheduler 
+
+Supported environment variables are `VERBOSE` and `MUTE`.  If either is set to
+any nonempty value, they will take effect.  `VERBOSE` simply dumps more output
+to stdout.  `MUTE` does the opposite and silences all output. `MUTE` supercedes
+`VERBOSE`.
+
+
+Plagurism alert
+---------------
+
+This was intended to be an extension to resque and so resulted in a lot of the
+code looking very similar to resque, particularly in resque-web and the views. I
+wanted it to be similar enough that someone familiar with resque could easily
+work on resque-scheduler.
+
+
+Contributing
+------------
+
+For bugs or suggestions, please just open an issue in github.

data/Rakefile

@@ -0,0 +1,54 @@
+load 'tasks/resque_scheduler.rake'
+
+$LOAD_PATH.unshift 'lib'
+
+task :default => :test
+
+desc "Run tests"
+task :test do
+  Dir['test/*_test.rb'].each do |f|
+    require f
+  end
+end
+
+
+desc "Build a gem"
+task :gem => [ :test, :gemspec, :build ]
+
+begin
+  begin
+    require 'jeweler'
+  rescue LoadError
+    puts "Jeweler not available. Install it with: "
+    puts "gem install jeweler"
+  end
+
+  require 'resque_scheduler/version'
+
+  Jeweler::Tasks.new do |gemspec|
+    gemspec.name = "thehenster-resque-scheduler"
+    gemspec.summary = "Light weight job scheduling on top of Resque"
+    gemspec.description = %{Light weight job scheduling on top of Resque.
+  Adds methods enqueue_at/enqueue_in to schedule jobs in the future.
+  Also supports queueing jobs on a fixed, cron-like schedule.}
+    gemspec.email = "bvandenbos@gmail.com"
+    gemspec.homepage = "http://github.com/bvandenbos/resque-scheduler"
+    gemspec.authors = ["Ben VandenBos"]
+    gemspec.version = ResqueScheduler::Version
+
+    gemspec.add_dependency "resque", ">= 1.5.0"
+    gemspec.add_dependency "rufus-scheduler"
+    gemspec.add_development_dependency "jeweler"
+    gemspec.add_development_dependency "mocha"
+  end
+end
+
+
+desc "Push a new version to Gemcutter"
+task :publish => [ :test, :gemspec, :build ] do
+  system "git tag v#{ResqueScheduler::Version}"
+  system "git push origin v#{ResqueScheduler::Version}"
+  system "git push origin master"
+  system "gem push pkg/resque-scheduler-#{ResqueScheduler::Version}.gem"
+  system "git clean -fd"
+end

data/lib/resque/scheduler.rb

@@ -0,0 +1,141 @@
+require 'rufus/scheduler'
+require 'thwait'
+
+module Resque
+
+  class Scheduler
+
+    extend Resque::Helpers
+
+    class << self
+
+      # If true, logs more stuff...
+      attr_accessor :verbose
+      
+      # If set, produces no output
+      attr_accessor :mute
+
+      # Schedule all jobs and continually look for delayed jobs (never returns)
+      def run
+
+        # trap signals
+        register_signal_handlers
+
+        # Load the schedule into rufus
+        load_schedule!
+
+        # Now start the scheduling part of the loop.
+        loop do
+          handle_delayed_items
+          poll_sleep
+        end
+
+        # never gets here.
+      end
+
+      # For all signals, set the shutdown flag and wait for current
+      # poll/enqueing to finish (should be almost istant).  In the
+      # case of sleeping, exit immediately.
+      def register_signal_handlers
+        trap("TERM") { shutdown }
+        trap("INT") { shutdown }
+        trap('QUIT') { shutdown } unless defined? JRUBY_VERSION
+      end
+
+      # Pulls the schedule from Resque.schedule and loads it into the
+      # rufus scheduler instance
+      def load_schedule!
+        log! "Schedule empty! Set Resque.schedule" if Resque.schedule.empty?
+
+        Resque.schedule.each do |name, config|
+          log! "Scheduling #{name} "
+          if !config['cron'].nil? && config['cron'].length > 0
+            rufus_scheduler.cron config['cron'] do
+              log! "queuing #{config['class']} (#{name})"
+              enqueue_from_config(config)
+            end
+          else
+            log! "no cron found for #{config['class']} (#{name}) - skipping"
+          end
+        end
+      end
+
+      # Handles queueing delayed items
+      def handle_delayed_items
+        item = nil
+        begin
+          if timestamp = Resque.next_delayed_timestamp
+            item = nil
+            begin
+              handle_shutdown do
+                if item = Resque.next_item_for_timestamp(timestamp)
+                  log "queuing #{item['class']} [delayed]"
+                  queue = item['queue'] || Resque.queue_from_class(constantize(item['class']))
+                  Job.create(queue, item['class'], *item['args'])
+                end
+              end
+            # continue processing until there are no more ready items in this timestamp
+            end while !item.nil?
+          end
+        # continue processing until there are no more ready timestamps
+        end while !timestamp.nil?
+      end
+
+      def handle_shutdown
+        exit if @shutdown
+        yield
+        exit if @shutdown
+      end
+
+      # Enqueues a job based on a config hash
+      def enqueue_from_config(config)
+        args = config['args'] || config[:args]
+        klass_name = config['class'] || config[:class]
+        params = args.nil? ? [] : Array(args)
+        queue = config['queue'] || config[:queue] || Resque.queue_from_class(constantize(klass_name))
+        if config[:just_once]
+          Resque::Job.destroy(queue, klass_name, *params)
+        end
+        Resque::Job.create(queue, klass_name, *params)
+      end
+
+      def rufus_scheduler
+        @rufus_scheduler ||= Rufus::Scheduler.start_new
+      end
+
+      # Stops old rufus scheduler and creates a new one.  Returns the new
+      # rufus scheduler
+      def clear_schedule!
+        rufus_scheduler.stop
+        @rufus_scheduler = nil
+        rufus_scheduler
+      end
+
+      # Sleeps and returns true
+      def poll_sleep
+        @sleeping = true
+        handle_shutdown { sleep 5 }
+        @sleeping = false
+        true
+      end
+
+      # Sets the shutdown flag, exits if sleeping
+      def shutdown
+        @shutdown = true
+        exit if @sleeping
+      end
+
+      def log!(msg)
+        puts "#{Time.now.strftime("%Y-%m-%d %H:%M:%S")} #{msg}" unless mute
+      end
+
+      def log(msg)
+        # add "verbose" logic later
+        log!(msg) if verbose
+      end
+
+    end
+
+  end
+
+end

data/lib/resque_scheduler.rb

@@ -0,0 +1,117 @@
+require 'rubygems'
+require 'resque'
+require 'resque/server'
+require 'resque_scheduler/version'
+require 'resque/scheduler'
+require 'resque_scheduler/server'
+
+module ResqueScheduler
+
+  #
+  # Accepts a new schedule configuration of the form:
+  #
+  #   {some_name => {"cron" => "5/* * * *",
+  #                  "class" => DoSomeWork,
+  #                  "args" => "work on this string",
+  #                  "description" => "this thing works it"s butter off"},
+  #    ...}
+  #
+  # :name can be anything and is used only to describe the scheduled job
+  # :cron can be any cron scheduling string :job can be any resque job class
+  # :class must be a resque worker class
+  # :args can be any yaml which will be converted to a ruby literal and passed
+  #   in a params. (optional)
+  # :description is just that, a description of the job (optional). If params is
+  #   an array, each element in the array is passed as a separate param,
+  #   otherwise params is passed in as the only parameter to perform.
+  def schedule=(schedule_hash)
+    redis[:resque_schedule_hash] = schedule_hash.to_yaml
+  end
+
+  # Returns the schedule hash
+  def schedule
+    YAML.load(redis[:resque_schedule_hash])
+  end
+
+  # This method is nearly identical to +enqueue+ only it also
+  # takes a timestamp which will be used to schedule the job
+  # for queueing.  Until timestamp is in the past, the job will
+  # sit in the schedule list.
+  def enqueue_at(timestamp, klass, *args)
+    delayed_push(timestamp, :class => klass.to_s, :args => args, :queue => queue_from_class(klass))
+  end
+
+  # Identical to enqueue_at but takes number_of_seconds_from_now
+  # instead of a timestamp.
+  def enqueue_in(number_of_seconds_from_now, klass, *args)
+    enqueue_at(Time.now + number_of_seconds_from_now, klass, *args)
+  end
+
+  # Used internally to stuff the item into the schedule sorted list.
+  # +timestamp+ can be either in seconds or a datetime object
+  # Insertion if O(log(n)).
+  # Returns true if it's the first job to be scheduled at that time, else false
+  def delayed_push(timestamp, item)
+    # First add this item to the list for this timestamp
+    redis.rpush("delayed:#{timestamp.to_i}", encode(item))
+
+    # Now, add this timestamp to the zsets.  The score and the value are
+    # the same since we'll be querying by timestamp, and we don't have
+    # anything else to store.
+    redis.zadd :delayed_queue_schedule, timestamp.to_i, timestamp.to_i
+  end
+
+  # Returns an array of timestamps based on start and count
+  def delayed_queue_peek(start, count)
+    redis.zrange(:delayed_queue_schedule, start, start+count).collect(&:to_i)
+  end
+
+  # Returns the size of the delayed queue schedule
+  def delayed_queue_schedule_size
+    redis.zcard :delayed_queue_schedule
+  end
+
+  # Returns the number of jobs for a given timestamp in the delayed queue schedule
+  def delayed_timestamp_size(timestamp)
+    redis.llen("delayed:#{timestamp.to_i}").to_i
+  end
+
+  # Returns an array of delayed items for the given timestamp
+  def delayed_timestamp_peek(timestamp, start, count)
+    if 1 == count
+      r = list_range "delayed:#{timestamp.to_i}", start, count
+      r.nil? ? [] : [r]
+    else
+      list_range "delayed:#{timestamp.to_i}", start, count
+    end
+  end
+
+  # Returns the next delayed queue timestamp
+  # (don't call directly)
+  def next_delayed_timestamp
+    timestamp = redis.zrangebyscore(:delayed_queue_schedule, '-inf', Time.now.to_i, 'limit', 0, 1).first
+    timestamp.to_i unless timestamp.nil?
+  end
+
+  # Returns the next item to be processed for a given timestamp, nil if
+  # done. (don't call directly)
+  # +timestamp+ can either be in seconds or a datetime
+  def next_item_for_timestamp(timestamp)
+    key = "delayed:#{timestamp.to_i}"
+
+    item = decode redis.lpop(key)
+
+    # If the list is empty, remove it.
+    if 0 == redis.llen(key).to_i
+      redis.del key
+      redis.zrem :delayed_queue_schedule, timestamp.to_i
+    end
+    item
+  end
+
+end
+
+Resque.extend ResqueScheduler
+Resque::Server.class_eval do
+  include ResqueScheduler::Server
+end

data/lib/resque_scheduler/server.rb

@@ -0,0 +1,51 @@
+
+# Extend Resque::Server to add tabs
+module ResqueScheduler
+  
+  module Server
+
+    def self.included(base)
+
+      base.class_eval do
+
+        helpers do
+          def format_time(t)
+            t.strftime("%Y-%m-%d %H:%M:%S")
+          end
+
+          def queue_from_class_name(class_name)
+            Resque.queue_from_class(Resque.constantize(class_name))
+          end
+        end
+
+        get "/schedule" do
+          # Is there a better way to specify alternate template locations with sinatra?
+          erb File.read(File.join(File.dirname(__FILE__), 'server/views/scheduler.erb'))
+        end
+
+        post "/schedule/requeue" do
+          config = Resque.schedule[params['job_name']]
+          Resque::Scheduler.enqueue_from_config(config)
+          redirect url("/overview")
+        end
+        
+        get "/delayed" do
+          # Is there a better way to specify alternate template locations with sinatra?
+          erb File.read(File.join(File.dirname(__FILE__), 'server/views/delayed.erb'))
+        end
+
+        get "/delayed/:timestamp" do
+          # Is there a better way to specify alternate template locations with sinatra?
+          erb File.read(File.join(File.dirname(__FILE__), 'server/views/delayed_timestamp.erb'))
+        end
+
+      end
+
+    end
+
+    Resque::Server.tabs << 'Schedule'
+    Resque::Server.tabs << 'Delayed'
+
+  end
+  
+end

data/lib/resque_scheduler/server/views/delayed.erb

@@ -0,0 +1,35 @@
+<h1>Delayed Jobs</h1>
+
+<p class='intro'>
+  This list below contains the timestamps for scheduled delayed jobs.
+</p>
+
+<p class='sub'>
+  Showing <%= start = params[:start].to_i %> to <%= start + 20 %> of <b><%=size = resque.delayed_queue_schedule_size %></b> timestamps
+</p>
+
+<table>
+  <tr>
+    <th>Timestamp</th>
+    <th>Job count</th>
+    <th>Class</th>
+    <th>Args</th>
+  </tr>
+  <% resque.delayed_queue_peek(start, start+20).each do |timestamp| %>
+    <tr>
+      <td><a href="<%= url "delayed/#{timestamp}" %>"><%= format_time(Time.at(timestamp)) %></a></td>
+      <td><%= delayed_timestamp_size = resque.delayed_timestamp_size(timestamp) %></td>
+      <% job = resque.delayed_timestamp_peek(timestamp, 0, 1).first %>
+      <td>
+        <% if job && delayed_timestamp_size == 1 %>
+          <%= h(job['class']) %>
+        <% else %>
+          <a href="<%= url "delayed/#{timestamp}" %>">see details</a>
+        <% end %>
+      </td>
+      <td><%= h(job['args'].inspect) if job && delayed_timestamp_size == 1 %></td>
+    </tr>
+  <% end %>
+</table>
+
+<%= partial :next_more, :start => start, :size => size %>

data/lib/resque_scheduler/server/views/delayed_timestamp.erb

@@ -0,0 +1,26 @@
+<% timestamp = params[:timestamp].to_i %>
+
+<h1>Delayed jobs scheduled for <%= format_time(Time.at(timestamp)) %></h1>
+
+<p class='sub'>Showing <%= start = params[:start].to_i %> to <%= start + 20 %> of <b><%=size = resque.delayed_timestamp_size(timestamp)%></b> jobs</p>
+
+<table class='jobs'>
+  <tr>
+    <th>Class</th>
+    <th>Args</th>
+  </tr>
+  <% jobs = resque.delayed_timestamp_peek(timestamp, start, 20) %>
+  <% jobs.each do |job| %>
+    <tr>
+      <td class='class'><%= job['class'] %></td>
+      <td class='args'><%=h job['args'].inspect %></td>
+    </tr>
+  <% end %>
+  <% if jobs.empty? %>
+    <tr>
+      <td class='no-data' colspan='2'>There are no pending jobs scheduled for this time.</td>
+    </tr>
+  <% end %>
+</table>
+
+<%= partial :next_more, :start => start, :size => size %>

data/lib/resque_scheduler/server/views/scheduler.erb

@@ -0,0 +1,34 @@
+<h1>Schedule</h1>
+
+<p class='intro'>
+  The list below contains all scheduled jobs.  Click &quot;Queue now&quot; to queue
+  a job immediately.
+</p>
+
+<table>
+  <tr>
+    <th></th>
+    <th>Name</th>
+    <th>Description</th>
+    <th>Cron</th>
+    <th>Class</th>
+    <th>Queue</th>
+    <th>Arguments</th>
+  </tr>
+  <% Resque.schedule.each do |name, config| %>
+    <tr>
+      <td>
+        <form action="<%= url "/schedule/requeue" %>" method="post">
+          <input type="hidden" name="job_name" value="<%= h name %>">
+          <input type="submit" value="Queue now">
+        </form>
+      </td>
+      <td><%= h name %></td>
+      <td><%= h config['description'] %></td>
+      <td style="white-space:nowrap"><%= h config['cron'] %></td>
+      <td><%= h config['class'] %></td>
+      <td><%= h config['queue'] || queue_from_class_name(config['class']) %></td>
+      <td><%= h config['args'].inspect %></td>
+    </tr>
+  <% end %>
+</table>

data/lib/resque_scheduler/tasks.rb

@@ -0,0 +1,16 @@
+# require 'resque/tasks'
+# will give you the resque tasks
+
+namespace :resque do
+  task :setup
+
+  desc "Start Resque Scheduler"
+  task :scheduler => :setup do
+    require 'resque'
+    require 'resque_scheduler'
+
+    Resque::Scheduler.verbose = true if ENV['VERBOSE']
+    Resque::Scheduler.run
+  end
+
+end

data/lib/resque_scheduler/version.rb

@@ -0,0 +1,3 @@
+module ResqueScheduler
+  Version = '1.0.6'
+end

data/tasks/resque_scheduler.rake

@@ -0,0 +1,2 @@
+$LOAD_PATH.unshift File.dirname(__FILE__) + '/../lib'
+require 'resque_scheduler/tasks'

data/test/delayed_queue_test.rb

@@ -0,0 +1,132 @@
+require File.dirname(__FILE__) + '/test_helper'
+
+class Resque::DelayedQueueTest < Test::Unit::TestCase
+
+  def setup
+    Resque::Scheduler.mute = true
+    Resque.redis.flush_all
+  end
+
+  def test_enqueue_at_adds_correct_list_and_zset
+
+    timestamp = Time.now - 1 # 1 second ago (in the past, should come out right away)
+
+    assert_equal(0, Resque.redis.llen("delayed:#{timestamp.to_i}").to_i, "delayed queue should be empty to start")
+
+    Resque.enqueue_at(timestamp, SomeIvarJob, "path")
+
+    # Confirm the correct keys were added
+    assert_equal(1, Resque.redis.llen("delayed:#{timestamp.to_i}").to_i, "delayed queue should have one entry now")
+    assert_equal(1, Resque.redis.zcard(:delayed_queue_schedule), "The delayed_queue_schedule should have 1 entry now")
+
+    read_timestamp = Resque.next_delayed_timestamp
+
+    # Confirm the timestamp came out correctly
+    assert_equal(timestamp.to_i, read_timestamp, "The timestamp we pull out of redis should match the one we put in")
+    item = Resque.next_item_for_timestamp(read_timestamp)
+
+    # Confirm the item came out correctly
+    assert_equal('SomeIvarJob', item['class'], "Should be the same class that we queued")
+    assert_equal(["path"], item['args'], "Should have the same arguments that we queued")
+    
+    # And now confirm the keys are gone
+    assert(!Resque.redis.exists("delayed:#{timestamp.to_i}"))
+    assert_equal(0, Resque.redis.zcard(:delayed_queue_schedule), "delayed queue should be empty")
+  end
+
+  def test_something_in_the_future_doesnt_come_out
+    timestamp = Time.now + 600 # 10 minutes from now (in the future, shouldn't come out)
+
+    assert_equal(0, Resque.redis.llen("delayed:#{timestamp.to_i}").to_i, "delayed queue should be empty to start")
+
+    Resque.enqueue_at(timestamp, SomeIvarJob, "path")
+
+    # Confirm the correct keys were added
+    assert_equal(1, Resque.redis.llen("delayed:#{timestamp.to_i}").to_i, "delayed queue should have one entry now")
+    assert_equal(1, Resque.redis.zcard(:delayed_queue_schedule), "The delayed_queue_schedule should have 1 entry now")
+
+    read_timestamp = Resque.next_delayed_timestamp
+
+    assert_nil(read_timestamp, "No timestamps should be ready for queueing")
+  end
+
+  def test_enqueue_at_and_enqueue_in_are_equivelent
+    timestamp = Time.now + 60
+
+    Resque.enqueue_at(timestamp, SomeIvarJob, "path")
+    Resque.enqueue_in(timestamp - Time.now, SomeIvarJob, "path")
+
+    assert_equal(1, Resque.redis.zcard(:delayed_queue_schedule), "should have one timestamp in the delayed queue")
+    assert_equal(2, Resque.redis.llen("delayed:#{timestamp.to_i}"), "should have 2 items in the timestamp queue")
+  end
+
+  def test_delayed_queue_peek
+    t = Time.now
+    expected_timestamps = (1..5).to_a.map do |i|
+      (t + 60 + i).to_i
+    end
+
+    expected_timestamps.each do |timestamp|
+      Resque.delayed_push(timestamp, {:class => SomeIvarJob, :args => 'blah1'})
+    end
+
+    timestamps = Resque.delayed_queue_peek(2,3)
+
+    assert_equal(expected_timestamps[2,3], timestamps)
+  end
+
+  def test_delayed_queue_schedule_size
+    assert_equal(0, Resque.delayed_queue_schedule_size)
+    Resque.enqueue_at(Time.now+60, SomeIvarJob)
+    assert_equal(1, Resque.delayed_queue_schedule_size)
+  end
+
+  def test_delayed_timestamp_size
+    t = Time.now + 60
+    assert_equal(0, Resque.delayed_timestamp_size(t))
+    Resque.enqueue_at(t, SomeIvarJob)
+    assert_equal(1, Resque.delayed_timestamp_size(t))
+    assert_equal(0, Resque.delayed_timestamp_size(t.to_i+1))
+  end
+
+  def test_delayed_timestamp_peek
+    t = Time.now + 60
+    assert_equal([], Resque.delayed_timestamp_peek(t, 0, 1), "make sure it's an empty array, not nil")
+    Resque.enqueue_at(t, SomeIvarJob)
+    assert_equal(1, Resque.delayed_timestamp_peek(t, 0, 1).length)
+    Resque.enqueue_at(t, SomeIvarJob)
+    assert_equal(1, Resque.delayed_timestamp_peek(t, 0, 1).length)
+    assert_equal(2, Resque.delayed_timestamp_peek(t, 0, 3).length)
+
+    assert_equal({'args' => [], 'class' => 'SomeIvarJob', 'queue' => 'ivar'}, Resque.delayed_timestamp_peek(t, 0, 1).first)
+  end
+
+  def test_handle_delayed_items_with_no_items
+    Resque::Scheduler.expects(:enqueue).never
+    Resque::Scheduler.handle_delayed_items
+  end
+
+  def test_handle_delayed_items_with_items
+    t = Time.now - 60 # in the past
+    Resque.enqueue_at(t, SomeIvarJob)
+    Resque.enqueue_at(t, SomeIvarJob)
+
+    # 2 SomeIvarJob jobs should be created in the "ivar" queue
+    Resque::Job.expects(:create).twice.with('ivar', 'SomeIvarJob', nil)
+    Resque.expects(:queue_from_class).never # Should NOT need to load the class
+    Resque::Scheduler.handle_delayed_items
+  end
+
+  def test_works_with_out_specifying_queue__upgrade_case
+    t = Time.now - 60
+    Resque.delayed_push(t, :class => 'SomeIvarJob')
+
+    # Since we didn't specify :queue when calling delayed_push, it will be forced
+    # to load the class to figure out the queue.  This is the upgrade case from 1.0.4
+    # to 1.0.5.
+    Resque::Job.expects(:create).once.with(:ivar, 'SomeIvarJob', nil)
+
+    Resque::Scheduler.handle_delayed_items
+  end
+  
+end

data/test/redis-test.conf

@@ -0,0 +1,132 @@
+# Redis configuration file example
+
+# By default Redis does not run as a daemon. Use 'yes' if you need it.
+# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
+daemonize yes
+
+# When run as a daemon, Redis write a pid file in /var/run/redis.pid by default.
+# You can specify a custom pid file location here.
+pidfile ./test/redis-test.pid
+
+# Accept connections on the specified port, default is 6379
+port 9736
+
+# If you want you can bind a single interface, if the bind option is not
+# specified all the interfaces will listen for connections.
+#
+# bind 127.0.0.1
+
+# Close the connection after a client is idle for N seconds (0 to disable)
+timeout 300
+
+# Save the DB on disk:
+#
+#   save <seconds> <changes>
+#
+#   Will save the DB if both the given number of seconds and the given
+#   number of write operations against the DB occurred.
+#
+#   In the example below the behaviour will be to save:
+#   after 900 sec (15 min) if at least 1 key changed
+#   after 300 sec (5 min) if at least 10 keys changed
+#   after 60 sec if at least 10000 keys changed
+save 900 1
+save 300 10
+save 60 10000
+
+# The filename where to dump the DB
+dbfilename dump.rdb
+
+# For default save/load DB in/from the working directory
+# Note that you must specify a directory not a file name.
+dir ./test/
+
+# Set server verbosity to 'debug'
+# it can be one of:
+# debug (a lot of information, useful for development/testing)
+# notice (moderately verbose, what you want in production probably)
+# warning (only very important / critical messages are logged)
+loglevel debug
+
+# Specify the log file name. Also 'stdout' can be used to force
+# the demon to log on the standard output. Note that if you use standard
+# output for logging but daemonize, logs will be sent to /dev/null
+logfile stdout
+
+# Set the number of databases. The default database is DB 0, you can select
+# a different one on a per-connection basis using SELECT <dbid> where
+# dbid is a number between 0 and 'databases'-1
+databases 16
+
+################################# REPLICATION #################################
+
+# Master-Slave replication. Use slaveof to make a Redis instance a copy of
+# another Redis server. Note that the configuration is local to the slave
+# so for example it is possible to configure the slave to save the DB with a
+# different interval, or to listen to another port, and so on.
+
+# slaveof <masterip> <masterport>
+
+################################## SECURITY ###################################
+
+# Require clients to issue AUTH <PASSWORD> before processing any other
+# commands.  This might be useful in environments in which you do not trust
+# others with access to the host running redis-server.
+#
+# This should stay commented out for backward compatibility and because most
+# people do not need auth (e.g. they run their own servers).
+
+# requirepass foobared
+
+################################### LIMITS ####################################
+
+# Set the max number of connected clients at the same time. By default there
+# is no limit, and it's up to the number of file descriptors the Redis process
+# is able to open. The special value '0' means no limts.
+# Once the limit is reached Redis will close all the new connections sending
+# an error 'max number of clients reached'.
+
+# maxclients 128
+
+# Don't use more memory than the specified amount of bytes.
+# When the memory limit is reached Redis will try to remove keys with an
+# EXPIRE set. It will try to start freeing keys that are going to expire
+# in little time and preserve keys with a longer time to live.
+# Redis will also try to remove objects from free lists if possible.
+#
+# If all this fails, Redis will start to reply with errors to commands
+# that will use more memory, like SET, LPUSH, and so on, and will continue
+# to reply to most read-only commands like GET.
+#
+# WARNING: maxmemory can be a good idea mainly if you want to use Redis as a
+# 'state' server or cache, not as a real DB. When Redis is used as a real
+# database the memory usage will grow over the weeks, it will be obvious if
+# it is going to use too much memory in the long run, and you'll have the time
+# to upgrade. With maxmemory after the limit is reached you'll start to get
+# errors for write operations, and this may even lead to DB inconsistency.
+
+# maxmemory <bytes>
+
+############################### ADVANCED CONFIG ###############################
+
+# Glue small output buffers together in order to send small replies in a
+# single TCP packet. Uses a bit more CPU but most of the times it is a win
+# in terms of number of queries per second. Use 'yes' if unsure.
+glueoutputbuf yes
+
+# Use object sharing. Can save a lot of memory if you have many common
+# string in your dataset, but performs lookups against the shared objects
+# pool so it uses more CPU and can be a bit slower. Usually it's a good
+# idea.
+#
+# When object sharing is enabled (shareobjects yes) you can use
+# shareobjectspoolsize to control the size of the pool used in order to try
+# object sharing. A bigger pool size will lead to better sharing capabilities.
+# In general you want this value to be at least the double of the number of
+# very common strings you have in your dataset.
+#
+# WARNING: object sharing is experimental, don't enable this feature
+# in production before of Redis 1.0-stable. Still please try this feature in
+# your development environment so that we can test it better.
+shareobjects no
+shareobjectspoolsize 1024

data/test/scheduler_test.rb

@@ -0,0 +1,28 @@
+require File.dirname(__FILE__) + '/test_helper'
+
+class Resque::SchedulerTest < Test::Unit::TestCase
+
+  def setup
+    Resque::Scheduler.clear_schedule!
+  end
+
+  def test_enqueue_from_config_puts_stuff_in_the_resque_queue_without_class_loaded
+    Resque::Job.stubs(:create).once.returns(true).with('joes_queue', 'BigJoesJob', '/tmp')
+    Resque::Scheduler.enqueue_from_config('cron' => "* * * * *", 'class' => 'BigJoesJob', 'args' => "/tmp", 'queue' => 'joes_queue')
+  end
+  
+  def test_enqueue_from_config_puts_stuff_in_the_resque_queue
+    Resque::Job.stubs(:create).once.returns(true).with(:ivar, 'SomeIvarJob', '/tmp')
+    Resque::Scheduler.enqueue_from_config('cron' => "* * * * *", 'class' => 'SomeIvarJob', 'args' => "/tmp")
+  end
+
+  def test_config_makes_it_into_the_rufus_scheduler
+    assert_equal(0, Resque::Scheduler.rufus_scheduler.all_jobs.size)
+
+    Resque.schedule = {:some_ivar_job => {'cron' => "* * * * *", 'class' => 'SomeIvarJob', 'args' => "/tmp"}}
+    Resque::Scheduler.load_schedule!
+
+    assert_equal(1, Resque::Scheduler.rufus_scheduler.all_jobs.size)
+  end
+
+end

data/test/test_helper.rb

@@ -0,0 +1,60 @@
+
+# Pretty much copied this file from the resque test_helper since we want
+# to do all the same stuff
+
+dir = File.dirname(File.expand_path(__FILE__))
+
+require 'rubygems'
+require 'test/unit'
+require 'mocha'
+require 'resque'
+require File.join(dir, '../lib/resque_scheduler')
+$LOAD_PATH.unshift File.dirname(File.expand_path(__FILE__)) + '/../lib'
+
+
+#
+# make sure we can run redis
+#
+
+if !system("which redis-server")
+  puts '', "** can't find `redis-server` in your path"
+  puts "** try running `sudo rake install`"
+  abort ''
+end
+
+
+#
+# start our own redis when the tests start,
+# kill it when they end
+#
+
+at_exit do
+  next if $!
+
+  if defined?(MiniTest)
+    exit_code = MiniTest::Unit.new.run(ARGV)
+  else
+    exit_code = Test::Unit::AutoRunner.run
+  end
+
+  pid = `ps -e -o pid,command | grep [r]edis-test`.split(" ")[0]
+  puts "Killing test redis server..."
+  `rm -f #{dir}/dump.rdb`
+  Process.kill("KILL", pid.to_i)
+  exit exit_code
+end
+
+puts "Starting redis for testing at localhost:9736..."
+`redis-server #{dir}/redis-test.conf`
+Resque.redis = 'localhost:9736'
+
+
+
+class SomeJob
+  def self.perform(repo_id, path)
+  end
+end
+
+class SomeIvarJob < SomeJob
+  @queue = :ivar
+end

metadata

@@ -0,0 +1,134 @@
+--- !ruby/object:Gem::Specification 
+name: thehenster-resque-scheduler
+version: !ruby/object:Gem::Version 
+  prerelease: false
+  segments: 
+  - 1
+  - 0
+  - 6
+  version: 1.0.6
+platform: ruby
+authors: 
+- Ben VandenBos
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2010-03-29 00:00:00 +01:00
+default_executable: 
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: resque
+  prerelease: false
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 1
+        - 5
+        - 0
+        version: 1.5.0
+  type: :runtime
+  version_requirements: *id001
+- !ruby/object:Gem::Dependency 
+  name: rufus-scheduler
+  prerelease: false
+  requirement: &id002 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id002
+- !ruby/object:Gem::Dependency 
+  name: jeweler
+  prerelease: false
+  requirement: &id003 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :development
+  version_requirements: *id003
+- !ruby/object:Gem::Dependency 
+  name: mocha
+  prerelease: false
+  requirement: &id004 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :development
+  version_requirements: *id004
+description: |-
+  Light weight job scheduling on top of Resque.
+    Adds methods enqueue_at/enqueue_in to schedule jobs in the future.
+    Also supports queueing jobs on a fixed, cron-like schedule.
+email: bvandenbos@gmail.com
+executables: []
+
+extensions: []
+
+extra_rdoc_files: 
+- LICENSE
+- README.markdown
+files: 
+- .gitignore
+- HISTORY.md
+- LICENSE
+- README.markdown
+- Rakefile
+- lib/resque/scheduler.rb
+- lib/resque_scheduler.rb
+- lib/resque_scheduler/server.rb
+- lib/resque_scheduler/server/views/delayed.erb
+- lib/resque_scheduler/server/views/delayed_timestamp.erb
+- lib/resque_scheduler/server/views/scheduler.erb
+- lib/resque_scheduler/tasks.rb
+- lib/resque_scheduler/version.rb
+- tasks/resque_scheduler.rake
+- test/delayed_queue_test.rb
+- test/redis-test.conf
+- test/scheduler_test.rb
+- test/test_helper.rb
+has_rdoc: true
+homepage: http://github.com/bvandenbos/resque-scheduler
+licenses: []
+
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.6
+signing_key: 
+specification_version: 3
+summary: Light weight job scheduling on top of Resque
+test_files: 
+- test/delayed_queue_test.rb
+- test/scheduler_test.rb
+- test/test_helper.rb