resque-scheduler 0.0.1 → 1.0.0

data/README.markdown

@@ -1,13 +1,22 @@
 resque-scheduler
 ===============
 
-Resque-scheduler is basically the union of rufus-scheduler and resque.  The goal
-is to provide simple job scheduling with centralized configuration and
-distributed workers.  
+Resque-scheduler is an extension to Resque that adds support for queueing items
+in the future.
+
+Requires redis >=1.1.
+
+
+Job scheduling is supported in two different way:
+
+### Recurring (scheduled)
+
+Recurring (or scheduled) jobs are logically no different than a standard cron
+job.  They are jobs that run based on a fixed schedule which is set at startup.
 
 The schedule is a list of Resque worker classes with arguments and a
 schedule frequency (in crontab syntax).  The schedule is just a hash, but
-is most likely stored in a YAML:
+is most likely stored in a YAML like so:
 
     queue_documents_for_indexing:
       cron: "0 0 * * *"
@@ -32,8 +41,69 @@ And then set the schedule wherever you configure Resque, like so:
     require 'resque-scheduler'
     ResqueScheduler.schedule = YAML.load_file(File.join(File.dirname(__FILE__), '../resque_schedule.yml'))
 
-The scheduler process is just a rake task which adds things to resque when they fire
-based on the schedule.  For obvious reasons, this process never exits.
+Keep in mind, scheduled jobs behave like crons: if your scheduler process (more
+on that later) is not running when a particular job is supposed to be queued,
+it will NOT be ran later when the scheduler process is started back up.  In that
+sense, you can sort of think of the scheduler process as crond.  Delayed jobs,
+however, are different.
+
+A big shout out to [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler) for handling the heavy lifting of the
+actual scheduling engine.
+
+### Delayed jobs
+
+Delayed jobs are one-off jobs that you want to be put into a queue at some point
+in the future.  The classic example is sending email:
+
+    Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :user_id => current_user.id)
+
+This will store the job for 5 days in the resque delayed queue at which time the
+scheduler process will pull it from the delayed queue and put it in the
+appropriate work queue for the given job and it will be processed as soon as
+a worker is available.
+
+NOTE: The job does not fire **exactly** at the time supplied.  Rather, once that
+time is in the past, the job moves from the delayed queue to the actual resque
+work queue and will be completed as workers as free to process it.
+
+Also supported is `Resque.enqueue_in` which takes an amount of time in seconds
+in which to queue the job.
+
+The delayed queue is stored in redis and is persisted in the same way the
+standard resque jobs are persisted (redis writing to disk). Delayed jobs differ
+from scheduled jobs in that if your scheduler process is down or workers are
+down when a particular job is supposed to be queue, they will simply "catch up"
+once they are started again.  Jobs are guaranteed to run (provided they make it
+into the delayed queue) after their given queue_at time has passed.
+
+One other thing to note is that insertion into the delayed queue is O(log(n))
+since the jobs are stored in a redis sorted set (zset).  I can't imagine this
+being an issue for someone since redis is stupidly fast even at log(n), but full
+disclosure is always best.
+
+
+Resque-web additions
+--------------------
+
+Resque-scheduler also adds to tabs to the resque-web UI.  One is for viewing
+(and manually queueing) the schedule and one is for viewing pending jobs in
+the delayed queue.
+
+The Schedule tab:
+
+![The Schedule Tab](http://img.skitch.com/20100111-km2f5gmtpbq23enpujbruj6mgk.png)
+
+The Delayed tab:
+
+![The Delayed Tab](http://img.skitch.com/20100111-ne4fcqtc5emkcuwc5qtais2kwx.jpg)
+
+
+The Scheduler process
+---------------------
+
+The scheduler process is just a rake task which is responsible for both queueing
+items from the schedule and polling the delayed queue for items ready to be
+pushed on to the work queues.  For obvious reasons, this process never exits.
 
     $ rake resque-scheduler 
 
@@ -42,4 +112,22 @@ You'll need to add this to your rakefile:
     require 'resque_scheduler/tasks'
     task "resque:setup" => :environment
 
+Supported environment variables are `VERBOSE` and `MUTE`.  If either is set to
+any nonempty value, they will take effect.  `VERBOSE` simply dumps more output
+to stdout.  `MUTE` does the opposite and silences all output. `MUTE` supercedes
+`VERBOSE`.
+
+
+Plagurism alert
+---------------
+
+This was intended to be an extension to resque and so resulted in a lot of the
+code looking very similar to resque, particularly in resque-web and the views. I
+wanted it to be similar enough that someone familiar with resque could easily
+work on resque-scheduler.
+
+
+Contributing
+------------
 
+For bugs or suggestions, please just open an issue in github.

data/Rakefile

@@ -27,14 +27,16 @@ begin
 
   Jeweler::Tasks.new do |gemspec|
     gemspec.name = "resque-scheduler"
-    gemspec.summary = ""
-    gemspec.description = ""
+    gemspec.summary = "Light weight job scheduling on top of Resque"
+    gemspec.description = %{Light weight job scheduling on top of Resque.
+  Adds methods enqueue_at/enqueue_in to schedule jobs in the future.
+  Also supports queueing jobs on a fixed, cron-like schedule.}
     gemspec.email = "bvandenbos@gmail.com"
     gemspec.homepage = "http://github.com/bvandenbos/resque-scheduler"
     gemspec.authors = ["Ben VandenBos"]
     gemspec.version = ResqueScheduler::Version
 
-    gemspec.add_dependency "resque"
+    gemspec.add_dependency "resque", ">= 1.3.0"
     gemspec.add_dependency "rufus-scheduler"
     gemspec.add_development_dependency "jeweler"
     gemspec.add_development_dependency "mocha"
@@ -49,5 +51,4 @@ task :publish => [ :test, :gemspec, :build ] do
   system "git push origin master"
   system "gem push pkg/resque-scheduler-#{ResqueScheduler::Version}.gem"
   system "git clean -fd"
-  exec "rake pages"
 end

data/lib/resque/scheduler.rb

@@ -10,24 +10,82 @@ module Resque
 
     class << self
 
-      # Schedule all jobs and sleep (never returns)
-      def run(wait = true)
-        puts "Schedule empty! Set Resque.schedule" if Resque.schedule.empty?
+      # If true, logs more stuff...
+      attr_accessor :verbose
+      
+      # If set, produces no output
+      attr_accessor :mute
+
+      # Schedule all jobs and continually look for delayed jobs (never returns)
+      def run
+
+        # trap signals
+        register_signal_handlers
+
+        # Load the schedule into rufus
+        load_schedule!
+
+        # Now start the scheduling part of the loop.
+        loop do
+          handle_delayed_items
+          poll_sleep
+        end
+
+        # never gets here.
+      end
+
+      # For all signals, set the shutdown flag and wait for current
+      # poll/enqueing to finish (should be almost istant).  In the
+      # case of sleeping, exit immediately.
+      def register_signal_handlers
+        trap("TERM") { shutdown }
+        trap("INT") { shutdown }
+        trap('QUIT') { shutdown } unless defined? JRUBY_VERSION
+      end
+
+      # Pulls the schedule from Resque.schedule and loads it into the
+      # rufus scheduler instance
+      def load_schedule!
+        log! "Schedule empty! Set Resque.schedule" if Resque.schedule.empty?
 
         Resque.schedule.each do |name, config|
-          puts "Scheduling #{name}..."
+          log! "Scheduling #{name} "
           rufus_scheduler.cron config['cron'] do
-            puts "#{Time.now} queuing #{name}"
+            log! "queuing #{config['class']} (#{name})"
             enqueue_from_config(config)
           end
         end
-        # sleep baby, sleep
-        ThreadsWait.all_waits(rufus_scheduler.instance_variable_get("@thread")) if wait
       end
 
+      # Handles queueing delayed items
+      def handle_delayed_items
+        item = nil
+        if timestamp = Resque.next_delayed_timestamp
+          item = nil
+          begin
+            handle_shutdown do
+              if item = Resque.next_item_for_timestamp(timestamp)
+                log "queuing #{item['class']} [delayed]"
+                klass = constantize(item['class'])
+                Resque.enqueue(klass, *item['args'])
+              end
+            end
+          end while !item.nil?
+        end
+      end
+
+      def handle_shutdown
+        exit if @shutdown
+        yield
+        exit if @shutdown
+      end
+
+      # Enqueues a job based on a config hash
       def enqueue_from_config(config)
-        params = config['args'].nil? ? [] : Array(config['args'])
-        Resque.enqueue(constantize(config['class']), *params)
+        args = config['args'] || config[:args]
+        klass_name = config['class'] || config[:class]
+        params = args.nil? ? [] : Array(args)
+        Resque.enqueue(constantize(klass_name), *params)
       end
 
       def rufus_scheduler
@@ -42,6 +100,29 @@ module Resque
         rufus_scheduler
       end
 
+      # Sleeps and returns true
+      def poll_sleep
+        @sleeping = true
+        handle_shutdown { sleep 5 }
+        @sleeping = false
+        true
+      end
+
+      # Sets the shutdown flag, exits if sleeping
+      def shutdown
+        @shutdown = true
+        exit if @sleeping
+      end
+
+      def log!(msg)
+        puts "#{Time.now.strftime("%Y-%m-%d %H:%M:%S")} #{msg}" unless mute
+      end
+
+      def log(msg)
+        # add "verbose" logic later
+        log!(msg) if verbose
+      end
+
     end
 
   end

data/lib/resque_scheduler.rb

@@ -1,7 +1,9 @@
 require 'rubygems'
 require 'resque'
+require 'resque/server'
 require 'resque_scheduler/version'
 require 'resque/scheduler'
+require 'resque_scheduler/server'
 
 module ResqueScheduler
 
@@ -31,6 +33,84 @@ module ResqueScheduler
     @schedule ||= {}
   end
 
+  # This method is nearly identical to +enqueue+ only it also
+  # takes a timestamp which will be used to schedule the job
+  # for queueing.  Until timestamp is in the past, the job will
+  # sit in the schedule list.
+  def enqueue_at(timestamp, klass, *args)
+    delayed_push(timestamp, :class => klass.to_s, :args => args)
+  end
+
+  # Identical to enqueue_at but takes number_of_seconds_from_now
+  # instead of a timestamp.
+  def enqueue_in(number_of_seconds_from_now, klass, *args)
+    enqueue_at(Time.now + number_of_seconds_from_now, klass, *args)
+  end
+
+  # Used internally to stuff the item into the schedule sorted list.
+  # +timestamp+ can be either in seconds or a datetime object
+  # Insertion if O(log(n)).
+  def delayed_push(timestamp, item)
+    # First add this item to the list for this timestamp
+    redis.rpush("delayed:#{timestamp.to_i}", encode(item))
+
+    # Now, add this timestamp to the zsets.  The score and the value are
+    # the same since we'll be querying by timestamp, and we don't have
+    # anything else to store.
+    redis.zset_add :delayed_queue_schedule, timestamp.to_i, timestamp.to_i
+  end
+
+  # Returns an array of timestamps based on start and count
+  def delayed_queue_peek(start, count)
+    (redis.zrange :delayed_queue_schedule, start, start+count).collect(&:to_i)
+  end
+
+  # Returns the size of the delayed queue schedule
+  def delayed_queue_schedule_size
+    redis.zcard :delayed_queue_schedule
+  end
+
+  # Returns the number of jobs for a given timestamp in the delayed queue schedule
+  def delayed_timestamp_size(timestamp)
+    redis.llen "delayed:#{timestamp.to_i}"
+  end
+
+  # Returns an array of delayed items for the given timestamp
+  def delayed_timestamp_peek(timestamp, start, count)
+    if 1 == count
+      r = list_range "delayed:#{timestamp.to_i}", start, count
+      r.nil? ? [] : [r]
+    else
+      list_range "delayed:#{timestamp.to_i}", start, count
+    end
+  end
+
+  # Returns the next delayed queue timestamp
+  # (don't call directly)
+  def next_delayed_timestamp
+    timestamp = redis.zrangebyscore(:delayed_queue_schedule, '-inf', Time.now.to_i, 'limit', 0, 1).first
+    timestamp.to_i unless timestamp.nil?
+  end
+
+  # Returns the next item to be processed for a given timestamp, nil if
+  # done. (don't call directly)
+  # +timestamp+ can either be in seconds or a datetime
+  def next_item_for_timestamp(timestamp)
+    key = "delayed:#{timestamp.to_i}"
+
+    item = decode redis.lpop(key)
+
+    # If the list is empty, remove it.
+    if 0 == redis.llen(key).to_i
+      redis.del key
+      redis.zrem :delayed_queue_schedule, timestamp.to_i
+    end
+    item
+  end
+
 end
 
-Resque.extend ResqueScheduler
+Resque.extend ResqueScheduler
+Resque::Server.class_eval do
+  include ResqueScheduler::Server
+end

data/lib/resque_scheduler/server.rb

@@ -0,0 +1,47 @@
+
+# Extend Resque::Server to add tabs
+module ResqueScheduler
+  
+  module Server
+
+    def self.included(base)
+
+      base.class_eval do
+
+        helpers do
+          def format_time(t)
+            t.strftime("%Y-%m-%d %H:%M:%S")
+          end
+        end
+
+        get "/schedule" do
+          # Is there a better way to specify alternate template locations with sinatra?
+          erb File.read(File.join(File.dirname(__FILE__), 'server/views/scheduler.erb'))
+        end
+
+        post "/schedule/requeue" do
+          config = Resque.schedule[params['job_name']]
+          Resque::Scheduler.enqueue_from_config(config)
+          redirect url("/queues")
+        end
+        
+        get "/delayed" do
+          # Is there a better way to specify alternate template locations with sinatra?
+          erb File.read(File.join(File.dirname(__FILE__), 'server/views/delayed.erb'))
+        end
+
+        get "/delayed/:timestamp" do
+          # Is there a better way to specify alternate template locations with sinatra?
+          erb File.read(File.join(File.dirname(__FILE__), 'server/views/delayed_timestamp.erb'))
+        end
+
+      end
+
+    end
+
+    Resque::Server.tabs << 'Schedule'
+    Resque::Server.tabs << 'Delayed'
+
+  end
+  
+end

data/lib/resque_scheduler/server/views/delayed.erb

@@ -0,0 +1,24 @@
+<h1>Delayed Jobs</h1>
+
+<p class='intro'>
+  This list below contains the timestamps for scheduled delayed jobs.
+</p>
+
+<p class='sub'>
+  Showing <%= start = params[:start].to_i %> to <%= start + 20 %> of <b><%=size = resque.delayed_queue_schedule_size %></b> timestamps
+</p>
+
+<table>
+  <tr>
+    <th>Timestamp</th>
+    <th>Job count</th>
+  </tr>
+  <% resque.delayed_queue_peek(start, start+20).each do |timestamp| %>
+    <tr>
+      <td><a href="<%= url "delayed/#{timestamp}" %>"><%= format_time(Time.at(timestamp)) %></a></td>
+      <td><%= resque.delayed_timestamp_size(timestamp) %></td>
+    </tr>
+  <% end %>
+</table>
+
+<%= partial :next_more, :start => start, :size => size %>

data/lib/resque_scheduler/server/views/delayed_timestamp.erb

@@ -0,0 +1,26 @@
+<% timestamp = params[:timestamp].to_i %>
+
+<h1>Delayed jobs scheduled for <%= format_time(Time.at(timestamp)) %></h1>
+
+<p class='sub'>Showing <%= start = params[:start].to_i %> to <%= start + 20 %> of <b><%=size = resque.delayed_timestamp_size(timestamp)%></b> jobs</p>
+
+<table class='jobs'>
+  <tr>
+    <th>Class</th>
+    <th>Args</th>
+  </tr>
+  <% jobs = resque.delayed_timestamp_peek(timestamp, start, 20) %>
+  <% jobs.each do |job| %>
+    <tr>
+      <td class='class'><%= job['class'] %></td>
+      <td class='args'><%=h job['args'].inspect %></td>
+    </tr>
+  <% end %>
+  <% if jobs.empty? %>
+    <tr>
+      <td class='no-data' colspan='2'>There are no pending jobs scheduled for this time.</td>
+    </tr>
+  <% end %>
+</table>
+
+<%= partial :next_more, :start => start, :size => size %>

data/lib/resque_scheduler/server/views/scheduler.erb

@@ -0,0 +1,32 @@
+<h1>Schedule</h1>
+
+<p class='intro'>
+  The list below contains all scheduled jobs.  Click &quot;Queue now&quot; to queue
+  a job immediately.
+</p>
+
+<table>
+  <tr>
+    <th></th>
+    <th>Name</th>
+    <th>Description</th>
+    <th>Cron</th>
+    <th>Class</th>
+    <th>Arguments</th>
+  </tr>
+  <% Resque.schedule.each do |name, config| %>
+    <tr>
+      <td>
+        <form action="<%= url "/schedule/requeue" %>" method="post">
+          <input type="hidden" name="job_name" value="<%= h name %>">
+          <input type="submit" value="Queue now">
+        </form>
+      </td>
+      <td><%= h name %></td>
+      <td><%= h config['description'] %></td>
+      <td style="white-space:nowrap"><%= h config['cron'] %></td>
+      <td><%= h config['class'] %></td>
+      <td><%= h config['args'].inspect %></td>
+    </tr>
+  <% end %>
+</table>

data/lib/resque_scheduler/tasks.rb

@@ -9,6 +9,7 @@ namespace :resque do
     require 'resque'
     require 'resque_scheduler'
 
+    Resque::Scheduler.verbose = true if ENV['VERBOSE']
     Resque::Scheduler.run
   end
 

data/lib/resque_scheduler/version.rb

@@ -1,3 +1,3 @@
 module ResqueScheduler
-  Version = '0.0.1'
-end
+  Version = '1.0.0'
+end

data/test/delayed_queue_test.rb

@@ -0,0 +1,117 @@
+require File.dirname(__FILE__) + '/test_helper'
+
+class Resque::DelayedQueueTest < Test::Unit::TestCase
+
+  def setup
+    Resque::Scheduler.mute = true
+    Resque.redis.flush_all
+  end
+
+  def test_enqueue_at_adds_correct_list_and_zset
+
+    timestamp = Time.now - 1 # 1 second ago (in the past, should come out right away)
+
+    assert_equal(0, Resque.redis.llen("delayed:#{timestamp.to_i}").to_i, "delayed queue should be empty to start")
+
+    Resque.enqueue_at(timestamp, SomeIvarJob, "path")
+
+    # Confirm the correct keys were added
+    assert_equal(1, Resque.redis.llen("delayed:#{timestamp.to_i}").to_i, "delayed queue should have one entry now")
+    assert_equal(1, Resque.redis.zcard(:delayed_queue_schedule), "The delayed_queue_schedule should have 1 entry now")
+
+    read_timestamp = Resque.next_delayed_timestamp
+
+    # Confirm the timestamp came out correctly
+    assert_equal(timestamp.to_i, read_timestamp, "The timestamp we pull out of redis should match the one we put in")
+    item = Resque.next_item_for_timestamp(read_timestamp)
+
+    # Confirm the item came out correctly
+    assert_equal('SomeIvarJob', item['class'], "Should be the same class that we queued")
+    assert_equal(["path"], item['args'], "Should have the same arguments that we queued")
+    
+    # And now confirm the keys are gone
+    assert(!Resque.redis.exists("delayed:#{timestamp.to_i}"))
+    assert_equal(0, Resque.redis.zcard(:delayed_queue_schedule), "delayed queue should be empty")
+  end
+
+  def test_something_in_the_future_doesnt_come_out
+    timestamp = Time.now + 600 # 10 minutes from now (in the future, shouldn't come out)
+
+    assert_equal(0, Resque.redis.llen("delayed:#{timestamp.to_i}").to_i, "delayed queue should be empty to start")
+
+    Resque.enqueue_at(timestamp, SomeIvarJob, "path")
+
+    # Confirm the correct keys were added
+    assert_equal(1, Resque.redis.llen("delayed:#{timestamp.to_i}").to_i, "delayed queue should have one entry now")
+    assert_equal(1, Resque.redis.zcard(:delayed_queue_schedule), "The delayed_queue_schedule should have 1 entry now")
+
+    read_timestamp = Resque.next_delayed_timestamp
+
+    assert_nil(read_timestamp, "No timestamps should be ready for queueing")
+  end
+
+  def test_enqueue_at_and_enqueue_in_are_equivelent
+    timestamp = Time.now + 60
+
+    Resque.enqueue_at(timestamp, SomeIvarJob, "path")
+    Resque.enqueue_in(timestamp - Time.now, SomeIvarJob, "path")
+
+    assert_equal(1, Resque.redis.zcard(:delayed_queue_schedule), "should have one timestamp in the delayed queue")
+    assert_equal(2, Resque.redis.llen("delayed:#{timestamp.to_i}"), "should have 2 items in the timestamp queue")
+  end
+
+  def test_delayed_queue_peek
+    t = Time.now
+    expected_timestamps = (1..5).to_a.map do |i|
+      (t + 60 + i).to_i
+    end
+
+    expected_timestamps.each do |timestamp|
+      Resque.delayed_push(timestamp, {:class => SomeIvarJob, :args => 'blah1'})
+    end
+
+    timestamps = Resque.delayed_queue_peek(2,3)
+
+    assert_equal(expected_timestamps[2,3], timestamps)
+  end
+
+  def test_delayed_queue_schedule_size
+    assert_equal(0, Resque.delayed_queue_schedule_size)
+    Resque.enqueue_at(Time.now+60, SomeIvarJob)
+    assert_equal(1, Resque.delayed_queue_schedule_size)
+  end
+
+  def test_delayed_timestamp_size
+    t = Time.now + 60
+    assert_equal(0, Resque.delayed_timestamp_size(t))
+    Resque.enqueue_at(t, SomeIvarJob)
+    assert_equal(1, Resque.delayed_timestamp_size(t))
+    assert_equal(0, Resque.delayed_timestamp_size(t.to_i+1))
+  end
+
+  def test_delayed_timestamp_peek
+    t = Time.now + 60
+    assert_equal([], Resque.delayed_timestamp_peek(t, 0, 1), "make sure it's an empty array, not nil")
+    Resque.enqueue_at(t, SomeIvarJob)
+    assert_equal(1, Resque.delayed_timestamp_peek(t, 0, 1).length)
+    Resque.enqueue_at(t, SomeIvarJob)
+    assert_equal(1, Resque.delayed_timestamp_peek(t, 0, 1).length)
+    assert_equal(2, Resque.delayed_timestamp_peek(t, 0, 3).length)
+
+    assert_equal({'args' => [], 'class' => 'SomeIvarJob'}, Resque.delayed_timestamp_peek(t, 0, 1).first)
+  end
+
+  def test_handle_delayed_items_with_no_items
+    Resque::Scheduler.expects(:enqueue).never
+    Resque::Scheduler.handle_delayed_items
+  end
+
+  def test_handle_delayed_items_with_items
+    t = Time.now - 60 # in the past
+    Resque.enqueue_at(t, SomeIvarJob)
+    Resque.enqueue_at(t, SomeIvarJob)
+    Resque.expects(:enqueue).twice
+    Resque::Scheduler.handle_delayed_items
+  end
+  
+end

data/test/redis-test.conf

@@ -0,0 +1,132 @@
+# Redis configuration file example
+
+# By default Redis does not run as a daemon. Use 'yes' if you need it.
+# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
+daemonize yes
+
+# When run as a daemon, Redis write a pid file in /var/run/redis.pid by default.
+# You can specify a custom pid file location here.
+pidfile ./test/redis-test.pid
+
+# Accept connections on the specified port, default is 6379
+port 9736
+
+# If you want you can bind a single interface, if the bind option is not
+# specified all the interfaces will listen for connections.
+#
+# bind 127.0.0.1
+
+# Close the connection after a client is idle for N seconds (0 to disable)
+timeout 300
+
+# Save the DB on disk:
+#
+#   save <seconds> <changes>
+#
+#   Will save the DB if both the given number of seconds and the given
+#   number of write operations against the DB occurred.
+#
+#   In the example below the behaviour will be to save:
+#   after 900 sec (15 min) if at least 1 key changed
+#   after 300 sec (5 min) if at least 10 keys changed
+#   after 60 sec if at least 10000 keys changed
+save 900 1
+save 300 10
+save 60 10000
+
+# The filename where to dump the DB
+dbfilename dump.rdb
+
+# For default save/load DB in/from the working directory
+# Note that you must specify a directory not a file name.
+dir ./test/
+
+# Set server verbosity to 'debug'
+# it can be one of:
+# debug (a lot of information, useful for development/testing)
+# notice (moderately verbose, what you want in production probably)
+# warning (only very important / critical messages are logged)
+loglevel debug
+
+# Specify the log file name. Also 'stdout' can be used to force
+# the demon to log on the standard output. Note that if you use standard
+# output for logging but daemonize, logs will be sent to /dev/null
+logfile stdout
+
+# Set the number of databases. The default database is DB 0, you can select
+# a different one on a per-connection basis using SELECT <dbid> where
+# dbid is a number between 0 and 'databases'-1
+databases 16
+
+################################# REPLICATION #################################
+
+# Master-Slave replication. Use slaveof to make a Redis instance a copy of
+# another Redis server. Note that the configuration is local to the slave
+# so for example it is possible to configure the slave to save the DB with a
+# different interval, or to listen to another port, and so on.
+
+# slaveof <masterip> <masterport>
+
+################################## SECURITY ###################################
+
+# Require clients to issue AUTH <PASSWORD> before processing any other
+# commands.  This might be useful in environments in which you do not trust
+# others with access to the host running redis-server.
+#
+# This should stay commented out for backward compatibility and because most
+# people do not need auth (e.g. they run their own servers).
+
+# requirepass foobared
+
+################################### LIMITS ####################################
+
+# Set the max number of connected clients at the same time. By default there
+# is no limit, and it's up to the number of file descriptors the Redis process
+# is able to open. The special value '0' means no limts.
+# Once the limit is reached Redis will close all the new connections sending
+# an error 'max number of clients reached'.
+
+# maxclients 128
+
+# Don't use more memory than the specified amount of bytes.
+# When the memory limit is reached Redis will try to remove keys with an
+# EXPIRE set. It will try to start freeing keys that are going to expire
+# in little time and preserve keys with a longer time to live.
+# Redis will also try to remove objects from free lists if possible.
+#
+# If all this fails, Redis will start to reply with errors to commands
+# that will use more memory, like SET, LPUSH, and so on, and will continue
+# to reply to most read-only commands like GET.
+#
+# WARNING: maxmemory can be a good idea mainly if you want to use Redis as a
+# 'state' server or cache, not as a real DB. When Redis is used as a real
+# database the memory usage will grow over the weeks, it will be obvious if
+# it is going to use too much memory in the long run, and you'll have the time
+# to upgrade. With maxmemory after the limit is reached you'll start to get
+# errors for write operations, and this may even lead to DB inconsistency.
+
+# maxmemory <bytes>
+
+############################### ADVANCED CONFIG ###############################
+
+# Glue small output buffers together in order to send small replies in a
+# single TCP packet. Uses a bit more CPU but most of the times it is a win
+# in terms of number of queries per second. Use 'yes' if unsure.
+glueoutputbuf yes
+
+# Use object sharing. Can save a lot of memory if you have many common
+# string in your dataset, but performs lookups against the shared objects
+# pool so it uses more CPU and can be a bit slower. Usually it's a good
+# idea.
+#
+# When object sharing is enabled (shareobjects yes) you can use
+# shareobjectspoolsize to control the size of the pool used in order to try
+# object sharing. A bigger pool size will lead to better sharing capabilities.
+# In general you want this value to be at least the double of the number of
+# very common strings you have in your dataset.
+#
+# WARNING: object sharing is experimental, don't enable this feature
+# in production before of Redis 1.0-stable. Still please try this feature in
+# your development environment so that we can test it better.
+shareobjects no
+shareobjectspoolsize 1024

data/test/scheduler_test.rb

@@ -15,7 +15,7 @@ class Resque::SchedulerTest < Test::Unit::TestCase
     assert_equal(0, Resque::Scheduler.rufus_scheduler.all_jobs.size)
 
     Resque.schedule = {:some_ivar_job => {'cron' => "* * * * *", 'class' => 'SomeIvarJob', 'args' => "/tmp"}}
-    Resque::Scheduler.run(false)
+    Resque::Scheduler.load_schedule!
 
     assert_equal(1, Resque::Scheduler.rufus_scheduler.all_jobs.size)
   end

data/test/test_helper.rb

@@ -1,8 +1,54 @@
+
+# Pretty much copied this file from the resque test_helper since we want
+# to do all the same stuff
+
+dir = File.dirname(File.expand_path(__FILE__))
+
 require 'rubygems'
 require 'test/unit'
 require 'mocha'
+require 'resque'
+require File.join(dir, '../lib/resque_scheduler')
 $LOAD_PATH.unshift File.dirname(File.expand_path(__FILE__)) + '/../lib'
-require 'resque_scheduler'
+
+
+#
+# make sure we can run redis
+#
+
+if !system("which redis-server")
+  puts '', "** can't find `redis-server` in your path"
+  puts "** try running `sudo rake install`"
+  abort ''
+end
+
+
+#
+# start our own redis when the tests start,
+# kill it when they end
+#
+
+at_exit do
+  next if $!
+
+  if defined?(MiniTest)
+    exit_code = MiniTest::Unit.new.run(ARGV)
+  else
+    exit_code = Test::Unit::AutoRunner.run
+  end
+
+  pid = `ps -e -o pid,command | grep [r]edis-test`.split(" ")[0]
+  puts "Killing test redis server..."
+  `rm -f #{dir}/dump.rdb`
+  Process.kill("KILL", pid.to_i)
+  exit exit_code
+end
+
+puts "Starting redis for testing at localhost:9736..."
+`redis-server #{dir}/redis-test.conf`
+Resque.redis = 'localhost:9736'
+
+
 
 class SomeJob
   def self.perform(repo_id, path)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: resque-scheduler
 version: !ruby/object:Gem::Version 
-  version: 0.0.1
+  version: 1.0.0
 platform: ruby
 authors: 
 - Ben VandenBos
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-12-21 00:00:00 -08:00
+date: 2010-01-11 00:00:00 -08:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -20,7 +20,7 @@ dependencies:
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
-        version: "0"
+        version: 1.3.0
     version: 
 - !ruby/object:Gem::Dependency 
   name: rufus-scheduler
@@ -52,7 +52,10 @@ dependencies:
       - !ruby/object:Gem::Version 
         version: "0"
     version: 
-description: ""
+description: |-
+  Light weight job scheduling on top of Resque.
+    Adds methods enqueue_at/enqueue_in to schedule jobs in the future.
+    Also supports queueing jobs on a fixed, cron-like schedule.
 email: bvandenbos@gmail.com
 executables: []
 
@@ -66,9 +69,15 @@ files:
 - Rakefile
 - lib/resque/scheduler.rb
 - lib/resque_scheduler.rb
+- lib/resque_scheduler/server.rb
+- lib/resque_scheduler/server/views/delayed.erb
+- lib/resque_scheduler/server/views/delayed_timestamp.erb
+- lib/resque_scheduler/server/views/scheduler.erb
 - lib/resque_scheduler/tasks.rb
 - lib/resque_scheduler/version.rb
 - tasks/resque_scheduler.rake
+- test/delayed_queue_test.rb
+- test/redis-test.conf
 - test/scheduler_test.rb
 - test/test_helper.rb
 has_rdoc: true
@@ -98,7 +107,8 @@ rubyforge_project:
 rubygems_version: 1.3.5
 signing_key: 
 specification_version: 3
-summary: ""
+summary: Light weight job scheduling on top of Resque
 test_files: 
+- test/delayed_queue_test.rb
 - test/scheduler_test.rb
 - test/test_helper.rb