sidekiq-scheduler 0.3.2 → 0.4.0

data/README.md

@@ -5,22 +5,38 @@
 sidekiq-scheduler is an extension to [Sidekiq](http://github.com/mperham/sidekiq)
 that adds support for queueing jobs in the future.
 
-At the moment job scheduling is only supported in a delayed fashion. Replacing cron
-is not the intention of this gem. Delayed jobs are Sidekiq jobs that you want to run
-at some point in the future.
+This table explains the version requirements for redis
 
+| sidekiq-scheduler version | required redis version|
+|:--------------------------|----------------------:|
+| ~> 1.0.0                  | >= 2.2.0              |
+
+Job scheduling is supported in two different way: Recurring (scheduled) and
+Delayed.
+
+Scheduled jobs are like cron jobs, recurring on a regular basis.  Delayed
+jobs are resque jobs that you want to run at some point in the future.
 The syntax is pretty explanatory:
 
     MyWorker.perform_in(5.days, 'arg1', 'arg2') # run a job in 5 days
     # or
     MyWorker.perform_at(5.days.from_now, 'arg1', 'arg2') # run job at a specific time
 
+### Documentation
+
+This README covers what most people need to know. If you're looking for
+details on individual methods, you might want to try the [rdoc](http://rdoc.info/github/yabawock/sidekiq-scheduler/master/frames).
+
+
 ## Installation
 
-    # Rails 3.x: add it to your Gemfile
+    #To install:
+    gem install resque-scheduler
+
+    #If you use a Gemfile:
     gem 'sidekiq-scheduler'
 
-    # Starting the scheduler
+    #Starting the scheduler
     bundle exec sidekiq-scheduler
 
 The scheduler will perform identically to a normal sidekiq worker with
@@ -30,9 +46,12 @@ node but all normal configuration options apply.
 
 NOTE: Since it's currently not possible to hook into the default option
 parsing provided by sidekiq you will need to use a configuration file to
-override the scheduler options. Currently the only option available is
+override the scheduler options.
+Available options are:
 
-    resolution: <seconds between schedule runs>
+    :resolution: <seconds between schedule runs>
+    :schedule: <the schedule to be run>
+    :dynamic: <if true the schedule can we modified in runtime>
 
 The scheduling thread will sleep this many seconds between looking for
 jobs that need moving to the worker queue. The default is 5 seconds
@@ -58,6 +77,9 @@ the scheduler will pull it from the delayed queue and put it in the appropriate
 work queue for the given job. It will then be processed as soon as a worker is 
 available (just like any other Sidekiq job).
 
+The `5.days` syntax will only work if you are using ActiveSupport (Rails). If you
+are not using Rails, just provide `perform_in` with the number of seconds.
+
 NOTE: The job does not fire **exactly** at the time supplied. Rather, once that
 time is in the past, the job moves from the delayed queue to the actual work
 queue and will be completed as workers are free to process it.
@@ -85,6 +107,81 @@ If you have the need to cancel a delayed job, you can do it like this:
     # remove the job with exactly the same parameters:
     MyWorker.remove_delayed(<timestamp>, 'arg1', 'arg2')
 
+## Scheduled Jobs (Recurring Jobs)
+
+Scheduled (or recurring) jobs are logically no different than a standard cron
+job.  They are jobs that run based on a fixed schedule which is set at
+startup.
+
+The schedule is a list of Resque worker classes with arguments and a
+schedule frequency (in crontab syntax).  The schedule is just a hash, but
+is most likely stored in a YAML like so:
+
+    CancelAbandonedOrders:
+      cron: "*/5 * * * *"
+
+    queue_documents_for_indexing:
+      cron: "0 0 * * *"
+      # you can use rufus-scheduler "every" syntax in place of cron if you prefer
+      # every: 1hr
+      # By default the job name (hash key) will be taken as worker class name.
+      # If you want to have a different job name and class name, provide the 'class' option
+      class: QueueDocuments
+      queue: high
+      args:
+      description: "This job queues all content for indexing in solr"
+
+    clear_leaderboards_contributors:
+      cron: "30 6 * * 1"
+      class: ClearLeaderboards
+      queue: low
+      args: contributors
+      description: "This job resets the weekly leaderboard for contributions"
+
+You can provide options to "every" or "cron" via Array:
+
+    clear_leaderboards_moderator:
+      every: ["30s", :first_in => '120s']
+      class: CheckDaemon
+      queue: daemons
+      description: "This job will check Daemon every 30 seconds after 120 seconds after start"
+
+
+NOTE: Six parameter cron's are also supported (as they supported by
+rufus-scheduler which powers the sidekiq-scheduler process).  This allows you
+to schedule jobs per second (ie: "30 * * * * *" would fire a job every 30
+seconds past the minute).
+
+A big shout out to [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler)
+for handling the heavy lifting of the actual scheduling engine.
+
+### Time zones
+
+Note that if you use the cron syntax, this will be interpreted as in the server time zone
+rather than the `config.time_zone` specified in Rails.
+
+You can explicitly specify the time zone that rufus-scheduler will use:
+
+    cron: "30 6 * * 1 Europe/Stockholm"
+
+Also note that `config.time_zone` in Rails allows for a shorthand (e.g. "Stockholm")
+that rufus-scheduler does not accept. If you write code to set the scheduler time zone
+from the `config.time_zone` value, make sure it's the right format, e.g. with:
+
+    ActiveSupport::TimeZone.find_tzinfo(Rails.configuration.time_zone).name
+
+A future version of sidekiq-scheduler may do this for you.
+
+## Using with Testing
+
+Sidekiq uses a jobs array on workers for testing, which is supported by sidekiq-scheduler when you require the test code:
+
+    require 'sidekiq/testing'
+    require 'sidekiq-scheduler/testing'
+    
+    MyWorker.perform_in 5, 'arg1'
+    puts MyWorker.jobs.inspect
+
 ## Note on Patches / Pull Requests
 
 * Fork the project.

data/lib/sidekiq-scheduler/capistrano.rb

@@ -0,0 +1,35 @@
+Capistrano::Configuration.instance.load do
+  before "deploy",        "sidekiq_scheduler:quiet"
+  after "deploy:stop",    "sidekiq_scheduler:stop"
+  after "deploy:start",   "sidekiq_scheduler:start"
+  after "deploy:restart", "sidekiq_scheduler:restart"
+
+  _cset(:sidekiq_timeout) { 10 }
+  _cset(:sidekiq_role) { :app }
+
+  namespace :sidekiq_scheduler do
+
+    desc "Quiet sidekiq-scheduler with sidekiq (stop accepting new work)"
+    task :quiet, :roles => lambda { fetch(:sidekiq_role) } do
+      run "cd #{current_path} && if [ -f #{current_path}/tmp/pids/sidekiq.pid ]; then #{fetch(:bundle_cmd, "bundle")} exec sidekiqctl quiet #{current_path}/tmp/pids/sidekiq.pid ; fi"
+    end
+
+    desc "Stop sidekiq-scheduler with sidekiq"
+    task :stop, :roles => lambda { fetch(:sidekiq_role) } do
+      run "cd #{current_path} && if [ -f #{current_path}/tmp/pids/sidekiq.pid ]; then #{fetch(:bundle_cmd, "bundle")} exec sidekiqctl stop #{current_path}/tmp/pids/sidekiq.pid #{fetch :sidekiq_timeout} ; fi"
+    end
+
+    desc "Start sidekiq-scheduler with sidekiq"
+    task :start, :roles => lambda { fetch(:sidekiq_role) } do
+      rails_env = fetch(:rails_env, "production")
+      run "cd #{current_path} ; nohup #{fetch(:bundle_cmd, "bundle")} exec sidekiq-scheduler -e #{rails_env} -C #{current_path}/config/sidekiq.yml -P #{current_path}/tmp/pids/sidekiq.pid >> #{current_path}/log/sidekiq.log 2>&1 &", :pty => false
+    end
+
+    desc "Restart sidekiq-scheduler with sidekiq"
+    task :restart, :roles => lambda { fetch(:sidekiq_role) } do
+      stop
+      start
+    end
+
+  end
+end

data/lib/sidekiq-scheduler/cli.rb

@@ -11,8 +11,15 @@ module SidekiqScheduler
     end
 
     def run_scheduler
-      scheduler_options = { :scheduler => true, :resolution => 5 }
+      scheduler_options = { :scheduler => true, :resolution => 5, :schedule => nil }
       scheduler_options.merge!(options)
+
+      if options[:config_file]
+        file_options = YAML.load_file(options[:config_file])
+        options.merge!(file_options)
+        options.delete(:config_file)
+      end
+
       scheduler = SidekiqScheduler::Manager.new(scheduler_options)
       scheduler.start!
       run_manager

data/lib/sidekiq-scheduler/manager.rb

@@ -4,6 +4,9 @@ require 'multi_json'
 
 require 'sidekiq/util'
 
+require 'sidekiq/scheduler'
+require 'sidekiq-scheduler/schedule'
+
 module SidekiqScheduler
 
   ##
@@ -16,10 +19,11 @@ module SidekiqScheduler
     include Celluloid
 
     def initialize(options={})
-      logger.info "Booting sidekiq scheduler #{SidekiqScheduler::VERSION} with Redis at #{redis { |r| r.client.location} }"
-      logger.debug { options.inspect }
       @enabled = options[:scheduler]
       @resolution = options[:resolution] || 5
+
+      Sidekiq::Scheduler.dynamic = options[:dynamic] || false
+      Sidekiq.schedule = options[:schedule] if options[:schedule]
     end
 
     def stop
@@ -27,6 +31,14 @@ module SidekiqScheduler
     end
 
     def start
+      #Load the schedule into rufus
+      #If dynamic is set, load that schedule otherwise use normal load
+      if @enabled && Sidekiq::Scheduler.dynamic
+        Sidekiq::Scheduler.reload_schedule!
+      elsif @enabled
+        Sidekiq::Scheduler.load_schedule!
+      end
+
       schedule(true)
     end
 
@@ -68,7 +80,8 @@ module SidekiqScheduler
 
         # Dispatch loop
         loop do
-          break logger.debug('no scheduler queues to process') unless timestamp = find_next_timestamp
+          Sidekiq::Scheduler.update_schedule if Sidekiq::Scheduler.dynamic
+          break unless timestamp = find_next_timestamp
           find_scheduled_work(timestamp)
         end
 

data/lib/sidekiq-scheduler/schedule.rb

@@ -0,0 +1,125 @@
+module SidekiqScheduler
+  module Schedule
+
+    # Accepts a new schedule configuration of the form:
+    #
+    #   {
+    #     "MakeTea" => {
+    #       "every" => "1m" },
+    #     "some_name" => {
+    #       "cron"        => "5/* * * *",
+    #       "class"       => "DoSomeWork",
+    #       "args"        => "work on this string",
+    #       "description" => "this thing works it"s butter off" },
+    #     ...
+    #   }
+    #
+    # Hash keys can be anything and are used to describe and reference
+    # the scheduled job. If the "class" argument is missing, the key
+    # is used implicitly as "class" argument - in the "MakeTea" example,
+    # "MakeTea" is used both as job name and sidekiq worker class.
+    #
+    # :cron can be any cron scheduling string
+    #
+    # :every can be used in lieu of :cron. see rufus-scheduler's 'every' usage
+    # for valid syntax. If :cron is present it will take precedence over :every.
+    #
+    # :class must be a sidekiq worker class. If it is missing, the job name (hash key)
+    # will be used as :class.
+    #
+    # :args can be any yaml which will be converted to a ruby literal and
+    # passed in a params. (optional)
+    #
+    # :rails_envs is the list of envs where the job gets loaded. Envs are
+    # comma separated (optional)
+    #
+    # :description is just that, a description of the job (optional). If
+    # params is an array, each element in the array is passed as a separate
+    # param, otherwise params is passed in as the only parameter to perform.
+    def schedule=(schedule_hash)
+      schedule_hash = prepare_schedule(schedule_hash)
+
+      if Sidekiq::Scheduler.dynamic
+        schedule_hash.each do |name, job_spec|
+          set_schedule(name, job_spec)
+        end
+      end
+      @schedule = schedule_hash
+    end
+
+    def schedule
+      @schedule ||= {}
+    end
+
+    # reloads the schedule from redis
+    def reload_schedule!
+      @schedule = get_schedule
+    end
+
+    # Retrive the schedule configuration for the given name
+    # if the name is nil it returns a hash with all the
+    # names end their schedules.
+    def get_schedule(name = nil)
+      if name.nil?
+        get_all_schedules
+      else
+        encoded_schedule = Sidekiq.redis { |r| r.hget(:schedules, name) }
+        encoded_schedule.nil? ? nil : MultiJson.decode(encoded_schedule)
+      end
+    end
+
+    # gets the schedule as it exists in redis
+    def get_all_schedules
+      schedules = nil
+      if Sidekiq.redis { |r| r.exists(:schedules) }
+        schedules = {}
+
+        Sidekiq.redis { |r| r.hgetall(:schedules) }.tap do |h|
+          h.each do |name, config|
+            schedules[name] = MultiJson.decode(config)
+          end
+        end
+      end
+
+      schedules
+    end
+
+    # Create or update a schedule with the provided name and configuration.
+    #
+    # Note: values for class and custom_job_class need to be strings,
+    # not constants.
+    #
+    #    Resque.set_schedule('some_job', {:class => 'SomeJob',
+    #                                     :every => '15mins',
+    #                                     :queue => 'high',
+    #                                     :args => '/tmp/poop'})
+    def set_schedule(name, config)
+      existing_config = get_schedule(name)
+      unless existing_config && existing_config == config
+        Sidekiq.redis { |r| r.hset(:schedules, name, MultiJson.encode(config)) }
+        Sidekiq.redis { |r| r.sadd(:schedules_changed, name) }
+      end
+      config
+    end
+
+    # remove a given schedule by name
+    def remove_schedule(name)
+      Sidekiq.redis { |r| r.hdel(:schedules, name) }
+      Sidekiq.redis { |r| r.sadd(:schedules_changed, name) }
+    end
+
+    private
+
+    def prepare_schedule(schedule_hash)
+      prepared_hash = {}
+      schedule_hash.each do |name, job_spec|
+        job_spec = job_spec.dup
+        job_spec['class'] = name unless job_spec.key?('class') || job_spec.key?(:class)
+        prepared_hash[name] = job_spec
+      end
+      prepared_hash
+    end
+  end
+end
+
+Sidekiq.extend SidekiqScheduler::Schedule

data/lib/sidekiq-scheduler/version.rb

@@ -1,3 +1,3 @@
 module SidekiqScheduler
-  VERSION = "0.3.2"
+  VERSION = "0.4.0"
 end

data/lib/sidekiq/scheduler.rb

@@ -0,0 +1,163 @@
+require 'rufus/scheduler'
+require 'thwait'
+require 'sidekiq/util'
+require 'sidekiq-scheduler/manager'
+
+module Sidekiq
+  class Scheduler
+    extend Sidekiq::Util
+
+    class << self
+      # If set, will try to update the schulde in the loop
+      attr_accessor :dynamic
+    end
+
+    # the Rufus::Scheduler jobs that are scheduled
+    def self.scheduled_jobs
+      @@scheduled_jobs
+    end
+
+    def self.print_schedule
+      if self.rufus_scheduler
+        logger.info "Scheduling Info\tLast Run"
+        scheduler_jobs = self.rufus_scheduler.all_jobs
+        scheduler_jobs.each do |k, v|
+          logger.info "#{v.t}\t#{v.last}\t"
+        end
+      end
+    end
+
+    # Pulls the schedule from Sidekiq.schedule and loads it into the
+    # rufus scheduler instance
+    def self.load_schedule!
+      logger.info 'Loading Schedule'
+
+      # Need to load the schedule from redis for the first time if dynamic
+      Sidekiq.reload_schedule! if dynamic
+
+      logger.info 'Schedule empty! Set Sidekiq.schedule' if Sidekiq.schedule.empty?
+
+      @@scheduled_jobs = {}
+
+      Sidekiq.schedule.each do |name, config|
+        self.load_schedule_job(name, config)
+      end
+
+      Sidekiq.redis { |r| r.del(:schedules_changed) }
+
+      logger.info 'Schedules Loaded'
+    end
+
+    # modify interval type value to value with options if options available
+    def self.optionizate_interval_value(value)
+      args = value
+      if args.is_a?(::Array)
+        return args.first if args.size > 2 || !args.last.is_a?(::Hash)
+        # symbolize keys of hash for options
+        args[1] = args[1].inject({}) do |m, i|
+          key, value = i
+          m[(key.to_sym rescue key) || key] = value
+          m
+        end
+      end
+      args
+    end
+
+    # Loads a job schedule into the Rufus::Scheduler and stores it in @@scheduled_jobs
+    def self.load_schedule_job(name, config)
+      # If rails_env is set in the config, enforce ENV['RAILS_ENV'] as
+      # required for the jobs to be scheduled.  If rails_env is missing, the
+      # job should be scheduled regardless of what ENV['RAILS_ENV'] is set
+      # to.
+      if config['rails_env'].nil? || self.rails_env_matches?(config)
+        logger.info "Scheduling #{name} "
+        interval_defined = false
+        interval_types = %w{cron every}
+        interval_types.each do |interval_type|
+          if !config[interval_type].nil? && config[interval_type].length > 0
+            args = self.optionizate_interval_value(config[interval_type])
+
+            @@scheduled_jobs[name] = self.rufus_scheduler.send(interval_type, *args) do
+              logger.info "queueing #{config['class']} (#{name})"
+              config.delete(interval_type)
+              self.handle_errors { self.enqueue_from_config(config) }
+            end
+
+            interval_defined = true
+
+            break
+          end
+        end
+
+        unless interval_defined
+          logger.info "no #{interval_types.join(' / ')} found for #{config['class']} (#{name}) - skipping"
+        end
+      end
+    end
+
+    # Returns true if the given schedule config hash matches the current
+    # ENV['RAILS_ENV']
+    def self.rails_env_matches?(config)
+      config['rails_env'] && ENV['RAILS_ENV'] && config['rails_env'].gsub(/\s/,'').split(',').include?(ENV['RAILS_ENV'])
+    end
+
+    def self.handle_errors
+      begin
+        yield
+      rescue Exception => e
+        logger.info "#{e.class.name}: #{e.message}"
+      end
+    end
+
+    # Enqueue a job based on a config hash
+    def self.enqueue_from_config(job_config)
+      job_config['class'] = constantize(job_config['class']) # Sidekiq expects the class to be constantized.
+      job_config['args'] = '' if job_config['args'].nil?
+
+      Sidekiq::Client.push(job_config)
+    end
+
+    def self.rufus_scheduler
+      @rufus_scheduler ||= Rufus::Scheduler.start_new
+    end
+
+    # Stops old rufus scheduler and creates a new one.  Returns the new
+    # rufus scheduler
+    def self.clear_schedule!
+      self.rufus_scheduler.stop
+      @rufus_scheduler = nil
+      @@scheduled_jobs = {}
+      self.rufus_scheduler
+    end
+
+    def self.reload_schedule!
+      logger.info 'Reloading Schedule'
+      self.clear_schedule!
+      self.load_schedule!
+    end
+
+    def self.update_schedule
+      if Sidekiq.redis { |r| r.scard(:schedules_changed) } > 0
+        logger.info 'Updating schedule'
+        Sidekiq.reload_schedule!
+        while schedule_name = Sidekiq.redis { |r| r.spop(:schedules_changed) }
+          if Sidekiq.schedule.keys.include?(schedule_name)
+            self.unschedule_job(schedule_name)
+            self.load_schedule_job(schedule_name, Sidekiq.schedule[schedule_name])
+          else
+            self.unschedule_job(schedule_name)
+          end
+        end
+        logger.info 'Schedules Loaded'
+      end
+    end
+
+    def self.unschedule_job(name)
+      if self.scheduled_jobs[name]
+        logger.debug "Removing schedule #{name}"
+        self.scheduled_jobs[name].unschedule
+        self.scheduled_jobs.delete(name)
+      end
+    end
+  end
+end

data/test/client_test.rb

@@ -4,7 +4,7 @@ require 'timecop'
 class ClientTest < MiniTest::Unit::TestCase
   describe 'with real redis' do
     before do
-      Sidekiq.redis = REDIS
+      Sidekiq.redis = Sidekiq::RedisConnection.create(:url => 'redis://localhost/15', :namespace => 'testy')
       Sidekiq.redis {|c| c.flushdb }
     end
 
@@ -82,7 +82,7 @@ class ClientTest < MiniTest::Unit::TestCase
     end
   end
 
-  describe 'with mock redis' do
+  describe 'redis calls' do
     before do
       @redis = MiniTest::Mock.new
       def @redis.multi; yield; end
@@ -100,9 +100,9 @@ class ClientTest < MiniTest::Unit::TestCase
     end
 
     it 'pushes delayed messages to redis' do
-      @redis.expect :rpush, 1, ['delayed:1331284491', String]
-      @redis.expect :zadd, 1, ['delayed_queue_schedule', 1331284491, 1331284491]
-      Sidekiq::Client.delayed_push('foo', 1331284491, 'class' => 'Foo', 'args' => [1, 2])
+      @redis.expect :rpush, 1, ['delayed:1661284491', String]
+      @redis.expect :zadd, 1, ['delayed_queue_schedule', 1661284491, 1661284491]
+      Sidekiq::Client.delayed_push('foo', 1661284491, 'class' => 'Foo', 'args' => [1, 2])
       @redis.verify
     end
 
@@ -114,17 +114,20 @@ class ClientTest < MiniTest::Unit::TestCase
     end
 
     it 'handles perform_at' do
-      @redis.expect :rpush, 1, ['delayed:1331284491', String]
-      @redis.expect :zadd, 1, ['delayed_queue_schedule', 1331284491, 1331284491]
+
+      #@redis.expect :rpush, 1, ['delayed:1331284491', String]
+      @redis.expect :zadd, 1, ['schedule', "1331284491.0", String]
       MyWorker.perform_at(1331284491, 1, 2)
+
       @redis.verify
     end
 
     it 'handles perform_in' do
+
       Timecop.freeze(Time.now) do
         timestamp = Time.now + 30
-        @redis.expect :rpush, 1, ["delayed:#{timestamp.to_i}", String]
-        @redis.expect :zadd, 1, ['delayed_queue_schedule', timestamp.to_i, timestamp.to_i]
+        #@redis.expect :rpush, 1, ["delayed:#{timestamp.to_i}", String]
+        @redis.expect :zadd, 1, ['schedule', "#{timestamp.to_f}", String]
         MyWorker.perform_in(30, 1, 2)
         @redis.verify
       end

data/test/config.yml

@@ -8,3 +8,4 @@
 :queues:
   - [often, 2]
   - [seldom, 1]
+---

data/test/lib/sidekiq/scheduler_test.rb

@@ -0,0 +1,286 @@
+require 'test_helper'
+
+class ManagerTest < MiniTest::Unit::TestCase
+  describe 'Sidekiq::Scheduler' do
+
+    before do
+      Sidekiq::Scheduler.dynamic = false
+      Sidekiq.redis { |r| r.del(:schedules) }
+      Sidekiq.redis { |r| r.del(:schedules_changed) }
+      Sidekiq::Scheduler.clear_schedule!
+      Sidekiq::Scheduler.send(:class_variable_set, :@@scheduled_jobs, {})
+    end
+
+    it 'enqueue constantizes' do
+      # The job should be loaded, since a missing rails_env means ALL envs.
+      ENV['RAILS_ENV'] = 'production'
+      config = {
+          'cron' => '* * * * *',
+          'class' => 'SomeRealClass',
+          'args' => '/tmp'
+      }
+
+      Sidekiq::Client.expects(:push).with(
+          {
+              'cron' => '* * * * *',
+              'class' => SomeRealClass,
+              'args' => '/tmp'
+          }
+      )
+      Sidekiq::Scheduler.enqueue_from_config(config)
+    end
+
+    it 'enqueue_from_config respects queue params' do
+      config = {
+          'cron' => '* * * * *',
+          'class' => 'SomeIvarJob',
+          'queue' => 'high'
+      }
+
+      Sidekiq::Client.expects(:push).with(
+          {
+              'cron' => '* * * * *',
+              'class' => SomeIvarJob,
+              'args' => '',
+              'queue' => 'high'
+          }
+      )
+
+      Sidekiq::Scheduler.enqueue_from_config(config)
+    end
+
+    it 'config makes it into the rufus_scheduler' do
+      assert_equal(0, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      Sidekiq.schedule = {
+          :some_ivar_job => {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp'
+          }
+      }
+
+
+      Sidekiq::Scheduler.load_schedule!
+
+      assert_equal(1, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      assert Sidekiq::Scheduler.scheduled_jobs.include?(:some_ivar_job)
+    end
+
+    it 'can reload schedule' do
+      Sidekiq::Scheduler.dynamic = true
+      Sidekiq.schedule = {
+          :some_ivar_job => {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp'
+          }
+      }
+
+      Sidekiq::Scheduler.load_schedule!
+
+      assert_equal(1, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      assert Sidekiq::Scheduler.scheduled_jobs.include?(:some_ivar_job)
+
+      Sidekiq.redis { |r| r.del(:schedules) }
+      Sidekiq.redis do |r|
+        r.hset(
+            :schedules,
+            'some_ivar_job2',
+            MultiJson.encode(
+                {
+                    'cron' => '* * * * *',
+                    'class' => 'SomeIvarJob',
+                    'args' => '/tmp/2'
+                }
+            )
+        )
+      end
+
+      Sidekiq::Scheduler.reload_schedule!
+
+      assert_equal(1, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+
+      assert_equal '/tmp/2', Sidekiq.schedule['some_ivar_job2']['args']
+      assert Sidekiq::Scheduler.scheduled_jobs.include?('some_ivar_job2')
+    end
+
+    it 'load_schedule_job loads a schedule' do
+      Sidekiq::Scheduler.load_schedule_job(
+          'some_ivar_job',
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp'
+          }
+      )
+
+      assert_equal(1, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      assert_equal(1, Sidekiq::Scheduler.scheduled_jobs.size)
+      assert Sidekiq::Scheduler.scheduled_jobs.keys.include?('some_ivar_job')
+    end
+
+    it 'load_schedule_job with every with options' do
+      Sidekiq::Scheduler.load_schedule_job(
+          'some_ivar_job',
+          {
+              'every' => ['30s', {'first_in' => '60s'}],
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp'
+          }
+      )
+
+      assert_equal(1, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      assert_equal(1, Sidekiq::Scheduler.scheduled_jobs.size)
+      assert Sidekiq::Scheduler.scheduled_jobs.keys.include?('some_ivar_job')
+      assert Sidekiq::Scheduler.scheduled_jobs['some_ivar_job'].params.keys.include?(:first_in)
+    end
+
+    it 'load_schedule_job with cron with options' do
+      Sidekiq::Scheduler.load_schedule_job(
+          'some_ivar_job',
+          {
+              'cron' => ['* * * * *', {'allow_overlapping' => 'true'}],
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp'
+          }
+      )
+
+      assert_equal(1, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      assert_equal(1, Sidekiq::Scheduler.scheduled_jobs.size)
+      assert Sidekiq::Scheduler.scheduled_jobs.keys.include?('some_ivar_job')
+      assert Sidekiq::Scheduler.scheduled_jobs['some_ivar_job'].params.keys.include?(:allow_overlapping)
+    end
+
+    it 'does not load the schedule without cron' do
+      Sidekiq::Scheduler.load_schedule_job(
+          'some_ivar_job',
+          {
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp'
+          }
+      )
+
+      assert_equal(0, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      assert_equal(0, Sidekiq::Scheduler.scheduled_jobs.size)
+      assert !Sidekiq::Scheduler.scheduled_jobs.keys.include?('some_ivar_job')
+    end
+
+    it 'does not load the schedule with an empty cron' do
+      Sidekiq::Scheduler.load_schedule_job(
+          'some_ivar_job',
+          {
+              'cron' => '',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp'
+          }
+      )
+
+      assert_equal(0, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      assert_equal(0, Sidekiq::Scheduler.scheduled_jobs.size)
+      assert !Sidekiq::Scheduler.scheduled_jobs.keys.include?('some_ivar_job')
+    end
+
+    it 'update_schedule' do
+      Sidekiq::Scheduler.dynamic = true
+      Sidekiq.schedule =
+          {
+              'some_ivar_job'    => {'cron' => '* * * * *', 'class' => 'SomeIvarJob', 'args' => '/tmp'},
+              'another_ivar_job' => {'cron' => '* * * * *', 'class' => 'SomeIvarJob', 'args' => '/tmp/5'},
+              'stay_put_job'     => {'cron' => '* * * * *', 'class' => 'SomeJob', 'args' => '/tmp'}
+          }
+
+      Sidekiq::Scheduler.load_schedule!
+
+      Sidekiq.set_schedule(
+          'some_ivar_job',
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp/2'
+          }
+      )
+      Sidekiq.set_schedule(
+          'new_ivar_job',
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeJob',
+              'args' => '/tmp/3'
+          }
+      )
+      Sidekiq.set_schedule(
+          'stay_put_job',
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeJob',
+              'args' => '/tmp'
+          }
+      )
+      Sidekiq.remove_schedule('another_ivar_job')
+
+      Sidekiq::Scheduler.update_schedule
+
+      assert_equal(3, Sidekiq::Scheduler.rufus_scheduler.all_jobs.size)
+      assert_equal(3, Sidekiq::Scheduler.scheduled_jobs.size)
+
+
+      %w(some_ivar_job new_ivar_job stay_put_job).each do |job_name|
+        assert Sidekiq::Scheduler.scheduled_jobs.keys.include?(job_name)
+        assert Sidekiq.schedule.keys.include?(job_name)
+      end
+      assert !Sidekiq::Scheduler.scheduled_jobs.keys.include?('another_ivar_job')
+      assert !Sidekiq.schedule.keys.include?('another_ivar_job')
+      assert_equal 0, Sidekiq.redis { |r| r.scard(:schedules_changed) }
+    end
+
+    it 'update_schedule with mocks' do
+      Sidekiq::Scheduler.dynamic = true
+      Sidekiq.schedule = {
+          'some_ivar_job' => {'cron' => '* * * * *', 'class' => 'SomeIvarJob', 'args' => '/tmp'},
+          'another_ivar_job'  => {'cron' => '* * * * *', 'class' => 'SomeIvarJob', 'args' => '/tmp/5'},
+          'stay_put_job'  => {'cron' => '* * * * *', 'class' => 'SomeJob', 'args' => '/tmp'}
+      }
+
+      Sidekiq::Scheduler.load_schedule!
+
+      Sidekiq::Scheduler.rufus_scheduler.expects(:unschedule).with(Sidekiq::Scheduler.scheduled_jobs['some_ivar_job'].job_id)
+      Sidekiq::Scheduler.rufus_scheduler.expects(:unschedule).with(Sidekiq::Scheduler.scheduled_jobs['another_ivar_job'].job_id)
+
+      Sidekiq.set_schedule(
+          'some_ivar_job',
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp/2'
+          }
+      )
+      Sidekiq.set_schedule(
+          'new_ivar_job',
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeJob',
+              'args' => '/tmp/3'
+          }
+      )
+      Sidekiq.set_schedule(
+          'stay_put_job',
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeJob',
+              'args' => '/tmp'
+          }
+      )
+      Sidekiq.remove_schedule('another_ivar_job')
+
+      Sidekiq::Scheduler.update_schedule
+
+      assert_equal(3, Sidekiq::Scheduler.scheduled_jobs.size)
+      %w(some_ivar_job new_ivar_job stay_put_job).each do |job_name|
+        assert Sidekiq::Scheduler.scheduled_jobs.keys.include?(job_name)
+        assert Sidekiq.schedule.keys.include?(job_name)
+      end
+      assert !Sidekiq::Scheduler.scheduled_jobs.keys.include?('another_ivar_job')
+      assert !Sidekiq.schedule.keys.include?('another_ivar_job')
+      assert_equal 0, Sidekiq.redis { |r| r.scard(:schedules_changed) }
+    end
+  end
+end

data/test/manager_test.rb

@@ -5,7 +5,7 @@ require 'sidekiq/manager'
 class ManagerTest < MiniTest::Unit::TestCase
   describe 'with redis' do
     before do
-      Sidekiq.redis = REDIS
+      Sidekiq.redis = Sidekiq::RedisConnection.create(:url => 'redis://localhost/15', :namespace => 'testy')
       Sidekiq.redis {|c| c.flushdb }
       @scheduler = SidekiqScheduler::Manager.new
       $processed = 0

data/test/schedule_test.rb

@@ -0,0 +1,124 @@
+require 'test_helper'
+
+class ScheduleTest < MiniTest::Unit::TestCase
+  #class ScheduleManager
+  #  extend SidekiqScheduler::ScheduleManager
+  #end
+
+
+  describe 'SidekiqScheduler::Schedule' do
+    it 'schedule= sets the schedule' do
+      Sidekiq::Scheduler.dynamic = true
+      Sidekiq.schedule = {
+          'my_ivar_job' => {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp/75'
+          }
+      }
+
+      assert_equal(
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp/75'
+          },
+          MultiJson.decode(Sidekiq.redis { |r|
+            r.hget(:schedules, 'my_ivar_job')
+          })
+      )
+    end
+
+    it "schedule= uses job name as 'class' argument if it's missing" do
+      Sidekiq::Scheduler.dynamic = true
+      Sidekiq.schedule = {
+          'SomeIvarJob' => {
+              'cron' => '* * * * *',
+              'args' => '/tmp/75'
+          }
+      }
+
+      assert_equal(
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp/75'
+          },
+          MultiJson.decode(Sidekiq.redis { |r| r.hget(:schedules, 'SomeIvarJob') })
+      )
+      assert_equal('SomeIvarJob', Sidekiq.schedule['SomeIvarJob']['class'])
+    end
+
+    it 'schedule= does not mutate argument' do
+      schedule = {
+          'SomeIvarJob' => {
+              'cron' => '* * * * *',
+              'args' => '/tmp/75'
+          }
+      }
+      Sidekiq.schedule = schedule
+      assert !schedule['SomeIvarJob'].key?('class')
+    end
+
+    it 'set_schedule can set an individual schedule' do
+      Sidekiq.set_schedule(
+          'some_ivar_job',
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp/22'
+          }
+      )
+      assert_equal(
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp/22'
+          },
+          MultiJson.decode(Sidekiq.redis { |r| r.hget(:schedules, 'some_ivar_job') })
+      )
+      assert Sidekiq.redis { |r| r.sismember(:schedules_changed, 'some_ivar_job') }
+    end
+
+    it 'get_schedule returns a schedule' do
+      Sidekiq.redis { |r| r.hset(
+          :schedules,
+          'some_ivar_job2',
+          MultiJson.encode(
+              {
+                  'cron' => '* * * * *',
+                  'class' => 'SomeIvarJob',
+                  'args' => '/tmp/33'
+              }
+          )
+      ) }
+      assert_equal(
+          {
+              'cron' => '* * * * *',
+              'class' => 'SomeIvarJob',
+              'args' => '/tmp/33'
+          },
+          Sidekiq.get_schedule('some_ivar_job2')
+      )
+    end
+
+    it 'remove_schedule removes a schedule' do
+      Sidekiq.redis do |r|
+        r.hset(
+            :schedules,
+            'some_ivar_job3',
+            MultiJson.encode(
+                {
+                    'cron' => '* * * * *',
+                    'class' => 'SomeIvarJob',
+                    'args' => '/tmp/44'
+                }
+            )
+        )
+      end
+      Sidekiq.remove_schedule('some_ivar_job3')
+      assert_equal nil, Sidekiq.redis{ |r| r.hget(:schedules, 'some_ivar_job3') }
+      assert Sidekiq.redis{ |r| r.sismember(:schedules_changed, 'some_ivar_job3') }
+    end
+  end
+end

data/test/test_helper.rb

@@ -2,13 +2,45 @@ require 'minitest/unit'
 require 'minitest/pride'
 require 'minitest/autorun'
 require 'sidekiq-scheduler'
+require 'mocha'
+require 'multi_json'
+require 'mock_redis'
 
 require 'sidekiq'
 require 'sidekiq/util'
-Sidekiq::Util.logger.level = Logger::ERROR
+if Sidekiq.respond_to?(:logger)
+  Sidekiq.logger.level = Logger::ERROR
+else
+  Sidekiq::Util.logger.level = Logger::ERROR
+end
 
 # Load support files
 Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each { |f| require f }
 
 require 'sidekiq/redis_connection'
-REDIS = Sidekiq::RedisConnection.create(:url => "redis://localhost/15", :namespace => 'testy')
+
+#Setup redis mock to avoid having a dependency
+# with redis server during tests
+$redis = ConnectionPool.new(:timeout => 1, :size => 1) { MockRedis.new }
+Sidekiq.redis = $redis
+
+class MiniTest::Spec
+  before :each do
+    $redis.with_connection { |conn| conn.flushdb }
+  end
+end
+
+class SomeJob
+  include Sidekiq::Worker
+  def self.perform(repo_id, path)
+  end
+end
+
+class SomeIvarJob < SomeJob
+  sidekiq_options :queue => :ivar
+end
+
+class SomeRealClass
+  include Sidekiq::Worker
+  sidekiq_options :queue => :some_real_queue
+end

data/test/testing_test.rb

@@ -11,7 +11,7 @@ class TestingTest < MiniTest::Unit::TestCase
         assert_equal 0, DirectWorker.jobs.size
         assert DirectWorker.perform_at(1331759054, 1, 2)
         assert_equal 1, DirectWorker.jobs.size
-        assert_equal 1331759054, DirectWorker.jobs[0]['timestamp']
+        assert_equal 1331759054, DirectWorker.jobs[0]['at']
         DirectWorker.jobs.clear
 
         # perform_in
@@ -20,7 +20,7 @@ class TestingTest < MiniTest::Unit::TestCase
           assert_equal 0, DirectWorker.jobs.size
           assert DirectWorker.perform_in(30, 1, 2)
           assert_equal 1, DirectWorker.jobs.size
-          assert_equal timestamp.to_i, DirectWorker.jobs[0]['timestamp']
+          assert_equal timestamp.to_f, DirectWorker.jobs[0]['at']
         end
       ensure
         # Undo override

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq-scheduler
 version: !ruby/object:Gem::Version
-  version: 0.3.2
+  version: 0.4.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,22 +9,59 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-04-24 00:00:00.000000000 Z
+date: 2012-08-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sidekiq
-  requirement: &70233824413420 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '2.0'
   type: :runtime
   prerelease: false
-  version_requirements: *70233824413420
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: redis
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.1
+- !ruby/object:Gem::Dependency
+  name: rufus-scheduler
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '2.0'
 - !ruby/object:Gem::Dependency
   name: rake
-  requirement: &70233824411780 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,10 +69,15 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70233824411780
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: timecop
-  requirement: &70233834978940 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -43,7 +85,60 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70233834978940
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: mocha
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: mock_redis
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Light weight job scheduling extension for Sidekiq that adds support for
   queueing items in the future.
 email:
@@ -54,9 +149,12 @@ extensions: []
 extra_rdoc_files: []
 files:
 - bin/sidekiq-scheduler
+- lib/sidekiq/scheduler.rb
+- lib/sidekiq-scheduler/capistrano.rb
 - lib/sidekiq-scheduler/cli.rb
 - lib/sidekiq-scheduler/client.rb
 - lib/sidekiq-scheduler/manager.rb
+- lib/sidekiq-scheduler/schedule.rb
 - lib/sidekiq-scheduler/testing.rb
 - lib/sidekiq-scheduler/version.rb
 - lib/sidekiq-scheduler/worker.rb
@@ -69,7 +167,9 @@ files:
 - test/client_test.rb
 - test/config.yml
 - test/fake_env.rb
+- test/lib/sidekiq/scheduler_test.rb
 - test/manager_test.rb
+- test/schedule_test.rb
 - test/support/direct_worker.rb
 - test/support/my_worker.rb
 - test/test_helper.rb
@@ -88,7 +188,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 4377536769909250181
+      hash: -1906038753253799220
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -97,10 +197,10 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 4377536769909250181
+      hash: -1906038753253799220
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.17
+rubygems_version: 1.8.23
 signing_key: 
 specification_version: 3
 summary: Light weight job scheduling extension for Sidekiq
@@ -109,7 +209,9 @@ test_files:
 - test/client_test.rb
 - test/config.yml
 - test/fake_env.rb
+- test/lib/sidekiq/scheduler_test.rb
 - test/manager_test.rb
+- test/schedule_test.rb
 - test/support/direct_worker.rb
 - test/support/my_worker.rb
 - test/test_helper.rb