resque-scheduler 2.0.0.g → 2.0.0.h

data/HISTORY.md

@@ -1,6 +1,6 @@
-## 2.0.0.g (2012-01-24)
+## 2.0.0.h (2012-03-19)
 
-* Fixed "already initialized constant" warning (skddc)
+* Adding plugin support with hooks (andreas)
 
 ## 2.0.0.f (2011-11-03)
 

data/README.markdown

@@ -128,12 +128,6 @@ down when a particular job is supposed to be queue, they will simply "catch up"
 once they are started again.  Jobs are guaranteed to run (provided they make it
 into the delayed queue) after their given queue_at time has passed.
 
-Similar to `before_enqueue` and `after_enqueue` hooks provided in Resque 
-(>= 1.19.1), your jobs can specify one or more `before_schedule` and
-`after_schedule` hooks, to be run before or after scheduling. If *any* of your
-`before_schedule` hooks returns `false`, the job will *not* be scheduled and
-your `after_schedule` hooks will *not* be run.
-
 One other thing to note is that insertion into the delayed queue is O(log(n))
 since the jobs are stored in a redis sorted set (zset).  I can't imagine this
 being an issue for someone since redis is stupidly fast even at log(n), but full
@@ -158,10 +152,15 @@ The schedule is a list of Resque worker classes with arguments and a
 schedule frequency (in crontab syntax).  The schedule is just a hash, but
 is most likely stored in a YAML like so:
 
+    CancelAbandonedOrders:
+      cron: "*/5 * * * *"
+
     queue_documents_for_indexing:
       cron: "0 0 * * *"
       # you can use rufus-scheduler "every" syntax in place of cron if you prefer
       # every: 1hr
+      # By default the job name (hash key) will be taken as worker class name.
+      # If you want to have a different job name and class name, provide the 'class' option
       class: QueueDocuments
       queue: high
       args: 
@@ -180,6 +179,15 @@ defined.  If you're getting "uninitialized constant" errors, you probably
 need to either set the queue in the schedule or require your jobs in your
 "resque:setup" rake task.
 
+You can provide options to "every" or "cron" via Array:
+
+    clear_leaderboards_moderator:
+      every: ["30s", :first_in => '120s']
+      class: CheckDaemon
+      queue: daemons
+      description: "This job will check Daemon every 30 seconds after 120 seconds after start"
+
+
 NOTE: Six parameter cron's are also supported (as they supported by
 rufus-scheduler which powers the resque-scheduler process).  This allows you
 to schedule jobs per second (ie: "30 * * * * *" would fire a job every 30
@@ -206,6 +214,21 @@ from the `config.time_zone` value, make sure it's the right format, e.g. with:
 
 A future version of resque-scheduler may do this for you.
 
+#### Hooks
+
+Similar to the `before_enqueue`- and `after_enqueue`-hooks provided in Resque
+(>= 1.19.1), your jobs can specify one or more of the following hooks:
+
+* `before_schedule`: Called with the job args before a job is placed on
+  the delayed queue. If the hook returns `false`, the job will not be placed on
+  the queue.
+* `after_schedule`: Called with the job args after a job is placed on the
+  delayed queue. Any exception raised propagates up to the code with queued the
+  job.
+* `before_delayed_enqueue`: Called with the job args after the job has been
+  removed from the delayed queue, but not yet put on a normal queue. It is
+  called before `before_enqueue`-hooks, and on the same job instance as the
+  `before_enqueue`-hooks will be invoked on. Return values are ignored.
 
 #### Support for resque-status (and other custom jobs)
 

data/lib/resque/scheduler.rb

@@ -103,6 +103,21 @@ module Resque
         Resque.redis.del(:schedules_changed)
         procline "Schedules Loaded"
       end
+      
+      # modify interval type value to value with options if options available
+      def optionizate_interval_value(value)
+        args = value
+        if args.is_a?(::Array)
+          return args.first if args.size > 2 || !args.last.is_a?(::Hash)
+          # symbolize keys of hash for options
+          args[1] = args[1].inject({}) do |m, i|
+            key, value = i
+            m[(key.to_sym rescue key) || key] = value
+            m
+          end
+        end
+        args
+      end
 
       # Loads a job schedule into the Rufus::Scheduler and stores it in @@scheduled_jobs
       def load_schedule_job(name, config)
@@ -116,7 +131,8 @@ module Resque
           interval_types = %w{cron every}
           interval_types.each do |interval_type|
             if !config[interval_type].nil? && config[interval_type].length > 0
-              @@scheduled_jobs[name] = rufus_scheduler.send(interval_type, config[interval_type]) do
+              args = optionizate_interval_value(config[interval_type])
+              @@scheduled_jobs[name] = rufus_scheduler.send(interval_type, *args) do
                 log! "queueing #{config['class']} (#{name})"
                 handle_errors { enqueue_from_config(config) }
               end
@@ -202,6 +218,7 @@ module Resque
           # for non-existent classes (for example: running scheduler in
           # one app that schedules for another
           if Class === klass
+            ResqueScheduler::Plugin.run_before_delayed_enqueue_hooks(klass, *params)
             Resque.enqueue(klass, *params)
           else
             # This will not run the before_hooks in rescue, but will at least

data/lib/resque_scheduler.rb

@@ -4,27 +4,36 @@ require 'resque/server'
 require 'resque_scheduler/version'
 require 'resque/scheduler'
 require 'resque_scheduler/server'
+require 'resque_scheduler/plugin'
 
 module ResqueScheduler
 
   #
   # Accepts a new schedule configuration of the form:
   #
-  #   {'some_name' => {"cron" => "5/* * * *",
-  #                  "class" => "DoSomeWork",
-  #                  "args" => "work on this string",
-  #                  "description" => "this thing works it"s butter off"},
-  #    ...}
+  #   {
+  #     "MakeTea" => {
+  #       "every" => "1m" },
+  #     "some_name" => {
+  #       "cron"        => "5/* * * *",
+  #       "class"       => "DoSomeWork",
+  #       "args"        => "work on this string",
+  #       "description" => "this thing works it"s butter off" },
+  #     ...
+  #   }
   #
-  # 'some_name' can be anything and is used only to describe and reference 
-  # the scheduled job
+  # Hash keys can be anything and are used to describe and reference
+  # the scheduled job. If the "class" argument is missing, the key
+  # is used implicitly as "class" argument - in the "MakeTea" example,
+  # "MakeTea" is used both as job name and resque worker class.
   #
-  # :cron can be any cron scheduling string :job can be any resque job class
+  # :cron can be any cron scheduling string
   #
   # :every can be used in lieu of :cron. see rufus-scheduler's 'every' usage
   # for valid syntax. If :cron is present it will take precedence over :every.
   #
-  # :class must be a resque worker class
+  # :class must be a resque worker class. If it is missing, the job name (hash key)
+  # will be used as :class.
   #
   # :args can be any yaml which will be converted to a ruby literal and
   # passed in a params. (optional)
@@ -36,6 +45,8 @@ module ResqueScheduler
   # params is an array, each element in the array is passed as a separate
   # param, otherwise params is passed in as the only parameter to perform.
   def schedule=(schedule_hash)
+    schedule_hash = prepare_schedule(schedule_hash)
+
     if Resque::Scheduler.dynamic
       schedule_hash.each do |name, job_spec|
         set_schedule(name, job_spec)
@@ -109,16 +120,11 @@ module ResqueScheduler
   # a queue in which the job will be placed after the 
   # timestamp has passed.
   def enqueue_at_with_queue(queue, timestamp, klass, *args)
-    before_hooks = before_schedule_hooks(klass).collect do |hook|
-      klass.send(hook,*args)
-    end
-    return false if before_hooks.any? { |result| result == false }
+    return false unless Plugin.run_before_schedule_hooks(klass, *args)
     
     delayed_push(timestamp, job_to_hash_with_queue(queue, klass, args))
     
-    after_schedule_hooks(klass).collect do |hook|
-      klass.send(hook,*args)
-    end
+    Plugin.run_after_schedule_hooks(klass, *args)
   end
 
   # Identical to enqueue_at but takes number_of_seconds_from_now
@@ -264,12 +270,14 @@ module ResqueScheduler
       end
     end
 
-    def before_schedule_hooks(klass)
-      klass.methods.grep(/^before_schedule/).sort
-    end
-
-    def after_schedule_hooks(klass)
-      klass.methods.grep(/^after_schedule/).sort
+    def prepare_schedule(schedule_hash)
+      prepared_hash = {}
+      schedule_hash.each do |name, job_spec|
+        job_spec = job_spec.dup
+        job_spec['class'] = name unless job_spec.key?('class') || job_spec.key?(:class)
+        prepared_hash[name] = job_spec
+      end
+      prepared_hash
     end
 
 end

data/lib/resque_scheduler/plugin.rb

@@ -0,0 +1,25 @@
+module ResqueScheduler
+  module Plugin
+    extend self
+    def hooks(job, pattern)
+      job.methods.grep(/^#{pattern}/).sort
+    end
+
+    def run_hooks(job, pattern, *args)
+      results = hooks(job, pattern).collect do |hook|
+        job.send(hook, *args)
+      end
+
+      results.all? { |result| result != false }
+    end
+
+    def method_missing(method_name, *args, &block)
+      if method_name =~ /^run_(.*)_hooks$/
+        job = args.shift
+        run_hooks job, $1, *args
+      else
+        super
+      end
+    end
+  end
+end

data/lib/resque_scheduler/version.rb

@@ -1,3 +1,3 @@
 module ResqueScheduler
-  VERSION = '2.0.0.g'
+  VERSION = '2.0.0.h'
 end

data/test/scheduler_hooks_test.rb

@@ -1,52 +1,23 @@
 require File.dirname(__FILE__) + '/test_helper'
 
 context "scheduling jobs with hooks" do
-  class JobThatCannotBeScheduledWithoutArguments < Resque::Job
-    @queue = :job_that_cannot_be_scheduled_without_arguments
-    def self.perform(*x);end
-    def self.before_schedule_return_nil_if_arguments_not_supplied(*args)
-      counters[:before_schedule] += 1
-      return false if args.empty?
-    end
-
-    def self.after_schedule_do_something(*args)
-      counters[:after_schedule] += 1
-    end
-
-    class << self
-      def counters
-        @counters ||= Hash.new{|h,k| h[k]=0}
-      end
-      def clean
-        counters.clear
-        self
-      end
-    end
-  end
-  
   setup do
-    Resque::Scheduler.dynamic = false
-    Resque.redis.del(:schedules)
-    Resque.redis.del(:schedules_changed)
-    Resque::Scheduler.mute = true
-    Resque::Scheduler.clear_schedule!
-    Resque::Scheduler.send(:class_variable_set, :@@scheduled_jobs, {})
+    Resque.redis.flushall
   end
 
-  test "before_schedule hook that does not return false should not block" do
-    enqueue_time = Time.now + 12
-    Resque.enqueue_at(enqueue_time, JobThatCannotBeScheduledWithoutArguments.clean, :foo)
-    assert_equal(1, Resque.delayed_timestamp_size(enqueue_time.to_i), "delayed queue should have one entry now")
-    assert_equal(1, JobThatCannotBeScheduledWithoutArguments.counters[:before_schedule], 'before_schedule was not run')
-    assert_equal(1, JobThatCannotBeScheduledWithoutArguments.counters[:after_schedule], 'after_schedule was not run')
+  test "before_schedule hook that does not return false should be enqueued" do
+    enqueue_time = Time.now
+    SomeRealClass.expects(:before_schedule_example).with(:foo)
+    SomeRealClass.expects(:after_schedule_example).with(:foo)
+    Resque.enqueue_at(enqueue_time.to_i, SomeRealClass, :foo)
+    assert_equal(1, Resque.delayed_timestamp_size(enqueue_time.to_i), "job should be enqueued")
   end
 
-  test "before_schedule hook that returns false should block" do
-    enqueue_time = Time.now + 60
-    assert_equal(0, JobThatCannotBeScheduledWithoutArguments.clean.counters[:before_schedule], 'before_schedule should be zero')
-    Resque.enqueue_at(enqueue_time, JobThatCannotBeScheduledWithoutArguments.clean)
-    assert_equal(0, Resque.delayed_timestamp_size(enqueue_time.to_i), "job should not have been put in queue")
-    assert_equal(1, JobThatCannotBeScheduledWithoutArguments.counters[:before_schedule], 'before_schedule was not run')
-    assert_equal(0, JobThatCannotBeScheduledWithoutArguments.counters[:after_schedule], 'after_schedule was run')
+  test "before_schedule hook that returns false should not be enqueued" do
+    enqueue_time = Time.now
+    SomeRealClass.expects(:before_schedule_example).with(:foo).returns(false)
+    SomeRealClass.expects(:after_schedule_example).never
+    Resque.enqueue_at(enqueue_time.to_i, SomeRealClass, :foo)
+    assert_equal(0, Resque.delayed_timestamp_size(enqueue_time.to_i), "job should not be enqueued")
   end
 end

data/test/scheduler_test.rb

@@ -25,7 +25,9 @@ context "Resque::Scheduler" do
     config = {'cron' => "* * * * *", 'class' => 'SomeRealClass', 'args' => "/tmp"}
 
     Resque::Job.expects(:create).with(SomeRealClass.queue, SomeRealClass, '/tmp')
-    SomeRealClass.expects(:after_enqueue_example)
+    SomeRealClass.expects(:before_delayed_enqueue_example).with("/tmp")
+    SomeRealClass.expects(:before_enqueue_example).with("/tmp")
+    SomeRealClass.expects(:after_enqueue_example).with("/tmp")
 
     Resque::Scheduler.enqueue_from_config(config)
   end
@@ -69,6 +71,24 @@ context "Resque::Scheduler" do
     assert_equal(1, Resque::Scheduler.scheduled_jobs.size)
     assert Resque::Scheduler.scheduled_jobs.keys.include?("some_ivar_job")
   end
+  
+  test "load_schedule_job with every with options" do 
+    Resque::Scheduler.load_schedule_job("some_ivar_job", {'every' => ['30s', {'first_in' => '60s'}], 'class' => 'SomeIvarJob', 'args' => "/tmp"})
+
+    assert_equal(1, Resque::Scheduler.rufus_scheduler.all_jobs.size)
+    assert_equal(1, Resque::Scheduler.scheduled_jobs.size)
+    assert Resque::Scheduler.scheduled_jobs.keys.include?("some_ivar_job")
+    assert Resque::Scheduler.scheduled_jobs["some_ivar_job"].params.keys.include?(:first_in)
+  end
+  
+  test "load_schedule_job with cron with options" do 
+    Resque::Scheduler.load_schedule_job("some_ivar_job", {'cron' => ['* * * * *', {'allow_overlapping' => 'true'}], 'class' => 'SomeIvarJob', 'args' => "/tmp"})
+
+    assert_equal(1, Resque::Scheduler.rufus_scheduler.all_jobs.size)
+    assert_equal(1, Resque::Scheduler.scheduled_jobs.size)
+    assert Resque::Scheduler.scheduled_jobs.keys.include?("some_ivar_job")
+    assert Resque::Scheduler.scheduled_jobs["some_ivar_job"].params.keys.include?(:allow_overlapping)
+  end
 
   test "load_schedule_job without cron" do
     Resque::Scheduler.load_schedule_job("some_ivar_job", {'class' => 'SomeIvarJob', 'args' => "/tmp"})
@@ -165,6 +185,24 @@ context "Resque::Scheduler" do
       Resque.decode(Resque.redis.hget(:schedules, "my_ivar_job")))
   end
 
+  test "schedule= uses job name as 'class' argument if it's missing" do
+    Resque::Scheduler.dynamic = true
+    Resque.schedule = {"SomeIvarJob" => {
+      'cron' => "* * * * *", 'args' => "/tmp/75"
+    }}
+    assert_equal({'cron' => "* * * * *", 'class' => 'SomeIvarJob', 'args' => "/tmp/75"},
+      Resque.decode(Resque.redis.hget(:schedules, "SomeIvarJob")))
+    assert_equal('SomeIvarJob', Resque.schedule['SomeIvarJob']['class'])
+  end
+
+  test "schedule= does not mutate argument" do
+    schedule = {"SomeIvarJob" => {
+      'cron' => "* * * * *", 'args' => "/tmp/75"
+    }}
+    Resque.schedule = schedule
+    assert !schedule['SomeIvarJob'].key?('class')
+  end
+
   test "set_schedule can set an individual schedule" do
     Resque.set_schedule("some_ivar_job", {
       'cron' => "* * * * *", 'class' => 'SomeIvarJob', 'args' => "/tmp/22"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: resque-scheduler
 version: !ruby/object:Gem::Version
-  version: 2.0.0.g
+  version: 2.0.0.h
   prerelease: 6
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-01-24 00:00:00.000000000Z
+date: 2012-03-19 00:00:00.000000000Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
-  requirement: &2160604100 !ruby/object:Gem::Requirement
+  requirement: &2169473600 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,10 +21,10 @@ dependencies:
         version: 1.0.0
   type: :development
   prerelease: false
-  version_requirements: *2160604100
+  version_requirements: *2169473600
 - !ruby/object:Gem::Dependency
   name: redis
-  requirement: &2160602220 !ruby/object:Gem::Requirement
+  requirement: &2169465640 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,10 +32,10 @@ dependencies:
         version: 2.0.1
   type: :runtime
   prerelease: false
-  version_requirements: *2160602220
+  version_requirements: *2169465640
 - !ruby/object:Gem::Dependency
   name: resque
-  requirement: &2160601740 !ruby/object:Gem::Requirement
+  requirement: &2169464480 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -43,10 +43,10 @@ dependencies:
         version: 1.19.0
   type: :runtime
   prerelease: false
-  version_requirements: *2160601740
+  version_requirements: *2169464480
 - !ruby/object:Gem::Dependency
   name: rufus-scheduler
-  requirement: &2160601260 !ruby/object:Gem::Requirement
+  requirement: &2169462660 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -54,7 +54,7 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *2160601260
+  version_requirements: *2169462660
 description: ! "Light weight job scheduling on top of Resque.\n    Adds methods enqueue_at/enqueue_in
   to schedule jobs in the future.\n    Also supports queueing jobs on a fixed, cron-like
   schedule."
@@ -72,6 +72,7 @@ files:
 - Rakefile
 - lib/resque/scheduler.rb
 - lib/resque_scheduler.rb
+- lib/resque_scheduler/plugin.rb
 - lib/resque_scheduler/server.rb
 - lib/resque_scheduler/server/views/delayed.erb
 - lib/resque_scheduler/server/views/delayed_timestamp.erb
@@ -101,7 +102,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -2970363216344829182
+      hash: -2739279222393439195
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -110,7 +111,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: 1.3.6
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.5
+rubygems_version: 1.8.17
 signing_key: 
 specification_version: 3
 summary: Light weight job scheduling on top of Resque