simple_scheduler 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 8d15cd2b0b48692188a8b470d258e667dae84d08
-  data.tar.gz: 20eca360c888dbf7e482377f8d78053e988906c5
+  metadata.gz: 7f15bb865956e8b2d8b84e65ba2baf7386333a5f
+  data.tar.gz: bf3cd7c9bac2b2d9d2085b8d4b4c55000977c98f
 SHA512:
-  metadata.gz: 452dae7f622c805289ade38235a8041d9ed23faeb9bd5a1acfa51577570afd07667dbb70d5d099c7630c9ca4fbe04245d80c53655f14464c50369bdb8747a52c
-  data.tar.gz: f1d49ad79d145d281ce24d4476c752c1d51ae8b37cba98e6e25830c0bc355a2b12e4c3b36a475a25f86a2456e74b0796c237ecae28f0eabd127ebf7d3ec79ac7
+  metadata.gz: ca37117e04fcb7c0e4582f702f6262a9051f33c7cfe6f0ee1ce6a80120515128f07ec4a71b3cd5014ce1da5ca1bc424e1b3b86005014bf897c431609b30c3762
+  data.tar.gz: df9d6c1ccbeb51d64c6ae5c5bd40bc5305cdeeaa4efb214ce7cf4b2f79cc2565aeffb449009fb2cdbc193ccfcbc32d223a3cdf87167f155d4d969d174d3be9de

data/README.md

@@ -37,7 +37,9 @@ $ bundle
 
 ## Getting Started
 
-Create a configuration file `config/simple_scheduler.yml`:
+### Create the Configuration File
+
+Create the file `config/simple_scheduler.yml` in your Rails project:
 
 ```yml
 # Global configuration options and their defaults. These can also be set on each task.
@@ -49,11 +51,13 @@ simple_task:
   class: "SomeActiveJob"
   every: "2.minutes"
 
-# Runs once every day at 4:00 AM
+# Runs once every day at 4:00 AM. The job will expire after 23 hours, which means the
+# job will not run if 23 hours passes (server downtime) before the job is actually run
 overnight_task:
   class: "SomeSidekiqWorker"
   every: "1.day"
   at: "4:00"
+  expires_after: "23.hours"
 
 # Runs once every hour at the half hour
 half_hour_task:
@@ -69,7 +73,21 @@ weekly_task:
   tz: "America/Chicago"
 ```
 
-### Task options
+### Set up Heroku Scheduler
+
+Add the rake task to Heroku Scheduler and set it to run every 10 minutes:
+
+```
+rake simple_scheduler
+```
+
+It may be useful to point to a specific configuration file in non-production environments:
+
+```
+rake simple_scheduler["config/simple_scheduler.staging.yml"]
+```
+
+### Task Options
 
 #### :class
 
@@ -81,12 +99,12 @@ on the `perform` method.
 
 How frequently the task should be performed as an ActiveSupport duration definition.
 
-```ruby
-1.day
-5.days
-12.hours
-20.minutes
-1.week
+```yml
+"1.day"
+"5.days"
+"12.hours"
+"20.minutes"
+"1.week"
 ```
 
 #### :at (optional)
@@ -97,60 +115,67 @@ follow the `every` duration to determine future execution times.
 
 Valid string formats/examples:
 
-```
-18:00
- 3:30
-**:00
- *:30
-Sun 2:00
-[Sun|Mon|Tue|Wed|Thu|Fri|Sat] 00:00
+```yml
+"18:00"
+"3:30"
+"**:00"
+"*:30"
+"Sun 2:00"
+"[Sun|Mon|Tue|Wed|Thu|Fri|Sat] 00:00"
 ```
 
-Add the rake task to Heroku Scheduler and set it to run every 10 minutes:
+#### :expires_after (optional)
 
-```
-rake simple_scheduler
-```
+If your worker process is down for an extended period of time, you may not want jobs
+to execute when the server comes back online. By specifying an `expires_after` value,
+your job will not fire if the time the job actually runs later, by the specified
+duration, than the scheduled run time.
 
-The file `config/simple_scheduler.yml` will be used by default, but it may be
-useful to point to another configuration file in non-production environments.
+The string should be in the form of an ActiveSupport duration.
 
-```
-rake simple_scheduler["config/simple_scheduler.staging.yml"]
+```yml
+"59.minutes"
+"23.hours"
 ```
 
 ## Writing Your Jobs
 
-Your Active Job or Sidekiq Worker must accept the task name and time stamp
-as arguments. This is so they can be queued properly and so your job can
-use the scheduled time to know when it was supposed to run vs the current time.
+There is no guarantee that the job will run at the exact time given in the
+configuration, so the time the job was scheduled to run will be passed to
+the job. This allows you to handle situations where the current time doesn't
+match the time it was expected to run. The `scheduled_time` argument is optional.
 
 ```ruby
 class ExampleJob < ActiveJob::Base
-  # @param task_name [String] This is the key used in the YAML file to define the task
-  # @param time [Integer] The epoch time for when the job was scheduled to be run
-  def perform(task_name, time)
-    puts task_name
-    puts Time.at(time)
+  # @param scheduled_time [Integer] The epoch time for when the job was scheduled to be run
+  def perform(scheduled_time)
+    puts Time.at(scheduled_time)
   end
 end
 ```
 
-When writing your jobs, you need to account for any possible server downtime.
-The most common downtime would be caused by Heroku's required daily restart.
+## Handling Expired Jobs
 
-To ensure that your tasks always run, the jobs are queued in advance and it's
-possible the jobs may not be executed at the exact time that you configured
-them to run. If there is extended downtime, your jobs may back up and there
-is no guarantee of the order they will be executed when your worker process
-comes back online.
+If you assign the `expires_after` option to your task, you may want to know if
+a job wasn't run because it expires. Add this block to an initializer file:
 
-Because there is no guarantee that the job is run at the exact time given in
-the configuration, the time the job was expected to run will be passed to
-the job so you can handle situations where the time it was run doesn't match
-the time it was expected to run.
+```ruby
+# config/initializers/simple_scheduler.rb
+
+# @param exception [SimpleScheduler::FutureJob::Expired]
+SimpleScheduler.expired_task do |exception|
+  ExceptionNotifier.notify_exception(
+    exception,
+    data: {
+      task:      exception.task.name,
+      scheduled: exception.scheduled_time,
+      actual:    exception.run_time
+    }
+  )
+end
+```
 
-### How It Works
+## How It Works
 
 Once the rake task is added to Heroku Scheduler, the Simple Scheduler library
 will load the configuration file every 10 minutes, and ensure that each task
@@ -163,10 +188,10 @@ even if one of the two was executed during the 10 minute scheduler wait time.
 
 ### Server Downtime Example
 
-If you're using a gem like [clockwork](https://github.com/Rykian/clockwork), there is no way for the clock process to
-know that the task was never run. If your task is scheduled for `12:00:00`, your
-clock process could possibly be restarted at `11:59:59` and your dyno might not
-be available until `12:00:20`.
+If you're using a gem like [clockwork](https://github.com/Rykian/clockwork),
+there is no way for the clock process to know that the task was never run.
+If your task is scheduled for `12:00:00`, your clock process could possibly
+be restarted at `11:59:59` and your dyno might not be available until `12:00:20`.
 
 Simple Scheduler would have already enqueued the task hours before the task should actually
 run, so you still have to worry about the worker dyno restarting, but when the worker
@@ -186,6 +211,7 @@ daily_digest_task:
   class: "DailyDigestEmailJob"
   every: "15.minutes"
   at: "*:00"
+  expires_after: "23.hours"
 ```
 
 app/jobs/daily_digest_email_job.rb:
@@ -196,12 +222,11 @@ class DailyDigestEmailJob < ApplicationJob
 
   # Called by Simple Scheduler and is given the scheduled time so decisions can be made
   # based on when the job was scheduled to be run rather than when it was actually run.
-  # @param task_name [String] This is the key used in the YAML file to define the task
-  # @param time [Integer] The epoch time for when the job was scheduled to be run
-  def perform(task_name, time)
+  # @param scheduled_time [Integer] The epoch time for when the job was scheduled to be run
+  def perform(scheduled_time)
     # Don't do this! This will be way too slow!
     User.find_each do |user|
-      if user.digest_time == Time.at(time)
+      if user.digest_time == Time.at(scheduled_time)
         DigestMailer.daily(user).deliver_later
       end
     end

data/lib/simple_scheduler.rb

@@ -1,5 +1,6 @@
 require "active_job"
 require "sidekiq/api"
+require_relative "./simple_scheduler/future_job"
 require_relative "./simple_scheduler/railtie"
 require_relative "./simple_scheduler/scheduler_job"
 require_relative "./simple_scheduler/task"
@@ -7,4 +8,24 @@ require_relative "./simple_scheduler/version"
 
 # Module for scheduling jobs at specific times using Sidekiq.
 module SimpleScheduler
+  # Used by a Rails initializer to handle expired tasks.
+  #   SimpleScheduler.expired_task do |exception|
+  #     ExceptionNotifier.notify_exception(
+  #       exception,
+  #       data: {
+  #         task:      exception.task.name,
+  #         scheduled: exception.scheduled_time,
+  #         actual:    exception.run_time
+  #       }
+  #     )
+  #   end
+  def self.expired_task(&block)
+    expired_task_blocks << block
+  end
+
+  # Blocks that should be called when a task doesn't run because it has expired.
+  # @return [Array]
+  def self.expired_task_blocks
+    @expired_task_blocks ||= []
+  end
 end

data/lib/simple_scheduler/future_job.rb

@@ -0,0 +1,83 @@
+module SimpleScheduler
+  # Active Job class that wraps the scheduled job and determines if the job
+  # should still be run based on the scheduled time and when the job expires.
+  class FutureJob < ActiveJob::Base
+    # An error class that is raised if a job does not run because the run time is
+    # too late when compared to the scheduled run time.
+    # @!attribute run_time
+    #   @return [Time] The actual run time
+    # @!attribute scheduled_time
+    #   @return [Time] The scheduled run time
+    # @!attribute task
+    #   @return [SimpleScheduler::Task] The expired task
+    class Expired < StandardError
+      attr_accessor :run_time, :scheduled_time, :task
+    end
+
+    rescue_from Expired, with: :handle_expired_task
+
+    # Perform the future job as defined by the task.
+    # @param task_params [Hash] The params from the scheduled task
+    # @param scheduled_time [Integer] The epoch time for when the job was scheduled to be run
+    def perform(task_params, scheduled_time)
+      @task = Task.new(task_params)
+      @scheduled_time = Time.at(scheduled_time).in_time_zone(@task.time_zone)
+      raise Expired if expired?
+
+      if @task.job_class.included_modules.include?(Sidekiq::Worker)
+        queue_sidekiq_worker
+      else
+        queue_active_job
+      end
+    end
+
+    private
+
+    # The duration between the scheduled run time and actual run time that
+    # will cause the job to expire. Expired jobs will not be executed.
+    def expire_duration
+      split_duration = @task.expires_after.split(".")
+      duration = split_duration[0].to_i
+      duration_units = split_duration[1]
+      duration.send(duration_units)
+    end
+
+    # Returns whether or not the job has expired based on the time
+    # between the scheduled run time and the current time.
+    # @return [Boolean]
+    def expired?
+      return false if @task.expires_after.blank?
+      expire_duration.from_now(@scheduled_time) < Time.now.in_time_zone(@task.time_zone)
+    end
+
+    # Handle the expired task by passing the task and run time information
+    # to a block that can be creating in a Rails initializer file.
+    def handle_expired_task(exception)
+      exception.run_time = Time.now.in_time_zone(@task.time_zone)
+      exception.scheduled_time = @scheduled_time
+      exception.task = @task
+
+      SimpleScheduler.expired_task_blocks.each do |block|
+        block.call(exception)
+      end
+    end
+
+    # Queue the job for immediate execution using Active Job.
+    def queue_active_job
+      if @task.job_class.instance_method(:perform).arity > 0
+        @task.job_class.perform_later(@scheduled_time.to_i)
+      else
+        @task.job_class.perform_later
+      end
+    end
+
+    # Queue the job for immediate execution using Sidekiq.
+    def queue_sidekiq_worker
+      if @task.job_class.instance_method(:perform).arity > 0
+        @task.job_class.perform_async(@scheduled_time.to_i)
+      else
+        @task.job_class.perform_async
+      end
+    end
+  end
+end

data/lib/simple_scheduler/scheduler_job.rb

@@ -1,34 +1,35 @@
 module SimpleScheduler
   # Active Job class that queues jobs defined in the config file.
   class SchedulerJob < ActiveJob::Base
+    # Accepts a file path to read the scheduler configuration.
+    # @param config_path [String]
+    def perform(config_path = nil)
+      config_path ||= "config/simple_scheduler.yml"
+      load_config(config_path)
+      queue_future_jobs
+    end
+
+    private
+
     # Load the global scheduler config from the YAML file.
     # @param config_path [String]
     def load_config(config_path)
       @config = YAML.load_file(config_path)
       @queue_ahead = @config["queue_ahead"] || Task::DEFAULT_QUEUE_AHEAD_MINUTES
-      @time_zone = @config["tz"] ? ActiveSupport::TimeZone.new(@config["tz"]) : Time.zone
+      @time_zone = @config["tz"] || Time.zone.tzinfo.name
       @config.delete("queue_ahead")
       @config.delete("tz")
     end
 
-    # Accepts a file path to read the scheduler configuration.
-    # @param config_path [String]
-    def perform(config_path = nil)
-      config_path ||= "config/simple_scheduler.yml"
-      load_config(config_path)
-      queue_future_jobs
-    end
-
     # Queue each of the future jobs into Sidekiq from the defined tasks.
     def queue_future_jobs
       tasks.each do |task|
         new_run_times = task.future_run_times - task.existing_run_times
         next if new_run_times.empty?
 
-        if task.job_class.included_modules.include?(Sidekiq::Worker)
-          queue_future_sidekiq_workers(task, new_run_times)
-        else
-          queue_future_active_jobs(task, new_run_times)
+        # Schedule the new run times using the future job wrapper.
+        new_run_times.each do |time|
+          SimpleScheduler::FutureJob.set(wait_until: time).perform_later(task.params, time.to_i)
         end
       end
     end
@@ -44,25 +45,5 @@ module SimpleScheduler
         Task.new(task_params)
       end
     end
-
-    private
-
-    # Queues jobs in the future using Active Job based on the task options.
-    # @param task [SimpleScheduler::Task]
-    # @param run_times [Array<Time>]
-    def queue_future_active_jobs(task, run_times)
-      run_times.each do |time|
-        task.job_class.set(wait_until: time).perform_later(task.name, time.to_i)
-      end
-    end
-
-    # Queues jobs in the future using Sidekiq based on the task options.
-    # @param task [SimpleScheduler::Task]
-    # @param run_times [Array<Time>]
-    def queue_future_sidekiq_workers(task, run_times)
-      run_times.each do |time|
-        task.job_class.perform_at(time, task.name, time.to_i)
-      end
-    end
   end
 end

data/lib/simple_scheduler/task.rb

@@ -1,8 +1,28 @@
 module SimpleScheduler
   # Class for parsing each task in the scheduler config YAML file and returning
   # the values needed to schedule the task in the future.
+  #
+  # @!attribute at
+  #   @return [String] The starting time for the interval
+  # @!attribute expires_after
+  #   @return [String] The time between the scheduled and actual run time that should cause the job not to run
+  # @!attribute frequency
+  #   @return [ActiveSupport::Duration] How often the job will be run
+  # @!attribute [r] job_class
+  #   @return [Class] The class of the job or worker
+  # @!attribute [r] job_class_name
+  #   @return [String] The class name of the job or worker
+  # @!attribute name
+  #   @return [String] The name of the task as defined in the YAML config
+  # @!attribute [r] params
+  #   @return [Hash] The params used to create the task
+  # @!attribute queue_ahead
+  #   @return [String] The name of the task as defined in the YAML config
+  # @!attribute time_zone
+  #   @return [ActiveSupport::TimeZone] The time zone to use when parsing the `at` option
   class Task
-    attr_accessor :at, :frequency, :job_class, :job_class_name, :name, :queue_ahead, :time_zone
+    attr_reader :at, :expires_after, :frequency, :job_class, :job_class_name
+    attr_reader :name, :params, :queue_ahead, :time_zone
 
     AT_PATTERN = /(Sun|Mon|Tue|Wed|Thu|Fri|Sat)?\s?(?:\*{1,2}|(\d{1,2})):(\d{1,2})/
     DAYS = %w(Sun Mon Tue Wed Thu Fri Sat).freeze
@@ -13,25 +33,30 @@ module SimpleScheduler
     # @option params [String] :class The class of the Active Job or Sidekiq Worker
     # @option params [String] :every How frequently the job will be performed
     # @option params [String] :at The starting time for the interval
+    # @option params [String] :expires_after The time between the scheduled and actual run time that should cause the job not to run
     # @option params [Integer] :queue_ahead The number of minutes that jobs should be queued in the future
     # @option params [String] :task_name The name of the task as defined in the YAML config
-    # @option params [ActiveSupport::TimeZone] :tz The time zone to use when parsing the `at` option
+    # @option params [String] :tz The time zone to use when parsing the `at` option
     def initialize(params)
       validate_params!(params)
-      @at = params[:at]
-      @frequency = parse_frequency(params[:every])
+      @at             = params[:at]
+      @expires_after  = params[:expires_after]
+      @frequency      = parse_frequency(params[:every])
       @job_class_name = params[:class]
-      @job_class = @job_class_name.constantize
-      @queue_ahead = params[:queue_ahead] || DEFAULT_QUEUE_AHEAD_MINUTES
-      @name = params[:name] || @job_class_name
-      @time_zone = params[:tz] || Time.zone
+      @job_class      = @job_class_name.constantize
+      @queue_ahead    = params[:queue_ahead] || DEFAULT_QUEUE_AHEAD_MINUTES
+      @name           = params[:name]
+      @params         = params
+      @time_zone      = params[:tz] ? ActiveSupport::TimeZone.new(params[:tz]) : Time.zone
     end
 
     # Returns an array of existing jobs matching the job class of the task.
     # @return [Array<Sidekiq::SortedEntry>]
     def existing_jobs
       @existing_jobs ||= SimpleScheduler::Task.scheduled_set.select do |job|
-        job.display_class == @job_class_name && job.display_args[0] == @name
+        next unless job.display_class == "SimpleScheduler::FutureJob"
+        task_params = job.display_args[0]
+        task_params[:class] == @job_class_name && task_params[:name] == @name
       end.to_a
     end
 
@@ -120,6 +145,7 @@ module SimpleScheduler
     end
 
     def validate_params!(params)
+      params[:name] ||= params[:class]
       raise ArgumentError, "Missing param `class` specifying the class of the job to run." unless params.key?(:class)
       raise ArgumentError, "Missing param `every` specifying how often the job should run." unless params.key?(:every)
     end

data/lib/simple_scheduler/version.rb

@@ -1,3 +1,3 @@
 module SimpleScheduler
-  VERSION = "0.1.0".freeze
+  VERSION = "0.2.0".freeze
 end

metadata

@@ -1,111 +1,97 @@
 --- !ruby/object:Gem::Specification
 name: simple_scheduler
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Brian Pattison
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-12-07 00:00:00.000000000 Z
+date: 2016-12-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '4.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '4.2'
 - !ruby/object:Gem::Dependency
   name: sidekiq
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '4.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '4.2'
 - !ruby/object:Gem::Dependency
   name: rspec-rails
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: rubocop
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: simplecov
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: simplecov-rcov
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: timecop
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 description: |2
@@ -118,15 +104,16 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- MIT-LICENSE
+- README.md
+- Rakefile
+- lib/simple_scheduler.rb
+- lib/simple_scheduler/future_job.rb
 - lib/simple_scheduler/railtie.rb
 - lib/simple_scheduler/scheduler_job.rb
 - lib/simple_scheduler/task.rb
 - lib/simple_scheduler/version.rb
-- lib/simple_scheduler.rb
 - lib/tasks/simple_scheduler_tasks.rake
-- MIT-LICENSE
-- Rakefile
-- README.md
 homepage: https://github.com/simplymadeapps/simple_scheduler
 licenses:
 - MIT
@@ -137,17 +124,17 @@ require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.0.14.1
+rubygems_version: 2.5.1
 signing_key: 
 specification_version: 4
 summary: An enhancement for Heroku Scheduler + Sidekiq for scheduling jobs at specific