que-scheduler 0.4.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 9640010d4ce493c0f9dc4e14cba6dd02a3b73b78
-  data.tar.gz: e30a1e6efc66e40f1946847008de970be9f23c44
+  metadata.gz: 6e9d071dc4b8b0357483a4a8aa5aa0b48b95e9d5
+  data.tar.gz: 8b8090e1eb7d1106a3a861d6c3ae2361a58a3eaf
 SHA512:
-  metadata.gz: dda04070144829a52281735bde7fce1c7ea7125a0273914524b4fa9d427ef26f17f08a6a097554b4cca9accf01dc72ee36c8b664ac173f754b59fedd597bda2a
-  data.tar.gz: 0e00aeedf7609aca824ce75c20ffcfd2a40786f57832c65164d921fa9a1589681d3f0755c47bfa80a1f190d5f1b6025abe254b7a11de29af2e59f9ad31a1a133
+  metadata.gz: 222d36133bf344092ad2ddff06031eecab82ee522f0da8da89d9f659f604b0770601acf676ba895003336a8076ef853538c554743f99db7571ac392e56548b92
+  data.tar.gz: 7cc612d46143d4fe9338e4d534c2b796c85692e2be77ed9204e02e8063e1430b6e07c5d40ee9624d93da2bcb5aa9f379911fd6fb52ae76aa70e264a6b01bddae

data/README.md

@@ -1,16 +1,19 @@
 que-scheduler
 ================
 
+[![Gem Version](https://badge.fury.io/rb/que-scheduler.svg)](https://badge.fury.io/rb/que-scheduler)
 [![Build Status](https://travis-ci.org/hlascelles/que-scheduler.svg?branch=master)](https://travis-ci.org/hlascelles/que-scheduler)
+[![Dependency Status](https://gemnasium.com/badges/github.com/hlascelles/que-scheduler.svg)](https://gemnasium.com/github.com/hlascelles/que-scheduler)
+[![Coverage Status](https://coveralls.io/repos/github/hlascelles/que-scheduler/badge.svg?branch=master)](https://coveralls.io/github/hlascelles/que-scheduler?branch=master)
 [![Code Climate Maintainability](https://api.codeclimate.com/v1/badges/710d2fc5202f95d76e8a/maintainability)](https://codeclimate.com/github/hlascelles/que-scheduler/maintainability)
 
-### Description
+## Description
 
 que-scheduler is an extension to [Que](https://github.com/chanks/que) that adds support for scheduling 
 items using a cron style configuration file. It works by running as a que job itself, determining what 
 needs to be run, enqueueing those jobs, then enqueueing itself to check again later.
 
-### Installation
+## Installation
 
 1. To install, add the gem to your Gemfile:
     ```ruby
@@ -26,7 +29,7 @@ files, but with additional features.
     Que::Scheduler::SchedulerJob.enqueue
     ```
     
-### Schedule configuration
+## Schedule configuration
 
 The schedule file is a list of que job classes with arguments and a schedule frequency (in crontab 
 syntax). The format is a superset of the resque-scheduler config format, so it they can be used
@@ -63,13 +66,13 @@ DailyBatchReport:
   unmissable: true
 ```
 
-### Environment Variables
+## Environment Variables
 
 You can configure some aspects of the gem with environment variables.
 
 * `QUE_SCHEDULER_CONFIG_LOCATION` - The location of the schedule configuration (default `config/que_schedule.yml`)
 
-### Redundancy and Fail-Over
+## Redundancy and Fail-Over
 
 Because of the way que-scheduler works, it requires no additional processes. It is, itself, a Que job.
 As long as there are Que workers functioning, then jobs will continue to be scheduled correctly. There
@@ -77,7 +80,7 @@ are no HA concerns to worry about and no namespace collisions between different
 
 Additionally, like Que, when your database is backed up, your scheduling state is stored too.
 
-### How it works
+## How it works
 
 que-scheduler is a job that reads a config file, enqueues any jobs it determines that need to be run,
 then reschedules itself. The flow is as follows:
@@ -87,6 +90,6 @@ then reschedules itself. The flow is as follows:
 1. Some time later it runs again. It knows what jobs it should be monitoring, and notices that some have are due. It enqueues those jobs and then itself. Repeat.
 1. After a deploy that changes the config, the job notices any new jobs to schedule, and knows which ones to forget. It does not need to be re-enqueued or restarted.
 
-### Thanks
+## Thanks
 
 This gem was inspired by the makers of the excellent [Que](https://github.com/chanks/que) job scheduler gem. 

data/lib/que/scheduler/schedule_parser.rb

@@ -6,7 +6,7 @@ module Que
 
     class ScheduleParser
       class << self
-        def parse(jobs_list, as_time, last_time, known_jobs)
+        def parse(scheduler_config, scheduler_job_args)
           missed_jobs = {}
           schedule_dictionary = []
 
@@ -14,12 +14,18 @@ module Que
           # new. Otherwise, check how many times we have missed the job since the last run time.
           # If it is "unmissable" then we schedule all of them, with the missed time as an arg,
           # otherwise just schedule it once.
-          jobs_list.each do |desc|
-            schedule_dictionary << desc[:name]
+          scheduler_config.each do |desc|
+            job_name = desc[:name]
+            schedule_dictionary << job_name
+
+            # If we have never seen this job before, we don't want to scheduled any jobs for it.
+            # But we have added it to the dictionary, so it will be used to enqueue jobs next time.
+            next unless scheduler_job_args.job_dictionary.include?(job_name)
 
-            next unless known_jobs.include?(desc[:name])
             # This has been seen before. We should check if we have missed any executions.
-            missed = calculate_missed_runs(desc, last_time, as_time)
+            missed = calculate_missed_runs(
+              desc, scheduler_job_args.last_run_time, scheduler_job_args.as_time
+            )
             missed_jobs[desc[:clazz]] = missed unless missed.empty?
           end
 
@@ -30,25 +36,15 @@ module Que
 
         # Given a job description, the last scheduler run time, and this run time, return all
         # the instances that should be enqueued for that job class.
-        def calculate_missed_runs(desc, last_scheduler_run_time, as_time)
-          jobs_for_class = []
+        def calculate_missed_runs(desc, last_run_time, as_time)
           missed_times = []
-          last_time = last_scheduler_run_time
+          last_time = last_run_time
           while (next_run = next_run_time(desc[:cron], last_time, as_time))
             missed_times << next_run
             last_time = next_run
           end
 
-          unless missed_times.empty?
-            if desc[:unmissable]
-              missed_times.each do |time_missed|
-                jobs_for_class << [time_missed] + desc[:args]
-              end
-            else
-              jobs_for_class << desc[:args]
-            end
-          end
-          jobs_for_class
+          generate_required_jobs_list(desc, missed_times)
         end
 
         # Given a cron, and a "last time", return the next Time the event will occur, or nil if it
@@ -59,6 +55,24 @@ module Que
           next_run = next_time.to_local_time.in_time_zone(next_time.zone)
           next_run <= to ? next_run : nil
         end
+
+        # Given a job description, and the timestamps of the missed events, generate a list of jobs
+        # that can be enqueued as an array of arrays of args.
+        def generate_required_jobs_list(desc, missed_times)
+          jobs_for_class = []
+          unless missed_times.empty?
+            job_args = desc[:args]
+
+            if desc[:unmissable]
+              missed_times.each do |time_missed|
+                jobs_for_class << [time_missed] + job_args
+              end
+            else
+              jobs_for_class << job_args
+            end
+          end
+          jobs_for_class
+        end
       end
     end
   end

data/lib/que/scheduler/scheduler_job.rb

@@ -1,6 +1,7 @@
 require 'que'
 require 'yaml'
 require_relative 'schedule_parser'
+require_relative 'scheduler_job_args'
 
 module Que
   module Scheduler
@@ -10,41 +11,39 @@ module Que
       # Highest possible priority.
       @priority = 0
 
-      def run(last_time = nil, known_jobs = [])
-        ::ActiveRecord::Base.transaction do
-          last_time = last_time.nil? ? Time.now : Time.zone.parse(last_time)
-          as_time = Time.zone.now
-
-          Que.log({ message: "que-scheduler last ran at #{last_time}." })
+      def run(options = nil, oldarg = nil)
+        # Early versions took separate args. We now just pass in a hash.
+        options = { last_run_time: options, job_dictionary: oldarg } if oldarg.present?
 
-          # Obtain the hash of required jobs. Keys are the job classes, and the values are arrays
-          # each containing more arrays for the arguments of that instance.
-          result = enqueue_required_jobs(last_time, as_time, known_jobs)
-          # And enqueue this job again.
-          enqueue_self_again(as_time, result.schedule_dictionary)
+        ::ActiveRecord::Base.transaction do
+          scheduler_job_args = SchedulerJobArgs.prepare_scheduler_job_args(options)
+          Que.log(message: "que-scheduler last ran at #{scheduler_job_args.last_run_time}.")
+          result = enqueue_required_jobs(scheduler_job_args)
+          enqueue_self_again(scheduler_job_args, result.schedule_dictionary)
           destroy
         end
       end
 
       private
 
-      def enqueue_required_jobs(last_time, as_time, known_jobs)
-        result =
-          ScheduleParser.parse(SchedulerJob.scheduler_config, as_time, last_time, known_jobs)
+      def enqueue_required_jobs(scheduler_job_args)
+        # Obtain the hash of missed jobs. Keys are the job classes, and the values are arrays
+        # each containing more arrays for the arguments of that instance.
+        result = ScheduleParser.parse(SchedulerJob.scheduler_config, scheduler_job_args)
         result.missed_jobs.each do |job_class, args_arrays|
           args_arrays.each do |args|
-            Que.log({ message: "que-scheduler enqueueing #{job_class} with args: #{args}" })
+            Que.log(message: "que-scheduler enqueueing #{job_class} with args: #{args}")
             job_class.enqueue(*args)
           end
         end
         result
       end
 
-      def enqueue_self_again(as_time, schedule_dictionary)
+      def enqueue_self_again(scheduler_job_args, new_job_dictionary)
         SchedulerJob.enqueue(
-          as_time,
-          schedule_dictionary,
-          run_at: as_time.beginning_of_minute + SCHEDULER_FREQUENCY
+          last_run_time: scheduler_job_args.as_time.iso8601,
+          job_dictionary: new_job_dictionary,
+          run_at: scheduler_job_args.as_time.beginning_of_minute + SCHEDULER_FREQUENCY
         )
       end
 

data/lib/que/scheduler/scheduler_job_args.rb

@@ -0,0 +1,28 @@
+require 'hashie'
+
+module Que
+  module Scheduler
+    class SchedulerJobArgs < Hashie::Dash
+      property :last_run_time, required: true
+      property :job_dictionary, required: true
+      property :as_time, required: true
+
+      def self.prepare_scheduler_job_args(options)
+        parsed =
+          if options.nil?
+            # First ever run
+            {
+              last_run_time: Time.zone.now,
+              job_dictionary: []
+            }
+          else
+            {
+              last_run_time: Time.zone.parse(options.fetch(:last_run_time)),
+              job_dictionary: options.fetch(:job_dictionary)
+            }
+          end
+        SchedulerJobArgs.new(parsed.merge(as_time: Time.zone.now))
+      end
+    end
+  end
+end

data/lib/que/scheduler/version.rb

@@ -1,5 +1,5 @@
 module Que
   module Scheduler
-    VERSION = '0.4.0'.freeze
+    VERSION = '0.5.0'.freeze
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: que-scheduler
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.5.0
 platform: ruby
 authors:
 - Harry Lascelles
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-10-29 00:00:00.000000000 Z
+date: 2017-11-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -16,168 +16,56 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '4.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '4.0'
 - !ruby/object:Gem::Dependency
   name: fugit
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '1'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '1'
 - !ruby/object:Gem::Dependency
-  name: que
+  name: hashie
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.12'
+        version: '3'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.12'
-- !ruby/object:Gem::Dependency
-  name: activesupport
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '4.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '4.0'
-- !ruby/object:Gem::Dependency
-  name: bundler
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.15'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.15'
-- !ruby/object:Gem::Dependency
-  name: pry-byebug
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '3'
 - !ruby/object:Gem::Dependency
-  name: que-testing
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.1'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.1'
-- !ruby/object:Gem::Dependency
-  name: rake
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '10.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '10.0'
-- !ruby/object:Gem::Dependency
-  name: rspec
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.0'
-- !ruby/object:Gem::Dependency
-  name: rubocop
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.51'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.51'
-- !ruby/object:Gem::Dependency
-  name: timecop
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.7'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.7'
-- !ruby/object:Gem::Dependency
-  name: zonebie
+  name: que
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.6'
-  type: :development
+        version: '0.10'
+  type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.6'
+        version: '0.10'
 description: A lightweight cron scheduler for the async job worker Que
 email:
 - harry@harrylascelles.com
@@ -189,8 +77,9 @@ files:
 - lib/que/scheduler.rb
 - lib/que/scheduler/schedule_parser.rb
 - lib/que/scheduler/scheduler_job.rb
+- lib/que/scheduler/scheduler_job_args.rb
 - lib/que/scheduler/version.rb
-homepage: https://rubygems.org/gems/que-scheduler
+homepage: https://github.com/hlascelles/que-scheduler
 licenses:
 - MIT
 metadata: {}