que-scheduler 3.2.8 → 3.4.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c2a35cace72621c452055873ea88a13aba5d00a63933df91b2d4d60840f11337
-  data.tar.gz: 1b2d3bbb599260b37f8ca8c10a12bd03992b08ed74b0c75f47905b1f8238dfd5
+  metadata.gz: 4ba02202915db229bcbd1c2e859f29299803507dbf61bd460008400abca3611c
+  data.tar.gz: 6293d7b6e6766d0e75fe65df01850f356b44b030c8b8ae539eb4c71059d9d15d
 SHA512:
-  metadata.gz: ab5ad0c46554887d523d7b9f90c8336da989489e76cc69a0bfe5465b66b34e7a7c3d0f2fc6c0dc026660926b51e4fff0fde029600b93e52386147565a4c10a2a
-  data.tar.gz: 7359bfdc4db605aea48c8bb87b1f02a8203dc103fa6e6c9ba5d6f33d051026c7e4abc7d4e5a503660d522d99f89c720455879a7b631929bd4e591961388afa75
+  metadata.gz: d8d18550867d7d54cfb5c03aaa68876d423876a870b972cde0fbc4e8399fb616d327ef0410ebbc8e3d9021a34d73a960774eabb808f2b33a526e11ea2fb579fe
+  data.tar.gz: bfc0e4be1f64d2b070dacb0863cc710ce13cf58cb30924fcf9f301d2e535be4ef9530740949bdb5abb90c0173924e71f7f151e4622405caf8c14a9d7d6f0469d

data/README.md

@@ -18,23 +18,28 @@ needs to be run, enqueueing those jobs, then enqueueing itself to check again la
     ```ruby
     gem 'que-scheduler'
     ```
-1. Specify a schedule in a yml file (see below). The default location that que-scheduler will 
-look for it is `config/que_schedule.yml`. They are essentially the same as resque-scheduler
-files, but with additional features.
+1. Specify a schedule in a yml file or programmatically (see below). The default location that 
+que-scheduler will look for it is `config/que_schedule.yml`. The format is essentially the same as
+resque-scheduler files, but with additional features.
 
-1. Add a migration to start the job scheduler and prepare the audit table. Note that this migration will fail if Que is set to execute jobs synchronously, i.e. `Que::Job.run_synchronously = true`.
+1. Add a migration to start the job scheduler and prepare the audit table. Note that this migration 
+   will fail if Que is set to execute jobs synchronously, i.e. `Que::Job.run_synchronously = true`.
 
     ```ruby
     class CreateQueSchedulerSchema < ActiveRecord::Migration
       def change
-        Que::Scheduler::Migrations.migrate!(version: 4)
+        Que::Scheduler::Migrations.migrate!(version: 5)
       end
     end
     ```
     
 ## Schedule configuration
 
-The schedule file is a list of que job classes with arguments and a schedule frequency (in crontab 
+The schedule file should be placed here: `config/que_schedule.yml`. Alternatively if you
+wish to generate the configuration dynamically, you can set it directly with
+`Que::Scheduler.schedule = some_hash`.
+
+The file is a list of que job classes with arguments and a schedule frequency (in crontab 
 syntax). The format is similar to the resque-scheduler format, though priorities must be supplied as
 integers, and job classes must be migrated from Resque to Que. Cron syntax can be anything
 understood by [fugit](https://github.com/floraison/fugit#fugitcron).
@@ -65,10 +70,21 @@ BatchOrders:
   cron: "0 0 * * *"
   priority: 25
   
-# Specify job arguments
+# Specify array job arguments
 SendOrders:
   cron: "0 0 * * *"
   args: ['open']
+
+# Specify hash job arguments. Note, this appears as a single hash to `run`, not as kwargs.
+SendPreorders:
+  cron: "0 0 * * *"
+  args:
+    order_type: special
+  
+# Specify a single nil argument
+SendPostorders:
+  cron: "0 0 * * *"
+  args: ~ # See https://stackoverflow.com/a/51990876/1267203
   
 # Use simpler cron syntax
 SendBilling:
@@ -136,8 +152,54 @@ end
 
 ## Scheduler Audit
 
-An audit table _que_scheduler_audit_ is written to by the scheduler to keep a history of what jobs 
-were enqueued when. It is created by the included migration tasks.
+An audit table `que_scheduler_audit` is written to by the scheduler to keep a history of when the 
+scheduler ran to calculate what was necessary to run (if anything). It is created by the included 
+migration tasks.
+
+Additionally, there is the audit table `que_scheduler_audit_enqueued`. This logs every job that 
+the scheduler enqueues.
+
+When there is a major version (breaking) change, a migration should be run in. The version of the 
+migration proceeds at a faster rate than the version of the gem. To run in all the migrations required
+up to a number, just migrate to that number with one line, and it will perform all the intermediary steps. 
+
+ie, This will perform all migrations necessary up to the latest version, skipping any already 
+performed.
+
+```ruby
+class CreateQueSchedulerSchema < ActiveRecord::Migration
+  def change
+    Que::Scheduler::Migrations.migrate!(version: 5)
+  end
+end
+```
+
+The changes in past migrations were: 
+
+| Version | Changes                                                                         |
+|:-------:|---------------------------------------------------------------------------------|
+|    1    | Enqueued the main Que::Scheduler. This is the job that performs the scheduling. |
+|    2    | Added the audit table `que_scheduler_audit`.                                    |
+|    3    | Added the audit table `que_scheduler_audit_enqueued`.                           |
+|    4    | Updated the the audit tables to use bigints                                     |
+|    5    | Dropped an unnecessary index                                                    |
+
+## Built in optional job for audit clear down
+
+que-scheduler comes with the `QueSchedulerAuditClearDownJob` job built in that you can optionally
+schedule to clear down audit rows if you don't need to retain them indefinitely. You should add this
+to your own scheduler config yaml.
+
+For example:
+
+```yaml
+# This will clear down the oldest que-scheduler audit rows. Since que-scheduler
+# runs approximately every minute, 129600 is 90 days.
+Que::Scheduler::Jobs::QueSchedulerAuditClearDownJob:
+  cron: "0 0 * * *"
+  args:
+    retain_row_count: 129600
+```
 
 ## HA Redundancy and DB restores
 
@@ -172,23 +234,6 @@ then reschedules itself. The flow is as follows:
 1. After a deploy that changes the schedule, the job notices any new jobs to schedule, and knows which
    ones to forget. It does not need to be re-enqueued or restarted.
    
-## DB Migrations
-
-When there is a major version (breaking) change, a migration should be run in. The version of the 
-migration proceeds at a faster rate than the version of the gem. To run in all the migrations required
-up to a number, just migrate to that number with one line, and it will perform all the intermediary steps. 
-
-ie, `Que::Scheduler::Migrations.migrate!(version: 4)` will perform all migrations necessary to 
-reach migration version `4`.
-
-As of migration `4`, two elements are added to the DB for que-scheduler to run. 
-
-1. The first is the scheduler job itself, which runs forever, re-enqueuing itself to performs its 
-   duties.
-1. The second part comprises the audit table `que_scheduler_audit` and the "enqueued" table 
-  `que_scheduler_audit_enqueued`. The first tracks when the scheduler calculated what was necessary to run 
-  (if anything). The second then logs every job that the scheduler enqueues. 
-
 ## Testing Configuration
 
 You can add tests to validate your configuration during the spec phase. This will perform a variety 
@@ -235,3 +280,6 @@ This gem was inspired by the makers of the excellent [Que](https://github.com/ch
 * @jish
 * @joehorsnell
 * @bnauta
+* @papodaca
+* @krzyzak
+* @JackDanger

data/lib/que-scheduler.rb

@@ -1,3 +1,3 @@
 # rubocop:disable Naming/FileName
-require 'que/scheduler'
+require "que/scheduler"
 # rubocop:enable Naming/FileName

data/lib/que/scheduler.rb

@@ -1,7 +1,8 @@
-require 'que/scheduler/version'
-require 'que/scheduler/version_support'
-require 'que/scheduler/config'
-require 'que/scheduler/scheduler_job'
-require 'que/scheduler/db'
-require 'que/scheduler/audit'
-require 'que/scheduler/migrations'
+require "que/scheduler/version"
+require "que/scheduler/version_support"
+require "que/scheduler/config"
+require "que/scheduler/scheduler_job"
+require "que/scheduler/db"
+require "que/scheduler/audit"
+require "que/scheduler/migrations"
+require "que/scheduler/jobs/que_scheduler_audit_clear_down_job"

data/lib/que/scheduler/audit.rb

@@ -1,10 +1,12 @@
 # frozen_string_literal: true
 
+require_relative "to_enqueue"
+
 module Que
   module Scheduler
     module Audit
-      TABLE_NAME = 'que_scheduler_audit'
-      ENQUEUED_TABLE_NAME = 'que_scheduler_audit_enqueued'
+      TABLE_NAME = "que_scheduler_audit"
+      ENQUEUED_TABLE_NAME = "que_scheduler_audit_enqueued"
       INSERT_AUDIT = %{
         INSERT INTO #{TABLE_NAME} (scheduler_job_id, executed_at)
         VALUES ($1::bigint, $2::timestamptz)
@@ -24,11 +26,10 @@ module Que
         def append(scheduler_job_id, executed_at, enqueued_jobs)
           ::Que::Scheduler::VersionSupport.execute(INSERT_AUDIT, [scheduler_job_id, executed_at])
           enqueued_jobs.each do |j|
-            attrs = Que::Scheduler::VersionSupport.job_attributes(j)
             inserted = ::Que::Scheduler::VersionSupport.execute(
               INSERT_AUDIT_ENQUEUED,
               [scheduler_job_id] +
-                attrs.values_at(:job_class, :queue, :priority, :args, :job_id, :run_at)
+                j.values_at(:job_class, :queue, :priority, :args, :job_id, :run_at)
             )
             raise "Cannot save audit row #{scheduler_job_id} #{executed_at} #{j}" if inserted.empty?
           end

data/lib/que/scheduler/config.rb

@@ -1,5 +1,5 @@
-require 'que'
-require_relative 'version_support'
+require "que"
+require_relative "version_support"
 
 module Que
   module Scheduler
@@ -21,7 +21,7 @@ module Que
 end
 
 Que::Scheduler.configure do |config|
-  config.schedule_location = ENV.fetch('QUE_SCHEDULER_CONFIG_LOCATION', 'config/que_schedule.yml')
+  config.schedule_location = ENV.fetch("QUE_SCHEDULER_CONFIG_LOCATION", "config/que_schedule.yml")
   config.transaction_adapter = ::Que.method(:transaction)
   config.que_scheduler_queue = Que::Scheduler::VersionSupport.default_scheduler_queue
 end

data/lib/que/scheduler/db.rb

@@ -1,13 +1,13 @@
 # frozen_string_literal: true
 
-require_relative 'config'
+require_relative "config"
 
 module Que
   module Scheduler
     module Db
       SCHEDULER_COUNT_SQL =
         "SELECT COUNT(*) FROM que_jobs WHERE job_class = 'Que::Scheduler::SchedulerJob'"
-      NOW_SQL = 'SELECT now()'
+      NOW_SQL = "SELECT now()"
 
       class << self
         def count_schedulers

data/lib/que/scheduler/defined_job.rb

@@ -1,6 +1,5 @@
-require 'hashie'
-require 'fugit'
-require 'backports/2.4.0/hash/compact'
+require "hashie"
+require "fugit"
 
 # This is the definition of one scheduleable job in the que-scheduler config yml file.
 module Que
@@ -10,23 +9,23 @@ module Que
 
       DEFINED_JOB_TYPES = [
         DEFINED_JOB_TYPE_DEFAULT = :default,
-        DEFINED_JOB_TYPE_EVERY_EVENT = :every_event
+        DEFINED_JOB_TYPE_EVERY_EVENT = :every_event,
       ].freeze
 
-      property :name, required: true
-      property :job_class, required: true, transform_with: lambda { |v|
-        job_class = Object.const_get(v)
-        job_class < Que::Job ? job_class : err_field(:job_class, v)
-      }
-      property :cron, required: true, transform_with: lambda { |v|
-        Fugit::Cron.parse(v) || err_field(:cron, v)
-      }
-      property :queue, transform_with: ->(v) { v.is_a?(String) ? v : err_field(:queue, v) }
-      property :priority, transform_with: ->(v) { v.is_a?(Integer) ? v : err_field(:priority, v) }
-      property :args
-      property :schedule_type, default: DEFINED_JOB_TYPE_DEFAULT, transform_with: lambda { |v|
-        v.to_sym.tap { |vs| DEFINED_JOB_TYPES.include?(vs) || err_field(:schedule_type, v) }
-      }
+      property :name
+      property :job_class, transform_with: ->(v) { Object.const_get(v) }
+      property :cron, transform_with: ->(v) { Fugit::Cron.parse(v) }
+      property :queue
+      property :priority
+      property :args_array
+      property :schedule_type, default: DEFINED_JOB_TYPE_DEFAULT
+
+      class << self
+        def create(options)
+          defined_job = new(options.compact)
+          defined_job.freeze.tap { |dj| dj.validate(options) }
+        end
+      end
 
       # Given a "last time", return the next Time the event will occur, or nil if it
       # is after "to".
@@ -49,36 +48,80 @@ module Que
         generate_to_enqueue_list(missed_times)
       end
 
-      class << self
-        private
+      def validate(options)
+        validate_fields_presence(options)
+        validate_fields_types(options)
+        validate_job_class_related(options)
+      end
+
+      private
 
-        def err_field(field, value)
-          schedule = Que::Scheduler.configuration.schedule_location
-          raise "Invalid #{field} '#{value}' in que-scheduler schedule #{schedule}"
+      # rubocop:disable Style/GuardClause This reads better as a conditional
+      def validate_fields_types(options)
+        unless queue.nil? || queue.is_a?(String)
+          err_field(:queue, options, "queue must be a string")
+        end
+        unless priority.nil? || priority.is_a?(Integer)
+          err_field(:priority, options, "priority must be an integer")
+        end
+        unless DEFINED_JOB_TYPES.include?(schedule_type)
+          err_field(:schedule_type, options, "Not in #{DEFINED_JOB_TYPES}")
         end
       end
+      # rubocop:enable Style/GuardClause
 
-      private
+      def validate_fields_presence(options)
+        err_field(:name, options, "name must be present") if name.nil?
+        err_field(:job_class, options, "job_class must be present") if job_class.nil?
+        # An invalid cron is nil
+        err_field(:cron, options, "cron must be present") if cron.nil?
+      end
+
+      def validate_job_class_related(options)
+        # Only support known job engines
+        unless Que::Scheduler::ToEnqueue.valid_job_class?(job_class)
+          err_field(:job_class, options, "Job #{job_class} was not a supported job type")
+        end
+
+        # queue name is only supported for a subrange of ActiveJob versions. Print this out as a
+        # warning.
+        if queue &&
+           Que::Scheduler::ToEnqueue.active_job_sufficient_version? &&
+           job_class < ::ActiveJob::Base &&
+           Que::Scheduler::ToEnqueue.active_job_version < Gem::Version.create("6.0.3")
+          puts <<-ERR
+            WARNING from que-scheduler....
+            Between versions 4.2.3 and 6.0.2 (inclusive) Rails did not support setting queue names
+            on que jobs with ActiveJob, so que-scheduler cannot support it.
+            See removed in Rails 4.2.3
+              https://github.com/rails/rails/pull/19498
+            And readded in Rails 6.0.3
+              https://github.com/rails/rails/pull/38635
+
+            Please remove all "queue" keys from ActiveJobs defined in the que-scheduler.yml config.
+            Specifically #{queue} for job #{name}.
+          ERR
+        end
+      end
 
-      class ToEnqueue < Hashie::Dash
-        property :args, required: true, default: []
-        property :queue
-        property :priority
-        property :job_class, required: true
+      def err_field(field, options, reason = "")
+        schedule = Que::Scheduler.configuration.schedule_location
+        value = options[field]
+        raise "Invalid #{field} '#{value}' for '#{name}' in que-scheduler schedule #{schedule}.\n" \
+              "#{reason}"
       end
 
       def generate_to_enqueue_list(missed_times)
         return [] if missed_times.empty?
 
         options = to_h.slice(:args, :queue, :priority, :job_class).compact
-        args_array = args.is_a?(Array) ? args : Array(args)
 
         if schedule_type == DefinedJob::DEFINED_JOB_TYPE_EVERY_EVENT
           missed_times.map do |time_missed|
-            ToEnqueue.new(options.merge(args: [time_missed] + args_array))
+            ToEnqueue.create(options.merge(args: [time_missed] + args_array))
           end
         else
-          [ToEnqueue.new(options.merge(args: args_array))]
+          [ToEnqueue.create(options.merge(args: args_array))]
         end
       end
     end

data/lib/que/scheduler/enqueueing_calculator.rb

@@ -1,4 +1,4 @@
-require 'fugit'
+require "fugit"
 
 module Que
   module Scheduler

data/lib/que/scheduler/jobs/que_scheduler_audit_clear_down_job.rb

@@ -0,0 +1,45 @@
+require "que"
+
+# This job can optionally be scheduled to clear down the que-scheduler audit log if it
+# isn't required in the long term.
+module Que
+  module Scheduler
+    module Jobs
+      class QueSchedulerAuditClearDownJob < Que::Job
+        class << self
+          def build_sql(table_name)
+            <<-SQL
+              WITH deleted AS (
+                DELETE FROM #{table_name}
+                WHERE scheduler_job_id <= (
+                  SELECT scheduler_job_id FROM que_scheduler_audit
+                  ORDER BY scheduler_job_id DESC
+                  LIMIT 1 OFFSET $1
+                ) RETURNING *
+              ) SELECT count(*) FROM deleted;
+            SQL
+          end
+        end
+
+        DELETE_AUDIT_ENQUEUED_SQL = build_sql("que_scheduler_audit_enqueued").freeze
+        DELETE_AUDIT_SQL = build_sql("que_scheduler_audit").freeze
+
+        # Very low priority
+        Que::Scheduler::VersionSupport.set_priority(self, 100)
+
+        def run(options)
+          retain_row_count = options.fetch(:retain_row_count)
+          Que::Scheduler::Db.transaction do
+            # This may delete zero or more than `retain_row_count` depending on if anything was
+            # scheduled in each of the past schedule runs
+            Que::Scheduler::VersionSupport.execute(DELETE_AUDIT_ENQUEUED_SQL, [retain_row_count])
+            # This will delete all but `retain_row_count` oldest rows
+            count = Que::Scheduler::VersionSupport.execute(DELETE_AUDIT_SQL, [retain_row_count])
+            log = "#{self.class} cleared down #{count.first.fetch(:count)} rows"
+            ::Que.log(event: "que-scheduler".to_sym, message: log)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/que/scheduler/migrations/5/down.sql

@@ -0,0 +1 @@
+CREATE UNIQUE INDEX index_que_scheduler_audit_on_scheduler_job_id ON que_scheduler_audit USING btree (scheduler_job_id);

data/lib/que/scheduler/migrations/5/up.sql

@@ -0,0 +1 @@
+DROP INDEX index_que_scheduler_audit_on_scheduler_job_id;

data/lib/que/scheduler/schedule.rb

@@ -1,4 +1,4 @@
-require_relative 'defined_job'
+require_relative "defined_job"
 
 module Que
   module Scheduler
@@ -12,9 +12,16 @@ module Que
             end
         end
 
+        def schedule=(schedule_config)
+          @schedule = schedule_config.nil? ? nil : from_yaml(schedule_config)
+        end
+
         def from_file(location)
-          yml = IO.read(location)
-          config_hash = YAML.safe_load(yml)
+          from_yaml(IO.read(location))
+        end
+
+        def from_yaml(config)
+          config_hash = YAML.safe_load(config)
           from_hash(config_hash)
         end
 
@@ -24,20 +31,36 @@ module Que
           end.to_h
         end
 
-        private
-
         def hash_item_to_defined_job(name, defined_job_hash)
-          Que::Scheduler::DefinedJob.new(
-            {
-              name: name,
-              job_class: defined_job_hash['class'] || name,
-              queue: defined_job_hash['queue'],
-              args: defined_job_hash['args'],
-              priority: defined_job_hash['priority'],
-              cron: defined_job_hash['cron'],
-              schedule_type: defined_job_hash['schedule_type'],
-            }.compact
-          ).freeze
+          # Que stores arguments as a json array. If the args we have to provide are already an
+          # array we can can simply pass them through. If it is a single non-nil value, then we make
+          # an array with one item which is that value (this includes if it is a hash). It could
+          # also be a single nil value.
+          args_array =
+            if !defined_job_hash.key?("args")
+              # No args were requested
+              []
+            else
+              args = defined_job_hash["args"]
+              if args.is_a?(Array)
+                # An array of args was requested
+                args
+              else
+                # A single value, a nil, or a hash was requested. que expects this to
+                # be enqueued as an array of 1 item
+                [args]
+              end
+            end
+
+          Que::Scheduler::DefinedJob.create(
+            name: name,
+            job_class: defined_job_hash["class"] || name,
+            queue: defined_job_hash["queue"],
+            args_array: args_array,
+            priority: defined_job_hash["priority"],
+            cron: defined_job_hash["cron"],
+            schedule_type: defined_job_hash["schedule_type"]&.to_sym
+          )
         end
       end
     end
@@ -46,6 +69,10 @@ module Que
       def schedule
         Schedule.schedule
       end
+
+      def schedule=(value)
+        Schedule.schedule = value
+      end
     end
   end
 end

data/lib/que/scheduler/scheduler_job.rb

@@ -1,10 +1,13 @@
-require 'que'
+require "que"
+require "active_support"
+require "active_support/core_ext/time"
 
-require_relative 'schedule'
-require_relative 'enqueueing_calculator'
-require_relative 'scheduler_job_args'
-require_relative 'state_checks'
-require_relative 'version_support'
+require_relative "schedule"
+require_relative "enqueueing_calculator"
+require_relative "scheduler_job_args"
+require_relative "state_checks"
+require_relative "to_enqueue"
+require_relative "version_support"
 
 # The main job that runs every minute, determining what needs to be enqueued, enqueues the required
 # jobs, then re-enqueues itself.
@@ -22,46 +25,36 @@ module Que
 
           scheduler_job_args = SchedulerJobArgs.build(options)
           logs = ["que-scheduler last ran at #{scheduler_job_args.last_run_time}."]
-
           result = EnqueueingCalculator.parse(Scheduler.schedule.values, scheduler_job_args)
           enqueued_jobs = enqueue_required_jobs(result, logs)
           enqueue_self_again(
             scheduler_job_args, scheduler_job_args.as_time, result.job_dictionary, enqueued_jobs
           )
-
           # Only now we're sure nothing errored, log the results
-          logs.each { |str| ::Que.log(event: 'que-scheduler'.to_sym, message: str) }
+          logs.each { |str| ::Que.log(event: "que-scheduler".to_sym, message: str) }
           destroy
         end
       end
 
-      def enqueue_required_jobs(result, logs)
-        result.missed_jobs.map do |to_enqueue|
-          job_class = to_enqueue.job_class
-          args = to_enqueue.args
-          remaining_hash = to_enqueue.except(:job_class, :args)
-          enqueued_job =
-            if args.is_a?(Hash)
-              job_class.enqueue(args.merge(remaining_hash))
-            else
-              job_class.enqueue(*args, remaining_hash)
-            end
-          check_enqueued_job(enqueued_job, job_class, args, logs)
+      def enqueue_required_jobs(calculator_result, logs)
+        calculator_result.missed_jobs.map do |to_enqueue|
+          to_enqueue.enqueue.tap do |enqueued_job|
+            check_enqueued_job(to_enqueue, enqueued_job, logs)
+          end
         end.compact
       end
 
       private
 
-      def check_enqueued_job(enqueued_job, job_class, args, logs)
-        if enqueued_job.is_a?(Que::Job)
-          job_id = Que::Scheduler::VersionSupport.job_attributes(enqueued_job).fetch(:job_id)
-          logs << "que-scheduler enqueueing #{job_class} #{job_id} with args: #{args}"
-          enqueued_job
-        else
-          # This can happen if a middleware nixes the enqueue call
-          logs << "que-scheduler called enqueue on #{job_class} but did not receive a #{Que::Job}"
-          nil
-        end
+      def check_enqueued_job(to_enqueue, enqueued_job, logs)
+        logs << if enqueued_job.present?
+                  "que-scheduler enqueueing #{enqueued_job.job_class} " \
+                              "#{enqueued_job.job_id} with args: #{enqueued_job.args}"
+                else
+                  # This can happen if a middleware nixes the enqueue call
+                  "que-scheduler called enqueue on #{to_enqueue.job_class} " \
+                              "but it reported no job was scheduled. Has `enqueue` been overridden?"
+                end
       end
 
       def enqueue_self_again(scheduler_job_args, last_full_execution, job_dictionary, enqueued_jobs)
@@ -79,8 +72,8 @@ module Que
         )
 
         # rubocop:disable Style/GuardClause This reads better as a conditional
-        unless check_enqueued_job(enqueued_job, SchedulerJob, {}, [])
-          raise 'SchedulerJob could not self-schedule. Has `.enqueue` been monkey patched?'
+        unless Que::Scheduler::VersionSupport.job_attributes(enqueued_job).fetch(:job_id)
+          raise "SchedulerJob could not self-schedule. Has `.enqueue` been monkey patched?"
         end
         # rubocop:enable Style/GuardClause
       end

data/lib/que/scheduler/scheduler_job_args.rb

@@ -1,6 +1,6 @@
-require 'hashie'
-require 'active_support'
-require 'active_support/time_with_zone'
+require "hashie"
+require "active_support"
+require "active_support/time_with_zone"
 
 # These are the args that are used for this particular run of the scheduler.
 module Que

data/lib/que/scheduler/state_checks.rb

@@ -1,6 +1,6 @@
-require_relative 'audit'
-require_relative 'db'
-require_relative 'migrations'
+require_relative "audit"
+require_relative "db"
+require_relative "migrations"
 
 module Que
   module Scheduler

data/lib/que/scheduler/to_enqueue.rb

@@ -0,0 +1,157 @@
+require "que"
+
+# This module uses polymorphic dispatch to centralise the differences between supporting Que::Job
+# and other job systems.
+module Que
+  module Scheduler
+    class ToEnqueue < Hashie::Dash
+      property :args, required: true, default: []
+      property :queue
+      property :priority
+      property :run_at, required: true
+      property :job_class, required: true
+
+      class << self
+        def create(options)
+          type_from_job_class(options.fetch(:job_class)).new(
+            options.merge(run_at: Que::Scheduler::Db.now)
+          )
+        end
+
+        def valid_job_class?(job_class)
+          type_from_job_class(job_class).present?
+        end
+
+        def active_job_version
+          Gem.loaded_specs["activejob"]&.version
+        end
+
+        def active_job_sufficient_version?
+          # ActiveJob 4.x does not support job_ids correctly
+          # https://github.com/rails/rails/pull/20056/files
+          active_job_version && active_job_version > Gem::Version.create("5")
+        end
+
+        def active_job_version_supports_queues?
+          # Supporting queue name in ActiveJob was removed in Rails 4.2.3
+          # https://github.com/rails/rails/pull/19498
+          # and readded in Rails 6.0.3
+          # https://github.com/rails/rails/pull/38635
+          ToEnqueue.active_job_version && ToEnqueue.active_job_version >=
+            Gem::Version.create("6.0.3")
+        end
+
+        private
+
+        def type_from_job_class(job_class)
+          types.each do |type, implementation|
+            return implementation if job_class < type
+          end
+          nil
+        end
+
+        def types
+          @types ||=
+            begin
+              hash = {
+                ::Que::Job => QueJobType,
+              }
+              hash[::ActiveJob::Base] = ActiveJobType if ToEnqueue.active_job_sufficient_version?
+              hash
+            end
+        end
+      end
+    end
+
+    # For jobs of type Que::Job
+    class QueJobType < ToEnqueue
+      def enqueue
+        job_settings = to_h.slice(:queue, :priority, :run_at).compact
+        job =
+          if args.is_a?(Hash)
+            job_class.enqueue(**args.merge(job_settings))
+          else
+            job_class.enqueue(*args, **job_settings)
+          end
+
+        return nil if job.nil? || !job # nil in Rails < 6.1, false after.
+
+        # Now read the just inserted job back out of the DB to get the actual values that will
+        # be used when the job is worked.
+        values = Que::Scheduler::VersionSupport.job_attributes(job).slice(
+          :args, :queue, :priority, :run_at, :job_class, :job_id
+        )
+        EnqueuedJobType.new(values)
+      end
+    end
+
+    # For jobs of type ActiveJob
+    class ActiveJobType < ToEnqueue
+      def enqueue
+        job = enqueue_active_job
+
+        return nil if job.nil? || !job # nil in Rails < 6.1, false after.
+
+        enqueued_values = calculate_enqueued_values(job)
+        EnqueuedJobType.new(enqueued_values)
+      end
+
+      def calculate_enqueued_values(job)
+        # Now read the just inserted job back out of the DB to get the actual values that will
+        # be used when the job is worked.
+        data = JSON.parse(job.to_json, symbolize_names: true)
+
+        # ActiveJob scheduled_at is returned as a float, where we want a Time for consistency
+        scheduled_at =
+          begin
+            scheduled_at_float = data[:scheduled_at]
+            scheduled_at_float ? Time.zone.at(scheduled_at_float) : nil
+          end
+
+        # Rails didn't support queues for ActiveJob for a while
+        used_queue = data[:queue_name] if ToEnqueue.active_job_version_supports_queues?
+
+        # We can't get the priority out of the DB, as the returned `job` doesn't give us access
+        # to the underlying ActiveJob that was scheduled. We have no option but to assume
+        # it was what we told it to use. If no priority was specified, we must assume it was
+        # the Que default, which is 100 t.ly/1jRK5
+        assume_used_priority = priority.nil? ? 100 : priority
+
+        {
+          args: data.fetch(:arguments),
+          queue: used_queue,
+          priority: assume_used_priority,
+          run_at: scheduled_at,
+          job_class: job_class.to_s,
+          job_id: data.fetch(:provider_job_id),
+        }
+      end
+
+      def enqueue_active_job
+        job_settings = {
+          priority: priority,
+          wait_until: run_at,
+          queue: queue || Que::Scheduler::VersionSupport.default_scheduler_queue,
+        }.compact
+
+        job_class_set = job_class.set(**job_settings)
+        if args.is_a?(Hash)
+          job_class_set.perform_later(**args)
+        else
+          job_class_set.perform_later(*args)
+        end
+      end
+    end
+
+    # A value object returned after a job has been enqueued. This is necessary as Que (normal) and
+    # ActiveJob return very different objects from the `enqueue` call.
+    class EnqueuedJobType < Hashie::Dash
+      property :args
+      property :queue
+      property :priority
+      property :run_at, required: true
+      property :job_class, required: true
+      property :job_id, required: true
+    end
+  end
+end

data/lib/que/scheduler/version.rb

@@ -1,5 +1,5 @@
 module Que
   module Scheduler
-    VERSION = '3.2.8'.freeze
+    VERSION = "3.4.3".freeze
   end
 end

data/lib/que/scheduler/version_support.rb

@@ -1,4 +1,4 @@
-require 'que'
+require "que"
 
 # The purpose of this module is to centralise the differences when supporting both que 0.x and
 # 1.x with the same gem.
@@ -15,7 +15,7 @@ module Que
         # the top of all jobs it enqueues.
         def set_priority(context, priority)
           if zero_major?
-            context.instance_variable_set('@priority', priority)
+            context.instance_variable_set("@priority", priority)
           else
             context.priority = priority
           end
@@ -24,7 +24,7 @@ module Que
         # Ensure the job runs at least once an hour when it is backing off due to errors
         def apply_retry_semantics(context)
           if zero_major?
-            context.instance_variable_set('@retry_interval', RETRY_PROC)
+            context.instance_variable_set("@retry_interval", RETRY_PROC)
           else
             context.maximum_retry_count = 1 << 128 # Heat death of universe
             context.retry_interval = RETRY_PROC
@@ -33,9 +33,9 @@ module Que
 
         def job_attributes(enqueued_job)
           if zero_major?
-            enqueued_job.attrs.transform_keys(&:to_sym)
+            enqueued_job.attrs.to_h.transform_keys(&:to_sym)
           else
-            enqueued_job.que_attrs.transform_keys(&:to_sym).tap do |hash|
+            enqueued_job.que_attrs.to_h.transform_keys(&:to_sym).tap do |hash|
               hash[:job_id] = hash.delete(:id)
             end
           end
@@ -48,7 +48,7 @@ module Que
         end
 
         def default_scheduler_queue
-          zero_major? ? '' : Que::DEFAULT_QUEUE
+          zero_major? ? "" : Que::DEFAULT_QUEUE
         end
 
         def running_synchronously?
@@ -56,18 +56,18 @@ module Que
         end
 
         def running_synchronously_code?
-          zero_major? ? 'Que.mode == :sync' : 'Que.run_synchronously = true'
+          zero_major? ? "Que.mode == :sync" : "Que.run_synchronously = true"
         end
 
         def zero_major?
           # This is the only way to handle beta releases too
-          @zero_major ||= Gem.loaded_specs['que'].version.to_s.split('.').first.to_i.zero?
+          @zero_major ||= Gem.loaded_specs["que"].version.to_s.split(".").first.to_i.zero?
         end
 
         private
 
         def normalise_array_of_hashes(array)
-          array.map { |row| row.transform_keys(&:to_sym) }
+          array.map { |row| row.to_h.transform_keys(&:to_sym) }
         end
       end
     end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: que-scheduler
 version: !ruby/object:Gem::Version
-  version: 3.2.8
+  version: 3.4.3
 platform: ruby
 authors:
 - Harry Lascelles
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-02-09 00:00:00.000000000 Z
+date: 2020-10-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -24,20 +24,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '4.0'
-- !ruby/object:Gem::Dependency
-  name: backports
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.10'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.10'
 - !ruby/object:Gem::Dependency
   name: fugit
   requirement: !ruby/object:Gem::Requirement
@@ -228,16 +214,16 @@ dependencies:
   name: reek
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '='
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 5.0.2
+        version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '='
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 5.0.2
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -256,16 +242,30 @@ dependencies:
   name: rubocop
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
-        version: 0.68.1
+        version: 0.84.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 0.84.0
+- !ruby/object:Gem::Dependency
+  name: rubocop-rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 0.68.1
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: sqlite3
   requirement: !ruby/object:Gem::Requirement
@@ -323,6 +323,7 @@ files:
 - lib/que/scheduler/db.rb
 - lib/que/scheduler/defined_job.rb
 - lib/que/scheduler/enqueueing_calculator.rb
+- lib/que/scheduler/jobs/que_scheduler_audit_clear_down_job.rb
 - lib/que/scheduler/migrations.rb
 - lib/que/scheduler/migrations/1/down.sql
 - lib/que/scheduler/migrations/1/up.sql
@@ -332,10 +333,13 @@ files:
 - lib/que/scheduler/migrations/3/up.sql
 - lib/que/scheduler/migrations/4/down.sql
 - lib/que/scheduler/migrations/4/up.sql
+- lib/que/scheduler/migrations/5/down.sql
+- lib/que/scheduler/migrations/5/up.sql
 - lib/que/scheduler/schedule.rb
 - lib/que/scheduler/scheduler_job.rb
 - lib/que/scheduler/scheduler_job_args.rb
 - lib/que/scheduler/state_checks.rb
+- lib/que/scheduler/to_enqueue.rb
 - lib/que/scheduler/version.rb
 - lib/que/scheduler/version_support.rb
 homepage: https://github.com/hlascelles/que-scheduler
@@ -347,7 +351,7 @@ metadata:
   changelog_uri: https://github.com/hlascelles/que-scheduler/blob/master/CHANGELOG.md
   source_code_uri: https://github.com/hlascelles/que-scheduler/
   bug_tracker_uri: https://github.com/hlascelles/que-scheduler/issues
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -362,9 +366,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.7.6.2
-signing_key: 
+rubygems_version: 3.0.3
+signing_key:
 specification_version: 4
 summary: A cron scheduler for Que
 test_files: []