que-scheduler 3.3.0 → 4.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5707036df23117e995d696388d17bfc0488eef9c7ba30211b1a3700e6f0fb94f
-  data.tar.gz: 3133e78529f3658c9509e0a1240189424caefc8219b7da035530e68fdbfb38c6
+  metadata.gz: f6250ff590c3c280c0219d93ba57f85758a993fba1185d2064f1ba714c8c9e4c
+  data.tar.gz: cb49ce9c63314888e05f48cc7cd5a98817deb0df03763b9aae756055261acdbb
 SHA512:
-  metadata.gz: 4609167d5adfc2e7b9c583d3cbece89dd34e8624512b0c8c257804c05934964a343336b192944ce2790fde4f4b6029173df2f76d28fe7daee72e0a1ae473bad2
-  data.tar.gz: d55658a7923c50010a8a38713dc25c0f08d182004cb5156a336921f55b35e91631edb042f5e76f194d58ed04245ea8f08c78dd64386c2479ee4bdb8ba9a4512a
+  metadata.gz: 9516e812e2d394d5ff7c23641db0344a34c3560ad4ed02a114cc9820beee8202da9416450fbfc68f92bfaa6aed4f5bf16d1df3d2476f487a58f6f0fb32141aa1
+  data.tar.gz: 4d29b9a9161713935ccf1dcea71303195131fa8c27570b05a528fdec68d146b72c316ebeed523fabecda625c0128e4accca76ba0f47ba63fb8ff1d6106c96120

data/README.md

@@ -18,23 +18,28 @@ needs to be run, enqueueing those jobs, then enqueueing itself to check again la
     ```ruby
     gem 'que-scheduler'
     ```
-1. Specify a schedule in a yml file (see below). The default location that que-scheduler will 
-look for it is `config/que_schedule.yml`. They are essentially the same as resque-scheduler
-files, but with additional features.
+1. Specify a schedule in a yml file or programmatically (see below). The default location that 
+que-scheduler will look for it is `config/que_schedule.yml`. The format is essentially the same as
+resque-scheduler files, but with additional features.
 
-1. Add a migration to start the job scheduler and prepare the audit table. Note that this migration will fail if Que is set to execute jobs synchronously, i.e. `Que::Job.run_synchronously = true`.
+1. Add a migration to start the job scheduler and prepare the audit table. Note that this migration 
+   will fail if Que is set to execute jobs synchronously, i.e. `Que::Job.run_synchronously = true`.
 
     ```ruby
     class CreateQueSchedulerSchema < ActiveRecord::Migration
       def change
-        Que::Scheduler::Migrations.migrate!(version: 4)
+        Que::Scheduler::Migrations.migrate!(version: 6)
       end
     end
     ```
     
 ## Schedule configuration
 
-The schedule file is a list of que job classes with arguments and a schedule frequency (in crontab 
+The schedule file should be placed here: `config/que_schedule.yml`. Alternatively if you
+wish to generate the configuration dynamically, you can set it directly with
+`Que::Scheduler.schedule = some_hash`.
+
+The file is a list of que job classes with arguments and a schedule frequency (in crontab 
 syntax). The format is similar to the resque-scheduler format, though priorities must be supplied as
 integers, and job classes must be migrated from Resque to Que. Cron syntax can be anything
 understood by [fugit](https://github.com/floraison/fugit#fugitcron).
@@ -65,10 +70,21 @@ BatchOrders:
   cron: "0 0 * * *"
   priority: 25
   
-# Specify job arguments
+# Specify array job arguments
 SendOrders:
   cron: "0 0 * * *"
   args: ['open']
+
+# Specify hash job arguments. Note, this appears as a single hash to `run`, not as kwargs.
+SendPreorders:
+  cron: "0 0 * * *"
+  args:
+    order_type: special
+  
+# Specify a single nil argument
+SendPostorders:
+  cron: "0 0 * * *"
+  args: ~ # See https://stackoverflow.com/a/51990876/1267203
   
 # Use simpler cron syntax
 SendBilling:
@@ -136,8 +152,58 @@ end
 
 ## Scheduler Audit
 
-An audit table _que_scheduler_audit_ is written to by the scheduler to keep a history of what jobs 
-were enqueued when. It is created by the included migration tasks.
+An audit table `que_scheduler_audit` is written to by the scheduler to keep a history of when the 
+scheduler ran to calculate what was necessary to run (if anything). It is created by the included 
+migration tasks.
+
+Additionally, there is the audit table `que_scheduler_audit_enqueued`. This logs every job that 
+the scheduler enqueues.
+
+When there is a major version (breaking) change, a migration should be run in. The version of the 
+latest migration proceeds at a faster rate than the version of the gem, eg if the gem is on version
+3 then the migrations may be on version 6). 
+
+To run in all the migrations required up to a number, just migrate to that number with one line, and
+it will perform all the intermediary steps. 
+
+ie, This will perform all migrations necessary up to the latest version, skipping any already 
+performed.
+
+```ruby
+class CreateQueSchedulerSchema < ActiveRecord::Migration
+  def change
+    Que::Scheduler::Migrations.migrate!(version: 6)
+  end
+end
+```
+
+The changes in past migrations were: 
+
+| Version | Changes                                                                         |
+|:-------:|---------------------------------------------------------------------------------|
+|    1    | Enqueued the main Que::Scheduler. This is the job that performs the scheduling. |
+|    2    | Added the audit table `que_scheduler_audit`.                                    |
+|    3    | Added the audit table `que_scheduler_audit_enqueued`.                           |
+|    4    | Updated the the audit tables to use bigints                                     |
+|    5    | Dropped an unnecessary index                                                    |
+|    6    | Enforced single scheduler job at the trigger level                              |
+
+## Built in optional job for audit clear down
+
+que-scheduler comes with the `QueSchedulerAuditClearDownJob` job built in that you can optionally
+schedule to clear down audit rows if you don't need to retain them indefinitely. You should add this
+to your own scheduler config yaml.
+
+For example:
+
+```yaml
+# This will clear down the oldest que-scheduler audit rows. Since que-scheduler
+# runs approximately every minute, 129600 is 90 days.
+Que::Scheduler::Jobs::QueSchedulerAuditClearDownJob:
+  cron: "0 0 * * *"
+  args:
+    retain_row_count: 129600
+```
 
 ## HA Redundancy and DB restores
 
@@ -152,8 +218,8 @@ in a coherent state with the rest of your database.
 ## Concurrent scheduler detection
 
 No matter how many tasks you have defined in your schedule, you will only ever need one que-scheduler
-job enqueued. que-scheduler knows this, and it will check before performing any operations that 
-there is only one of itself present.
+job enqueued. que-scheduler knows this, and there are DB constraints in place to ensure there is
+only ever exactly one scheduler job.
 
 It also follows que job design [best practices](https://github.com/chanks/que/blob/master/docs/writing_reliable_jobs.md),
 using ACID guarantees, to ensure that it will never run multiple times. If the scheduler crashes for any reason,
@@ -172,23 +238,6 @@ then reschedules itself. The flow is as follows:
 1. After a deploy that changes the schedule, the job notices any new jobs to schedule, and knows which
    ones to forget. It does not need to be re-enqueued or restarted.
    
-## DB Migrations
-
-When there is a major version (breaking) change, a migration should be run in. The version of the 
-migration proceeds at a faster rate than the version of the gem. To run in all the migrations required
-up to a number, just migrate to that number with one line, and it will perform all the intermediary steps. 
-
-ie, `Que::Scheduler::Migrations.migrate!(version: 4)` will perform all migrations necessary to 
-reach migration version `4`.
-
-As of migration `4`, two elements are added to the DB for que-scheduler to run. 
-
-1. The first is the scheduler job itself, which runs forever, re-enqueuing itself to performs its 
-   duties.
-1. The second part comprises the audit table `que_scheduler_audit` and the "enqueued" table 
-  `que_scheduler_audit_enqueued`. The first tracks when the scheduler calculated what was necessary to run 
-  (if anything). The second then logs every job that the scheduler enqueues. 
-
 ## Testing Configuration
 
 You can add tests to validate your configuration during the spec phase. This will perform a variety 
@@ -222,6 +271,10 @@ The scheduler will then continue to retry indefinitely.
 que-scheduler uses [semantic versioning](https://semver.org/), so major version changes will usually 
 require additional actions to be taken upgrading from one major version to another. 
 
+## Changelog
+
+A full changelog can be found here: [CHANGELOG.md](https://github.com/hlascelles/que-scheduler/blob/master/CHANGELOG.md)
+
 ## System requirements
 
 Your [postgres](https://www.postgresql.org/) database must be at least version 9.4.0.
@@ -232,7 +285,9 @@ This gem was inspired by the makers of the excellent [Que](https://github.com/ch
 
 ## Contributors
 
+* @bnauta
+* @JackDanger
 * @jish
 * @joehorsnell
-* @bnauta
+* @krzyzak
 * @papodaca

data/lib/que-scheduler.rb

@@ -1,3 +1,3 @@
 # rubocop:disable Naming/FileName
-require 'que/scheduler'
+require "que/scheduler"
 # rubocop:enable Naming/FileName

data/lib/que/scheduler.rb

@@ -1,7 +1,8 @@
-require 'que/scheduler/version'
-require 'que/scheduler/version_support'
-require 'que/scheduler/config'
-require 'que/scheduler/scheduler_job'
-require 'que/scheduler/db'
-require 'que/scheduler/audit'
-require 'que/scheduler/migrations'
+require "que/scheduler/version"
+require "que/scheduler/version_support"
+require "que/scheduler/config"
+require "que/scheduler/scheduler_job"
+require "que/scheduler/db"
+require "que/scheduler/audit"
+require "que/scheduler/migrations"
+require "que/scheduler/jobs/que_scheduler_audit_clear_down_job"

data/lib/que/scheduler/audit.rb

@@ -1,12 +1,12 @@
 # frozen_string_literal: true
 
-require_relative 'to_enqueue'
+require_relative "to_enqueue"
 
 module Que
   module Scheduler
     module Audit
-      TABLE_NAME = 'que_scheduler_audit'
-      ENQUEUED_TABLE_NAME = 'que_scheduler_audit_enqueued'
+      TABLE_NAME = "que_scheduler_audit"
+      ENQUEUED_TABLE_NAME = "que_scheduler_audit_enqueued"
       INSERT_AUDIT = %{
         INSERT INTO #{TABLE_NAME} (scheduler_job_id, executed_at)
         VALUES ($1::bigint, $2::timestamptz)

data/lib/que/scheduler/config.rb

@@ -1,5 +1,5 @@
-require 'que'
-require_relative 'version_support'
+require "que"
+require_relative "version_support"
 
 module Que
   module Scheduler
@@ -21,7 +21,7 @@ module Que
 end
 
 Que::Scheduler.configure do |config|
-  config.schedule_location = ENV.fetch('QUE_SCHEDULER_CONFIG_LOCATION', 'config/que_schedule.yml')
+  config.schedule_location = ENV.fetch("QUE_SCHEDULER_CONFIG_LOCATION", "config/que_schedule.yml")
   config.transaction_adapter = ::Que.method(:transaction)
   config.que_scheduler_queue = Que::Scheduler::VersionSupport.default_scheduler_queue
 end

data/lib/que/scheduler/db.rb

@@ -1,13 +1,13 @@
 # frozen_string_literal: true
 
-require_relative 'config'
+require_relative "config"
 
 module Que
   module Scheduler
     module Db
       SCHEDULER_COUNT_SQL =
         "SELECT COUNT(*) FROM que_jobs WHERE job_class = 'Que::Scheduler::SchedulerJob'"
-      NOW_SQL = 'SELECT now()'
+      NOW_SQL = "SELECT now()"
 
       class << self
         def count_schedulers

data/lib/que/scheduler/defined_job.rb

@@ -1,6 +1,5 @@
-require 'hashie'
-require 'fugit'
-require 'backports/2.4.0/hash/compact'
+require "hashie"
+require "fugit"
 
 # This is the definition of one scheduleable job in the que-scheduler config yml file.
 module Que
@@ -18,7 +17,7 @@ module Que
       property :cron, transform_with: ->(v) { Fugit::Cron.parse(v) }
       property :queue
       property :priority
-      property :args
+      property :args_array
       property :schedule_type, default: DEFINED_JOB_TYPE_DEFAULT
 
       class << self
@@ -60,10 +59,10 @@ module Que
       # rubocop:disable Style/GuardClause This reads better as a conditional
       def validate_fields_types(options)
         unless queue.nil? || queue.is_a?(String)
-          err_field(:queue, options, 'queue must be a string')
+          err_field(:queue, options, "queue must be a string")
         end
         unless priority.nil? || priority.is_a?(Integer)
-          err_field(:priority, options, 'priority must be an integer')
+          err_field(:priority, options, "priority must be an integer")
         end
         unless DEFINED_JOB_TYPES.include?(schedule_type)
           err_field(:schedule_type, options, "Not in #{DEFINED_JOB_TYPES}")
@@ -72,10 +71,10 @@ module Que
       # rubocop:enable Style/GuardClause
 
       def validate_fields_presence(options)
-        err_field(:name, options, 'name must be present') if name.nil?
-        err_field(:job_class, options, 'job_class must be present') if job_class.nil?
+        err_field(:name, options, "name must be present") if name.nil?
+        err_field(:job_class, options, "job_class must be present") if job_class.nil?
         # An invalid cron is nil
-        err_field(:cron, options, 'cron must be present') if cron.nil?
+        err_field(:cron, options, "cron must be present") if cron.nil?
       end
 
       def validate_job_class_related(options)
@@ -89,7 +88,7 @@ module Que
         if queue &&
            Que::Scheduler::ToEnqueue.active_job_sufficient_version? &&
            job_class < ::ActiveJob::Base &&
-           Que::Scheduler::ToEnqueue.active_job_version < Gem::Version.create('6.0.3')
+           Que::Scheduler::ToEnqueue.active_job_version < Gem::Version.create("6.0.3")
           puts <<-ERR
             WARNING from que-scheduler....
             Between versions 4.2.3 and 6.0.2 (inclusive) Rails did not support setting queue names
@@ -105,7 +104,7 @@ module Que
         end
       end
 
-      def err_field(field, options, reason = '')
+      def err_field(field, options, reason = "")
         schedule = Que::Scheduler.configuration.schedule_location
         value = options[field]
         raise "Invalid #{field} '#{value}' for '#{name}' in que-scheduler schedule #{schedule}.\n" \
@@ -116,7 +115,6 @@ module Que
         return [] if missed_times.empty?
 
         options = to_h.slice(:args, :queue, :priority, :job_class).compact
-        args_array = args.is_a?(Array) ? args : Array(args)
 
         if schedule_type == DefinedJob::DEFINED_JOB_TYPE_EVERY_EVENT
           missed_times.map do |time_missed|

data/lib/que/scheduler/enqueueing_calculator.rb

@@ -1,4 +1,4 @@
-require 'fugit'
+require "fugit"
 
 module Que
   module Scheduler

data/lib/que/scheduler/jobs/que_scheduler_audit_clear_down_job.rb

@@ -0,0 +1,45 @@
+require "que"
+
+# This job can optionally be scheduled to clear down the que-scheduler audit log if it
+# isn't required in the long term.
+module Que
+  module Scheduler
+    module Jobs
+      class QueSchedulerAuditClearDownJob < Que::Job
+        class << self
+          def build_sql(table_name)
+            <<-SQL
+              WITH deleted AS (
+                DELETE FROM #{table_name}
+                WHERE scheduler_job_id <= (
+                  SELECT scheduler_job_id FROM que_scheduler_audit
+                  ORDER BY scheduler_job_id DESC
+                  LIMIT 1 OFFSET $1
+                ) RETURNING *
+              ) SELECT count(*) FROM deleted;
+            SQL
+          end
+        end
+
+        DELETE_AUDIT_ENQUEUED_SQL = build_sql("que_scheduler_audit_enqueued").freeze
+        DELETE_AUDIT_SQL = build_sql("que_scheduler_audit").freeze
+
+        # Very low priority
+        Que::Scheduler::VersionSupport.set_priority(self, 100)
+
+        def run(options)
+          retain_row_count = options.fetch(:retain_row_count)
+          Que::Scheduler::Db.transaction do
+            # This may delete zero or more than `retain_row_count` depending on if anything was
+            # scheduled in each of the past schedule runs
+            Que::Scheduler::VersionSupport.execute(DELETE_AUDIT_ENQUEUED_SQL, [retain_row_count])
+            # This will delete all but `retain_row_count` oldest rows
+            count = Que::Scheduler::VersionSupport.execute(DELETE_AUDIT_SQL, [retain_row_count])
+            log = "#{self.class} cleared down #{count.first.fetch(:count)} rows"
+            ::Que.log(event: "que-scheduler".to_sym, message: log)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/que/scheduler/migrations/5/down.sql

@@ -0,0 +1 @@
+CREATE UNIQUE INDEX index_que_scheduler_audit_on_scheduler_job_id ON que_scheduler_audit USING btree (scheduler_job_id);

data/lib/que/scheduler/migrations/5/up.sql

@@ -0,0 +1 @@
+DROP INDEX index_que_scheduler_audit_on_scheduler_job_id;

data/lib/que/scheduler/migrations/6/down.sql

@@ -0,0 +1,7 @@
+DROP TRIGGER que_scheduler_prevent_job_deletion_trigger ON que_jobs;
+
+DROP FUNCTION que_scheduler_prevent_job_deletion();
+
+DROP FUNCTION que_scheduler_check_job_exists();
+
+DROP INDEX que_scheduler_job_in_que_jobs_unique_index;

data/lib/que/scheduler/migrations/6/up.sql

@@ -0,0 +1,26 @@
+-- Ensure there is no more than one scheduler
+CREATE UNIQUE INDEX que_scheduler_job_in_que_jobs_unique_index ON que_jobs(job_class)
+WHERE job_class = 'Que::Scheduler::SchedulerJob';
+
+-- Ensure there is at least one scheduler
+CREATE OR REPLACE FUNCTION que_scheduler_check_job_exists() RETURNS bool AS $$
+SELECT EXISTS(SELECT * FROM que_jobs WHERE job_class = 'Que::Scheduler::SchedulerJob');
+$$ LANGUAGE SQL;
+
+CREATE OR REPLACE FUNCTION que_scheduler_prevent_job_deletion() RETURNS TRIGGER AS
+$BODY$
+DECLARE
+BEGIN
+    IF OLD.job_class = 'Que::Scheduler::SchedulerJob' THEN
+        IF NOT que_scheduler_check_job_exists() THEN
+            raise exception 'Deletion of que_scheduler job % prevented. Deleting the que_scheduler job is almost certainly a mistake.', OLD.job_id;
+        END IF;
+    END IF;
+    RETURN OLD;
+END;
+$BODY$
+LANGUAGE 'plpgsql';
+
+CREATE CONSTRAINT TRIGGER que_scheduler_prevent_job_deletion_trigger AFTER UPDATE OR DELETE ON que_jobs
+DEFERRABLE INITIALLY DEFERRED
+FOR EACH ROW EXECUTE PROCEDURE que_scheduler_prevent_job_deletion();

data/lib/que/scheduler/schedule.rb

@@ -1,4 +1,4 @@
-require_relative 'defined_job'
+require_relative "defined_job"
 
 module Que
   module Scheduler
@@ -12,9 +12,16 @@ module Que
             end
         end
 
+        def schedule=(schedule_config)
+          @schedule = schedule_config.nil? ? nil : from_yaml(schedule_config)
+        end
+
         def from_file(location)
-          yml = IO.read(location)
-          config_hash = YAML.safe_load(yml)
+          from_yaml(IO.read(location))
+        end
+
+        def from_yaml(config)
+          config_hash = YAML.safe_load(config)
           from_hash(config_hash)
         end
 
@@ -24,17 +31,35 @@ module Que
           end.to_h
         end
 
-        private
-
         def hash_item_to_defined_job(name, defined_job_hash)
+          # Que stores arguments as a json array. If the args we have to provide are already an
+          # array we can can simply pass them through. If it is a single non-nil value, then we make
+          # an array with one item which is that value (this includes if it is a hash). It could
+          # also be a single nil value.
+          args_array =
+            if !defined_job_hash.key?("args")
+              # No args were requested
+              []
+            else
+              args = defined_job_hash["args"]
+              if args.is_a?(Array)
+                # An array of args was requested
+                args
+              else
+                # A single value, a nil, or a hash was requested. que expects this to
+                # be enqueued as an array of 1 item
+                [args]
+              end
+            end
+
           Que::Scheduler::DefinedJob.create(
             name: name,
-            job_class: defined_job_hash['class'] || name,
-            queue: defined_job_hash['queue'],
-            args: defined_job_hash['args'],
-            priority: defined_job_hash['priority'],
-            cron: defined_job_hash['cron'],
-            schedule_type: defined_job_hash['schedule_type']&.to_sym
+            job_class: defined_job_hash["class"] || name,
+            queue: defined_job_hash["queue"],
+            args_array: args_array,
+            priority: defined_job_hash["priority"],
+            cron: defined_job_hash["cron"],
+            schedule_type: defined_job_hash["schedule_type"]&.to_sym
           )
         end
       end
@@ -44,6 +69,10 @@ module Que
       def schedule
         Schedule.schedule
       end
+
+      def schedule=(value)
+        Schedule.schedule = value
+      end
     end
   end
 end

data/lib/que/scheduler/scheduler_job.rb

@@ -1,11 +1,13 @@
-require 'que'
+require "que"
+require "active_support"
+require "active_support/core_ext/time"
 
-require_relative 'schedule'
-require_relative 'enqueueing_calculator'
-require_relative 'scheduler_job_args'
-require_relative 'state_checks'
-require_relative 'to_enqueue'
-require_relative 'version_support'
+require_relative "schedule"
+require_relative "enqueueing_calculator"
+require_relative "scheduler_job_args"
+require_relative "state_checks"
+require_relative "to_enqueue"
+require_relative "version_support"
 
 # The main job that runs every minute, determining what needs to be enqueued, enqueues the required
 # jobs, then re-enqueues itself.
@@ -14,8 +16,8 @@ module Que
     class SchedulerJob < Que::Job
       SCHEDULER_FREQUENCY = 60
 
-      Que::Scheduler::VersionSupport.set_priority(self, 0)
-      Que::Scheduler::VersionSupport.apply_retry_semantics(self)
+      VersionSupport.set_priority(self, 0)
+      VersionSupport.apply_retry_semantics(self)
 
       def run(options = nil)
         Que::Scheduler::Db.transaction do
@@ -25,17 +27,18 @@ module Que
           logs = ["que-scheduler last ran at #{scheduler_job_args.last_run_time}."]
           result = EnqueueingCalculator.parse(Scheduler.schedule.values, scheduler_job_args)
           enqueued_jobs = enqueue_required_jobs(result, logs)
+          # Remove this job and schedule self again
+          destroy
           enqueue_self_again(
             scheduler_job_args, scheduler_job_args.as_time, result.job_dictionary, enqueued_jobs
           )
           # Only now we're sure nothing errored, log the results
-          logs.each { |str| ::Que.log(event: 'que-scheduler'.to_sym, message: str) }
-          destroy
+          logs.each { |str| ::Que.log(event: "que-scheduler".to_sym, message: str) }
         end
       end
 
-      def enqueue_required_jobs(result, logs)
-        result.missed_jobs.map do |to_enqueue|
+      def enqueue_required_jobs(calculator_result, logs)
+        calculator_result.missed_jobs.map do |to_enqueue|
           to_enqueue.enqueue.tap do |enqueued_job|
             check_enqueued_job(to_enqueue, enqueued_job, logs)
           end
@@ -51,13 +54,13 @@ module Que
                 else
                   # This can happen if a middleware nixes the enqueue call
                   "que-scheduler called enqueue on #{to_enqueue.job_class} " \
-                              'but it reported no job was scheduled. Has `enqueue` been overridden?'
+                              "but it reported no job was scheduled. Has `enqueue` been overridden?"
                 end
       end
 
       def enqueue_self_again(scheduler_job_args, last_full_execution, job_dictionary, enqueued_jobs)
         # Log last run...
-        job_id = Que::Scheduler::VersionSupport.job_attributes(self).fetch(:job_id)
+        job_id = VersionSupport.job_attributes(self).fetch(:job_id)
         Audit.append(job_id, scheduler_job_args.as_time, enqueued_jobs)
 
         # And rerun...
@@ -70,8 +73,8 @@ module Que
         )
 
         # rubocop:disable Style/GuardClause This reads better as a conditional
-        unless Que::Scheduler::VersionSupport.job_attributes(enqueued_job).fetch(:job_id)
-          raise 'SchedulerJob could not self-schedule. Has `.enqueue` been monkey patched?'
+        unless enqueued_job && VersionSupport.job_attributes(enqueued_job).fetch(:job_id)
+          raise "SchedulerJob could not self-schedule. Has `.enqueue` been monkey patched?"
         end
         # rubocop:enable Style/GuardClause
       end

data/lib/que/scheduler/scheduler_job_args.rb

@@ -1,6 +1,6 @@
-require 'hashie'
-require 'active_support'
-require 'active_support/time_with_zone'
+require "hashie"
+require "active_support"
+require "active_support/time_with_zone"
 
 # These are the args that are used for this particular run of the scheduler.
 module Que

data/lib/que/scheduler/state_checks.rb

@@ -1,6 +1,6 @@
-require_relative 'audit'
-require_relative 'db'
-require_relative 'migrations'
+require_relative "audit"
+require_relative "db"
+require_relative "migrations"
 
 module Que
   module Scheduler
@@ -8,7 +8,6 @@ module Que
       class << self
         def check
           assert_db_migrated
-          assert_one_scheduler_job
         end
 
         private
@@ -60,21 +59,6 @@ module Que
             synchronously. This will fail as que-scheduler needs the above tables to work.
           ERR
         end
-
-        def assert_one_scheduler_job
-          schedulers = Que::Scheduler::Db.count_schedulers
-          return if schedulers == 1
-
-          raise(<<-ERR)
-            Only one #{Que::Scheduler::SchedulerJob.name} should be enqueued. #{schedulers} were found.
-
-            que-scheduler works by running a self-enqueueing version of itself that determines which
-            jobs should be enqueued based on the provided config. If two or more que-schedulers were
-            to run at once, then duplicate jobs would occur.
-
-            To resolve this problem, please remove any duplicate scheduler jobs from the que_jobs table.
-          ERR
-        end
       end
     end
   end

data/lib/que/scheduler/to_enqueue.rb

@@ -1,4 +1,4 @@
-require 'que'
+require "que"
 
 # This module uses polymorphic dispatch to centralise the differences between supporting Que::Job
 # and other job systems.
@@ -23,13 +23,22 @@ module Que
         end
 
         def active_job_version
-          Gem.loaded_specs['activejob']&.version
+          Gem.loaded_specs["activejob"]&.version
         end
 
         def active_job_sufficient_version?
           # ActiveJob 4.x does not support job_ids correctly
           # https://github.com/rails/rails/pull/20056/files
-          active_job_version && active_job_version > Gem::Version.create('5')
+          active_job_version && active_job_version > Gem::Version.create("5")
+        end
+
+        def active_job_version_supports_queues?
+          # Supporting queue name in ActiveJob was removed in Rails 4.2.3
+          # https://github.com/rails/rails/pull/19498
+          # and readded in Rails 6.0.3
+          # https://github.com/rails/rails/pull/38635
+          ToEnqueue.active_job_version && ToEnqueue.active_job_version >=
+            Gem::Version.create("6.0.3")
         end
 
         private
@@ -87,8 +96,6 @@ module Que
         EnqueuedJobType.new(enqueued_values)
       end
 
-      private
-
       def calculate_enqueued_values(job)
         # Now read the just inserted job back out of the DB to get the actual values that will
         # be used when the job is worked.
@@ -101,9 +108,8 @@ module Que
             scheduled_at_float ? Time.zone.at(scheduled_at_float) : nil
           end
 
-        # Rails doesn't support queues for ActiveJob
-        # https://github.com/rails/rails/pull/19498
-        used_queue = nil
+        # Rails didn't support queues for ActiveJob for a while
+        used_queue = data[:queue_name] if ToEnqueue.active_job_version_supports_queues?
 
         # We can't get the priority out of the DB, as the returned `job` doesn't give us access
         # to the underlying ActiveJob that was scheduled. We have no option but to assume
@@ -125,6 +131,7 @@ module Que
         job_settings = {
           priority: priority,
           wait_until: run_at,
+          queue: queue || Que::Scheduler::VersionSupport.default_scheduler_queue,
         }.compact
 
         job_class_set = job_class.set(**job_settings)

data/lib/que/scheduler/version.rb

@@ -1,5 +1,5 @@
 module Que
   module Scheduler
-    VERSION = '3.3.0'.freeze
+    VERSION = "4.0.0".freeze
   end
 end

data/lib/que/scheduler/version_support.rb

@@ -1,4 +1,4 @@
-require 'que'
+require "que"
 
 # The purpose of this module is to centralise the differences when supporting both que 0.x and
 # 1.x with the same gem.
@@ -15,7 +15,7 @@ module Que
         # the top of all jobs it enqueues.
         def set_priority(context, priority)
           if zero_major?
-            context.instance_variable_set('@priority', priority)
+            context.instance_variable_set("@priority", priority)
           else
             context.priority = priority
           end
@@ -24,7 +24,7 @@ module Que
         # Ensure the job runs at least once an hour when it is backing off due to errors
         def apply_retry_semantics(context)
           if zero_major?
-            context.instance_variable_set('@retry_interval', RETRY_PROC)
+            context.instance_variable_set("@retry_interval", RETRY_PROC)
           else
             context.maximum_retry_count = 1 << 128 # Heat death of universe
             context.retry_interval = RETRY_PROC
@@ -33,9 +33,9 @@ module Que
 
         def job_attributes(enqueued_job)
           if zero_major?
-            enqueued_job.attrs.transform_keys(&:to_sym)
+            enqueued_job.attrs.to_h.transform_keys(&:to_sym)
           else
-            enqueued_job.que_attrs.transform_keys(&:to_sym).tap do |hash|
+            enqueued_job.que_attrs.to_h.transform_keys(&:to_sym).tap do |hash|
               hash[:job_id] = hash.delete(:id)
             end
           end
@@ -48,7 +48,7 @@ module Que
         end
 
         def default_scheduler_queue
-          zero_major? ? '' : Que::DEFAULT_QUEUE
+          zero_major? ? "" : Que::DEFAULT_QUEUE
         end
 
         def running_synchronously?
@@ -56,18 +56,18 @@ module Que
         end
 
         def running_synchronously_code?
-          zero_major? ? 'Que.mode == :sync' : 'Que.run_synchronously = true'
+          zero_major? ? "Que.mode == :sync" : "Que.run_synchronously = true"
         end
 
         def zero_major?
           # This is the only way to handle beta releases too
-          @zero_major ||= Gem.loaded_specs['que'].version.to_s.split('.').first.to_i.zero?
+          @zero_major ||= Gem.loaded_specs["que"].version.to_s.split(".").first.to_i.zero?
         end
 
         private
 
         def normalise_array_of_hashes(array)
-          array.map { |row| row.transform_keys(&:to_sym) }
+          array.map { |row| row.to_h.transform_keys(&:to_sym) }
         end
       end
     end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: que-scheduler
 version: !ruby/object:Gem::Version
-  version: 3.3.0
+  version: 4.0.0
 platform: ruby
 authors:
 - Harry Lascelles
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-04-04 00:00:00.000000000 Z
+date: 2020-11-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -24,20 +24,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '4.0'
-- !ruby/object:Gem::Dependency
-  name: backports
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.10'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '3.10'
 - !ruby/object:Gem::Dependency
   name: fugit
   requirement: !ruby/object:Gem::Requirement
@@ -258,14 +244,28 @@ dependencies:
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 0.80.0
+        version: 0.84.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 0.80.0
+        version: 0.84.0
+- !ruby/object:Gem::Dependency
+  name: rubocop-rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: sqlite3
   requirement: !ruby/object:Gem::Requirement
@@ -323,6 +323,7 @@ files:
 - lib/que/scheduler/db.rb
 - lib/que/scheduler/defined_job.rb
 - lib/que/scheduler/enqueueing_calculator.rb
+- lib/que/scheduler/jobs/que_scheduler_audit_clear_down_job.rb
 - lib/que/scheduler/migrations.rb
 - lib/que/scheduler/migrations/1/down.sql
 - lib/que/scheduler/migrations/1/up.sql
@@ -332,6 +333,10 @@ files:
 - lib/que/scheduler/migrations/3/up.sql
 - lib/que/scheduler/migrations/4/down.sql
 - lib/que/scheduler/migrations/4/up.sql
+- lib/que/scheduler/migrations/5/down.sql
+- lib/que/scheduler/migrations/5/up.sql
+- lib/que/scheduler/migrations/6/down.sql
+- lib/que/scheduler/migrations/6/up.sql
 - lib/que/scheduler/schedule.rb
 - lib/que/scheduler/scheduler_job.rb
 - lib/que/scheduler/scheduler_job_args.rb
@@ -348,7 +353,7 @@ metadata:
   changelog_uri: https://github.com/hlascelles/que-scheduler/blob/master/CHANGELOG.md
   source_code_uri: https://github.com/hlascelles/que-scheduler/
   bug_tracker_uri: https://github.com/hlascelles/que-scheduler/issues
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -364,7 +369,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubygems_version: 3.0.3
-signing_key: 
+signing_key:
 specification_version: 4
 summary: A cron scheduler for Que
 test_files: []