sidekiq-cron 1.9.1 → 1.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4d751ce0280b4d30f1f43a1e0ed973166c2f398d7ac976e6fea267e66e999bb8
-  data.tar.gz: 352b64238d89eb4eedb0e9c046f9d54f6f298c376804de1066cbdad3481c43af
+  metadata.gz: aaa6af217265c4e60b29018b984b43a4316b2618e52ce874db670e41ecadf85b
+  data.tar.gz: f70a50a90508ceb32bad90b8b2b6a880fb460f308dfa81f7380152742bb939cf
 SHA512:
-  metadata.gz: ee7843b8dd62b31954de3db17c4cef76229050d2c4590bde276c77d6ad2c96bbb523620631ad41574d4b976ace4da6425130b1aa5c4a20986c4d42fdcdde1ef4
-  data.tar.gz: 95e74c836bdae7eb10fe1fe540fdc91ecb730c3a73bc1dd7e578d9324cf448e32054e9ad1b487036119076e966b0df1011978aa34b9215a334d996759d4bb684
+  metadata.gz: ef95b33d15c1867a3dc6cc096080af6f2fab50c4460d8be24948bcd887c6c278dc3ca1a8c92db8d1ce386d24e0ceb95f53f6add336c3315fec04f808b7451475
+  data.tar.gz: e30e02e3bcc13f8604426d5d9e5f30e24c46a85bfd7679544975a3bdfcc0510631d363ef041f948c8a51e289a02a222fd27bba572cf57ac1697aabb917673a01

data/CHANGELOG.md

@@ -2,6 +2,32 @@
 
 All notable changes to this project will be documented in this file.
 
+## 1.12.0
+
+- Remove Sidekiq.server? check from schedule loader (https://github.com/sidekiq-cron/sidekiq-cron/pull/436)
+- Parse arguments on `args=` method (https://github.com/sidekiq-cron/sidekiq-cron/pull/442)
+- Only check out a Redis connection if necessary (https://github.com/sidekiq-cron/sidekiq-cron/pull/438)
+
+## 1.11.0
+
+- Differentiates b/w "schedule" vs "dynamic" jobs (https://github.com/sidekiq-cron/sidekiq-cron/pull/431)
+- Clears scheduled jobs upon schedule load (https://github.com/sidekiq-cron/sidekiq-cron/pull/431)
+- Reduce gem size by excluding test files (https://github.com/sidekiq-cron/sidekiq-cron/pull/414)
+
+## 1.10.1
+
+- Use `hset` instead of deprecated `hmset` (https://github.com/sidekiq-cron/sidekiq-cron/pull/410)
+
+## 1.10.0
+
+- Remove EOL Ruby 2.6 support (https://github.com/sidekiq-cron/sidekiq-cron/pull/399)
+- Add a logo for the project! (https://github.com/sidekiq-cron/sidekiq-cron/pull/402)
+- Added support for ActiveRecord serialize/deserialize using GlobalID (https://github.com/sidekiq-cron/sidekiq-cron/pull/395)
+- Allow for keyword args (`embedded: true`) in Poller (https://github.com/sidekiq-cron/sidekiq-cron/pull/398)
+- Make last_enqueue_time be always an instance of Time (https://github.com/sidekiq-cron/sidekiq-cron/pull/354)
+- Fix argument error problem update from 1.6.0 to newer (https://github.com/sidekiq-cron/sidekiq-cron/pull/392)
+- Clear old jobs while loading the jobs from schedule via the schedule loader (https://github.com/sidekiq-cron/sidekiq-cron/pull/405)
+
 ## 1.9.1
 
 - Always enqueue via Active Job interface when defined in cron job config (https://github.com/sidekiq-cron/sidekiq-cron/pull/381)
@@ -45,7 +71,7 @@ All notable changes to this project will be documented in this file.
 ## 1.4.0
 
 - Fix buttons order in job show view (https://github.com/sidekiq-cron/sidekiq-cron/pull/302)
-- Dark Mode support in UI (https://github.com/sidekiq-cron/sidekiq-cron/pull/317/282)
+- Dark Mode support in UI (https://github.com/sidekiq-cron/sidekiq-cron/pull/282)
 - Remove invocation of deprecated Redis functionality (https://github.com/sidekiq-cron/sidekiq-cron/pull/318)
 - Internal code cleanup (https://github.com/sidekiq-cron/sidekiq-cron/pull/317)
 - Optimize gem size (https://github.com/sidekiq-cron/sidekiq-cron/pull/322)

data/Gemfile

@@ -1,3 +1,6 @@
 source 'https://rubygems.org'
 
 gemspec
+
+# To test different Sidekiq versions
+gem "sidekiq", ENV.fetch("SIDEKIQ_VERSION", ">= 6")

data/README.md

@@ -1,8 +1,8 @@
-# Sidekiq-Cron
+![Sidekiq-Cron](logos/cover.png)
 
 [![Gem Version](https://badge.fury.io/rb/sidekiq-cron.svg)](https://badge.fury.io/rb/sidekiq-cron)
 [![Build Status](https://github.com/sidekiq-cron/sidekiq-cron/workflows/CI/badge.svg?branch=master)](https://github.com/sidekiq-cron/sidekiq-cron/actions)
-[![Coverage Status](https://coveralls.io/repos/github/ondrejbartas/sidekiq-cron/badge.svg?branch=master)](https://coveralls.io/github/ondrejbartas/sidekiq-cron?branch=master)
+[![codecov](https://codecov.io/gh/sidekiq-cron/sidekiq-cron/branch/master/graph/badge.svg?token=VK9IVLIaY8)](https://codecov.io/gh/sidekiq-cron/sidekiq-cron)
 
 > A scheduling add-on for [Sidekiq](https://sidekiq.org/)
 
@@ -26,12 +26,6 @@ Before upgrading to a new version, please read our [Changelog](CHANGELOG.md).
 
 ## Installation
 
-### Requirements
-
-- Redis 2.8 or greater is required (Redis 3.0.3 or greater is recommended for large scale use)
-- Sidekiq 4.2 or greater is required (for Sidekiq < 4 use version sidekiq-cron 0.3.1)
-- Sidekiq 6.5 requires Sidekiq-Cron 1.5+
-
 Install the gem:
 
 ```
@@ -48,7 +42,7 @@ gem "sidekiq-cron"
 
 ## Getting Started
 
- ### Job properties
+### Job properties
 
 ```ruby
 {
@@ -57,12 +51,13 @@ gem "sidekiq-cron"
   'cron' => '1 * * * *',  # execute at 1 minute of every hour, ex: 12:01, 13:01, 14:01, ...
   'class' => 'MyClass',
   # OPTIONAL
+  'source' => 'dynamic', # source of the job, `schedule`/`dynamic` (default: `dynamic`)
   'queue' => 'name of queue',
   'args' => '[Array or Hash] of arguments which will be passed to perform method',
   'date_as_argument' => true, # add the time of execution as last argument of the perform method
   'active_job' => true,  # enqueue job through Rails 4.2+ Active Job interface
   'queue_name_prefix' => 'prefix', # Rails 4.2+ Active Job queue with prefix
-  'queue_name_delimiter' => '.', # Rails 4.2+ Active Job queue with custom delimiter
+  'queue_name_delimiter' => '.', # Rails 4.2+ Active Job queue with custom delimiter (default: '_')
   'description' => 'A sentence describing what work this job performs'
   'status' => 'disabled' # default: enabled
 }
@@ -141,6 +136,8 @@ which will ensure that arguments you are passing to it will be symbolized when p
 
 #### Adding Cron job
 
+Refer to [Schedule vs Dynamic jobs](#schedule-vs-dynamic-jobs) to understand the difference.
+
 ```ruby
 class HardWorker
   include Sidekiq::Worker
@@ -171,6 +168,26 @@ unless job.save
 end
 ```
 
+Use ActiveRecord models as arguments
+
+```rb
+class Person < ApplicationRecord
+end
+
+class HardWorker < ActiveJob::Base
+  queue_as :default
+
+  def perform(person)
+    puts "person: #{person}"
+  end
+end
+
+
+person = Person.create(id: 1)
+Sidekiq::Cron::Job.create(name: 'Hard worker - every 5min', cron: '*/5 * * * *', class: 'HardWorker', args: person)
+# => true
+```
+
 Load more jobs from hash:
 
 ```ruby
@@ -209,7 +226,7 @@ array = [
 Sidekiq::Cron::Job.load_from_array array
 ```
 
-Bang-suffixed methods will remove jobs that are not present in the given hash/array, update jobs that have the same names, and create new ones when the names are previously unknown.
+Bang-suffixed methods will remove jobs where source is `schedule` and are not present in the given hash/array, update jobs that have the same names, and create new ones when the names are previously unknown.
 
 ```ruby
 Sidekiq::Cron::Job.load_from_hash! hash
@@ -248,7 +265,9 @@ Sidekiq.configure_server do |config|
     schedule_file = "config/users_schedule.yml"
 
     if File.exist?(schedule_file)
-      Sidekiq::Cron::Job.load_from_hash YAML.load_file(schedule_file)
+      schedule = YAML.load_file(schedule_file)
+
+      Sidekiq::Cron::Job.load_from_hash!(schedule, source: "schedule")
     end
   end
 end
@@ -301,6 +320,16 @@ job.status
 job.enque!
 ```
 
+### Schedule vs Dynamic jobs
+
+There are two potential job sources: `schedule` and `dynamic`.
+Jobs associated with schedule files are labeled as `schedule` as their source,
+whereas jobs created at runtime without the `source=schedule` argument are classified as `dynamic`.
+
+The key distinction lies in how these jobs are managed.
+When a schedule is loaded, any stale `schedule` jobs are automatically removed to ensure synchronization within the schedule.
+The `dynamic` jobs remain unaffected by this process.
+
 ### How to start scheduling?
 
 Just start Sidekiq workers by running:
@@ -316,7 +345,7 @@ add `require 'sidekiq/cron/web'` after `require 'sidekiq/web'`.
 
 With this, you will get:
 
-![Web UI](examples/web-cron-ui.jpeg)
+![Web UI](docs/images/web-cron-ui.jpeg)
 
 ## Under the hood
 

data/lib/sidekiq/cron/job.rb

@@ -1,4 +1,5 @@
 require 'fugit'
+require 'globalid'
 require 'sidekiq'
 require 'sidekiq/cron/support'
 require 'sidekiq/options'
@@ -15,13 +16,17 @@ module Sidekiq
       # Use the exists? method if we're on a newer version of Redis.
       REDIS_EXISTS_METHOD = Gem::Version.new(Sidekiq::VERSION) >= Gem::Version.new("7.0.0") || Gem.loaded_specs['redis'].version < Gem::Version.new('4.2') ? :exists : :exists?
 
+      # Use serialize/deserialize key of GlobalID.
+      GLOBALID_KEY = "_sc_globalid"
+
       # Crucial part of whole enqueuing job.
       def should_enque? time
+        return false unless status == "enabled"
+        return false unless not_past_scheduled_time?(time)
+        return false unless not_enqueued_after?(time)
+
         enqueue = Sidekiq.redis do |conn|
-          status == "enabled" &&
-            not_past_scheduled_time?(time) &&
-            not_enqueued_after?(time) &&
-            conn.zadd(job_enqueued_key, formatted_enqueue_time(time), formatted_last_time(time))
+          conn.zadd(job_enqueued_key, formatted_enqueue_time(time), formatted_last_time(time))
         end
         enqueue == true || enqueue == 1
       end
@@ -45,7 +50,7 @@ module Sidekiq
 
       # Enqueue cron job to queue.
       def enque! time = Time.now.utc
-        @last_enqueue_time = time.strftime(LAST_ENQUEUE_TIME_FORMAT)
+        @last_enqueue_time = time
 
         klass_const =
             begin
@@ -85,7 +90,8 @@ module Sidekiq
       end
 
       def enqueue_args
-        date_as_argument? ? @args + [Time.now.to_f] : @args
+        args = date_as_argument? ? @args + [Time.now.to_f] : @args
+        deserialize_argument(args)
       end
 
       def enqueue_active_job(klass_const)
@@ -157,19 +163,19 @@ module Sidekiq
       #   }
       # }
       #
-      def self.load_from_hash hash
+      def self.load_from_hash(hash, options = {})
         array = hash.map do |key, job|
           job['name'] = key
           job
         end
-        load_from_array array
+        load_from_array(array, options)
       end
 
       # Like #load_from_hash.
       # If exists old jobs in Redis but removed from args, destroy old jobs.
-      def self.load_from_hash! hash
+      def self.load_from_hash!(hash, options = {})
         destroy_removed_jobs(hash.keys)
-        load_from_hash(hash)
+        load_from_hash(hash, options)
       end
 
       # Load cron jobs from Array.
@@ -189,10 +195,10 @@ module Sidekiq
       #   }
       # ]
       #
-      def self.load_from_array array
+      def self.load_from_array(array, options = {})
         errors = {}
         array.each do |job_data|
-          job = new(job_data)
+          job = new(job_data.merge(options))
           errors[job.name] = job.errors unless job.save
         end
         errors
@@ -200,10 +206,10 @@ module Sidekiq
 
       # Like #load_from_array.
       # If exists old jobs in Redis but removed from args, destroy old jobs.
-      def self.load_from_array! array
+      def self.load_from_array!(array, options = {})
         job_names = array.map { |job| job["name"] }
         destroy_removed_jobs(job_names)
-        load_from_array(array)
+        load_from_array(array, options)
       end
 
       # Get all cron jobs.
@@ -234,12 +240,11 @@ module Sidekiq
       def self.find name
         # If name is hash try to get name from it.
         name = name[:name] || name['name'] if name.is_a?(Hash)
+        return unless exists? name
 
         output = nil
         Sidekiq.redis do |conn|
-          if exists? name
-            output = Job.new conn.hgetall( redis_key(name) )
-          end
+          output = Job.new conn.hgetall( redis_key(name) )
         end
         output if output && output.valid?
       end
@@ -262,7 +267,7 @@ module Sidekiq
       end
 
       attr_accessor :name, :cron, :description, :klass, :args, :message
-      attr_reader   :last_enqueue_time, :fetch_missing_args
+      attr_reader   :last_enqueue_time, :fetch_missing_args, :source
 
       def initialize input_args = {}
         args = Hash[input_args.map{ |k, v| [k.to_s, v] }]
@@ -272,6 +277,7 @@ module Sidekiq
         @name = args["name"]
         @cron = args["cron"]
         @description = args["description"] if args["description"]
+        @source = args["source"] == "schedule" ? "schedule" : "dynamic"
 
         # Get class from klass or class.
         @klass = args["klass"] || args["class"]
@@ -288,7 +294,7 @@ module Sidekiq
 
         # Get right arguments for job.
         @symbolize_args = args["symbolize_args"] == true || ("#{args["symbolize_args"]}" =~ (/^(true|t|yes|y|1)$/i)) == 0 || false
-        @args = args["args"].nil? ? [] : parse_args( args["args"] )
+        @args = parse_args(args["args"])
 
         @date_as_argument = args["date_as_argument"] == true || ("#{args["date_as_argument"]}" =~ (/^(true|t|yes|y|1)$/i)) == 0 || false
 
@@ -398,21 +404,27 @@ module Sidekiq
 
       # Export job data to hash.
       def to_hash
-        {
+        hash = {
           name: @name,
           klass: @klass.to_s,
           cron: @cron,
           description: @description,
+          source: @source,
           args: @args.is_a?(String) ? @args : Sidekiq.dump_json(@args || []),
-          date_as_argument: date_as_argument? ? "1" : "0",
           message: @message.is_a?(String) ? @message : Sidekiq.dump_json(@message || {}),
           status: @status,
           active_job: @active_job ? "1" : "0",
           queue_name_prefix: @active_job_queue_name_prefix,
           queue_name_delimiter: @active_job_queue_name_delimiter,
-          last_enqueue_time: @last_enqueue_time.to_s,
+          last_enqueue_time: serialized_last_enqueue_time,
           symbolize_args: symbolize_args? ? "1" : "0",
         }
+
+        if date_as_argument?
+          hash.merge!(date_as_argument: "1")
+        end
+
+        hash
       end
 
       def errors
@@ -459,7 +471,7 @@ module Sidekiq
           conn.sadd self.class.jobs_key, [redis_key]
 
           # Add informations for this job!
-          conn.hmset redis_key, *hash_to_redis(to_hash)
+          conn.hset redis_key, to_hash.transform_values! { |v| v || "" }
 
           # Add information about last time! - don't enque right after scheduler poller starts!
           time = Time.now.utc
@@ -472,7 +484,7 @@ module Sidekiq
       def save_last_enqueue_time
         Sidekiq.redis do |conn|
           # Update last enqueue time.
-          conn.hset redis_key, 'last_enqueue_time', @last_enqueue_time
+          conn.hset redis_key, 'last_enqueue_time', serialized_last_enqueue_time
         end
       end
 
@@ -519,7 +531,7 @@ module Sidekiq
 
       # Remove "removed jobs" between current jobs and new jobs
       def self.destroy_removed_jobs new_job_names
-        current_job_names = Sidekiq::Cron::Job.all.map(&:name)
+        current_job_names = Sidekiq::Cron::Job.all.filter_map { |j| j.name if j.source == "schedule" }
         removed_job_names = current_job_names - new_job_names
         removed_job_names.each { |j| Sidekiq::Cron::Job.destroy(j) }
         removed_job_names
@@ -554,6 +566,10 @@ module Sidekiq
         "#{status == "enabled" ? 0 : 1}_#{name}".downcase
       end
 
+      def args=(args)
+        @args = parse_args(args)
+      end
+
       private
 
       def parsed_cron
@@ -569,6 +585,8 @@ module Sidekiq
       # try to load JSON, then failover to string array.
       def parse_args(args)
         case args
+        when GlobalID::Identification
+          [convert_to_global_id_hash(args)]
         when String
           begin
             parsed_args = Sidekiq.load_json(args)
@@ -577,8 +595,10 @@ module Sidekiq
             [*args]
           end
         when Hash
+          args = serialize_argument(args)
           symbolize_args? ? [symbolize_args(args)] : [args]
         when Array
+          args = serialize_argument(args)
           symbolize_args? ? symbolize_args(args) : args
         else
           [*args]
@@ -649,9 +669,55 @@ module Sidekiq
         self.class.jid_history_key @name
       end
 
-      # Give Hash returns array for using it for redis.hmset
-      def hash_to_redis hash
-        hash.flat_map{ |key, value| [key, value || ""] }
+      def serialized_last_enqueue_time
+        @last_enqueue_time&.strftime(LAST_ENQUEUE_TIME_FORMAT)
+      end
+      
+      def convert_to_global_id_hash(argument)
+        { GLOBALID_KEY => argument.to_global_id.to_s }
+      rescue URI::GID::MissingModelIdError
+        raise "Unable to serialize #{argument.class} " \
+          "without an id. (Maybe you forgot to call save?)"
+      end
+
+      def deserialize_argument(argument)
+        case argument
+        when String
+          argument
+        when Array
+          argument.map { |arg| deserialize_argument(arg) }
+        when Hash
+          if serialized_global_id?(argument)
+            deserialize_global_id argument
+          else
+            argument.transform_values { |v| deserialize_argument(v) }
+          end
+        else
+          argument
+        end
+      end
+
+      def serialized_global_id?(hash)
+        hash.size == 1 && hash.include?(GLOBALID_KEY)
+      end
+
+      def deserialize_global_id(hash)
+        GlobalID::Locator.locate hash[GLOBALID_KEY]
+      end
+
+      def serialize_argument(argument)
+        case argument
+        when GlobalID::Identification
+          convert_to_global_id_hash(argument)
+        when Array
+          argument.map { |arg| serialize_argument(arg) }
+        when Hash
+          argument.each_with_object({}) do |(key, value), hash|
+            hash[key] = serialize_argument(value)
+          end
+        else
+          argument
+        end
       end
     end
   end

data/lib/sidekiq/cron/launcher.rb

@@ -14,7 +14,7 @@ module Sidekiq
       attr_reader :cron_poller
 
       # Add cron poller and execute normal initialize of Sidekiq launcher.
-      def initialize(config)
+      def initialize(config, **kwargs)
         config[:cron_poll_interval] = DEFAULT_POLL_INTERVAL if config[:cron_poll_interval].nil?
 
         @cron_poller = Sidekiq::Cron::Poller.new(config) if config[:cron_poll_interval] > 0

data/lib/sidekiq/cron/schedule_loader.rb

@@ -2,20 +2,18 @@ require 'sidekiq'
 require 'sidekiq/cron/job'
 require 'sidekiq/options'
 
-if Sidekiq.server?
-  Sidekiq.configure_server do |config|
-    schedule_file = Sidekiq::Options[:cron_schedule_file] || 'config/schedule.yml'
+Sidekiq.configure_server do |config|
+  schedule_file = Sidekiq::Options[:cron_schedule_file] || 'config/schedule.yml'
 
-    if File.exist?(schedule_file)
-      config.on(:startup) do
-        schedule = Sidekiq::Cron::Support.load_yaml(ERB.new(IO.read(schedule_file)).result)
-        if schedule.kind_of?(Hash)
-          Sidekiq::Cron::Job.load_from_hash schedule
-        elsif schedule.kind_of?(Array)
-          Sidekiq::Cron::Job.load_from_array schedule
-        else
-          raise "Not supported schedule format. Confirm your #{schedule_file}"
-        end
+  if File.exist?(schedule_file)
+    config.on(:startup) do
+      schedule = Sidekiq::Cron::Support.load_yaml(ERB.new(IO.read(schedule_file)).result)
+      if schedule.kind_of?(Hash)
+        Sidekiq::Cron::Job.load_from_hash!(schedule, source: "schedule")
+      elsif schedule.kind_of?(Array)
+        Sidekiq::Cron::Job.load_from_array!(schedule, source: "schedule")
+      else
+        raise "Not supported schedule format. Confirm your #{schedule_file}"
       end
     end
   end

data/lib/sidekiq/cron/version.rb

@@ -2,6 +2,6 @@
 
 module Sidekiq
   module Cron
-    VERSION = "1.9.1"
+    VERSION = "1.12.0"
   end
 end

data/lib/sidekiq/options.rb

@@ -3,11 +3,15 @@ require 'sidekiq'
 module Sidekiq
   module Options
     def self.[](key)
-      options_field ? Sidekiq.public_send(options_field)[key] : Sidekiq[key]
+      self.config[key]
     end
 
     def self.[]=(key, value)
-      options_field ? Sidekiq.public_send(options_field)[key] = value : Sidekiq[key] = value
+      self.config[key] = value
+    end
+
+    def self.config
+      options_field ? Sidekiq.public_send(options_field) : Sidekiq
     end
 
     def self.options_field

data/sidekiq-cron.gemspec

@@ -15,7 +15,7 @@ Gem::Specification.new do |s|
     "LICENSE.txt",
     "README.md"
   ]
-  s.files = Dir.glob('lib/**/*') + Dir.glob('test/**/*') + [
+  s.files = Dir.glob('lib/**/*') + [
     "CHANGELOG.md",
     "Gemfile",
     "LICENSE.txt",
@@ -24,15 +24,17 @@ Gem::Specification.new do |s|
     "sidekiq-cron.gemspec",
   ]
 
-  s.required_ruby_version = ">= 2.6"
+  s.required_ruby_version = ">= 2.7"
 
   s.add_dependency("fugit", "~> 1.8")
-  s.add_dependency("sidekiq", ">= 4.2.1")
+  s.add_dependency("sidekiq", ">= 6")
+  s.add_dependency("globalid", ">= 1.0.1")
 
   s.add_development_dependency("minitest", "~> 5.15")
-  s.add_development_dependency("mocha", "~> 1.14")
+  s.add_development_dependency("mocha", "~> 2.1")
   s.add_development_dependency("rack", "~> 2.2")
   s.add_development_dependency("rack-test", "~> 1.1")
   s.add_development_dependency("rake", "~> 13.0")
   s.add_development_dependency("simplecov", "~> 0.21")
+  s.add_development_dependency("simplecov-cobertura", "~> 2.1")
 end