sidekiq-scheduler 2.1.10 → 3.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/README.md +45 -11
- data/lib/sidekiq-scheduler.rb +4 -27
- data/lib/sidekiq-scheduler/extensions/schedule.rb +4 -0
- data/lib/sidekiq-scheduler/extensions/web.rb +5 -0
- data/lib/sidekiq-scheduler/job_presenter.rb +2 -2
- data/lib/sidekiq-scheduler/manager.rb +31 -9
- data/lib/sidekiq-scheduler/redis_manager.rb +7 -1
- data/lib/sidekiq-scheduler/schedule.rb +0 -2
- data/lib/sidekiq-scheduler/scheduler.rb +357 -0
- data/lib/sidekiq-scheduler/utils.rb +86 -0
- data/lib/sidekiq-scheduler/version.rb +1 -1
- data/lib/sidekiq-scheduler/web.rb +3 -6
- data/lib/sidekiq/scheduler.rb +2 -406
- data/web/locales/ja.yml +14 -0
- data/web/locales/nl.yml +14 -0
- data/web/locales/pl.yml +14 -0
- data/web/locales/pt-BR.yml +14 -0
- data/web/locales/ru.yml +14 -0
- data/web/views/recurring_jobs.erb +2 -2
- metadata +45 -10
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 7c9e10cdd3a2bfc144f07860038c95ef459bb2bf8800d763ea90379b6c709472
|
4
|
+
data.tar.gz: 8f4be75200f8a5c44f53227ba73586e5e6ac9e8885972163382b372372086af0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e8dc47d6600676d0de6cac9b508026315c2eefcf6a055b496abd84ed663ec9bb9eddc267246ba877588ec4abb67d79a773664238d1028e23ca2859869e76dedd
|
7
|
+
data.tar.gz: 78764a88d67d2c501a4df10a8d7b79331146912a4d5fa2c83a04e1b06165bd45ceec0a11232e8e841fc452d6fd79e985142ad72a945e4fc04a8d79aa7090f79c
|
data/README.md
CHANGED
@@ -30,6 +30,8 @@
|
|
30
30
|
`sidekiq-scheduler` is an extension to [Sidekiq](http://github.com/mperham/sidekiq) that
|
31
31
|
pushes jobs in a scheduled way, mimicking cron utility.
|
32
32
|
|
33
|
+
__Note:__ If you are looking for version 2.2.*, go to [2.2-stable branch](https://github.com/moove-it/sidekiq-scheduler/tree/2.2-stable).
|
34
|
+
|
33
35
|
## Installation
|
34
36
|
|
35
37
|
``` shell
|
@@ -180,6 +182,8 @@ Cron, every, and interval types push jobs into sidekiq in a recurrent manner.
|
|
180
182
|
`interval` is similar to `every`, the difference between them is that `interval` type schedules the
|
181
183
|
next execution after the interval has elapsed counting from its last job enqueue.
|
182
184
|
|
185
|
+
Note that `every` and `interval` count from when the Sidekiq process (re)starts. So `every: '48h'` will never run if the Sidekiq process is restarted daily, for example. You can do `every: ['48h', first_in: '0s']` to make the job run immediately after a restart, and then have the worker check when it was last run.
|
186
|
+
|
183
187
|
At, and in types push jobs only once. `at` schedules in a point in time:
|
184
188
|
``` yaml
|
185
189
|
at: '3001/01/01'
|
@@ -223,12 +227,12 @@ To load the schedule:
|
|
223
227
|
|
224
228
|
``` ruby
|
225
229
|
require 'sidekiq'
|
226
|
-
require 'sidekiq
|
230
|
+
require 'sidekiq-scheduler'
|
227
231
|
|
228
232
|
Sidekiq.configure_server do |config|
|
229
233
|
config.on(:startup) do
|
230
234
|
Sidekiq.schedule = YAML.load_file(File.expand_path('../../sidekiq_scheduler.yml', __FILE__))
|
231
|
-
|
235
|
+
SidekiqScheduler::Scheduler.instance.reload_schedule!
|
232
236
|
end
|
233
237
|
end
|
234
238
|
```
|
@@ -256,7 +260,7 @@ If `:dynamic` flag is set to `false`, you'll have to reload the schedule manuall
|
|
256
260
|
side:
|
257
261
|
|
258
262
|
``` ruby
|
259
|
-
|
263
|
+
SidekiqScheduler::Scheduler.instance.reload_schedule!
|
260
264
|
```
|
261
265
|
|
262
266
|
Invoke `Sidekiq.get_schedule` to obtain the current schedule:
|
@@ -268,8 +272,9 @@ Sidekiq.get_schedule
|
|
268
272
|
|
269
273
|
## Time zones
|
270
274
|
|
271
|
-
Note that if you use the cron syntax, this will be interpreted
|
272
|
-
|
275
|
+
Note that if you use the cron syntax and are not running a Rails app, this will be interpreted in the server time zone.
|
276
|
+
|
277
|
+
In a Rails app, [rufus-scheduler](https://github.com/jmettraux/rufus-scheduler) (>= 3.3.3) will use the `config.time_zone` specified in Rails.
|
273
278
|
|
274
279
|
You can explicitly specify the time zone that rufus-scheduler will use:
|
275
280
|
|
@@ -285,11 +290,27 @@ from the `config.time_zone` value, make sure it's the right format, e.g. with:
|
|
285
290
|
ActiveSupport::TimeZone.find_tzinfo(Rails.configuration.time_zone).name
|
286
291
|
```
|
287
292
|
|
293
|
+
## Notes about connection pooling
|
294
|
+
|
295
|
+
If you're configuring your own Redis connection pool, you need to make sure the size is adequate to be inclusive of both Sidekiq's own connection pool and Rufus Scheduler's.
|
296
|
+
|
297
|
+
That's a minimum of `concurrency` + 5 (per the [Sidekiq wiki](https://github.com/mperham/sidekiq/wiki/Using-Redis#complete-control)) + `Rufus::Scheduler::MAX_WORK_THREADS` (28 as of this writing; per the [Rufus README](https://github.com/jmettraux/rufus-scheduler#max_work_threads)), for a total of 58 with the default `concurrency` of 25.
|
298
|
+
|
299
|
+
You can also override the thread pool size in Rufus Scheduler by setting e.g.:
|
300
|
+
|
301
|
+
```
|
302
|
+
SidekiqScheduler::Scheduler.instance.rufus_scheduler_options = { max_work_threads: 5 }
|
303
|
+
```
|
304
|
+
|
288
305
|
## Notes about running on Multiple Hosts
|
289
306
|
|
290
|
-
`cron` and `at` jobs are pushed once regardless of the number of `sidekiq-scheduler` running instances,
|
307
|
+
Under normal conditions, `cron` and `at` jobs are pushed once regardless of the number of `sidekiq-scheduler` running instances,
|
291
308
|
assumming that time deltas between hosts is less than 24 hours.
|
292
309
|
|
310
|
+
Non-normal conditions that could push a specific job multiple times are:
|
311
|
+
- high cpu load + a high number of jobs scheduled at the same time, like 100 jobs
|
312
|
+
- network / redis latency + 28 (see `MAX_WORK_THREADS` https://github.com/jmettraux/rufus-scheduler/blob/master/lib/rufus/scheduler.rb#L41) or more jobs scheduled within the same network latency window
|
313
|
+
|
293
314
|
`every`, `interval` and `in` jobs will be pushed once per host.
|
294
315
|
|
295
316
|
## Sidekiq Web Integration
|
@@ -307,6 +328,18 @@ run Sidekiq::Web
|
|
307
328
|
|
308
329
|

|
309
330
|
|
331
|
+
## ActiveJob integration
|
332
|
+
|
333
|
+
When using sidekiq-scheduler with ActiveJob your jobs can just extend `ApplicationJob` as usual, without the `require` and `include` boilerplate. Under the hood Rails will load up the scheduler and include the worker module for you.
|
334
|
+
|
335
|
+
```rb
|
336
|
+
class HelloWorld < ApplicationJob
|
337
|
+
def perform
|
338
|
+
puts 'Hello world'
|
339
|
+
end
|
340
|
+
end
|
341
|
+
```
|
342
|
+
|
310
343
|
## The Spring preloader and Testing your initializer via Rails console
|
311
344
|
|
312
345
|
If you're pulling in your schedule from a YML file via an initializer as shown, be aware that the Spring application preloader included with Rails will interefere with testing via the Rails console.
|
@@ -327,7 +360,8 @@ something like this in an initializer:
|
|
327
360
|
|
328
361
|
``` ruby
|
329
362
|
# config/initializers/sidekiq_scheduler.rb
|
330
|
-
require 'sidekiq
|
363
|
+
require 'sidekiq'
|
364
|
+
require 'sidekiq-scheduler'
|
331
365
|
|
332
366
|
puts "Sidekiq.server? is #{Sidekiq.server?.inspect}"
|
333
367
|
puts "defined?(Rails::Server) is #{defined?(Rails::Server).inspect}"
|
@@ -338,12 +372,12 @@ if Rails.env == 'production' && (defined?(Rails::Server) || defined?(Unicorn))
|
|
338
372
|
|
339
373
|
config.on(:startup) do
|
340
374
|
Sidekiq.schedule = YAML.load_file(File.expand_path('../../scheduler.yml', __FILE__))
|
341
|
-
|
375
|
+
SidekiqScheduler::Scheduler.instance.reload_schedule!
|
342
376
|
end
|
343
377
|
end
|
344
378
|
else
|
345
|
-
|
346
|
-
puts "
|
379
|
+
SidekiqScheduler::Scheduler.instance.enabled = false
|
380
|
+
puts "SidekiqScheduler::Scheduler.instance.enabled is #{SidekiqScheduler::Scheduler.instance.enabled.inspect}"
|
347
381
|
end
|
348
382
|
```
|
349
383
|
|
@@ -353,6 +387,6 @@ MIT License
|
|
353
387
|
|
354
388
|
## Copyright
|
355
389
|
|
356
|
-
Copyright 2013 -
|
390
|
+
Copyright 2013 - 2018 Moove-IT.
|
357
391
|
Copyright 2012 Morton Jonuschat.
|
358
392
|
Some parts copyright 2010 Ben VandenBos.
|
data/lib/sidekiq-scheduler.rb
CHANGED
@@ -1,47 +1,24 @@
|
|
1
1
|
require 'sidekiq'
|
2
2
|
require 'tilt/erb'
|
3
3
|
|
4
|
+
require_relative 'sidekiq/scheduler'
|
4
5
|
require_relative 'sidekiq-scheduler/version'
|
5
6
|
require_relative 'sidekiq-scheduler/manager'
|
6
7
|
require_relative 'sidekiq-scheduler/redis_manager'
|
8
|
+
require_relative 'sidekiq-scheduler/extensions/schedule'
|
7
9
|
|
8
10
|
Sidekiq.configure_server do |config|
|
9
11
|
|
10
12
|
config.on(:startup) do
|
11
|
-
dynamic = Sidekiq::Scheduler.dynamic
|
12
|
-
dynamic = dynamic.nil? ? config.options.fetch(:dynamic, false) : dynamic
|
13
|
-
|
14
|
-
dynamic_every = Sidekiq::Scheduler.dynamic_every
|
15
|
-
dynamic_every = dynamic_every.nil? ? config.options.fetch(:dynamic_every, '5s') : dynamic_every
|
16
|
-
|
17
|
-
enabled = Sidekiq::Scheduler.enabled
|
18
|
-
enabled = enabled.nil? ? config.options.fetch(:enabled, true) : enabled
|
19
|
-
|
20
|
-
scheduler = config.options.fetch(:scheduler, {})
|
21
|
-
|
22
|
-
listened_queues_only = Sidekiq::Scheduler.listened_queues_only
|
23
|
-
listened_queues_only = listened_queues_only.nil? ? scheduler[:listened_queues_only] : listened_queues_only
|
24
|
-
|
25
|
-
schedule = Sidekiq.schedule
|
26
|
-
schedule ||= config.options[:schedule] || {}
|
27
|
-
|
28
|
-
scheduler_options = {
|
29
|
-
dynamic: dynamic,
|
30
|
-
dynamic_every: dynamic_every,
|
31
|
-
enabled: enabled,
|
32
|
-
schedule: schedule,
|
33
|
-
listened_queues_only: listened_queues_only
|
34
|
-
}
|
35
|
-
|
36
13
|
# schedules_changed's type was changed from SET to ZSET, so we remove old versions at startup
|
37
14
|
SidekiqScheduler::RedisManager.clean_schedules_changed
|
38
15
|
|
39
|
-
schedule_manager = SidekiqScheduler::Manager.new(
|
16
|
+
schedule_manager = SidekiqScheduler::Manager.new(config.options)
|
40
17
|
config.options[:schedule_manager] = schedule_manager
|
41
18
|
config.options[:schedule_manager].start
|
42
19
|
end
|
43
20
|
|
44
|
-
config.on(:
|
21
|
+
config.on(:quiet) do
|
45
22
|
config.options[:schedule_manager].stop
|
46
23
|
end
|
47
24
|
|
@@ -56,7 +56,7 @@ module SidekiqScheduler
|
|
56
56
|
end
|
57
57
|
|
58
58
|
def enabled?
|
59
|
-
|
59
|
+
SidekiqScheduler::Scheduler.job_enabled?(@name)
|
60
60
|
end
|
61
61
|
|
62
62
|
# Builds the presenter instances for the schedule hash
|
@@ -66,7 +66,7 @@ module SidekiqScheduler
|
|
66
66
|
def self.build_collection(schedule_hash)
|
67
67
|
schedule_hash ||= {}
|
68
68
|
|
69
|
-
schedule_hash.map do |name, job_spec|
|
69
|
+
schedule_hash.sort.map do |name, job_spec|
|
70
70
|
new(name, job_spec)
|
71
71
|
end
|
72
72
|
end
|
@@ -2,8 +2,8 @@ require 'redis'
|
|
2
2
|
|
3
3
|
require 'sidekiq/util'
|
4
4
|
|
5
|
-
require 'sidekiq/scheduler'
|
6
5
|
require 'sidekiq-scheduler/schedule'
|
6
|
+
require 'sidekiq-scheduler/scheduler'
|
7
7
|
|
8
8
|
module SidekiqScheduler
|
9
9
|
|
@@ -14,26 +14,48 @@ module SidekiqScheduler
|
|
14
14
|
class Manager
|
15
15
|
include Sidekiq::Util
|
16
16
|
|
17
|
+
DEFAULT_SCHEDULER_OPTIONS = {
|
18
|
+
enabled: true,
|
19
|
+
dynamic: false,
|
20
|
+
dynamic_every: '5s',
|
21
|
+
schedule: {}
|
22
|
+
}
|
23
|
+
|
17
24
|
def initialize(options)
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
Sidekiq.schedule =
|
25
|
+
scheduler_options = load_scheduler_options(options)
|
26
|
+
|
27
|
+
@scheduler_instance = SidekiqScheduler::Scheduler.new(scheduler_options)
|
28
|
+
SidekiqScheduler::Scheduler.instance = @scheduler_instance
|
29
|
+
Sidekiq.schedule = scheduler_options[:schedule] if @scheduler_instance.enabled
|
23
30
|
end
|
24
31
|
|
25
32
|
def stop
|
26
|
-
|
33
|
+
@scheduler_instance.clear_schedule!
|
27
34
|
end
|
28
35
|
|
29
36
|
def start
|
30
|
-
|
37
|
+
@scheduler_instance.load_schedule!
|
31
38
|
end
|
32
39
|
|
33
40
|
def reset
|
34
41
|
clear_scheduled_work
|
35
42
|
end
|
36
43
|
|
37
|
-
|
44
|
+
private
|
45
|
+
|
46
|
+
def load_scheduler_options(options)
|
47
|
+
options[:listened_queues_only] = options.fetch(:scheduler, {})[:listened_queues_only]
|
48
|
+
scheduler_options = DEFAULT_SCHEDULER_OPTIONS.merge(options)
|
38
49
|
|
50
|
+
current_options = {
|
51
|
+
enabled: SidekiqScheduler::Scheduler.enabled,
|
52
|
+
dynamic: SidekiqScheduler::Scheduler.dynamic,
|
53
|
+
dynamic_every: SidekiqScheduler::Scheduler.dynamic_every,
|
54
|
+
schedule: Sidekiq.schedule,
|
55
|
+
listened_queues_only: SidekiqScheduler::Scheduler.listened_queues_only
|
56
|
+
}.delete_if { |_, value| value.nil? }
|
57
|
+
|
58
|
+
scheduler_options.merge(current_options)
|
59
|
+
end
|
60
|
+
end
|
39
61
|
end
|
@@ -96,7 +96,13 @@ module SidekiqScheduler
|
|
96
96
|
#
|
97
97
|
# @return [Boolean] true if the schedules key is set, false otherwise
|
98
98
|
def self.schedule_exist?
|
99
|
-
Sidekiq.redis
|
99
|
+
Sidekiq.redis do |r|
|
100
|
+
if r.respond_to?(:exists?)
|
101
|
+
r.exists?(:schedules)
|
102
|
+
else
|
103
|
+
!!r.exists(:schedules)
|
104
|
+
end
|
105
|
+
end
|
100
106
|
end
|
101
107
|
|
102
108
|
# Returns all the schedule changes for a given time range.
|
@@ -0,0 +1,357 @@
|
|
1
|
+
require 'rufus/scheduler'
|
2
|
+
require 'thwait'
|
3
|
+
require 'sidekiq/util'
|
4
|
+
require 'json'
|
5
|
+
require 'sidekiq-scheduler/rufus_utils'
|
6
|
+
require 'sidekiq-scheduler/redis_manager'
|
7
|
+
|
8
|
+
module SidekiqScheduler
|
9
|
+
class Scheduler
|
10
|
+
extend Sidekiq::Util
|
11
|
+
|
12
|
+
# We expect rufus jobs to have #params
|
13
|
+
Rufus::Scheduler::Job.module_eval do
|
14
|
+
alias_method :params, :opts
|
15
|
+
end
|
16
|
+
|
17
|
+
# Set to enable or disable the scheduler.
|
18
|
+
attr_accessor :enabled
|
19
|
+
|
20
|
+
# Set to update the schedule in runtime in a given time period.
|
21
|
+
attr_accessor :dynamic
|
22
|
+
|
23
|
+
# Set to update the schedule in runtime dynamically per this period.
|
24
|
+
attr_accessor :dynamic_every
|
25
|
+
|
26
|
+
# Set to schedule jobs only when will be pushed to queues listened by sidekiq
|
27
|
+
attr_accessor :listened_queues_only
|
28
|
+
|
29
|
+
class << self
|
30
|
+
|
31
|
+
def instance
|
32
|
+
@instance = new unless @instance
|
33
|
+
@instance
|
34
|
+
end
|
35
|
+
|
36
|
+
def instance=(value)
|
37
|
+
@instance = value
|
38
|
+
end
|
39
|
+
|
40
|
+
def method_missing(method, *arguments, &block)
|
41
|
+
instance_methods.include?(method) ? instance.public_send(method, *arguments) : super
|
42
|
+
end
|
43
|
+
end
|
44
|
+
|
45
|
+
def initialize(options = {})
|
46
|
+
self.enabled = options[:enabled]
|
47
|
+
self.dynamic = options[:dynamic]
|
48
|
+
self.dynamic_every = options[:dynamic_every]
|
49
|
+
self.listened_queues_only = options[:listened_queues_only]
|
50
|
+
end
|
51
|
+
|
52
|
+
# the Rufus::Scheduler jobs that are scheduled
|
53
|
+
def scheduled_jobs
|
54
|
+
@scheduled_jobs
|
55
|
+
end
|
56
|
+
|
57
|
+
def print_schedule
|
58
|
+
if rufus_scheduler
|
59
|
+
Sidekiq.logger.info "Scheduling Info\tLast Run"
|
60
|
+
scheduler_jobs = rufus_scheduler.all_jobs
|
61
|
+
scheduler_jobs.each_value do |v|
|
62
|
+
Sidekiq.logger.info "#{v.t}\t#{v.last}\t"
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
66
|
+
|
67
|
+
# Pulls the schedule from Sidekiq.schedule and loads it into the
|
68
|
+
# rufus scheduler instance
|
69
|
+
def load_schedule!
|
70
|
+
if enabled
|
71
|
+
Sidekiq.logger.info 'Loading Schedule'
|
72
|
+
|
73
|
+
# Load schedule from redis for the first time if dynamic
|
74
|
+
if dynamic
|
75
|
+
Sidekiq.reload_schedule!
|
76
|
+
@current_changed_score = Time.now.to_f
|
77
|
+
rufus_scheduler.every(dynamic_every) do
|
78
|
+
update_schedule
|
79
|
+
end
|
80
|
+
end
|
81
|
+
|
82
|
+
Sidekiq.logger.info 'Schedule empty! Set Sidekiq.schedule' if Sidekiq.schedule.empty?
|
83
|
+
|
84
|
+
@scheduled_jobs = {}
|
85
|
+
queues = sidekiq_queues
|
86
|
+
|
87
|
+
Sidekiq.schedule.each do |name, config|
|
88
|
+
if !listened_queues_only || enabled_queue?(config['queue'].to_s, queues)
|
89
|
+
load_schedule_job(name, config)
|
90
|
+
else
|
91
|
+
Sidekiq.logger.info { "Ignoring #{name}, job's queue is not enabled." }
|
92
|
+
end
|
93
|
+
end
|
94
|
+
|
95
|
+
Sidekiq.logger.info 'Schedules Loaded'
|
96
|
+
else
|
97
|
+
Sidekiq.logger.info 'SidekiqScheduler is disabled'
|
98
|
+
end
|
99
|
+
end
|
100
|
+
|
101
|
+
# Loads a job schedule into the Rufus::Scheduler and stores it in @scheduled_jobs
|
102
|
+
def load_schedule_job(name, config)
|
103
|
+
# If rails_env is set in the config, enforce ENV['RAILS_ENV'] as
|
104
|
+
# required for the jobs to be scheduled. If rails_env is missing, the
|
105
|
+
# job should be scheduled regardless of what ENV['RAILS_ENV'] is set
|
106
|
+
# to.
|
107
|
+
if config['rails_env'].nil? || rails_env_matches?(config)
|
108
|
+
Sidekiq.logger.info "Scheduling #{name} #{config}"
|
109
|
+
interval_defined = false
|
110
|
+
interval_types = %w(cron every at in interval)
|
111
|
+
interval_types.each do |interval_type|
|
112
|
+
config_interval_type = config[interval_type]
|
113
|
+
|
114
|
+
if !config_interval_type.nil? && config_interval_type.length > 0
|
115
|
+
|
116
|
+
schedule, options = SidekiqScheduler::RufusUtils.normalize_schedule_options(config_interval_type)
|
117
|
+
|
118
|
+
rufus_job = new_job(name, interval_type, config, schedule, options)
|
119
|
+
@scheduled_jobs[name] = rufus_job
|
120
|
+
SidekiqScheduler::Utils.update_job_next_time(name, rufus_job.next_time)
|
121
|
+
|
122
|
+
interval_defined = true
|
123
|
+
|
124
|
+
break
|
125
|
+
end
|
126
|
+
end
|
127
|
+
|
128
|
+
unless interval_defined
|
129
|
+
Sidekiq.logger.info "no #{interval_types.join(' / ')} found for #{config['class']} (#{name}) - skipping"
|
130
|
+
end
|
131
|
+
end
|
132
|
+
end
|
133
|
+
|
134
|
+
# Pushes the job into Sidekiq if not already pushed for the given time
|
135
|
+
#
|
136
|
+
# @param [String] job_name The job's name
|
137
|
+
# @param [Time] time The time when the job got cleared for triggering
|
138
|
+
# @param [Hash] config Job's config hash
|
139
|
+
def idempotent_job_enqueue(job_name, time, config)
|
140
|
+
registered = SidekiqScheduler::RedisManager.register_job_instance(job_name, time)
|
141
|
+
|
142
|
+
if registered
|
143
|
+
Sidekiq.logger.info "queueing #{config['class']} (#{job_name})"
|
144
|
+
|
145
|
+
handle_errors { enqueue_job(config, time) }
|
146
|
+
|
147
|
+
SidekiqScheduler::RedisManager.remove_elder_job_instances(job_name)
|
148
|
+
else
|
149
|
+
Sidekiq.logger.debug { "Ignoring #{job_name} job as it has been already enqueued" }
|
150
|
+
end
|
151
|
+
end
|
152
|
+
|
153
|
+
# Enqueue a job based on a config hash
|
154
|
+
#
|
155
|
+
# @param job_config [Hash] the job configuration
|
156
|
+
# @param time [Time] time the job is enqueued
|
157
|
+
def enqueue_job(job_config, time = Time.now)
|
158
|
+
config = prepare_arguments(job_config.dup)
|
159
|
+
|
160
|
+
if config.delete('include_metadata')
|
161
|
+
config['args'] = arguments_with_metadata(config['args'], scheduled_at: time.to_f)
|
162
|
+
end
|
163
|
+
|
164
|
+
if active_job_enqueue?(config['class'])
|
165
|
+
SidekiqScheduler::Utils.enqueue_with_active_job(config)
|
166
|
+
else
|
167
|
+
SidekiqScheduler::Utils.enqueue_with_sidekiq(config)
|
168
|
+
end
|
169
|
+
end
|
170
|
+
|
171
|
+
def rufus_scheduler_options
|
172
|
+
@rufus_scheduler_options ||= {}
|
173
|
+
end
|
174
|
+
|
175
|
+
def rufus_scheduler_options=(options)
|
176
|
+
@rufus_scheduler_options = options
|
177
|
+
end
|
178
|
+
|
179
|
+
def rufus_scheduler
|
180
|
+
@rufus_scheduler ||= SidekiqScheduler::Utils.new_rufus_scheduler(rufus_scheduler_options)
|
181
|
+
end
|
182
|
+
|
183
|
+
# Stops old rufus scheduler and creates a new one. Returns the new
|
184
|
+
# rufus scheduler
|
185
|
+
#
|
186
|
+
# @param [Symbol] stop_option The option to be passed to Rufus::Scheduler#stop
|
187
|
+
def clear_schedule!(stop_option = :wait)
|
188
|
+
if @rufus_scheduler
|
189
|
+
@rufus_scheduler.stop(stop_option)
|
190
|
+
@rufus_scheduler = nil
|
191
|
+
end
|
192
|
+
|
193
|
+
@@scheduled_jobs = {}
|
194
|
+
|
195
|
+
rufus_scheduler
|
196
|
+
end
|
197
|
+
|
198
|
+
def reload_schedule!
|
199
|
+
if enabled
|
200
|
+
Sidekiq.logger.info 'Reloading Schedule'
|
201
|
+
clear_schedule!
|
202
|
+
load_schedule!
|
203
|
+
else
|
204
|
+
Sidekiq.logger.info 'SidekiqScheduler is disabled'
|
205
|
+
end
|
206
|
+
end
|
207
|
+
|
208
|
+
def update_schedule
|
209
|
+
last_changed_score, @current_changed_score = @current_changed_score, Time.now.to_f
|
210
|
+
schedule_changes = SidekiqScheduler::RedisManager.get_schedule_changes(last_changed_score, @current_changed_score)
|
211
|
+
|
212
|
+
if schedule_changes.size > 0
|
213
|
+
Sidekiq.logger.info 'Updating schedule'
|
214
|
+
|
215
|
+
Sidekiq.reload_schedule!
|
216
|
+
schedule_changes.each do |schedule_name|
|
217
|
+
if Sidekiq.schedule.keys.include?(schedule_name)
|
218
|
+
unschedule_job(schedule_name)
|
219
|
+
load_schedule_job(schedule_name, Sidekiq.schedule[schedule_name])
|
220
|
+
else
|
221
|
+
unschedule_job(schedule_name)
|
222
|
+
end
|
223
|
+
end
|
224
|
+
Sidekiq.logger.info 'Schedule updated'
|
225
|
+
end
|
226
|
+
end
|
227
|
+
|
228
|
+
def job_enabled?(name)
|
229
|
+
job = Sidekiq.schedule[name]
|
230
|
+
schedule_state(name).fetch('enabled', job.fetch('enabled', true)) if job
|
231
|
+
end
|
232
|
+
|
233
|
+
def toggle_job_enabled(name)
|
234
|
+
state = schedule_state(name)
|
235
|
+
state['enabled'] = !job_enabled?(name)
|
236
|
+
set_schedule_state(name, state)
|
237
|
+
end
|
238
|
+
|
239
|
+
private
|
240
|
+
|
241
|
+
def new_job(name, interval_type, config, schedule, options)
|
242
|
+
options = options.merge({ :job => true, :tags => [name] })
|
243
|
+
|
244
|
+
rufus_scheduler.send(interval_type, schedule, options) do |job, time|
|
245
|
+
idempotent_job_enqueue(name, time, SidekiqScheduler::Utils.sanitize_job_config(config)) if job_enabled?(name)
|
246
|
+
end
|
247
|
+
end
|
248
|
+
|
249
|
+
def unschedule_job(name)
|
250
|
+
if scheduled_jobs[name]
|
251
|
+
Sidekiq.logger.debug "Removing schedule #{name}"
|
252
|
+
scheduled_jobs[name].unschedule
|
253
|
+
scheduled_jobs.delete(name)
|
254
|
+
end
|
255
|
+
end
|
256
|
+
|
257
|
+
# Retrieves a schedule state
|
258
|
+
#
|
259
|
+
# @param name [String] with the schedule's name
|
260
|
+
# @return [Hash] with the schedule's state
|
261
|
+
def schedule_state(name)
|
262
|
+
state = SidekiqScheduler::RedisManager.get_job_state(name)
|
263
|
+
|
264
|
+
state ? JSON.parse(state) : {}
|
265
|
+
end
|
266
|
+
|
267
|
+
# Saves a schedule state
|
268
|
+
#
|
269
|
+
# @param name [String] with the schedule's name
|
270
|
+
# @param name [Hash] with the schedule's state
|
271
|
+
def set_schedule_state(name, state)
|
272
|
+
SidekiqScheduler::RedisManager.set_job_state(name, state)
|
273
|
+
end
|
274
|
+
|
275
|
+
# Adds a Hash with schedule metadata as the last argument to call the worker.
|
276
|
+
# It currently returns the schedule time as a Float number representing the milisencods
|
277
|
+
# since epoch.
|
278
|
+
#
|
279
|
+
# @example with hash argument
|
280
|
+
# arguments_with_metadata({value: 1}, scheduled_at: Time.now)
|
281
|
+
# #=> [{value: 1}, {scheduled_at: <miliseconds since epoch>}]
|
282
|
+
#
|
283
|
+
# @param args [Array|Hash]
|
284
|
+
# @param metadata [Hash]
|
285
|
+
# @return [Array] arguments with added metadata
|
286
|
+
def arguments_with_metadata(args, metadata)
|
287
|
+
if args.is_a? Array
|
288
|
+
[*args, metadata]
|
289
|
+
else
|
290
|
+
[args, metadata]
|
291
|
+
end
|
292
|
+
end
|
293
|
+
|
294
|
+
def sidekiq_queues
|
295
|
+
Sidekiq.options[:queues].map(&:to_s)
|
296
|
+
end
|
297
|
+
|
298
|
+
# Returns true if a job's queue is included in the array of queues
|
299
|
+
#
|
300
|
+
# If queues are empty, returns true.
|
301
|
+
#
|
302
|
+
# @param [String] job_queue Job's queue name
|
303
|
+
# @param [Array<String>] queues
|
304
|
+
#
|
305
|
+
# @return [Boolean]
|
306
|
+
def enabled_queue?(job_queue, queues)
|
307
|
+
queues.empty? || queues.include?(job_queue)
|
308
|
+
end
|
309
|
+
|
310
|
+
# Returns true if the enqueuing needs to be done for an ActiveJob
|
311
|
+
# class false otherwise.
|
312
|
+
#
|
313
|
+
# @param [Class] klass the class to check is decendant from ActiveJob
|
314
|
+
#
|
315
|
+
# @return [Boolean]
|
316
|
+
def active_job_enqueue?(klass)
|
317
|
+
klass.is_a?(Class) && defined?(ActiveJob::Enqueuing) &&
|
318
|
+
klass.included_modules.include?(ActiveJob::Enqueuing)
|
319
|
+
end
|
320
|
+
|
321
|
+
# Convert the given arguments in the format expected to be enqueued.
|
322
|
+
#
|
323
|
+
# @param [Hash] config the options to be converted
|
324
|
+
# @option config [String] class the job class
|
325
|
+
# @option config [Hash/Array] args the arguments to be passed to the job
|
326
|
+
# class
|
327
|
+
#
|
328
|
+
# @return [Hash]
|
329
|
+
def prepare_arguments(config)
|
330
|
+
config['class'] = SidekiqScheduler::Utils.try_to_constantize(config['class'])
|
331
|
+
|
332
|
+
if config['args'].is_a?(Hash)
|
333
|
+
config['args'].symbolize_keys! if config['args'].respond_to?(:symbolize_keys!)
|
334
|
+
else
|
335
|
+
config['args'] = Array(config['args'])
|
336
|
+
end
|
337
|
+
|
338
|
+
config
|
339
|
+
end
|
340
|
+
|
341
|
+
# Returns true if the given schedule config hash matches the current ENV['RAILS_ENV']
|
342
|
+
# @param [Hash] config The schedule job configuration
|
343
|
+
#
|
344
|
+
# @return [Boolean] true if the schedule config matches the current ENV['RAILS_ENV']
|
345
|
+
def rails_env_matches?(config)
|
346
|
+
config['rails_env'] && ENV['RAILS_ENV'] && config['rails_env'].gsub(/\s/, '').split(',').include?(ENV['RAILS_ENV'])
|
347
|
+
end
|
348
|
+
|
349
|
+
def handle_errors
|
350
|
+
begin
|
351
|
+
yield
|
352
|
+
rescue StandardError => e
|
353
|
+
Sidekiq.logger.info "#{e.class.name}: #{e.message}"
|
354
|
+
end
|
355
|
+
end
|
356
|
+
end
|
357
|
+
end
|