rails-mongoid-scheduler 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/MIT-LICENSE +20 -0
- data/README.md +48 -0
- data/Rakefile +29 -0
- data/app/assets/config/scheduler_manifest.js +0 -0
- data/app/jobs/example_scheduler_job.rb +18 -0
- data/app/jobs/scheduler_job.rb +55 -0
- data/app/models/example_schedulable_model.rb +5 -0
- data/app/models/scheduler/schedulable.rb +201 -0
- data/config/routes.rb +2 -0
- data/lib/generators/scheduler/config_generator.rb +18 -0
- data/lib/generators/templates/scheduler.rb +42 -0
- data/lib/scheduler.rb +88 -0
- data/lib/scheduler/configuration.rb +24 -0
- data/lib/scheduler/engine.rb +8 -0
- data/lib/scheduler/main_process.rb +136 -0
- data/lib/scheduler/version.rb +3 -0
- data/lib/tasks/scheduler_tasks.rake +18 -0
- metadata +133 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 7400669410e9e4b5a7c85e4add4c6a6d0767f02ac4f52818e816c4dcc90391c8
|
4
|
+
data.tar.gz: de7abad7b8984bfbdb5512f69480eca81fc493d9c9b48582d1cab9e58eb31456
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: ff83ba06f6652263118cfe78c0d2f869ef1ae1e13dee21049872988a3464a0cf6fce7843bfae2801c069be0d4523c3fd518d4c95d618f62a729952b9a8b33f2a
|
7
|
+
data.tar.gz: b3a7de098fc62ba744e425a5c4224fa9e590b12fec074d43db440882b7edc917a628a02c731236ee98993eacfbc5fadc716fda60cbe68d750c96d7da90e907d3
|
data/MIT-LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright 2018 Francesco Ballardin
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,48 @@
|
|
1
|
+
# Scheduler
|
2
|
+
This gem aims to create a simple yet efficient framework to handle job scheduling and execution. Currently it supports only MongoDB as database.
|
3
|
+
|
4
|
+
## Installation
|
5
|
+
Add this line to your application's Gemfile:
|
6
|
+
|
7
|
+
```ruby
|
8
|
+
gem 'scheduler'
|
9
|
+
```
|
10
|
+
|
11
|
+
And then execute:
|
12
|
+
```bash
|
13
|
+
$ bundle install
|
14
|
+
```
|
15
|
+
|
16
|
+
And then install the config file with:
|
17
|
+
```bash
|
18
|
+
$ rails generate scheduler:config
|
19
|
+
```
|
20
|
+
|
21
|
+
## Usage
|
22
|
+
This gem adds a `Scheduler` module which can be started, stopped or restarted with their corresponding rake task:
|
23
|
+
```bash
|
24
|
+
$ rails scheduler:start
|
25
|
+
$ rails scheduler:stop
|
26
|
+
$ rails scheduler:restart
|
27
|
+
```
|
28
|
+
A `Scheduler` is a process that keeps running looking for jobs to perform. The jobs are documents of a specific collection that you can specify in the scheduler configuration file. You can specify your own model to act as a schedulable entity, as long as it includes the `Schedulable` module. The other configuration options are explained in the generated `config/initializers/scheduler.rb` file.
|
29
|
+
|
30
|
+
This gem also gives you a base ActiveJob implementation, called `SchedulerJob`, which you can subclass in order to implement your jobs.
|
31
|
+
|
32
|
+
As an example, the gem comes with a `ExampleSchedulableModel` which is a bare class that just includes the `Schedulable` module, and also an `ExampleSchedulerJob` job which is a bare implementation of the `SchedulableJob` job that just sleeps for a given amount of time.
|
33
|
+
|
34
|
+
First start by running the scheduler:
|
35
|
+
```bash
|
36
|
+
$ rails scheduler:start
|
37
|
+
```
|
38
|
+
|
39
|
+
You can then queue or run jobs by calling:
|
40
|
+
```ruby
|
41
|
+
ExampleSchedulableModel.perform_later('ExampleSchedulableJob', 10) # to queue
|
42
|
+
ExampleSchedulableModel.perform_now('ExampleSchedulableJob', 10) # to perform immediately
|
43
|
+
```
|
44
|
+
Both methods create a document of `ExampleSchedulableModel` and put it in queue.
|
45
|
+
The __perform_now__ method skips the scheduler and performs the job immediately, instead the __perform_later__ leaves the performing task to the scheduler.
|
46
|
+
|
47
|
+
## License
|
48
|
+
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
data/Rakefile
ADDED
@@ -0,0 +1,29 @@
|
|
1
|
+
begin
|
2
|
+
require 'bundler/setup'
|
3
|
+
rescue LoadError
|
4
|
+
puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
|
5
|
+
end
|
6
|
+
|
7
|
+
require 'rdoc/task'
|
8
|
+
|
9
|
+
RDoc::Task.new(:rdoc) do |rdoc|
|
10
|
+
rdoc.rdoc_dir = 'rdoc'
|
11
|
+
rdoc.title = 'Scheduler'
|
12
|
+
rdoc.options << '--line-numbers'
|
13
|
+
rdoc.rdoc_files.include('README.md')
|
14
|
+
rdoc.rdoc_files.include('lib/**/*.rb')
|
15
|
+
end
|
16
|
+
|
17
|
+
load 'rails/tasks/statistics.rake'
|
18
|
+
|
19
|
+
require 'bundler/gem_tasks'
|
20
|
+
|
21
|
+
require 'rake/testtask'
|
22
|
+
|
23
|
+
Rake::TestTask.new(:test) do |t|
|
24
|
+
t.libs << 'test'
|
25
|
+
t.pattern = 'test/**/*_test.rb'
|
26
|
+
t.verbose = false
|
27
|
+
end
|
28
|
+
|
29
|
+
task default: :test
|
File without changes
|
@@ -0,0 +1,18 @@
|
|
1
|
+
class ExampleSchedulerJob < SchedulerJob
|
2
|
+
|
3
|
+
##
|
4
|
+
# Performs job with ActiveJob framework.
|
5
|
+
#
|
6
|
+
# @param [String] job_id the id of the corresponding Job.
|
7
|
+
# @param [Integer] work_time an example amount of time to simulate work.
|
8
|
+
#
|
9
|
+
# @return [nil]
|
10
|
+
def perform(job_class, job_id, *args, &block)
|
11
|
+
super do |job, work_time|
|
12
|
+
job.log :info, "Preparing to do some work for #{work_time} seconds."
|
13
|
+
sleep work_time
|
14
|
+
job.log :info, "Work done!"
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
end
|
@@ -0,0 +1,55 @@
|
|
1
|
+
class SchedulerJob < ActiveJob::Base
|
2
|
+
|
3
|
+
queue_as :default
|
4
|
+
|
5
|
+
##
|
6
|
+
# Performs job with ActiveJob framework.
|
7
|
+
#
|
8
|
+
# @param [String] job_class the class of the corresponding Job.
|
9
|
+
# @param [String] job_id the id of the corresponding Job.
|
10
|
+
# @param [Array] *args additional arguments.
|
11
|
+
# @param [Proc] &block extra block to define custom jobs.
|
12
|
+
#
|
13
|
+
# @return [nil]
|
14
|
+
def perform(job_class, job_id, *args, &block)
|
15
|
+
begin
|
16
|
+
@job = job_class.constantize.find(job_id)
|
17
|
+
@job.executed_at = Time.current
|
18
|
+
@job.status!(:running)
|
19
|
+
yield @job, *args if block_given?
|
20
|
+
@job.completed_at = Time.current
|
21
|
+
@job.progress!(100)
|
22
|
+
@job.status!(:completed)
|
23
|
+
rescue StandardError => error
|
24
|
+
handle_error(error, job_class, job_id)
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
##
|
29
|
+
# Method to handle any error raised when performing a job.
|
30
|
+
#
|
31
|
+
# @param [StandardError] error the raised error.
|
32
|
+
# @param [String] job_class the class of the corresponding Job.
|
33
|
+
# @param [String] job_id the id of the corresponding Job.
|
34
|
+
# @param [Proc] &block extra block to define custom error handling.
|
35
|
+
#
|
36
|
+
# @return [nil]
|
37
|
+
def handle_error(error, job_class, job_id, &block)
|
38
|
+
if @job.present?
|
39
|
+
@job.completed_at = Time.current
|
40
|
+
if block_given?
|
41
|
+
yield error
|
42
|
+
else
|
43
|
+
backtrace = error.backtrace.select { |line| line.include?('app') }.join("\n")
|
44
|
+
@job.log(:error, "#{error.class}: #{error.message}")
|
45
|
+
@job.log(:error, backtrace)
|
46
|
+
@job.backtrace = backtrace
|
47
|
+
@job.status!(:error)
|
48
|
+
end
|
49
|
+
@job.save
|
50
|
+
else
|
51
|
+
raise "Unable to find #{job_class} with id '#{job_id}'."
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
end
|
@@ -0,0 +1,201 @@
|
|
1
|
+
module Scheduler
|
2
|
+
module Schedulable
|
3
|
+
|
4
|
+
##
|
5
|
+
# Possible schedulable statuses.
|
6
|
+
STATUSES = [ :queued, :running, :completed, :error, :locked ]
|
7
|
+
|
8
|
+
##
|
9
|
+
# Possible log levels.
|
10
|
+
LOG_LEVELS = [ :debug, :info, :warn, :error ]
|
11
|
+
|
12
|
+
def self.included(base)
|
13
|
+
|
14
|
+
base.class_eval do
|
15
|
+
include Mongoid::Document
|
16
|
+
|
17
|
+
field :class_name, type: String
|
18
|
+
field :args, type: Array, default: []
|
19
|
+
field :scheduled_at, type: DateTime
|
20
|
+
field :executed_at, type: DateTime
|
21
|
+
field :completed_at, type: DateTime
|
22
|
+
field :pid, type: Integer
|
23
|
+
field :status, type: Symbol, default: :queued
|
24
|
+
field :logs, type: Array, default: []
|
25
|
+
field :progress, type: Float, default: 0.0
|
26
|
+
field :error, type: String
|
27
|
+
field :backtrace, type: String
|
28
|
+
|
29
|
+
scope :queued, -> { where(status: :queued) }
|
30
|
+
scope :running, -> { where(status: :running) }
|
31
|
+
scope :completed, -> { where(status: :completed) }
|
32
|
+
scope :in_error, -> { where(status: :error) }
|
33
|
+
scope :locked, -> { where(status: :locked) }
|
34
|
+
|
35
|
+
validates_presence_of :class_name
|
36
|
+
|
37
|
+
after_save :broadcast
|
38
|
+
|
39
|
+
class << self
|
40
|
+
|
41
|
+
##
|
42
|
+
# Returns possible statuses.
|
43
|
+
#
|
44
|
+
# @return [Array<Symbol>] possible statuses.
|
45
|
+
def statuses
|
46
|
+
Scheduler::Schedulable::STATUSES
|
47
|
+
end
|
48
|
+
|
49
|
+
##
|
50
|
+
# Returns possible log levels.
|
51
|
+
#
|
52
|
+
# @return [Array<Symbol>] possible log levels.
|
53
|
+
def log_levels
|
54
|
+
Scheduler::Schedulable::LOG_LEVELS
|
55
|
+
end
|
56
|
+
|
57
|
+
##
|
58
|
+
# Returns the corresponding log color if this level.
|
59
|
+
#
|
60
|
+
# @param [Symbol] level log level.
|
61
|
+
#
|
62
|
+
# @return [Symbol] the color.
|
63
|
+
def log_color(level)
|
64
|
+
case level
|
65
|
+
when :debug then :green
|
66
|
+
when :info then :cyan
|
67
|
+
when :warn then :yellow
|
68
|
+
when :error then :red
|
69
|
+
end
|
70
|
+
end
|
71
|
+
|
72
|
+
##
|
73
|
+
# Creates an instance of this class and calls :perform_later.
|
74
|
+
#
|
75
|
+
# @param [String] job_class the class of the job to run.
|
76
|
+
# @param [Array] *job_args job arguments
|
77
|
+
#
|
78
|
+
# @return [Object] the created job.
|
79
|
+
def perform_later(job_class, *job_args)
|
80
|
+
self.create(class_name: job_class, args: job_args).perform_later
|
81
|
+
end
|
82
|
+
|
83
|
+
##
|
84
|
+
# Creates an instance of this class and calls :perform_now.
|
85
|
+
#
|
86
|
+
# @param [String] job_class the class of the job to run.
|
87
|
+
# @param [Array] *job_args job arguments
|
88
|
+
#
|
89
|
+
# @return [Object] the created job.
|
90
|
+
def perform_now(job_class, *job_args)
|
91
|
+
self.create(class_name: job_class, args: job_args).perform_now
|
92
|
+
end
|
93
|
+
|
94
|
+
end
|
95
|
+
|
96
|
+
end
|
97
|
+
|
98
|
+
##
|
99
|
+
# Gets ActiveJob's job class.
|
100
|
+
#
|
101
|
+
# @return [Class] the ActiveJob's job class.
|
102
|
+
def job_class
|
103
|
+
self.class_name.constantize
|
104
|
+
end
|
105
|
+
|
106
|
+
##
|
107
|
+
# Schedules the job.
|
108
|
+
#
|
109
|
+
# @return [Object] itself.
|
110
|
+
def schedule
|
111
|
+
self.scheduled_at = Time.current
|
112
|
+
self.status = :queued
|
113
|
+
self.logs = []
|
114
|
+
self.progress = 0.0
|
115
|
+
self.unset(:error)
|
116
|
+
self.unset(:backtrace)
|
117
|
+
self.unset(:completed_at)
|
118
|
+
self.unset(:executed_at)
|
119
|
+
self.save
|
120
|
+
if block_given?
|
121
|
+
yield self
|
122
|
+
else self end
|
123
|
+
end
|
124
|
+
|
125
|
+
##
|
126
|
+
# Performs job when queue is available.
|
127
|
+
# On test or development env, the job is performed by ActiveJob queue,
|
128
|
+
# if configured on the Scheduler configuration.
|
129
|
+
# On production env, the job is performed only with a Scheduler::MainProcess.
|
130
|
+
#
|
131
|
+
# @return [Object] the job class.
|
132
|
+
def perform_later
|
133
|
+
self.schedule
|
134
|
+
if Rails.env.development? or Rails.env.test?
|
135
|
+
if Scheduler.configuration.perform_jobs_in_test_or_development
|
136
|
+
job_class.set(wait: 5.seconds).perform_later(self.class.name, self.id.to_s, *self.args)
|
137
|
+
end
|
138
|
+
end
|
139
|
+
self
|
140
|
+
end
|
141
|
+
|
142
|
+
##
|
143
|
+
# Performs job when queue is available.
|
144
|
+
# On test or development env, the job is performed with ActiveJob.
|
145
|
+
# On production env, the job is performed with Scheduler.
|
146
|
+
#
|
147
|
+
# @return [Object] the job class.
|
148
|
+
def perform_now
|
149
|
+
self.schedule
|
150
|
+
job_class.perform_now(self.class.name, self.id.to_s, *self.args)
|
151
|
+
self
|
152
|
+
end
|
153
|
+
|
154
|
+
##
|
155
|
+
# Immediately update the status to the given one.
|
156
|
+
#
|
157
|
+
# @param [Symbol] status the status to update.
|
158
|
+
#
|
159
|
+
# @return [nil]
|
160
|
+
def status!(status)
|
161
|
+
self.update(status: status)
|
162
|
+
end
|
163
|
+
|
164
|
+
##
|
165
|
+
# Immediately increases progress to the given amount.
|
166
|
+
#
|
167
|
+
# @param [Float] amount the given progress amount.
|
168
|
+
def progress!(amount)
|
169
|
+
self.update(progress: amount.to_f) if amount.numeric?
|
170
|
+
end
|
171
|
+
|
172
|
+
##
|
173
|
+
# Immediately increases progress by the given amount.
|
174
|
+
#
|
175
|
+
# @param [Float] amount the given progress amount.
|
176
|
+
def progress_by!(amount)
|
177
|
+
self.update(progress: progress + amount.to_f) if amount.numeric?
|
178
|
+
end
|
179
|
+
|
180
|
+
##
|
181
|
+
# Registers a log message with the given level.
|
182
|
+
def log(level, message)
|
183
|
+
raise ArgumentError.new("The given log level '#{level}' is not valid. "\
|
184
|
+
"Valid log levels are: #{LOG_LEVELS.join(', ')}") unless level.in? LOG_LEVELS
|
185
|
+
Scheduler.configuration.logger.send level,
|
186
|
+
"[#{self.class}:#{self.id}] #{message}".send(self.class.log_color level)
|
187
|
+
self.update(logs: logs.push([level, message]))
|
188
|
+
end
|
189
|
+
|
190
|
+
##
|
191
|
+
# Broadcasts a job updating event.
|
192
|
+
#
|
193
|
+
# @return [nil]
|
194
|
+
def broadcast
|
195
|
+
{ status: status, logs: logs }
|
196
|
+
end
|
197
|
+
|
198
|
+
end
|
199
|
+
|
200
|
+
end
|
201
|
+
end
|
data/config/routes.rb
ADDED
@@ -0,0 +1,18 @@
|
|
1
|
+
require 'rails/generators'
|
2
|
+
|
3
|
+
module Scheduler
|
4
|
+
module Generators
|
5
|
+
|
6
|
+
class ConfigGenerator < Rails::Generators::Base
|
7
|
+
source_root File.expand_path("../../templates", __FILE__)
|
8
|
+
|
9
|
+
desc "Creates a scheduler configuration file."
|
10
|
+
|
11
|
+
def copy_config
|
12
|
+
template "scheduler.rb", "config/initializers/scheduler.rb"
|
13
|
+
end
|
14
|
+
|
15
|
+
end
|
16
|
+
|
17
|
+
end
|
18
|
+
end
|
@@ -0,0 +1,42 @@
|
|
1
|
+
##
|
2
|
+
# Example configuration file for rails-scheduler gem.
|
3
|
+
Scheduler.configure do |config|
|
4
|
+
|
5
|
+
##
|
6
|
+
# A logger interface to save scheduler logs.
|
7
|
+
# Defaults to Rails.logger.
|
8
|
+
#
|
9
|
+
# config.logger = Rails.logger
|
10
|
+
|
11
|
+
##
|
12
|
+
# A custom class to handle the execution of jobs.
|
13
|
+
# This class must include Scheduler::Schedulable module
|
14
|
+
# in order to work.
|
15
|
+
# Defaults to ExampleSchedulableModel, which is a bare class that
|
16
|
+
# just includes Scheduler::Schedulable module.
|
17
|
+
#
|
18
|
+
# config.job_class = ExampleSchedulableModel
|
19
|
+
|
20
|
+
##
|
21
|
+
# How often the scheduler has to check for new jobs to run.
|
22
|
+
# Defaults to 5 seconds.
|
23
|
+
#
|
24
|
+
# config.polling_interval = 5
|
25
|
+
|
26
|
+
##
|
27
|
+
# How many jobs can run at a given time.
|
28
|
+
# Defaults to the minimum value between the number of the current
|
29
|
+
# machine CPU cores or 24.
|
30
|
+
#
|
31
|
+
# config.max_concurrent_jobs = [ Etc.nprocessors, 24 ].min
|
32
|
+
|
33
|
+
##
|
34
|
+
# Sets whether to perform jobs when in test or development env.
|
35
|
+
# Usually jobs are performed only when a Scheduler::MainProcess is running.
|
36
|
+
# But for convenience, you can set this parameter to true so you
|
37
|
+
# don't need to keep a Scheduler::MainProcess running.
|
38
|
+
# Defaults to false.
|
39
|
+
#
|
40
|
+
# config.perform_jobs_in_test_or_development = false
|
41
|
+
|
42
|
+
end
|
data/lib/scheduler.rb
ADDED
@@ -0,0 +1,88 @@
|
|
1
|
+
require "scheduler/engine"
|
2
|
+
require "scheduler/configuration"
|
3
|
+
require "scheduler/main_process"
|
4
|
+
|
5
|
+
module Scheduler
|
6
|
+
|
7
|
+
class << self
|
8
|
+
|
9
|
+
# @return [Scheduler::Configuration] the configuration class for Scheduler.
|
10
|
+
attr_accessor :configuration
|
11
|
+
|
12
|
+
##
|
13
|
+
# Initializes configuration.
|
14
|
+
#
|
15
|
+
# @return [Scheduler::Configuration] the configuration class for Scheduler.
|
16
|
+
def configuration
|
17
|
+
@configuration || Scheduler::Configuration.new
|
18
|
+
end
|
19
|
+
|
20
|
+
##
|
21
|
+
# Method to configure various Scheduler options.
|
22
|
+
#
|
23
|
+
# @return [nil]
|
24
|
+
def configure
|
25
|
+
@configuration ||= Scheduler::Configuration.new
|
26
|
+
yield @configuration
|
27
|
+
end
|
28
|
+
|
29
|
+
##
|
30
|
+
# Gets scheduler pid file.
|
31
|
+
#
|
32
|
+
# @return [String] the pid file.
|
33
|
+
def pid_file
|
34
|
+
'/tmp/rails-scheduler.pid'
|
35
|
+
end
|
36
|
+
|
37
|
+
##
|
38
|
+
# Gets scheduler main process pid.
|
39
|
+
#
|
40
|
+
# @return [Integer] main process pid.
|
41
|
+
def pid
|
42
|
+
File.read(self.pid_file).to_i rescue nil
|
43
|
+
end
|
44
|
+
|
45
|
+
##
|
46
|
+
# Starts a Scheduler::MainProcess in a separate process.
|
47
|
+
#
|
48
|
+
# @return [nil]
|
49
|
+
def start
|
50
|
+
scheduler_pid = Process.fork do
|
51
|
+
begin
|
52
|
+
Process.daemon(true)
|
53
|
+
File.open(self.pid_file, 'w+') do |pidfile|
|
54
|
+
pidfile.puts Process.pid
|
55
|
+
end
|
56
|
+
scheduler = Scheduler::MainProcess.new
|
57
|
+
rescue StandardError => e
|
58
|
+
Rails.logger.error "#{e.class}: #{e.message} (#{e.backtrace.first})".red
|
59
|
+
end
|
60
|
+
end
|
61
|
+
Process.detach(scheduler_pid)
|
62
|
+
scheduler_pid
|
63
|
+
end
|
64
|
+
|
65
|
+
##
|
66
|
+
# Reschedules all running jobs and stops the scheduler main process.
|
67
|
+
#
|
68
|
+
# @return [nil]
|
69
|
+
def stop
|
70
|
+
begin
|
71
|
+
Process.kill :TERM, Scheduler.pid
|
72
|
+
FileUtils.rm(self.pid_file)
|
73
|
+
rescue Errno::ENOENT, Errno::ESRCH
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
##
|
78
|
+
# Restarts the scheduler.
|
79
|
+
#
|
80
|
+
# @return [nil]
|
81
|
+
def restart
|
82
|
+
self.stop
|
83
|
+
self.start
|
84
|
+
end
|
85
|
+
|
86
|
+
end
|
87
|
+
|
88
|
+
end
|
@@ -0,0 +1,24 @@
|
|
1
|
+
module Scheduler
|
2
|
+
class Configuration
|
3
|
+
|
4
|
+
# @return [String] a logger file.
|
5
|
+
attr_accessor :logger
|
6
|
+
# @return [Class] the class of the main job model.
|
7
|
+
attr_accessor :job_class
|
8
|
+
# @return [Integer] how much time to wait before each iteration.
|
9
|
+
attr_accessor :polling_interval
|
10
|
+
# @return [Integer] maximum number of concurent jobs.
|
11
|
+
attr_accessor :max_concurrent_jobs
|
12
|
+
# @return [Boolean] whether to perform jobs when in test or development env.
|
13
|
+
attr_accessor :perform_jobs_in_test_or_development
|
14
|
+
|
15
|
+
def initialize
|
16
|
+
@logger = Rails.logger
|
17
|
+
@job_class = ExampleSchedulableModel
|
18
|
+
@polling_interval = 5
|
19
|
+
@max_concurrent_jobs = [ Etc.nprocessors, 24 ].min
|
20
|
+
@perform_jobs_in_test_or_development = false
|
21
|
+
end
|
22
|
+
|
23
|
+
end
|
24
|
+
end
|
@@ -0,0 +1,136 @@
|
|
1
|
+
module Scheduler
|
2
|
+
class MainProcess
|
3
|
+
|
4
|
+
# @return [Integer] pid of the main process.
|
5
|
+
attr_accessor :pid
|
6
|
+
# @return [String] a logger file.
|
7
|
+
attr_accessor :logger
|
8
|
+
# @return [Class] the class of the main job model.
|
9
|
+
attr_accessor :job_class
|
10
|
+
# @return [Integer] how much time to wait before each iteration.
|
11
|
+
attr_accessor :polling_interval
|
12
|
+
# @return [Integer] maximum number of concurent jobs.
|
13
|
+
attr_accessor :max_concurrent_jobs
|
14
|
+
|
15
|
+
##
|
16
|
+
# Creates a MainProcess which keeps running
|
17
|
+
# and continuously checks if new jobs are queued.
|
18
|
+
#
|
19
|
+
# @return [Scheduler::MainProcess] the created MainProcess.
|
20
|
+
def initialize
|
21
|
+
@pid = Process.pid
|
22
|
+
@logger = Scheduler.configuration.logger
|
23
|
+
@job_class = Scheduler.configuration.job_class
|
24
|
+
@polling_interval = Scheduler.configuration.polling_interval
|
25
|
+
@max_concurrent_jobs = Scheduler.configuration.max_concurrent_jobs
|
26
|
+
|
27
|
+
unless @logger.instance_of?(ActiveSupport::Logger) or @logger.instance_of?(Logger)
|
28
|
+
@logger = Logger.new(@logger)
|
29
|
+
end
|
30
|
+
|
31
|
+
if @polling_interval < 1
|
32
|
+
@logger.warn "[Scheduler:#{@pid}] Warning: specified a polling interval lesser than 1: "\
|
33
|
+
"it will be forced to 1.".yellow
|
34
|
+
@polling_interval = 1
|
35
|
+
end
|
36
|
+
|
37
|
+
unless @job_class.included_modules.include? Scheduler::Schedulable
|
38
|
+
raise "The given job class '#{@job_class}' is not a Schedulable class. "\
|
39
|
+
"Make sure to add 'include Scheduler::Schedulable' to your class."
|
40
|
+
end
|
41
|
+
|
42
|
+
@logger.info "[Scheduler:#{@pid}] Starting scheduler..".cyan
|
43
|
+
self.start_loop
|
44
|
+
end
|
45
|
+
|
46
|
+
##
|
47
|
+
# Main loop.
|
48
|
+
#
|
49
|
+
# @return [nil]
|
50
|
+
def start_loop
|
51
|
+
loop do
|
52
|
+
begin
|
53
|
+
# Loads up a job queue.
|
54
|
+
queue = []
|
55
|
+
|
56
|
+
# Counts jobs to schedule.
|
57
|
+
running_jobs = @job_class.running.entries
|
58
|
+
schedulable_jobs = @job_class.queued.order_by(scheduled_at: :asc).entries
|
59
|
+
jobs_to_schedule = @max_concurrent_jobs - running_jobs.count
|
60
|
+
jobs_to_schedule = 0 if jobs_to_schedule < 0
|
61
|
+
|
62
|
+
# Finds out scheduled jobs waiting to be performed.
|
63
|
+
scheduled_jobs = []
|
64
|
+
schedulable_jobs.first(jobs_to_schedule).each do |job|
|
65
|
+
job_pid = Process.fork do
|
66
|
+
begin
|
67
|
+
job.perform_now
|
68
|
+
rescue StandardError => e
|
69
|
+
@logger.error "[Scheduler:#{@pid}] Error #{e.class}: #{e.message} "\
|
70
|
+
"(#{e.backtrace.select { |l| l.include?('app') }.first}).".red
|
71
|
+
end
|
72
|
+
end
|
73
|
+
Process.detach(job_pid)
|
74
|
+
job.update_attribute(:pid, job_pid)
|
75
|
+
scheduled_jobs << job
|
76
|
+
queue << job.id.to_s
|
77
|
+
end
|
78
|
+
|
79
|
+
# Logs launched jobs
|
80
|
+
if scheduled_jobs.any?
|
81
|
+
@logger.info "[Scheduler:#{@pid}] Launched #{scheduled_jobs.count} "\
|
82
|
+
"jobs: #{scheduled_jobs.map(&:id).map(&:to_s).join(', ')}.".cyan
|
83
|
+
else
|
84
|
+
if schedulable_jobs.count == 0
|
85
|
+
@logger.info "[Scheduler:#{@pid}] No jobs in queue.".cyan
|
86
|
+
else
|
87
|
+
@logger.warn "[Scheduler:#{@pid}] No jobs launched, reached maximum "\
|
88
|
+
"number of concurrent jobs. Jobs in queue: #{schedulable_jobs.count}.".yellow
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
# Checks for completed jobs: clears up queue and kills any zombie pid
|
93
|
+
queue.delete_if do |job_id|
|
94
|
+
job = @job_class.find(job_id)
|
95
|
+
if job.present? and job.status.in? [ :completed, :error ]
|
96
|
+
begin
|
97
|
+
@logger.info "[Scheduler:#{@pid}] Rimosso processo #{job.pid} per lavoro completato".cyan
|
98
|
+
Process.kill :QUIT, job.pid
|
99
|
+
rescue Errno::ENOENT, Errno::ESRCH
|
100
|
+
end
|
101
|
+
true
|
102
|
+
else false end
|
103
|
+
end
|
104
|
+
|
105
|
+
# Waits the specified amount of time before next iteration
|
106
|
+
sleep @polling_interval
|
107
|
+
rescue StandardError => error
|
108
|
+
@logger.error "[Scheduler:#{@pid}] Error #{error.message}".red
|
109
|
+
@logger.error error.backtrace.select { |line| line.include?('app') }.join("\n").red
|
110
|
+
rescue SignalException => signal
|
111
|
+
if signal.message.in? [ 'SIGINT', 'SIGTERM', 'SIGQUIT' ]
|
112
|
+
@logger.warn "[Scheduler:#{@pid}] Received interrupt, terminating scheduler..".yellow
|
113
|
+
reschedule_running_jobs
|
114
|
+
break
|
115
|
+
end
|
116
|
+
end
|
117
|
+
end
|
118
|
+
end
|
119
|
+
|
120
|
+
##
|
121
|
+
# Reschedules currently running jobs.
|
122
|
+
#
|
123
|
+
# @return [nil]
|
124
|
+
def reschedule_running_jobs
|
125
|
+
@job_class.running.each do |job|
|
126
|
+
begin
|
127
|
+
Process.kill :QUIT, job.pid if job.pid.present?
|
128
|
+
rescue Errno::ESRCH, Errno::EPERM
|
129
|
+
ensure
|
130
|
+
job.schedule
|
131
|
+
end
|
132
|
+
end
|
133
|
+
end
|
134
|
+
|
135
|
+
end
|
136
|
+
end
|
@@ -0,0 +1,18 @@
|
|
1
|
+
namespace :scheduler do
|
2
|
+
|
3
|
+
desc 'Scheduler Start'
|
4
|
+
task :start => :environment do |t, args|
|
5
|
+
Scheduler.start
|
6
|
+
end
|
7
|
+
|
8
|
+
desc 'Scheduler Stop'
|
9
|
+
task :stop => :environment do |t, args|
|
10
|
+
Scheduler.stop
|
11
|
+
end
|
12
|
+
|
13
|
+
desc 'Scheduler Restart'
|
14
|
+
task :restart => :environment do |t, args|
|
15
|
+
Scheduler.restart
|
16
|
+
end
|
17
|
+
|
18
|
+
end
|
metadata
ADDED
@@ -0,0 +1,133 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: rails-mongoid-scheduler
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Francesco Ballardin
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2018-12-13 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: rails
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: 5.2.2
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: 5.2.2
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: mongoid
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: 7.0.2
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: 7.0.2
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: bson_ext
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :runtime
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: whenever
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: 0.10.0
|
62
|
+
type: :runtime
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: 0.10.0
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: rails-dev-tools
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - ">="
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '0'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - ">="
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '0'
|
83
|
+
description: A Rails engine to schedule jobs, handle parallel execution and manage
|
84
|
+
the jobs queue.
|
85
|
+
email:
|
86
|
+
- francesco.ballardin@gmail.com
|
87
|
+
executables: []
|
88
|
+
extensions: []
|
89
|
+
extra_rdoc_files: []
|
90
|
+
files:
|
91
|
+
- MIT-LICENSE
|
92
|
+
- README.md
|
93
|
+
- Rakefile
|
94
|
+
- app/assets/config/scheduler_manifest.js
|
95
|
+
- app/jobs/example_scheduler_job.rb
|
96
|
+
- app/jobs/scheduler_job.rb
|
97
|
+
- app/models/example_schedulable_model.rb
|
98
|
+
- app/models/scheduler/schedulable.rb
|
99
|
+
- config/routes.rb
|
100
|
+
- lib/generators/scheduler/config_generator.rb
|
101
|
+
- lib/generators/templates/scheduler.rb
|
102
|
+
- lib/scheduler.rb
|
103
|
+
- lib/scheduler/configuration.rb
|
104
|
+
- lib/scheduler/engine.rb
|
105
|
+
- lib/scheduler/main_process.rb
|
106
|
+
- lib/scheduler/version.rb
|
107
|
+
- lib/tasks/scheduler_tasks.rake
|
108
|
+
homepage: https://github.com/Pluvie/rails-scheduler
|
109
|
+
licenses:
|
110
|
+
- MIT
|
111
|
+
metadata: {}
|
112
|
+
post_install_message:
|
113
|
+
rdoc_options: []
|
114
|
+
require_paths:
|
115
|
+
- lib
|
116
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
117
|
+
requirements:
|
118
|
+
- - ">="
|
119
|
+
- !ruby/object:Gem::Version
|
120
|
+
version: '0'
|
121
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
122
|
+
requirements:
|
123
|
+
- - ">="
|
124
|
+
- !ruby/object:Gem::Version
|
125
|
+
version: '0'
|
126
|
+
requirements: []
|
127
|
+
rubyforge_project:
|
128
|
+
rubygems_version: 2.7.6
|
129
|
+
signing_key:
|
130
|
+
specification_version: 4
|
131
|
+
summary: A Rails engine to schedule jobs, handle parallel execution and manage the
|
132
|
+
jobs queue.
|
133
|
+
test_files: []
|