resque-scheduler 2.3.1 → 2.4.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of resque-scheduler might be problematic. Click here for more details.
- data/.gitignore +3 -0
- data/.rubocop.yml +11 -11
- data/.simplecov +1 -0
- data/.travis.yml +5 -2
- data/AUTHORS.md +3 -0
- data/HISTORY.md +26 -2
- data/LICENSE +1 -1
- data/README.md +120 -31
- data/ROADMAP.md +10 -0
- data/Rakefile +7 -19
- data/bin/resque-scheduler +5 -0
- data/examples/Rakefile +2 -0
- data/examples/config/initializers/resque-web.rb +37 -0
- data/examples/dynamic-scheduling/README.md +28 -0
- data/examples/dynamic-scheduling/app/jobs/fix_schedules_job.rb +54 -0
- data/examples/dynamic-scheduling/app/jobs/send_email_job.rb +9 -0
- data/examples/dynamic-scheduling/app/models/user.rb +16 -0
- data/examples/dynamic-scheduling/config/resque.yml +4 -0
- data/examples/dynamic-scheduling/config/static_schedule.yml +7 -0
- data/examples/dynamic-scheduling/lib/tasks/resque.rake +48 -0
- data/examples/run-resque-web +3 -0
- data/lib/resque-scheduler.rb +2 -0
- data/lib/resque/scheduler.rb +130 -41
- data/lib/resque/scheduler/lock/resilient.rb +1 -1
- data/lib/resque/scheduler_locking.rb +3 -1
- data/lib/resque_scheduler.rb +73 -31
- data/lib/resque_scheduler/cli.rb +160 -0
- data/lib/resque_scheduler/logger_builder.rb +27 -8
- data/lib/resque_scheduler/plugin.rb +10 -7
- data/lib/resque_scheduler/server.rb +52 -11
- data/lib/resque_scheduler/server/views/delayed.erb +2 -0
- data/lib/resque_scheduler/server/views/delayed_schedules.erb +20 -0
- data/lib/resque_scheduler/server/views/scheduler.erb +4 -12
- data/lib/resque_scheduler/tasks.rb +15 -27
- data/lib/resque_scheduler/version.rb +1 -1
- data/resque-scheduler.gemspec +2 -0
- data/test/cli_test.rb +286 -0
- data/test/delayed_queue_test.rb +70 -1
- data/test/resque-web_test.rb +36 -1
- data/test/scheduler_args_test.rb +51 -17
- data/test/scheduler_hooks_test.rb +1 -1
- data/test/scheduler_locking_test.rb +63 -1
- data/test/scheduler_setup_test.rb +54 -18
- data/test/scheduler_task_test.rb +35 -0
- data/test/scheduler_test.rb +130 -42
- data/test/support/redis_instance.rb +8 -3
- data/test/test_helper.rb +47 -20
- metadata +77 -6
- checksums.yaml +0 -15
data/.gitignore
CHANGED
data/.rubocop.yml
CHANGED
@@ -2,6 +2,9 @@
|
|
2
2
|
# The point is for the user to remove these configuration records
|
3
3
|
# one by one as the offences are removed from the code base.
|
4
4
|
|
5
|
+
AccessorMethodName:
|
6
|
+
Enabled: false
|
7
|
+
|
5
8
|
AlignParameters:
|
6
9
|
Enabled: false
|
7
10
|
|
@@ -15,7 +18,7 @@ CaseEquality:
|
|
15
18
|
Enabled: false
|
16
19
|
|
17
20
|
ClassLength:
|
18
|
-
|
21
|
+
Max: 300
|
19
22
|
|
20
23
|
ClassVars:
|
21
24
|
Enabled: false
|
@@ -30,17 +33,11 @@ CommentAnnotation:
|
|
30
33
|
Enabled: false
|
31
34
|
|
32
35
|
CyclomaticComplexity:
|
33
|
-
|
36
|
+
Max: 18
|
34
37
|
|
35
38
|
Documentation:
|
36
39
|
Enabled: false
|
37
40
|
|
38
|
-
EmptyLines:
|
39
|
-
Enabled: false
|
40
|
-
|
41
|
-
EmptyLinesAroundBody:
|
42
|
-
Enabled: false
|
43
|
-
|
44
41
|
Encoding:
|
45
42
|
Enabled: false
|
46
43
|
|
@@ -59,9 +56,12 @@ HashSyntax:
|
|
59
56
|
IfUnlessModifier:
|
60
57
|
Enabled: false
|
61
58
|
|
62
|
-
|
59
|
+
IndentationWidth:
|
63
60
|
Enabled: false
|
64
61
|
|
62
|
+
LineLength:
|
63
|
+
Max: 152
|
64
|
+
|
65
65
|
Loop:
|
66
66
|
Enabled: false
|
67
67
|
|
@@ -69,7 +69,7 @@ MethodCallParentheses:
|
|
69
69
|
Enabled: false
|
70
70
|
|
71
71
|
MethodLength:
|
72
|
-
|
72
|
+
Max: 90
|
73
73
|
|
74
74
|
ModuleFunction:
|
75
75
|
Enabled: false
|
@@ -80,7 +80,7 @@ NumericLiterals:
|
|
80
80
|
ParenthesesAroundCondition:
|
81
81
|
Enabled: false
|
82
82
|
|
83
|
-
|
83
|
+
PredicateName:
|
84
84
|
Enabled: false
|
85
85
|
|
86
86
|
RedundantBegin:
|
data/.simplecov
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
SimpleCov.start { add_filter '/test/' } if ENV['COVERAGE']
|
data/.travis.yml
CHANGED
@@ -1,9 +1,12 @@
|
|
1
1
|
language: ruby
|
2
2
|
rvm:
|
3
|
-
- ree
|
4
3
|
- 1.9.3
|
5
4
|
- 2.0.0
|
6
|
-
|
5
|
+
env:
|
6
|
+
global:
|
7
|
+
- RESQUE_SCHEDULER_DISABLE_TEST_REDIS_SERVER=1
|
8
|
+
services:
|
9
|
+
- redis-server
|
7
10
|
notifications:
|
8
11
|
email:
|
9
12
|
recipients: daniel.buch+resque-scheduler@gmail.com
|
data/AUTHORS.md
CHANGED
data/HISTORY.md
CHANGED
@@ -1,3 +1,22 @@
|
|
1
|
+
# Resque Scheduler History / ChangeLog / Release Notes
|
2
|
+
|
3
|
+
## 2.4.0 (2014-01-14)
|
4
|
+
|
5
|
+
* Including optional env name in procline
|
6
|
+
* Fixing an explosion regarding `every` in the views
|
7
|
+
* Bumping the copyright year
|
8
|
+
* Corrected doc for syntax of class and every keys
|
9
|
+
* Adding a standalone executable
|
10
|
+
* **POSSIBLE BREAKING CHANGE**: Dropping support for ree
|
11
|
+
* Add support for persistence of dynamic schedules
|
12
|
+
* Fix unsafe shutdown in Ruby 2
|
13
|
+
* Adding `.configure` convenience method for block-style configuration
|
14
|
+
* Add `.remove_delayed_selection` method to remove based on result of a block
|
15
|
+
* Add support for viewing all schedules for a job in web UI
|
16
|
+
* Use resque redis namespace in the master lock key
|
17
|
+
* Including optional app name in procline
|
18
|
+
* Various test improvements, :bug: fixes, and documentation updates!
|
19
|
+
|
1
20
|
## 2.3.1 (2013-11-20)
|
2
21
|
|
3
22
|
* Correcting `require_paths` in gemspec
|
@@ -93,6 +112,11 @@
|
|
93
112
|
* Dynamic schedule support (brianjlandau, davidyang)
|
94
113
|
* Now depends on redis >=1.3
|
95
114
|
|
115
|
+
## 1.9.11 (2013-11-20)
|
116
|
+
|
117
|
+
* Fixed behavior of `#validate_job!` via `#enqueue_at_with_queue` #286
|
118
|
+
* Correcting `require_paths` in gemspec #288
|
119
|
+
|
96
120
|
## 1.9.10 (2013-09-19)
|
97
121
|
|
98
122
|
* Backported `#enqueue_at_with_queue`
|
@@ -151,8 +175,8 @@
|
|
151
175
|
|
152
176
|
## 1.8.1 (2010-05-19)
|
153
177
|
|
154
|
-
* Adding rails_env for scheduled jobs to support scoping jobs by
|
155
|
-
RAILS_ENV (gravis).
|
178
|
+
* Adding `rails_env` for scheduled jobs to support scoping jobs by
|
179
|
+
`RAILS_ENV` (gravis).
|
156
180
|
* Fixing ruby 1.8.6 compatibility issue.
|
157
181
|
* Adding gemspec for bundler support.
|
158
182
|
|
data/LICENSE
CHANGED
data/README.md
CHANGED
@@ -33,8 +33,9 @@ Resque.enqueue_at(5.days.from_now, SomeJob) # run SomeJob at a specific time
|
|
33
33
|
|
34
34
|
### Documentation
|
35
35
|
|
36
|
-
This README covers what most people need to know. If you're looking
|
37
|
-
details on individual methods, you might want to try the
|
36
|
+
This `README` covers what most people need to know. If you're looking
|
37
|
+
for details on individual methods, you might want to try the
|
38
|
+
[rdoc](http://rdoc.info/github/resque/resque-scheduler/master/frames).
|
38
39
|
|
39
40
|
### Installation
|
40
41
|
|
@@ -45,7 +46,7 @@ To install:
|
|
45
46
|
If you use a Gemfile, you may want to specify the `:require` explicitly:
|
46
47
|
|
47
48
|
```ruby
|
48
|
-
gem 'resque-scheduler'
|
49
|
+
gem 'resque-scheduler'
|
49
50
|
```
|
50
51
|
|
51
52
|
Adding the resque:scheduler rake task:
|
@@ -71,7 +72,6 @@ namespace :resque do
|
|
71
72
|
task :setup do
|
72
73
|
require 'resque'
|
73
74
|
require 'resque_scheduler'
|
74
|
-
require 'resque/scheduler'
|
75
75
|
|
76
76
|
# you probably already have this somewhere
|
77
77
|
Resque.redis = 'localhost:6379'
|
@@ -105,6 +105,10 @@ never exits.
|
|
105
105
|
|
106
106
|
$ rake resque:scheduler
|
107
107
|
|
108
|
+
or, if you want to load the environment first:
|
109
|
+
|
110
|
+
$ rake environment resque:scheduler
|
111
|
+
|
108
112
|
Supported environment variables are `VERBOSE` and `MUTE`. If either is set to
|
109
113
|
any nonempty value, they will take effect. `VERBOSE` simply dumps more output
|
110
114
|
to stdout. `MUTE` does the opposite and silences all output. `MUTE`
|
@@ -112,7 +116,10 @@ supersedes `VERBOSE`.
|
|
112
116
|
|
113
117
|
### Resque Pool integration
|
114
118
|
|
115
|
-
For normal work with
|
119
|
+
For normal work with the
|
120
|
+
[resque-pool](https://github.com/nevans/resque-pool) gem, add the
|
121
|
+
following task to wherever tasks are kept, such as
|
122
|
+
`./lib/tasks/resque.rake`:
|
116
123
|
|
117
124
|
```ruby
|
118
125
|
task 'resque:pool:setup' do
|
@@ -150,7 +157,7 @@ standard resque jobs are persisted (redis writing to disk). Delayed jobs differ
|
|
150
157
|
from scheduled jobs in that if your scheduler process is down or workers are
|
151
158
|
down when a particular job is supposed to be queue, they will simply "catch up"
|
152
159
|
once they are started again. Jobs are guaranteed to run (provided they make it
|
153
|
-
into the delayed queue) after their given queue_at time has passed.
|
160
|
+
into the delayed queue) after their given `queue_at` time has passed.
|
154
161
|
|
155
162
|
One other thing to note is that insertion into the delayed queue is O(log(n))
|
156
163
|
since the jobs are stored in a redis sorted set (zset). I can't imagine this
|
@@ -168,15 +175,38 @@ Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :user_id => current_user.i
|
|
168
175
|
Resque.remove_delayed(SendFollowUpEmail, :user_id => current_user.id)
|
169
176
|
```
|
170
177
|
|
178
|
+
If you need to cancel a delayed job based on some matching arguments, but don't wish to specify each argument from when the job was created, you can do like so:
|
179
|
+
|
180
|
+
``` ruby
|
181
|
+
# after you've enqueued a job like:
|
182
|
+
Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :account_id => current_account.id, :user_id => current_user.id)
|
183
|
+
# remove jobs matching just the account:
|
184
|
+
Resque.remove_delayed_selection { |args| args[0]['account_id'] == current_account.id }
|
185
|
+
# or remove jobs matching just the user:
|
186
|
+
Resque.remove_delayed_selection { |args| args[0]['user_id'] == current_user.id }
|
187
|
+
```
|
188
|
+
|
171
189
|
### Scheduled Jobs (Recurring Jobs)
|
172
190
|
|
173
191
|
Scheduled (or recurring) jobs are logically no different than a standard cron
|
174
|
-
job. They are jobs that run based on a
|
175
|
-
|
192
|
+
job. They are jobs that run based on a schedule which can be static or dynamic.
|
193
|
+
|
194
|
+
#### Static schedules
|
176
195
|
|
177
|
-
|
196
|
+
Static schedules are set when `resque-scheduler` starts by passing a schedule file
|
197
|
+
to `resque-scheduler` initialization like this (see *Installation* above for a more complete example):
|
198
|
+
|
199
|
+
```ruby
|
200
|
+
Resque.schedule = YAML.load_file('your_resque_schedule.yml')
|
201
|
+
```
|
202
|
+
|
203
|
+
If a static schedule is not set `resque-scheduler` will issue a "Schedule empty!" warning on
|
204
|
+
startup, but despite that warning setting a static schedule is totally optional. It is possible
|
205
|
+
to use only dynamic schedules (see below).
|
206
|
+
|
207
|
+
The schedule file is a list of Resque job classes with arguments and a
|
178
208
|
schedule frequency (in crontab syntax). The schedule is just a hash, but
|
179
|
-
is
|
209
|
+
is usually stored in a YAML like this:
|
180
210
|
|
181
211
|
```yaml
|
182
212
|
CancelAbandonedOrders:
|
@@ -185,17 +215,17 @@ CancelAbandonedOrders:
|
|
185
215
|
queue_documents_for_indexing:
|
186
216
|
cron: "0 0 * * *"
|
187
217
|
# you can use rufus-scheduler "every" syntax in place of cron if you prefer
|
188
|
-
# every:
|
218
|
+
# every: 1h
|
189
219
|
# By default the job name (hash key) will be taken as worker class name.
|
190
220
|
# If you want to have a different job name and class name, provide the 'class' option
|
191
|
-
class: QueueDocuments
|
221
|
+
class: "QueueDocuments"
|
192
222
|
queue: high
|
193
223
|
args:
|
194
224
|
description: "This job queues all content for indexing in solr"
|
195
225
|
|
196
226
|
clear_leaderboards_contributors:
|
197
227
|
cron: "30 6 * * 1"
|
198
|
-
class: ClearLeaderboards
|
228
|
+
class: "ClearLeaderboards"
|
199
229
|
queue: low
|
200
230
|
args: contributors
|
201
231
|
description: "This job resets the weekly leaderboard for contributions"
|
@@ -213,8 +243,8 @@ You can provide options to "every" or "cron" via Array:
|
|
213
243
|
clear_leaderboards_moderator:
|
214
244
|
every:
|
215
245
|
- "30s"
|
216
|
-
- :first_in:
|
217
|
-
class: CheckDaemon
|
246
|
+
- :first_in: '120s'
|
247
|
+
class: "CheckDaemon"
|
218
248
|
queue: daemons
|
219
249
|
description: "This job will check Daemon every 30 seconds after 120 seconds after start"
|
220
250
|
```
|
@@ -232,6 +262,60 @@ seconds past the minute).
|
|
232
262
|
A big shout out to [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler)
|
233
263
|
for handling the heavy lifting of the actual scheduling engine.
|
234
264
|
|
265
|
+
#### Dynamic schedules
|
266
|
+
|
267
|
+
Dynamic schedules are programmatically set on a running `resque-scheduler`.
|
268
|
+
All [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler) options are supported
|
269
|
+
when setting schedules.
|
270
|
+
|
271
|
+
Dynamic schedules are not enabled by default. To be able to dynamically set schedules, you
|
272
|
+
must pass the following to `resque-scheduler` initialization (see *Installation* above for a more complete example):
|
273
|
+
|
274
|
+
```ruby
|
275
|
+
Resque::Scheduler.dynamic = true
|
276
|
+
```
|
277
|
+
|
278
|
+
Dynamic schedules allow for greater flexibility than static schedules as they can be set,
|
279
|
+
unset or changed without having to restart `resque-scheduler`. You can specify, if the schedule
|
280
|
+
must survive a resque-scheduler restart or not. This is done by setting the `persist` configuration
|
281
|
+
for the schedule: it is a boolean value, if set the schedule will persist a restart. By default,
|
282
|
+
a schedule will not be persisted.
|
283
|
+
|
284
|
+
The job to be scheduled must be a valid Resque job class.
|
285
|
+
|
286
|
+
For example, suppose you have a SendEmail job which sends emails. The `perform` method of the
|
287
|
+
job receives a string argument with the email subject. To run the SendEmail job every hour
|
288
|
+
starting five minutes from now, you can do:
|
289
|
+
|
290
|
+
```ruby
|
291
|
+
name = 'send_emails'
|
292
|
+
config = {}
|
293
|
+
config[:class] = 'SendEmail'
|
294
|
+
config[:args] = 'POC email subject'
|
295
|
+
config[:every] = ['1h', {first_in: 5.minutes}]
|
296
|
+
config[:persist] = true
|
297
|
+
Resque.set_schedule(name, config)
|
298
|
+
```
|
299
|
+
|
300
|
+
Schedules can later be removed by passing their name to the `remove_schedule` method:
|
301
|
+
|
302
|
+
```ruby
|
303
|
+
name = 'send_emails'
|
304
|
+
Resque.remove_schedule(name)
|
305
|
+
```
|
306
|
+
|
307
|
+
Schedule names are unique; i.e. two dynamic schedules cannot have the same name. If `set_schedule` is
|
308
|
+
passed the name of an existing schedule, that schedule is updated. E.g. if after setting the above schedule
|
309
|
+
we want the job to run every day instead of every hour from now on, we can do:
|
310
|
+
|
311
|
+
```ruby
|
312
|
+
name = 'send_emails'
|
313
|
+
config = {}
|
314
|
+
config[:class] = 'SendEmail'
|
315
|
+
config[:args] = 'POC email subject'
|
316
|
+
config[:every] = '1d'
|
317
|
+
Resque.set_schedule(name, config)
|
318
|
+
```
|
235
319
|
|
236
320
|
#### Time zones
|
237
321
|
|
@@ -295,7 +379,7 @@ And then a schedule:
|
|
295
379
|
create_fake_leaderboards:
|
296
380
|
cron: "30 6 * * 1"
|
297
381
|
queue: scoring
|
298
|
-
custom_job_class: FakeLeaderboard
|
382
|
+
custom_job_class: "FakeLeaderboard"
|
299
383
|
args:
|
300
384
|
rails_env: demo
|
301
385
|
description: "This job will auto-create leaderboards for our online demo and the status will update as the worker makes progress"
|
@@ -413,24 +497,29 @@ worker is started.
|
|
413
497
|
There are several options to toggle the way scheduler logs its actions. They
|
414
498
|
are toggled by environment variables:
|
415
499
|
|
416
|
-
- `MUTE` will stop logging anything. Completely silent
|
417
|
-
- `VERBOSE` opposite
|
418
|
-
- `LOGFILE` specifies the file to write logs to.
|
500
|
+
- `MUTE` will stop logging anything. Completely silent.
|
501
|
+
- `VERBOSE` opposite of 'mute'; will log even debug information
|
502
|
+
- `LOGFILE` specifies the file to write logs to. (default standard output)
|
503
|
+
- `LOGFORMAT` specifies either "text" or "json" output format
|
504
|
+
(default "text")
|
419
505
|
|
420
|
-
All
|
506
|
+
All of these variables are optional and will be given the following default
|
507
|
+
values:
|
421
508
|
|
422
509
|
```ruby
|
423
|
-
Resque::Scheduler.
|
424
|
-
|
425
|
-
|
510
|
+
Resque::Scheduler.configure do |c|
|
511
|
+
c.mute = false
|
512
|
+
c.verbose = false
|
513
|
+
c.logfile = nil # meaning all messages go to $stdout
|
514
|
+
c.logformat = 'text'
|
515
|
+
end
|
426
516
|
```
|
427
517
|
|
428
|
-
|
429
518
|
### Polling frequency
|
430
519
|
|
431
|
-
You can pass a `RESQUE_SCHEDULER_INTERVAL` option which is an integer or
|
432
|
-
representing the polling frequency. The default is 5 seconds, but
|
433
|
-
semi-active app you may want to use a smaller value.
|
520
|
+
You can pass a `RESQUE_SCHEDULER_INTERVAL` option which is an integer or
|
521
|
+
float representing the polling frequency. The default is 5 seconds, but
|
522
|
+
for a semi-active app you may want to use a smaller value.
|
434
523
|
|
435
524
|
$ RESQUE_SCHEDULER_INTERVAL=1 rake resque:scheduler
|
436
525
|
|
@@ -440,10 +529,10 @@ uses for its jobs.
|
|
440
529
|
|
441
530
|
### Plagiarism alert
|
442
531
|
|
443
|
-
This was intended to be an extension to resque and so resulted in a lot
|
444
|
-
code looking very similar to resque, particularly in resque-web
|
445
|
-
wanted it to be similar enough that someone familiar
|
446
|
-
work on resque-scheduler.
|
532
|
+
This was intended to be an extension to resque and so resulted in a lot
|
533
|
+
of the code looking very similar to resque, particularly in resque-web
|
534
|
+
and the views. I wanted it to be similar enough that someone familiar
|
535
|
+
with resque could easily work on resque-scheduler.
|
447
536
|
|
448
537
|
|
449
538
|
### Contributing
|
data/ROADMAP.md
ADDED
data/Rakefile
CHANGED
@@ -1,25 +1,13 @@
|
|
1
1
|
require 'bundler/gem_tasks'
|
2
|
+
require 'rake/testtask'
|
2
3
|
|
3
|
-
|
4
|
+
task default: [:rubocop, :test]
|
4
5
|
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
unless ENV['SEED']
|
11
|
-
srand
|
12
|
-
ENV['SEED'] = (srand % 0xFFFF).to_s
|
13
|
-
end
|
14
|
-
|
15
|
-
$stdout.puts "Running with SEED=#{ENV['SEED']}"
|
16
|
-
srand Integer(ENV['SEED'])
|
17
|
-
elsif ENV['SEED']
|
18
|
-
ARGV += %W(--seed #{ENV['SEED']})
|
19
|
-
end
|
20
|
-
Dir['test/*_test.rb'].each do |f|
|
21
|
-
require File.expand_path(f)
|
22
|
-
end
|
6
|
+
Rake::TestTask.new do |t|
|
7
|
+
t.libs << 'test'
|
8
|
+
t.pattern = ENV['PATTERN'] || 'test/*_test.rb'
|
9
|
+
t.verbose = !!ENV['VERBOSE']
|
10
|
+
t.options = "--seed #{ENV['SEED']}" if ENV['SEED']
|
23
11
|
end
|
24
12
|
|
25
13
|
desc 'Run rubocop'
|