resque-scheduler 2.3.1 → 2.4.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of resque-scheduler might be problematic. Click here for more details.

Files changed (49) hide show
  1. data/.gitignore +3 -0
  2. data/.rubocop.yml +11 -11
  3. data/.simplecov +1 -0
  4. data/.travis.yml +5 -2
  5. data/AUTHORS.md +3 -0
  6. data/HISTORY.md +26 -2
  7. data/LICENSE +1 -1
  8. data/README.md +120 -31
  9. data/ROADMAP.md +10 -0
  10. data/Rakefile +7 -19
  11. data/bin/resque-scheduler +5 -0
  12. data/examples/Rakefile +2 -0
  13. data/examples/config/initializers/resque-web.rb +37 -0
  14. data/examples/dynamic-scheduling/README.md +28 -0
  15. data/examples/dynamic-scheduling/app/jobs/fix_schedules_job.rb +54 -0
  16. data/examples/dynamic-scheduling/app/jobs/send_email_job.rb +9 -0
  17. data/examples/dynamic-scheduling/app/models/user.rb +16 -0
  18. data/examples/dynamic-scheduling/config/resque.yml +4 -0
  19. data/examples/dynamic-scheduling/config/static_schedule.yml +7 -0
  20. data/examples/dynamic-scheduling/lib/tasks/resque.rake +48 -0
  21. data/examples/run-resque-web +3 -0
  22. data/lib/resque-scheduler.rb +2 -0
  23. data/lib/resque/scheduler.rb +130 -41
  24. data/lib/resque/scheduler/lock/resilient.rb +1 -1
  25. data/lib/resque/scheduler_locking.rb +3 -1
  26. data/lib/resque_scheduler.rb +73 -31
  27. data/lib/resque_scheduler/cli.rb +160 -0
  28. data/lib/resque_scheduler/logger_builder.rb +27 -8
  29. data/lib/resque_scheduler/plugin.rb +10 -7
  30. data/lib/resque_scheduler/server.rb +52 -11
  31. data/lib/resque_scheduler/server/views/delayed.erb +2 -0
  32. data/lib/resque_scheduler/server/views/delayed_schedules.erb +20 -0
  33. data/lib/resque_scheduler/server/views/scheduler.erb +4 -12
  34. data/lib/resque_scheduler/tasks.rb +15 -27
  35. data/lib/resque_scheduler/version.rb +1 -1
  36. data/resque-scheduler.gemspec +2 -0
  37. data/test/cli_test.rb +286 -0
  38. data/test/delayed_queue_test.rb +70 -1
  39. data/test/resque-web_test.rb +36 -1
  40. data/test/scheduler_args_test.rb +51 -17
  41. data/test/scheduler_hooks_test.rb +1 -1
  42. data/test/scheduler_locking_test.rb +63 -1
  43. data/test/scheduler_setup_test.rb +54 -18
  44. data/test/scheduler_task_test.rb +35 -0
  45. data/test/scheduler_test.rb +130 -42
  46. data/test/support/redis_instance.rb +8 -3
  47. data/test/test_helper.rb +47 -20
  48. metadata +77 -6
  49. checksums.yaml +0 -15
data/.gitignore CHANGED
@@ -1,4 +1,6 @@
1
1
  .bundle/
2
+ .idea/
3
+
2
4
  doc/
3
5
  pkg
4
6
  nbproject
@@ -6,3 +8,4 @@ Gemfile.lock
6
8
  .rvmrc
7
9
  *.swp
8
10
 
11
+ /coverage/
@@ -2,6 +2,9 @@
2
2
  # The point is for the user to remove these configuration records
3
3
  # one by one as the offences are removed from the code base.
4
4
 
5
+ AccessorMethodName:
6
+ Enabled: false
7
+
5
8
  AlignParameters:
6
9
  Enabled: false
7
10
 
@@ -15,7 +18,7 @@ CaseEquality:
15
18
  Enabled: false
16
19
 
17
20
  ClassLength:
18
- Enabled: false
21
+ Max: 300
19
22
 
20
23
  ClassVars:
21
24
  Enabled: false
@@ -30,17 +33,11 @@ CommentAnnotation:
30
33
  Enabled: false
31
34
 
32
35
  CyclomaticComplexity:
33
- Enabled: false
36
+ Max: 18
34
37
 
35
38
  Documentation:
36
39
  Enabled: false
37
40
 
38
- EmptyLines:
39
- Enabled: false
40
-
41
- EmptyLinesAroundBody:
42
- Enabled: false
43
-
44
41
  Encoding:
45
42
  Enabled: false
46
43
 
@@ -59,9 +56,12 @@ HashSyntax:
59
56
  IfUnlessModifier:
60
57
  Enabled: false
61
58
 
62
- LineLength:
59
+ IndentationWidth:
63
60
  Enabled: false
64
61
 
62
+ LineLength:
63
+ Max: 152
64
+
65
65
  Loop:
66
66
  Enabled: false
67
67
 
@@ -69,7 +69,7 @@ MethodCallParentheses:
69
69
  Enabled: false
70
70
 
71
71
  MethodLength:
72
- Enabled: false
72
+ Max: 90
73
73
 
74
74
  ModuleFunction:
75
75
  Enabled: false
@@ -80,7 +80,7 @@ NumericLiterals:
80
80
  ParenthesesAroundCondition:
81
81
  Enabled: false
82
82
 
83
- PerlBackrefs:
83
+ PredicateName:
84
84
  Enabled: false
85
85
 
86
86
  RedundantBegin:
@@ -0,0 +1 @@
1
+ SimpleCov.start { add_filter '/test/' } if ENV['COVERAGE']
@@ -1,9 +1,12 @@
1
1
  language: ruby
2
2
  rvm:
3
- - ree
4
3
  - 1.9.3
5
4
  - 2.0.0
6
- script: bundle exec rake rubocop test
5
+ env:
6
+ global:
7
+ - RESQUE_SCHEDULER_DISABLE_TEST_REDIS_SERVER=1
8
+ services:
9
+ - redis-server
7
10
  notifications:
8
11
  email:
9
12
  recipients: daniel.buch+resque-scheduler@gmail.com
data/AUTHORS.md CHANGED
@@ -45,9 +45,12 @@ Resque Scheduler authors
45
45
  - Olivier Brisse
46
46
  - Petteri Räty
47
47
  - Phil Cohen
48
+ - Rob Olson
48
49
  - Russell Branca
49
50
  - Ryan Biesemeyer
50
51
  - Ryan Carver
52
+ - Sameer Siruguri
53
+ - Scott Francis
51
54
  - Sebastian Kippe
52
55
  - Spring MC
53
56
  - Tim Liner
data/HISTORY.md CHANGED
@@ -1,3 +1,22 @@
1
+ # Resque Scheduler History / ChangeLog / Release Notes
2
+
3
+ ## 2.4.0 (2014-01-14)
4
+
5
+ * Including optional env name in procline
6
+ * Fixing an explosion regarding `every` in the views
7
+ * Bumping the copyright year
8
+ * Corrected doc for syntax of class and every keys
9
+ * Adding a standalone executable
10
+ * **POSSIBLE BREAKING CHANGE**: Dropping support for ree
11
+ * Add support for persistence of dynamic schedules
12
+ * Fix unsafe shutdown in Ruby 2
13
+ * Adding `.configure` convenience method for block-style configuration
14
+ * Add `.remove_delayed_selection` method to remove based on result of a block
15
+ * Add support for viewing all schedules for a job in web UI
16
+ * Use resque redis namespace in the master lock key
17
+ * Including optional app name in procline
18
+ * Various test improvements, :bug: fixes, and documentation updates!
19
+
1
20
  ## 2.3.1 (2013-11-20)
2
21
 
3
22
  * Correcting `require_paths` in gemspec
@@ -93,6 +112,11 @@
93
112
  * Dynamic schedule support (brianjlandau, davidyang)
94
113
  * Now depends on redis >=1.3
95
114
 
115
+ ## 1.9.11 (2013-11-20)
116
+
117
+ * Fixed behavior of `#validate_job!` via `#enqueue_at_with_queue` #286
118
+ * Correcting `require_paths` in gemspec #288
119
+
96
120
  ## 1.9.10 (2013-09-19)
97
121
 
98
122
  * Backported `#enqueue_at_with_queue`
@@ -151,8 +175,8 @@
151
175
 
152
176
  ## 1.8.1 (2010-05-19)
153
177
 
154
- * Adding rails_env for scheduled jobs to support scoping jobs by
155
- RAILS_ENV (gravis).
178
+ * Adding `rails_env` for scheduled jobs to support scoping jobs by
179
+ `RAILS_ENV` (gravis).
156
180
  * Fixing ruby 1.8.6 compatibility issue.
157
181
  * Adding gemspec for bundler support.
158
182
 
data/LICENSE CHANGED
@@ -1,4 +1,4 @@
1
- Copyright (c) 2013 Ben VandenBos
1
+ Copyright (c) 2014 Ben VandenBos
2
2
 
3
3
  MIT License
4
4
 
data/README.md CHANGED
@@ -33,8 +33,9 @@ Resque.enqueue_at(5.days.from_now, SomeJob) # run SomeJob at a specific time
33
33
 
34
34
  ### Documentation
35
35
 
36
- This README covers what most people need to know. If you're looking for
37
- details on individual methods, you might want to try the [rdoc](http://rdoc.info/github/bvandenbos/resque-scheduler/master/frames).
36
+ This `README` covers what most people need to know. If you're looking
37
+ for details on individual methods, you might want to try the
38
+ [rdoc](http://rdoc.info/github/resque/resque-scheduler/master/frames).
38
39
 
39
40
  ### Installation
40
41
 
@@ -45,7 +46,7 @@ To install:
45
46
  If you use a Gemfile, you may want to specify the `:require` explicitly:
46
47
 
47
48
  ```ruby
48
- gem 'resque-scheduler', :require => 'resque_scheduler'
49
+ gem 'resque-scheduler'
49
50
  ```
50
51
 
51
52
  Adding the resque:scheduler rake task:
@@ -71,7 +72,6 @@ namespace :resque do
71
72
  task :setup do
72
73
  require 'resque'
73
74
  require 'resque_scheduler'
74
- require 'resque/scheduler'
75
75
 
76
76
  # you probably already have this somewhere
77
77
  Resque.redis = 'localhost:6379'
@@ -105,6 +105,10 @@ never exits.
105
105
 
106
106
  $ rake resque:scheduler
107
107
 
108
+ or, if you want to load the environment first:
109
+
110
+ $ rake environment resque:scheduler
111
+
108
112
  Supported environment variables are `VERBOSE` and `MUTE`. If either is set to
109
113
  any nonempty value, they will take effect. `VERBOSE` simply dumps more output
110
114
  to stdout. `MUTE` does the opposite and silences all output. `MUTE`
@@ -112,7 +116,10 @@ supersedes `VERBOSE`.
112
116
 
113
117
  ### Resque Pool integration
114
118
 
115
- For normal work with [resque-pool](https://github.com/nevans/resque-pool) gem add next lines in lib/rake/resque.rake
119
+ For normal work with the
120
+ [resque-pool](https://github.com/nevans/resque-pool) gem, add the
121
+ following task to wherever tasks are kept, such as
122
+ `./lib/tasks/resque.rake`:
116
123
 
117
124
  ```ruby
118
125
  task 'resque:pool:setup' do
@@ -150,7 +157,7 @@ standard resque jobs are persisted (redis writing to disk). Delayed jobs differ
150
157
  from scheduled jobs in that if your scheduler process is down or workers are
151
158
  down when a particular job is supposed to be queue, they will simply "catch up"
152
159
  once they are started again. Jobs are guaranteed to run (provided they make it
153
- into the delayed queue) after their given queue_at time has passed.
160
+ into the delayed queue) after their given `queue_at` time has passed.
154
161
 
155
162
  One other thing to note is that insertion into the delayed queue is O(log(n))
156
163
  since the jobs are stored in a redis sorted set (zset). I can't imagine this
@@ -168,15 +175,38 @@ Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :user_id => current_user.i
168
175
  Resque.remove_delayed(SendFollowUpEmail, :user_id => current_user.id)
169
176
  ```
170
177
 
178
+ If you need to cancel a delayed job based on some matching arguments, but don't wish to specify each argument from when the job was created, you can do like so:
179
+
180
+ ``` ruby
181
+ # after you've enqueued a job like:
182
+ Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :account_id => current_account.id, :user_id => current_user.id)
183
+ # remove jobs matching just the account:
184
+ Resque.remove_delayed_selection { |args| args[0]['account_id'] == current_account.id }
185
+ # or remove jobs matching just the user:
186
+ Resque.remove_delayed_selection { |args| args[0]['user_id'] == current_user.id }
187
+ ```
188
+
171
189
  ### Scheduled Jobs (Recurring Jobs)
172
190
 
173
191
  Scheduled (or recurring) jobs are logically no different than a standard cron
174
- job. They are jobs that run based on a fixed schedule which is set at
175
- startup.
192
+ job. They are jobs that run based on a schedule which can be static or dynamic.
193
+
194
+ #### Static schedules
176
195
 
177
- The schedule is a list of Resque worker classes with arguments and a
196
+ Static schedules are set when `resque-scheduler` starts by passing a schedule file
197
+ to `resque-scheduler` initialization like this (see *Installation* above for a more complete example):
198
+
199
+ ```ruby
200
+ Resque.schedule = YAML.load_file('your_resque_schedule.yml')
201
+ ```
202
+
203
+ If a static schedule is not set `resque-scheduler` will issue a "Schedule empty!" warning on
204
+ startup, but despite that warning setting a static schedule is totally optional. It is possible
205
+ to use only dynamic schedules (see below).
206
+
207
+ The schedule file is a list of Resque job classes with arguments and a
178
208
  schedule frequency (in crontab syntax). The schedule is just a hash, but
179
- is most likely stored in a YAML like so:
209
+ is usually stored in a YAML like this:
180
210
 
181
211
  ```yaml
182
212
  CancelAbandonedOrders:
@@ -185,17 +215,17 @@ CancelAbandonedOrders:
185
215
  queue_documents_for_indexing:
186
216
  cron: "0 0 * * *"
187
217
  # you can use rufus-scheduler "every" syntax in place of cron if you prefer
188
- # every: 1hr
218
+ # every: 1h
189
219
  # By default the job name (hash key) will be taken as worker class name.
190
220
  # If you want to have a different job name and class name, provide the 'class' option
191
- class: QueueDocuments
221
+ class: "QueueDocuments"
192
222
  queue: high
193
223
  args:
194
224
  description: "This job queues all content for indexing in solr"
195
225
 
196
226
  clear_leaderboards_contributors:
197
227
  cron: "30 6 * * 1"
198
- class: ClearLeaderboards
228
+ class: "ClearLeaderboards"
199
229
  queue: low
200
230
  args: contributors
201
231
  description: "This job resets the weekly leaderboard for contributions"
@@ -213,8 +243,8 @@ You can provide options to "every" or "cron" via Array:
213
243
  clear_leaderboards_moderator:
214
244
  every:
215
245
  - "30s"
216
- - :first_in: "120s"
217
- class: CheckDaemon
246
+ - :first_in: '120s'
247
+ class: "CheckDaemon"
218
248
  queue: daemons
219
249
  description: "This job will check Daemon every 30 seconds after 120 seconds after start"
220
250
  ```
@@ -232,6 +262,60 @@ seconds past the minute).
232
262
  A big shout out to [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler)
233
263
  for handling the heavy lifting of the actual scheduling engine.
234
264
 
265
+ #### Dynamic schedules
266
+
267
+ Dynamic schedules are programmatically set on a running `resque-scheduler`.
268
+ All [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler) options are supported
269
+ when setting schedules.
270
+
271
+ Dynamic schedules are not enabled by default. To be able to dynamically set schedules, you
272
+ must pass the following to `resque-scheduler` initialization (see *Installation* above for a more complete example):
273
+
274
+ ```ruby
275
+ Resque::Scheduler.dynamic = true
276
+ ```
277
+
278
+ Dynamic schedules allow for greater flexibility than static schedules as they can be set,
279
+ unset or changed without having to restart `resque-scheduler`. You can specify, if the schedule
280
+ must survive a resque-scheduler restart or not. This is done by setting the `persist` configuration
281
+ for the schedule: it is a boolean value, if set the schedule will persist a restart. By default,
282
+ a schedule will not be persisted.
283
+
284
+ The job to be scheduled must be a valid Resque job class.
285
+
286
+ For example, suppose you have a SendEmail job which sends emails. The `perform` method of the
287
+ job receives a string argument with the email subject. To run the SendEmail job every hour
288
+ starting five minutes from now, you can do:
289
+
290
+ ```ruby
291
+ name = 'send_emails'
292
+ config = {}
293
+ config[:class] = 'SendEmail'
294
+ config[:args] = 'POC email subject'
295
+ config[:every] = ['1h', {first_in: 5.minutes}]
296
+ config[:persist] = true
297
+ Resque.set_schedule(name, config)
298
+ ```
299
+
300
+ Schedules can later be removed by passing their name to the `remove_schedule` method:
301
+
302
+ ```ruby
303
+ name = 'send_emails'
304
+ Resque.remove_schedule(name)
305
+ ```
306
+
307
+ Schedule names are unique; i.e. two dynamic schedules cannot have the same name. If `set_schedule` is
308
+ passed the name of an existing schedule, that schedule is updated. E.g. if after setting the above schedule
309
+ we want the job to run every day instead of every hour from now on, we can do:
310
+
311
+ ```ruby
312
+ name = 'send_emails'
313
+ config = {}
314
+ config[:class] = 'SendEmail'
315
+ config[:args] = 'POC email subject'
316
+ config[:every] = '1d'
317
+ Resque.set_schedule(name, config)
318
+ ```
235
319
 
236
320
  #### Time zones
237
321
 
@@ -295,7 +379,7 @@ And then a schedule:
295
379
  create_fake_leaderboards:
296
380
  cron: "30 6 * * 1"
297
381
  queue: scoring
298
- custom_job_class: FakeLeaderboard
382
+ custom_job_class: "FakeLeaderboard"
299
383
  args:
300
384
  rails_env: demo
301
385
  description: "This job will auto-create leaderboards for our online demo and the status will update as the worker makes progress"
@@ -413,24 +497,29 @@ worker is started.
413
497
  There are several options to toggle the way scheduler logs its actions. They
414
498
  are toggled by environment variables:
415
499
 
416
- - `MUTE` will stop logging anything. Completely silent;
417
- - `VERBOSE` opposite to 'mute' will log even debug information;
418
- - `LOGFILE` specifies the file to write logs to. Default is standard output.
500
+ - `MUTE` will stop logging anything. Completely silent.
501
+ - `VERBOSE` opposite of 'mute'; will log even debug information
502
+ - `LOGFILE` specifies the file to write logs to. (default standard output)
503
+ - `LOGFORMAT` specifies either "text" or "json" output format
504
+ (default "text")
419
505
 
420
- All those variables are non-mandatory and default values are
506
+ All of these variables are optional and will be given the following default
507
+ values:
421
508
 
422
509
  ```ruby
423
- Resque::Scheduler.mute = false
424
- Resque::Scheduler.verbose = false
425
- Resque::Scheduler.logfile = nil # that means, all messages go to STDOUT
510
+ Resque::Scheduler.configure do |c|
511
+ c.mute = false
512
+ c.verbose = false
513
+ c.logfile = nil # meaning all messages go to $stdout
514
+ c.logformat = 'text'
515
+ end
426
516
  ```
427
517
 
428
-
429
518
  ### Polling frequency
430
519
 
431
- You can pass a `RESQUE_SCHEDULER_INTERVAL` option which is an integer or float
432
- representing the polling frequency. The default is 5 seconds, but for a
433
- semi-active app you may want to use a smaller value.
520
+ You can pass a `RESQUE_SCHEDULER_INTERVAL` option which is an integer or
521
+ float representing the polling frequency. The default is 5 seconds, but
522
+ for a semi-active app you may want to use a smaller value.
434
523
 
435
524
  $ RESQUE_SCHEDULER_INTERVAL=1 rake resque:scheduler
436
525
 
@@ -440,10 +529,10 @@ uses for its jobs.
440
529
 
441
530
  ### Plagiarism alert
442
531
 
443
- This was intended to be an extension to resque and so resulted in a lot of the
444
- code looking very similar to resque, particularly in resque-web and the views. I
445
- wanted it to be similar enough that someone familiar with resque could easily
446
- work on resque-scheduler.
532
+ This was intended to be an extension to resque and so resulted in a lot
533
+ of the code looking very similar to resque, particularly in resque-web
534
+ and the views. I wanted it to be similar enough that someone familiar
535
+ with resque could easily work on resque-scheduler.
447
536
 
448
537
 
449
538
  ### Contributing
@@ -0,0 +1,10 @@
1
+ TODO for v3.0.0
2
+ ===============
3
+
4
+ - Clean up all RuboCop offences, breaking the API if necessary
5
+ - Get to at least 95% test coverage
6
+
7
+ TODO for v3.1.0
8
+ ===============
9
+
10
+ - Compatibility with Resque 2
data/Rakefile CHANGED
@@ -1,25 +1,13 @@
1
1
  require 'bundler/gem_tasks'
2
+ require 'rake/testtask'
2
3
 
3
- $LOAD_PATH.unshift 'lib'
4
+ task default: [:rubocop, :test]
4
5
 
5
- task :default => :test
6
-
7
- desc 'Run tests'
8
- task :test do
9
- if RUBY_VERSION =~ /^1\.8/
10
- unless ENV['SEED']
11
- srand
12
- ENV['SEED'] = (srand % 0xFFFF).to_s
13
- end
14
-
15
- $stdout.puts "Running with SEED=#{ENV['SEED']}"
16
- srand Integer(ENV['SEED'])
17
- elsif ENV['SEED']
18
- ARGV += %W(--seed #{ENV['SEED']})
19
- end
20
- Dir['test/*_test.rb'].each do |f|
21
- require File.expand_path(f)
22
- end
6
+ Rake::TestTask.new do |t|
7
+ t.libs << 'test'
8
+ t.pattern = ENV['PATTERN'] || 'test/*_test.rb'
9
+ t.verbose = !!ENV['VERBOSE']
10
+ t.options = "--seed #{ENV['SEED']}" if ENV['SEED']
23
11
  end
24
12
 
25
13
  desc 'Run rubocop'