resque-scheduler 2.2.0 → 4.10.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (68) hide show
  1. checksums.yaml +7 -0
  2. data/.github/dependabot.yml +12 -0
  3. data/.github/funding.yml +4 -0
  4. data/.github/workflows/codeql-analysis.yml +59 -0
  5. data/.github/workflows/rubocop.yml +27 -0
  6. data/.github/workflows/ruby.yml +81 -0
  7. data/AUTHORS.md +31 -0
  8. data/CHANGELOG.md +539 -0
  9. data/CODE_OF_CONDUCT.md +74 -0
  10. data/Gemfile +26 -2
  11. data/LICENSE +1 -1
  12. data/README.md +423 -91
  13. data/Rakefile +12 -35
  14. data/exe/resque-scheduler +5 -0
  15. data/lib/resque/scheduler/cli.rb +147 -0
  16. data/lib/resque/scheduler/configuration.rb +102 -0
  17. data/lib/resque/scheduler/delaying_extensions.rb +371 -0
  18. data/lib/resque/scheduler/env.rb +85 -0
  19. data/lib/resque/scheduler/extension.rb +13 -0
  20. data/lib/resque/scheduler/failure_handler.rb +11 -0
  21. data/lib/resque/scheduler/lock/base.rb +12 -3
  22. data/lib/resque/scheduler/lock/basic.rb +4 -5
  23. data/lib/resque/scheduler/lock/resilient.rb +52 -43
  24. data/lib/resque/scheduler/lock.rb +2 -1
  25. data/lib/resque/scheduler/locking.rb +104 -0
  26. data/lib/resque/scheduler/logger_builder.rb +83 -0
  27. data/lib/resque/scheduler/plugin.rb +31 -0
  28. data/lib/resque/scheduler/scheduling_extensions.rb +142 -0
  29. data/lib/{resque_scheduler → resque/scheduler}/server/views/delayed.erb +23 -8
  30. data/lib/resque/scheduler/server/views/delayed_schedules.erb +20 -0
  31. data/lib/{resque_scheduler → resque/scheduler}/server/views/delayed_timestamp.erb +1 -1
  32. data/lib/resque/scheduler/server/views/scheduler.erb +58 -0
  33. data/lib/resque/scheduler/server/views/search.erb +69 -0
  34. data/lib/resque/scheduler/server/views/search_form.erb +4 -0
  35. data/lib/resque/scheduler/server.rb +268 -0
  36. data/lib/resque/scheduler/signal_handling.rb +40 -0
  37. data/lib/resque/scheduler/tasks.rb +25 -0
  38. data/lib/resque/scheduler/util.rb +39 -0
  39. data/lib/resque/scheduler/version.rb +7 -0
  40. data/lib/resque/scheduler.rb +333 -149
  41. data/lib/resque-scheduler.rb +4 -0
  42. data/resque-scheduler.gemspec +56 -20
  43. metadata +205 -104
  44. data/.gitignore +0 -8
  45. data/.rubocop.yml +0 -120
  46. data/.travis.yml +0 -10
  47. data/HISTORY.md +0 -180
  48. data/lib/resque/scheduler_locking.rb +0 -90
  49. data/lib/resque_scheduler/logger_builder.rb +0 -51
  50. data/lib/resque_scheduler/plugin.rb +0 -25
  51. data/lib/resque_scheduler/server/views/scheduler.erb +0 -44
  52. data/lib/resque_scheduler/server.rb +0 -92
  53. data/lib/resque_scheduler/tasks.rb +0 -40
  54. data/lib/resque_scheduler/version.rb +0 -3
  55. data/lib/resque_scheduler.rb +0 -355
  56. data/script/migrate_to_timestamps_set.rb +0 -14
  57. data/tasks/resque_scheduler.rake +0 -2
  58. data/test/delayed_queue_test.rb +0 -383
  59. data/test/redis-test.conf +0 -108
  60. data/test/resque-web_test.rb +0 -116
  61. data/test/scheduler_args_test.rb +0 -156
  62. data/test/scheduler_hooks_test.rb +0 -23
  63. data/test/scheduler_locking_test.rb +0 -180
  64. data/test/scheduler_setup_test.rb +0 -59
  65. data/test/scheduler_test.rb +0 -256
  66. data/test/support/redis_instance.rb +0 -129
  67. data/test/test_helper.rb +0 -92
  68. /data/lib/{resque_scheduler → resque/scheduler}/server/views/requeue-params.erb +0 -0
data/README.md CHANGED
@@ -1,24 +1,17 @@
1
1
  resque-scheduler
2
2
  ================
3
3
 
4
- [![Dependency Status](https://gemnasium.com/resque/resque-scheduler.png)](https://gemnasium.com/resque/resque-scheduler)
5
- [![Gem Version](https://badge.fury.io/rb/resque-scheduler.png)](http://badge.fury.io/rb/resque-scheduler)
6
- [![Build Status](https://travis-ci.org/resque/resque-scheduler.png?branch=master)](https://travis-ci.org/resque/resque-scheduler)
4
+
5
+ [![Gem Version](https://badge.fury.io/rb/resque-scheduler.svg)](https://badge.fury.io/rb/resque-scheduler)
6
+ [![Ruby specs](https://github.com/resque/resque-scheduler/actions/workflows/ruby.yml/badge.svg)](https://github.com/resque/resque-scheduler/actions)
7
+ [![Code Climate](https://codeclimate.com/github/resque/resque-scheduler/badges/gpa.svg)](https://codeclimate.com/github/resque/resque-scheduler)
7
8
 
8
9
  ### Description
9
10
 
10
11
  Resque-scheduler is an extension to [Resque](http://github.com/resque/resque)
11
12
  that adds support for queueing items in the future.
12
13
 
13
- This table explains the version requirements for redis
14
-
15
- | resque-scheduler version | required redis version|
16
- |:-------------------------|----------------------:|
17
- | >= 2.0.0 | >= 2.2.0 |
18
- | >= 0.0.1 | >= 1.3 |
19
-
20
-
21
- Job scheduling is supported in two different way: Recurring (scheduled) and
14
+ Job scheduling is supported in two different ways: Recurring (scheduled) and
22
15
  Delayed.
23
16
 
24
17
  Scheduled jobs are like cron jobs, recurring on a regular basis. Delayed
@@ -26,15 +19,16 @@ jobs are resque jobs that you want to run at some point in the future.
26
19
  The syntax is pretty explanatory:
27
20
 
28
21
  ```ruby
29
- Resque.enqueue_in(5.days, SendFollowupEmail) # run a job in 5 days
22
+ Resque.enqueue_in(5.days, SendFollowupEmail, argument) # runs a job in 5 days, calling SendFollowupEmail.perform(argument)
30
23
  # or
31
- Resque.enqueue_at(5.days.from_now, SomeJob) # run SomeJob at a specific time
24
+ Resque.enqueue_at(5.days.from_now, SomeJob, argument) # runs a job at a specific time, calling SomeJob.perform(argument)
32
25
  ```
33
26
 
34
27
  ### Documentation
35
28
 
36
- This README covers what most people need to know. If you're looking for
37
- details on individual methods, you might want to try the [rdoc](http://rdoc.info/github/bvandenbos/resque-scheduler/master/frames).
29
+ This `README` covers what most people need to know. If you're looking
30
+ for details on individual methods, you might want to try the
31
+ [rdoc](http://rdoc.info/github/resque/resque-scheduler/master/frames).
38
32
 
39
33
  ### Installation
40
34
 
@@ -42,39 +36,40 @@ To install:
42
36
 
43
37
  gem install resque-scheduler
44
38
 
45
- If you use a Gemfile, you may want to specify the `:require` explicitly:
39
+ If you use a Gemfile:
46
40
 
47
41
  ```ruby
48
- gem 'resque-scheduler', :require => 'resque_scheduler'
42
+ gem 'resque-scheduler'
49
43
  ```
50
44
 
51
45
  Adding the resque:scheduler rake task:
52
46
 
53
47
  ```ruby
54
- require 'resque_scheduler/tasks'
48
+ require 'resque/scheduler/tasks'
55
49
  ```
56
50
 
57
- There are three things `resque-scheduler` needs to know about in order to do
58
- it's jobs: the schedule, where redis lives, and which queues to use. The
59
- easiest way to configure these things is via the rake task. By default,
60
- `resque-scheduler` depends on the "resque:setup" rake task. Since you
61
- probably already have this task, lets just put our configuration there.
62
- `resque-scheduler` pretty much needs to know everything `resque` needs
63
- to know.
51
+ ### Rake integration
52
+
53
+ By default, `resque-scheduler` depends on the "resque:setup" rake task.
54
+ Since you probably already have this task, lets just put our
55
+ configuration there. `resque-scheduler` pretty much needs to know
56
+ everything `resque` needs to know.
64
57
 
65
58
  ```ruby
66
59
  # Resque tasks
67
60
  require 'resque/tasks'
68
- require 'resque_scheduler/tasks'
61
+ require 'resque/scheduler/tasks'
69
62
 
70
63
  namespace :resque do
71
64
  task :setup do
72
65
  require 'resque'
73
- require 'resque_scheduler'
74
- require 'resque/scheduler'
75
66
 
76
67
  # you probably already have this somewhere
77
68
  Resque.redis = 'localhost:6379'
69
+ end
70
+
71
+ task :setup_schedule => :setup do
72
+ require 'resque-scheduler'
78
73
 
79
74
  # If you want to be able to dynamically change the schedule,
80
75
  # uncomment this line. A dynamic schedule can be updated via the
@@ -82,7 +77,7 @@ namespace :resque do
82
77
  # When dynamic is set to true, the scheduler process looks for
83
78
  # schedule changes and applies them on the fly.
84
79
  # Note: This feature is only available in >=2.0.0.
85
- #Resque::Scheduler.dynamic = true
80
+ # Resque::Scheduler.dynamic = true
86
81
 
87
82
  # The schedule doesn't need to be stored in a YAML, it just needs to
88
83
  # be a hash. YAML is usually the easiest.
@@ -95,20 +90,78 @@ namespace :resque do
95
90
  # So, something like this:
96
91
  require 'jobs'
97
92
  end
93
+
94
+ task :scheduler => :setup_schedule
98
95
  end
99
96
  ```
100
97
 
101
- The scheduler process is just a rake task which is responsible for both
102
- queueing items from the schedule and polling the delayed queue for items
103
- ready to be pushed on to the work queues. For obvious reasons, this process
104
- never exits.
98
+ The scheduler rake task is responsible for both queueing items from the
99
+ schedule and polling the delayed queue for items ready to be pushed on
100
+ to the work queues. For obvious reasons, this process never exits.
101
+
102
+ ``` bash
103
+ rake resque:scheduler
104
+ ```
105
+
106
+ or, if you want to load the environment first:
105
107
 
106
- $ rake resque:scheduler
108
+ ``` bash
109
+ rake environment resque:scheduler
110
+ ```
107
111
 
108
- Supported environment variables are `VERBOSE` and `MUTE`. If either is set to
109
- any nonempty value, they will take effect. `VERBOSE` simply dumps more output
110
- to stdout. `MUTE` does the opposite and silences all output. `MUTE`
111
- supersedes `VERBOSE`.
112
+
113
+ ### Standalone Executable
114
+
115
+ The scheduler may also be run via a standalone `resque-scheduler`
116
+ executable, which will be available once the gem is installed.
117
+
118
+ ``` bash
119
+ # Get some help
120
+ resque-scheduler --help
121
+ ```
122
+
123
+ The executable accepts options via option flags as well as via
124
+ [environment variables](#environment-variables).
125
+
126
+ ### Environment Variables
127
+
128
+ Both the Rake task and standalone executable support the following
129
+ environment variables:
130
+
131
+ * `APP_NAME` - Application name used in procline (`$0`) (default empty)
132
+ * `BACKGROUND` - [Run in the background](#running-in-the-background) if
133
+ non-empty (via `Process.daemon`, if supported) (default `false`)
134
+ * `DYNAMIC_SCHEDULE` - Enables [dynamic scheduling](#dynamic-schedules)
135
+ if non-empty (default `false`)
136
+ * `RAILS_ENV` - Environment to use in procline (`$0`) (default empty)
137
+ * `INITIALIZER_PATH` - Path to a Ruby file that will be loaded *before*
138
+ requiring `resque` and `resque/scheduler` (default empty).
139
+ * `RESQUE_SCHEDULER_INTERVAL` - Interval in seconds for checking if a
140
+ scheduled job must run (coerced with `Kernel#Float()`) (default `5`)
141
+ * `LOGFILE` - Log file name (default empty, meaning `$stdout`)
142
+ * `LOGFORMAT` - Log output format to use (either `'text'`, `'json'` or `'logfmt'`,
143
+ default `'text'`)
144
+ * `PIDFILE` - If non-empty, write process PID to file (default empty)
145
+ * `QUIET` - Silence most output if non-empty (equivalent to a level of
146
+ `MonoLogger::FATAL`, default `false`)
147
+ * `VERBOSE` - Maximize log verbosity if non-empty (equivalent to a level
148
+ of `MonoLogger::DEBUG`, default `false`)
149
+
150
+
151
+ ### Resque Pool integration
152
+
153
+ For normal work with the
154
+ [resque-pool](https://github.com/nevans/resque-pool) gem, add the
155
+ following task to wherever tasks are kept, such as
156
+ `./lib/tasks/resque.rake`:
157
+
158
+ ```ruby
159
+ task 'resque:pool:setup' do
160
+ Resque::Pool.after_prefork do |job|
161
+ Resque.redis.reconnect
162
+ end
163
+ end
164
+ ```
112
165
 
113
166
 
114
167
  ### Delayed jobs
@@ -117,7 +170,11 @@ Delayed jobs are one-off jobs that you want to be put into a queue at some point
117
170
  in the future. The classic example is sending email:
118
171
 
119
172
  ```ruby
120
- Resque.enqueue_in(5.days, SendFollowUpEmail, :user_id => current_user.id)
173
+ Resque.enqueue_in(
174
+ 5.days,
175
+ SendFollowUpEmail,
176
+ user_id: current_user.id
177
+ )
121
178
  ```
122
179
 
123
180
  This will store the job for 5 days in the resque delayed queue at which time
@@ -125,27 +182,36 @@ the scheduler process will pull it from the delayed queue and put it in the
125
182
  appropriate work queue for the given job and it will be processed as soon as
126
183
  a worker is available (just like any other resque job).
127
184
 
128
- NOTE: The job does not fire **exactly** at the time supplied. Rather, once that
185
+ **NOTE**: The job does not fire **exactly** at the time supplied. Rather, once that
129
186
  time is in the past, the job moves from the delayed queue to the actual resque
130
- work queue and will be completed as workers as free to process it.
187
+ work queue and will be completed as workers are free to process it.
131
188
 
132
189
  Also supported is `Resque.enqueue_at` which takes a timestamp to queue the
133
190
  job, and `Resque.enqueue_at_with_queue` which takes both a timestamp and a
134
- queue name.
191
+ queue name:
192
+
193
+ ```ruby
194
+ Resque.enqueue_at_with_queue(
195
+ 'queue_name',
196
+ 5.days.from_now,
197
+ SendFollowUpEmail,
198
+ user_id: current_user.id
199
+ )
200
+ ```
135
201
 
136
202
  The delayed queue is stored in redis and is persisted in the same way the
137
203
  standard resque jobs are persisted (redis writing to disk). Delayed jobs differ
138
204
  from scheduled jobs in that if your scheduler process is down or workers are
139
205
  down when a particular job is supposed to be queue, they will simply "catch up"
140
206
  once they are started again. Jobs are guaranteed to run (provided they make it
141
- into the delayed queue) after their given queue_at time has passed.
207
+ into the delayed queue) after their given `queue_at` time has passed.
142
208
 
143
209
  One other thing to note is that insertion into the delayed queue is O(log(n))
144
210
  since the jobs are stored in a redis sorted set (zset). I can't imagine this
145
211
  being an issue for someone since redis is stupidly fast even at log(n), but full
146
212
  disclosure is always best.
147
213
 
148
- #### Removing Delayed jobs
214
+ #### Removing Delayed Jobs
149
215
 
150
216
  If you have the need to cancel a delayed job, you can do like so:
151
217
 
@@ -156,15 +222,103 @@ Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :user_id => current_user.i
156
222
  Resque.remove_delayed(SendFollowUpEmail, :user_id => current_user.id)
157
223
  ```
158
224
 
225
+ If you need to cancel a delayed job based on some matching arguments, but don't wish to specify each argument from when the job was created, you can do like so:
226
+
227
+ ``` ruby
228
+ # after you've enqueued a job like:
229
+ Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :account_id => current_account.id, :user_id => current_user.id)
230
+ # remove jobs matching just the account:
231
+ Resque.remove_delayed_selection { |args| args[0]['account_id'] == current_account.id }
232
+ # or remove jobs matching just the user:
233
+ Resque.remove_delayed_selection { |args| args[0]['user_id'] == current_user.id }
234
+ ```
235
+
236
+ If you need to cancel a delayed job based on some matching arguments AND by which class the job is, but don't wish to specify each argument from when the job was created, you can do like so:
237
+
238
+ ``` ruby
239
+ # after you've enqueued a job like:
240
+ Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :account_id => current_account.id, :user_id => current_user.id)
241
+ # remove jobs matching just the account and that were of the class SendFollowUpEmail:
242
+ Resque.remove_delayed_selection(SendFollowUpEmail) { |args| args[0]['account_id'] == current_account.id }
243
+ # or remove jobs matching just the user and that were of the class SendFollowUpEmail:
244
+ Resque.remove_delayed_selection(SendFollowUpEmail) { |args| args[0]['user_id'] == current_user.id }
245
+ ```
246
+
247
+ If you need to enqueue immediately a delayed job based on some matching arguments, but don't wish to specify each argument from when the job was created, you can do like so:
248
+
249
+ ``` ruby
250
+ # after you've enqueued a job like:
251
+ Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :account_id => current_account.id, :user_id => current_user.id)
252
+ # enqueue immediately jobs matching just the account:
253
+ Resque.enqueue_delayed_selection { |args| args[0]['account_id'] == current_account.id }
254
+ # or enqueue immediately jobs matching just the user:
255
+ Resque.enqueue_delayed_selection { |args| args[0]['user_id'] == current_user.id }
256
+ ```
257
+
258
+ #### Updating Delayed Jobs
259
+
260
+ Previously delayed jobs may be delayed even further into the future like so:
261
+
262
+ ```ruby
263
+ # after you've enqueued a job like:
264
+ Resque.enqueue_at(1.minute.from_now, SendNotifications, :user_id => current_user.id)
265
+ # delay running the job until two minutes from now
266
+ Resque.delay_or_enqueue_at(2.minutes.from_now, SendNotifications, :user_id => current_user.id)
267
+ ```
268
+
269
+ You don't need to worry if a matching job has already been queued, because if no matching jobs are found a new job is created and enqueued as if you had called `enqueue_at`. This means you don't need any special conditionals to know if a job has already been queued. You simply create the job like so:
270
+
271
+ ```ruby
272
+ Resque.delay_or_enqueue_at(1.minute.from_now, SendNotifications, :user_id => current_user.id)
273
+ ```
274
+
275
+ If multiple matching jobs are found, all of the matching jobs will be updated to have the same timestamp even if their original timestamps were not the same.
276
+
277
+ ```ruby
278
+ # enqueue multiple jobs with different delay timestamps
279
+ Resque.enqueue_at(1.minute.from_now, SendNotifications, :user_id => current_user.id)
280
+ Resque.enqueue_at(2.minutes.from_now, SendNotifications, :user_id => current_user.id)
281
+
282
+ # delay running the two jobs until 5 minutes from now
283
+ Resque.delay_or_enqueue_at(5.minutes.from_now, SendNotifications, :user_id => current_user.id)
284
+ ```
285
+
286
+ The most useful case for increasing the delay of an already delayed job is to batch together work based on multiple events. For example, if you wanted to send a notification email to a user when an event triggers but didn't want to send 10 emails if many events happened within a short period, you could use this technique to delay the noficication email until no events have triggered for a period of time. This way you could send 1 email containing the 10 notifications once no events have triggered for 2 minutes. You could implement this like so:
287
+
288
+ ```ruby
289
+ # Send a notification when an event is created.
290
+ # app/models/event.rb
291
+ after_commit on: :create do
292
+ Resque.delay_or_enqueue_in(2.minutes, SendNotifications, :user_id => user.id)
293
+ end
294
+ ```
295
+
296
+ When the first event is created a job will be scheduled to send unsent notifications to the associated user. If another event is created within the 2 minute window, the timer will be reset to 2 minutes. This will continue as long as new events are created for the specific user before the 2 minute timer expires. Once the timer expires and the job is scheduled any new events that are created will schedule a new job and start the process over. By adjusting the window you can tweak the trade-off between sending notification emails quickly after an event happens and sending fewer emails.
297
+
298
+ Read more in the [original PR](https://github.com/resque/resque-scheduler/pull/645)
299
+
300
+
159
301
  ### Scheduled Jobs (Recurring Jobs)
160
302
 
161
303
  Scheduled (or recurring) jobs are logically no different than a standard cron
162
- job. They are jobs that run based on a fixed schedule which is set at
163
- startup.
304
+ job. They are jobs that run based on a schedule which can be static or dynamic.
305
+
306
+ #### Static schedules
164
307
 
165
- The schedule is a list of Resque worker classes with arguments and a
308
+ Static schedules are set when `resque-scheduler` starts by passing a schedule file
309
+ to `resque-scheduler` initialization like this (see *Installation* above for a more complete example):
310
+
311
+ ```ruby
312
+ Resque.schedule = YAML.load_file('your_resque_schedule.yml')
313
+ ```
314
+
315
+ If a static schedule is not set `resque-scheduler` will issue a "Schedule empty!" warning on
316
+ startup, but despite that warning setting a static schedule is totally optional. It is possible
317
+ to use only dynamic schedules (see below).
318
+
319
+ The schedule file is a list of Resque job classes with arguments and a
166
320
  schedule frequency (in crontab syntax). The schedule is just a hash, but
167
- is most likely stored in a YAML like so:
321
+ is usually stored in a YAML like this:
168
322
 
169
323
  ```yaml
170
324
  CancelAbandonedOrders:
@@ -173,22 +327,34 @@ CancelAbandonedOrders:
173
327
  queue_documents_for_indexing:
174
328
  cron: "0 0 * * *"
175
329
  # you can use rufus-scheduler "every" syntax in place of cron if you prefer
176
- # every: 1hr
330
+ # every: 1h
177
331
  # By default the job name (hash key) will be taken as worker class name.
178
332
  # If you want to have a different job name and class name, provide the 'class' option
179
- class: QueueDocuments
333
+ class: "QueueDocuments"
180
334
  queue: high
181
335
  args:
182
336
  description: "This job queues all content for indexing in solr"
183
337
 
184
338
  clear_leaderboards_contributors:
185
339
  cron: "30 6 * * 1"
186
- class: ClearLeaderboards
340
+ class: "ClearLeaderboards"
187
341
  queue: low
188
342
  args: contributors
189
343
  description: "This job resets the weekly leaderboard for contributions"
190
344
  ```
191
345
 
346
+ If you would like to setup a job that is executed manually you can configure like this in your YAML file.
347
+
348
+ ```yaml
349
+ ImportOrdersManual:
350
+ custom_job_class: 'AmazonMws::ImportOrdersJob'
351
+ never: "* * * * *"
352
+ queue: high
353
+ description: "This is a manual job for importing orders."
354
+ args:
355
+ days_in_arrears: 7
356
+ ```
357
+
192
358
  The queue value is optional, but if left unspecified resque-scheduler will
193
359
  attempt to get the queue from the job class, which means it needs to be
194
360
  defined. If you're getting "uninitialized constant" errors, you probably
@@ -201,13 +367,18 @@ You can provide options to "every" or "cron" via Array:
201
367
  clear_leaderboards_moderator:
202
368
  every:
203
369
  - "30s"
204
- - :first_in: "120s"
205
- class: CheckDaemon
370
+ - :first_in: '120s'
371
+ class: "CheckDaemon"
206
372
  queue: daemons
207
373
  description: "This job will check Daemon every 30 seconds after 120 seconds after start"
208
374
  ```
209
375
 
210
- NOTE: Six parameter cron's are also supported (as they supported by
376
+ IMPORTANT: Rufus `every` syntax will calculate jobs scheduling time starting from the moment of deploy,
377
+ resulting in resetting schedule time on every deploy, so it's probably a good idea to use it only for
378
+ frequent jobs (like every 10-30 minutes), otherwise - when you use something like `every 20h` and deploy once-twice per day -
379
+ it will schedule the job for 20 hours from deploy, resulting in a job to never be run.
380
+
381
+ **NOTE**: Six parameter cron's are also supported (as they supported by
211
382
  rufus-scheduler which powers the resque-scheduler process). This allows you
212
383
  to schedule jobs per second (ie: `"30 * * * * *"` would fire a job every 30
213
384
  seconds past the minute).
@@ -215,21 +386,110 @@ seconds past the minute).
215
386
  A big shout out to [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler)
216
387
  for handling the heavy lifting of the actual scheduling engine.
217
388
 
389
+ ##### Queue with parameters
218
390
 
219
- #### Time zones
391
+ It's possible to specify parameters, that must be given by the user when they manually queue the job. To enable this feature add `parameters` key to scheduled job definition.
392
+
393
+ ```yaml
394
+ queue_documents_for_indexing:
395
+ cron: "0 0 * * *"
396
+ class: "QueueDocuments"
397
+ queue: high
398
+ args:
399
+ foo: "bar"
400
+ a: "b"
401
+ parameters:
402
+ foo:
403
+ description: "value of foo"
404
+ default: "baz"
220
405
 
221
- Note that if you use the cron syntax, this will be interpreted as in the server time zone
222
- rather than the `config.time_zone` specified in Rails.
406
+ description: "This job queues all content for indexing in solr"
407
+ ```
223
408
 
224
- You can explicitly specify the time zone that rufus-scheduler will use:
409
+ One can use following options for each parameter:
410
+ * description - tooltip to be shown next to the parameter input
411
+ * default - prefilled value in the parameter input
412
+
413
+ **NOTE**: When sheduling the job, parameters are merged into job args. Assuming the example above and default parametr value, the job will be run with the following args:
414
+
415
+ ```ruby
416
+ {"foo"=>"baz", "a"=>"b"}
417
+ ```
225
418
 
419
+ **NOTE**: If user leaves the parameter value empty, it'll be sent as empty string.
420
+
421
+ #### Dynamic schedules
422
+
423
+ Dynamic schedules are programmatically set on a running `resque-scheduler`.
424
+ Most [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler) options are supported
425
+ when setting schedules. Specifically the `overlap` option will not work.
426
+
427
+ Dynamic schedules are not enabled by default. To be able to dynamically set schedules, you
428
+ must pass the following to `resque-scheduler` initialization (see *Installation* above for a more complete example):
429
+
430
+ ```ruby
431
+ Resque::Scheduler.dynamic = true
432
+ ```
433
+
434
+ **NOTE**: In order to delete dynamic schedules via `resque-web` in the
435
+ "Schedule" tab, you must include the `Rack::MethodOverride` middleware (in
436
+ `config.ru` or equivalent).
437
+
438
+ Dynamic schedules allow for greater flexibility than static schedules as they can be set,
439
+ unset or changed without having to restart `resque-scheduler`. You can specify, if the schedule
440
+ must survive a resque-scheduler restart or not. This is done by setting the `persist` configuration
441
+ for the schedule: it is a boolean value, if set the schedule will persist a restart. By default,
442
+ a schedule will not be persisted.
443
+
444
+ The job to be scheduled must be a valid Resque job class.
445
+
446
+ For example, suppose you have a SendEmail job which sends emails. The `perform` method of the
447
+ job receives a string argument with the email subject. To run the SendEmail job every hour
448
+ starting five minutes from now, you can do:
449
+
450
+ ```ruby
451
+ name = 'send_emails'
452
+ config = {}
453
+ config[:class] = 'SendEmail'
454
+ config[:args] = 'POC email subject'
455
+ config[:every] = ['1h', {first_in: 5.minutes}]
456
+ config[:persist] = true
457
+ Resque.set_schedule(name, config)
458
+ ```
459
+
460
+ Schedules can later be removed by passing their name to the `remove_schedule` method:
461
+
462
+ ```ruby
463
+ name = 'send_emails'
464
+ Resque.remove_schedule(name)
465
+ ```
466
+
467
+ Schedule names are unique; i.e. two dynamic schedules cannot have the same name. If `set_schedule` is
468
+ passed the name of an existing schedule, that schedule is updated. E.g. if after setting the above schedule
469
+ we want the job to run every day instead of every hour from now on, we can do:
470
+
471
+ ```ruby
472
+ name = 'send_emails'
473
+ config = {}
474
+ config[:class] = 'SendEmail'
475
+ config[:args] = 'POC email subject'
476
+ config[:every] = '1d'
477
+ Resque.set_schedule(name, config)
478
+ ```
479
+
480
+ #### Time zones
481
+
482
+ If you use the cron syntax, by default it is interpreted in the server time zone.
483
+ You can explicitly specify the time zone that rufus-scheduler will use:
226
484
  ```yaml
227
485
  cron: "30 6 * * 1 Europe/Stockholm"
228
486
  ```
229
487
 
230
- Also note that `config.time_zone` in Rails allows for a shorthand (e.g. "Stockholm")
231
- that rufus-scheduler does not accept. If you write code to set the scheduler time zone
232
- from the `config.time_zone` value, make sure it's the right format, e.g. with:
488
+ ##### Rails
489
+ In Rails, `config.time_zone` will be used to determine the time zone for `resque-scheduler`.
490
+
491
+ Note that `config.time_zone` allows for a shorthand (e.g. "Stockholm")
492
+ that rufus-scheduler does not accept, so make sure it's the right format, e.g. with:
233
493
 
234
494
  ```ruby
235
495
  ActiveSupport::TimeZone.find_tzinfo(Rails.configuration.time_zone).name
@@ -252,6 +512,13 @@ Similar to the `before_enqueue`- and `after_enqueue`-hooks provided in Resque
252
512
  removed from the delayed queue, but not yet put on a normal queue. It is
253
513
  called before `before_enqueue`-hooks, and on the same job instance as the
254
514
  `before_enqueue`-hooks will be invoked on. Return values are ignored.
515
+ * `on_enqueue_failure`: Called with the job args and the exception that was raised
516
+ while enqueueing a job to resque or external application fails. Return
517
+ values are ignored. For example:
518
+
519
+ ```ruby
520
+ Resque::Scheduler.failure_handler = ExceptionHandlerClass
521
+ ```
255
522
 
256
523
  #### Support for resque-status (and other custom jobs)
257
524
 
@@ -278,7 +545,7 @@ And then a schedule:
278
545
  create_fake_leaderboards:
279
546
  cron: "30 6 * * 1"
280
547
  queue: scoring
281
- custom_job_class: FakeLeaderboard
548
+ custom_job_class: "FakeLeaderboard"
282
549
  args:
283
550
  rails_env: demo
284
551
  description: "This job will auto-create leaderboards for our online demo and the status will update as the worker makes progress"
@@ -304,13 +571,13 @@ end
304
571
  *>= 2.0.1 only. Prior to 2.0.1, it is not recommended to run multiple resque-scheduler processes and will result in duplicate jobs.*
305
572
 
306
573
  You may want to have resque-scheduler running on multiple machines for
307
- redudancy. Electing a master and failover is built in and default. Simply
574
+ redundancy. Electing a master and failover is built in and default. Simply
308
575
  run resque-scheduler on as many machine as you want pointing to the same
309
576
  redis instance and schedule. The scheduler processes will use redis to
310
577
  elect a master process and detect failover when the master dies. Precautions are
311
578
  taken to prevent jobs from potentially being queued twice during failover even
312
579
  when the clocks of the scheduler machines are slightly out of sync (or load affects
313
- scheduled job firing time). If you want the gory details, look at Resque::SchedulerLocking.
580
+ scheduled job firing time). If you want the gory details, look at Resque::Scheduler::Locking.
314
581
 
315
582
  If the scheduler process(es) goes down for whatever reason, the delayed items
316
583
  that should have fired during the outage will fire once the scheduler process
@@ -325,11 +592,13 @@ run, leading to undesired behaviour. To allow different scheduler configs run at
325
592
  on one redis, you can either namespace your redis connections, or supply an environment variable
326
593
  to split the shared lock key resque-scheduler uses thus:
327
594
 
328
- RESQUE_SCHEDULER_MASTER_LOCK_PREFIX=MyApp: rake resque:scheduler
595
+ ``` bash
596
+ RESQUE_SCHEDULER_MASTER_LOCK_PREFIX=MyApp: rake resque:scheduler
597
+ ```
329
598
 
330
599
  ### resque-web Additions
331
600
 
332
- Resque-scheduler also adds to tabs to the resque-web UI. One is for viewing
601
+ Resque-scheduler also adds two tabs to the resque-web UI. One is for viewing
333
602
  (and manually queueing) the schedule and one is for viewing pending jobs in
334
603
  the delayed queue.
335
604
 
@@ -357,18 +626,13 @@ Now, you want to add the following:
357
626
 
358
627
  ```ruby
359
628
  # This will make the tabs show up.
360
- require 'resque_scheduler'
361
- require 'resque_scheduler/server'
629
+ require 'resque-scheduler'
630
+ require 'resque/scheduler/server'
362
631
  ```
363
632
 
364
633
  That should make the scheduler tabs show up in `resque-web`.
365
634
 
366
-
367
- #### Changes as of 2.0.0
368
-
369
- As of resque-scheduler 2.0.0, it's no longer necessary to have the resque-web
370
- process aware of the schedule because it reads it from redis. But prior to
371
- 2.0, you'll want to make sure you load the schedule in this file as well.
635
+ You'll want to make sure you load the schedule in this file as well.
372
636
  Something like this:
373
637
 
374
638
  ```ruby
@@ -382,7 +646,7 @@ Now make sure you're passing that file to resque-web like so:
382
646
 
383
647
  ### Running in the background
384
648
 
385
- (Only supported with ruby >= 1.9). There are scenarios where it's helpful for
649
+ There are scenarios where it's helpful for
386
650
  the resque worker to run itself in the background (usually in combination with
387
651
  PIDFILE). Use the BACKGROUND option so that rake will return as soon as the
388
652
  worker is started.
@@ -396,32 +660,100 @@ worker is started.
396
660
  There are several options to toggle the way scheduler logs its actions. They
397
661
  are toggled by environment variables:
398
662
 
399
- - `MUTE` will stop logging anything. Completely silent;
400
- - `VERBOSE` opposite to 'mute' will log even debug information;
401
- - `LOGFILE` specifies the file to write logs to. Default is standard output.
663
+ - `QUIET` will stop logging anything. Completely silent.
664
+ - `VERBOSE` opposite of 'QUIET'; will log even debug information
665
+ - `LOGFILE` specifies the file to write logs to. (default standard output)
666
+ - `LOGFORMAT` specifies either "text", "json" or "logfmt" output format
667
+ (default "text")
402
668
 
403
- All those variables are non-mandatory and default values are
669
+ All of these variables are optional and will be given the following default
670
+ values:
404
671
 
405
672
  ```ruby
406
- Resque::Scheduler.mute = false
407
- Resque::Scheduler.verbose = false
408
- Resque::Scheduler.logfile = nil # that means, all messages go to STDOUT
673
+ Resque::Scheduler.configure do |c|
674
+ c.quiet = false
675
+ c.verbose = false
676
+ c.logfile = nil # meaning all messages go to $stdout
677
+ c.logformat = 'text'
678
+ end
409
679
  ```
410
680
 
411
-
412
681
  ### Polling frequency
413
682
 
414
- You can pass an INTERVAL option which is a integer representing the polling frequency.
415
- The default is 5 seconds, but for a semi-active app you may want to use a smaller (integer) value.
683
+ You can pass a `RESQUE_SCHEDULER_INTERVAL` option which is an integer or
684
+ float representing the polling frequency. The default is 5 seconds, but
685
+ for a semi-active app you may want to use a smaller value.
416
686
 
417
- $ INTERVAL=1 rake resque:scheduler
687
+ $ RESQUE_SCHEDULER_INTERVAL=1 rake resque:scheduler
688
+
689
+ **NOTE** This value was previously `INTERVAL` but was renamed to
690
+ `RESQUE_SCHEDULER_INTERVAL` to avoid clashing with the interval Resque
691
+ uses for its jobs.
418
692
 
419
693
  ### Plagiarism alert
420
694
 
421
- This was intended to be an extension to resque and so resulted in a lot of the
422
- code looking very similar to resque, particularly in resque-web and the views. I
423
- wanted it to be similar enough that someone familiar with resque could easily
424
- work on resque-scheduler.
695
+ This was intended to be an extension to resque and so resulted in a lot
696
+ of the code looking very similar to resque, particularly in resque-web
697
+ and the views. I wanted it to be similar enough that someone familiar
698
+ with resque could easily work on resque-scheduler.
699
+
700
+ ### Development
701
+
702
+ Working on resque-scheduler requires the following:
703
+
704
+ * A relatively modern Ruby interpreter
705
+ * bundler
706
+
707
+ The development setup looks like this, which is roughly the same thing
708
+ that happens on Travis CI and Appveyor:
709
+
710
+ ``` bash
711
+ # Install everything
712
+ bundle install
713
+
714
+ # Make sure tests are green before you change stuff
715
+ bundle exec rubocop && bundle exec rake
716
+ # Change stuff
717
+ # Repeat
718
+ ```
719
+
720
+ If you have [vagrant](http://www.vagrantup.com) installed, there is a
721
+ development box available that requires no plugins or external
722
+ provisioners:
723
+
724
+ ``` bash
725
+ vagrant up
726
+ ```
727
+
728
+ ### Deployment Notes
729
+
730
+ It is recommended that a production deployment of `resque-scheduler` be hosted
731
+ on a dedicated Redis database. While making and managing scheduled tasks,
732
+ `resque-scheduler` currently scans the entire Redis keyspace, which may cause
733
+ latency and stability issues if `resque-scheduler` is hosted on a Redis instance
734
+ storing a large number of keys (such as those written by a different system
735
+ hosted on the same Redis instance).
736
+
737
+ #### Compatibility Notes
738
+
739
+ Different versions of the `redis` and `rufus-scheduler` gems are needed
740
+ depending on your version of `resque-scheduler`. This is typically not a
741
+ problem with `resque-scheduler` itself, but when mixing dependencies with an
742
+ existing application.
743
+
744
+ This table explains the version requirements for redis gem
745
+
746
+ | resque-scheduler | redis gem |
747
+ |:-----------------|-----------:|
748
+ | `~> 2.0` | `>= 3.0.0` |
749
+ | `>= 0.0.1` | `~> 1.3` |
750
+
751
+ This table explains the version requirements for rufus-scheduler
752
+
753
+ | resque-scheduler | rufus-scheduler |
754
+ |:-----------------|----------------:|
755
+ | `~> 4.0` | `~> 3.0` |
756
+ | `< 4.0` | `~> 2.0` |
425
757
 
426
758
 
427
759
  ### Contributing