resque-scheduler 4.3.0 → 4.7.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of resque-scheduler might be problematic. Click here for more details.

data/README.md CHANGED
@@ -1,10 +1,9 @@
1
1
  resque-scheduler
2
2
  ================
3
3
 
4
- [![Dependency Status](https://gemnasium.com/badges/github.com/resque/resque-scheduler.svg)](https://gemnasium.com/github.com/resque/resque-scheduler)
4
+
5
5
  [![Gem Version](https://badge.fury.io/rb/resque-scheduler.svg)](https://badge.fury.io/rb/resque-scheduler)
6
- [![Build Status](https://travis-ci.org/resque/resque-scheduler.svg?branch=master)](https://travis-ci.org/resque/resque-scheduler)
7
- [![Windows Build Status](https://ci.appveyor.com/api/projects/status/sxvf2086v5j0absb/branch/master?svg=true)](https://ci.appveyor.com/project/resque/resque-scheduler/branch/master)
6
+ [![Ruby specs](https://github.com/resque/resque-scheduler/actions/workflows/ruby.yml/badge.svg)](https://github.com/resque/resque-scheduler/actions)
8
7
  [![Code Climate](https://codeclimate.com/github/resque/resque-scheduler/badges/gpa.svg)](https://codeclimate.com/github/resque/resque-scheduler)
9
8
 
10
9
  ### Description
@@ -12,7 +11,7 @@ resque-scheduler
12
11
  Resque-scheduler is an extension to [Resque](http://github.com/resque/resque)
13
12
  that adds support for queueing items in the future.
14
13
 
15
- Job scheduling is supported in two different way: Recurring (scheduled) and
14
+ Job scheduling is supported in two different ways: Recurring (scheduled) and
16
15
  Delayed.
17
16
 
18
17
  Scheduled jobs are like cron jobs, recurring on a regular basis. Delayed
@@ -159,7 +158,7 @@ following task to wherever tasks are kept, such as
159
158
  ```ruby
160
159
  task 'resque:pool:setup' do
161
160
  Resque::Pool.after_prefork do |job|
162
- Resque.redis.client.reconnect
161
+ Resque.redis.reconnect
163
162
  end
164
163
  end
165
164
  ```
@@ -212,7 +211,7 @@ since the jobs are stored in a redis sorted set (zset). I can't imagine this
212
211
  being an issue for someone since redis is stupidly fast even at log(n), but full
213
212
  disclosure is always best.
214
213
 
215
- #### Removing Delayed jobs
214
+ #### Removing Delayed Jobs
216
215
 
217
216
  If you have the need to cancel a delayed job, you can do like so:
218
217
 
@@ -234,6 +233,17 @@ Resque.remove_delayed_selection { |args| args[0]['account_id'] == current_accoun
234
233
  Resque.remove_delayed_selection { |args| args[0]['user_id'] == current_user.id }
235
234
  ```
236
235
 
236
+ If you need to cancel a delayed job based on some matching arguments AND by which class the job is, but don't wish to specify each argument from when the job was created, you can do like so:
237
+
238
+ ``` ruby
239
+ # after you've enqueued a job like:
240
+ Resque.enqueue_at(5.days.from_now, SendFollowUpEmail, :account_id => current_account.id, :user_id => current_user.id)
241
+ # remove jobs matching just the account and that were of the class SendFollowUpEmail:
242
+ Resque.remove_delayed_selection(SendFollowUpEmail) { |args| args[0]['account_id'] == current_account.id }
243
+ # or remove jobs matching just the user and that were of the class SendFollowUpEmail:
244
+ Resque.remove_delayed_selection(SendFollowUpEmail) { |args| args[0]['user_id'] == current_user.id }
245
+ ```
246
+
237
247
  If you need to enqueue immediately a delayed job based on some matching arguments, but don't wish to specify each argument from when the job was created, you can do like so:
238
248
 
239
249
  ``` ruby
@@ -245,6 +255,49 @@ Resque.enqueue_delayed_selection { |args| args[0]['account_id'] == current_accou
245
255
  Resque.enqueue_delayed_selection { |args| args[0]['user_id'] == current_user.id }
246
256
  ```
247
257
 
258
+ #### Updating Delayed Jobs
259
+
260
+ Previously delayed jobs may be delayed even further into the future like so:
261
+
262
+ ```ruby
263
+ # after you've enqueued a job like:
264
+ Resque.enqueue_at(1.minute.from_now, SendNotifications, :user_id => current_user.id)
265
+ # delay running the job until two minutes from now
266
+ Resque.delay_or_enqueue_at(2.minutes.from_now, SendNotifications, :user_id => current_user.id)
267
+ ```
268
+
269
+ You don't need to worry if a matching job has already been queued, because if no matching jobs are found a new job is created and enqueued as if you had called `enqueue_at`. This means you don't need any special conditionals to know if a job has already been queued. You simply create the job like so:
270
+
271
+ ```ruby
272
+ Resque.delay_or_enqueue_at(1.minute.from_now, SendNotifications, :user_id => current_user.id)
273
+ ```
274
+
275
+ If multiple matching jobs are found, all of the matching jobs will be updated to have the same timestamp even if their original timestamps were not the same.
276
+
277
+ ```ruby
278
+ # enqueue multiple jobs with different delay timestamps
279
+ Resque.enqueue_at(1.minute.from_now, SendNotifications, :user_id => current_user.id)
280
+ Resque.enqueue_at(2.minutes.from_now, SendNotifications, :user_id => current_user.id)
281
+
282
+ # delay running the two jobs until 5 minutes from now
283
+ Resque.delay_or_enqueue_at(5.minutes.from_now, SendNotifications, :user_id => current_user.id)
284
+ ```
285
+
286
+ The most useful case for increasing the delay of an already delayed job is to batch together work based on multiple events. For example, if you wanted to send a notification email to a user when an event triggers but didn't want to send 10 emails if many events happened within a short period, you could use this technique to delay the noficication email until no events have triggered for a period of time. This way you could send 1 email containing the 10 notifications once no events have triggered for 2 minutes. You could implement this like so:
287
+
288
+ ```ruby
289
+ # Send a notification when an event is created.
290
+ # app/models/event.rb
291
+ after_commit on: :create do
292
+ Resque.delay_or_enqueue_in(2.minutes, SendNotifications, :user_id => user.id)
293
+ end
294
+ ```
295
+
296
+ When the first event is created a job will be scheduled to send unsent notifications to the associated user. If another event is created within the 2 minute window, the timer will be reset to 2 minutes. This will continue as long as new events are created for the specific user before the 2 minute timer expires. Once the timer expires and the job is scheduled any new events that are created will schedule a new job and start the process over. By adjusting the window you can tweak the trade-off between sending notification emails quickly after an event happens and sending fewer emails.
297
+
298
+ Read more in the [original PR](https://github.com/resque/resque-scheduler/pull/645)
299
+
300
+
248
301
  ### Scheduled Jobs (Recurring Jobs)
249
302
 
250
303
  Scheduled (or recurring) jobs are logically no different than a standard cron
@@ -290,6 +343,18 @@ clear_leaderboards_contributors:
290
343
  description: "This job resets the weekly leaderboard for contributions"
291
344
  ```
292
345
 
346
+ If you would like to setup a job that is executed manually you can configure like this in your YAML file.
347
+
348
+ ```yaml
349
+ ImportOrdersManual:
350
+ custom_job_class: 'AmazonMws::ImportOrdersJob'
351
+ never: "* * * * *"
352
+ queue: high
353
+ description: "This is a manual job for importing orders."
354
+ args:
355
+ days_in_arrears: 7
356
+ ```
357
+
293
358
  The queue value is optional, but if left unspecified resque-scheduler will
294
359
  attempt to get the queue from the job class, which means it needs to be
295
360
  defined. If you're getting "uninitialized constant" errors, you probably
@@ -321,11 +386,43 @@ seconds past the minute).
321
386
  A big shout out to [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler)
322
387
  for handling the heavy lifting of the actual scheduling engine.
323
388
 
389
+ ##### Queue with parameters
390
+
391
+ It's possible to specify parameters, that must be given by the user when they manually queue the job. To enable this feature add `parameters` key to scheduled job definition.
392
+
393
+ ```yaml
394
+ queue_documents_for_indexing:
395
+ cron: "0 0 * * *"
396
+ class: "QueueDocuments"
397
+ queue: high
398
+ args:
399
+ foo: "bar"
400
+ a: "b"
401
+ parameters:
402
+ foo:
403
+ description: "value of foo"
404
+ default: "baz"
405
+
406
+ description: "This job queues all content for indexing in solr"
407
+ ```
408
+
409
+ One can use following options for each parameter:
410
+ * description - tooltip to be shown next to the parameter input
411
+ * default - prefilled value in the parameter input
412
+
413
+ **NOTE**: When sheduling the job, parameters are merged into job args. Assuming the example above and default parametr value, the job will be run with the following args:
414
+
415
+ ```ruby
416
+ {"foo"=>"baz", "a"=>"b"}
417
+ ```
418
+
419
+ **NOTE**: If user leaves the parameter value empty, it'll be sent as empty string.
420
+
324
421
  #### Dynamic schedules
325
422
 
326
423
  Dynamic schedules are programmatically set on a running `resque-scheduler`.
327
- All [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler) options are supported
328
- when setting schedules.
424
+ Most [rufus-scheduler](http://github.com/jmettraux/rufus-scheduler) options are supported
425
+ when setting schedules. Specifically the `overlap` option will not work.
329
426
 
330
427
  Dynamic schedules are not enabled by default. To be able to dynamically set schedules, you
331
428
  must pass the following to `resque-scheduler` initialization (see *Installation* above for a more complete example):
@@ -382,18 +479,17 @@ Resque.set_schedule(name, config)
382
479
 
383
480
  #### Time zones
384
481
 
385
- Note that if you use the cron syntax, this will be interpreted as in the server time zone
386
- rather than the `config.time_zone` specified in Rails.
387
-
482
+ If you use the cron syntax, by default it is interpreted in the server time zone.
388
483
  You can explicitly specify the time zone that rufus-scheduler will use:
389
-
390
484
  ```yaml
391
485
  cron: "30 6 * * 1 Europe/Stockholm"
392
486
  ```
393
487
 
394
- Also note that `config.time_zone` in Rails allows for a shorthand (e.g. "Stockholm")
395
- that rufus-scheduler does not accept. If you write code to set the scheduler time zone
396
- from the `config.time_zone` value, make sure it's the right format, e.g. with:
488
+ ##### Rails
489
+ In Rails, `config.time_zone` will be used to determine the time zone for `resque-scheduler`.
490
+
491
+ Note that `config.time_zone` allows for a shorthand (e.g. "Stockholm")
492
+ that rufus-scheduler does not accept, so make sure it's the right format, e.g. with:
397
493
 
398
494
  ```ruby
399
495
  ActiveSupport::TimeZone.find_tzinfo(Rails.configuration.time_zone).name
@@ -502,7 +598,7 @@ RESQUE_SCHEDULER_MASTER_LOCK_PREFIX=MyApp: rake resque:scheduler
502
598
 
503
599
  ### resque-web Additions
504
600
 
505
- Resque-scheduler also adds to tabs to the resque-web UI. One is for viewing
601
+ Resque-scheduler also adds two tabs to the resque-web UI. One is for viewing
506
602
  (and manually queueing) the schedule and one is for viewing pending jobs in
507
603
  the delayed queue.
508
604
 
@@ -536,11 +632,7 @@ require 'resque/scheduler/server'
536
632
 
537
633
  That should make the scheduler tabs show up in `resque-web`.
538
634
 
539
- #### Changes as of 2.0.0
540
-
541
- As of resque-scheduler 2.0.0, it's no longer necessary to have the resque-web
542
- process aware of the schedule because it reads it from redis. But prior to
543
- 2.0, you'll want to make sure you load the schedule in this file as well.
635
+ You'll want to make sure you load the schedule in this file as well.
544
636
  Something like this:
545
637
 
546
638
  ```ruby
@@ -554,7 +646,7 @@ Now make sure you're passing that file to resque-web like so:
554
646
 
555
647
  ### Running in the background
556
648
 
557
- (Only supported with ruby >= 1.9). There are scenarios where it's helpful for
649
+ There are scenarios where it's helpful for
558
650
  the resque worker to run itself in the background (usually in combination with
559
651
  PIDFILE). Use the BACKGROUND option so that rake will return as soon as the
560
652
  worker is started.
@@ -620,7 +712,7 @@ that happens on Travis CI and Appveyor:
620
712
  bundle install
621
713
 
622
714
  # Make sure tests are green before you change stuff
623
- bundle exec rake
715
+ bundle exec rubocop && bundle exec rake
624
716
  # Change stuff
625
717
  # Repeat
626
718
  ```
data/Rakefile CHANGED
@@ -1,13 +1,9 @@
1
1
  # vim:fileencoding=utf-8
2
2
  require 'bundler/gem_tasks'
3
3
  require 'rake/testtask'
4
- require 'rubocop/rake_task'
5
4
  require 'yard'
6
5
 
7
- task default: [:rubocop, :test] unless RUBY_PLATFORM =~ /java/
8
- task default: [:test] if RUBY_PLATFORM =~ /java/
9
-
10
- RuboCop::RakeTask.new
6
+ task default: :test
11
7
 
12
8
  Rake::TestTask.new do |t|
13
9
  t.libs << 'test'
@@ -16,6 +12,7 @@ Rake::TestTask.new do |t|
16
12
  o << "--seed #{ENV['SEED']} " if ENV['SEED']
17
13
  o << '--verbose ' if ENV['VERBOSE']
18
14
  end
15
+ t.warning = false
19
16
  end
20
17
 
21
18
  YARD::Rake::YardocTask.new
@@ -128,6 +128,7 @@ module Resque
128
128
  def option_parser
129
129
  OptionParser.new do |opts|
130
130
  opts.banner = BANNER
131
+ opts.version = Resque::Scheduler::VERSION
131
132
  OPTIONS.each do |opt|
132
133
  opts.on(*opt[:args], &opt[:callback].call(options))
133
134
  end
@@ -8,13 +8,18 @@ module Resque
8
8
  yield self
9
9
  end
10
10
 
11
+ attr_writer :environment
12
+ def environment
13
+ @environment ||= ENV
14
+ end
15
+
11
16
  # Used in `#load_schedule_job`
12
17
  attr_writer :env
13
18
 
14
19
  def env
15
20
  return @env if @env
16
21
  @env ||= Rails.env if defined?(Rails) && Rails.respond_to?(:env)
17
- @env ||= ENV['RAILS_ENV']
22
+ @env ||= environment['RAILS_ENV']
18
23
  @env
19
24
  end
20
25
 
@@ -22,42 +27,42 @@ module Resque
22
27
  attr_writer :verbose
23
28
 
24
29
  def verbose
25
- @verbose ||= !!ENV['VERBOSE']
30
+ @verbose ||= to_bool(environment['VERBOSE'])
26
31
  end
27
32
 
28
33
  # If set, produces no output
29
34
  attr_writer :quiet
30
35
 
31
36
  def quiet
32
- @quiet ||= !!ENV['QUIET']
37
+ @quiet ||= to_bool(environment['QUIET'])
33
38
  end
34
39
 
35
40
  # If set, will write messages to the file
36
41
  attr_writer :logfile
37
42
 
38
43
  def logfile
39
- @logfile ||= ENV['LOGFILE']
44
+ @logfile ||= environment['LOGFILE']
40
45
  end
41
46
 
42
47
  # Sets whether to log in 'text' or 'json'
43
48
  attr_writer :logformat
44
49
 
45
50
  def logformat
46
- @logformat ||= ENV['LOGFORMAT']
51
+ @logformat ||= environment['LOGFORMAT']
47
52
  end
48
53
 
49
54
  # If set, will try to update the schedule in the loop
50
55
  attr_writer :dynamic
51
56
 
52
57
  def dynamic
53
- @dynamic ||= !!ENV['DYNAMIC_SCHEDULE']
58
+ @dynamic ||= to_bool(environment['DYNAMIC_SCHEDULE'])
54
59
  end
55
60
 
56
61
  # If set, will append the app name to procline
57
62
  attr_writer :app_name
58
63
 
59
64
  def app_name
60
- @app_name ||= ENV['APP_NAME']
65
+ @app_name ||= environment['APP_NAME']
61
66
  end
62
67
 
63
68
  # Amount of time in seconds to sleep between polls of the delayed
@@ -66,7 +71,25 @@ module Resque
66
71
 
67
72
  def poll_sleep_amount
68
73
  @poll_sleep_amount ||=
69
- Float(ENV.fetch('RESQUE_SCHEDULER_INTERVAL', '5'))
74
+ Float(environment.fetch('RESQUE_SCHEDULER_INTERVAL', '5'))
75
+ end
76
+
77
+ private
78
+
79
+ # Copied from https://github.com/rails/rails/blob/main/activemodel/lib/active_model/type/boolean.rb#L17
80
+ TRUE_VALUES = [
81
+ true, 1,
82
+ '1', :'1',
83
+ 't', :t,
84
+ 'T', :T,
85
+ 'true', :true,
86
+ 'TRUE', :TRUE,
87
+ 'on', :on,
88
+ 'ON', :ON
89
+ ].to_set.freeze
90
+
91
+ def to_bool(value)
92
+ TRUE_VALUES.include?(value)
70
93
  end
71
94
  end
72
95
  end
@@ -24,7 +24,7 @@ module Resque
24
24
  def enqueue_at_with_queue(queue, timestamp, klass, *args)
25
25
  return false unless plugin.run_before_schedule_hooks(klass, *args)
26
26
 
27
- if Resque.inline? || timestamp.to_i < Time.now.to_i
27
+ if Resque.inline? || timestamp.to_i <= Time.now.to_i
28
28
  # Just create the job and let resque perform it right away with
29
29
  # inline. If the class is a custom job class, call self#scheduled
30
30
  # on it. This allows you to do things like
@@ -33,7 +33,7 @@ module Resque
33
33
  if klass.respond_to?(:scheduled)
34
34
  klass.scheduled(queue, klass.to_s, *args)
35
35
  else
36
- Resque::Job.create(queue, klass, *args)
36
+ Resque.enqueue_to(queue, klass, *args)
37
37
  end
38
38
  else
39
39
  delayed_push(timestamp, job_to_hash_with_queue(queue, klass, args))
@@ -45,6 +45,9 @@ module Resque
45
45
  # Identical to enqueue_at but takes number_of_seconds_from_now
46
46
  # instead of a timestamp.
47
47
  def enqueue_in(number_of_seconds_from_now, klass, *args)
48
+ unless number_of_seconds_from_now.is_a?(Numeric)
49
+ raise ArgumentError, 'Please supply a numeric number of seconds'
50
+ end
48
51
  enqueue_at(Time.now + number_of_seconds_from_now, klass, *args)
49
52
  end
50
53
 
@@ -53,14 +56,35 @@ module Resque
53
56
  # number of seconds has passed.
54
57
  def enqueue_in_with_queue(queue, number_of_seconds_from_now,
55
58
  klass, *args)
59
+ unless number_of_seconds_from_now.is_a?(Numeric)
60
+ raise ArgumentError, 'Please supply a numeric number of seconds'
61
+ end
56
62
  enqueue_at_with_queue(queue, Time.now + number_of_seconds_from_now,
57
63
  klass, *args)
58
64
  end
59
65
 
66
+ # Update the delayed timestamp of any matching delayed jobs or enqueue a
67
+ # new job if no matching jobs are found. Returns the number of delayed or
68
+ # enqueued jobs.
69
+ def delay_or_enqueue_at(timestamp, klass, *args)
70
+ count = remove_delayed(klass, *args)
71
+ count = 1 if count == 0
72
+
73
+ count.times do
74
+ enqueue_at(timestamp, klass, *args)
75
+ end
76
+ end
77
+
78
+ # Identical to +delay_or_enqueue_at+, except it takes
79
+ # number_of_seconds_from_now instead of a timestamp
80
+ def delay_or_enqueue_in(number_of_seconds_from_now, klass, *args)
81
+ delay_or_enqueue_at(Time.now + number_of_seconds_from_now, klass, *args)
82
+ end
83
+
60
84
  # Used internally to stuff the item into the schedule sorted list.
61
- # +timestamp+ can be either in seconds or a datetime object Insertion
62
- # if O(log(n)). Returns true if it's the first job to be scheduled at
63
- # that time, else false
85
+ # +timestamp+ can be either in seconds or a datetime object. The
86
+ # insertion time complexity is O(log(n)). Returns true if it's
87
+ # the first job to be scheduled at that time, else false.
64
88
  def delayed_push(timestamp, item)
65
89
  # First add this item to the list for this timestamp
66
90
  redis.rpush("delayed:#{timestamp.to_i}", encode(item))
@@ -82,6 +106,7 @@ module Resque
82
106
  end
83
107
 
84
108
  # Returns the size of the delayed queue schedule
109
+ # this does not represent the number of items in the queue to be scheduled
85
110
  def delayed_queue_schedule_size
86
111
  redis.zcard :delayed_queue_schedule
87
112
  end
@@ -128,10 +153,7 @@ module Resque
128
153
  Array(redis.zrange(:delayed_queue_schedule, 0, -1)).each do |item|
129
154
  key = "delayed:#{item}"
130
155
  items = redis.lrange(key, 0, -1)
131
- redis.pipelined do
132
- items.each { |ts_item| redis.del("timestamps:#{ts_item}") }
133
- end
134
- redis.del key
156
+ redis.del(key, items.map { |ts_item| "timestamps:#{ts_item}" })
135
157
  end
136
158
 
137
159
  redis.del :delayed_queue_schedule
@@ -143,6 +165,11 @@ module Resque
143
165
  remove_delayed_job(search)
144
166
  end
145
167
 
168
+ def remove_delayed_in_queue(klass, queue, *args)
169
+ search = encode(job_to_hash_with_queue(queue, klass, args))
170
+ remove_delayed_job(search)
171
+ end
172
+
146
173
  # Given an encoded item, enqueue it now
147
174
  def enqueue_delayed(klass, *args)
148
175
  hash = job_to_hash(klass, args)
@@ -151,6 +178,13 @@ module Resque
151
178
  end
152
179
  end
153
180
 
181
+ def enqueue_delayed_with_queue(klass, queue, *args)
182
+ hash = job_to_hash_with_queue(queue, klass, args)
183
+ remove_delayed_in_queue(klass, queue, *args).times do
184
+ Resque::Scheduler.enqueue_from_config(hash)
185
+ end
186
+ end
187
+
154
188
  # Given a block, remove jobs that return true from a block
155
189
  #
156
190
  # This allows for removal of delayed jobs that have arguments matching
@@ -175,7 +209,15 @@ module Resque
175
209
  found_jobs.reduce(0) do |sum, encoded_job|
176
210
  decoded_job = decode(encoded_job)
177
211
  klass = Util.constantize(decoded_job['class'])
178
- sum + enqueue_delayed(klass, *decoded_job['args'])
212
+ queue = decoded_job['queue']
213
+
214
+ if queue
215
+ jobs_queued = enqueue_delayed_with_queue(klass, queue, *decoded_job['args'])
216
+ else
217
+ jobs_queued = enqueue_delayed(klass, *decoded_job['args'])
218
+ end
219
+
220
+ jobs_queued + sum
179
221
  end
180
222
  end
181
223
 
@@ -190,9 +232,9 @@ module Resque
190
232
 
191
233
  # Beyond 100 there's almost no improvement in speed
192
234
  found = timestamps.each_slice(100).map do |ts_group|
193
- jobs = redis.pipelined do |r|
235
+ jobs = redis.pipelined do |pipeline|
194
236
  ts_group.each do |ts|
195
- r.lrange("delayed:#{ts}", 0, -1)
237
+ pipeline.lrange("delayed:#{ts}", 0, -1)
196
238
  end
197
239
  end
198
240
 
@@ -265,18 +307,23 @@ module Resque
265
307
  { class: klass.to_s, args: args, queue: queue }
266
308
  end
267
309
 
310
+ # Removes a job from the queue, but not modify the timestamp schedule. This method
311
+ # will not effect the output of `delayed_queue_schedule_size`
268
312
  def remove_delayed_job(encoded_job)
269
313
  return 0 if Resque.inline?
270
314
 
271
315
  timestamps = redis.smembers("timestamps:#{encoded_job}")
272
316
 
273
- replies = redis.pipelined do
317
+ replies = redis.pipelined do |pipeline|
274
318
  timestamps.each do |key|
275
- redis.lrem(key, 0, encoded_job)
276
- redis.srem("timestamps:#{encoded_job}", key)
319
+ pipeline.lrem(key, 0, encoded_job)
320
+ pipeline.srem("timestamps:#{encoded_job}", key)
277
321
  end
278
322
  end
279
323
 
324
+ # timestamp key is not removed from the schedule, this is done later
325
+ # by the scheduler loop
326
+
280
327
  return 0 if replies.nil? || replies.empty?
281
328
  replies.each_slice(2).map(&:first).inject(:+)
282
329
  end
@@ -287,9 +334,9 @@ module Resque
287
334
  redis.watch(key) do
288
335
  if redis.llen(key).to_i == 0
289
336
  # If the list is empty, remove it.
290
- redis.multi do
291
- redis.del(key)
292
- redis.zrem(:delayed_queue_schedule, timestamp.to_i)
337
+ redis.multi do |transaction|
338
+ transaction.del(key)
339
+ transaction.zrem(:delayed_queue_schedule, timestamp.to_i)
293
340
  end
294
341
  else
295
342
  redis.redis.unwatch
@@ -38,12 +38,8 @@ module Resque
38
38
  true
39
39
  end
40
40
 
41
- unless Process.respond_to?('daemon')
42
- abort 'background option is set, which requires ruby >= 1.9'
43
- end
44
-
45
41
  Process.daemon(true, !Resque::Scheduler.quiet)
46
- Resque.redis.client.reconnect
42
+ Resque.redis.reconnect
47
43
  end
48
44
 
49
45
  def setup_pid_file
@@ -64,13 +60,13 @@ module Resque
64
60
 
65
61
  c.dynamic = !!options[:dynamic] if options.key?(:dynamic)
66
62
 
67
- c.env = options[:env] if options.key(:env)
63
+ c.env = options[:env] if options.key?(:env)
68
64
 
69
65
  c.logfile = options[:logfile] if options.key?(:logfile)
70
66
 
71
67
  c.logformat = options[:logformat] if options.key?(:logformat)
72
68
 
73
- if psleep = options[:poll_sleep_amount] && !psleep.nil?
69
+ if (psleep = options[:poll_sleep_amount]) && !psleep.nil?
74
70
  c.poll_sleep_amount = Float(psleep)
75
71
  end
76
72
 
@@ -6,19 +6,11 @@ module Resque
6
6
  module Lock
7
7
  class Resilient < Base
8
8
  def acquire!
9
- Resque.redis.evalsha(
10
- acquire_sha,
11
- keys: [key],
12
- argv: [value]
13
- ).to_i == 1
9
+ evalsha(:acquire, [key], [value]).to_i == 1
14
10
  end
15
11
 
16
12
  def locked?
17
- Resque.redis.evalsha(
18
- locked_sha,
19
- keys: [key],
20
- argv: [value]
21
- ).to_i == 1
13
+ evalsha(:locked, [key], [value]).to_i == 1
22
14
  end
23
15
 
24
16
  def timeout=(seconds)
@@ -32,11 +24,26 @@ module Resque
32
24
 
33
25
  private
34
26
 
27
+ def evalsha(script, keys, argv, refresh: false)
28
+ sha_method_name = "#{script}_sha"
29
+ Resque.redis.evalsha(
30
+ send(sha_method_name, refresh),
31
+ keys: keys,
32
+ argv: argv
33
+ )
34
+ rescue Redis::CommandError => e
35
+ if e.message =~ /NOSCRIPT/
36
+ refresh = true
37
+ retry
38
+ end
39
+ raise
40
+ end
41
+
35
42
  def locked_sha(refresh = false)
36
43
  @locked_sha = nil if refresh
37
44
 
38
45
  @locked_sha ||=
39
- Resque.redis.script(:load, <<-EOF.gsub(/^ {14}/, ''))
46
+ Resque.data_store.redis.script(:load, <<-EOF.gsub(/^ {14}/, ''))
40
47
  if redis.call('GET', KEYS[1]) == ARGV[1]
41
48
  then
42
49
  redis.call('EXPIRE', KEYS[1], #{timeout})
@@ -55,7 +62,7 @@ module Resque
55
62
  @acquire_sha = nil if refresh
56
63
 
57
64
  @acquire_sha ||=
58
- Resque.redis.script(:load, <<-EOF.gsub(/^ {14}/, ''))
65
+ Resque.data_store.redis.script(:load, <<-EOF.gsub(/^ {14}/, ''))
59
66
  if redis.call('SETNX', KEYS[1], ARGV[1]) == 1
60
67
  then
61
68
  redis.call('EXPIRE', KEYS[1], #{timeout})
@@ -2,7 +2,7 @@
2
2
 
3
3
  # ### Locking the scheduler process
4
4
  #
5
- # There are two places in resque-scheduler that need to be synchonized in order
5
+ # There are two places in resque-scheduler that need to be synchronized in order
6
6
  # to be able to run redundant scheduler processes while ensuring jobs don't get
7
7
  # queued multiple times when the master process changes.
8
8
  #
@@ -76,7 +76,7 @@ module Resque
76
76
 
77
77
  def release_master_lock
78
78
  master_lock.release
79
- rescue Errno::EAGAIN, Errno::ECONNRESET, Redis::CannotConnectError
79
+ rescue *INTERMITTENT_ERRORS
80
80
  @master_lock = nil
81
81
  end
82
82
 
@@ -97,7 +97,7 @@ module Resque
97
97
  end
98
98
 
99
99
  def redis_master_version
100
- Resque.redis.info['redis_version'].to_f
100
+ Resque.data_store.redis.info['redis_version'].to_f
101
101
  end
102
102
  end
103
103
  end
@@ -36,7 +36,7 @@ module Resque
36
36
  # :args can be any yaml which will be converted to a ruby literal and
37
37
  # passed in a params. (optional)
38
38
  #
39
- # :rails_envs is the list of envs where the job gets loaded. Envs are
39
+ # :rails_env is the list of envs where the job gets loaded. Envs are
40
40
  # comma separated (optional)
41
41
  #
42
42
  # :description is just that, a description of the job (optional). If
@@ -101,12 +101,13 @@ module Resque
101
101
  end
102
102
 
103
103
  # remove a given schedule by name
104
- def remove_schedule(name)
104
+ # Preventing a reload is optional and available to batch operations
105
+ def remove_schedule(name, reload = true)
105
106
  non_persistent_schedules.delete(name)
106
107
  redis.hdel(:persistent_schedules, name)
107
108
  redis.sadd(:schedules_changed, name)
108
109
 
109
- reload_schedule!
110
+ reload_schedule! if reload
110
111
  end
111
112
 
112
113
  private