resque-retry 0.2.2 → 1.0.0.a
Sign up to get free protection for your applications and to get access to all the features.
- data/HISTORY.md +16 -11
- data/README.md +15 -8
- data/lib/resque-retry/server.rb +10 -7
- data/lib/resque-retry/server/views/retry.erb +4 -6
- data/lib/resque-retry/server/views/retry_timestamp.erb +1 -1
- data/lib/resque/failure/multiple_with_retry_suppression.rb +3 -3
- data/lib/resque/plugins/exponential_backoff.rb +1 -1
- data/lib/resque/plugins/retry.rb +48 -12
- data/test/multiple_failure_test.rb +41 -5
- data/test/retry_inheriting_checks_test.rb +12 -0
- data/test/retry_test.rb +25 -0
- data/test/test_helper.rb +9 -11
- data/test/test_jobs.rb +83 -2
- metadata +39 -28
data/HISTORY.md
CHANGED
@@ -1,13 +1,21 @@
|
|
1
|
-
## HEAD
|
1
|
+
## HEAD - will become v1.0.0
|
2
2
|
|
3
|
-
|
3
|
+
**INCLUDES NON-BACKWARDS COMPATIBLE CHANGES**
|
4
|
+
|
5
|
+
* Fixed issues related to infinate job retries and v1.20.0 of resque.
|
6
|
+
* Minimum gem dependency versions changed: resque >= 1.10.0, resque-scheduler >= 1.9.9
|
7
|
+
* Feature: Setting `@retry_job_delegate` allows you to seperate the orignal job from a the retry job. (@tanob/@jniesen)
|
8
|
+
* Web interface will work without needing to `require` your job code. (n.b. less details avaialble via web).
|
9
|
+
* IMPORTANT: `#identifier` method has been namedspaced to `#retry_identifier`.
|
10
|
+
* Bugfix: `Remove` button on retry web interface was not working.
|
11
|
+
|
12
|
+
## 0.2.2 (2011-12-08)
|
4
13
|
|
5
14
|
* Feature: Ability to set `retry_delay` per exception type. (Dave Benvenuti)
|
6
15
|
|
7
16
|
## 0.2.1 (2011-11-23)
|
8
17
|
|
9
|
-
* Bugfix: Fixed error when we tried to parse a number/string as JSON on the
|
10
|
-
reque-retry web interface.
|
18
|
+
* Bugfix: Fixed error when we tried to parse a number/string as JSON on the reque-retry web interface.
|
11
19
|
|
12
20
|
## 0.2.0 (2011-11-22)
|
13
21
|
|
@@ -17,10 +25,9 @@
|
|
17
25
|
PREVIOUSLY: 0 == infinite retries.
|
18
26
|
NOW: -1 == infinite retries; 0 == means never retry.
|
19
27
|
|
20
|
-
* Bugfix: `#redis_retry_key` incorrectly built key when custom identifier
|
21
|
-
was used. (Bogdan Gusiev)
|
28
|
+
* Bugfix: `#redis_retry_key` incorrectly built key when custom identifier was used. (Bogdan Gusiev)
|
22
29
|
* Feature: Ability to sleep worker after re-queuing a job, may be used to bias
|
23
|
-
|
30
|
+
against the same worker from picking up the job again. (Michael Keirnan)
|
24
31
|
* Feature: Ability to remove retry jobs using resque-web. (Thiago Morello)
|
25
32
|
* Added example demo application.
|
26
33
|
* Added Bundler `Gemfile`.
|
@@ -50,10 +57,8 @@
|
|
50
57
|
## 0.0.2 (2010-05-06)
|
51
58
|
|
52
59
|
* Bugfix: Were calling non-existent method to delete redis key.
|
53
|
-
* Delay no-longer falls back to `sleep`. resque-scheduler is a required
|
54
|
-
|
55
|
-
* Redis key doesn't include ending colon `:` if no args were passed
|
56
|
-
to the job.
|
60
|
+
* Delay no-longer falls back to `sleep`. resque-scheduler is a required dependancy.
|
61
|
+
* Redis key doesn't include ending colon `:` if no args were passed to the job.
|
57
62
|
|
58
63
|
## 0.0.1 (2010-04-27)
|
59
64
|
|
data/README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
resque-retry
|
2
2
|
============
|
3
3
|
|
4
|
-
A [Resque][rq] plugin. Requires Resque >= 1.
|
4
|
+
A [Resque][rq] plugin. Requires Resque >= 1.10.0 & [resque-scheduler][rqs] >= 1.9.9.
|
5
5
|
|
6
6
|
resque-retry provides retry, delay and exponential backoff support for
|
7
7
|
resque jobs.
|
@@ -12,6 +12,8 @@ resque jobs.
|
|
12
12
|
* Multiple failure backend with retry suppression & resque-web tab.
|
13
13
|
* Small & Extendable - plenty of places to override retry logic/settings.
|
14
14
|
|
15
|
+
[![Build Status](https://secure.travis-ci.org/lantins/resque-retry.png?branch=master)](http://travis-ci.org/lantins/resque-retry)
|
16
|
+
|
15
17
|
Install & Quick Start
|
16
18
|
---------------------
|
17
19
|
|
@@ -19,6 +21,9 @@ To install:
|
|
19
21
|
|
20
22
|
$ gem install resque-retry
|
21
23
|
|
24
|
+
If your using [Bundler][bundler] to manage your dependencies, you should add `gem
|
25
|
+
'resque-retry'` to your projects `Gemfile`.
|
26
|
+
|
22
27
|
Add this to your `Rakefile`:
|
23
28
|
|
24
29
|
require 'resque/tasks'
|
@@ -299,17 +304,17 @@ job arguments, to modify the arguments for the next retry attempt.
|
|
299
304
|
end
|
300
305
|
end
|
301
306
|
|
302
|
-
### Job Identifier/Key
|
307
|
+
### Job Retry Identifier/Key
|
303
308
|
|
304
309
|
The retry attempt is incremented and stored in a Redis key. The key is
|
305
|
-
built using the `
|
306
|
-
ones, you should consider overriding `
|
307
|
-
or loose custom identifier.
|
310
|
+
built using the `retry_identifier`. If you have a lot of arguments or really long
|
311
|
+
ones, you should consider overriding `retry_identifier` to define a more precise
|
312
|
+
or loose custom retry identifier.
|
308
313
|
|
309
|
-
The default identifier is just your job arguments joined with a dash `-`.
|
314
|
+
The default retry identifier is just your job arguments joined with a dash `-`.
|
310
315
|
|
311
316
|
By default the key uses this format:
|
312
|
-
`resque-retry:<job class name>:<
|
317
|
+
`resque-retry:<job class name>:<retry_identifier>`.
|
313
318
|
|
314
319
|
Or you can define the entire key by overriding `redis_retry_key`.
|
315
320
|
|
@@ -317,7 +322,7 @@ Or you can define the entire key by overriding `redis_retry_key`.
|
|
317
322
|
extend Resque::Plugins::Retry
|
318
323
|
@queue = :mt_messages
|
319
324
|
|
320
|
-
def self.
|
325
|
+
def self.retry_identifier(mt_id, mobile_number, message)
|
321
326
|
"#{mobile_number}:#{mt_id}"
|
322
327
|
end
|
323
328
|
|
@@ -340,3 +345,5 @@ Contributing/Pull Requests
|
|
340
345
|
[god]: http://github.com/mojombo/god
|
341
346
|
[rq]: http://github.com/defunkt/resque
|
342
347
|
[rqs]: http://github.com/bvandenbos/resque-scheduler
|
348
|
+
[bundler]: http://gembundler.com/
|
349
|
+
|
data/lib/resque-retry/server.rb
CHANGED
@@ -9,10 +9,14 @@ module ResqueRetry
|
|
9
9
|
helpers do
|
10
10
|
# builds a retry key for the specified job.
|
11
11
|
def retry_key_for_job(job)
|
12
|
-
|
13
|
-
|
14
|
-
klass.redis_retry_key
|
15
|
-
|
12
|
+
begin
|
13
|
+
klass = Resque.constantize(job['class'])
|
14
|
+
if klass.respond_to?(:redis_retry_key)
|
15
|
+
klass.redis_retry_key(job['args'])
|
16
|
+
else
|
17
|
+
nil
|
18
|
+
end
|
19
|
+
rescue NameError
|
16
20
|
nil
|
17
21
|
end
|
18
22
|
end
|
@@ -21,7 +25,7 @@ module ResqueRetry
|
|
21
25
|
def retry_attempts_for_job(job)
|
22
26
|
Resque.redis.get(retry_key_for_job(job))
|
23
27
|
end
|
24
|
-
|
28
|
+
|
25
29
|
# gets the failure details hash for a job.
|
26
30
|
def retry_failure_details(retry_key)
|
27
31
|
Resque.decode(Resque.redis.get("failure_#{retry_key}"))
|
@@ -33,8 +37,7 @@ module ResqueRetry
|
|
33
37
|
File.read(File.join(File.dirname(__FILE__), "server/views/#{path}"))
|
34
38
|
end
|
35
39
|
|
36
|
-
#
|
37
|
-
#
|
40
|
+
# cancels job retry
|
38
41
|
def cancel_retry(job)
|
39
42
|
klass = Resque.constantize(job['class'])
|
40
43
|
retry_key = retry_key_for_job(job)
|
@@ -23,26 +23,24 @@
|
|
23
23
|
<% timestamps.each do |timestamp| %>
|
24
24
|
<% job = resque.delayed_timestamp_peek(timestamp, 0, 1).first %>
|
25
25
|
<% next unless job %>
|
26
|
-
<% retry_key = retry_key_for_job(job) %>
|
27
26
|
<tr>
|
28
27
|
<td>
|
29
|
-
<form action="<%=
|
30
|
-
<input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
|
28
|
+
<form action="<%= u "retry/#{timestamp}/remove" %>" method="post">
|
31
29
|
<input type="submit" value="Remove">
|
32
30
|
</form>
|
33
|
-
<form action="<%=
|
31
|
+
<form action="<%= u "/delayed/queue_now" %>" method="post">
|
34
32
|
<input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
|
35
33
|
<input type="submit" value="Queue now">
|
36
34
|
</form>
|
37
35
|
</td>
|
38
|
-
<td><a href="<%=
|
36
|
+
<td><a href="<%= u "retry/#{timestamp}" %>"><%= format_time(Time.at(timestamp)) %></a></td>
|
39
37
|
<td><%= delayed_timestamp_size = resque.delayed_timestamp_size(timestamp) %></td>
|
40
38
|
<% if job && delayed_timestamp_size == 1 %>
|
41
39
|
<td><%= h job['class'] %></td>
|
42
40
|
<td><%= h job['args'].inspect %></td>
|
43
41
|
<td><%= retry_attempts_for_job(job) || '<i>n/a</i>' %></td>
|
44
42
|
<% else %>
|
45
|
-
<td><a href="<%=
|
43
|
+
<td><a href="<%= u "retry/#{timestamp}" %>">see details</a></td>
|
46
44
|
<td></td>
|
47
45
|
<td></td>
|
48
46
|
<% end %>
|
@@ -51,7 +51,7 @@
|
|
51
51
|
<% end %>
|
52
52
|
<% end %>
|
53
53
|
<td>
|
54
|
-
<form action="<%=
|
54
|
+
<form action="<%= u "retry/#{timestamp}/jobs/#{CGI.escape(Resque.encode(job))}/remove" %>" method="post">
|
55
55
|
<input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
|
56
56
|
<input type="submit" value="Remove">
|
57
57
|
</form>
|
@@ -25,7 +25,7 @@ module Resque
|
|
25
25
|
# Store the lastest failure information in redis, used by the web
|
26
26
|
# interface.
|
27
27
|
def save
|
28
|
-
if !
|
28
|
+
if !(retryable? && retrying?)
|
29
29
|
cleanup_retry_failure_log!
|
30
30
|
super
|
31
31
|
elsif retry_delay > 0
|
@@ -39,13 +39,13 @@ module Resque
|
|
39
39
|
:queue => queue
|
40
40
|
}
|
41
41
|
|
42
|
-
redis.setex(failure_key, 2*retry_delay,
|
42
|
+
redis.setex(failure_key, 2*retry_delay, encode(data))
|
43
43
|
end
|
44
44
|
end
|
45
45
|
|
46
46
|
# Expose this for the hook's use.
|
47
47
|
def self.failure_key(retry_key)
|
48
|
-
'
|
48
|
+
'failure-' + retry_key
|
49
49
|
end
|
50
50
|
|
51
51
|
protected
|
@@ -25,7 +25,7 @@ module Resque
|
|
25
25
|
# @backoff_strategy = [0, 60]
|
26
26
|
#
|
27
27
|
# # used to build redis key, for counting job attempts.
|
28
|
-
# def self.
|
28
|
+
# def self.retry_identifier(mt_id, mobile_number, message)
|
29
29
|
# "#{mobile_number}:#{mt_id}"
|
30
30
|
# end
|
31
31
|
#
|
data/lib/resque/plugins/retry.rb
CHANGED
@@ -23,7 +23,7 @@ module Resque
|
|
23
23
|
# @retry_delay = 60 # default: 0
|
24
24
|
#
|
25
25
|
# # used to build redis key, for counting job attempts.
|
26
|
-
# def self.
|
26
|
+
# def self.retry_identifier(url, hook_id, hmac_key)
|
27
27
|
# "#{url}-#{hook_id}"
|
28
28
|
# end
|
29
29
|
#
|
@@ -40,16 +40,16 @@ module Resque
|
|
40
40
|
subclass.instance_variable_set("@retry_criteria_checks", retry_criteria_checks.dup)
|
41
41
|
end
|
42
42
|
|
43
|
-
# @abstract You may override to implement a custom identifier,
|
43
|
+
# @abstract You may override to implement a custom retry identifier,
|
44
44
|
# you should consider doing this if your job arguments
|
45
|
-
# are many/long or may not cleanly
|
45
|
+
# are many/long or may not cleanly convert to strings.
|
46
46
|
#
|
47
|
-
# Builds
|
47
|
+
# Builds a retry identifier using the job arguments. This identifier
|
48
48
|
# is used as part of the redis key.
|
49
49
|
#
|
50
50
|
# @param [Array] args job arguments
|
51
51
|
# @return [String] job identifier
|
52
|
-
def
|
52
|
+
def retry_identifier(*args)
|
53
53
|
args_string = args.join('-')
|
54
54
|
args_string.empty? ? nil : args_string
|
55
55
|
end
|
@@ -59,7 +59,7 @@ module Resque
|
|
59
59
|
#
|
60
60
|
# @return [String] redis key
|
61
61
|
def redis_retry_key(*args)
|
62
|
-
['resque-retry', name,
|
62
|
+
['resque-retry', name, retry_identifier(*args)].compact.join(":").gsub(/\s/, '')
|
63
63
|
end
|
64
64
|
|
65
65
|
# Maximum number of retrys we can attempt to successfully perform the job.
|
@@ -103,7 +103,11 @@ module Resque
|
|
103
103
|
def sleep_after_requeue
|
104
104
|
@sleep_after_requeue ||= 0
|
105
105
|
end
|
106
|
-
|
106
|
+
|
107
|
+
def retry_job_delegate
|
108
|
+
@retry_job_delegate ||= nil
|
109
|
+
end
|
110
|
+
|
107
111
|
# @abstract
|
108
112
|
# Modify the arguments used to retry the job. Use this to do something
|
109
113
|
# other than try the exact same job again.
|
@@ -149,7 +153,7 @@ module Resque
|
|
149
153
|
|
150
154
|
# call user retry criteria check blocks.
|
151
155
|
retry_criteria_checks.each do |criteria_check|
|
152
|
-
should_retry ||= !!
|
156
|
+
should_retry ||= !!instance_exec(exception, *args, &criteria_check)
|
153
157
|
end
|
154
158
|
|
155
159
|
should_retry
|
@@ -206,12 +210,18 @@ module Resque
|
|
206
210
|
# we'll just check here to see whether it takes the additional exception class argument or not
|
207
211
|
temp_retry_delay = ([-1, 1].include?(method(:retry_delay).arity) ? retry_delay(exception.class) : retry_delay)
|
208
212
|
|
213
|
+
retry_in_queue = retry_job_delegate ? retry_job_delegate : self
|
209
214
|
if temp_retry_delay <= 0
|
210
215
|
# If the delay is 0, no point passing it through the scheduler
|
211
|
-
Resque.enqueue(
|
216
|
+
Resque.enqueue(retry_in_queue, *args_for_retry(*args))
|
212
217
|
else
|
213
|
-
Resque.enqueue_in(temp_retry_delay,
|
218
|
+
Resque.enqueue_in(temp_retry_delay, retry_in_queue, *args_for_retry(*args))
|
214
219
|
end
|
220
|
+
|
221
|
+
# remove retry key from redis if we handed retry off to another queue.
|
222
|
+
clean_retry_key(*args) if retry_job_delegate
|
223
|
+
|
224
|
+
# sleep after requeue if enabled.
|
215
225
|
sleep(sleep_after_requeue) if sleep_after_requeue > 0
|
216
226
|
end
|
217
227
|
|
@@ -219,6 +229,9 @@ module Resque
|
|
219
229
|
#
|
220
230
|
# Increments and sets the `@retry_attempt` count.
|
221
231
|
def before_perform_retry(*args)
|
232
|
+
@on_failure_retry_hook_already_called = false
|
233
|
+
|
234
|
+
# store number of retry attempts.
|
222
235
|
retry_key = redis_retry_key(*args)
|
223
236
|
Resque.redis.setnx(retry_key, -1) # default to -1 if not set.
|
224
237
|
@retry_attempt = Resque.redis.incr(retry_key) # increment by 1.
|
@@ -228,19 +241,42 @@ module Resque
|
|
228
241
|
#
|
229
242
|
# Deletes retry attempt count from Redis.
|
230
243
|
def after_perform_retry(*args)
|
231
|
-
|
244
|
+
clean_retry_key(*args)
|
232
245
|
end
|
233
246
|
|
234
247
|
# Resque on_failure hook.
|
235
248
|
#
|
236
249
|
# Checks if our retry criteria is valid, if it is we try again.
|
237
250
|
# Otherwise the retry attempt count is deleted from Redis.
|
251
|
+
#
|
252
|
+
# NOTE: This hook will only allow execution once per job perform attempt.
|
253
|
+
# This was added because Resque v1.20.0 calls the hook twice.
|
254
|
+
# IMO; this isn't something resque-retry should have to worry about!
|
238
255
|
def on_failure_retry(exception, *args)
|
256
|
+
return if @on_failure_retry_hook_already_called
|
257
|
+
|
239
258
|
if retry_criteria_valid?(exception, *args)
|
240
259
|
try_again(exception, *args)
|
241
260
|
else
|
242
|
-
|
261
|
+
clean_retry_key(*args)
|
262
|
+
end
|
263
|
+
|
264
|
+
@on_failure_retry_hook_already_called = true
|
265
|
+
end
|
266
|
+
|
267
|
+
def instance_exec(*args, &block)
|
268
|
+
mname = "__instance_exec_#{Thread.current.object_id.abs}"
|
269
|
+
class << self; self end.class_eval{ define_method(mname, &block) }
|
270
|
+
begin
|
271
|
+
ret = send(mname, *args)
|
272
|
+
ensure
|
273
|
+
class << self; self end.class_eval{ undef_method(mname) } rescue nil
|
243
274
|
end
|
275
|
+
ret
|
276
|
+
end
|
277
|
+
|
278
|
+
def clean_retry_key(*args)
|
279
|
+
Resque.redis.del(redis_retry_key(*args))
|
244
280
|
end
|
245
281
|
|
246
282
|
end
|
@@ -1,6 +1,7 @@
|
|
1
1
|
require File.expand_path(File.dirname(__FILE__) + '/test_helper')
|
2
2
|
|
3
3
|
class MultipleFailureTest < MiniTest::Unit::TestCase
|
4
|
+
|
4
5
|
def setup
|
5
6
|
Resque.redis.flushall
|
6
7
|
@worker = Resque::Worker.new(:testing)
|
@@ -8,13 +9,26 @@ class MultipleFailureTest < MiniTest::Unit::TestCase
|
|
8
9
|
|
9
10
|
@old_failure_backend = Resque::Failure.backend
|
10
11
|
MockFailureBackend.errors = []
|
11
|
-
Resque::Failure::MultipleWithRetrySuppression.classes = [MockFailureBackend]
|
12
|
+
Resque::Failure::MultipleWithRetrySuppression.classes = [ MockFailureBackend ]
|
12
13
|
Resque::Failure.backend = Resque::Failure::MultipleWithRetrySuppression
|
13
14
|
end
|
14
15
|
|
15
16
|
def failure_key_for(klass)
|
16
17
|
args = []
|
17
|
-
key =
|
18
|
+
key = 'failure-' + klass.redis_retry_key(args)
|
19
|
+
end
|
20
|
+
|
21
|
+
def test_failure_is_passed_on_when_job_class_not_found
|
22
|
+
skip 'commit 7113b0df to `resque` gem means the failure backend is never called'
|
23
|
+
new_job_class = Class.new(LimitThreeJob).tap { |klass| klass.send(:instance_variable_set, :@queue, LimitThreeJob.instance_variable_get(:@queue)) }
|
24
|
+
Object.send(:const_set, 'LimitThreeJobTemp', new_job_class)
|
25
|
+
Resque.enqueue(LimitThreeJobTemp)
|
26
|
+
|
27
|
+
Object.send(:remove_const, 'LimitThreeJobTemp')
|
28
|
+
perform_next_job(@worker)
|
29
|
+
|
30
|
+
assert_equal 1, MockFailureBackend.errors.count, 'should have one error'
|
31
|
+
assert_match /uninitialized constant LimitThreeJobTemp/, MockFailureBackend.errors.first
|
18
32
|
end
|
19
33
|
|
20
34
|
def test_last_failure_is_saved_in_redis_if_delay
|
@@ -26,6 +40,29 @@ class MultipleFailureTest < MiniTest::Unit::TestCase
|
|
26
40
|
assert Resque.redis.exists(key)
|
27
41
|
end
|
28
42
|
|
43
|
+
def test_retry_key_splatting_args
|
44
|
+
# were expecting this to be called twice:
|
45
|
+
# - once before the job is executed.
|
46
|
+
# - once by the failure backend.
|
47
|
+
RetryDefaultsJob.expects(:redis_retry_key).with({'a' => 1, 'b' => 2}).times(2)
|
48
|
+
|
49
|
+
Resque.enqueue(RetryDefaultsJob, {'a' => 1, 'b' => 2})
|
50
|
+
perform_next_job(@worker)
|
51
|
+
end
|
52
|
+
|
53
|
+
def test_last_failure_removed_from_redis_after_error_limit
|
54
|
+
3.times do
|
55
|
+
Resque.enqueue(LimitThreeJobDelay1Hour)
|
56
|
+
perform_next_job(@worker)
|
57
|
+
end
|
58
|
+
|
59
|
+
key = failure_key_for(LimitThreeJobDelay1Hour)
|
60
|
+
assert Resque.redis.exists(key), 'key should still exist'
|
61
|
+
|
62
|
+
Resque.enqueue(LimitThreeJobDelay1Hour)
|
63
|
+
perform_next_job(@worker)
|
64
|
+
assert !Resque.redis.exists(key), 'key should have been removed.'
|
65
|
+
end
|
29
66
|
|
30
67
|
def test_last_failure_has_double_delay_redis_expiry_if_delay
|
31
68
|
Resque.enqueue(LimitThreeJobDelay1Hour)
|
@@ -45,7 +82,6 @@ class MultipleFailureTest < MiniTest::Unit::TestCase
|
|
45
82
|
assert !Resque.redis.exists(key)
|
46
83
|
end
|
47
84
|
|
48
|
-
|
49
85
|
def test_errors_are_suppressed_up_to_retry_limit
|
50
86
|
Resque.enqueue(LimitThreeJob)
|
51
87
|
3.times do
|
@@ -73,8 +109,8 @@ class MultipleFailureTest < MiniTest::Unit::TestCase
|
|
73
109
|
assert_equal 5, MockFailureBackend.errors.size
|
74
110
|
end
|
75
111
|
|
76
|
-
def
|
77
|
-
Resque.enqueue(
|
112
|
+
def test_custom_retry_identifier_job
|
113
|
+
Resque.enqueue(CustomRetryIdentifierFailingJob, 'qq', 2)
|
78
114
|
4.times do
|
79
115
|
perform_next_job(@worker)
|
80
116
|
end
|
@@ -30,4 +30,16 @@ class RetryInheritingChecksTest < MiniTest::Unit::TestCase
|
|
30
30
|
assert_equal 1, klass.retry_criteria_checks.size
|
31
31
|
assert_equal 'test', klass.test_value
|
32
32
|
end
|
33
|
+
|
34
|
+
def test_retry_criteria_check_should_be_evaluated_under_child_context
|
35
|
+
Resque.enqueue(InheritedJob, 'arg')
|
36
|
+
|
37
|
+
10.times do
|
38
|
+
perform_next_job(@worker)
|
39
|
+
end
|
40
|
+
|
41
|
+
assert_equal 0, BaseJob.retry_attempt, "BaseJob retry attempts"
|
42
|
+
assert_equal 0, InheritedJob.retry_attempt, "InheritedJob retry attempts"
|
43
|
+
assert_equal 5, InheritedRetryJob.retry_attempt, "InheritedRetryJob retry attempts"
|
44
|
+
end
|
33
45
|
end
|
data/test/retry_test.rb
CHANGED
@@ -142,6 +142,31 @@ class RetryTest < MiniTest::Unit::TestCase
|
|
142
142
|
assert_equal 0, Resque.info[:pending], 'pending jobs'
|
143
143
|
end
|
144
144
|
|
145
|
+
def test_retry_failed_jobs_in_separate_queue
|
146
|
+
Resque.enqueue(JobWithRetryQueue, 'arg1')
|
147
|
+
|
148
|
+
perform_next_job(@worker)
|
149
|
+
|
150
|
+
assert job_from_retry_queue = Resque.pop(:testing_retry)
|
151
|
+
assert_equal ['arg1'], job_from_retry_queue['args']
|
152
|
+
assert_equal nil, Resque.redis.get(JobWithRetryQueue.redis_retry_key('arg1'))
|
153
|
+
end
|
154
|
+
|
155
|
+
def test_clean_retry_key_should_splat_args
|
156
|
+
JobWithRetryQueue.expects(:clean_retry_key).once.with({"a" => 1, "b" => 2})
|
157
|
+
|
158
|
+
Resque.enqueue(JobWithRetryQueue, {"a" => 1, "b" => 2})
|
159
|
+
|
160
|
+
perform_next_job(@worker)
|
161
|
+
end
|
162
|
+
|
163
|
+
def test_retry_delayed_failed_jobs_in_separate_queue
|
164
|
+
Resque.enqueue(DelayedJobWithRetryQueue, 'arg1')
|
165
|
+
Resque.expects(:enqueue_in).with(1, JobRetryQueue, 'arg1')
|
166
|
+
|
167
|
+
perform_next_job(@worker)
|
168
|
+
end
|
169
|
+
|
145
170
|
def test_delete_redis_key_when_job_is_successful
|
146
171
|
Resque.enqueue(GoodJob, 'arg1')
|
147
172
|
|
data/test/test_helper.rb
CHANGED
@@ -2,20 +2,22 @@ dir = File.dirname(File.expand_path(__FILE__))
|
|
2
2
|
$LOAD_PATH.unshift dir + '/../lib'
|
3
3
|
$TESTING = true
|
4
4
|
|
5
|
-
|
5
|
+
require 'rubygems'
|
6
6
|
require 'minitest/unit'
|
7
7
|
require 'minitest/pride'
|
8
8
|
require 'rack/test'
|
9
|
-
require '
|
9
|
+
require 'mocha'
|
10
10
|
|
11
|
-
|
12
|
-
|
11
|
+
if RUBY_ENGINE == 'ruby' && RUBY_VERSION >= '1.9'
|
12
|
+
require 'simplecov'
|
13
|
+
SimpleCov.start do
|
14
|
+
add_filter '/test/'
|
15
|
+
end
|
13
16
|
end
|
14
17
|
|
15
18
|
require 'resque-retry'
|
16
19
|
require dir + '/test_jobs'
|
17
20
|
|
18
|
-
|
19
21
|
# make sure we can run redis
|
20
22
|
if !system("which redis-server")
|
21
23
|
puts '', "** can't find `redis-server` in your path"
|
@@ -29,16 +31,12 @@ end
|
|
29
31
|
at_exit do
|
30
32
|
next if $!
|
31
33
|
|
32
|
-
|
33
|
-
exit_code = MiniTest::Unit.new.run(ARGV)
|
34
|
-
else
|
35
|
-
exit_code = Test::Unit::AutoRunner.run
|
36
|
-
end
|
34
|
+
exit_code = MiniTest::Unit.new.run(ARGV)
|
37
35
|
|
38
36
|
pid = `ps -e -o pid,command | grep [r]edis-test`.split(" ")[0]
|
39
37
|
puts "Killing test redis server..."
|
40
38
|
`rm -f #{dir}/dump.rdb`
|
41
|
-
|
39
|
+
`kill -9 #{pid}`
|
42
40
|
exit exit_code
|
43
41
|
end
|
44
42
|
|
data/test/test_jobs.rb
CHANGED
@@ -44,6 +44,35 @@ class SleepDelay1SecondJob < RetryDefaultsJob
|
|
44
44
|
end
|
45
45
|
end
|
46
46
|
|
47
|
+
class JobRetryQueue
|
48
|
+
extend Resque::Plugins::Retry
|
49
|
+
@queue = :testing_retry
|
50
|
+
|
51
|
+
def self.perform(*args)
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
class JobWithRetryQueue
|
56
|
+
extend Resque::Plugins::Retry
|
57
|
+
@queue = :testing
|
58
|
+
@retry_job_delegate = JobRetryQueue
|
59
|
+
|
60
|
+
def self.perform(*args)
|
61
|
+
raise
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
65
|
+
class DelayedJobWithRetryQueue
|
66
|
+
extend Resque::Plugins::Retry
|
67
|
+
@queue = :testing
|
68
|
+
@retry_delay = 1
|
69
|
+
@retry_job_delegate = JobRetryQueue
|
70
|
+
|
71
|
+
def self.perform(*args)
|
72
|
+
raise
|
73
|
+
end
|
74
|
+
end
|
75
|
+
|
47
76
|
class InheritTestJob < RetryDefaultsJob
|
48
77
|
end
|
49
78
|
|
@@ -147,6 +176,58 @@ module RetryModuleDefaultsJob
|
|
147
176
|
end
|
148
177
|
end
|
149
178
|
|
179
|
+
class AsyncJob
|
180
|
+
extend Resque::Plugins::Retry
|
181
|
+
|
182
|
+
class << self
|
183
|
+
def perform(*opts)
|
184
|
+
process
|
185
|
+
end
|
186
|
+
|
187
|
+
def process
|
188
|
+
raise "Shouldn't be called"
|
189
|
+
end
|
190
|
+
end
|
191
|
+
end
|
192
|
+
|
193
|
+
class BaseJob < AsyncJob
|
194
|
+
@retry_limit = -1
|
195
|
+
@auto_retry_limit = 5
|
196
|
+
@retry_exceptions = []
|
197
|
+
|
198
|
+
retry_criteria_check do |exception, *args|
|
199
|
+
keep_trying?
|
200
|
+
end
|
201
|
+
|
202
|
+
class << self
|
203
|
+
def keep_trying?
|
204
|
+
retry_attempt < @auto_retry_limit
|
205
|
+
end
|
206
|
+
|
207
|
+
def inherited(subclass)
|
208
|
+
super
|
209
|
+
%w(@retry_exceptions @retry_delay @retry_limit @auto_retry_limit).each do |variable|
|
210
|
+
value = BaseJob.instance_variable_get(variable)
|
211
|
+
value = value.dup rescue value
|
212
|
+
subclass.instance_variable_set(variable, value)
|
213
|
+
end
|
214
|
+
end
|
215
|
+
|
216
|
+
def process
|
217
|
+
raise "Got called #{Time.now}"
|
218
|
+
end
|
219
|
+
end
|
220
|
+
end
|
221
|
+
|
222
|
+
class InheritedRetryJob < BaseJob
|
223
|
+
@queue = :testing
|
224
|
+
end
|
225
|
+
|
226
|
+
class InheritedJob < BaseJob
|
227
|
+
@queue = :testing
|
228
|
+
@retry_job_delegate = InheritedRetryJob
|
229
|
+
end
|
230
|
+
|
150
231
|
module RetryModuleCustomRetryCriteriaCheck
|
151
232
|
extend Resque::Plugins::Retry
|
152
233
|
@queue = :testing
|
@@ -254,14 +335,14 @@ end
|
|
254
335
|
class InheritOrderingJobExtendFirstSubclass < InheritOrderingJobExtendFirst; end
|
255
336
|
class InheritOrderingJobExtendLastSubclass < InheritOrderingJobExtendLast; end
|
256
337
|
|
257
|
-
class
|
338
|
+
class CustomRetryIdentifierFailingJob
|
258
339
|
extend Resque::Plugins::Retry
|
259
340
|
|
260
341
|
@queue = :testing
|
261
342
|
@retry_limit = 2
|
262
343
|
@retry_delay = 0
|
263
344
|
|
264
|
-
def self.
|
345
|
+
def self.retry_identifier(*args)
|
265
346
|
args.first.to_s
|
266
347
|
end
|
267
348
|
|
metadata
CHANGED
@@ -1,8 +1,8 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: resque-retry
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
5
|
-
prerelease:
|
4
|
+
version: 1.0.0.a
|
5
|
+
prerelease: 6
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
8
8
|
- Luke Antins
|
@@ -10,44 +10,44 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2012-03-11 00:00:00.000000000Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
|
-
name:
|
17
|
-
requirement: &
|
16
|
+
name: resque
|
17
|
+
requirement: &2158569780 !ruby/object:Gem::Requirement
|
18
18
|
none: false
|
19
19
|
requirements:
|
20
20
|
- - ! '>='
|
21
21
|
- !ruby/object:Gem::Version
|
22
|
-
version:
|
22
|
+
version: 1.10.0
|
23
23
|
type: :runtime
|
24
24
|
prerelease: false
|
25
|
-
version_requirements: *
|
25
|
+
version_requirements: *2158569780
|
26
26
|
- !ruby/object:Gem::Dependency
|
27
|
-
name: resque
|
28
|
-
requirement: &
|
27
|
+
name: resque-scheduler
|
28
|
+
requirement: &2158568800 !ruby/object:Gem::Requirement
|
29
29
|
none: false
|
30
30
|
requirements:
|
31
31
|
- - ! '>='
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: 1.
|
33
|
+
version: 1.9.9
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
|
-
version_requirements: *
|
36
|
+
version_requirements: *2158568800
|
37
37
|
- !ruby/object:Gem::Dependency
|
38
|
-
name:
|
39
|
-
requirement: &
|
38
|
+
name: rake
|
39
|
+
requirement: &2158568180 !ruby/object:Gem::Requirement
|
40
40
|
none: false
|
41
41
|
requirements:
|
42
42
|
- - ! '>='
|
43
43
|
- !ruby/object:Gem::Version
|
44
|
-
version:
|
45
|
-
type: :
|
44
|
+
version: '0'
|
45
|
+
type: :development
|
46
46
|
prerelease: false
|
47
|
-
version_requirements: *
|
47
|
+
version_requirements: *2158568180
|
48
48
|
- !ruby/object:Gem::Dependency
|
49
49
|
name: minitest
|
50
|
-
requirement: &
|
50
|
+
requirement: &2158567580 !ruby/object:Gem::Requirement
|
51
51
|
none: false
|
52
52
|
requirements:
|
53
53
|
- - ! '>='
|
@@ -55,10 +55,10 @@ dependencies:
|
|
55
55
|
version: '0'
|
56
56
|
type: :development
|
57
57
|
prerelease: false
|
58
|
-
version_requirements: *
|
58
|
+
version_requirements: *2158567580
|
59
59
|
- !ruby/object:Gem::Dependency
|
60
60
|
name: rack-test
|
61
|
-
requirement: &
|
61
|
+
requirement: &2158567120 !ruby/object:Gem::Requirement
|
62
62
|
none: false
|
63
63
|
requirements:
|
64
64
|
- - ! '>='
|
@@ -66,10 +66,10 @@ dependencies:
|
|
66
66
|
version: '0'
|
67
67
|
type: :development
|
68
68
|
prerelease: false
|
69
|
-
version_requirements: *
|
69
|
+
version_requirements: *2158567120
|
70
70
|
- !ruby/object:Gem::Dependency
|
71
71
|
name: yard
|
72
|
-
requirement: &
|
72
|
+
requirement: &2158566580 !ruby/object:Gem::Requirement
|
73
73
|
none: false
|
74
74
|
requirements:
|
75
75
|
- - ! '>='
|
@@ -77,10 +77,10 @@ dependencies:
|
|
77
77
|
version: '0'
|
78
78
|
type: :development
|
79
79
|
prerelease: false
|
80
|
-
version_requirements: *
|
80
|
+
version_requirements: *2158566580
|
81
81
|
- !ruby/object:Gem::Dependency
|
82
82
|
name: json
|
83
|
-
requirement: &
|
83
|
+
requirement: &2158566080 !ruby/object:Gem::Requirement
|
84
84
|
none: false
|
85
85
|
requirements:
|
86
86
|
- - ! '>='
|
@@ -88,10 +88,10 @@ dependencies:
|
|
88
88
|
version: '0'
|
89
89
|
type: :development
|
90
90
|
prerelease: false
|
91
|
-
version_requirements: *
|
91
|
+
version_requirements: *2158566080
|
92
92
|
- !ruby/object:Gem::Dependency
|
93
93
|
name: simplecov
|
94
|
-
requirement: &
|
94
|
+
requirement: &2158565300 !ruby/object:Gem::Requirement
|
95
95
|
none: false
|
96
96
|
requirements:
|
97
97
|
- - ! '>='
|
@@ -99,7 +99,18 @@ dependencies:
|
|
99
99
|
version: 0.3.0
|
100
100
|
type: :development
|
101
101
|
prerelease: false
|
102
|
-
version_requirements: *
|
102
|
+
version_requirements: *2158565300
|
103
|
+
- !ruby/object:Gem::Dependency
|
104
|
+
name: mocha
|
105
|
+
requirement: &2158564740 !ruby/object:Gem::Requirement
|
106
|
+
none: false
|
107
|
+
requirements:
|
108
|
+
- - ! '>='
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '0'
|
111
|
+
type: :development
|
112
|
+
prerelease: false
|
113
|
+
version_requirements: *2158564740
|
103
114
|
description: ! " resque-retry provides retry, delay and exponential backoff support
|
104
115
|
for\n resque jobs.\n\n Features:\n\n * Redis backed retry count/limit.\n * Retry
|
105
116
|
on all or specific exceptions.\n * Exponential backoff (varying the delay between
|
@@ -147,9 +158,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
147
158
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
148
159
|
none: false
|
149
160
|
requirements:
|
150
|
-
- - ! '
|
161
|
+
- - ! '>'
|
151
162
|
- !ruby/object:Gem::Version
|
152
|
-
version:
|
163
|
+
version: 1.3.1
|
153
164
|
requirements: []
|
154
165
|
rubyforge_project:
|
155
166
|
rubygems_version: 1.8.10
|