resque-retry 0.2.2 → 1.0.0.a
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/HISTORY.md +16 -11
- data/README.md +15 -8
- data/lib/resque-retry/server.rb +10 -7
- data/lib/resque-retry/server/views/retry.erb +4 -6
- data/lib/resque-retry/server/views/retry_timestamp.erb +1 -1
- data/lib/resque/failure/multiple_with_retry_suppression.rb +3 -3
- data/lib/resque/plugins/exponential_backoff.rb +1 -1
- data/lib/resque/plugins/retry.rb +48 -12
- data/test/multiple_failure_test.rb +41 -5
- data/test/retry_inheriting_checks_test.rb +12 -0
- data/test/retry_test.rb +25 -0
- data/test/test_helper.rb +9 -11
- data/test/test_jobs.rb +83 -2
- metadata +39 -28
data/HISTORY.md
CHANGED
@@ -1,13 +1,21 @@
|
|
1
|
-
## HEAD
|
1
|
+
## HEAD - will become v1.0.0
|
2
2
|
|
3
|
-
|
3
|
+
**INCLUDES NON-BACKWARDS COMPATIBLE CHANGES**
|
4
|
+
|
5
|
+
* Fixed issues related to infinate job retries and v1.20.0 of resque.
|
6
|
+
* Minimum gem dependency versions changed: resque >= 1.10.0, resque-scheduler >= 1.9.9
|
7
|
+
* Feature: Setting `@retry_job_delegate` allows you to seperate the orignal job from a the retry job. (@tanob/@jniesen)
|
8
|
+
* Web interface will work without needing to `require` your job code. (n.b. less details avaialble via web).
|
9
|
+
* IMPORTANT: `#identifier` method has been namedspaced to `#retry_identifier`.
|
10
|
+
* Bugfix: `Remove` button on retry web interface was not working.
|
11
|
+
|
12
|
+
## 0.2.2 (2011-12-08)
|
4
13
|
|
5
14
|
* Feature: Ability to set `retry_delay` per exception type. (Dave Benvenuti)
|
6
15
|
|
7
16
|
## 0.2.1 (2011-11-23)
|
8
17
|
|
9
|
-
* Bugfix: Fixed error when we tried to parse a number/string as JSON on the
|
10
|
-
reque-retry web interface.
|
18
|
+
* Bugfix: Fixed error when we tried to parse a number/string as JSON on the reque-retry web interface.
|
11
19
|
|
12
20
|
## 0.2.0 (2011-11-22)
|
13
21
|
|
@@ -17,10 +25,9 @@
|
|
17
25
|
PREVIOUSLY: 0 == infinite retries.
|
18
26
|
NOW: -1 == infinite retries; 0 == means never retry.
|
19
27
|
|
20
|
-
* Bugfix: `#redis_retry_key` incorrectly built key when custom identifier
|
21
|
-
was used. (Bogdan Gusiev)
|
28
|
+
* Bugfix: `#redis_retry_key` incorrectly built key when custom identifier was used. (Bogdan Gusiev)
|
22
29
|
* Feature: Ability to sleep worker after re-queuing a job, may be used to bias
|
23
|
-
|
30
|
+
against the same worker from picking up the job again. (Michael Keirnan)
|
24
31
|
* Feature: Ability to remove retry jobs using resque-web. (Thiago Morello)
|
25
32
|
* Added example demo application.
|
26
33
|
* Added Bundler `Gemfile`.
|
@@ -50,10 +57,8 @@
|
|
50
57
|
## 0.0.2 (2010-05-06)
|
51
58
|
|
52
59
|
* Bugfix: Were calling non-existent method to delete redis key.
|
53
|
-
* Delay no-longer falls back to `sleep`. resque-scheduler is a required
|
54
|
-
|
55
|
-
* Redis key doesn't include ending colon `:` if no args were passed
|
56
|
-
to the job.
|
60
|
+
* Delay no-longer falls back to `sleep`. resque-scheduler is a required dependancy.
|
61
|
+
* Redis key doesn't include ending colon `:` if no args were passed to the job.
|
57
62
|
|
58
63
|
## 0.0.1 (2010-04-27)
|
59
64
|
|
data/README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
resque-retry
|
2
2
|
============
|
3
3
|
|
4
|
-
A [Resque][rq] plugin. Requires Resque >= 1.
|
4
|
+
A [Resque][rq] plugin. Requires Resque >= 1.10.0 & [resque-scheduler][rqs] >= 1.9.9.
|
5
5
|
|
6
6
|
resque-retry provides retry, delay and exponential backoff support for
|
7
7
|
resque jobs.
|
@@ -12,6 +12,8 @@ resque jobs.
|
|
12
12
|
* Multiple failure backend with retry suppression & resque-web tab.
|
13
13
|
* Small & Extendable - plenty of places to override retry logic/settings.
|
14
14
|
|
15
|
+
[](http://travis-ci.org/lantins/resque-retry)
|
16
|
+
|
15
17
|
Install & Quick Start
|
16
18
|
---------------------
|
17
19
|
|
@@ -19,6 +21,9 @@ To install:
|
|
19
21
|
|
20
22
|
$ gem install resque-retry
|
21
23
|
|
24
|
+
If your using [Bundler][bundler] to manage your dependencies, you should add `gem
|
25
|
+
'resque-retry'` to your projects `Gemfile`.
|
26
|
+
|
22
27
|
Add this to your `Rakefile`:
|
23
28
|
|
24
29
|
require 'resque/tasks'
|
@@ -299,17 +304,17 @@ job arguments, to modify the arguments for the next retry attempt.
|
|
299
304
|
end
|
300
305
|
end
|
301
306
|
|
302
|
-
### Job Identifier/Key
|
307
|
+
### Job Retry Identifier/Key
|
303
308
|
|
304
309
|
The retry attempt is incremented and stored in a Redis key. The key is
|
305
|
-
built using the `
|
306
|
-
ones, you should consider overriding `
|
307
|
-
or loose custom identifier.
|
310
|
+
built using the `retry_identifier`. If you have a lot of arguments or really long
|
311
|
+
ones, you should consider overriding `retry_identifier` to define a more precise
|
312
|
+
or loose custom retry identifier.
|
308
313
|
|
309
|
-
The default identifier is just your job arguments joined with a dash `-`.
|
314
|
+
The default retry identifier is just your job arguments joined with a dash `-`.
|
310
315
|
|
311
316
|
By default the key uses this format:
|
312
|
-
`resque-retry:<job class name>:<
|
317
|
+
`resque-retry:<job class name>:<retry_identifier>`.
|
313
318
|
|
314
319
|
Or you can define the entire key by overriding `redis_retry_key`.
|
315
320
|
|
@@ -317,7 +322,7 @@ Or you can define the entire key by overriding `redis_retry_key`.
|
|
317
322
|
extend Resque::Plugins::Retry
|
318
323
|
@queue = :mt_messages
|
319
324
|
|
320
|
-
def self.
|
325
|
+
def self.retry_identifier(mt_id, mobile_number, message)
|
321
326
|
"#{mobile_number}:#{mt_id}"
|
322
327
|
end
|
323
328
|
|
@@ -340,3 +345,5 @@ Contributing/Pull Requests
|
|
340
345
|
[god]: http://github.com/mojombo/god
|
341
346
|
[rq]: http://github.com/defunkt/resque
|
342
347
|
[rqs]: http://github.com/bvandenbos/resque-scheduler
|
348
|
+
[bundler]: http://gembundler.com/
|
349
|
+
|
data/lib/resque-retry/server.rb
CHANGED
@@ -9,10 +9,14 @@ module ResqueRetry
|
|
9
9
|
helpers do
|
10
10
|
# builds a retry key for the specified job.
|
11
11
|
def retry_key_for_job(job)
|
12
|
-
|
13
|
-
|
14
|
-
klass.redis_retry_key
|
15
|
-
|
12
|
+
begin
|
13
|
+
klass = Resque.constantize(job['class'])
|
14
|
+
if klass.respond_to?(:redis_retry_key)
|
15
|
+
klass.redis_retry_key(job['args'])
|
16
|
+
else
|
17
|
+
nil
|
18
|
+
end
|
19
|
+
rescue NameError
|
16
20
|
nil
|
17
21
|
end
|
18
22
|
end
|
@@ -21,7 +25,7 @@ module ResqueRetry
|
|
21
25
|
def retry_attempts_for_job(job)
|
22
26
|
Resque.redis.get(retry_key_for_job(job))
|
23
27
|
end
|
24
|
-
|
28
|
+
|
25
29
|
# gets the failure details hash for a job.
|
26
30
|
def retry_failure_details(retry_key)
|
27
31
|
Resque.decode(Resque.redis.get("failure_#{retry_key}"))
|
@@ -33,8 +37,7 @@ module ResqueRetry
|
|
33
37
|
File.read(File.join(File.dirname(__FILE__), "server/views/#{path}"))
|
34
38
|
end
|
35
39
|
|
36
|
-
#
|
37
|
-
#
|
40
|
+
# cancels job retry
|
38
41
|
def cancel_retry(job)
|
39
42
|
klass = Resque.constantize(job['class'])
|
40
43
|
retry_key = retry_key_for_job(job)
|
@@ -23,26 +23,24 @@
|
|
23
23
|
<% timestamps.each do |timestamp| %>
|
24
24
|
<% job = resque.delayed_timestamp_peek(timestamp, 0, 1).first %>
|
25
25
|
<% next unless job %>
|
26
|
-
<% retry_key = retry_key_for_job(job) %>
|
27
26
|
<tr>
|
28
27
|
<td>
|
29
|
-
<form action="<%=
|
30
|
-
<input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
|
28
|
+
<form action="<%= u "retry/#{timestamp}/remove" %>" method="post">
|
31
29
|
<input type="submit" value="Remove">
|
32
30
|
</form>
|
33
|
-
<form action="<%=
|
31
|
+
<form action="<%= u "/delayed/queue_now" %>" method="post">
|
34
32
|
<input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
|
35
33
|
<input type="submit" value="Queue now">
|
36
34
|
</form>
|
37
35
|
</td>
|
38
|
-
<td><a href="<%=
|
36
|
+
<td><a href="<%= u "retry/#{timestamp}" %>"><%= format_time(Time.at(timestamp)) %></a></td>
|
39
37
|
<td><%= delayed_timestamp_size = resque.delayed_timestamp_size(timestamp) %></td>
|
40
38
|
<% if job && delayed_timestamp_size == 1 %>
|
41
39
|
<td><%= h job['class'] %></td>
|
42
40
|
<td><%= h job['args'].inspect %></td>
|
43
41
|
<td><%= retry_attempts_for_job(job) || '<i>n/a</i>' %></td>
|
44
42
|
<% else %>
|
45
|
-
<td><a href="<%=
|
43
|
+
<td><a href="<%= u "retry/#{timestamp}" %>">see details</a></td>
|
46
44
|
<td></td>
|
47
45
|
<td></td>
|
48
46
|
<% end %>
|
@@ -51,7 +51,7 @@
|
|
51
51
|
<% end %>
|
52
52
|
<% end %>
|
53
53
|
<td>
|
54
|
-
<form action="<%=
|
54
|
+
<form action="<%= u "retry/#{timestamp}/jobs/#{CGI.escape(Resque.encode(job))}/remove" %>" method="post">
|
55
55
|
<input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
|
56
56
|
<input type="submit" value="Remove">
|
57
57
|
</form>
|
@@ -25,7 +25,7 @@ module Resque
|
|
25
25
|
# Store the lastest failure information in redis, used by the web
|
26
26
|
# interface.
|
27
27
|
def save
|
28
|
-
if !
|
28
|
+
if !(retryable? && retrying?)
|
29
29
|
cleanup_retry_failure_log!
|
30
30
|
super
|
31
31
|
elsif retry_delay > 0
|
@@ -39,13 +39,13 @@ module Resque
|
|
39
39
|
:queue => queue
|
40
40
|
}
|
41
41
|
|
42
|
-
redis.setex(failure_key, 2*retry_delay,
|
42
|
+
redis.setex(failure_key, 2*retry_delay, encode(data))
|
43
43
|
end
|
44
44
|
end
|
45
45
|
|
46
46
|
# Expose this for the hook's use.
|
47
47
|
def self.failure_key(retry_key)
|
48
|
-
'
|
48
|
+
'failure-' + retry_key
|
49
49
|
end
|
50
50
|
|
51
51
|
protected
|
@@ -25,7 +25,7 @@ module Resque
|
|
25
25
|
# @backoff_strategy = [0, 60]
|
26
26
|
#
|
27
27
|
# # used to build redis key, for counting job attempts.
|
28
|
-
# def self.
|
28
|
+
# def self.retry_identifier(mt_id, mobile_number, message)
|
29
29
|
# "#{mobile_number}:#{mt_id}"
|
30
30
|
# end
|
31
31
|
#
|
data/lib/resque/plugins/retry.rb
CHANGED
@@ -23,7 +23,7 @@ module Resque
|
|
23
23
|
# @retry_delay = 60 # default: 0
|
24
24
|
#
|
25
25
|
# # used to build redis key, for counting job attempts.
|
26
|
-
# def self.
|
26
|
+
# def self.retry_identifier(url, hook_id, hmac_key)
|
27
27
|
# "#{url}-#{hook_id}"
|
28
28
|
# end
|
29
29
|
#
|
@@ -40,16 +40,16 @@ module Resque
|
|
40
40
|
subclass.instance_variable_set("@retry_criteria_checks", retry_criteria_checks.dup)
|
41
41
|
end
|
42
42
|
|
43
|
-
# @abstract You may override to implement a custom identifier,
|
43
|
+
# @abstract You may override to implement a custom retry identifier,
|
44
44
|
# you should consider doing this if your job arguments
|
45
|
-
# are many/long or may not cleanly
|
45
|
+
# are many/long or may not cleanly convert to strings.
|
46
46
|
#
|
47
|
-
# Builds
|
47
|
+
# Builds a retry identifier using the job arguments. This identifier
|
48
48
|
# is used as part of the redis key.
|
49
49
|
#
|
50
50
|
# @param [Array] args job arguments
|
51
51
|
# @return [String] job identifier
|
52
|
-
def
|
52
|
+
def retry_identifier(*args)
|
53
53
|
args_string = args.join('-')
|
54
54
|
args_string.empty? ? nil : args_string
|
55
55
|
end
|
@@ -59,7 +59,7 @@ module Resque
|
|
59
59
|
#
|
60
60
|
# @return [String] redis key
|
61
61
|
def redis_retry_key(*args)
|
62
|
-
['resque-retry', name,
|
62
|
+
['resque-retry', name, retry_identifier(*args)].compact.join(":").gsub(/\s/, '')
|
63
63
|
end
|
64
64
|
|
65
65
|
# Maximum number of retrys we can attempt to successfully perform the job.
|
@@ -103,7 +103,11 @@ module Resque
|
|
103
103
|
def sleep_after_requeue
|
104
104
|
@sleep_after_requeue ||= 0
|
105
105
|
end
|
106
|
-
|
106
|
+
|
107
|
+
def retry_job_delegate
|
108
|
+
@retry_job_delegate ||= nil
|
109
|
+
end
|
110
|
+
|
107
111
|
# @abstract
|
108
112
|
# Modify the arguments used to retry the job. Use this to do something
|
109
113
|
# other than try the exact same job again.
|
@@ -149,7 +153,7 @@ module Resque
|
|
149
153
|
|
150
154
|
# call user retry criteria check blocks.
|
151
155
|
retry_criteria_checks.each do |criteria_check|
|
152
|
-
should_retry ||= !!
|
156
|
+
should_retry ||= !!instance_exec(exception, *args, &criteria_check)
|
153
157
|
end
|
154
158
|
|
155
159
|
should_retry
|
@@ -206,12 +210,18 @@ module Resque
|
|
206
210
|
# we'll just check here to see whether it takes the additional exception class argument or not
|
207
211
|
temp_retry_delay = ([-1, 1].include?(method(:retry_delay).arity) ? retry_delay(exception.class) : retry_delay)
|
208
212
|
|
213
|
+
retry_in_queue = retry_job_delegate ? retry_job_delegate : self
|
209
214
|
if temp_retry_delay <= 0
|
210
215
|
# If the delay is 0, no point passing it through the scheduler
|
211
|
-
Resque.enqueue(
|
216
|
+
Resque.enqueue(retry_in_queue, *args_for_retry(*args))
|
212
217
|
else
|
213
|
-
Resque.enqueue_in(temp_retry_delay,
|
218
|
+
Resque.enqueue_in(temp_retry_delay, retry_in_queue, *args_for_retry(*args))
|
214
219
|
end
|
220
|
+
|
221
|
+
# remove retry key from redis if we handed retry off to another queue.
|
222
|
+
clean_retry_key(*args) if retry_job_delegate
|
223
|
+
|
224
|
+
# sleep after requeue if enabled.
|
215
225
|
sleep(sleep_after_requeue) if sleep_after_requeue > 0
|
216
226
|
end
|
217
227
|
|
@@ -219,6 +229,9 @@ module Resque
|
|
219
229
|
#
|
220
230
|
# Increments and sets the `@retry_attempt` count.
|
221
231
|
def before_perform_retry(*args)
|
232
|
+
@on_failure_retry_hook_already_called = false
|
233
|
+
|
234
|
+
# store number of retry attempts.
|
222
235
|
retry_key = redis_retry_key(*args)
|
223
236
|
Resque.redis.setnx(retry_key, -1) # default to -1 if not set.
|
224
237
|
@retry_attempt = Resque.redis.incr(retry_key) # increment by 1.
|
@@ -228,19 +241,42 @@ module Resque
|
|
228
241
|
#
|
229
242
|
# Deletes retry attempt count from Redis.
|
230
243
|
def after_perform_retry(*args)
|
231
|
-
|
244
|
+
clean_retry_key(*args)
|
232
245
|
end
|
233
246
|
|
234
247
|
# Resque on_failure hook.
|
235
248
|
#
|
236
249
|
# Checks if our retry criteria is valid, if it is we try again.
|
237
250
|
# Otherwise the retry attempt count is deleted from Redis.
|
251
|
+
#
|
252
|
+
# NOTE: This hook will only allow execution once per job perform attempt.
|
253
|
+
# This was added because Resque v1.20.0 calls the hook twice.
|
254
|
+
# IMO; this isn't something resque-retry should have to worry about!
|
238
255
|
def on_failure_retry(exception, *args)
|
256
|
+
return if @on_failure_retry_hook_already_called
|
257
|
+
|
239
258
|
if retry_criteria_valid?(exception, *args)
|
240
259
|
try_again(exception, *args)
|
241
260
|
else
|
242
|
-
|
261
|
+
clean_retry_key(*args)
|
262
|
+
end
|
263
|
+
|
264
|
+
@on_failure_retry_hook_already_called = true
|
265
|
+
end
|
266
|
+
|
267
|
+
def instance_exec(*args, &block)
|
268
|
+
mname = "__instance_exec_#{Thread.current.object_id.abs}"
|
269
|
+
class << self; self end.class_eval{ define_method(mname, &block) }
|
270
|
+
begin
|
271
|
+
ret = send(mname, *args)
|
272
|
+
ensure
|
273
|
+
class << self; self end.class_eval{ undef_method(mname) } rescue nil
|
243
274
|
end
|
275
|
+
ret
|
276
|
+
end
|
277
|
+
|
278
|
+
def clean_retry_key(*args)
|
279
|
+
Resque.redis.del(redis_retry_key(*args))
|
244
280
|
end
|
245
281
|
|
246
282
|
end
|
@@ -1,6 +1,7 @@
|
|
1
1
|
require File.expand_path(File.dirname(__FILE__) + '/test_helper')
|
2
2
|
|
3
3
|
class MultipleFailureTest < MiniTest::Unit::TestCase
|
4
|
+
|
4
5
|
def setup
|
5
6
|
Resque.redis.flushall
|
6
7
|
@worker = Resque::Worker.new(:testing)
|
@@ -8,13 +9,26 @@ class MultipleFailureTest < MiniTest::Unit::TestCase
|
|
8
9
|
|
9
10
|
@old_failure_backend = Resque::Failure.backend
|
10
11
|
MockFailureBackend.errors = []
|
11
|
-
Resque::Failure::MultipleWithRetrySuppression.classes = [MockFailureBackend]
|
12
|
+
Resque::Failure::MultipleWithRetrySuppression.classes = [ MockFailureBackend ]
|
12
13
|
Resque::Failure.backend = Resque::Failure::MultipleWithRetrySuppression
|
13
14
|
end
|
14
15
|
|
15
16
|
def failure_key_for(klass)
|
16
17
|
args = []
|
17
|
-
key =
|
18
|
+
key = 'failure-' + klass.redis_retry_key(args)
|
19
|
+
end
|
20
|
+
|
21
|
+
def test_failure_is_passed_on_when_job_class_not_found
|
22
|
+
skip 'commit 7113b0df to `resque` gem means the failure backend is never called'
|
23
|
+
new_job_class = Class.new(LimitThreeJob).tap { |klass| klass.send(:instance_variable_set, :@queue, LimitThreeJob.instance_variable_get(:@queue)) }
|
24
|
+
Object.send(:const_set, 'LimitThreeJobTemp', new_job_class)
|
25
|
+
Resque.enqueue(LimitThreeJobTemp)
|
26
|
+
|
27
|
+
Object.send(:remove_const, 'LimitThreeJobTemp')
|
28
|
+
perform_next_job(@worker)
|
29
|
+
|
30
|
+
assert_equal 1, MockFailureBackend.errors.count, 'should have one error'
|
31
|
+
assert_match /uninitialized constant LimitThreeJobTemp/, MockFailureBackend.errors.first
|
18
32
|
end
|
19
33
|
|
20
34
|
def test_last_failure_is_saved_in_redis_if_delay
|
@@ -26,6 +40,29 @@ class MultipleFailureTest < MiniTest::Unit::TestCase
|
|
26
40
|
assert Resque.redis.exists(key)
|
27
41
|
end
|
28
42
|
|
43
|
+
def test_retry_key_splatting_args
|
44
|
+
# were expecting this to be called twice:
|
45
|
+
# - once before the job is executed.
|
46
|
+
# - once by the failure backend.
|
47
|
+
RetryDefaultsJob.expects(:redis_retry_key).with({'a' => 1, 'b' => 2}).times(2)
|
48
|
+
|
49
|
+
Resque.enqueue(RetryDefaultsJob, {'a' => 1, 'b' => 2})
|
50
|
+
perform_next_job(@worker)
|
51
|
+
end
|
52
|
+
|
53
|
+
def test_last_failure_removed_from_redis_after_error_limit
|
54
|
+
3.times do
|
55
|
+
Resque.enqueue(LimitThreeJobDelay1Hour)
|
56
|
+
perform_next_job(@worker)
|
57
|
+
end
|
58
|
+
|
59
|
+
key = failure_key_for(LimitThreeJobDelay1Hour)
|
60
|
+
assert Resque.redis.exists(key), 'key should still exist'
|
61
|
+
|
62
|
+
Resque.enqueue(LimitThreeJobDelay1Hour)
|
63
|
+
perform_next_job(@worker)
|
64
|
+
assert !Resque.redis.exists(key), 'key should have been removed.'
|
65
|
+
end
|
29
66
|
|
30
67
|
def test_last_failure_has_double_delay_redis_expiry_if_delay
|
31
68
|
Resque.enqueue(LimitThreeJobDelay1Hour)
|
@@ -45,7 +82,6 @@ class MultipleFailureTest < MiniTest::Unit::TestCase
|
|
45
82
|
assert !Resque.redis.exists(key)
|
46
83
|
end
|
47
84
|
|
48
|
-
|
49
85
|
def test_errors_are_suppressed_up_to_retry_limit
|
50
86
|
Resque.enqueue(LimitThreeJob)
|
51
87
|
3.times do
|
@@ -73,8 +109,8 @@ class MultipleFailureTest < MiniTest::Unit::TestCase
|
|
73
109
|
assert_equal 5, MockFailureBackend.errors.size
|
74
110
|
end
|
75
111
|
|
76
|
-
def
|
77
|
-
Resque.enqueue(
|
112
|
+
def test_custom_retry_identifier_job
|
113
|
+
Resque.enqueue(CustomRetryIdentifierFailingJob, 'qq', 2)
|
78
114
|
4.times do
|
79
115
|
perform_next_job(@worker)
|
80
116
|
end
|
@@ -30,4 +30,16 @@ class RetryInheritingChecksTest < MiniTest::Unit::TestCase
|
|
30
30
|
assert_equal 1, klass.retry_criteria_checks.size
|
31
31
|
assert_equal 'test', klass.test_value
|
32
32
|
end
|
33
|
+
|
34
|
+
def test_retry_criteria_check_should_be_evaluated_under_child_context
|
35
|
+
Resque.enqueue(InheritedJob, 'arg')
|
36
|
+
|
37
|
+
10.times do
|
38
|
+
perform_next_job(@worker)
|
39
|
+
end
|
40
|
+
|
41
|
+
assert_equal 0, BaseJob.retry_attempt, "BaseJob retry attempts"
|
42
|
+
assert_equal 0, InheritedJob.retry_attempt, "InheritedJob retry attempts"
|
43
|
+
assert_equal 5, InheritedRetryJob.retry_attempt, "InheritedRetryJob retry attempts"
|
44
|
+
end
|
33
45
|
end
|
data/test/retry_test.rb
CHANGED
@@ -142,6 +142,31 @@ class RetryTest < MiniTest::Unit::TestCase
|
|
142
142
|
assert_equal 0, Resque.info[:pending], 'pending jobs'
|
143
143
|
end
|
144
144
|
|
145
|
+
def test_retry_failed_jobs_in_separate_queue
|
146
|
+
Resque.enqueue(JobWithRetryQueue, 'arg1')
|
147
|
+
|
148
|
+
perform_next_job(@worker)
|
149
|
+
|
150
|
+
assert job_from_retry_queue = Resque.pop(:testing_retry)
|
151
|
+
assert_equal ['arg1'], job_from_retry_queue['args']
|
152
|
+
assert_equal nil, Resque.redis.get(JobWithRetryQueue.redis_retry_key('arg1'))
|
153
|
+
end
|
154
|
+
|
155
|
+
def test_clean_retry_key_should_splat_args
|
156
|
+
JobWithRetryQueue.expects(:clean_retry_key).once.with({"a" => 1, "b" => 2})
|
157
|
+
|
158
|
+
Resque.enqueue(JobWithRetryQueue, {"a" => 1, "b" => 2})
|
159
|
+
|
160
|
+
perform_next_job(@worker)
|
161
|
+
end
|
162
|
+
|
163
|
+
def test_retry_delayed_failed_jobs_in_separate_queue
|
164
|
+
Resque.enqueue(DelayedJobWithRetryQueue, 'arg1')
|
165
|
+
Resque.expects(:enqueue_in).with(1, JobRetryQueue, 'arg1')
|
166
|
+
|
167
|
+
perform_next_job(@worker)
|
168
|
+
end
|
169
|
+
|
145
170
|
def test_delete_redis_key_when_job_is_successful
|
146
171
|
Resque.enqueue(GoodJob, 'arg1')
|
147
172
|
|
data/test/test_helper.rb
CHANGED
@@ -2,20 +2,22 @@ dir = File.dirname(File.expand_path(__FILE__))
|
|
2
2
|
$LOAD_PATH.unshift dir + '/../lib'
|
3
3
|
$TESTING = true
|
4
4
|
|
5
|
-
|
5
|
+
require 'rubygems'
|
6
6
|
require 'minitest/unit'
|
7
7
|
require 'minitest/pride'
|
8
8
|
require 'rack/test'
|
9
|
-
require '
|
9
|
+
require 'mocha'
|
10
10
|
|
11
|
-
|
12
|
-
|
11
|
+
if RUBY_ENGINE == 'ruby' && RUBY_VERSION >= '1.9'
|
12
|
+
require 'simplecov'
|
13
|
+
SimpleCov.start do
|
14
|
+
add_filter '/test/'
|
15
|
+
end
|
13
16
|
end
|
14
17
|
|
15
18
|
require 'resque-retry'
|
16
19
|
require dir + '/test_jobs'
|
17
20
|
|
18
|
-
|
19
21
|
# make sure we can run redis
|
20
22
|
if !system("which redis-server")
|
21
23
|
puts '', "** can't find `redis-server` in your path"
|
@@ -29,16 +31,12 @@ end
|
|
29
31
|
at_exit do
|
30
32
|
next if $!
|
31
33
|
|
32
|
-
|
33
|
-
exit_code = MiniTest::Unit.new.run(ARGV)
|
34
|
-
else
|
35
|
-
exit_code = Test::Unit::AutoRunner.run
|
36
|
-
end
|
34
|
+
exit_code = MiniTest::Unit.new.run(ARGV)
|
37
35
|
|
38
36
|
pid = `ps -e -o pid,command | grep [r]edis-test`.split(" ")[0]
|
39
37
|
puts "Killing test redis server..."
|
40
38
|
`rm -f #{dir}/dump.rdb`
|
41
|
-
|
39
|
+
`kill -9 #{pid}`
|
42
40
|
exit exit_code
|
43
41
|
end
|
44
42
|
|
data/test/test_jobs.rb
CHANGED
@@ -44,6 +44,35 @@ class SleepDelay1SecondJob < RetryDefaultsJob
|
|
44
44
|
end
|
45
45
|
end
|
46
46
|
|
47
|
+
class JobRetryQueue
|
48
|
+
extend Resque::Plugins::Retry
|
49
|
+
@queue = :testing_retry
|
50
|
+
|
51
|
+
def self.perform(*args)
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
class JobWithRetryQueue
|
56
|
+
extend Resque::Plugins::Retry
|
57
|
+
@queue = :testing
|
58
|
+
@retry_job_delegate = JobRetryQueue
|
59
|
+
|
60
|
+
def self.perform(*args)
|
61
|
+
raise
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
65
|
+
class DelayedJobWithRetryQueue
|
66
|
+
extend Resque::Plugins::Retry
|
67
|
+
@queue = :testing
|
68
|
+
@retry_delay = 1
|
69
|
+
@retry_job_delegate = JobRetryQueue
|
70
|
+
|
71
|
+
def self.perform(*args)
|
72
|
+
raise
|
73
|
+
end
|
74
|
+
end
|
75
|
+
|
47
76
|
class InheritTestJob < RetryDefaultsJob
|
48
77
|
end
|
49
78
|
|
@@ -147,6 +176,58 @@ module RetryModuleDefaultsJob
|
|
147
176
|
end
|
148
177
|
end
|
149
178
|
|
179
|
+
class AsyncJob
|
180
|
+
extend Resque::Plugins::Retry
|
181
|
+
|
182
|
+
class << self
|
183
|
+
def perform(*opts)
|
184
|
+
process
|
185
|
+
end
|
186
|
+
|
187
|
+
def process
|
188
|
+
raise "Shouldn't be called"
|
189
|
+
end
|
190
|
+
end
|
191
|
+
end
|
192
|
+
|
193
|
+
class BaseJob < AsyncJob
|
194
|
+
@retry_limit = -1
|
195
|
+
@auto_retry_limit = 5
|
196
|
+
@retry_exceptions = []
|
197
|
+
|
198
|
+
retry_criteria_check do |exception, *args|
|
199
|
+
keep_trying?
|
200
|
+
end
|
201
|
+
|
202
|
+
class << self
|
203
|
+
def keep_trying?
|
204
|
+
retry_attempt < @auto_retry_limit
|
205
|
+
end
|
206
|
+
|
207
|
+
def inherited(subclass)
|
208
|
+
super
|
209
|
+
%w(@retry_exceptions @retry_delay @retry_limit @auto_retry_limit).each do |variable|
|
210
|
+
value = BaseJob.instance_variable_get(variable)
|
211
|
+
value = value.dup rescue value
|
212
|
+
subclass.instance_variable_set(variable, value)
|
213
|
+
end
|
214
|
+
end
|
215
|
+
|
216
|
+
def process
|
217
|
+
raise "Got called #{Time.now}"
|
218
|
+
end
|
219
|
+
end
|
220
|
+
end
|
221
|
+
|
222
|
+
class InheritedRetryJob < BaseJob
|
223
|
+
@queue = :testing
|
224
|
+
end
|
225
|
+
|
226
|
+
class InheritedJob < BaseJob
|
227
|
+
@queue = :testing
|
228
|
+
@retry_job_delegate = InheritedRetryJob
|
229
|
+
end
|
230
|
+
|
150
231
|
module RetryModuleCustomRetryCriteriaCheck
|
151
232
|
extend Resque::Plugins::Retry
|
152
233
|
@queue = :testing
|
@@ -254,14 +335,14 @@ end
|
|
254
335
|
class InheritOrderingJobExtendFirstSubclass < InheritOrderingJobExtendFirst; end
|
255
336
|
class InheritOrderingJobExtendLastSubclass < InheritOrderingJobExtendLast; end
|
256
337
|
|
257
|
-
class
|
338
|
+
class CustomRetryIdentifierFailingJob
|
258
339
|
extend Resque::Plugins::Retry
|
259
340
|
|
260
341
|
@queue = :testing
|
261
342
|
@retry_limit = 2
|
262
343
|
@retry_delay = 0
|
263
344
|
|
264
|
-
def self.
|
345
|
+
def self.retry_identifier(*args)
|
265
346
|
args.first.to_s
|
266
347
|
end
|
267
348
|
|
metadata
CHANGED
@@ -1,8 +1,8 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: resque-retry
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
5
|
-
prerelease:
|
4
|
+
version: 1.0.0.a
|
5
|
+
prerelease: 6
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
8
8
|
- Luke Antins
|
@@ -10,44 +10,44 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2012-03-11 00:00:00.000000000Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
|
-
name:
|
17
|
-
requirement: &
|
16
|
+
name: resque
|
17
|
+
requirement: &2158569780 !ruby/object:Gem::Requirement
|
18
18
|
none: false
|
19
19
|
requirements:
|
20
20
|
- - ! '>='
|
21
21
|
- !ruby/object:Gem::Version
|
22
|
-
version:
|
22
|
+
version: 1.10.0
|
23
23
|
type: :runtime
|
24
24
|
prerelease: false
|
25
|
-
version_requirements: *
|
25
|
+
version_requirements: *2158569780
|
26
26
|
- !ruby/object:Gem::Dependency
|
27
|
-
name: resque
|
28
|
-
requirement: &
|
27
|
+
name: resque-scheduler
|
28
|
+
requirement: &2158568800 !ruby/object:Gem::Requirement
|
29
29
|
none: false
|
30
30
|
requirements:
|
31
31
|
- - ! '>='
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: 1.
|
33
|
+
version: 1.9.9
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
|
-
version_requirements: *
|
36
|
+
version_requirements: *2158568800
|
37
37
|
- !ruby/object:Gem::Dependency
|
38
|
-
name:
|
39
|
-
requirement: &
|
38
|
+
name: rake
|
39
|
+
requirement: &2158568180 !ruby/object:Gem::Requirement
|
40
40
|
none: false
|
41
41
|
requirements:
|
42
42
|
- - ! '>='
|
43
43
|
- !ruby/object:Gem::Version
|
44
|
-
version:
|
45
|
-
type: :
|
44
|
+
version: '0'
|
45
|
+
type: :development
|
46
46
|
prerelease: false
|
47
|
-
version_requirements: *
|
47
|
+
version_requirements: *2158568180
|
48
48
|
- !ruby/object:Gem::Dependency
|
49
49
|
name: minitest
|
50
|
-
requirement: &
|
50
|
+
requirement: &2158567580 !ruby/object:Gem::Requirement
|
51
51
|
none: false
|
52
52
|
requirements:
|
53
53
|
- - ! '>='
|
@@ -55,10 +55,10 @@ dependencies:
|
|
55
55
|
version: '0'
|
56
56
|
type: :development
|
57
57
|
prerelease: false
|
58
|
-
version_requirements: *
|
58
|
+
version_requirements: *2158567580
|
59
59
|
- !ruby/object:Gem::Dependency
|
60
60
|
name: rack-test
|
61
|
-
requirement: &
|
61
|
+
requirement: &2158567120 !ruby/object:Gem::Requirement
|
62
62
|
none: false
|
63
63
|
requirements:
|
64
64
|
- - ! '>='
|
@@ -66,10 +66,10 @@ dependencies:
|
|
66
66
|
version: '0'
|
67
67
|
type: :development
|
68
68
|
prerelease: false
|
69
|
-
version_requirements: *
|
69
|
+
version_requirements: *2158567120
|
70
70
|
- !ruby/object:Gem::Dependency
|
71
71
|
name: yard
|
72
|
-
requirement: &
|
72
|
+
requirement: &2158566580 !ruby/object:Gem::Requirement
|
73
73
|
none: false
|
74
74
|
requirements:
|
75
75
|
- - ! '>='
|
@@ -77,10 +77,10 @@ dependencies:
|
|
77
77
|
version: '0'
|
78
78
|
type: :development
|
79
79
|
prerelease: false
|
80
|
-
version_requirements: *
|
80
|
+
version_requirements: *2158566580
|
81
81
|
- !ruby/object:Gem::Dependency
|
82
82
|
name: json
|
83
|
-
requirement: &
|
83
|
+
requirement: &2158566080 !ruby/object:Gem::Requirement
|
84
84
|
none: false
|
85
85
|
requirements:
|
86
86
|
- - ! '>='
|
@@ -88,10 +88,10 @@ dependencies:
|
|
88
88
|
version: '0'
|
89
89
|
type: :development
|
90
90
|
prerelease: false
|
91
|
-
version_requirements: *
|
91
|
+
version_requirements: *2158566080
|
92
92
|
- !ruby/object:Gem::Dependency
|
93
93
|
name: simplecov
|
94
|
-
requirement: &
|
94
|
+
requirement: &2158565300 !ruby/object:Gem::Requirement
|
95
95
|
none: false
|
96
96
|
requirements:
|
97
97
|
- - ! '>='
|
@@ -99,7 +99,18 @@ dependencies:
|
|
99
99
|
version: 0.3.0
|
100
100
|
type: :development
|
101
101
|
prerelease: false
|
102
|
-
version_requirements: *
|
102
|
+
version_requirements: *2158565300
|
103
|
+
- !ruby/object:Gem::Dependency
|
104
|
+
name: mocha
|
105
|
+
requirement: &2158564740 !ruby/object:Gem::Requirement
|
106
|
+
none: false
|
107
|
+
requirements:
|
108
|
+
- - ! '>='
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '0'
|
111
|
+
type: :development
|
112
|
+
prerelease: false
|
113
|
+
version_requirements: *2158564740
|
103
114
|
description: ! " resque-retry provides retry, delay and exponential backoff support
|
104
115
|
for\n resque jobs.\n\n Features:\n\n * Redis backed retry count/limit.\n * Retry
|
105
116
|
on all or specific exceptions.\n * Exponential backoff (varying the delay between
|
@@ -147,9 +158,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
147
158
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
148
159
|
none: false
|
149
160
|
requirements:
|
150
|
-
- - ! '
|
161
|
+
- - ! '>'
|
151
162
|
- !ruby/object:Gem::Version
|
152
|
-
version:
|
163
|
+
version: 1.3.1
|
153
164
|
requirements: []
|
154
165
|
rubyforge_project:
|
155
166
|
rubygems_version: 1.8.10
|