resque-retry 0.1.0 → 0.2.1

Sign up to get free protection for your applications and to get access to all the features.
data/HISTORY.md CHANGED
@@ -1,3 +1,24 @@
1
+ ## 0.2.1 (2011-11-23)
2
+
3
+ * Bugfix: Fixed error when we tried to parse a number/string as JSON on the
4
+ reque-retry web interface.
5
+
6
+ ## 0.2.0 (2011-11-22)
7
+
8
+ **INCLUDES NON-BACKWARDS COMPATIBLE CHANGES**
9
+
10
+ * IMPORTANT: `retry_limit` behaviour has changed. (Nicolas Fouché)
11
+ PREVIOUSLY: 0 == infinite retries.
12
+ NOW: -1 == infinite retries; 0 == means never retry.
13
+
14
+ * Bugfix: `#redis_retry_key` incorrectly built key when custom identifier
15
+ was used. (Bogdan Gusiev)
16
+ * Feature: Ability to sleep worker after re-queuing a job, may be used to bias
17
+ against the same worker from picking up the job again. (Michael Keirnan)
18
+ * Feature: Ability to remove retry jobs using resque-web. (Thiago Morello)
19
+ * Added example demo application.
20
+ * Added Bundler `Gemfile`.
21
+
1
22
  ## 0.1.0 (2010-08-29)
2
23
 
3
24
  * Feature: Multiple failure backend with retry suppression.
@@ -30,4 +51,4 @@
30
51
 
31
52
  ## 0.0.1 (2010-04-27)
32
53
 
33
- * First release.
54
+ * First release.
data/README.md CHANGED
@@ -1,7 +1,7 @@
1
1
  resque-retry
2
2
  ============
3
3
 
4
- A [Resque][rq] plugin. Requires Resque 1.8.0 & [resque-scheduler][rqs].
4
+ A [Resque][rq] plugin. Requires Resque >= 1.8.0 & [resque-scheduler][rqs].
5
5
 
6
6
  resque-retry provides retry, delay and exponential backoff support for
7
7
  resque jobs.
@@ -19,7 +19,7 @@ To install:
19
19
 
20
20
  $ gem install resque-retry
21
21
 
22
- You'll want add this to your `Rakefile`:
22
+ Add this to your `Rakefile`:
23
23
 
24
24
  require 'resque/tasks'
25
25
  require 'resque_scheduler/tasks'
@@ -118,7 +118,7 @@ some ideas =), adapt for your own usage and feel free to pick and mix!
118
118
 
119
119
  Retry the job **once** on failure, with zero delay.
120
120
 
121
- require 'require-retry'
121
+ require 'resque-retry'
122
122
 
123
123
  class DeliverWebHook
124
124
  extend Resque::Plugins::Retry
@@ -147,12 +147,37 @@ determine if we can requeue the job for another go.
147
147
  end
148
148
  end
149
149
 
150
- The above modification will allow your job to retry upto 10 times, with
150
+ The above modification will allow your job to retry up to 10 times, with
151
151
  a delay of 120 seconds, or 2 minutes between retry attempts.
152
152
 
153
153
  Alternatively you could override the `retry_delay` method to do something
154
154
  more special.
155
155
 
156
+ ### Sleep After Requeuing
157
+
158
+ Sometimes it is useful to delay the worker that failed a job attempt, but still requeue the job for immediate processing
159
+ by other workers. This can be done with `@sleep_after_requeue`:
160
+
161
+ class DeliverWebHook
162
+ extend Resque::Plugins::Retry
163
+ @queue = :web_hooks
164
+
165
+ @sleep_after_requeue = 5
166
+
167
+ def self.perform(url, hook_id, hmac_key)
168
+ heavy_lifting
169
+ end
170
+ end
171
+
172
+ This retries the job once and causes the worker that failed to sleep for 5 seconds after requeuing the job. If there are
173
+ multiple workers in the system this allows the job to be retried immediately while the original worker heals itself. For
174
+ example failed jobs may cause other (non-worker) OS processes to die. A system monitor such as [god][god] can fix
175
+ the server while the job is being retried on a different worker.
176
+
177
+ `@sleep_after_requeue` is independent of `@retry_delay`. If you set both, they both take effect.
178
+
179
+ You can override the method `sleep_after_requeue` to set the sleep value dynamically.
180
+
156
181
  ### Exponential Backoff
157
182
 
158
183
  Use this if you wish to vary the delay between retry attempts:
@@ -176,7 +201,7 @@ Use this if you wish to vary the delay between retry attempts:
176
201
  The first delay will be 0 seconds, the 2nd will be 60 seconds, etc...
177
202
  Again, tweak to your own needs.
178
203
 
179
- The number if retrys is equal to the size of the `backoff_strategy`
204
+ The number of retries is equal to the size of the `backoff_strategy`
180
205
  array, unless you set `retry_limit` yourself.
181
206
 
182
207
  ### Retry Specific Exceptions
@@ -285,5 +310,6 @@ Contributing/Pull Requests
285
310
  * Send me a pull request. Bonus points for topic branches.
286
311
  * If you edit the gemspec/version etc, do it in another commit please.
287
312
 
313
+ [god]: http://github.com/mojombo/god
288
314
  [rq]: http://github.com/defunkt/resque
289
315
  [rqs]: http://github.com/bvandenbos/resque-scheduler
@@ -3,4 +3,4 @@ require 'resque_scheduler'
3
3
 
4
4
  require 'resque/plugins/retry'
5
5
  require 'resque/plugins/exponential_backoff'
6
- require 'resque/failure/multiple_with_retry_suppression'
6
+ require 'resque/failure/multiple_with_retry_suppression'
@@ -1,3 +1,5 @@
1
+ require 'cgi'
2
+
1
3
  # Extend Resque::Server to add tabs.
2
4
  module ResqueRetry
3
5
  module Server
@@ -22,7 +24,7 @@ module ResqueRetry
22
24
 
23
25
  # gets the failure details hash for a job.
24
26
  def retry_failure_details(retry_key)
25
- Resque.decode(Resque.redis["failure_#{retry_key}"])
27
+ Resque.decode(Resque.redis.get("failure_#{retry_key}"))
26
28
  end
27
29
 
28
30
  # reads a 'local' template file.
@@ -30,6 +32,16 @@ module ResqueRetry
30
32
  # Is there a better way to specify alternate template locations with sinatra?
31
33
  File.read(File.join(File.dirname(__FILE__), "server/views/#{path}"))
32
34
  end
35
+
36
+ # Cancels job retry
37
+ #
38
+ def cancel_retry(job)
39
+ klass = Resque.constantize(job['class'])
40
+ retry_key = retry_key_for_job(job)
41
+ Resque.remove_delayed(klass, *job['args'])
42
+ Resque.redis.del("failure_#{retry_key}")
43
+ Resque.redis.del(retry_key)
44
+ end
33
45
  end
34
46
 
35
47
  get '/retry' do
@@ -39,6 +51,19 @@ module ResqueRetry
39
51
  get '/retry/:timestamp' do
40
52
  erb local_template('retry_timestamp.erb')
41
53
  end
54
+
55
+ post '/retry/:timestamp/remove' do
56
+ Resque.delayed_timestamp_peek(params[:timestamp], 0, 0).each do |job|
57
+ cancel_retry(job)
58
+ end
59
+ redirect u('retry')
60
+ end
61
+
62
+ post '/retry/:timestamp/jobs/:id/remove' do
63
+ job = Resque.decode(params[:id])
64
+ cancel_retry(job)
65
+ redirect u("retry/#{params[:timestamp]}")
66
+ end
42
67
  }
43
68
  end
44
69
 
@@ -22,9 +22,14 @@
22
22
  <% timestamps = resque.delayed_queue_peek(start, start+20) %>
23
23
  <% timestamps.each do |timestamp| %>
24
24
  <% job = resque.delayed_timestamp_peek(timestamp, 0, 1).first %>
25
+ <% next unless job %>
25
26
  <% retry_key = retry_key_for_job(job) %>
26
27
  <tr>
27
28
  <td>
29
+ <form action="<%= url "retry/#{timestamp}/remove" %>" method="post">
30
+ <input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
31
+ <input type="submit" value="Remove">
32
+ </form>
28
33
  <form action="<%= url "/delayed/queue_now" %>" method="post">
29
34
  <input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
30
35
  <input type="submit" value="Queue now">
@@ -45,4 +50,4 @@
45
50
  <% end %>
46
51
  </table>
47
52
 
48
- <%= partial :next_more, :start => start, :size => size %>
53
+ <%= partial :next_more, :start => start, :size => size %>
@@ -22,6 +22,7 @@
22
22
  <th>Retry Attempts</th>
23
23
  <th>Exception</th>
24
24
  <th>Backtrace</th>
25
+ <th>Actions</th>
25
26
  </tr>
26
27
  <% jobs = resque.delayed_timestamp_peek(timestamp, start, 20) %>
27
28
  <% jobs.each do |job| %>
@@ -35,21 +36,31 @@
35
36
  <% else %>
36
37
  <td><%= retry_attempts %></td>
37
38
  <% failure = retry_failure_details(retry_key) %>
38
- <td><code><%= failure['exception'] %></code></td>
39
- <td class="error">
40
- <% if failure['backtrace'] %>
41
- <a href="#" class="backtrace"><%= h(failure['error']) %></a>
42
- <pre style="display:none"><%= h failure['backtrace'].join("\n") %></pre>
43
- <% else %>
44
- <%= h failure['error'] %>
45
- <% end %>
46
- </td>
39
+ <% if failure.nil? %>
40
+ <td colspan="2"><i>n/a - not using resque-retry failure backend</i></td>
41
+ <% else %>
42
+ <td><code><%= failure['exception'] %></code></td>
43
+ <td class="error">
44
+ <% if failure['backtrace'] %>
45
+ <a href="#" class="backtrace"><%= h(failure['error']) %></a>
46
+ <pre style="display:none"><%= h failure['backtrace'].join("\n") %></pre>
47
+ <% else %>
48
+ <%= h failure['error'] %>
49
+ <% end %>
50
+ </td>
51
+ <% end %>
47
52
  <% end %>
53
+ <td>
54
+ <form action="<%= url "retry/#{timestamp}/jobs/#{CGI.escape(Resque.encode(job))}/remove" %>" method="post">
55
+ <input type="hidden" name="timestamp" value="<%= timestamp.to_i %>">
56
+ <input type="submit" value="Remove">
57
+ </form>
58
+ </td>
48
59
  </tr>
49
60
  <% end %>
50
61
  <% if jobs.empty? %>
51
62
  <tr>
52
- <td class="no-data" colspan="5">
63
+ <td class="no-data" colspan="6">
53
64
  There are no pending jobs scheduled for this time.
54
65
  </td>
55
66
  </tr>
@@ -19,27 +19,16 @@ module Resque
19
19
  class MultipleWithRetrySuppression < Multiple
20
20
  include Resque::Helpers
21
21
 
22
- module CleanupHooks
23
- # Resque after_perform hook.
24
- #
25
- # Deletes retry failure information from Redis.
26
- def after_perform_retry_failure_cleanup(*args)
27
- retry_key = redis_retry_key(*args)
28
- failure_key = Resque::Failure::MultipleWithRetrySuppression.failure_key(retry_key)
29
- Resque.redis.del(failure_key)
30
- end
31
- end
32
-
33
22
  # Called when the job fails.
34
23
  #
35
24
  # If the job will retry, suppress the failure from the other backends.
36
25
  # Store the lastest failure information in redis, used by the web
37
26
  # interface.
38
27
  def save
39
- unless retryable? && retrying?
28
+ if ! (retryable? && retrying?)
40
29
  cleanup_retry_failure_log!
41
30
  super
42
- else
31
+ elsif retry_delay > 0
43
32
  data = {
44
33
  :failed_at => Time.now.strftime("%Y/%m/%d %H:%M:%S"),
45
34
  :payload => payload,
@@ -50,12 +39,7 @@ module Resque
50
39
  :queue => queue
51
40
  }
52
41
 
53
- # Register cleanup hooks.
54
- unless klass.respond_to?(:after_perform_retry_failure_cleanup)
55
- klass.send(:extend, CleanupHooks)
56
- end
57
-
58
- redis[failure_key] = Resque.encode(data)
42
+ redis.setex(failure_key, 2*retry_delay, Resque.encode(data))
59
43
  end
60
44
  end
61
45
 
@@ -69,8 +53,12 @@ module Resque
69
53
  constantize(payload['class'])
70
54
  end
71
55
 
56
+ def retry_delay
57
+ klass.retry_delay
58
+ end
59
+
72
60
  def retry_key
73
- klass.redis_retry_key(payload['args'])
61
+ klass.redis_retry_key(*payload['args'])
74
62
  end
75
63
 
76
64
  def failure_key
@@ -79,6 +67,8 @@ module Resque
79
67
 
80
68
  def retryable?
81
69
  klass.respond_to?(:redis_retry_key)
70
+ rescue NameError
71
+ false
82
72
  end
83
73
 
84
74
  def retrying?
@@ -63,7 +63,9 @@ module Resque
63
63
  end
64
64
 
65
65
  # Maximum number of retrys we can attempt to successfully perform the job.
66
- # A retry limit of 0 or below will retry forever.
66
+ #
67
+ # A retry limit of 0 will *never* retry.
68
+ # A retry limit of -1 or below will retry forever.
67
69
  #
68
70
  # @return [Fixnum]
69
71
  def retry_limit
@@ -88,6 +90,14 @@ module Resque
88
90
  @retry_delay ||= 0
89
91
  end
90
92
 
93
+ # @abstract
94
+ # Number of seconds to sleep after job is requeued
95
+ #
96
+ # @return [Number] number of seconds to sleep
97
+ def sleep_after_requeue
98
+ @sleep_after_requeue ||= 0
99
+ end
100
+
91
101
  # @abstract
92
102
  # Modify the arguments used to retry the job. Use this to do something
93
103
  # other than try the exact same job again.
@@ -147,10 +157,13 @@ module Resque
147
157
  #
148
158
  # @return [Boolean]
149
159
  def retry_limit_reached?
150
- if retry_limit > 0
151
- return true if retry_attempt >= retry_limit
160
+ if retry_limit == 0
161
+ true
162
+ elsif retry_limit > 0
163
+ true if retry_attempt >= retry_limit
164
+ else
165
+ false
152
166
  end
153
- false
154
167
  end
155
168
 
156
169
  # Register a retry criteria check callback to be run before retrying
@@ -177,7 +190,7 @@ module Resque
177
190
  retry_criteria_checks << block
178
191
  end
179
192
 
180
- # Will retry the job.
193
+ # Retries the job.
181
194
  def try_again(*args)
182
195
  if retry_delay <= 0
183
196
  # If the delay is 0, no point passing it through the scheduler
@@ -185,6 +198,7 @@ module Resque
185
198
  else
186
199
  Resque.enqueue_in(retry_delay, self, *args_for_retry(*args))
187
200
  end
201
+ sleep(sleep_after_requeue) if sleep_after_requeue > 0
188
202
  end
189
203
 
190
204
  # Resque before_perform hook.
@@ -1,6 +1,6 @@
1
1
  require File.dirname(__FILE__) + '/test_helper'
2
2
 
3
- class ExponentialBackoffTest < Test::Unit::TestCase
3
+ class ExponentialBackoffTest < MiniTest::Unit::TestCase
4
4
  def setup
5
5
  Resque.redis.flushall
6
6
  @worker = Resque::Worker.new(:testing)
@@ -8,9 +8,8 @@ class ExponentialBackoffTest < Test::Unit::TestCase
8
8
  end
9
9
 
10
10
  def test_resque_plugin_lint
11
- assert_nothing_raised do
12
- Resque::Plugin.lint(Resque::Plugins::ExponentialBackoff)
13
- end
11
+ # will raise exception if were not a good plugin.
12
+ assert Resque::Plugin.lint(Resque::Plugins::ExponentialBackoff)
14
13
  end
15
14
 
16
15
  def test_default_backoff_strategy
@@ -19,7 +18,7 @@ class ExponentialBackoffTest < Test::Unit::TestCase
19
18
 
20
19
  perform_next_job @worker
21
20
  assert_equal 1, Resque.info[:processed], '1 processed job'
22
- assert_equal 1, Resque.info[:failed], 'first ever run, and it should of failed, but never retried'
21
+ assert_equal 1, Resque.info[:failed], 'first ever run, and it should have failed, but never retried'
23
22
  assert_equal 1, Resque.info[:pending], '1 pending job, because it never hits the scheduler'
24
23
 
25
24
  perform_next_job @worker
@@ -1,6 +1,6 @@
1
1
  require File.dirname(__FILE__) + '/test_helper'
2
2
 
3
- class MultipleFailureTest < Test::Unit::TestCase
3
+ class MultipleFailureTest < MiniTest::Unit::TestCase
4
4
  def setup
5
5
  Resque.redis.flushall
6
6
  @worker = Resque::Worker.new(:testing)
@@ -17,42 +17,35 @@ class MultipleFailureTest < Test::Unit::TestCase
17
17
  key = "failure_" + klass.redis_retry_key(args)
18
18
  end
19
19
 
20
- def test_last_failure_is_saved_in_redis
21
- Resque.enqueue(LimitThreeJob)
20
+ def test_last_failure_is_saved_in_redis_if_delay
21
+ Resque.enqueue(LimitThreeJobDelay1Hour)
22
22
  perform_next_job(@worker)
23
23
 
24
24
  # I don't like this, but...
25
- key = failure_key_for(LimitThreeJob)
25
+ key = failure_key_for(LimitThreeJobDelay1Hour)
26
26
  assert Resque.redis.exists(key)
27
27
  end
28
28
 
29
- def test_last_failure_removed_from_redis_after_error_limit
30
- Resque.enqueue(LimitThreeJob)
31
- 3.times do
32
- perform_next_job(@worker)
33
- end
34
-
35
- key = failure_key_for(LimitThreeJob)
36
- assert Resque.redis.exists(key)
37
29
 
30
+ def test_last_failure_has_double_delay_redis_expiry_if_delay
31
+ Resque.enqueue(LimitThreeJobDelay1Hour)
38
32
  perform_next_job(@worker)
39
- assert !Resque.redis.exists(key)
33
+
34
+ # I don't like this, but...
35
+ key = failure_key_for(LimitThreeJobDelay1Hour)
36
+ assert_equal 7200, Resque.redis.ttl(key)
40
37
  end
41
38
 
42
- def test_on_success_failure_log_removed_from_redis
43
- SwitchToSuccessJob.successful_after = 1
44
- Resque.enqueue(SwitchToSuccessJob)
39
+ def test_last_failure_is_not_saved_in_redis_if_no_delay
40
+ Resque.enqueue(LimitThreeJob)
45
41
  perform_next_job(@worker)
46
42
 
47
- key = failure_key_for(SwitchToSuccessJob)
48
- assert Resque.redis.exists(key)
49
-
50
- perform_next_job(@worker)
51
- assert !Resque.redis.exists(key), 'key removed on success'
52
- ensure
53
- SwitchToSuccessJob.reset_defaults
43
+ # I don't like this, but...
44
+ key = failure_key_for(LimitThreeJob)
45
+ assert !Resque.redis.exists(key)
54
46
  end
55
47
 
48
+
56
49
  def test_errors_are_suppressed_up_to_retry_limit
57
50
  Resque.enqueue(LimitThreeJob)
58
51
  3.times do
@@ -80,6 +73,14 @@ class MultipleFailureTest < Test::Unit::TestCase
80
73
  assert_equal 5, MockFailureBackend.errors.size
81
74
  end
82
75
 
76
+ def test_custom_identifier_job
77
+ Resque.enqueue(CustomIdentifierFailingJob, 'qq', 2)
78
+ 4.times do
79
+ perform_next_job(@worker)
80
+ end
81
+ assert_equal 1, MockFailureBackend.errors.size
82
+ end
83
+
83
84
  def teardown
84
85
  Resque::Failure.backend = @old_failure_backend
85
86
  end
@@ -112,21 +112,4 @@ databases 16
112
112
  # Glue small output buffers together in order to send small replies in a
113
113
  # single TCP packet. Uses a bit more CPU but most of the times it is a win
114
114
  # in terms of number of queries per second. Use 'yes' if unsure.
115
- glueoutputbuf yes
116
-
117
- # Use object sharing. Can save a lot of memory if you have many common
118
- # string in your dataset, but performs lookups against the shared objects
119
- # pool so it uses more CPU and can be a bit slower. Usually it's a good
120
- # idea.
121
- #
122
- # When object sharing is enabled (shareobjects yes) you can use
123
- # shareobjectspoolsize to control the size of the pool used in order to try
124
- # object sharing. A bigger pool size will lead to better sharing capabilities.
125
- # In general you want this value to be at least the double of the number of
126
- # very common strings you have in your dataset.
127
- #
128
- # WARNING: object sharing is experimental, don't enable this feature
129
- # in production before of Redis 1.0-stable. Still please try this feature in
130
- # your development environment so that we can test it better.
131
- # shareobjects no
132
- # shareobjectspoolsize 1024
115
+ glueoutputbuf yes
@@ -1,7 +1,7 @@
1
1
  require File.dirname(__FILE__) + '/test_helper'
2
2
 
3
3
  # make sure the worlds not fallen from beneith us.
4
- class ResqueTest < Test::Unit::TestCase
4
+ class ResqueTest < MiniTest::Unit::TestCase
5
5
  def test_resque_version
6
6
  major, minor, patch = Resque::Version.split('.')
7
7
  assert_equal 1, major.to_i, 'major version does not match'
@@ -1,6 +1,6 @@
1
1
  require File.dirname(__FILE__) + '/test_helper'
2
2
 
3
- class RetryCriteriaTest < Test::Unit::TestCase
3
+ class RetryCriteriaTest < MiniTest::Unit::TestCase
4
4
  def setup
5
5
  Resque.redis.flushall
6
6
  @worker = Resque::Worker.new(:testing)
@@ -1,6 +1,6 @@
1
1
  require File.dirname(__FILE__) + '/test_helper'
2
2
 
3
- class RetryInheritingChecksTest < Test::Unit::TestCase
3
+ class RetryInheritingChecksTest < MiniTest::Unit::TestCase
4
4
  def setup
5
5
  Resque.redis.flushall
6
6
  @worker = Resque::Worker.new(:testing)
@@ -1,6 +1,6 @@
1
1
  require File.dirname(__FILE__) + '/test_helper'
2
2
 
3
- class RetryTest < Test::Unit::TestCase
3
+ class RetryTest < MiniTest::Unit::TestCase
4
4
  def setup
5
5
  Resque.redis.flushall
6
6
  @worker = Resque::Worker.new(:testing)
@@ -8,16 +8,15 @@ class RetryTest < Test::Unit::TestCase
8
8
  end
9
9
 
10
10
  def test_resque_plugin_lint
11
- assert_nothing_raised do
12
- Resque::Plugin.lint(Resque::Plugins::Retry)
13
- end
11
+ # will raise exception if were not a good plugin.
12
+ assert Resque::Plugin.lint(Resque::Plugins::Retry)
14
13
  end
15
14
 
16
15
  def test_default_settings
17
- assert_equal 1, RetryDefaultsJob.retry_limit, 'default retry limit'
18
- assert_equal 0, RetryDefaultsJob.retry_attempt, 'default number of retry attempts'
19
- assert_equal nil, RetryDefaultsJob.retry_exceptions, 'default retry exceptions; nil = any'
20
- assert_equal 0, RetryDefaultsJob.retry_delay, 'default seconds until retry'
16
+ assert_equal 1, RetryDefaultSettingsJob.retry_limit, 'default retry limit'
17
+ assert_equal 0, RetryDefaultSettingsJob.retry_attempt, 'default number of retry attempts'
18
+ assert_equal nil, RetryDefaultSettingsJob.retry_exceptions, 'default retry exceptions; nil = any'
19
+ assert_equal 0, RetryDefaultSettingsJob.retry_delay, 'default seconds until retry'
21
20
  end
22
21
 
23
22
  def test_retry_once_by_default
@@ -71,6 +70,17 @@ class RetryTest < Test::Unit::TestCase
71
70
  assert_equal 10, Resque.info[:processed], 'processed job'
72
71
  end
73
72
 
73
+ def test_retry_never_retry
74
+ Resque.enqueue(NeverRetryJob)
75
+ 10.times do
76
+ perform_next_job(@worker)
77
+ end
78
+
79
+ assert_equal 0, Resque.info[:pending], 'pending jobs'
80
+ assert_equal 1, Resque.info[:failed], 'failed jobs'
81
+ assert_equal 1, Resque.info[:processed], 'processed job'
82
+ end
83
+
74
84
  def test_fail_five_times_then_succeed
75
85
  Resque.enqueue(FailFiveTimesJob)
76
86
  7.times do
@@ -82,6 +92,22 @@ class RetryTest < Test::Unit::TestCase
82
92
  assert_equal 0, Resque.info[:pending], 'pending jobs'
83
93
  end
84
94
 
95
+ def test_retry_delay_sleep
96
+ assert_equal 0, Resque.info[:failed], 'failed jobs'
97
+ Resque.enqueue(SleepDelay1SecondJob)
98
+ before = Time.now
99
+ 3.times do
100
+ perform_next_job(@worker)
101
+ end
102
+ actual_delay = Time.now - before
103
+
104
+ assert actual_delay >= 1, "did not sleep long enough: #{actual_delay} seconds"
105
+ assert actual_delay < 2, "slept too long: #{actual_delay} seconds"
106
+ assert_equal 1, Resque.info[:failed], 'failed jobs'
107
+ assert_equal 2, Resque.info[:processed], 'processed job'
108
+ assert_equal 0, Resque.info[:pending], 'pending jobs'
109
+ end
110
+
85
111
  def test_can_determine_if_exception_may_be_retried
86
112
  assert_equal true, RetryDefaultsJob.retry_exception?(StandardError), 'StandardError may retry'
87
113
  assert_equal true, RetryDefaultsJob.retry_exception?(CustomException), 'CustomException may retry'
@@ -0,0 +1,25 @@
1
+ require File.dirname(__FILE__) + '/test_helper'
2
+
3
+ require 'resque-retry/server'
4
+ ENV['RACK_ENV'] = 'test'
5
+
6
+ # Testing the Resque web interface additions.
7
+ class ServerTest < MiniTest::Unit::TestCase
8
+ include Rack::Test::Methods
9
+
10
+ def app
11
+ Resque::Server
12
+ end
13
+
14
+ # make sure the world looks okay first =)
15
+ def test_were_able_to_access_normal_resque_web_overview
16
+ get '/overview'
17
+ assert_equal 200, last_response.status, 'HTTP status code should be 200'
18
+ end
19
+
20
+ def test_should_include_retry_tab
21
+ get '/overview'
22
+ assert last_response.body.include?('/retry')
23
+ end
24
+
25
+ end
@@ -2,10 +2,10 @@ dir = File.dirname(File.expand_path(__FILE__))
2
2
  $LOAD_PATH.unshift dir + '/../lib'
3
3
  $TESTING = true
4
4
 
5
- require 'test/unit'
6
- require 'rubygems'
7
- require 'turn'
8
- require 'simplecov-html'
5
+ require 'minitest/unit'
6
+ require 'minitest/pride'
7
+ require 'rack/test'
8
+ require 'simplecov'
9
9
 
10
10
  SimpleCov.start do
11
11
  add_filter "/test/"
@@ -59,7 +59,7 @@ class MockFailureBackend < Resque::Failure::Base
59
59
  end
60
60
 
61
61
  # Test helpers
62
- class Test::Unit::TestCase
62
+ class MiniTest::Unit::TestCase
63
63
  def perform_next_job(worker, &block)
64
64
  return unless job = @worker.reserve
65
65
  @worker.perform(job, &block)
@@ -13,6 +13,15 @@ end
13
13
  class GoodJob
14
14
  extend Resque::Plugins::Retry
15
15
  @queue = :testing
16
+
17
+ def self.perform(*args)
18
+ end
19
+ end
20
+
21
+ class RetryDefaultSettingsJob
22
+ extend Resque::Plugins::Retry
23
+ @queue = :testing
24
+
16
25
  def self.perform(*args)
17
26
  end
18
27
  end
@@ -26,6 +35,15 @@ class RetryDefaultsJob
26
35
  end
27
36
  end
28
37
 
38
+ class SleepDelay1SecondJob < RetryDefaultsJob
39
+ @queue = :testing
40
+ @sleep_after_requeue = 1
41
+
42
+ def self.perform(*args)
43
+ raise if retry_attempt == 0
44
+ end
45
+ end
46
+
29
47
  class InheritTestJob < RetryDefaultsJob
30
48
  end
31
49
 
@@ -54,6 +72,11 @@ class RetryWithModifiedArgsJob < RetryDefaultsJob
54
72
  end
55
73
 
56
74
  class NeverGiveUpJob < RetryDefaultsJob
75
+ @queue = :testing
76
+ @retry_limit = -1
77
+ end
78
+
79
+ class NeverRetryJob < RetryDefaultsJob
57
80
  @queue = :testing
58
81
  @retry_limit = 0
59
82
  end
@@ -67,6 +90,12 @@ class LimitThreeJob < RetryDefaultsJob
67
90
  end
68
91
  end
69
92
 
93
+ class LimitThreeJobDelay1Hour < LimitThreeJob
94
+ @queue = :testing
95
+ @retry_limit = 3
96
+ @retry_delay = 3600
97
+ end
98
+
70
99
  class FailFiveTimesJob < RetryDefaultsJob
71
100
  @queue = :testing
72
101
  @retry_limit = 6
@@ -204,32 +233,6 @@ class InheritOrderingJobExtendFirst
204
233
  end
205
234
  end
206
235
 
207
- # This job switches to successful after
208
- # n +tries+.
209
- class SwitchToSuccessJob < GoodJob
210
- @queue = :testing
211
- @max_retries = 3
212
-
213
- class << self
214
- attr_accessor :successful_after
215
- attr_accessor :tries
216
-
217
- def reset_defaults
218
- self.tries = 0
219
- self.successful_after = 2
220
- end
221
- end
222
-
223
- reset_defaults
224
-
225
- def self.perform(*args)
226
- if self.tries < self.successful_after
227
- self.tries += 1
228
- raise "error"
229
- end
230
- end
231
- end
232
-
233
236
  class InheritOrderingJobExtendLast
234
237
  class << self
235
238
  attr_accessor :test_value
@@ -247,5 +250,22 @@ class InheritOrderingJobExtendLast
247
250
  end
248
251
  end
249
252
 
253
+
250
254
  class InheritOrderingJobExtendFirstSubclass < InheritOrderingJobExtendFirst; end
251
255
  class InheritOrderingJobExtendLastSubclass < InheritOrderingJobExtendLast; end
256
+
257
+ class CustomIdentifierFailingJob
258
+ extend Resque::Plugins::Retry
259
+
260
+ @queue = :testing
261
+ @retry_limit = 2
262
+ @retry_delay = 0
263
+
264
+ def self.identifier(*args)
265
+ args.first.to_s
266
+ end
267
+
268
+ def self.perform(*args)
269
+ raise 'failed'
270
+ end
271
+ end
metadata CHANGED
@@ -1,123 +1,115 @@
1
- --- !ruby/object:Gem::Specification
1
+ --- !ruby/object:Gem::Specification
2
2
  name: resque-retry
3
- version: !ruby/object:Gem::Version
4
- hash: 27
5
- prerelease: false
6
- segments:
7
- - 0
8
- - 1
9
- - 0
10
- version: 0.1.0
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.2.1
5
+ prerelease:
11
6
  platform: ruby
12
- authors:
7
+ authors:
13
8
  - Luke Antins
14
9
  - Ryan Carver
15
10
  autorequire:
16
11
  bindir: bin
17
12
  cert_chain: []
18
-
19
- date: 2010-08-29 00:00:00 +01:00
20
- default_executable:
21
- dependencies:
22
- - !ruby/object:Gem::Dependency
23
- name: resque
13
+ date: 2011-11-23 00:00:00.000000000Z
14
+ dependencies:
15
+ - !ruby/object:Gem::Dependency
16
+ name: rake
17
+ requirement: &2165851780 !ruby/object:Gem::Requirement
18
+ none: false
19
+ requirements:
20
+ - - ! '>='
21
+ - !ruby/object:Gem::Version
22
+ version: '0'
23
+ type: :runtime
24
24
  prerelease: false
25
- requirement: &id001 !ruby/object:Gem::Requirement
25
+ version_requirements: *2165851780
26
+ - !ruby/object:Gem::Dependency
27
+ name: resque
28
+ requirement: &2165851020 !ruby/object:Gem::Requirement
26
29
  none: false
27
- requirements:
28
- - - ">="
29
- - !ruby/object:Gem::Version
30
- hash: 55
31
- segments:
32
- - 1
33
- - 8
34
- - 0
30
+ requirements:
31
+ - - ! '>='
32
+ - !ruby/object:Gem::Version
35
33
  version: 1.8.0
36
34
  type: :runtime
37
- version_requirements: *id001
38
- - !ruby/object:Gem::Dependency
39
- name: resque-scheduler
40
35
  prerelease: false
41
- requirement: &id002 !ruby/object:Gem::Requirement
36
+ version_requirements: *2165851020
37
+ - !ruby/object:Gem::Dependency
38
+ name: resque-scheduler
39
+ requirement: &2165835400 !ruby/object:Gem::Requirement
42
40
  none: false
43
- requirements:
44
- - - ">="
45
- - !ruby/object:Gem::Version
46
- hash: 55
47
- segments:
48
- - 1
49
- - 8
50
- - 0
41
+ requirements:
42
+ - - ! '>='
43
+ - !ruby/object:Gem::Version
51
44
  version: 1.8.0
52
45
  type: :runtime
53
- version_requirements: *id002
54
- - !ruby/object:Gem::Dependency
55
- name: test-unit
56
46
  prerelease: false
57
- requirement: &id003 !ruby/object:Gem::Requirement
47
+ version_requirements: *2165835400
48
+ - !ruby/object:Gem::Dependency
49
+ name: minitest
50
+ requirement: &2165834800 !ruby/object:Gem::Requirement
58
51
  none: false
59
- requirements:
60
- - - ">="
61
- - !ruby/object:Gem::Version
62
- hash: 3
63
- segments:
64
- - 0
65
- version: "0"
52
+ requirements:
53
+ - - ! '>='
54
+ - !ruby/object:Gem::Version
55
+ version: '0'
66
56
  type: :development
67
- version_requirements: *id003
68
- - !ruby/object:Gem::Dependency
69
- name: turn
70
57
  prerelease: false
71
- requirement: &id004 !ruby/object:Gem::Requirement
58
+ version_requirements: *2165834800
59
+ - !ruby/object:Gem::Dependency
60
+ name: rack-test
61
+ requirement: &2165834000 !ruby/object:Gem::Requirement
72
62
  none: false
73
- requirements:
74
- - - ">="
75
- - !ruby/object:Gem::Version
76
- hash: 3
77
- segments:
78
- - 0
79
- version: "0"
63
+ requirements:
64
+ - - ! '>='
65
+ - !ruby/object:Gem::Version
66
+ version: '0'
80
67
  type: :development
81
- version_requirements: *id004
82
- - !ruby/object:Gem::Dependency
68
+ prerelease: false
69
+ version_requirements: *2165834000
70
+ - !ruby/object:Gem::Dependency
83
71
  name: yard
72
+ requirement: &2165833040 !ruby/object:Gem::Requirement
73
+ none: false
74
+ requirements:
75
+ - - ! '>='
76
+ - !ruby/object:Gem::Version
77
+ version: '0'
78
+ type: :development
84
79
  prerelease: false
85
- requirement: &id005 !ruby/object:Gem::Requirement
80
+ version_requirements: *2165833040
81
+ - !ruby/object:Gem::Dependency
82
+ name: json
83
+ requirement: &2165832180 !ruby/object:Gem::Requirement
86
84
  none: false
87
- requirements:
88
- - - ">="
89
- - !ruby/object:Gem::Version
90
- hash: 3
91
- segments:
92
- - 0
93
- version: "0"
85
+ requirements:
86
+ - - ! '>='
87
+ - !ruby/object:Gem::Version
88
+ version: '0'
94
89
  type: :development
95
- version_requirements: *id005
96
- - !ruby/object:Gem::Dependency
97
- name: simplecov-html
98
90
  prerelease: false
99
- requirement: &id006 !ruby/object:Gem::Requirement
91
+ version_requirements: *2165832180
92
+ - !ruby/object:Gem::Dependency
93
+ name: simplecov
94
+ requirement: &2165831320 !ruby/object:Gem::Requirement
100
95
  none: false
101
- requirements:
102
- - - ">="
103
- - !ruby/object:Gem::Version
104
- hash: 19
105
- segments:
106
- - 0
107
- - 3
108
- - 0
96
+ requirements:
97
+ - - ! '>='
98
+ - !ruby/object:Gem::Version
109
99
  version: 0.3.0
110
100
  type: :development
111
- version_requirements: *id006
112
- description: " resque-retry provides retry, delay and exponential backoff support for\n resque jobs.\n\n Features:\n\n * Redis backed retry count/limit.\n * Retry on all or specific exceptions.\n * Exponential backoff (varying the delay between retrys).\n * Multiple failure backend with retry suppression & resque-web tab.\n * Small & Extendable - plenty of places to override retry logic/settings.\n"
101
+ prerelease: false
102
+ version_requirements: *2165831320
103
+ description: ! " resque-retry provides retry, delay and exponential backoff support
104
+ for\n resque jobs.\n\n Features:\n\n * Redis backed retry count/limit.\n * Retry
105
+ on all or specific exceptions.\n * Exponential backoff (varying the delay between
106
+ retrys).\n * Multiple failure backend with retry suppression & resque-web tab.\n
107
+ \ * Small & Extendable - plenty of places to override retry logic/settings.\n"
113
108
  email: luke@lividpenguin.com
114
109
  executables: []
115
-
116
110
  extensions: []
117
-
118
111
  extra_rdoc_files: []
119
-
120
- files:
112
+ files:
121
113
  - LICENSE
122
114
  - Rakefile
123
115
  - README.md
@@ -129,6 +121,7 @@ files:
129
121
  - test/retry_criteria_test.rb
130
122
  - test/retry_inheriting_checks_test.rb
131
123
  - test/retry_test.rb
124
+ - test/server_test.rb
132
125
  - test/test_helper.rb
133
126
  - test/test_jobs.rb
134
127
  - lib/resque/failure/multiple_with_retry_suppression.rb
@@ -138,39 +131,30 @@ files:
138
131
  - lib/resque-retry/server/views/retry_timestamp.erb
139
132
  - lib/resque-retry/server.rb
140
133
  - lib/resque-retry.rb
141
- has_rdoc: false
142
134
  homepage: http://github.com/lantins/resque-retry
143
135
  licenses: []
144
-
145
136
  post_install_message:
146
137
  rdoc_options: []
147
-
148
- require_paths:
138
+ require_paths:
149
139
  - lib
150
- required_ruby_version: !ruby/object:Gem::Requirement
140
+ required_ruby_version: !ruby/object:Gem::Requirement
151
141
  none: false
152
- requirements:
153
- - - ">="
154
- - !ruby/object:Gem::Version
155
- hash: 3
156
- segments:
157
- - 0
158
- version: "0"
159
- required_rubygems_version: !ruby/object:Gem::Requirement
142
+ requirements:
143
+ - - ! '>='
144
+ - !ruby/object:Gem::Version
145
+ version: '0'
146
+ required_rubygems_version: !ruby/object:Gem::Requirement
160
147
  none: false
161
- requirements:
162
- - - ">="
163
- - !ruby/object:Gem::Version
164
- hash: 3
165
- segments:
166
- - 0
167
- version: "0"
148
+ requirements:
149
+ - - ! '>='
150
+ - !ruby/object:Gem::Version
151
+ version: '0'
168
152
  requirements: []
169
-
170
153
  rubyforge_project:
171
- rubygems_version: 1.3.7
154
+ rubygems_version: 1.8.10
172
155
  signing_key:
173
156
  specification_version: 3
174
- summary: A resque plugin; provides retry, delay and exponential backoff support for resque jobs.
157
+ summary: A resque plugin; provides retry, delay and exponential backoff support for
158
+ resque jobs.
175
159
  test_files: []
176
-
160
+ has_rdoc: false