resque-retry 1.2.1 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 105706892be947dafeb239f354c775e3bcb8df9d
4
- data.tar.gz: 43c4248749336f1dd2a56fd561fcc0f56676ab30
3
+ metadata.gz: 66d34a270a535f97586c21a6c259d780c7a1ce9f
4
+ data.tar.gz: 9ab5f4591d2a99e262cc1ab2501d70e845f710aa
5
5
  SHA512:
6
- metadata.gz: a6cc62c680c943615e628f65e64e7391bd8db8b8f9257ce8080690650bd55e7a0812e8ecc62001a8a608d1f82f9daa92ef6ca36c5498d6094f01a327a3d8f126
7
- data.tar.gz: baf25c455e64044e2b3849df250d1b9a594b278b83903f41703d04a6c2b0ac56d3ec9ef6aa7bcadb0a672b658bd1251f8169b9f90893ed925754b265954b856e
6
+ metadata.gz: 03b4b70d9ef0e69d46d738fb3b6c1dd987e3e0e46cf3a2724300b50e1fbdcfc9f44d2af1b18b53999a21e74d48e335fe1a689e9885e14e784070ad5384f1fcac
7
+ data.tar.gz: 781a313ebd129ad8db24df9822b47bec750879aa2fa1abcc524df00228767c6a50eca03c780b19aee8e576304fcb294fe83f963af8fb75beacf4cb89b3f631fb
data/.travis.yml CHANGED
@@ -10,5 +10,6 @@ rvm:
10
10
  - 2.0.0
11
11
  - 2.1.0
12
12
  - 2.1.1
13
+ - 2.1.2
13
14
  - jruby-19mode
14
15
  - rbx-2
data/HISTORY.md CHANGED
@@ -1,5 +1,12 @@
1
1
  ## HEAD
2
2
 
3
+ ## 1.3.0 (2014-07-25)
4
+
5
+ * Adjust gem dependency on `resque-scheduler` to ~> 3.0
6
+ * Deprecated: `args_for_retry` in favor of `retry_args` (will output deprecation warnings if your using the older method).
7
+ * Feature: Allow changing the args for a given exception using `retry_args_for_exception` (@jonp)
8
+ * Feature: Allow setting `@expire_retry_key_after` on the fly (@orenmazor)
9
+
3
10
  ## 1.2.1 (2014-06-09)
4
11
 
5
12
  * Fixed Kernel.rand: "invalid argument - 0.0 (ArgumentError)" error with "ExponentialBackoff" (on "Rubinius") when `retry_delay_multiplicand_min` and `retry_delay_multiplicand_max` were the same value (@saizai)
data/README.md CHANGED
@@ -1,7 +1,7 @@
1
1
  resque-retry
2
2
  ============
3
3
 
4
- A [Resque][rq] plugin. Requires Resque ~> 1.25 & [resque-scheduler][rqs] ~> 2.5.
4
+ A [Resque][rq] plugin. Requires Resque ~> 1.25 & [resque-scheduler][rqs] ~> 3.0.
5
5
 
6
6
  resque-retry provides retry, delay and exponential backoff support for
7
7
  resque jobs.
@@ -28,7 +28,7 @@ If you're using [Bundler][bundler] to manage your dependencies, you should add `
28
28
  Add this to your `Rakefile`:
29
29
  ```ruby
30
30
  require 'resque/tasks'
31
- require 'resque_scheduler/tasks'
31
+ require 'resque/scheduler/tasks'
32
32
  ```
33
33
 
34
34
  The delay between retry attempts is provided by [resque-scheduler][rqs].
@@ -120,8 +120,8 @@ run Resque::Server.new
120
120
  Retry Options & Logic
121
121
  ---------------------
122
122
 
123
- Please take a look at the yardoc/code for more details on methods you may
124
- wish to override.
123
+ Please take a look at the [yardoc](http://rubydoc.info/gems/resque-retry)/code
124
+ for more details on methods you may wish to override.
125
125
 
126
126
  Customisation is pretty easy, the below examples should give you
127
127
  some ideas =), adapt for your own usage and feel free to pick and mix!
@@ -332,7 +332,7 @@ job should retry.
332
332
 
333
333
  ### Retry Arguments
334
334
 
335
- You may override `args_for_retry`, which is passed the current
335
+ You may override `retry_args`, which is passed the current
336
336
  job arguments, to modify the arguments for the next retry attempt.
337
337
  ```ruby
338
338
  class DeliverViaSMSC
@@ -340,7 +340,7 @@ class DeliverViaSMSC
340
340
  @queue = :mt_smsc_messages
341
341
 
342
342
  # retry using the emergency SMSC.
343
- def self.args_for_retry(smsc_id, mt_message)
343
+ def self.retry_args(smsc_id, mt_message)
344
344
  [999, mt_message]
345
345
  end
346
346
 
@@ -350,6 +350,25 @@ class DeliverViaSMSC
350
350
  end
351
351
  ```
352
352
 
353
+ Alternatively, if you require finer control of the args based on the
354
+ exception thrown, you may override `retry_args_for_exception`, which is passed
355
+ the exception and the current job arguments, to modify the arguments for the
356
+ next retry attempt.
357
+ ```ruby
358
+ class DeliverViaSMSC
359
+ extend Resque::Plugins::Retry
360
+ @queue = :mt_smsc_messages
361
+
362
+ # retry using the emergency SMSC.
363
+ def self.retry_args_for_exception(exception, smsc_id, mt_message)
364
+ [999, mt_message + exception.message]
365
+ end
366
+
367
+ self.perform(smsc_id, mt_message)
368
+ heavy_lifting
369
+ end
370
+ end
371
+ ```
353
372
  ### Job Retry Identifier/Key
354
373
 
355
374
  The retry attempt is incremented and stored in a Redis key. The key is
@@ -401,7 +420,7 @@ This saves you from having to run a "house cleaning" or "errand" job.
401
420
  The expiary timeout is "pushed forward" or "touched" after each failure to
402
421
  ensure its not expired too soon.
403
422
 
404
- ### Debug Plugging Logging
423
+ ### Debug Plugin Logging
405
424
 
406
425
  The inner-workings of the plugin are output to the Resque [Logger](https://github.com/resque/resque/wiki/Logging)
407
426
  when `Resque.logger.level` is set to `Logger::DEBUG`.
@@ -12,7 +12,7 @@ require 'resque/failure/redis'
12
12
 
13
13
  # Require Rakefile related resque things.
14
14
  require 'resque/tasks'
15
- require 'resque_scheduler/tasks'
15
+ require 'resque/scheduler/tasks'
16
16
 
17
17
  # Enable resque-retry failure backend.
18
18
  Resque::Failure::MultipleWithRetrySuppression.classes = [Resque::Failure::Redis]
data/lib/resque-retry.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  require 'resque'
2
- require 'resque_scheduler'
2
+ require 'resque-scheduler'
3
3
 
4
4
  require 'resque/plugins/retry'
5
5
  require 'resque/plugins/exponential_backoff'
@@ -1,6 +1,6 @@
1
1
  require 'cgi'
2
2
  require 'resque/server'
3
- require 'resque_scheduler/server'
3
+ require 'resque/scheduler/server'
4
4
 
5
5
  # Extend Resque::Server to add tabs.
6
6
  module ResqueRetry
@@ -1,3 +1,3 @@
1
1
  module ResqueRetry
2
- VERSION = '1.2.1'
2
+ VERSION = '1.3.0'
3
3
  end
@@ -154,8 +154,27 @@ module Resque
154
154
  # @return [Array] new job arguments
155
155
  #
156
156
  # @api public
157
- def args_for_retry(*args)
158
- args
157
+ def retry_args(*args)
158
+ # Here for backwards compatibility. If an "args_for_retry" method exists
159
+ # invoke it, but warn that it is deprecated (and will be removed in a
160
+ # future revision)
161
+ if respond_to?(:args_for_retry)
162
+ warn "`Resque::Plugins::Retry#args_for_retry` is deprecated, please use `Resque::Plugins::Retry#retry_args` instead."
163
+ args_for_retry(*args)
164
+ else
165
+ args
166
+ end
167
+ end
168
+
169
+ # @abstract
170
+ # Modify the arguments used to retry the job based on the exception.
171
+ # Use this to do something other than try the exact same job again.
172
+ #
173
+ # @return [Array] new job arguments
174
+ #
175
+ # @api public
176
+ def retry_args_for_exception(exception, *args)
177
+ retry_args(*args)
159
178
  end
160
179
 
161
180
  # Convenience method to test whether you may retry on a given
@@ -223,6 +242,14 @@ module Resque
223
242
  end
224
243
  end
225
244
 
245
+ # @abstract
246
+ # The number of seconds to set the TTL to on the resque-retry key in redis
247
+ #
248
+ # @return [Number] number of seconds
249
+ #
250
+ # @api public
251
+ attr_reader :expire_retry_key_after
252
+
226
253
  # Test if the retry criteria is valid
227
254
  #
228
255
  # @param [Exception] exception
@@ -322,11 +349,13 @@ module Resque
322
349
  # jobs that fail before ::perform will be both retried and sent to the failed queue.
323
350
  Resque.redis.setnx(redis_retry_key(*args), -1)
324
351
 
352
+ retry_args = retry_args_for_exception(exception, *args)
353
+
325
354
  if temp_retry_delay <= 0
326
355
  # If the delay is 0, no point passing it through the scheduler
327
- Resque.enqueue(retry_in_queue, *args_for_retry(*args))
356
+ Resque.enqueue(retry_in_queue, *retry_args)
328
357
  else
329
- Resque.enqueue_in(temp_retry_delay, retry_in_queue, *args_for_retry(*args))
358
+ Resque.enqueue_in(temp_retry_delay, retry_in_queue, *retry_args)
330
359
  end
331
360
 
332
361
  # remove retry key from redis if we handed retry off to another queue.
@@ -338,7 +367,8 @@ module Resque
338
367
 
339
368
  # Resque before_perform hook
340
369
  #
341
- # Increments and sets the `@retry_attempt` count.
370
+ # Increments `@retry_attempt` count and updates the "retry_key" expiration
371
+ # time (if applicable)
342
372
  #
343
373
  # @api private
344
374
  def before_perform_retry(*args)
@@ -347,10 +377,15 @@ module Resque
347
377
 
348
378
  # store number of retry attempts.
349
379
  retry_key = redis_retry_key(*args)
350
- Resque.redis.setnx(retry_key, -1) # default to -1 if not set.
351
- @retry_attempt = Resque.redis.incr(retry_key) # increment by 1.
380
+ Resque.redis.setnx(retry_key, -1)
381
+ @retry_attempt = Resque.redis.incr(retry_key)
352
382
  log_message "attempt: #{@retry_attempt} set in Redis", args
353
- Resque.redis.expire(retry_key, @retry_delay.to_i + @expire_retry_key_after.to_i) if @expire_retry_key_after
383
+
384
+ # set/update the "retry_key" expiration
385
+ if expire_retry_key_after
386
+ log_message "updating expiration for retry key: #{retry_key}", args
387
+ Resque.redis.expire(retry_key, retry_delay + expire_retry_key_after)
388
+ end
354
389
  end
355
390
 
356
391
  # Resque after_perform hook
data/resque-retry.gemspec CHANGED
@@ -30,7 +30,7 @@ Gem::Specification.new do |s|
30
30
  s.require_paths = %w[lib]
31
31
 
32
32
  s.add_dependency('resque', '~> 1.25')
33
- s.add_dependency('resque-scheduler', '~> 2.5')
33
+ s.add_dependency('resque-scheduler', '~> 3.0')
34
34
 
35
35
  s.add_development_dependency('rake', '~> 10.1')
36
36
  s.add_development_dependency('minitest', '~> 4.0')
data/test/retry_test.rb CHANGED
@@ -55,12 +55,23 @@ class RetryTest < MiniTest::Unit::TestCase
55
55
  assert_equal test_args, job['args']
56
56
  end
57
57
 
58
- def test_job_args_may_be_modified
59
- Resque.enqueue(RetryWithModifiedArgsJob, 'foo', 'bar')
58
+ def test_job_args_can_be_modified_by_overriding_args_for_retry
59
+ Resque.enqueue(DeprecatedRetryWithModifiedArgsJob)
60
+ DeprecatedRetryWithModifiedArgsJob.expects(:warn)
61
+ DeprecatedRetryWithModifiedArgsJob.expects(:args_for_retry)
60
62
  perform_next_job(@worker)
63
+ end
61
64
 
62
- assert job = Resque.pop(:testing)
63
- assert_equal ['foobar', 'barbar'], job['args']
65
+ def test_job_args_can_be_modified_by_overriding_retry_args
66
+ Resque.enqueue(RetryWithModifiedArgsJob)
67
+ RetryWithModifiedArgsJob.expects(:retry_args)
68
+ perform_next_job(@worker)
69
+ end
70
+
71
+ def test_job_args_can_be_modified_by_overriding_retry_args_for_exception
72
+ Resque.enqueue(RetryWithExceptionBasedArgsJob)
73
+ RetryWithExceptionBasedArgsJob.expects(:retry_args_for_exception)
74
+ perform_next_job(@worker)
64
75
  end
65
76
 
66
77
  def test_retry_never_give_up
@@ -286,22 +297,19 @@ class RetryTest < MiniTest::Unit::TestCase
286
297
  perform_next_job(@worker)
287
298
  end
288
299
 
289
- if Process.respond_to?(:fork) && Gem::Version.new(Resque::VERSION) >= Gem::Version.new('1.20.0')
290
- def test_retry_on_dirty_exit
291
- Resque.enqueue(RetryKilledJob)
292
- RetryKilledJob.expects(:clean_retry_key).once
293
- 2.times do
294
- job = @worker.reserve
295
- child = fork do
296
- Resque.redis.client.reconnect
297
- job.perform
298
- end
299
- Process.waitpid(child)
300
- job.fail(Resque::DirtyExit.new)
301
- end
300
+ def test_expire_key_setting_on_the_fly
301
+ retry_key = 'resque-retry:FailFiveTimesWithCustomExpiryJob'
302
+
303
+ Resque.redis.expects(:expire).
304
+ with(retry_key, 100).then.
305
+ with(retry_key, 101).then.
306
+ with(retry_key, 102).then.
307
+ with(retry_key, 103).then.
308
+ with(retry_key, 104)
302
309
 
303
- assert_equal nil, @worker.reserve
310
+ Resque.enqueue(FailFiveTimesWithCustomExpiryJob)
311
+ 5.times do
312
+ perform_next_job(@worker)
304
313
  end
305
314
  end
306
-
307
315
  end
data/test/test_jobs.rb CHANGED
@@ -41,7 +41,7 @@ class RetryDefaultsJob
41
41
  @queue = :testing
42
42
 
43
43
  def self.perform(*args)
44
- raise
44
+ raise 'error'
45
45
  end
46
46
  end
47
47
 
@@ -102,11 +102,27 @@ class InheritTestWithMoreExtraJob < InheritTestWithExtraJob
102
102
  end
103
103
  end
104
104
 
105
- class RetryWithModifiedArgsJob < RetryDefaultsJob
105
+ class DeprecatedRetryWithModifiedArgsJob < RetryDefaultsJob
106
106
  @queue = :testing
107
107
 
108
108
  def self.args_for_retry(*args)
109
- args.each { |arg| arg << 'bar' }
109
+ # NOTE: implementation is irrelevant we only care that it's invoked
110
+ end
111
+ end
112
+
113
+ class RetryWithModifiedArgsJob < RetryDefaultsJob
114
+ @queue = :testing
115
+
116
+ def self.retry_args(*args)
117
+ # NOTE: implementation is irrelevant we only care that it's invoked
118
+ end
119
+ end
120
+
121
+ class RetryWithExceptionBasedArgsJob < RetryDefaultsJob
122
+ @queue = :testing
123
+
124
+ def self.retry_args_for_exception(exception, *args)
125
+ # NOTE: implementation is irrelevant we only care that it's invoked
110
126
  end
111
127
  end
112
128
 
@@ -154,6 +170,19 @@ class FailFiveTimesWithExpiryJob < RetryDefaultsJob
154
170
  end
155
171
  end
156
172
 
173
+ class FailFiveTimesWithCustomExpiryJob < RetryDefaultsJob
174
+ @queue = :testing
175
+ @retry_limit = 6
176
+
177
+ def self.expire_retry_key_after
178
+ retry_attempt + 100
179
+ end
180
+
181
+ def self.perform(*args)
182
+ raise if retry_attempt <= 4
183
+ end
184
+ end
185
+
157
186
  class ExponentialBackoffJob < RetryDefaultsJob
158
187
  extend Resque::Plugins::ExponentialBackoff
159
188
  @queue = :testing
@@ -474,13 +503,3 @@ class FailsDuringConnectJob < RetryDefaultsJob
474
503
  @retry_limit = 3
475
504
  @retry_delay = 10
476
505
  end
477
-
478
-
479
- class RetryKilledJob
480
- extend Resque::Plugins::Retry
481
- @queue = :testing
482
-
483
- def self.perform(*args)
484
- Process.kill("KILL", Process.pid)
485
- end
486
- end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: resque-retry
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.2.1
4
+ version: 1.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Luke Antins
@@ -10,7 +10,7 @@ authors:
10
10
  autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
- date: 2014-06-09 00:00:00.000000000 Z
13
+ date: 2014-07-25 00:00:00.000000000 Z
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency
16
16
  name: resque
@@ -32,14 +32,14 @@ dependencies:
32
32
  requirements:
33
33
  - - "~>"
34
34
  - !ruby/object:Gem::Version
35
- version: '2.5'
35
+ version: '3.0'
36
36
  type: :runtime
37
37
  prerelease: false
38
38
  version_requirements: !ruby/object:Gem::Requirement
39
39
  requirements:
40
40
  - - "~>"
41
41
  - !ruby/object:Gem::Version
42
- version: '2.5'
42
+ version: '3.0'
43
43
  - !ruby/object:Gem::Dependency
44
44
  name: rake
45
45
  requirement: !ruby/object:Gem::Requirement