retriable 2.0.0.beta3 → 2.0.0.beta4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +30 -18
- data/lib/retriable/core_ext/kernel.rb +1 -1
- data/lib/retriable/version.rb +1 -1
- data/lib/retriable.rb +32 -18
- data/spec/retriable_spec.rb +70 -25
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 6fb94b8f9e411a4f0939d03f213928e28e002375
|
4
|
+
data.tar.gz: 703fe21cb7abce08770c7cd5d308099233e8c614
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d5ed763d75076b92d366854009dd0a0a2df4eeafa184a664c13ba3a01d74968274bff2a61d9a66d201d3c0ad3d913c6e4caf4c75761ba74de447f6aa9eed3d86
|
7
|
+
data.tar.gz: 06b33e84de325d990cf32790460dfc30dc5eb765db574c344fa64762463ad668ec47d23db41646f748fdb8d658fd3feff069cfac0f748c7ddf238a84479a188b
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,7 @@
|
|
1
|
+
## 2.0.0.beta4
|
2
|
+
* Change #retry back to #retriable. Didn't like the idea of defining a method that is also a reserved word.
|
3
|
+
* Add ability for `:on` argument to accept a `Hash` where the keys are exception types and the values are a single or list of `Regexp` pattern(s) to match against exception messages for retrial.
|
4
|
+
|
1
5
|
## 2.0.0.beta3
|
2
6
|
* Accept `intervals` array argument to provide your own custom intervals.
|
3
7
|
* Refactor the exponential backoff code into it's own class.
|
data/README.md
CHANGED
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
[![Build Status](https://secure.travis-ci.org/kamui/retriable.png)](http://travis-ci.org/kamui/retriable)
|
4
4
|
|
5
|
-
Retriable is an simple DSL to retry failed code blocks with randomized [exponential backoff](http://en.wikipedia.org/wiki/Exponential_backoff). This is especially useful when interacting external api/services or file system calls.
|
5
|
+
Retriable is an simple DSL to retry failed code blocks with randomized [exponential backoff](http://en.wikipedia.org/wiki/Exponential_backoff) time intervals. This is especially useful when interacting external api/services or file system calls.
|
6
6
|
|
7
7
|
## Requirements
|
8
8
|
|
@@ -34,7 +34,7 @@ gem 'retriable'
|
|
34
34
|
|
35
35
|
## Usage
|
36
36
|
|
37
|
-
Code in a `Retriable.
|
37
|
+
Code in a `Retriable.retriable` block will be retried if an exception is raised. By default, Retriable will rescue any exception inherited from `StandardError`, make 3 retry attempts before raising the last exception, and also use randomized exponential backoff to calculate each succeeding attempt interval. The default interval table with 10 attempts looks like this (in seconds):
|
38
38
|
|
39
39
|
| request# | retry interval | randomized interval |
|
40
40
|
| -------- | -------------- | ------------------- |
|
@@ -55,7 +55,7 @@ require 'retriable'
|
|
55
55
|
class Api
|
56
56
|
# Use it in methods that interact with unreliable services
|
57
57
|
def get
|
58
|
-
Retriable.
|
58
|
+
Retriable.retriable do
|
59
59
|
# code here...
|
60
60
|
end
|
61
61
|
end
|
@@ -80,15 +80,15 @@ randomized_interval = retry_interval * (random value in range [1 - randomization
|
|
80
80
|
|
81
81
|
`multiplier` (default: 1.5) - Each successive interval grows by this factor. A multipler of 1.5 means the next interval will be 1.5x the current interval.
|
82
82
|
|
83
|
-
`max_elapsed_time` (default: 900 (15 min)) - The maximum amount of total time that code is allowed to keep being retried
|
83
|
+
`max_elapsed_time` (default: 900 (15 min)) - The maximum amount of total time that code is allowed to keep being retried.
|
84
84
|
|
85
85
|
`intervals` (default: nil) - Skip generated intervals and provide your own array of intervals in seconds. Setting this option will ignore `max_tries`, `base_interval`, `max_interval`, `rand_factor`, and `multiplier` values.
|
86
86
|
|
87
87
|
`timeout` (default: 0) - Number of seconds to allow the code block to run before raising a Timeout::Error
|
88
88
|
|
89
|
-
`on` (default: [StandardError]) - An
|
89
|
+
`on` (default: [StandardError]) - An `Array` of exceptions to rescue for each attempt, a `Hash` where the keys are `Exception` classes and the values can be a single `Regexp` pattern or a list of patterns, or a single `Exception` type.
|
90
90
|
|
91
|
-
`on_retry` - (default: nil) - Proc to call after each attempt is rescued
|
91
|
+
`on_retry` - (default: nil) - Proc to call after each attempt is rescued.
|
92
92
|
|
93
93
|
### Config
|
94
94
|
|
@@ -103,10 +103,10 @@ end
|
|
103
103
|
|
104
104
|
### Examples
|
105
105
|
|
106
|
-
`Retriable.
|
106
|
+
`Retriable.retriable` accepts custom arguments. This example will only retry on a `Timeout::Error`, retry 3 times and sleep for a full second before each attempt.
|
107
107
|
|
108
108
|
```ruby
|
109
|
-
Retriable.
|
109
|
+
Retriable.retriable on: Timeout::Error, max_tries: 3, base_interval: 1 do
|
110
110
|
# code here...
|
111
111
|
end
|
112
112
|
```
|
@@ -114,7 +114,19 @@ end
|
|
114
114
|
You can also specify multiple errors to retry on by passing an array of exceptions.
|
115
115
|
|
116
116
|
```ruby
|
117
|
-
Retriable.
|
117
|
+
Retriable.retriable on: [Timeout::Error, Errno::ECONNRESET] do
|
118
|
+
# code here...
|
119
|
+
end
|
120
|
+
```
|
121
|
+
|
122
|
+
You can also specify a Hash of exceptions where the values are a list or single Regexp pattern.
|
123
|
+
|
124
|
+
```ruby
|
125
|
+
Retriable.retriable on: {
|
126
|
+
ActiveRecord::RecordNotUnique => nil,
|
127
|
+
ActiveRecord::RecordInvalid => [/Email has already been taken/, /Username has already been taken/],
|
128
|
+
Mysql2::Error => /Duplicate entry/
|
129
|
+
} do
|
118
130
|
# code here...
|
119
131
|
end
|
120
132
|
```
|
@@ -122,7 +134,7 @@ end
|
|
122
134
|
You can also specify a timeout if you want the code block to only make an attempt for X amount of seconds. This timeout is per attempt.
|
123
135
|
|
124
136
|
```ruby
|
125
|
-
Retriable.
|
137
|
+
Retriable.retriable timeout: 60 do
|
126
138
|
# code here...
|
127
139
|
end
|
128
140
|
```
|
@@ -130,7 +142,7 @@ end
|
|
130
142
|
If you need millisecond units of time for the sleep or the timeout:
|
131
143
|
|
132
144
|
```ruby
|
133
|
-
Retriable.
|
145
|
+
Retriable.retriable base_interval: (200/1000.0), timeout: (500/1000.0) do
|
134
146
|
# code here...
|
135
147
|
end
|
136
148
|
```
|
@@ -140,7 +152,7 @@ end
|
|
140
152
|
You can also bypass the built-in interval generation and provide your own array of intervals. Supplying your own intervals overrides the `max_tries`, `base_interval`, `max_interval`, `rand_factor`, and `multiplier` parameters.
|
141
153
|
|
142
154
|
```ruby
|
143
|
-
Retriable.
|
155
|
+
Retriable.retriable intervals: [0.5, 1.0, 2.0, 2.5] do
|
144
156
|
# code here...
|
145
157
|
end
|
146
158
|
```
|
@@ -150,7 +162,7 @@ end
|
|
150
162
|
Exponential backoff is enabled by default, if you want to simply execute code every second, you can do this:
|
151
163
|
|
152
164
|
```ruby
|
153
|
-
Retriable.
|
165
|
+
Retriable.retriable base_interval: 1.0, multiplier: 1.0, rand_factor: 0.0 do
|
154
166
|
# code here...
|
155
167
|
end
|
156
168
|
```
|
@@ -158,21 +170,21 @@ end
|
|
158
170
|
If you don't want exponential backoff, but you still want some randomization between intervals, this code will run every 1 seconds with a randomization factor of 0.2, which means each interval will be a random value between 0.8 and 1.2 (1 second +/- 0.2):
|
159
171
|
|
160
172
|
```ruby
|
161
|
-
Retriable.
|
173
|
+
Retriable.retriable base_interval: 1.0, multiplier: 1.0, rand_factor: 0.2 do
|
162
174
|
# code here...
|
163
175
|
end
|
164
176
|
```
|
165
177
|
|
166
178
|
### Callbacks
|
167
179
|
|
168
|
-
|
180
|
+
`#retriable` also provides a callback called `:on_retry` that will run after an exception is rescued. This callback provides the `exception` that was raised in the current attempt, the `try_number`, the `elapsed_time` for all attempts so far, and the time in seconds of the `next_interval`. As these are specified in a `Proc`, unnecessary variables can be left out of the parameter list.
|
169
181
|
|
170
182
|
```ruby
|
171
183
|
do_this_on_each_retry = Proc.new do |exception, try_number, elapsed_time, next_interval|
|
172
184
|
log "#{exception.class}: '#{exception.message}' - #{try_number} attempts in #{elapsed_time} seconds and #{next_interval} seconds until the next attempt."}
|
173
185
|
end
|
174
186
|
|
175
|
-
Retriable.
|
187
|
+
Retriable.retriable on_retry: do_this_on_each_retry do
|
176
188
|
# code here...
|
177
189
|
end
|
178
190
|
```
|
@@ -183,7 +195,7 @@ What if I want to execute a code block at the end, whether or not an exception w
|
|
183
195
|
|
184
196
|
```ruby
|
185
197
|
begin
|
186
|
-
Retriable.
|
198
|
+
Retriable.retriable do
|
187
199
|
# some code
|
188
200
|
end
|
189
201
|
rescue => e
|
@@ -197,7 +209,7 @@ end
|
|
197
209
|
|
198
210
|
## Kernel Extension
|
199
211
|
|
200
|
-
If you want to call `Retriable.
|
212
|
+
If you want to call `Retriable.retriable` without the `Retriable` module prefix and you don't mind extending `Kernel`,
|
201
213
|
there is a kernel extension available for this.
|
202
214
|
|
203
215
|
In your ruby script:
|
data/lib/retriable/version.rb
CHANGED
data/lib/retriable.rb
CHANGED
@@ -16,7 +16,7 @@ module Retriable
|
|
16
16
|
@config ||= Config.new
|
17
17
|
end
|
18
18
|
|
19
|
-
def
|
19
|
+
def retriable(
|
20
20
|
max_tries: config.max_tries,
|
21
21
|
base_interval: config.base_interval,
|
22
22
|
max_interval: config.max_interval,
|
@@ -26,35 +26,49 @@ module Retriable
|
|
26
26
|
intervals: config.intervals,
|
27
27
|
timeout: config.timeout,
|
28
28
|
on: config.on,
|
29
|
-
on_retry: config.on_retry
|
30
|
-
&block
|
29
|
+
on_retry: config.on_retry
|
31
30
|
)
|
32
31
|
|
33
|
-
raise LocalJumpError unless block_given?
|
34
|
-
|
35
32
|
start_time = Time.now
|
36
33
|
elapsed_time = -> { Time.now - start_time }
|
37
34
|
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
|
44
|
-
|
45
|
-
|
35
|
+
if intervals
|
36
|
+
max_tries = intervals.size
|
37
|
+
else
|
38
|
+
intervals = ExponentialBackoff.new(
|
39
|
+
max_tries: max_tries,
|
40
|
+
base_interval: base_interval,
|
41
|
+
multiplier: multiplier,
|
42
|
+
max_interval: max_interval,
|
43
|
+
rand_factor: rand_factor
|
44
|
+
).intervals
|
45
|
+
end
|
46
|
+
|
47
|
+
exception_list = on.kind_of?(Hash) ? on.keys : on
|
46
48
|
|
47
49
|
intervals.each.with_index(1) do |interval, attempt|
|
48
50
|
begin
|
49
51
|
if timeout
|
50
|
-
Timeout::timeout(timeout) { return
|
52
|
+
Timeout::timeout(timeout) { return yield(attempt) }
|
51
53
|
else
|
52
|
-
return
|
54
|
+
return yield(attempt)
|
53
55
|
end
|
54
|
-
rescue *[*
|
55
|
-
|
56
|
+
rescue *[*exception_list] => exception
|
57
|
+
if on.kind_of?(Hash)
|
58
|
+
patterns = [*on[exception.class]]
|
59
|
+
message_match = patterns.empty? ? true : false
|
60
|
+
patterns.each do |pattern|
|
61
|
+
if !!(exception.message =~ pattern)
|
62
|
+
message_match = true
|
63
|
+
break
|
64
|
+
end
|
65
|
+
end
|
66
|
+
raise unless message_match
|
67
|
+
end
|
68
|
+
|
69
|
+
on_retry.call(exception, attempt, elapsed_time.call, interval) if on_retry
|
56
70
|
raise if attempt >= max_tries || (elapsed_time.call + interval) > max_elapsed_time
|
57
|
-
sleep interval if
|
71
|
+
sleep interval if config.sleep_disabled != true
|
58
72
|
end
|
59
73
|
end
|
60
74
|
end
|
data/spec/retriable_spec.rb
CHANGED
@@ -16,24 +16,28 @@ describe Retriable do
|
|
16
16
|
|
17
17
|
it "stops at first attempt if the block does not raise an exception" do
|
18
18
|
attempts = 0
|
19
|
-
subject.
|
19
|
+
subject.retriable do
|
20
20
|
attempts += 1
|
21
21
|
end
|
22
22
|
|
23
23
|
attempts.must_equal 1
|
24
24
|
end
|
25
25
|
|
26
|
-
it "raises a LocalJumpError if
|
26
|
+
it "raises a LocalJumpError if #retriable is not given a block" do
|
27
27
|
-> do
|
28
|
-
subject.
|
28
|
+
subject.retriable on: EOFError
|
29
|
+
end.must_raise LocalJumpError
|
30
|
+
|
31
|
+
-> do
|
32
|
+
subject.retriable on: EOFError, timeout: 2
|
29
33
|
end.must_raise LocalJumpError
|
30
34
|
end
|
31
35
|
|
32
|
-
describe "
|
36
|
+
describe "#retriable block of code raising EOFError with no arguments" do
|
33
37
|
before do
|
34
38
|
@attempts = 0
|
35
39
|
|
36
|
-
subject.
|
40
|
+
subject.retriable do
|
37
41
|
@attempts += 1
|
38
42
|
raise EOFError.new if @attempts < 3
|
39
43
|
end
|
@@ -44,18 +48,18 @@ describe Retriable do
|
|
44
48
|
end
|
45
49
|
end
|
46
50
|
|
47
|
-
it "
|
51
|
+
it "#retriable on custom exception and re-raises the exception" do
|
48
52
|
-> do
|
49
|
-
subject.
|
53
|
+
subject.retriable on: TestError do
|
50
54
|
raise TestError.new
|
51
55
|
end
|
52
56
|
end.must_raise TestError
|
53
57
|
end
|
54
58
|
|
55
|
-
it "
|
59
|
+
it "#retriable with 10 max tries" do
|
56
60
|
attempts = 0
|
57
61
|
|
58
|
-
subject.
|
62
|
+
subject.retriable(
|
59
63
|
max_tries: 10
|
60
64
|
) do
|
61
65
|
attempts += 1
|
@@ -65,9 +69,9 @@ describe Retriable do
|
|
65
69
|
attempts.must_equal 10
|
66
70
|
end
|
67
71
|
|
68
|
-
it "
|
72
|
+
it "#retriable will timeout after 1 second" do
|
69
73
|
-> do
|
70
|
-
subject.
|
74
|
+
subject.retriable timeout: 1 do
|
71
75
|
sleep 2
|
72
76
|
end
|
73
77
|
end.must_raise Timeout::Error
|
@@ -83,7 +87,7 @@ describe Retriable do
|
|
83
87
|
end
|
84
88
|
|
85
89
|
-> do
|
86
|
-
Retriable.
|
90
|
+
Retriable.retriable(
|
87
91
|
on: [EOFError, ArgumentError],
|
88
92
|
on_retry: handler,
|
89
93
|
rand_factor: 0.0,
|
@@ -105,7 +109,7 @@ describe Retriable do
|
|
105
109
|
@time_table[9].between?(6.403, 19.210).must_equal true
|
106
110
|
end
|
107
111
|
|
108
|
-
describe "retries with an
|
112
|
+
describe "retries with an on_#retriable handler, 6 max retries, and a 0.0 rand_factor" do
|
109
113
|
before do
|
110
114
|
max_tries = 6
|
111
115
|
@attempts = 0
|
@@ -116,7 +120,7 @@ describe Retriable do
|
|
116
120
|
@time_table[attempt] = next_interval
|
117
121
|
end
|
118
122
|
|
119
|
-
Retriable.
|
123
|
+
Retriable.retriable(
|
120
124
|
on: [EOFError, ArgumentError],
|
121
125
|
on_retry: handler,
|
122
126
|
rand_factor: 0.0,
|
@@ -142,7 +146,7 @@ describe Retriable do
|
|
142
146
|
end
|
143
147
|
end
|
144
148
|
|
145
|
-
it "
|
149
|
+
it "#retriable has a max interval of 1.5 seconds" do
|
146
150
|
max_tries = 6
|
147
151
|
attempts = 0
|
148
152
|
time_table = {}
|
@@ -151,7 +155,7 @@ describe Retriable do
|
|
151
155
|
time_table[attempt] = next_interval
|
152
156
|
end
|
153
157
|
|
154
|
-
subject.
|
158
|
+
subject.retriable(
|
155
159
|
on: EOFError,
|
156
160
|
on_retry: handler,
|
157
161
|
rand_factor: 0.0,
|
@@ -171,7 +175,7 @@ describe Retriable do
|
|
171
175
|
})
|
172
176
|
end
|
173
177
|
|
174
|
-
it "
|
178
|
+
it "#retriable with defined intervals" do
|
175
179
|
intervals = [
|
176
180
|
0.5,
|
177
181
|
0.75,
|
@@ -186,7 +190,7 @@ describe Retriable do
|
|
186
190
|
end
|
187
191
|
|
188
192
|
-> do
|
189
|
-
subject.
|
193
|
+
subject.retriable(
|
190
194
|
on: EOFError,
|
191
195
|
on_retry: handler,
|
192
196
|
intervals: intervals
|
@@ -204,7 +208,48 @@ describe Retriable do
|
|
204
208
|
})
|
205
209
|
end
|
206
210
|
|
207
|
-
it "
|
211
|
+
it "#retriable with a hash exception where the value is an exception message pattern" do
|
212
|
+
e = -> do
|
213
|
+
subject.retriable on: { TestError => /something went wrong/ } do
|
214
|
+
raise TestError.new('something went wrong')
|
215
|
+
end
|
216
|
+
end.must_raise TestError
|
217
|
+
|
218
|
+
e.message.must_equal "something went wrong"
|
219
|
+
end
|
220
|
+
|
221
|
+
it "#retriable with a hash exception list where the values are exception message patterns" do
|
222
|
+
attempts = 0
|
223
|
+
tries = []
|
224
|
+
handler = ->(exception, attempt, elapsed_time, next_interval) do
|
225
|
+
tries[attempt] = exception
|
226
|
+
end
|
227
|
+
|
228
|
+
e = -> do
|
229
|
+
subject.retriable max_tries: 4, on: { EOFError => nil, TestError => [/foo/, /bar/] }, on_retry: handler do
|
230
|
+
attempts += 1
|
231
|
+
case attempts
|
232
|
+
when 1
|
233
|
+
raise TestError.new('foo')
|
234
|
+
when 2
|
235
|
+
raise TestError.new('bar')
|
236
|
+
when 3
|
237
|
+
raise EOFError.new
|
238
|
+
else
|
239
|
+
raise TestError.new('crash')
|
240
|
+
end
|
241
|
+
end
|
242
|
+
end.must_raise TestError
|
243
|
+
|
244
|
+
e.message.must_equal "crash"
|
245
|
+
tries[1].class.must_equal TestError
|
246
|
+
tries[1].message.must_equal "foo"
|
247
|
+
tries[2].class.must_equal TestError
|
248
|
+
tries[2].message.must_equal "bar"
|
249
|
+
tries[3].class.must_equal EOFError
|
250
|
+
end
|
251
|
+
|
252
|
+
it "#retriable can be called in the global scope" do
|
208
253
|
-> do
|
209
254
|
retriable do
|
210
255
|
puts "should raise NoMethodError"
|
@@ -213,16 +258,16 @@ describe Retriable do
|
|
213
258
|
|
214
259
|
require_relative "../lib/retriable/core_ext/kernel"
|
215
260
|
|
216
|
-
|
261
|
+
attempts = 0
|
217
262
|
retriable do
|
218
|
-
|
219
|
-
raise EOFError.new if
|
263
|
+
attempts += 1
|
264
|
+
raise EOFError.new if attempts < 3
|
220
265
|
end
|
221
|
-
|
266
|
+
attempts.must_equal 3
|
222
267
|
end
|
223
268
|
end
|
224
269
|
|
225
|
-
it "
|
270
|
+
it "#retriable runs for a max elapsed time of 2 seconds" do
|
226
271
|
subject.configure do |c|
|
227
272
|
c.sleep_disabled = false
|
228
273
|
end
|
@@ -237,7 +282,7 @@ describe Retriable do
|
|
237
282
|
end
|
238
283
|
|
239
284
|
-> do
|
240
|
-
subject.
|
285
|
+
subject.retriable(
|
241
286
|
base_interval: 1.0,
|
242
287
|
multiplier: 1.0,
|
243
288
|
rand_factor: 0.0,
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: retriable
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0.0.
|
4
|
+
version: 2.0.0.beta4
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Jack Chu
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-10-
|
11
|
+
date: 2014-10-09 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rake
|