retriable 3.5.0 → 4.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/main.yml +38 -9
- data/.hound.yml +1 -1
- data/.rubocop.yml +4 -1
- data/CHANGELOG.md +113 -0
- data/Gemfile +4 -1
- data/README.md +197 -110
- data/docs/testing.md +212 -0
- data/lib/retriable/config.rb +81 -10
- data/lib/retriable/core_ext/kernel.rb +6 -4
- data/lib/retriable/exponential_backoff.rb +44 -10
- data/lib/retriable/validation.rb +95 -0
- data/lib/retriable/version.rb +1 -1
- data/lib/retriable.rb +121 -57
- data/retriable.gemspec +2 -7
- data/sig/retriable.rbs +29 -1
- data/spec/config_spec.rb +157 -4
- data/spec/exponential_backoff_spec.rb +45 -26
- data/spec/retriable_spec.rb +739 -87
- data/spec/spec_helper.rb +3 -1
- metadata +9 -53
data/README.md
CHANGED
|
@@ -1,19 +1,78 @@
|
|
|
1
1
|
# Retriable
|
|
2
2
|
|
|
3
3
|
|
|
4
|
-
[](https://houndci.com)
|
|
5
4
|
|
|
6
5
|
Retriable is a simple DSL to retry failed code blocks with randomized [exponential backoff](http://en.wikipedia.org/wiki/Exponential_backoff) time intervals. This is especially useful when interacting external APIs, remote services, or file system calls.
|
|
7
6
|
|
|
7
|
+
## Table of Contents
|
|
8
|
+
|
|
9
|
+
- [Requirements](#requirements)
|
|
10
|
+
- [Migration from 3.x to 4.0](#migration-from-3x-to-40)
|
|
11
|
+
- [Installation](#installation)
|
|
12
|
+
- [Usage](#usage)
|
|
13
|
+
- [Defaults](#defaults)
|
|
14
|
+
- [Options](#options)
|
|
15
|
+
- [Configuring Which Options to Retry With :on](#configuring-which-options-to-retry-with-on)
|
|
16
|
+
- [Advanced Retry Matching With :retry_if](#advanced-retry-matching-with-retry_if)
|
|
17
|
+
- [Configuration](#configuration)
|
|
18
|
+
- [Override](#override)
|
|
19
|
+
- [Example Usage](#example-usage)
|
|
20
|
+
- [Custom Interval Array](#custom-interval-array)
|
|
21
|
+
- [Unbounded Retries (Opt-in)](#unbounded-retries-opt-in)
|
|
22
|
+
- [Turn off Exponential Backoff](#turn-off-exponential-backoff)
|
|
23
|
+
- [Callbacks](#callbacks)
|
|
24
|
+
- [Disabling a Configured Callback Per Call](#disabling-a-configured-callback-per-call)
|
|
25
|
+
- [Ensure/Else](#ensureelse)
|
|
26
|
+
- [Contexts](#contexts)
|
|
27
|
+
- [Kernel Extension](#kernel-extension)
|
|
28
|
+
- [Testing](#testing)
|
|
29
|
+
- [Credits](#credits)
|
|
30
|
+
- [Development](#development)
|
|
31
|
+
- [Running Specs](#running-specs)
|
|
32
|
+
|
|
8
33
|
## Requirements
|
|
9
34
|
|
|
10
|
-
Ruby
|
|
35
|
+
Ruby 3.2+
|
|
36
|
+
|
|
37
|
+
If you need Ruby 2.3.0-3.1.x support, use the [3.8.x branch](https://github.com/kamui/retriable/tree/3.8.x) by specifying `~> 3.8` in your Gemfile.
|
|
38
|
+
|
|
39
|
+
If you need Ruby 2.0.0-2.2.x support, use the [3.1 branch](https://github.com/kamui/retriable/tree/3.1.x) by specifying `~3.1` in your Gemfile.
|
|
40
|
+
|
|
41
|
+
If you need Ruby 1.9.3 support, use the [2.x branch](https://github.com/kamui/retriable/tree/2.x) by specifying `~2.1` in your Gemfile.
|
|
42
|
+
|
|
43
|
+
If you need Ruby 1.8.x to 1.9.2 support, use the [1.x branch](https://github.com/kamui/retriable/tree/1.x) by specifying `~1.4` in your Gemfile.
|
|
44
|
+
|
|
45
|
+
## Migration from 3.x to 4.0
|
|
46
|
+
|
|
47
|
+
### Ruby version
|
|
48
|
+
|
|
49
|
+
Retriable 4.0 requires Ruby 3.2 or later. If you run Ruby 2.3.0-3.1.x, or want to stay on the 3.x gem line, use Retriable 3.8.x by specifying `~> 3.8` in your Gemfile.
|
|
50
|
+
|
|
51
|
+
### `timeout:` option removed
|
|
52
|
+
|
|
53
|
+
The `timeout:` option was deprecated in Retriable 3.8.0 and has been removed in Retriable 4.0. It was a thin wrapper around `Timeout.timeout`, which has well-documented safety issues: it interrupts execution at arbitrary lines and can corrupt internal state in libraries that are not interrupt-safe. See [issue #96](https://github.com/kamui/retriable/issues/96) for the original report of this problem.
|
|
54
|
+
|
|
55
|
+
If you previously used `Retriable.retriable(timeout: 5) { ... }`, you have two recommended alternatives:
|
|
11
56
|
|
|
12
|
-
|
|
57
|
+
1. **Use your library's native timeout** (preferred). For example, configure `Net::HTTP#read_timeout`, Faraday's `request.timeout`, or your database client's statement timeout. Library-native timeouts do not have the safety issues of `Timeout.timeout`.
|
|
13
58
|
|
|
14
|
-
|
|
59
|
+
2. **Manage the timeout yourself inside the block** if no native option exists:
|
|
15
60
|
|
|
16
|
-
|
|
61
|
+
```ruby
|
|
62
|
+
require "timeout"
|
|
63
|
+
|
|
64
|
+
Retriable.retriable do
|
|
65
|
+
Timeout.timeout(5) do
|
|
66
|
+
# code here...
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
**Note:** This still uses `Timeout.timeout`, which has the same safety issues that motivated removing the option — interruption can happen at any line, including inside non-interrupt-safe library code (mutexes, file handles, network sockets, allocator state). Prefer option 1 wherever possible. For background, see [why Ruby's `Timeout` is dangerous](https://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/), [Headius on Thread#raise and Timeout](http://blog.headius.com/2008/02/ruby-threadraise-threadkill-timeoutrb.html), [In Ruby, don't use `Timeout`](https://adamhooper.medium.com/in-ruby-dont-use-timeout-77d9d4e5a001), and [Timeout: Ruby's most dangerous API](https://www.mikeperham.com/2015/05/08/timeout-rubys-most-dangerous-api/).
|
|
72
|
+
|
|
73
|
+
Like the removed `timeout:` option, `Timeout.timeout(5)` inside the block is per-try — each retry gets a fresh 5-second budget. For an overall cap across all retries, use `max_elapsed_time:` instead.
|
|
74
|
+
|
|
75
|
+
Passing `timeout:` to `Retriable.retriable` or `Retriable.with_override` now raises `ArgumentError`. The `timeout` configuration attribute has also been removed, so `Retriable.configure { |c| c.timeout = 5 }` now raises `NoMethodError`.
|
|
17
76
|
|
|
18
77
|
## Installation
|
|
19
78
|
|
|
@@ -32,7 +91,7 @@ require 'retriable'
|
|
|
32
91
|
In your Gemfile:
|
|
33
92
|
|
|
34
93
|
```ruby
|
|
35
|
-
gem 'retriable', '~>
|
|
94
|
+
gem 'retriable', '~> 4.0'
|
|
36
95
|
```
|
|
37
96
|
|
|
38
97
|
## Usage
|
|
@@ -79,27 +138,29 @@ The default interval table with 10 tries looks like this (in seconds, rounded to
|
|
|
79
138
|
|
|
80
139
|
Here are the available options, in some vague order of relevance to most common use patterns:
|
|
81
140
|
|
|
82
|
-
| Option | Default | Definition
|
|
83
|
-
| ---------------------- | ----------------- |
|
|
84
|
-
| **`tries`** | `3` | Number of attempts to make at running your code block (includes initial attempt).
|
|
85
|
-
| **`on`** | `[StandardError]` | Type of exceptions to retry. [Read more](#configuring-which-options-to-retry-with-on).
|
|
86
|
-
| **`retry_if`** | `nil` | Callable (for example a `Proc` or lambda) that receives the rescued exception and returns true/false to decide whether to retry. [Read more](#advanced-retry-matching-with-retry_if).
|
|
87
|
-
| **`on_retry`** | `nil` | `Proc` to call after each try is rescued. [Read more](#callbacks).
|
|
88
|
-
| **`
|
|
89
|
-
| **`
|
|
90
|
-
| **`
|
|
91
|
-
| **`
|
|
92
|
-
| **`
|
|
93
|
-
| **`
|
|
94
|
-
| **`
|
|
95
|
-
| **`
|
|
141
|
+
| Option | Default | Definition |
|
|
142
|
+
| ---------------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
143
|
+
| **`tries`** | `3` | Number of attempts to make at running your code block (includes initial attempt). Pass `Float::INFINITY` to keep retrying until success or until `max_elapsed_time` is reached. |
|
|
144
|
+
| **`on`** | `[StandardError]` | Type of exceptions to retry. [Read more](#configuring-which-options-to-retry-with-on). |
|
|
145
|
+
| **`retry_if`** | `nil` | Callable (for example a `Proc` or lambda) that receives the rescued exception and returns true/false to decide whether to retry. [Read more](#advanced-retry-matching-with-retry_if). |
|
|
146
|
+
| **`on_retry`** | `nil` | `Proc` to call after each try is rescued. Pass `false` to disable a callback set in `#configure` for a single call. [Read more](#callbacks). |
|
|
147
|
+
| **`on_give_up`** | `nil` | `Proc` to call when Retriable stops retrying after a rescued retriable exception. [Read more](#callbacks). |
|
|
148
|
+
| **`sleep_disabled`** | `false` | When true, disable exponential backoff and attempt retries immediately. |
|
|
149
|
+
| **`base_interval`** | `0.5` | The initial interval in seconds between tries. |
|
|
150
|
+
| **`max_elapsed_time`** | `900` (15 min) | The maximum amount of total time in seconds that code is allowed to keep being retried. Set to `nil` to disable the time limit and retry based solely on `tries`. |
|
|
151
|
+
| **`max_interval`** | `60` | The maximum interval in seconds that any individual retry can reach. |
|
|
152
|
+
| **`multiplier`** | `1.5` | Each successive interval grows by this factor. A multipler of 1.5 means the next interval will be 1.5x the current interval. |
|
|
153
|
+
| **`rand_factor`** | `0.5` | The percentage to randomize the next retry interval time. The next interval calculation is `randomized_interval = retry_interval * (random value in range [1 - randomization_factor, 1 + randomization_factor])` |
|
|
154
|
+
| **`intervals`** | `nil` | Skip generated intervals and provide your own array of intervals in seconds. [Read more](#custom-interval-array). |
|
|
155
|
+
|
|
156
|
+
Timing options are validated before retrying. `tries` must be a positive integer when Retriable generates intervals, or `Float::INFINITY` for unbounded retries. `base_interval`, `max_interval`, `multiplier`, and `max_elapsed_time` must be non-negative numbers, with `max_elapsed_time` also accepting `nil`. `rand_factor` must be a number from `0` through `1`. If provided, `intervals` must be an array of non-negative numbers; because it replaces generated intervals, it also overrides `tries`, `base_interval`, `max_interval`, `rand_factor`, and `multiplier` validation. `intervals` cannot be combined with `tries: Float::INFINITY`.
|
|
96
157
|
|
|
97
158
|
#### Configuring Which Options to Retry With :on
|
|
98
159
|
|
|
99
160
|
**`:on`** Can take the form:
|
|
100
161
|
|
|
101
162
|
- An `Exception` class (retry every exception of this type, including subclasses)
|
|
102
|
-
- An `Array` of `Exception` classes (retry any exception of one of these types, including subclasses)
|
|
163
|
+
- An `Array` or `Set` of `Exception` classes (retry any exception of one of these types, including subclasses)
|
|
103
164
|
- A `Hash` where the keys are `Exception` classes and the values are one of:
|
|
104
165
|
- `nil` (retry every exception of the key's type, including subclasses)
|
|
105
166
|
- A single `Regexp` pattern (retries exceptions ONLY if their `message` matches the pattern)
|
|
@@ -145,39 +206,66 @@ end
|
|
|
145
206
|
`#configure` sets defaults only. Per-call options passed to `Retriable.retriable` and
|
|
146
207
|
`Retriable.with_context` still take precedence.
|
|
147
208
|
|
|
209
|
+
When a higher-precedence layer sets `tries:` without `intervals:`, it clears any
|
|
210
|
+
`intervals:` inherited from a lower layer (so `retriable(tries: 1)` runs once even
|
|
211
|
+
if `intervals` was configured). Within a single call, passing `intervals:` still
|
|
212
|
+
overrides `tries:`.
|
|
213
|
+
|
|
148
214
|
### Override
|
|
149
215
|
|
|
150
|
-
|
|
151
|
-
`#
|
|
216
|
+
`#with_override` is a block-scoped API for forcing retry options that should
|
|
217
|
+
take precedence over both `#configure` defaults and per-call options. It is
|
|
218
|
+
primarily intended for tests — it lets a test force values like `tries: 1` or
|
|
219
|
+
`base_interval: 0` so the suite runs quickly and predictably, regardless of
|
|
220
|
+
the application's `#configure` defaults. In application code, prefer
|
|
221
|
+
`#configure` for app-level defaults and per-call options for caller-specific
|
|
222
|
+
values.
|
|
152
223
|
|
|
153
224
|
```ruby
|
|
154
|
-
Retriable.
|
|
225
|
+
Retriable.with_override(tries: 1, base_interval: 0) do
|
|
226
|
+
Retriable.retriable do
|
|
227
|
+
# code here...
|
|
228
|
+
end
|
|
229
|
+
end
|
|
155
230
|
```
|
|
156
231
|
|
|
157
|
-
|
|
232
|
+
Precedence inside the block:
|
|
158
233
|
|
|
159
234
|
```
|
|
160
|
-
|
|
235
|
+
with_override > local options > configure defaults
|
|
161
236
|
```
|
|
162
237
|
|
|
163
|
-
`#
|
|
164
|
-
|
|
165
|
-
|
|
238
|
+
`#with_override` requires a block and raises `ArgumentError` if called without
|
|
239
|
+
one. The override is active only while the block is executing, and is
|
|
240
|
+
automatically restored to its previous value when the block returns or raises.
|
|
241
|
+
Nested `#with_override` calls work as expected: the inner block temporarily
|
|
242
|
+
replaces the active override and the outer override is restored when the
|
|
243
|
+
inner block exits.
|
|
166
244
|
|
|
167
|
-
`#
|
|
168
|
-
|
|
245
|
+
`#with_override` is scoped to the **current thread**. The active override
|
|
246
|
+
does not affect any other thread, and child threads spawned inside the block
|
|
247
|
+
do not inherit it. This makes `#with_override` safe to use in parallel test
|
|
248
|
+
runners. Fibers running inside the same thread share the thread's active
|
|
249
|
+
override.
|
|
169
250
|
|
|
170
|
-
|
|
251
|
+
`#with_override` stores the provided options hash **by reference** and reads
|
|
252
|
+
from it on every attempt while the block runs. Treat the hash and all of its
|
|
253
|
+
nested values as immutable for the duration of the block: do not mutate them
|
|
254
|
+
from inside the block, and do not mutate them from another thread or fiber that
|
|
255
|
+
shares this thread's active override. Mutating the options mid-block results in
|
|
256
|
+
undefined retry behavior. If options must be computed, build the hash before
|
|
257
|
+
calling `#with_override` and do not retain a reference you will later mutate.
|
|
171
258
|
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
```
|
|
259
|
+
For test-integration patterns (RSpec `around`, helper methods, Minitest, etc.),
|
|
260
|
+
see [docs/testing.md](docs/testing.md).
|
|
175
261
|
|
|
176
262
|
### Example Usage
|
|
177
263
|
|
|
178
264
|
This example will only retry on a `Timeout::Error`, retry 3 times and sleep for a full second before each try.
|
|
179
265
|
|
|
180
266
|
```ruby
|
|
267
|
+
require "timeout"
|
|
268
|
+
|
|
181
269
|
Retriable.retriable(on: Timeout::Error, tries: 3, base_interval: 1) do
|
|
182
270
|
# code here...
|
|
183
271
|
end
|
|
@@ -186,6 +274,8 @@ end
|
|
|
186
274
|
You can also specify multiple errors to retry on by passing an array of exceptions.
|
|
187
275
|
|
|
188
276
|
```ruby
|
|
277
|
+
require "timeout"
|
|
278
|
+
|
|
189
279
|
Retriable.retriable(on: [Timeout::Error, Errno::ECONNRESET]) do
|
|
190
280
|
# code here...
|
|
191
281
|
end
|
|
@@ -203,35 +293,40 @@ Retriable.retriable(on: {
|
|
|
203
293
|
end
|
|
204
294
|
```
|
|
205
295
|
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
The implementation uses `Timeout::timeout`, which may be [unsafe](https://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/) [and](http://blog.headius.com/2008/02/ruby-threadraise-threadkill-timeoutrb.html) [even](https://adamhooper.medium.com/in-ruby-dont-use-timeout-77d9d4e5a001) [dangerous](https://www.mikeperham.com/2015/05/08/timeout-rubys-most-dangerous-api/). You can use this option, but you need to be very careful because the code in the block, including libraries or other code it calls, could be interrupted by the timeout at any line. You must ensure you have the right rescue logic and guards in place ([Thread.handle_interrupt](https://www.rubydoc.info/stdlib/core/Thread.handle_interrupt)) to handle that possible behavior. If that's not possible, the recommendation is that you're better off impelenting your own timeout methods depending on what your code is doing than use this feature.
|
|
296
|
+
If you need millisecond units of time for the sleep interval:
|
|
209
297
|
|
|
210
298
|
```ruby
|
|
211
|
-
Retriable.retriable(
|
|
299
|
+
Retriable.retriable(base_interval: (200 / 1000.0)) do
|
|
212
300
|
# code here...
|
|
213
301
|
end
|
|
214
302
|
```
|
|
215
303
|
|
|
216
|
-
|
|
304
|
+
### Custom Interval Array
|
|
305
|
+
|
|
306
|
+
You can also bypass the built-in interval generation and provide your own array of intervals. Supplying your own intervals overrides the `tries`, `base_interval`, `max_interval`, `rand_factor`, and `multiplier` parameters.
|
|
217
307
|
|
|
218
308
|
```ruby
|
|
219
|
-
Retriable.retriable(
|
|
309
|
+
Retriable.retriable(intervals: [0.5, 1.0, 2.0, 2.5]) do
|
|
220
310
|
# code here...
|
|
221
311
|
end
|
|
222
312
|
```
|
|
223
313
|
|
|
224
|
-
|
|
314
|
+
This example makes 5 total attempts. If the first attempt fails, the 2nd attempt occurs 0.5 seconds later.
|
|
225
315
|
|
|
226
|
-
|
|
316
|
+
### Unbounded Retries (Opt-in)
|
|
317
|
+
|
|
318
|
+
You can opt in to unbounded retries with `tries: Float::INFINITY`. This is useful for long-running worker processes where retrying should continue indefinitely, but it must be used with care.
|
|
227
319
|
|
|
228
320
|
```ruby
|
|
229
|
-
Retriable.retriable(
|
|
321
|
+
Retriable.retriable(tries: Float::INFINITY, max_elapsed_time: 300) do
|
|
230
322
|
# code here...
|
|
231
323
|
end
|
|
232
324
|
```
|
|
233
325
|
|
|
234
|
-
|
|
326
|
+
When `tries: Float::INFINITY` is set:
|
|
327
|
+
|
|
328
|
+
- `max_elapsed_time` must be a finite number. Retriable raises `ArgumentError` if it is `nil` or `Float::INFINITY`. This is a safety bound that prevents accidentally unbounded loops.
|
|
329
|
+
- Custom `intervals:` cannot be combined with `Float::INFINITY` and raises `ArgumentError`. Use the exponential backoff settings (`base_interval`, `multiplier`, `max_interval`, `rand_factor`) instead.
|
|
235
330
|
|
|
236
331
|
### Turn off Exponential Backoff
|
|
237
332
|
|
|
@@ -277,6 +372,50 @@ Retriable.retriable(on_retry: do_this_on_each_retry) do
|
|
|
277
372
|
end
|
|
278
373
|
```
|
|
279
374
|
|
|
375
|
+
> **Note:** On the final rescued attempt — when Retriable is about to give up because `tries` are exhausted — `on_retry` still fires (before `on_give_up`; see below), but `next_interval` is **`nil`** because there is no next retry. Guard any handler that does arithmetic or formatting on `next_interval` (for example `next_interval&.*(1000)`, or `if next_interval`), and avoid unconditionally logging messages like `"retrying in #{next_interval}s"` since no retry is coming. This mirrors the `nil` contract documented for [`on_give_up`](#callbacks) below.
|
|
376
|
+
|
|
377
|
+
#### Disabling a Configured Callback Per Call
|
|
378
|
+
|
|
379
|
+
If `on_retry` is set in `Retriable.configure`, every call uses it by default. To opt a specific call out — for example, a critical call site that should not log on retry — pass `on_retry: false` or `on_retry: nil`.
|
|
380
|
+
|
|
381
|
+
```ruby
|
|
382
|
+
Retriable.configure do |c|
|
|
383
|
+
c.on_retry = ->(exception, try, elapsed_time, next_interval) { log(...) }
|
|
384
|
+
end
|
|
385
|
+
|
|
386
|
+
# Most calls use the configured callback.
|
|
387
|
+
Retriable.retriable do
|
|
388
|
+
# ...
|
|
389
|
+
end
|
|
390
|
+
|
|
391
|
+
# This specific call opts out of the configured callback.
|
|
392
|
+
Retriable.retriable(on_retry: false) do
|
|
393
|
+
# ...
|
|
394
|
+
end
|
|
395
|
+
```
|
|
396
|
+
|
|
397
|
+
You can also use `:on_give_up` to run a callback when Retriable stops retrying after a rescued retriable exception. This callback receives the `exception`, the `try_number`, the `elapsed_time` for all tries so far, the `next_interval`, and the `reason` Retriable is giving up. The `reason` is either `:tries_exhausted` or `:max_elapsed_time`.
|
|
398
|
+
|
|
399
|
+
```ruby
|
|
400
|
+
do_this_when_retries_stop = Proc.new do |exception, try, elapsed_time, next_interval, reason|
|
|
401
|
+
log "#{exception.class}: '#{exception.message}' - gave up after #{try} tries because #{reason}."
|
|
402
|
+
end
|
|
403
|
+
|
|
404
|
+
Retriable.retriable(on_give_up: do_this_when_retries_stop) do
|
|
405
|
+
# code here...
|
|
406
|
+
end
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
When the reason is `:tries_exhausted`, `next_interval` is `nil` because there is no next retry. When the reason is `:max_elapsed_time`, `next_interval` is the interval that would have been slept before the next try. This reason means the next retry would exceed `max_elapsed_time`, not necessarily that the elapsed time has already exceeded it.
|
|
410
|
+
|
|
411
|
+
If both `:on_retry` and `:on_give_up` are configured, `:on_retry` still runs first for the final rescued retriable exception. This preserves the existing behavior that `:on_retry` runs whenever Retriable rescues an exception that matches its retry rules.
|
|
412
|
+
|
|
413
|
+
If you configure a default `:on_give_up` callback but want to suppress it for a specific call, pass `on_give_up: false` (or `nil`). Both are treated as "no callback".
|
|
414
|
+
|
|
415
|
+
`:on_give_up` is invoked only when Retriable rescued an exception that matched the retry rules and then decided to stop. It does **not** fire when the block raises an exception that is not in `:on`, nor when `:retry_if` returns false. Both of those cases are immediate re-raises, not retry exhaustion, and should be handled with normal Ruby `rescue` blocks around the `Retriable.retriable` call.
|
|
416
|
+
|
|
417
|
+
If `:on_give_up` itself raises, that exception propagates to the caller and replaces the original retried exception. Keep the handler defensive (rescue inside it) if you need the original exception to surface.
|
|
418
|
+
|
|
280
419
|
### Ensure/Else
|
|
281
420
|
|
|
282
421
|
What if I want to execute a code block at the end, whether or not an exception was rescued ([ensure](http://ruby-doc.org/docs/keywords/1.9/Object.html#method-i-ensure))? Or what if I want to execute a code block if no exception is raised ([else](http://ruby-doc.org/docs/keywords/1.9/Object.html#method-i-else))? Instead of providing more callbacks, I recommend you just wrap retriable in a begin/retry/else/ensure block:
|
|
@@ -304,7 +443,8 @@ Retriable.configure do |c|
|
|
|
304
443
|
c.contexts[:aws] = {
|
|
305
444
|
tries: 3,
|
|
306
445
|
base_interval: 5,
|
|
307
|
-
on_retry: Proc.new { puts 'Curse you, AWS!' }
|
|
446
|
+
on_retry: Proc.new { puts 'Curse you, AWS!' },
|
|
447
|
+
on_give_up: Proc.new { |_e, _try, _elapsed, _interval, reason| puts "Gave up on AWS: #{reason}" }
|
|
308
448
|
}
|
|
309
449
|
c.contexts[:mysql] = {
|
|
310
450
|
tries: 10,
|
|
@@ -338,6 +478,9 @@ Retriable.with_context(:mysql, tries: 30) do
|
|
|
338
478
|
end
|
|
339
479
|
```
|
|
340
480
|
|
|
481
|
+
`#with_context` requires a block and raises `ArgumentError` if called without
|
|
482
|
+
one.
|
|
483
|
+
|
|
341
484
|
## Kernel Extension
|
|
342
485
|
|
|
343
486
|
If you want to call `Retriable.retriable` without the `Retriable` module prefix and you don't mind extending `Kernel`,
|
|
@@ -367,72 +510,16 @@ retriable_with_context(:api) do
|
|
|
367
510
|
end
|
|
368
511
|
```
|
|
369
512
|
|
|
370
|
-
##
|
|
371
|
-
|
|
372
|
-
When you are running tests for your app it often takes a long time to retry blocks that fail. This is because Retriable will default to 3 tries with exponential backoff. Ideally your tests will run as quickly as possible.
|
|
373
|
-
|
|
374
|
-
If you want to short-circuit retries in tests, including calls that pass local options, use `Retriable.override` and set `tries` to `1`.
|
|
375
|
-
|
|
376
|
-
Under Rails, keep shared defaults in `Retriable.configure` and apply test-only overrides conditionally:
|
|
377
|
-
|
|
378
|
-
```ruby
|
|
379
|
-
# config/initializers/retriable.rb
|
|
380
|
-
Retriable.configure do |c|
|
|
381
|
-
c.tries = 3
|
|
382
|
-
c.base_interval = 0.5
|
|
383
|
-
c.rand_factor = 0.5
|
|
384
|
-
end
|
|
385
|
-
|
|
386
|
-
if Rails.env.test?
|
|
387
|
-
Retriable.override(tries: 1, base_interval: 0, rand_factor: 0)
|
|
388
|
-
end
|
|
389
|
-
```
|
|
390
|
-
|
|
391
|
-
If you need to run a specific test with normal retry behavior, call `Retriable.reset_override` for that example and then reapply your test override afterward.
|
|
392
|
-
|
|
393
|
-
Alternately, if you are using RSpec, you could override the Retriable configuration in your `spec_helper`.
|
|
394
|
-
|
|
395
|
-
```ruby
|
|
396
|
-
# spec/spec_helper.rb
|
|
397
|
-
Retriable.override(tries: 1, base_interval: 0, rand_factor: 0)
|
|
398
|
-
```
|
|
399
|
-
|
|
400
|
-
If you have defined contexts for your configuration, top-level override values (such as `tries: 1`) already take precedence over context-specific values. However, if you need to override context-specific options (for example, clearing a context's `:intervals` array or changing its `:on` exception list), pass `:contexts` to `Retriable.override`:
|
|
401
|
-
|
|
402
|
-
For example assuming you have configured a `google_api` context:
|
|
403
|
-
|
|
404
|
-
```ruby
|
|
405
|
-
# config/initializers/retriable.rb
|
|
406
|
-
Retriable.configure do |c|
|
|
407
|
-
c.contexts[:google_api] = {
|
|
408
|
-
tries: 5,
|
|
409
|
-
base_interval: 3,
|
|
410
|
-
on: [
|
|
411
|
-
Net::ReadTimeout,
|
|
412
|
-
Signet::AuthorizationError,
|
|
413
|
-
Errno::ECONNRESET,
|
|
414
|
-
OpenSSL::SSL::SSLError
|
|
415
|
-
]
|
|
416
|
-
}
|
|
417
|
-
end
|
|
418
|
-
```
|
|
419
|
-
|
|
420
|
-
Then in your test environment, you can override both top-level defaults and per-context options:
|
|
513
|
+
## Testing
|
|
421
514
|
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
context_overrides[key] = { tries: 1, base_interval: 0 }
|
|
427
|
-
end
|
|
515
|
+
`Retriable.with_override` is designed to short-circuit retries in your test
|
|
516
|
+
suite so failing blocks do not slow tests down. The simplest pattern is an
|
|
517
|
+
RSpec `around(:each)` hook (or your test framework's equivalent) that wraps
|
|
518
|
+
every example in `with_override(tries: 1, base_interval: 0)`.
|
|
428
519
|
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
base_interval: 0,
|
|
433
|
-
contexts: context_overrides,
|
|
434
|
-
)
|
|
435
|
-
```
|
|
520
|
+
For Rails integration, opting out of the override for specific tests, and
|
|
521
|
+
overriding configured contexts in tests, see
|
|
522
|
+
[docs/testing.md](docs/testing.md).
|
|
436
523
|
|
|
437
524
|
## Credits
|
|
438
525
|
|
data/docs/testing.md
ADDED
|
@@ -0,0 +1,212 @@
|
|
|
1
|
+
# Testing with Retriable
|
|
2
|
+
|
|
3
|
+
`Retriable.with_override` exists primarily for tests. It lets a test force
|
|
4
|
+
retry options like `tries: 1` or `base_interval: 0` so the suite runs quickly
|
|
5
|
+
and predictably, regardless of what the application's `Retriable.configure`
|
|
6
|
+
defaults are.
|
|
7
|
+
|
|
8
|
+
`with_override` is block-scoped: the override is active inside the block and
|
|
9
|
+
restored to its previous value (which is usually "no override") when the block
|
|
10
|
+
exits, even if the block raises. It is also thread-local — overrides set in
|
|
11
|
+
one thread do not affect other threads — so it is safe for parallel test
|
|
12
|
+
runners. See the README for the full API contract.
|
|
13
|
+
|
|
14
|
+
## RSpec
|
|
15
|
+
|
|
16
|
+
### Apply an override to every test
|
|
17
|
+
|
|
18
|
+
Use `around(:each)` in `RSpec.configure` so every test in the suite runs inside
|
|
19
|
+
the override. This is the most common pattern:
|
|
20
|
+
|
|
21
|
+
```ruby
|
|
22
|
+
RSpec.configure do |config|
|
|
23
|
+
config.around(:each) do |example|
|
|
24
|
+
Retriable.with_override(tries: 1, base_interval: 0) do
|
|
25
|
+
example.run
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### Apply an override to a specific context
|
|
32
|
+
|
|
33
|
+
```ruby
|
|
34
|
+
describe MyClient do
|
|
35
|
+
context "when external calls should not retry" do
|
|
36
|
+
around(:each) do |example|
|
|
37
|
+
Retriable.with_override(tries: 1) { example.run }
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
it "fails fast" do
|
|
41
|
+
# `with_override(tries: 1)` is active here
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
### Apply an override to a single test
|
|
48
|
+
|
|
49
|
+
Wrap the test body directly:
|
|
50
|
+
|
|
51
|
+
```ruby
|
|
52
|
+
it "does the thing without waiting" do
|
|
53
|
+
Retriable.with_override(tries: 1, base_interval: 0) do
|
|
54
|
+
# test body
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### Reusable helper
|
|
60
|
+
|
|
61
|
+
Wrap a common configuration in a helper to keep tests readable:
|
|
62
|
+
|
|
63
|
+
```ruby
|
|
64
|
+
module RetriableHelpers
|
|
65
|
+
def with_fast_retries(&block)
|
|
66
|
+
Retriable.with_override(tries: 1, base_interval: 0, &block)
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
RSpec.configure do |config|
|
|
71
|
+
config.include RetriableHelpers
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
# In a spec:
|
|
75
|
+
it "does the thing" do
|
|
76
|
+
with_fast_retries do
|
|
77
|
+
# test body
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## Minitest
|
|
83
|
+
|
|
84
|
+
```ruby
|
|
85
|
+
class MyClientTest < Minitest::Test
|
|
86
|
+
def around
|
|
87
|
+
Retriable.with_override(tries: 1, base_interval: 0) { yield }
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def test_fails_fast
|
|
91
|
+
# `with_override(tries: 1)` is active here
|
|
92
|
+
end
|
|
93
|
+
end
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Older Minitest versions without `around` can wrap the test body directly:
|
|
97
|
+
|
|
98
|
+
```ruby
|
|
99
|
+
def test_fails_fast
|
|
100
|
+
Retriable.with_override(tries: 1) do
|
|
101
|
+
# test body
|
|
102
|
+
end
|
|
103
|
+
end
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## Short-Circuiting Retriable in Your Test Suite
|
|
107
|
+
|
|
108
|
+
When you are running tests for your app, the default retry behavior (3 tries
|
|
109
|
+
with exponential backoff) makes failing blocks take a long time. To short-circuit
|
|
110
|
+
retries — including calls that pass local options — set `tries: 1` and disable
|
|
111
|
+
backoff using `with_override`.
|
|
112
|
+
|
|
113
|
+
### Under Rails
|
|
114
|
+
|
|
115
|
+
Keep shared defaults in `Retriable.configure` and apply test-only overrides via
|
|
116
|
+
RSpec's `around` hook (or your test framework's equivalent):
|
|
117
|
+
|
|
118
|
+
```ruby
|
|
119
|
+
# config/initializers/retriable.rb
|
|
120
|
+
Retriable.configure do |c|
|
|
121
|
+
c.tries = 3
|
|
122
|
+
c.base_interval = 0.5
|
|
123
|
+
c.rand_factor = 0.5
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
# spec/spec_helper.rb (or equivalent)
|
|
127
|
+
RSpec.configure do |config|
|
|
128
|
+
config.around(:each) do |example|
|
|
129
|
+
Retriable.with_override(tries: 1, base_interval: 0, rand_factor: 0) do
|
|
130
|
+
example.run
|
|
131
|
+
end
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
If a specific test needs normal retry behavior, opt out by running outside the
|
|
137
|
+
`around` hook. The cleanest way is to tag the example and skip the hook for
|
|
138
|
+
tagged examples:
|
|
139
|
+
|
|
140
|
+
```ruby
|
|
141
|
+
config.around(:each, retriable: :real) { |example| example.run }
|
|
142
|
+
config.around(:each) do |example|
|
|
143
|
+
next example.run if example.metadata[:retriable] == :real
|
|
144
|
+
|
|
145
|
+
Retriable.with_override(tries: 1, base_interval: 0, rand_factor: 0) do
|
|
146
|
+
example.run
|
|
147
|
+
end
|
|
148
|
+
end
|
|
149
|
+
|
|
150
|
+
it "exercises the real retry behavior", retriable: :real do
|
|
151
|
+
# `with_override` is not applied here
|
|
152
|
+
end
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
### Overriding Configured Contexts in Tests
|
|
156
|
+
|
|
157
|
+
If you have configured contexts, top-level override values (such as `tries: 1`)
|
|
158
|
+
already take precedence over context-specific values. To override
|
|
159
|
+
context-specific options as well (for example, clearing a context's
|
|
160
|
+
`:intervals` array or shrinking its `:on` exception list), pass `:contexts` to
|
|
161
|
+
`with_override`.
|
|
162
|
+
|
|
163
|
+
Given a configured `google_api` context:
|
|
164
|
+
|
|
165
|
+
```ruby
|
|
166
|
+
# config/initializers/retriable.rb
|
|
167
|
+
Retriable.configure do |c|
|
|
168
|
+
c.contexts[:google_api] = {
|
|
169
|
+
tries: 5,
|
|
170
|
+
base_interval: 3,
|
|
171
|
+
on: [
|
|
172
|
+
Net::ReadTimeout,
|
|
173
|
+
Signet::AuthorizationError,
|
|
174
|
+
Errno::ECONNRESET,
|
|
175
|
+
OpenSSL::SSL::SSLError,
|
|
176
|
+
],
|
|
177
|
+
}
|
|
178
|
+
end
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
You can override both top-level defaults and per-context options in your
|
|
182
|
+
test setup:
|
|
183
|
+
|
|
184
|
+
```ruby
|
|
185
|
+
RSpec.configure do |config|
|
|
186
|
+
config.around(:each) do |example|
|
|
187
|
+
context_overrides = Retriable.config.contexts.each_key.with_object({}) do |key, h|
|
|
188
|
+
h[key] = { tries: 1, base_interval: 0 }
|
|
189
|
+
end
|
|
190
|
+
|
|
191
|
+
Retriable.with_override(
|
|
192
|
+
multiplier: 1.0,
|
|
193
|
+
rand_factor: 0.0,
|
|
194
|
+
base_interval: 0,
|
|
195
|
+
contexts: context_overrides,
|
|
196
|
+
) do
|
|
197
|
+
example.run
|
|
198
|
+
end
|
|
199
|
+
end
|
|
200
|
+
end
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
## Notes
|
|
204
|
+
|
|
205
|
+
- The override is automatically cleared when the block exits, including when
|
|
206
|
+
the block raises. You do not need to clean up after the block.
|
|
207
|
+
- `with_override` calls nest: an inner block temporarily replaces the active
|
|
208
|
+
override, and the outer override is restored when the inner block exits.
|
|
209
|
+
- Overrides are thread-local. Child threads spawned inside the block do not
|
|
210
|
+
inherit it. If a test spawns background threads that themselves call
|
|
211
|
+
`Retriable.retriable`, wrap each background thread's body in its own
|
|
212
|
+
`with_override` call.
|