philiprehberger-retry_queue 0.4.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -0
- data/README.md +25 -1
- data/lib/philiprehberger/retry_queue/processor.rb +19 -3
- data/lib/philiprehberger/retry_queue/result.rb +26 -0
- data/lib/philiprehberger/retry_queue/version.rb +1 -1
- data/lib/philiprehberger/retry_queue.rb +8 -3
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 96d4078aaeb7a082f0b36dc2121077882c4ae88d5acba91acdf668b2de76d42c
|
|
4
|
+
data.tar.gz: 92e49204be845308e033260af04fa8aa1d40fa328ab14a7cb2c31a5e4c54cbb7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a43f5db92ac6e907a95b51b066852a55f9203f88f44600649dd12bfd6b93568dea33c12722869dc951b3f2927f70adca69a68a6cc423bdd4ac2463d815259319
|
|
7
|
+
data.tar.gz: 03466f8aeb298b0d4ff161b43a913967662ed59d2add9e5a9ff6cdbc7b027a849a556d370e9edd41badc3e18b409f05d43079f5b3ab28e08e0dc98839525cc4d
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.6.0] - 2026-05-07
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
- `Result#failure_rate` — counterpart to `#success_rate`, returns the ratio of failed items to total. Returns `0.0` for empty batches.
|
|
14
|
+
|
|
15
|
+
## [0.5.0] - 2026-04-23
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
- Optional `jitter:` (0.0..1.0) on `RetryQueue.process` — multiplies backoff delay by `1 + rand * jitter` to reduce thundering-herd risk.
|
|
19
|
+
- `Result#empty?` and `Result#size` convenience methods.
|
|
20
|
+
- YARD clarification that `max_retries: 0` means one attempt, no retries.
|
|
21
|
+
|
|
10
22
|
## [0.4.0] - 2026-04-19
|
|
11
23
|
|
|
12
24
|
### Added
|
data/README.md
CHANGED
|
@@ -45,6 +45,20 @@ result = Philiprehberger::RetryQueue.process(items, max_retries: 5, backoff: ->(
|
|
|
45
45
|
end
|
|
46
46
|
```
|
|
47
47
|
|
|
48
|
+
### Jitter
|
|
49
|
+
|
|
50
|
+
Reduce thundering-herd risk by randomizing the backoff delay. Pass a fraction in
|
|
51
|
+
`0.0..1.0`; the computed delay is multiplied by `1 + rand * jitter`. A value of
|
|
52
|
+
`0.0` (default) disables jitter.
|
|
53
|
+
|
|
54
|
+
```ruby
|
|
55
|
+
result = Philiprehberger::RetryQueue.process(items, max_retries: 3, jitter: 0.3) do |item|
|
|
56
|
+
external_api_call(item)
|
|
57
|
+
end
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Values outside `0.0..1.0` or non-Numeric values raise `ArgumentError`.
|
|
61
|
+
|
|
48
62
|
### Selective Retry
|
|
49
63
|
|
|
50
64
|
```ruby
|
|
@@ -112,18 +126,28 @@ stats = result.stats
|
|
|
112
126
|
```ruby
|
|
113
127
|
result = Philiprehberger::RetryQueue.process(items, max_retries: 3) { |item| call(item) }
|
|
114
128
|
result.success_rate # => 0.92
|
|
129
|
+
result.failure_rate # => 0.08
|
|
115
130
|
```
|
|
116
131
|
|
|
132
|
+
`Result#failure_rate` is the counterpart to `#success_rate` and pairs naturally with
|
|
133
|
+
it — for any non-empty Result, `success_rate + failure_rate` sums to `1.0`. Empty
|
|
134
|
+
batches return `0.0` for both.
|
|
135
|
+
|
|
117
136
|
## API
|
|
118
137
|
|
|
119
138
|
| Method | Description |
|
|
120
139
|
|--------|-------------|
|
|
121
|
-
| `.process(items, max_retries:, concurrency:, backoff:, retry_on:, on_retry:, on_failure:) { \|item\| }` | Process items with retry logic |
|
|
140
|
+
| `.process(items, max_retries:, concurrency:, backoff:, retry_on:, on_retry:, on_failure:, jitter:) { \|item\| }` | Process items with retry logic |
|
|
141
|
+
| `max_retries:` | Integer >= 0. `0` means one attempt with no retries (not zero attempts) |
|
|
142
|
+
| `jitter:` | Numeric in `0.0..1.0`; multiplies backoff delay by `1 + rand * jitter`. Defaults to `0.0` |
|
|
122
143
|
| `on_failure:` | Callable `(item, error)` invoked once per item that exhausts retries; hook errors are swallowed |
|
|
123
144
|
| `Result#succeeded` | Array of successfully processed items |
|
|
124
145
|
| `Result#failed` | Array of hashes with `:item`, `:error`, `:attempts` |
|
|
125
146
|
| `Result#stats` | Hash with `:total`, `:succeeded`, `:failed`, `:success_rate`, `:elapsed` |
|
|
126
147
|
| `Result#success_rate` | Float in `[0.0, 1.0]`; ratio of succeeded to total items (`0.0` for empty batches) |
|
|
148
|
+
| `Result#failure_rate` | Ratio of failed items to total (0.0..1.0); pairs with success_rate |
|
|
149
|
+
| `Result#empty?` | `true` when `stats[:total]` is 0 |
|
|
150
|
+
| `Result#size` | Returns `stats[:total]` (succeeded + failed count) |
|
|
127
151
|
| `Result#reprocess_failed { \|item, error\| }` | Reprocess failed items, returns a new Result |
|
|
128
152
|
|
|
129
153
|
## Development
|
|
@@ -7,7 +7,8 @@ module Philiprehberger
|
|
|
7
7
|
# Default backoff strategy: exponential with base 0.1s.
|
|
8
8
|
DEFAULT_BACKOFF = ->(attempt) { 0.1 * (2**attempt) }
|
|
9
9
|
|
|
10
|
-
# @param max_retries [Integer] maximum retry attempts per item
|
|
10
|
+
# @param max_retries [Integer] maximum retry attempts per item. `max_retries: 0` means
|
|
11
|
+
# one attempt with no retries (not zero attempts).
|
|
11
12
|
# @param concurrency [Integer] number of concurrent workers (reserved for future use)
|
|
12
13
|
# @param backoff [Proc, nil] proc receiving attempt number, returns sleep duration
|
|
13
14
|
# @param retry_on [Array<Class>, nil] exception classes to retry on; nil means retry all
|
|
@@ -15,8 +16,12 @@ module Philiprehberger
|
|
|
15
16
|
# @param on_failure [Proc, nil] callable invoked with `(item, error)` once per item that
|
|
16
17
|
# exhausts retries and moves to the dead-letter list; exceptions raised by the hook are
|
|
17
18
|
# swallowed so a faulty hook cannot break the queue
|
|
18
|
-
|
|
19
|
+
# @param jitter [Numeric] fraction in `0.0..1.0` applied to the computed backoff delay as
|
|
20
|
+
# `delay * (1 + rand * jitter)`. Defaults to `0.0` (no jitter).
|
|
21
|
+
def initialize(max_retries: 3, concurrency: 1, backoff: nil, retry_on: nil, on_retry: nil, on_failure: nil,
|
|
22
|
+
jitter: 0.0)
|
|
19
23
|
raise Error, 'max_retries must be non-negative' unless max_retries.is_a?(Integer) && max_retries >= 0
|
|
24
|
+
raise ArgumentError, 'jitter must be a Numeric in 0.0..1.0' unless valid_jitter?(jitter)
|
|
20
25
|
|
|
21
26
|
@max_retries = max_retries
|
|
22
27
|
@concurrency = concurrency
|
|
@@ -24,6 +29,7 @@ module Philiprehberger
|
|
|
24
29
|
@retry_on = retry_on
|
|
25
30
|
@on_retry_hooks = Array(on_retry)
|
|
26
31
|
@on_failure = on_failure
|
|
32
|
+
@jitter = jitter.to_f
|
|
27
33
|
end
|
|
28
34
|
|
|
29
35
|
# Process a collection of items with retry logic.
|
|
@@ -64,11 +70,21 @@ module Philiprehberger
|
|
|
64
70
|
|
|
65
71
|
fire_on_retry_hooks(item, e, attempts)
|
|
66
72
|
|
|
67
|
-
sleep_duration = @backoff.call(attempts - 1)
|
|
73
|
+
sleep_duration = apply_jitter(@backoff.call(attempts - 1))
|
|
68
74
|
sleep(sleep_duration) if sleep_duration.positive?
|
|
69
75
|
end
|
|
70
76
|
end
|
|
71
77
|
|
|
78
|
+
def apply_jitter(delay)
|
|
79
|
+
return delay if @jitter.zero?
|
|
80
|
+
|
|
81
|
+
delay * (1 + (rand * @jitter))
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def valid_jitter?(jitter)
|
|
85
|
+
jitter.is_a?(Numeric) && jitter >= 0.0 && jitter <= 1.0
|
|
86
|
+
end
|
|
87
|
+
|
|
72
88
|
def retryable?(error)
|
|
73
89
|
return true if @retry_on.nil?
|
|
74
90
|
|
|
@@ -33,6 +33,20 @@ module Philiprehberger
|
|
|
33
33
|
}
|
|
34
34
|
end
|
|
35
35
|
|
|
36
|
+
# Whether the batch processed zero items.
|
|
37
|
+
#
|
|
38
|
+
# @return [Boolean] true when no items were processed (succeeded and failed both empty)
|
|
39
|
+
def empty?
|
|
40
|
+
stats[:total].zero?
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# Total number of items processed, including both succeeded and failed.
|
|
44
|
+
#
|
|
45
|
+
# @return [Integer] `stats[:total]`
|
|
46
|
+
def size
|
|
47
|
+
stats[:total]
|
|
48
|
+
end
|
|
49
|
+
|
|
36
50
|
# Ratio of succeeded items to total processed items.
|
|
37
51
|
#
|
|
38
52
|
# Returns 0.0 for an empty batch to avoid ZeroDivisionError.
|
|
@@ -45,6 +59,18 @@ module Philiprehberger
|
|
|
45
59
|
@succeeded.size.to_f / total
|
|
46
60
|
end
|
|
47
61
|
|
|
62
|
+
# Ratio of failed items to total processed items.
|
|
63
|
+
#
|
|
64
|
+
# Returns 0.0 for an empty batch to avoid ZeroDivisionError.
|
|
65
|
+
#
|
|
66
|
+
# @return [Float] ratio in the range [0.0, 1.0]
|
|
67
|
+
def failure_rate
|
|
68
|
+
total = @succeeded.size + @failed.size
|
|
69
|
+
return 0.0 if total.zero?
|
|
70
|
+
|
|
71
|
+
@failed.size.to_f / total
|
|
72
|
+
end
|
|
73
|
+
|
|
48
74
|
# Reprocess failed items by yielding each item and its last error to the block.
|
|
49
75
|
# Returns a new Result with the reprocessing outcomes.
|
|
50
76
|
#
|
|
@@ -11,7 +11,9 @@ module Philiprehberger
|
|
|
11
11
|
# Process items with per-item retry, backoff, and dead letter collection.
|
|
12
12
|
#
|
|
13
13
|
# @param items [Array] items to process
|
|
14
|
-
# @param max_retries [Integer] maximum retry attempts per item
|
|
14
|
+
# @param max_retries [Integer] maximum retry attempts per item. Note: `max_retries: 0`
|
|
15
|
+
# means one attempt with no retries (not zero attempts); the item is processed once
|
|
16
|
+
# and moves straight to the dead-letter list on failure.
|
|
15
17
|
# @param concurrency [Integer] number of concurrent workers
|
|
16
18
|
# @param backoff [Proc, nil] custom backoff strategy
|
|
17
19
|
# @param retry_on [Array<Class>, nil] exception classes to retry on; others go straight to failed
|
|
@@ -19,17 +21,20 @@ module Philiprehberger
|
|
|
19
21
|
# @param on_failure [Proc, nil] callable invoked with `(item, error)` once per item that
|
|
20
22
|
# exhausts retries and moves to the dead-letter list; exceptions raised by the hook are
|
|
21
23
|
# swallowed so a faulty hook cannot break the queue
|
|
24
|
+
# @param jitter [Numeric] fraction in `0.0..1.0` applied to the computed backoff delay as
|
|
25
|
+
# `delay * (1 + rand * jitter)` to reduce thundering-herd risk. Defaults to `0.0` (no jitter).
|
|
22
26
|
# @yield [item] block that processes a single item
|
|
23
27
|
# @return [Result] processing result
|
|
24
28
|
def self.process(items, max_retries: 3, concurrency: 1, backoff: nil, retry_on: nil, on_retry: nil,
|
|
25
|
-
on_failure: nil, &block)
|
|
29
|
+
on_failure: nil, jitter: 0.0, &block)
|
|
26
30
|
processor = Processor.new(
|
|
27
31
|
max_retries: max_retries,
|
|
28
32
|
concurrency: concurrency,
|
|
29
33
|
backoff: backoff,
|
|
30
34
|
retry_on: retry_on,
|
|
31
35
|
on_retry: on_retry,
|
|
32
|
-
on_failure: on_failure
|
|
36
|
+
on_failure: on_failure,
|
|
37
|
+
jitter: jitter
|
|
33
38
|
)
|
|
34
39
|
processor.call(items, &block)
|
|
35
40
|
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: philiprehberger-retry_queue
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.6.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Philip Rehberger
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-05-07 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: Processes collections of items with configurable per-item retry logic,
|
|
14
14
|
exponential backoff, and dead letter collection for failed items. Returns detailed
|