retriable 4.2.0 → 5.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/AGENTS.md +15 -0
- data/CHANGELOG.md +54 -1
- data/Gemfile +2 -0
- data/README.md +70 -44
- data/benchmark/config_publication.rb +79 -0
- data/docs/adr/0001-copy-on-write-config-publication.md +124 -0
- data/docs/agents/domain.md +38 -0
- data/docs/agents/issue-tracker.md +45 -0
- data/docs/agents/triage-labels.md +17 -0
- data/docs/migration.md +84 -0
- data/lib/retriable/config.rb +104 -0
- data/lib/retriable/version.rb +1 -1
- data/lib/retriable.rb +102 -15
- data/sig/retriable.rbs +1 -1
- data/spec/config_spec.rb +149 -0
- data/spec/retriable_spec.rb +308 -9
- metadata +9 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b1518899ae471b83e7167d0f2dce18e198a5069d30e6f05f04c0754ebd1c3b3a
|
|
4
|
+
data.tar.gz: 5fdd9babcd9dcb9dceeec99ce861018fd58435f50a184e62a64fa5142523d235
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: af50822074d864951bc299bea96cb52ea5d35cbe7d5b0d079e5e633dec30cce8131a8db30339613562155015ee396a8fc7eb21ea841cf8beb0a826842ce314f0
|
|
7
|
+
data.tar.gz: d9dfa3ae45a889ee8326ba5aa7fb119af47302dc087e6c4a6b4033cbe0da9178ea41214b3fd5360f904b72f056ff709b5526c7298c2ff173d115803d88ca661e
|
data/AGENTS.md
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Retriable
|
|
2
|
+
|
|
3
|
+
## Agent skills
|
|
4
|
+
|
|
5
|
+
### Issue tracker
|
|
6
|
+
|
|
7
|
+
Issues live in GitHub Issues on `kamui/retriable`, via the `gh` CLI. See `docs/agents/issue-tracker.md`.
|
|
8
|
+
|
|
9
|
+
### Triage labels
|
|
10
|
+
|
|
11
|
+
The five canonical triage roles, using their default label strings. See `docs/agents/triage-labels.md`.
|
|
12
|
+
|
|
13
|
+
### Domain docs
|
|
14
|
+
|
|
15
|
+
Single-context: `CONTEXT.md` and `docs/adr/` at the repo root. See `docs/agents/domain.md`.
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,58 @@
|
|
|
1
1
|
# HEAD
|
|
2
2
|
|
|
3
|
+
## 5.0.0
|
|
4
|
+
|
|
5
|
+
**This is a major release with a breaking change. Please read carefully before
|
|
6
|
+
upgrading.**
|
|
7
|
+
|
|
8
|
+
### Breaking changes
|
|
9
|
+
|
|
10
|
+
Retriable 5.0 makes the thread-safety change below. Because it changes direct
|
|
11
|
+
config mutation, it is released as a major version.
|
|
12
|
+
([#151](https://github.com/kamui/retriable/pull/151))
|
|
13
|
+
|
|
14
|
+
The published config is frozen. Mutating `Retriable.config` directly
|
|
15
|
+
raises `FrozenError`:
|
|
16
|
+
|
|
17
|
+
```ruby
|
|
18
|
+
Retriable.config.sleep_disabled = true # => FrozenError
|
|
19
|
+
Retriable.config.contexts[:api] = {} # => FrozenError
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Go through `configure` instead:
|
|
23
|
+
|
|
24
|
+
```ruby
|
|
25
|
+
Retriable.configure { |c| c.sleep_disabled = true }
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Check your test setup first: `Retriable.config.sleep_disabled = true` in a
|
|
29
|
+
`spec_helper` or `rails_helper` is the likeliest place this bites. Reading
|
|
30
|
+
`Retriable.config` is unaffected.
|
|
31
|
+
|
|
32
|
+
A published config is shared by every thread reading it, so an in-place write
|
|
33
|
+
was a data race that could corrupt another thread's retry behavior with no sign
|
|
34
|
+
anything had gone wrong. Freezing it is what makes the copy-on-write guarantee
|
|
35
|
+
below hold in practice rather than only on paper.
|
|
36
|
+
|
|
37
|
+
The full upgrade guide lives in [docs/migration.md](docs/migration.md#4x-to-5x).
|
|
38
|
+
|
|
39
|
+
### Bug fixes
|
|
40
|
+
|
|
41
|
+
- `Retriable.configure` is now thread-safe. Configuration is copy-on-write: the
|
|
42
|
+
block mutates a duplicate, which is published atomically only if the block
|
|
43
|
+
returns without raising. Concurrent readers therefore see either the whole
|
|
44
|
+
previous config or the whole new one, never a half-applied mix, and a raising
|
|
45
|
+
block leaves the existing config in place. Nested mutable values (`on`,
|
|
46
|
+
`intervals`, `contexts`, and anything inside them) are deep-copied, so a
|
|
47
|
+
mutation inside a `configure` block can no longer reach back into the config
|
|
48
|
+
other threads are reading. `Retriable.with_context` now resolves the context
|
|
49
|
+
lookup and the global options against a single snapshot, closing a race where
|
|
50
|
+
a concurrent `configure` could drop a context's retry options. Nested
|
|
51
|
+
`configure` calls remain supported: they share the outer working copy and
|
|
52
|
+
publish once when the outermost block returns. See **Breaking changes** above
|
|
53
|
+
for the direct-mutation behavior change this required.
|
|
54
|
+
([#151](https://github.com/kamui/retriable/pull/151))
|
|
55
|
+
|
|
3
56
|
## 4.2.0
|
|
4
57
|
|
|
5
58
|
### Bug fixes
|
|
@@ -77,7 +130,7 @@
|
|
|
77
130
|
|
|
78
131
|
### Breaking changes
|
|
79
132
|
|
|
80
|
-
- Removed `timeout:` option. The `timeout:` option has been removed from `Retriable.retriable`, `Retriable.configure`, and `Retriable.with_override`. It was a thin wrapper around Ruby's `Timeout.timeout`, which has well-documented safety issues: it interrupts execution at arbitrary lines and can corrupt internal state in libraries that are not interrupt-safe (mutexes, file handles, network sockets, allocator state). This was first raised against this gem in [#96](https://github.com/kamui/retriable/issues/96) in 2021; Retriable 3.8.0 deprecated the option, and 4.0 removes the footgun entirely. As a side effect, the historical bug where Retriable's own internal `Timeout::Error` was silently retried by default is no longer reachable, since Retriable no longer raises a timeout itself. User-raised `Timeout::Error` (for example, from a `Timeout.timeout` block you write inside the retried block) is still matched by the default `on: [StandardError]` because `Timeout::Error < RuntimeError < StandardError`. Passing `timeout:` to `Retriable.retriable` or `Retriable.with_override` now raises `ArgumentError`; setting `config.timeout` in `Retriable.configure` now raises `NoMethodError` because the configuration attribute has been removed. See the [4.0 migration
|
|
133
|
+
- Removed `timeout:` option. The `timeout:` option has been removed from `Retriable.retriable`, `Retriable.configure`, and `Retriable.with_override`. It was a thin wrapper around Ruby's `Timeout.timeout`, which has well-documented safety issues: it interrupts execution at arbitrary lines and can corrupt internal state in libraries that are not interrupt-safe (mutexes, file handles, network sockets, allocator state). This was first raised against this gem in [#96](https://github.com/kamui/retriable/issues/96) in 2021; Retriable 3.8.0 deprecated the option, and 4.0 removes the footgun entirely. As a side effect, the historical bug where Retriable's own internal `Timeout::Error` was silently retried by default is no longer reachable, since Retriable no longer raises a timeout itself. User-raised `Timeout::Error` (for example, from a `Timeout.timeout` block you write inside the retried block) is still matched by the default `on: [StandardError]` because `Timeout::Error < RuntimeError < StandardError`. Passing `timeout:` to `Retriable.retriable` or `Retriable.with_override` now raises `ArgumentError`; setting `config.timeout` in `Retriable.configure` now raises `NoMethodError` because the configuration attribute has been removed. See the [4.0 migration guide](docs/migration.md#3x-to-40) for replacement patterns.
|
|
81
134
|
- Minimum Ruby version is now 3.2. Support for Ruby 2.x, 3.0, and 3.1 has been dropped in Retriable 4.0. If you need Retriable on Ruby 2.3.0-3.1.x, the 3.8.x line (`~> 3.8`) remains available.
|
|
82
135
|
|
|
83
136
|
### Features
|
data/Gemfile
CHANGED
data/README.md
CHANGED
|
@@ -7,7 +7,7 @@ Retriable is a simple DSL to retry failed code blocks with randomized [exponenti
|
|
|
7
7
|
## Table of Contents
|
|
8
8
|
|
|
9
9
|
- [Requirements](#requirements)
|
|
10
|
-
- [
|
|
10
|
+
- [Upgrading](#upgrading)
|
|
11
11
|
- [Installation](#installation)
|
|
12
12
|
- [Usage](#usage)
|
|
13
13
|
- [Defaults](#defaults)
|
|
@@ -42,37 +42,19 @@ If you need Ruby 1.9.3 support, use the [2.x branch](https://github.com/kamui/re
|
|
|
42
42
|
|
|
43
43
|
If you need Ruby 1.8.x to 1.9.2 support, use the [1.x branch](https://github.com/kamui/retriable/tree/1.x) by specifying `~1.4` in your Gemfile.
|
|
44
44
|
|
|
45
|
-
##
|
|
45
|
+
## Upgrading
|
|
46
46
|
|
|
47
|
-
|
|
47
|
+
Retriable 5.0 changes one thing for existing users: `Retriable.config` is now a
|
|
48
|
+
frozen snapshot, so direct writes raise `FrozenError`.
|
|
48
49
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
The `timeout:` option was deprecated in Retriable 3.8.0 and has been removed in Retriable 4.0. It was a thin wrapper around `Timeout.timeout`, which has well-documented safety issues: it interrupts execution at arbitrary lines and can corrupt internal state in libraries that are not interrupt-safe. See [issue #96](https://github.com/kamui/retriable/issues/96) for the original report of this problem.
|
|
54
|
-
|
|
55
|
-
If you previously used `Retriable.retriable(timeout: 5) { ... }`, you have two recommended alternatives:
|
|
56
|
-
|
|
57
|
-
1. **Use your library's native timeout** (preferred). For example, configure `Net::HTTP#read_timeout`, Faraday's `request.timeout`, or your database client's statement timeout. Library-native timeouts do not have the safety issues of `Timeout.timeout`.
|
|
58
|
-
|
|
59
|
-
2. **Manage the timeout yourself inside the block** if no native option exists:
|
|
60
|
-
|
|
61
|
-
```ruby
|
|
62
|
-
require "timeout"
|
|
63
|
-
|
|
64
|
-
Retriable.retriable do
|
|
65
|
-
Timeout.timeout(5) do
|
|
66
|
-
# code here...
|
|
67
|
-
end
|
|
68
|
-
end
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
**Note:** This still uses `Timeout.timeout`, which has the same safety issues that motivated removing the option — interruption can happen at any line, including inside non-interrupt-safe library code (mutexes, file handles, network sockets, allocator state). Prefer option 1 wherever possible. For background, see [why Ruby's `Timeout` is dangerous](https://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/), [Headius on Thread#raise and Timeout](http://blog.headius.com/2008/02/ruby-threadraise-threadkill-timeoutrb.html), [In Ruby, don't use `Timeout`](https://adamhooper.medium.com/in-ruby-dont-use-timeout-77d9d4e5a001), and [Timeout: Ruby's most dangerous API](https://www.mikeperham.com/2015/05/08/timeout-rubys-most-dangerous-api/).
|
|
72
|
-
|
|
73
|
-
Like the removed `timeout:` option, `Timeout.timeout(5)` inside the block is per-try — each retry gets a fresh 5-second budget. For an overall cap across all retries, use `max_elapsed_time:` instead.
|
|
50
|
+
```ruby
|
|
51
|
+
Retriable.config.sleep_disabled = true # => FrozenError
|
|
52
|
+
Retriable.configure { |c| c.sleep_disabled = true } # do this instead
|
|
53
|
+
```
|
|
74
54
|
|
|
75
|
-
|
|
55
|
+
Test setup files such as `spec_helper` and `rails_helper` are the likeliest
|
|
56
|
+
place this bites. For the full 4.x to 5.x and 3.x to 4.0 guides, see
|
|
57
|
+
[docs/migration.md](docs/migration.md).
|
|
76
58
|
|
|
77
59
|
## Installation
|
|
78
60
|
|
|
@@ -91,7 +73,7 @@ require 'retriable'
|
|
|
91
73
|
In your Gemfile:
|
|
92
74
|
|
|
93
75
|
```ruby
|
|
94
|
-
gem 'retriable', '~>
|
|
76
|
+
gem 'retriable', '~> 5.0'
|
|
95
77
|
```
|
|
96
78
|
|
|
97
79
|
## Usage
|
|
@@ -138,20 +120,20 @@ The default interval table with 10 tries looks like this (in seconds, rounded to
|
|
|
138
120
|
|
|
139
121
|
Here are the available options, in some vague order of relevance to most common use patterns:
|
|
140
122
|
|
|
141
|
-
| Option | Default | Definition
|
|
142
|
-
| ---------------------- | ----------------- |
|
|
143
|
-
| **`tries`** | `3` | Number of attempts to make at running your code block (includes initial attempt). Pass `Float::INFINITY` to keep retrying until success or until `max_elapsed_time` is reached.
|
|
144
|
-
| **`on`** | `[StandardError]` | Type of exceptions to retry. [Read more](#configuring-which-options-to-retry-with-on).
|
|
145
|
-
| **`retry_if`** | `nil` | Callable (for example a `Proc` or lambda) that receives the rescued exception and returns true/false to decide whether to retry. [Read more](#advanced-retry-matching-with-retry_if).
|
|
146
|
-
| **`on_retry`** | `nil` | `Proc` to call after each try is rescued. Pass `false` to disable a callback set in `#configure` for a single call. [Read more](#callbacks).
|
|
147
|
-
| **`on_give_up`** | `nil` | `Proc` to call when Retriable stops retrying after a rescued retriable exception. [Read more](#callbacks).
|
|
148
|
-
| **`sleep_disabled`** | `false` | When true, disable exponential backoff and attempt retries immediately.
|
|
149
|
-
| **`base_interval`** | `0.5` | The initial interval in seconds between tries.
|
|
150
|
-
| **`max_elapsed_time`** | `900` (15 min) | The maximum amount of total time in seconds that code is allowed to keep being retried. Set to `nil` to disable the time limit and retry based solely on `tries`.
|
|
151
|
-
| **`max_interval`** | `60` | The maximum interval in seconds that any individual retry can reach.
|
|
152
|
-
| **`multiplier`** | `1.5` | Each successive interval grows by this factor. A multipler of 1.5 means the next interval will be 1.5x the current interval.
|
|
153
|
-
| **`rand_factor`** | `0.5` | The percentage to randomize the next retry interval time. The next interval calculation is `randomized_interval = retry_interval * (random value in range [1 - randomization_factor, 1 + randomization_factor])`
|
|
154
|
-
| **`intervals`** | `nil` | Skip generated intervals and provide your own array of intervals in seconds. [Read more](#custom-interval-array).
|
|
123
|
+
| Option | Default | Definition |
|
|
124
|
+
| ---------------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
125
|
+
| **`tries`** | `3` | Number of attempts to make at running your code block (includes initial attempt). Pass `Float::INFINITY` to keep retrying until success or until `max_elapsed_time` is reached. |
|
|
126
|
+
| **`on`** | `[StandardError]` | Type of exceptions to retry. [Read more](#configuring-which-options-to-retry-with-on). |
|
|
127
|
+
| **`retry_if`** | `nil` | Callable (for example a `Proc` or lambda) that receives the rescued exception and returns true/false to decide whether to retry. [Read more](#advanced-retry-matching-with-retry_if). |
|
|
128
|
+
| **`on_retry`** | `nil` | `Proc` to call after each try is rescued. Pass `false` to disable a callback set in `#configure` for a single call. [Read more](#callbacks). |
|
|
129
|
+
| **`on_give_up`** | `nil` | `Proc` to call when Retriable stops retrying after a rescued retriable exception. [Read more](#callbacks). |
|
|
130
|
+
| **`sleep_disabled`** | `false` | When true, disable exponential backoff and attempt retries immediately. |
|
|
131
|
+
| **`base_interval`** | `0.5` | The initial interval in seconds between tries. |
|
|
132
|
+
| **`max_elapsed_time`** | `900` (15 min) | The maximum amount of total time in seconds that code is allowed to keep being retried. Set to `nil` to disable the time limit and retry based solely on `tries`. |
|
|
133
|
+
| **`max_interval`** | `60` | The maximum interval in seconds that any individual retry can reach. |
|
|
134
|
+
| **`multiplier`** | `1.5` | Each successive interval grows by this factor. A multipler of 1.5 means the next interval will be 1.5x the current interval. |
|
|
135
|
+
| **`rand_factor`** | `0.5` | The percentage to randomize the next retry interval time. The next interval calculation is `randomized_interval = retry_interval * (random value in range [1 - randomization_factor, 1 + randomization_factor])` |
|
|
136
|
+
| **`intervals`** | `nil` | Skip generated intervals and provide your own array of intervals in seconds. [Read more](#custom-interval-array). |
|
|
155
137
|
|
|
156
138
|
Timing options are validated before retrying. `tries` must be a positive integer when Retriable generates intervals, or `Float::INFINITY` for unbounded retries. `base_interval`, `max_interval`, `multiplier`, and `max_elapsed_time` must be non-negative numbers, with `max_elapsed_time` also accepting `nil`. `rand_factor` must be a number from `0` through `1`. If provided, `intervals` must be an array of non-negative numbers; because it replaces generated intervals, it also overrides `tries`, `base_interval`, `max_interval`, `rand_factor`, and `multiplier` validation. `intervals` cannot be combined with `tries: Float::INFINITY`.
|
|
157
139
|
|
|
@@ -211,6 +193,50 @@ When a higher-precedence layer sets `tries:` without `intervals:`, it clears any
|
|
|
211
193
|
if `intervals` was configured). Within a single call, passing `intervals:` still
|
|
212
194
|
overrides `tries:`.
|
|
213
195
|
|
|
196
|
+
#### Thread safety
|
|
197
|
+
|
|
198
|
+
`#configure` is the only supported way to change configuration, and it is safe to
|
|
199
|
+
call from multiple threads.
|
|
200
|
+
|
|
201
|
+
Configuration is copy-on-write. `#configure` duplicates the current config, hands
|
|
202
|
+
your block the copy, and publishes it only if the block returns without raising.
|
|
203
|
+
So a reader in another thread always sees either the whole previous config or the
|
|
204
|
+
whole new one, never a half-applied mix, and a block that raises leaves the
|
|
205
|
+
existing config in place.
|
|
206
|
+
|
|
207
|
+
Configuration blocks are serialized. Keep them short, and do not wait inside one
|
|
208
|
+
for work that may call `#configure`, because that work cannot begin until the
|
|
209
|
+
current block returns. Readers are unaffected and continue using the last
|
|
210
|
+
published config while a block runs.
|
|
211
|
+
|
|
212
|
+
The published config is deeply frozen. Reaching around `#configure` to mutate it
|
|
213
|
+
raises `FrozenError`:
|
|
214
|
+
|
|
215
|
+
```ruby
|
|
216
|
+
Retriable.config.tries = 5 # => FrozenError
|
|
217
|
+
Retriable.config.contexts[:api] = {} # => FrozenError
|
|
218
|
+
|
|
219
|
+
Retriable.configure { |c| c.tries = 5 } # this is the supported path
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
That is deliberate. A published config is shared by every thread reading it, so an
|
|
223
|
+
in-place write is a data race that used to corrupt other threads' retry behavior
|
|
224
|
+
silently. `Retriable.config` remains fine to **read**.
|
|
225
|
+
|
|
226
|
+
Two more details:
|
|
227
|
+
|
|
228
|
+
- `#configure` calls can nest. Nested calls on the configuring thread, including
|
|
229
|
+
calls from its fibers, share the outer working copy. Only the outermost call
|
|
230
|
+
publishes. If its block raises, none of the nested changes are published. A
|
|
231
|
+
nested call does not create an independent commit or savepoint.
|
|
232
|
+
- Inside a `#configure` block, every fiber on the configuring thread sees the
|
|
233
|
+
in-progress config. Other threads keep seeing the last published one until the
|
|
234
|
+
block completes.
|
|
235
|
+
|
|
236
|
+
Thread safety covers the config structure. User-supplied callbacks such as
|
|
237
|
+
`retry_if`, `on_retry`, and `on_give_up` can still hold mutable state. The caller
|
|
238
|
+
must synchronize that state if the callback can run from multiple threads.
|
|
239
|
+
|
|
214
240
|
### Override
|
|
215
241
|
|
|
216
242
|
`#with_override` is a block-scoped API for forcing retry options that should
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
$LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
|
|
4
|
+
|
|
5
|
+
require "rbconfig"
|
|
6
|
+
require "retriable"
|
|
7
|
+
|
|
8
|
+
RUN_SECONDS = Float(ENV.fetch("RETRIABLE_BENCH_SECONDS", "1.0"))
|
|
9
|
+
THREAD_COUNTS = ENV.fetch("RETRIABLE_BENCH_THREADS", "1,2,4,8")
|
|
10
|
+
.split(",")
|
|
11
|
+
.map { |value| Integer(value, 10) }
|
|
12
|
+
.uniq
|
|
13
|
+
.freeze
|
|
14
|
+
BATCH_SIZE = 100
|
|
15
|
+
|
|
16
|
+
raise ArgumentError, "RETRIABLE_BENCH_SECONDS must be positive" unless RUN_SECONDS.positive?
|
|
17
|
+
raise ArgumentError, "RETRIABLE_BENCH_THREADS must contain positive integers" unless THREAD_COUNTS.all?(&:positive?)
|
|
18
|
+
|
|
19
|
+
SnapshotHolder = Struct.new(:value)
|
|
20
|
+
snapshot_holder = SnapshotHolder.new(Retriable.config)
|
|
21
|
+
|
|
22
|
+
BENCHMARK_CASES = {
|
|
23
|
+
"plain_snapshot_read" => -> { snapshot_holder.value },
|
|
24
|
+
"published_config_read" => -> { Retriable.config },
|
|
25
|
+
"successful_retriable" => -> { Retriable.retriable { nil } }
|
|
26
|
+
}.freeze
|
|
27
|
+
|
|
28
|
+
def monotonic_time
|
|
29
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def measure(operation, thread_count)
|
|
33
|
+
ready = Queue.new
|
|
34
|
+
start = Queue.new
|
|
35
|
+
threads = Array.new(thread_count) do
|
|
36
|
+
Thread.new do
|
|
37
|
+
ready << true
|
|
38
|
+
start.pop
|
|
39
|
+
count = 0
|
|
40
|
+
deadline = monotonic_time + RUN_SECONDS
|
|
41
|
+
|
|
42
|
+
loop do
|
|
43
|
+
BATCH_SIZE.times { operation.call }
|
|
44
|
+
count += BATCH_SIZE
|
|
45
|
+
break if monotonic_time >= deadline
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
count
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
thread_count.times { ready.pop }
|
|
53
|
+
started_at = monotonic_time
|
|
54
|
+
thread_count.times { start << true }
|
|
55
|
+
operation_count = threads.sum(&:value)
|
|
56
|
+
elapsed = monotonic_time - started_at
|
|
57
|
+
|
|
58
|
+
operation_count / elapsed
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
BENCHMARK_CASES.each_value do |operation|
|
|
62
|
+
5_000.times { operation.call }
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
puts "ruby=#{RUBY_DESCRIPTION}"
|
|
66
|
+
puts "engine=#{RUBY_ENGINE}"
|
|
67
|
+
puts "host_cpu=#{RbConfig::CONFIG.fetch("host_cpu")}"
|
|
68
|
+
puts "seconds_per_case=#{RUN_SECONDS}"
|
|
69
|
+
puts "case,threads,operations_per_second"
|
|
70
|
+
|
|
71
|
+
BENCHMARK_CASES.each do |name, operation|
|
|
72
|
+
THREAD_COUNTS.each do |thread_count|
|
|
73
|
+
operations_per_second = measure(operation, thread_count)
|
|
74
|
+
puts format(
|
|
75
|
+
"%<name>s,%<threads>d,%<operations>d",
|
|
76
|
+
name: name, threads: thread_count, operations: operations_per_second.round,
|
|
77
|
+
)
|
|
78
|
+
end
|
|
79
|
+
end
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
# 1. Copy-on-write config publication
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
`Retriable.config` is a single mutable object read on every `Retriable.retriable`
|
|
10
|
+
call and written rarely, usually once at boot. That shape had three races:
|
|
11
|
+
|
|
12
|
+
- `@config ||= Config.new` was check-then-act, so two threads booting at once
|
|
13
|
+
could build two different `Config` objects.
|
|
14
|
+
- `retriable` read attributes off the live shared config one at a time, so a
|
|
15
|
+
concurrent `configure` could hand a single call an inconsistent mix of old and
|
|
16
|
+
new values.
|
|
17
|
+
- `with_context` read the config more than once. The existence check and the
|
|
18
|
+
option resolution could straddle a `configure`. The check could pass against
|
|
19
|
+
the old snapshot before resolution silently dropped the context's retry
|
|
20
|
+
options from the new one.
|
|
21
|
+
|
|
22
|
+
Reads vastly outnumber writes, so a scheme that keeps readers cheap and pays the
|
|
23
|
+
cost on the writer is the right trade.
|
|
24
|
+
|
|
25
|
+
## Decision
|
|
26
|
+
|
|
27
|
+
Configuration is copy-on-write.
|
|
28
|
+
|
|
29
|
+
`configure` takes `CONFIG_MUTEX`, duplicates the published config, yields the
|
|
30
|
+
duplicate, and publishes it only if the block returns without raising. Writers
|
|
31
|
+
serialize against each other; a raising block leaves the previous config in
|
|
32
|
+
place. Publication does not change the method's return contract: `configure`
|
|
33
|
+
returns the block's result, not the candidate or published snapshot.
|
|
34
|
+
|
|
35
|
+
**One publication seam, one engine path.** `CONFIG_PUBLICATION_MUTEX` guards the
|
|
36
|
+
`@config` reference and is held only for a reference read or the publishing
|
|
37
|
+
write, never across the user's block, so a writer blocks a reader for no longer
|
|
38
|
+
than a pointer swap. We do not special-case MRI. An unsynchronized read would be
|
|
39
|
+
safe there thanks to the GVL, but it would leave a second memory model to
|
|
40
|
+
maintain for JRuby and TruffleRuby.
|
|
41
|
+
|
|
42
|
+
The mutex is part of the hot path. A `retriable` call with no local options or
|
|
43
|
+
override uses the published `Config` directly, so it does not otherwise pay for
|
|
44
|
+
`Config.new`, `to_h`, or a merge. `benchmark/config_publication.rb` measures both
|
|
45
|
+
raw config reads and successful retry calls with one or more reader threads. We
|
|
46
|
+
accept the mutex cost for a portable memory-visibility guarantee, but we do not
|
|
47
|
+
assume that cost is free. If the benchmark shows material contention on a
|
|
48
|
+
supported engine, `Concurrent::AtomicReference` is the preferred alternative.
|
|
49
|
+
An engine-conditional unsynchronized read is not.
|
|
50
|
+
|
|
51
|
+
**The published snapshot is deeply frozen.** Atomic publication alone does not
|
|
52
|
+
deliver a consistent read: if the published object stays mutable, any caller can
|
|
53
|
+
still do `Retriable.config.contexts[:api][:tries] = 1` and corrupt what every
|
|
54
|
+
other thread is reading. `Config#freeze` therefore freezes `on`, `intervals` and
|
|
55
|
+
`contexts` recursively before freezing the config itself. `configure` publishes a
|
|
56
|
+
*copy* of the candidate so freezing never reaches a container the caller still
|
|
57
|
+
owns (`c.on = my_array` must not leave `my_array` frozen).
|
|
58
|
+
|
|
59
|
+
**`dup` copies containers, not leaves, and never preserves frozen state.**
|
|
60
|
+
`Config#initialize_copy` deep-copies `on`, `intervals` and `contexts`; scalars,
|
|
61
|
+
procs, exception classes and regexps are shared by reference. A Hash's mutable
|
|
62
|
+
default *value* is part of the copied graph, because a shared one would let
|
|
63
|
+
`config.contexts[:absent] << x` reach the caller's object; a `default_proc`
|
|
64
|
+
remains a shared callable leaf. Copies start from `#dup` rather than a fresh
|
|
65
|
+
literal so a container's class and a Hash's default behavior survive. Rebuilding
|
|
66
|
+
into a bare `{}` would silently downgrade an indifferent-access `contexts` hash
|
|
67
|
+
and break string-key lookups. Frozen state is deliberately dropped, because a dup
|
|
68
|
+
is the mutable working copy a `configure` block mutates; publication re-freezes
|
|
69
|
+
it.
|
|
70
|
+
|
|
71
|
+
Hash *keys* are left as-is. Ruby already dups and freezes an unfrozen String key
|
|
72
|
+
on assignment. The supported key types, Symbols for `contexts` and exception
|
|
73
|
+
classes for `on`, are immutable. Copying keys would buy immutability only for
|
|
74
|
+
container keys, still miss arbitrary mutable objects, and break `compare_by_identity`
|
|
75
|
+
lookups, so the boundary stays where Ruby puts it.
|
|
76
|
+
|
|
77
|
+
Deep immutability covers the containers owned by `Config`. Callable leaves can
|
|
78
|
+
hold their own mutable state, and a shared `default_proc` can mutate state outside
|
|
79
|
+
the config. Callers remain responsible for synchronizing that state when the
|
|
80
|
+
callable runs from multiple threads.
|
|
81
|
+
|
|
82
|
+
**Two mechanisms, two questions.** A thread-local (`CONFIGURING_THREAD_KEY`)
|
|
83
|
+
answers "is this thread mid-`configure`?", which is what lets the configuring
|
|
84
|
+
thread and its fibers see their own candidate. Which snapshot a *single*
|
|
85
|
+
`retriable`/`with_context` call resolves against is a different question,
|
|
86
|
+
answered by passing that snapshot as an argument to `retriable_with_config`. A
|
|
87
|
+
thread-local would leak the resolved snapshot across the caller's block and
|
|
88
|
+
change what `Retriable.config` returns inside it, so the two are kept apart. The
|
|
89
|
+
snapshot travels one hop; `with_context` resolves its contexts hash once and
|
|
90
|
+
passes that.
|
|
91
|
+
|
|
92
|
+
**Nested calls join the outer transaction.** A nested `configure` sees the
|
|
93
|
+
candidate stored in `CONFIGURING_THREAD_KEY` and yields that same object without
|
|
94
|
+
taking `CONFIG_MUTEX` again. This preserves the behavior supported before
|
|
95
|
+
copy-on-write and avoids recursive locking. The outermost block alone publishes.
|
|
96
|
+
If it raises, Retriable discards every change made by nested blocks. A nested call
|
|
97
|
+
does not create an independent commit or savepoint. Fibers on the configuring
|
|
98
|
+
thread share the transaction because `CONFIGURING_THREAD_KEY` is a true thread
|
|
99
|
+
local.
|
|
100
|
+
|
|
101
|
+
## Consequences
|
|
102
|
+
|
|
103
|
+
- Direct mutation of `Retriable.config` now raises `FrozenError`. This is a
|
|
104
|
+
breaking change for code that reached around `configure`; the migration is to
|
|
105
|
+
use `configure`. Reading `Retriable.config` is unaffected. This change will
|
|
106
|
+
ship in Retriable 5.0.
|
|
107
|
+
- Nested `configure` remains supported. Nested blocks mutate the outer working
|
|
108
|
+
copy and do not publish separately.
|
|
109
|
+
- Writer blocks are serialized for their full duration. A block must not wait
|
|
110
|
+
for work that may call `configure`, because that work cannot acquire
|
|
111
|
+
`CONFIG_MUTEX` until the current block returns. Readers do not take that mutex
|
|
112
|
+
and continue using the last published snapshot.
|
|
113
|
+
- `configure` pays two deep copies per call (one to build the candidate, one to
|
|
114
|
+
take ownership before freezing). `configure` is a rare, usually boot-time
|
|
115
|
+
operation, so this is not on any hot path.
|
|
116
|
+
- Every `retriable` call takes one uncontended mutex to read the config
|
|
117
|
+
reference. The checked-in benchmark measures the cost on CRuby and JRuby. A
|
|
118
|
+
measured optimization may replace the publication mechanism, but it must keep
|
|
119
|
+
one memory model across supported engines.
|
|
120
|
+
- The structural snapshot is thread-safe. User-supplied callbacks and other
|
|
121
|
+
callable leaves must synchronize their own mutable state.
|
|
122
|
+
- `Retriable.config` is still a global. This ADR makes the global safe to read
|
|
123
|
+
concurrently; it does not introduce per-thread, per-instance, or Ractor-safe
|
|
124
|
+
configuration.
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# Domain Docs
|
|
2
|
+
|
|
3
|
+
How the engineering skills should consume this repo's domain documentation when exploring the codebase.
|
|
4
|
+
|
|
5
|
+
This repo is **single-context**: one `CONTEXT.md` and one `docs/adr/` at the root.
|
|
6
|
+
|
|
7
|
+
## Before exploring, read these
|
|
8
|
+
|
|
9
|
+
- **`CONTEXT.md`** at the repo root
|
|
10
|
+
- **`docs/adr/`**: read ADRs that touch the area you're about to work in.
|
|
11
|
+
|
|
12
|
+
If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The `/domain-modeling` skill (reached via `/grill-with-docs` and `/improve-codebase-architecture`) creates them lazily when terms or decisions actually get resolved.
|
|
13
|
+
|
|
14
|
+
## File structure
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
/
|
|
18
|
+
├── CONTEXT.md
|
|
19
|
+
├── docs/adr/
|
|
20
|
+
│ ├── 0001-copy-on-write-config.md
|
|
21
|
+
│ └── 0002-randomized-exponential-backoff.md
|
|
22
|
+
├── lib/
|
|
23
|
+
└── spec/
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Design decisions belong in `docs/adr/` with a numbered filename. Don't scatter them into other `docs/` subdirectories.
|
|
27
|
+
|
|
28
|
+
## Use the glossary's vocabulary
|
|
29
|
+
|
|
30
|
+
When your output names a domain concept (in an issue title, a refactor proposal, a hypothesis, a test name), use the term as defined in `CONTEXT.md`. Don't drift to synonyms the glossary explicitly avoids.
|
|
31
|
+
|
|
32
|
+
If the concept you need isn't in the glossary yet, that's a signal: either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/domain-modeling`).
|
|
33
|
+
|
|
34
|
+
## Flag ADR conflicts
|
|
35
|
+
|
|
36
|
+
If your output contradicts an existing ADR, surface it explicitly rather than silently overriding:
|
|
37
|
+
|
|
38
|
+
> _Contradicts ADR-0007 (event-sourced orders), but worth reopening because…_
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Issue tracker: GitHub
|
|
2
|
+
|
|
3
|
+
Issues and specs for this repo live as GitHub issues on `kamui/retriable`. Use the `gh` CLI for all operations.
|
|
4
|
+
|
|
5
|
+
## Conventions
|
|
6
|
+
|
|
7
|
+
- **Create an issue**: `gh issue create --title "..." --body "..."`. Use a heredoc for multi-line bodies.
|
|
8
|
+
- **Read an issue**: `gh issue view <number> --comments`, filtering comments by `jq` and also fetching labels.
|
|
9
|
+
- **List issues**: `gh issue list --state open --json number,title,body,labels,comments --jq '[.[] | {number, title, body, labels: [.labels[].name], comments: [.comments[].body]}]'` with appropriate `--label` and `--state` filters.
|
|
10
|
+
- **Comment on an issue**: `gh issue comment <number> --body "..."`
|
|
11
|
+
- **Apply / remove labels**: `gh issue edit <number> --add-label "..."` / `--remove-label "..."`
|
|
12
|
+
- **Close**: `gh issue close <number> --comment "..."`
|
|
13
|
+
|
|
14
|
+
Infer the repo from `git remote -v`; `gh` does this automatically when run inside a clone.
|
|
15
|
+
|
|
16
|
+
## Pull requests as a triage surface
|
|
17
|
+
|
|
18
|
+
**PRs as a request surface: no.** _(Set to `yes` if this repo treats external PRs as feature requests; `/triage` reads this flag.)_
|
|
19
|
+
|
|
20
|
+
When set to `yes`, PRs run through the same labels and states as issues, using the `gh pr` equivalents:
|
|
21
|
+
|
|
22
|
+
- **Read a PR**: `gh pr view <number> --comments` and `gh pr diff <number>` for the diff.
|
|
23
|
+
- **List external PRs for triage**: `gh pr list --state open --json number,title,body,labels,author,authorAssociation,comments` then keep only `authorAssociation` of `CONTRIBUTOR`, `FIRST_TIME_CONTRIBUTOR`, or `NONE` (drop `OWNER`/`MEMBER`/`COLLABORATOR`).
|
|
24
|
+
- **Comment / label / close**: `gh pr comment`, `gh pr edit --add-label`/`--remove-label`, `gh pr close`.
|
|
25
|
+
|
|
26
|
+
GitHub shares one number space across issues and PRs, so a bare `#42` may be either: resolve with `gh pr view 42` and fall back to `gh issue view 42`.
|
|
27
|
+
|
|
28
|
+
## When a skill says "publish to the issue tracker"
|
|
29
|
+
|
|
30
|
+
Create a GitHub issue.
|
|
31
|
+
|
|
32
|
+
## When a skill says "fetch the relevant ticket"
|
|
33
|
+
|
|
34
|
+
Run `gh issue view <number> --comments`.
|
|
35
|
+
|
|
36
|
+
## Wayfinding operations
|
|
37
|
+
|
|
38
|
+
Used by `/wayfinder`. The **map** is a single issue with **child** issues as tickets.
|
|
39
|
+
|
|
40
|
+
- **Map**: a single issue labelled `wayfinder:map`, holding the Notes / Decisions-so-far / Fog body. `gh issue create --label wayfinder:map`.
|
|
41
|
+
- **Child ticket**: an issue linked to the map as a GitHub sub-issue (`gh api` on the sub-issues endpoint). Where sub-issues aren't enabled, add the child to a task list in the map body and put `Part of #<map>` at the top of the child body. Labels: `wayfinder:<type>` (`research`/`prototype`/`grilling`/`task`). Once claimed, the ticket is assigned to the driving dev.
|
|
42
|
+
- **Blocking**: GitHub's **native issue dependencies**, the canonical, UI-visible representation. Add an edge with `gh api --method POST repos/<owner>/<repo>/issues/<child>/dependencies/blocked_by -F issue_id=<blocker-db-id>`, where `<blocker-db-id>` is the blocker's numeric **database id** (`gh api repos/<owner>/<repo>/issues/<n> --jq .id`, _not_ the `#number` or `node_id`). GitHub reports `issue_dependencies_summary.blocked_by` (open blockers only, the live gate). Where dependencies aren't available, fall back to a `Blocked by: #<n>, #<n>` line at the top of the child body. A ticket is unblocked when every blocker is closed.
|
|
43
|
+
- **Frontier query**: list the map's open children (`gh issue list --state open`, scoped to the map's sub-issues / task list), drop any with an open blocker (`issue_dependencies_summary.blocked_by > 0`, or an open issue in the `Blocked by` line) or an assignee; first in map order wins.
|
|
44
|
+
- **Claim**: `gh issue edit <n> --add-assignee @me`, the session's first write.
|
|
45
|
+
- **Resolve**: `gh issue comment <n> --body "<answer>"`, then `gh issue close <n>`, then append a context pointer (gist + link) to the map's Decisions-so-far.
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Triage Labels
|
|
2
|
+
|
|
3
|
+
The skills speak in terms of five canonical triage roles. This file maps those roles to the actual label strings used in this repo's issue tracker.
|
|
4
|
+
|
|
5
|
+
| Label in mattpocock/skills | Label in our tracker | Meaning |
|
|
6
|
+
| -------------------------- | -------------------- | ---------------------------------------- |
|
|
7
|
+
| `needs-triage` | `needs-triage` | Maintainer needs to evaluate this issue |
|
|
8
|
+
| `needs-info` | `needs-info` | Waiting on reporter for more information |
|
|
9
|
+
| `ready-for-agent` | `ready-for-agent` | Fully specified, ready for an AFK agent |
|
|
10
|
+
| `ready-for-human` | `ready-for-human` | Requires human implementation |
|
|
11
|
+
| `wontfix` | `wontfix` | Will not be actioned |
|
|
12
|
+
|
|
13
|
+
When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the corresponding label string from this table.
|
|
14
|
+
|
|
15
|
+
Edit the right-hand column to match whatever vocabulary you actually use.
|
|
16
|
+
|
|
17
|
+
None of these labels exist in `kamui/retriable` yet. Create one on first use with `gh label create <name>`; the repo's existing labels (`question`, `dependencies`, `ruby`) don't collide with any of them.
|