patient_http-sidekiq 1.4.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/ARCHITECTURE.md +53 -46
- data/CHANGELOG.md +30 -0
- data/README.md +286 -282
- data/VERSION +1 -1
- data/lib/generators/patient_http/sidekiq/install_generator.rb +42 -0
- data/lib/generators/patient_http/sidekiq/templates/initializer.rb +96 -0
- data/lib/patient_http/sidekiq/callback_worker.rb +16 -13
- data/lib/patient_http/sidekiq/configuration.rb +261 -153
- data/lib/patient_http/sidekiq/context.rb +21 -14
- data/lib/patient_http/sidekiq/direct_task_handler.rb +26 -20
- data/lib/patient_http/sidekiq/lifecycle_hooks.rb +13 -9
- data/lib/patient_http/sidekiq/processor_observer.rb +94 -50
- data/lib/patient_http/sidekiq/redis_pool.rb +33 -28
- data/lib/patient_http/sidekiq/request_executor.rb +54 -37
- data/lib/patient_http/sidekiq/request_worker.rb +23 -26
- data/lib/patient_http/sidekiq/stats.rb +79 -54
- data/lib/patient_http/sidekiq/task_handler.rb +49 -36
- data/lib/patient_http/sidekiq/task_monitor.rb +248 -215
- data/lib/patient_http/sidekiq/task_monitor_thread.rb +25 -21
- data/lib/patient_http/sidekiq/web.rb +2 -2
- data/lib/patient_http/sidekiq/web_ui.rb +31 -17
- data/lib/patient_http/sidekiq.rb +417 -254
- data/patient_http-sidekiq.gemspec +2 -2
- metadata +6 -4
data/README.md
CHANGED
|
@@ -6,13 +6,13 @@
|
|
|
6
6
|
|
|
7
7
|
*Built for APIs that like to think.*
|
|
8
8
|
|
|
9
|
-
This gem
|
|
9
|
+
This gem runs HTTP requests from Sidekiq on a dedicated async I/O processor in your Sidekiq process, using the [patient_http gem](https://github.com/bdurand/patient_http). Worker threads don't wait for HTTP responses, so they're free to run other jobs while requests are in flight.
|
|
10
10
|
|
|
11
11
|
## Motivation
|
|
12
12
|
|
|
13
|
-
Sidekiq
|
|
13
|
+
Sidekiq works best when jobs finish quickly. A long HTTP request blocks a worker thread, so other jobs wait, latency rises, and throughput drops. LLM and other AI APIs make this worse, because a request can take many seconds to finish.
|
|
14
14
|
|
|
15
|
-
|
|
15
|
+
Without this gem, each slow request holds a worker thread for its full duration:
|
|
16
16
|
|
|
17
17
|
```
|
|
18
18
|
┌────────────────────────────────────────────────────────────────────────┐
|
|
@@ -26,7 +26,7 @@ Sidekiq is designed with the assumption that jobs are short-lived and complete q
|
|
|
26
26
|
└────────────────────────────────────────────────────────────────────────┘
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
|
|
29
|
+
With this gem, a worker thread only hands off the request, and the async processor waits for the response:
|
|
30
30
|
|
|
31
31
|
```
|
|
32
32
|
┌────────────────────────────────────────────────────────────────────────┐
|
|
@@ -42,21 +42,33 @@ Sidekiq is designed with the assumption that jobs are short-lived and complete q
|
|
|
42
42
|
└────────────────────────────────────────────────────────────────────────┘
|
|
43
43
|
```
|
|
44
44
|
|
|
45
|
-
The async processor runs in a dedicated thread
|
|
45
|
+
The async processor runs in a dedicated thread in your Sidekiq process. It uses Ruby's fiber-based concurrency to run hundreds of HTTP requests at the same time without blocking. When a request finishes, the gem calls your callback service.
|
|
46
46
|
|
|
47
|
-
## Quick
|
|
47
|
+
## Quick start
|
|
48
48
|
|
|
49
|
-
### 1.
|
|
49
|
+
### 1. Install the gem
|
|
50
50
|
|
|
51
|
-
|
|
51
|
+
Add the gem to your Gemfile:
|
|
52
52
|
|
|
53
53
|
```ruby
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
54
|
+
gem "patient_http-sidekiq"
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Then install it:
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
bundle install
|
|
57
61
|
```
|
|
58
62
|
|
|
59
|
-
|
|
63
|
+
No other setup is required. When the gem loads, it registers the request handler and connects the processor to Sidekiq's startup and shutdown. You don't need to write an initializer or call a setup method. Every option has a working default. To change the defaults, see [Configuration](#configuration).
|
|
64
|
+
|
|
65
|
+
To create a commented initializer to start from, run the generator:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
bin/rails generate patient_http:sidekiq:install
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
### 2. Create a callback service
|
|
60
72
|
|
|
61
73
|
Define a callback service class with `on_complete` and `on_error` methods:
|
|
62
74
|
|
|
@@ -79,9 +91,9 @@ class FetchDataCallback
|
|
|
79
91
|
end
|
|
80
92
|
```
|
|
81
93
|
|
|
82
|
-
### 3. Make HTTP
|
|
94
|
+
### 3. Make HTTP requests
|
|
83
95
|
|
|
84
|
-
Make HTTP requests from anywhere in your code
|
|
96
|
+
Make HTTP requests from anywhere in your code with the `PatientHttp` module:
|
|
85
97
|
|
|
86
98
|
```ruby
|
|
87
99
|
PatientHttp.get(
|
|
@@ -92,24 +104,80 @@ PatientHttp.get(
|
|
|
92
104
|
)
|
|
93
105
|
```
|
|
94
106
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
The request will be enqueued as a Sidekiq job and passed to a [PatientHttp](https://github.com/bdurand/patient_http) processor to execute asynchronously. When the HTTP request completes, your callback's `on_complete` method is executed in another Sidekiq job.
|
|
107
|
+
The gem enqueues the request as a Sidekiq job, which runs the request on a [PatientHttp](https://github.com/bdurand/patient_http) processor. If a processor runs in the current process, the request goes straight to it instead; see [Direct execution](#direct-execution). When the request finishes, another Sidekiq job calls your callback's `on_complete` method. If the request fails, the job calls `on_error` instead.
|
|
98
108
|
|
|
99
|
-
|
|
109
|
+
The `response.callback_args` and `error.callback_args` methods return the arguments that you passed with the `callback_args` option.
|
|
100
110
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
The `response.callback_args` and `error.callback_args` provide access to the arguments you passed via the `callback_args` option.
|
|
111
|
+
For other HTTP methods, use `PatientHttp.post`, `PatientHttp.put`, `PatientHttp.patch`, `PatientHttp.delete`, `PatientHttp.head`, and `PatientHttp.query`. For the full API reference, see the [patient_http documentation](https://github.com/bdurand/patient_http).
|
|
104
112
|
|
|
105
113
|
> [!IMPORTANT]
|
|
106
|
-
>
|
|
114
|
+
> Don't raise an error in `on_error` to retry the request. Sidekiq retries the callback job, not the request. To retry the request, make a new request from `on_error`. Make sure that the retries stop if the error persists, or they can loop forever.
|
|
107
115
|
>
|
|
108
|
-
>
|
|
116
|
+
> The `on_error` callback runs only when the request raises an exception, such as a timeout or a connection failure. By default, HTTP error status codes (4xx and 5xx) don't call `on_error`. The gem treats these responses as completed requests and passes them to `on_complete`. To treat HTTP errors as exceptions, see [Handle HTTP error responses](#handle-http-error-responses).
|
|
117
|
+
|
|
118
|
+
## Usage
|
|
119
|
+
|
|
120
|
+
### Make requests
|
|
121
|
+
|
|
122
|
+
Use the `PatientHttp` module methods to make requests. There's a method for each HTTP method:
|
|
123
|
+
|
|
124
|
+
```ruby
|
|
125
|
+
# GET request
|
|
126
|
+
PatientHttp.get("https://api.example.com/users/123",
|
|
127
|
+
callback: MyCallback, callback_args: {user_id: 123})
|
|
128
|
+
|
|
129
|
+
# POST request with a JSON body
|
|
130
|
+
PatientHttp.post("https://api.example.com/users",
|
|
131
|
+
json: {name: "John", email: "john@example.com"},
|
|
132
|
+
callback: MyCallback)
|
|
133
|
+
|
|
134
|
+
# PUT request
|
|
135
|
+
PatientHttp.put("https://api.example.com/users/123",
|
|
136
|
+
json: {name: "Updated Name"},
|
|
137
|
+
callback: MyCallback)
|
|
138
|
+
|
|
139
|
+
# PATCH request
|
|
140
|
+
PatientHttp.patch("https://api.example.com/users/123",
|
|
141
|
+
json: {status: "active"},
|
|
142
|
+
callback: MyCallback)
|
|
143
|
+
|
|
144
|
+
# DELETE request
|
|
145
|
+
PatientHttp.delete("https://api.example.com/users/123",
|
|
146
|
+
callback: MyCallback)
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
The methods take these options:
|
|
109
150
|
|
|
110
|
-
|
|
151
|
+
| Option | Description |
|
|
152
|
+
| --- | --- |
|
|
153
|
+
| `callback:` | Required. The callback service class, or its name. |
|
|
154
|
+
| `callback_args:` | A Hash of arguments that the callback reads from the response or error. See [Callback arguments](#callback-arguments). |
|
|
155
|
+
| `headers:` | The request headers. |
|
|
156
|
+
| `body:` | The request body. GET, HEAD, and DELETE requests can't have a body. |
|
|
157
|
+
| `json:` | An object to send as a JSON body. Can't be combined with `body:`. |
|
|
158
|
+
| `params:` | Query parameters to add to the URL. |
|
|
159
|
+
| `timeout:` | The request timeout in seconds. |
|
|
160
|
+
| `raise_error_responses:` | Whether to treat non-2xx responses as errors. See [Handle HTTP error responses](#handle-http-error-responses). |
|
|
161
|
+
| `processor:` | The name of the processor that runs the request. See [Named processors](#named-processors). |
|
|
111
162
|
|
|
112
|
-
|
|
163
|
+
For all options, see the [patient_http documentation](https://github.com/bdurand/patient_http#make-requests).
|
|
164
|
+
|
|
165
|
+
For more control, build a `PatientHttp::Request` object and pass it to `PatientHttp.execute`:
|
|
166
|
+
|
|
167
|
+
```ruby
|
|
168
|
+
request = PatientHttp::Request.new(:get, "https://api.example.com/users/123",
|
|
169
|
+
headers: {"Authorization" => "Bearer token"},
|
|
170
|
+
params: {include: "profile"},
|
|
171
|
+
timeout: 30
|
|
172
|
+
)
|
|
173
|
+
PatientHttp.execute(request: request, callback: MyCallback, callback_args: {user_id: 123})
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
For the full `Request` and `Response` API reference, see the [patient_http documentation](https://github.com/bdurand/patient_http).
|
|
177
|
+
|
|
178
|
+
### Handle HTTP error responses
|
|
179
|
+
|
|
180
|
+
By default, the gem treats HTTP error status codes (4xx and 5xx) as completed requests and passes them to `on_complete`. To check the status, use `response.success?`, `response.client_error?`, or `response.server_error?`:
|
|
113
181
|
|
|
114
182
|
```ruby
|
|
115
183
|
class ApiCallback
|
|
@@ -134,22 +202,22 @@ PatientHttp.get(
|
|
|
134
202
|
)
|
|
135
203
|
```
|
|
136
204
|
|
|
137
|
-
|
|
205
|
+
To treat HTTP errors as exceptions, set the `raise_error_responses` option. With this option, a non-2xx response calls `on_error` with a `PatientHttp::HttpError` instead:
|
|
138
206
|
|
|
139
207
|
```ruby
|
|
140
208
|
class ApiCallback
|
|
141
209
|
def on_complete(response)
|
|
142
|
-
#
|
|
210
|
+
# Called only for 2xx responses.
|
|
143
211
|
process_data(response.json)
|
|
144
212
|
end
|
|
145
213
|
|
|
146
214
|
def on_error(error)
|
|
147
|
-
# Called for exceptions
|
|
215
|
+
# Called for exceptions, and for HTTP errors when raise_error_responses is set.
|
|
148
216
|
if error.is_a?(PatientHttp::HttpError)
|
|
149
|
-
#
|
|
217
|
+
# The response is available from error.response.
|
|
150
218
|
Rails.logger.error("HTTP #{error.status} from #{error.url}: #{error.response.body}")
|
|
151
219
|
else
|
|
152
|
-
#
|
|
220
|
+
# Request errors, such as timeouts and connection failures.
|
|
153
221
|
Rails.logger.error("Request failed: #{error.message}")
|
|
154
222
|
end
|
|
155
223
|
end
|
|
@@ -162,7 +230,7 @@ PatientHttp.get(
|
|
|
162
230
|
)
|
|
163
231
|
```
|
|
164
232
|
|
|
165
|
-
|
|
233
|
+
An `HttpError` gives you access to the request and the response:
|
|
166
234
|
|
|
167
235
|
```ruby
|
|
168
236
|
def on_error(error)
|
|
@@ -172,69 +240,14 @@ def on_error(error)
|
|
|
172
240
|
puts error.http_method # HTTP method
|
|
173
241
|
puts error.response.body # Response body
|
|
174
242
|
puts error.response.headers # Response headers
|
|
175
|
-
puts error.response.json #
|
|
243
|
+
puts error.response.json # Response body parsed as JSON
|
|
176
244
|
end
|
|
177
245
|
end
|
|
178
246
|
```
|
|
179
247
|
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
### Making Requests
|
|
248
|
+
### Set Sidekiq options at runtime
|
|
183
249
|
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
```ruby
|
|
187
|
-
# GET request
|
|
188
|
-
PatientHttp.get("https://api.example.com/users/123",
|
|
189
|
-
callback: MyCallback, callback_args: {user_id: 123})
|
|
190
|
-
|
|
191
|
-
# POST request with JSON body
|
|
192
|
-
PatientHttp.post("https://api.example.com/users",
|
|
193
|
-
json: {name: "John", email: "john@example.com"},
|
|
194
|
-
callback: MyCallback)
|
|
195
|
-
|
|
196
|
-
# PUT request
|
|
197
|
-
PatientHttp.put("https://api.example.com/users/123",
|
|
198
|
-
json: {name: "Updated Name"},
|
|
199
|
-
callback: MyCallback)
|
|
200
|
-
|
|
201
|
-
# PATCH request
|
|
202
|
-
PatientHttp.patch("https://api.example.com/users/123",
|
|
203
|
-
json: {status: "active"},
|
|
204
|
-
callback: MyCallback)
|
|
205
|
-
|
|
206
|
-
# DELETE request
|
|
207
|
-
PatientHttp.delete("https://api.example.com/users/123",
|
|
208
|
-
callback: MyCallback)
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
Available request options:
|
|
212
|
-
|
|
213
|
-
- `callback:` - (required) Callback service class or class name
|
|
214
|
-
- `callback_args:` - Hash of arguments passed to callback via response/error
|
|
215
|
-
- `headers:` - Request headers
|
|
216
|
-
- `body:` - Request body (for POST/PUT/PATCH)
|
|
217
|
-
- `json:` - Object to serialize as JSON body (cannot use with body)
|
|
218
|
-
- `params:` - Query parameters to append to URL
|
|
219
|
-
- `timeout:` - Request timeout in seconds
|
|
220
|
-
- `raise_error_responses:` - Treat non-2xx responses as errors
|
|
221
|
-
|
|
222
|
-
You can also build a `PatientHttp::Request` object and pass it to `PatientHttp.execute` for more control:
|
|
223
|
-
|
|
224
|
-
```ruby
|
|
225
|
-
request = PatientHttp::Request.new(:get, "https://api.example.com/users/123",
|
|
226
|
-
headers: {"Authorization" => "Bearer token"},
|
|
227
|
-
params: {include: "profile"},
|
|
228
|
-
timeout: 30
|
|
229
|
-
)
|
|
230
|
-
PatientHttp.execute(request: request, callback: MyCallback, callback_args: {user_id: 123})
|
|
231
|
-
```
|
|
232
|
-
|
|
233
|
-
See the [patient_http docs](https://github.com/bdurand/patient_http) for the full `Request` and `Response` API reference.
|
|
234
|
-
|
|
235
|
-
### Setting Sidekiq Options At Runtime
|
|
236
|
-
|
|
237
|
-
You can set Sidekiq job options for individual requests with `PatientHttp::Sidekiq.with_sidekiq_options`. The options apply to all requests enqueued within the block. Use this, for example, to route urgent requests to a higher priority queue:
|
|
250
|
+
To set Sidekiq job options for specific requests, use `PatientHttp::Sidekiq.with_sidekiq_options`. The options apply to all requests made in the block. For example, to send urgent requests to a higher priority queue:
|
|
238
251
|
|
|
239
252
|
```ruby
|
|
240
253
|
PatientHttp::Sidekiq.with_sidekiq_options(queue: "high_priority") do
|
|
@@ -242,60 +255,69 @@ PatientHttp::Sidekiq.with_sidekiq_options(queue: "high_priority") do
|
|
|
242
255
|
end
|
|
243
256
|
```
|
|
244
257
|
|
|
245
|
-
|
|
258
|
+
Sidekiq applies the options with its `set` method, so any Sidekiq job option, such as `queue` or `retry`, is allowed. The options work as follows:
|
|
259
|
+
|
|
260
|
+
- If the options include a `queue`, the callback job that calls `on_complete` or `on_error` uses that queue as well. As a result, the request keeps the same priority from start to finish.
|
|
261
|
+
- Nested blocks merge their options, and the innermost values take precedence. When a block exits, the previous options are restored, even if the block raises an error.
|
|
262
|
+
- Requests made in the block always go through the Sidekiq queue, even when the processor runs in the current process, so that Sidekiq applies the options. For more information, see [Direct execution](#direct-execution).
|
|
246
263
|
|
|
247
|
-
|
|
248
|
-
- Nested blocks merge their options, and the innermost values take precedence. The previous options are restored when the block exits, even if the block raises an error.
|
|
249
|
-
- Requests made in the block always go through the Sidekiq queue, even when the processor runs in the current process, so that Sidekiq applies the options (see Direct Execution below).
|
|
264
|
+
### Direct execution
|
|
250
265
|
|
|
251
|
-
|
|
266
|
+
When a request is made in a process where the processor is running, usually a Sidekiq server process, the request skips the Sidekiq queue and goes straight to the processor. This saves a round trip through Redis. Otherwise, the request behaves the same as an enqueued request:
|
|
252
267
|
|
|
253
|
-
|
|
268
|
+
- The request can always be re-enqueued. The gem adds the request to the crash-recovery registry before the call returns. If the processor shuts down or the process crashes, the request is enqueued as a `RequestWorker` job. If the gem can't write the registry entry, for example because Redis is unavailable, the call raises an error, as a failed enqueue does.
|
|
269
|
+
- If the processor is at capacity or stops accepting requests, the request is enqueued through Sidekiq instead, and the normal Sidekiq retry behavior applies.
|
|
254
270
|
|
|
255
|
-
|
|
256
|
-
- If the processor is at max capacity or stops accepting requests, the request is enqueued through Sidekiq instead, and the normal Sidekiq retry behavior applies from there.
|
|
257
|
-
- Requests made in a `with_sidekiq_options` block always go through the Sidekiq queue, so that Sidekiq applies the options (queue routing, scheduling, retry). Use this to route specific requests to a dedicated Sidekiq process.
|
|
258
|
-
- Options set with `config.sidekiq_options` (including a `queue`) do not apply to direct-executed requests, because no Sidekiq job is created. If every request must go through the configured queue (for example, to run all requests on a dedicated Sidekiq process), set `config.direct_execution = false`.
|
|
259
|
-
- Direct execution is disabled when `Sidekiq::Testing` is enabled, so tests can observe enqueued jobs as usual.
|
|
271
|
+
Direct execution has these limits:
|
|
260
272
|
|
|
261
|
-
|
|
273
|
+
- Requests made in a `with_sidekiq_options` block always go through the Sidekiq queue, so that Sidekiq applies the options, such as queue routing, scheduling, and retries. Use a block to send specific requests to a dedicated Sidekiq process.
|
|
274
|
+
- Options set with `config.sidekiq_options`, including `queue`, don't apply to direct requests, because no Sidekiq job is created.
|
|
275
|
+
- Direct execution is off when `Sidekiq::Testing` is enabled, so that tests can check enqueued jobs as usual.
|
|
262
276
|
|
|
263
|
-
|
|
277
|
+
To turn off direct execution, set `config.direct_execution = false`. Turn it off in any of these cases:
|
|
264
278
|
|
|
265
|
-
|
|
279
|
+
- You send all requests to a dedicated queue with `config.sidekiq_options`, for example to run them on a dedicated Sidekiq process.
|
|
280
|
+
- Sidekiq client or server middleware must run for every request.
|
|
281
|
+
- Every request must appear as an enqueued job in Sidekiq metrics and the Web UI.
|
|
282
|
+
|
|
283
|
+
### Named processors
|
|
284
|
+
|
|
285
|
+
By default, all requests share one processor and one `max_connections` limit. If one process runs workloads with very different profiles, such as slow LLM API calls and fast webhook deliveries, a burst of one workload can use all the capacity that the other needs. Named processor profiles keep the workloads separate:
|
|
266
286
|
|
|
267
287
|
```ruby
|
|
268
|
-
PatientHttp
|
|
288
|
+
PatientHttp.configure do |config|
|
|
269
289
|
config.processor(:llm, max_connections: 200, request_timeout: 120)
|
|
270
290
|
config.processor(:webhooks, max_connections: 64, request_timeout: 10)
|
|
271
291
|
end
|
|
272
292
|
```
|
|
273
293
|
|
|
274
|
-
Each profile runs as an independent processor in the process, with its own capacity, timeouts, and threads. Profile options override the top-level configuration
|
|
294
|
+
Each profile runs as an independent processor in the process, with its own capacity, timeouts, and threads. Profile options override the top-level configuration. The profiles share every option that they don't override, such as secrets, preprocessors, payload stores, encryption, and the logger. Declare a profile with no options, such as `config.processor(:bulk)`, to run a separate processor that uses the top-level options. The `:default` processor always exists. To override its options, declare `config.processor(:default, ...)`.
|
|
275
295
|
|
|
276
|
-
|
|
296
|
+
To send a request to a processor, use any of these methods:
|
|
277
297
|
|
|
278
298
|
```ruby
|
|
279
|
-
#
|
|
280
|
-
PatientHttp
|
|
299
|
+
# An option on the request method.
|
|
300
|
+
PatientHttp.get(url, callback: MyCallback, processor: :llm)
|
|
281
301
|
|
|
282
|
-
#
|
|
302
|
+
# A request object. The processor is kept through serialization, retries, and crash recovery.
|
|
283
303
|
request = PatientHttp::Request.new(:get, url, processor: :llm)
|
|
284
304
|
|
|
285
|
-
#
|
|
305
|
+
# A request template.
|
|
286
306
|
template = PatientHttp::RequestTemplate.new(base_url: url, processor: :llm)
|
|
287
307
|
|
|
288
|
-
#
|
|
308
|
+
# A block.
|
|
289
309
|
PatientHttp::Sidekiq.with_sidekiq_options("processor" => "webhooks") do
|
|
290
|
-
PatientHttp
|
|
310
|
+
PatientHttp.get(url, callback: MyCallback)
|
|
291
311
|
end
|
|
292
312
|
```
|
|
293
313
|
|
|
294
|
-
|
|
314
|
+
A request that names a processor that isn't declared in the process making the request raises `PatientHttp::UnknownProcessorError`, so a misspelled name fails where the request is made. Declare processors in every process that makes requests, not only in the Sidekiq server, for example by declaring them outside a `Sidekiq.configure_server` block.
|
|
295
315
|
|
|
296
|
-
|
|
316
|
+
The processor name is saved in the job arguments, so Sidekiq retries and crash recovery send the request to the same processor. If a job names a processor that isn't configured in the process that runs it, the job raises `PatientHttp::UnknownProcessorError`, and Sidekiq retries it. As a result, you can roll out a new profile name gradually. Jobs enqueued by earlier versions of the gem run on the `:default` processor.
|
|
297
317
|
|
|
298
|
-
|
|
318
|
+
### Use request templates
|
|
319
|
+
|
|
320
|
+
To share settings across requests to the same API, use `PatientHttp::RequestTemplate`:
|
|
299
321
|
|
|
300
322
|
```ruby
|
|
301
323
|
class ApiService
|
|
@@ -327,11 +349,11 @@ class ApiService
|
|
|
327
349
|
end
|
|
328
350
|
```
|
|
329
351
|
|
|
330
|
-
|
|
352
|
+
If the template doesn't set a `timeout`, the configured `request_timeout` applies.
|
|
331
353
|
|
|
332
|
-
|
|
354
|
+
### Use the RequestHelper module
|
|
333
355
|
|
|
334
|
-
|
|
356
|
+
For a class that makes many requests, include `PatientHttp::RequestHelper`. The module adds the `async_get`, `async_head`, `async_post`, `async_put`, `async_patch`, `async_delete`, `async_query`, and `async_request` instance methods. To set shared options such as `base_url`, `headers`, and `timeout`, use the `request_template` class method:
|
|
335
357
|
|
|
336
358
|
```ruby
|
|
337
359
|
class NotificationService
|
|
@@ -358,18 +380,18 @@ class NotificationService
|
|
|
358
380
|
end
|
|
359
381
|
```
|
|
360
382
|
|
|
361
|
-
The `async_*` methods
|
|
383
|
+
The `async_*` methods take the same options as `PatientHttp.get`, `PatientHttp.post`, and the other module methods. Paths are relative to the template's `base_url`.
|
|
362
384
|
|
|
363
|
-
|
|
385
|
+
For the full `RequestHelper` documentation, see the [patient_http documentation](https://github.com/bdurand/patient_http#use-the-requesthelper-module).
|
|
364
386
|
|
|
365
|
-
### Callback
|
|
387
|
+
### Callback arguments
|
|
366
388
|
|
|
367
|
-
|
|
389
|
+
To pass data to your callbacks, use the `callback_args` option:
|
|
368
390
|
|
|
369
391
|
```ruby
|
|
370
392
|
class FetchDataCallback
|
|
371
393
|
def on_complete(response)
|
|
372
|
-
#
|
|
394
|
+
# Read callback_args with symbol or string keys.
|
|
373
395
|
user_id = response.callback_args[:user_id]
|
|
374
396
|
request_timestamp = response.callback_args[:request_timestamp]
|
|
375
397
|
|
|
@@ -389,7 +411,7 @@ class FetchDataCallback
|
|
|
389
411
|
end
|
|
390
412
|
end
|
|
391
413
|
|
|
392
|
-
# Pass data
|
|
414
|
+
# Pass data with the callback_args option.
|
|
393
415
|
PatientHttp.get(
|
|
394
416
|
"https://api.example.com/users/#{user_id}",
|
|
395
417
|
callback: FetchDataCallback,
|
|
@@ -400,43 +422,43 @@ PatientHttp.get(
|
|
|
400
422
|
)
|
|
401
423
|
```
|
|
402
424
|
|
|
403
|
-
|
|
425
|
+
The `callback_args` value follows these rules:
|
|
404
426
|
|
|
405
|
-
-
|
|
406
|
-
- Hash keys
|
|
407
|
-
-
|
|
408
|
-
-
|
|
427
|
+
- It must be a Hash, or respond to `to_h`, and contain only JSON-native types: `nil`, `true`, `false`, `String`, `Integer`, `Float`, `Array`, and `Hash`.
|
|
428
|
+
- Hash keys are converted to strings, including the keys of nested hashes and of hashes in arrays.
|
|
429
|
+
- You can read the arguments with symbol or string keys: `callback_args[:user_id]` or `callback_args["user_id"]`.
|
|
430
|
+
- Reading a key that isn't set raises a `KeyError`. To get a default value instead, use `callback_args.fetch(:user_id, nil)`.
|
|
409
431
|
|
|
410
|
-
###
|
|
432
|
+
### Protect sensitive data
|
|
411
433
|
|
|
412
|
-
|
|
434
|
+
The gem stores requests and responses in Redis so that it can run the callback job. If they contain sensitive data, that data is stored in plain text.
|
|
413
435
|
|
|
414
|
-
|
|
436
|
+
To protect the data, configure encryption. The gem then encrypts all request and response data before it stores the data in Redis, and decrypts the data when it reads it.
|
|
415
437
|
|
|
416
|
-
####
|
|
438
|
+
#### Use an encryption key
|
|
417
439
|
|
|
418
|
-
The simplest option is `encryption_key
|
|
440
|
+
The simplest option is `encryption_key=`. It uses [ActiveSupport::MessageEncryptor](https://api.rubyonrails.org/classes/ActiveSupport/MessageEncryptor.html) with AES-256-GCM:
|
|
419
441
|
|
|
420
442
|
```ruby
|
|
421
|
-
PatientHttp
|
|
443
|
+
PatientHttp.configure do |config|
|
|
422
444
|
config.encryption_key = ENV["PATIENT_HTTP_ENCRYPTION_KEY"]
|
|
423
445
|
end
|
|
424
446
|
```
|
|
425
447
|
|
|
426
|
-
|
|
448
|
+
To rotate keys, pass an array. The first key encrypts data, and all keys are tried for decryption:
|
|
427
449
|
|
|
428
450
|
```ruby
|
|
429
|
-
PatientHttp
|
|
451
|
+
PatientHttp.configure do |config|
|
|
430
452
|
config.encryption_key = [ENV["PATIENT_HTTP_ENCRYPTION_KEY"], ENV["PATIENT_HTTP_OLD_KEY"]]
|
|
431
453
|
end
|
|
432
454
|
```
|
|
433
455
|
|
|
434
|
-
####
|
|
456
|
+
#### Use custom callables
|
|
435
457
|
|
|
436
|
-
|
|
458
|
+
To use another encryption library, provide callables that take and return raw bytes as a String:
|
|
437
459
|
|
|
438
460
|
```ruby
|
|
439
|
-
PatientHttp
|
|
461
|
+
PatientHttp.configure do |config|
|
|
440
462
|
config.encryption { |bytes| MyEncryption.encrypt(bytes) }
|
|
441
463
|
config.decryption { |bytes| MyEncryption.decrypt(bytes) }
|
|
442
464
|
end
|
|
@@ -445,152 +467,166 @@ end
|
|
|
445
467
|
You can also pass any object that responds to `call`:
|
|
446
468
|
|
|
447
469
|
```ruby
|
|
448
|
-
PatientHttp
|
|
470
|
+
PatientHttp.configure do |config|
|
|
449
471
|
config.encryption(->(bytes) { MyEncryption.encrypt(bytes) })
|
|
450
472
|
config.decryption(->(bytes) { MyEncryption.decrypt(bytes) })
|
|
451
473
|
end
|
|
452
474
|
```
|
|
453
475
|
|
|
476
|
+
To keep API tokens out of the queue entirely, use secrets instead. For secrets, request preprocessors, and payload stores for large payloads, see the [patient_http documentation](https://github.com/bdurand/patient_http#sensitive-and-large-payloads).
|
|
477
|
+
|
|
454
478
|
## Configuration
|
|
455
479
|
|
|
456
|
-
|
|
480
|
+
All configuration is optional. To set options, call `PatientHttp.configure` in an initializer. The method yields this gem's configuration. `PatientHttp::Sidekiq.configure` does the same thing, but `PatientHttp.configure` keeps the initializer free of references to the job system.
|
|
481
|
+
|
|
482
|
+
Every call yields the same configuration object, so options accumulate. Several initializers can each set options without overwriting one another.
|
|
457
483
|
|
|
458
484
|
```ruby
|
|
459
|
-
PatientHttp
|
|
460
|
-
# Maximum concurrent HTTP requests (default: 256)
|
|
485
|
+
PatientHttp.configure do |config|
|
|
486
|
+
# Maximum concurrent HTTP requests (default: 256).
|
|
461
487
|
config.max_connections = 256
|
|
462
488
|
|
|
463
|
-
# Default timeout for HTTP requests in seconds (default: 60)
|
|
489
|
+
# Default timeout for HTTP requests in seconds (default: 60).
|
|
464
490
|
config.request_timeout = 60
|
|
465
491
|
|
|
466
|
-
# Maximum number of host clients to pool (default: 100)
|
|
492
|
+
# Maximum number of host clients to pool (default: 100).
|
|
467
493
|
config.connection_pool_size = 100
|
|
468
494
|
|
|
469
|
-
#
|
|
495
|
+
# Timeout in seconds to open a connection, including the TCP connect and the
|
|
496
|
+
# TLS handshake (default: nil, no limit). It doesn't limit the wait for a
|
|
497
|
+
# response; request_timeout does that.
|
|
470
498
|
config.connection_timeout = 10
|
|
471
499
|
|
|
472
|
-
#
|
|
500
|
+
# TCP keepalive for pooled connections (default: nil, the kernel sends no
|
|
501
|
+
# probes). A number sets the idle seconds before the first probe. A Hash also
|
|
502
|
+
# sets the interval and the probe count, for example
|
|
503
|
+
# {idle: 30, interval: 10, count: 3}. The Hash must contain :idle. The
|
|
504
|
+
# :interval default is 10 seconds, and the :count default is 3 probes.
|
|
505
|
+
config.tcp_keepalive = 30
|
|
506
|
+
|
|
507
|
+
# Seconds that sent data can stay unacknowledged before the kernel closes the
|
|
508
|
+
# connection (default: nil, the kernel default applies). Sets
|
|
509
|
+
# TCP_USER_TIMEOUT, which is available only on Linux.
|
|
510
|
+
config.tcp_user_timeout = 30
|
|
511
|
+
|
|
512
|
+
# Number of retries for failed requests (default: 3).
|
|
473
513
|
config.retries = 3
|
|
474
514
|
|
|
475
|
-
# HTTP
|
|
476
|
-
#
|
|
515
|
+
# HTTP or HTTPS proxy URL (default: nil). Supports authentication, for
|
|
516
|
+
# example "http://user:pass@proxy.example.com:8080".
|
|
477
517
|
config.proxy_url = "http://proxy.example.com:8080"
|
|
478
518
|
|
|
479
|
-
# Default User-Agent header for all requests (default: "PatientHttp")
|
|
519
|
+
# Default User-Agent header for all requests (default: "PatientHttp").
|
|
480
520
|
config.user_agent = "MyApp/1.0"
|
|
481
521
|
|
|
482
|
-
# Timeout for graceful shutdown in seconds (default: the Sidekiq
|
|
483
|
-
#
|
|
484
|
-
# shutdown timeout.
|
|
522
|
+
# Timeout for graceful shutdown in seconds (default: the Sidekiq shutdown
|
|
523
|
+
# timeout minus 2 seconds). Must be less than Sidekiq's shutdown timeout.
|
|
485
524
|
config.shutdown_timeout = 23
|
|
486
525
|
|
|
487
|
-
# Maximum response body size in bytes (default: 1MB)
|
|
488
|
-
#
|
|
526
|
+
# Maximum response body size in bytes (default: 1MB). Larger responses raise
|
|
527
|
+
# ResponseTooLargeError.
|
|
489
528
|
config.max_response_size = 1024 * 1024
|
|
490
529
|
|
|
491
|
-
# Maximum number of redirects to follow (default: 5
|
|
530
|
+
# Maximum number of redirects to follow (default: 5; 0 turns off redirects).
|
|
492
531
|
config.max_redirects = 5
|
|
493
532
|
|
|
494
|
-
# Whether to raise HttpError for non-2xx responses by default (default: false)
|
|
533
|
+
# Whether to raise HttpError for non-2xx responses by default (default: false).
|
|
495
534
|
config.raise_error_responses = false
|
|
496
535
|
|
|
497
|
-
# Heartbeat interval for crash recovery in seconds (default: 60)
|
|
536
|
+
# Heartbeat interval for crash recovery in seconds (default: 60).
|
|
498
537
|
config.heartbeat_interval = 60
|
|
499
538
|
|
|
500
|
-
#
|
|
501
|
-
#
|
|
539
|
+
# Seconds without a heartbeat after which a request is re-enqueued
|
|
540
|
+
# (default: 300).
|
|
502
541
|
config.orphan_threshold = 300
|
|
503
542
|
|
|
504
|
-
# Size
|
|
505
|
-
#
|
|
506
|
-
# store is configured.
|
|
543
|
+
# Size in bytes above which payloads are stored externally when a payload
|
|
544
|
+
# store is configured (default: 64KB).
|
|
507
545
|
config.payload_store_threshold = 64 * 1024
|
|
508
546
|
|
|
509
|
-
# Sidekiq options for RequestWorker and CallbackWorker
|
|
510
|
-
#
|
|
547
|
+
# Sidekiq options for RequestWorker and CallbackWorker (default: nil). To
|
|
548
|
+
# override them for specific requests, use
|
|
549
|
+
# PatientHttp::Sidekiq.with_sidekiq_options.
|
|
511
550
|
config.sidekiq_options = {queue: "patient_http", retry: 5}
|
|
512
551
|
|
|
513
552
|
# Whether the URL, HTTP method, and processor of each in-flight request are
|
|
514
|
-
# recorded so the Web UI can list them (default: true).
|
|
553
|
+
# recorded so that the Web UI can list them (default: true).
|
|
515
554
|
config.inflight_details = true
|
|
516
555
|
|
|
517
|
-
# Sanitizer
|
|
518
|
-
# name, password, query string, and fragment
|
|
556
|
+
# Sanitizer that runs on a URL before it's recorded (default: removes the
|
|
557
|
+
# user name, password, query string, and fragment).
|
|
519
558
|
config.inflight_url_sanitizer { |url| url.sub(%r{/users/\d+}, "/users/:id") }
|
|
520
559
|
|
|
521
560
|
# Whether requests made in a process with a running processor skip the
|
|
522
|
-
# Sidekiq queue and go straight to the processor (default: true).
|
|
523
|
-
#
|
|
524
|
-
#
|
|
561
|
+
# Sidekiq queue and go straight to the processor (default: true). Sidekiq
|
|
562
|
+
# options, including a queue, don't apply to these requests. Set this to
|
|
563
|
+
# false to send every request through the queue.
|
|
525
564
|
config.direct_execution = true
|
|
526
565
|
|
|
527
|
-
# Size of the gem's dedicated Redis pool
|
|
528
|
-
# (default: nil,
|
|
529
|
-
# floor of 10)
|
|
566
|
+
# Size of the gem's dedicated Redis pool, which the gem's own threads use
|
|
567
|
+
# (default: nil, based on completion_threads with a minimum of 10).
|
|
530
568
|
config.redis_pool_size = nil
|
|
531
569
|
|
|
532
|
-
# Checkout timeout in seconds for the dedicated Redis pool (default: 5)
|
|
570
|
+
# Checkout timeout in seconds for the dedicated Redis pool (default: 5).
|
|
533
571
|
config.redis_pool_timeout = 5
|
|
534
572
|
|
|
535
|
-
# Seconds between flushes of
|
|
536
|
-
#
|
|
573
|
+
# Seconds between flushes of local stats to Redis (default: 5; 0 writes every
|
|
574
|
+
# event immediately).
|
|
537
575
|
config.stats_flush_interval = 5
|
|
538
576
|
|
|
539
|
-
# Number of threads that decode responses and deliver results (default: 2)
|
|
577
|
+
# Number of threads that decode responses and deliver results (default: 2).
|
|
540
578
|
config.completion_threads = 2
|
|
541
579
|
|
|
542
|
-
# Maximum connections
|
|
580
|
+
# Maximum connections to each host (default: nil, no limit).
|
|
543
581
|
config.max_connections_per_host = 32
|
|
544
582
|
|
|
545
|
-
# Named processor profiles for workload isolation
|
|
583
|
+
# Named processor profiles for workload isolation. See Named processors.
|
|
546
584
|
config.processor(:llm, max_connections: 200, request_timeout: 120)
|
|
547
585
|
config.processor(:webhooks, max_connections: 64, request_timeout: 10)
|
|
548
586
|
|
|
549
|
-
# Handler
|
|
587
|
+
# Handler that runs when a callback job uses up all of its Sidekiq retries.
|
|
550
588
|
config.on_retries_exhausted { |error| MyAlertService.notify(error) }
|
|
551
589
|
|
|
552
|
-
#
|
|
590
|
+
# Logger (default: Sidekiq.logger).
|
|
553
591
|
config.logger = Rails.logger
|
|
554
592
|
|
|
555
|
-
# Encryption for sensitive data
|
|
593
|
+
# Encryption for sensitive data. See Protect sensitive data.
|
|
556
594
|
config.encryption_key = ENV["PATIENT_HTTP_ENCRYPTION_KEY"]
|
|
557
595
|
end
|
|
558
596
|
```
|
|
559
597
|
|
|
560
|
-
|
|
598
|
+
For all options, see the [Configuration](lib/patient_http/sidekiq/configuration.rb) class. For the HTTP options that this gem inherits, see the [patient_http documentation](https://github.com/bdurand/patient_http#configuration).
|
|
561
599
|
|
|
562
|
-
|
|
563
|
-
> You **must** call `PatientHttp::Sidekiq.configure` or `PatientHttp::Sidekiq.register_handler` in order to register the request handler and start processing requests. If you do not call either method, you will get errors making HTTP requests with `PatientHttp.execute`.
|
|
600
|
+
### Tuning tips
|
|
564
601
|
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
- `
|
|
568
|
-
- `
|
|
569
|
-
- `
|
|
570
|
-
- `
|
|
571
|
-
- `
|
|
572
|
-
- `
|
|
573
|
-
- `
|
|
574
|
-
- `
|
|
575
|
-
- `
|
|
576
|
-
- `
|
|
602
|
+
- `max_connections`: Set this based on your system's resources. Each connection uses memory and a file descriptor. A tuned system with enough resources can handle thousands of concurrent connections.
|
|
603
|
+
- `request_timeout`: Set this based on the response times of the APIs that you call. AI APIs can take minutes to respond while they generate content.
|
|
604
|
+
- `connection_pool_size`: Sets the maximum number of hosts whose connections are kept open. Increase it if your application calls many different hosts.
|
|
605
|
+
- `connection_timeout`: Limits only the TCP connect and the TLS handshake. Set it to fail fast when a host doesn't answer. It doesn't limit the wait for a response, because `request_timeout` controls the full exchange.
|
|
606
|
+
- `retries`: Sets the number of times to retry a failed request before the gem calls the error callback.
|
|
607
|
+
- `max_response_size`: Limits the size of HTTP responses to prevent high memory use from unexpectedly large responses. Responses are serialized in Sidekiq job arguments, and very large responses can slow Redis down. Text response bodies are compressed to save space. Binary bodies are Base64 encoded, which increases their size by about 33%.
|
|
608
|
+
- `payload_store_threshold`: Lower this if large payloads slow your queue down. Higher values avoid extra reads and writes to the payload store.
|
|
609
|
+
- `max_connections_per_host`: Limits the sockets open to each host. Make sure that the process file descriptor limit covers `max_connections`, plus idle pooled connections, plus the application's own connections. Raise the limit if needed.
|
|
610
|
+
- `shutdown_timeout`: Must be less than the process supervisor's stop timeout, so that in-flight requests finish before a hard kill. The default is based on Sidekiq's shutdown timeout. If a container orchestrator or init system also stops the process, check its stop timeout as well.
|
|
611
|
+
- `completion_threads`: Increase this when result delivery does heavy work, such as serialization or encryption, and finished requests wait for a thread.
|
|
612
|
+
- `redis_pool_size`: The automatic size covers the gem's own threads. Increase it if a high request rate makes request registration or result delivery wait for a connection.
|
|
613
|
+
- `heartbeat_interval` and `orphan_threshold`: For high-volume workloads, set `heartbeat_interval` as high as your recovery objective allows, while you keep it less than `orphan_threshold`. Fewer heartbeats mean fewer writes to Redis.
|
|
577
614
|
|
|
578
615
|
> [!WARNING]
|
|
579
|
-
>
|
|
616
|
+
> Don't install `hiredis-client` in processes that run the async processor. The hiredis driver does blocking I/O that doesn't yield to the fiber scheduler. A Redis call on the reactor thread, for example from a custom processor observer, stalls every in-flight HTTP request. The gem logs a warning at startup if it detects the hiredis driver.
|
|
580
617
|
|
|
581
618
|
> [!IMPORTANT]
|
|
619
|
+
> When the processor reaches `max_connections`, a new request raises an error in its Sidekiq job, and Sidekiq retries the job.
|
|
582
620
|
>
|
|
583
|
-
>
|
|
584
|
-
>
|
|
585
|
-
> In contrast, slow synchronous HTTP requests will fill up the Sidekiq worker pool and block new jobs from being dequeued until a worker thread becomes free.
|
|
621
|
+
> Synchronous HTTP requests in Sidekiq jobs behave differently. Slow synchronous requests fill the Sidekiq worker pool, and no new jobs start until a worker thread is free.
|
|
586
622
|
>
|
|
587
|
-
>
|
|
623
|
+
> The asynchronous behavior is usually better, because Sidekiq keeps running other jobs, and thousands of jobs don't pile up in the queue.
|
|
588
624
|
|
|
589
|
-
## Metrics and
|
|
625
|
+
## Metrics and monitoring
|
|
590
626
|
|
|
591
627
|
### Web UI
|
|
592
628
|
|
|
593
|
-
If you
|
|
629
|
+
If you use the Sidekiq Web UI, you can add a tab that shows the async HTTP processor stats. The tab requires Sidekiq 7.3 or later.
|
|
594
630
|
|
|
595
631
|
```ruby
|
|
596
632
|
# config/routes.rb (Rails)
|
|
@@ -599,45 +635,36 @@ require "patient_http/sidekiq/web"
|
|
|
599
635
|
mount Sidekiq::Web => "/sidekiq"
|
|
600
636
|
```
|
|
601
637
|
|
|
602
|
-
The
|
|
603
|
-
|
|
604
|
-
-
|
|
605
|
-
|
|
606
|
-
- The requests that have been in flight the longest, with their URL, HTTP method, processor, and age
|
|
607
|
-
-
|
|
638
|
+
The tab shows the following information:
|
|
639
|
+
|
|
640
|
+
- Total requests, errors, times at capacity, average duration, and current capacity utilization.
|
|
641
|
+
- Capacity, utilization, requests, errors, times at capacity, average duration, and the in-flight high-water mark for each processor, when more than one [named processor](#named-processors) is configured.
|
|
642
|
+
- The requests that have been in flight the longest, with their URL, HTTP method, processor, and age.
|
|
643
|
+
- The number of in-flight requests in each process.
|
|
608
644
|
|
|
609
|
-
The
|
|
610
|
-
cover the processes that are currently running and can lag by a few seconds.
|
|
645
|
+
The processor numbers come from the capacity that each process publishes with its heartbeat. They cover the processes that are running, and they can be a few seconds old.
|
|
611
646
|
|
|
612
|
-
The high-water mark is the most requests one process held on that processor at once, so
|
|
613
|
-
with `max_connections`, which is also per process, rather than with the capacity column, which is the
|
|
614
|
-
sum across the running processes. The count only rises when a processor accepts a request, so the
|
|
615
|
-
mark is exact rather than sampled. It covers everything since the statistics were last cleared.
|
|
647
|
+
The high-water mark is the most requests that one process held on that processor at once. Compare it with `max_connections`, which is also per process, and not with the capacity column, which is the sum across running processes. The count rises only when a processor accepts a request, so the mark is exact, not sampled. It covers everything since the stats were last cleared.
|
|
616
648
|
|
|
617
|
-
#### In-
|
|
649
|
+
#### In-flight requests
|
|
618
650
|
|
|
619
|
-
The URL, HTTP method, and processor of each in-flight request
|
|
620
|
-
crash-recovery record, and the dashboard lists the 50 oldest. A request left behind by a process
|
|
621
|
-
that died stays listed until the orphan collector re-enqueues it, so this is also where you see what
|
|
622
|
-
a process was working on when it stopped.
|
|
651
|
+
The gem records the URL, HTTP method, and processor of each in-flight request next to its crash-recovery record. The dashboard lists the 50 oldest requests. A request left behind by a process that died stays listed until the orphan collector re-enqueues it. As a result, the list also shows what a process was working on when it stopped.
|
|
623
652
|
|
|
624
|
-
The
|
|
625
|
-
removed and the scheme, host, and path are kept. Paths can still carry identifiers, so you can
|
|
626
|
-
redact more, or record nothing at all:
|
|
653
|
+
The gem sanitizes each URL before it records it. It removes the user name, password, query string, and fragment, and keeps the scheme, host, and path. Paths can still contain identifiers, so you can remove more, or record nothing:
|
|
627
654
|
|
|
628
655
|
```ruby
|
|
629
|
-
PatientHttp
|
|
630
|
-
#
|
|
656
|
+
PatientHttp.configure do |config|
|
|
657
|
+
# Remove more of the URL.
|
|
631
658
|
config.inflight_url_sanitizer { |url| url.sub(%r{/users/\d+}, "/users/:id") }
|
|
632
659
|
|
|
633
|
-
# Or keep URLs out of Redis
|
|
660
|
+
# Or keep URLs out of Redis.
|
|
634
661
|
config.inflight_details = false
|
|
635
662
|
end
|
|
636
663
|
```
|
|
637
664
|
|
|
638
|
-
###
|
|
665
|
+
### Monitoring callbacks
|
|
639
666
|
|
|
640
|
-
|
|
667
|
+
To send metrics to your monitoring system, register `after_completion` and `after_error` callbacks:
|
|
641
668
|
|
|
642
669
|
```ruby
|
|
643
670
|
PatientHttp::Sidekiq.after_completion do |response|
|
|
@@ -651,14 +678,14 @@ PatientHttp::Sidekiq.after_error do |error|
|
|
|
651
678
|
end
|
|
652
679
|
```
|
|
653
680
|
|
|
654
|
-
You can register
|
|
681
|
+
You can register more than one callback. Callbacks run in the order that you register them.
|
|
655
682
|
|
|
656
|
-
###
|
|
683
|
+
### Handle exhausted retries
|
|
657
684
|
|
|
658
|
-
When a callback
|
|
685
|
+
When a callback job uses up all of its Sidekiq retries, the gem can call an `on_retries_exhausted` handler. Use the handler to send an alert or to record that a callback failed permanently. The handler receives the same error object as `on_error`:
|
|
659
686
|
|
|
660
687
|
```ruby
|
|
661
|
-
PatientHttp
|
|
688
|
+
PatientHttp.configure do |config|
|
|
662
689
|
config.on_retries_exhausted do |error|
|
|
663
690
|
Sentry.capture_message("Callback permanently failed: #{error.message}")
|
|
664
691
|
DeadLetterRecord.create!(
|
|
@@ -672,101 +699,78 @@ end
|
|
|
672
699
|
You can also assign any object that responds to `call`:
|
|
673
700
|
|
|
674
701
|
```ruby
|
|
675
|
-
PatientHttp
|
|
702
|
+
PatientHttp.configure do |config|
|
|
676
703
|
config.on_retries_exhausted = ->(error) { MyAlertService.notify(error) }
|
|
677
704
|
end
|
|
678
705
|
```
|
|
679
706
|
|
|
680
707
|
> [!NOTE]
|
|
681
|
-
> The `on_retries_exhausted` handler
|
|
708
|
+
> The gem calls the `on_retries_exhausted` handler only for callback jobs that deliver an error to `on_error`. If the handler raises an exception, the gem logs a warning, and the dead job cleanup continues as usual.
|
|
682
709
|
|
|
683
|
-
## Shutdown
|
|
710
|
+
## Shutdown behavior
|
|
684
711
|
|
|
685
|
-
The async HTTP processor
|
|
712
|
+
The async HTTP processor follows Sidekiq's lifecycle events:
|
|
686
713
|
|
|
687
|
-
1. **Startup
|
|
688
|
-
2. **Quiet (TSTP signal)
|
|
689
|
-
3. **Shutdown
|
|
714
|
+
1. **Startup**: The processor starts when Sidekiq starts.
|
|
715
|
+
2. **Quiet** (TSTP signal): The processor stops accepting new requests but continues to run in-flight requests.
|
|
716
|
+
3. **Shutdown**: The processor waits up to `shutdown_timeout` seconds for in-flight requests to finish.
|
|
690
717
|
|
|
691
|
-
### Incomplete
|
|
718
|
+
### Incomplete requests
|
|
692
719
|
|
|
693
|
-
If requests are still in
|
|
720
|
+
If requests are still in flight when the shutdown timeout ends, the gem interrupts them and re-enqueues their Sidekiq jobs. The jobs run again when Sidekiq restarts or on another Sidekiq process, so no work is lost during deployments or restarts.
|
|
694
721
|
|
|
695
|
-
|
|
696
|
-
- The **original Sidekiq job** is automatically re-enqueued
|
|
697
|
-
- Re-enqueued jobs will be processed again when Sidekiq restarts
|
|
722
|
+
### Crash recovery
|
|
698
723
|
|
|
699
|
-
|
|
724
|
+
The gem recovers requests from processes that crash:
|
|
700
725
|
|
|
701
|
-
|
|
726
|
+
1. **Heartbeats**: Every `heartbeat_interval` seconds, each process updates the heartbeat times of its in-flight requests in Redis.
|
|
727
|
+
2. **Orphan detection**: One process at a time checks for requests that haven't had a heartbeat in `orphan_threshold` seconds.
|
|
728
|
+
3. **Re-enqueue**: The gem re-enqueues the Sidekiq jobs of the orphaned requests.
|
|
702
729
|
|
|
703
|
-
|
|
730
|
+
As a result, if a Sidekiq process crashes, another process retries its in-flight requests.
|
|
704
731
|
|
|
705
|
-
|
|
706
|
-
2. **Orphan Detection:** One processor periodically checks for requests that haven't received a heartbeat update in `orphan_threshold` seconds
|
|
707
|
-
3. **Automatic Re-enqueue:** Orphaned requests have their original Sidekiq jobs re-enqueued
|
|
732
|
+
Crash recovery gives at-least-once delivery. If a process crashes at the wrong moment, such as between a re-enqueue and the removal of the registry entry, a request can run more than once, and its callback can run more than once. Make your callbacks idempotent. A request is durable once the call that makes it returns. A crash during the call behaves like a failed enqueue, and the caller never gets an acknowledgment.
|
|
708
733
|
|
|
709
|
-
|
|
710
|
-
|
|
711
|
-
Crash recovery gives at-least-once delivery. If a process crashes at the wrong moment (for example, between a re-enqueue and the removal of the registry entry), a request can execute more than once and its callback can fire more than once. Make your callbacks idempotent. A request is durable once the call that submits it returns; a crash during the call behaves like a failed enqueue, and the caller never received an acknowledgment.
|
|
712
|
-
|
|
713
|
-
A request whose result cannot be handed to a callback job keeps its registry entry as well, so the same recovery re-enqueues it. When the failure is one that trying again cannot fix, because the result cannot be serialized, the request is not kept: it is counted as an `undeliverable_result` error and its job is moved to the Sidekiq dead set, where you can inspect it and retry it by hand.
|
|
734
|
+
If the gem can't hand a result to a callback job, it keeps the request's registry entry, so crash recovery re-enqueues the request. The exception is a failure that a retry can't fix, because the result can't be serialized. In that case, the gem records an `undeliverable_result` error and moves the job to the Sidekiq dead set, where you can inspect it and retry it by hand.
|
|
714
735
|
|
|
715
736
|
## Testing
|
|
716
737
|
|
|
717
|
-
The gem supports `Sidekiq::Testing.inline
|
|
718
|
-
|
|
719
|
-
## Installation
|
|
720
|
-
|
|
721
|
-
Add this line to your application's Gemfile:
|
|
722
|
-
|
|
723
|
-
```ruby
|
|
724
|
-
gem "patient_http-sidekiq"
|
|
725
|
-
```
|
|
726
|
-
|
|
727
|
-
Then execute:
|
|
728
|
-
|
|
729
|
-
```bash
|
|
730
|
-
bundle install
|
|
731
|
-
```
|
|
738
|
+
The gem supports `Sidekiq::Testing.inline!`. In inline mode, requests run immediately in the worker thread and block until they finish. As a result, tests can check the full request and response cycle without a running processor.
|
|
732
739
|
|
|
733
740
|
## Contributing
|
|
734
741
|
|
|
735
742
|
Open a pull request on [GitHub](https://github.com/bdurand/patient_http-sidekiq).
|
|
736
743
|
|
|
737
|
-
|
|
744
|
+
Follow the [standardrb](https://github.com/testdouble/standard) style, and run `standardrb --fix` before you submit a pull request.
|
|
738
745
|
|
|
739
|
-
|
|
746
|
+
The tests require a Redis-compatible server. To start one in a local container on port 24455, run this script:
|
|
740
747
|
|
|
741
748
|
```bash
|
|
742
749
|
bin/run-valkey
|
|
743
750
|
```
|
|
744
751
|
|
|
745
|
-
Then run the
|
|
752
|
+
Then run the tests:
|
|
746
753
|
|
|
747
754
|
```bash
|
|
748
755
|
bundle exec rake
|
|
749
756
|
```
|
|
750
757
|
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
To run the test app, first install the dependencies:
|
|
758
|
+
The `test_app` directory has a test app for manual testing. To run it, install its dependencies:
|
|
754
759
|
|
|
755
760
|
```bash
|
|
756
761
|
bundle exec rake test_app:bundle
|
|
757
762
|
```
|
|
758
763
|
|
|
759
|
-
|
|
764
|
+
Then start the server, which runs at http://localhost:9292:
|
|
760
765
|
|
|
761
766
|
```bash
|
|
762
767
|
bundle exec rake test_app
|
|
763
768
|
```
|
|
764
769
|
|
|
765
|
-
## Further
|
|
770
|
+
## Further reading
|
|
766
771
|
|
|
767
772
|
- [Architecture](ARCHITECTURE.md)
|
|
768
773
|
|
|
769
|
-
|
|
770
774
|
## License
|
|
771
775
|
|
|
772
776
|
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|