grape-idempotency 1.1.0 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 115100d8cd1141c7c801a1501ede2db9eff3419a5b2974de93c4b79e7c812fd2
4
- data.tar.gz: 567472c4ab8ad2aae2561d7af0e808c803a025caae34f75492c2b889b6468d7e
3
+ metadata.gz: 482197066de124a0866f6e8a96a3506b0301f892f8cab0ec04ea550a88ba8046
4
+ data.tar.gz: b0eb8cd9a3f5fd8979180ca234501e0be9781d33f079dc1085ef2021713960e9
5
5
  SHA512:
6
- metadata.gz: fde07abe226552ba88086f20fcece9778ef13ee04c17c27376aea2d13386955944caea65ae28ff1d2abdda72bc501600a9afb107229e6da47b4dd02a56b2e5f5
7
- data.tar.gz: 3a9a9fcd8f30d5f1deab9da2c719c54c1a0cbecafb5ce3b1196301bc60983bd3271153f4b02fed795d49d4188255507e0a8445d1fdd37cad1bbe0a0851b32e48
6
+ metadata.gz: 6bb352d73edaf86330aa4585df7ad5124fec16cb10314cf5eb2b338be83b6e8754592615a7ab833e989c3e11d65df15030378e698d51b435d2613078eaf2c779
7
+ data.tar.gz: 40d0a3e83170df6bed8b8daece96dec1241ae953fab54c56490e79dc18ca439a342b26a6cbc9214cfca3f14c059dd2bbd9e2e30bc9a694353693307acf9f2dcb
data/CHANGELOG.md CHANGED
@@ -4,20 +4,59 @@ All changes to `grape-idempotency` will be documented in this file.
4
4
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
5
5
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
- ## [1.1.0] - (Next)
7
+ ## [1.2.0] - 2026-10-15
8
+
9
+ ### Feature
10
+
11
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): Replay the response headers of the original request (e.g. `Content-Type`, `Location` or custom headers), except `Set-Cookie`, hop-by-hop headers, `Content-Length`, `Date`, `Age`, `Via`, `Server-Timing`, `X-Runtime`, `Original-Request` and the configured idempotency key and request id headers - [@jcagarcia](https://github.com/jcagarcia).
12
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): Add the `replay_headers_denylist` configuration option to exclude more response headers from being stored and replayed - [@jcagarcia](https://github.com/jcagarcia).
13
+ * Add `encryption_key` and `encryptor` configuration options for encrypting the requests and responses stored in `Redis` - [@jcagarcia](https://github.com/jcagarcia).
8
14
 
9
15
  ### Fix
10
16
 
11
- * Your contribution here.
17
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): Store and replay binary and non UTF-8 response bodies, which raised an error while storing the response and replaced the original response - [@jcagarcia](https://github.com/jcagarcia).
18
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): Replay the formatted response body byte for byte. APIs not using the JSON format returned a different body when replaying (e.g. `{:id=>1}` became `{"id"=>1}`) - [@jcagarcia](https://github.com/jcagarcia).
19
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): Keep the headers passed to `error!` inside an `idempotent` block in the first response, and replay them - [@jcagarcia](https://github.com/jcagarcia).
20
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): Return the `Original-Request` and idempotency key headers in responses generated by `rescue_from` handlers with Grape < 2.4 - [@jcagarcia](https://github.com/jcagarcia).
12
21
 
13
22
  ### Changed
14
23
 
15
- * Your contribution here.
24
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): Store the final response (status, headers and formatted body) built by Grape, for both `idempotent` blocks and `rescue_from` handlers. Responses stored by previous versions are still replayed - [@jcagarcia](https://github.com/jcagarcia).
25
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): When replaying a response, the code placed after the `idempotent` block in the endpoint (and the `after` callbacks) is no longer run. Previously it ran on every replay - [@jcagarcia](https://github.com/jcagarcia).
26
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): `error!` called inside an `idempotent` block is rendered by Grape's error formatter, as it is without the gem. Previously the message was rendered by the regular formatter (e.g. `error!('Not allowed', 403)` returned `"Not allowed"` instead of `{"error":"Not allowed"}` with `format :json`) - [@jcagarcia](https://github.com/jcagarcia).
27
+ * [#27](https://github.com/jcagarcia/grape-idempotency/issues/27): Responses whose body is a stream or a file are not stored. The idempotency key is released and a warning is logged, so these endpoints are not idempotent - [@jcagarcia](https://github.com/jcagarcia).
28
+ * Store a SHA-256 digest of the request params instead of the params. Requests stored by previous versions are still read. While deploying, instances running a previous version return the conflict response for retries of requests stored by this version - [@jcagarcia](https://github.com/jcagarcia).
29
+ * Do not log idempotency keys, stored requests or response bodies. Fix the inverted values of the "Same params?" and "Same path?" debug log - [@jcagarcia](https://github.com/jcagarcia).
30
+ * Log a warning the first time an idempotent endpoint is performed when the stored data is not encrypted - [@jcagarcia](https://github.com/jcagarcia).
31
+ * Document the data stored in `Redis` and for how long - [@jcagarcia](https://github.com/jcagarcia).
32
+ * Improve the gem summary and description, and add source code, changelog, bug tracker and documentation links to the gem metadata - [@jcagarcia](https://github.com/jcagarcia).
33
+
34
+
35
+ ## [1.1.1] - 2026-10-15
36
+
37
+ ### Fix
38
+
39
+ * Catch errors thrown with `error!` inside `rescue_from` handlers, which raised `UncaughtThrowError` with Grape >= 2.1 - [@jcagarcia](https://github.com/jcagarcia).
40
+ * Return Grape's `500 Invalid response` when a `rescue_from` handler does not return a valid response, instead of raising `NoMethodError: undefined method 'default_rescue_handler'` with Grape >= 2.1 - [@jcagarcia](https://github.com/jcagarcia).
41
+ * Store that `500 Invalid response` for the idempotency key, so retrying the request replays it instead of returning `409 Conflict` until the key expires - [@jcagarcia](https://github.com/jcagarcia).
42
+ * Support `rescue_from ..., with: :helper_method` with Grape >= 3.0, which resolves symbol handlers on the endpoint - [@jcagarcia](https://github.com/jcagarcia).
43
+ * Only treat a Hash returned by a `rescue_from` handler as an error response when it contains `:message`, `:status` and `:headers`, matching Grape - [@jcagarcia](https://github.com/jcagarcia).
44
+ * Handle `Grape::Exceptions::ErrorResponse` thrown by `error!` with Grape >= 3.3, both inside `idempotent` blocks and inside `rescue_from` handlers - [@jcagarcia](https://github.com/jcagarcia).
45
+ * [#22](https://github.com/jcagarcia/grape-idempotency/pull/22): Do not turn `rescue_from` responses into 500 errors when a stored error request expires or is invalid while being looked up. - [@salrepe](https://github.com/salrepe).
46
+ * [#22](https://github.com/jcagarcia/grape-idempotency/pull/22): Only access the storage from the error middleware for requests that stored an error inside `idempotent`, reading that exact key instead of scanning all error keys. - [@salrepe](https://github.com/salrepe).
47
+ * [#22](https://github.com/jcagarcia/grape-idempotency/pull/22): Do not store error requests when no idempotency key was provided. - [@salrepe](https://github.com/salrepe).
48
+
49
+ ### Changed
50
+
51
+ * [#24](https://github.com/jcagarcia/grape-idempotency/issues/24): Test Ruby (2.6 - 4.0) and Grape (1.5 - 4.0) in CI: the combinations where behaviour changes on every push and pull request, and every supported combination plus the latest Grape release weekly or on demand - [@jcagarcia](https://github.com/jcagarcia).
52
+ * Document Ruby and Grape compatibility in the README - [@jcagarcia](https://github.com/jcagarcia).
53
+
54
+
55
+ ## [1.1.0] - 2023-12-12
16
56
 
17
57
  ### Feature
18
58
 
19
- * [#20](https://github.com/jcagarcia/grape-idempotency/pull/20): Manage `Redis` exceptions by default and allow the gem's consumer to manage them by itself - [@jcagarcia](https://github.com/jcagarcia).
20
- * Your contribution here.
59
+ * [#20](https://github.com/jcagarcia/grape-idempotency/pull/20): Allow consumers to configure the gem for handling `Redis` exceptions - [@jcagarcia](https://github.com/jcagarcia).
21
60
 
22
61
  ## [1.0.0] - 2023-11-23
23
62
 
data/README.md CHANGED
@@ -10,10 +10,12 @@ Implementation based on the Active Internet-Draft [draft-ietf-httpapi-idempotenc
10
10
  Topics covered in this README:
11
11
 
12
12
  - [Installation](#installation-)
13
+ - [Compatibility](#compatibility-)
13
14
  - [Basic Usage](#basic-usage-)
14
15
  - [How it works](#how-it-works-)
15
16
  - [Making idempotency key header mandatory](#making-idempotency-key-header-mandatory-)
16
17
  - [Redis Storage Connectivity Issue](#redis-storage-connectivity-issue)
18
+ - [Stored data](#stored-data-)
17
19
  - [Configuration](#configuration-)
18
20
  - [Changelog](#changelog)
19
21
  - [Contributing](#contributing)
@@ -30,6 +32,20 @@ Ruby 2.6 or newer is required.
30
32
  gem install grape-idempotency
31
33
  ```
32
34
 
35
+ ## Compatibility 🤝
36
+
37
+ - Ruby 2.6 or newer.
38
+ - Grape 1.x and 2.x or newer are supported from a single codebase.
39
+ - Grape 1.0 - 1.4 are allowed by the gemspec but are not tested.
40
+ - Grape 1.x does not work with Rack 3, so if you are using Grape 1.x add `gem 'rack', '< 3'` to your `Gemfile`.
41
+ - Grape 1.5 does not load with ActiveSupport 7.1.x (7.0 and 7.2+ work). On Ruby 2.7 / 3.0, where 7.1 is the newest installable ActiveSupport, add `gem 'activesupport', '< 7.1'` to your `Gemfile`.
42
+
43
+ The tested Ruby and Grape combinations live in the [CI workflow](.github/workflows/ruby.yml): pushes and pull requests run the combinations where Ruby or Grape behaviour changes, and a weekly (or manually triggered) run tests every Grape minor line from 1.5 onwards against every Ruby version it supports, plus the latest Grape release on the newest Ruby. To run the specs against a specific Grape line locally, set `GRAPE_VERSION` (this rewrites your local `Gemfile.lock`, do not commit it):
44
+
45
+ ```shell
46
+ GRAPE_VERSION="~> 1.8.0" bundle update grape && bundle exec rspec
47
+ ```
48
+
33
49
  ## Basic Usage 📖
34
50
 
35
51
  Configure the `Grape::Idempotency` class with a `Redis` instance.
@@ -73,9 +89,15 @@ This key allows your consumer to make the same request again in case of a connec
73
89
 
74
90
  To execute an idempotent request, simply request your user to include an extra `Idempotency-Key: <key>` header as part of his request.
75
91
 
76
- This gem operates by storing the initial request's status code and response body, regardless of whether the request succeeded or failed, using a specific idempotency key. Subsequent requests with the same key will consistently yield the same result, even if there were 500 errors.
92
+ This gem operates by storing the initial request's status code, response headers and response body, regardless of whether the request succeeded or failed, using a specific idempotency key. Subsequent requests with the same key will consistently yield the same result, even if there were 500 errors.
93
+
94
+ The stored response is the final one built by Grape (the formatted body, also for responses generated by `error!` or by `rescue_from` handlers), and it is returned byte for byte. When a response is replayed, your endpoint is not run, including any code placed after the `idempotent` block.
77
95
 
78
- Keys are automatically removed from the system if they are at least 24 hours old, and a new request is generated when a key is reused after the original has been removed. The idempotency layer compares incoming parameters to those of the original request and returns a `422 - Unprocessable Entity` status code if they don't match, preventing accidental misuse.
96
+ Some response headers are never stored nor replayed: `Set-Cookie`, hop-by-hop headers (e.g. `Connection` or `Transfer-Encoding`), `Content-Length`, `Date`, `Age`, `Via`, `Server-Timing`, `X-Runtime`, `Original-Request` and the configured idempotency key and request id headers. You can exclude more headers with the [replay_headers_denylist](#replay_headers_denylist) configuration option.
97
+
98
+ Responses whose body is a stream or a file (e.g. using `stream` or `sendfile`) are not stored: the idempotency key is released and a warning is logged, so those endpoints are not idempotent.
99
+
100
+ Keys are automatically removed from the system once they are older than the configured [expires_in](#expires_in) (60 hours by default), and a new request is generated when a key is reused after the original has been removed. The idempotency layer compares incoming parameters to those of the original request and returns a `422 - Unprocessable Entity` status code if they don't match, preventing accidental misuse.
79
101
  If a request is received while another one with the same idempotency key is still being processed the idempotency layer returns a `409 - Conflict` status
80
102
 
81
103
  Results are only saved if an API endpoint begins its execution. If incoming parameters fail validation or if the request conflicts with another one executing concurrently, no idempotent result is stored because no API endpoint has initiated execution. In such cases, retrying these requests is safe.
@@ -145,13 +167,30 @@ If you want to avoid this functionality, and you want the gem handles the potent
145
167
 
146
168
  🚨 WARNING: If a `Redis` exception appears AFTER performing the wrapped code, nothing will be re-raised. The process will continue working and the response will be returned to the consumer of your API. However, a `409 Conflict` response can be returned to your consumer if it retried the same call with the same idempotency key. This is because the gem was not able to associate the response of the original request to the original idempotency key because those connectivity issues.
147
169
 
170
+ ### Stored data 🔐
171
+
172
+ For each request with an idempotency key, the gem stores in `Redis`, under the `grape:idempotency:<idempotency-key>` key:
173
+
174
+ * The request path.
175
+ * A SHA-256 digest of the request params, used to detect when an idempotency key is reused with different params. The params themselves are NOT stored.
176
+ * The response status, headers (except the ones listed in [How it works](#how-it-works-)) and body, which are returned again when the request is retried.
177
+ * The original request id.
178
+
179
+ Stored requests are removed after [expires_in](#expires_in) seconds (60 hours by default).
180
+
181
+ Since `1.2.0` the request params are not stored. While deploying `1.2.0`, instances still running an older version of the gem return the [conflict_error_response](#conflict_error_response) for retries of requests stored by `1.2.0`, as they compare the params of the request with the stored ones.
182
+
183
+ The response is stored as returned by your endpoint, so it may include sensitive data. Configure an [encryption_key](#encryption_key) or an [encryptor](#encryptor) to encrypt everything the gem stores. Until you do, the gem logs a warning the first time an idempotent endpoint is performed.
184
+
185
+ The gem never logs idempotency keys, request params or response bodies.
186
+
148
187
  ## Configuration 🪚
149
188
 
150
189
  In addition to the storage aspect, you have the option to supply additional configuration details to tailor the gem to the specific requirements of your project.
151
190
 
152
191
  ### expires_in
153
192
 
154
- As we have mentioned in the [How it works](#how-it-works-) section, keys are automatically removed from the system if they are at least 24 hours old. However, a 24-hour timeframe may not be suitable for your project. To accommodate your specific needs, you can adjust this duration by using the `expires_in` parameter for configuration:
193
+ As we have mentioned in the [How it works](#how-it-works-) section, keys are automatically removed from the system once they are older than `expires_in` (60 hours by default). However, this timeframe may not be suitable for your project. To accommodate your specific needs, you can adjust this duration by using the `expires_in` parameter for configuration:
155
194
 
156
195
  ```ruby
157
196
  Grape::Idempotency.configure do |c|
@@ -194,7 +233,7 @@ In the case above, you request your consumers to use the `X-Trace-Id: <trace-id>
194
233
 
195
234
  ### logger, logger_level and logger_prefix
196
235
 
197
- By default, the logger used by the gem is configured like `Logger.new(STDOUT)` and `INFO` level. As this gem does not log any message with `INFO` level, only `ERROR` messages will be logged.
236
+ By default, the logger used by the gem is configured like `Logger.new(STDOUT)` and `INFO` level. As this gem does not log any message with `INFO` level, only `WARN` and `ERROR` messages will be logged.
198
237
 
199
238
 
200
239
  If you want to provide your own logger, you want to change the level to `DEBUG` or you want to provide your own prefix, you can configure the gem like:
@@ -213,7 +252,7 @@ An example of the logged information when changing the level of the log to `DEBU
213
252
  ```shell
214
253
  I, [2023-11-23T22:41:39.148163 #1] DEBUG -- : [my-own-prefix] Performing endpoint "/payments" with idempotency.
215
254
  I, [2023-11-23T22:41:39.148176 #1] DEBUG -- : [my-own-prefix] Idempotency key is NOT mandatory for this endpoint.
216
- I, [2023-11-23T22:41:39.148192 #1] DEBUG -- : [my-own-prefix] Idempotency key received in request header "x-custom-idempotency-key" => "fd77c9d6-b7da-4966-aac8-40ee258f24aa"
255
+ I, [2023-11-23T22:41:39.148192 #1] DEBUG -- : [my-own-prefix] Idempotency key received in request header "x-custom-idempotency-key".
217
256
  I, [2023-11-23T22:41:39.148210 #1] DEBUG -- : [my-own-prefix] Previous request information has NOT been found for the provided idempotency key.
218
257
  I, [2023-11-23T22:41:39.148248 #1] DEBUG -- : [my-own-prefix] Request stored as processing.
219
258
  I, [2023-11-23T22:41:39.148261 #1] DEBUG -- : [my-own-prefix] Performing the provided block.
@@ -222,8 +261,8 @@ I, [2023-11-23T22:41:39.148287 #1] DEBUG -- : [my-own-prefix] Storing response.
222
261
  I, [2023-11-23T22:41:39.148317 #1] DEBUG -- : [my-own-prefix] Response stored.
223
262
  I, [2023-11-23T22:41:39.148473 #1] DEBUG -- : [my-own-prefix] Performing endpoint "/payments" with idempotency.
224
263
  I, [2023-11-23T22:41:39.148486 #1] DEBUG -- : [my-own-prefix] Idempotency key is NOT mandatory for this endpoint.
225
- I, [2023-11-23T22:41:39.148502 #1] DEBUG -- : [my-own-prefix] Idempotency key received in request header "x-custom-idempotency-key" => "fd77c9d6-b7da-4966-aac8-40ee258f24aa"
226
- I, [2023-11-23T22:41:39.148523 #1] DEBUG -- : [my-own-prefix] Request has been found for the provided idempotency key => {"path"=>"/payments", "params"=>{"locale"=>"undefined", "{\"amount\":10000}"=>nil}, "status"=>500, "original_request"=>"wadus", "response"=>"{\"error\":\"Internal Server Error\"}"}
264
+ I, [2023-11-23T22:41:39.148502 #1] DEBUG -- : [my-own-prefix] Idempotency key received in request header "x-custom-idempotency-key".
265
+ I, [2023-11-23T22:41:39.148523 #1] DEBUG -- : [my-own-prefix] Request has been found for the provided idempotency key.
227
266
  I, [2023-11-23T22:41:39.148537 #1] DEBUG -- : [my-own-prefix] Returning the response from the original request.
228
267
  ```
229
268
 
@@ -242,6 +281,59 @@ end
242
281
 
243
282
  However, this approach carries a certain level of risk. In the case that `Redis` experiences an outage, the idempotent functionality will be lost, the endpoint will behave as no idempotent, and this issue may go unnoticed.
244
283
 
284
+ ### replay_headers_denylist
285
+
286
+ By default, all the response headers are stored and replayed except the ones listed in the [How it works](#how-it-works-) section. If your API returns other headers that must not be replayed, you can add them using the `replay_headers_denylist` configuration option. Header names are case-insensitive, and the headers excluded by default are always excluded:
287
+
288
+ ```ruby
289
+ Grape::Idempotency.configure do |c|
290
+ c.storage = @storage
291
+ c.replay_headers_denylist = ["X-Session-Token", "X-Rate-Limit-Remaining"]
292
+ end
293
+ ```
294
+
295
+ ### encryption_key
296
+
297
+ Encrypts everything the gem stores in `Redis` (see [Stored data](#stored-data-)) using `ActiveSupport::MessageEncryptor` with `aes-256-gcm`. The encryption key is derived from the provided value, so use a long random secret and share the same value across all the instances of your API, otherwise a retried request reaching another instance would not be able to read the stored response.
298
+
299
+ ```ruby
300
+ Grape::Idempotency.configure do |c|
301
+ c.storage = @storage
302
+ c.encryption_key = ENV.fetch("IDEMPOTENCY_ENCRYPTION_KEY") # e.g. generated with `SecureRandom.hex(32)`
303
+ end
304
+ ```
305
+
306
+ Requests stored before configuring the encryption are still read, so you can enable it without affecting the retries in progress. While deploying it, instances running a version older than `1.2.0` are not able to read the encrypted requests, so upgrade the gem before configuring the encryption.
307
+
308
+ If a stored request cannot be decrypted (e.g. after changing the key, or after removing the encryption), the gem returns the [conflict_error_response](#conflict_error_response) instead of performing the request again. Those requests are removed once [expires_in](#expires_in) has elapsed.
309
+
310
+ ### encryptor
311
+
312
+ If you want to use your own encryption (e.g. a KMS, or your existing `ActiveSupport::MessageEncryptor`), provide an object responding to `encrypt(string)` and `decrypt(string)`, both returning a `String`. `decrypt` must raise an error (or return `nil`) when the value cannot be decrypted.
313
+
314
+ ```ruby
315
+ class IdempotencyEncryptor
316
+ def initialize(message_encryptor)
317
+ @message_encryptor = message_encryptor
318
+ end
319
+
320
+ def encrypt(value)
321
+ @message_encryptor.encrypt_and_sign(value)
322
+ end
323
+
324
+ def decrypt(value)
325
+ @message_encryptor.decrypt_and_verify(value)
326
+ end
327
+ end
328
+
329
+ Grape::Idempotency.configure do |c|
330
+ c.storage = @storage
331
+ c.encryptor = IdempotencyEncryptor.new(my_message_encryptor)
332
+ end
333
+ ```
334
+
335
+ Configure either an `encryption_key` or an `encryptor`, not both.
336
+
245
337
  ### conflict_error_response
246
338
 
247
339
  When providing a `Idempotency-Key: <key>` header, this gem compares incoming parameters to those of the original request (if exists) and returns a `409 - Conflict` status code if they don't match, preventing accidental misuse. The response body returned by the gem looks like:
@@ -368,4 +460,5 @@ Open issues on the GitHub issue tracker with clear information.
368
460
  ### Contributors
369
461
 
370
462
  * Juan Carlos García - Creator - https://github.com/jcagarcia
371
- * Carlos Cabanero - Contributor - https://github.com/Flip120
463
+ * Carlos Cabanero - Contributor - https://github.com/Flip120
464
+ * Salvador Revert Pedron - Contributor - https://github.com/salrepe
@@ -8,10 +8,21 @@ Gem::Specification.new do |spec|
8
8
  spec.version = Grape::Idempotency::VERSION
9
9
  spec.authors = ['Juan Carlos García']
10
10
  spec.email = ['jugade92@gmail.com']
11
- spec.description = 'Add idempotency support to your Grape APIs for safely retrying requests without accidentally performing the same operation twice. When creating or updating an object, use an idempotency key. Then, if a connection error occurs, you can safely repeat the request without risk of creating a second object or performing the update twice.'
12
- spec.summary = 'Gem for supporting idempotency in your Grape APIs'
11
+ spec.description = 'Idempotency-Key header support for Grape REST APIs, based on the IETF Idempotency-Key HTTP header draft. ' \
12
+ 'Wrap a POST or PATCH endpoint in an idempotent block and clients can safely retry requests after timeouts or ' \
13
+ 'network errors without creating duplicate payments, orders or records. The first response (status, headers and ' \
14
+ 'body) is stored in Redis and replayed for retries with the same key. Concurrent duplicate requests get a 409 Conflict, ' \
15
+ 'and reusing a key with a different payload gets a 422 Unprocessable Entity. Works with Grape 1.x to 4.x and Ruby 2.6+.'
16
+ spec.summary = 'Idempotency-Key support for Grape APIs: safe retries and duplicate request prevention backed by Redis'
13
17
  spec.homepage = 'https://github.com/jcagarcia/grape-idempotency'
14
18
  spec.license = 'MIT'
19
+ spec.metadata = {
20
+ 'homepage_uri' => spec.homepage,
21
+ 'source_code_uri' => 'https://github.com/jcagarcia/grape-idempotency',
22
+ 'changelog_uri' => 'https://github.com/jcagarcia/grape-idempotency/blob/main/CHANGELOG.md',
23
+ 'bug_tracker_uri' => 'https://github.com/jcagarcia/grape-idempotency/issues',
24
+ 'documentation_uri' => 'https://github.com/jcagarcia/grape-idempotency#readme'
25
+ }
15
26
 
16
27
  files = Dir["lib/**/*.rb"]
17
28
  rootfiles = ["CHANGELOG.md", "grape-idempotency.gemspec", "Rakefile", "README.md"]
@@ -0,0 +1,38 @@
1
+ require 'openssl'
2
+
3
+ module Grape
4
+ module Idempotency
5
+ # Encryptor built by the gem from the configured `encryption_key`.
6
+ class Encryptor
7
+ CIPHER = "aes-256-gcm".freeze
8
+ KEY_SALT = "grape-idempotency".freeze
9
+ KEY_ITERATIONS = 65_536
10
+
11
+ # Stored records are already JSON strings, so they are encrypted as they are
12
+ module RawSerializer
13
+ def self.dump(value)
14
+ value
15
+ end
16
+
17
+ def self.load(value)
18
+ value
19
+ end
20
+ end
21
+
22
+ def initialize(encryption_key)
23
+ require 'active_support/message_encryptor'
24
+
25
+ key = OpenSSL::PKCS5.pbkdf2_hmac(encryption_key, KEY_SALT, KEY_ITERATIONS, 32, OpenSSL::Digest::SHA256.new)
26
+ @message_encryptor = ActiveSupport::MessageEncryptor.new(key, cipher: CIPHER, serializer: RawSerializer)
27
+ end
28
+
29
+ def encrypt(value)
30
+ @message_encryptor.encrypt_and_sign(value)
31
+ end
32
+
33
+ def decrypt(value)
34
+ @message_encryptor.decrypt_and_verify(value)
35
+ end
36
+ end
37
+ end
38
+ end
@@ -5,36 +5,39 @@ module Grape
5
5
  class Error < Base
6
6
  def run_rescue_handler(handler, error, endpoint=nil)
7
7
  if handler.instance_of?(Symbol)
8
- raise NoMethodError, "undefined method '#{handler}'" unless respond_to?(handler)
8
+ # grape >= 3.0 resolves symbol handlers (`rescue_from ..., with: :helper`) on the endpoint
9
+ handler_owner = respond_to?(handler) || endpoint.nil? ? self : endpoint
10
+ raise NoMethodError, "undefined method '#{handler}'" unless handler_owner.respond_to?(handler)
9
11
 
10
- handler = public_method(handler)
12
+ handler = handler_owner.public_method(handler)
11
13
  end
12
14
 
13
- if endpoint
14
- response = handler.arity.zero? ? endpoint.instance_exec(&handler) : endpoint.instance_exec(error, &handler)
15
- else
16
- response = handler.arity.zero? ? instance_exec(&handler) : instance_exec(error, &handler)
15
+ response = catch(:error) do
16
+ if endpoint
17
+ handler.arity.zero? ? endpoint.instance_exec(&handler) : endpoint.instance_exec(error, &handler)
18
+ else
19
+ handler.arity.zero? ? instance_exec(&handler) : instance_exec(error, &handler)
20
+ end
17
21
  end
18
22
 
23
+ response = error_response(response) if thrown_error?(response)
24
+
19
25
  if response.is_a?(Rack::Response)
20
- update_idempotency_error_with(error, response)
21
26
  response
22
27
  else
23
- run_rescue_handler(:default_rescue_handler, Grape::Exceptions::InvalidResponse.new)
28
+ # default_rescue_handler is private since grape 2.1
29
+ run_rescue_handler(method(:default_rescue_handler), Grape::Exceptions::InvalidResponse.new, endpoint)
24
30
  end
25
31
  end
26
32
 
27
33
  private
28
34
 
29
- def update_idempotency_error_with(error, response)
30
- begin
31
- body = JSON.parse(response.body.join)
32
- rescue JSON::ParserError
33
- body = response.body.join
34
- end
35
+ def thrown_error?(response)
36
+ # grape >= 3.3 throws Grape::Exceptions::ErrorResponse instead of a Hash
37
+ return true if defined?(Grape::Exceptions::ErrorResponse) && response.is_a?(Grape::Exceptions::ErrorResponse)
35
38
 
36
- Grape::Idempotency.update_error_with_rescue_from_result(error, response.status, body)
39
+ response.is_a?(Hash) && response.key?(:message) && response.key?(:status) && response.key?(:headers)
37
40
  end
38
41
  end
39
42
  end
40
- end
43
+ end
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Grape
4
+ module Idempotency
5
+ module Middleware
6
+ # Prepended to Grape::Middleware::Error, the outermost Grape middleware of every endpoint.
7
+ # It receives the final response (formatted body, status and headers) of the success path,
8
+ # of `error!` and of rescue_from handlers, so it can be stored or replayed as it is.
9
+ module ResponseCapture
10
+ def call!(env)
11
+ Grape::Idempotency.finalize_response(env, super)
12
+ end
13
+ end
14
+ end
15
+ end
16
+ end
17
+
18
+ Grape::Middleware::Error.prepend(Grape::Idempotency::Middleware::ResponseCapture)
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Grape
4
4
  module Idempotency
5
- VERSION = '1.1.0'
5
+ VERSION = '1.2.0'
6
6
  end
7
7
  end
@@ -2,16 +2,34 @@ require 'grape'
2
2
  require 'redis'
3
3
  require 'logger'
4
4
  require 'securerandom'
5
+ require 'digest'
6
+ require 'json'
5
7
  require 'grape/idempotency/version'
8
+ require 'grape/idempotency/encryptor'
6
9
  require 'grape/idempotency/middleware/error'
10
+ require 'grape/idempotency/middleware/response_capture'
7
11
 
8
12
  module Grape
9
13
  module Idempotency
10
14
  autoload :Helpers, 'grape/idempotency/helpers'
11
15
  Grape::Endpoint.send :include, Grape::Idempotency::Helpers
12
16
 
17
+ ENCRYPTED_RECORD_PREFIX = "encrypted:v1:".freeze
18
+
19
+ class UnreadableRecordError < StandardError; end
20
+
13
21
  class << self
14
22
  ORIGINAL_REQUEST_HEADER = "Original-Request".freeze
23
+ STORAGE_SCHEMA_VERSION = 2
24
+ BODY_ENCODING_UTF8 = "utf8".freeze
25
+ BODY_ENCODING_BASE64 = "base64".freeze
26
+ PENDING_ENV_KEY = "grape.idempotency.pending".freeze
27
+ REPLAY_ENV_KEY = "grape.idempotency.replay".freeze
28
+ # Response headers that are never stored nor replayed
29
+ EXCLUDED_REPLAY_HEADERS = %w[
30
+ set-cookie connection keep-alive proxy-connection transfer-encoding te trailer upgrade
31
+ content-length date age via server-timing x-runtime original-request
32
+ ].freeze
15
33
 
16
34
  def configure(&block)
17
35
  yield(configuration)
@@ -21,30 +39,46 @@ module Grape
21
39
  clean_configuration = Configuration.new
22
40
  clean_configuration.storage = storage
23
41
  @configuration = clean_configuration
42
+ @unencrypted_storage_warned = false
24
43
  end
25
44
 
26
45
  def idempotent(grape, required: false, &block)
27
46
  validate_config!
47
+ warn_unencrypted_storage unless encryptor
28
48
  log(:debug, "Performing endpoint \"#{grape.request.path}\" with idempotency.")
29
49
  log(:debug, "Idempotency key is #{!required ? 'NOT' : ''} mandatory for this endpoint.")
30
50
 
31
51
  idempotency_key = get_idempotency_key(grape.request.headers)
32
- log(:debug, "Idempotency key received in request header \"#{configuration.idempotency_key_header.downcase}\" => \"#{idempotency_key}\"") if idempotency_key
52
+ log(:debug, "Idempotency key received in request header \"#{configuration.idempotency_key_header.downcase}\".") if idempotency_key
33
53
 
34
54
  grape.error!(configuration.mandatory_header_response, 400) if required && !idempotency_key
35
55
  return block.call if !idempotency_key
36
56
 
37
- stored_request = get_from_storage(idempotency_key)
38
- log(:debug, "Request has been found for the provided idempotency key => #{stored_request}") if stored_request
39
- if stored_request && (stored_request["params"] != grape.request.params || stored_request["path"] != grape.request.path)
40
- log(:debug, "Request has conflicts. Same params? => #{stored_request["params"] != grape.request.params}. Same path? => #{stored_request["path"] != grape.request.path}")
57
+ request_params_digest = params_digest(grape.request.params)
58
+ begin
59
+ stored_request = get_from_storage(idempotency_key)
60
+ rescue UnreadableRecordError
61
+ log(:error, "The request stored for the provided idempotency key could NOT be read. Check the encryption configuration.")
62
+ log(:debug, "Returning conflict error response.")
63
+ grape.error!(configuration.conflict_error_response, 422)
64
+ end
65
+ log(:debug, "Request has been found for the provided idempotency key.") if stored_request
66
+ same_params = stored_request && stored_params_digest(stored_request) == request_params_digest
67
+ same_path = stored_request && stored_request["path"] == grape.request.path
68
+ if stored_request && (!same_params || !same_path)
69
+ log(:debug, "Request has conflicts. Same params? => #{same_params}. Same path? => #{same_path}")
41
70
  log(:debug, "Returning conflict error response.")
42
71
  grape.error!(configuration.conflict_error_response, 422)
43
72
  elsif stored_request && stored_request["processing"] == true
44
73
  log(:debug, "Returning processing error response.")
45
74
  grape.error!(configuration.processing_response, 409)
46
- elsif stored_request
75
+ elsif stored_request && stored_request["v"].to_i >= STORAGE_SCHEMA_VERSION
47
76
  log(:debug, "Returning the response from the original request.")
77
+ # Stop the endpoint here. The stored response replaces Grape's response in #finalize_response.
78
+ grape.env[REPLAY_ENV_KEY] = { idempotency_key: idempotency_key, stored_request: stored_request }
79
+ grape.error!('', stored_request["status"])
80
+ elsif stored_request
81
+ log(:debug, "Returning the response from the original request (stored by a previous version of the gem).")
48
82
  grape.status stored_request["status"]
49
83
  grape.header(ORIGINAL_REQUEST_HEADER, stored_request["original_request"])
50
84
  grape.header(configuration.idempotency_key_header, idempotency_key)
@@ -54,7 +88,7 @@ module Grape
54
88
  log(:debug, "Previous request information has NOT been found for the provided idempotency key.")
55
89
 
56
90
  original_request_id = get_request_id(grape.request.headers)
57
- success = store_processing_request(idempotency_key, grape.request.path, grape.request.params, original_request_id)
91
+ success = store_processing_request(idempotency_key, grape.request.path, request_params_digest, original_request_id)
58
92
  if !success
59
93
  log(:error, "Request NOT stored as processing. Concurrent requests for the same idempotency key appeared.")
60
94
  grape.error!(configuration.processing_response, 409)
@@ -62,66 +96,60 @@ module Grape
62
96
  log(:debug, "Request stored as processing.")
63
97
  end
64
98
 
99
+ # The final response (formatted body, status and headers) is stored by #finalize_response,
100
+ # once Grape has built it, both for the success path and for the rescue_from path.
101
+ grape.env[PENDING_ENV_KEY] = {
102
+ idempotency_key: idempotency_key,
103
+ path: grape.request.path,
104
+ params_digest: request_params_digest,
105
+ original_request: original_request_id
106
+ }
107
+ grape.header(ORIGINAL_REQUEST_HEADER, original_request_id)
108
+ grape.header(configuration.idempotency_key_header, idempotency_key)
109
+
65
110
  log(:debug, "Performing the provided block.")
111
+ block.call
112
+ end
66
113
 
67
- response = catch(:error) do
68
- block.call
69
- end
114
+ # Called with the final Rack response of an endpoint (see Grape::Idempotency::ResponseCapture).
115
+ # Stores it for a pending idempotent request, or replaces it with the stored one when replaying.
116
+ # It never raises: on any failure the original response is returned.
117
+ def finalize_response(env, response)
118
+ replay = env.delete(REPLAY_ENV_KEY)
119
+ pending = env.delete(PENDING_ENV_KEY)
120
+ return response unless replay || pending
70
121
 
71
- log(:debug, "Block has been performed.")
122
+ parts = rack_response_parts(response)
72
123
 
73
- if is_an_error?(response)
74
- log(:debug, "An error response was returned by the performed block. => #{response}")
75
- response = response[:message]
76
- end
124
+ if replay
125
+ return response unless parts
77
126
 
78
- grape.header(ORIGINAL_REQUEST_HEADER, original_request_id)
79
- grape.body response
80
- rescue Redis::BaseError => e
81
- raise
82
- rescue => e
83
- log(:debug, "An unexpected error was raised when performing the block.")
84
- if !stored_request && !response
85
- validate_config!
86
- log(:debug, "Storing error response.")
87
- original_request_id = get_request_id(grape.request.headers)
88
- stored_key = store_error_request(idempotency_key, grape.request.path, grape.request.params, grape.status, original_request_id, e)
89
- if stored_key
90
- log(:debug, "Error response stored.")
91
- grape.header(ORIGINAL_REQUEST_HEADER, original_request_id)
92
- grape.header(configuration.idempotency_key_header, stored_key)
93
- end
127
+ log(:debug, "Replaying the stored response.")
128
+ return replay_response(replay[:stored_request], replay[:idempotency_key], parts[1])
94
129
  end
95
- log(:debug, "Re-raising the error.")
96
- raise
97
- ensure
98
- if !stored_request && response
99
- validate_config!
100
- log(:debug, "Storing response.")
101
- stored_key = store_request_response(idempotency_key, grape.request.path, grape.request.params, grape.status, original_request_id, response)
102
- if stored_key
103
- log(:debug, "Response stored.")
104
- grape.header(configuration.idempotency_key_header, stored_key)
105
- end
106
- end
107
- end
108
130
 
109
- def update_error_with_rescue_from_result(error, status, response)
110
- validate_config!
131
+ if parts.nil? || !storable_body?(parts[2])
132
+ log(:warn, "The response body of \"#{pending[:path]}\" is a stream or a file. It is NOT stored and its idempotency key is released.")
133
+ release_idempotency_key(pending[:idempotency_key])
134
+ return response
135
+ end
111
136
 
112
- stored_error = get_error_request_for(error)
113
- return unless stored_error
137
+ status, headers, body = parts
138
+ log(:debug, "Storing response.")
139
+ store_request_response(pending, status, headers, body.join)
140
+ log(:debug, "Response stored.")
114
141
 
115
- request_with_unmanaged_error = stored_error[:request]
116
- idempotency_key = stored_error[:idempotency_key]
117
- path = request_with_unmanaged_error["path"]
118
- params = request_with_unmanaged_error["params"]
119
- original_request_id = request_with_unmanaged_error["original_request"]
142
+ set_response_header(headers, ORIGINAL_REQUEST_HEADER, pending[:original_request])
143
+ set_response_header(headers, configuration.idempotency_key_header, pending[:idempotency_key])
144
+ response
145
+ rescue => e
146
+ log(:error, "The response could NOT be stored or replayed => #{e.class}")
147
+ response
148
+ end
120
149
 
121
- store_request_response(idempotency_key, path, params, status, original_request_id, response)
122
- storage.del(stored_error[:error_key])
123
- rescue Redis::BaseError => e
124
- log(:error, "Storage error => #{e.message} - #{e}")
150
+ # Kept for backwards compatibility. Responses generated by rescue_from handlers are now
151
+ # stored by #finalize_response.
152
+ def update_error_with_rescue_from_result(_error, _status, _response, error_key: nil)
125
153
  nil
126
154
  end
127
155
 
@@ -129,6 +157,21 @@ module Grape
129
157
 
130
158
  def validate_config!
131
159
  raise Configuration::Error.new("A Redis instance must be configured as cache storage") unless valid_storage?
160
+ validate_encryption_config!
161
+ end
162
+
163
+ def validate_encryption_config!
164
+ if configuration.encryption_key && configuration.encryptor
165
+ raise Configuration::Error.new("Configure either an encryption_key or an encryptor, not both")
166
+ end
167
+
168
+ if configuration.encryption_key && !(configuration.encryption_key.is_a?(String) && !configuration.encryption_key.empty?)
169
+ raise Configuration::Error.new("The encryption_key must be a non-empty String")
170
+ end
171
+
172
+ if configuration.encryptor && !(configuration.encryptor.respond_to?(:encrypt) && configuration.encryptor.respond_to?(:decrypt))
173
+ raise Configuration::Error.new("The encryptor must respond to encrypt and decrypt")
174
+ end
132
175
  end
133
176
 
134
177
  def valid_storage?
@@ -158,100 +201,179 @@ module Grape
158
201
  value = storage.get(key(idempotency_key))
159
202
  return unless value
160
203
 
161
- JSON.parse(value)
204
+ deserialize(value)
162
205
  rescue Redis::BaseError => e
163
206
  log(:error, "Storage error => #{e.message} - #{e}")
164
207
  return if configuration.manage_redis_exceptions
165
208
  raise
166
209
  end
167
210
 
168
- def store_processing_request(idempotency_key, path, params, request_id)
211
+ def store_processing_request(idempotency_key, path, request_params_digest, request_id)
169
212
  body = {
170
213
  path: path,
171
- params: params,
214
+ params_digest: request_params_digest,
172
215
  original_request: request_id,
173
216
  processing: true
174
217
  }
175
218
 
176
- storage.set(key(idempotency_key), body.to_json, ex: configuration.expires_in, nx: true)
219
+ storage.set(key(idempotency_key), serialize(body), ex: configuration.expires_in, nx: true)
177
220
  rescue Redis::BaseError => e
178
221
  return true if configuration.manage_redis_exceptions
179
222
  raise
180
223
  end
181
224
 
182
- def store_request_response(idempotency_key, path, params, status, request_id, response)
183
- body = {
184
- path: path,
185
- params: params,
186
- status: status,
187
- original_request: request_id,
188
- response: response
225
+ def store_request_response(pending, status, headers, body)
226
+ utf8_body = body.dup.force_encoding(Encoding::UTF_8)
227
+ body_encoding = utf8_body.valid_encoding? ? BODY_ENCODING_UTF8 : BODY_ENCODING_BASE64
228
+ replayable_headers = replayable_headers(headers)
229
+
230
+ stored_response = {
231
+ v: STORAGE_SCHEMA_VERSION,
232
+ path: pending[:path],
233
+ params_digest: pending[:params_digest],
234
+ status: status.to_i,
235
+ original_request: pending[:original_request],
236
+ headers: replayable_headers,
237
+ body_encoding: body_encoding,
238
+ body: body_encoding == BODY_ENCODING_UTF8 ? utf8_body : [body].pack('m0')
189
239
  }
240
+ # Read by versions of the gem that do not know the schema above (e.g. during a rolling deploy)
241
+ stored_response[:response] = legacy_response(replayable_headers, utf8_body) if body_encoding == BODY_ENCODING_UTF8
190
242
 
191
- storage.set(key(idempotency_key), body.to_json, ex: configuration.expires_in, nx: false)
243
+ storage.set(key(pending[:idempotency_key]), serialize(stored_response), ex: configuration.expires_in, nx: false)
192
244
 
193
- idempotency_key
245
+ pending[:idempotency_key]
194
246
  rescue Redis::BaseError => e
195
247
  log(:error, "Storage error => #{e.message} - #{e}")
196
- idempotency_key
248
+ pending[:idempotency_key]
197
249
  end
198
250
 
199
- def store_error_request(idempotency_key, path, params, status, request_id, error)
200
- body = {
201
- path: path,
202
- params: params,
203
- status: status,
204
- original_request: request_id,
205
- error: {
206
- class_name: error.class.to_s,
207
- message: error.message
208
- }
209
- }.to_json
210
-
211
- storage.set(error_key(idempotency_key), body, ex: 30, nx: false)
212
-
213
- idempotency_key
251
+ def release_idempotency_key(idempotency_key)
252
+ storage.del(key(idempotency_key))
214
253
  rescue Redis::BaseError => e
215
254
  log(:error, "Storage error => #{e.message} - #{e}")
216
- idempotency_key
255
+ nil
217
256
  end
218
257
 
219
- def get_error_request_for(error)
220
- error_keys = storage.keys("#{error_key_prefix}*")
221
- return if error_keys.empty?
222
-
223
- error_keys.map do |key|
224
- request_with_error = JSON.parse(storage.get(key))
225
- error_class_name = request_with_error["error"]["class_name"]
226
- error_message = request_with_error["error"]["message"]
227
-
228
- if error_class_name == error.class.to_s && error_message == error.message
229
- {
230
- error_key: key,
231
- request: request_with_error,
232
- idempotency_key: key.gsub(error_key_prefix, '')
233
- }
234
- end
235
- end.first
236
- rescue Redis::BaseError => e
237
- log(:error, "Storage error => #{e.message} - #{e}")
238
- nil
258
+ def replay_response(stored_request, idempotency_key, headers)
259
+ headers.keys.each do |name|
260
+ headers.delete(name) if %w[content-type content-length].include?(name.to_s.downcase)
261
+ end
262
+ (stored_request["headers"] || {}).each { |name, value| set_response_header(headers, name, value) }
263
+ set_response_header(headers, ORIGINAL_REQUEST_HEADER, stored_request["original_request"])
264
+ set_response_header(headers, configuration.idempotency_key_header, idempotency_key)
265
+
266
+ body = stored_request["body"].to_s
267
+ body = body.unpack1('m0') if stored_request["body_encoding"] == BODY_ENCODING_BASE64
268
+
269
+ [stored_request["status"].to_i, headers, [body]]
239
270
  end
240
271
 
241
- def is_an_error?(response)
242
- response.is_a?(Hash) && response.has_key?(:message) && response.has_key?(:headers) && response.has_key?(:status)
272
+ def rack_response_parts(response)
273
+ if response.is_a?(Rack::Response)
274
+ [response.status, response.headers, response.body]
275
+ elsif response.is_a?(Array) && response.size == 3
276
+ response
277
+ end
243
278
  end
244
279
 
245
- def key(idempotency_key)
246
- "#{gem_prefix}#{idempotency_key}"
280
+ # Streams and files are not buffered, so they cannot be stored
281
+ def storable_body?(body)
282
+ body.is_a?(Array) && body.all? { |chunk| chunk.is_a?(String) }
283
+ end
284
+
285
+ def replayable_headers(headers)
286
+ excluded = excluded_headers
287
+ headers.each_with_object({}) do |(name, value), replayable|
288
+ replayable[name.to_s] = value unless excluded.include?(name.to_s.downcase)
289
+ end
290
+ end
291
+
292
+ def excluded_headers
293
+ EXCLUDED_REPLAY_HEADERS +
294
+ [configuration.idempotency_key_header, configuration.request_id_header].map(&:downcase) +
295
+ Array(configuration.replay_headers_denylist).map { |name| name.to_s.downcase }
296
+ end
297
+
298
+ def legacy_response(headers, body)
299
+ content_type = headers.find { |name, _| name.downcase == 'content-type' }&.last.to_s
300
+ return body unless content_type =~ /\bjson\b/i
301
+
302
+ JSON.parse(body)
303
+ rescue JSON::ParserError
304
+ body
305
+ end
306
+
307
+ def set_response_header(headers, name, value)
308
+ headers.keys.each { |existing| headers.delete(existing) if existing.to_s.casecmp?(name) }
309
+ # Rack 3 requires lowercase header names
310
+ headers[defined?(Rack::Headers) ? name.downcase : name] = value
311
+ end
312
+
313
+ def serialize(record)
314
+ json = record.to_json
315
+ return json unless encryptor
316
+
317
+ "#{ENCRYPTED_RECORD_PREFIX}#{encryptor.encrypt(json)}"
247
318
  end
248
319
 
249
- def error_key(idempotency_key)
250
- "#{error_key_prefix}#{idempotency_key}"
320
+ def deserialize(value)
321
+ if value.start_with?(ENCRYPTED_RECORD_PREFIX)
322
+ raise UnreadableRecordError, "Stored request is encrypted but no encryption is configured" unless encryptor
323
+
324
+ value = encryptor.decrypt(value.delete_prefix(ENCRYPTED_RECORD_PREFIX))
325
+ end
326
+
327
+ # Records stored before configuring the encryption are not encrypted and are still accepted
328
+ JSON.parse(value)
329
+ rescue UnreadableRecordError
330
+ raise
331
+ rescue StandardError => e
332
+ raise UnreadableRecordError, "Stored request could not be read (#{e.class})"
333
+ end
334
+
335
+ def encryptor
336
+ return configuration.encryptor if configuration.encryptor
337
+ return unless configuration.encryption_key
338
+
339
+ if @encryptor.nil? || @encryptor_key != configuration.encryption_key
340
+ @encryptor = Encryptor.new(configuration.encryption_key)
341
+ @encryptor_key = configuration.encryption_key
342
+ end
343
+ @encryptor
344
+ end
345
+
346
+ def warn_unencrypted_storage
347
+ return if @unencrypted_storage_warned
348
+
349
+ @unencrypted_storage_warned = true
350
+ log(:warn, "Stored requests and responses are NOT encrypted. Configure an encryption_key or an encryptor to encrypt them.")
251
351
  end
252
352
 
253
- def error_key_prefix
254
- "#{gem_prefix}error:"
353
+ def params_digest(params)
354
+ digest(JSON.generate(canonical(JSON.parse(params.to_json))))
355
+ end
356
+
357
+ def stored_params_digest(stored_request)
358
+ # Requests stored by versions < 1.2.0 contain the params instead of their digest
359
+ stored_request["params_digest"] || params_digest(stored_request["params"])
360
+ end
361
+
362
+ # Sorts hash keys so the digest does not depend on the params order
363
+ def canonical(value)
364
+ case value
365
+ when Hash then value.keys.sort.map { |k| [k, canonical(value[k])] }
366
+ when Array then value.map { |v| canonical(v) }
367
+ else value
368
+ end
369
+ end
370
+
371
+ def digest(value)
372
+ Digest::SHA256.hexdigest(value)
373
+ end
374
+
375
+ def key(idempotency_key)
376
+ "#{gem_prefix}#{idempotency_key}"
255
377
  end
256
378
 
257
379
  def gem_prefix
@@ -267,7 +389,7 @@ module Grape
267
389
  end
268
390
 
269
391
  def logger
270
- return @logger if @logger
392
+ return @logger if @logger && @logger.equal?(configuration.logger)
271
393
 
272
394
  @logger = configuration.logger
273
395
  @logger.level = configuration.logger_level
@@ -282,7 +404,7 @@ module Grape
282
404
  class Configuration
283
405
  attr_accessor :storage, :logger, :logger_level, :logger_prefix, :expires_in, :idempotency_key_header,
284
406
  :request_id_header, :conflict_error_response, :processing_response, :mandatory_header_response,
285
- :manage_redis_exceptions
407
+ :manage_redis_exceptions, :replay_headers_denylist, :encryption_key, :encryptor
286
408
 
287
409
  class Error < StandardError; end
288
410
 
@@ -295,6 +417,9 @@ module Grape
295
417
  @idempotency_key_header = "idempotency-key"
296
418
  @request_id_header = "x-request-id"
297
419
  @manage_redis_exceptions = false
420
+ @replay_headers_denylist = []
421
+ @encryption_key = nil
422
+ @encryptor = nil
298
423
  @conflict_error_response = {
299
424
  "title" => "Idempotency-Key is already used",
300
425
  "detail" => "This operation is idempotent and it requires correct usage of Idempotency Key. Idempotency Key MUST not be reused across different payloads of this operation."
@@ -310,4 +435,4 @@ module Grape
310
435
  end
311
436
  end
312
437
  end
313
- end
438
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: grape-idempotency
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 1.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Juan Carlos García
8
- autorequire:
8
+ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2023-12-12 00:00:00.000000000 Z
11
+ date: 2026-10-15 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: grape
@@ -94,11 +94,13 @@ dependencies:
94
94
  - - ">="
95
95
  - !ruby/object:Gem::Version
96
96
  version: '0'
97
- description: Add idempotency support to your Grape APIs for safely retrying requests
98
- without accidentally performing the same operation twice. When creating or updating
99
- an object, use an idempotency key. Then, if a connection error occurs, you can safely
100
- repeat the request without risk of creating a second object or performing the update
101
- twice.
97
+ description: Idempotency-Key header support for Grape REST APIs, based on the IETF
98
+ Idempotency-Key HTTP header draft. Wrap a POST or PATCH endpoint in an idempotent
99
+ block and clients can safely retry requests after timeouts or network errors without
100
+ creating duplicate payments, orders or records. The first response (status, headers
101
+ and body) is stored in Redis and replayed for retries with the same key. Concurrent
102
+ duplicate requests get a 409 Conflict, and reusing a key with a different payload
103
+ gets a 422 Unprocessable Entity. Works with Grape 1.x to 4.x and Ruby 2.6+.
102
104
  email:
103
105
  - jugade92@gmail.com
104
106
  executables: []
@@ -111,14 +113,21 @@ files:
111
113
  - grape-idempotency.gemspec
112
114
  - lib/grape-idempotency.rb
113
115
  - lib/grape/idempotency.rb
116
+ - lib/grape/idempotency/encryptor.rb
114
117
  - lib/grape/idempotency/helpers.rb
115
118
  - lib/grape/idempotency/middleware/error.rb
119
+ - lib/grape/idempotency/middleware/response_capture.rb
116
120
  - lib/grape/idempotency/version.rb
117
121
  homepage: https://github.com/jcagarcia/grape-idempotency
118
122
  licenses:
119
123
  - MIT
120
- metadata: {}
121
- post_install_message:
124
+ metadata:
125
+ homepage_uri: https://github.com/jcagarcia/grape-idempotency
126
+ source_code_uri: https://github.com/jcagarcia/grape-idempotency
127
+ changelog_uri: https://github.com/jcagarcia/grape-idempotency/blob/main/CHANGELOG.md
128
+ bug_tracker_uri: https://github.com/jcagarcia/grape-idempotency/issues
129
+ documentation_uri: https://github.com/jcagarcia/grape-idempotency#readme
130
+ post_install_message:
122
131
  rdoc_options: []
123
132
  require_paths:
124
133
  - lib
@@ -134,7 +143,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
134
143
  version: '0'
135
144
  requirements: []
136
145
  rubygems_version: 3.4.10
137
- signing_key:
146
+ signing_key:
138
147
  specification_version: 4
139
- summary: Gem for supporting idempotency in your Grape APIs
148
+ summary: 'Idempotency-Key support for Grape APIs: safe retries and duplicate request
149
+ prevention backed by Redis'
140
150
  test_files: []