faraday-retry 1.0.3 → 2.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2605913141029cfe8448ca6b6007a1c892a7ce8629689c0b979035ad59a6e975
-  data.tar.gz: ceb3abfc225c30b4ecc0868be11bbb52e88083e4edf149eb8e40f2be3a87d70b
+  metadata.gz: f4c985b60e619c7a2161b8643fda86adcd4927847a66d903bbc22a32ed851d13
+  data.tar.gz: 010d8a002a7310f82c44395344cc81cb3f8acfe9e53ca8aae8ae84cd3b66d90a
 SHA512:
-  metadata.gz: 21bfc830b3c1fd43f16b9dcb1b44a682cdb0fd3fd9c7cbc708cd998f676d7bcdbab5beea1d7f7905ba5cb6d631ba4e5dd2347be8ae7c73def1baf8bbe45f9b8e
-  data.tar.gz: 7dea5532d1c3045da07ad24dd76157083cecbbfa96be0fea7b07f45be6114be0e18f191b875d8e76349c7dd5bad3965de073a8ddc1f20b83b18ab1dfe7d2a133
+  metadata.gz: b6dc78fe8c3b3ccc189c086271eb646e06f6e598a0a2b311ce3f0831321ba8f00b01d24be1c29521f7bbef5367eb17c43be7f6dfaa6402f0ec9cf206483f8794
+  data.tar.gz: 6338884c1806a7d06f33457db3939d71aaf761c9cc8c427a14c68257600d0601ff6765dde9b061254fd5af8f4ff77315d018a8d42d5326842fd0d26d0e52219f

data/CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## v2.2.0 (2023-06-01)
+
+* Support new `header_parser_block` option. [PR #28](https://github.com/lostisland/faraday-retry/pull/28). Thanks, [@zavan]!
+
+## v2.1.0 (2023-03-03)
+
+* Support for custom RateLimit headers. [PR #13](https://github.com/lostisland/faraday-retry/pull/13). Thanks, [@brookemckim]!
+
+v2.1.1 (2023-02-17) is a spurious not-released version that you may have seen mentioned in this CHANGELOG.
+
+## v2.0.0 (2022-06-08)
+
+### Changed
+
+* `retry_block` now takes keyword arguments instead of positional (backwards incompatible)
+* `retry_block`'s `retry_count` argument now counts up from 0, instead of old `retries_remaining`
+
+### Added
+
+* Support for the `RateLimit-Reset` header. [PR #9](https://github.com/lostisland/faraday-retry/pull/9). Thanks, [@maxprokopiev]!
+* `retry_block` has additional `will_retry_in` argument with upcoming delay before retry in seconds.
+
 ## v1.0
 
 Initial release.
@@ -8,3 +30,7 @@ This release consists of the same middleware that was previously bundled with Fa
 ### Fixed
 
 *  Retry middleware `retry_block` is not called if retry will not happen due to `max_interval`, https://github.com/lostisland/faraday/pull/1350
+
+[@maxprokopiev]: https://github.com/maxprokopiev
+[@brookemckim]: https://github.com/brookemckim
+[@zavan]: https://github.com/zavan

data/README.md

@@ -1,6 +1,6 @@
 # Faraday Retry
 
-[![GitHub Workflow Status](https://img.shields.io/github/workflow/status/lostisland/faraday-retry/CI)](https://github.com/lostisland/faraday-retry/actions?query=branch%3Amain)
+[![CI](https://github.com/lostisland/faraday-retry/actions/workflows/ci.yaml/badge.svg)](https://github.com/lostisland/faraday-retry/actions/workflows/ci.yaml)
 [![Gem](https://img.shields.io/gem/v/faraday-retry.svg?style=flat-square)](https://rubygems.org/gems/faraday-retry)
 [![License](https://img.shields.io/github/license/lostisland/faraday-retry.svg?style=flat-square)](LICENSE.md)
 
@@ -96,11 +96,14 @@ retry_options = {
 }
 ```
 
-#### Automatically handle the `Retry-After` header
+#### Automatically handle the `Retry-After` and `RateLimit-Reset` headers
 
-Some APIs, like the [Slack API](https://api.slack.com/docs/rate-limits), will inform you when you reach their API limits by replying with a response status code of `429` and a response header of `Retry-After` containing a time in seconds. You should then only retry querying after the amount of time provided by the `Retry-After` header, otherwise you won't get a response.
+Some APIs, like the [Slack API](https://api.slack.com/docs/rate-limits), will inform you when you reach their API limits by replying with a response status code of `429`
+and a response header of `Retry-After` containing a time in seconds. You should then only retry querying after the amount of time provided by the `Retry-After` header,
+otherwise you won't get a response. Other APIs communicate their rate limits via the [RateLimit-xxx](https://www.ietf.org/archive/id/draft-ietf-httpapi-ratelimit-headers-05.html#name-providing-ratelimit-fields) headers
+where `RateLimit-Reset` behaves similarly to the `Retry-After`.
 
-You can automatically handle this and have Faraday pause and retry for the right amount of time by including the `429` status code in the retry statuses list:
+You can automatically handle both headers and have Faraday pause and retry for the right amount of time by including the `429` status code in the retry statuses list:
 
 ```ruby
 retry_options = {
@@ -108,6 +111,17 @@ retry_options = {
 }
 ```
 
+If you are working with an API which does not comply with the Rate Limit RFC you can specify custom headers to be used for retry and reset, as well as a block to parse the headers:
+
+```ruby
+retry_options = {
+  retry_statuses: [429],
+  rate_limit_retry_header: 'x-rate-limit-retry-after',
+  rate_limit_reset_header: 'x-rate-limit-reset',
+  header_parser_block: ->(value) { Time.at(value.to_i).utc - Time.now.utc }
+}
+```
+
 #### Specify a custom retry logic
 
 You can also specify a custom retry logic with the `retry_if` option.
@@ -127,15 +141,19 @@ retry_options = {
 
 ### Call a block on every retry
 
-You can specify a block through the `retry_block` option that will be called before every retry.
-There are many different applications for this feature, spacing from instrumentation to monitoring.
-Request environment, middleware options, current number of retries and the exception is passed to the block as parameters.
+You can specify a proc object through the `retry_block` option that will be called before every
+retry, before  There are many different applications for this feature, spacing from instrumentation to monitoring.
+
+
+The block is passed keyword arguments with contextual information: Request environment, middleware options, current number of retries, exception, and amount of time we will wait before retrying. (retry_block is called before the wait time happens)
+
+
 For example, you might want to keep track of the response statuses:
 
 ```ruby
 response_statuses = []
 retry_options = {
-  retry_block: -> (env, options, retries, exc) { response_statuses << env.status }
+  retry_block: -> (env:, options:, retry_count:, exception:, will_retry_in:) { response_statuses << env.status }
 }
 ```
 
@@ -147,8 +165,9 @@ Then, run `bin/test` to run the tests.
 
 To install this gem onto your local machine, run `rake build`.
 
-To release a new version, make a commit with a message such as "Bumped to 0.0.2" and then run `rake release`.
-See how it works [here](https://bundler.io/guides/creating_gem.html#releasing-the-gem).
+### Releasing a new version
+
+To release a new version, make a commit with a message such as "Bumped to 0.0.2", and change the _Unreleased_ heading in `CHANGELOG.md` to a heading like "0.0.2 (2022-01-01)", and then use GitHub Releases to author a release. A GitHub Actions workflow then publishes a new gem to [RubyGems.org](https://rubygems.org/gems/faraday-retry).
 
 ## Contributing
 

data/lib/faraday/retry/middleware.rb

@@ -26,7 +26,8 @@ module Faraday
                                            :interval_randomness,
                                            :backoff_factor, :exceptions,
                                            :methods, :retry_if, :retry_block,
-                                           :retry_statuses)
+                                           :retry_statuses, :rate_limit_retry_header,
+                                           :rate_limit_reset_header, :header_parser_block)
 
         DEFAULT_CHECK = ->(_env, _exception) { false }
 
@@ -71,7 +72,7 @@ module Faraday
         end
 
         def retry_block
-          self[:retry_block] ||= proc {} # rubocop:disable Lint/EmptyBlock
+          self[:retry_block] ||= proc {}
         end
 
         def retry_statuses
@@ -105,12 +106,24 @@ module Faraday
       #   if the exception produced is non-recoverable or if the
       #   the HTTP method called is not idempotent.
       # @option options [Block] :retry_block block that is executed before
-      #   every retry. Request environment, middleware options, current number
-      #   of retries and the exception is passed to the block as parameters.
+      #   every retry. The block will be yielded keyword arguments:
+      #     * env [Faraday::Env]: Request environment
+      #     * options [Faraday::Options]: middleware options
+      #     * retry_count [Integer]: how many retries have already occured (starts at 0)
+      #     * exception [Exception]: exception that triggered the retry,
+      #       will be the synthetic `Faraday::RetriableResponse` if the
+      #       retry was triggered by something other than an exception.
+      #     * will_retry_in [Float]: retry_block is called *before* the retry
+      #       delay, actual retry will happen in will_retry_in number of
+      #       seconds.
       # @option options [Array] :retry_statuses Array of Integer HTTP status
       #   codes or a single Integer value that determines whether to raise
       #   a Faraday::RetriableResponse exception based on the HTTP status code
       #   of an HTTP response.
+      # @option options [Block] :header_parser_block block that will receive
+      #   the the value of the retry header and should return the number of
+      #   seconds to wait before retrying the request. This is useful if the
+      #   value of the header is not a number of seconds or a RFC 2822 formatted date.
       def initialize(app, options = nil)
         super(app)
         @options = Options.from(options)
@@ -118,7 +131,7 @@ module Faraday
       end
 
       def calculate_sleep_amount(retries, env)
-        retry_after = calculate_retry_after(env)
+        retry_after = [calculate_retry_after(env), calculate_rate_limit_reset(env)].compact.max
         retry_interval = calculate_retry_interval(retries)
 
         return if retry_after && retry_after > @options.max_interval
@@ -145,7 +158,13 @@ module Faraday
             retries -= 1
             rewind_files(request_body)
             if (sleep_amount = calculate_sleep_amount(retries + 1, env))
-              @options.retry_block.call(env, @options, retries, e)
+              @options.retry_block.call(
+                env: env,
+                options: @options,
+                retry_count: @options.max - (retries + 1),
+                exception: e,
+                will_retry_in: sleep_amount
+              )
               sleep sleep_amount
               retry
             end
@@ -198,21 +217,18 @@ module Faraday
         end
       end
 
+      # RFC for RateLimit Header Fields for HTTP:
+      # https://www.ietf.org/archive/id/draft-ietf-httpapi-ratelimit-headers-05.html#name-fields-definition
+      def calculate_rate_limit_reset(env)
+        reset_header = @options.rate_limit_reset_header || 'RateLimit-Reset'
+        parse_retry_header(env, reset_header)
+      end
+
       # MDN spec for Retry-After header:
       # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After
       def calculate_retry_after(env)
-        response_headers = env[:response_headers]
-        return unless response_headers
-
-        retry_after_value = env[:response_headers]['Retry-After']
-
-        # Try to parse date from the header value
-        begin
-          datetime = DateTime.rfc2822(retry_after_value)
-          datetime.to_time - Time.now.utc
-        rescue ArgumentError
-          retry_after_value.to_f
-        end
+        retry_header = @options.rate_limit_retry_header || 'Retry-After'
+        parse_retry_header(env, retry_header)
       end
 
       def calculate_retry_interval(retries)
@@ -225,6 +241,25 @@ module Faraday
 
         current_interval + random_interval
       end
+
+      def parse_retry_header(env, header)
+        response_headers = env[:response_headers]
+        return unless response_headers
+
+        retry_after_value = env[:response_headers][header]
+
+        if @options.header_parser_block
+          @options.header_parser_block.call(retry_after_value)
+        else
+          # Try to parse date from the header value
+          begin
+            datetime = DateTime.rfc2822(retry_after_value)
+            datetime.to_time - Time.now.utc
+          rescue ArgumentError
+            retry_after_value.to_f
+          end
+        end
+      end
     end
   end
 end

data/lib/faraday/retry/version.rb

@@ -2,6 +2,6 @@
 
 module Faraday
   module Retry
-    VERSION = '1.0.3'
+    VERSION = '2.2.0'
   end
 end

data/lib/faraday/retry.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require 'faraday'
 require_relative 'retriable_response'
 require_relative 'retry/middleware'
 require_relative 'retry/version'

metadata

@@ -1,15 +1,141 @@
 --- !ruby/object:Gem::Specification
 name: faraday-retry
 version: !ruby/object:Gem::Version
-  version: 1.0.3
+  version: 2.2.0
 platform: ruby
 authors:
 - Mattia Giuffrida
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2022-01-06 00:00:00.000000000 Z
-dependencies: []
+date: 2023-06-01 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: faraday
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: simplecov
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.21.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.21.0
+- !ruby/object:Gem::Dependency
+  name: rubocop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.21.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.21.0
+- !ruby/object:Gem::Dependency
+  name: rubocop-packaging
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.5.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.5.0
+- !ruby/object:Gem::Dependency
+  name: rubocop-performance
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: rubocop-rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
 description: 'Catches exceptions and retries each request a limited number of times.
 
   '
@@ -31,11 +157,10 @@ licenses:
 - MIT
 metadata:
   bug_tracker_uri: https://github.com/lostisland/faraday-retry/issues
-  changelog_uri: https://github.com/lostisland/faraday-retry/blob/v1.0.3/CHANGELOG.md
-  documentation_uri: http://www.rubydoc.info/gems/faraday-retry/1.0.3
+  changelog_uri: https://github.com/lostisland/faraday-retry/blob/v2.2.0/CHANGELOG.md
+  documentation_uri: http://www.rubydoc.info/gems/faraday-retry/2.2.0
   homepage_uri: https://github.com/lostisland/faraday-retry
   source_code_uri: https://github.com/lostisland/faraday-retry
-  wiki_uri: https://github.com/lostisland/faraday-retry/wiki
 post_install_message: 
 rdoc_options: []
 require_paths:
@@ -44,7 +169,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '2.4'
+      version: '2.6'
   - - "<"
     - !ruby/object:Gem::Version
       version: '4'