faraday_dynamic_timeout 1.0.0 → 1.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8e0d5c3fe787914d19d470b70182719f5c4863106a5d9fc4854df38e219f1a5a
-  data.tar.gz: 485e82cfdd0b9098801cffd08fad0426a763ec4a51c746b97a3a553c12281d2d
+  metadata.gz: aba9d8b057d18dc3912677fd37e02148e5bc69ea78fbbb57cb8b4c3a790c4624
+  data.tar.gz: 3f17c539413375236113a2037a89717639b35669e0625926ea0d727525c37ab6
 SHA512:
-  metadata.gz: a5025c609b11ae25188e685a15dc44ae55499fea41f16bad4c1cf925b3465942cf42ea77d734eeff8af417f6df7610c8c4dbe5032169dac22100004d1348bda0
-  data.tar.gz: 3443e7adc22b8bcc9c8b3adc2a25d24187b2f6b284e03429fcaabee8ff80299e976f8fe019cce1ffb3ff55a2020eb6b2be7b31cfe39c8773922327bb86fdc4de
+  metadata.gz: 57bca13ad57839868f5dec34419271325e72fde1d0e45e0c7a7cd0bedce5350ae62a41ec7b026e95bf3c57fadf50dfc76f44ec630d0bf14d86191d41b9e5b07f
+  data.tar.gz: e0f0c827dfdd3b16a9f38e45f7127aa4781ab8d25060ad7bc4a47705e9d0d8410ea7a997abbb8f27d81e332dfd0ee6ea16602bc2f4c9b9e91201f3b8e2cd58ed

data/CHANGELOG.md

@@ -4,6 +4,33 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 1.1.1
+
+### Added
+
+- The middleware now fails open on any Redis error: instead of raising and taking down HTTP traffic, the request is made using the highest configured timeout without throttling.
+
+### Fixed
+
+- Padded the TTL used to clean up orphaned Redis entries for throttle slots and request counters. Previously the TTL was set to the bucket timeout, but a request's total wall time can legitimately exceed the timeout (adapters apply it to the open/read/write phases individually), so slots could expire and be reclaimed while requests were still in flight, allowing the concurrency limit to be exceeded.
+- A `Restrainer::ThrottledError` raised from within the request itself (e.g. from a nested middleware) is no longer mistaken for a full bucket. Previously such an error would cause the request to be retried on the next bucket, executing the HTTP request a second time.
+- Made bucket configuration memoization safe when the configuration array is mutated concurrently by another thread.
+- Added missing `require "securerandom"`, which previously only worked because a dependency loaded it.
+
+### Documentation
+
+- Documented the `RequestInfo` fields, the behavior of `timed_out?` with respect to connection-level timeouts, the fact that `:callback` runs inline and can mask the request result if it raises, and the streaming/parallel adapter limitations.
+
+### Removed
+
+- Removed unused internal `Strategy` class.
+
+## 1.1.0
+
+### Added
+
+- Add `before_request` option on middleware to allow making custom changes to the request based on the timeout value being used.
+
 ## 1.0.0
 
 ### Added

data/README.md

@@ -50,11 +50,35 @@ In this example, the timeout will be set to 8 seconds if there are 5 or fewer re
 
 - `:name` - An optional name for the resource. By default the hostname and port of the request URL will be used to identify the resource. Each resource will report a separate count of concurrent requests and processes. You can group multiple resources from different hosts together with the `:name` option.
 
-- `:callback` - An optional callback that will be called after each request. The callback can be a `Proc` or any object that responds to `call`. It will be called with a `FaradayDyamicTimeout::RequestInfo` argument. You can use this to log the number of concurrent requests or to report metrics to a monitoring system. This can be very useful for tuning the bucket settings.
+- `:before_request` - An optional callback that will be called before each request is made. The callback can be a `Proc` or any object that responds to `call`. It will be called before the request is made with the `Faraday::Env` object and the timeout being used. You can use this to make changes to the request based on the timeout being used. This can be used, for example, to add the timeout to the request payload.
+
+- `:filter` - An optional callback that will be called before each request with the `Faraday::Env` object. The callback can be a `Proc` or any object that responds to `call`. If it returns a falsey value, the middleware will pass the request through without applying any timeout or throttling. You can use this to limit the middleware to specific requests (for example, only search requests).
+
+- `:callback` - An optional callback that will be called after each request. The callback can be a `Proc` or any object that responds to `call`. It will be called with a `FaradayDynamicTimeout::RequestInfo` argument. You can use this to log the number of concurrent requests or to report metrics to a monitoring system. This can be very useful for tuning the bucket settings. Note that the callback runs inline before the response is returned, so any exception it raises will propagate out of the request and can mask the original result. If your callback might raise (for example, when a metrics backend is unavailable), rescue inside the callback itself.
+
+### The RequestInfo object
+
+The `:callback` option receives a `FaradayDynamicTimeout::RequestInfo` object with details about the completed request:
+
+- `duration` - The time spent making the request in seconds.
+- `timeout` - The timeout that was applied, or `nil` if none was applied (for example when the request was throttled before it was made).
+- `request_count` - The number of concurrent requests observed for the resource.
+- `error` - The exception raised by the request, if any.
+- `throttled?` - True if the request was rejected because all buckets were full.
+- `timed_out?` - True if the request failed with a `Faraday::TimeoutError`. This reflects the read/request timeout that this middleware sets. Connection-level timeouts are reported by adapters as `Faraday::ConnectionFailed` on some adapters rather than `Faraday::TimeoutError`, so they are not counted as timeouts here; inspect `error` directly if you need to distinguish them.
+- `error?` - True if the request raised any error.
+
+### Redis availability
+
+The middleware depends on Redis to coordinate counts across processes, but Redis problems will not take down your HTTP traffic. If any Redis call fails (a connection failure, a timeout, a command error, etc.), the middleware fails open: the request is made using the highest configured timeout without throttling, and the request count reported to the `:callback` will be `1`. Cleanup of Redis bookkeeping is best effort and any leftover entries expire automatically.
+
+### Limitations
+
+- **Streaming and parallel adapters.** A request occupies its bucket slot only for the duration of the underlying `app.call`. If you use a streaming response where the body is consumed after the middleware returns, or a Faraday adapter running requests in parallel, the concurrency accounting will not reflect the time spent streaming or the true number of in-flight requests.
 
 ### Capacity Strategy
 
-You can use the `FaraadyDynamicTimeout::CapacityStrategy` class to build a bucket configuration based on the current capacity of your application rather than hard coding bucket limits. This can be useful if you have a system that can scale up and down based on load. It works by estimating the total number of threads available in your application and uses that value to calculate bucket limits based on a percentage provided by the `:capacity` option.
+You can use the `FaradayDynamicTimeout::CapacityStrategy` class to build a bucket configuration based on the current capacity of your application rather than hard coding bucket limits. This can be useful if you have a system that can scale up and down based on load. It works by estimating the total number of threads available in your application and uses that value to calculate bucket limits based on a percentage provided by the `:capacity` option.
 
 ```ruby
 capacity = FaradayDynamicTimeout::CapacityStrategy.new(
@@ -80,6 +104,14 @@ For this example, we will configure the `opensearch` gem with this middleware al
 # Set up a redis connection to coordinate counting concurrent requests.
 redis = Redis.new(url: ENV.fetch("REDIS_URL"))
 
+# Set the query timeout to match the request timeout so the search nodes will stop
+# processing if the request times out.
+set_query_timeout = ->(env, timeout) do
+  query_params = (Faraday::Utils.parse_query(env.url.query) || {})
+  query_params["timeout"] = "#{(timeout * 1000).ceil}ms"
+  env.url.query = Faraday::Utils.build_query(query_params)
+end
+
 # Set up a statsd client to report metrics with the DataDog extensions.
 statsd = Statsd.new(ENV.fetch("STATSD_HOST"), ENV.fetch("STATSD_PORT"))
 
@@ -95,13 +127,14 @@ end
 client = OpenSearch::Client.new(host: 'localhost', port: '9200') do |faraday|
   faraday.request :dynamic_timeout,
                   buckets: [
-                    {timeout: 8, max_requests: 5},
-                    {timeout: 1, max_requests: 10},
-                    {timeout: 0.5, max_requests: 20}
+                    {timeout: 8, limit: 5},
+                    {timeout: 1, limit: 10},
+                    {timeout: 0.5, limit: 20}
                   ],
                   name: "opensearch",
                   redis: redis,
                   filter: ->(env) { env.url.path.end_with?("/_search") },
+                  before_request: set_query_timeout,
                   callback: metrics_callback
 end
 ```

data/VERSION

@@ -1 +1 @@
-1.0.0
+1.1.1

data/faraday_dynamic_timeout.gemspec

@@ -14,6 +14,8 @@ Gem::Specification.new do |spec|
     .
     Gemfile
     Gemfile.lock
+    Appraisals
+    gemfiles/
     Rakefile
     bin/
     spec/
@@ -29,5 +31,5 @@ Gem::Specification.new do |spec|
 
   spec.add_development_dependency "bundler"
 
-  spec.required_ruby_version = ">= 2.5"
+  spec.required_ruby_version = ">= 2.6"
 end

data/lib/faraday_dynamic_timeout/middleware.rb

@@ -11,13 +11,15 @@ module FaradayDynamicTimeout
       end
 
       @memoized_buckets = []
-      @mutex = Mutex.new
     end
 
     def call(env)
-      buckets = sorted_buckets
+      # A Redis outage must never take down HTTP traffic, so if building the bucket
+      # configuration touches Redis (e.g. a capacity based strategy) and it is
+      # unavailable, fail open and let the request through with no dynamic timeout.
+      buckets = safe_redis { sorted_buckets }
       redis = redis_client
-      return app.call(env) if !enabled?(env) || buckets.empty? || redis.nil?
+      return app.call(env) if !enabled?(env) || buckets.nil? || buckets.empty? || redis.nil?
 
       error = nil
       bucket_timeout = nil
@@ -27,7 +29,7 @@ module FaradayDynamicTimeout
       count_request(env.url, redis, buckets, callback) do |request_count|
         execute_with_timeout(env.url, buckets, request_count, redis) do |timeout|
           bucket_timeout = timeout
-          set_timeout(env.request, timeout) if timeout
+          set_timeout(env, timeout) if timeout
 
           # Resetting the start time to more accurately reflect the time spent in the request.
           start_time = monotonic_time if callback
@@ -55,15 +57,16 @@ module FaradayDynamicTimeout
       config = config.call if config.respond_to?(:call)
       config = Array(config)
       memoized_config, memoized_buckets = @memoized_buckets
-
-      if config == memoized_config
-        memoized_buckets
-      else
-        duplicated_config = @mutex.synchronize { config.collect(&:dup) }
-        buckets = Bucket.from_hashes(duplicated_config)
-        @memoized_buckets = [duplicated_config, buckets]
-        buckets
-      end
+      return memoized_buckets if config == memoized_config
+
+      # Duplicate the config before storing it so a caller mutating the array or its
+      # hashes cannot change the memoized snapshot out from under us. The memo is
+      # published with a single assignment; concurrent threads may redundantly rebuild
+      # it, which is harmless.
+      config = config.collect(&:dup)
+      buckets = Bucket.from_hashes(config)
+      @memoized_buckets = [config, buckets]
+      buckets
     end
 
     def enabled?(env)
@@ -84,10 +87,13 @@ module FaradayDynamicTimeout
           retval = yield(bucket.timeout)
           break
         else
-          restrainer = Restrainer.new(restrainer_name(uri, bucket.timeout), limit: bucket.limit, timeout: bucket.timeout, redis: redis)
+          restrainer = Restrainer.new(restrainer_name(uri, bucket.timeout), limit: bucket.limit, timeout: slot_ttl(bucket.timeout), redis: redis)
           begin
-            retval = restrainer.throttle { yield(bucket.timeout) }
-            break
+            # Acquire the slot explicitly rather than using Restrainer#throttle so that a
+            # ThrottledError raised from within the request itself (e.g. from a nested
+            # middleware) is not mistaken for this bucket being full, which would retry
+            # the request on the next bucket and execute it a second time.
+            process_id = restrainer.lock!
           rescue Restrainer::ThrottledError
             total_requests += bucket.limit
             if buckets.empty?
@@ -96,6 +102,21 @@ module FaradayDynamicTimeout
               request_count = [request_count, total_requests + 1].max
               raise ThrottledError.new("Request to #{base_url(uri)} aborted due to #{request_count} concurrent requests", request_count: request_count)
             end
+          rescue Redis::BaseError
+            # Redis is unavailable, so throttling cannot be enforced. Fail open using the
+            # current (highest available) timeout rather than failing the request.
+            retval = yield(bucket.timeout)
+            break
+          else
+            begin
+              retval = yield(bucket.timeout)
+            ensure
+              # Releasing the slot is best effort; if Redis is unavailable the slot will
+              # expire on its own via the TTL. A cleanup failure must not mask the result
+              # of a request that has already been made.
+              safe_redis { restrainer.release!(process_id) }
+            end
+            break
           end
         end
       end
@@ -103,27 +124,56 @@ module FaradayDynamicTimeout
       retval
     end
 
-    def set_timeout(request, timeout)
+    def set_timeout(env, timeout)
+      request = env.request
       request.timeout = timeout
       request.open_timeout = nil
       request.write_timeout = nil
       request.read_timeout = nil
+
+      option(:before_request)&.call(env, timeout)
+    end
+
+    # The TTL used to clean up orphaned slot and counter entries in Redis. The bucket
+    # timeout only bounds each phase of the request (open/read/write), not its total
+    # wall time, so the TTL is padded to avoid expiring entries for requests that are
+    # still legitimately in flight.
+    def slot_ttl(timeout)
+      [timeout * 3, 60].max
     end
 
     # Track how many requests are currently being executed only if a callback has been configured.
+    #
+    # Each Redis operation is guarded so that a Redis outage degrades to a pass through
+    # (reporting a request count of 1) rather than blocking the request. The counter entry
+    # is released on the way out, but only when it was successfully added.
     def count_request(uri, redis, buckets, callback)
-      if callback
-        ttl = buckets.last.timeout
-        ttl = 60 if ttl <= 0
-        request_counter = Counter.new(name: request_counter_name(uri), redis: redis, ttl: ttl)
-        request_counter.execute do
-          yield request_counter.value
-        end
-      else
-        yield 1
+      return yield(1) unless callback
+
+      # The counter entry is added before a bucket has been selected, so the TTL must
+      # conservatively cover the highest timeout (the last bucket); a shorter TTL could
+      # expire entries for requests still legitimately in flight. The TTL only comes
+      # into play for orphaned entries since entries are normally removed when the
+      # request finishes.
+      ttl = slot_ttl(buckets.last.timeout)
+      request_counter = Counter.new(name: request_counter_name(uri), redis: redis, ttl: ttl)
+      id = safe_redis { request_counter.track! }
+      begin
+        yield(safe_redis { request_counter.value } || 1)
+      ensure
+        safe_redis { request_counter.release!(id) } if id
       end
     end
 
+    # Run a block that talks to Redis, returning nil instead of raising if the Redis
+    # call fails for any reason. Used to keep Redis problems from taking down HTTP
+    # traffic.
+    def safe_redis
+      yield
+    rescue Redis::BaseError
+      nil
+    end
+
     def request_counter_name(uri)
       "#{redis_key_namespace(uri)}.requests"
     end

data/lib/faraday_dynamic_timeout.rb

@@ -2,6 +2,7 @@
 
 require "faraday"
 require "restrainer"
+require "securerandom"
 require "socket"
 
 require_relative "faraday_dynamic_timeout/bucket"

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: faraday_dynamic_timeout
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.1
 platform: ruby
 authors:
 - Brian Durand
-autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-12-29 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: faraday
@@ -52,33 +51,27 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description:
 email:
 - bbdurand@gmail.com
 executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- Appraisals
 - CHANGELOG.md
 - MIT-LICENSE.txt
 - README.md
 - VERSION
 - faraday_dynamic_timeout.gemspec
-- gemfiles/faraday_1.gemfile
-- gemfiles/faraday_2.gemfile
 - lib/faraday_dynamic_timeout.rb
 - lib/faraday_dynamic_timeout/bucket.rb
 - lib/faraday_dynamic_timeout/capacity_strategy.rb
 - lib/faraday_dynamic_timeout/counter.rb
 - lib/faraday_dynamic_timeout/middleware.rb
 - lib/faraday_dynamic_timeout/request_info.rb
-- lib/faraday_dynamic_timeout/strategy.rb
 homepage: https://github.com/bdurand/faraday_dynamic_timeout
 licenses:
 - MIT
 metadata: {}
-post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -86,15 +79,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '2.5'
+      version: '2.6'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.12
-signing_key:
+rubygems_version: 4.0.3
 specification_version: 4
 summary: Faraday middleware to dynamically set a request timeout based on the number
   of concurrent requests and throttle the number of requests that can be made.

data/Appraisals

@@ -1,7 +0,0 @@
-appraise "faraday_2" do
-  gem "faraday", "~> 2.0"
-end
-
-appraise "faraday_1" do
-  gem "faraday", "~> 1.0"
-end

data/gemfiles/faraday_1.gemfile

@@ -1,15 +0,0 @@
-# This file was generated by Appraisal
-
-source "https://rubygems.org"
-
-gem "webmock"
-gem "appraisal"
-gem "dotenv"
-gem "rake"
-gem "rspec", "~> 3.10"
-gem "standard", "~> 1.0"
-gem "simplecov", "~> 0.21", require: false
-gem "yard"
-gem "faraday", "~> 1.0"
-
-gemspec path: "../"

data/gemfiles/faraday_2.gemfile

@@ -1,15 +0,0 @@
-# This file was generated by Appraisal
-
-source "https://rubygems.org"
-
-gem "webmock"
-gem "appraisal"
-gem "dotenv"
-gem "rake"
-gem "rspec", "~> 3.10"
-gem "standard", "~> 1.0"
-gem "simplecov", "~> 0.21", require: false
-gem "yard"
-gem "faraday", "~> 2.0"
-
-gemspec path: "../"

data/lib/faraday_dynamic_timeout/strategy.rb

@@ -1,10 +0,0 @@
-# frozen_string_literal: true
-
-module FaradayDynamicTimeout
-  class Strategy
-    def initialize(buckets:, options: {})
-      @config = buckets
-      @options = options
-    end
-  end
-end