faraday_dynamic_timeout 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 8e0d5c3fe787914d19d470b70182719f5c4863106a5d9fc4854df38e219f1a5a
+  data.tar.gz: 485e82cfdd0b9098801cffd08fad0426a763ec4a51c746b97a3a553c12281d2d
+SHA512:
+  metadata.gz: a5025c609b11ae25188e685a15dc44ae55499fea41f16bad4c1cf925b3465942cf42ea77d734eeff8af417f6df7610c8c4dbe5032169dac22100004d1348bda0
+  data.tar.gz: 3443e7adc22b8bcc9c8b3adc2a25d24187b2f6b284e03429fcaabee8ff80299e976f8fe019cce1ffb3ff55a2020eb6b2be7b31cfe39c8773922327bb86fdc4de

data/Appraisals

@@ -0,0 +1,7 @@
+appraise "faraday_2" do
+  gem "faraday", "~> 2.0"
+end
+
+appraise "faraday_1" do
+  gem "faraday", "~> 1.0"
+end

data/CHANGELOG.md

@@ -0,0 +1,11 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## 1.0.0
+
+### Added
+
+- Initial release

data/MIT-LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright 2023 Brian Durand
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,135 @@
+# Faraday Dynamic Timeout Middleware
+
+[![Continuous Integration](https://github.com/bdurand/faraday_dynamic_timeout/actions/workflows/continuous_integration.yml/badge.svg)](https://github.com/bdurand/faraday_dynamic_timeout/actions/workflows/continuous_integration.yml)
+[![Regression Test](https://github.com/bdurand/faraday_dynamic_timeout/actions/workflows/regression_test.yml/badge.svg)](https://github.com/bdurand/faraday_dynamic_timeout/actions/workflows/regression_test.yml)
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
+[![Gem Version](https://badge.fury.io/rb/faraday_dynamic_timeout.svg)](https://badge.fury.io/rb/faraday_dynamic_timeout)
+
+This gem provides Faraday middleware that allows you to set dynamic timeouts on HTTP requests based on the current number of requests being made to an endpoint. This allows you to set a long enough timeout to handle your slowest requests when the system is healthy, but then progressively set shorter timeouts as load increases so requests can fail fast in an unhealthy system.
+
+It's always hard to figure out what the proper timeout is for an HTTP request. If you set it too short, then you will get errors on an occasional request that takes just a little longer than normal. If you set it too long, then you will end up with a system that becomes unresponsive when something goes wrong and most of your application resouces can end up waiting on an external system that just isn't working.
+
+This middleware works by letting you set "buckets" of timeouts. In a simple use case, you would setup a small bucket with a long timeout and a large bucket with a short timeout. Each concurrent request to a service will use up one slot in each bucket and will use the highest available request timeout available.
+
+Under normal load, the long timeout will be used for all requests (if you've set the bucket limit high enough). However, as the number of concurrent requests increases (for example, when the external service starts responding more slowly), the request timeouts will be automatically reduced as requests start falling over to the next bucket with a lower timeout. If the number of concurrent requests exceeds the limit of the last bucket, then an exception will be raised without even making the request.
+
+This chart shows the basic logic:
+
+[![](https://mermaid.ink/img/pako:eNptk8tuwyAQRX8F0U0qOZWjKlJltZUSJ91102ZV2wuExwkKjxSwqijJvxcMVHnUq5m5Z4C5mAOmqgVc4I6rH7oh2qLVopbIfR_w3YOxoyoGzX2oz6p5T7dg61paJkD1tkBPJnMpZ4K5ZNoEcP4PODkHJ3kky3_I_GF6waZVl1orPao0YQbQG9GkJfvFXhLB6IoJ31qsNlpZy6Ed2HRuY_ccQruLtdpCcdfleUYVV3oIA7cKJ6gMWBRPE7d-J1uIZlSVT5JHTZMsMzslDXjPQpQ2jyAaj19RuQG6PZRK0l5rkH7qKJtT9MMTnj0-v0yPaHZbnbjy_Lb8mB9ReVkee-XVC8Po8Q4H_s_rqTmmsePVXeqTa7281P1dXRExGQY-8-3GyAFIbuEMC9CCsNb9kQfP1thuQECNCxe20JGe2xpnQRLMTzSjVmnjiY5wA1GTysKMs7UMrRw611fLk9uC9FZ97iXFhdU9ZLjftcTCgpG1JgIXYRUMLXPrvofXMTySRC4H5Q_cEfmlVGo8_QJywxkA?type=png)](https://mermaid-js.github.io/mermaid-live-editor/edit#pako:eNptk8tuwyAQRX8F0U0qOZWjKlJltZUSJ91102ZV2wuExwkKjxSwqijJvxcMVHnUq5m5Z4C5mAOmqgVc4I6rH7oh2qLVopbIfR_w3YOxoyoGzX2oz6p5T7dg61paJkD1tkBPJnMpZ4K5ZNoEcP4PODkHJ3kky3_I_GF6waZVl1orPao0YQbQG9GkJfvFXhLB6IoJ31qsNlpZy6Ed2HRuY_ccQruLtdpCcdfleUYVV3oIA7cKJ6gMWBRPE7d-J1uIZlSVT5JHTZMsMzslDXjPQpQ2jyAaj19RuQG6PZRK0l5rkH7qKJtT9MMTnj0-v0yPaHZbnbjy_Lb8mB9ReVkee-XVC8Po8Q4H_s_rqTmmsePVXeqTa7281P1dXRExGQY-8-3GyAFIbuEMC9CCsNb9kQfP1thuQECNCxe20JGe2xpnQRLMTzSjVmnjiY5wA1GTysKMs7UMrRw611fLk9uC9FZ97iXFhdU9ZLjftcTCgpG1JgIXYRUMLXPrvofXMTySRC4H5Q_cEfmlVGo8_QJywxkA)
+
+This setup makes for a system that degrades gracefully and predictably. When the external system starts having issues, you will still be sending requests to it, but they will start failing fast and at a certain limit they will fail immediately without even trying to make a request (which would have most likely only added to the issues on the external system). This can allow your system to remain functional (albeit in a degraded state) and to automatically recover when the external system recovers.
+
+The middleware requires a redis server which is used to count the number of concurrent requests being made to an endpoint across all processes.
+
+## Usage
+
+The middleware can be installed in your Faraday connection like this:
+
+```ruby
+require "faraday_dynamic_timeout"
+
+connection = Faraday.new do |faraday|
+  faraday.use(
+    :dynamic_timeout,
+    buckets: [
+      {timeout: 8, limit: 5},
+      {timeout: 1, limit: 10},
+      {timeout: 0.5, limit: 20}
+    ]
+  )
+end
+```
+
+In this example, the timeout will be set to 8 seconds if there are 5 or fewer requests being made to an endpoint, 1 second if there are 10 or fewer requests, and 0.5 seconds if there are 20 or fewer requests. If there are more than 20 requests being made to an endpoint, an exception will be raised.
+
+### Configuration Options
+
+- `:buckets` - An array of bucket configurations. Each bucket is a hash with two keys: `:timeout` and `:limit`. The `:timeout` value is the timeout to use for requests when it falls into that bucket. The `:limit` value is the maximum number of concurrent requests that can use that bucket. Requests will always try to use the bucket with the highest timeout, so order does not matter. If a bucket has a limit less than zero, it will be considered unlimited and requests will never fall over to the next bucket. This value can also be set as a `Proc` (or any object that responds to `call`) that will be evaluated at runtime when each request is made.
+
+- `:redis` - The redis connection to use. This should be a `Redis` object or a `Proc` that yields a `Redis` object. If not provided, a default `Redis` connection will be used (configured using environment variables). If the value is explicitly set to nil, then the middleware will pass through all requests without doing anything.
+
+- `:name` - An optional name for the resource. By default the hostname and port of the request URL will be used to identify the resource. Each resource will report a separate count of concurrent requests and processes. You can group multiple resources from different hosts together with the `:name` option.
+
+- `:callback` - An optional callback that will be called after each request. The callback can be a `Proc` or any object that responds to `call`. It will be called with a `FaradayDyamicTimeout::RequestInfo` argument. You can use this to log the number of concurrent requests or to report metrics to a monitoring system. This can be very useful for tuning the bucket settings.
+
+### Capacity Strategy
+
+You can use the `FaraadyDynamicTimeout::CapacityStrategy` class to build a bucket configuration based on the current capacity of your application rather than hard coding bucket limits. This can be useful if you have a system that can scale up and down based on load. It works by estimating the total number of threads available in your application and uses that value to calculate bucket limits based on a percentage provided by the `:capacity` option.
+
+```ruby
+capacity = FaradayDynamicTimeout::CapacityStrategy.new(
+  buckets: [
+    {timeout: 8, capacity: 0.05, limit: 3},
+    {timeout: 1, capacity: 0.05},
+    {timeout: 0.5, capacity: 0.10},
+    {timeout: 0.2, capacity: 1.0}
+  ],
+  threads_per_process: 4
+)
+```
+
+In this example, it will be assumed that each process has 4 threads, The first bucket will have a limit of 5% of the application threads but it will never be less than 3 (because of the hardcoded `:limit` option). The second bucket will have a limit of 5% of the application threads (rounded up). The third bucket will have a limit of 10% of the application threads. The forth bucket will have unlimited capacity (100%) and will serve all requests that exceed the limits of the previous buckets.
+
+The number of processes is estimated. Every time a process makes a request through the middleware, it will be remembered for 60 seconds. So if you have processes that have not made any requests through the middleware, they will not be counted. Processes that have been terminated will also be considered in the calculation for up to 60 seconds. You should still get a pretty good estimate of the number of processes that are currently running. The number will be less accurate if you application is scaling up and down very quickly (i.e. during a deployment) or when it is not very active. In the deployment case the estimate will be high so it should not cause any problems. In the low activity case you aren't really worried about load and the bucket sizes will never be lower than 1. In any case, it's a good idea to set a minimum limit on the first bucket.
+
+### Full Example
+
+For this example, we will configure the `opensearch` gem with this middleware along with metrics tracking to a statsd servers.
+
+```ruby
+# Set up a redis connection to coordinate counting concurrent requests.
+redis = Redis.new(url: ENV.fetch("REDIS_URL"))
+
+# Set up a statsd client to report metrics with the DataDog extensions.
+statsd = Statsd.new(ENV.fetch("STATSD_HOST"), ENV.fetch("STATSD_PORT"))
+
+metrics_callback = ->(request_info) do
+  batch = Statsd::Batch.new(statsd)
+  batch.gauge("opensearch.concurrent_requests", request_info.request_count)
+  batch.timing("opensearch.duration", (request_info.duration * 1000).round)
+  batch.increment("opensearch.throttled") if request_info.throttled?
+  batch.increment("opensearch.timed_out") if request_info.timed_out?
+  batch.flush
+end
+
+client = OpenSearch::Client.new(host: 'localhost', port: '9200') do |faraday|
+  faraday.request :dynamic_timeout,
+                  buckets: [
+                    {timeout: 8, max_requests: 5},
+                    {timeout: 1, max_requests: 10},
+                    {timeout: 0.5, max_requests: 20}
+                  ],
+                  name: "opensearch",
+                  redis: redis,
+                  filter: ->(env) { env.url.path.end_with?("/_search") },
+                  callback: metrics_callback
+end
+```
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem "faraday_dynamic_timeout"
+```
+
+And then execute:
+```bash
+$ bundle
+```
+
+Or install it yourself as:
+```bash
+$ gem install faraday_dynamic_timeout
+```
+
+## Contributing
+
+Open a pull request on GitHub.
+
+Please use the [standardrb](https://github.com/testdouble/standard) syntax and lint your code with `standardrb --fix` before submitting.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/VERSION

@@ -0,0 +1 @@
+1.0.0

data/faraday_dynamic_timeout.gemspec

@@ -0,0 +1,33 @@
+Gem::Specification.new do |spec|
+  spec.name = "faraday_dynamic_timeout"
+  spec.version = File.read(File.expand_path("VERSION", __dir__)).strip
+  spec.authors = ["Brian Durand"]
+  spec.email = ["bbdurand@gmail.com"]
+
+  spec.summary = "Faraday middleware to dynamically set a request timeout based on the number of concurrent requests and throttle the number of requests that can be made."
+  spec.homepage = "https://github.com/bdurand/faraday_dynamic_timeout"
+  spec.license = "MIT"
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  ignore_files = %w[
+    .
+    Gemfile
+    Gemfile.lock
+    Rakefile
+    bin/
+    spec/
+  ]
+  spec.files = Dir.chdir(File.expand_path("..", __FILE__)) do
+    `git ls-files -z`.split("\x0").reject { |f| ignore_files.any? { |path| f.start_with?(path) } }
+  end
+
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "faraday"
+  spec.add_dependency "restrainer", ">= 1.1.3"
+
+  spec.add_development_dependency "bundler"
+
+  spec.required_ruby_version = ">= 2.5"
+end

data/gemfiles/faraday_1.gemfile

@@ -0,0 +1,15 @@
+# This file was generated by Appraisal
+
+source "https://rubygems.org"
+
+gem "webmock"
+gem "appraisal"
+gem "dotenv"
+gem "rake"
+gem "rspec", "~> 3.10"
+gem "standard", "~> 1.0"
+gem "simplecov", "~> 0.21", require: false
+gem "yard"
+gem "faraday", "~> 1.0"
+
+gemspec path: "../"

data/gemfiles/faraday_2.gemfile

@@ -0,0 +1,15 @@
+# This file was generated by Appraisal
+
+source "https://rubygems.org"
+
+gem "webmock"
+gem "appraisal"
+gem "dotenv"
+gem "rake"
+gem "rspec", "~> 3.10"
+gem "standard", "~> 1.0"
+gem "simplecov", "~> 0.21", require: false
+gem "yard"
+gem "faraday", "~> 2.0"
+
+gemspec path: "../"

data/lib/faraday_dynamic_timeout/bucket.rb

@@ -0,0 +1,70 @@
+# frozen_string_literal: true
+
+module FaradayDynamicTimeout
+  # Internal class for storing bucket configuration.
+  # @api private
+  class Bucket
+    attr_reader :timeout, :limit
+
+    class << self
+      def from_hashes(hashes)
+        all_buckets = hashes.collect { |hash| new(timeout: fetch_indifferent_key(hash, :timeout), limit: fetch_indifferent_key(hash, :limit)) }
+        grouped_buckets = all_buckets.select(&:valid?).group_by(&:timeout).values
+        return [] if grouped_buckets.empty?
+
+        unique_buckets = grouped_buckets.collect do |buckets|
+          buckets.reduce do |merged, bucket|
+            merged.nil? ? bucket : merged.merge(bucket)
+          end
+        end
+
+        unique_buckets.sort_by(&:timeout)
+      end
+
+      private
+
+      def fetch_indifferent_key(hash, key)
+        hash[key] || hash[key.to_s]
+      end
+    end
+
+    # @param timeout [Float] The timeout in seconds.
+    # @param limit [Integer] The limit.
+    def initialize(timeout:, limit: 0)
+      @timeout = timeout.to_f.round(3)
+      @limit = limit.to_i
+    end
+
+    # Return true if the bucket has no limit. A bucket has no limit if the limit is negative.
+    # @return [Boolean] True if the bucket has no limit.
+    def no_limit?
+      limit < 0
+    end
+
+    # Return true if the bucket is valid. A bucket is valid if the timeout is positive and
+    # the limit is non-zero.
+    def valid?
+      timeout.positive? && limit != 0
+    end
+
+    def ==(other)
+      return false unless other.is_a?(self.class)
+
+      timeout == other.timeout && limit == other.limit
+    end
+
+    def merge(bucket)
+      combined_limit = if no_limit?
+        limit
+      elsif bucket.no_limit?
+        bucket.limit
+      else
+        limit + bucket.limit
+      end
+
+      combined_timeout = [timeout, bucket.timeout].max
+
+      self.class.new(timeout: combined_timeout, limit: combined_limit)
+    end
+  end
+end

data/lib/faraday_dynamic_timeout/capacity_strategy.rb

@@ -0,0 +1,79 @@
+# frozen_string_literal: true
+
+module FaradayDynamicTimeout
+  class CapacityStrategy
+    def initialize(buckets:, redis:, name: nil, threads_per_process: 1)
+      @config = buckets
+      @redis = redis
+      @name = (name || "default").to_s
+      @threads_per_process = threads_per_process
+    end
+
+    def call
+      process_count = count_processes
+      threads_count = total_threads(process_count)
+
+      buckets_config.collect do |bucket|
+        timeout = fetch_indifferent_key(bucket, :timeout)&.to_f
+        limit = fetch_indifferent_key(bucket, :limit)&.to_i
+        capacity = fetch_indifferent_key(bucket, :capacity)&.to_f
+        {timeout: timeout, limit: capacity_limit(capacity, limit, threads_count)}
+      end
+    end
+
+    private
+
+    def capacity_limit(capacity, limit, threads_count)
+      if capacity && (limit.nil? || limit >= 0)
+        if capacity < 0 || capacity >= 1.0
+          limit = -1
+        else
+          capacity_limit = (capacity * threads_count).ceil
+          limit = [limit, capacity_limit].compact.max
+        end
+      end
+
+      limit
+    end
+
+    def count_processes
+      process_counter = Counter.new(name: process_counter_name, redis: redis_client, ttl: 60)
+      process_counter.track!(process_id)
+      process_counter.value
+    end
+
+    def buckets_config
+      if @config.respond_to?(:call)
+        @config.call
+      else
+        @config
+      end
+    end
+
+    def redis_client
+      if @redis.is_a?(Proc)
+        @redis.call
+      else
+        @redis
+      end
+    end
+
+    def total_threads(process_count)
+      threads_per_process = (@threads_per_process.respond_to?(:call) ? @threads_per_process.call : @threads_per_process).to_i
+      threads_per_process = 1 if threads_per_process <= 0
+      [process_count, 1].max * threads_per_process
+    end
+
+    def process_id
+      "#{Socket.gethostname}:#{Process.pid}"
+    end
+
+    def process_counter_name
+      "#{self.class.name}:#{@name}.processes"
+    end
+
+    def fetch_indifferent_key(hash, key)
+      hash[key] || hash[key.to_s]
+    end
+  end
+end

data/lib/faraday_dynamic_timeout/counter.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+module FaradayDynamicTimeout
+  class Counter
+    def initialize(name:, redis:, ttl: 60.0)
+      @ttl = ttl.to_f
+      @ttl = 60.0 if @ttl <= 0.0
+      @redis = redis
+      @key = "FaradayDynamicTimeout:#{name}"
+    end
+
+    def execute
+      id = track!
+      begin
+        yield
+      ensure
+        release!(id)
+      end
+    end
+
+    def value
+      total_count, expired_count = @redis.multi do |transaction|
+        transaction.zcard(@key)
+        transaction.zremrangebyscore(@key, "-inf", Time.now.to_f - @ttl)
+      end
+
+      total_count - expired_count
+    end
+
+    def track!(id = nil)
+      id ||= SecureRandom.hex
+      @redis.multi do |transaction|
+        transaction.zadd(@key, Time.now.to_f, id)
+        transaction.pexpire(@key, (@ttl * 1000).round)
+      end
+      id
+    end
+
+    def release!(id)
+      @redis.zrem(@key, id)
+    end
+  end
+end

data/lib/faraday_dynamic_timeout/middleware.rb

@@ -0,0 +1,161 @@
+# frozen_string_literal: true
+
+module FaradayDynamicTimeout
+  class Middleware < Faraday::Middleware
+    def initialize(*)
+      super
+
+      @redis_client = option(:redis)
+      if @redis_client.nil? && !(options.include?(:redis) || options.include?("redis"))
+        @redis_client = Redis.new
+      end
+
+      @memoized_buckets = []
+      @mutex = Mutex.new
+    end
+
+    def call(env)
+      buckets = sorted_buckets
+      redis = redis_client
+      return app.call(env) if !enabled?(env) || buckets.empty? || redis.nil?
+
+      error = nil
+      bucket_timeout = nil
+      callback = option(:callback)
+      start_time = monotonic_time if callback
+
+      count_request(env.url, redis, buckets, callback) do |request_count|
+        execute_with_timeout(env.url, buckets, request_count, redis) do |timeout|
+          bucket_timeout = timeout
+          set_timeout(env.request, timeout) if timeout
+
+          # Resetting the start time to more accurately reflect the time spent in the request.
+          start_time = monotonic_time if callback
+          app.call(env)
+        end
+      rescue => e
+        error = e
+        raise
+      ensure
+        if callback
+          duration = monotonic_time - start_time
+          request_info = RequestInfo.new(env: env, duration: duration, timeout: bucket_timeout, request_count: request_count, error: error)
+          callback.call(request_info)
+        end
+      end
+    end
+
+    private
+
+    # Return the valid buckets sorted by timeout.
+    # @return [Array<Bucket>] The sorted buckets.
+    # @api private
+    def sorted_buckets
+      config = option(:buckets)
+      config = config.call if config.respond_to?(:call)
+      config = Array(config)
+      memoized_config, memoized_buckets = @memoized_buckets
+
+      if config == memoized_config
+        memoized_buckets
+      else
+        duplicated_config = @mutex.synchronize { config.collect(&:dup) }
+        buckets = Bucket.from_hashes(duplicated_config)
+        @memoized_buckets = [duplicated_config, buckets]
+        buckets
+      end
+    end
+
+    def enabled?(env)
+      filter = option(:filter)
+      if filter
+        filter.call(env)
+      else
+        true
+      end
+    end
+
+    def execute_with_timeout(uri, buckets, request_count, redis)
+      buckets = buckets.dup
+      total_requests = 0
+
+      while (bucket = buckets.pop)
+        if bucket.no_limit?
+          retval = yield(bucket.timeout)
+          break
+        else
+          restrainer = Restrainer.new(restrainer_name(uri, bucket.timeout), limit: bucket.limit, timeout: bucket.timeout, redis: redis)
+          begin
+            retval = restrainer.throttle { yield(bucket.timeout) }
+            break
+          rescue Restrainer::ThrottledError
+            total_requests += bucket.limit
+            if buckets.empty?
+              # Since request_count is a snapshot before the request was started it is subject to
+              # race conditions, so we'll make sure to report a higher number if we calculated one.
+              request_count = [request_count, total_requests + 1].max
+              raise ThrottledError.new("Request to #{base_url(uri)} aborted due to #{request_count} concurrent requests", request_count: request_count)
+            end
+          end
+        end
+      end
+
+      retval
+    end
+
+    def set_timeout(request, timeout)
+      request.timeout = timeout
+      request.open_timeout = nil
+      request.write_timeout = nil
+      request.read_timeout = nil
+    end
+
+    # Track how many requests are currently being executed only if a callback has been configured.
+    def count_request(uri, redis, buckets, callback)
+      if callback
+        ttl = buckets.last.timeout
+        ttl = 60 if ttl <= 0
+        request_counter = Counter.new(name: request_counter_name(uri), redis: redis, ttl: ttl)
+        request_counter.execute do
+          yield request_counter.value
+        end
+      else
+        yield 1
+      end
+    end
+
+    def request_counter_name(uri)
+      "#{redis_key_namespace(uri)}.requests"
+    end
+
+    def restrainer_name(uri, timeout)
+      "#{redis_key_namespace(uri)}.#{timeout}"
+    end
+
+    def redis_key_namespace(uri)
+      name = option(:name).to_s
+      name = base_url(uri) if name.empty?
+      "FaradayDynamicTimeout:#{name}"
+    end
+
+    def base_url(uri)
+      url = "#{uri.scheme}://#{uri.host.downcase}"
+      url = "#{url}:#{uri.port}" unless uri.port == uri.default_port
+      url
+    end
+
+    def redis_client
+      redis = option(:redis) || @redis_client
+      redis = redis.call if redis.is_a?(Proc)
+      redis
+    end
+
+    def monotonic_time
+      Process.clock_gettime(Process::CLOCK_MONOTONIC)
+    end
+
+    def option(key)
+      options[key] || options[key.to_s]
+    end
+  end
+end

data/lib/faraday_dynamic_timeout/request_info.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+module FaradayDynamicTimeout
+  class RequestInfo
+    attr_reader :env, :duration, :timeout, :error
+
+    def initialize(env:, duration:, timeout:, request_count:, error: nil)
+      @env = env
+      @duration = duration
+      @timeout = timeout
+      @request_count = request_count
+      @error = error
+    end
+
+    def http_method
+      env.method
+    end
+
+    def uri
+      env.url
+    end
+
+    def status
+      env.status
+    end
+
+    def request_count
+      throttled? ? error.request_count : @request_count
+    end
+
+    def throttled?
+      @error.is_a?(ThrottledError)
+    end
+
+    def timed_out?
+      @error.is_a?(Faraday::TimeoutError)
+    end
+
+    def error?
+      !!@error
+    end
+  end
+end

data/lib/faraday_dynamic_timeout/strategy.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module FaradayDynamicTimeout
+  class Strategy
+    def initialize(buckets:, options: {})
+      @config = buckets
+      @options = options
+    end
+  end
+end

data/lib/faraday_dynamic_timeout.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+require "faraday"
+require "restrainer"
+require "socket"
+
+require_relative "faraday_dynamic_timeout/bucket"
+require_relative "faraday_dynamic_timeout/capacity_strategy"
+require_relative "faraday_dynamic_timeout/counter"
+require_relative "faraday_dynamic_timeout/middleware"
+require_relative "faraday_dynamic_timeout/request_info"
+
+module FaradayDynamicTimeout
+  VERSION = File.read(File.expand_path("../VERSION", __dir__)).strip
+
+  # Error raised when a request is not executed due to too many concurrent requests.
+  class ThrottledError < Restrainer::ThrottledError
+    attr_reader :request_count
+
+    def initialize(message, request_count:)
+      super(message)
+      @request_count = request_count
+    end
+  end
+end
+
+Faraday::Middleware.register_middleware(dynamic_timeout: FaradayDynamicTimeout::Middleware)

metadata

@@ -0,0 +1,101 @@
+--- !ruby/object:Gem::Specification
+name: faraday_dynamic_timeout
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- Brian Durand
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2023-12-29 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: faraday
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: restrainer
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.1.3
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.1.3
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description:
+email:
+- bbdurand@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- Appraisals
+- CHANGELOG.md
+- MIT-LICENSE.txt
+- README.md
+- VERSION
+- faraday_dynamic_timeout.gemspec
+- gemfiles/faraday_1.gemfile
+- gemfiles/faraday_2.gemfile
+- lib/faraday_dynamic_timeout.rb
+- lib/faraday_dynamic_timeout/bucket.rb
+- lib/faraday_dynamic_timeout/capacity_strategy.rb
+- lib/faraday_dynamic_timeout/counter.rb
+- lib/faraday_dynamic_timeout/middleware.rb
+- lib/faraday_dynamic_timeout/request_info.rb
+- lib/faraday_dynamic_timeout/strategy.rb
+homepage: https://github.com/bdurand/faraday_dynamic_timeout
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '2.5'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.4.12
+signing_key:
+specification_version: 4
+summary: Faraday middleware to dynamically set a request timeout based on the number
+  of concurrent requests and throttle the number of requests that can be made.
+test_files: []