faraday-highly_available_retries 0.1.0.pre.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 0d99996c87cd7bfc2792430d4a7be1dc810889e2d59c1460da5c56bf18a32990
+  data.tar.gz: 4eae695335d5848847bc4e9a34295575dd9ca306c5380b79c77cba059bdc643a
+SHA512:
+  metadata.gz: 6262767b39d59b4a7a4e1b1ecf31a8160ca6b0c3bda88d74e6ec49379ee8508d6e7e7235e581ba9c72c6429074a786def5c4369a80f9566e268c8b05789aa020
+  data.tar.gz: b2a1fc4a67ea0c9616194183e164a458dac26e93d186d2ee15f22f076820641404080c4d539cebd815fee2daae907baa5b410c7b6a9f51700fbda984a2a6c4d9

data/CHANGELOG.md

@@ -0,0 +1,9 @@
+# Changelog for `faraday-highly_available_retries`
+
+Inspired by [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
+
+**Note:** this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - Unreleased
+
+*   Initial release.

data/LICENSE.md

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2023 James Ebentier
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,110 @@
+# Faraday Highly Available Retries
+
+[![GitHub Workflow Status](https://img.shields.io/github/workflow/status/invoca/faraday-highly_available_retries/ci)](https://github.com/invoca/faraday-highly_available_retries/actions?query=branch%3Amain)
+
+An extension for the Faraday::Retry middleware allowing retries to failover across multiple resolved endpoints.
+
+## Why should I use this gem?
+
+At the time a request is made, the list of hosts pulled from the connection and request is resolved to an array of IP addresses.
+This list of IP addresses is then shuffled, and if a request fails to connect to the first IP address, it will try the next one.
+And so on and so forth, until the retries are exhausted. If all of the IPs fail, then we cycle back to the first one and try again.
+
+The reason this is impactful and should be used in conjunction with the retry middleware is that the retry middleware will
+leave DNS resolution to the OS, which has the potential of caching results and not try resolving to a different IP address.
+This means that if a DNS entry resolves to three IPs, `[A, B, C]`, the retry middleware may try all three retries against
+`A`, where as this middleware guarantees that it will try `A`, `B`, and `C`.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'faraday-highly_available_retries'
+```
+
+And then execute:
+
+```shell
+bundle install
+```
+
+Or install it yourself as:
+
+```shell
+gem install faraday-highly_available_retries
+```
+
+## Usage
+
+This extension can be used on its own but is ultimately meant to be used in tandem with the [Faraday::Retry](https://github.com/lostisland/faraday-retry) middleware.
+You should make sure that this middleware is added in the request stack ***after*** the retry middleware to ensure
+it can alter the environment on retries.
+
+When you configure the connection, you can do it a few different ways for the failover to work.
+
+### Setting the URL in the request
+
+This simplest way is to make a generic Faraday connection, and pass the full URI in the request.
+When this is done, the failover middleware will parse the URI and use the host and port to determine
+the failover endpoints using DNS resolution.
+
+```ruby
+require 'faraday/retry/failover'
+
+conn = Faraday.new do |f|
+  f.request :retry, { max: 2 } # This makes sure we retry 2, resulting in 3 attempts total before failing finally
+  f.request :highly_available_retries
+end
+
+conn.get('https://api.invoca.net/api/2020-10-01/transaction/33.json')
+```
+
+### Setting the base URL in the connection
+
+This is the same as the previous example, but the base URL is set in the connection. This is useful
+when the base URL is always the same, and you have a DNS entry that will resolve to multiple IPs.
+
+```ruby
+require 'faraday/retry/failover'
+
+conn = Faraday.new('https://api.invoca.net') do |f|
+  f.request :retry, { max: 2 }
+  f.request :highly_available_retries
+end
+
+conn.get('/api/2020-10-01/transaction/33.json')
+```
+
+### Specifying multiple hosts
+
+This is the most useful when you already have multiple IPs or separate hostnames that you want to
+use for failover. The hosts list provided can contain hostnames and IPs both with and without ports.
+
+```ruby
+conn = Faraday.new do |f|
+  f.request :retry, { max: 2 }
+  f.request :highly_available_retries, { hosts: ['api.invoca.net', 'api.invoca.com'] }
+end
+
+conn.get('/api/2020-10-01/transaction/33.json')
+```
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies.
+
+Then, run `bin/test` to run the tests.
+
+To install this gem onto your local machine, run `rake build`.
+
+To release a new version, make a commit with a message such as "Bumped to 0.0.2" and then run `rake release`.
+See how it works [here](https://bundler.io/guides/creating_gem.html#releasing-the-gem).
+
+## Contributing
+
+Bug reports and pull requests are welcome on [GitHub](https://github.com/invoca/faraday-highly_available_retries).
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/lib/faraday/highly_available_retries/endpoint.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+require 'ipaddr'
+
+require_relative 'ip_validator'
+
+module Faraday
+  module HighlyAvailableRetries
+    class Endpoint
+      class DNSResolutionError < StandardError; end
+
+      include Comparable
+
+      class << self
+        def from_host_and_port(host, port)
+          host_ips = Socket.getaddrinfo(host, nil, Socket::AF_INET, Socket::SOCK_STREAM) or raise DNSResolutionError, "getaddrinfo(#{host}) failed"
+          host_ips.any? or raise DNSResolutionError, "No IP addrs returned from #{host}"
+          host_ips.map { |host_ip| new(host_ip[3], port, hostname: host) }
+        end
+      end
+
+      attr_reader :ip_addr, :port, :request_host
+
+      def initialize(ip_addr, port, hostname: nil)
+        IpValidator.ip_addr?(ip_addr) or raise ArgumentError, "ip_addr must be a valid IP address but received #{ip_addr.inspect}"
+
+        @ip_addr      = ip_addr
+        @port         = port
+        @request_host = if hostname && !IpValidator.ip_addr?(hostname)
+                          "#{hostname}:#{port}"
+                        end
+      end
+
+      def <=>(other)
+        ip_addr <=> other.ip_addr &&
+          port <=> other.port &&
+          request_host <=> other.request_host
+      end
+    end
+  end
+end

data/lib/faraday/highly_available_retries/ip_validator.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+require 'ipaddr'
+
+module Faraday
+  module HighlyAvailableRetries
+    class IpValidator
+      class << self
+        def ip_addr?(addr)
+          IPAddr.new(addr)
+          true
+        rescue
+          false
+        end
+      end
+    end
+  end
+end

data/lib/faraday/highly_available_retries/middleware.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+require 'ipaddr'
+
+require_relative 'endpoint'
+require_relative 'ip_validator'
+require_relative 'options'
+
+module Faraday
+  module HighlyAvailableRetries
+    # This class provides the main implementation for your middleware.
+    # Your middleware can implement any of the following methods:
+    # * on_request - called when the request is being prepared
+    # * on_complete - called when the response is being processed
+    #
+    # Optionally, you can also override the following methods from Faraday::Middleware
+    # * initialize(app, options = {}) - the initializer method
+    # * call(env) - the main middleware invocation method.
+    #   This already calls on_request and on_complete, so you normally don't need to override it.
+    #   You may need to in case you need to "wrap" the request or need more control
+    #   (see "retry" middleware: https://github.com/lostisland/faraday/blob/main/lib/faraday/request/retry.rb#L142).
+    #   IMPORTANT: Remember to call `@app.call(env)` or `super` to not interrupt the middleware chain!
+    class Middleware < Faraday::Middleware
+      FAILOVER_ENDPOINTS_ENV_KEY     = :failover_endpoints
+      FAILOVER_COUNTER_ENV_KEY       = :failover_counter
+      FAILOVER_ORIGINAL_HOST_ENV_KEY = :failover_original_hostname
+      FAILOVER_ORIGINAL_PORT_ENV_KEY = :failover_original_port
+
+      def initialize(app, options = nil)
+        super(app)
+        @options = Faraday::HighlyAvailableRetries::Options.from(options)
+      end
+
+      # This method will be called when the request is being prepared.
+      # You can alter it as you like, accessing things like request_body, request_headers, and more.
+      # Refer to Faraday::Env for a list of accessible fields:
+      # https://github.com/lostisland/faraday/blob/main/lib/faraday/options/env.rb
+      #
+      # @param env [Faraday::Env] the environment of the request being processed
+      def on_request(env)
+        # If we have not already resolved the failover endpoints, do so now
+        setup_resolution_env(env) unless env[FAILOVER_ENDPOINTS_ENV_KEY]&.any?
+
+        # If there is already a response associated with the request, it means that the request failed and we're retrying it.
+        # Increment the failover counter so we push on to the next endpoint
+        env[FAILOVER_COUNTER_ENV_KEY] += 1 unless env[:response].nil?
+
+        # Assign the next Failover endpoint to the request
+        setup_next_failover_endpoint(env)
+      end
+
+      private
+
+      def setup_next_failover_endpoint(env)
+        next_failover_endpoint = env[FAILOVER_ENDPOINTS_ENV_KEY][env[FAILOVER_COUNTER_ENV_KEY] % env[FAILOVER_ENDPOINTS_ENV_KEY].size]
+
+        if next_failover_endpoint.request_host
+          env[:request_headers] ||= {}
+          env[:request_headers]['Host'] = next_failover_endpoint.request_host
+        end
+
+        env[:url].hostname = next_failover_endpoint.ip_addr
+      end
+
+      def setup_resolution_env(env)
+        env[FAILOVER_ORIGINAL_HOST_ENV_KEY] = env[:url].hostname
+        env[FAILOVER_ORIGINAL_PORT_ENV_KEY] = env[:url].port
+        env[FAILOVER_COUNTER_ENV_KEY]       = 0
+        env[FAILOVER_ENDPOINTS_ENV_KEY]     = resolve_endpoints(env)
+      end
+
+      def resolve_endpoints(env)
+        host_list = if env[FAILOVER_ORIGINAL_HOST_ENV_KEY]
+                      [[env[FAILOVER_ORIGINAL_HOST_ENV_KEY], env[FAILOVER_ORIGINAL_PORT_ENV_KEY]]] + @options.hosts(refresh: true)
+                    else
+                      @options.hosts
+                    end
+
+        host_list.map { |host, port| Endpoint.from_host_and_port(host, port) }.flatten
+      end
+    end
+  end
+end

data/lib/faraday/highly_available_retries/options.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module Faraday
+  module HighlyAvailableRetries
+    # Options contains the configurable parameters for the middleware.
+    class Options < Faraday::Options.new(:hosts, :default_port)
+      DEFAULT_PORT = 80
+
+      def hosts(refresh: false)
+        if refresh
+          @hosts = nil
+        end
+
+        @hosts ||= load_host_list
+      end
+
+      def default_port
+        self[:default_port] ||= DEFAULT_PORT
+      end
+
+      private
+
+      def load_host_list
+        hosts = self[:hosts] ||= []
+        host_list = case hosts
+                    when Proc
+                      hosts.call
+                    when Array
+                      hosts
+                    else
+                      [hosts]
+                    end
+        host_list.map { |host| host_to_hostname_and_port(host) }
+      end
+
+      def host_to_hostname_and_port(host)
+        hostname, port = host.split(':', 2)
+        [hostname, port ? port.to_i : default_port]
+      end
+    end
+  end
+end

data/lib/faraday/highly_available_retries/version.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+module Faraday
+  module HighlyAvailableRetries
+    VERSION = '0.1.0.pre.1'
+  end
+end

data/lib/faraday/highly_available_retries.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+require_relative 'highly_available_retries/middleware'
+require_relative 'highly_available_retries/version'
+
+module Faraday
+  # This will be your middleware main module, though the actual middleware implementation will go
+  # into Faraday::HighlyAvailableRetries::Middleware for the correct namespacing.
+  module HighlyAvailableRetries
+    # Faraday allows you to register your middleware for easier configuration.
+    # This step is totally optional, but it basically allows users to use a
+    # custom symbol (in this case, `:highly_available_retries`), to use your middleware in their connections.
+    # After calling this line, the following are both valid ways to set the middleware in a connection:
+    # * conn.use Faraday::HighlyAvailableRetries::Middleware
+    # * conn.use :highly_available_retries
+    # Without this line, only the former method is valid.
+    # Faraday::Middleware.register_middleware(highly_available_retries: Faraday::HighlyAvailableRetries::Middleware)
+
+    # Alternatively, you can register your middleware under Faraday::Request or Faraday::Response.
+    # This will allow to load your middleware using the `request` or `response` methods respectively.
+    #
+    # Load middleware with conn.request :failover
+    Faraday::Request.register_middleware(highly_available_retries: Faraday::HighlyAvailableRetries::Middleware)
+    #
+    # Load middleware with conn.response :highly_available_retries
+    # Faraday::Response.register_middleware(highly_available_retries: Faraday::HighlyAvailableRetries::Middleware)
+  end
+end

metadata

@@ -0,0 +1,80 @@
+--- !ruby/object:Gem::Specification
+name: faraday-highly_available_retries
+version: !ruby/object:Gem::Version
+  version: 0.1.0.pre.1
+platform: ruby
+authors:
+- Invoca Development
+- James Ebentier
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2023-08-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: faraday
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+description: An extension for the Faraday::Retry middleware allowing retries to failover
+  across multiple resolved endpoints.
+email:
+- development@invoca.com
+- jebentier@invoca.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.md
+- README.md
+- lib/faraday/highly_available_retries.rb
+- lib/faraday/highly_available_retries/endpoint.rb
+- lib/faraday/highly_available_retries/ip_validator.rb
+- lib/faraday/highly_available_retries/middleware.rb
+- lib/faraday/highly_available_retries/options.rb
+- lib/faraday/highly_available_retries/version.rb
+homepage: https://github.com/invoca/faraday-highly_available_retries
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org
+  bug_tracker_uri: https://github.com/invoca/faraday-highly_available_retries/issues
+  changelog_uri: https://github.com/invoca/faraday-highly_available_retries/blob/v0.1.0.pre.1/CHANGELOG.md
+  documentation_uri: http://www.rubydoc.info/gems/faraday-highly_available_retries/0.1.0.pre.1
+  homepage_uri: https://github.com/invoca/faraday-highly_available_retries
+  source_code_uri: https://github.com/invoca/faraday-highly_available_retries
+  wiki_uri: https://github.com/invoca/faraday-highly_available_retries/wiki
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '2.7'
+  - - "<"
+    - !ruby/object:Gem::Version
+      version: '4'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">"
+    - !ruby/object:Gem::Version
+      version: 1.3.1
+requirements: []
+rubygems_version: 3.4.17
+signing_key:
+specification_version: 4
+summary: An extension for the Faraday::Retry middleware allowing retries to failover
+  across multiple resolved endpoints.
+test_files: []