httpx 0.10.2 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2dee09976c993c9a7cb840410ce8758e68faaae6d13d1101a0cf207b59eebdd4
-  data.tar.gz: 444e4c0373f4c3faed53b1b2238e338aaac70af61b24657b2b79ce3f509bffde
+  metadata.gz: e5a33fd07fae25308cb791a086313c8dd8aa05cc7bc24504c92c17e20b692688
+  data.tar.gz: 8be24aad50c50bfea6667a94f7afade82dada7d2512177430ea5b04593f2e296
 SHA512:
-  metadata.gz: 4d113f172d81c88f155b0bc00c1a42c76fc01c6eed31df693c2ed5c43190e4887c6331492b33acb3104c16de49ec34023fd3c22eb199bfa363392af5fd8ce414
-  data.tar.gz: defb46daafc3fba4f13e765531b61892e5fedd96de9ffdd4ab0e10132ac4df3a5e99e94baf032578cef2919c29069ac3c080c25e56b93907e8451c573a245972
+  metadata.gz: 3b03f47c4ef852af01d9d4eb6895921e6601d33e8c0261b25e34309c663c481a77ce9c5161124efdb02518b1a7c89f3e3440b823348106d492e3d27daf66ab6e
+  data.tar.gz: e4b2961e9d0b6d11348cf81eae7dbb0e744878c47a51ef584429ab89f32eadc24f9f195ceeb68ecfcb553c19b2a333818cba0e867c0cb992bec66b99dcd1def7

data/README.md

@@ -1,7 +1,7 @@
 # HTTPX: A Ruby HTTP library for tomorrow... and beyond!
 
 [![Gem Version](https://badge.fury.io/rb/httpx.svg)](http://rubygems.org/gems/httpx)
-[![pipeline status](https://gitlab.com/honeyryderchuck/httpx/badges/master/pipeline.svg)](https://gitlab.com/honeyryderchuck/httpx/commits/master)
+[![pipeline status](https://gitlab.com/honeyryderchuck/httpx/badges/master/pipeline.svg)](https://gitlab.com/honeyryderchuck/httpx/pipelines?page=1&scope=all&ref=master)
 [![coverage report](https://gitlab.com/honeyryderchuck/httpx/badges/master/coverage.svg?job=coverage)](https://honeyryderchuck.gitlab.io/httpx/coverage/#_AllFiles)
 
 HTTPX is an HTTP client library for the Ruby programming language.
@@ -85,7 +85,15 @@ However if the server supports HTTP/1.1, it will try to use HTTP pipelining, fal
 
 ### Clean API
 
-`HTTPX` acknowledges the ease-of-use of the [http gem](https://github.com/httprb/http) API (itself inspired by python [requests](http://docs.python-requests.org/en/latest/) library). It therefore aims at reusing the same facade, extending it for the use cases which the http gem doesn't support.
+`httpx` builds all functions around the `HTTPX` module, so that all calls can compose of each other. Here are a few examples:
+
+```ruby
+response = HTTPX.get("https://www.google.com")
+response = HTTPX.post("https://www.nghttp2.org/httpbin/post", params: {name: "John", age: "22"})
+response = HTTPX.plugin(:basic_authentication)
+                .basic_authentication("user", "pass")
+                .get("https://www.google.com")
+```
 
 ### Lightweight
 
@@ -135,7 +143,7 @@ Doesn't work with ruby 2.4.0 for Windows (see [#36](https://gitlab.com/honeyryde
 
 * Discuss your contribution in an issue
 * Fork it
-* Make your changes, add some test
+* Make your changes, add some tests
 * Ensure all tests pass (`bundle exec rake test`)
 * Open a Merge Request (that's Pull Request in Github-ish)
 * Wait for feedback

data/doc/release_notes/0_11_0.md

@@ -0,0 +1,76 @@
+# 0.11.0
+
+## Features
+
+### Webmock Adapter
+
+`httpx` can now be integrated with `webmock`, a popular HTTP requests stubbing library.
+
+```ruby
+# minitest
+require "webmock/minitest"
+require "httpx/adapters/webmock"
+
+# in rspec
+require "webmock/rspec"
+require "httpx/adapters/webmock"
+
+# and now you're free for mocking
+WebMock.enable!
+stub_http_request(:get, "https://www.google.com").and_return(status: 200, body: "here's google")
+
+```
+
+Read more about it in the [webmock integration documentation](https://honeyryderchuck.gitlab.io/httpx/wiki/Webmock-Adapter).
+
+### Datadog Adapter
+
+`httpx` ships with integration for [ddtrace, datadog's official tracing client](https://github.com/DataDog/dd-trace-rb). You just need to initialize it the following way:
+
+```ruby
+require "ddtrace"
+require "httpx/adapters/datadog"
+
+Datadog.configure do |c|
+  c.use :httpx
+end
+```
+
+A trace will be emitted for every request, so this should be an interesting visualization if concurrent requests are sent.
+
+Customization options and traces are similar to what [the net-http adapter provides](https://docs.datadoghq.com/tracing/setup_overview/setup/ruby/#nethttp).
+
+Read more about it in the [datadog integration documentation](https://honeyryderchuck.gitlab.io/httpx/wiki/Datadog-Adapter).
+
+## Improvements
+
+### Own multipart request encoder
+
+`httpx` now ships with its own multipart formdata encoder, and does not rely on `http-form_data` anymore:
+
+```ruby
+HTTPX.plugin(:multipart).post(uri, form: {file: File.new("path/to/file")})
+```
+
+Read more about it in the [multipart plugin documentation](https://honeyryderchuck.gitlab.io/httpx/wiki/Multipart-Uploads), including also about why this was made.
+
+### Expect Plugin
+
+The `:expect` plugin now works reliably when the server does not support the `expect: 100-continue` header, i.e. it'll upload the body after a certain timeout. Building onn that, two behaviours are now implemented:
+
+* A cache of domains which did not respond to the `expect` header is now kept, so that subsequent requests can skip the timeout and immediately upload the payload.
+* If the "100 Continue" response arrives **after** the timeout expired and the body has been uploaded, the domain is removed from the cache, and subsequent requests will send the `expect` header.
+
+### SNI/Host options
+
+Some extension of the API was applied in order to support custom TLS negotiation parameters. You can now pass `:hostname` under the `:ssl` options, and this will be used for the SNI part of the TLS negotiation. This is useful in scenarios where a proxy certificate doesn't apply for the host one wants to send the request to:
+
+```ruby
+response = session.get(proxy_ip, headers: { "host" => upstream_hostname }, ssl: { hostname: sni_hostname }
+```
+
+## Bugfixes
+
+A default 5 second timeout is in-place when using the DNS `:system` resolver, as it was found out that. when using the `resolv` library, the DNS query will not be retried otherwise. You can change this setting py passing `resolver_options: { timeouts: ANOTHER_TIMEOUT}`. In the future, this may become another timeout option, however.
+
+

data/lib/httpx/adapters/datadog.rb

@@ -0,0 +1,205 @@
+# frozen_string_literal: true
+
+require "ddtrace/contrib/integration"
+require "ddtrace/contrib/rest_client/configuration/settings"
+require "ddtrace/contrib/rest_client/patcher"
+
+module Datadog
+  module Contrib
+    module HTTPX
+      # HTTPX Datadog Plugin
+      #
+      # Enables tracing for httpx requests. A span will be created for each individual requests,
+      # and it'll trace since the moment it is fed to the connection, until the moment the response is
+      # fed back to the session.
+      #
+      module Plugin
+        class RequestTracer
+          SPAN_REQUEST = "httpx.request"
+
+          def initialize(request)
+            @request = request
+          end
+
+          def call
+            return if skip_tracing?
+
+            @request.on(:response, &method(:finish))
+
+            verb = @request.verb.to_s.upcase
+            uri = @request.uri
+
+            @span = datadog_pin.tracer.trace(SPAN_REQUEST)
+            service_name = datadog_config[:split_by_domain] ? uri.host : datadog_pin.service_name
+
+            begin
+              @span.service = service_name
+              @span.span_type = Datadog::Ext::HTTP::TYPE_OUTBOUND
+              @span.resource = verb
+
+              Datadog::HTTPPropagator.inject!(@span.context, @request.headers) if datadog_pin.tracer.enabled && !skip_distributed_tracing?
+
+              # Add additional request specific tags to the span.
+
+              @span.set_tag(Datadog::Ext::HTTP::URL, @request.path)
+              @span.set_tag(Datadog::Ext::HTTP::METHOD, verb)
+
+              @span.set_tag(Datadog::Ext::NET::TARGET_HOST, uri.host)
+              @span.set_tag(Datadog::Ext::NET::TARGET_PORT, uri.port.to_s)
+
+              # Tag as an external peer service
+              @span.set_tag(Datadog::Ext::Integration::TAG_PEER_SERVICE, @span.service)
+
+              # Set analytics sample rate
+              if Contrib::Analytics.enabled?(datadog_config[:analytics_enabled])
+                Contrib::Analytics.set_sample_rate(@span, datadog_config[:analytics_sample_rate])
+              end
+            rescue StandardError => e
+              Datadog.logger.error("error preparing span for http request: #{e}")
+            end
+          rescue StandardError => e
+            Datadog.logger.debug("Failed to start span: #{e}")
+          end
+
+          def finish(response)
+            return unless @span
+
+            if response.respond_to?(:error)
+              @span.set_error(response.error)
+            else
+              @span.set_tag(Datadog::Ext::HTTP::STATUS_CODE, response.status.to_s)
+
+              @span.set_error(::HTTPX::HTTPError.new(response)) if response.status >= 400 && response.status <= 599
+            end
+
+            @span.finish
+          end
+
+          private
+
+          def skip_tracing?
+            return true if @request.headers.key?(Datadog::Ext::Transport::HTTP::HEADER_META_TRACER_VERSION)
+
+            return false unless @datadog_pin
+
+            span = @datadog_pin.tracer.active_span
+
+            return true if span && (span.name == SPAN_REQUEST)
+
+            false
+          end
+
+          def skip_distributed_tracing?
+            return !datadog_pin.config[:distributed_tracing] if datadog_pin.config && datadog_pin.config.key?(:distributed_tracing)
+
+            !Datadog.configuration[:httpx][:distributed_tracing]
+          end
+
+          def datadog_pin
+            @datadog_pin ||= begin
+              service = datadog_config[:service_name]
+              tracer = datadog_config[:tracer]
+
+              Datadog::Pin.new(
+                service,
+                app: "httpx",
+                app_type: Datadog::Ext::AppTypes::WEB,
+                tracer: -> { tracer }
+              )
+            end
+          end
+
+          def datadog_config
+            @datadog_config ||= Datadog.configuration[:httpx, @request.uri.host]
+          end
+        end
+
+        module ConnectionMethods
+          def send(request)
+            RequestTracer.new(request).call
+            super
+          end
+        end
+      end
+
+      module Configuration
+        # Default settings for httpx
+        #
+        class Settings < Datadog::Contrib::Configuration::Settings
+          option :service_name, default: "httpx"
+          option :distributed_tracing, default: true
+          option :split_by_domain, default: false
+
+          option :enabled do |o|
+            o.default { env_to_bool("DD_TRACE_HTTPX_ENABLED", true) }
+            o.lazy
+          end
+
+          option :analytics_enabled do |o|
+            o.default { env_to_bool(%w[DD_TRACE_HTTPX_ANALYTICS_ENABLED DD_HTTPX_ANALYTICS_ENABLED], false) }
+            o.lazy
+          end
+
+          option :analytics_sample_rate do |o|
+            o.default { env_to_float(%w[DD_TRACE_HTTPX_ANALYTICS_SAMPLE_RATE DD_HTTPX_ANALYTICS_SAMPLE_RATE], 1.0) }
+            o.lazy
+          end
+
+          option :error_handler, default: Datadog::Tracer::DEFAULT_ON_ERROR
+        end
+      end
+
+      # Patcher enables patching of 'httpx' with datadog components.
+      #
+      module Patcher
+        include Datadog::Contrib::Patcher
+
+        module_function
+
+        def target_version
+          Integration.version
+        end
+
+        # loads a session instannce with the datadog plugin, and replaces the
+        # base HTTPX::Session with the patched session class.
+        def patch
+          datadog_session = ::HTTPX.plugin(Plugin)
+
+          ::HTTPX.send(:remove_const, :Session)
+          ::HTTPX.send(:const_set, :Session, datadog_session.class)
+        end
+      end
+
+      # Datadog Integration for HTTPX.
+      #
+      class Integration
+        include Contrib::Integration
+
+        # MINIMUM_VERSION = Gem::Version.new('0.11.0')
+        MINIMUM_VERSION = Gem::Version.new("0.10.2")
+
+        register_as :httpx
+
+        def self.version
+          Gem.loaded_specs["httpx"] && Gem.loaded_specs["httpx"].version
+        end
+
+        def self.loaded?
+          defined?(::HTTPX::Request)
+        end
+
+        def self.compatible?
+          super && version >= MINIMUM_VERSION
+        end
+
+        def default_configuration
+          Configuration::Settings.new
+        end
+
+        def patcher
+          Patcher
+        end
+      end
+    end
+  end
+end

data/lib/httpx/adapters/faraday.rb

@@ -65,7 +65,6 @@ module Faraday
         plugin(:compression)
         plugin(:persistent)
 
-        # :nocov:
         module ReasonPlugin
           if RUBY_VERSION < "2.5"
             def self.load_dependencies(*)
@@ -88,7 +87,6 @@ module Faraday
             end
           end
         end
-        # :nocov:
         plugin(ReasonPlugin)
       end
 

data/lib/httpx/adapters/webmock.rb

@@ -0,0 +1,123 @@
+# frozen_string_literal: true
+
+module WebMock
+  module HttpLibAdapters
+    if RUBY_VERSION < "2.5"
+      require "webrick/httpstatus"
+      HTTP_REASONS = WEBrick::HTTPStatus::StatusMessage
+    else
+      require "net/http/status"
+      HTTP_REASONS = Net::HTTP::STATUS_CODES
+    end
+
+    #
+    # HTTPX plugin for webmock.
+    #
+    # Requests are "hijacked" at the session, before they're distributed to a connection.
+    #
+    module Plugin
+      module InstanceMethods
+        private
+
+        def send_requests(*requests, options)
+          request_signatures = requests.map do |request|
+            request_signature = _build_webmock_request_signature(request)
+            WebMock::RequestRegistry.instance.requested_signatures.put(request_signature)
+            request_signature
+          end
+
+          responses = request_signatures.map do |request_signature|
+            WebMock::StubRegistry.instance.response_for_request(request_signature)
+          end
+
+          real_requests = {}
+
+          requests.each_with_index.each_with_object([request_signatures, responses]) do |(request, idx), (sig_reqs, mock_responses)|
+            if (webmock_response = mock_responses[idx])
+              mock_responses[idx] = _build_from_webmock_response(request, webmock_response)
+              WebMock::CallbackRegistry.invoke_callbacks({ lib: :httpx }, sig_reqs[idx], webmock_response)
+              log { "mocking #{request.uri} with #{mock_responses[idx].inspect}" }
+            elsif WebMock.net_connect_allowed?(sig_reqs[idx].uri)
+              log { "performing #{request.uri}" }
+              real_requests[request] = idx
+            else
+              raise WebMock::NetConnectNotAllowedError, sig_reqs[idx]
+            end
+          end
+
+          unless real_requests.empty?
+            reqs = real_requests.keys
+            reqs.zip(super(*reqs, options)).each do |req, res|
+              idx = real_requests[req]
+
+              if WebMock::CallbackRegistry.any_callbacks?
+                webmock_response = _build_webmock_response(req, res)
+                WebMock::CallbackRegistry.invoke_callbacks(
+                  { lib: :httpx, real_request: true }, request_signatures[idx],
+                  webmock_response
+                )
+              end
+
+              responses[idx] = res
+            end
+          end
+
+          responses
+        end
+
+        def _build_webmock_request_signature(request)
+          uri = WebMock::Util::URI.heuristic_parse(request.uri)
+          uri.path = uri.normalized_path.gsub("[^:]//", "/")
+
+          WebMock::RequestSignature.new(
+            request.verb,
+            uri.to_s,
+            body: request.body.each.to_a.join,
+            headers: request.headers.to_h
+          )
+        end
+
+        def _build_webmock_response(_request, response)
+          webmock_response = WebMock::Response.new
+          webmock_response.status = [response.status, HTTP_REASONS[response.status]]
+          webmock_response.body = response.body.to_s
+          webmock_response.headers = response.headers.to_h
+          webmock_response
+        end
+
+        def _build_from_webmock_response(request, webmock_response)
+          return ErrorResponse.new(request, webmock_response.exception, request.options) if webmock_response.exception
+
+          response = request.options.response_class.new(request,
+                                                        webmock_response.status[0],
+                                                        "2.0",
+                                                        webmock_response.headers)
+          response << webmock_response.body.dup
+          response
+        end
+      end
+    end
+
+    class HttpxAdapter < HttpLibAdapter
+      adapter_for :httpx
+
+      class << self
+        def enable!
+          @original_session = ::HTTPX::Session
+
+          webmock_session = ::HTTPX.plugin(Plugin)
+
+          ::HTTPX.send(:remove_const, :Session)
+          ::HTTPX.send(:const_set, :Session, webmock_session.class)
+        end
+
+        def disable!
+          return unless @original_session
+
+          HTTPX.send(:remove_const, :Session)
+          HTTPX.send(:const_set, :Session, @original_session)
+        end
+      end
+    end
+  end
+end