httpx 0.10.1 → 0.11.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ed5ebccf22cf5c4719a81db15ee29a7d05e08fa43bd83b14c3b2682c083615be
-  data.tar.gz: 2befada7d0c17d0c20d0afce44fc0a9d3192ffb2bb311322ee171107d285d050
+  metadata.gz: e1612c9f4696a7a6ae0508ea66f864654e28470e742b91b57cd14ab5b43adad2
+  data.tar.gz: 791569d7282f7cb3e451d245e332871c170e4571ca5e10ead757f93afffe3d78
 SHA512:
-  metadata.gz: 9abffdd09d11f8c22c51f38064525d6d5250fe1e72a2e369f79c98e7d02ee6fe7a7442ac76758bf60cbc87eabe51a3faa30a675f4a30cb83a01e09aa98334d0a
-  data.tar.gz: 9c0706ac930dc0e082845795784a5007d0cd9bb25f4a81435be6d8e081ba29528a0df05fc03713dc07615329944ca586e4df8386766c6b5ff440c96d90dc472e
+  metadata.gz: 96e8bc5d07f59b21ccdbc54bb11bce9b1deb53eb128c4d6c2652a0dfca6c925cbe084e82f352857004420cc5753fbacc3a68e2bc4d885937a2afbfa5c6138836
+  data.tar.gz: b74cbac2346d110a97e6ab9a13f2a62db7ed23ce1454df510f9c8cb6ea58314edff2724b9cc269cdf8d5fdf00e9ce6c5c28eff864605f6ce01a9e2f4b9078911

data/README.md

@@ -1,7 +1,7 @@
 # HTTPX: A Ruby HTTP library for tomorrow... and beyond!
 
 [![Gem Version](https://badge.fury.io/rb/httpx.svg)](http://rubygems.org/gems/httpx)
-[![pipeline status](https://gitlab.com/honeyryderchuck/httpx/badges/master/pipeline.svg)](https://gitlab.com/honeyryderchuck/httpx/commits/master)
+[![pipeline status](https://gitlab.com/honeyryderchuck/httpx/badges/master/pipeline.svg)](https://gitlab.com/honeyryderchuck/httpx/pipelines?page=1&scope=all&ref=master)
 [![coverage report](https://gitlab.com/honeyryderchuck/httpx/badges/master/coverage.svg?job=coverage)](https://honeyryderchuck.gitlab.io/httpx/coverage/#_AllFiles)
 
 HTTPX is an HTTP client library for the Ruby programming language.
@@ -85,7 +85,15 @@ However if the server supports HTTP/1.1, it will try to use HTTP pipelining, fal
 
 ### Clean API
 
-`HTTPX` acknowledges the ease-of-use of the [http gem](https://github.com/httprb/http) API (itself inspired by python [requests](http://docs.python-requests.org/en/latest/) library). It therefore aims at reusing the same facade, extending it for the use cases which the http gem doesn't support.
+`httpx` builds all functions around the `HTTPX` module, so that all calls can compose of each other. Here are a few examples:
+
+```ruby
+response = HTTPX.get("https://www.google.com")
+response = HTTPX.post("https://www.nghttp2.org/httpbin/post", params: {name: "John", age: "22"})
+response = HTTPX.plugin(:basic_authentication)
+                .basic_authentication("user", "pass")
+                .get("https://www.google.com")
+```
 
 ### Lightweight
 
@@ -135,7 +143,7 @@ Doesn't work with ruby 2.4.0 for Windows (see [#36](https://gitlab.com/honeyryde
 
 * Discuss your contribution in an issue
 * Fork it
-* Make your changes, add some test
+* Make your changes, add some tests
 * Ensure all tests pass (`bundle exec rake test`)
 * Open a Merge Request (that's Pull Request in Github-ish)
 * Wait for feedback

data/doc/release_notes/0_10_1.md

@@ -29,9 +29,7 @@ From now on, both headers and the responnse payload will also appear, so expecte
 * HTTP/2 421 responses will now correctly migrate the connection and pendign requests to HTTP/1.1 (a hanging loop was being caused);
 * HTTP/2 connection failed with a GOAWAY settings timeout will now return error responses (instead of hanging indefinitely);
 * Non-IP proxy name-resolving errors will now move on to the next available proxy in the list (instead of hanging indefinitely);
-* Non-IP DNS resolve errors for `native` and `https` variannts will now return the appropriate error response (instead of hanging indefinitely);
-* 
-
+* Non-IP DNS resolve errors for `native` and `https` variants will now return the appropriate error response (instead of hanging indefinitely);
 
 ## Chore
 

data/doc/release_notes/0_10_2.md

@@ -0,0 +1,5 @@
+# 0.10.2
+
+## Bugfixes
+
+Support for nested params in multipart forms introduced a bug where top-level params weren't being spread out correctly in the request body.

data/doc/release_notes/0_11_0.md

@@ -0,0 +1,76 @@
+# 0.11.0
+
+## Features
+
+### Webmock Adapter
+
+`httpx` can now be integrated with `webmock`, a popular HTTP requests stubbing library.
+
+```ruby
+# minitest
+require "webmock/minitest"
+require "httpx/adapters/webmock"
+
+# in rspec
+require "webmock/rspec"
+require "httpx/adapters/webmock"
+
+# and now you're free for mocking
+WebMock.enable!
+stub_http_request(:get, "https://www.google.com").and_return(status: 200, body: "here's google")
+
+```
+
+Read more about it in the [webmock integration documentation](https://honeyryderchuck.gitlab.io/httpx/wiki/Webmock-Adapter).
+
+### Datadog Adapter
+
+`httpx` ships with integration for [ddtrace, datadog's official tracing client](https://github.com/DataDog/dd-trace-rb). You just need to initialize it the following way:
+
+```ruby
+require "ddtrace"
+require "httpx/adapters/datadog"
+
+Datadog.configure do |c|
+  c.use :httpx
+end
+```
+
+A trace will be emitted for every request, so this should be an interesting visualization if concurrent requests are sent.
+
+Customization options and traces are similar to what [the net-http adapter provides](https://docs.datadoghq.com/tracing/setup_overview/setup/ruby/#nethttp).
+
+Read more about it in the [datadog integration documentation](https://honeyryderchuck.gitlab.io/httpx/wiki/Datadog-Adapter).
+
+## Improvements
+
+### Own multipart request encoder
+
+`httpx` now ships with its own multipart formdata encoder, and does not rely on `http-form_data` anymore:
+
+```ruby
+HTTPX.plugin(:multipart).post(uri, form: {file: File.new("path/to/file")})
+```
+
+Read more about it in the [multipart plugin documentation](https://honeyryderchuck.gitlab.io/httpx/wiki/Multipart-Uploads), including also about why this was made.
+
+### Expect Plugin
+
+The `:expect` plugin now works reliably when the server does not support the `expect: 100-continue` header, i.e. it'll upload the body after a certain timeout. Building onn that, two behaviours are now implemented:
+
+* A cache of domains which did not respond to the `expect` header is now kept, so that subsequent requests can skip the timeout and immediately upload the payload.
+* If the "100 Continue" response arrives **after** the timeout expired and the body has been uploaded, the domain is removed from the cache, and subsequent requests will send the `expect` header.
+
+### SNI/Host options
+
+Some extension of the API was applied in order to support custom TLS negotiation parameters. You can now pass `:hostname` under the `:ssl` options, and this will be used for the SNI part of the TLS negotiation. This is useful in scenarios where a proxy certificate doesn't apply for the host one wants to send the request to:
+
+```ruby
+response = session.get(proxy_ip, headers: { "host" => upstream_hostname }, ssl: { hostname: sni_hostname }
+```
+
+## Bugfixes
+
+A default 5 second timeout is in-place when using the DNS `:system` resolver, as it was found out that. when using the `resolv` library, the DNS query will not be retried otherwise. You can change this setting py passing `resolver_options: { timeouts: ANOTHER_TIMEOUT}`. In the future, this may become another timeout option, however.
+
+

data/doc/release_notes/0_11_1.md

@@ -0,0 +1 @@
+0_11_1.md

data/doc/release_notes/0_11_2.md

@@ -0,0 +1,5 @@
+# 0.11.2
+
+## Bugfixes
+
+The `:cookies` plugin wasn't able to parse `Expires` values, as it was using `Time.httpdate` to parse timestamps, which is RFC 2616-compliant, whereas cookies datetime values need to be RFC 6265-compliant.

data/doc/release_notes/0_11_3.md

@@ -0,0 +1,5 @@
+# 0.11.3
+
+## Bugfixes
+
+`EOFError` was being thrown when HTTP/1.1 non-chunked responses which don't contain a "content-length" header. The RFC mandates that the socket should be consumed until the server closes it, so a patch was implemented, to return the available response in such cases.

data/lib/httpx/adapters/datadog.rb

@@ -0,0 +1,205 @@
+# frozen_string_literal: true
+
+require "ddtrace/contrib/integration"
+require "ddtrace/contrib/rest_client/configuration/settings"
+require "ddtrace/contrib/rest_client/patcher"
+
+module Datadog
+  module Contrib
+    module HTTPX
+      # HTTPX Datadog Plugin
+      #
+      # Enables tracing for httpx requests. A span will be created for each individual requests,
+      # and it'll trace since the moment it is fed to the connection, until the moment the response is
+      # fed back to the session.
+      #
+      module Plugin
+        class RequestTracer
+          SPAN_REQUEST = "httpx.request"
+
+          def initialize(request)
+            @request = request
+          end
+
+          def call
+            return if skip_tracing?
+
+            @request.on(:response, &method(:finish))
+
+            verb = @request.verb.to_s.upcase
+            uri = @request.uri
+
+            @span = datadog_pin.tracer.trace(SPAN_REQUEST)
+            service_name = datadog_config[:split_by_domain] ? uri.host : datadog_pin.service_name
+
+            begin
+              @span.service = service_name
+              @span.span_type = Datadog::Ext::HTTP::TYPE_OUTBOUND
+              @span.resource = verb
+
+              Datadog::HTTPPropagator.inject!(@span.context, @request.headers) if datadog_pin.tracer.enabled && !skip_distributed_tracing?
+
+              # Add additional request specific tags to the span.
+
+              @span.set_tag(Datadog::Ext::HTTP::URL, @request.path)
+              @span.set_tag(Datadog::Ext::HTTP::METHOD, verb)
+
+              @span.set_tag(Datadog::Ext::NET::TARGET_HOST, uri.host)
+              @span.set_tag(Datadog::Ext::NET::TARGET_PORT, uri.port.to_s)
+
+              # Tag as an external peer service
+              @span.set_tag(Datadog::Ext::Integration::TAG_PEER_SERVICE, @span.service)
+
+              # Set analytics sample rate
+              if Contrib::Analytics.enabled?(datadog_config[:analytics_enabled])
+                Contrib::Analytics.set_sample_rate(@span, datadog_config[:analytics_sample_rate])
+              end
+            rescue StandardError => e
+              Datadog.logger.error("error preparing span for http request: #{e}")
+            end
+          rescue StandardError => e
+            Datadog.logger.debug("Failed to start span: #{e}")
+          end
+
+          def finish(response)
+            return unless @span
+
+            if response.respond_to?(:error)
+              @span.set_error(response.error)
+            else
+              @span.set_tag(Datadog::Ext::HTTP::STATUS_CODE, response.status.to_s)
+
+              @span.set_error(::HTTPX::HTTPError.new(response)) if response.status >= 400 && response.status <= 599
+            end
+
+            @span.finish
+          end
+
+          private
+
+          def skip_tracing?
+            return true if @request.headers.key?(Datadog::Ext::Transport::HTTP::HEADER_META_TRACER_VERSION)
+
+            return false unless @datadog_pin
+
+            span = @datadog_pin.tracer.active_span
+
+            return true if span && (span.name == SPAN_REQUEST)
+
+            false
+          end
+
+          def skip_distributed_tracing?
+            return !datadog_pin.config[:distributed_tracing] if datadog_pin.config && datadog_pin.config.key?(:distributed_tracing)
+
+            !Datadog.configuration[:httpx][:distributed_tracing]
+          end
+
+          def datadog_pin
+            @datadog_pin ||= begin
+              service = datadog_config[:service_name]
+              tracer = datadog_config[:tracer]
+
+              Datadog::Pin.new(
+                service,
+                app: "httpx",
+                app_type: Datadog::Ext::AppTypes::WEB,
+                tracer: -> { tracer }
+              )
+            end
+          end
+
+          def datadog_config
+            @datadog_config ||= Datadog.configuration[:httpx, @request.uri.host]
+          end
+        end
+
+        module ConnectionMethods
+          def send(request)
+            RequestTracer.new(request).call
+            super
+          end
+        end
+      end
+
+      module Configuration
+        # Default settings for httpx
+        #
+        class Settings < Datadog::Contrib::Configuration::Settings
+          option :service_name, default: "httpx"
+          option :distributed_tracing, default: true
+          option :split_by_domain, default: false
+
+          option :enabled do |o|
+            o.default { env_to_bool("DD_TRACE_HTTPX_ENABLED", true) }
+            o.lazy
+          end
+
+          option :analytics_enabled do |o|
+            o.default { env_to_bool(%w[DD_TRACE_HTTPX_ANALYTICS_ENABLED DD_HTTPX_ANALYTICS_ENABLED], false) }
+            o.lazy
+          end
+
+          option :analytics_sample_rate do |o|
+            o.default { env_to_float(%w[DD_TRACE_HTTPX_ANALYTICS_SAMPLE_RATE DD_HTTPX_ANALYTICS_SAMPLE_RATE], 1.0) }
+            o.lazy
+          end
+
+          option :error_handler, default: Datadog::Tracer::DEFAULT_ON_ERROR
+        end
+      end
+
+      # Patcher enables patching of 'httpx' with datadog components.
+      #
+      module Patcher
+        include Datadog::Contrib::Patcher
+
+        module_function
+
+        def target_version
+          Integration.version
+        end
+
+        # loads a session instannce with the datadog plugin, and replaces the
+        # base HTTPX::Session with the patched session class.
+        def patch
+          datadog_session = ::HTTPX.plugin(Plugin)
+
+          ::HTTPX.send(:remove_const, :Session)
+          ::HTTPX.send(:const_set, :Session, datadog_session.class)
+        end
+      end
+
+      # Datadog Integration for HTTPX.
+      #
+      class Integration
+        include Contrib::Integration
+
+        # MINIMUM_VERSION = Gem::Version.new('0.11.0')
+        MINIMUM_VERSION = Gem::Version.new("0.10.2")
+
+        register_as :httpx
+
+        def self.version
+          Gem.loaded_specs["httpx"] && Gem.loaded_specs["httpx"].version
+        end
+
+        def self.loaded?
+          defined?(::HTTPX::Request)
+        end
+
+        def self.compatible?
+          super && version >= MINIMUM_VERSION
+        end
+
+        def default_configuration
+          Configuration::Settings.new
+        end
+
+        def patcher
+          Patcher
+        end
+      end
+    end
+  end
+end

data/lib/httpx/adapters/faraday.rb

@@ -65,7 +65,6 @@ module Faraday
         plugin(:compression)
         plugin(:persistent)
 
-        # :nocov:
         module ReasonPlugin
           if RUBY_VERSION < "2.5"
             def self.load_dependencies(*)
@@ -88,7 +87,6 @@ module Faraday
             end
           end
         end
-        # :nocov:
         plugin(ReasonPlugin)
       end
 

data/lib/httpx/adapters/webmock.rb

@@ -0,0 +1,123 @@
+# frozen_string_literal: true
+
+module WebMock
+  module HttpLibAdapters
+    if RUBY_VERSION < "2.5"
+      require "webrick/httpstatus"
+      HTTP_REASONS = WEBrick::HTTPStatus::StatusMessage
+    else
+      require "net/http/status"
+      HTTP_REASONS = Net::HTTP::STATUS_CODES
+    end
+
+    #
+    # HTTPX plugin for webmock.
+    #
+    # Requests are "hijacked" at the session, before they're distributed to a connection.
+    #
+    module Plugin
+      module InstanceMethods
+        private
+
+        def send_requests(*requests, options)
+          request_signatures = requests.map do |request|
+            request_signature = _build_webmock_request_signature(request)
+            WebMock::RequestRegistry.instance.requested_signatures.put(request_signature)
+            request_signature
+          end
+
+          responses = request_signatures.map do |request_signature|
+            WebMock::StubRegistry.instance.response_for_request(request_signature)
+          end
+
+          real_requests = {}
+
+          requests.each_with_index.each_with_object([request_signatures, responses]) do |(request, idx), (sig_reqs, mock_responses)|
+            if (webmock_response = mock_responses[idx])
+              mock_responses[idx] = _build_from_webmock_response(request, webmock_response)
+              WebMock::CallbackRegistry.invoke_callbacks({ lib: :httpx }, sig_reqs[idx], webmock_response)
+              log { "mocking #{request.uri} with #{mock_responses[idx].inspect}" }
+            elsif WebMock.net_connect_allowed?(sig_reqs[idx].uri)
+              log { "performing #{request.uri}" }
+              real_requests[request] = idx
+            else
+              raise WebMock::NetConnectNotAllowedError, sig_reqs[idx]
+            end
+          end
+
+          unless real_requests.empty?
+            reqs = real_requests.keys
+            reqs.zip(super(*reqs, options)).each do |req, res|
+              idx = real_requests[req]
+
+              if WebMock::CallbackRegistry.any_callbacks?
+                webmock_response = _build_webmock_response(req, res)
+                WebMock::CallbackRegistry.invoke_callbacks(
+                  { lib: :httpx, real_request: true }, request_signatures[idx],
+                  webmock_response
+                )
+              end
+
+              responses[idx] = res
+            end
+          end
+
+          responses
+        end
+
+        def _build_webmock_request_signature(request)
+          uri = WebMock::Util::URI.heuristic_parse(request.uri)
+          uri.path = uri.normalized_path.gsub("[^:]//", "/")
+
+          WebMock::RequestSignature.new(
+            request.verb,
+            uri.to_s,
+            body: request.body.each.to_a.join,
+            headers: request.headers.to_h
+          )
+        end
+
+        def _build_webmock_response(_request, response)
+          webmock_response = WebMock::Response.new
+          webmock_response.status = [response.status, HTTP_REASONS[response.status]]
+          webmock_response.body = response.body.to_s
+          webmock_response.headers = response.headers.to_h
+          webmock_response
+        end
+
+        def _build_from_webmock_response(request, webmock_response)
+          return ErrorResponse.new(request, webmock_response.exception, request.options) if webmock_response.exception
+
+          response = request.options.response_class.new(request,
+                                                        webmock_response.status[0],
+                                                        "2.0",
+                                                        webmock_response.headers)
+          response << webmock_response.body.dup
+          response
+        end
+      end
+    end
+
+    class HttpxAdapter < HttpLibAdapter
+      adapter_for :httpx
+
+      class << self
+        def enable!
+          @original_session = ::HTTPX::Session
+
+          webmock_session = ::HTTPX.plugin(Plugin)
+
+          ::HTTPX.send(:remove_const, :Session)
+          ::HTTPX.send(:const_set, :Session, webmock_session.class)
+        end
+
+        def disable!
+          return unless @original_session
+
+          HTTPX.send(:remove_const, :Session)
+          HTTPX.send(:const_set, :Session, @original_session)
+        end
+      end
+    end
+  end
+end