httpx 0.21.0 → 1.2.1

data/doc/release_notes/0_23_0.md

@@ -0,0 +1,42 @@
+# 0.23.0
+
+## Features
+
+### `:retries` plugin: resumable requests
+
+The `:retries` plugin will now support scenarios where, if the request being retried supports the `range` header, and a partial response has been already buffered, the retry will resume from there and only download the missing data.
+
+#### HTTPX::ErrorResponse#response
+
+As a result, ´HTTPX::ErrorResponse#response` has also been introduced; error responses may have an actual response. This happens in cases where the request failed **after** a partial response was initiated.
+
+#### `:buffer_size` option
+
+A nnew option, `:buffer_size`, can be used to tweak the buffers used by the read/write socket routines (16k by default, you can lower it in memory-constrained environments).
+
+## Improvements
+
+### `:native` resolver falls back to TCP for truncated messages
+
+The `:native` resolver will repeat DNS queries to a nameserver via TCP when the first attempt is marked as truncated. This behaviour is both aligned with `getaddrinfo` and the `resolv` standard library.
+
+This introduces a new `resolver_options` option, `:socket_type`, which can now be `:tcp` if it is to remain the default.
+
+## Chore
+
+### HTTPX.build_request should receive upcased string (i.e. "GET")
+
+Functions which receive an HTTP verb should be given he verb in "upcased string" format now. The usage of symbols is still possible, but a deprecation warning will be emitted, and support will be removed in v1.0.0 .
+
+### Remove HTTPX::Registry
+
+These internal registries were a bit magical to use, difficult to debug, not thread-safe, and overall a nuisance when it came to type checking. While there is the possibility that someone was relying on it existing, nothing had ever been publicly documented.
+
+## Bugfixes
+
+* fixed proxy discovery using proxy env vars (`HTTPS_PROXY`, `NO_PROXY`...) being enabled/disabled based on first host uused in the session;
+* fixed `:no_proxy` option usage inn the `:proxy` plugin.
+* fixed `webmock` adapter to correctly disable it when `Webmock.disable!` is called.
+* fixed bug in `:digest_authentication` plugin when enabled and no credentials were passed.
+* fixed several bugs in the `sentry` adapter around breadcrumb handling.
+* fixed `:native` resolver candidate calculation by putting absolute domain at the bottom of the list.

data/doc/release_notes/0_23_1.md

@@ -0,0 +1,5 @@
+# 0.23.1
+
+## Bugfixes
+
+* fixed regression causing dns candidate names not being tried after first one fails.

data/doc/release_notes/0_23_2.md

@@ -0,0 +1,5 @@
+# 0.23.2
+
+## Bugfixes
+
+* fix missing variable on code path in the native resolver.

data/doc/release_notes/0_23_3.md

@@ -0,0 +1,6 @@
+# 0.23.3
+
+## Bugfixes
+
+* native resolver: fix missing exception variable in the DNS error code path.
+* native resolver: fixed short DNS packet handling when using TCP.

data/doc/release_notes/0_23_4.md

@@ -0,0 +1,5 @@
+# 0.23.4
+
+## Bugfixes
+
+* fix `Response::Body#read` which rewinds on every call.

data/doc/release_notes/0_24_0.md

@@ -0,0 +1,48 @@
+# 0.24.0
+
+## Features
+
+### `:oauth` plugin
+
+The `:oauth` plugin manages the handling of a given OAuth session, in that it ships with convenience methods to generate a new access token, which it then injects in all requests.
+
+More info under https://honeyryderchuck.gitlab.io/httpx/wiki/OAuth
+
+### session callbacks
+
+HTTP request/response lifecycle events have now the ability of being intercepted via public API callback methods:
+
+```ruby
+HTTPX.on_request_completed do |request|
+  puts "request to #{request.uri} sent"
+end.get(...)
+```
+
+More info under https://honeyryderchuck.gitlab.io/httpx/wiki/Events to know which events and callback methods are supported.
+
+### `:circuit_breaker` plugin `on_circuit_open` callback
+
+A callback has been introduced for the `:circuit_breaker` plugin, which is triggered when a circuit is opened.
+
+```ruby
+http = HTTPX.plugin(:circuit_breaker).on_circuit_open do |req|
+  puts "circuit opened for #{req.uri}"
+end
+http.get(...)
+```
+
+## Improvements
+
+Several `:response_cache` features have been improved:
+
+* `:response_cache` plugin: response cache store has been made thread-safe.
+* cached response sharing across threads is made safer, as stringio/tempfile instances are copied instead of shared (without copying the underling string/file).
+* stale cached responses are eliminate on cache store lookup/store operations.
+* already closed responses are evicted from the cache store.
+* fallback for lack of compatible response "date" header has been fixed to return a `Time` object.
+
+## Bugfixes
+
+* Ability to recover from errors happening during response chunk processing (required for overriding behaviour and response chunk callbacks); error bubbling up will result in the connection being closed.
+* Happy eyeballs support for multi-homed early-resolved domain names (such as `localhost` under `/etc/hosts`) was broken, as it would try the first given IP; so, if given `::1` and connection would fail, it wouldn't try `127.0.0.1`, which would have succeeded.
+* `:digest_authentication` plugin was removing the "algorithm" header on `-sess` declared algorithms, which is required for HTTP digest auth negotiation.

data/doc/release_notes/0_24_1.md

@@ -0,0 +1,12 @@
+# 0.24.1
+
+## Improvements
+
+* datadog adapter: support `:service_name` configuration option.
+* datadog adapter: set `:distributed_tracing` to `true` by default.
+* `:proxy` plugin: when the proxy uri uses an unsupported scheme (i.e.: "scp://125.24.2.1"), a more user friendly error is raised (instead of the previous broken stacktrace).
+
+## Bugfixes
+
+* datadog adapter: fix tracing enable call, which was wrongly calling `super`.
++ `:proxy` plugin: fix for bug which was turning off plugins overriding `HTTPX::Connection#send` (such as the datadog adapter).

data/doc/release_notes/0_24_2.md

@@ -0,0 +1,12 @@
+# 0.24.2
+
+## Improvements
+
+* besides an array, `:resolver_options` can now receive a hash for `:nameserver`, which **must** be indexed by IP family (`Socket::AF_INET6` or `Socket::AF_INET`); each group of nameservers will be used for emitting DNS queries of that iP family.
+* `:authentication` plugin: Added `#bearer_auth` helper, which receives a token, and sets it as `"Bearer $TOKEN` in the `"authorization"` header.
+* `faraday` adapter: now implements `#build_connection` and `#close`, will now interact with `faraday` native timeouts (`:read`, `:write` and `:connect`).
+
+
+## Bugfixes
+
+* fixed native resolver bug when queries involving intermediate alias would be kept after the original query and mess with re-queries.

data/doc/release_notes/0_24_3.md

@@ -0,0 +1,12 @@
+# 0.24.3
+
+## Improvements
+
+* faraday adapter: reraise httpx timeout errors as faraday errors.
+* faraday adapter: support `:bind` option, which expects a host and port to connect to.
+
+## Bugfixes
+
+* faraday adapter: fix `#close` implementation using the wrong ivar.
+* faraday adapter: fix usage of `requestt_timeout` translation of faraday timeouts into httpx timeouts.
+* faraday adapter: `ssl: { verify: false }` was being ignored, and certification verification was still proceeding.

data/doc/release_notes/0_24_4.md

@@ -0,0 +1,18 @@
+# 0.24.4
+
+## Improvements
+
+* `digest_authentication` plugin now supports passing HA1hashed with password HA1s (common to store in htdigest files for example) when setting the`:hashed` kwarg to `true` in the `.digest_auth` call.
+  * ex: `http.digest_auth(user, get_hashed_passwd_from_htdigest(user), hashed: true)`
+* TLS session resumption is now supported
+  * whenever possible, `httpx` sessions will recycle used connections so that, in the case of TLS connections, the first session will keep being reusedd, thereby diminishing the overhead of subsequent TLS handshakes on the same host.
+  * TLS sessions are only reused in the scope of the same `httpx` session, unless the `:persistent` plugin is used, in which case, the persisted `httpx` session will always try to resume TLS sessions.
+
+## Bugfixes
+
+* When explicitly using IP addresses in the URL host, TLS handshake will now verify tif he IP address is included in the certificate.
+  * IP address will keep not be used for SNI, as per RFC 6066, section 3.
+  * ex: `http.get("https://10.12.0.12/get")`
+  * if you want the prior behavior, set `HTTPX.with(ssl: {verify_hostname: false})`
+* Turn TLS hostname verification on for `jruby` (it's turned off by default).
+  * if you want the prior behavior, set `HTTPX.with(ssl: {verify_hostname: false})`

data/doc/release_notes/0_24_5.md

@@ -0,0 +1,6 @@
+# 0.24.5
+
+## Bugfixes
+
+* fix for SSL handshake post connection SAN check using IPv6 address.
+* fix bug in DoH impl when the request returned no answer.

data/doc/release_notes/0_24_6.md

@@ -0,0 +1,5 @@
+# 0.24.6
+
+## Bugfixes
+
+* fix Session class assertions not prepared for class overrides, which could break some plugins which override the Session class on load (such as `datadog` or `webmock` adapters).

data/doc/release_notes/0_24_7.md

@@ -0,0 +1,10 @@
+# 0.24.6
+
+## dependencies
+
+`http-2-next` last supported version for the 0.x series is the last version before v1. This shoul ensure that older versions of `httpx` won't be affected by any of the recent breaking changes.
+
+## Bugfixes
+
+* `grpc`: setup of rpc calls from camel-cased symbols has been fixed. As an improvement, the GRPC-enabled session will now support both snake-cased, as well as camel-cased calls.
+* `datadog` adapter has now been patched to support the most recent breaking changes of `ddtrace` configuration DSL (`env_to_bool` is no longer supported).

data/doc/release_notes/1_0_0.md

@@ -0,0 +1,60 @@
+# 1.0.0
+
+## Breaking changes
+
+* the minimum supported ruby version is 2.7.0 .
+* The fallback support for IDNA 2003 has been removed. If you require this feature, install the [idnx gem](https://github.com/HoneyryderChuck/idnx), which `httpx` automatically integrates with when available (and supports IDNA 2008).
+* `:total_timeout` option has been removed (no session-wide timeout supported, use `:request_timeout`).
+* `:read_timeout` and `:write_timeout` are now set to 60 seconds by default, and preferred over `:operation_timeout`;
+  * the exception being in the `:stream` plugin, as the response is theoretically endless (so `:read_timeout` is unset).
+* The `:multipart` plugin is removed, as its functionality and API are now loaded by default (no API changes).
+* The `:compression` plugin is removed, as its functionality and API are now loaded by default (no API changes).
+  * `:compression_threshold_size` was removed (formats in `"content-encoding"` request header will always encode the request body).
+  * the new `:compress_request_body` and `:decompress_response_body` can be set to `false` to (respectively) disable compression of passed input body, or decompression of the response body.
+* `:retries` plugin: the `:retry_on` condition will **not** replace default retriable error checks, it will now instead be triggered **only if** no retryable error has been found.
+
+### plugins
+
+* `:authentication` plugin becomes `:auth`.
+  * `.authentication` helper becomes `.authorization`.
+* `:basic_authentication` plugin becomes `:basic_auth`.
+  * `:basic_authentication` helper is removed.
+* `:digest_authentication` plugin becomes `:digest_auth`.
+  * `:digest_authentication` helper is removed.
+* `:ntlm_authentication` plugin becomes `:ntlm_auth`.
+  * `:ntlm_authentication` helper is removed.
+* OAuth plugin: `:oauth_authentication` helper is rename to `:oauth_auth`.
+* `:compression/brotli` plugin becomes `:brotli`.
+
+### Support removed for deprecated APIs
+
+* The deprecated `HTTPX::Client` constant lookup has been removed (use `HTTPX::Session` instead).
+* The deprecated `HTTPX.timeout({...})` function has been removed (use `HTTPX.with(timeout: {...})` instead).
+* The deprecated `HTTPX.headers({...})` function has been removed (use `HTTPX.with(headers: {...})` instead).
+* The deprecated `HTTPX.plugins(...)` function has been removed (use `HTTPX.plugin(...).plugin(...)...` instead).
+* The deprecated `:transport_options` option, which was only valid for UNIX connections, has been removed (use `:addresses` instead).
+* The deprecated `def_option(...)` function, previously used to define additional options in plugins, has been removed (use `def option_$new_option)` instead).
+* The deprecated `:loop_timeout` timeout option has been removed.
+* `:stream` plugin: the deprecated `HTTPX::InstanceMethods::StreamResponse` has been removed (use `HTTPX::StreamResponse` instead).
+* The deprecated usage of symbols to indicate HTTP verbs (i.e. `HTTPX.request(:get, ...)` or `HTTPX.build_request(:get, ...)`) is not supported anymore (use the upcase string always, i.e. `HTTPX.request("GET", ...)` or `HTTPX.build_request("GET", ...)`, instead).
+* The deprecated `HTTPX::ErrorResponse#status` method has been removed (use `HTTPX::ErrorResponse#error` instead).
+
+### dependencies
+
+* `http-2-next` minimum supported version is 1.0.0.
+* `:datadog` adapter only supports `ddtrace` gem 1.x or higher.
+* `:faraday` adapter only supports `faraday` gem 1.x or higher.
+
+## Improvements
+
+* `circuit_breaker`: the drip rate of real request during the "half-open" stage of a circuit will reliably distribute real requests (as per the drip rate) over the `max_attempts`, before the circuit is closed.
+
+## Bugfixes
+
+* Tempfiles are now correctly identified as file inputs for multipart requests.
+* fixed `proxy` plugin behaviour when loaded with the `follow_redirects` plugin and processing a 305 response (request needs to be retried on a different proxy).
+
+## Chore
+
+* `:grpc` plugin: connection won't buffer requests before HTTP/2 handshake is commpleted, i.e. works the same as plain `httpx` HTTP/2 connection establishment.
+  * if you are relying on this, you can keep the old behavior this way: `HTTPX.plugin(:grpc, http2_settings: { wait_for_handshake: false })`.

data/doc/release_notes/1_0_1.md

@@ -0,0 +1,5 @@
+# 1.0.1
+
+## Bugfixes
+
+* do not try to inflate empty chunks (it triggered an error during response decoding).

data/doc/release_notes/1_0_2.md

@@ -0,0 +1,7 @@
+# 1.0.2
+
+## bugfixes
+
+* bump `http-2-next` to 1.0.1, which fixes a bug where http/2 connection interprets MAX_CONCURRENT_STREAMS as request cap.
+* `grpc`: setup of rpc calls from camel-cased symbols has been fixed. As an improvement, the GRPC-enabled session will now support both snake-cased, as well as camel-cased calls.
+* `datadog` adapter has now been patched to support the most recent breaking changes of `ddtrace` configuration DSL (`env_to_bool` is no longer supported).

data/doc/release_notes/1_1_0.md

@@ -0,0 +1,32 @@
+# 1.1.0
+
+## Features
+
+A function, `#peer_address`, was added to the response object, which returns the IP (either a string or an `IPAddr` object) from the socket used to get the response from.
+
+```ruby
+response = HTTPX.get("https://example.com")
+response.peer_address #=> #<IPAddr: IPv4:93.184.216.34/255.255.255.255>
+```
+
+error responses will also expose an IP address via `#peer_address` as long a connection happened before the error.
+
+## Improvements
+
+* A performance regression involving the new default timeouts has been fixed, which could cause significant overhead in "multiple requests in sequence" scenarios, and was clearly visible in benchmarks.
+  * this regression will still be seen in jruby due to a bug, which fix will be released in jruby 9.4.5.0.
+* HTTP/1.1 connections are now set to handle as many requests as they can by default (instead of the past default of max 200, at which point they'd be recycled).
+* tolerate the inexistence of `openssl` in the installed ruby, like `net-http` does.
+* `on_connection_opened` and `on_connection_closed` will yield the `OpenSSL::SSL::SSLSocket` instance for `https` backed origins (instead of always the `Socket` instance).
+
+## Bugfixes
+
+* when using the `:native` resolver (default option), a default of 1 for ndots is set, for systems which do not set one.
+* replaced usage of `Float::INFINITY` with `nil` for timeout defaults, as the former can't be used in IO wait functions.
+* `faraday` adapter timeout setup now maps to `:read_timeout` and `:write_timeout` options from `httpx`.
+* fixed HTTP/1.1 connection recycling on number of max requests exhausted.
+* `response.json` will now work when "content-type" header is set to "application/hal+json".
+
+## Chore
+
+* when using the `:cookies` plugin, a warning message to install the idnx message will only be emitted if the cookie domain is an IDN (this message was being shown all the time since v1 release).

data/doc/release_notes/1_1_1.md

@@ -0,0 +1,17 @@
+# 1.1.1
+
+## improvements
+
+* (Re-)enabling default retries in DNS name queries; this had been disabled as a result of revamping timeouts, and resulted in queries only being sent once, which is very little for UDP-related traffic, and breaks if using DNs rate-limiting software. Retries the query just once, for now.
+
+## bugfixes
+
+* reset timers when adding new intervals, as these may be added as a result on after-select connection handling, and must wait for the next tick cycle (before the patch, they were triggering too soon).
+* fixed "on close" callback leak on connection reuse, which caused linear performance regression in benchmarks performing one request per connection.
+* fixed hanging connection when an HTTP/1.1 emitted a "connection: close" header but the server would not emit one (it closes the connection now).
+* fixed recursive dns cached lookups which may have already expired, and created nil entries in the returned address list.
+* dns system resolver is now able to retry on failure.
+
+## chore
+
+* remove duplicated callback unregitering connections.

data/doc/release_notes/1_1_2.md

@@ -0,0 +1,12 @@
+# 1.1.2
+
+## improvements
+
+* only moving eden connections to idle when they're recycled.
+
+## bugfixes
+
+* skip closing a connection which is already closed during reset.
+* sentry adapter: fixed `super` call which didn't have a super method (this prevented usinng sentry-enabled sessions with the `:retries` plugin).
+* sentry adapter: fixing registering of sentry config.
+* sentry adapter: do not propagate traces when relevant sdk options are disabled (such as `propagate_traces`).

data/doc/release_notes/1_1_3.md

@@ -0,0 +1,18 @@
+# 1.1.3
+
+## improvements
+
+## security
+
+* when using `:follow_redirects` plugin, the "authorization" header will be removed when following redirect responses to a different origin.
+
+## bugfixes
+
+* fixed `:stream` plugin not following redirect responses when used with the `:follow_redirects` plugin.
+* fixed `:stream` plugin not doing content decoding when responses were p.ex. gzip-compressed.
+* fixed bug preventing usage of IPv6 loopback or link-local addresses in the request URL in systems with no IPv6 internet connectivity (the request was left hanging).
+* protect all code which may initiate a new connection from abrupt errors (such as internet turned off), as it was done on the initial request call.
+
+## chore
+
+internal usage of `mutex_m` has been removed (`mutex_m` is going to be deprecated in ruby 3.3).

data/doc/release_notes/1_1_4.md

@@ -0,0 +1,6 @@
+# 1.1.4
+
+## bugfixes
+
+* datadog adapter: use `Gem::Version` to invoke the correct configuration API.
+* stream plugin: do not preempt request enqueuing (this was making integration with the `:follow_redirects` plugin fail when set up with `webmock`).

data/doc/release_notes/1_1_5.md

@@ -0,0 +1,12 @@
+# 1.1.5
+
+## improvements
+
+* pattern matching support for responses has been backported to ruby 2.7 as well.
+
+## bugfixes
+
+* `stream` plugin: fix for `HTTPX::StreamResponse#each_line` not yielding the last line of the payload when not delimiter-terminated.
+* `stream` plugin: fix `webmock` adapter integration when methods calls would happen in the `HTTPX::StreamResponse#each` block.
+* `stream` plugin: fix `:follow_redirects` plugin integration which was caching the redirect response and using it for method calls inside the `HTTPX::StreamResponse#each` block.
+* "103 early hints" responses will be ignored when processing the response (it was causing the response returned by sesssions to hold its headers, instead of the following 200 response, while keeping the 200 response body).

data/doc/release_notes/1_2_0.md

@@ -0,0 +1,49 @@
+# 1.2.0
+
+## Features
+
+### `:ssrf_filter` plugin
+
+The `:ssrf_filter` plugin prevents server-side request forgery attacks, by blocking requests to the internal network. This is useful when the URLs used to perform requests aren’t under the developer control (such as when they are inserted via a web application form).
+
+```ruby
+http = HTTPX.plugin(:ssrf_filter)
+
+# this works
+response = http.get("https://example.com")
+
+# this doesn't
+response = http.get("http://localhost:3002")
+response = http.get("http://[::1]:3002")
+response = http.get("http://169.254.169.254/latest/meta-data/")
+```
+
+More info under https://honeyryderchuck.gitlab.io/httpx/wiki/SSRF-Filter
+
+### `:callbacks` plugin
+
+The session callbacks introduced in v0.24.0 are in its own plugin. Older code will still work and emit a deprecation warning.
+
+More info under https://honeyryderchuck.gitlab.io/httpx/wiki/Callbacks
+
+### `:redirect_on` option for `:follow_redirects` plugin
+
+This option allows passing a callback which, when returning `false`, can interrupt the redirect loop.
+
+```ruby
+http = HTTPX.plugin(:follow_redirects).with(redirect_on: ->(location_uri) { BLACKLIST_HOSTS.include?(location_uri.host) })
+```
+
+### `:close_on_handshake_timeout` timeout
+
+A new `:timeout` option, `:close_handshake_timeout`, is added, which monitors connection readiness when performing HTTP/2 connection termination handshake.
+
+## Improvements
+
+* Internal "eden connections" concept was removed, and connection objects are now kept-and-reused during the lifetime of a session, even when closed. This simplified connectio pool implementation and improved performance.
+* request using `:proxy` and `:retries` plugin enabled sessions will now retry on proxy connection establishment related errors.
+
+## Bugfixes
+
+* webmock adapter: mocked responses storing decoded payloads won't try to decode them again (fixes vcr/webmock integrations).
+* webmock adapter: fix issue related with making real requests over webmock-enabled connection.

data/doc/release_notes/1_2_1.md

@@ -0,0 +1,6 @@
+# 1.2.1
+
+## Bugfixes
+
+* DoH resolver: try resolving other candidates on "domain not found" error (same behaviour as with native resolver).
+* Allow HTTP/2 connections to exit cleanly when TLS session gets corrupted and termination handshake can't be performed.