httpx 1.0.2 → 1.1.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (47) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +2 -2
  3. data/doc/release_notes/1_1_0.md +32 -0
  4. data/doc/release_notes/1_1_1.md +17 -0
  5. data/lib/httpx/adapters/faraday.rb +28 -19
  6. data/lib/httpx/connection/http1.rb +13 -6
  7. data/lib/httpx/connection/http2.rb +1 -1
  8. data/lib/httpx/connection.rb +53 -15
  9. data/lib/httpx/domain_name.rb +6 -2
  10. data/lib/httpx/errors.rb +32 -0
  11. data/lib/httpx/io/ssl.rb +3 -1
  12. data/lib/httpx/io/tcp.rb +4 -2
  13. data/lib/httpx/io.rb +5 -1
  14. data/lib/httpx/options.rb +48 -1
  15. data/lib/httpx/plugins/expect.rb +10 -8
  16. data/lib/httpx/plugins/proxy/http.rb +0 -1
  17. data/lib/httpx/pool.rb +0 -4
  18. data/lib/httpx/request/body.rb +22 -9
  19. data/lib/httpx/request.rb +63 -4
  20. data/lib/httpx/resolver/native.rb +2 -2
  21. data/lib/httpx/resolver/resolver.rb +5 -2
  22. data/lib/httpx/resolver/system.rb +5 -2
  23. data/lib/httpx/resolver.rb +6 -4
  24. data/lib/httpx/response/body.rb +30 -5
  25. data/lib/httpx/response/buffer.rb +20 -14
  26. data/lib/httpx/response.rb +95 -16
  27. data/lib/httpx/selector.rb +2 -2
  28. data/lib/httpx/session.rb +64 -2
  29. data/lib/httpx/timers.rb +35 -8
  30. data/lib/httpx/transcoder/json.rb +1 -1
  31. data/lib/httpx/transcoder/utils/inflater.rb +19 -0
  32. data/lib/httpx/version.rb +1 -1
  33. data/sig/connection/http1.rbs +3 -3
  34. data/sig/connection/http2.rbs +1 -1
  35. data/sig/connection.rbs +4 -1
  36. data/sig/io/tcp.rbs +1 -1
  37. data/sig/options.rbs +2 -2
  38. data/sig/pool.rbs +1 -1
  39. data/sig/request/body.rbs +0 -2
  40. data/sig/request.rbs +9 -3
  41. data/sig/resolver/native.rbs +1 -1
  42. data/sig/resolver.rbs +1 -1
  43. data/sig/response/body.rbs +0 -1
  44. data/sig/response.rbs +11 -3
  45. data/sig/timers.rbs +17 -7
  46. data/sig/transcoder/utils/inflater.rbs +12 -0
  47. metadata +8 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a0f330c17658b6d488b83008df23e73eb276c04fb93f6e72093224a1a6e1d839
4
- data.tar.gz: 75fea2fe69162c00d371ca83f121883d435be3d7d0749f1f2dce71332c6f18d2
3
+ metadata.gz: 39a2d410f391dedb077e3704edeafde92786201ebd7b1d3e4e2f6aa7e254797b
4
+ data.tar.gz: bdcc46a37c8cc4ba8edd11355c5dc439a7401569c8ef014c7ca45ac22e97d284
5
5
  SHA512:
6
- metadata.gz: 4c3bf06316177777f8802fc7662021e565ddc19590a2cad092ab3e05759fa778e7edd4f9b6ee9359b582d8828f669236be517129af57a12d17cfe71cde7adf06
7
- data.tar.gz: 66ef58dbd0c3f4681bbde3e211afaaa7272d88aa4aa70e5791636147439a5bacb2bf85f89d0b33a792ddcf6444ae14e3c64685436b9cb849cb7fad30f3db1ef1
6
+ metadata.gz: df50d7db27a18a2f7d610e12b618fc1a8ab2d18a6a1f3bbd70054a4fdab9bec8a8e428115696d5f042f28c7409e7539eaca0f3798d13bc6ab98d88e43d41808d
7
+ data.tar.gz: 63563a10039713d75e6d3d795ddfc08f91a46c2f73225b9987e96acbed9e23d0c5e69154589afbf0163ec69ff99eb9c34f517fb9d360ad7f951a06fb5569ce95
data/README.md CHANGED
@@ -108,8 +108,8 @@ HTTPX.get(
108
108
  ```ruby
109
109
  response = HTTPX.get("https://www.google.com", params: { q: "me" })
110
110
  response = HTTPX.post("https://www.nghttp2.org/httpbin/post", form: {name: "John", age: "22"})
111
- response = HTTPX.plugin(:basic_authentication)
112
- .basic_authentication("user", "pass")
111
+ response = HTTPX.plugin(:basic_auth)
112
+ .basic_auth("user", "pass")
113
113
  .get("https://www.google.com")
114
114
 
115
115
  # more complex client objects can be cached, and are thread-safe
@@ -0,0 +1,32 @@
1
+ # 1.1.0
2
+
3
+ ## Features
4
+
5
+ A function, `#peer_address`, was added to the response object, which returns the IP (either a string or an `IPAddr` object) from the socket used to get the response from.
6
+
7
+ ```ruby
8
+ response = HTTPX.get("https://example.com")
9
+ response.peer_address #=> #<IPAddr: IPv4:93.184.216.34/255.255.255.255>
10
+ ```
11
+
12
+ error responses will also expose an IP address via `#peer_address` as long a connection happened before the error.
13
+
14
+ ## Improvements
15
+
16
+ * A performance regression involving the new default timeouts has been fixed, which could cause significant overhead in "multiple requests in sequence" scenarios, and was clearly visible in benchmarks.
17
+ * this regression will still be seen in jruby due to a bug, which fix will be released in jruby 9.4.5.0.
18
+ * HTTP/1.1 connections are now set to handle as many requests as they can by default (instead of the past default of max 200, at which point they'd be recycled).
19
+ * tolerate the inexistence of `openssl` in the installed ruby, like `net-http` does.
20
+ * `on_connection_opened` and `on_connection_closed` will yield the `OpenSSL::SSL::SSLSocket` instance for `https` backed origins (instead of always the `Socket` instance).
21
+
22
+ ## Bugfixes
23
+
24
+ * when using the `:native` resolver (default option), a default of 1 for ndots is set, for systems which do not set one.
25
+ * replaced usage of `Float::INFINITY` with `nil` for timeout defaults, as the former can't be used in IO wait functions.
26
+ * `faraday` adapter timeout setup now maps to `:read_timeout` and `:write_timeout` options from `httpx`.
27
+ * fixed HTTP/1.1 connection recycling on number of max requests exhausted.
28
+ * `response.json` will now work when "content-type" header is set to "application/hal+json".
29
+
30
+ ## Chore
31
+
32
+ * when using the `:cookies` plugin, a warning message to install the idnx message will only be emitted if the cookie domain is an IDN (this message was being shown all the time since v1 release).
@@ -0,0 +1,17 @@
1
+ # 1.1.1
2
+
3
+ ## improvements
4
+
5
+ * (Re-)enabling default retries in DNS name queries; this had been disabled as a result of revamping timouts, and resulted in queries only being sent once, which is very little for UDP-related traffic, and breaks if using DNs rate-limiting software. Retries the query just once, for now.
6
+
7
+ ## bugfixes
8
+
9
+ * reset timers when adding new intervals, as these may be added as a result on after-select connection handling, and must wait for the next tick cycle (before the patch, they were triggering too soon).
10
+ * fixed "on close" callback leak on connection reuse, which caused linear performance regression in benchmarks performing one request per connection.
11
+ * fixed hanging connection whan an HTTP/1.1 emitted a "connection: close" header but the server would not emit one (it closes the connection now).
12
+ * fixed recursive dns cached lookups which may have already expired, and created nil entries in the returned address list.
13
+ * dns system resolver is now able to retry on failure.
14
+
15
+ ## chore
16
+
17
+ * remove duplicated callback unregitering connections.
@@ -69,38 +69,47 @@ module Faraday
69
69
  timeout_options = {}
70
70
  req_opts = env.request
71
71
  if (sec = request_timeout(:read, req_opts))
72
- timeout_options[:operation_timeout] = sec
72
+ timeout_options[:read_timeout] = sec
73
73
  end
74
74
 
75
75
  if (sec = request_timeout(:write, req_opts))
76
- timeout_options[:operation_timeout] = sec
76
+ timeout_options[:write_timeout] = sec
77
77
  end
78
78
 
79
79
  if (sec = request_timeout(:open, req_opts))
80
80
  timeout_options[:connect_timeout] = sec
81
81
  end
82
82
 
83
- ssl_options = {}
84
-
85
- unless env.ssl.verify.nil?
86
- ssl_options[:verify_mode] = env.ssl.verify ? OpenSSL::SSL::VERIFY_PEER : OpenSSL::SSL::VERIFY_NONE
87
- end
88
-
89
- ssl_options[:ca_file] = env.ssl.ca_file if env.ssl.ca_file
90
- ssl_options[:ca_path] = env.ssl.ca_path if env.ssl.ca_path
91
- ssl_options[:cert_store] = env.ssl.cert_store if env.ssl.cert_store
92
- ssl_options[:cert] = env.ssl.client_cert if env.ssl.client_cert
93
- ssl_options[:key] = env.ssl.client_key if env.ssl.client_key
94
- ssl_options[:ssl_version] = env.ssl.version if env.ssl.version
95
- ssl_options[:verify_depth] = env.ssl.verify_depth if env.ssl.verify_depth
96
- ssl_options[:min_version] = env.ssl.min_version if env.ssl.min_version
97
- ssl_options[:max_version] = env.ssl.max_version if env.ssl.max_version
98
-
99
83
  {
100
- ssl: ssl_options,
84
+ ssl: ssl_options_from_env(env),
101
85
  timeout: timeout_options,
102
86
  }
103
87
  end
88
+
89
+ if defined?(::OpenSSL)
90
+ def ssl_options_from_env(env)
91
+ ssl_options = {}
92
+
93
+ unless env.ssl.verify.nil?
94
+ ssl_options[:verify_mode] = env.ssl.verify ? OpenSSL::SSL::VERIFY_PEER : OpenSSL::SSL::VERIFY_NONE
95
+ end
96
+
97
+ ssl_options[:ca_file] = env.ssl.ca_file if env.ssl.ca_file
98
+ ssl_options[:ca_path] = env.ssl.ca_path if env.ssl.ca_path
99
+ ssl_options[:cert_store] = env.ssl.cert_store if env.ssl.cert_store
100
+ ssl_options[:cert] = env.ssl.client_cert if env.ssl.client_cert
101
+ ssl_options[:key] = env.ssl.client_key if env.ssl.client_key
102
+ ssl_options[:ssl_version] = env.ssl.version if env.ssl.version
103
+ ssl_options[:verify_depth] = env.ssl.verify_depth if env.ssl.verify_depth
104
+ ssl_options[:min_version] = env.ssl.min_version if env.ssl.min_version
105
+ ssl_options[:max_version] = env.ssl.max_version if env.ssl.max_version
106
+ ssl_options
107
+ end
108
+ else
109
+ def ssl_options_from_env(*)
110
+ {}
111
+ end
112
+ end
104
113
  end
105
114
 
106
115
  include RequestMixin
@@ -15,7 +15,7 @@ module HTTPX
15
15
  def initialize(buffer, options)
16
16
  @options = Options.new(options)
17
17
  @max_concurrent_requests = @options.max_concurrent_requests || MAX_REQUESTS
18
- @max_requests = @options.max_requests || MAX_REQUESTS
18
+ @max_requests = @options.max_requests
19
19
  @parser = Parser::HTTP1.new(self)
20
20
  @buffer = buffer
21
21
  @version = [1, 1]
@@ -181,10 +181,17 @@ module HTTPX
181
181
  if response.is_a?(ErrorResponse)
182
182
  disable
183
183
  else
184
- manage_connection(response)
184
+ manage_connection(request, response)
185
185
  end
186
186
 
187
- send(@pending.shift) unless @pending.empty?
187
+ if exhausted?
188
+ @pending.concat(@requests)
189
+ @requests.clear
190
+
191
+ emit(:exhausted)
192
+ else
193
+ send(@pending.shift) unless @pending.empty?
194
+ end
188
195
  end
189
196
 
190
197
  def handle_error(ex)
@@ -217,7 +224,7 @@ module HTTPX
217
224
 
218
225
  private
219
226
 
220
- def manage_connection(response)
227
+ def manage_connection(request, response)
221
228
  connection = response.headers["connection"]
222
229
  case connection
223
230
  when /keep-alive/i
@@ -247,7 +254,7 @@ module HTTPX
247
254
  disable
248
255
  when nil
249
256
  # In HTTP/1.1, it's keep alive by default
250
- return if response.version == "1.1"
257
+ return if response.version == "1.1" && request.headers["connection"] != "close"
251
258
 
252
259
  disable
253
260
  end
@@ -287,7 +294,7 @@ module HTTPX
287
294
 
288
295
  connection = request.headers["connection"]
289
296
 
290
- connection ||= if request.options.persistent
297
+ connection ||= if request.persistent?
291
298
  # when in a persistent connection, the request can't be at
292
299
  # the edge of a renegotiation
293
300
  if @requests.index(request) + 1 < @max_requests
@@ -35,7 +35,7 @@ module HTTPX
35
35
  @handshake_completed = false
36
36
  @wait_for_handshake = @settings.key?(:wait_for_handshake) ? @settings.delete(:wait_for_handshake) : true
37
37
  @max_concurrent_requests = @options.max_concurrent_requests || MAX_CONCURRENT_REQUESTS
38
- @max_requests = @options.max_requests || Float::INFINITY
38
+ @max_requests = @options.max_requests
39
39
  init_connection
40
40
  end
41
41
 
@@ -70,6 +70,8 @@ module HTTPX
70
70
  @inflight = 0
71
71
  @keep_alive_timeout = @options.timeout[:keep_alive_timeout]
72
72
 
73
+ @intervals = []
74
+
73
75
  self.addresses = @options.addresses if @options.addresses
74
76
  end
75
77
 
@@ -213,6 +215,9 @@ module HTTPX
213
215
 
214
216
  def call
215
217
  case @state
218
+ when :idle
219
+ connect
220
+ consume
216
221
  when :closed
217
222
  return
218
223
  when :closing
@@ -268,7 +273,7 @@ module HTTPX
268
273
  end
269
274
 
270
275
  def timeout
271
- return @timeout if defined?(@timeout)
276
+ return @timeout if @timeout
272
277
 
273
278
  return @options.timeout[:connect_timeout] if @state == :idle
274
279
 
@@ -294,7 +299,15 @@ module HTTPX
294
299
  @state == :open || @state == :inactive
295
300
  end
296
301
 
297
- def raise_timeout_error(interval)
302
+ def handle_socket_timeout(interval)
303
+ @intervals.delete_if(&:elapsed?)
304
+
305
+ unless @intervals.empty?
306
+ # remove the intervals which will elapse
307
+
308
+ return
309
+ end
310
+
298
311
  error = HTTPX::TimeoutError.new(interval, "timed out while waiting on select")
299
312
  error.set_backtrace(caller)
300
313
  on_error(error)
@@ -449,6 +462,7 @@ module HTTPX
449
462
 
450
463
  def send_request_to_parser(request)
451
464
  @inflight += 1
465
+ request.peer_address = @io.ip
452
466
  parser.send(request)
453
467
 
454
468
  set_request_timeouts(request)
@@ -504,7 +518,6 @@ module HTTPX
504
518
  else
505
519
  transition(:closing)
506
520
  transition(:closed)
507
- emit(:reset)
508
521
 
509
522
  @parser.reset if @parser
510
523
  transition(:idle)
@@ -603,7 +616,7 @@ module HTTPX
603
616
  def purge_after_closed
604
617
  @io.close if @io
605
618
  @read_buffer.clear
606
- remove_instance_variable(:@timeout) if defined?(@timeout)
619
+ @timeout = nil
607
620
  end
608
621
 
609
622
  def build_socket(addrs = nil)
@@ -655,19 +668,26 @@ module HTTPX
655
668
 
656
669
  def set_request_timeouts(request)
657
670
  write_timeout = request.write_timeout
658
- request.once(:headers) do
659
- @timers.after(write_timeout) { write_timeout_callback(request, write_timeout) }
660
- end unless write_timeout.nil? || write_timeout.infinite?
661
-
662
671
  read_timeout = request.read_timeout
663
- request.once(:done) do
664
- @timers.after(read_timeout) { read_timeout_callback(request, read_timeout) }
665
- end unless read_timeout.nil? || read_timeout.infinite?
666
-
667
672
  request_timeout = request.request_timeout
668
- request.once(:headers) do
669
- @timers.after(request_timeout) { read_timeout_callback(request, request_timeout, RequestTimeoutError) }
670
- end unless request_timeout.nil? || request_timeout.infinite?
673
+
674
+ unless write_timeout.nil? || write_timeout.infinite?
675
+ set_request_timeout(request, write_timeout, :headers, %i[done response]) do
676
+ write_timeout_callback(request, write_timeout)
677
+ end
678
+ end
679
+
680
+ unless read_timeout.nil? || read_timeout.infinite?
681
+ set_request_timeout(request, read_timeout, :done, :response) do
682
+ read_timeout_callback(request, read_timeout)
683
+ end
684
+ end
685
+
686
+ return if request_timeout.nil? || request_timeout.infinite?
687
+
688
+ set_request_timeout(request, request_timeout, :headers, :response) do
689
+ read_timeout_callback(request, request_timeout, RequestTimeoutError)
690
+ end
671
691
  end
672
692
 
673
693
  def write_timeout_callback(request, write_timeout)
@@ -688,6 +708,24 @@ module HTTPX
688
708
  on_error(error)
689
709
  end
690
710
 
711
+ def set_request_timeout(request, timeout, start_event, finish_events, &callback)
712
+ request.once(start_event) do
713
+ interval = @timers.after(timeout, callback)
714
+
715
+ Array(finish_events).each do |event|
716
+ # clean up reques timeouts if the connection errors out
717
+ request.once(event) do
718
+ if @intervals.include?(interval)
719
+ interval.delete(callback)
720
+ @intervals.delete(interval) if interval.no_callbacks?
721
+ end
722
+ end
723
+ end
724
+
725
+ @intervals << interval
726
+ end
727
+ end
728
+
691
729
  class << self
692
730
  def parser_type(protocol)
693
731
  case protocol
@@ -61,8 +61,12 @@ module HTTPX
61
61
  # Normalizes a _domain_ using the Punycode algorithm as necessary.
62
62
  # The result will be a downcased, ASCII-only string.
63
63
  def normalize(domain)
64
- domain = domain.chomp(".").unicode_normalize(:nfc) unless domain.ascii_only?
65
- Punycode.encode_hostname(domain).downcase
64
+ unless domain.ascii_only?
65
+ domain = domain.chomp(".").unicode_normalize(:nfc)
66
+ domain = Punycode.encode_hostname(domain)
67
+ end
68
+
69
+ domain.downcase
66
70
  end
67
71
  end
68
72
 
data/lib/httpx/errors.rb CHANGED
@@ -1,20 +1,27 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module HTTPX
4
+ # the default exception class for exceptions raised by HTTPX.
4
5
  class Error < StandardError; end
5
6
 
6
7
  class UnsupportedSchemeError < Error; end
7
8
 
8
9
  class ConnectionError < Error; end
9
10
 
11
+ # Error raised when there was a timeout. Its subclasses allow for finer-grained
12
+ # control of which timeout happened.
10
13
  class TimeoutError < Error
14
+ # The timeout value which caused this error to be raised.
11
15
  attr_reader :timeout
12
16
 
17
+ # initializes the timeout exception with the +timeout+ causing the error, and the
18
+ # error +message+ for it.
13
19
  def initialize(timeout, message)
14
20
  @timeout = timeout
15
21
  super(message)
16
22
  end
17
23
 
24
+ # clones this error into a HTTPX::ConnectionTimeoutError.
18
25
  def to_connection_error
19
26
  ex = ConnectTimeoutError.new(@timeout, message)
20
27
  ex.set_backtrace(backtrace)
@@ -22,11 +29,19 @@ module HTTPX
22
29
  end
23
30
  end
24
31
 
32
+ # Error raised when there was a timeout establishing the connection to a server.
33
+ # This may be raised due to timeouts during TCP and TLS (when applicable) connection
34
+ # establishment.
25
35
  class ConnectTimeoutError < TimeoutError; end
26
36
 
37
+ # Error raised when there was a timeout while sending a request, or receiving a response
38
+ # from the server.
27
39
  class RequestTimeoutError < TimeoutError
40
+ # The HTTPX::Request request object this exception refers to.
28
41
  attr_reader :request
29
42
 
43
+ # initializes the exception with the +request+ and +response+ it refers to, and the
44
+ # +timeout+ causing the error, and the
30
45
  def initialize(request, response, timeout)
31
46
  @request = request
32
47
  @response = response
@@ -38,19 +53,28 @@ module HTTPX
38
53
  end
39
54
  end
40
55
 
56
+ # Error raised when there was a timeout while receiving a response from the server.
41
57
  class ReadTimeoutError < RequestTimeoutError; end
42
58
 
59
+ # Error raised when there was a timeout while sending a request from the server.
43
60
  class WriteTimeoutError < RequestTimeoutError; end
44
61
 
62
+ # Error raised when there was a timeout while waiting for the HTTP/2 settings frame from the server.
45
63
  class SettingsTimeoutError < TimeoutError; end
46
64
 
65
+ # Error raised when there was a timeout while resolving a domain to an IP.
47
66
  class ResolveTimeoutError < TimeoutError; end
48
67
 
68
+ # Error raised when there was an error while resolving a domain to an IP.
49
69
  class ResolveError < Error; end
50
70
 
71
+ # Error raised when there was an error while resolving a domain to an IP
72
+ # using a HTTPX::Resolver::Native resolver.
51
73
  class NativeResolveError < ResolveError
52
74
  attr_reader :connection, :host
53
75
 
76
+ # initializes the exception with the +connection+ it refers to, the +host+ domain
77
+ # which failed to resolve, and the error +message+.
54
78
  def initialize(connection, host, message = "Can't resolve #{host}")
55
79
  @connection = connection
56
80
  @host = host
@@ -58,18 +82,26 @@ module HTTPX
58
82
  end
59
83
  end
60
84
 
85
+ # The exception class for HTTP responses with 4xx or 5xx status.
61
86
  class HTTPError < Error
87
+ # The HTTPX::Response response object this exception refers to.
62
88
  attr_reader :response
63
89
 
90
+ # Creates the instance and assigns the HTTPX::Response +response+.
64
91
  def initialize(response)
65
92
  @response = response
66
93
  super("HTTP Error: #{@response.status} #{@response.headers}\n#{@response.body}")
67
94
  end
68
95
 
96
+ # The HTTP response status.
97
+ #
98
+ # error.status #=> 404
69
99
  def status
70
100
  @response.status
71
101
  end
72
102
  end
73
103
 
104
+ # error raised when a request was sent a server which can't reproduce a response, and
105
+ # has therefore returned an HTTP response using the 421 status code.
74
106
  class MisdirectedRequestError < HTTPError; end
75
107
  end
data/lib/httpx/io/ssl.rb CHANGED
@@ -114,7 +114,9 @@ module HTTPX
114
114
  end
115
115
 
116
116
  def try_ssl_connect
117
- case @io.connect_nonblock(exception: false)
117
+ ret = @io.connect_nonblock(exception: false)
118
+ log(level: 3, color: :cyan) { "TLS CONNECT: #{ret}..." }
119
+ case ret
118
120
  when :wait_readable
119
121
  @interests = :r
120
122
  return
data/lib/httpx/io/tcp.rb CHANGED
@@ -41,7 +41,7 @@ module HTTPX
41
41
  end
42
42
 
43
43
  def socket
44
- @io.to_io
44
+ @io
45
45
  end
46
46
 
47
47
  def add_addresses(addrs)
@@ -95,7 +95,9 @@ module HTTPX
95
95
  end
96
96
 
97
97
  def try_connect
98
- case @io.connect_nonblock(Socket.sockaddr_in(@port, @ip.to_s), exception: false)
98
+ ret = @io.connect_nonblock(Socket.sockaddr_in(@port, @ip.to_s), exception: false)
99
+ log(level: 3, color: :cyan) { "TCP CONNECT: #{ret}..." }
100
+ case ret
99
101
  when :wait_readable
100
102
  @interests = :r
101
103
  return
data/lib/httpx/io.rb CHANGED
@@ -4,4 +4,8 @@ require "socket"
4
4
  require "httpx/io/udp"
5
5
  require "httpx/io/tcp"
6
6
  require "httpx/io/unix"
7
- require "httpx/io/ssl"
7
+
8
+ begin
9
+ require "httpx/io/ssl"
10
+ rescue LoadError
11
+ end
data/lib/httpx/options.rb CHANGED
@@ -3,6 +3,8 @@
3
3
  require "socket"
4
4
 
5
5
  module HTTPX
6
+ # Contains a set of options which are passed and shared across from session to its requests or
7
+ # responses.
6
8
  class Options
7
9
  BUFFER_SIZE = 1 << 14
8
10
  WINDOW_SIZE = 1 << 14 # 16K
@@ -10,7 +12,7 @@ module HTTPX
10
12
  KEEP_ALIVE_TIMEOUT = 20
11
13
  SETTINGS_TIMEOUT = 10
12
14
  CONNECT_TIMEOUT = READ_TIMEOUT = WRITE_TIMEOUT = 60
13
- REQUEST_TIMEOUT = OPERATION_TIMEOUT = Float::INFINITY
15
+ REQUEST_TIMEOUT = OPERATION_TIMEOUT = nil
14
16
 
15
17
  # https://github.com/ruby/resolv/blob/095f1c003f6073730500f02acbdbc55f83d70987/lib/resolv.rb#L408
16
18
  ip_address_families = begin
@@ -25,6 +27,7 @@ module HTTPX
25
27
  end
26
28
 
27
29
  DEFAULT_OPTIONS = {
30
+ :max_requests => Float::INFINITY,
28
31
  :debug => ENV.key?("HTTPX_DEBUG") ? $stderr : nil,
29
32
  :debug_level => (ENV["HTTPX_DEBUG"] || 1).to_i,
30
33
  :ssl => {},
@@ -81,6 +84,50 @@ module HTTPX
81
84
  end
82
85
  end
83
86
 
87
+ # creates a new options instance from a given hash, which optionally define the following:
88
+ #
89
+ # :debug :: an object which log messages are written to (must respond to <tt><<</tt>)
90
+ # :debug_level :: the log level of messages (can be 1, 2, or 3).
91
+ # :ssl :: a hash of options which can be set as params of OpenSSL::SSL::SSLContext (see HTTPX::IO::SSL)
92
+ # :http2_settings :: a hash of options to be passed to a HTTP2Next::Connection (ex: <tt>{ max_concurrent_streams: 2 }</tt>)
93
+ # :fallback_protocol :: version of HTTP protocol to use by default in the absence of protocol negotiation
94
+ # like ALPN (defaults to <tt>"http/1.1"</tt>)
95
+ # :supported_compression_formats :: list of compressions supported by the transcoder layer (defaults to <tt>%w[gzip deflate]</tt>).
96
+ # :decompress_response_body :: whether to auto-decompress response body (defaults to <tt>true</tt>).
97
+ # :compress_request_body :: whether to auto-decompress response body (defaults to <tt>true</tt>)
98
+ # :timeout :: hash of timeout configurations (supports <tt>:connect_timeout</tt>, <tt>:settings_timeout</tt>,
99
+ # <tt>:operation_timeout</tt>, <tt>:keep_alive_timeout</tt>, <tt>:read_timeout</tt>, <tt>:write_timeout</tt>
100
+ # and <tt>:request_timeout</tt>
101
+ # :headers :: hash of HTTP headers (ex: <tt>{ "x-custom-foo" => "bar" }</tt>)
102
+ # :window_size :: number of bytes to read from a socket
103
+ # :buffer_size :: internal read and write buffer size in bytes
104
+ # :body_threshold_size :: maximum size in bytes of response payload that is buffered in memory.
105
+ # :request_class :: class used to instantiate a request
106
+ # :response_class :: class used to instantiate a response
107
+ # :headers_class :: class used to instantiate headers
108
+ # :request_body_class :: class used to instantiate a request body
109
+ # :response_body_class :: class used to instantiate a response body
110
+ # :connection_class :: class used to instantiate connections
111
+ # :options_class :: class used to instantiate options
112
+ # :transport :: type of transport to use (set to "unix" for UNIX sockets)
113
+ # :addresses :: bucket of peer addresses (can be a list of IP addresses, a hash of domain to list of adddresses;
114
+ # paths should be used for UNIX sockets instead)
115
+ # :io :: open socket, or domain/ip-to-socket hash, which requests should be sent to
116
+ # :persistent :: whether to persist connections in between requests (defaults to <tt>true</tt>)
117
+ # :resolver_class :: which resolver to use (defaults to <tt>:native</tt>, can also be <tt>:system<tt> for
118
+ # using getaddrinfo or <tt>:https</tt> for DoH resolver, or a custom class)
119
+ # :resolver_options :: hash of options passed to the resolver
120
+ # :ip_families :: which socket families are supported (system-dependent)
121
+ # :origin :: HTTP origin to set on requests with relative path (ex: "https://api.serv.com")
122
+ # :base_path :: path to prefix given relative paths with (ex: "/v2")
123
+ # :max_concurrent_requests :: max number of requests which can be set concurrently
124
+ # :max_requests :: max number of requests which can be made on socket before it reconnects.
125
+ # :params :: hash or array of key-values which will be encoded and set in the query string of request uris.
126
+ # :form :: hash of array of key-values which will be form-or-multipart-encoded in requests body payload.
127
+ # :json :: hash of array of key-values which will be JSON-encoded in requests body payload.
128
+ # :xml :: Nokogiri XML nodes which will be encoded in requests body payload.
129
+ #
130
+ # This list of options are enhanced with each loaded plugin, see the plugin docs for details.
84
131
  def initialize(options = {})
85
132
  do_initialize(options)
86
133
  freeze
@@ -75,14 +75,16 @@ module HTTPX
75
75
 
76
76
  return unless request.headers["expect"] == "100-continue"
77
77
 
78
- request.once(:expect) do
79
- @timers.after(request.options.expect_timeout) do
80
- # expect timeout expired
81
- if request.state == :expect && !request.expects?
82
- Expect.no_expect_store << request.origin
83
- request.headers.delete("expect")
84
- consume
85
- end
78
+ expect_timeout = request.options.expect_timeout
79
+
80
+ return if expect_timeout.nil? || expect_timeout.infinite?
81
+
82
+ set_request_timeout(request, expect_timeout, :expect, %i[body response]) do
83
+ # expect timeout expired
84
+ if request.state == :expect && !request.expects?
85
+ Expect.no_expect_store << request.origin
86
+ request.headers.delete("expect")
87
+ consume
86
88
  end
87
89
  end
88
90
  end
@@ -76,7 +76,6 @@ module HTTPX
76
76
  else
77
77
  transition(:closing)
78
78
  transition(:closed)
79
- emit(:reset)
80
79
 
81
80
  parser.reset if @parser
82
81
  transition(:idle)
data/lib/httpx/pool.rb CHANGED
@@ -52,7 +52,6 @@ module HTTPX
52
52
  def close(connections = @connections)
53
53
  return if connections.empty?
54
54
 
55
- @timers.cancel
56
55
  @eden_connections.clear
57
56
  connections = connections.reject(&:inflight?)
58
57
  connections.each(&:close)
@@ -224,9 +223,6 @@ module HTTPX
224
223
  @connected_connections += 1
225
224
  end
226
225
  select_connection(connection)
227
- connection.on(:close) do
228
- unregister_connection(connection)
229
- end
230
226
  end
231
227
 
232
228
  def unregister_connection(connection)