httpx 1.7.3 → 1.7.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 54c87d9d8b2be0d12570204fd3c60d37f82127b624dde3a233085d5bcb43778c
4
- data.tar.gz: 8bfec9fadfe697d083d37a9317e9ec9d6e23016f174bd62cabf24275f18fee79
3
+ metadata.gz: 125df778093197c9a1fe8872211cf4085a5a885a40536b58501ce1938f8d28ee
4
+ data.tar.gz: 5a17b5b02212f65ba3472f4dfda96a017fc1ef9a4fb7a3dce85d7b4ac6a8c6bb
5
5
  SHA512:
6
- metadata.gz: 2c425e8714c36bdca0ab5d068157caa94ca591829466ccdb888e903487aaab50c9a8756eaa53f1cd692a109160deca10ec9ce4405a7690f38149a9e95ec9fb17
7
- data.tar.gz: 20c6ae3067f1fe187402c743fb1741a131ba09e5039ed86e5c5d2eb8c94d590c84644ca3e6ab5fd3948b635d328bba4149df3a1d94dde3dfbf257bd694730275
6
+ metadata.gz: b6a4014db970d25a1ac51ea73ac5bd828bf8c0f1012ec778abdf054d9dd6dcfe6cf7fc5f73f915a39d74e4e420329e1293e020d63955601e6f669ae6164a016b
7
+ data.tar.gz: 96b98469fe6194bbd3427adfd1bfc6032be9c44f2c68ff301a6da20a90741b1979aa194c113a11e1b82cb9a54c3559ef7c35b908d41c10297746e438b557eca3
@@ -0,0 +1,42 @@
1
+ # 1.7.4
2
+
3
+ ## Features
4
+
5
+ ### Tracing plugin
6
+
7
+ A new `:tracing` plugin was introduced. It adds support for a new option, `:tracer`, which accepts an object which responds to the following callbacks:
8
+
9
+ * `#enabled?(request)` - should return true or false depending on whether tracing is enabled
10
+ * `#start(request)` - called when a request is about to be sent
11
+ * `#finish(request, response)` - called when a response is received
12
+ * `#reset(request)` - called when a request is being prepared to be resent, in cases where it makes sense (i.e. when a request is retried).
13
+
14
+ You can pass chain several tracers, and callbacks will be relayed to all of them:
15
+
16
+ ```ruby
17
+ HTTP.plugin(:tracing).with(tracer: telemetry_platform_tracer).with(tracer: telemetry2_platform_tracer)
18
+ ```
19
+
20
+ This was developed to be the foundation on top of which the datadag and OTel integrations will be built.
21
+
22
+ ## Improvements
23
+
24
+ * try fetching response immediately after send the request to the connection; this allows returning from errors much earlier and bug free than doing another round of waits on I/O.
25
+ * when a connection is reconnected, and it was established the first time that the peer can accept only 1 request at a time, the connection will keep that informaation and keep sending requests 1 at a time afterwards.
26
+
27
+ ## Bugfixes
28
+
29
+ * fix regression from introducing connection post state transition callbacks, by foregoing disconnect when there's pending backlog.
30
+ * transition requests to `:idle` before routing them to a different connection on merge (this could possibly leave dangling timeout callbacks otherwise).
31
+ * `:brotli` plugin was integrated with the stream writer component which allows writing compressed payload in chunks.
32
+ * `:brotli` plugin integrates with the `brotli` gem v0.8.0, which fixed an issue dealing with large payload responses due to the lack of support for decoding payloads in chunks.
33
+ * http1 parser: reset before early returning on `Upgrade` responses (it was left in an invalid "parsing headers", which in the case of a keep-alive connection, would cause the next request to fail being parsed).
34
+ * `datadog` adapter: fixed initialization of the request start time after connections were opened (it was being set to connection initialization time every time, instead of just on the first request before connection is established).
35
+ * parsers: also reroute non-completed in-flight requests back to the connection so they can be retried (previously, only pending requests were).
36
+ * `:proxy` plugin: do not try disconnecting unnecessarily when resetting may already do so (if conditions apply).
37
+ * `:proxy` plugin: removed call to unexisting `#reset!` function.
38
+ * `:proxy` plugin: also close wrapped sockets.
39
+ * connection: on force_close, move connection disconnection logic below so that, if requests are reenqueued from the parser, this can be halted.
40
+ * connection: when transition to `:idle`, reenqueue requests from parser before resetting it.
41
+ * implement `#lazy_resolve` on resolvers, as when they're picked from the selector (instead of from the pool), they may not be wrapped by a Multi proxy.
42
+ * allow resolvers transitioning from `:idle` to `:closed` and forego disconnecting when the resolver is not able to transition to `:closed` (which protects from a possible fiber scheduler context switch which changed the state under the hood).
@@ -46,35 +46,25 @@ module Datadog::Tracing
46
46
 
47
47
  SPAN_REQUEST = "httpx.request"
48
48
 
49
- # initializes tracing on the +request+.
50
- def call(request)
51
- return unless configuration(request).enabled
52
-
53
- span = nil
54
-
55
- # request objects are reused, when already buffered requests get rerouted to a different
56
- # connection due to connection issues, or when they already got a response, but need to
57
- # be retried. In such situations, the original span needs to be extended for the former,
58
- # while a new is required for the latter.
59
- request.on(:idle) do
60
- span = nil
61
- end
62
- # the span is initialized when the request is buffered in the parser, which is the closest
63
- # one gets to actually sending the request.
64
- request.on(:headers) do
65
- next if span
49
+ def enabled?(request)
50
+ configuration(request).enabled
51
+ end
66
52
 
67
- span = initialize_span(request, now)
68
- end
53
+ def start(request)
54
+ request.datadog_span = initialize_span(request, now)
55
+ end
56
+
57
+ def reset(request)
58
+ request.datadog_span = nil
59
+ end
69
60
 
70
- request.on(:response) do |response|
71
- span = initialize_span(request, request.init_time) if !span && request.init_time
61
+ def finish(request, response)
62
+ request.datadog_span ||= initialize_span(request, request.init_time) if request.init_time
72
63
 
73
- finish(response, span)
74
- end
64
+ finish_span(response, request.datadog_span)
75
65
  end
76
66
 
77
- def finish(response, span)
67
+ def finish_span(response, span)
78
68
  if response.is_a?(::HTTPX::ErrorResponse)
79
69
  span.set_error(response.error)
80
70
  else
@@ -137,9 +127,9 @@ module Datadog::Tracing
137
127
  ) if Datadog.configuration.tracing.respond_to?(:header_tags)
138
128
 
139
129
  span
140
- rescue StandardError => e
141
- Datadog.logger.error("error preparing span for http request: #{e}")
142
- Datadog.logger.error(e.backtrace)
130
+ rescue StandardError => e
131
+ Datadog.logger.error("error preparing span for http request: #{e}")
132
+ Datadog.logger.error(e.backtrace)
143
133
  end
144
134
 
145
135
  def now
@@ -179,44 +169,18 @@ module Datadog::Tracing
179
169
  end
180
170
  end
181
171
 
182
- module RequestMethods
183
- attr_accessor :init_time
184
-
185
- # intercepts request initialization to inject the tracing logic.
186
- def initialize(*)
187
- super
188
-
189
- @init_time = nil
190
-
191
- return unless Datadog::Tracing.enabled?
192
-
193
- RequestTracer.call(self)
172
+ class << self
173
+ def load_dependencies(klass)
174
+ klass.plugin(:tracing)
194
175
  end
195
176
 
196
- def response=(*)
197
- # init_time should be set when it's send to a connection.
198
- # However, there are situations where connection initialization fails.
199
- # Example is the :ssrf_filter plugin, which raises an error on
200
- # initialize if the host is an IP which matches against the known set.
201
- # in such cases, we'll just set here right here.
202
- @init_time ||= ::Datadog::Core::Utils::Time.now.utc
203
-
204
- super
177
+ def extra_options(options)
178
+ options.merge(tracer: RequestTracer)
205
179
  end
206
180
  end
207
181
 
208
- module ConnectionMethods
209
- def initialize(*)
210
- super
211
-
212
- @init_time = ::Datadog::Core::Utils::Time.now.utc
213
- end
214
-
215
- def send(request)
216
- request.init_time ||= @init_time
217
-
218
- super
219
- end
182
+ module RequestMethods
183
+ attr_accessor :datadog_span
220
184
  end
221
185
  end
222
186
 
@@ -49,7 +49,12 @@ module HTTPX
49
49
  @max_requests = @options.max_requests || MAX_REQUESTS
50
50
  @parser.reset!
51
51
  @handshake_completed = false
52
+ reset_requests
53
+ end
54
+
55
+ def reset_requests
52
56
  @pending.unshift(*@requests)
57
+ @requests.clear
53
58
  end
54
59
 
55
60
  def close
@@ -175,6 +180,7 @@ module HTTPX
175
180
 
176
181
  if @parser.upgrade?
177
182
  response << @parser.upgrade_data
183
+ @parser.reset!
178
184
  throw(:called)
179
185
  end
180
186
 
@@ -161,6 +161,8 @@ module HTTPX
161
161
  @pings.any?
162
162
  end
163
163
 
164
+ def reset_requests; end
165
+
164
166
  private
165
167
 
166
168
  def can_buffer_more_requests?
@@ -45,10 +45,10 @@ module HTTPX
45
45
  protected :ssl_session, :sibling
46
46
 
47
47
  def initialize(uri, options)
48
- @current_session = @current_selector =
49
- @parser = @sibling = @coalesced_connection = @altsvc_connection =
50
- @family = @io = @ssl_session = @timeout =
51
- @connected_at = @response_received_at = nil
48
+ @current_session = @current_selector = @max_concurrent_requests =
49
+ @parser = @sibling = @coalesced_connection = @altsvc_connection =
50
+ @family = @io = @ssl_session = @timeout =
51
+ @connected_at = @response_received_at = nil
52
52
 
53
53
  @exhausted = @cloned = @main_sibling = false
54
54
 
@@ -154,6 +154,7 @@ module HTTPX
154
154
  end if @io
155
155
  end
156
156
  connection.purge_pending do |req|
157
+ req.transition(:idle)
157
158
  send(req)
158
159
  end
159
160
  end
@@ -161,8 +162,9 @@ module HTTPX
161
162
  def purge_pending(&block)
162
163
  pendings = []
163
164
  if @parser
164
- @inflight -= @parser.pending.size
165
- pendings << @parser.pending
165
+ pending = @parser.pending
166
+ @inflight -= pending.size
167
+ pendings << pending
166
168
  end
167
169
  pendings << @pending
168
170
  pendings.each do |pending|
@@ -259,16 +261,17 @@ module HTTPX
259
261
  # bypasses state machine rules while setting the connection in the
260
262
  # :closed state.
261
263
  def force_close(delete_pending = false)
264
+ force_purge
265
+ return unless @state == :closed
266
+
262
267
  if delete_pending
263
268
  @pending.clear
264
269
  elsif (parser = @parser)
265
270
  enqueue_pending_requests_from_parser(parser)
266
271
  end
267
- return if @state == :closed
268
272
 
269
- @state = :closed
270
- @write_buffer.clear
271
- purge_after_closed
273
+ return unless @pending.empty?
274
+
272
275
  disconnect
273
276
  emit(:force_closed, delete_pending)
274
277
  end
@@ -284,6 +287,15 @@ module HTTPX
284
287
  def reset
285
288
  return if @state == :closing || @state == :closed
286
289
 
290
+ parser = @parser
291
+
292
+ if parser && parser.respond_to?(:max_concurrent_requests)
293
+ # if connection being reset has at some downgraded the number of concurrent
294
+ # requests, such as in the case where an attempt to use HTTP/1 pipelining failed,
295
+ # keep that information around.
296
+ @max_concurrent_requests = parser.max_concurrent_requests
297
+ end
298
+
287
299
  transition(:closing)
288
300
 
289
301
  transition(:closed)
@@ -326,7 +338,10 @@ module HTTPX
326
338
  purge_after_closed
327
339
  @write_buffer.clear
328
340
  transition(:idle)
329
- @parser = nil if @parser
341
+ return unless @parser
342
+
343
+ enqueue_pending_requests_from_parser(parser)
344
+ @parser = nil
330
345
  end
331
346
 
332
347
  def used?
@@ -378,6 +393,14 @@ module HTTPX
378
393
  current_session.deselect_connection(self, current_selector, @cloned)
379
394
  end
380
395
 
396
+ def on_connect_error(e)
397
+ # connect errors, exit gracefully
398
+ error = ConnectionError.new(e.message)
399
+ error.set_backtrace(e.backtrace)
400
+ handle_connect_error(error) if connecting?
401
+ force_close
402
+ end
403
+
381
404
  def on_io_error(e)
382
405
  on_error(e)
383
406
  force_close(true)
@@ -586,6 +609,7 @@ module HTTPX
586
609
  end
587
610
 
588
611
  def enqueue_pending_requests_from_parser(parser)
612
+ parser.reset_requests # move sequential requests back to pending queue.
589
613
  parser_pending_requests = parser.pending
590
614
 
591
615
  return if parser_pending_requests.empty?
@@ -601,6 +625,7 @@ module HTTPX
601
625
  def build_parser(protocol = @io.protocol)
602
626
  parser = parser_type(protocol).new(@write_buffer, @options)
603
627
  set_parser_callbacks(parser)
628
+ parser.max_concurrent_requests = @max_concurrent_requests if @max_concurrent_requests && parser.respond_to?(:max_concurrent_requests=)
604
629
  parser
605
630
  end
606
631
 
@@ -637,7 +662,6 @@ module HTTPX
637
662
  end
638
663
  parser.on(:close) do
639
664
  reset
640
- disconnect
641
665
  end
642
666
  parser.on(:close_handshake) do
643
667
  consume unless @state == :closed
@@ -694,11 +718,7 @@ module HTTPX
694
718
  Errno::ENOENT,
695
719
  SocketError,
696
720
  IOError => e
697
- # connect errors, exit gracefully
698
- error = ConnectionError.new(e.message)
699
- error.set_backtrace(e.backtrace)
700
- handle_connect_error(error) if connecting?
701
- force_close
721
+ on_connect_error(e)
702
722
  rescue TLSError, ::HTTP2::Error::ProtocolError, ::HTTP2::Error::HandshakeError => e
703
723
  # connect errors, exit gracefully
704
724
  handle_error(e)
@@ -729,6 +749,8 @@ module HTTPX
729
749
  when :inactive
730
750
  return unless @state == :open
731
751
 
752
+ # @type ivar @parser: HTTP1 | HTTP2
753
+
732
754
  # do not deactivate connection in use
733
755
  return if @inflight.positive? || @parser.waiting_for_ping?
734
756
  when :closing
@@ -748,9 +770,6 @@ module HTTPX
748
770
  return unless @write_buffer.empty?
749
771
 
750
772
  purge_after_closed
751
-
752
- # TODO: should this raise an error instead?
753
- return unless @pending.empty?
754
773
  when :already_open
755
774
  nextstate = :open
756
775
  # the first check for given io readiness must still use a timeout.
@@ -769,11 +788,30 @@ module HTTPX
769
788
  @state = nextstate
770
789
  # post state change
771
790
  case nextstate
772
- when :closed, :inactive
791
+ when :inactive
792
+ disconnect
793
+ when :closed
794
+ # TODO: should this raise an error instead?
795
+ return unless @pending.empty?
796
+
773
797
  disconnect
774
798
  end
775
799
  end
776
800
 
801
+ def force_purge
802
+ return if @state == :closed
803
+
804
+ @state = :closed
805
+ @write_buffer.clear
806
+ begin
807
+ purge_after_closed
808
+ rescue IOError
809
+ # may be raised when closing the socket.
810
+ # due to connection reuse / fiber scheduling, it may
811
+ # have been reopened, to bail out in that case.
812
+ end
813
+ end
814
+
777
815
  def close_sibling
778
816
  return unless @sibling
779
817
 
@@ -3,11 +3,33 @@
3
3
  module HTTPX
4
4
  module Plugins
5
5
  module Brotli
6
+ class Error < HTTPX::Error; end
7
+
6
8
  class Deflater < Transcoder::Deflater
9
+ def initialize(body)
10
+ @compressor = ::Brotli::Compressor.new
11
+ super
12
+ end
13
+
7
14
  def deflate(chunk)
8
- return unless chunk
15
+ return @compressor.process(chunk) << @compressor.flush if chunk
16
+
17
+ @compressor.finish
18
+ end
19
+ end
20
+
21
+ class Inflater
22
+ def initialize(bytesize)
23
+ @inflater = ::Brotli::Decompressor.new
24
+ @bytesize = bytesize
25
+ end
26
+
27
+ def call(chunk)
28
+ buffer = @inflater.process(chunk)
29
+ @bytesize -= chunk.bytesize
30
+ raise Error, "Unexpected end of compressed stream" if @bytesize <= 0 && !@inflater.finished?
9
31
 
10
- ::Brotli.deflate(chunk)
32
+ buffer
11
33
  end
12
34
  end
13
35
 
@@ -30,19 +52,25 @@ module HTTPX
30
52
  module_function
31
53
 
32
54
  def load_dependencies(*)
55
+ gem "brotli", ">= 0.8.0"
33
56
  require "brotli"
34
57
  end
35
58
 
36
59
  def self.extra_options(options)
37
- options.merge(supported_compression_formats: %w[br] + options.supported_compression_formats)
60
+ supported_compression_formats = (%w[br] + options.supported_compression_formats).freeze
61
+ options.merge(
62
+ supported_compression_formats: supported_compression_formats,
63
+ headers: options.headers_class.new(options.headers.merge("accept-encoding" => supported_compression_formats))
64
+ )
38
65
  end
39
66
 
40
67
  def encode(body)
41
68
  Deflater.new(body)
42
69
  end
43
70
 
44
- def decode(_response, **)
45
- ::Brotli.method(:inflate)
71
+ def decode(response, bytesize: nil)
72
+ bytesize ||= response.headers.key?("content-length") ? response.headers["content-length"].to_i : Float::INFINITY
73
+ Inflater.new(bytesize)
46
74
  end
47
75
  end
48
76
  register_plugin :brotli, Brotli
@@ -132,7 +132,10 @@ module HTTPX
132
132
  request.headers.delete("expect")
133
133
  request.transition(:idle)
134
134
  send_request(request, selector, options)
135
- return
135
+
136
+ # recalling itself, in case an error was triggered by the above, and we can
137
+ # verify retriability again.
138
+ return fetch_response(request, selector, options)
136
139
  end
137
140
 
138
141
  response
@@ -165,6 +165,10 @@ module HTTPX
165
165
  end
166
166
  else
167
167
  send_request(retry_request, selector, options)
168
+
169
+ # recalling itself, in case an error was triggered by the above, and we can
170
+ # verify retriability again.
171
+ return fetch_response(request, selector, options)
168
172
  end
169
173
  nil
170
174
  end
@@ -81,7 +81,7 @@ module HTTPX
81
81
  @parser.upgrade(request, response)
82
82
  @upgrade_protocol = "h2c"
83
83
 
84
- prev_parser.requests.each do |req|
84
+ prev_parser.pending.each do |req|
85
85
  req.transition(:idle)
86
86
  send(req)
87
87
  end
@@ -35,7 +35,10 @@ module HTTPX
35
35
  request.headers["proxy-authorization"] =
36
36
  options.proxy.authenticate(request, response.headers["proxy-authenticate"])
37
37
  send_request(request, selector, options)
38
- return
38
+
39
+ # recalling itself, in case an error was triggered by the above, and we can
40
+ # verify retriability again.
41
+ return fetch_response(request, selector, options)
39
42
  end
40
43
 
41
44
  response
@@ -46,7 +49,7 @@ module HTTPX
46
49
  def force_close(*)
47
50
  if @state == :connecting
48
51
  # proxy connect related requests should not be reenqueed
49
- @parser.reset!
52
+ @parser.reset
50
53
  @inflight -= @parser.pending.size
51
54
  @parser.pending.clear
52
55
  end
@@ -67,18 +70,16 @@ module HTTPX
67
70
  return unless @io.connected?
68
71
 
69
72
  @parser || begin
70
- @parser = parser_type(@io.protocol).new(@write_buffer, @options.merge(max_concurrent_requests: 1))
71
- parser = @parser
73
+ @parser = parser = parser_type(@io.protocol).new(@write_buffer, @options.merge(max_concurrent_requests: 1))
72
74
  parser.extend(ProxyParser)
73
75
  parser.on(:response, &method(:__http_on_connect))
74
76
  parser.on(:close) do
75
77
  next unless @parser
76
78
 
77
79
  reset
78
- disconnect
79
80
  end
80
81
  parser.on(:reset) do
81
- if parser.empty?
82
+ if parser.pending.empty? && parser.empty?
82
83
  reset
83
84
  else
84
85
  enqueue_pending_requests_from_parser(parser)
@@ -94,17 +95,23 @@ module HTTPX
94
95
  # keep parser state around due to proxy auth protocol;
95
96
  # intermediate authenticated request is already inside
96
97
  # the parser
97
- parser = nil
98
+ connect_request = parser = nil
98
99
 
99
100
  if initial_state == :connecting
100
101
  parser = @parser
101
102
  @parser.reset
103
+ if @pending.first.is_a?(ConnectRequest)
104
+ connect_request = @pending.shift # this happened when reenqueing
105
+ end
102
106
  end
103
107
 
104
108
  idling
105
109
 
106
110
  @parser = parser
107
-
111
+ if connect_request
112
+ @inflight += 1
113
+ parser.send(connect_request)
114
+ end
108
115
  transition(:connecting)
109
116
  end
110
117
  end
@@ -202,7 +202,10 @@ module HTTPX
202
202
  log { "failed connecting to proxy, trying next..." }
203
203
  request.transition(:idle)
204
204
  send_request(request, selector, options)
205
- return
205
+
206
+ # recalling itself, in case an error was triggered by the above, and we can
207
+ # verify retriability again.
208
+ return fetch_response(request, selector, options)
206
209
  end
207
210
  response
208
211
  rescue ProxyError
@@ -320,7 +323,12 @@ module HTTPX
320
323
 
321
324
  def purge_after_closed
322
325
  super
323
- @io = @io.proxy_io if @io.respond_to?(:proxy_io)
326
+
327
+ while @io.respond_to?(:proxy_io)
328
+ @io = @io.proxy_io
329
+
330
+ super
331
+ end
324
332
  end
325
333
  end
326
334
 
@@ -166,11 +166,15 @@ module HTTPX
166
166
  send_request(request, selector, options)
167
167
  end
168
168
  end
169
+
170
+ return
169
171
  else
170
172
  send_request(request, selector, options)
171
- end
172
173
 
173
- return
174
+ # recalling itself, in case an error was triggered by the above, and we can
175
+ # verify retriability again.
176
+ return fetch_response(request, selector, options)
177
+ end
174
178
  end
175
179
  response
176
180
  end
@@ -0,0 +1,137 @@
1
+ # frozen_string_literal: true
2
+
3
+ module HTTPX::Plugins
4
+ #
5
+ # This plugin adds a simple interface to integrate request tracing SDKs.
6
+ #
7
+ # An example of such an integration is the datadog adapter.
8
+ #
9
+ # https://gitlab.com/os85/httpx/wikis/Tracing
10
+ #
11
+ module Tracing
12
+ class Wrapper
13
+ attr_reader :tracers
14
+ protected :tracers
15
+
16
+ def initialize(*tracers)
17
+ @tracers = tracers.flat_map do |tracer|
18
+ case tracer
19
+ when Wrapper
20
+ tracer.tracers
21
+ else
22
+ tracer
23
+ end
24
+ end.uniq
25
+ end
26
+
27
+ def merge(tracer)
28
+ case tracer
29
+ when Wrapper
30
+ Wrapper.new(*@tracers, *tracer.tracers)
31
+ else
32
+ Wrapper.new(*@tracers, tracer)
33
+ end
34
+ end
35
+
36
+ def freeze
37
+ @tracers.each(&:freeze).freeze
38
+ super
39
+ end
40
+
41
+ %i[start finish reset enabled?].each do |callback|
42
+ class_eval(<<-OUT, __FILE__, __LINE__ + 1)
43
+ # proxies ##{callback} calls to wrapper tracers.
44
+ def #{callback}(*args) # def start(*args)
45
+ @tracers.each { |t| t.#{callback}(*args) } # @tracers.each { |t| t.start(*args) }
46
+ end # end
47
+ OUT
48
+ end
49
+ end
50
+
51
+ # adds support for the following options:
52
+ #
53
+ # :tracer :: object which responds to #start, #finish and #reset.
54
+ module OptionsMethods
55
+ private
56
+
57
+ def option_tracer(tracer)
58
+ unless tracer.respond_to?(:start) &&
59
+ tracer.respond_to?(:finish) &&
60
+ tracer.respond_to?(:reset) &&
61
+ tracer.respond_to?(:enabled?)
62
+ raise TypeError, "#{tracer} must to respond to `#start(r)`, `#finish` and `#reset` and `#enabled?"
63
+ end
64
+
65
+ tracer = Wrapper.new(@tracer, tracer) if @tracer
66
+ tracer
67
+ end
68
+ end
69
+
70
+ module RequestMethods
71
+ attr_accessor :init_time
72
+
73
+ # intercepts request initialization to inject the tracing logic.
74
+ def initialize(*)
75
+ super
76
+
77
+ @init_time = nil
78
+
79
+ tracer = @options.tracer
80
+
81
+ return unless tracer && tracer.enabled?(self)
82
+
83
+ on(:idle) do
84
+ tracer.reset(self)
85
+
86
+ # request is reset when it's retried.
87
+ @init_time = nil
88
+ end
89
+ on(:headers) do
90
+ # the usual request init time (when not including the connection handshake)
91
+ # should be the time the request is buffered the first time.
92
+ @init_time ||= ::Time.now
93
+
94
+ tracer.start(self)
95
+ end
96
+ on(:response) { |response| tracer.finish(self, response) }
97
+ end
98
+
99
+ def response=(*)
100
+ # init_time should be set when it's send to a connection.
101
+ # However, there are situations where connection initialization fails.
102
+ # Example is the :ssrf_filter plugin, which raises an error on
103
+ # initialize if the host is an IP which matches against the known set.
104
+ # in such cases, we'll just set here right here.
105
+ @init_time ||= ::Time.now
106
+
107
+ super
108
+ end
109
+ end
110
+
111
+ # Connection mixin
112
+ module ConnectionMethods
113
+ def initialize(*)
114
+ super
115
+
116
+ @init_time = ::Time.now
117
+ end
118
+
119
+ def send(request)
120
+ # request init time is only the same as the connection init time
121
+ # if the connection is going through the connection handshake.
122
+ request.init_time ||= @init_time unless open?
123
+
124
+ super
125
+ end
126
+
127
+ def idling
128
+ super
129
+
130
+ # time of initial request(s) is accounted from the moment
131
+ # the connection is back to :idle, and ready to connect again.
132
+ @init_time = ::Time.now
133
+ end
134
+ end
135
+ end
136
+ register_plugin :tracing, Tracing
137
+ end
@@ -71,14 +71,7 @@ module HTTPX
71
71
 
72
72
  def lazy_resolve(connection)
73
73
  @resolvers.each do |resolver|
74
- conn_to_resolve = @current_session.try_clone_connection(connection, @current_selector, resolver.family)
75
- resolver << conn_to_resolve
76
-
77
- next if resolver.empty?
78
-
79
- # both the resolver and the connection it's resolving must be pineed to the session
80
- @current_session.pin(conn_to_resolve, @current_selector)
81
- @current_session.select_resolver(resolver, @current_selector)
74
+ resolver.lazy_resolve(connection)
82
75
  end
83
76
  end
84
77
 
@@ -529,7 +529,7 @@ module HTTPX
529
529
 
530
530
  resolve if @queries.empty? && !@connections.empty?
531
531
  when :closed
532
- return unless @state == :open
532
+ return if @state == :closed
533
533
 
534
534
  @io.close if @io
535
535
  @start_timeout = nil
@@ -143,6 +143,19 @@ module HTTPX
143
143
  true
144
144
  end
145
145
 
146
+ def lazy_resolve(connection)
147
+ return unless @current_session && @current_selector
148
+
149
+ conn_to_resolve = @current_session.try_clone_connection(connection, @current_selector, @family)
150
+ self << conn_to_resolve
151
+
152
+ return if empty?
153
+
154
+ # both the resolver and the connection it's resolving must be pinned to the session
155
+ @current_session.pin(conn_to_resolve, @current_selector)
156
+ @current_session.select_resolver(self, @current_selector)
157
+ end
158
+
146
159
  private
147
160
 
148
161
  def emit_resolved_connection(connection, addresses, early_resolve)
@@ -186,9 +199,10 @@ module HTTPX
186
199
  end
187
200
 
188
201
  def disconnect
189
- return if closed?
190
-
191
202
  close
203
+
204
+ return unless closed?
205
+
192
206
  @current_session.deselect_resolver(self, @current_selector)
193
207
  end
194
208
  end
@@ -116,7 +116,9 @@ module HTTPX
116
116
  @current_session.select_resolver(self, @current_selector)
117
117
  end
118
118
 
119
- def early_resolve(connection, **); end
119
+ def early_resolve(_, **) # rubocop:disable Naming/PredicateMethod
120
+ false
121
+ end
120
122
 
121
123
  def handle_socket_timeout(interval)
122
124
  error = HTTPX::ResolveTimeoutError.new(interval, "timed out while waiting on select")
data/lib/httpx/session.rb CHANGED
@@ -392,7 +392,11 @@ module HTTPX
392
392
  resolver = find_resolver_for(connection, selector)
393
393
 
394
394
  pin(connection, selector)
395
- resolver.early_resolve(connection) || resolver.lazy_resolve(connection)
395
+ early_resolve(resolver, connection) || resolver.lazy_resolve(connection)
396
+ end
397
+
398
+ def early_resolve(resolver, connection)
399
+ resolver.early_resolve(connection)
396
400
  end
397
401
 
398
402
  def on_resolver_connection(connection, selector)
data/lib/httpx/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module HTTPX
4
- VERSION = "1.7.3"
4
+ VERSION = "1.7.4"
5
5
  end
data/sig/chainable.rbs CHANGED
@@ -43,6 +43,7 @@ module HTTPX
43
43
  | (:webdav, ?options) -> Plugins::sessionWebDav
44
44
  | (:xml, ?options) -> Plugins::sessionXML
45
45
  | (:query, ?options) -> Plugins::sessionQuery
46
+ | (:tracing, ?options) -> Plugins::sessionTracing
46
47
  | (Symbol | Module, ?options) { (Class) -> void } -> Session
47
48
  | (Symbol | Module, ?options) -> Session
48
49
 
@@ -26,6 +26,8 @@ module HTTPX
26
26
 
27
27
  def reset: () -> void
28
28
 
29
+ def reset_requests: () -> void
30
+
29
31
  def close: () -> void
30
32
 
31
33
  def empty?: () -> bool
@@ -6,6 +6,7 @@ module HTTPX
6
6
  MAX_CONCURRENT_REQUESTS: Integer
7
7
 
8
8
  attr_reader streams: Hash[Request, ::HTTP2::Stream]
9
+
9
10
  attr_reader pending: Array[Request]
10
11
 
11
12
  @connection: HTTP2::Client
@@ -43,6 +44,8 @@ module HTTPX
43
44
 
44
45
  def timeout: () -> Numeric?
45
46
 
47
+ def reset_requests: () -> void
48
+
46
49
  private
47
50
 
48
51
  def initialize: (Buffer buffer, Options options) -> untyped
data/sig/connection.rbs CHANGED
@@ -122,6 +122,8 @@ module HTTPX
122
122
 
123
123
  def on_io_error: (IOError error) -> void
124
124
 
125
+ def on_connect_error: (Exception error) -> void
126
+
125
127
  private
126
128
 
127
129
  def initialize: (http_uri uri, Options options) -> void
@@ -156,6 +158,8 @@ module HTTPX
156
158
 
157
159
  def handle_error: (StandardError error, ?Request? request) -> void
158
160
 
161
+ def force_purge: () -> void
162
+
159
163
  def close_sibling: () -> void
160
164
 
161
165
  def purge_after_closed: () -> void
@@ -5,17 +5,22 @@ module HTTPX
5
5
 
6
6
  def self?.encode: (body_encoder body) -> Deflater
7
7
 
8
- def self?.decode: (HTTPX::Response response, ?bytesize: Integer) -> Transcoder::_Decoder
8
+ def self?.decode: (HTTPX::Response response, ?bytesize: Integer | Float) -> Inflater
9
9
 
10
- class Deflater < Transcoder::Deflater
10
+ class Error < ::HTTPX::Error
11
11
  end
12
12
 
13
- module RequestBodyClassMethods
14
- def initialize_deflater_body: (body_encoder body, Encoding | String encoding) -> body_encoder
13
+ class Deflater < Transcoder::Deflater
14
+ @compressor: ::Brotli::Compressor
15
15
  end
16
16
 
17
- module ResponseBodyClassMethods
18
- def initialize_inflater_by_encoding: (Encoding | String encoding, Response response, ?bytesize: Integer) -> (Transcoder::_Decoder | Transcoder::GZIP::Inflater)
17
+ class Inflater
18
+ @inflater: ::Brotli::Decompressor
19
+ @bytesize: Integer | Float
20
+
21
+ def initialize: (Integer | Float bytesize) -> void
22
+
23
+ def call: (String chunk) -> String
19
24
  end
20
25
  end
21
26
  end
@@ -0,0 +1,41 @@
1
+ module HTTPX
2
+ module Plugins
3
+ module Tracing
4
+ interface _Tracer
5
+ def enabled?: (retriesRequest request) -> boolish
6
+
7
+ def start: (retriesRequest request) -> void
8
+
9
+ def reset: (retriesRequest request) -> void
10
+
11
+ def finish: (retriesRequest request, response response) -> void
12
+ end
13
+
14
+ class Wrapper
15
+ include _Tracer
16
+
17
+ attr_reader tracers: Array[_Tracer]
18
+
19
+ def initialize: (*_Tracer tracers) -> void
20
+
21
+ def merge: (instance | _Tracer) -> void
22
+ end
23
+
24
+ interface _RetriesOptions
25
+ def tracer: () -> _Tracer
26
+ end
27
+
28
+ module RequestMethods
29
+ attr_accessor init_time: Time?
30
+ end
31
+
32
+ module ConnectionMethods
33
+ @init_time: Time
34
+ end
35
+
36
+ type retriesRequest = Request & RequestMethods
37
+ end
38
+
39
+ type sessionTracing = Session
40
+ end
41
+ end
@@ -41,6 +41,8 @@ module HTTPX
41
41
 
42
42
  def early_resolve: (Connection connection, ?hostname: String) -> bool
43
43
 
44
+ def lazy_resolve: (Connection connection) -> void
45
+
44
46
  private
45
47
 
46
48
  def resolve: (?Connection connection, ?String hostname) -> void
@@ -18,8 +18,6 @@ module HTTPX
18
18
 
19
19
  def initialize: (Options options) -> void
20
20
 
21
- def lazy_resolve: (Connection connection) -> void
22
-
23
21
  private
24
22
 
25
23
  def transition: (Symbol nextstate) -> void
@@ -43,7 +43,7 @@ module HTTPX
43
43
 
44
44
  def initialize_inflaters: () -> void
45
45
 
46
- def self.initialize_inflater_by_encoding: (Encoding | String encoding, Response response, ?bytesize: Integer) -> Transcoder::GZIP::Inflater
46
+ def self.initialize_inflater_by_encoding: (Encoding | String encoding, Response response, ?bytesize: Integer) -> (Object & Transcoder::_Inflater)
47
47
 
48
48
  def decode_chunk: (String chunk) -> String
49
49
 
data/sig/session.rbs CHANGED
@@ -66,6 +66,8 @@ module HTTPX
66
66
 
67
67
  def receive_requests: (Array[Request] requests, Selector selector) -> Array[response]
68
68
 
69
+ def early_resolve: (resolver resolver, Connection connection) -> bool
70
+
69
71
  def resolve_connection: (Connection connection, Selector selector) -> void
70
72
 
71
73
  def on_resolver_connection: (Connection connection, Selector selector) -> void
@@ -16,7 +16,7 @@ module HTTPX
16
16
 
17
17
  class Inflater
18
18
  @inflater: Zlib::Inflate
19
- @bytesize: Integer
19
+ @bytesize: Integer | Float
20
20
 
21
21
  def initialize: (Integer | Float bytesize) -> void
22
22
 
data/sig/transcoder.rbs CHANGED
@@ -31,8 +31,6 @@ module HTTPX
31
31
  end
32
32
 
33
33
  interface _Inflater
34
- def initialize: (Integer | Float bytesize) -> void
35
-
36
34
  def call: (String chunk) -> String
37
35
  end
38
36
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: httpx
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.7.3
4
+ version: 1.7.4
5
5
  platform: ruby
6
6
  authors:
7
7
  - Tiago Cardoso
@@ -163,6 +163,7 @@ extra_rdoc_files:
163
163
  - doc/release_notes/1_7_1.md
164
164
  - doc/release_notes/1_7_2.md
165
165
  - doc/release_notes/1_7_3.md
166
+ - doc/release_notes/1_7_4.md
166
167
  files:
167
168
  - LICENSE.txt
168
169
  - README.md
@@ -298,6 +299,7 @@ files:
298
299
  - doc/release_notes/1_7_1.md
299
300
  - doc/release_notes/1_7_2.md
300
301
  - doc/release_notes/1_7_3.md
302
+ - doc/release_notes/1_7_4.md
301
303
  - lib/httpx.rb
302
304
  - lib/httpx/adapters/datadog.rb
303
305
  - lib/httpx/adapters/faraday.rb
@@ -369,6 +371,7 @@ files:
369
371
  - lib/httpx/plugins/ssrf_filter.rb
370
372
  - lib/httpx/plugins/stream.rb
371
373
  - lib/httpx/plugins/stream_bidi.rb
374
+ - lib/httpx/plugins/tracing.rb
372
375
  - lib/httpx/plugins/upgrade.rb
373
376
  - lib/httpx/plugins/upgrade/h2.rb
374
377
  - lib/httpx/plugins/webdav.rb
@@ -474,6 +477,7 @@ files:
474
477
  - sig/plugins/ssrf_filter.rbs
475
478
  - sig/plugins/stream.rbs
476
479
  - sig/plugins/stream_bidi.rbs
480
+ - sig/plugins/tracing.rbs
477
481
  - sig/plugins/upgrade.rbs
478
482
  - sig/plugins/upgrade/h2.rbs
479
483
  - sig/plugins/webdav.rbs