excon 1.8.0 → 1.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d7a177488f1722e063f04d9d2e0f82eb6b06c29febdc7823ac09006257c84ef3
4
- data.tar.gz: baf57fb9742edd1fc5e28cfa272a60a59bd9d476e53f3a753e0b9537ede6c7b3
3
+ metadata.gz: a8f339eac15303582618d7c092e0c31facc11e717477d11676ddf1cece1f4df8
4
+ data.tar.gz: f1225a0e544fb64eed9411d5f0c5028103a853e07acfa5df5b9c8d1057101297
5
5
  SHA512:
6
- metadata.gz: b378a72d6a9c703e9150adb6f6d18e2bc5045f4c273b7caec53948ab301f03a5df9142f0f6dff6b93b38cddd51403ed1a1acd575e18dc76eb172d3a71d931970
7
- data.tar.gz: d1699bfde750945682253c74f5763168ef605aff5bd4a6cebb37be7e9af7051f833fd42eb22cebd9651d2df3a1dc7c6ed8884241812602f0e26ed1f0511d1bb0
6
+ metadata.gz: 37951c154ffa841570e55ec4bd9bd6894ccd4617cbf51f03e2606d341defde41e114d85b78af7cb67ffc9f46ba4fb8ca59398dcd219f4aeb0267ee79f5a20649
7
+ data.tar.gz: bf5f8bb104fcdcc6fe87d8f9a221775214af61b81df535f603ff1c1c47b44b5bfd441aef1bb35ede8761cdb2badd5ff52be93ecd2a3639a4809cf0fc3371f383
data/CONTRIBUTING.md CHANGED
@@ -9,6 +9,7 @@ New contributors are always welcome, when it doubt please ask questions. We stri
9
9
  * Review open [issues](https://github.com/excon/excon/issues) for things to help on.
10
10
  * [Create an issue](https://github.com/excon/excon/issues/new) to start a discussion on additions or features.
11
11
  * Fork the project, add your changes and tests to cover them in a topic branch.
12
+ * Tests go in `test/` (minitest), fixtures in `test/fixtures/`. See [AGENTS.md](AGENTS.md#tests).
12
13
  * Commit your changes and rebase against `excon/excon` to ensure everything is up to date.
13
14
  * [Submit a pull request](https://github.com/excon/excon/compare/).
14
15
 
data/README.md CHANGED
@@ -158,6 +158,8 @@ connection = Excon.new('https://...')
158
158
 
159
159
  You can modify timeouts and define whether and how many (blocking) retries Excon should attempt if errors occur.
160
160
 
161
+ Excon only retries requests marked `:idempotent`. Otherwise `:retry_limit`, `:retry_interval` and `:retry_errors` have no effect, whether they are set on the request, the connection or `Excon.defaults`.
162
+
161
163
  ```ruby
162
164
  # this request can be repeated safely, so retry on errors up to 4 times
163
165
  connection.request(:idempotent => true)
@@ -466,7 +468,7 @@ The #instrument method will be called for each HTTP request, response, retry, an
466
468
 
467
469
  For debugging purposes you can also use `Excon::StandardInstrumentor` to output all events to stderr. This can also be specified by setting the `EXCON_DEBUG` ENV var.
468
470
 
469
- See [the documentation for ActiveSupport::Notifications](http://api.rubyonrails.org/classes/ActiveSupport/Notifications.html) for more detail on using the subscription interface. See excon's [instrumentation_test.rb](https://github.com/excon/excon/blob/master/tests/middlewares/instrumentation_tests.rb) for more examples of instrumenting excon.
471
+ See [the documentation for ActiveSupport::Notifications](http://api.rubyonrails.org/classes/ActiveSupport/Notifications.html) for more detail on using the subscription interface. See excon's [instrumentation_test.rb](https://github.com/excon/excon/blob/master/test/middlewares/instrumentation_test.rb) for more examples of instrumenting excon.
470
472
 
471
473
  ## HTTPS client certificate
472
474
 
data/excon.gemspec CHANGED
@@ -23,17 +23,9 @@ Gem::Specification.new do |s|
23
23
  ]
24
24
  s.required_ruby_version = '>= 3.3.0'
25
25
 
26
- s.add_dependency 'logger'
27
-
28
- s.add_development_dependency('activesupport')
29
- s.add_development_dependency('delorean')
26
+ s.add_development_dependency('minitest', '>= 6')
30
27
  s.add_development_dependency('puma')
31
28
  s.add_development_dependency('rake')
32
- s.add_development_dependency('rdoc')
33
- s.add_development_dependency('rspec', '>= 3.5.0')
34
- s.add_development_dependency('shindo')
35
- s.add_development_dependency('sinatra')
36
- s.add_development_dependency('sinatra-contrib')
37
29
  s.add_development_dependency('webrick')
38
30
 
39
31
  s.metadata = {
@@ -56,14 +56,15 @@ module Excon
56
56
  # @option params [String] :socket The path to the unix socket (required for 'unix://' connections)
57
57
  # @option params [String] :ciphers Only use the specified SSL/TLS cipher suites; use OpenSSL cipher spec format e.g. 'HIGH:!aNULL:!3DES' or 'AES256-SHA:DES-CBC3-SHA'
58
58
  # @option params [String] :proxy Proxy server; e.g. 'http://myproxy.com:8888'
59
- # @option params [Fixnum] :retry_limit Set how many times we'll retry a failed request. (Default 4)
60
- # @option params [Fixnum] :retry_interval Set how long to wait between retries. (Default 0)
59
+ # @option params [Fixnum] :retry_limit Set how many times we'll retry a failed request. (Default 4) Only applies to :idempotent requests.
60
+ # @option params [Fixnum] :retry_interval Set how long to wait between retries. (Default 0) Only applies to :idempotent requests.
61
61
  # @option params [Class] :instrumentor Responds to #instrument as in ActiveSupport::Notifications
62
62
  # @option params [String] :instrumentor_name Name prefix for #instrument events. Defaults to 'excon'
63
63
  # @option params [Resolv, nil] :resolv_resolver A ready to use +Resolv+ resolver instance.
64
64
  # @option params [Class] :resolver_factory Const of the resolver factory. Defaults to 'Excon::ResolverFactory'
65
65
  def initialize(params = {})
66
66
  @pid = Process.pid
67
+ @sockets_mutex = Mutex.new
67
68
  @data = Excon.defaults.dup
68
69
  # merge does not deep-dup, so make sure headers is not the original
69
70
  @data[:headers] = @data[:headers].dup
@@ -233,6 +234,12 @@ module Excon
233
234
  # @option params [String] :path appears after 'scheme://host:port/'
234
235
  # @option params [Hash] :query appended to the 'scheme://host:port/path/' in the form of '?key=value'
235
236
  def request(params={}, &block)
237
+ # Set once the socket is left fit for the next request. Leaving early
238
+ # without passing through the rescue below (Timeout.timeout,
239
+ # Thread#raise, Thread#kill, Interrupt, throw) can leave a request
240
+ # half-written or a response unread on it, so ensure then resets it.
241
+ socket_reusable = false
242
+
236
243
  # @data has defaults, merge in new params to override
237
244
  datum = @data.merge(params)
238
245
 
@@ -294,11 +301,6 @@ module Excon
294
301
 
295
302
  datum[:connection] = self
296
303
 
297
- # cleanup data left behind on persistent connection after interrupt
298
- if datum[:persistent] && !@persistent_socket_reusable
299
- reset
300
- end
301
-
302
304
  datum[:stack] = datum[:middlewares].map do |middleware|
303
305
  lambda {|stack| middleware.new(stack)}
304
306
  end.reverse.inject(self) do |middlewares, middleware|
@@ -307,9 +309,7 @@ module Excon
307
309
  datum = datum[:stack].request_call(datum)
308
310
 
309
311
  unless datum[:pipeline]
310
- @persistent_socket_reusable = false
311
312
  datum = response(datum)
312
- @persistent_socket_reusable = true
313
313
 
314
314
  if datum[:persistent]
315
315
  if (key = datum[:response][:headers].keys.detect {|k| k.casecmp?('Connection') })
@@ -321,12 +321,16 @@ module Excon
321
321
  reset
322
322
  end
323
323
 
324
+ socket_reusable = true
324
325
  Excon::Response.new(datum[:response])
325
326
  else
327
+ # the response is read later, by #requests
328
+ socket_reusable = true
326
329
  datum
327
330
  end
328
331
  rescue => error
329
332
  reset
333
+ socket_reusable = true
330
334
 
331
335
  # If we didn't get far enough to initialize datum and the middleware stack, just raise
332
336
  raise error if !datum
@@ -337,11 +341,16 @@ module Excon
337
341
  else
338
342
  raise error
339
343
  end
344
+ ensure
345
+ reset unless socket_reusable
340
346
  end
341
347
 
342
348
  # Sends the supplied requests to the destination host using pipelining.
343
349
  # @param pipeline_params [Array<Hash>] An array of one or more optional params, override defaults set in Connection.new, see #request for details
344
350
  def requests(pipeline_params)
351
+ # as in #request: reset unless every response was read
352
+ socket_reusable = false
353
+
345
354
  pipeline_params.each {|params| params.merge!(:pipeline => true, :persistent => true) }
346
355
  pipeline_params.last[:persistent] = @data[:persistent]
347
356
 
@@ -361,7 +370,10 @@ module Excon
361
370
  reset
362
371
  end
363
372
 
373
+ socket_reusable = true
364
374
  responses
375
+ ensure
376
+ reset unless socket_reusable
365
377
  end
366
378
 
367
379
  # Sends the supplied requests to the destination host using pipelining in
@@ -383,7 +395,6 @@ module Excon
383
395
  if (old_socket = sockets.delete(@socket_key))
384
396
  old_socket.close rescue nil
385
397
  end
386
- @persistent_socket_reusable = true
387
398
  end
388
399
 
389
400
  # Generate HTTP request verb methods
@@ -512,21 +523,26 @@ module Excon
512
523
  end
513
524
 
514
525
  def sockets
515
- if @pid != Process.pid
516
- @_excon_sockets&.clear # GC will take care of closing sockets
517
- @pid = Process.pid
518
- end
526
+ # Threads sharing this connection all read and write @_excon_sockets.
527
+ # On a Ruby without a GVL (JRuby) concurrent updates can lose another
528
+ # thread's entry, so that thread opens a new socket mid-request.
529
+ @sockets_mutex.synchronize do
530
+ if @pid != Process.pid
531
+ @_excon_sockets&.clear # GC will take care of closing sockets
532
+ @pid = Process.pid
533
+ end
519
534
 
520
- if @data[:thread_safe_sockets]
521
- @_excon_sockets ||= ObjectSpace::WeakKeyMap.new
535
+ if @data[:thread_safe_sockets]
536
+ @_excon_sockets ||= ObjectSpace::WeakKeyMap.new
522
537
 
523
- # In a multi-threaded world, if the same connection is used by multiple
524
- # threads at the same time to connect to the same destination, they may
525
- # stomp on each other's sockets. This ensures every thread gets their
526
- # own socket cache, within the context of a single connection.
527
- @_excon_sockets[Thread.current] ||= {}
528
- else
529
- @_excon_sockets ||= {}.compare_by_identity
538
+ # In a multi-threaded world, if the same connection is used by multiple
539
+ # threads at the same time to connect to the same destination, they may
540
+ # stomp on each other's sockets. This ensures every thread gets their
541
+ # own socket cache, within the context of a single connection.
542
+ @_excon_sockets[Thread.current] ||= {}
543
+ else
544
+ @_excon_sockets ||= {}.compare_by_identity
545
+ end
530
546
  end
531
547
  end
532
548
 
@@ -545,8 +561,10 @@ module Excon
545
561
  rescue IPAddr::Error
546
562
  false
547
563
  end
564
+ elsif host == '*'
565
+ true
548
566
  else
549
- /(^|\.)#{host}$/.match(@data[:host])
567
+ /(^|\.)#{Regexp.escape(host)}$/.match(@data[:host])
550
568
  end
551
569
  host_match && (port.nil? || port.to_i == @data[:port])
552
570
  end
@@ -1,12 +1,19 @@
1
- require 'logger'
2
-
3
1
  module Excon
4
2
  class LoggingInstrumentor
5
3
 
4
+ # Stand-in for Logger.new($stderr), matching Logger's default format, so
5
+ # excon does not need to depend on the logger gem.
6
+ module StderrLogger
7
+ def self.info(message)
8
+ timestamp = Time.now.strftime('%Y-%m-%dT%H:%M:%S.%6N')
9
+ $stderr.write("I, [#{timestamp} ##{Process.pid}] INFO -- : #{message}\n")
10
+ end
11
+ end
12
+
6
13
  def self.instrument(name, params = {})
7
14
  params = params.dup
8
15
 
9
- logger = params[:logger] || Logger.new($stderr)
16
+ logger = params[:logger] || StderrLogger
10
17
 
11
18
  # reduce duplication/noise of output
12
19
  params.delete(:connection)
@@ -2,6 +2,14 @@
2
2
  module Excon
3
3
  module Middleware
4
4
  class RedirectFollower < Excon::Middleware::Base
5
+ CONTENT_HEADERS = %w[
6
+ Content-Encoding
7
+ Content-Language
8
+ Content-Length
9
+ Content-Location
10
+ Content-Type
11
+ ].freeze
12
+
5
13
  REDACTION_HEADERS = %w[
6
14
  Authorization
7
15
  Cookie
@@ -39,6 +47,10 @@ module Excon
39
47
  if datum.has_key?(:response)
40
48
  case datum[:response][:status]
41
49
  when 301, 302, 303, 307, 308
50
+ location = get_header(datum, 'Location')
51
+ # Without a Location there is nowhere to go, so return the response as is
52
+ return @stack.response_call(datum) unless location
53
+
42
54
  if datum[:redirects_remaining] <= 0
43
55
  raise Excon::Errors::TooManyRedirects
44
56
  end
@@ -47,7 +59,6 @@ module Excon
47
59
 
48
60
  uri_parser = datum[:uri_parser] || Excon.defaults[:uri_parser]
49
61
 
50
- location = get_header(datum, 'Location')
51
62
  base_uri = Excon::Utils.request_uri(datum)
52
63
  uri = uri_parser.join(base_uri, location)
53
64
 
@@ -61,14 +72,18 @@ module Excon
61
72
  params.delete(:stack)
62
73
  params.delete(:user)
63
74
 
75
+ params[:headers] = datum[:headers].dup
64
76
  if [301, 302, 303].include?(response[:status])
65
77
  params[:method] = :get
66
78
  params.delete(:body)
67
- params[:headers].delete('Content-Length')
79
+ # The body is not sent again, so neither are the header fields describing it
80
+ params[:headers].delete_if do |key, _|
81
+ CONTENT_HEADERS.any? { |header| key.to_s.casecmp?(header) }
82
+ end
68
83
  end
69
- params[:headers] = datum[:headers].dup
70
- REDACTION_HEADERS.each do |header|
71
- params[:headers].delete(header)
84
+ # drop sensitive headers in any casing
85
+ params[:headers].delete_if do |key, _|
86
+ REDACTION_HEADERS.any? { |header| key.to_s.casecmp?(header) }
72
87
  end
73
88
  params.merge!(
74
89
  :scheme => uri.scheme || datum[:scheme],
data/lib/excon/socket.rb CHANGED
@@ -174,6 +174,10 @@ module Excon
174
174
  rescue *CONNECT_RETRY_EXCEPTION_CLASSES
175
175
  select_with_timeout(socket, :connect_write)
176
176
  begin
177
+ # macOS 27 raises EISCONN below even when the connect failed, so check SO_ERROR first
178
+ errno = socket.getsockopt(::Socket::SOL_SOCKET, ::Socket::SO_ERROR).int
179
+ raise(SystemCallError.new("connect(2) for #{addrinfo.inspect_sockaddr}", errno)) unless errno.zero?
180
+
177
181
  socket.connect_nonblock(sockaddr)
178
182
  @socket = socket
179
183
  rescue Errno::EISCONN
data/lib/excon/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Excon
4
- VERSION = "1.8.0"
4
+ VERSION = "1.10.0"
5
5
  end
data/lib/excon.rb CHANGED
@@ -139,6 +139,9 @@ module Excon
139
139
  # @param request_params [Hash<Symbol, >] request params to match against, omitted params match all
140
140
  # @param response_params [Hash<Symbol, >] response params to return or block to call with matched params
141
141
  def stub(request_params = {}, response_params = nil, &block)
142
+ # copy, so later changes to the caller's hash (e.g. stub_for adding
143
+ # :captures) don't change the stored stub
144
+ request_params = request_params.dup
142
145
  if (method = request_params.delete(:method))
143
146
  request_params[:method] = method.to_s.downcase.to_sym
144
147
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: excon
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.8.0
4
+ version: 1.10.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - dpiddy (Dan Peterson)
@@ -12,47 +12,19 @@ cert_chain: []
12
12
  date: 1980-01-02 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
- name: logger
15
+ name: minitest
16
16
  requirement: !ruby/object:Gem::Requirement
17
17
  requirements:
18
18
  - - ">="
19
19
  - !ruby/object:Gem::Version
20
- version: '0'
21
- type: :runtime
22
- prerelease: false
23
- version_requirements: !ruby/object:Gem::Requirement
24
- requirements:
25
- - - ">="
26
- - !ruby/object:Gem::Version
27
- version: '0'
28
- - !ruby/object:Gem::Dependency
29
- name: activesupport
30
- requirement: !ruby/object:Gem::Requirement
31
- requirements:
32
- - - ">="
33
- - !ruby/object:Gem::Version
34
- version: '0'
20
+ version: '6'
35
21
  type: :development
36
22
  prerelease: false
37
23
  version_requirements: !ruby/object:Gem::Requirement
38
24
  requirements:
39
25
  - - ">="
40
26
  - !ruby/object:Gem::Version
41
- version: '0'
42
- - !ruby/object:Gem::Dependency
43
- name: delorean
44
- requirement: !ruby/object:Gem::Requirement
45
- requirements:
46
- - - ">="
47
- - !ruby/object:Gem::Version
48
- version: '0'
49
- type: :development
50
- prerelease: false
51
- version_requirements: !ruby/object:Gem::Requirement
52
- requirements:
53
- - - ">="
54
- - !ruby/object:Gem::Version
55
- version: '0'
27
+ version: '6'
56
28
  - !ruby/object:Gem::Dependency
57
29
  name: puma
58
30
  requirement: !ruby/object:Gem::Requirement
@@ -81,76 +53,6 @@ dependencies:
81
53
  - - ">="
82
54
  - !ruby/object:Gem::Version
83
55
  version: '0'
84
- - !ruby/object:Gem::Dependency
85
- name: rdoc
86
- requirement: !ruby/object:Gem::Requirement
87
- requirements:
88
- - - ">="
89
- - !ruby/object:Gem::Version
90
- version: '0'
91
- type: :development
92
- prerelease: false
93
- version_requirements: !ruby/object:Gem::Requirement
94
- requirements:
95
- - - ">="
96
- - !ruby/object:Gem::Version
97
- version: '0'
98
- - !ruby/object:Gem::Dependency
99
- name: rspec
100
- requirement: !ruby/object:Gem::Requirement
101
- requirements:
102
- - - ">="
103
- - !ruby/object:Gem::Version
104
- version: 3.5.0
105
- type: :development
106
- prerelease: false
107
- version_requirements: !ruby/object:Gem::Requirement
108
- requirements:
109
- - - ">="
110
- - !ruby/object:Gem::Version
111
- version: 3.5.0
112
- - !ruby/object:Gem::Dependency
113
- name: shindo
114
- requirement: !ruby/object:Gem::Requirement
115
- requirements:
116
- - - ">="
117
- - !ruby/object:Gem::Version
118
- version: '0'
119
- type: :development
120
- prerelease: false
121
- version_requirements: !ruby/object:Gem::Requirement
122
- requirements:
123
- - - ">="
124
- - !ruby/object:Gem::Version
125
- version: '0'
126
- - !ruby/object:Gem::Dependency
127
- name: sinatra
128
- requirement: !ruby/object:Gem::Requirement
129
- requirements:
130
- - - ">="
131
- - !ruby/object:Gem::Version
132
- version: '0'
133
- type: :development
134
- prerelease: false
135
- version_requirements: !ruby/object:Gem::Requirement
136
- requirements:
137
- - - ">="
138
- - !ruby/object:Gem::Version
139
- version: '0'
140
- - !ruby/object:Gem::Dependency
141
- name: sinatra-contrib
142
- requirement: !ruby/object:Gem::Requirement
143
- requirements:
144
- - - ">="
145
- - !ruby/object:Gem::Version
146
- version: '0'
147
- type: :development
148
- prerelease: false
149
- version_requirements: !ruby/object:Gem::Requirement
150
- requirements:
151
- - - ">="
152
- - !ruby/object:Gem::Version
153
- version: '0'
154
56
  - !ruby/object:Gem::Dependency
155
57
  name: webrick
156
58
  requirement: !ruby/object:Gem::Requirement