softwaregravy-net-http-persistent 2.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.autotest +7 -0
- data/.gemtest +0 -0
- data/History.txt +257 -0
- data/Manifest.txt +11 -0
- data/README.rdoc +82 -0
- data/Rakefile +20 -0
- data/lib/net/http/faster.rb +27 -0
- data/lib/net/http/persistent.rb +998 -0
- data/lib/net/http/persistent/ssl_reuse.rb +129 -0
- data/test/test_net_http_persistent.rb +1302 -0
- data/test/test_net_http_persistent_ssl_reuse.rb +101 -0
- metadata +103 -0
data/.autotest
ADDED
data/.gemtest
ADDED
File without changes
|
data/History.txt
ADDED
@@ -0,0 +1,257 @@
|
|
1
|
+
=== 2.6 / 2010-03-26
|
2
|
+
|
3
|
+
* Minor enhancement
|
4
|
+
* Net::HTTP::Persistent#idle_timeout may be set to nil to disable expiration
|
5
|
+
of connections. Pull Request #21 by Aaron Stone
|
6
|
+
|
7
|
+
=== 2.5.2 / 2012-02-13
|
8
|
+
|
9
|
+
* Bug fix
|
10
|
+
* Fix variable shadowing warning.
|
11
|
+
|
12
|
+
=== 2.5.1 / 2012-02-10
|
13
|
+
|
14
|
+
* Bug fix
|
15
|
+
* Reusing SSL connections with HTTP proxies now works. Issue #15 by Paul
|
16
|
+
Ingham and mcrmfc
|
17
|
+
|
18
|
+
=== 2.5 / 2012-02-07
|
19
|
+
|
20
|
+
* Minor enhancements
|
21
|
+
* The proxy may be changed at any time.
|
22
|
+
* The allowed SSL version may now be set via #ssl_version.
|
23
|
+
Issue #16 by astera
|
24
|
+
* Added Net::HTTP::Persistent#override_headers which allows overriding
|
25
|
+
* Net::HTTP default headers like User-Agent. See
|
26
|
+
Net::HTTP::Persistent@Headers for details. Issue #17 by andkerosine
|
27
|
+
|
28
|
+
* Bug fixes
|
29
|
+
* The ruby 1.8 speed monkeypatch now handles EAGAIN for windows users.
|
30
|
+
Issue #12 by Alwyn Schoeman
|
31
|
+
* Fixed POST example in README. Submitted by injekt.
|
32
|
+
* Fixed various bugs in the shutdown of connections especially cross-thread
|
33
|
+
(which you shouldn't be doing anyways).
|
34
|
+
|
35
|
+
=== 2.4.1 / 2012-02-03
|
36
|
+
|
37
|
+
* Bug fixes
|
38
|
+
* When FakeWeb or WebMock are loaded SSL sessions will not be reused to
|
39
|
+
prevent breakage of testing frameworks. Issue #13 by Matt Brictson, pull
|
40
|
+
request #14 by Zachary Scott
|
41
|
+
* SSL connections are reset when the SSL parameters change.
|
42
|
+
Mechanize issue #194 by dsisnero
|
43
|
+
* Last-use times are now cleaned up in #shutdown.
|
44
|
+
|
45
|
+
=== 2.4 / 2012-01-31
|
46
|
+
|
47
|
+
* Minor enhancement
|
48
|
+
* net-http-persistent now complains if OpenSSL::SSL::VERIFY_PEER is equal to
|
49
|
+
OpenSSL::SSL::VERIFY_NONE. If you have a platform that is broken this way
|
50
|
+
you must define the constant:
|
51
|
+
|
52
|
+
I_KNOW_THAT_OPENSSL_VERIFY_PEER_EQUALS_VERIFY_NONE_IS_WRONG = nil
|
53
|
+
|
54
|
+
at the top level of your application to disable the warning.
|
55
|
+
|
56
|
+
* Bug fix
|
57
|
+
* Fix persisting SSL sessions through HTTP proxies. Mechanize issue #178 by
|
58
|
+
Robert Poor, net-http-persistent issues #10, #11.
|
59
|
+
|
60
|
+
=== 2.3.2 / 2011-12-21
|
61
|
+
|
62
|
+
* Bug fix
|
63
|
+
* Finish connections that were closed by Net::HTTP so they can be restarted.
|
64
|
+
|
65
|
+
=== 2.3.1 / 2011-10-26
|
66
|
+
|
67
|
+
* Bug fix
|
68
|
+
* If a request object already contains a Connection header it will no longer
|
69
|
+
be overridden. This allows keep-alive connections to be disabled on a
|
70
|
+
per-request basis.
|
71
|
+
|
72
|
+
=== 2.3 / 2011-10-25
|
73
|
+
|
74
|
+
* Minor Enhancement
|
75
|
+
* The time since last use for a connection is now recorded in error
|
76
|
+
messages for the connection.
|
77
|
+
|
78
|
+
=== 2.2 / 2011-10-24
|
79
|
+
|
80
|
+
* Minor Enhancements
|
81
|
+
* Added timeouts for idle connections which are set through #idle_timeout.
|
82
|
+
The default timeout is 5 seconds. Reducing the idle timeout is preferred
|
83
|
+
over setting #retry_change_requests to true if you wish to avoid the "too
|
84
|
+
many connection resets" error when POSTing data.
|
85
|
+
* Documented tunables and settings in one place in Net::HTTP::Persistent
|
86
|
+
|
87
|
+
=== 2.1 / 2011-09-19
|
88
|
+
|
89
|
+
* Minor Enhancement
|
90
|
+
* For HTTPS connections, SSL sessions are now reused avoiding the extra
|
91
|
+
round trips and computations of extra SSL handshakes. If you have
|
92
|
+
problems with SSL session reuse it can be disabled by
|
93
|
+
Net::HTTP::Persistent#reuse_ssl_sessions
|
94
|
+
|
95
|
+
* Bug Fixes
|
96
|
+
* The default certificate store is now used even if #verify_mode was not
|
97
|
+
set. Issue #7, Pull Request #8 by Matthew M. Boedicker
|
98
|
+
|
99
|
+
=== 2.0 / 2011-08-26
|
100
|
+
|
101
|
+
* Incompatibility
|
102
|
+
* Net::HTTP::Persistent#verify_mode now defaults to
|
103
|
+
OpenSSL::SSL::VERIFY_PEER. This may cause HTTPS request failures if your
|
104
|
+
default certificate store lacks the correct certificates.
|
105
|
+
|
106
|
+
=== 1.9 / 2011-08-26
|
107
|
+
|
108
|
+
* Minor Enhancement
|
109
|
+
* Added Net::HTTP::Persistent#cert_store to set an SSL certificate store
|
110
|
+
which defaults to the OpenSSL default certificate store.
|
111
|
+
|
112
|
+
HTTPS server certificates will be validated when this option is combined
|
113
|
+
with setting Net::HTTP::Persistent#verify_mode to
|
114
|
+
OpenSSL::SSL::VERIFY_PEER.
|
115
|
+
|
116
|
+
=== 1.8.1 / 2011-08-08
|
117
|
+
|
118
|
+
* Bug Fix
|
119
|
+
* Requests with OpenSSL errors are retried now. Pull Request #5 by James
|
120
|
+
Tucker.
|
121
|
+
|
122
|
+
=== 1.8 / 2011-06-27
|
123
|
+
|
124
|
+
* Minor Enhancement
|
125
|
+
* Added Net::HTTP::Persistent#retry_change_requests which allows POST and
|
126
|
+
other non-idempotent requests to be retried automatically. Take care when
|
127
|
+
enabling this option to ensure the server will handle multiple POSTs with
|
128
|
+
the same data in a sane manner.
|
129
|
+
|
130
|
+
=== 1.7 / 2011-04-17
|
131
|
+
|
132
|
+
* Minor Enhancement
|
133
|
+
* Added Net::HTTP::Persistent#pipeline which integrates with
|
134
|
+
net-http-pipeline when it is present.
|
135
|
+
* Bug Fix
|
136
|
+
* Perform a case-insensitive check of the URI scheme for HTTPS URIs
|
137
|
+
|
138
|
+
=== 1.6.1 / 2011-03-08
|
139
|
+
|
140
|
+
* Bug Fix
|
141
|
+
* Net::HTTP::Persistent#request now handles Errno::EINVAL as a connection
|
142
|
+
reset and will be retried for idempotent requests. Reported by Aaron
|
143
|
+
Qian.
|
144
|
+
|
145
|
+
=== 1.6 / 2011-03-01
|
146
|
+
|
147
|
+
* Minor Enhancement
|
148
|
+
* Added Net::HTTP::Persistent#socket_options to set multiple socket options
|
149
|
+
at socket startup.
|
150
|
+
|
151
|
+
=== 1.5.2 / 2011-02-24
|
152
|
+
|
153
|
+
* Bug Fix
|
154
|
+
* Only set TCP_NODELAY if the connection has an @socket. Allows
|
155
|
+
net-http-persistent to be used with fake_web. Reported by Sathish
|
156
|
+
Pasupunuri.
|
157
|
+
|
158
|
+
=== 1.5.1 / 2011-02-10
|
159
|
+
|
160
|
+
* Bug fix
|
161
|
+
* Only set TCP_NODELAY at connection start. Reported by Brian Henderson.
|
162
|
+
|
163
|
+
=== 1.5 / 2011-01-25
|
164
|
+
|
165
|
+
* Minor Enhancements
|
166
|
+
* Set TCP_NODELAY on created socket if possible. (This will only help for
|
167
|
+
requests that send bodies.)
|
168
|
+
|
169
|
+
=== 1.4.1 / 2010-10-13
|
170
|
+
|
171
|
+
* Bug Fixes
|
172
|
+
* Don't finish the connection when we're retrying, reset it. Patch by James
|
173
|
+
Tucker.
|
174
|
+
|
175
|
+
=== 1.4 / 2010-09-29
|
176
|
+
|
177
|
+
* Minor Enhancements
|
178
|
+
* Added the very dangerous #shutdown_in_all_threads. IT IS DANGEROUS!.
|
179
|
+
Patch by Torsten Schönebaum.
|
180
|
+
|
181
|
+
=== 1.3.1 / 2010-09-13
|
182
|
+
|
183
|
+
* Bug Fixes
|
184
|
+
* #connection_for no longer tries to ssl-enable an existing connection.
|
185
|
+
Patch by Joseph West.
|
186
|
+
|
187
|
+
=== 1.3 / 2010-09-08
|
188
|
+
|
189
|
+
* Minor Enhancements
|
190
|
+
* HTTP versions are now recorded. This information is not currently used.
|
191
|
+
|
192
|
+
* Bug Fixes
|
193
|
+
* #shutdown no longer fails when an unstarted HTTP connection is shut down.
|
194
|
+
|
195
|
+
=== 1.2.5 / 2010-07-27
|
196
|
+
|
197
|
+
* Bug Fixes
|
198
|
+
* Fix duplicated test name. Noted by Peter Higgins.
|
199
|
+
* #shutdown now works even when no connections were made.
|
200
|
+
|
201
|
+
=== 1.2.4 / 2010-07-26
|
202
|
+
|
203
|
+
* Bug Fixes
|
204
|
+
* Actually have #request only finish a connection. Somehow this got
|
205
|
+
missed.
|
206
|
+
|
207
|
+
=== 1.2.3 / 2010-06-29
|
208
|
+
|
209
|
+
* Bug Fixes
|
210
|
+
* Fix example code (pointed out by Alex Stahl)
|
211
|
+
|
212
|
+
=== 1.2.2 / 2010-06-22
|
213
|
+
|
214
|
+
* Bug Fixes
|
215
|
+
* #request only finishes a connection instead of restarting it. This helps
|
216
|
+
prevents errors on non-idempotent HTTP requests after errors.
|
217
|
+
* #connection_for handles EHOSTDOWN like #reset
|
218
|
+
|
219
|
+
=== 1.2.1 / 2010-05-25
|
220
|
+
|
221
|
+
* Bug Fixes
|
222
|
+
* Don't alter Net::BufferedIO#rbuf_fill on 1.9+
|
223
|
+
|
224
|
+
=== 1.2 / 2010-05-20
|
225
|
+
|
226
|
+
* Minor Enhancements
|
227
|
+
* Net::HTTP#read_timeout is now supported
|
228
|
+
* Net::HTTP#open_timeout is now supported
|
229
|
+
* Net::HTTP::Persistent#request now supports a block like Net::HTTP#request
|
230
|
+
|
231
|
+
=== 1.1 / 2010-05-18
|
232
|
+
|
233
|
+
* Minor Enhancements
|
234
|
+
* Proxy support, see Net::HTTP::Persistent::new,
|
235
|
+
Net::HTTP::Persistent#proxy_from_env
|
236
|
+
* Added +name+ parameter to Net::HTTP::Persistent::new for separation of
|
237
|
+
connection pools.
|
238
|
+
* Added Net::HTTP::Persistent#shutdown so you can clean up after yourself
|
239
|
+
* Net::HTTP::Persistent now suppresses "peer certificate won't be verified
|
240
|
+
in this SSL session" warning.
|
241
|
+
|
242
|
+
* Bug Fixes
|
243
|
+
* Net::HTTP::Persistent retries requests in accordance with RFC 2616.
|
244
|
+
|
245
|
+
=== 1.0.1 / 2010-05-05
|
246
|
+
|
247
|
+
* Minor Enhancements
|
248
|
+
* Added #debug_output
|
249
|
+
* Now uses Hoe minitest plugin
|
250
|
+
* Bug Fixes
|
251
|
+
* Tests pass on 1.9
|
252
|
+
|
253
|
+
=== 1.0.0 / 2010-05-04
|
254
|
+
|
255
|
+
* Major Enhancements
|
256
|
+
* Birthday!
|
257
|
+
|
data/Manifest.txt
ADDED
data/README.rdoc
ADDED
@@ -0,0 +1,82 @@
|
|
1
|
+
= net-http-persistent
|
2
|
+
|
3
|
+
* http://docs.seattlerb.org/net-http-persistent
|
4
|
+
* https://github.com/drbrain/net-http-persistent
|
5
|
+
|
6
|
+
== DESCRIPTION:
|
7
|
+
|
8
|
+
Manages persistent connections using Net::HTTP plus a speed fix for Ruby 1.8.
|
9
|
+
It's thread-safe too!
|
10
|
+
|
11
|
+
Using persistent HTTP connections can dramatically increase the speed of HTTP.
|
12
|
+
Creating a new HTTP connection for every request involves an extra TCP
|
13
|
+
round-trip and causes TCP congestion avoidance negotiation to start over.
|
14
|
+
|
15
|
+
Net::HTTP supports persistent connections with some API methods but does not
|
16
|
+
handle reconnection gracefully. Net::HTTP::Persistent supports reconnection
|
17
|
+
and retry according to RFC 2616.
|
18
|
+
|
19
|
+
== FEATURES/PROBLEMS:
|
20
|
+
|
21
|
+
* Supports SSL
|
22
|
+
* Thread-safe
|
23
|
+
* Pure ruby
|
24
|
+
* Timeout-less speed boost for Ruby 1.8 (by Aaron Patterson)
|
25
|
+
|
26
|
+
== SYNOPSIS
|
27
|
+
|
28
|
+
The following example will make two requests to the same server. The
|
29
|
+
connection is kept alive between requests:
|
30
|
+
|
31
|
+
require 'net/http/persistent'
|
32
|
+
|
33
|
+
uri = URI 'http://example.com/awesome/web/service'
|
34
|
+
|
35
|
+
http = Net::HTTP::Persistent.new 'my_app_name'
|
36
|
+
|
37
|
+
# perform a GET
|
38
|
+
response = http.request uri
|
39
|
+
|
40
|
+
# create a POST
|
41
|
+
post_uri = uri + 'create'
|
42
|
+
post = Net::HTTP::Post.new post_uri.path
|
43
|
+
post.set_form_data 'some' => 'cool data'
|
44
|
+
|
45
|
+
# perform the POST, the URI is always required
|
46
|
+
response = http.request post_uri, post
|
47
|
+
|
48
|
+
# if you are done making http requests, or won't make requests for several
|
49
|
+
# minutes
|
50
|
+
http.shutdown
|
51
|
+
|
52
|
+
Please see the documentation on Net::HTTP::Persistent for more information,
|
53
|
+
including SSL connection verification, header handling and tunable options.
|
54
|
+
|
55
|
+
== INSTALL:
|
56
|
+
|
57
|
+
gem install net-http-persistent
|
58
|
+
|
59
|
+
== LICENSE:
|
60
|
+
|
61
|
+
(The MIT License)
|
62
|
+
|
63
|
+
Copyright (c) 2010 Eric Hodel, Aaron Patterson
|
64
|
+
|
65
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
66
|
+
a copy of this software and associated documentation files (the
|
67
|
+
'Software'), to deal in the Software without restriction, including
|
68
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
69
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
70
|
+
permit persons to whom the Software is furnished to do so, subject to
|
71
|
+
the following conditions:
|
72
|
+
|
73
|
+
The above copyright notice and this permission notice shall be
|
74
|
+
included in all copies or substantial portions of the Software.
|
75
|
+
|
76
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
77
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
78
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
79
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
80
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
81
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
82
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/Rakefile
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
# -*- ruby -*-
|
2
|
+
|
3
|
+
require 'rubygems'
|
4
|
+
require 'hoe'
|
5
|
+
|
6
|
+
Hoe.plugin :git
|
7
|
+
Hoe.plugin :minitest
|
8
|
+
Hoe.plugin :travis
|
9
|
+
|
10
|
+
Hoe.spec 'softwaregravy-net-http-persistent' do |p|
|
11
|
+
developer 'Eric Hodel', 'drbrain@segment7.net'
|
12
|
+
|
13
|
+
self.readme_file = 'README.rdoc'
|
14
|
+
self.extra_rdoc_files += Dir['*.rdoc']
|
15
|
+
|
16
|
+
rdoc_locations <<
|
17
|
+
'docs.seattlerb.org:/data/www/docs.seattlerb.org/net-http-persistent/'
|
18
|
+
end
|
19
|
+
|
20
|
+
# vim: syntax=Ruby
|
@@ -0,0 +1,27 @@
|
|
1
|
+
require 'net/protocol'
|
2
|
+
|
3
|
+
##
|
4
|
+
# Aaron Patterson's monkeypatch (accepted into 1.9.1) to fix Net::HTTP's speed
|
5
|
+
# problems.
|
6
|
+
#
|
7
|
+
# http://gist.github.com/251244
|
8
|
+
|
9
|
+
class Net::BufferedIO #:nodoc:
|
10
|
+
alias :old_rbuf_fill :rbuf_fill
|
11
|
+
|
12
|
+
def rbuf_fill
|
13
|
+
if @io.respond_to? :read_nonblock then
|
14
|
+
begin
|
15
|
+
@rbuf << @io.read_nonblock(65536)
|
16
|
+
rescue Errno::EWOULDBLOCK, Errno::EAGAIN => e
|
17
|
+
retry if IO.select [@io], nil, nil, @read_timeout
|
18
|
+
raise Timeout::Error, e.message
|
19
|
+
end
|
20
|
+
else # SSL sockets do not have read_nonblock
|
21
|
+
timeout @read_timeout do
|
22
|
+
@rbuf << @io.sysread(65536)
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end if RUBY_VERSION < '1.9'
|
27
|
+
|
@@ -0,0 +1,998 @@
|
|
1
|
+
require 'net/http'
|
2
|
+
require 'net/https'
|
3
|
+
require 'net/http/faster'
|
4
|
+
require 'uri'
|
5
|
+
require 'cgi' # for escaping
|
6
|
+
|
7
|
+
begin
|
8
|
+
require 'net/http/pipeline'
|
9
|
+
rescue LoadError
|
10
|
+
end
|
11
|
+
|
12
|
+
##
|
13
|
+
# Persistent connections for Net::HTTP
|
14
|
+
#
|
15
|
+
# Net::HTTP::Persistent maintains persistent connections across all the
|
16
|
+
# servers you wish to talk to. For each host:port you communicate with a
|
17
|
+
# single persistent connection is created.
|
18
|
+
#
|
19
|
+
# Multiple Net::HTTP::Persistent objects will share the same set of
|
20
|
+
# connections.
|
21
|
+
#
|
22
|
+
# For each thread you start a new connection will be created. A
|
23
|
+
# Net::HTTP::Persistent connection will not be shared across threads.
|
24
|
+
#
|
25
|
+
# You can shut down the HTTP connections when done by calling #shutdown. You
|
26
|
+
# should name your Net::HTTP::Persistent object if you intend to call this
|
27
|
+
# method.
|
28
|
+
#
|
29
|
+
# Example:
|
30
|
+
#
|
31
|
+
# require 'net/http/persistent'
|
32
|
+
#
|
33
|
+
# uri = URI 'http://example.com/awesome/web/service'
|
34
|
+
#
|
35
|
+
# http = Net::HTTP::Persistent.new 'my_app_name'
|
36
|
+
#
|
37
|
+
# # perform a GET
|
38
|
+
# response = http.request uri
|
39
|
+
#
|
40
|
+
# # create a POST
|
41
|
+
# post_uri = uri + 'create'
|
42
|
+
# post = Net::HTTP::Post.new post_uri.path
|
43
|
+
# post.set_form_data 'some' => 'cool data'
|
44
|
+
#
|
45
|
+
# # perform the POST, the URI is always required
|
46
|
+
# response http.request post_uri, post
|
47
|
+
#
|
48
|
+
# == SSL
|
49
|
+
#
|
50
|
+
# SSL connections are automatically created depending upon the scheme of the
|
51
|
+
# URI. SSL connections are automatically verified against the default
|
52
|
+
# certificate store for your computer. You can override this by changing
|
53
|
+
# verify_mode or by specifying an alternate cert_store.
|
54
|
+
#
|
55
|
+
# Here are the SSL settings, see the individual methods for documentation:
|
56
|
+
#
|
57
|
+
# #certificate :: This client's certificate
|
58
|
+
# #ca_file :: The certificate-authority
|
59
|
+
# #cert_store :: An SSL certificate store
|
60
|
+
# #private_key :: The client's SSL private key
|
61
|
+
# #reuse_ssl_sessions :: Reuse a previously opened SSL session for a new
|
62
|
+
# connection
|
63
|
+
# #ssl_version :: Which specific SSL version to use
|
64
|
+
# #verify_callback :: For server certificate verification
|
65
|
+
# #verify_mode :: How connections should be verified
|
66
|
+
#
|
67
|
+
# == Proxies
|
68
|
+
#
|
69
|
+
# A proxy can be set through #proxy= or at initialization time by providing a
|
70
|
+
# second argument to ::new. The proxy may be the URI of the proxy server or
|
71
|
+
# <code>:ENV</code> which will consult environment variables.
|
72
|
+
#
|
73
|
+
# See #proxy= and #proxy_from_env for details.
|
74
|
+
#
|
75
|
+
# == Headers
|
76
|
+
#
|
77
|
+
# Headers may be specified for use in every request. #headers are appended to
|
78
|
+
# any headers on the request. #override_headers replace existing headers on
|
79
|
+
# the request.
|
80
|
+
#
|
81
|
+
# The difference between the two can be seen in setting the User-Agent. Using
|
82
|
+
# <code>http.headers['User-Agent'] = 'MyUserAgent'</code> will send "Ruby,
|
83
|
+
# MyUserAgent" while <code>http.override_headers['User-Agent'] =
|
84
|
+
# 'MyUserAgent'</code> will send "MyUserAgent".
|
85
|
+
#
|
86
|
+
# == Tuning
|
87
|
+
#
|
88
|
+
# === Segregation
|
89
|
+
#
|
90
|
+
# By providing an application name to ::new you can separate your connections
|
91
|
+
# from the connections of other applications.
|
92
|
+
#
|
93
|
+
# === Idle Timeout
|
94
|
+
#
|
95
|
+
# If a connection hasn't been used for this number of seconds it will automatically be
|
96
|
+
# reset upon the next use to avoid attempting to send to a closed connection.
|
97
|
+
# The default value is 5 seconds. nil means no timeout. Set through #idle_timeout.
|
98
|
+
#
|
99
|
+
# Reducing this value may help avoid the "too many connection resets" error
|
100
|
+
# when sending non-idempotent requests while increasing this value will cause
|
101
|
+
# fewer round-trips.
|
102
|
+
#
|
103
|
+
# === Read Timeout
|
104
|
+
#
|
105
|
+
# The amount of time allowed between reading two chunks from the socket. Set
|
106
|
+
# through #read_timeout
|
107
|
+
#
|
108
|
+
# === Open Timeout
|
109
|
+
#
|
110
|
+
# The amount of time to wait for a connection to be opened. Set through
|
111
|
+
# #open_timeout.
|
112
|
+
#
|
113
|
+
# === Socket Options
|
114
|
+
#
|
115
|
+
# Socket options may be set on newly-created connections. See #socket_options
|
116
|
+
# for details.
|
117
|
+
#
|
118
|
+
# === Non-Idempotent Requests
|
119
|
+
#
|
120
|
+
# By default non-idempotent requests will not be retried per RFC 2616. By
|
121
|
+
# setting retry_change_requests to true requests will automatically be retried
|
122
|
+
# once.
|
123
|
+
#
|
124
|
+
# Only do this when you know that retrying a POST or other non-idempotent
|
125
|
+
# request is safe for your application and will not create duplicate
|
126
|
+
# resources.
|
127
|
+
#
|
128
|
+
# The recommended way to handle non-idempotent requests is the following:
|
129
|
+
#
|
130
|
+
# require 'net/http/persistent'
|
131
|
+
#
|
132
|
+
# uri = URI 'http://example.com/awesome/web/service'
|
133
|
+
# post_uri = uri + 'create'
|
134
|
+
#
|
135
|
+
# http = Net::HTTP::Persistent.new 'my_app_name'
|
136
|
+
#
|
137
|
+
# post = Net::HTTP::Post.new post_uri.path
|
138
|
+
# # ... fill in POST request
|
139
|
+
#
|
140
|
+
# begin
|
141
|
+
# response = http.request post_uri, post
|
142
|
+
# rescue Net::HTTP::Persistent::Error
|
143
|
+
#
|
144
|
+
# # POST failed, make a new request to verify the server did not process
|
145
|
+
# # the request
|
146
|
+
# exists_uri = uri + '...'
|
147
|
+
# response = http.get exists_uri
|
148
|
+
#
|
149
|
+
# # Retry if it failed
|
150
|
+
# retry if response.code == '404'
|
151
|
+
# end
|
152
|
+
#
|
153
|
+
# The method of determining if the resource was created or not is unique to
|
154
|
+
# the particular service you are using. Of course, you will want to add
|
155
|
+
# protection from infinite looping.
|
156
|
+
#
|
157
|
+
# === Connection Termination
|
158
|
+
#
|
159
|
+
# If you are done using the Net::HTTP::Persistent instance you may shut down
|
160
|
+
# all the connections in the current thread with #shutdown. This is not
|
161
|
+
# recommended for normal use, it should only be used when it will be several
|
162
|
+
# minutes before you make another HTTP request.
|
163
|
+
#
|
164
|
+
# If you are using multiple threads, call #shutdown in each thread when the
|
165
|
+
# thread is done making requests. If you don't call shutdown, that's OK.
|
166
|
+
# Ruby will automatically garbage collect and shutdown your HTTP connections
|
167
|
+
# when the thread terminates.
|
168
|
+
|
169
|
+
class Net::HTTP::Persistent
|
170
|
+
|
171
|
+
##
|
172
|
+
# The beginning of Time
|
173
|
+
|
174
|
+
EPOCH = Time.at 0 # :nodoc:
|
175
|
+
|
176
|
+
##
|
177
|
+
# The version of Net::HTTP::Persistent you are using
|
178
|
+
|
179
|
+
VERSION = '2.6'
|
180
|
+
|
181
|
+
##
|
182
|
+
# Error class for errors raised by Net::HTTP::Persistent. Various
|
183
|
+
# SystemCallErrors are re-raised with a human-readable message under this
|
184
|
+
# class.
|
185
|
+
|
186
|
+
class Error < StandardError; end
|
187
|
+
|
188
|
+
##
|
189
|
+
# This client's OpenSSL::X509::Certificate
|
190
|
+
|
191
|
+
attr_reader :certificate
|
192
|
+
|
193
|
+
##
|
194
|
+
# An SSL certificate authority. Setting this will set verify_mode to
|
195
|
+
# VERIFY_PEER.
|
196
|
+
|
197
|
+
attr_reader :ca_file
|
198
|
+
|
199
|
+
##
|
200
|
+
# An SSL certificate store. Setting this will override the default
|
201
|
+
# certificate store. See verify_mode for more information.
|
202
|
+
|
203
|
+
attr_reader :cert_store
|
204
|
+
|
205
|
+
##
|
206
|
+
# Sends debug_output to this IO via Net::HTTP#set_debug_output.
|
207
|
+
#
|
208
|
+
# Never use this method in production code, it causes a serious security
|
209
|
+
# hole.
|
210
|
+
|
211
|
+
attr_accessor :debug_output
|
212
|
+
|
213
|
+
##
|
214
|
+
# Current connection generation
|
215
|
+
|
216
|
+
attr_reader :generation # :nodoc:
|
217
|
+
|
218
|
+
##
|
219
|
+
# Where this instance's connections live in the thread local variables
|
220
|
+
|
221
|
+
attr_reader :generation_key # :nodoc:
|
222
|
+
|
223
|
+
##
|
224
|
+
# Headers that are added to every request using Net::HTTP#add_field
|
225
|
+
|
226
|
+
attr_reader :headers
|
227
|
+
|
228
|
+
##
|
229
|
+
# Maps host:port to an HTTP version. This allows us to enable version
|
230
|
+
# specific features.
|
231
|
+
|
232
|
+
attr_reader :http_versions
|
233
|
+
|
234
|
+
##
|
235
|
+
# Maximum time an unused connection can remain idle before being
|
236
|
+
# automatically closed.
|
237
|
+
|
238
|
+
attr_accessor :idle_timeout
|
239
|
+
|
240
|
+
##
|
241
|
+
# The value sent in the Keep-Alive header. Defaults to 30. Not needed for
|
242
|
+
# HTTP/1.1 servers.
|
243
|
+
#
|
244
|
+
# This may not work correctly for HTTP/1.0 servers
|
245
|
+
#
|
246
|
+
# This method may be removed in a future version as RFC 2616 does not
|
247
|
+
# require this header.
|
248
|
+
|
249
|
+
attr_accessor :keep_alive
|
250
|
+
|
251
|
+
##
|
252
|
+
# A name for this connection. Allows you to keep your connections apart
|
253
|
+
# from everybody else's.
|
254
|
+
|
255
|
+
attr_reader :name
|
256
|
+
|
257
|
+
##
|
258
|
+
# Seconds to wait until a connection is opened. See Net::HTTP#open_timeout
|
259
|
+
|
260
|
+
attr_accessor :open_timeout
|
261
|
+
|
262
|
+
##
|
263
|
+
# Headers that are added to every request using Net::HTTP#[]=
|
264
|
+
|
265
|
+
attr_reader :override_headers
|
266
|
+
|
267
|
+
##
|
268
|
+
# This client's SSL private key
|
269
|
+
|
270
|
+
attr_reader :private_key
|
271
|
+
|
272
|
+
##
|
273
|
+
# The URL through which requests will be proxied
|
274
|
+
|
275
|
+
attr_reader :proxy_uri
|
276
|
+
|
277
|
+
##
|
278
|
+
# Seconds to wait until reading one block. See Net::HTTP#read_timeout
|
279
|
+
|
280
|
+
attr_accessor :read_timeout
|
281
|
+
|
282
|
+
##
|
283
|
+
# Where this instance's request counts live in the thread local variables
|
284
|
+
|
285
|
+
attr_reader :request_key # :nodoc:
|
286
|
+
|
287
|
+
##
|
288
|
+
# By default SSL sessions are reused to avoid extra SSL handshakes. Set
|
289
|
+
# this to false if you have problems communicating with an HTTPS server
|
290
|
+
# like:
|
291
|
+
#
|
292
|
+
# SSL_connect [...] read finished A: unexpected message (OpenSSL::SSL::SSLError)
|
293
|
+
|
294
|
+
attr_accessor :reuse_ssl_sessions
|
295
|
+
|
296
|
+
##
|
297
|
+
# An array of options for Socket#setsockopt.
|
298
|
+
#
|
299
|
+
# By default the TCP_NODELAY option is set on sockets.
|
300
|
+
#
|
301
|
+
# To set additional options append them to this array:
|
302
|
+
#
|
303
|
+
# http.socket_options << [Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, 1]
|
304
|
+
|
305
|
+
attr_reader :socket_options
|
306
|
+
|
307
|
+
##
|
308
|
+
# Current SSL connection generation
|
309
|
+
|
310
|
+
attr_reader :ssl_generation # :nodoc:
|
311
|
+
|
312
|
+
##
|
313
|
+
# Where this instance's SSL connections live in the thread local variables
|
314
|
+
|
315
|
+
attr_reader :ssl_generation_key # :nodoc:
|
316
|
+
|
317
|
+
##
|
318
|
+
# SSL version to use.
|
319
|
+
#
|
320
|
+
# By default, the version will be negotiated automatically between client
|
321
|
+
# and server. Ruby 1.9 and newer only.
|
322
|
+
|
323
|
+
attr_reader :ssl_version if RUBY_VERSION > '1.9'
|
324
|
+
|
325
|
+
##
|
326
|
+
# Where this instance's last-use times live in the thread local variables
|
327
|
+
|
328
|
+
attr_reader :timeout_key # :nodoc:
|
329
|
+
|
330
|
+
##
|
331
|
+
# SSL verification callback. Used when ca_file is set.
|
332
|
+
|
333
|
+
attr_reader :verify_callback
|
334
|
+
|
335
|
+
##
|
336
|
+
# HTTPS verify mode. Defaults to OpenSSL::SSL::VERIFY_PEER which verifies
|
337
|
+
# the server certificate.
|
338
|
+
#
|
339
|
+
# If no ca_file or cert_store is set the default system certificate store is
|
340
|
+
# used.
|
341
|
+
#
|
342
|
+
# You can use +verify_mode+ to override any default values.
|
343
|
+
|
344
|
+
attr_reader :verify_mode
|
345
|
+
|
346
|
+
##
|
347
|
+
# Enable retries of non-idempotent requests that change data (e.g. POST
|
348
|
+
# requests) when the server has disconnected.
|
349
|
+
#
|
350
|
+
# This will in the worst case lead to multiple requests with the same data,
|
351
|
+
# but it may be useful for some applications. Take care when enabling
|
352
|
+
# this option to ensure it is safe to POST or perform other non-idempotent
|
353
|
+
# requests to the server.
|
354
|
+
|
355
|
+
attr_accessor :retry_change_requests
|
356
|
+
|
357
|
+
##
|
358
|
+
# Creates a new Net::HTTP::Persistent.
|
359
|
+
#
|
360
|
+
# Set +name+ to keep your connections apart from everybody else's. Not
|
361
|
+
# required currently, but highly recommended. Your library name should be
|
362
|
+
# good enough. This parameter will be required in a future version.
|
363
|
+
#
|
364
|
+
# +proxy+ may be set to a URI::HTTP or :ENV to pick up proxy options from
|
365
|
+
# the environment. See proxy_from_env for details.
|
366
|
+
#
|
367
|
+
# In order to use a URI for the proxy you may need to do some extra work
|
368
|
+
# beyond URI parsing if the proxy requires a password:
|
369
|
+
#
|
370
|
+
# proxy = URI 'http://proxy.example'
|
371
|
+
# proxy.user = 'AzureDiamond'
|
372
|
+
# proxy.password = 'hunter2'
|
373
|
+
|
374
|
+
def initialize name = nil, proxy = nil
|
375
|
+
@name = name
|
376
|
+
|
377
|
+
@debug_output = nil
|
378
|
+
@proxy_uri = nil
|
379
|
+
@headers = {}
|
380
|
+
@override_headers = {}
|
381
|
+
@http_versions = {}
|
382
|
+
@keep_alive = 30
|
383
|
+
@open_timeout = nil
|
384
|
+
@read_timeout = nil
|
385
|
+
@idle_timeout = 5
|
386
|
+
@socket_options = []
|
387
|
+
|
388
|
+
@socket_options << [Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1] if
|
389
|
+
Socket.const_defined? :TCP_NODELAY
|
390
|
+
|
391
|
+
key = ['net_http_persistent', name].compact
|
392
|
+
@generation_key = [key, 'generations' ].join('_').intern
|
393
|
+
@ssl_generation_key = [key, 'ssl_generations'].join('_').intern
|
394
|
+
@request_key = [key, 'requests' ].join('_').intern
|
395
|
+
@timeout_key = [key, 'timeouts' ].join('_').intern
|
396
|
+
|
397
|
+
@certificate = nil
|
398
|
+
@ca_file = nil
|
399
|
+
@private_key = nil
|
400
|
+
@ssl_version = nil
|
401
|
+
@verify_callback = nil
|
402
|
+
@verify_mode = OpenSSL::SSL::VERIFY_PEER
|
403
|
+
@cert_store = nil
|
404
|
+
|
405
|
+
@generation = 0 # incremented when proxy URI changes
|
406
|
+
@ssl_generation = 0 # incremented when SSL session variables change
|
407
|
+
@reuse_ssl_sessions = true
|
408
|
+
|
409
|
+
@retry_change_requests = false
|
410
|
+
|
411
|
+
self.proxy = proxy if proxy
|
412
|
+
end
|
413
|
+
|
414
|
+
##
|
415
|
+
# Sets this client's OpenSSL::X509::Certificate
|
416
|
+
|
417
|
+
def certificate= certificate
|
418
|
+
@certificate = certificate
|
419
|
+
|
420
|
+
reconnect_ssl
|
421
|
+
end
|
422
|
+
|
423
|
+
##
|
424
|
+
# Sets the SSL certificate authority file.
|
425
|
+
|
426
|
+
def ca_file= file
|
427
|
+
@ca_file = file
|
428
|
+
|
429
|
+
reconnect_ssl
|
430
|
+
end
|
431
|
+
|
432
|
+
##
|
433
|
+
# Overrides the default SSL certificate store used for verifying
|
434
|
+
# connections.
|
435
|
+
|
436
|
+
def cert_store= store
|
437
|
+
@cert_store = store
|
438
|
+
|
439
|
+
reconnect_ssl
|
440
|
+
end
|
441
|
+
|
442
|
+
##
|
443
|
+
# Finishes all connections on the given +thread+ that were created before
|
444
|
+
# the given +generation+ in the threads +generation_key+ list.
|
445
|
+
#
|
446
|
+
# See #shutdown for a bunch of scary warning about misusing this method.
|
447
|
+
|
448
|
+
def cleanup(generation, thread = Thread.current,
|
449
|
+
generation_key = @generation_key) # :nodoc:
|
450
|
+
timeouts = thread[@timeout_key]
|
451
|
+
|
452
|
+
(0...generation).each do |old_generation|
|
453
|
+
next unless thread[generation_key]
|
454
|
+
|
455
|
+
conns = thread[generation_key].delete old_generation
|
456
|
+
|
457
|
+
conns.each_value do |conn|
|
458
|
+
finish conn, thread
|
459
|
+
|
460
|
+
timeouts.delete conn.object_id if timeouts
|
461
|
+
end if conns
|
462
|
+
end
|
463
|
+
end
|
464
|
+
|
465
|
+
##
|
466
|
+
# Creates a new connection for +uri+
|
467
|
+
|
468
|
+
def connection_for uri
|
469
|
+
Thread.current[@generation_key] ||= Hash.new { |h,k| h[k] = {} }
|
470
|
+
Thread.current[@ssl_generation_key] ||= Hash.new { |h,k| h[k] = {} }
|
471
|
+
Thread.current[@request_key] ||= Hash.new 0
|
472
|
+
Thread.current[@timeout_key] ||= Hash.new EPOCH
|
473
|
+
|
474
|
+
use_ssl = uri.scheme.downcase == 'https'
|
475
|
+
|
476
|
+
if use_ssl then
|
477
|
+
ssl_generation = @ssl_generation
|
478
|
+
|
479
|
+
ssl_cleanup ssl_generation
|
480
|
+
|
481
|
+
connections = Thread.current[@ssl_generation_key][ssl_generation]
|
482
|
+
else
|
483
|
+
generation = @generation
|
484
|
+
|
485
|
+
cleanup generation
|
486
|
+
|
487
|
+
connections = Thread.current[@generation_key][generation]
|
488
|
+
end
|
489
|
+
|
490
|
+
net_http_args = [uri.host, uri.port]
|
491
|
+
connection_id = net_http_args.join ':'
|
492
|
+
|
493
|
+
if @proxy_uri then
|
494
|
+
connection_id << @proxy_connection_id
|
495
|
+
net_http_args.concat @proxy_args
|
496
|
+
end
|
497
|
+
|
498
|
+
connection = connections[connection_id]
|
499
|
+
|
500
|
+
unless connection = connections[connection_id] then
|
501
|
+
connections[connection_id] = http_class.new(*net_http_args)
|
502
|
+
connection = connections[connection_id]
|
503
|
+
ssl connection if use_ssl
|
504
|
+
else
|
505
|
+
reset connection if expired? connection
|
506
|
+
end
|
507
|
+
|
508
|
+
unless connection.started? then
|
509
|
+
connection.set_debug_output @debug_output if @debug_output
|
510
|
+
connection.open_timeout = @open_timeout if @open_timeout
|
511
|
+
connection.read_timeout = @read_timeout if @read_timeout
|
512
|
+
|
513
|
+
connection.start
|
514
|
+
|
515
|
+
socket = connection.instance_variable_get :@socket
|
516
|
+
|
517
|
+
if socket then # for fakeweb
|
518
|
+
@socket_options.each do |option|
|
519
|
+
socket.io.setsockopt(*option)
|
520
|
+
end
|
521
|
+
end
|
522
|
+
end
|
523
|
+
|
524
|
+
connection
|
525
|
+
rescue Errno::ECONNREFUSED
|
526
|
+
raise Error, "connection refused: #{connection.address}:#{connection.port}"
|
527
|
+
rescue Errno::EHOSTDOWN
|
528
|
+
raise Error, "host down: #{connection.address}:#{connection.port}"
|
529
|
+
end
|
530
|
+
|
531
|
+
##
|
532
|
+
# Returns an error message containing the number of requests performed on
|
533
|
+
# this connection
|
534
|
+
|
535
|
+
def error_message connection
|
536
|
+
requests = Thread.current[@request_key][connection.object_id] - 1 # fixup
|
537
|
+
last_use = Thread.current[@timeout_key][connection.object_id]
|
538
|
+
|
539
|
+
age = Time.now - last_use
|
540
|
+
|
541
|
+
"after #{requests} requests on #{connection.object_id}, " \
|
542
|
+
"last used #{age} seconds ago"
|
543
|
+
end
|
544
|
+
|
545
|
+
##
|
546
|
+
# URI::escape wrapper
|
547
|
+
|
548
|
+
def escape str
|
549
|
+
CGI.escape str if str
|
550
|
+
end
|
551
|
+
|
552
|
+
##
|
553
|
+
# Returns true if the connection should be reset due to an idle timeout,
|
554
|
+
# false otherwise.
|
555
|
+
|
556
|
+
def expired? connection
|
557
|
+
return false unless @idle_timeout
|
558
|
+
return true if @idle_timeout.zero?
|
559
|
+
|
560
|
+
last_used = Thread.current[@timeout_key][connection.object_id]
|
561
|
+
|
562
|
+
Time.now - last_used > @idle_timeout
|
563
|
+
end
|
564
|
+
|
565
|
+
##
|
566
|
+
# Finishes the Net::HTTP +connection+
|
567
|
+
|
568
|
+
def finish connection, thread = Thread.current
|
569
|
+
if requests = thread[@request_key] then
|
570
|
+
requests.delete connection.object_id
|
571
|
+
end
|
572
|
+
|
573
|
+
connection.finish
|
574
|
+
rescue IOError
|
575
|
+
end
|
576
|
+
|
577
|
+
def http_class # :nodoc:
|
578
|
+
if [:FakeWeb, :WebMock].any? { |klass| Object.const_defined?(klass) } or
|
579
|
+
not @reuse_ssl_sessions then
|
580
|
+
Net::HTTP
|
581
|
+
else
|
582
|
+
Net::HTTP::Persistent::SSLReuse
|
583
|
+
end
|
584
|
+
end
|
585
|
+
|
586
|
+
##
|
587
|
+
# Returns the HTTP protocol version for +uri+
|
588
|
+
|
589
|
+
def http_version uri
|
590
|
+
@http_versions["#{uri.host}:#{uri.port}"]
|
591
|
+
end
|
592
|
+
|
593
|
+
##
|
594
|
+
# Is +req+ idempotent according to RFC 2616?
|
595
|
+
|
596
|
+
def idempotent? req
|
597
|
+
case req
|
598
|
+
when Net::HTTP::Delete, Net::HTTP::Get, Net::HTTP::Head,
|
599
|
+
Net::HTTP::Options, Net::HTTP::Put, Net::HTTP::Trace then
|
600
|
+
true
|
601
|
+
end
|
602
|
+
end
|
603
|
+
|
604
|
+
##
|
605
|
+
# Is the request idempotent or is retry_change_requests allowed
|
606
|
+
|
607
|
+
def can_retry? req
|
608
|
+
retry_change_requests or idempotent?(req)
|
609
|
+
end
|
610
|
+
|
611
|
+
if RUBY_VERSION > '1.9' then
|
612
|
+
##
|
613
|
+
# Workaround for missing Net::HTTPHeader#connection_close? on Ruby 1.8
|
614
|
+
|
615
|
+
def connection_close? header
|
616
|
+
header.connection_close?
|
617
|
+
end
|
618
|
+
|
619
|
+
##
|
620
|
+
# Workaround for missing Net::HTTPHeader#connection_keep_alive? on Ruby 1.8
|
621
|
+
|
622
|
+
def connection_keep_alive? header
|
623
|
+
header.connection_keep_alive?
|
624
|
+
end
|
625
|
+
else
|
626
|
+
##
|
627
|
+
# Workaround for missing Net::HTTPRequest#connection_close? on Ruby 1.8
|
628
|
+
|
629
|
+
def connection_close? header
|
630
|
+
header['connection'] =~ /close/ or header['proxy-connection'] =~ /close/
|
631
|
+
end
|
632
|
+
|
633
|
+
##
|
634
|
+
# Workaround for missing Net::HTTPRequest#connection_keep_alive? on Ruby
|
635
|
+
# 1.8
|
636
|
+
|
637
|
+
def connection_keep_alive? header
|
638
|
+
header['connection'] =~ /keep-alive/ or
|
639
|
+
header['proxy-connection'] =~ /keep-alive/
|
640
|
+
end
|
641
|
+
end
|
642
|
+
|
643
|
+
##
|
644
|
+
# Deprecated in favor of #expired?
|
645
|
+
|
646
|
+
def max_age # :nodoc:
|
647
|
+
return Time.now + 1 unless @idle_timeout
|
648
|
+
|
649
|
+
Time.now - @idle_timeout
|
650
|
+
end
|
651
|
+
|
652
|
+
##
|
653
|
+
# Adds "http://" to the String +uri+ if it is missing.
|
654
|
+
|
655
|
+
def normalize_uri uri
|
656
|
+
(uri =~ /^https?:/) ? uri : "http://#{uri}"
|
657
|
+
end
|
658
|
+
|
659
|
+
##
|
660
|
+
# Pipelines +requests+ to the HTTP server at +uri+ yielding responses if a
|
661
|
+
# block is given. Returns all responses recieved.
|
662
|
+
#
|
663
|
+
# See
|
664
|
+
# Net::HTTP::Pipeline[http://docs.seattlerb.org/net-http-pipeline/Net/HTTP/Pipeline.html]
|
665
|
+
# for further details.
|
666
|
+
#
|
667
|
+
# Only if <tt>net-http-pipeline</tt> was required before
|
668
|
+
# <tt>net-http-persistent</tt> #pipeline will be present.
|
669
|
+
|
670
|
+
def pipeline uri, requests, &block # :yields: responses
|
671
|
+
connection = connection_for uri
|
672
|
+
|
673
|
+
connection.pipeline requests, &block
|
674
|
+
end
|
675
|
+
|
676
|
+
##
|
677
|
+
# Sets this client's SSL private key
|
678
|
+
|
679
|
+
def private_key= key
|
680
|
+
@private_key = key
|
681
|
+
|
682
|
+
reconnect_ssl
|
683
|
+
end
|
684
|
+
|
685
|
+
##
|
686
|
+
# Sets the proxy server. The +proxy+ may be the URI of the proxy server,
|
687
|
+
# the symbol +:ENV+ which will read the proxy from the environment or nil to
|
688
|
+
# disable use of a proxy. See #proxy_from_env for details on setting the
|
689
|
+
# proxy from the environment.
|
690
|
+
#
|
691
|
+
# If the proxy URI is set after requests have been made, the next request
|
692
|
+
# will shut-down and re-open all connections.
|
693
|
+
#
|
694
|
+
# If you are making some requests through a proxy and others without a proxy
|
695
|
+
# use separate Net::Http::Persistent instances.
|
696
|
+
|
697
|
+
def proxy= proxy
|
698
|
+
@proxy_uri = case proxy
|
699
|
+
when :ENV then proxy_from_env
|
700
|
+
when URI::HTTP then proxy
|
701
|
+
when nil then # ignore
|
702
|
+
else raise ArgumentError, 'proxy must be :ENV or a URI::HTTP'
|
703
|
+
end
|
704
|
+
|
705
|
+
if @proxy_uri then
|
706
|
+
@proxy_args = [
|
707
|
+
@proxy_uri.host,
|
708
|
+
@proxy_uri.port,
|
709
|
+
@proxy_uri.user,
|
710
|
+
@proxy_uri.password,
|
711
|
+
]
|
712
|
+
|
713
|
+
@proxy_connection_id = [nil, *@proxy_args].join ':'
|
714
|
+
end
|
715
|
+
|
716
|
+
reconnect
|
717
|
+
reconnect_ssl
|
718
|
+
end
|
719
|
+
|
720
|
+
##
|
721
|
+
# Creates a URI for an HTTP proxy server from ENV variables.
|
722
|
+
#
|
723
|
+
# If +HTTP_PROXY+ is set a proxy will be returned.
|
724
|
+
#
|
725
|
+
# If +HTTP_PROXY_USER+ or +HTTP_PROXY_PASS+ are set the URI is given the
|
726
|
+
# indicated user and password unless HTTP_PROXY contains either of these in
|
727
|
+
# the URI.
|
728
|
+
#
|
729
|
+
# For Windows users, lowercase ENV variables are preferred over uppercase ENV
|
730
|
+
# variables.
|
731
|
+
|
732
|
+
def proxy_from_env
|
733
|
+
env_proxy = ENV['http_proxy'] || ENV['HTTP_PROXY']
|
734
|
+
|
735
|
+
return nil if env_proxy.nil? or env_proxy.empty?
|
736
|
+
|
737
|
+
uri = URI normalize_uri env_proxy
|
738
|
+
|
739
|
+
unless uri.user or uri.password then
|
740
|
+
uri.user = escape ENV['http_proxy_user'] || ENV['HTTP_PROXY_USER']
|
741
|
+
uri.password = escape ENV['http_proxy_pass'] || ENV['HTTP_PROXY_PASS']
|
742
|
+
end
|
743
|
+
|
744
|
+
uri
|
745
|
+
end
|
746
|
+
|
747
|
+
##
|
748
|
+
# Forces reconnection of HTTP connections.
|
749
|
+
|
750
|
+
def reconnect
|
751
|
+
@generation += 1
|
752
|
+
end
|
753
|
+
|
754
|
+
##
|
755
|
+
# Forces reconnection of SSL connections.
|
756
|
+
|
757
|
+
def reconnect_ssl
|
758
|
+
@ssl_generation += 1
|
759
|
+
end
|
760
|
+
|
761
|
+
##
|
762
|
+
# Finishes then restarts the Net::HTTP +connection+
|
763
|
+
|
764
|
+
def reset connection
|
765
|
+
Thread.current[@request_key].delete connection.object_id
|
766
|
+
Thread.current[@timeout_key].delete connection.object_id
|
767
|
+
|
768
|
+
finish connection
|
769
|
+
|
770
|
+
connection.start
|
771
|
+
rescue Errno::ECONNREFUSED
|
772
|
+
raise Error, "connection refused: #{connection.address}:#{connection.port}"
|
773
|
+
rescue Errno::EHOSTDOWN
|
774
|
+
raise Error, "host down: #{connection.address}:#{connection.port}"
|
775
|
+
end
|
776
|
+
|
777
|
+
##
|
778
|
+
# Makes a request on +uri+. If +req+ is nil a Net::HTTP::Get is performed
|
779
|
+
# against +uri+.
|
780
|
+
#
|
781
|
+
# If a block is passed #request behaves like Net::HTTP#request (the body of
|
782
|
+
# the response will not have been read).
|
783
|
+
#
|
784
|
+
# +req+ must be a Net::HTTPRequest subclass (see Net::HTTP for a list).
|
785
|
+
#
|
786
|
+
# If there is an error and the request is idempontent according to RFC 2616
|
787
|
+
# it will be retried automatically.
|
788
|
+
|
789
|
+
def request uri, req = nil, &block
|
790
|
+
retried = false
|
791
|
+
bad_response = false
|
792
|
+
|
793
|
+
req = Net::HTTP::Get.new uri.request_uri unless req
|
794
|
+
|
795
|
+
@headers.each do |pair|
|
796
|
+
req.add_field(*pair)
|
797
|
+
end
|
798
|
+
|
799
|
+
@override_headers.each do |name, value|
|
800
|
+
req[name] = value
|
801
|
+
end
|
802
|
+
|
803
|
+
unless req['Connection'] then
|
804
|
+
req.add_field 'Connection', 'keep-alive'
|
805
|
+
req.add_field 'Keep-Alive', @keep_alive
|
806
|
+
end
|
807
|
+
|
808
|
+
connection = connection_for uri
|
809
|
+
connection_id = connection.object_id
|
810
|
+
|
811
|
+
begin
|
812
|
+
Thread.current[@request_key][connection_id] += 1
|
813
|
+
response = connection.request req, &block
|
814
|
+
|
815
|
+
if connection_close?(req) or
|
816
|
+
(response.http_version <= '1.0' and
|
817
|
+
not connection_keep_alive?(response)) or
|
818
|
+
connection_close?(response) then
|
819
|
+
connection.finish
|
820
|
+
end
|
821
|
+
rescue Net::HTTPBadResponse => e
|
822
|
+
message = error_message connection
|
823
|
+
|
824
|
+
finish connection
|
825
|
+
|
826
|
+
raise Error, "too many bad responses #{message}" if
|
827
|
+
bad_response or not can_retry? req
|
828
|
+
|
829
|
+
bad_response = true
|
830
|
+
retry
|
831
|
+
rescue IOError, EOFError, Timeout::Error,
|
832
|
+
Errno::ECONNABORTED, Errno::ECONNRESET, Errno::EPIPE,
|
833
|
+
Errno::EINVAL, OpenSSL::SSL::SSLError => e
|
834
|
+
|
835
|
+
if retried or not can_retry? req
|
836
|
+
due_to = "(due to #{e.message} - #{e.class})"
|
837
|
+
message = error_message connection
|
838
|
+
|
839
|
+
finish connection
|
840
|
+
|
841
|
+
raise Error, "too many connection resets #{due_to} #{message}"
|
842
|
+
end
|
843
|
+
|
844
|
+
reset connection
|
845
|
+
|
846
|
+
retried = true
|
847
|
+
retry
|
848
|
+
ensure
|
849
|
+
Thread.current[@timeout_key][connection_id] = Time.now
|
850
|
+
end
|
851
|
+
|
852
|
+
@http_versions["#{uri.host}:#{uri.port}"] ||= response.http_version
|
853
|
+
|
854
|
+
response
|
855
|
+
end
|
856
|
+
|
857
|
+
##
|
858
|
+
# Shuts down all connections for +thread+.
|
859
|
+
#
|
860
|
+
# Uses the current thread by default.
|
861
|
+
#
|
862
|
+
# If you've used Net::HTTP::Persistent across multiple threads you should
|
863
|
+
# call this in each thread when you're done making HTTP requests.
|
864
|
+
#
|
865
|
+
# *NOTE*: Calling shutdown for another thread can be dangerous!
|
866
|
+
#
|
867
|
+
# If the thread is still using the connection it may cause an error! It is
|
868
|
+
# best to call #shutdown in the thread at the appropriate time instead!
|
869
|
+
|
870
|
+
def shutdown thread = Thread.current
|
871
|
+
generation = reconnect
|
872
|
+
cleanup generation, thread, @generation_key
|
873
|
+
|
874
|
+
ssl_generation = reconnect_ssl
|
875
|
+
cleanup ssl_generation, thread, @ssl_generation_key
|
876
|
+
|
877
|
+
thread[@request_key] = nil
|
878
|
+
thread[@timeout_key] = nil
|
879
|
+
end
|
880
|
+
|
881
|
+
##
|
882
|
+
# Shuts down all connections in all threads
|
883
|
+
#
|
884
|
+
# *NOTE*: THIS METHOD IS VERY DANGEROUS!
|
885
|
+
#
|
886
|
+
# Do not call this method if other threads are still using their
|
887
|
+
# connections! Call #shutdown at the appropriate time instead!
|
888
|
+
#
|
889
|
+
# Use this method only as a last resort!
|
890
|
+
|
891
|
+
def shutdown_in_all_threads
|
892
|
+
Thread.list.each do |thread|
|
893
|
+
shutdown thread
|
894
|
+
end
|
895
|
+
|
896
|
+
nil
|
897
|
+
end
|
898
|
+
|
899
|
+
##
|
900
|
+
# Enables SSL on +connection+
|
901
|
+
|
902
|
+
def ssl connection
|
903
|
+
connection.use_ssl = true
|
904
|
+
|
905
|
+
connection.ssl_version = @ssl_version if @ssl_version
|
906
|
+
|
907
|
+
connection.verify_mode = @verify_mode
|
908
|
+
|
909
|
+
if OpenSSL::SSL::VERIFY_PEER == OpenSSL::SSL::VERIFY_NONE and
|
910
|
+
not Object.const_defined?(:I_KNOW_THAT_OPENSSL_VERIFY_PEER_EQUALS_VERIFY_NONE_IS_WRONG) then
|
911
|
+
warn <<-WARNING
|
912
|
+
!!!SECURITY WARNING!!!
|
913
|
+
|
914
|
+
The SSL HTTP connection to:
|
915
|
+
|
916
|
+
#{connection.address}:#{connection.port}
|
917
|
+
|
918
|
+
!!!MAY NOT BE VERIFIED!!!
|
919
|
+
|
920
|
+
On your platform your OpenSSL implementation is broken.
|
921
|
+
|
922
|
+
There is no difference between the values of VERIFY_NONE and VERIFY_PEER.
|
923
|
+
|
924
|
+
This means that attempting to verify the security of SSL connections may not
|
925
|
+
work. This exposes you to man-in-the-middle exploits, snooping on the
|
926
|
+
contents of your connection and other dangers to the security of your data.
|
927
|
+
|
928
|
+
To disable this warning define the following constant at top-level in your
|
929
|
+
application:
|
930
|
+
|
931
|
+
I_KNOW_THAT_OPENSSL_VERIFY_PEER_EQUALS_VERIFY_NONE_IS_WRONG = nil
|
932
|
+
|
933
|
+
WARNING
|
934
|
+
end
|
935
|
+
|
936
|
+
if @ca_file then
|
937
|
+
connection.ca_file = @ca_file
|
938
|
+
connection.verify_mode = OpenSSL::SSL::VERIFY_PEER
|
939
|
+
connection.verify_callback = @verify_callback if @verify_callback
|
940
|
+
end
|
941
|
+
|
942
|
+
if @certificate and @private_key then
|
943
|
+
connection.cert = @certificate
|
944
|
+
connection.key = @private_key
|
945
|
+
end
|
946
|
+
|
947
|
+
connection.cert_store = if @cert_store then
|
948
|
+
@cert_store
|
949
|
+
else
|
950
|
+
store = OpenSSL::X509::Store.new
|
951
|
+
store.set_default_paths
|
952
|
+
store
|
953
|
+
end
|
954
|
+
end
|
955
|
+
|
956
|
+
##
|
957
|
+
# Finishes all connections that existed before the given SSL parameter
|
958
|
+
# +generation+.
|
959
|
+
|
960
|
+
def ssl_cleanup generation # :nodoc:
|
961
|
+
cleanup generation, Thread.current, @ssl_generation_key
|
962
|
+
end
|
963
|
+
|
964
|
+
##
|
965
|
+
# SSL version to use
|
966
|
+
|
967
|
+
def ssl_version= ssl_version
|
968
|
+
@ssl_version = ssl_version
|
969
|
+
|
970
|
+
reconnect_ssl
|
971
|
+
end if RUBY_VERSION > '1.9'
|
972
|
+
|
973
|
+
##
|
974
|
+
# Sets the HTTPS verify mode. Defaults to OpenSSL::SSL::VERIFY_PEER.
|
975
|
+
#
|
976
|
+
# Setting this to VERIFY_NONE is a VERY BAD IDEA and should NEVER be used.
|
977
|
+
# Securely transfer the correct certificate and update the default
|
978
|
+
# certificate store or set the ca file instead.
|
979
|
+
|
980
|
+
def verify_mode= verify_mode
|
981
|
+
@verify_mode = verify_mode
|
982
|
+
|
983
|
+
reconnect_ssl
|
984
|
+
end
|
985
|
+
|
986
|
+
##
|
987
|
+
# SSL verification callback.
|
988
|
+
|
989
|
+
def verify_callback= callback
|
990
|
+
@verify_callback = callback
|
991
|
+
|
992
|
+
reconnect_ssl
|
993
|
+
end
|
994
|
+
|
995
|
+
end
|
996
|
+
|
997
|
+
require 'net/http/persistent/ssl_reuse'
|
998
|
+
|