manticore 0.2.1-java → 0.3.0-java
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +1 -0
- data/CHANGELOG.md +31 -0
- data/README.md +164 -41
- data/gem-public_cert.pem +21 -0
- data/lib/manticore/client/proxies.rb +57 -0
- data/lib/manticore/client.rb +78 -63
- data/lib/manticore/cookie.rb +23 -3
- data/lib/manticore/response.rb +138 -36
- data/lib/manticore/stubbed_response.rb +91 -0
- data/lib/manticore/version.rb +1 -1
- data/lib/manticore.rb +15 -5
- data/manticore.gemspec +6 -0
- data/spec/manticore/client_proxy_spec.rb +92 -0
- data/spec/manticore/client_spec.rb +122 -37
- data/spec/manticore/cookie_spec.rb +23 -4
- data/spec/manticore/response_spec.rb +8 -0
- data/spec/manticore/stubbed_response_spec.rb +44 -0
- data/spec/spec_helper.rb +3 -0
- data.tar.gz.sig +0 -0
- metadata +33 -4
- metadata.gz.sig +0 -0
- data/lib/manticore/async_response.rb +0 -88
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 0078d294015d5ca40a8e4afadc48973fb9246de1
|
4
|
+
data.tar.gz: bcff310d2a92c1e592cac51f63859a3ee483b63f
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ef8bfdbaf46d56810e0f137fb4cf96425a46bc22ecf157f9451093bcf9acf721ff4cff86161593ec7c95112a64793856e5345c19454569d0b3ca457c3216d99f
|
7
|
+
data.tar.gz: cd2bcc65d8d377d8622fae139e0f4c5e91816bcd0cea3926c8d1a5503eda2e68c28ef179cae5f153a389cd8c1474bb5ffbb3f789408e197f8d297b385b741e0a
|
checksums.yaml.gz.sig
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
h>Ĭ��odd(��`���P��{F�҆<U����$�ѐ�/#�5q�6�>q�>��NJ�YE�'��P�[���8EP���>��a�W�"{vh��|�0���W`Or���x?a�L:{�=������� ѫ�5c���>.��n�[5n%ph����,�Ҳ{b��L�D��l�G����Q�-hn�K�:�X[v^d�ܨjj�X���ޔO�i!����tmK�U�k��z9���힏3Ͽ~�2�x�<uY
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,31 @@
|
|
1
|
+
## v0.3
|
2
|
+
### v0.3.0 (pending)
|
3
|
+
|
4
|
+
* Major refactor of `Response`/`AsyncResponse` to eliminate redundant code. `AsyncResponse` has been removed and
|
5
|
+
its functionality has been rolled into `Response`.
|
6
|
+
* Added `StubbedResponse`, a subclass of `Response`, to be used for stubbing requests/responses for testing.
|
7
|
+
* Added `Client#stub`, `Client#unstub` and `Client#respond_with`
|
8
|
+
* Responses are now lazy-evaluated by default (similar to how `AsyncResponse` used to behave). The following
|
9
|
+
rules apply:
|
10
|
+
* Synchronous responses which do NOT pass a block are lazy-evaluated the first time one of their results is requested.
|
11
|
+
* Synchronous responses which DO pass a block are evaluated immediately, and are passed to the handler block.
|
12
|
+
* Async responses are always evaluted when `Client#execute!` is called.
|
13
|
+
* You can evaluate a `Response` at any time by invoking `#call` on it. Invoking an async response before `Client#execute`
|
14
|
+
is called on it will cause `Client#execute` to throw an exception.
|
15
|
+
* Responses (both synchronous and async) may use on_success handlers and the like.
|
16
|
+
|
17
|
+
## v0.2
|
18
|
+
### v0.2.1
|
19
|
+
|
20
|
+
* Added basic auth support
|
21
|
+
* Added proxy support
|
22
|
+
* Added support for per-request cookies (as opposed to per-session cookies)
|
23
|
+
* Added a `Response#cookies` convenience method.
|
24
|
+
|
25
|
+
### v0.2.0
|
26
|
+
|
27
|
+
* Added documentation and licenses
|
28
|
+
* Significant performance overhaul
|
29
|
+
* Response handler blocks are now only yielded the Response. `#request` is available on
|
30
|
+
the response object.
|
31
|
+
* Patched httpclient.jar to address https://issues.apache.org/jira/browse/HTTPCLIENT-1461
|
data/README.md
CHANGED
@@ -34,6 +34,7 @@ Or install it yourself as:
|
|
34
34
|
* Transparent gzip and deflate handling
|
35
35
|
* Transparent cookie handling
|
36
36
|
* Both synchronous and asynchronous execution models
|
37
|
+
* Lazy evaluation
|
37
38
|
* Authentication
|
38
39
|
* Proxy support
|
39
40
|
* SSL
|
@@ -44,77 +45,199 @@ Or install it yourself as:
|
|
44
45
|
|
45
46
|
If you don't want to worry about setting up and maintaining client pools, Manticore comes with a facade that you can use to start making requests right away:
|
46
47
|
|
47
|
-
|
48
|
+
```ruby
|
49
|
+
document_body = Manticore.get("http://www.google.com/").body
|
50
|
+
```
|
48
51
|
|
49
52
|
Additionally, you can mix the `Manticore::Facade` into your own class for similar behavior:
|
50
53
|
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
54
|
+
```ruby
|
55
|
+
class MyClient
|
56
|
+
include Manticore::Facade
|
57
|
+
include_http_client user_agent: "MyClient/1.0"
|
58
|
+
end
|
55
59
|
|
56
|
-
|
60
|
+
response_code = MyClient.get("http://www.google.com/").code
|
61
|
+
```
|
57
62
|
|
58
63
|
Mixing the client into a class will create a new new pool. If you want to share a single pool between clients, specify the `shared_pool` option:
|
59
64
|
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
|
65
|
+
```ruby
|
66
|
+
class MyClient
|
67
|
+
include Manticore::Facade
|
68
|
+
include_http_client shared_pool: true
|
69
|
+
end
|
64
70
|
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
71
|
+
class MyOtherClient
|
72
|
+
include Manticore::Facade
|
73
|
+
include_http_client shared_pool: true
|
74
|
+
end
|
75
|
+
```
|
69
76
|
|
70
77
|
### More Control
|
71
78
|
|
72
79
|
Manticore is built around a connection pool. When you create a `Client`, you will pass various parameters that it will use to set up the pool.
|
73
80
|
|
74
|
-
|
81
|
+
```ruby
|
82
|
+
client = Manticore::Client.new(request_timeout: 5, connect_timeout: 5, socket_timeout: 5, pool_max: 10, pool_max_per_route: 2)
|
83
|
+
```
|
75
84
|
|
76
85
|
Then, you can make requests from the client. Pooling and route maximum constraints are automatically managed:
|
77
86
|
|
78
|
-
|
87
|
+
```ruby
|
88
|
+
response = client.get("http://www.google.com/")
|
89
|
+
body = response.body
|
90
|
+
```
|
79
91
|
|
80
92
|
It is recommend that you instantiate a client once, then re-use it, rather than instantiating a new client per request.
|
81
93
|
|
82
94
|
Additionally, if you pass a block to the initializer, the underlying [HttpClientBuilder](http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/HttpClientBuilder.html) and [RequestConfig.Builder](http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/config/RequestConfig.Builder.html) will be yielded so that you can operate on them directly:
|
83
95
|
|
84
|
-
|
85
|
-
|
86
|
-
|
96
|
+
```ruby
|
97
|
+
client = Manticore::Client.new(socket_timeout: 5) do |http_client_builder, request_builder|
|
98
|
+
http_client_builder.disable_redirect_handling
|
99
|
+
end
|
100
|
+
```
|
87
101
|
|
88
102
|
### Parallel execution
|
89
103
|
|
90
|
-
Manticore can perform
|
104
|
+
Manticore can perform concurrent execution of multiple requests.
|
91
105
|
|
92
|
-
|
106
|
+
```ruby
|
107
|
+
client = Manticore::Client.new
|
93
108
|
|
94
|
-
|
95
|
-
|
96
|
-
|
97
|
-
|
98
|
-
|
99
|
-
|
109
|
+
# These aren't actually executed until #execute! is called.
|
110
|
+
# You can define response handlers in a block when you queue the request:
|
111
|
+
client.async.get("http://www.google.com") {|req|
|
112
|
+
req.on_success do |response|
|
113
|
+
puts response.body
|
114
|
+
end
|
100
115
|
|
101
|
-
|
102
|
-
|
103
|
-
|
104
|
-
|
116
|
+
req.on_failure do |exception|
|
117
|
+
puts "Boom! #{exception.message}"
|
118
|
+
end
|
119
|
+
}
|
105
120
|
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
121
|
+
# ...or by invoking the method on the queued response returned:
|
122
|
+
response = client.async.get("http://www.yahoo.com")
|
123
|
+
response.on_success do |response|
|
124
|
+
puts "The length of the Yahoo! homepage is #{response.body.length}"
|
125
|
+
end
|
111
126
|
|
112
|
-
|
113
|
-
|
114
|
-
|
115
|
-
|
127
|
+
# ...or even by chaining them onto the call
|
128
|
+
client.async.get("http://bing.com").
|
129
|
+
on_success {|r| puts r.code }.
|
130
|
+
on_failure {|e| puts "on noes!"}
|
116
131
|
|
117
|
-
|
132
|
+
client.execute!
|
133
|
+
```
|
134
|
+
|
135
|
+
### Lazy Evaluation
|
136
|
+
|
137
|
+
Manticore attempts to avoid doing any actual work until right before you need results. As a result,
|
138
|
+
responses are lazy-evaluated as late as possible. The following rules apply:
|
139
|
+
|
140
|
+
1. Synchronous responses are evaluted when you call an accessor on them, like `#body` or `#headers`.
|
141
|
+
2. Synchronous responses which pass a handler block are evaluated immediately.
|
142
|
+
3. Asynchronous responses are always evaluated when you call `Client#execute!`
|
143
|
+
4. Background responses are always immediately evaluated, but return a `Future`.
|
144
|
+
|
145
|
+
As a result, with the exception of background requests, this allows you to attach handlers to synchronous
|
146
|
+
and asynchronous responses in the same fashion:
|
147
|
+
|
148
|
+
```ruby
|
149
|
+
# Response doesn't evaluate when you call get, since you don't need any results from it yet
|
150
|
+
response = client.get("http://google.com").on_success {|r| "Success handler!" }
|
151
|
+
# As soon as you request #body, the response will evaluate to a result.
|
152
|
+
body = response.body
|
153
|
+
|
154
|
+
response = client.async.get("http://google.com").on_success {|r| "Success handler!" }
|
155
|
+
client.execute!
|
156
|
+
body = response.body
|
157
|
+
```
|
158
|
+
|
159
|
+
If you want to make a response that is not lazy-evaluated, you can either pass a handler block to it, or you can
|
160
|
+
call `#call` on the resulting response:
|
161
|
+
|
162
|
+
```ruby
|
163
|
+
# This will evaluate immediately
|
164
|
+
client.get("http://google.com") {r| r.body }
|
165
|
+
|
166
|
+
# As will this, via explicit invocation of #call
|
167
|
+
client.get("http://google.com").call
|
168
|
+
```
|
169
|
+
|
170
|
+
### Stubbing
|
171
|
+
|
172
|
+
Manticore provides a stubbing interface somewhat similar to Typhoeus'
|
173
|
+
|
174
|
+
```ruby
|
175
|
+
client.stub("http://google.com", body: "response body", code: 200)
|
176
|
+
client.get("http://google.com") do |response|
|
177
|
+
response.body.should == "response body"
|
178
|
+
end
|
179
|
+
client.clear_stubs!
|
180
|
+
```
|
181
|
+
|
182
|
+
This works for async requests as well:
|
183
|
+
|
184
|
+
```ruby
|
185
|
+
client.stub("http://google.com", body: "response body", code: 200)
|
186
|
+
|
187
|
+
# The request to google.com returns a stub as expected
|
188
|
+
client.async.get("http://google.com").on_success do |response|
|
189
|
+
response.should be_a Manticore::ResponseStub
|
190
|
+
end
|
191
|
+
|
192
|
+
# Since yahoo.com isn't stubbed, a full request will be performed
|
193
|
+
client.async.get("http://yahoo.com").on_success do |response|
|
194
|
+
response.should be_a Manticore::Response
|
195
|
+
end
|
196
|
+
client.clear_stubs!
|
197
|
+
```
|
198
|
+
|
199
|
+
If you don't want to worry about stub teardown, you can just use `#respond_with`, which will stub the next
|
200
|
+
response the client makes with a ResponseStub rather than permitting it to execute a remote request.
|
201
|
+
|
202
|
+
```ruby
|
203
|
+
client.respond_with(body: "body").get("http://google.com") do |response|
|
204
|
+
response.body.should == "body"
|
205
|
+
end
|
206
|
+
```
|
207
|
+
|
208
|
+
You can also chain proxies to, say, stub an async request:
|
209
|
+
|
210
|
+
```ruby
|
211
|
+
response = client.async.respond_with(body: "response body").get("http://google.com")
|
212
|
+
client.execute!
|
213
|
+
|
214
|
+
response.body.should == "response body"
|
215
|
+
```
|
216
|
+
|
217
|
+
Additionally, you can stub and unstub individual URLs as desired:
|
218
|
+
```ruby
|
219
|
+
client.stub("http://google.com", body: "response body", code: 200)
|
220
|
+
client.stub("http://yahoo.com", body: "response body", code: 200)
|
221
|
+
|
222
|
+
# The request to google.com returns a stub as expected
|
223
|
+
client.get("http://google.com") do |response|
|
224
|
+
response.should be_a Manticore::ResponseStub
|
225
|
+
end
|
226
|
+
|
227
|
+
# After this point, yahoo will remain stubbed, while google will not.
|
228
|
+
client.unstub("http://google.com")
|
229
|
+
```
|
230
|
+
|
231
|
+
### Background requests
|
232
|
+
|
233
|
+
You might want to fire-and-forget requests without blocking your calling thread. You can do this with `Client#background`:
|
234
|
+
|
235
|
+
```ruby
|
236
|
+
future = client.background.get("http://google.com")
|
237
|
+
# The request is now running, but the calling thread isn't blocked
|
238
|
+
# Do whatever stuff you need to right now. At some point, if you want the result of the request, you can call `Future#get`:
|
239
|
+
response = future.get
|
240
|
+
```
|
118
241
|
|
119
242
|
## Contributing
|
120
243
|
|
data/gem-public_cert.pem
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
-----BEGIN CERTIFICATE-----
|
2
|
+
MIIDaDCCAlCgAwIBAgIBATANBgkqhkiG9w0BAQUFADA9MQ8wDQYDVQQDDAZjaGVh
|
3
|
+
bGQxFTATBgoJkiaJk/IsZAEZFgVnbWFpbDETMBEGCgmSJomT8ixkARkWA2NvbTAe
|
4
|
+
Fw0xNDAzMDEyMTE4MzJaFw0xNTAzMDEyMTE4MzJaMD0xDzANBgNVBAMMBmNoZWFs
|
5
|
+
ZDEVMBMGCgmSJomT8ixkARkWBWdtYWlsMRMwEQYKCZImiZPyLGQBGRYDY29tMIIB
|
6
|
+
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwcnX2yl+rbjUztC4V+iUJgWv
|
7
|
+
NPxqU4bQBaL+w00wVABWr04Hjg+cEkqiJ6A0kXxZIz5uXKhhvsaO50NvHfplVcUf
|
8
|
+
BxQabIfCS79xdvexXN0or3F5saatGaGa4cj/0uUHjX7w+K5MpEVfbjJp0uAKp62B
|
9
|
+
wUU2ilmn7EvZhEUPOMQi01t8z8OsOGc8kF2UtU1kGCxLV7eThWqu8CdXrux5E140
|
10
|
+
7SnFnPlnXNeHqwZdOMZzQ9PiTQAPCKO3AY0aBFQeG3wlFPqkcEjOrtV1h7werUwz
|
11
|
+
aNb4t5sAuu0f/9B646BOjiMgj1WeUlhgiAsaF5kVvLFNCxwS/xpcN3X01M2KdQID
|
12
|
+
AQABo3MwcTAJBgNVHRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUtFVpwfEG
|
13
|
+
78mBd2ulzsS+SlffdOcwGwYDVR0RBBQwEoEQY2hlYWxkQGdtYWlsLmNvbTAbBgNV
|
14
|
+
HRIEFDASgRBjaGVhbGRAZ21haWwuY29tMA0GCSqGSIb3DQEBBQUAA4IBAQANcBIJ
|
15
|
+
vIk27iNu4Ic6awNRy4Rd2YchzW4DaMhfpTXxtYWgAE502OdqUTFtmihd2PmnVRIP
|
16
|
+
xdP5K2SpbCLuLKFCZ825qp8R02JSNMdgi2d4Btl0vHSO8O3lQ1ZAOM1BZPn2+Kbn
|
17
|
+
KsiSURg3SnDXItcoedMsS+CQdNyBSdqvpkQ5gn22dN4BSwJg5ApwEYJ9tTuTpRXt
|
18
|
+
1KCQFe5hamoenfoHO6uUXPEs24PUHnXl8QibWCLbt1FIBpTrJYNGiMHHnB7ip+zv
|
19
|
+
E7PWS50D9moUJ6xWcemf0qKYC87qBFh0ng73awjG9uf+13lMslqJRMtek8C92cvh
|
20
|
+
+R9zgQlbeNjy9O1i
|
21
|
+
-----END CERTIFICATE-----
|
@@ -0,0 +1,57 @@
|
|
1
|
+
module Manticore
|
2
|
+
class Client
|
3
|
+
module ProxiesInterface
|
4
|
+
def respond_with(stubs)
|
5
|
+
StubProxy.new(self, stubs)
|
6
|
+
end
|
7
|
+
|
8
|
+
# Causes the next request to be made asynchronously
|
9
|
+
def async
|
10
|
+
AsyncProxy.new(self)
|
11
|
+
end
|
12
|
+
|
13
|
+
# Causes the next request to be made immediately in the background
|
14
|
+
def background
|
15
|
+
BackgroundProxy.new(self)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
class BaseProxy
|
20
|
+
include ProxiesInterface
|
21
|
+
def initialize(client)
|
22
|
+
@client = client
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
class AsyncProxy < BaseProxy
|
27
|
+
%w(get put head post options patch).each do |func|
|
28
|
+
define_method func do |url, options = {}, &block|
|
29
|
+
@client.send(func, url, options.merge(async: true), &block)
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
class StubProxy < BaseProxy
|
35
|
+
def initialize(client, stubs)
|
36
|
+
super(client)
|
37
|
+
@stubs = stubs
|
38
|
+
end
|
39
|
+
|
40
|
+
%w(get put head post options patch).each do |func|
|
41
|
+
define_method func do |url, options = {}, &block|
|
42
|
+
@client.stub(url, @stubs)
|
43
|
+
@client.send(func, url, options, &block).complete { @client.unstub url }
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
class BackgroundProxy < BaseProxy
|
49
|
+
%w(get put head post options patch).each do |func|
|
50
|
+
define_method func do |url, options = {}, &block|
|
51
|
+
request = @client.send(func, url, options.merge(async: true), &block)
|
52
|
+
@client.executor.java_method(:submit, [java.util.concurrent.Callable.java_class]).call request
|
53
|
+
end
|
54
|
+
end
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
data/lib/manticore/client.rb
CHANGED
@@ -37,7 +37,12 @@ module Manticore
|
|
37
37
|
#
|
38
38
|
# @!macro [new] http_method_shared_sync
|
39
39
|
# @example Simple usage
|
40
|
-
# client.$0("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"})
|
40
|
+
# body = client.$0("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).body
|
41
|
+
# @example Passing a block as the success handler:
|
42
|
+
# body = client.$0("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}) {|response| response.body }
|
43
|
+
# @example Explicit success handler:
|
44
|
+
# body = client.$0("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).
|
45
|
+
# on_success {|response| response.body }
|
41
46
|
# @macro http_method_shared
|
42
47
|
# @macro http_request_exceptions
|
43
48
|
#
|
@@ -66,13 +71,11 @@ module Manticore
|
|
66
71
|
include_package "java.util.concurrent"
|
67
72
|
include_package "org.apache.http.client.protocol"
|
68
73
|
java_import "org.apache.http.HttpHost"
|
74
|
+
include ProxiesInterface
|
69
75
|
|
70
76
|
# The default maximum pool size for requests
|
71
77
|
DEFAULT_MAX_POOL_SIZE = 50
|
72
78
|
|
73
|
-
# The default maximum number of threads per route that will be permitted
|
74
|
-
DEFAULT_MAX_PER_ROUTE = 10
|
75
|
-
|
76
79
|
DEFAULT_REQUEST_TIMEOUT = 60
|
77
80
|
DEFAULT_SOCKET_TIMEOUT = 10
|
78
81
|
DEFAULT_CONNECT_TIMEOUT = 10
|
@@ -99,7 +102,7 @@ module Manticore
|
|
99
102
|
# @param options [Hash] Client pool options
|
100
103
|
# @option options [String] user_agent The user agent used in requests.
|
101
104
|
# @option options [Integer] pool_max (50) The maximum number of active connections in the pool
|
102
|
-
# @option options [integer] pool_max_per_route (
|
105
|
+
# @option options [integer] pool_max_per_route (pool_max) Sets the maximum number of active connections for a given target endpoint
|
103
106
|
# @option options [boolean] cookies (true) enable or disable automatic cookie management between requests
|
104
107
|
# @option options [boolean] compression (true) enable or disable transparent gzip/deflate support
|
105
108
|
# @option options [integer] request_timeout (60) Sets the timeout for requests. Raises {Manticore::Timeout} on failure.
|
@@ -112,6 +115,9 @@ module Manticore
|
|
112
115
|
# @option options [String] proxy Proxy host in form: http://proxy.org:1234
|
113
116
|
# @option options [Hash] proxy Proxy host in form: {host: 'proxy.org'[, port: 80[, scheme: 'http']]}
|
114
117
|
# @option options [URI] proxy Proxy host as a URI object
|
118
|
+
# @option options [Boolean/Integer] keepalive (true) Whether to allow connections to be reused. Defaults to true. If an integer,
|
119
|
+
# then connections will be kept alive for this long when Connection: keep-alive
|
120
|
+
# is sent, but no Keep-Alive header is sent.
|
115
121
|
def initialize(options = {})
|
116
122
|
builder = client_builder
|
117
123
|
builder.set_user_agent options.fetch(:user_agent, "Manticore #{VERSION}")
|
@@ -121,10 +127,20 @@ module Manticore
|
|
121
127
|
builder.set_proxy get_proxy_host(options[:proxy]) if options.key?(:proxy)
|
122
128
|
|
123
129
|
# This should make it easier to reuse connections
|
124
|
-
|
125
|
-
builder.
|
130
|
+
# TODO: Determine what this actually does!
|
131
|
+
# builder.disable_connection_state
|
132
|
+
|
133
|
+
keepalive = options.fetch(:keepalive, true)
|
134
|
+
if keepalive == false
|
135
|
+
builder.set_connection_reuse_strategy {|response, context| false }
|
136
|
+
else
|
137
|
+
builder.set_connection_reuse_strategy DefaultConnectionReuseStrategy.new
|
138
|
+
end
|
139
|
+
|
140
|
+
socket_config_builder = SocketConfig.custom
|
141
|
+
socket_config_builder.setSoTimeout( options.fetch(:socket_timeout, DEFAULT_SOCKET_TIMEOUT) * 1000 )
|
142
|
+
builder.set_default_socket_config socket_config_builder.build
|
126
143
|
|
127
|
-
# socket_config = SocketConfig.custom.set_tcp_no_delay(true).build
|
128
144
|
builder.set_connection_manager pool(options)
|
129
145
|
|
130
146
|
request_config = RequestConfig.custom
|
@@ -134,7 +150,6 @@ module Manticore
|
|
134
150
|
request_config.set_max_redirects options.fetch(:max_redirects, DEFAULT_MAX_REDIRECTS)
|
135
151
|
request_config.set_expect_continue_enabled options.fetch(:expect_continue, DEFAULT_EXPECT_CONTINUE)
|
136
152
|
request_config.set_stale_connection_check_enabled options.fetch(:stale_check, DEFAULT_STALE_CHECK)
|
137
|
-
# request_config.set_authentication_enabled options.fetch(:use_auth, false)
|
138
153
|
request_config.set_circular_redirects_allowed false
|
139
154
|
|
140
155
|
yield builder, request_config if block_given?
|
@@ -143,6 +158,18 @@ module Manticore
|
|
143
158
|
@client = builder.build
|
144
159
|
@options = options
|
145
160
|
@async_requests = []
|
161
|
+
@stubs = {}
|
162
|
+
end
|
163
|
+
|
164
|
+
# Return a hash of statistics about this client's HTTP connection pool
|
165
|
+
def pool_stats
|
166
|
+
stats = @pool.get_total_stats
|
167
|
+
{
|
168
|
+
max: stats.get_max,
|
169
|
+
leased: stats.get_leased,
|
170
|
+
pending: stats.get_pending,
|
171
|
+
available: stats.get_available
|
172
|
+
}
|
146
173
|
end
|
147
174
|
|
148
175
|
### Sync methods
|
@@ -177,58 +204,39 @@ module Manticore
|
|
177
204
|
request HttpDelete, url, options, &block
|
178
205
|
end
|
179
206
|
|
207
|
+
# Perform a HTTP OPTIONS request
|
180
208
|
# @macro http_method_shared_sync
|
181
209
|
def options(url, options = {}, &block)
|
182
210
|
request HttpOptions, url, options, &block
|
183
211
|
end
|
184
212
|
|
213
|
+
# Perform a HTTP PATCH request
|
185
214
|
# @macro http_method_shared_sync_with_body
|
186
215
|
def patch(url, options = {}, &block)
|
187
216
|
request HttpPatch, url, options, &block
|
188
217
|
end
|
189
218
|
|
190
|
-
|
191
|
-
|
192
|
-
|
193
|
-
|
194
|
-
def async_get(url, options = {}, &block)
|
195
|
-
get url, options.merge(async: true), &block
|
196
|
-
end
|
197
|
-
|
198
|
-
# Queue an asynchronous HTTP HEAD request
|
199
|
-
# @macro http_method_shared_async
|
200
|
-
def async_head(url, options = {}, &block)
|
201
|
-
head url, options.merge(async: true), &block
|
202
|
-
end
|
203
|
-
|
204
|
-
# Queue an asynchronous HTTP PUT request
|
205
|
-
# @macro http_method_shared_async_with_body
|
206
|
-
def async_put(url, options = {}, &block)
|
207
|
-
put url, options.merge(async: true), &block
|
208
|
-
end
|
209
|
-
|
210
|
-
# Queue an asynchronous HTTP POST request
|
211
|
-
# @macro http_method_shared_async_with_body
|
212
|
-
def async_post(url, options = {}, &block)
|
213
|
-
post url, options.merge(async: true), &block
|
219
|
+
%w(get put head post options patch).each do |func|
|
220
|
+
define_method "#{func}!" do |url, options, &block|
|
221
|
+
send(func, url, options, &block).call
|
222
|
+
end
|
214
223
|
end
|
215
224
|
|
216
|
-
#
|
217
|
-
# @
|
218
|
-
|
219
|
-
|
225
|
+
# Cause this client to return a stubbed response for this URL
|
226
|
+
# @param url [String] URL to stub for
|
227
|
+
# @param stubs [Hash] Hash of options to return for the stubbed response
|
228
|
+
def stub(url, stubs)
|
229
|
+
@stubs[url] = stubs
|
220
230
|
end
|
221
231
|
|
222
|
-
#
|
223
|
-
|
224
|
-
|
225
|
-
options url, options.merge(async: true), &block
|
232
|
+
# Cause this client to unstubbed previously-stubbed URL
|
233
|
+
def unstub(url)
|
234
|
+
@stubs.delete(url)
|
226
235
|
end
|
227
236
|
|
228
|
-
#
|
229
|
-
|
230
|
-
|
231
|
-
patch url, options.merge(async: true), &block
|
237
|
+
# Wipe all currently-set stubs.
|
238
|
+
def clear_stubs!
|
239
|
+
@stubs.clear
|
232
240
|
end
|
233
241
|
|
234
242
|
# Remove all pending asynchronous requests.
|
@@ -243,9 +251,16 @@ module Manticore
|
|
243
251
|
#
|
244
252
|
# @return [Array] An array of the responses from the requests executed.
|
245
253
|
def execute!
|
246
|
-
|
254
|
+
method = executor.java_method(:submit, [java.util.concurrent.Callable.java_class])
|
255
|
+
result = @async_requests.map {|r| method.call r }
|
247
256
|
@async_requests.clear
|
248
|
-
result
|
257
|
+
result.map(&:get)
|
258
|
+
end
|
259
|
+
|
260
|
+
# Get at the underlying ExecutorService used to invoke asynchronous calls.
|
261
|
+
def executor
|
262
|
+
create_executor_if_needed
|
263
|
+
@executor
|
249
264
|
end
|
250
265
|
|
251
266
|
protected
|
@@ -262,7 +277,7 @@ module Manticore
|
|
262
277
|
@pool ||= begin
|
263
278
|
@max_pool_size = options.fetch(:pool_max, DEFAULT_MAX_POOL_SIZE)
|
264
279
|
cm = pool_builder
|
265
|
-
cm.set_default_max_per_route options.fetch(:pool_max_per_route,
|
280
|
+
cm.set_default_max_per_route options.fetch(:pool_max_per_route, @max_pool_size)
|
266
281
|
cm.set_max_total @max_pool_size
|
267
282
|
Thread.new {
|
268
283
|
loop {
|
@@ -283,35 +298,35 @@ module Manticore
|
|
283
298
|
def request(klass, url, options, &block)
|
284
299
|
req, context = request_from_options(klass, url, options)
|
285
300
|
if options.delete(:async)
|
286
|
-
async_request req, context
|
301
|
+
async_request req, context
|
287
302
|
else
|
288
303
|
sync_request req, context, &block
|
289
304
|
end
|
290
305
|
end
|
291
306
|
|
292
|
-
def async_request(request, context
|
307
|
+
def async_request(request, context)
|
293
308
|
create_executor_if_needed
|
294
|
-
response =
|
309
|
+
response = response_object_for(@client, request, context)
|
295
310
|
@async_requests << response
|
296
311
|
response
|
297
312
|
end
|
298
313
|
|
299
314
|
def sync_request(request, context, &block)
|
300
|
-
response =
|
301
|
-
|
302
|
-
|
315
|
+
response = response_object_for(@client, request, context, &block)
|
316
|
+
if block_given?
|
317
|
+
response.call
|
318
|
+
else
|
303
319
|
response
|
304
|
-
rescue Java::JavaNet::SocketTimeoutException, Java::OrgApacheHttpConn::ConnectTimeoutException, Java::OrgApacheHttp::NoHttpResponseException => e
|
305
|
-
raise Manticore::Timeout.new(e.get_cause)
|
306
|
-
rescue Java::JavaNet::SocketException => e
|
307
|
-
raise Manticore::SocketException.new(e.get_cause)
|
308
|
-
rescue Java::OrgApacheHttpClient::ClientProtocolException, Java::JavaxNetSsl::SSLHandshakeException,
|
309
|
-
Java::OrgApacheHttpConn::HttpHostConnectException, Java::JavaxNetSsl::SSLException => e
|
310
|
-
raise Manticore::ClientProtocolException.new(e.get_cause)
|
311
|
-
rescue Java::JavaNet::UnknownHostException => e
|
312
|
-
raise Manticore::ResolutionFailure.new(e.get_cause)
|
313
320
|
end
|
321
|
+
end
|
314
322
|
|
323
|
+
def response_object_for(client, request, context, &block)
|
324
|
+
request_uri = request.getURI.to_s
|
325
|
+
if @stubs.key?(request_uri)
|
326
|
+
StubbedResponse.new(client, request, context, &block).stub( @stubs[request_uri] )
|
327
|
+
else
|
328
|
+
Response.new(client, request, context, &block)
|
329
|
+
end
|
315
330
|
end
|
316
331
|
|
317
332
|
def uri_from_url_and_options(url, options)
|