async-http 0.52.3 → 0.54.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/bake/async/http/h2spec.rb +1 -1
- data/lib/async/http/body/delayed.rb +2 -2
- data/lib/async/http/body/hijack.rb +5 -0
- data/lib/async/http/body/pipe.rb +15 -7
- data/lib/async/http/body/slowloris.rb +2 -2
- data/lib/async/http/body/stream.rb +1 -1
- data/lib/async/http/client.rb +3 -3
- data/lib/async/http/endpoint.rb +2 -2
- data/lib/async/http/internet.rb +4 -0
- data/lib/async/http/protocol/http1/connection.rb +0 -5
- data/lib/async/http/protocol/http1/server.rb +4 -3
- data/lib/async/http/protocol/http2.rb +0 -18
- data/lib/async/http/protocol/http2/output.rb +1 -1
- data/lib/async/http/protocol/http2/request.rb +0 -38
- data/lib/async/http/protocol/http2/response.rb +6 -13
- data/lib/async/http/protocol/request.rb +0 -4
- data/lib/async/http/proxy.rb +24 -8
- data/lib/async/http/server.rb +3 -3
- data/lib/async/http/version.rb +1 -1
- metadata +23 -65
- data/.editorconfig +0 -6
- data/.github/workflows/development.yml +0 -52
- data/.gitignore +0 -15
- data/.rspec +0 -3
- data/.travis.yml +0 -35
- data/README.md +0 -365
- data/async-http.gemspec +0 -39
- data/bake.rb +0 -0
- data/examples/compare/Gemfile +0 -9
- data/examples/compare/benchmark.rb +0 -78
- data/examples/download/chunked.rb +0 -86
- data/examples/fetch/Gemfile +0 -3
- data/examples/fetch/Gemfile.lock +0 -74
- data/examples/fetch/README.md +0 -3
- data/examples/fetch/config.ru +0 -28
- data/examples/fetch/public/index.html +0 -23
- data/examples/fetch/public/stream.js +0 -56
- data/examples/google/search.rb +0 -47
- data/examples/request.rb +0 -38
- data/examples/stream/stop.rb +0 -28
- data/examples/trenni/Gemfile +0 -5
- data/examples/trenni/streaming.rb +0 -35
- data/examples/upload/client.rb +0 -39
- data/examples/upload/data.txt +0 -41
- data/examples/upload/server.rb +0 -19
- data/examples/upload/upload.rb +0 -26
- data/gems.rb +0 -11
data/.editorconfig
DELETED
@@ -1,52 +0,0 @@
|
|
1
|
-
name: Development
|
2
|
-
|
3
|
-
on: [push, pull_request]
|
4
|
-
|
5
|
-
jobs:
|
6
|
-
test:
|
7
|
-
runs-on: ${{matrix.os}}-latest
|
8
|
-
continue-on-error: ${{matrix.experimental}}
|
9
|
-
|
10
|
-
strategy:
|
11
|
-
matrix:
|
12
|
-
experimental: [false]
|
13
|
-
|
14
|
-
os:
|
15
|
-
- ubuntu
|
16
|
-
- macos
|
17
|
-
|
18
|
-
ruby:
|
19
|
-
- 2.5
|
20
|
-
- 2.6
|
21
|
-
- 2.7
|
22
|
-
|
23
|
-
include:
|
24
|
-
- experimental: true
|
25
|
-
os: ubuntu
|
26
|
-
ruby: truffleruby
|
27
|
-
- experimental: true
|
28
|
-
os: ubuntu
|
29
|
-
ruby: jruby
|
30
|
-
- experimental: true
|
31
|
-
os: ubuntu
|
32
|
-
ruby: head
|
33
|
-
- experimental: true
|
34
|
-
os: ubuntu
|
35
|
-
ruby: 2.6
|
36
|
-
env: COVERAGE=PartialSummary,Coveralls
|
37
|
-
|
38
|
-
steps:
|
39
|
-
- uses: actions/checkout@v1
|
40
|
-
- uses: ruby/setup-ruby@v1
|
41
|
-
with:
|
42
|
-
ruby-version: ${{matrix.ruby}}
|
43
|
-
|
44
|
-
- name: Installing packages (ubuntu)
|
45
|
-
if: matrix.os == 'ubuntu'
|
46
|
-
run: sudo apt-get install apache2-utils
|
47
|
-
|
48
|
-
- name: Install dependencies
|
49
|
-
run: ${{matrix.env}} bundle install
|
50
|
-
|
51
|
-
- name: Run tests
|
52
|
-
run: ${{matrix.env}} bundle exec rspec
|
data/.gitignore
DELETED
data/.rspec
DELETED
data/.travis.yml
DELETED
@@ -1,35 +0,0 @@
|
|
1
|
-
language: ruby
|
2
|
-
dist: xenial
|
3
|
-
cache: bundler
|
4
|
-
|
5
|
-
addons:
|
6
|
-
apt:
|
7
|
-
packages:
|
8
|
-
- wrk
|
9
|
-
- apache2-utils
|
10
|
-
homebrew:
|
11
|
-
packages:
|
12
|
-
- wrk
|
13
|
-
|
14
|
-
script: bundle exec rspec
|
15
|
-
|
16
|
-
matrix:
|
17
|
-
include:
|
18
|
-
- rvm: 2.5
|
19
|
-
- rvm: 2.6
|
20
|
-
- rvm: 2.7
|
21
|
-
- rvm: 2.6
|
22
|
-
env: COVERAGE=PartialSummary,Coveralls
|
23
|
-
- rvm: truffleruby
|
24
|
-
- rvm: jruby-head
|
25
|
-
env: JRUBY_OPTS="--debug -X+O"
|
26
|
-
- rvm: ruby-head
|
27
|
-
- rvm: 2.7
|
28
|
-
os: osx
|
29
|
-
allow_failures:
|
30
|
-
- rvm: ruby-head
|
31
|
-
- rvm: jruby-head
|
32
|
-
- rvm: truffleruby
|
33
|
-
|
34
|
-
# after_success:
|
35
|
-
# - bundle exec rake h2spec:all
|
data/README.md
DELETED
@@ -1,365 +0,0 @@
|
|
1
|
-
# Async::HTTP
|
2
|
-
|
3
|
-
An asynchronous client and server implementation of HTTP/1.0, HTTP/1.1 and HTTP/2 including TLS. Support for streaming requests and responses. Built on top of [async] and [async-io]. [falcon] provides a rack-compatible server.
|
4
|
-
|
5
|
-
[](https://travis-ci.com/socketry/async-http)
|
6
|
-
[](https://codeclimate.com/github/socketry/async-http)
|
7
|
-
[](https://coveralls.io/r/socketry/async-http)
|
8
|
-
|
9
|
-
[async]: https://github.com/socketry/async
|
10
|
-
[async-io]: https://github.com/socketry/async-io
|
11
|
-
[falcon]: https://github.com/socketry/falcon
|
12
|
-
|
13
|
-
## Installation
|
14
|
-
|
15
|
-
Add this line to your application's Gemfile:
|
16
|
-
|
17
|
-
```ruby
|
18
|
-
gem 'async-http'
|
19
|
-
```
|
20
|
-
|
21
|
-
And then execute:
|
22
|
-
|
23
|
-
$ bundle
|
24
|
-
|
25
|
-
Or install it yourself as:
|
26
|
-
|
27
|
-
$ gem install async-http
|
28
|
-
|
29
|
-
## Usage
|
30
|
-
|
31
|
-
### Post JSON data
|
32
|
-
|
33
|
-
Here is an example showing how to post a data structure as JSON to a remote resource:
|
34
|
-
|
35
|
-
```ruby
|
36
|
-
#!/usr/bin/env ruby
|
37
|
-
|
38
|
-
require 'json'
|
39
|
-
require 'async'
|
40
|
-
require 'async/http/internet'
|
41
|
-
|
42
|
-
data = {'life' => 42}
|
43
|
-
|
44
|
-
Async do
|
45
|
-
# Make a new internet:
|
46
|
-
internet = Async::HTTP::Internet.new
|
47
|
-
|
48
|
-
# Prepare the request:
|
49
|
-
headers = [['accept', 'application/json']]
|
50
|
-
body = [JSON.dump(data)]
|
51
|
-
|
52
|
-
# Issues a POST request:
|
53
|
-
response = internet.post("https://httpbin.org/anything", headers, body)
|
54
|
-
|
55
|
-
# Save the response body to a local file:
|
56
|
-
pp JSON.parse(response.read)
|
57
|
-
ensure
|
58
|
-
# The internet is closed for business:
|
59
|
-
internet.close
|
60
|
-
end
|
61
|
-
```
|
62
|
-
|
63
|
-
Consider using [async-rest](https://github.com/socketry/async-rest) instead.
|
64
|
-
|
65
|
-
### Multiple Requests
|
66
|
-
|
67
|
-
To issue multiple requests concurrently, you should use a barrier, e.g.
|
68
|
-
|
69
|
-
```ruby
|
70
|
-
#!/usr/bin/env ruby
|
71
|
-
|
72
|
-
require 'async'
|
73
|
-
require 'async/barrier'
|
74
|
-
require 'async/http/internet'
|
75
|
-
|
76
|
-
TOPICS = ["ruby", "python", "rust"]
|
77
|
-
|
78
|
-
Async do
|
79
|
-
internet = Async::HTTP::Internet.new
|
80
|
-
barrier = Async::Barrier.new
|
81
|
-
|
82
|
-
# Spawn an asynchronous task for each topic:
|
83
|
-
TOPICS.each do |topic|
|
84
|
-
barrier.async do
|
85
|
-
response = internet.get "https://www.google.com/search?q=#{topic}"
|
86
|
-
puts "Found #{topic}: #{response.read.scan(topic).size} times."
|
87
|
-
end
|
88
|
-
end
|
89
|
-
|
90
|
-
# Ensure we wait for all requests to complete before continuing:
|
91
|
-
barrier.wait
|
92
|
-
ensure
|
93
|
-
internet&.close
|
94
|
-
end
|
95
|
-
```
|
96
|
-
|
97
|
-
#### Limiting Requests
|
98
|
-
|
99
|
-
If you need to limit the number of simultaneous requests, use a semaphore.
|
100
|
-
|
101
|
-
```ruby
|
102
|
-
#!/usr/bin/env ruby
|
103
|
-
|
104
|
-
require 'async'
|
105
|
-
require 'async/barrier'
|
106
|
-
require 'async/semaphore'
|
107
|
-
require 'async/http/internet'
|
108
|
-
|
109
|
-
TOPICS = ["ruby", "python", "rust"]
|
110
|
-
|
111
|
-
Async do
|
112
|
-
internet = Async::HTTP::Internet.new
|
113
|
-
barrier = Async::Barrier.new
|
114
|
-
semaphore = Async::Semaphore.new(2, parent: barrier)
|
115
|
-
|
116
|
-
# Spawn an asynchronous task for each topic:
|
117
|
-
TOPICS.each do |topic|
|
118
|
-
semaphore.async do
|
119
|
-
response = internet.get "https://www.google.com/search?q=#{topic}"
|
120
|
-
puts "Found #{topic}: #{response.read.scan(topic).size} times."
|
121
|
-
end
|
122
|
-
end
|
123
|
-
|
124
|
-
# Ensure we wait for all requests to complete before continuing:
|
125
|
-
barrier.wait
|
126
|
-
ensure
|
127
|
-
internet&.close
|
128
|
-
end
|
129
|
-
```
|
130
|
-
|
131
|
-
### Downloading a File
|
132
|
-
|
133
|
-
Here is an example showing how to download a file and save it to a local path:
|
134
|
-
|
135
|
-
```ruby
|
136
|
-
#!/usr/bin/env ruby
|
137
|
-
|
138
|
-
require 'async'
|
139
|
-
require 'async/http/internet'
|
140
|
-
|
141
|
-
Async do
|
142
|
-
# Make a new internet:
|
143
|
-
internet = Async::HTTP::Internet.new
|
144
|
-
|
145
|
-
# Issues a GET request to Google:
|
146
|
-
response = internet.get("https://www.google.com/search?q=kittens")
|
147
|
-
|
148
|
-
# Save the response body to a local file:
|
149
|
-
response.save("/tmp/search.html")
|
150
|
-
ensure
|
151
|
-
# The internet is closed for business:
|
152
|
-
internet.close
|
153
|
-
end
|
154
|
-
```
|
155
|
-
|
156
|
-
### Basic Client/Server
|
157
|
-
|
158
|
-
Here is a basic example of a client/server running in the same reactor:
|
159
|
-
|
160
|
-
```ruby
|
161
|
-
#!/usr/bin/env ruby
|
162
|
-
|
163
|
-
require 'async'
|
164
|
-
require 'async/http/server'
|
165
|
-
require 'async/http/client'
|
166
|
-
require 'async/http/endpoint'
|
167
|
-
require 'async/http/protocol/response'
|
168
|
-
|
169
|
-
endpoint = Async::HTTP::Endpoint.parse('http://127.0.0.1:9294')
|
170
|
-
|
171
|
-
app = lambda do |request|
|
172
|
-
Protocol::HTTP::Response[200, {}, ["Hello World"]]
|
173
|
-
end
|
174
|
-
|
175
|
-
server = Async::HTTP::Server.new(app, endpoint)
|
176
|
-
client = Async::HTTP::Client.new(endpoint)
|
177
|
-
|
178
|
-
Async do |task|
|
179
|
-
server_task = task.async do
|
180
|
-
server.run
|
181
|
-
end
|
182
|
-
|
183
|
-
response = client.get("/")
|
184
|
-
|
185
|
-
puts response.status
|
186
|
-
puts response.read
|
187
|
-
|
188
|
-
server_task.stop
|
189
|
-
end
|
190
|
-
```
|
191
|
-
|
192
|
-
### Advanced Verification
|
193
|
-
|
194
|
-
You can hook into SSL certificate verification to improve server verification.
|
195
|
-
|
196
|
-
```ruby
|
197
|
-
require 'async'
|
198
|
-
require 'async/http'
|
199
|
-
|
200
|
-
# These are generated from the certificate chain that the server presented.
|
201
|
-
trusted_fingerprints = {
|
202
|
-
"dac9024f54d8f6df94935fb1732638ca6ad77c13" => true,
|
203
|
-
"e6a3b45b062d509b3382282d196efe97d5956ccb" => true,
|
204
|
-
"07d63f4c05a03f1c306f9941b8ebf57598719ea2" => true,
|
205
|
-
"e8d994f44ff20dc78dbff4e59d7da93900572bbf" => true,
|
206
|
-
}
|
207
|
-
|
208
|
-
Async do
|
209
|
-
endpoint = Async::HTTP::Endpoint.parse("https://www.codeotaku.com/index")
|
210
|
-
|
211
|
-
# This is a quick hack/POC:
|
212
|
-
ssl_context = endpoint.ssl_context
|
213
|
-
|
214
|
-
ssl_context.verify_callback = proc do |verified, store_context|
|
215
|
-
certificate = store_context.current_cert
|
216
|
-
fingerprint = OpenSSL::Digest::SHA1.new(certificate.to_der).to_s
|
217
|
-
|
218
|
-
if trusted_fingerprints.include? fingerprint
|
219
|
-
true
|
220
|
-
else
|
221
|
-
Async.logger.warn("Untrusted Certificate Fingerprint"){fingerprint}
|
222
|
-
false
|
223
|
-
end
|
224
|
-
end
|
225
|
-
|
226
|
-
endpoint = endpoint.with(ssl_context: ssl_context)
|
227
|
-
|
228
|
-
client = Async::HTTP::Client.new(endpoint)
|
229
|
-
|
230
|
-
response = client.get(endpoint.path)
|
231
|
-
|
232
|
-
pp response.status, response.headers.fields, response.read
|
233
|
-
end
|
234
|
-
```
|
235
|
-
|
236
|
-
## Performance
|
237
|
-
|
238
|
-
On a 4-core 8-thread i7, running `ab` which uses discrete (non-keep-alive) connections:
|
239
|
-
|
240
|
-
```
|
241
|
-
$ ab -c 8 -t 10 http://127.0.0.1:9294/
|
242
|
-
This is ApacheBench, Version 2.3 <$Revision: 1757674 $>
|
243
|
-
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
|
244
|
-
Licensed to The Apache Software Foundation, http://www.apache.org/
|
245
|
-
|
246
|
-
Benchmarking 127.0.0.1 (be patient)
|
247
|
-
Completed 5000 requests
|
248
|
-
Completed 10000 requests
|
249
|
-
Completed 15000 requests
|
250
|
-
Completed 20000 requests
|
251
|
-
Completed 25000 requests
|
252
|
-
Completed 30000 requests
|
253
|
-
Completed 35000 requests
|
254
|
-
Completed 40000 requests
|
255
|
-
Completed 45000 requests
|
256
|
-
Completed 50000 requests
|
257
|
-
Finished 50000 requests
|
258
|
-
|
259
|
-
|
260
|
-
Server Software:
|
261
|
-
Server Hostname: 127.0.0.1
|
262
|
-
Server Port: 9294
|
263
|
-
|
264
|
-
Document Path: /
|
265
|
-
Document Length: 13 bytes
|
266
|
-
|
267
|
-
Concurrency Level: 8
|
268
|
-
Time taken for tests: 1.869 seconds
|
269
|
-
Complete requests: 50000
|
270
|
-
Failed requests: 0
|
271
|
-
Total transferred: 2450000 bytes
|
272
|
-
HTML transferred: 650000 bytes
|
273
|
-
Requests per second: 26755.55 [#/sec] (mean)
|
274
|
-
Time per request: 0.299 [ms] (mean)
|
275
|
-
Time per request: 0.037 [ms] (mean, across all concurrent requests)
|
276
|
-
Transfer rate: 1280.29 [Kbytes/sec] received
|
277
|
-
|
278
|
-
Connection Times (ms)
|
279
|
-
min mean[+/-sd] median max
|
280
|
-
Connect: 0 0 0.0 0 0
|
281
|
-
Processing: 0 0 0.2 0 6
|
282
|
-
Waiting: 0 0 0.2 0 6
|
283
|
-
Total: 0 0 0.2 0 6
|
284
|
-
|
285
|
-
Percentage of the requests served within a certain time (ms)
|
286
|
-
50% 0
|
287
|
-
66% 0
|
288
|
-
75% 0
|
289
|
-
80% 0
|
290
|
-
90% 0
|
291
|
-
95% 1
|
292
|
-
98% 1
|
293
|
-
99% 1
|
294
|
-
100% 6 (longest request)
|
295
|
-
```
|
296
|
-
|
297
|
-
On a 4-core 8-thread i7, running `wrk`, which uses 8 keep-alive connections:
|
298
|
-
|
299
|
-
```
|
300
|
-
$ wrk -c 8 -d 10 -t 8 http://127.0.0.1:9294/
|
301
|
-
Running 10s test @ http://127.0.0.1:9294/
|
302
|
-
8 threads and 8 connections
|
303
|
-
Thread Stats Avg Stdev Max +/- Stdev
|
304
|
-
Latency 217.69us 0.99ms 23.21ms 97.39%
|
305
|
-
Req/Sec 12.18k 1.58k 17.67k 83.21%
|
306
|
-
974480 requests in 10.10s, 60.41MB read
|
307
|
-
Requests/sec: 96485.00
|
308
|
-
Transfer/sec: 5.98MB
|
309
|
-
```
|
310
|
-
|
311
|
-
According to these results, the cost of handling connections is quite high, while general throughput seems pretty decent.
|
312
|
-
|
313
|
-
## Semantic Model
|
314
|
-
|
315
|
-
### Scheme
|
316
|
-
|
317
|
-
HTTP/1 has an implicit scheme determined by the kind of connection made to the server (either `http` or `https`), while HTTP/2 models this explicitly and the client indicates this in the request using the `:scheme` pseudo-header (typically `https`). To normalize this, `Async::HTTP::Client` and `Async::HTTP::Server` have a default scheme which is used if none is supplied.
|
318
|
-
|
319
|
-
### Version
|
320
|
-
|
321
|
-
HTTP/1 has an explicit version while HTTP/2 does not expose the version in any way.
|
322
|
-
|
323
|
-
### Reason
|
324
|
-
|
325
|
-
HTTP/1 responses contain a reason field which is largely irrelevant. HTTP/2 does not support this field.
|
326
|
-
|
327
|
-
## Contributing
|
328
|
-
|
329
|
-
1. Fork it
|
330
|
-
2. Create your feature branch (`git checkout -b my-new-feature`)
|
331
|
-
3. Commit your changes (`git commit -am 'Add some feature'`)
|
332
|
-
4. Push to the branch (`git push origin my-new-feature`)
|
333
|
-
5. Create new Pull Request
|
334
|
-
|
335
|
-
## See Also
|
336
|
-
|
337
|
-
- [benchmark-http](https://github.com/socketry/benchmark-http) — A benchmarking tool to report on web server concurrency.
|
338
|
-
- [falcon](https://github.com/socketry/falcon) — A rack compatible server built on top of `async-http`.
|
339
|
-
- [async-websocket](https://github.com/socketry/async-websocket) — Asynchronous client and server websockets.
|
340
|
-
- [async-rest](https://github.com/socketry/async-rest) — A RESTful resource layer built on top of `async-http`.
|
341
|
-
- [async-http-faraday](https://github.com/socketry/async-http-faraday) — A faraday adapter to use `async-http`.
|
342
|
-
|
343
|
-
## License
|
344
|
-
|
345
|
-
Released under the MIT license.
|
346
|
-
|
347
|
-
Copyright, 2018, by [Samuel G. D. Williams](http://www.codeotaku.com/samuel-williams).
|
348
|
-
|
349
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
350
|
-
of this software and associated documentation files (the "Software"), to deal
|
351
|
-
in the Software without restriction, including without limitation the rights
|
352
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
353
|
-
copies of the Software, and to permit persons to whom the Software is
|
354
|
-
furnished to do so, subject to the following conditions:
|
355
|
-
|
356
|
-
The above copyright notice and this permission notice shall be included in
|
357
|
-
all copies or substantial portions of the Software.
|
358
|
-
|
359
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
360
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
361
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
362
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
363
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
364
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
365
|
-
THE SOFTWARE.
|