async-http 0.52.2 → 0.53.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (44) hide show
  1. checksums.yaml +4 -4
  2. data/lib/async/http/body/delayed.rb +2 -2
  3. data/lib/async/http/body/hijack.rb +5 -0
  4. data/lib/async/http/body/pipe.rb +15 -7
  5. data/lib/async/http/body/slowloris.rb +2 -2
  6. data/lib/async/http/body/stream.rb +1 -1
  7. data/lib/async/http/client.rb +2 -2
  8. data/lib/async/http/endpoint.rb +2 -2
  9. data/lib/async/http/internet.rb +4 -0
  10. data/lib/async/http/protocol/http1/connection.rb +0 -5
  11. data/lib/async/http/protocol/http1/server.rb +4 -3
  12. data/lib/async/http/protocol/http2/output.rb +1 -1
  13. data/lib/async/http/protocol/http2/response.rb +5 -0
  14. data/lib/async/http/proxy.rb +24 -8
  15. data/lib/async/http/version.rb +1 -1
  16. metadata +22 -65
  17. data/.editorconfig +0 -6
  18. data/.github/workflows/development.yml +0 -52
  19. data/.gitignore +0 -15
  20. data/.rspec +0 -3
  21. data/.travis.yml +0 -35
  22. data/README.md +0 -365
  23. data/async-http.gemspec +0 -39
  24. data/bake.rb +0 -0
  25. data/examples/compare/Gemfile +0 -9
  26. data/examples/compare/benchmark.rb +0 -78
  27. data/examples/download/chunked.rb +0 -86
  28. data/examples/fetch/Gemfile +0 -3
  29. data/examples/fetch/Gemfile.lock +0 -74
  30. data/examples/fetch/README.md +0 -3
  31. data/examples/fetch/config.ru +0 -28
  32. data/examples/fetch/public/index.html +0 -23
  33. data/examples/fetch/public/stream.js +0 -56
  34. data/examples/google/search.rb +0 -47
  35. data/examples/request.rb +0 -38
  36. data/examples/stream/stop.rb +0 -28
  37. data/examples/trenni/Gemfile +0 -5
  38. data/examples/trenni/streaming.rb +0 -35
  39. data/examples/upload/client.rb +0 -39
  40. data/examples/upload/data.txt +0 -41
  41. data/examples/upload/server.rb +0 -19
  42. data/examples/upload/upload.rb +0 -26
  43. data/gems.locked +0 -103
  44. data/gems.rb +0 -11
@@ -1,6 +0,0 @@
1
- root = true
2
-
3
- [*]
4
- indent_style = tab
5
- indent_size = 2
6
-
@@ -1,52 +0,0 @@
1
- name: Development
2
-
3
- on: [push, pull_request]
4
-
5
- jobs:
6
- test:
7
- runs-on: ${{matrix.os}}-latest
8
- continue-on-error: ${{matrix.experimental}}
9
-
10
- strategy:
11
- matrix:
12
- experimental: [false]
13
-
14
- os:
15
- - ubuntu
16
- - macos
17
-
18
- ruby:
19
- - 2.5
20
- - 2.6
21
- - 2.7
22
-
23
- include:
24
- - experimental: true
25
- os: ubuntu
26
- ruby: truffleruby
27
- - experimental: true
28
- os: ubuntu
29
- ruby: jruby
30
- - experimental: true
31
- os: ubuntu
32
- ruby: head
33
- - experimental: true
34
- os: ubuntu
35
- ruby: 2.6
36
- env: COVERAGE=PartialSummary,Coveralls
37
-
38
- steps:
39
- - uses: actions/checkout@v1
40
- - uses: ruby/setup-ruby@v1
41
- with:
42
- ruby-version: ${{matrix.ruby}}
43
-
44
- - name: Installing packages (ubuntu)
45
- if: matrix.os == 'ubuntu'
46
- run: sudo apt-get install apache2-utils
47
-
48
- - name: Install dependencies
49
- run: ${{matrix.env}} bundle install
50
-
51
- - name: Run tests
52
- run: ${{matrix.env}} bundle exec rspec
data/.gitignore DELETED
@@ -1,15 +0,0 @@
1
- .tags
2
-
3
- /.bundle/
4
- /.yardoc
5
- /Gemfile.lock
6
- /_yardoc/
7
- /coverage/
8
- /doc/
9
- /pkg/
10
- /spec/reports/
11
- /tmp/
12
-
13
- .rspec_status
14
- .covered.db
15
- /h2spec
data/.rspec DELETED
@@ -1,3 +0,0 @@
1
- --format documentation
2
- --warnings
3
- --require spec_helper
@@ -1,35 +0,0 @@
1
- language: ruby
2
- dist: xenial
3
- cache: bundler
4
-
5
- addons:
6
- apt:
7
- packages:
8
- - wrk
9
- - apache2-utils
10
- homebrew:
11
- packages:
12
- - wrk
13
-
14
- script: bundle exec rspec
15
-
16
- matrix:
17
- include:
18
- - rvm: 2.5
19
- - rvm: 2.6
20
- - rvm: 2.7
21
- - rvm: 2.6
22
- env: COVERAGE=PartialSummary,Coveralls
23
- - rvm: truffleruby
24
- - rvm: jruby-head
25
- env: JRUBY_OPTS="--debug -X+O"
26
- - rvm: ruby-head
27
- - rvm: 2.7
28
- os: osx
29
- allow_failures:
30
- - rvm: ruby-head
31
- - rvm: jruby-head
32
- - rvm: truffleruby
33
-
34
- # after_success:
35
- # - bundle exec rake h2spec:all
data/README.md DELETED
@@ -1,365 +0,0 @@
1
- # Async::HTTP
2
-
3
- An asynchronous client and server implementation of HTTP/1.0, HTTP/1.1 and HTTP/2 including TLS. Support for streaming requests and responses. Built on top of [async] and [async-io]. [falcon] provides a rack-compatible server.
4
-
5
- [![Build Status](https://travis-ci.com/socketry/async-http.svg?branch=master)](https://travis-ci.com/socketry/async-http)
6
- [![Code Climate](https://codeclimate.com/github/socketry/async-http.svg)](https://codeclimate.com/github/socketry/async-http)
7
- [![Coverage Status](https://coveralls.io/repos/socketry/async-http/badge.svg)](https://coveralls.io/r/socketry/async-http)
8
-
9
- [async]: https://github.com/socketry/async
10
- [async-io]: https://github.com/socketry/async-io
11
- [falcon]: https://github.com/socketry/falcon
12
-
13
- ## Installation
14
-
15
- Add this line to your application's Gemfile:
16
-
17
- ```ruby
18
- gem 'async-http'
19
- ```
20
-
21
- And then execute:
22
-
23
- $ bundle
24
-
25
- Or install it yourself as:
26
-
27
- $ gem install async-http
28
-
29
- ## Usage
30
-
31
- ### Post JSON data
32
-
33
- Here is an example showing how to post a data structure as JSON to a remote resource:
34
-
35
- ```ruby
36
- #!/usr/bin/env ruby
37
-
38
- require 'json'
39
- require 'async'
40
- require 'async/http/internet'
41
-
42
- data = {'life' => 42}
43
-
44
- Async do
45
- # Make a new internet:
46
- internet = Async::HTTP::Internet.new
47
-
48
- # Prepare the request:
49
- headers = [['accept', 'application/json']]
50
- body = [JSON.dump(data)]
51
-
52
- # Issues a POST request:
53
- response = internet.post("https://httpbin.org/anything", headers, body)
54
-
55
- # Save the response body to a local file:
56
- pp JSON.parse(response.read)
57
- ensure
58
- # The internet is closed for business:
59
- internet.close
60
- end
61
- ```
62
-
63
- Consider using [async-rest](https://github.com/socketry/async-rest) instead.
64
-
65
- ### Multiple Requests
66
-
67
- To issue multiple requests concurrently, you should use a barrier, e.g.
68
-
69
- ```ruby
70
- #!/usr/bin/env ruby
71
-
72
- require 'async'
73
- require 'async/barrier'
74
- require 'async/http/internet'
75
-
76
- TOPICS = ["ruby", "python", "rust"]
77
-
78
- Async do
79
- internet = Async::HTTP::Internet.new
80
- barrier = Async::Barrier.new
81
-
82
- # Spawn an asynchronous task for each topic:
83
- TOPICS.each do |topic|
84
- barrier.async do
85
- response = internet.get "https://www.google.com/search?q=#{topic}"
86
- puts "Found #{topic}: #{response.read.scan(topic).size} times."
87
- end
88
- end
89
-
90
- # Ensure we wait for all requests to complete before continuing:
91
- barrier.wait
92
- ensure
93
- internet&.close
94
- end
95
- ```
96
-
97
- #### Limiting Requests
98
-
99
- If you need to limit the number of simultaneous requests, use a semaphore.
100
-
101
- ```ruby
102
- #!/usr/bin/env ruby
103
-
104
- require 'async'
105
- require 'async/barrier'
106
- require 'async/semaphore'
107
- require 'async/http/internet'
108
-
109
- TOPICS = ["ruby", "python", "rust"]
110
-
111
- Async do
112
- internet = Async::HTTP::Internet.new
113
- barrier = Async::Barrier.new
114
- semaphore = Async::Semaphore.new(2, parent: barrier)
115
-
116
- # Spawn an asynchronous task for each topic:
117
- TOPICS.each do |topic|
118
- semaphore.async do
119
- response = internet.get "https://www.google.com/search?q=#{topic}"
120
- puts "Found #{topic}: #{response.read.scan(topic).size} times."
121
- end
122
- end
123
-
124
- # Ensure we wait for all requests to complete before continuing:
125
- barrier.wait
126
- ensure
127
- internet&.close
128
- end
129
- ```
130
-
131
- ### Downloading a File
132
-
133
- Here is an example showing how to download a file and save it to a local path:
134
-
135
- ```ruby
136
- #!/usr/bin/env ruby
137
-
138
- require 'async'
139
- require 'async/http/internet'
140
-
141
- Async do
142
- # Make a new internet:
143
- internet = Async::HTTP::Internet.new
144
-
145
- # Issues a GET request to Google:
146
- response = internet.get("https://www.google.com/search?q=kittens")
147
-
148
- # Save the response body to a local file:
149
- response.save("/tmp/search.html")
150
- ensure
151
- # The internet is closed for business:
152
- internet.close
153
- end
154
- ```
155
-
156
- ### Basic Client/Server
157
-
158
- Here is a basic example of a client/server running in the same reactor:
159
-
160
- ```ruby
161
- #!/usr/bin/env ruby
162
-
163
- require 'async'
164
- require 'async/http/server'
165
- require 'async/http/client'
166
- require 'async/http/endpoint'
167
- require 'async/http/protocol/response'
168
-
169
- endpoint = Async::HTTP::Endpoint.parse('http://127.0.0.1:9294')
170
-
171
- app = lambda do |request|
172
- Protocol::HTTP::Response[200, {}, ["Hello World"]]
173
- end
174
-
175
- server = Async::HTTP::Server.new(app, endpoint)
176
- client = Async::HTTP::Client.new(endpoint)
177
-
178
- Async do |task|
179
- server_task = task.async do
180
- server.run
181
- end
182
-
183
- response = client.get("/")
184
-
185
- puts response.status
186
- puts response.read
187
-
188
- server_task.stop
189
- end
190
- ```
191
-
192
- ### Advanced Verification
193
-
194
- You can hook into SSL certificate verification to improve server verification.
195
-
196
- ```ruby
197
- require 'async'
198
- require 'async/http'
199
-
200
- # These are generated from the certificate chain that the server presented.
201
- trusted_fingerprints = {
202
- "dac9024f54d8f6df94935fb1732638ca6ad77c13" => true,
203
- "e6a3b45b062d509b3382282d196efe97d5956ccb" => true,
204
- "07d63f4c05a03f1c306f9941b8ebf57598719ea2" => true,
205
- "e8d994f44ff20dc78dbff4e59d7da93900572bbf" => true,
206
- }
207
-
208
- Async do
209
- endpoint = Async::HTTP::Endpoint.parse("https://www.codeotaku.com/index")
210
-
211
- # This is a quick hack/POC:
212
- ssl_context = endpoint.ssl_context
213
-
214
- ssl_context.verify_callback = proc do |verified, store_context|
215
- certificate = store_context.current_cert
216
- fingerprint = OpenSSL::Digest::SHA1.new(certificate.to_der).to_s
217
-
218
- if trusted_fingerprints.include? fingerprint
219
- true
220
- else
221
- Async.logger.warn("Untrusted Certificate Fingerprint"){fingerprint}
222
- false
223
- end
224
- end
225
-
226
- endpoint = endpoint.with(ssl_context: ssl_context)
227
-
228
- client = Async::HTTP::Client.new(endpoint)
229
-
230
- response = client.get(endpoint.path)
231
-
232
- pp response.status, response.headers.fields, response.read
233
- end
234
- ```
235
-
236
- ## Performance
237
-
238
- On a 4-core 8-thread i7, running `ab` which uses discrete (non-keep-alive) connections:
239
-
240
- ```
241
- $ ab -c 8 -t 10 http://127.0.0.1:9294/
242
- This is ApacheBench, Version 2.3 <$Revision: 1757674 $>
243
- Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
244
- Licensed to The Apache Software Foundation, http://www.apache.org/
245
-
246
- Benchmarking 127.0.0.1 (be patient)
247
- Completed 5000 requests
248
- Completed 10000 requests
249
- Completed 15000 requests
250
- Completed 20000 requests
251
- Completed 25000 requests
252
- Completed 30000 requests
253
- Completed 35000 requests
254
- Completed 40000 requests
255
- Completed 45000 requests
256
- Completed 50000 requests
257
- Finished 50000 requests
258
-
259
-
260
- Server Software:
261
- Server Hostname: 127.0.0.1
262
- Server Port: 9294
263
-
264
- Document Path: /
265
- Document Length: 13 bytes
266
-
267
- Concurrency Level: 8
268
- Time taken for tests: 1.869 seconds
269
- Complete requests: 50000
270
- Failed requests: 0
271
- Total transferred: 2450000 bytes
272
- HTML transferred: 650000 bytes
273
- Requests per second: 26755.55 [#/sec] (mean)
274
- Time per request: 0.299 [ms] (mean)
275
- Time per request: 0.037 [ms] (mean, across all concurrent requests)
276
- Transfer rate: 1280.29 [Kbytes/sec] received
277
-
278
- Connection Times (ms)
279
- min mean[+/-sd] median max
280
- Connect: 0 0 0.0 0 0
281
- Processing: 0 0 0.2 0 6
282
- Waiting: 0 0 0.2 0 6
283
- Total: 0 0 0.2 0 6
284
-
285
- Percentage of the requests served within a certain time (ms)
286
- 50% 0
287
- 66% 0
288
- 75% 0
289
- 80% 0
290
- 90% 0
291
- 95% 1
292
- 98% 1
293
- 99% 1
294
- 100% 6 (longest request)
295
- ```
296
-
297
- On a 4-core 8-thread i7, running `wrk`, which uses 8 keep-alive connections:
298
-
299
- ```
300
- $ wrk -c 8 -d 10 -t 8 http://127.0.0.1:9294/
301
- Running 10s test @ http://127.0.0.1:9294/
302
- 8 threads and 8 connections
303
- Thread Stats Avg Stdev Max +/- Stdev
304
- Latency 217.69us 0.99ms 23.21ms 97.39%
305
- Req/Sec 12.18k 1.58k 17.67k 83.21%
306
- 974480 requests in 10.10s, 60.41MB read
307
- Requests/sec: 96485.00
308
- Transfer/sec: 5.98MB
309
- ```
310
-
311
- According to these results, the cost of handling connections is quite high, while general throughput seems pretty decent.
312
-
313
- ## Semantic Model
314
-
315
- ### Scheme
316
-
317
- HTTP/1 has an implicit scheme determined by the kind of connection made to the server (either `http` or `https`), while HTTP/2 models this explicitly and the client indicates this in the request using the `:scheme` pseudo-header (typically `https`). To normalize this, `Async::HTTP::Client` and `Async::HTTP::Server` have a default scheme which is used if none is supplied.
318
-
319
- ### Version
320
-
321
- HTTP/1 has an explicit version while HTTP/2 does not expose the version in any way.
322
-
323
- ### Reason
324
-
325
- HTTP/1 responses contain a reason field which is largely irrelevant. HTTP/2 does not support this field.
326
-
327
- ## Contributing
328
-
329
- 1. Fork it
330
- 2. Create your feature branch (`git checkout -b my-new-feature`)
331
- 3. Commit your changes (`git commit -am 'Add some feature'`)
332
- 4. Push to the branch (`git push origin my-new-feature`)
333
- 5. Create new Pull Request
334
-
335
- ## See Also
336
-
337
- - [benchmark-http](https://github.com/socketry/benchmark-http) — A benchmarking tool to report on web server concurrency.
338
- - [falcon](https://github.com/socketry/falcon) — A rack compatible server built on top of `async-http`.
339
- - [async-websocket](https://github.com/socketry/async-websocket) — Asynchronous client and server websockets.
340
- - [async-rest](https://github.com/socketry/async-rest) — A RESTful resource layer built on top of `async-http`.
341
- - [async-http-faraday](https://github.com/socketry/async-http-faraday) — A faraday adapter to use `async-http`.
342
-
343
- ## License
344
-
345
- Released under the MIT license.
346
-
347
- Copyright, 2018, by [Samuel G. D. Williams](http://www.codeotaku.com/samuel-williams).
348
-
349
- Permission is hereby granted, free of charge, to any person obtaining a copy
350
- of this software and associated documentation files (the "Software"), to deal
351
- in the Software without restriction, including without limitation the rights
352
- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
353
- copies of the Software, and to permit persons to whom the Software is
354
- furnished to do so, subject to the following conditions:
355
-
356
- The above copyright notice and this permission notice shall be included in
357
- all copies or substantial portions of the Software.
358
-
359
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
360
- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
361
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
362
- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
363
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
364
- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
365
- THE SOFTWARE.