requests_ruby 1.0.3 → 1.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 308045ed13c88c9a9eb1f89a483692c177b605e6aaac49e4eafa7aa0de2a3a74
4
- data.tar.gz: 7f62813cf1d17285f51b10a79bd4b5f08ea1f91e71172b7b50a9ab416330bcfd
3
+ metadata.gz: 8b4146e6fbead1e412259712bcd0939b7971d542bd94a41007e210c9c51de37c
4
+ data.tar.gz: bbc39cd987b00d948be87f9b02251d2e058016123c6ad2fda16724d921575f24
5
5
  SHA512:
6
- metadata.gz: a26999d7ae3f856351a416736f08ff8d1ebd5fb21addd94bcee508d30f3d49de7af9e09c8ea52dc4d4c61bd81126ad884bb3203c5349552863930f2e33cc7a82
7
- data.tar.gz: f199826c39a900bab90eba7445218b0c9c0f8830a7e33f588591667551c58f53f2c206c26424fcaae51cb207652611baba2ec70648c1138487b9c8ab66f89dc9
6
+ metadata.gz: 658ca79dbadae76f08300daa6ef77b5302bf8c8b2a42877659b490ffdc8359cc07f4c61ff79b56e181f3a4173d9474f5447170420198ee3bb8ded744404f58d1
7
+ data.tar.gz: 37e529b01d2a6406e86332e1ce10835c1b124b7076ceba696da237c215a0e3dab48a570e526dd75b614a95bce426bd1ce87c2a76035867d78fd45f33fa2fef5e
data/CHANGELOG.md CHANGED
@@ -1,7 +1,36 @@
1
1
  # Changelog
2
2
 
3
- <<<<<<< HEAD
4
- =======
3
+ ## 1.0.4
4
+
5
+ Adds real HTTP/2 support and rounds off a handful of gaps that showed up
6
+ against Python's `requests` (and `requests`+`urllib3`) feature set.
7
+
8
+ ### Added
9
+ - **HTTP/2 support**, implemented from scratch on top of stdlib
10
+ `Socket`/`OpenSSL` (ALPN negotiation, HPACK header compression with
11
+ Huffman coding, frame multiplexing, flow control) - no external gems.
12
+ `Requests::Http2Adapter` is a drop-in `HTTPAdapter` replacement:
13
+ `Requests::Session.new(http2: true)`, `session.http2!`, or per-call
14
+ `http2: true` on any request/`download`. Transparently falls back to
15
+ HTTP/1.1 for plain `http://` URLs, proxied requests, or servers that
16
+ don't negotiate `h2` over ALPN.
17
+ - `Session#trust_env` (default `true`) - proxies are now picked up from
18
+ `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY` (and lowercase variants) when no
19
+ explicit `proxies:` is set, matching python `requests`' default
20
+ behavior. Set `session.trust_env = false` to opt out.
21
+ - Retry backoff now honors a server's `Retry-After` header (seconds or
22
+ HTTP-date) when retrying a status in `status_forcelist`, instead of
23
+ always using the fixed `backoff_factor * attempt` formula.
24
+ - `Session#head!` / `#options!` and module-level `Requests.get!` /
25
+ `#post!` / `#put!` / `#patch!` / `#delete!` / `#head!` / `#options!` -
26
+ the raise-on-error helpers now cover every verb at both levels.
27
+ - `Requests.session(**opts)` now forwards keyword args (e.g.
28
+ `Requests.session(http2: true)`) instead of always building a bare
29
+ `Session.new`.
30
+
31
+ ### Fixed
32
+ - Resolved a leftover unresolved git merge conflict in this file.
33
+
5
34
  ## 1.0.3
6
35
 
7
36
  Focused on the two things people actually hit walls on with a stdlib-only
@@ -40,7 +69,6 @@ behave like cookies instead of a flat global hash.
40
69
  `backoff_factor`, or `status_forcelist` no longer gets silently
41
70
  overwritten back to the session's defaults on the next request.
42
71
 
43
- >>>>>>> cf89f78 (v1.0.3)
44
72
  ## 1.0.2
45
73
 
46
74
  Big cleanup pass. Split the single `lib/requests.rb` file into a proper
data/README.md CHANGED
@@ -4,6 +4,7 @@ A zero-dependency, stdlib-only HTTP client for Ruby, modeled closely on Python's
4
4
 
5
5
  [![Gem Version](https://img.shields.io/badge/gem-requests__ruby-red?logo=rubygems)](https://rubygems.org/gems/requests_ruby)
6
6
  [![Ruby](https://img.shields.io/badge/ruby-%3E%3D%202.7-CC342D?logo=ruby)](https://www.ruby-lang.org)
7
+ [![Gem Downloads](https://img.shields.io/gem/dt/requests_ruby)](https://rubygems.org/gems/requests_ruby)
7
8
  [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
8
9
  [![Zero Dependencies](https://img.shields.io/badge/dependencies-zero-brightgreen)](requests_ruby.gemspec)
9
10
 
@@ -29,6 +30,7 @@ A zero-dependency, stdlib-only HTTP client for Ruby, modeled closely on Python's
29
30
  - [Streaming & downloads](#streaming--downloads)
30
31
  - [Downloading big files in chunks](#downloading-big-files-in-chunks)
31
32
  - [Connection pooling](#connection-pooling)
33
+ - [HTTP/2](#http2)
32
34
  - [Custom transport adapters](#custom-transport-adapters)
33
35
  - [Status codes](#status-codes)
34
36
  - [API reference](#-api-reference)
@@ -221,13 +223,18 @@ Requests.get(url, timeout: [3, 10]) # [connect, read]
221
223
  # retry transient connection errors on idempotent methods (GET/HEAD/OPTIONS/PUT/DELETE)
222
224
  s = Requests::Session.new
223
225
  s.retries = 3
224
- s.backoff_factor = 0.5 # sleeps 0.5s, 1s, 1.5s between attempts
226
+ s.backoff_factor = 0.5 # sleeps 0.5s, 1s, 1.5s between attempts, unless...
227
+ s.status_forcelist = [429, 502, 503, 504] # ...also retry these response codes
225
228
  s.get(url)
226
229
 
227
230
  # or per-request
228
231
  Requests.get(url, retries: 3)
229
232
  ```
230
233
 
234
+ A response in `status_forcelist` that comes back with a `Retry-After`
235
+ header (seconds or an HTTP date) is retried after that delay instead of
236
+ the `backoff_factor` formula.
237
+
231
238
  ### Proxies & SSL
232
239
 
233
240
  ```ruby
@@ -236,6 +243,12 @@ Requests.get(url, verify: false) # skip cert verification (careful
236
243
  Requests.get(url, verify: '/path/to/ca.pem') # custom CA bundle
237
244
  ```
238
245
 
246
+ By default (`session.trust_env = true`), a `Session` without an explicit
247
+ `proxies:` picks one up from the `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY`
248
+ environment variables (lowercase names also work), same as python
249
+ `requests`. Set `session.trust_env = false` to ignore the environment
250
+ entirely.
251
+
239
252
  A bundled `cacert.pem` (Mozilla's CA bundle) ships with the gem and is used by default. Override it globally with the `REQUESTS_CA_FILE` environment variable.
240
253
 
241
254
  ### Cookies
@@ -332,6 +345,28 @@ it's transparently dropped and reopened on the next request; combined with
332
345
  `retries`/`backoff_factor` this makes long-lived sessions much more resilient
333
346
  than opening a fresh connection per call.
334
347
 
348
+ ### HTTP/2
349
+
350
+ As of 1.0.4, `Requests::Http2Adapter` speaks HTTP/2 directly over
351
+ `Socket`/`OpenSSL` - ALPN negotiation, HPACK header compression, and
352
+ stream multiplexing/flow control, all stdlib, no external gem:
353
+
354
+ ```ruby
355
+ s = Requests::Session.new(http2: true)
356
+ s.get('https://api.example.com/a') # negotiates h2 over ALPN, reuses the connection
357
+ s.get('https://api.example.com/b') # same connection, new stream
358
+
359
+ # or opt in per call on a plain session
360
+ Requests.get('https://api.example.com', http2: true)
361
+ resp = Requests::Session.new.tap(&:http2!).get('https://api.example.com')
362
+ ```
363
+
364
+ If the server doesn't advertise `h2` during the TLS handshake, or the URL
365
+ is plain `http://`, or a proxy is configured, requests on that host
366
+ transparently fall back to the regular `HTTPAdapter` (HTTP/1.1) - you get
367
+ the same `Response` object either way and don't need to know which
368
+ transport actually served the request.
369
+
335
370
  ### Custom transport adapters
336
371
 
337
372
  Loosely inspired by Python's `Session.mount()` / `HTTPAdapter`:
@@ -362,15 +397,16 @@ Requests.codes.im_a_teapot # 418, yes really
362
397
  |---|---|---|
363
398
  | `Requests.get/post/put/patch/delete/head/options` | `session.get/post/put/patch/delete/head/options` | `status_code`, `status` |
364
399
  | `Requests.request(method, url, **opts)` | `session.request(method, url, **opts)` | `ok?`, `redirect?` |
365
- | `Requests.session` new `Session` | `session.get!/post!/put!/patch!/delete!` (raise on error) | `text`, `content`, `json` |
366
- | `Requests.download(url, to:, chunk_size:, resume:, progress:)` | `session.download(url, to:, ...)` (same, but reuses the session's pool/auth/headers) | `headers`, `cookies` |
367
- | `Requests.codes` | `session.mount(prefix, adapter)`, `session.close` | `elapsed`, `history`, `url` |
368
- | | `session.hooks[:response]` | `raise_for_status`, `save_to`, `links` |
369
- | | `session.headers`, `.cookies`, `.auth`, `.proxies`, `.retries`, `.backoff_factor`, `.status_forcelist` | |
400
+ | `Requests.get!/post!/put!/patch!/delete!/head!/options!` (raise on error) | `session.get!/post!/put!/patch!/delete!/head!/options!` (raise on error) | `text`, `content`, `json` |
401
+ | `Requests.session(**opts)` new `Session` | `session.mount(prefix, adapter)`, `session.close` | `headers`, `cookies` |
402
+ | `Requests.download(url, to:, chunk_size:, resume:, progress:)` | `session.download(url, to:, ...)` (same, but reuses the session's pool/auth/headers) | `elapsed`, `history`, `url` |
403
+ | `Requests.codes` | `session.http2!`, `Session.new(http2: true)`, or `http2: true` per request | `raise_for_status`, `save_to`, `links` |
404
+ | | `session.hooks[:response]` | |
405
+ | | `session.headers`, `.cookies`, `.auth`, `.proxies`, `.trust_env`, `.retries`, `.backoff_factor`, `.status_forcelist` | |
370
406
  | | `Requests::Jar#save(path)` / `Requests::Jar.load(path)` | |
371
- | | `HTTPAdapter.new(max_retries:, backoff_factor:, status_forcelist:)` | |
407
+ | | `HTTPAdapter.new(max_retries:, backoff_factor:, status_forcelist:)`, `Http2Adapter.new(...)` | |
372
408
 
373
- Every request-level method accepts: `params`, `data`, `json`, `headers`, `cookies`, `files`, `auth`, `timeout`, `allow_redirects`, `proxies`, `verify`, `cert`, `hooks`, `retries`.
409
+ Every request-level method accepts: `params`, `data`, `json`, `headers`, `cookies`, `files`, `auth`, `timeout`, `allow_redirects`, `proxies`, `verify`, `cert`, `hooks`, `retries`, `http2`.
374
410
 
375
411
  ## Python parity cheat sheet
376
412
 
@@ -397,15 +433,22 @@ Every request-level method accepts: `params`, `data`, `json`, `headers`, `cookie
397
433
  - `iter_content`/`iter_lines` on a regular response read the whole body up
398
434
  front rather than lazily streaming it off the socket — `download` (added
399
435
  in 1.0.3) is the one that streams for real, use that for big files.
400
- - No HTTP/2, no automatic NTLM/Kerberos/OAuth bring your own `auth:`
401
- object (anything with `#call(headers)` works) if you need those.
436
+ - HTTP/2 (1.0.4) only applies to direct `https://` connections - there's
437
+ no `h2c` (cleartext HTTP/2) support, and requests through a proxy always
438
+ use HTTP/1.1. No automatic NTLM/Kerberos/OAuth either — bring your own
439
+ `auth:` object (anything with `#call(headers)` works) if you need those.
440
+ - The HTTP/2 adapter applies `timeout:` as one coarse deadline around the
441
+ whole request/response exchange rather than resetting it on every
442
+ individual socket read the way the HTTP/1.1 adapter's `Net::HTTP`
443
+ read-timeout does.
402
444
  - `verify: '/path/to/ca.pem'` accepts a single bundle file, not a
403
445
  directory of certs (`Net::HTTP` itself only takes `ca_file` or
404
446
  `ca_path`, and this gem only wires up the former today).
405
- - The connection pool added in 1.0.3 is per-`Session`/per-`HTTPAdapter`
406
- instance and isn't thread-safe for concurrent requests on the *same*
407
- `Session` object — use one `Session` per thread, or one connection per
408
- thread, same advice as most connection-pooling HTTP clients.
447
+ - The connection pool added in 1.0.3 (HTTP/1.1 and HTTP/2 alike) is per-
448
+ `Session`/per-adapter instance and isn't thread-safe for concurrent
449
+ requests on the *same* `Session` object — use one `Session` per thread,
450
+ or one connection per thread, same advice as most connection-pooling
451
+ HTTP clients.
409
452
 
410
453
 
411
454
  ## 🔧 Development
@@ -428,6 +471,10 @@ Bug reports and pull requests are welcome. A few ground rules:
428
471
  2. Add a test in `spec/requests_spec.rb` for anything you fix or add.
429
472
  3. Keep the Python-parity naming where it makes sense, but don't force it where it doesn't fit Ruby idiom (`?`-suffixed predicates, etc.).
430
473
 
474
+ ## Support the project
475
+
476
+ If requests_ruby helped you, consider starring the repository ⭐
477
+
431
478
  ## License
432
479
 
433
480
  [MIT](LICENSE)
@@ -5,6 +5,7 @@ require 'openssl'
5
5
  require 'uri'
6
6
  require 'zlib'
7
7
  require 'stringio'
8
+ require 'time'
8
9
 
9
10
  module Requests
10
11
  class HTTPAdapter
@@ -30,7 +31,7 @@ module Requests
30
31
  r = do_send(meth, url, hdrs, body, timeout, proxies, verify, cert, auth)
31
32
  if @status_forcelist.include?(r.code.to_i) && attempt < @max_retries
32
33
  attempt += 1
33
- sleep(@backoff_factor * attempt) if @backoff_factor.to_f > 0
34
+ sleep(retry_delay(r, attempt))
34
35
  next
35
36
  end
36
37
  return r
@@ -104,6 +105,19 @@ module Requests
104
105
  def idempotent?(meth)
105
106
  %w[GET HEAD OPTIONS PUT DELETE].include?(meth.to_s.upcase)
106
107
  end
108
+ def retry_delay(resp, attempt)
109
+ ra = resp['retry-after']
110
+ if ra
111
+ return ra.to_i if ra =~ /\A\d+\z/
112
+ begin
113
+ d = Time.httpdate(ra) - Time.now
114
+ return d.positive? ? d : 0
115
+ rescue ArgumentError
116
+ nil
117
+ end
118
+ end
119
+ @backoff_factor.to_f > 0 ? @backoff_factor * attempt : 0
120
+ end
107
121
  def do_send(meth, url, hdrs, body, timeout, proxies, verify, cert, auth)
108
122
  u = URI.parse(url)
109
123
  auth.call(hdrs) if auth.respond_to?(:call) && !auth.is_a?(Requests::DigestAuth)
data/lib/requests/api.rb CHANGED
@@ -14,8 +14,15 @@ module Requests
14
14
  kw[:allow_redirects] = kw.fetch(:allow_redirects, false)
15
15
  request('HEAD', url, **kw)
16
16
  end
17
- def self.session
18
- Session.new
17
+ def self.get!(url, **kw); get(url, **kw).raise_for_status; end
18
+ def self.post!(url, **kw); post(url, **kw).raise_for_status; end
19
+ def self.put!(url, **kw); put(url, **kw).raise_for_status; end
20
+ def self.patch!(url, **kw); patch(url, **kw).raise_for_status; end
21
+ def self.delete!(url, **kw); delete(url, **kw).raise_for_status; end
22
+ def self.head!(url, **kw); head(url, **kw).raise_for_status; end
23
+ def self.options!(url, **kw); options(url, **kw).raise_for_status; end
24
+ def self.session(**kw)
25
+ Session.new(**kw)
19
26
  end
20
27
  def self.download(url, to:, **kw)
21
28
  Session.new.download(url, to: to, **kw)
@@ -0,0 +1,241 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Requests
4
+ module HPACK
5
+ STATIC_TABLE = [
6
+ [':authority', ''],[':method', 'GET'],[':method', 'POST'],[':path', '/'],
7
+ [':path', '/index.html'],[':scheme', 'http'],[':scheme', 'https'],[':status', '200'],
8
+ [':status', '204'],[':status', '206'],[':status', '304'],[':status', '400'],
9
+ [':status', '404'],[':status', '500'],['accept-charset', ''],['accept-encoding', 'gzip, deflate'],
10
+ ['accept-language', ''],['accept-ranges', ''],['accept', ''],['access-control-allow-origin', ''],
11
+ ['age', ''],['allow', ''],['authorization', ''],['cache-control', ''],
12
+ ['content-disposition', ''],['content-encoding', ''],['content-language', ''],['content-length', ''],
13
+ ['content-location', ''],['content-range', ''],['content-type', ''],['cookie', ''],
14
+ ['date', ''],['etag', ''],['expect', ''],['expires', ''],
15
+ ['from', ''],['host', ''],['if-match', ''],['if-modified-since', ''],
16
+ ['if-none-match', ''],['if-range', ''],['if-unmodified-since', ''],['last-modified', ''],
17
+ ['link', ''],['location', ''],['max-forwards', ''],['proxy-authenticate', ''],
18
+ ['proxy-authorization', ''],['range', ''],['referer', ''],['refresh', ''],
19
+ ['retry-after', ''],['server', ''],['set-cookie', ''],['strict-transport-security', ''],
20
+ ['transfer-encoding', ''],['user-agent', ''],['vary', ''],['via', ''],
21
+ ['www-authenticate', '']
22
+ ].freeze
23
+
24
+ module Huffman
25
+ CODES = [
26
+ [0x1ff8, 13],[0x7fffd8, 23],[0xfffffe2, 28],[0xfffffe3, 28],[0xfffffe4, 28],[0xfffffe5, 28],
27
+ [0xfffffe6, 28],[0xfffffe7, 28],[0xfffffe8, 28],[0xffffea, 24],[0x3ffffffc, 30],[0xfffffe9, 28],
28
+ [0xfffffea, 28],[0x3ffffffd, 30],[0xfffffeb, 28],[0xfffffec, 28],[0xfffffed, 28],[0xfffffee, 28],
29
+ [0xfffffef, 28],[0xffffff0, 28],[0xffffff1, 28],[0xffffff2, 28],[0x3ffffffe, 30],[0xffffff3, 28],
30
+ [0xffffff4, 28],[0xffffff5, 28],[0xffffff6, 28],[0xffffff7, 28],[0xffffff8, 28],[0xffffff9, 28],
31
+ [0xffffffa, 28],[0xffffffb, 28],[0x14, 6],[0x3f8, 10],[0x3f9, 10],[0xffa, 12],
32
+ [0x1ff9, 13],[0x15, 6],[0xf8, 8],[0x7fa, 11],[0x3fa, 10],[0x3fb, 10],
33
+ [0xf9, 8],[0x7fb, 11],[0xfa, 8],[0x16, 6],[0x17, 6],[0x18, 6],
34
+ [0x0, 5],[0x1, 5],[0x2, 5],[0x19, 6],[0x1a, 6],[0x1b, 6],
35
+ [0x1c, 6],[0x1d, 6],[0x1e, 6],[0x1f, 6],[0x5c, 7],[0xfb, 8],
36
+ [0x7ffc, 15],[0x20, 6],[0xffb, 12],[0x3fc, 10],[0x1ffa, 13],[0x21, 6],
37
+ [0x5d, 7],[0x5e, 7],[0x5f, 7],[0x60, 7],[0x61, 7],[0x62, 7],
38
+ [0x63, 7],[0x64, 7],[0x65, 7],[0x66, 7],[0x67, 7],[0x68, 7],
39
+ [0x69, 7],[0x6a, 7],[0x6b, 7],[0x6c, 7],[0x6d, 7],[0x6e, 7],
40
+ [0x6f, 7],[0x70, 7],[0x71, 7],[0x72, 7],[0xfc, 8],[0x73, 7],
41
+ [0xfd, 8],[0x1ffb, 13],[0x7fff0, 19],[0x1ffc, 13],[0x3ffc, 14],[0x22, 6],
42
+ [0x7ffd, 15],[0x3, 5],[0x23, 6],[0x4, 5],[0x24, 6],[0x5, 5],
43
+ [0x25, 6],[0x26, 6],[0x27, 6],[0x6, 5],[0x74, 7],[0x75, 7],
44
+ [0x28, 6],[0x29, 6],[0x2a, 6],[0x7, 5],[0x2b, 6],[0x76, 7],
45
+ [0x2c, 6],[0x8, 5],[0x9, 5],[0x2d, 6],[0x77, 7],[0x78, 7],
46
+ [0x79, 7],[0x7a, 7],[0x7b, 7],[0x7ffe, 15],[0x7fc, 11],[0x3ffd, 14],
47
+ [0x1ffd, 13],[0xffffffc, 28],[0xfffe6, 20],[0x3fffd2, 22],[0xfffe7, 20],[0xfffe8, 20],
48
+ [0x3fffd3, 22],[0x3fffd4, 22],[0x3fffd5, 22],[0x7fffd9, 23],[0x3fffd6, 22],[0x7fffda, 23],
49
+ [0x7fffdb, 23],[0x7fffdc, 23],[0x7fffdd, 23],[0x7fffde, 23],[0xffffeb, 24],[0x7fffdf, 23],
50
+ [0xffffec, 24],[0xffffed, 24],[0x3fffd7, 22],[0x7fffe0, 23],[0xffffee, 24],[0x7fffe1, 23],
51
+ [0x7fffe2, 23],[0x7fffe3, 23],[0x7fffe4, 23],[0x1fffdc, 21],[0x3fffd8, 22],[0x7fffe5, 23],
52
+ [0x3fffd9, 22],[0x7fffe6, 23],[0x7fffe7, 23],[0xffffef, 24],[0x3fffda, 22],[0x1fffdd, 21],
53
+ [0xfffe9, 20],[0x3fffdb, 22],[0x3fffdc, 22],[0x7fffe8, 23],[0x7fffe9, 23],[0x1fffde, 21],
54
+ [0x7fffea, 23],[0x3fffdd, 22],[0x3fffde, 22],[0xfffff0, 24],[0x1fffdf, 21],[0x3fffdf, 22],
55
+ [0x7fffeb, 23],[0x7fffec, 23],[0x1fffe0, 21],[0x1fffe1, 21],[0x3fffe0, 22],[0x1fffe2, 21],
56
+ [0x7fffed, 23],[0x3fffe1, 22],[0x7fffee, 23],[0x7fffef, 23],[0xfffea, 20],[0x3fffe2, 22],
57
+ [0x3fffe3, 22],[0x3fffe4, 22],[0x7ffff0, 23],[0x3fffe5, 22],[0x3fffe6, 22],[0x7ffff1, 23],
58
+ [0x3ffffe0, 26],[0x3ffffe1, 26],[0xfffeb, 20],[0x7fff1, 19],[0x3fffe7, 22],[0x7ffff2, 23],
59
+ [0x3fffe8, 22],[0x1ffffec, 25],[0x3ffffe2, 26],[0x3ffffe3, 26],[0x3ffffe4, 26],[0x7ffffde, 27],
60
+ [0x7ffffdf, 27],[0x3ffffe5, 26],[0xfffff1, 24],[0x1ffffed, 25],[0x7fff2, 19],[0x1fffe3, 21],
61
+ [0x3ffffe6, 26],[0x7ffffe0, 27],[0x7ffffe1, 27],[0x3ffffe7, 26],[0x7ffffe2, 27],[0xfffff2, 24],
62
+ [0x1fffe4, 21],[0x1fffe5, 21],[0x3ffffe8, 26],[0x3ffffe9, 26],[0xffffffd, 28],[0x7ffffe3, 27],
63
+ [0x7ffffe4, 27],[0x7ffffe5, 27],[0xfffec, 20],[0xfffff3, 24],[0xfffed, 20],[0x1fffe6, 21],
64
+ [0x3fffe9, 22],[0x1fffe7, 21],[0x1fffe8, 21],[0x7ffff3, 23],[0x3fffea, 22],[0x3fffeb, 22],
65
+ [0x1ffffee, 25],[0x1ffffef, 25],[0xfffff4, 24],[0xfffff5, 24],[0x3ffffea, 26],[0x7ffff4, 23],
66
+ [0x3ffffeb, 26],[0x7ffffe6, 27],[0x3ffffec, 26],[0x3ffffed, 26],[0x7ffffe7, 27],[0x7ffffe8, 27],
67
+ [0x7ffffe9, 27],[0x7ffffea, 27],[0x7ffffeb, 27],[0xffffffe, 28],[0x7ffffec, 27],[0x7ffffed, 27],
68
+ [0x7ffffee, 27],[0x7ffffef, 27],[0x7fffff0, 27],[0x3ffffee, 26],[0x3fffffff, 30]
69
+ ].freeze
70
+ EOS = 256
71
+ ENCODE_TABLE = CODES.map { |c, l| [c].pack('N').unpack1('B*')[-l..-1] }.freeze
72
+ def self.build_tree
73
+ root = {}
74
+ CODES.each_with_index do |(code, len), sym|
75
+ node = root
76
+ (len - 1).downto(0) do |i|
77
+ bit = (code >> i) & 1
78
+ node = (node[bit] ||= {})
79
+ end
80
+ node[:sym] = sym
81
+ end
82
+ root
83
+ end
84
+ DECODE_TREE = build_tree
85
+ def self.encode(str)
86
+ bits = String.new
87
+ str.each_byte { |b| bits << ENCODE_TABLE[b] }
88
+ pad = (8 - bits.size % 8) % 8
89
+ bits << ('1' * pad) if pad > 0
90
+ [bits].pack('B*')
91
+ end
92
+ def self.decode(bytes)
93
+ out = String.new(encoding: Encoding::BINARY)
94
+ node = DECODE_TREE
95
+ bitcount = bytes.bytesize * 8
96
+ pos = 0
97
+ bytes.each_byte do |byte|
98
+ 7.downto(0) do |i|
99
+ pos += 1
100
+ bit = (byte >> i) & 1
101
+ nxt = node[bit]
102
+ unless nxt
103
+ raise Requests::ContentDecodingError, 'invalid huffman padding' if pos > bitcount - 8 || pos < bitcount
104
+ return out
105
+ end
106
+ node = nxt
107
+ if node.key?(:sym)
108
+ sym = node[:sym]
109
+ raise Requests::ContentDecodingError, 'unexpected huffman EOS' if sym == EOS
110
+ out << sym.chr(Encoding::BINARY)
111
+ node = DECODE_TREE
112
+ end
113
+ end
114
+ end
115
+ out
116
+ end
117
+ end
118
+
119
+ module Integer
120
+ module_function
121
+ def encode(value, prefix_bits, first_byte = 0)
122
+ max = (1 << prefix_bits) - 1
123
+ if value < max
124
+ [first_byte | value].pack('C')
125
+ else
126
+ out = [first_byte | max].pack('C')
127
+ value -= max
128
+ while value >= 128
129
+ out << [(value % 128) | 128].pack('C')
130
+ value /= 128
131
+ end
132
+ out << [value].pack('C')
133
+ end
134
+ end
135
+ def decode(bytes, pos, prefix_bits)
136
+ max = (1 << prefix_bits) - 1
137
+ value = bytes.getbyte(pos) & max
138
+ pos += 1
139
+ if value == max
140
+ m = 0
141
+ loop do
142
+ b = bytes.getbyte(pos)
143
+ raise Requests::ContentDecodingError, 'truncated hpack integer' if b.nil?
144
+ pos += 1
145
+ value += (b & 127) * (2**m)
146
+ m += 7
147
+ break if b & 128 == 0
148
+ end
149
+ end
150
+ [value, pos]
151
+ end
152
+ end
153
+
154
+ class Encoder
155
+ def encode(headers)
156
+ out = String.new(encoding: Encoding::BINARY)
157
+ headers.each { |name, value| out << encode_field(name.to_s.downcase, value.to_s) }
158
+ out
159
+ end
160
+ private
161
+ def encode_field(name, value)
162
+ exact = STATIC_TABLE.index { |n, v| n == name && v == value }
163
+ return Integer.encode(exact + 1, 7, 0x80) if exact
164
+ name_idx = STATIC_TABLE.index { |n, _| n == name }
165
+ head = name_idx ? Integer.encode(name_idx + 1, 4, 0x00) : Integer.encode(0, 4, 0x00) + literal(name)
166
+ head + literal(value)
167
+ end
168
+ def literal(str)
169
+ h = Huffman.encode(str)
170
+ if h.bytesize < str.bytesize
171
+ Integer.encode(h.bytesize, 7, 0x80) + h
172
+ else
173
+ Integer.encode(str.bytesize, 7, 0x00) + str.dup.force_encoding(Encoding::BINARY)
174
+ end
175
+ end
176
+ end
177
+
178
+ class Decoder
179
+ def initialize(max_size: 4096)
180
+ @dynamic = []
181
+ @max_size = max_size
182
+ @size = 0
183
+ end
184
+ def decode(bytes)
185
+ pos = 0
186
+ out = []
187
+ while pos < bytes.bytesize
188
+ byte = bytes.getbyte(pos)
189
+ if byte & 0x80 != 0
190
+ idx, pos = Integer.decode(bytes, pos, 7)
191
+ out << lookup(idx)
192
+ elsif byte & 0x40 != 0
193
+ name, value, pos = read_literal(bytes, pos, 6)
194
+ add_dynamic(name, value)
195
+ out << [name, value]
196
+ elsif byte & 0x20 != 0
197
+ size, pos = Integer.decode(bytes, pos, 5)
198
+ @max_size = size
199
+ evict
200
+ else
201
+ name, value, pos = read_literal(bytes, pos, 4)
202
+ out << [name, value]
203
+ end
204
+ end
205
+ out
206
+ end
207
+ private
208
+ def lookup(idx)
209
+ return STATIC_TABLE[idx - 1] if idx <= STATIC_TABLE.size
210
+ d = @dynamic[idx - STATIC_TABLE.size - 1]
211
+ raise Requests::ContentDecodingError, "invalid hpack index #{idx}" unless d
212
+ d
213
+ end
214
+ def read_literal(bytes, pos, prefix_bits)
215
+ idx, pos = Integer.decode(bytes, pos, prefix_bits)
216
+ name, pos = idx.zero? ? read_string(bytes, pos) : [lookup(idx)[0], pos]
217
+ value, pos = read_string(bytes, pos)
218
+ [name, value, pos]
219
+ end
220
+ def read_string(bytes, pos)
221
+ huff = bytes.getbyte(pos) & 0x80 != 0
222
+ len, pos = Integer.decode(bytes, pos, 7)
223
+ raw = bytes.byteslice(pos, len)
224
+ pos += len
225
+ [huff ? Huffman.decode(raw) : raw, pos]
226
+ end
227
+ def add_dynamic(name, value)
228
+ entry_size = name.bytesize + value.bytesize + 32
229
+ @dynamic.unshift([name, value])
230
+ @size += entry_size
231
+ evict
232
+ end
233
+ def evict
234
+ while @size > @max_size && !@dynamic.empty?
235
+ name, value = @dynamic.pop
236
+ @size -= name.bytesize + value.bytesize + 32
237
+ end
238
+ end
239
+ end
240
+ end
241
+ end
@@ -0,0 +1,469 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'socket'
4
+ require 'openssl'
5
+ require 'uri'
6
+ require 'zlib'
7
+ require 'timeout'
8
+ require 'time'
9
+
10
+ module Requests
11
+ class Http2Response
12
+ def initialize(header_pairs, body)
13
+ @status = nil
14
+ @fields = Hash.new { |h, k| h[k] = [] }
15
+ @headers = CIHash.new
16
+ header_pairs.each do |k, v|
17
+ if k == ':status'
18
+ @status = v
19
+ else
20
+ @fields[k.downcase] << v
21
+ @headers[k] = v
22
+ end
23
+ end
24
+ @body = body
25
+ end
26
+ def code
27
+ @status
28
+ end
29
+ def message
30
+ (Requests::STATUS_CODES[@status.to_i] || :unknown).to_s.split('_').map(&:capitalize).join(' ')
31
+ end
32
+ def [](k)
33
+ @headers[k]
34
+ end
35
+ def each_header
36
+ return enum_for(:each_header) unless block_given?
37
+ @headers.each { |k, v| yield k, v }
38
+ end
39
+ def get_fields(name)
40
+ f = @fields[name.to_s.downcase]
41
+ f.empty? ? nil : f
42
+ end
43
+ def body
44
+ @body
45
+ end
46
+ end
47
+
48
+ class Http2Connection
49
+ TYPE_DATA = 0x0
50
+ TYPE_HEADERS = 0x1
51
+ TYPE_PRIORITY = 0x2
52
+ TYPE_RST_STREAM = 0x3
53
+ TYPE_SETTINGS = 0x4
54
+ TYPE_PUSH_PROMISE = 0x5
55
+ TYPE_PING = 0x6
56
+ TYPE_GOAWAY = 0x7
57
+ TYPE_WINDOW_UPDATE = 0x8
58
+ TYPE_CONTINUATION = 0x9
59
+ FLAG_END_STREAM = 0x1
60
+ FLAG_ACK = 0x1
61
+ FLAG_END_HEADERS = 0x4
62
+ FLAG_PADDED = 0x8
63
+ FLAG_PRIORITY = 0x20
64
+ PREFACE = "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"
65
+ DEFAULT_WINDOW = 65_535
66
+
67
+ def initialize(socket)
68
+ @socket = socket
69
+ @next_stream_id = 1
70
+ @encoder = HPACK::Encoder.new
71
+ @decoder = HPACK::Decoder.new
72
+ @send_window = DEFAULT_WINDOW
73
+ @conn_recv_window = DEFAULT_WINDOW
74
+ @peer_max_frame_size = 16_384
75
+ @peer_initial_window = DEFAULT_WINDOW
76
+ @streams = {}
77
+ @closed = false
78
+ handshake!
79
+ end
80
+ def alive?
81
+ !@closed && !@socket.closed?
82
+ end
83
+ def close
84
+ write_frame(TYPE_GOAWAY, 0, 0, [0, 0].pack('NN')) if alive?
85
+ rescue StandardError
86
+ nil
87
+ ensure
88
+ @closed = true
89
+ @socket.close unless @socket.closed?
90
+ end
91
+ def request(method, uri, header_list, body, io: nil, progress: nil)
92
+ raise Requests::ConnectionError, 'http/2 connection is closed' unless alive?
93
+ stream_id = @next_stream_id
94
+ @next_stream_id += 2
95
+ @streams[stream_id] = { send_window: @peer_initial_window, recv_window: DEFAULT_WINDOW }
96
+ end_stream = body.nil? || body.to_s.empty?
97
+ send_headers(stream_id, header_list, end_stream)
98
+ send_body(stream_id, body) unless end_stream
99
+ read_response(stream_id, io: io, progress: progress)
100
+ ensure
101
+ @streams.delete(stream_id) if stream_id
102
+ end
103
+ private
104
+ def handshake!
105
+ @socket.write(PREFACE)
106
+ write_frame(TYPE_SETTINGS, 0, 0, '')
107
+ loop do
108
+ type, flags, sid, payload = read_frame
109
+ case type
110
+ when TYPE_SETTINGS
111
+ if flags & FLAG_ACK == 0
112
+ apply_settings(payload)
113
+ write_frame(TYPE_SETTINGS, FLAG_ACK, 0, '')
114
+ end
115
+ break
116
+ when TYPE_WINDOW_UPDATE
117
+ apply_window_update(sid, payload)
118
+ end
119
+ end
120
+ end
121
+ def send_headers(stream_id, header_list, end_stream)
122
+ blob = @encoder.encode(header_list)
123
+ offset = 0
124
+ first = true
125
+ loop do
126
+ chunk = blob.byteslice(offset, @peer_max_frame_size) || ''
127
+ offset += chunk.bytesize
128
+ last = offset >= blob.bytesize
129
+ flags = 0
130
+ flags |= FLAG_END_HEADERS if last
131
+ flags |= FLAG_END_STREAM if last && end_stream
132
+ write_frame(first ? TYPE_HEADERS : TYPE_CONTINUATION, flags, stream_id, chunk)
133
+ first = false
134
+ break if last
135
+ end
136
+ end
137
+ def send_body(stream_id, body)
138
+ pos = 0
139
+ st = @streams[stream_id]
140
+ body = body.dup.force_encoding(Encoding::BINARY)
141
+ loop do
142
+ avail = [st[:send_window], @send_window, @peer_max_frame_size].min
143
+ if avail <= 0
144
+ type, flags, sid, payload = read_frame
145
+ dispatch_control_frame(type, flags, sid, payload)
146
+ next
147
+ end
148
+ chunk = body.byteslice(pos, [avail, body.bytesize - pos].min)
149
+ pos += chunk.bytesize
150
+ st[:send_window] -= chunk.bytesize
151
+ @send_window -= chunk.bytesize
152
+ last = pos >= body.bytesize
153
+ write_frame(TYPE_DATA, last ? FLAG_END_STREAM : 0, stream_id, chunk)
154
+ break if last
155
+ end
156
+ end
157
+ def read_response(stream_id, io: nil, progress: nil)
158
+ header_blob = String.new(encoding: Encoding::BINARY)
159
+ headers = nil
160
+ body = io ? nil : String.new(encoding: Encoding::BINARY)
161
+ decoder = nil
162
+ total = 0
163
+ content_length = nil
164
+ done = false
165
+ until done
166
+ type, flags, sid, payload = read_frame
167
+ case type
168
+ when TYPE_HEADERS, TYPE_CONTINUATION
169
+ next unless sid == stream_id || headers.nil?
170
+ block = type == TYPE_HEADERS ? strip_header_padding(payload, flags) : payload
171
+ header_blob << block
172
+ if flags & FLAG_END_HEADERS != 0
173
+ headers = @decoder.decode(header_blob)
174
+ if io
175
+ hdrs = CIHash.new
176
+ headers.each { |k, v| hdrs[k] = v unless k == ':status' }
177
+ decoder = stream_decoder(hdrs)
178
+ content_length = hdrs['content-length']&.to_i
179
+ end
180
+ end
181
+ done = true if sid == stream_id && flags & FLAG_END_STREAM != 0
182
+ when TYPE_DATA
183
+ if sid == stream_id
184
+ chunk = strip_data_padding(payload, flags)
185
+ update_recv_window(stream_id, payload.bytesize)
186
+ if io
187
+ piece = decoder == :raw ? chunk : decoder.inflate(chunk)
188
+ io.write(piece)
189
+ total += piece.bytesize
190
+ progress.call(total, content_length) if progress
191
+ else
192
+ body << chunk
193
+ end
194
+ end
195
+ done = true if sid == stream_id && flags & FLAG_END_STREAM != 0
196
+ else
197
+ dispatch_control_frame(type, flags, sid, payload)
198
+ done = true if type == TYPE_RST_STREAM && sid == stream_id
199
+ end
200
+ end
201
+ Http2Response.new(headers || [], body)
202
+ end
203
+ def dispatch_control_frame(type, flags, sid, payload)
204
+ case type
205
+ when TYPE_WINDOW_UPDATE
206
+ apply_window_update(sid, payload)
207
+ when TYPE_SETTINGS
208
+ if flags & FLAG_ACK == 0
209
+ apply_settings(payload)
210
+ write_frame(TYPE_SETTINGS, FLAG_ACK, 0, '')
211
+ end
212
+ when TYPE_PING
213
+ write_frame(TYPE_PING, FLAG_ACK, 0, payload) if flags & FLAG_ACK == 0
214
+ when TYPE_GOAWAY
215
+ @closed = true
216
+ when TYPE_RST_STREAM
217
+ raise Requests::ConnectionError, 'stream reset by server'
218
+ when TYPE_PUSH_PROMISE
219
+ psid = payload.byteslice(0, 4).unpack1('N') & 0x7fffffff
220
+ write_frame(TYPE_RST_STREAM, 0, psid, [0x8].pack('N'))
221
+ end
222
+ end
223
+ def apply_settings(payload)
224
+ i = 0
225
+ while i + 6 <= payload.bytesize
226
+ id = payload.byteslice(i, 2).unpack1('n')
227
+ val = payload.byteslice(i + 2, 4).unpack1('N')
228
+ @peer_initial_window = val if id == 4
229
+ @peer_max_frame_size = val if id == 5
230
+ i += 6
231
+ end
232
+ end
233
+ def apply_window_update(sid, payload)
234
+ inc = payload.unpack1('N') & 0x7fffffff
235
+ if sid == 0
236
+ @send_window += inc
237
+ else
238
+ (@streams[sid] ||= { send_window: @peer_initial_window, recv_window: DEFAULT_WINDOW })[:send_window] += inc
239
+ end
240
+ end
241
+ def update_recv_window(stream_id, n)
242
+ st = @streams[stream_id]
243
+ return unless st
244
+ st[:recv_window] -= n
245
+ @conn_recv_window -= n
246
+ if st[:recv_window] < 32_768
247
+ inc = DEFAULT_WINDOW - st[:recv_window]
248
+ write_frame(TYPE_WINDOW_UPDATE, 0, stream_id, [inc].pack('N'))
249
+ st[:recv_window] += inc
250
+ end
251
+ return unless @conn_recv_window < 32_768
252
+ inc = DEFAULT_WINDOW - @conn_recv_window
253
+ write_frame(TYPE_WINDOW_UPDATE, 0, 0, [inc].pack('N'))
254
+ @conn_recv_window += inc
255
+ end
256
+ def stream_decoder(hdrs)
257
+ case hdrs['content-encoding'].to_s.downcase
258
+ when 'gzip', 'x-gzip' then Zlib::Inflate.new(32 + Zlib::MAX_WBITS)
259
+ when 'deflate' then Zlib::Inflate.new
260
+ else :raw
261
+ end
262
+ end
263
+ def strip_header_padding(payload, flags)
264
+ pos = 0
265
+ padlen = 0
266
+ if flags & FLAG_PADDED != 0
267
+ padlen = payload.getbyte(0)
268
+ pos += 1
269
+ end
270
+ pos += 5 if flags & FLAG_PRIORITY != 0
271
+ payload.byteslice(pos, payload.bytesize - pos - padlen)
272
+ end
273
+ def strip_data_padding(payload, flags)
274
+ return payload if flags & FLAG_PADDED == 0
275
+ padlen = payload.getbyte(0)
276
+ payload.byteslice(1, payload.bytesize - 1 - padlen)
277
+ end
278
+ def read_frame
279
+ hdr = read_exactly(9)
280
+ length = (hdr.getbyte(0) << 16) | (hdr.getbyte(1) << 8) | hdr.getbyte(2)
281
+ type = hdr.getbyte(3)
282
+ flags = hdr.getbyte(4)
283
+ sid = hdr.byteslice(5, 4).unpack1('N') & 0x7fffffff
284
+ payload = length.positive? ? read_exactly(length) : ''
285
+ [type, flags, sid, payload]
286
+ end
287
+ def write_frame(type, flags, stream_id, payload)
288
+ len = payload.bytesize
289
+ hdr = [(len >> 16) & 0xff, (len >> 8) & 0xff, len & 0xff, type, flags].pack('C5') + [stream_id & 0x7fffffff].pack('N')
290
+ @socket.write(hdr + payload)
291
+ end
292
+ def read_exactly(n)
293
+ buf = String.new(encoding: Encoding::BINARY)
294
+ while buf.bytesize < n
295
+ chunk = @socket.read(n - buf.bytesize)
296
+ raise Requests::ConnectionError, 'connection closed by peer' if chunk.nil?
297
+ buf << chunk
298
+ end
299
+ buf
300
+ end
301
+ end
302
+
303
+ class Http2Adapter
304
+ attr_accessor :max_retries, :backoff_factor, :status_forcelist
305
+ def initialize(max_retries: 0, backoff_factor: 0, status_forcelist: [])
306
+ @max_retries = max_retries
307
+ @backoff_factor = backoff_factor
308
+ @status_forcelist = status_forcelist
309
+ @pool = {}
310
+ @alpn = {}
311
+ @fallback = HTTPAdapter.new(max_retries: max_retries, backoff_factor: backoff_factor, status_forcelist: status_forcelist)
312
+ end
313
+ def send_once(meth, url, hdrs, body, timeout, proxies, verify, cert, auth)
314
+ u = URI.parse(url)
315
+ sync_fallback!
316
+ return @fallback.send_once(meth, url, hdrs, body, timeout, proxies, verify, cert, auth) unless usable?(u, proxies)
317
+ conn = conn_for(u, verify, cert, timeout)
318
+ return @fallback.send_once(meth, url, hdrs, body, timeout, proxies, verify, cert, auth) if conn == :http1
319
+ attempt = 0
320
+ loop do
321
+ begin
322
+ auth.call(hdrs) if auth.respond_to?(:call) && !auth.is_a?(Requests::DigestAuth)
323
+ resp = perform(conn, meth, u, hdrs, body, timeout)
324
+ if @status_forcelist.include?(resp.code.to_i) && attempt < @max_retries
325
+ attempt += 1
326
+ sleep(retry_delay(resp, attempt))
327
+ conn = conn_for(u, verify, cert, timeout)
328
+ next
329
+ end
330
+ return resp
331
+ rescue Requests::ConnectionError, Requests::ConnectTimeout => e
332
+ drop_conn(u)
333
+ attempt += 1
334
+ raise e unless attempt <= @max_retries && idempotent?(meth)
335
+ sleep(@backoff_factor * attempt) if @backoff_factor.to_f > 0
336
+ conn = conn_for(u, verify, cert, timeout)
337
+ return @fallback.send_once(meth, url, hdrs, body, timeout, proxies, verify, cert, auth) if conn == :http1
338
+ end
339
+ end
340
+ end
341
+ def stream_to(meth, url, hdrs, body, timeout, proxies, verify, cert, auth, io, chunk_size: 65_536, progress: nil)
342
+ u = URI.parse(url)
343
+ sync_fallback!
344
+ unless usable?(u, proxies)
345
+ return @fallback.stream_to(meth, url, hdrs, body, timeout, proxies, verify, cert, auth, io, chunk_size: chunk_size, progress: progress)
346
+ end
347
+ conn = conn_for(u, verify, cert, timeout)
348
+ if conn == :http1
349
+ return @fallback.stream_to(meth, url, hdrs, body, timeout, proxies, verify, cert, auth, io, chunk_size: chunk_size, progress: progress)
350
+ end
351
+ auth.call(hdrs) if auth.respond_to?(:call) && !auth.is_a?(Requests::DigestAuth)
352
+ begin
353
+ header_list = build_headers(meth, u, hdrs)
354
+ _open_t, read_t = split_timeout(timeout)
355
+ with_timeout(read_t) { conn.request(meth, u, header_list, body, io: io, progress: progress) }
356
+ rescue Requests::ConnectionError, Requests::ConnectTimeout => e
357
+ drop_conn(u)
358
+ raise e
359
+ end
360
+ end
361
+ def decode_body(net_resp)
362
+ @fallback.decode_body(net_resp)
363
+ end
364
+ def close_pool
365
+ @pool.each_value { |c| c.close }
366
+ @pool.clear
367
+ @fallback.close_pool
368
+ end
369
+ private
370
+ def perform(conn, meth, u, hdrs, body, timeout)
371
+ header_list = build_headers(meth, u, hdrs)
372
+ _open_t, read_t = split_timeout(timeout)
373
+ with_timeout(read_t) { conn.request(meth, u, header_list, body) }
374
+ end
375
+ def with_timeout(read_t, &blk)
376
+ return blk.call unless read_t
377
+ ::Timeout.timeout(read_t, Requests::ReadTimeout, &blk)
378
+ end
379
+ def sync_fallback!
380
+ @fallback.max_retries = @max_retries
381
+ @fallback.backoff_factor = @backoff_factor
382
+ @fallback.status_forcelist = @status_forcelist
383
+ end
384
+ def idempotent?(meth)
385
+ %w[GET HEAD OPTIONS PUT DELETE].include?(meth.to_s.upcase)
386
+ end
387
+ def retry_delay(resp, attempt)
388
+ ra = resp['retry-after']
389
+ if ra
390
+ return ra.to_i if ra =~ /\A\d+\z/
391
+ begin
392
+ d = Time.httpdate(ra) - Time.now
393
+ return d.positive? ? d : 0
394
+ rescue ArgumentError
395
+ nil
396
+ end
397
+ end
398
+ @backoff_factor.to_f > 0 ? @backoff_factor * attempt : 0
399
+ end
400
+ def usable?(u, proxies)
401
+ return false unless u.scheme == 'https'
402
+ return false if proxies && !proxies.empty? && (proxies[u.scheme] || proxies[u.scheme.to_sym])
403
+ true
404
+ end
405
+ def build_headers(meth, u, hdrs)
406
+ port = u.port && u.port != 443 ? ":#{u.port}" : ''
407
+ list = [[':method', meth.to_s.upcase], [':scheme', 'https'], [':path', u.request_uri], [':authority', "#{u.host}#{port}"]]
408
+ hdrs.each { |k, v| list << [k.to_s, v.to_s] unless k.to_s.downcase == 'host' }
409
+ list
410
+ end
411
+ def key_for(u)
412
+ "#{u.host}:#{u.port}"
413
+ end
414
+ def conn_for(u, verify, cert, timeout)
415
+ k = key_for(u)
416
+ cached = @pool[k]
417
+ return cached if cached && cached.alive?
418
+ @pool.delete(k)
419
+ return :http1 if @alpn[k] == 'http1'
420
+ sock = open_tls(u, verify, cert, timeout)
421
+ if sock.alpn_protocol == 'h2'
422
+ @alpn[k] = 'h2'
423
+ @pool[k] = Http2Connection.new(sock)
424
+ else
425
+ @alpn[k] = 'http1'
426
+ sock.close rescue nil
427
+ :http1
428
+ end
429
+ end
430
+ def drop_conn(u)
431
+ c = @pool.delete(key_for(u))
432
+ c.close if c
433
+ end
434
+ def open_tls(u, verify, cert, timeout)
435
+ open_t, _read_t = split_timeout(timeout)
436
+ tcp = open_t ? Socket.tcp(u.host, u.port, connect_timeout: open_t) : Socket.tcp(u.host, u.port)
437
+ ctx = OpenSSL::SSL::SSLContext.new
438
+ ctx.alpn_protocols = ['h2', 'http/1.1']
439
+ if verify == false
440
+ ctx.verify_mode = OpenSSL::SSL::VERIFY_NONE
441
+ else
442
+ ctx.verify_mode = OpenSSL::SSL::VERIFY_PEER
443
+ ctx.ca_file = verify.is_a?(String) ? verify : Requests::CA_FILE
444
+ end
445
+ if cert
446
+ ctx.cert = OpenSSL::X509::Certificate.new(File.read(cert[0]))
447
+ ctx.key = OpenSSL::PKey::RSA.new(File.read(cert[1]))
448
+ end
449
+ ssl = OpenSSL::SSL::SSLSocket.new(tcp, ctx)
450
+ ssl.hostname = u.host
451
+ ssl.sync_close = true
452
+ ssl.connect
453
+ ssl
454
+ rescue Errno::ETIMEDOUT, IO::TimeoutError
455
+ raise Requests::ConnectTimeout, "connect timeout: #{u}"
456
+ rescue OpenSSL::SSL::SSLError => e
457
+ raise Requests::SSLError, e.message
458
+ rescue SocketError, Errno::ECONNREFUSED, Errno::EHOSTUNREACH, Errno::ECONNRESET => e
459
+ raise Requests::ConnectionError, e.message
460
+ end
461
+ def split_timeout(t)
462
+ case t
463
+ when nil then [nil, nil]
464
+ when Array then [t[0], t[1]]
465
+ else [t, t]
466
+ end
467
+ end
468
+ end
469
+ end
@@ -6,8 +6,8 @@ require 'time'
6
6
  module Requests
7
7
  class Session
8
8
  attr_accessor :headers, :cookies, :auth, :params, :proxies, :verify,
9
- :cert, :max_redirects, :timeout, :retries, :backoff_factor, :status_forcelist
10
- def initialize
9
+ :cert, :max_redirects, :timeout, :retries, :backoff_factor, :status_forcelist, :trust_env
10
+ def initialize(http2: false)
11
11
  @headers = Requests::Utils.default_headers
12
12
  @cookies = Jar.new
13
13
  @auth = nil
@@ -20,9 +20,11 @@ module Requests
20
20
  @retries = 0
21
21
  @backoff_factor = 0
22
22
  @status_forcelist = []
23
+ @trust_env = true
23
24
  @hooks = { response: [] }
24
- @adapters = { 'https://' => HTTPAdapter.new, 'http://' => HTTPAdapter.new }
25
+ @adapters = { 'https://' => http2 ? Http2Adapter.new : HTTPAdapter.new, 'http://' => HTTPAdapter.new }
25
26
  @default_adapters = @adapters.values.dup
27
+ @http2_adapter = http2 ? @adapters['https://'] : nil
26
28
  end
27
29
  def hooks
28
30
  @hooks
@@ -31,6 +33,12 @@ module Requests
31
33
  @adapters[prefix] = adapter
32
34
  self
33
35
  end
36
+ def http2!
37
+ @http2_adapter ||= Http2Adapter.new
38
+ @adapters['https://'] = @http2_adapter
39
+ @default_adapters = @adapters.values.dup
40
+ self
41
+ end
34
42
  def get(url, **kw); request('GET', url, **kw); end
35
43
  def post(url, **kw); request('POST', url, **kw); end
36
44
  def put(url, **kw); request('PUT', url, **kw); end
@@ -46,12 +54,14 @@ module Requests
46
54
  def put!(url, **kw); put(url, **kw).raise_for_status; end
47
55
  def patch!(url, **kw); patch(url, **kw).raise_for_status; end
48
56
  def delete!(url, **kw); delete(url, **kw).raise_for_status; end
57
+ def head!(url, **kw); head(url, **kw).raise_for_status; end
58
+ def options!(url, **kw); options(url, **kw).raise_for_status; end
49
59
  def close
50
60
  @adapters.values.uniq.each { |a| a.close_pool if a.respond_to?(:close_pool) }
51
61
  true
52
62
  end
53
63
  def download(url, to:, chunk_size: 65536, resume: false, progress: nil, headers: nil, params: nil,
54
- timeout: nil, proxies: nil, verify: nil, cert: nil, auth: nil)
64
+ timeout: nil, proxies: nil, verify: nil, cert: nil, auth: nil, http2: nil)
55
65
  validate_url!(url)
56
66
  full_url = build_url(url, merge_hash(@params, params))
57
67
  hdrs = CIHash.new(@headers.to_h)
@@ -61,13 +71,13 @@ module Requests
61
71
  hdrs['Range'] = "bytes=#{File.size(to)}-"
62
72
  mode = 'ab'
63
73
  end
64
- adapter = adapter_for(full_url)
74
+ adapter = pick_adapter(full_url, http2)
65
75
  adapter.max_retries = @retries
66
76
  adapter.backoff_factor = @backoff_factor
67
77
  use_auth = auth || @auth
68
78
  use_auth.call(hdrs) if use_auth.respond_to?(:call) && !use_auth.is_a?(DigestAuth)
69
79
  File.open(to, mode) do |f|
70
- net_resp = adapter.stream_to('GET', full_url, hdrs, nil, timeout || @timeout, proxies || @proxies,
80
+ net_resp = adapter.stream_to('GET', full_url, hdrs, nil, timeout || @timeout, effective_proxies(full_url, proxies),
71
81
  verify.nil? ? @verify : verify, cert || @cert, nil, f,
72
82
  chunk_size: chunk_size, progress: progress)
73
83
  unless [200, 206].include?(net_resp.code.to_i)
@@ -78,7 +88,7 @@ module Requests
78
88
  end
79
89
  def request(method, url, params: nil, data: nil, json: nil, headers: nil, cookies: nil, files: nil,
80
90
  auth: nil, timeout: nil, allow_redirects: true, proxies: nil, verify: nil, stream: false,
81
- cert: nil, hooks: nil, retries: nil)
91
+ cert: nil, hooks: nil, retries: nil, http2: nil)
82
92
  validate_url!(url)
83
93
  use_auth = auth || @auth
84
94
  merged_params = merge_hash(@params, params)
@@ -97,13 +107,14 @@ module Requests
97
107
  ck = @cookies.to_header(cur_url)
98
108
  ck = [ck, cookies.map { |k, v| "#{k}=#{v}" }.join('; ')].reject { |x| x.nil? || x.empty? }.join('; ') if cookies
99
109
  hdrs['Cookie'] = ck unless ck.empty?
100
- adapter = adapter_for(cur_url)
110
+ adapter = pick_adapter(cur_url, http2)
101
111
  adapter.max_retries = retries unless retries.nil?
102
112
  adapter.max_retries = @retries if @retries.to_i > 0 && retries.nil? && default_adapter?(adapter)
103
113
  adapter.backoff_factor = @backoff_factor if @backoff_factor.to_f > 0 && default_adapter?(adapter)
104
114
  adapter.status_forcelist = @status_forcelist if !@status_forcelist.empty? && adapter.respond_to?(:status_forcelist=) && default_adapter?(adapter)
115
+ eff_proxies = effective_proxies(cur_url, proxies)
105
116
  t0 = Time.now
106
- net_resp = adapter.send_once(cur_method, cur_url, hdrs, body, timeout || @timeout,proxies || @proxies, verify.nil? ? @verify : verify,cert || @cert, use_auth)
117
+ net_resp = adapter.send_once(cur_method, cur_url, hdrs, body, timeout || @timeout, eff_proxies, verify.nil? ? @verify : verify,cert || @cert, use_auth)
107
118
  elapsed = Time.now - t0
108
119
  resp = to_response(adapter, net_resp, cur_url, elapsed, cur_method, hdrs, body)
109
120
  @cookies.update(net_resp, cur_url)
@@ -111,7 +122,7 @@ module Requests
111
122
  if use_auth.is_a?(DigestAuth) && resp.status_code == 401 && redirs.zero? && hist.empty?
112
123
  use_auth.call(hdrs, meth: cur_method, url: cur_url, prev_resp: resp)
113
124
  t1 = Time.now
114
- net_resp = adapter.send_once(cur_method, cur_url, hdrs, body, timeout || @timeout,proxies || @proxies, verify.nil? ? @verify : verify,cert || @cert, nil)
125
+ net_resp = adapter.send_once(cur_method, cur_url, hdrs, body, timeout || @timeout, eff_proxies, verify.nil? ? @verify : verify,cert || @cert, nil)
115
126
  resp = to_response(adapter, net_resp, cur_url, Time.now - t1, cur_method, hdrs, body)
116
127
  @cookies.update(net_resp, cur_url)
117
128
  resp.cookies = @cookies
@@ -149,6 +160,19 @@ module Requests
149
160
  match = @adapters.keys.select { |prefix| url.start_with?(prefix) }.max_by(&:length)
150
161
  match ? @adapters[match] : @adapters['https://']
151
162
  end
163
+ def pick_adapter(url, http2)
164
+ return adapter_for(url) unless http2 == true
165
+ unless @http2_adapter
166
+ @http2_adapter = Http2Adapter.new
167
+ @default_adapters << @http2_adapter
168
+ end
169
+ @http2_adapter
170
+ end
171
+ def effective_proxies(url, proxies)
172
+ p = proxies || @proxies
173
+ return p if p && !p.empty?
174
+ @trust_env ? Requests::Utils.env_proxies(url) : (p || {})
175
+ end
152
176
  def merge_hash(a, b)
153
177
  return b unless a
154
178
  return a unless b
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require 'uri'
4
+
3
5
  module Requests
4
6
  module Utils
5
7
  module_function
@@ -51,5 +53,21 @@ module Requests
51
53
  def looks_like_html?(str)
52
54
  !!(str.to_s.lstrip =~ /\A(<!DOCTYPE html|<html)/i)
53
55
  end
56
+ def env_proxies(url)
57
+ host = URI.parse(url).host.to_s
58
+ no_proxy = ENV['NO_PROXY'] || ENV['no_proxy']
59
+ if no_proxy
60
+ list = no_proxy.split(',').map { |s| s.strip.downcase }.reject(&:empty?)
61
+ return {} if list.include?('*') || list.any? { |p| host.downcase == p || host.downcase.end_with?(".#{p.sub(/\A\./, '')}") }
62
+ end
63
+ out = {}
64
+ http = ENV['HTTP_PROXY'] || ENV['http_proxy']
65
+ https = ENV['HTTPS_PROXY'] || ENV['https_proxy']
66
+ out['http'] = http if http && !http.empty?
67
+ out['https'] = https if https && !https.empty?
68
+ out
69
+ rescue URI::InvalidURIError
70
+ {}
71
+ end
54
72
  end
55
73
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Requests
4
- VERSION = '1.0.3'
4
+ VERSION = '1.0.4'
5
5
  end
data/lib/requests.rb CHANGED
@@ -15,5 +15,7 @@ require_relative 'requests/auth'
15
15
  require_relative 'requests/utils'
16
16
  require_relative 'requests/models'
17
17
  require_relative 'requests/adapters'
18
+ require_relative 'requests/hpack'
19
+ require_relative 'requests/http2'
18
20
  require_relative 'requests/sessions'
19
21
  require_relative 'requests/api'
metadata CHANGED
@@ -1,13 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: requests_ruby
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.3
4
+ version: 1.0.4
5
5
  platform: ruby
6
6
  authors:
7
7
  - monji024
8
+ autorequire:
8
9
  bindir: bin
9
10
  cert_chain: []
10
- date: 2026-07-30 00:00:00.000000000 Z
11
+ date: 2026-08-25 00:00:00.000000000 Z
11
12
  dependencies:
12
13
  - !ruby/object:Gem::Dependency
13
14
  name: minitest
@@ -26,6 +27,7 @@ dependencies:
26
27
  description: A stdlib-only Ruby HTTP client modeled after python's requests library.
27
28
  Sessions, cookies, redirects, auth (Basic/Bearer/Digest), multipart uploads, streaming
28
29
  responses, hooks, and a friendly exception hierarchy - no external gems required.
30
+ email:
29
31
  executables: []
30
32
  extensions: []
31
33
  extra_rdoc_files: []
@@ -40,6 +42,8 @@ files:
40
42
  - lib/requests/auth.rb
41
43
  - lib/requests/cookies.rb
42
44
  - lib/requests/exceptions.rb
45
+ - lib/requests/hpack.rb
46
+ - lib/requests/http2.rb
43
47
  - lib/requests/models.rb
44
48
  - lib/requests/sessions.rb
45
49
  - lib/requests/status_codes.rb
@@ -56,6 +60,7 @@ metadata:
56
60
  changelog_uri: https://github.com/monji024/requests_ruby/blob/main/CHANGELOG.md
57
61
  bug_tracker_uri: https://github.com/monji024/requests_ruby/issues
58
62
  documentation_uri: https://rubydoc.info/gems/requests_ruby
63
+ post_install_message:
59
64
  rdoc_options: []
60
65
  require_paths:
61
66
  - lib
@@ -70,7 +75,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
70
75
  - !ruby/object:Gem::Version
71
76
  version: '0'
72
77
  requirements: []
73
- rubygems_version: 3.6.3
78
+ rubygems_version: 3.3.15
79
+ signing_key:
74
80
  specification_version: 4
75
81
  summary: Requests is a simple, yet elegant, HTTP library.
76
82
  test_files: []