httparty 0.18.1 → 0.20.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of httparty might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +23 -0
- data/Changelog.md +21 -0
- data/README.md +2 -4
- data/docs/README.md +67 -2
- data/examples/README.md +3 -0
- data/examples/aaws.rb +6 -2
- data/examples/idn.rb +10 -0
- data/httparty.gemspec +1 -1
- data/lib/httparty/connection_adapter.rb +3 -1
- data/lib/httparty/cookie_hash.rb +3 -1
- data/lib/httparty/decompressor.rb +92 -0
- data/lib/httparty/exceptions.rb +3 -1
- data/lib/httparty/hash_conversions.rb +4 -2
- data/lib/httparty/headers_processor.rb +2 -0
- data/lib/httparty/logger/apache_formatter.rb +4 -2
- data/lib/httparty/logger/curl_formatter.rb +6 -4
- data/lib/httparty/logger/logger.rb +2 -0
- data/lib/httparty/logger/logstash_formatter.rb +4 -2
- data/lib/httparty/module_inheritable_attributes.rb +3 -1
- data/lib/httparty/net_digest_auth.rb +9 -10
- data/lib/httparty/parser.rb +9 -5
- data/lib/httparty/request/body.rb +24 -10
- data/lib/httparty/request/multipart_boundary.rb +2 -0
- data/lib/httparty/request.rb +45 -10
- data/lib/httparty/response/headers.rb +2 -0
- data/lib/httparty/response.rb +6 -4
- data/lib/httparty/response_fragment.rb +2 -0
- data/lib/httparty/text_encoder.rb +6 -4
- data/lib/httparty/utils.rb +2 -0
- data/lib/httparty/version.rb +3 -1
- data/lib/httparty.rb +22 -3
- metadata +7 -5
- data/.travis.yml +0 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: bde34a85d8b010341759c1b2b42dfe58cad693c39f047b9803ac90afbc13290b
|
4
|
+
data.tar.gz: b5248420c90ea545dfa9e6e234d6c597b4c7306dfc8d58ff94dca82e405e0ed1
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 13316910a85a75d1204c64be41e995475b26abf9aad1165c06349c296d0b3f0f6ae644b00d8bcf14adc3d9a11659bffd060a5101dc88718af74d0324adde9e30
|
7
|
+
data.tar.gz: 668fac0ca6a513e64dead9b633ecb12c4ff926cb017e4e7e244249cfb361f93e1aa515ef3fb6b80e6245264e7da38d0599abec9931c5b33b88d7235b966b0aa1
|
@@ -0,0 +1,23 @@
|
|
1
|
+
name: CI
|
2
|
+
on: [push, pull_request]
|
3
|
+
jobs:
|
4
|
+
build:
|
5
|
+
runs-on: ubuntu-latest
|
6
|
+
strategy:
|
7
|
+
matrix:
|
8
|
+
ruby:
|
9
|
+
- 2.3
|
10
|
+
- 2.4
|
11
|
+
- 2.5
|
12
|
+
- 2.6
|
13
|
+
- 2.7
|
14
|
+
steps:
|
15
|
+
- name: Check out repository code
|
16
|
+
uses: actions/checkout@v2
|
17
|
+
- name: Set up Ruby
|
18
|
+
uses: ruby/setup-ruby@v1
|
19
|
+
with:
|
20
|
+
ruby-version: ${{ matrix.ruby }}
|
21
|
+
bundler-cache: true # Run "bundle install", and cache the result automatically.
|
22
|
+
- name: Run Rake
|
23
|
+
run: bundle exec rake
|
data/Changelog.md
CHANGED
@@ -1,3 +1,24 @@
|
|
1
|
+
## 0.20.0
|
2
|
+
|
3
|
+
Breaking changes
|
4
|
+
|
5
|
+
* Require Ruby >= 2.3.0
|
6
|
+
|
7
|
+
Fixes
|
8
|
+
|
9
|
+
* [`Marshal.dump` fails on response objects when request option `:logger` is set or `:parser` is a proc](https://github.com/jnunemaker/httparty/pull/714)
|
10
|
+
* [Switch `:pem` option to to `OpenSSL::PKey.read` to support other algorithms](https://github.com/jnunemaker/httparty/pull/720)
|
11
|
+
|
12
|
+
## 0.19.1
|
13
|
+
|
14
|
+
* [Remove use of unary + method for creating non-frozen string to increase compatibility with older versions of ruby](https://github.com/jnunemaker/httparty/commit/4416141d37fd71bdba4f37589ec265f55aa446ce)
|
15
|
+
|
16
|
+
## 0.19.0
|
17
|
+
|
18
|
+
* [Multipart/Form-Data: rewind files after read](https://github.com/jnunemaker/httparty/pull/709)
|
19
|
+
* [add frozen_string_literal pragma to all files](https://github.com/jnunemaker/httparty/pull/711)
|
20
|
+
* [Better handling of Accept-Encoding / Content-Encoding decompression (fixes #562)](https://github.com/jnunemaker/httparty/pull/729)
|
21
|
+
|
1
22
|
## 0.18.1
|
2
23
|
|
3
24
|
* [Rename cop Lint/HandleExceptions to Lint/SuppressedException](https://github.com/jnunemaker/httparty/pull/699).
|
data/README.md
CHANGED
@@ -12,7 +12,7 @@ gem install httparty
|
|
12
12
|
|
13
13
|
## Requirements
|
14
14
|
|
15
|
-
* Ruby 2.
|
15
|
+
* Ruby 2.3.0 or higher
|
16
16
|
* multi_xml
|
17
17
|
* You like to party!
|
18
18
|
|
@@ -48,7 +48,6 @@ puts stack_exchange.users
|
|
48
48
|
```
|
49
49
|
|
50
50
|
See the [examples directory](http://github.com/jnunemaker/httparty/tree/master/examples) for even more goodies.
|
51
|
-
|
52
51
|
## Command Line Interface
|
53
52
|
|
54
53
|
httparty also includes the executable `httparty` which can be
|
@@ -65,9 +64,8 @@ httparty "https://api.stackexchange.com/2.2/questions?site=stackoverflow"
|
|
65
64
|
## Help and Docs
|
66
65
|
|
67
66
|
* [Docs](https://github.com/jnunemaker/httparty/tree/master/docs)
|
68
|
-
* https://
|
67
|
+
* https://github.com/jnunemaker/httparty/discussions
|
69
68
|
* https://www.rubydoc.info/github/jnunemaker/httparty
|
70
|
-
* http://stackoverflow.com/questions/tagged/httparty
|
71
69
|
|
72
70
|
## Contributing
|
73
71
|
|
data/docs/README.md
CHANGED
@@ -70,7 +70,7 @@ class Client
|
|
70
70
|
end
|
71
71
|
```
|
72
72
|
|
73
|
-
You can also include
|
73
|
+
You can also include all of these options with the call:
|
74
74
|
|
75
75
|
```ruby
|
76
76
|
class Client
|
@@ -86,7 +86,7 @@ end
|
|
86
86
|
|
87
87
|
### Avoid SSL verification
|
88
88
|
|
89
|
-
In some cases you may want to skip SSL verification, because the entity that
|
89
|
+
In some cases you may want to skip SSL verification, because the entity that issued the certificate is not a valid one, but you still want to work with it. You can achieve this through:
|
90
90
|
|
91
91
|
```ruby
|
92
92
|
# Skips SSL certificate verification
|
@@ -104,3 +104,68 @@ class Client
|
|
104
104
|
end
|
105
105
|
end
|
106
106
|
```
|
107
|
+
|
108
|
+
### HTTP Compression
|
109
|
+
|
110
|
+
The `Accept-Encoding` request header and `Content-Encoding` response header
|
111
|
+
are used to control compression (gzip, etc.) over the wire. Refer to
|
112
|
+
[RFC-2616](https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html) for details.
|
113
|
+
(For clarity: these headers are **not** used for character encoding i.e. `utf-8`
|
114
|
+
which is specified in the `Accept` and `Content-Type` headers.)
|
115
|
+
|
116
|
+
Unless you have specific requirements otherwise, we recommend to **not** set
|
117
|
+
set the `Accept-Encoding` header on HTTParty requests. In this case, `Net::HTTP`
|
118
|
+
will set a sensible default compression scheme and automatically decompress the response.
|
119
|
+
|
120
|
+
If you explicitly set `Accept-Encoding`, there be dragons:
|
121
|
+
|
122
|
+
* If the HTTP response `Content-Encoding` received on the wire is `gzip` or `deflate`,
|
123
|
+
`Net::HTTP` will automatically decompress it, and will omit `Content-Encoding`
|
124
|
+
from your `HTTParty::Response` headers.
|
125
|
+
|
126
|
+
* For encodings `br` (Brotli) or `compress` (LZW), HTTParty will automatically
|
127
|
+
decompress if you include the `brotli` or `ruby-lzws` gems respectively into your project.
|
128
|
+
**Warning:** Support for these encodings is experimental and not fully battle-tested.
|
129
|
+
Similar to above, if decompression succeeds, `Content-Encoding` will be omitted
|
130
|
+
from your `HTTParty::Response` headers.
|
131
|
+
|
132
|
+
* For other encodings, `HTTParty::Response#body` will return the raw uncompressed byte string,
|
133
|
+
and you'll need to inspect the `Content-Encoding` response header and decompress it yourself.
|
134
|
+
In this case, `HTTParty::Response#parsed_response` will be `nil`.
|
135
|
+
|
136
|
+
* Lastly, you may use the `skip_decompression` option to disable all automatic decompression
|
137
|
+
and always get `HTTParty::Response#body` in its raw form along with the `Content-Encoding` header.
|
138
|
+
|
139
|
+
```ruby
|
140
|
+
# Accept-Encoding=gzip,deflate can be safely assumed to be auto-decompressed
|
141
|
+
|
142
|
+
res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => 'gzip,deflate,identity' })
|
143
|
+
JSON.parse(res.body) # safe
|
144
|
+
|
145
|
+
|
146
|
+
# Accept-Encoding=br,compress requires third-party gems
|
147
|
+
|
148
|
+
require 'brotli'
|
149
|
+
require 'lzws'
|
150
|
+
res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => 'br,compress' })
|
151
|
+
JSON.parse(res.body)
|
152
|
+
|
153
|
+
|
154
|
+
# Accept-Encoding=* may return unhandled Content-Encoding
|
155
|
+
|
156
|
+
res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => '*' })
|
157
|
+
encoding = res.headers['Content-Encoding']
|
158
|
+
if encoding
|
159
|
+
JSON.parse(your_decompression_handling(res.body, encoding))
|
160
|
+
else
|
161
|
+
# Content-Encoding not present implies decompressed
|
162
|
+
JSON.parse(res.body)
|
163
|
+
end
|
164
|
+
|
165
|
+
|
166
|
+
# Gimme the raw data!
|
167
|
+
|
168
|
+
res = HTTParty.get('https://example.com/test.json', skip_decompression: true)
|
169
|
+
encoding = res.headers['Content-Encoding']
|
170
|
+
JSON.parse(your_decompression_handling(res.body, encoding))
|
171
|
+
```
|
data/examples/README.md
CHANGED
@@ -84,3 +84,6 @@
|
|
84
84
|
|
85
85
|
* [Accessing x509 Peer Certificate](peer_cert.rb)
|
86
86
|
* Provides access to the server's TLS certificate
|
87
|
+
|
88
|
+
* [Accessing IDNs](idn.rb)
|
89
|
+
* Uses a `get` request with an International domain names, which are Urls with emojis and non-ASCII characters such as accented letters.
|
data/examples/aaws.rb
CHANGED
@@ -1,5 +1,7 @@
|
|
1
1
|
require 'rubygems'
|
2
2
|
require 'active_support'
|
3
|
+
require 'active_support/core_ext/hash'
|
4
|
+
require 'active_support/core_ext/string'
|
3
5
|
|
4
6
|
dir = File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib'))
|
5
7
|
require File.join(dir, 'httparty')
|
@@ -13,14 +15,16 @@ module AAWS
|
|
13
15
|
default_params Service: 'AWSECommerceService', Operation: 'ItemSearch', SearchIndex: 'Books'
|
14
16
|
|
15
17
|
def initialize(key)
|
16
|
-
|
18
|
+
@auth = { AWSAccessKeyId: key }
|
17
19
|
end
|
18
20
|
|
19
21
|
def search(options = {})
|
20
22
|
raise ArgumentError, 'You must search for something' if options[:query].blank?
|
21
23
|
|
22
24
|
# amazon uses nasty camelized query params
|
23
|
-
options[:query] = options[:query]
|
25
|
+
options[:query] = options[:query]
|
26
|
+
.reverse_merge(@auth)
|
27
|
+
.transform_keys { |k| k.to_s.camelize }
|
24
28
|
|
25
29
|
# make a request and return the items (NOTE: this doesn't handle errors at this point)
|
26
30
|
self.class.get('/onca/xml', options)['ItemSearchResponse']['Items']
|
data/examples/idn.rb
ADDED
data/httparty.gemspec
CHANGED
@@ -13,7 +13,7 @@ Gem::Specification.new do |s|
|
|
13
13
|
s.summary = 'Makes http fun! Also, makes consuming restful web services dead easy.'
|
14
14
|
s.description = 'Makes http fun! Also, makes consuming restful web services dead easy.'
|
15
15
|
|
16
|
-
s.required_ruby_version = '>= 2.
|
16
|
+
s.required_ruby_version = '>= 2.3.0'
|
17
17
|
|
18
18
|
s.add_dependency 'multi_xml', ">= 0.5.2"
|
19
19
|
s.add_dependency('mime-types', "~> 3.0")
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
# Default connection adapter that returns a new Net::HTTP each time
|
3
5
|
#
|
@@ -221,7 +223,7 @@ module HTTParty
|
|
221
223
|
# Note: options[:pem] must contain the content of a PEM file having the private key appended
|
222
224
|
if options[:pem]
|
223
225
|
http.cert = OpenSSL::X509::Certificate.new(options[:pem])
|
224
|
-
http.key = OpenSSL::PKey
|
226
|
+
http.key = OpenSSL::PKey.read(options[:pem], options[:pem_password])
|
225
227
|
http.verify_mode = verify_ssl_certificate? ? OpenSSL::SSL::VERIFY_PEER : OpenSSL::SSL::VERIFY_NONE
|
226
228
|
end
|
227
229
|
|
data/lib/httparty/cookie_hash.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
class HTTParty::CookieHash < Hash #:nodoc:
|
2
4
|
CLIENT_COOKIES = %w(path expires domain path secure httponly samesite)
|
3
5
|
|
@@ -16,6 +18,6 @@ class HTTParty::CookieHash < Hash #:nodoc:
|
|
16
18
|
end
|
17
19
|
|
18
20
|
def to_cookie_string
|
19
|
-
select { |k, v| !CLIENT_COOKIES.include?(k.to_s.downcase) }.collect { |k, v| "#{k}=#{v}" }.join(
|
21
|
+
select { |k, v| !CLIENT_COOKIES.include?(k.to_s.downcase) }.collect { |k, v| "#{k}=#{v}" }.join('; ')
|
20
22
|
end
|
21
23
|
end
|
@@ -0,0 +1,92 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module HTTParty
|
4
|
+
# Decompresses the response body based on the Content-Encoding header.
|
5
|
+
#
|
6
|
+
# Net::HTTP automatically decompresses Content-Encoding values "gzip" and "deflate".
|
7
|
+
# This class will handle "br" (Brotli) and "compress" (LZW) if the requisite
|
8
|
+
# gems are installed. Otherwise, it returns nil if the body data cannot be
|
9
|
+
# decompressed.
|
10
|
+
#
|
11
|
+
# @abstract Read the HTTP Compression section for more information.
|
12
|
+
class Decompressor
|
13
|
+
|
14
|
+
# "gzip" and "deflate" are handled by Net::HTTP
|
15
|
+
# hence they do not need to be handled by HTTParty
|
16
|
+
SupportedEncodings = {
|
17
|
+
'none' => :none,
|
18
|
+
'identity' => :none,
|
19
|
+
'br' => :brotli,
|
20
|
+
'compress' => :lzw
|
21
|
+
}.freeze
|
22
|
+
|
23
|
+
# The response body of the request
|
24
|
+
# @return [String]
|
25
|
+
attr_reader :body
|
26
|
+
|
27
|
+
# The Content-Encoding algorithm used to encode the body
|
28
|
+
# @return [Symbol] e.g. :gzip
|
29
|
+
attr_reader :encoding
|
30
|
+
|
31
|
+
# @param [String] body - the response body of the request
|
32
|
+
# @param [Symbol] encoding - the Content-Encoding algorithm used to encode the body
|
33
|
+
def initialize(body, encoding)
|
34
|
+
@body = body
|
35
|
+
@encoding = encoding
|
36
|
+
end
|
37
|
+
|
38
|
+
# Perform decompression on the response body
|
39
|
+
# @return [String] the decompressed body
|
40
|
+
# @return [nil] when the response body is nil or cannot decompressed
|
41
|
+
def decompress
|
42
|
+
return nil if body.nil?
|
43
|
+
return body if encoding.nil? || encoding.strip.empty?
|
44
|
+
|
45
|
+
if supports_encoding?
|
46
|
+
decompress_supported_encoding
|
47
|
+
else
|
48
|
+
nil
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
protected
|
53
|
+
|
54
|
+
def supports_encoding?
|
55
|
+
SupportedEncodings.keys.include?(encoding)
|
56
|
+
end
|
57
|
+
|
58
|
+
def decompress_supported_encoding
|
59
|
+
method = SupportedEncodings[encoding]
|
60
|
+
if respond_to?(method, true)
|
61
|
+
send(method)
|
62
|
+
else
|
63
|
+
raise NotImplementedError, "#{self.class.name} has not implemented a decompression method for #{encoding.inspect} encoding."
|
64
|
+
end
|
65
|
+
end
|
66
|
+
|
67
|
+
def none
|
68
|
+
body
|
69
|
+
end
|
70
|
+
|
71
|
+
def brotli
|
72
|
+
return nil unless defined?(::Brotli)
|
73
|
+
begin
|
74
|
+
::Brotli.inflate(body)
|
75
|
+
rescue StandardError
|
76
|
+
nil
|
77
|
+
end
|
78
|
+
end
|
79
|
+
|
80
|
+
def lzw
|
81
|
+
begin
|
82
|
+
if defined?(::LZWS::String)
|
83
|
+
::LZWS::String.decompress(body)
|
84
|
+
elsif defined?(::LZW::Simple)
|
85
|
+
::LZW::Simple.new.decompress(body)
|
86
|
+
end
|
87
|
+
rescue StandardError
|
88
|
+
nil
|
89
|
+
end
|
90
|
+
end
|
91
|
+
end
|
92
|
+
end
|
data/lib/httparty/exceptions.rb
CHANGED
@@ -1,5 +1,7 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
|
-
# @
|
4
|
+
# @abstract Exceptions raised by HTTParty inherit from Error
|
3
5
|
class Error < StandardError; end
|
4
6
|
|
5
7
|
# Exception raised when you attempt to set a non-existent format
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'erb'
|
2
4
|
|
3
5
|
module HTTParty
|
@@ -26,8 +28,8 @@ module HTTParty
|
|
26
28
|
def self.normalize_param(key, value)
|
27
29
|
normalized_keys = normalize_keys(key, value)
|
28
30
|
|
29
|
-
normalized_keys.flatten.each_slice(2).inject('') do |string, (k, v)|
|
30
|
-
string
|
31
|
+
normalized_keys.flatten.each_slice(2).inject(''.dup) do |string, (k, v)|
|
32
|
+
string << "#{ERB::Util.url_encode(k)}=#{ERB::Util.url_encode(v.to_s)}&"
|
31
33
|
end
|
32
34
|
end
|
33
35
|
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
module Logger
|
3
5
|
class ApacheFormatter #:nodoc:
|
@@ -26,11 +28,11 @@ module HTTParty
|
|
26
28
|
end
|
27
29
|
|
28
30
|
def current_time
|
29
|
-
Time.now.strftime(
|
31
|
+
Time.now.strftime('%Y-%m-%d %H:%M:%S %z')
|
30
32
|
end
|
31
33
|
|
32
34
|
def http_method
|
33
|
-
request.http_method.name.split(
|
35
|
+
request.http_method.name.split('::').last.upcase
|
34
36
|
end
|
35
37
|
|
36
38
|
def path
|
@@ -1,9 +1,11 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
module Logger
|
3
5
|
class CurlFormatter #:nodoc:
|
4
6
|
TAG_NAME = HTTParty.name
|
5
|
-
OUT = '>'
|
6
|
-
IN = '<'
|
7
|
+
OUT = '>'
|
8
|
+
IN = '<'
|
7
9
|
|
8
10
|
attr_accessor :level, :logger
|
9
11
|
|
@@ -20,7 +22,7 @@ module HTTParty
|
|
20
22
|
log_request
|
21
23
|
log_response
|
22
24
|
|
23
|
-
logger.public_send level, messages.join(
|
25
|
+
logger.public_send level, messages.join('\n')
|
24
26
|
end
|
25
27
|
|
26
28
|
private
|
@@ -44,7 +46,7 @@ module HTTParty
|
|
44
46
|
end
|
45
47
|
|
46
48
|
def log_url
|
47
|
-
http_method = request.http_method.name.split(
|
49
|
+
http_method = request.http_method.name.split('::').last.upcase
|
48
50
|
uri = if request.options[:base_uri]
|
49
51
|
request.options[:base_uri] + request.path.path
|
50
52
|
else
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
module Logger
|
3
5
|
class LogstashFormatter #:nodoc:
|
@@ -40,11 +42,11 @@ module HTTParty
|
|
40
42
|
end
|
41
43
|
|
42
44
|
def current_time
|
43
|
-
Time.now.strftime(
|
45
|
+
Time.now.strftime('%Y-%m-%d %H:%M:%S %z')
|
44
46
|
end
|
45
47
|
|
46
48
|
def http_method
|
47
|
-
@http_method ||= request.http_method.name.split(
|
49
|
+
@http_method ||= request.http_method.name.split('::').last.upcase
|
48
50
|
end
|
49
51
|
|
50
52
|
def path
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
module ModuleInheritableAttributes #:nodoc:
|
3
5
|
def self.included(base)
|
@@ -36,7 +38,7 @@ module HTTParty
|
|
36
38
|
def inherited(subclass)
|
37
39
|
super
|
38
40
|
@mattr_inheritable_attrs.each do |inheritable_attribute|
|
39
|
-
ivar = "@#{inheritable_attribute}"
|
41
|
+
ivar = :"@#{inheritable_attribute}"
|
40
42
|
subclass.instance_variable_set(ivar, instance_variable_get(ivar).clone)
|
41
43
|
|
42
44
|
if instance_variable_get(ivar).respond_to?(:merge)
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'digest/md5'
|
2
4
|
require 'net/http'
|
3
5
|
|
@@ -44,12 +46,9 @@ module Net
|
|
44
46
|
header << %(algorithm="#{@response['algorithm']}") if algorithm_present?
|
45
47
|
|
46
48
|
if qop_present?
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
"nc=00000001"
|
51
|
-
]
|
52
|
-
fields.each { |field| header << field }
|
49
|
+
header << %(cnonce="#{@cnonce}")
|
50
|
+
header << %(qop="#{@response['qop']}")
|
51
|
+
header << 'nc=00000001'
|
53
52
|
end
|
54
53
|
|
55
54
|
header << %(opaque="#{@response['opaque']}") if opaque_present?
|
@@ -98,13 +97,13 @@ module Net
|
|
98
97
|
end
|
99
98
|
|
100
99
|
def random
|
101
|
-
format
|
100
|
+
format '%x', (Time.now.to_i + rand(65535))
|
102
101
|
end
|
103
102
|
|
104
103
|
def request_digest
|
105
104
|
a = [md5(a1), @response['nonce'], md5(a2)]
|
106
|
-
a.insert(2,
|
107
|
-
md5(a.join(
|
105
|
+
a.insert(2, '00000001', @cnonce, @response['qop']) if qop_present?
|
106
|
+
md5(a.join(':'))
|
108
107
|
end
|
109
108
|
|
110
109
|
def md5(str)
|
@@ -129,7 +128,7 @@ module Net
|
|
129
128
|
end
|
130
129
|
|
131
130
|
def a2
|
132
|
-
[@method, @path].join(
|
131
|
+
[@method, @path].join(':')
|
133
132
|
end
|
134
133
|
end
|
135
134
|
end
|
data/lib/httparty/parser.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
# The default parser used by HTTParty, supports xml, json, html, csv and
|
3
5
|
# plain text.
|
@@ -101,7 +103,7 @@ module HTTParty
|
|
101
103
|
# @return [nil] when the response body is nil, an empty string, spaces only or "null"
|
102
104
|
def parse
|
103
105
|
return nil if body.nil?
|
104
|
-
return nil if body ==
|
106
|
+
return nil if body == 'null'
|
105
107
|
return nil if body.valid_encoding? && body.strip.empty?
|
106
108
|
if body.valid_encoding? && body.encoding == Encoding::UTF_8
|
107
109
|
@body = body.gsub(/\A#{UTF8_BOM}/, '')
|
@@ -119,7 +121,7 @@ module HTTParty
|
|
119
121
|
MultiXml.parse(body)
|
120
122
|
end
|
121
123
|
|
122
|
-
UTF8_BOM = "\xEF\xBB\xBF"
|
124
|
+
UTF8_BOM = "\xEF\xBB\xBF"
|
123
125
|
|
124
126
|
def json
|
125
127
|
JSON.parse(body, :quirks_mode => true, :allow_nan => true)
|
@@ -142,9 +144,11 @@ module HTTParty
|
|
142
144
|
end
|
143
145
|
|
144
146
|
def parse_supported_format
|
145
|
-
|
146
|
-
|
147
|
-
|
147
|
+
if respond_to?(format, true)
|
148
|
+
send(format)
|
149
|
+
else
|
150
|
+
raise NotImplementedError, "#{self.class.name} has not implemented a parsing method for the #{format.inspect} format."
|
151
|
+
end
|
148
152
|
end
|
149
153
|
end
|
150
154
|
end
|
@@ -1,8 +1,13 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require_relative 'multipart_boundary'
|
2
4
|
|
3
5
|
module HTTParty
|
4
6
|
class Request
|
5
7
|
class Body
|
8
|
+
NEWLINE = "\r\n"
|
9
|
+
private_constant :NEWLINE
|
10
|
+
|
6
11
|
def initialize(params, query_string_normalizer: nil, force_multipart: false)
|
7
12
|
@params = params
|
8
13
|
@query_string_normalizer = query_string_normalizer
|
@@ -30,20 +35,20 @@ module HTTParty
|
|
30
35
|
def generate_multipart
|
31
36
|
normalized_params = params.flat_map { |key, value| HashConversions.normalize_keys(key, value) }
|
32
37
|
|
33
|
-
multipart = normalized_params.inject('') do |memo, (key, value)|
|
34
|
-
memo
|
35
|
-
memo
|
38
|
+
multipart = normalized_params.inject(''.dup) do |memo, (key, value)|
|
39
|
+
memo << "--#{boundary}#{NEWLINE}"
|
40
|
+
memo << %(Content-Disposition: form-data; name="#{key}")
|
36
41
|
# value.path is used to support ActionDispatch::Http::UploadedFile
|
37
42
|
# https://github.com/jnunemaker/httparty/pull/585
|
38
|
-
memo
|
39
|
-
memo
|
40
|
-
memo
|
41
|
-
memo
|
42
|
-
memo
|
43
|
-
memo
|
43
|
+
memo << %(; filename="#{file_name(value)}") if file?(value)
|
44
|
+
memo << NEWLINE
|
45
|
+
memo << "Content-Type: #{content_type(value)}#{NEWLINE}" if file?(value)
|
46
|
+
memo << NEWLINE
|
47
|
+
memo << content_body(value)
|
48
|
+
memo << NEWLINE
|
44
49
|
end
|
45
50
|
|
46
|
-
multipart
|
51
|
+
multipart << "--#{boundary}--#{NEWLINE}"
|
47
52
|
end
|
48
53
|
|
49
54
|
def has_file?(value)
|
@@ -68,6 +73,15 @@ module HTTParty
|
|
68
73
|
end
|
69
74
|
end
|
70
75
|
|
76
|
+
def content_body(object)
|
77
|
+
if file?(object)
|
78
|
+
object = (file = object).read
|
79
|
+
file.rewind if file.respond_to?(:rewind)
|
80
|
+
end
|
81
|
+
|
82
|
+
object.to_s
|
83
|
+
end
|
84
|
+
|
71
85
|
def content_type(object)
|
72
86
|
return object.content_type if object.respond_to?(:content_type)
|
73
87
|
mime = MIME::Types.type_for(object.path)
|
data/lib/httparty/request.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'erb'
|
2
4
|
|
3
5
|
module HTTParty
|
@@ -44,6 +46,11 @@ module HTTParty
|
|
44
46
|
end.flatten.join('&')
|
45
47
|
end
|
46
48
|
|
49
|
+
def self._load(data)
|
50
|
+
http_method, path, options = Marshal.load(data)
|
51
|
+
new(http_method, path, options)
|
52
|
+
end
|
53
|
+
|
47
54
|
attr_accessor :http_method, :options, :last_response, :redirect, :last_uri
|
48
55
|
attr_reader :path
|
49
56
|
|
@@ -87,11 +94,11 @@ module HTTParty
|
|
87
94
|
end
|
88
95
|
|
89
96
|
def uri
|
90
|
-
if redirect && path.relative? && path.path[0] !=
|
91
|
-
last_uri_host = @last_uri.path.gsub(/[^\/]+$/,
|
97
|
+
if redirect && path.relative? && path.path[0] != '/'
|
98
|
+
last_uri_host = @last_uri.path.gsub(/[^\/]+$/, '')
|
92
99
|
|
93
|
-
path.path = "/#{path.path}" if last_uri_host[-1] !=
|
94
|
-
path.path = last_uri_host
|
100
|
+
path.path = "/#{path.path}" if last_uri_host[-1] != '/'
|
101
|
+
path.path = "#{last_uri_host}#{path.path}"
|
95
102
|
end
|
96
103
|
|
97
104
|
if path.relative? && path.host
|
@@ -117,7 +124,7 @@ module HTTParty
|
|
117
124
|
def base_uri
|
118
125
|
if redirect
|
119
126
|
base_uri = "#{@last_uri.scheme}://#{@last_uri.host}"
|
120
|
-
base_uri
|
127
|
+
base_uri = "#{base_uri}:#{@last_uri.port}" if @last_uri.port != 80
|
121
128
|
base_uri
|
122
129
|
else
|
123
130
|
options[:base_uri] && HTTParty.normalize_base_uri(options[:base_uri])
|
@@ -173,6 +180,13 @@ module HTTParty
|
|
173
180
|
@raw_request.body
|
174
181
|
end
|
175
182
|
|
183
|
+
def _dump(_level)
|
184
|
+
opts = options.dup
|
185
|
+
opts.delete(:logger)
|
186
|
+
opts.delete(:parser) if opts[:parser] && opts[:parser].is_a?(Proc)
|
187
|
+
Marshal.dump([http_method, path, opts])
|
188
|
+
end
|
189
|
+
|
176
190
|
private
|
177
191
|
|
178
192
|
def http
|
@@ -227,6 +241,8 @@ module HTTParty
|
|
227
241
|
@raw_request.body = body.call
|
228
242
|
end
|
229
243
|
|
244
|
+
@raw_request.instance_variable_set(:@decode_content, decompress_content?)
|
245
|
+
|
230
246
|
if options[:basic_auth] && send_authorization_header?
|
231
247
|
@raw_request.basic_auth(username, password)
|
232
248
|
@credentials_sent = true
|
@@ -238,6 +254,10 @@ module HTTParty
|
|
238
254
|
!!options[:digest_auth]
|
239
255
|
end
|
240
256
|
|
257
|
+
def decompress_content?
|
258
|
+
!options[:skip_decompression]
|
259
|
+
end
|
260
|
+
|
241
261
|
def response_unauthorized?
|
242
262
|
!!last_response && last_response.code == '401'
|
243
263
|
end
|
@@ -261,7 +281,7 @@ module HTTParty
|
|
261
281
|
query_string_parts << options[:query] unless options[:query].nil?
|
262
282
|
end
|
263
283
|
|
264
|
-
query_string_parts.reject!(&:empty?) unless query_string_parts == [
|
284
|
+
query_string_parts.reject!(&:empty?) unless query_string_parts == ['']
|
265
285
|
query_string_parts.size > 0 ? query_string_parts.join('&') : nil
|
266
286
|
end
|
267
287
|
|
@@ -269,7 +289,7 @@ module HTTParty
|
|
269
289
|
options[:assume_utf16_is_big_endian]
|
270
290
|
end
|
271
291
|
|
272
|
-
def handle_response(
|
292
|
+
def handle_response(raw_body, &block)
|
273
293
|
if response_redirects?
|
274
294
|
options[:limit] -= 1
|
275
295
|
if options[:logger]
|
@@ -290,9 +310,20 @@ module HTTParty
|
|
290
310
|
capture_cookies(last_response)
|
291
311
|
perform(&block)
|
292
312
|
else
|
293
|
-
|
294
|
-
|
295
|
-
|
313
|
+
raw_body ||= last_response.body
|
314
|
+
|
315
|
+
body = decompress(raw_body, last_response['content-encoding']) unless raw_body.nil?
|
316
|
+
|
317
|
+
unless body.nil?
|
318
|
+
body = encode_text(body, last_response['content-type'])
|
319
|
+
|
320
|
+
if decompress_content?
|
321
|
+
last_response.delete('content-encoding')
|
322
|
+
raw_body = body
|
323
|
+
end
|
324
|
+
end
|
325
|
+
|
326
|
+
Response.new(self, last_response, lambda { parse_response(body) }, body: raw_body)
|
296
327
|
end
|
297
328
|
end
|
298
329
|
|
@@ -368,6 +399,10 @@ module HTTParty
|
|
368
399
|
end
|
369
400
|
end
|
370
401
|
|
402
|
+
def decompress(body, encoding)
|
403
|
+
Decompressor.new(body, encoding).decompress
|
404
|
+
end
|
405
|
+
|
371
406
|
def encode_text(text, content_type)
|
372
407
|
TextEncoder.new(
|
373
408
|
text,
|
data/lib/httparty/response.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
class Response < Object
|
3
5
|
def self.underscore(string)
|
@@ -49,14 +51,14 @@ module HTTParty
|
|
49
51
|
end
|
50
52
|
|
51
53
|
def inspect
|
52
|
-
inspect_id = ::Kernel::format
|
54
|
+
inspect_id = ::Kernel::format '%x', (object_id * 2)
|
53
55
|
%(#<#{self.class}:0x#{inspect_id} parsed_response=#{parsed_response.inspect}, @response=#{response.inspect}, @headers=#{headers.inspect}>)
|
54
56
|
end
|
55
57
|
|
56
58
|
CODES_TO_OBJ = ::Net::HTTPResponse::CODE_CLASS_TO_OBJ.merge ::Net::HTTPResponse::CODE_TO_OBJ
|
57
59
|
|
58
60
|
CODES_TO_OBJ.each do |response_code, klass|
|
59
|
-
name = klass.name.sub(
|
61
|
+
name = klass.name.sub('Net::HTTP', '')
|
60
62
|
name = "#{underscore(name)}?".to_sym
|
61
63
|
|
62
64
|
define_method(name) do
|
@@ -65,12 +67,12 @@ module HTTParty
|
|
65
67
|
end
|
66
68
|
|
67
69
|
# Support old multiple_choice? method from pre 2.0.0 era.
|
68
|
-
if ::RUBY_VERSION >=
|
70
|
+
if ::RUBY_VERSION >= '2.0.0' && ::RUBY_PLATFORM != 'java'
|
69
71
|
alias_method :multiple_choice?, :multiple_choices?
|
70
72
|
end
|
71
73
|
|
72
74
|
# Support old status codes method from pre 2.6.0 era.
|
73
|
-
if ::RUBY_VERSION >=
|
75
|
+
if ::RUBY_VERSION >= '2.6.0' && ::RUBY_PLATFORM != 'java'
|
74
76
|
alias_method :gateway_time_out?, :gateway_timeout?
|
75
77
|
alias_method :request_entity_too_large?, :payload_too_large?
|
76
78
|
alias_method :request_time_out?, :request_timeout?
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
class TextEncoder
|
3
5
|
attr_reader :text, :content_type, :assume_utf16_is_big_endian
|
@@ -33,16 +35,16 @@ module HTTParty
|
|
33
35
|
def encode_utf_16
|
34
36
|
if text.bytesize >= 2
|
35
37
|
if text.getbyte(0) == 0xFF && text.getbyte(1) == 0xFE
|
36
|
-
return text.force_encoding(
|
38
|
+
return text.force_encoding('UTF-16LE')
|
37
39
|
elsif text.getbyte(0) == 0xFE && text.getbyte(1) == 0xFF
|
38
|
-
return text.force_encoding(
|
40
|
+
return text.force_encoding('UTF-16BE')
|
39
41
|
end
|
40
42
|
end
|
41
43
|
|
42
44
|
if assume_utf16_is_big_endian # option
|
43
|
-
text.force_encoding(
|
45
|
+
text.force_encoding('UTF-16BE')
|
44
46
|
else
|
45
|
-
text.force_encoding(
|
47
|
+
text.force_encoding('UTF-16LE')
|
46
48
|
end
|
47
49
|
end
|
48
50
|
|
data/lib/httparty/utils.rb
CHANGED
data/lib/httparty/version.rb
CHANGED
data/lib/httparty.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'pathname'
|
2
4
|
require 'net/http'
|
3
5
|
require 'net/https'
|
@@ -16,6 +18,7 @@ require 'httparty/connection_adapter'
|
|
16
18
|
require 'httparty/logger/logger'
|
17
19
|
require 'httparty/request/body'
|
18
20
|
require 'httparty/response_fragment'
|
21
|
+
require 'httparty/decompressor'
|
19
22
|
require 'httparty/text_encoder'
|
20
23
|
require 'httparty/headers_processor'
|
21
24
|
|
@@ -26,8 +29,8 @@ module HTTParty
|
|
26
29
|
base.send :include, ModuleInheritableAttributes
|
27
30
|
base.send(:mattr_inheritable, :default_options)
|
28
31
|
base.send(:mattr_inheritable, :default_cookies)
|
29
|
-
base.instance_variable_set(
|
30
|
-
base.instance_variable_set(
|
32
|
+
base.instance_variable_set(:@default_options, {})
|
33
|
+
base.instance_variable_set(:@default_cookies, CookieHash.new)
|
31
34
|
end
|
32
35
|
|
33
36
|
# == Common Request Options
|
@@ -399,6 +402,22 @@ module HTTParty
|
|
399
402
|
default_options[:ssl_version] = version
|
400
403
|
end
|
401
404
|
|
405
|
+
# Deactivate automatic decompression of the response body.
|
406
|
+
# This will require you to explicitly handle body decompression
|
407
|
+
# by inspecting the Content-Encoding response header.
|
408
|
+
#
|
409
|
+
# Refer to docs/README.md "HTTP Compression" section for
|
410
|
+
# further details.
|
411
|
+
#
|
412
|
+
# @example
|
413
|
+
# class Foo
|
414
|
+
# include HTTParty
|
415
|
+
# skip_decompression
|
416
|
+
# end
|
417
|
+
def skip_decompression(value = true)
|
418
|
+
default_options[:skip_decompression] = !!value
|
419
|
+
end
|
420
|
+
|
402
421
|
# Allows setting of SSL ciphers to use. This only works in Ruby 1.9+.
|
403
422
|
# You can get a list of valid specific ciphers from OpenSSL::Cipher.ciphers.
|
404
423
|
# You also can specify a cipher suite here, listed here at openssl.org:
|
@@ -597,7 +616,7 @@ module HTTParty
|
|
597
616
|
def process_cookies(options) #:nodoc:
|
598
617
|
return unless options[:cookies] || default_cookies.any?
|
599
618
|
options[:headers] ||= headers.dup
|
600
|
-
options[:headers][
|
619
|
+
options[:headers]['cookie'] = cookies.merge(options.delete(:cookies) || {}).to_cookie_string
|
601
620
|
end
|
602
621
|
|
603
622
|
def validate_format
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: httparty
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.20.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- John Nunemaker
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2021-09-29 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: multi_xml
|
@@ -48,11 +48,11 @@ extensions: []
|
|
48
48
|
extra_rdoc_files: []
|
49
49
|
files:
|
50
50
|
- ".editorconfig"
|
51
|
+
- ".github/workflows/ci.yml"
|
51
52
|
- ".gitignore"
|
52
53
|
- ".rubocop.yml"
|
53
54
|
- ".rubocop_todo.yml"
|
54
55
|
- ".simplecov"
|
55
|
-
- ".travis.yml"
|
56
56
|
- CONTRIBUTING.md
|
57
57
|
- Changelog.md
|
58
58
|
- Gemfile
|
@@ -72,6 +72,7 @@ files:
|
|
72
72
|
- examples/delicious.rb
|
73
73
|
- examples/google.rb
|
74
74
|
- examples/headers_and_user_agents.rb
|
75
|
+
- examples/idn.rb
|
75
76
|
- examples/logging.rb
|
76
77
|
- examples/microsoft_graph.rb
|
77
78
|
- examples/multipart.rb
|
@@ -88,6 +89,7 @@ files:
|
|
88
89
|
- lib/httparty.rb
|
89
90
|
- lib/httparty/connection_adapter.rb
|
90
91
|
- lib/httparty/cookie_hash.rb
|
92
|
+
- lib/httparty/decompressor.rb
|
91
93
|
- lib/httparty/exceptions.rb
|
92
94
|
- lib/httparty/hash_conversions.rb
|
93
95
|
- lib/httparty/headers_processor.rb
|
@@ -122,14 +124,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
122
124
|
requirements:
|
123
125
|
- - ">="
|
124
126
|
- !ruby/object:Gem::Version
|
125
|
-
version: 2.
|
127
|
+
version: 2.3.0
|
126
128
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
127
129
|
requirements:
|
128
130
|
- - ">="
|
129
131
|
- !ruby/object:Gem::Version
|
130
132
|
version: '0'
|
131
133
|
requirements: []
|
132
|
-
rubygems_version: 3.
|
134
|
+
rubygems_version: 3.0.3
|
133
135
|
signing_key:
|
134
136
|
specification_version: 4
|
135
137
|
summary: Makes http fun! Also, makes consuming restful web services dead easy.
|