httparty 0.18.1 → 0.19.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of httparty might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +28 -0
- data/Changelog.md +6 -0
- data/README.md +1 -3
- data/docs/README.md +67 -2
- data/examples/README.md +3 -0
- data/examples/aaws.rb +6 -2
- data/examples/idn.rb +10 -0
- data/lib/httparty/connection_adapter.rb +2 -0
- data/lib/httparty/cookie_hash.rb +3 -1
- data/lib/httparty/decompressor.rb +92 -0
- data/lib/httparty/exceptions.rb +3 -1
- data/lib/httparty/hash_conversions.rb +4 -2
- data/lib/httparty/headers_processor.rb +2 -0
- data/lib/httparty/logger/apache_formatter.rb +4 -2
- data/lib/httparty/logger/curl_formatter.rb +6 -4
- data/lib/httparty/logger/logger.rb +2 -0
- data/lib/httparty/logger/logstash_formatter.rb +4 -2
- data/lib/httparty/module_inheritable_attributes.rb +3 -1
- data/lib/httparty/net_digest_auth.rb +9 -10
- data/lib/httparty/parser.rb +9 -5
- data/lib/httparty/request/body.rb +24 -10
- data/lib/httparty/request/multipart_boundary.rb +2 -0
- data/lib/httparty/request.rb +33 -10
- data/lib/httparty/response/headers.rb +2 -0
- data/lib/httparty/response.rb +6 -4
- data/lib/httparty/response_fragment.rb +2 -0
- data/lib/httparty/text_encoder.rb +6 -4
- data/lib/httparty/utils.rb +2 -0
- data/lib/httparty/version.rb +3 -1
- data/lib/httparty.rb +22 -3
- metadata +6 -4
- data/.travis.yml +0 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 11f43b51a670445d42c8eef37875fa9b2c4340edd67bff72cd1ba6b3dd7a0001
|
4
|
+
data.tar.gz: b3a0a431cc81a2efde61d232468839001661ef07837a5a91471bc1123601d212
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8621cfbba4e743544cfdf603a44bd347b4854a099eac6db284b7594151dbf68bf62b393ff445f67fbe4723e4156bb8e0e7760d72b4971a777968c8cdd3fd78e6
|
7
|
+
data.tar.gz: 1f6f0ea5181a674b91cce360b9d9c3dab1b8a97affc78827aa1d07520066d4712ac93fad73583e439a876ddc80a75b20135c6656d396e3f394d5b3baf04eb127
|
@@ -0,0 +1,28 @@
|
|
1
|
+
name: CI
|
2
|
+
on: [push, pull_request]
|
3
|
+
jobs:
|
4
|
+
build:
|
5
|
+
runs-on: ubuntu-latest
|
6
|
+
strategy:
|
7
|
+
matrix:
|
8
|
+
ruby: ['2.5', '2.6', '2.7']
|
9
|
+
steps:
|
10
|
+
- name: Check out repository code
|
11
|
+
uses: actions/checkout@v2
|
12
|
+
- name: Do some action caching
|
13
|
+
uses: actions/cache@v1
|
14
|
+
with:
|
15
|
+
path: vendor/bundle
|
16
|
+
key: ${{ runner.os }}-gem-${{ hashFiles('**/Gemfile.lock') }}
|
17
|
+
restore-keys: |
|
18
|
+
${{ runner.os }}-gem-
|
19
|
+
- name: Set up Ruby
|
20
|
+
uses: actions/setup-ruby@v1
|
21
|
+
with:
|
22
|
+
ruby-version: ${{ matrix.ruby }}
|
23
|
+
- name: Install bundler
|
24
|
+
run: gem install bundler
|
25
|
+
- name: Run bundler
|
26
|
+
run: bundle install --jobs 4 --retry 3
|
27
|
+
- name: Run Rake
|
28
|
+
run: bundle exec rake
|
data/Changelog.md
CHANGED
@@ -1,3 +1,9 @@
|
|
1
|
+
## 0.19.0
|
2
|
+
|
3
|
+
* [Multipart/Form-Data: rewind files after read](https://github.com/jnunemaker/httparty/pull/709)
|
4
|
+
* [add frozen_string_literal pragma to all files](https://github.com/jnunemaker/httparty/pull/711)
|
5
|
+
* [Better handling of Accept-Encoding / Content-Encoding decompression (fixes #562)](https://github.com/jnunemaker/httparty/pull/729)
|
6
|
+
|
1
7
|
## 0.18.1
|
2
8
|
|
3
9
|
* [Rename cop Lint/HandleExceptions to Lint/SuppressedException](https://github.com/jnunemaker/httparty/pull/699).
|
data/README.md
CHANGED
@@ -48,7 +48,6 @@ puts stack_exchange.users
|
|
48
48
|
```
|
49
49
|
|
50
50
|
See the [examples directory](http://github.com/jnunemaker/httparty/tree/master/examples) for even more goodies.
|
51
|
-
|
52
51
|
## Command Line Interface
|
53
52
|
|
54
53
|
httparty also includes the executable `httparty` which can be
|
@@ -65,9 +64,8 @@ httparty "https://api.stackexchange.com/2.2/questions?site=stackoverflow"
|
|
65
64
|
## Help and Docs
|
66
65
|
|
67
66
|
* [Docs](https://github.com/jnunemaker/httparty/tree/master/docs)
|
68
|
-
* https://
|
67
|
+
* https://github.com/jnunemaker/httparty/discussions
|
69
68
|
* https://www.rubydoc.info/github/jnunemaker/httparty
|
70
|
-
* http://stackoverflow.com/questions/tagged/httparty
|
71
69
|
|
72
70
|
## Contributing
|
73
71
|
|
data/docs/README.md
CHANGED
@@ -70,7 +70,7 @@ class Client
|
|
70
70
|
end
|
71
71
|
```
|
72
72
|
|
73
|
-
You can also include
|
73
|
+
You can also include all of these options with the call:
|
74
74
|
|
75
75
|
```ruby
|
76
76
|
class Client
|
@@ -86,7 +86,7 @@ end
|
|
86
86
|
|
87
87
|
### Avoid SSL verification
|
88
88
|
|
89
|
-
In some cases you may want to skip SSL verification, because the entity that
|
89
|
+
In some cases you may want to skip SSL verification, because the entity that issued the certificate is not a valid one, but you still want to work with it. You can achieve this through:
|
90
90
|
|
91
91
|
```ruby
|
92
92
|
# Skips SSL certificate verification
|
@@ -104,3 +104,68 @@ class Client
|
|
104
104
|
end
|
105
105
|
end
|
106
106
|
```
|
107
|
+
|
108
|
+
### HTTP Compression
|
109
|
+
|
110
|
+
The `Accept-Encoding` request header and `Content-Encoding` response header
|
111
|
+
are used to control compression (gzip, etc.) over the wire. Refer to
|
112
|
+
[RFC-2616](https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html) for details.
|
113
|
+
(For clarity: these headers are **not** used for character encoding i.e. `utf-8`
|
114
|
+
which is specified in the `Accept` and `Content-Type` headers.)
|
115
|
+
|
116
|
+
Unless you have specific requirements otherwise, we recommend to **not** set
|
117
|
+
set the `Accept-Encoding` header on HTTParty requests. In this case, `Net::HTTP`
|
118
|
+
will set a sensible default compression scheme and automatically decompress the response.
|
119
|
+
|
120
|
+
If you explicitly set `Accept-Encoding`, there be dragons:
|
121
|
+
|
122
|
+
* If the HTTP response `Content-Encoding` received on the wire is `gzip` or `deflate`,
|
123
|
+
`Net::HTTP` will automatically decompress it, and will omit `Content-Encoding`
|
124
|
+
from your `HTTParty::Response` headers.
|
125
|
+
|
126
|
+
* For encodings `br` (Brotli) or `compress` (LZW), HTTParty will automatically
|
127
|
+
decompress if you include the `brotli` or `ruby-lzws` gems respectively into your project.
|
128
|
+
**Warning:** Support for these encodings is experimental and not fully battle-tested.
|
129
|
+
Similar to above, if decompression succeeds, `Content-Encoding` will be omitted
|
130
|
+
from your `HTTParty::Response` headers.
|
131
|
+
|
132
|
+
* For other encodings, `HTTParty::Response#body` will return the raw uncompressed byte string,
|
133
|
+
and you'll need to inspect the `Content-Encoding` response header and decompress it yourself.
|
134
|
+
In this case, `HTTParty::Response#parsed_response` will be `nil`.
|
135
|
+
|
136
|
+
* Lastly, you may use the `skip_decompression` option to disable all automatic decompression
|
137
|
+
and always get `HTTParty::Response#body` in its raw form along with the `Content-Encoding` header.
|
138
|
+
|
139
|
+
```ruby
|
140
|
+
# Accept-Encoding=gzip,deflate can be safely assumed to be auto-decompressed
|
141
|
+
|
142
|
+
res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => 'gzip,deflate,identity' })
|
143
|
+
JSON.parse(res.body) # safe
|
144
|
+
|
145
|
+
|
146
|
+
# Accept-Encoding=br,compress requires third-party gems
|
147
|
+
|
148
|
+
require 'brotli'
|
149
|
+
require 'lzws'
|
150
|
+
res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => 'br,compress' })
|
151
|
+
JSON.parse(res.body)
|
152
|
+
|
153
|
+
|
154
|
+
# Accept-Encoding=* may return unhandled Content-Encoding
|
155
|
+
|
156
|
+
res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => '*' })
|
157
|
+
encoding = res.headers['Content-Encoding']
|
158
|
+
if encoding
|
159
|
+
JSON.parse(your_decompression_handling(res.body, encoding))
|
160
|
+
else
|
161
|
+
# Content-Encoding not present implies decompressed
|
162
|
+
JSON.parse(res.body)
|
163
|
+
end
|
164
|
+
|
165
|
+
|
166
|
+
# Gimme the raw data!
|
167
|
+
|
168
|
+
res = HTTParty.get('https://example.com/test.json', skip_decompression: true)
|
169
|
+
encoding = res.headers['Content-Encoding']
|
170
|
+
JSON.parse(your_decompression_handling(res.body, encoding))
|
171
|
+
```
|
data/examples/README.md
CHANGED
@@ -84,3 +84,6 @@
|
|
84
84
|
|
85
85
|
* [Accessing x509 Peer Certificate](peer_cert.rb)
|
86
86
|
* Provides access to the server's TLS certificate
|
87
|
+
|
88
|
+
* [Accessing IDNs](idn.rb)
|
89
|
+
* Uses a `get` request with an International domain names, which are Urls with emojis and non-ASCII characters such as accented letters.
|
data/examples/aaws.rb
CHANGED
@@ -1,5 +1,7 @@
|
|
1
1
|
require 'rubygems'
|
2
2
|
require 'active_support'
|
3
|
+
require 'active_support/core_ext/hash'
|
4
|
+
require 'active_support/core_ext/string'
|
3
5
|
|
4
6
|
dir = File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib'))
|
5
7
|
require File.join(dir, 'httparty')
|
@@ -13,14 +15,16 @@ module AAWS
|
|
13
15
|
default_params Service: 'AWSECommerceService', Operation: 'ItemSearch', SearchIndex: 'Books'
|
14
16
|
|
15
17
|
def initialize(key)
|
16
|
-
|
18
|
+
@auth = { AWSAccessKeyId: key }
|
17
19
|
end
|
18
20
|
|
19
21
|
def search(options = {})
|
20
22
|
raise ArgumentError, 'You must search for something' if options[:query].blank?
|
21
23
|
|
22
24
|
# amazon uses nasty camelized query params
|
23
|
-
options[:query] = options[:query]
|
25
|
+
options[:query] = options[:query]
|
26
|
+
.reverse_merge(@auth)
|
27
|
+
.transform_keys { |k| k.to_s.camelize }
|
24
28
|
|
25
29
|
# make a request and return the items (NOTE: this doesn't handle errors at this point)
|
26
30
|
self.class.get('/onca/xml', options)['ItemSearchResponse']['Items']
|
data/examples/idn.rb
ADDED
data/lib/httparty/cookie_hash.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
class HTTParty::CookieHash < Hash #:nodoc:
|
2
4
|
CLIENT_COOKIES = %w(path expires domain path secure httponly samesite)
|
3
5
|
|
@@ -16,6 +18,6 @@ class HTTParty::CookieHash < Hash #:nodoc:
|
|
16
18
|
end
|
17
19
|
|
18
20
|
def to_cookie_string
|
19
|
-
select { |k, v| !CLIENT_COOKIES.include?(k.to_s.downcase) }.collect { |k, v| "#{k}=#{v}" }.join(
|
21
|
+
select { |k, v| !CLIENT_COOKIES.include?(k.to_s.downcase) }.collect { |k, v| "#{k}=#{v}" }.join('; ')
|
20
22
|
end
|
21
23
|
end
|
@@ -0,0 +1,92 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module HTTParty
|
4
|
+
# Decompresses the response body based on the Content-Encoding header.
|
5
|
+
#
|
6
|
+
# Net::HTTP automatically decompresses Content-Encoding values "gzip" and "deflate".
|
7
|
+
# This class will handle "br" (Brotli) and "compress" (LZW) if the requisite
|
8
|
+
# gems are installed. Otherwise, it returns nil if the body data cannot be
|
9
|
+
# decompressed.
|
10
|
+
#
|
11
|
+
# @abstract Read the HTTP Compression section for more information.
|
12
|
+
class Decompressor
|
13
|
+
|
14
|
+
# "gzip" and "deflate" are handled by Net::HTTP
|
15
|
+
# hence they do not need to be handled by HTTParty
|
16
|
+
SupportedEncodings = {
|
17
|
+
'none' => :none,
|
18
|
+
'identity' => :none,
|
19
|
+
'br' => :brotli,
|
20
|
+
'compress' => :lzw
|
21
|
+
}.freeze
|
22
|
+
|
23
|
+
# The response body of the request
|
24
|
+
# @return [String]
|
25
|
+
attr_reader :body
|
26
|
+
|
27
|
+
# The Content-Encoding algorithm used to encode the body
|
28
|
+
# @return [Symbol] e.g. :gzip
|
29
|
+
attr_reader :encoding
|
30
|
+
|
31
|
+
# @param [String] body - the response body of the request
|
32
|
+
# @param [Symbol] encoding - the Content-Encoding algorithm used to encode the body
|
33
|
+
def initialize(body, encoding)
|
34
|
+
@body = body
|
35
|
+
@encoding = encoding
|
36
|
+
end
|
37
|
+
|
38
|
+
# Perform decompression on the response body
|
39
|
+
# @return [String] the decompressed body
|
40
|
+
# @return [nil] when the response body is nil or cannot decompressed
|
41
|
+
def decompress
|
42
|
+
return nil if body.nil?
|
43
|
+
return body if encoding.nil? || encoding.strip.empty?
|
44
|
+
|
45
|
+
if supports_encoding?
|
46
|
+
decompress_supported_encoding
|
47
|
+
else
|
48
|
+
nil
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
protected
|
53
|
+
|
54
|
+
def supports_encoding?
|
55
|
+
SupportedEncodings.keys.include?(encoding)
|
56
|
+
end
|
57
|
+
|
58
|
+
def decompress_supported_encoding
|
59
|
+
method = SupportedEncodings[encoding]
|
60
|
+
if respond_to?(method, true)
|
61
|
+
send(method)
|
62
|
+
else
|
63
|
+
raise NotImplementedError, "#{self.class.name} has not implemented a decompression method for #{encoding.inspect} encoding."
|
64
|
+
end
|
65
|
+
end
|
66
|
+
|
67
|
+
def none
|
68
|
+
body
|
69
|
+
end
|
70
|
+
|
71
|
+
def brotli
|
72
|
+
return nil unless defined?(::Brotli)
|
73
|
+
begin
|
74
|
+
::Brotli.inflate(body)
|
75
|
+
rescue StandardError
|
76
|
+
nil
|
77
|
+
end
|
78
|
+
end
|
79
|
+
|
80
|
+
def lzw
|
81
|
+
begin
|
82
|
+
if defined?(::LZWS::String)
|
83
|
+
::LZWS::String.decompress(body)
|
84
|
+
elsif defined?(::LZW::Simple)
|
85
|
+
::LZW::Simple.new.decompress(body)
|
86
|
+
end
|
87
|
+
rescue StandardError
|
88
|
+
nil
|
89
|
+
end
|
90
|
+
end
|
91
|
+
end
|
92
|
+
end
|
data/lib/httparty/exceptions.rb
CHANGED
@@ -1,5 +1,7 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
|
-
# @
|
4
|
+
# @abstract Exceptions raised by HTTParty inherit from Error
|
3
5
|
class Error < StandardError; end
|
4
6
|
|
5
7
|
# Exception raised when you attempt to set a non-existent format
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'erb'
|
2
4
|
|
3
5
|
module HTTParty
|
@@ -26,8 +28,8 @@ module HTTParty
|
|
26
28
|
def self.normalize_param(key, value)
|
27
29
|
normalized_keys = normalize_keys(key, value)
|
28
30
|
|
29
|
-
normalized_keys.flatten.each_slice(2).inject('') do |string, (k, v)|
|
30
|
-
string
|
31
|
+
normalized_keys.flatten.each_slice(2).inject(+'') do |string, (k, v)|
|
32
|
+
string << "#{ERB::Util.url_encode(k)}=#{ERB::Util.url_encode(v.to_s)}&"
|
31
33
|
end
|
32
34
|
end
|
33
35
|
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
module Logger
|
3
5
|
class ApacheFormatter #:nodoc:
|
@@ -26,11 +28,11 @@ module HTTParty
|
|
26
28
|
end
|
27
29
|
|
28
30
|
def current_time
|
29
|
-
Time.now.strftime(
|
31
|
+
Time.now.strftime('%Y-%m-%d %H:%M:%S %z')
|
30
32
|
end
|
31
33
|
|
32
34
|
def http_method
|
33
|
-
request.http_method.name.split(
|
35
|
+
request.http_method.name.split('::').last.upcase
|
34
36
|
end
|
35
37
|
|
36
38
|
def path
|
@@ -1,9 +1,11 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
module Logger
|
3
5
|
class CurlFormatter #:nodoc:
|
4
6
|
TAG_NAME = HTTParty.name
|
5
|
-
OUT = '>'
|
6
|
-
IN = '<'
|
7
|
+
OUT = '>'
|
8
|
+
IN = '<'
|
7
9
|
|
8
10
|
attr_accessor :level, :logger
|
9
11
|
|
@@ -20,7 +22,7 @@ module HTTParty
|
|
20
22
|
log_request
|
21
23
|
log_response
|
22
24
|
|
23
|
-
logger.public_send level, messages.join(
|
25
|
+
logger.public_send level, messages.join('\n')
|
24
26
|
end
|
25
27
|
|
26
28
|
private
|
@@ -44,7 +46,7 @@ module HTTParty
|
|
44
46
|
end
|
45
47
|
|
46
48
|
def log_url
|
47
|
-
http_method = request.http_method.name.split(
|
49
|
+
http_method = request.http_method.name.split('::').last.upcase
|
48
50
|
uri = if request.options[:base_uri]
|
49
51
|
request.options[:base_uri] + request.path.path
|
50
52
|
else
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
module Logger
|
3
5
|
class LogstashFormatter #:nodoc:
|
@@ -40,11 +42,11 @@ module HTTParty
|
|
40
42
|
end
|
41
43
|
|
42
44
|
def current_time
|
43
|
-
Time.now.strftime(
|
45
|
+
Time.now.strftime('%Y-%m-%d %H:%M:%S %z')
|
44
46
|
end
|
45
47
|
|
46
48
|
def http_method
|
47
|
-
@http_method ||= request.http_method.name.split(
|
49
|
+
@http_method ||= request.http_method.name.split('::').last.upcase
|
48
50
|
end
|
49
51
|
|
50
52
|
def path
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
module ModuleInheritableAttributes #:nodoc:
|
3
5
|
def self.included(base)
|
@@ -36,7 +38,7 @@ module HTTParty
|
|
36
38
|
def inherited(subclass)
|
37
39
|
super
|
38
40
|
@mattr_inheritable_attrs.each do |inheritable_attribute|
|
39
|
-
ivar = "@#{inheritable_attribute}"
|
41
|
+
ivar = :"@#{inheritable_attribute}"
|
40
42
|
subclass.instance_variable_set(ivar, instance_variable_get(ivar).clone)
|
41
43
|
|
42
44
|
if instance_variable_get(ivar).respond_to?(:merge)
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'digest/md5'
|
2
4
|
require 'net/http'
|
3
5
|
|
@@ -44,12 +46,9 @@ module Net
|
|
44
46
|
header << %(algorithm="#{@response['algorithm']}") if algorithm_present?
|
45
47
|
|
46
48
|
if qop_present?
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
"nc=00000001"
|
51
|
-
]
|
52
|
-
fields.each { |field| header << field }
|
49
|
+
header << %(cnonce="#{@cnonce}")
|
50
|
+
header << %(qop="#{@response['qop']}")
|
51
|
+
header << 'nc=00000001'
|
53
52
|
end
|
54
53
|
|
55
54
|
header << %(opaque="#{@response['opaque']}") if opaque_present?
|
@@ -98,13 +97,13 @@ module Net
|
|
98
97
|
end
|
99
98
|
|
100
99
|
def random
|
101
|
-
format
|
100
|
+
format '%x', (Time.now.to_i + rand(65535))
|
102
101
|
end
|
103
102
|
|
104
103
|
def request_digest
|
105
104
|
a = [md5(a1), @response['nonce'], md5(a2)]
|
106
|
-
a.insert(2,
|
107
|
-
md5(a.join(
|
105
|
+
a.insert(2, '00000001', @cnonce, @response['qop']) if qop_present?
|
106
|
+
md5(a.join(':'))
|
108
107
|
end
|
109
108
|
|
110
109
|
def md5(str)
|
@@ -129,7 +128,7 @@ module Net
|
|
129
128
|
end
|
130
129
|
|
131
130
|
def a2
|
132
|
-
[@method, @path].join(
|
131
|
+
[@method, @path].join(':')
|
133
132
|
end
|
134
133
|
end
|
135
134
|
end
|
data/lib/httparty/parser.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
# The default parser used by HTTParty, supports xml, json, html, csv and
|
3
5
|
# plain text.
|
@@ -101,7 +103,7 @@ module HTTParty
|
|
101
103
|
# @return [nil] when the response body is nil, an empty string, spaces only or "null"
|
102
104
|
def parse
|
103
105
|
return nil if body.nil?
|
104
|
-
return nil if body ==
|
106
|
+
return nil if body == 'null'
|
105
107
|
return nil if body.valid_encoding? && body.strip.empty?
|
106
108
|
if body.valid_encoding? && body.encoding == Encoding::UTF_8
|
107
109
|
@body = body.gsub(/\A#{UTF8_BOM}/, '')
|
@@ -119,7 +121,7 @@ module HTTParty
|
|
119
121
|
MultiXml.parse(body)
|
120
122
|
end
|
121
123
|
|
122
|
-
UTF8_BOM = "\xEF\xBB\xBF"
|
124
|
+
UTF8_BOM = "\xEF\xBB\xBF"
|
123
125
|
|
124
126
|
def json
|
125
127
|
JSON.parse(body, :quirks_mode => true, :allow_nan => true)
|
@@ -142,9 +144,11 @@ module HTTParty
|
|
142
144
|
end
|
143
145
|
|
144
146
|
def parse_supported_format
|
145
|
-
|
146
|
-
|
147
|
-
|
147
|
+
if respond_to?(format, true)
|
148
|
+
send(format)
|
149
|
+
else
|
150
|
+
raise NotImplementedError, "#{self.class.name} has not implemented a parsing method for the #{format.inspect} format."
|
151
|
+
end
|
148
152
|
end
|
149
153
|
end
|
150
154
|
end
|
@@ -1,8 +1,13 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require_relative 'multipart_boundary'
|
2
4
|
|
3
5
|
module HTTParty
|
4
6
|
class Request
|
5
7
|
class Body
|
8
|
+
NEWLINE = "\r\n"
|
9
|
+
private_constant :NEWLINE
|
10
|
+
|
6
11
|
def initialize(params, query_string_normalizer: nil, force_multipart: false)
|
7
12
|
@params = params
|
8
13
|
@query_string_normalizer = query_string_normalizer
|
@@ -30,20 +35,20 @@ module HTTParty
|
|
30
35
|
def generate_multipart
|
31
36
|
normalized_params = params.flat_map { |key, value| HashConversions.normalize_keys(key, value) }
|
32
37
|
|
33
|
-
multipart = normalized_params.inject('') do |memo, (key, value)|
|
34
|
-
memo
|
35
|
-
memo
|
38
|
+
multipart = normalized_params.inject(+'') do |memo, (key, value)|
|
39
|
+
memo << "--#{boundary}#{NEWLINE}"
|
40
|
+
memo << %(Content-Disposition: form-data; name="#{key}")
|
36
41
|
# value.path is used to support ActionDispatch::Http::UploadedFile
|
37
42
|
# https://github.com/jnunemaker/httparty/pull/585
|
38
|
-
memo
|
39
|
-
memo
|
40
|
-
memo
|
41
|
-
memo
|
42
|
-
memo
|
43
|
-
memo
|
43
|
+
memo << %(; filename="#{file_name(value)}") if file?(value)
|
44
|
+
memo << NEWLINE
|
45
|
+
memo << "Content-Type: #{content_type(value)}#{NEWLINE}" if file?(value)
|
46
|
+
memo << NEWLINE
|
47
|
+
memo << content_body(value)
|
48
|
+
memo << NEWLINE
|
44
49
|
end
|
45
50
|
|
46
|
-
multipart
|
51
|
+
multipart << "--#{boundary}--#{NEWLINE}"
|
47
52
|
end
|
48
53
|
|
49
54
|
def has_file?(value)
|
@@ -68,6 +73,15 @@ module HTTParty
|
|
68
73
|
end
|
69
74
|
end
|
70
75
|
|
76
|
+
def content_body(object)
|
77
|
+
if file?(object)
|
78
|
+
object = (file = object).read
|
79
|
+
file.rewind if file.respond_to?(:rewind)
|
80
|
+
end
|
81
|
+
|
82
|
+
object.to_s
|
83
|
+
end
|
84
|
+
|
71
85
|
def content_type(object)
|
72
86
|
return object.content_type if object.respond_to?(:content_type)
|
73
87
|
mime = MIME::Types.type_for(object.path)
|
data/lib/httparty/request.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'erb'
|
2
4
|
|
3
5
|
module HTTParty
|
@@ -87,11 +89,11 @@ module HTTParty
|
|
87
89
|
end
|
88
90
|
|
89
91
|
def uri
|
90
|
-
if redirect && path.relative? && path.path[0] !=
|
91
|
-
last_uri_host = @last_uri.path.gsub(/[^\/]+$/,
|
92
|
+
if redirect && path.relative? && path.path[0] != '/'
|
93
|
+
last_uri_host = @last_uri.path.gsub(/[^\/]+$/, '')
|
92
94
|
|
93
|
-
path.path = "/#{path.path}" if last_uri_host[-1] !=
|
94
|
-
path.path = last_uri_host
|
95
|
+
path.path = "/#{path.path}" if last_uri_host[-1] != '/'
|
96
|
+
path.path = "#{last_uri_host}#{path.path}"
|
95
97
|
end
|
96
98
|
|
97
99
|
if path.relative? && path.host
|
@@ -117,7 +119,7 @@ module HTTParty
|
|
117
119
|
def base_uri
|
118
120
|
if redirect
|
119
121
|
base_uri = "#{@last_uri.scheme}://#{@last_uri.host}"
|
120
|
-
base_uri
|
122
|
+
base_uri = "#{base_uri}:#{@last_uri.port}" if @last_uri.port != 80
|
121
123
|
base_uri
|
122
124
|
else
|
123
125
|
options[:base_uri] && HTTParty.normalize_base_uri(options[:base_uri])
|
@@ -227,6 +229,8 @@ module HTTParty
|
|
227
229
|
@raw_request.body = body.call
|
228
230
|
end
|
229
231
|
|
232
|
+
@raw_request.instance_variable_set(:@decode_content, decompress_content?)
|
233
|
+
|
230
234
|
if options[:basic_auth] && send_authorization_header?
|
231
235
|
@raw_request.basic_auth(username, password)
|
232
236
|
@credentials_sent = true
|
@@ -238,6 +242,10 @@ module HTTParty
|
|
238
242
|
!!options[:digest_auth]
|
239
243
|
end
|
240
244
|
|
245
|
+
def decompress_content?
|
246
|
+
!options[:skip_decompression]
|
247
|
+
end
|
248
|
+
|
241
249
|
def response_unauthorized?
|
242
250
|
!!last_response && last_response.code == '401'
|
243
251
|
end
|
@@ -261,7 +269,7 @@ module HTTParty
|
|
261
269
|
query_string_parts << options[:query] unless options[:query].nil?
|
262
270
|
end
|
263
271
|
|
264
|
-
query_string_parts.reject!(&:empty?) unless query_string_parts == [
|
272
|
+
query_string_parts.reject!(&:empty?) unless query_string_parts == ['']
|
265
273
|
query_string_parts.size > 0 ? query_string_parts.join('&') : nil
|
266
274
|
end
|
267
275
|
|
@@ -269,7 +277,7 @@ module HTTParty
|
|
269
277
|
options[:assume_utf16_is_big_endian]
|
270
278
|
end
|
271
279
|
|
272
|
-
def handle_response(
|
280
|
+
def handle_response(raw_body, &block)
|
273
281
|
if response_redirects?
|
274
282
|
options[:limit] -= 1
|
275
283
|
if options[:logger]
|
@@ -290,9 +298,20 @@ module HTTParty
|
|
290
298
|
capture_cookies(last_response)
|
291
299
|
perform(&block)
|
292
300
|
else
|
293
|
-
|
294
|
-
|
295
|
-
|
301
|
+
raw_body ||= last_response.body
|
302
|
+
|
303
|
+
body = decompress(raw_body, last_response['content-encoding']) unless raw_body.nil?
|
304
|
+
|
305
|
+
unless body.nil?
|
306
|
+
body = encode_text(body, last_response['content-type'])
|
307
|
+
|
308
|
+
if decompress_content?
|
309
|
+
last_response.delete('content-encoding')
|
310
|
+
raw_body = body
|
311
|
+
end
|
312
|
+
end
|
313
|
+
|
314
|
+
Response.new(self, last_response, lambda { parse_response(body) }, body: raw_body)
|
296
315
|
end
|
297
316
|
end
|
298
317
|
|
@@ -368,6 +387,10 @@ module HTTParty
|
|
368
387
|
end
|
369
388
|
end
|
370
389
|
|
390
|
+
def decompress(body, encoding)
|
391
|
+
Decompressor.new(body, encoding).decompress
|
392
|
+
end
|
393
|
+
|
371
394
|
def encode_text(text, content_type)
|
372
395
|
TextEncoder.new(
|
373
396
|
text,
|
data/lib/httparty/response.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
class Response < Object
|
3
5
|
def self.underscore(string)
|
@@ -49,14 +51,14 @@ module HTTParty
|
|
49
51
|
end
|
50
52
|
|
51
53
|
def inspect
|
52
|
-
inspect_id = ::Kernel::format
|
54
|
+
inspect_id = ::Kernel::format '%x', (object_id * 2)
|
53
55
|
%(#<#{self.class}:0x#{inspect_id} parsed_response=#{parsed_response.inspect}, @response=#{response.inspect}, @headers=#{headers.inspect}>)
|
54
56
|
end
|
55
57
|
|
56
58
|
CODES_TO_OBJ = ::Net::HTTPResponse::CODE_CLASS_TO_OBJ.merge ::Net::HTTPResponse::CODE_TO_OBJ
|
57
59
|
|
58
60
|
CODES_TO_OBJ.each do |response_code, klass|
|
59
|
-
name = klass.name.sub(
|
61
|
+
name = klass.name.sub('Net::HTTP', '')
|
60
62
|
name = "#{underscore(name)}?".to_sym
|
61
63
|
|
62
64
|
define_method(name) do
|
@@ -65,12 +67,12 @@ module HTTParty
|
|
65
67
|
end
|
66
68
|
|
67
69
|
# Support old multiple_choice? method from pre 2.0.0 era.
|
68
|
-
if ::RUBY_VERSION >=
|
70
|
+
if ::RUBY_VERSION >= '2.0.0' && ::RUBY_PLATFORM != 'java'
|
69
71
|
alias_method :multiple_choice?, :multiple_choices?
|
70
72
|
end
|
71
73
|
|
72
74
|
# Support old status codes method from pre 2.6.0 era.
|
73
|
-
if ::RUBY_VERSION >=
|
75
|
+
if ::RUBY_VERSION >= '2.6.0' && ::RUBY_PLATFORM != 'java'
|
74
76
|
alias_method :gateway_time_out?, :gateway_timeout?
|
75
77
|
alias_method :request_entity_too_large?, :payload_too_large?
|
76
78
|
alias_method :request_time_out?, :request_timeout?
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module HTTParty
|
2
4
|
class TextEncoder
|
3
5
|
attr_reader :text, :content_type, :assume_utf16_is_big_endian
|
@@ -33,16 +35,16 @@ module HTTParty
|
|
33
35
|
def encode_utf_16
|
34
36
|
if text.bytesize >= 2
|
35
37
|
if text.getbyte(0) == 0xFF && text.getbyte(1) == 0xFE
|
36
|
-
return text.force_encoding(
|
38
|
+
return text.force_encoding('UTF-16LE')
|
37
39
|
elsif text.getbyte(0) == 0xFE && text.getbyte(1) == 0xFF
|
38
|
-
return text.force_encoding(
|
40
|
+
return text.force_encoding('UTF-16BE')
|
39
41
|
end
|
40
42
|
end
|
41
43
|
|
42
44
|
if assume_utf16_is_big_endian # option
|
43
|
-
text.force_encoding(
|
45
|
+
text.force_encoding('UTF-16BE')
|
44
46
|
else
|
45
|
-
text.force_encoding(
|
47
|
+
text.force_encoding('UTF-16LE')
|
46
48
|
end
|
47
49
|
end
|
48
50
|
|
data/lib/httparty/utils.rb
CHANGED
data/lib/httparty/version.rb
CHANGED
data/lib/httparty.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'pathname'
|
2
4
|
require 'net/http'
|
3
5
|
require 'net/https'
|
@@ -16,6 +18,7 @@ require 'httparty/connection_adapter'
|
|
16
18
|
require 'httparty/logger/logger'
|
17
19
|
require 'httparty/request/body'
|
18
20
|
require 'httparty/response_fragment'
|
21
|
+
require 'httparty/decompressor'
|
19
22
|
require 'httparty/text_encoder'
|
20
23
|
require 'httparty/headers_processor'
|
21
24
|
|
@@ -26,8 +29,8 @@ module HTTParty
|
|
26
29
|
base.send :include, ModuleInheritableAttributes
|
27
30
|
base.send(:mattr_inheritable, :default_options)
|
28
31
|
base.send(:mattr_inheritable, :default_cookies)
|
29
|
-
base.instance_variable_set(
|
30
|
-
base.instance_variable_set(
|
32
|
+
base.instance_variable_set(:@default_options, {})
|
33
|
+
base.instance_variable_set(:@default_cookies, CookieHash.new)
|
31
34
|
end
|
32
35
|
|
33
36
|
# == Common Request Options
|
@@ -399,6 +402,22 @@ module HTTParty
|
|
399
402
|
default_options[:ssl_version] = version
|
400
403
|
end
|
401
404
|
|
405
|
+
# Deactivate automatic decompression of the response body.
|
406
|
+
# This will require you to explicitly handle body decompression
|
407
|
+
# by inspecting the Content-Encoding response header.
|
408
|
+
#
|
409
|
+
# Refer to docs/README.md "HTTP Compression" section for
|
410
|
+
# further details.
|
411
|
+
#
|
412
|
+
# @example
|
413
|
+
# class Foo
|
414
|
+
# include HTTParty
|
415
|
+
# skip_decompression
|
416
|
+
# end
|
417
|
+
def skip_decompression(value = true)
|
418
|
+
default_options[:skip_decompression] = !!value
|
419
|
+
end
|
420
|
+
|
402
421
|
# Allows setting of SSL ciphers to use. This only works in Ruby 1.9+.
|
403
422
|
# You can get a list of valid specific ciphers from OpenSSL::Cipher.ciphers.
|
404
423
|
# You also can specify a cipher suite here, listed here at openssl.org:
|
@@ -597,7 +616,7 @@ module HTTParty
|
|
597
616
|
def process_cookies(options) #:nodoc:
|
598
617
|
return unless options[:cookies] || default_cookies.any?
|
599
618
|
options[:headers] ||= headers.dup
|
600
|
-
options[:headers][
|
619
|
+
options[:headers]['cookie'] = cookies.merge(options.delete(:cookies) || {}).to_cookie_string
|
601
620
|
end
|
602
621
|
|
603
622
|
def validate_format
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: httparty
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.19.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- John Nunemaker
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2021-09-09 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: multi_xml
|
@@ -48,11 +48,11 @@ extensions: []
|
|
48
48
|
extra_rdoc_files: []
|
49
49
|
files:
|
50
50
|
- ".editorconfig"
|
51
|
+
- ".github/workflows/ci.yml"
|
51
52
|
- ".gitignore"
|
52
53
|
- ".rubocop.yml"
|
53
54
|
- ".rubocop_todo.yml"
|
54
55
|
- ".simplecov"
|
55
|
-
- ".travis.yml"
|
56
56
|
- CONTRIBUTING.md
|
57
57
|
- Changelog.md
|
58
58
|
- Gemfile
|
@@ -72,6 +72,7 @@ files:
|
|
72
72
|
- examples/delicious.rb
|
73
73
|
- examples/google.rb
|
74
74
|
- examples/headers_and_user_agents.rb
|
75
|
+
- examples/idn.rb
|
75
76
|
- examples/logging.rb
|
76
77
|
- examples/microsoft_graph.rb
|
77
78
|
- examples/multipart.rb
|
@@ -88,6 +89,7 @@ files:
|
|
88
89
|
- lib/httparty.rb
|
89
90
|
- lib/httparty/connection_adapter.rb
|
90
91
|
- lib/httparty/cookie_hash.rb
|
92
|
+
- lib/httparty/decompressor.rb
|
91
93
|
- lib/httparty/exceptions.rb
|
92
94
|
- lib/httparty/hash_conversions.rb
|
93
95
|
- lib/httparty/headers_processor.rb
|
@@ -129,7 +131,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
129
131
|
- !ruby/object:Gem::Version
|
130
132
|
version: '0'
|
131
133
|
requirements: []
|
132
|
-
rubygems_version: 3.
|
134
|
+
rubygems_version: 3.0.3
|
133
135
|
signing_key:
|
134
136
|
specification_version: 4
|
135
137
|
summary: Makes http fun! Also, makes consuming restful web services dead easy.
|