httparty 0.13.7 → 0.21.0

data/README.md

@@ -1,6 +1,8 @@
 # httparty
 
-Makes http fun again!
+[![CI](https://github.com/jnunemaker/httparty/actions/workflows/ci.yml/badge.svg)](https://github.com/jnunemaker/httparty/actions/workflows/ci.yml)
+
+Makes http fun again!  Ain't no party like a httparty, because a httparty don't stop.
 
 ## Install
 
@@ -10,7 +12,7 @@ gem install httparty
 
 ## Requirements
 
-* Ruby 1.9.3 or higher
+* Ruby 2.3.0 or higher
 * multi_xml
 * You like to party!
 
@@ -28,7 +30,7 @@ class StackExchange
   base_uri 'api.stackexchange.com'
 
   def initialize(service, page)
-    @options = { query: {site: service, page: page} }
+    @options = { query: { site: service, page: page } }
   end
 
   def questions
@@ -46,7 +48,6 @@ puts stack_exchange.users
 ```
 
 See the [examples directory](http://github.com/jnunemaker/httparty/tree/master/examples) for even more goodies.
-
 ## Command Line Interface
 
 httparty also includes the executable `httparty` which can be
@@ -62,9 +63,9 @@ httparty "https://api.stackexchange.com/2.2/questions?site=stackoverflow"
 
 ## Help and Docs
 
-* https://groups.google.com/forum/#!forum/httparty-gem
-* http://rdoc.info/projects/jnunemaker/httparty
-* http://stackoverflow.com/questions/tagged/httparty
+* [Docs](https://github.com/jnunemaker/httparty/tree/master/docs)
+* https://github.com/jnunemaker/httparty/discussions
+* https://www.rubydoc.info/github/jnunemaker/httparty
 
 ## Contributing
 

data/bin/httparty

@@ -103,9 +103,11 @@ else
     when :json
       begin
         require 'json'
-        puts JSON.pretty_generate(response)
+        puts JSON.pretty_generate(response.parsed_response)
       rescue LoadError
         puts YAML.dump(response)
+      rescue JSON::JSONError
+        puts response.inspect
       end
     when :xml
       require 'rexml/document'

data/docs/README.md

@@ -0,0 +1,191 @@
+# httparty
+
+Makes http fun again!
+
+## Table of contents
+- [Parsing JSON](#parsing-json)
+- [Working with SSL](#working-with-ssl)
+
+## Parsing JSON
+If the response Content Type is `application/json`, HTTParty will parse the response and return Ruby objects such as a hash or array. The default behavior for parsing JSON will return keys as strings. This can be supressed with the `format` option. To get hash keys as symbols:
+
+```ruby
+response = HTTParty.get('http://example.com', format: :plain)
+JSON.parse response, symbolize_names: true
+```
+
+## Posting JSON
+When using Content Type `application/json` with `POST`, `PUT` or `PATCH` requests, the body should be a string of valid JSON:
+
+```ruby
+# With written JSON
+HTTParty.post('http://example.com', body: "{\"foo\":\"bar\"}", headers: { 'Content-Type' => 'application/json' })
+
+# Using JSON.generate
+HTTParty.post('http://example.com', body: JSON.generate({ foo: 'bar' }), headers: { 'Content-Type' => 'application/json' })
+
+# Using object.to_json
+HTTParty.post('http://example.com', body: { foo: 'bar' }.to_json, headers: { 'Content-Type' => 'application/json' })
+```
+
+## Working with SSL
+
+You can use this guide to work with SSL certificates.
+
+#### Using `pem` option
+
+```ruby
+# Use this example if you are using a pem file
+
+class Client
+  include HTTParty
+
+  base_uri "https://example.com"
+  pem File.read("#{File.expand_path('.')}/path/to/certs/cert.pem"), "123456"
+end
+```
+
+#### Using `pkcs12` option
+
+```ruby
+# Use this example if you are using a pkcs12 file
+
+class Client
+  include HTTParty
+
+  base_uri "https://example.com"
+  pkcs12 File.read("#{File.expand_path('.')}/path/to/certs/cert.p12"), "123456"
+end
+```
+
+#### Using `ssl_ca_file` option
+
+```ruby
+# Use this example if you are using a pkcs12 file
+
+class Client
+  include HTTParty
+
+  base_uri "https://example.com"
+  ssl_ca_file "#{File.expand_path('.')}/path/to/certs/cert.pem"
+end
+```
+
+#### Using `ssl_ca_path` option
+
+```ruby
+# Use this example if you are using a pkcs12 file
+
+class Client
+  include HTTParty
+
+  base_uri "https://example.com"
+  ssl_ca_path '/path/to/certs'
+end
+```
+
+You can also include all of these options with the call:
+
+```ruby
+class Client
+  include HTTParty
+
+  base_uri "https://example.com"
+
+  def self.fetch
+    get("/resources", pem: File.read("#{File.expand_path('.')}/path/to/certs/cert.pem"), pem_password: "123456")
+  end
+end
+```
+
+### Avoid SSL verification
+
+In some cases you may want to skip SSL verification, because the entity that issued the certificate is not a valid one, but you still want to work with it. You can achieve this through:
+
+```ruby
+# Skips SSL certificate verification
+
+class Client
+  include HTTParty
+
+  base_uri "https://example.com"
+  pem File.read("#{File.expand_path('.')}/path/to/certs/cert.pem"), "123456"
+
+  def self.fetch
+    get("/resources", verify: false)
+    # You can also use something like:
+    # get("resources", verify_peer: false)
+  end
+end
+```
+
+### HTTP Compression
+
+The `Accept-Encoding` request header and `Content-Encoding` response header
+are used to control compression (gzip, etc.) over the wire. Refer to
+[RFC-2616](https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html) for details.
+(For clarity: these headers are **not** used for character encoding i.e. `utf-8`
+which is specified in the `Accept` and `Content-Type` headers.)
+
+Unless you have specific requirements otherwise, we recommend to **not** set
+set the `Accept-Encoding` header on HTTParty requests. In this case, `Net::HTTP`
+will set a sensible default compression scheme and automatically decompress the response.
+
+If you explicitly set `Accept-Encoding`, there be dragons:
+
+* If the HTTP response `Content-Encoding` received on the wire is `gzip` or `deflate`,
+  `Net::HTTP` will automatically decompress it, and will omit `Content-Encoding`
+  from your `HTTParty::Response` headers.
+
+* For the following encodings, HTTParty will automatically decompress them if you include
+  the required gem into your project. Similar to above, if decompression succeeds,
+  `Content-Encoding` will be omitted from your `HTTParty::Response` headers.
+  **Warning:** Support for these encodings is experimental and not fully battle-tested.
+
+  | Content-Encoding | Required Gem |
+  | --- | --- |
+  | `br` (Brotli)      | [brotli](https://rubygems.org/gems/brotli) |
+  | `compress` (LZW)   | [ruby-lzws](https://rubygems.org/gems/ruby-lzws) |
+  | `zstd` (Zstandard) | [zstd-ruby](https://rubygems.org/gems/zstd-ruby) |
+
+* For other encodings, `HTTParty::Response#body` will return the raw uncompressed byte string,
+  and you'll need to inspect the `Content-Encoding` response header and decompress it yourself.
+  In this case, `HTTParty::Response#parsed_response` will be `nil`.
+
+* Lastly, you may use the `skip_decompression` option to disable all automatic decompression
+  and always get `HTTParty::Response#body` in its raw form along with the `Content-Encoding` header.
+
+```ruby
+# Accept-Encoding=gzip,deflate can be safely assumed to be auto-decompressed
+
+res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => 'gzip,deflate,identity' })
+JSON.parse(res.body) # safe
+
+
+# Accept-Encoding=br,compress requires third-party gems
+
+require 'brotli'
+require 'lzws'
+require 'zstd-ruby'
+res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => 'br,compress,zstd' })
+JSON.parse(res.body)
+
+
+# Accept-Encoding=* may return unhandled Content-Encoding
+
+res = HTTParty.get('https://example.com/test.json', headers: { 'Accept-Encoding' => '*' })
+encoding = res.headers['Content-Encoding']
+if encoding
+JSON.parse(your_decompression_handling(res.body, encoding))
+else
+# Content-Encoding not present implies decompressed
+JSON.parse(res.body)
+end
+
+
+# Gimme the raw data!
+
+res = HTTParty.get('https://example.com/test.json', skip_decompression: true)
+encoding = res.headers['Content-Encoding']
+JSON.parse(your_decompression_handling(res.body, encoding))
+```

data/examples/README.md

@@ -13,22 +13,22 @@
     * Creates a custom parser for XML using crack gem
     * Uses `get` request
 
-* [Create HTML Nokogiri parser](nokogiri_html_parser.rb)   
+* [Create HTML Nokogiri parser](nokogiri_html_parser.rb)
     * Adds Html as a format
     * passed the body of request to Nokogiri
-    
+
 * [More Custom Parsers](custom_parsers.rb)
   * Create an additional parser for atom or make it the ONLY parser
-  
+
 * [Basic Auth, Delicious](delicious.rb)
   * Basic Auth, shows how to merge those into options
   * Uses `get` requests
-  
+
 * [Passing Headers, User Agent](headers_and_user_agents.rb)
   * Use the class method of Httparty
   * Pass the User-Agent in the headers
   * Uses `get` requests
-  
+
 * [Basic Post Request](basic.rb)
     * Httparty included into poro class
     * Uses `post` requests
@@ -36,7 +36,7 @@
 * [Access Rubyurl Shortener](rubyurl.rb)
   * Httparty included into poro class
   * Uses `post` requests
-  
+
 * [Add a custom log file](logging.rb)
   * create a log file and have httparty log requests
 
@@ -44,24 +44,46 @@
   * Httparty included into poro class
   * Creates methods for different endpoints
   * Uses `get` requests
-  
+
 * [Accessing Tripit](tripit_sign_in.rb)
   * Httparty included into poro class
   * Example of using `debug_output` to see headers/urls passed
   * Getting and using Cookies
   * Uses `get` requests
-  
+
 * [Accessing Twitter](twitter.rb)
   * Httparty included into poro class
   * Basic Auth
-  * Loads settings from a config file 
+  * Loads settings from a config file
   * Uses `get` requests
   * Uses `post` requests
-  
+
 * [Accessing WhoIsMyRep](whoismyrep.rb)
   * Httparty included into poro class
-  * Uses `get` requests     
+  * Uses `get` requests
   * Two ways to pass params to get, inline on the url or in query hash
 
 * [Rescue Json Error](rescue_json.rb)
-  * Rescue errors due to parsing response
+  * Rescue errors due to parsing response
+
+* [Download file using stream mode](stream_download.rb)
+  * Uses `get` requests
+  * Uses `stream_body` mode
+  * Download file without using the memory
+
+* [Microsoft graph](microsoft_graph.rb)
+  * Basic Auth
+  * Uses `post` requests
+  * Uses multipart
+
+* [Multipart](multipart.rb)
+  * Multipart data upload _(with and without file)_
+
+* [Uploading File](body_stream.rb)
+  * Uses `body_stream` to upload file
+
+* [Accessing x509 Peer Certificate](peer_cert.rb)
+  * Provides access to the server's TLS certificate
+
+* [Accessing IDNs](idn.rb)
+  * Uses a `get` request with an International domain names, which are Urls with emojis and non-ASCII characters such as accented letters.

data/examples/aaws.rb

@@ -1,5 +1,7 @@
 require 'rubygems'
 require 'active_support'
+require 'active_support/core_ext/hash'
+require 'active_support/core_ext/string'
 
 dir = File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib'))
 require File.join(dir, 'httparty')
@@ -13,14 +15,16 @@ module AAWS
     default_params Service: 'AWSECommerceService', Operation: 'ItemSearch', SearchIndex: 'Books'
 
     def initialize(key)
-      self.class.default_params AWSAccessKeyId: key
+      @auth = { AWSAccessKeyId: key }
     end
 
     def search(options = {})
       raise ArgumentError, 'You must search for something' if options[:query].blank?
 
       # amazon uses nasty camelized query params
-      options[:query] = options[:query].inject({}) { |h, q| h[q[0].to_s.camelize] = q[1]; h }
+      options[:query] = options[:query]
+        .reverse_merge(@auth)
+        .transform_keys { |k| k.to_s.camelize }
 
       # make a request and return the items (NOTE: this doesn't handle errors at this point)
       self.class.get('/onca/xml', options)['ItemSearchResponse']['Items']
@@ -29,4 +33,4 @@ module AAWS
 end
 
 aaws = AAWS::Book.new(config[:access_key])
-pp aaws.search(query: {title: 'Ruby On Rails'})
+pp aaws.search(query: { title: 'Ruby On Rails' })

data/examples/body_stream.rb

@@ -0,0 +1,14 @@
+# To upload file to a server use :body_stream
+
+HTTParty.put(
+  'http://localhost:3000/train',
+  body_stream: File.open('sample_configs/config_train_server_md.yml', 'r')
+)
+
+
+# Actually, it works with any IO object
+
+HTTParty.put(
+  'http://localhost:3000/train',
+  body_stream: StringIO.new('foo')
+)

data/examples/crack.rb

@@ -16,4 +16,4 @@ class Rep
 end
 
 pp Rep.get('http://whoismyrepresentative.com/getall_mems.php?zip=46544')
-pp Rep.get('http://whoismyrepresentative.com/getall_mems.php', query: {zip: 46544})
+pp Rep.get('http://whoismyrepresentative.com/getall_mems.php', query: { zip: 46544 })

data/examples/custom_parsers.rb

@@ -1,3 +1,7 @@
+dir = File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib'))
+require File.join(dir, 'httparty')
+require 'pp'
+
 class ParseAtom
   include HTTParty
 
@@ -21,7 +25,7 @@ class OnlyParseAtom
 
   # Only support Atom
   class Parser::OnlyAtom < HTTParty::Parser
-    SupportedFormats = {"application/atom+xml" => :atom}
+    SupportedFormats = { "application/atom+xml" => :atom }
 
     protected
 

data/examples/delicious.rb

@@ -8,7 +8,7 @@ class Delicious
   base_uri 'https://api.del.icio.us/v1'
 
   def initialize(u, p)
-    @auth = {username: u, password: p}
+    @auth = { username: u, password: p }
   end
 
   # query params that filter the posts are:
@@ -17,7 +17,7 @@ class Delicious
   #   url (optional). Filter by this url.
   #   ie: posts(query: {tag: 'ruby'})
   def posts(options = {})
-    options.merge!({basic_auth: @auth})
+    options.merge!({ basic_auth: @auth })
     self.class.get('/posts/get', options)
   end
 
@@ -25,13 +25,13 @@ class Delicious
   #   tag (optional). Filter by this tag.
   #   count (optional). Number of items to retrieve (Default:15, Maximum:100).
   def recent(options = {})
-    options.merge!({basic_auth: @auth})
+    options.merge!({ basic_auth: @auth })
     self.class.get('/posts/recent', options)
   end
 end
 
 delicious = Delicious.new(config['username'], config['password'])
-pp delicious.posts(query: {tag: 'ruby'})
+pp delicious.posts(query: { tag: 'ruby' })
 pp delicious.recent
 
 delicious.recent['posts']['post'].each { |post| puts post['href'] }

data/examples/headers_and_user_agents.rb

@@ -1,6 +1,10 @@
 # To send custom user agents to identify your application to a web service (or mask as a specific browser for testing), send "User-Agent" as a hash to headers as shown below.
 
-require 'httparty'
+dir = File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib'))
+require File.join(dir, 'httparty')
+require 'pp'
 
-APPLICATION_NAME = "Httparty"
-response = HTTParty.get('http://example.com', headers: {"User-Agent" => APPLICATION_NAME})
+response = HTTParty.get('http://example.com', {
+  headers: {"User-Agent" => "Httparty"},
+  debug_output: STDOUT, # To show that User-Agent is Httparty
+})

data/examples/idn.rb

@@ -0,0 +1,10 @@
+dir = File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib'))
+require File.join(dir, 'httparty')
+require 'pp'
+
+class Idn
+    include HTTParty
+    uri_adapter Addressable::URI
+end
+
+pp Idn.get("https://i❤️.ws/emojidomain/💎?format=json")

data/examples/logging.rb

@@ -3,7 +3,7 @@ require File.join(dir, 'httparty')
 require 'logger'
 require 'pp'
 
-my_logger = Logger.new "httparty.log"
+my_logger = Logger.new STDOUT
 
 my_logger.info "Logging can be used on the main HTTParty class. It logs redirects too."
 HTTParty.get "http://google.com", logger: my_logger
@@ -14,7 +14,7 @@ my_logger.info "It can be used also on a custom class."
 
 class Google
   include HTTParty
-  logger ::Logger.new "httparty.log"
+  logger ::Logger.new STDOUT
 end
 
 Google.get "http://google.com"
@@ -22,7 +22,7 @@ Google.get "http://google.com"
 my_logger.info '*' * 70
 
 my_logger.info "The default formatter is :apache. The :curl formatter can also be used."
-my_logger.info "You can tell wich method to call on the logger too. It is info by default."
+my_logger.info "You can tell which method to call on the logger too. It is info by default."
 HTTParty.get "http://google.com", logger: my_logger, log_level: :debug, log_format: :curl
 
 my_logger.info '*' * 70
@@ -30,7 +30,7 @@ my_logger.info '*' * 70
 my_logger.info "These configs are also available on custom classes."
 class Google
   include HTTParty
-  logger ::Logger.new("httparty.log"), :debug, :curl
+  logger ::Logger.new(STDOUT), :debug, :curl
 end
 
 Google.get "http://google.com"

data/examples/microsoft_graph.rb

@@ -0,0 +1,52 @@
+require 'httparty'
+
+class MicrosoftGraph
+  MS_BASE_URL = "https://login.microsoftonline.com".freeze
+  TOKEN_REQUEST_PATH = "oauth2/v2.0/token".freeze
+
+  def initialize(tenant_id)
+    @tenant_id = tenant_id
+  end
+
+  # Make a request to the Microsoft Graph API, for instance https://graph.microsoft.com/v1.0/users
+  def request(url)
+    return false unless (token = bearer_token)
+
+    response = HTTParty.get(
+      url,
+      headers: {
+        Authorization: "Bearer #{token}"
+      }
+    )
+
+    return false unless response.code == 200
+
+    return JSON.parse(response.body)
+  end
+
+  private
+
+  # A post to the Microsoft Graph to get a bearer token for the specified tenant.  In this example
+  # our Rails application has already been given permission to request these tokens by the admin of
+  # the specified tenant_id.
+  #
+  # See here for more information https://developer.microsoft.com/en-us/graph/docs/concepts/auth_v2_service
+  #
+  # This request also makes use of the multipart/form-data post body.
+  def bearer_token
+    response = HTTParty.post(
+      "#{MS_BASE_URL}/#{@tenant_id}/#{TOKEN_REQUEST_PATH}",
+      multipart: true,
+      body: {
+        client_id: Rails.application.credentials[Rails.env.to_sym][:microsoft_client_id],
+        client_secret: Rails.application.credentials[Rails.env.to_sym][:microsoft_client_secret],
+        scope: 'https://graph.microsoft.com/.default',
+        grant_type: 'client_credentials'
+      }
+    )
+
+    return false unless response.code == 200
+
+    JSON.parse(response.body)['access_token']
+  end
+end

data/examples/multipart.rb

@@ -0,0 +1,22 @@
+# If you are uploading file in params, multipart will used as content-type automatically
+
+HTTParty.post(
+  'http://localhost:3000/user',
+  body: {
+    name: 'Foo Bar',
+    email: 'example@email.com',
+    avatar: File.open('/full/path/to/avatar.jpg')
+  }
+)
+
+
+# However, you can force it yourself
+
+HTTParty.post(
+  'http://localhost:3000/user',
+  multipart: true,
+  body: {
+    name: 'Foo Bar',
+    email: 'example@email.com'
+  }
+)

data/examples/peer_cert.rb

@@ -0,0 +1,9 @@
+dir = File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib'))
+require File.join(dir, 'httparty')
+
+peer_cert = nil
+HTTParty.get("https://www.example.com") do |fragment|
+  peer_cert ||= fragment.connection.peer_cert
+end
+
+puts "The server's certificate expires #{peer_cert.not_after}"

data/examples/stackexchange.rb

@@ -7,7 +7,7 @@ class StackExchange
   base_uri 'api.stackexchange.com'
 
   def initialize(service, page)
-    @options = { query: {site: service, page: page} }
+    @options = { query: { site: service, page: page } }
   end
 
   def questions

data/examples/stream_download.rb

@@ -0,0 +1,26 @@
+dir = File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib'))
+require File.join(dir, 'httparty')
+require 'pp'
+
+# download file linux-4.6.4.tar.xz without using the memory
+response = nil
+filename = "linux-4.6.4.tar.xz"
+url = "https://cdn.kernel.org/pub/linux/kernel/v4.x/#{filename}"
+
+File.open(filename, "w") do |file|
+  response = HTTParty.get(url, stream_body: true) do |fragment|
+    if [301, 302].include?(fragment.code)
+      print "skip writing for redirect"
+    elsif fragment.code == 200
+      print "."
+      file.write(fragment)
+    else
+      raise StandardError, "Non-success status code while streaming #{fragment.code}"
+    end
+  end
+end
+puts
+
+pp "Success: #{response.success?}"
+pp File.stat(filename).inspect
+File.unlink(filename)