typhoeus 0.4.2 → 1.4.0

data/README.md

@@ -1,440 +1,569 @@
-#  Typhoeus [![Build Status](https://secure.travis-ci.org/typhoeus/typhoeus.png?branch=master)](http://travis-ci.org/typhoeus/typhoeus)
-
-[the mailing list](http://groups.google.com/group/typhoeus)
-
-##  Summary
-
-Like a modern code version of the mythical beast with 100 serpent heads,
-Typhoeus runs HTTP requests in parallel while cleanly encapsulating handling
-logic. To be a little more specific, it’s a library for accessing web services
-in Ruby. It’s specifically designed for building RESTful service oriented
-architectures in Ruby that need to be fast enough to process calls to multiple
-services within the client’s HTTP request/response life cycle.
-
-Some of the awesome features are parallel request execution, memoization of
-request responses (so you don’t make the same request multiple times in a
-single group), built in support for caching responses to memcached (or
-whatever), and mocking capability baked in. It uses libcurl and libcurl-multi
-to work this speedy magic. I wrote the bindings myself so it’s yet another
-Ruby libcurl library, but with some extra awesomeness added in. FFI is used to
-interface with the library so it works with any Ruby implementation.
-
-##  Installation
-
-Typhoeus requires you to have a current version of libcurl installed. The
-easiest solution is to use your system’s package manager to install it. If
-that doesn’t work, you can grab a package off of [the curl
-website](http://curl.haxx.se/download.html) and manually install it following
-the instructions given there. Typhoeus will work with version 7.19.4 or higher
-(earlier versions might work but no guarantees are provided).
-
-To install Typhoeus, simply run:
-
-    gem install typhoeus
-
-If you’re on Debian or Ubuntu and getting errors while trying to install, it
-could be because you don’t have the latest version of libcurl installed. Do
-this to fix:
-
-    sudo apt-get install libcurl4-gnutls-dev
-
-If you’re still having issues, please let me know on [the mailing
-list](http://groups.google.com/group/typhoeus).
-
-There’s one other thing you should know. The Easy object (which is just a
-libcurl thing) allows you to set timeout values in milliseconds. However, for
-this to work you need to build libcurl with c-ares support built in.
-
-##  Windows Support
-
-Typhoeus runs perfectly on Windows. The tricky part is knowing how to install
-libcurl in the absence of a package manager.
-
-To install libcurl, simply grab [the latest libcurl
-package](http://curl.haxx.se/download.html#Win32) off of the curl website,
-extract the bin directory, and then add the path to the bin directory into the
-PATH environment variable. Ruby with then be able to find libcurl properly and
-everything will just work.
-
-##  Usage
-
-The primary interface for Typhoeus is comprised of three classes: Request,
-Response, and Hydra. Request represents an HTTP request object, response
-represents an HTTP response, and Hydra manages making parallel HTTP
-connections.
-
-    require 'rubygems'
-    require 'typhoeus'
-    require 'json'
-
-    # the request object
-    request = Typhoeus::Request.new("http://www.pauldix.net",
-                                    :body          => "this is a request body",
-                                    :method        => :post,
-                                    :headers       => {:Accept => "text/html"},
-                                    :timeout       => 100, # milliseconds
-                                    :cache_timeout => 60, # seconds
-                                    :params        => {:field1 => "a field"})
-    # we can see from this that the first argument is the url. the second is a set of options.
-    # the options are all optional. The default for :method is :get. Timeout is measured in milliseconds.
-    # cache_timeout is measured in seconds.
-
-    # Run the request via Hydra.
-    hydra = Typhoeus::Hydra.new
-    hydra.queue(request)
-    hydra.run
-
-    # the response object will be set after the request is run
-    response = request.response
-    response.code    # http status code
-    response.time    # time in seconds the request took
-    response.headers # the http headers
-    response.headers_hash # http headers put into a hash
-    response.body    # the response body
-
-**Making Quick Requests**
-
-The request object has some convenience methods for performing single HTTP
-requests. The arguments are the same as those you pass into the request
-constructor.
-
-    response = Typhoeus::Request.get("http://www.pauldix.net")
-    response = Typhoeus::Request.head("http://www.pauldix.net")
-    response = Typhoeus::Request.put("http://localhost:3000/posts/1", :body => "whoo, a body")
-    response = Typhoeus::Request.post("http://localhost:3000/posts", :params => {:title => "test post", :content => "this is my test"})
-    response = Typhoeus::Request.delete("http://localhost:3000/posts/1")
-
-**Handling HTTP errors**
+# Typhoeus [![Build Status](https://img.shields.io/travis/typhoeus/typhoeus/master.svg)](https://travis-ci.org/typhoeus/typhoeus) [![Code Climate](https://img.shields.io/codeclimate/maintainability/typhoeus/typhoeus.svg)](https://codeclimate.com/github/typhoeus/typhoeus) [![Gem Version](https://img.shields.io/gem/v/typhoeus.svg)](https://rubygems.org/gems/typhoeus)
+
+Like a modern code version of the mythical beast with 100 serpent heads, Typhoeus runs HTTP requests in parallel while cleanly encapsulating handling logic.
+
+## Example
+
+A single request:
+
+```ruby
+Typhoeus.get("www.example.com", followlocation: true)
+```
+
+Parallel requests:
+
+```ruby
+hydra = Typhoeus::Hydra.new
+10.times.map{ hydra.queue(Typhoeus::Request.new("www.example.com", followlocation: true)) }
+hydra.run
+```
+
+## Installation
+Add the following line to your Gemfile:
+```
+gem "typhoeus"
+```
+Then run `bundle install`
+
+Or install it yourself as:
+
+```
+gem install typhoeus
+```
+
+## Project Tracking
+
+* [Documentation](http://rubydoc.info/github/typhoeus/typhoeus/frames/Typhoeus) (GitHub master)
+* [Mailing list](http://groups.google.com/group/typhoeus)
+
+## Usage
+
+### Introduction
+
+The primary interface for Typhoeus is comprised of three classes: Request, Response, and Hydra. Request represents an HTTP request object, response represents an HTTP response, and Hydra manages making parallel HTTP connections.
+
+```ruby
+request = Typhoeus::Request.new(
+  "www.example.com",
+  method: :post,
+  body: "this is a request body",
+  params: { field1: "a field" },
+  headers: { Accept: "text/html" }
+)
+```
+
+We can see from this that the first argument is the url. The second is a set of options.
+The options are all optional. The default for `:method` is `:get`.
+
+When you want to send URL parameters, you can use `:params` hash to do so. Please note that in case of you should send a request via `x-www-form-urlencoded` parameters, you need to use `:body` hash instead. `params` are for URL parameters and `:body` is for the request body.
+
+#### Sending requests through the proxy
+
+Add a proxy url to the list of options:
+
+```ruby
+options = {proxy: 'http://myproxy.org'}
+req = Typhoeus::Request.new(url, options)
+```
+
+If your proxy requires authentication, add it with `proxyuserpwd` option key:
+
+```ruby
+options = {proxy: 'http://proxyurl.com', proxyuserpwd: 'user:password'}
+req = Typhoeus::Request.new(url, options)
+```
+
+Note that `proxyuserpwd` is a colon-separated username and password, in the vein of basic auth `userpwd` option.
+
+
+You can run the query either on its own or through the hydra:
+
+``` ruby
+request.run
+#=> <Typhoeus::Response ... >
+```
+
+```ruby
+hydra = Typhoeus::Hydra.hydra
+hydra.queue(request)
+hydra.run
+```
+
+The response object will be set after the request is run.
+
+```ruby
+response = request.response
+response.code
+response.total_time
+response.headers
+response.body
+```
+
+### Making Quick Requests
+
+Typhoeus has some convenience methods for performing single HTTP requests. The arguments are the same as those you pass into the request constructor.
+
+```ruby
+Typhoeus.get("www.example.com")
+Typhoeus.head("www.example.com")
+Typhoeus.put("www.example.com/posts/1", body: "whoo, a body")
+Typhoeus.patch("www.example.com/posts/1", body: "a new body")
+Typhoeus.post("www.example.com/posts", body: { title: "test post", content: "this is my test"})
+Typhoeus.delete("www.example.com/posts/1")
+Typhoeus.options("www.example.com")
+```
+#### Sending params in the body with PUT
+When using POST the content-type is set automatically to 'application/x-www-form-urlencoded'. That's not the case for any other method like PUT, PATCH, HEAD and so on,  irrespective of whether you are using body or not. To get the same result as POST, i.e. a hash in the body coming through as params in the receiver, you need to set the content-type as shown below:
+```ruby
+Typhoeus.put("www.example.com/posts/1",
+        headers: {'Content-Type'=> "application/x-www-form-urlencoded"},
+        body: {title:"test post updated title", content: "this is my updated content"}
+    )
+```
+
+### Handling HTTP errors
 
 You can query the response object to figure out if you had a successful
 request or not. Here’s some example code that you might use to handle errors.
-
-    request.on_complete do |response|
-      if response.success?
-        # hell yeah
-      elsif response.timed_out?
-        # aw hell no
-        log("got a time out")
-      elsif response.code == 0
-        # Could not get an http response, something's wrong.
-        log(response.curl_error_message)
-      else
-        # Received a non-successful http response.
-        log("HTTP request failed: " + response.code.to_s)
-      end
-    end
+The callbacks are executed right after the request is finished, make sure to define
+them before running the request.
+
+```ruby
+request = Typhoeus::Request.new("www.example.com", followlocation: true)
+
+request.on_complete do |response|
+  if response.success?
+    # hell yeah
+  elsif response.timed_out?
+    # aw hell no
+    log("got a time out")
+  elsif response.code == 0
+    # Could not get an http response, something's wrong.
+    log(response.return_message)
+  else
+    # Received a non-successful http response.
+    log("HTTP request failed: " + response.code.to_s)
+  end
+end
+
+request.run
+```
 
 This also works with serial (blocking) requests in the same fashion. Both
 serial and parallel requests return a Response object.
 
-**Handling file uploads**
+### Handling file uploads
 
 A File object can be passed as a param for a POST request to handle uploading
 files to the server. Typhoeus will upload the file as the original file name
 and use Mime::Types to set the content type.
 
-    response = Typhoeus::Request.post("http://localhost:3000/posts",
-      :params => {
-        :title => "test post", :content => "this is my test",
-        :file => File.open("thesis.txt","r")
-      }
-    )
+```ruby
+Typhoeus.post(
+  "http://localhost:3000/posts",
+  body: {
+    title: "test post",
+    content: "this is my test",
+    file: File.open("thesis.txt","r")
+  }
+)
+```
+
+### Streaming the response body
+
+Typhoeus can stream responses. When you're expecting a large response,
+set the `on_body` callback on a request. Typhoeus will yield to the callback
+with chunks of the response, as they're read. When you set an `on_body` callback,
+Typhoeus will not store the complete response.
+
+```ruby
+downloaded_file = File.open 'huge.iso', 'wb'
+request = Typhoeus::Request.new("www.example.com/huge.iso")
+request.on_headers do |response|
+  if response.code != 200
+    raise "Request failed"
+  end
+end
+request.on_body do |chunk|
+  downloaded_file.write(chunk)
+end
+request.on_complete do |response|
+  downloaded_file.close
+  # Note that response.body is ""
+end
+request.run
+```
+
+If you need to interrupt the stream halfway,
+you can return the `:abort` symbol from the `on_body` block, example:
+
+```ruby
+request.on_body do |chunk|
+  buffer << chunk
+  :abort if buffer.size > 1024 * 1024
+end
+```
+
+This will properly stop the stream internally and avoid any memory leak which
+may happen if you interrupt with something like a `return`, `throw` or `raise`.
+
+### Making Parallel Requests
+
+Generally, you should be running requests through hydra. Here is how that looks:
+
+```ruby
+hydra = Typhoeus::Hydra.hydra
+
+first_request = Typhoeus::Request.new("http://example.com/posts/1")
+first_request.on_complete do |response|
+  third_url = response.body
+  third_request = Typhoeus::Request.new(third_url)
+  hydra.queue third_request
+end
+second_request = Typhoeus::Request.new("http://example.com/posts/2")
+
+hydra.queue first_request
+hydra.queue second_request
+hydra.run # this is a blocking call that returns once all requests are complete
+```
+
+The execution of that code goes something like this. The first and second requests are built and queued. When hydra is run the first and second requests run in parallel. When the first request completes, the third request is then built and queued, in this example based on the result of the first request. The moment it is queued Hydra starts executing it.  Meanwhile the second request would continue to run (or it could have completed before the first). Once the third request is done, `hydra.run` returns.
+
+How to get an array of response bodies back after executing a queue:
+
+```ruby
+hydra = Typhoeus::Hydra.new
+requests = 10.times.map {
+  request = Typhoeus::Request.new("www.example.com", followlocation: true)
+  hydra.queue(request)
+  request
+}
+hydra.run
+
+responses = requests.map { |request|
+  request.response.body
+}
+```
+`hydra.run` is a blocking request. You can also use the `on_complete` callback to handle each request as it completes:
+
+```ruby
+hydra = Typhoeus::Hydra.new
+10.times do
+  request = Typhoeus::Request.new("www.example.com", followlocation: true)
+  request.on_complete do |response|
+    #do_something_with response
+  end
+  hydra.queue(request)
+end
+hydra.run
+```
+
+### Making Parallel Requests with Faraday + Typhoeus
+
+```ruby
+require 'faraday'
+
+conn = Faraday.new(:url => 'http://httppage.com') do |builder|
+  builder.request  :url_encoded
+  builder.response :logger
+  builder.adapter  :typhoeus
+end
+
+conn.in_parallel do
+  response1 = conn.get('/first')
+  response2 = conn.get('/second')
+
+  # these will return nil here since the
+  # requests have not been completed
+  response1.body
+  response2.body
+end
+
+# after it has been completed the response information is fully available
+# response1.status, etc
+response1.body
+response2.body
+```
+
+### Specifying Max Concurrency
+
+Hydra will also handle how many requests you can make in parallel. Things will get flakey if you try to make too many requests at the same time. The built in limit is 200. When more requests than that are queued up, hydra will save them for later and start the requests as others are finished. You can raise or lower the concurrency limit through the Hydra constructor.
+
+```ruby
+Typhoeus::Hydra.new(max_concurrency: 20)
+```
 
-**Making Parallel Requests**
-
-    # Generally, you should be running requests through hydra. Here is how that looks
-    hydra = Typhoeus::Hydra.new
-
-    first_request = Typhoeus::Request.new("http://localhost:3000/posts/1.json")
-    first_request.on_complete do |response|
-      post = JSON.parse(response.body)
-      third_request = Typhoeus::Request.new(post.links.first) # get the first url in the post
-      third_request.on_complete do |response|
-        # do something with that
-      end
-      hydra.queue third_request
-      return post
-    end
-    second_request = Typhoeus::Request.new("http://localhost:3000/users/1.json")
-    second_request.on_complete do |response|
-      JSON.parse(response.body)
-    end
-    hydra.queue first_request
-    hydra.queue second_request
-    hydra.run # this is a blocking call that returns once all requests are complete
-
-    first_request.handled_response # the value returned from the on_complete block
-    second_request.handled_response # the value returned from the on_complete block (parsed JSON)
-
-The execution of that code goes something like this. The first and second
-requests are built and queued. When hydra is run the first and second requests
-run in parallel. When the first request completes, the third request is then
-built and queued up. The moment it is queued Hydra starts executing it.
-Meanwhile the second request would continue to run (or it could have completed
-before the first). Once the third request is done, hydra.run returns.
-
-**Specifying Max Concurrency**
-
-Hydra will also handle how many requests you can make in parallel. Things will
-get flakey if you try to make too many requests at the same time. The built in
-limit is 200. When more requests than that are queued up, hydra will save them
-for later and start the requests as others are finished. You can raise or
-lower the concurrency limit through the Hydra constructor.
-
-    hydra = Typhoeus::Hydra.new(:max_concurrency => 20) # keep from killing some servers
-
-**Memoization**
-
-Hydra memoizes requests within a single run call. You can also disable
-memoization.
-
-    hydra = Typhoeus::Hydra.new
-    2.times do
-      r = Typhoeus::Request.new("http://localhost/3000/users/1")
-      hydra.queue r
-    end
-    hydra.run # this will result in a single request being issued. However, the on_complete handlers of both will be called.
-    hydra.disable_memoization
-    2.times do
-      r = Typhoeus::Request.new("http://localhost/3000/users/1")
-      hydra.queue r
-    end
-    hydra.run # this will result in a two requests.
-
-**Caching**
-
-Hydra includes built in support for creating cache getters and setters. In the
-following example, if there is a cache hit, the cached object is passed to the
-on\_complete handler of the request object.
-
-    hydra = Typhoeus::Hydra.new
-    hydra.cache_setter do |request|
-      @cache.set(request.cache_key, request.response, request.cache_timeout)
-    end
-
-    hydra.cache_getter do |request|
-      @cache.get(request.cache_key) rescue nil
-    end
-
-**Direct Stubbing**
+### Memoization
 
-Hydra allows you to stub out specific urls and patterns to avoid hitting
-remote servers while testing.
+Hydra memoizes requests within a single run call. You have to enable memoization.
+This will result in a single request being issued. However, the on_complete handlers of both will be called.
 
-    hydra = Typhoeus::Hydra.new
-    response = Response.new(:code => 200, :headers => "", :body => "{'name' : 'paul'}", :time => 0.3)
-    hydra.stub(:get, "http://localhost:3000/users/1").and_return(response)
+```ruby
+Typhoeus::Config.memoize = true
 
-    request = Typhoeus::Request.new("http://localhost:3000/users/1")
-    request.on_complete do |response|
-      JSON.parse(response.body)
-    end
-    hydra.queue request
-    hydra.run
+hydra = Typhoeus::Hydra.new(max_concurrency: 1)
+2.times do
+  hydra.queue Typhoeus::Request.new("www.example.com")
+end
+hydra.run
+```
 
-The queued request will hit the stub. The on\_complete handler will be called
-and will be passed the response object. You can also specify a regex to match
-urls.
+This will result in two requests.
 
-    hydra.stub(:get, /http\:\/\/localhost\:3000\/users\/.*/).and_return(response)
-    # any requests for a user will be stubbed out with the pre built response.
+```ruby
+Typhoeus::Config.memoize = false
+
+hydra = Typhoeus::Hydra.new(max_concurrency: 1)
+2.times do
+  hydra.queue Typhoeus::Request.new("www.example.com")
+end
+hydra.run
+```
 
-**The Singleton**
+### Caching
 
-All of the quick requests are done using the singleton hydra object. If you
-want to enable caching or stubbing on the quick requests, set those options on
-the singleton.
+Typhoeus includes built in support for caching. In the following example, if there is a cache hit, the cached object is passed to the on_complete handler of the request object.
 
-    hydra = Typhoeus::Hydra.hydra
-    hydra.stub(:get, "http://localhost:3000/users")
+```ruby
+class Cache
+  def initialize
+    @memory = {}
+  end
 
-**Timeouts**
+  def get(request)
+    @memory[request]
+  end
 
-No exceptions are raised on HTTP timeouts. You can check whether a request
-timed out with the following methods:
+  def set(request, response)
+    @memory[request] = response
+  end
+end
 
-    easy.timed_out?  # for a raw Easy handle
-    response.timed_out?  # for a Response handle
+Typhoeus::Config.cache = Cache.new
 
-**Following Redirections**
+Typhoeus.get("www.example.com").cached?
+#=> false
+Typhoeus.get("www.example.com").cached?
+#=> true
+```
 
-Use `:follow_location => true`, eg:
+For use with [Dalli](https://github.com/mperham/dalli):
 
-    Typhoeus::Request.new(“www.example.com”, :follow_location => true)
+```ruby
+dalli = Dalli::Client.new(...)
+Typhoeus::Config.cache = Typhoeus::Cache::Dalli.new(dalli)
+```
 
-**Basic Authentication**
+For use with Rails:
 
-    response = Typhoeus::Request.get("http://twitter.com/statuses/followers.json",
-                                     :username => username, :password => password)
+```ruby
+Typhoeus::Config.cache = Typhoeus::Cache::Rails.new
+```
 
-**SSL**
+For use with [Redis](https://github.com/redis/redis-rb):
 
-SSL comes built in to libcurl so it’s in Typhoeus as well. If you pass in a
-url with “https” it should just work assuming that you have your [cert
-bundle](http://curl.haxx.se/docs/caextract.html) in order and the server is
-verifiable. You must also have libcurl built with SSL support enabled. You can
-check that by doing this:
+```ruby
+redis = Redis.new(...)
+Typhoeus::Config.cache = Typhoeus::Cache::Redis.new(redis)
+```
 
-    Typhoeus::Easy.new.curl_version # output should include OpenSSL/...
+All three of these adapters take an optional keyword argument `default_ttl`, which sets a default
+TTL on cached responses (in seconds), for requests which do not have a cache TTL set.
 
-Now, even if you have libcurl built with OpenSSL you may still have a messed
-up cert bundle or if you’re hitting a non-verifiable SSL server then you’ll
-have to disable peer verification to make SSL work. Like this:
+You may also selectively choose not to cache by setting `cache` to `false` on a request or to use
+a different adapter.
 
-    Typhoeus::Request.get("https://mail.google.com/mail", :disable_ssl_peer_verification => true)
+```ruby
+cache = Cache.new
+Typhoeus.get("www.example.com", cache: cache)
+```
 
-If you are getting “SSL: certificate subject name does not match target host
-name” from curl (ex:- you are trying to access to b.c.host.com when the
-certificate subject is \*.host.com). You can disable host verification. Like
-this:
+### Direct Stubbing
 
-    Typhoeus::Request.get("https://mail.google.com/mail", :disable_ssl_host_verification => true)
+Hydra allows you to stub out specific urls and patterns to avoid hitting
+remote servers while testing.
 
-**LibCurl**
+```ruby
+response = Typhoeus::Response.new(code: 200, body: "{'name' : 'paul'}")
+Typhoeus.stub('www.example.com').and_return(response)
 
-Typhoeus also has a more raw libcurl interface. These are the Easy and Multi
-objects. If you’re into accessing just the raw libcurl style, those are your
-best bet.
+Typhoeus.get("www.example.com") == response
+#=> true
+```
 
-However, by using this raw interface, you do not get access to Hydra-specific
-features, such as stubbing/mocking.
+The queued request will hit the stub. You can also specify a regex to match urls.
 
-SSL Certs can be provided to the Easy interface:
+```ruby
+response = Typhoeus::Response.new(code: 200, body: "{'name' : 'paul'}")
+Typhoeus.stub(/example/).and_return(response)
 
-    e = Typhoeus::Easy.new
-    e.url = "https://example.com/action"
-    s.ssl_cacert = "ca_file.cer"
-    e.ssl_cert = "acert.crt"
-    e.ssl_key = "akey.key"
-    [...]
-    e.perform
+Typhoeus.get("www.example.com") == response
+#=> true
+```
 
-or directly to a Typhoeus::Request :
+You may also specify an array for the stub to return sequentially.
 
-    e = Typhoeus::Request.get("https://example.com/action",
-      :ssl_cacert => "ca_file.cer",
-      :ssl_cert => "acert.crt",
-      :ssl_key => "akey.key",
-      [...]
-    end
+```ruby
+Typhoeus.stub('www.example.com').and_return([response1, response2])
 
-##  Advanced authentication
+Typhoeus.get('www.example.com') == response1 #=> true
+Typhoeus.get('www.example.com') == response2 #=> true
+```
 
-Thanks for the authentication piece and this description go to Oleg Ivanov
-(morhekil). The major reason to start this fork was the need to perform NTLM
-authentication in Ruby, and other libcurl’s authentications method were made
-possible as a result. Now you can do it via Typhoeus::Easy interface using the
-following API.
+When testing make sure to clear your expectations or the stubs will persist between tests. The following can be included in your spec_helper.rb file to do this automatically.
 
-    e = Typhoeus::Easy.new
-    e.auth = {
-      :username => 'username',
-      :password => 'password',
-      :method => Typhoeus::Easy::AUTH_TYPES[:CURLAUTH_NTLM]
-    }
-    e.url = "http://example.com/auth_ntlm"
-    e.method = :get
-    e.perform
+```ruby
+RSpec.configure do |config|
+  config.before :each do
+    Typhoeus::Expectation.clear
+  end
+end
+```
 
-**Other authentication types**
+### Timeouts
 
-The following authentication types are available:
+No exceptions are raised on HTTP timeouts. You can check whether a request timed out with the following method:
 
-  * CURLAUTH\_BASIC
-  * CURLAUTH\_DIGEST
-  * CURLAUTH\_GSSNEGOTIATE
-  * CURLAUTH\_NTLM
-  * CURLAUTH\_DIGEST\_IE
-  * CURLAUTH\_AUTO
+```ruby
+Typhoeus.get("www.example.com", timeout: 1).timed_out?
+```
 
-The last one (CURLAUTH\_AUTO) is really a combination of all previous methods
-and is provided by Typhoeus for convenience. When you set authentication to
-auto, Typhoeus will retrieve the given URL first and examine it’s headers to
-confirm what auth types are supported by the server. The it will select the
-strongest of available auth methods and will send the second request using the
-selected authentication method.
+Timed out responses also have their success? method return false.
 
-**Authentication via the quick request interface**
+There are two different timeouts available: [`timeout`](http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTTIMEOUT)
+and [`connecttimeout`](http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTCONNECTTIMEOUT).
+`timeout` is the time limit for the entire request in seconds.
+`connecttimeout` is the time limit for just the connection phase, again in seconds.
 
-There’s also an easy way to perform any kind of authentication via the quick
-request interface:
+There are two additional more fine grained options `timeout_ms` and
+`connecttimeout_ms`. These options offer millisecond precision but are not always available (for instance on linux if `nosignal` is not set to true).
 
-    e = Typhoeus::Request.get("http://example.com",
-      :username => 'username',
-      :password => 'password',
-      :auth_method => :ntlm)
+When you pass a floating point `timeout` (or `connecttimeout`) Typhoeus will set `timeout_ms` for you if it has not been defined. The actual timeout values passed to curl will always be rounded up.
 
-All methods listed above is available in a shorter form – :basic, :digest,
-:gssnegotiate, :ntlm, :digest\_ie, :auto.
+DNS timeouts of less than one second are not supported unless curl is compiled with an asynchronous resolver.
 
-**Query of available auth types**
+The default `timeout` is 0 (zero) which means curl never times out during transfer. The default `connecttimeout` is 300 seconds. A `connecttimeout` of 0 will also result in the default `connecttimeout` of 300 seconds.
 
-After the initial request you can get the authentication types available on
-the server via Typhoues::Easy#auth\_methods call. It will return a number
+### Following Redirections
 
-that you’ll need to decode yourself, please refer to easy.rb source code to
-see the numeric values of different auth types.
+Use `followlocation: true`, eg:
 
-##  Verbose debug output
+```ruby
+Typhoeus.get("www.example.com", followlocation: true)
+```
 
-Sometime it’s useful to see verbose output from curl. You may now enable it:
+### Basic Authentication
 
-    e = Typhoeus::Easy.new
-    e.verbose = 1
+```ruby
+Typhoeus::Request.get("www.example.com", userpwd: "user:password")
+```
 
-or using the quick request:
+### Compression
 
-    e = Typhoeus::Request.get("http://example.com", :verbose => true)
+```ruby
+Typhoeus.get("www.example.com", accept_encoding: "gzip")
+```
 
-Just remember that libcurl prints it’s debug output to the console (to
-STDERR), so you’ll need to run your scripts from the console to see it.
+The above has a different behavior than setting the header directly in the header hash, eg:
+```ruby
+Typhoeus.get("www.example.com", headers: {"Accept-Encoding" => "gzip"})
+```
+
+Setting the header hash directly will not include the `--compressed` flag in the libcurl command and therefore libcurl will not decompress the response.  If you want the `--compressed` flag to be added automatically, set `:accept_encoding` Typhoeus option.
+
+
+### Cookies
+
+```ruby
+Typhoeus::Request.get("www.example.com", cookiefile: "/path/to/file", cookiejar: "/path/to/file")
+```
 
-##  Benchmarks
+Here, `cookiefile` is a file to read cookies from, and `cookiejar` is a file to write received cookies to.
+If you just want cookies enabled, you need to pass the same filename for both options.
 
-I set up a benchmark to test how the parallel performance works vs Ruby’s
-built in NET::HTTP. The setup was a local evented HTTP server that would take
-a request, sleep for 500 milliseconds and then issued a blank response. I set
-up the client to call this 20 times. Here are the results:
+### Other CURL options
 
-      net::http  0.030000   0.010000   0.040000 ( 10.054327)
-      typhoeus   0.020000   0.070000   0.090000 (  0.508817)
+Are available and documented [here](http://rubydoc.info/github/typhoeus/ethon/Ethon/Easy/Options)
+
+### SSL
+
+SSL comes built in to libcurl so it’s in Typhoeus as well. If you pass in a
+url with "https" it should just work assuming that you have your [cert
+bundle](http://curl.haxx.se/docs/caextract.html) in order and the server is
+verifiable. You must also have libcurl built with SSL support enabled. You can
+check that by doing this:
+
+```
+curl --version
+```
+
+Now, even if you have libcurl built with OpenSSL you may still have a messed
+up cert bundle or if you’re hitting a non-verifiable SSL server then you’ll
+have to disable peer verification to make SSL work. Like this:
+
+```ruby
+Typhoeus.get("https://www.example.com", ssl_verifypeer: false)
+```
+
+If you are getting "SSL: certificate subject name does not match target host
+name" from curl (ex:- you are trying to access to b.c.host.com when the
+certificate subject is \*.host.com). You can disable host verification. Like
+this:
+
+```ruby
+# host checking enabled
+Typhoeus.get("https://www.example.com", ssl_verifyhost: 2)
+# host checking disabled
+Typhoeus.get("https://www.example.com", ssl_verifyhost: 0)
+```
+
+### Verbose debug output
+
+It’s sometimes useful to see verbose output from curl. You can enable it on a per-request basis:
+
+```ruby
+Typhoeus.get("http://example.com", verbose: true)
+```
+
+or globally:
+
+```ruby
+Typhoeus::Config.verbose = true
+```
+
+Just remember that libcurl prints it’s debug output to the console (to
+STDERR), so you’ll need to run your scripts from the console to see it.
 
-We can see from this that NET::HTTP performs as expected, taking 10 seconds to
-run 20 500ms requests. Typhoeus only takes 500ms (the time of the response
-that took the longest.) One other thing to note is that Typhoeus keeps a pool
-of libcurl Easy handles to use. For this benchmark I warmed the pool first. So
-if you test this out it may be a bit slower until the Easy handle pool has
-enough in it to run all the simultaneous requests. For some reason the easy
-handles can take quite some time to allocate.
+### Default User Agent Header
 
-##  Running the specs
+In many cases, all HTTP requests made by an application require the same User-Agent header set. Instead of supplying it on a per-request basis by supplying a custom header, it is possible to override it for all requests using:
 
-Running the specs requires a couple of Sinatra servers to be booted. rake spec
-will do this for you, but if you’re needing to run the specs a lot, spinning
-up the servers manually and leaving them running should speed things up a bit.
-Do this:
 
-      # Start up the test servers (in another terminal)
-      rake start_test_servers
+```ruby
+Typhoeus::Config.user_agent = "custom user agent"
+```
 
-      # Run the specs
-      rake spec
+### Running the specs
 
+Running the specs should be as easy as:
 
-##  Next Steps
+```
+bundle install
+bundle exec rake
+```
+## Semantic Versioning
 
-  * Add in ability to keep-alive requests and reuse them within hydra.
-  * Add support for automatic retry, exponential back-off, and queuing for later.
+This project conforms to [semver](http://semver.org/).
 
-##  LICENSE
+## LICENSE
 
 (The MIT License)
 
-Copyright © 2009-2010 Paul Dix
+Copyright © 2009-2010 [Paul Dix](http://www.pauldix.net/)
 
-Copyright © 2011 David Balatero
+Copyright © 2011-2012 [David Balatero](https://github.com/dbalatero/)
 
-Copyright © 2012 [Hans Hasselberg](http://www.hans.io)
+Copyright © 2012-2016 [Hans Hasselberg](http://github.com/i0rek/)
 
 Permission is hereby granted, free of charge, to any person obtaining a
 copy of this software and associated documentation files (the "Software"),