rest-client 1.8.0 → 2.0.0.rc1

data/README.rdoc

@@ -10,28 +10,26 @@ of specifying actions: get, put, post, delete.
 
 == Requirements
 
-MRI Ruby 1.9.2 and newer are supported. Alternative interpreters compatible with
-1.9.1+ should work as well.
+MRI Ruby 1.9.3 and newer are supported. Alternative interpreters compatible with
+1.9+ should work as well.
 
-Ruby 1.8.7 is no longer supported.  That's because the Ruby 1.8.7 interpreter
-itself no longer has official support, _not_ _even_ _security_ _patches!_ If you
-have been putting off upgrading your servers, now is the time.
-({More info is on the Ruby developers'
-blog.}[http://www.ruby-lang.org/en/news/2013/06/30/we-retire-1-8-7/])
+Earlier Ruby versions such as 1.8.7 and 1.9.2 are no longer supported. These
+versions no longer have any official support, and do not receive security
+updates.
 
-The rest-client gem depends on these other gems for installation and usage:
+The rest-client gem depends on these other gems for usage at runtime:
 
 * {mime-types}[http://rubygems.org/gems/mime-types]
 * {netrc}[http://rubygems.org/gems/netrc]
-* {rdoc}[http://rubygems.org/gems/rdoc]
+* {http-cookie}[https://rubygems.org/gems/http-cookie]
 
-If you want to hack on the code, you should also have {the Bundler
-gem}[http://bundler.io/] installed so it can manage all necessary development
-dependencies for you.
+There are also several development dependencies. It's recommended to use
+{bundler}[http://bundler.io/] to manage these dependencies for hacking on
+rest-client.
 
 == Usage: Raw URL
 
-  require 'rest_client'
+  require 'rest-client'
 
   RestClient.get 'http://example.com/resource'
 
@@ -67,6 +65,32 @@ dependencies for you.
       }
     })
 
+== Passing advanced options
+
+The top level helper methods like RestClient.get accept a headers hash as
+their last argument and don't allow passing more complex options. But these
+helpers are just thin wrappers around RestClient::Request.execute.
+
+  RestClient::Request.execute(method: :get, url: 'http://example.com/resource',
+                              timeout: 10)
+
+You can also use this to pass a payload for HTTP verbs like DELETE, where the
+RestClient.delete helper doesn't accept a payload.
+
+  RestClient::Request.execute(method: :delete, url: 'http://example.com/resource',
+                              payload: 'foo', headers: {myheader: 'bar'})
+
+Due to unfortunate choices in the original API, the params used to populate the
+query string are actually taken out of the headers hash. So if you want to pass
+both the params hash and more complex options, use the special key
+<tt>:params</tt> in the headers hash. This design may change in a future major
+release.
+
+  RestClient::Request.execute(method: :get, url: 'http://example.com/resource',
+                              timeout: 10, headers: {params: {foo: 'bar'}})
+
+  ➔ GET http://example.com/resource?foo=bar
+
 == Multipart
 
 Yeah, that's right!  This does multipart sends for you!
@@ -105,6 +129,7 @@ See RestClient::Resource docs for details.
 * for result codes 301, 302 or 307, the redirection will be followed if the request is a GET or a HEAD
 * for result code 303, the redirection will be followed and the request transformed into a GET
 * for other cases, a RestClient::Exception holding the Response will be raised; a specific exception class will be thrown for known error codes
+* call <tt>.response</tt> on the exception to get the server's response
 
    RestClient.get 'http://example.com/resource'
    ➔ RestClient::ResourceNotFound: RestClient::ResourceNotFound
@@ -118,6 +143,34 @@ See RestClient::Resource docs for details.
 
 == Result handling
 
+The result of a RestClient::Request is a RestClient::Response object.
+
+<b>New in 2.0:</b> RestClient::Response objects are now a subclass of String.
+Previously, they were a real String object with response functionality mixed
+in, which was very confusing to work with.
+
+Response objects have several useful methods. (See the class rdoc for more details.)
+
+* Response#code: The HTTP response code
+* Response#body: The response body as a string. (AKA .to_s)
+* Response#headers: A hash of HTTP response headers
+* Response#cookies: A hash of HTTP cookies set by the server
+* Response#cookie_jar: <em>New in 1.8</em> An HTTP::CookieJar of cookies
+* Response#request: The RestClient::Request object used to make the request
+* Response#history: If redirection was followed, a list of prior Response objects
+
+   >> RestClient.get('http://example.com')
+   => <RestClient::Response 200 "<!doctype h...">
+
+   >> begin
+   >>   RestClient.get('http://example.com/notfound')
+   >> rescue RestClient::ExceptionWithResponse => err
+   >>   err.response
+   >> end
+   => <RestClient::Response 404 "<!doctype h...">
+
+=== Response callbacks
+
 A block can be passed to the RestClient method. This block will then be called with the Response.
 Response.return! can be called to invoke the default response's behavior.
 
@@ -249,6 +302,20 @@ the URL for GET, HEAD and DELETE requests, escaping the keys and values as neede
   RestClient.get 'http://example.com/resource', :params => {:foo => 'bar', :baz => 'qux'}
   # will GET http://example.com/resource?foo=bar&baz=qux
 
+== Headers
+
+Request headers can be set by passing a ruby hash containing keys and values
+representing header names and values:
+
+  # GET request with modified headers
+  RestClient.get 'http://example.com/resource', {:Authorization => 'Bearer cT0febFoD5lxAlNAXHo6g'}
+
+  # POST request with modified headers
+  RestClient.post 'http://example.com/resource', {:foo => 'bar', :baz => 'qux'}, {:Authorization => 'Bearer cT0febFoD5lxAlNAXHo6g'}
+
+  # DELETE request with modified headers
+  RestClient.delete 'http://example.com/resource', {:Authorization => 'Bearer cT0febFoD5lxAlNAXHo6g'}
+
 == Cookies
 
 Request and Response objects know about HTTP cookies, and will automatically
@@ -302,15 +369,16 @@ Have a look at rest-client-components: http://github.com/crohr/rest-client-compo
 
 == Credits
 
-REST Client Team:: Matthew Manning, Lawrence Leonard Gilbert, Andy Brody
+REST Client Team:: Andy Brody
 
 Creator:: Adam Wiggins
 
-Maintainer Emeritus:: Julien Kirch
+Maintainers Emeriti:: Lawrence Leonard Gilbert, Matthew Manning, Julien Kirch
 
 Major contributions:: Blake Mizerany, Julien Kirch
 
-Patches contributed by many, including Chris Anderson, Greg Borenstein, Ardekantur, Pedro Belo, Rafael Souza, Rick Olson, Aman Gupta, François Beausoleil and Nick Plante.
+A great many generous folks have contributed features and patches.
+See AUTHORS for the full list.
 
 == Legal
 

data/Rakefile

@@ -34,6 +34,22 @@ RSpec::Core::RakeTask.new('rcov') do |t|
   t.rcov_opts = ['--exclude', 'examples']
 end
 
+desc 'Regenerate authors file'
+task :authors do
+  Dir.chdir(File.dirname(__FILE__)) do
+    File.open('AUTHORS', 'w') do |f|
+      f.write( <<-EOM
+The Ruby REST Client would not be what it is today without the help of
+the following kind souls:
+
+      EOM
+      )
+    end
+
+    sh 'git shortlog -s | cut -f 2 >> AUTHORS'
+  end
+end
+
 task :default do
   sh 'rake -T'
 end

data/bin/restclient

@@ -56,11 +56,9 @@ if @verb
 end
 
 POSSIBLE_VERBS.each do |m|
-  eval <<-end_eval
-def  #{m}(path, *args, &b)
-  r[path].#{m}(*args, &b)
-end
-  end_eval
+  define_method(m.to_sym) do |path, *args, &b|
+    r[path].public_send(m.to_sym, *args, &b)
+  end
 end
 
 def method_missing(s, * args, & b)

data/history.md

@@ -1,3 +1,64 @@
+# 2.0.0
+
+This release is largely API compatible, but makes several breaking changes.
+
+- Drop support for Ruby 1.9.2
+- Respect Content-Type charset header provided by server. Previously,
+  rest-client would not override the string encoding chosen by Net::HTTP. Now
+  responses that specify a charset will yield a body string in that encoding.
+  For example, `Content-Type: text/plain; charset=EUC-JP` will return a String
+  encoded with `Encoding::EUC_JP`.
+- Change exceptions raised on request timeout. Instead of
+  RestClient::RequestTimeout (which is still used for HTTP 408), network
+  timeouts will now raise either RestClient::Exceptions::ReadTimeout or
+  RestClient::Exceptions::OpenTimeout, both of which inherit from
+  RestClient::Exceptions::Timeout. This class also makes the original wrapped
+  exception available as `#original_exception`.
+- Rename `:timeout` to `:read_timeout`. When `:timeout` is passed, now set both
+  `:read_timeout` and `:open_timeout`.
+- Change default HTTP Accept header to `*/*`
+- Use a more descriptive User-Agent header by default
+- Drop RC4-MD5 from default cipher list
+- Only prepend http:// to URIs without a scheme
+- Fix some support for using IPv6 addresses in URLs (still affected by Ruby
+  2.0+ bug https://bugs.ruby-lang.org/issues/9129, with the fix expected to be
+  backported to 2.0 and 2.1)
+- `Response` objects are now a subclass of `String` rather than a `String` that
+  mixes in the response functionality. Most of the methods remain unchanged,
+  but this makes it much easier to understand what is happening when you look
+  at a RestClient response object. There are a few additional changes:
+  - Response objects now implement `.inspect` to make this distinction clearer.
+  - `Response#to_i` will now behave like `String#to_i` instead of returning the
+    HTTP response code, which was very surprising behavior.
+  - `Response#body` and `#to_s` will now return a true `String` object rather
+    than self. Previously there was no easy way to get the true `String` response
+    instead of the Frankenstein response string object with AbstractResponse
+    mixed in.
+- Handle multiple HTTP response headers with the same name (except for
+  Set-Cookie, which is special) by joining the values with a comma space,
+  compliant with RFC 7230
+- Don't set basic auth header if explicit `Authorization` header is specified
+- Add `:proxy` option to requests, which can be used for thread-safe
+  per-request proxy configuration, overriding `RestClient.proxy`
+- Allow overriding `ENV['http_proxy']` to disable proxies by setting
+  `RestClient.proxy` to a falsey value. Previously there was no way in Ruby 2.x
+  to turn off a proxy specified in the environment without changing `ENV`.
+- Add actual support for streaming request payloads. Previously rest-client
+  would call `.to_s` even on RestClient::Payload::Streamed objects. Instead,
+  treat any object that responds to `.read` as a streaming payload and pass it
+  through to `.body_stream=` on the Net:HTTP object. This massively reduces the
+  memory required for large file uploads.
+- Remove `RestClient::MaxRedirectsReached` in favor of the normal
+  `ExceptionWithResponse` subclasses. This makes the response accessible on the
+  exception object as `.response`, making it possible for callers to tell what
+  has actually happened when the redirect limit is reached.
+- When following HTTP redirection, store a list of each previous response on
+  the response object as `.history`. This makes it possible to access the
+  original response headers and body before the redirection was followed.
+- Add `:before_execution_proc` option to `RestClient::Request`. This makes it
+  possible to add procs like `RestClient.add_before_execution_proc` to a single
+  request without global state.
+
 # 1.8.0
 
 - Security: implement standards compliant cookie handling by adding a

data/lib/restclient.rb

@@ -7,6 +7,7 @@ require 'zlib'
 require File.dirname(__FILE__) + '/restclient/version'
 require File.dirname(__FILE__) + '/restclient/platform'
 require File.dirname(__FILE__) + '/restclient/exceptions'
+require File.dirname(__FILE__) + '/restclient/utils'
 require File.dirname(__FILE__) + '/restclient/request'
 require File.dirname(__FILE__) + '/restclient/abstract_response'
 require File.dirname(__FILE__) + '/restclient/response'
@@ -89,8 +90,23 @@ module RestClient
     Request.execute(:method => :options, :url => url, :headers => headers, &block)
   end
 
-  class << self
-    attr_accessor :proxy
+  # A global proxy URL to use for all requests. This can be overridden on a
+  # per-request basis by passing `:proxy` to RestClient::Request.
+  def self.proxy
+    @proxy
+  end
+  def self.proxy=(value)
+    @proxy = value
+    @proxy_set = true
+  end
+
+  # Return whether RestClient.proxy was set explicitly. We use this to
+  # differentiate between no value being set and a value explicitly set to nil.
+  #
+  # @return [Boolean]
+  #
+  def self.proxy_set?
+    !!@proxy_set
   end
 
   # Setup the log for RestClient calls.
@@ -150,6 +166,7 @@ module RestClient
   # Add a Proc to be called before each request in executed.
   # The proc parameters will be the http request and the request params.
   def self.add_before_execution_proc &proc
+    raise ArgumentError.new('block is required') unless proc
     @@before_execution_procs << proc
   end
 

data/lib/restclient/abstract_response.rb

@@ -7,11 +7,19 @@ module RestClient
 
     attr_reader :net_http_res, :args, :request
 
+    def inspect
+      raise NotImplementedError.new('must override in subclass')
+    end
+
     # HTTP status code
     def code
       @code ||= @net_http_res.code.to_i
     end
 
+    def history
+      @history ||= request.redirection_history || []
+    end
+
     # A hash of the headers, beautified with symbols and underscores.
     # e.g. "Content-type" will become :content_type.
     def headers
@@ -27,6 +35,9 @@ module RestClient
       @net_http_res = net_http_res
       @args = args
       @request = request
+
+      # prime redirection history
+      history
     end
 
     # Hash of cookies extracted from response headers
@@ -57,79 +68,133 @@ module RestClient
 
     # Return the default behavior corresponding to the response code:
     # the response itself for code in 200..206, redirection for 301, 302 and 307 in get and head cases, redirection for 303 and an exception in other cases
-    def return! request = nil, result = nil, & block
+    def return!(&block)
       if (200..207).include? code
         self
       elsif [301, 302, 307].include? code
         unless [:get, :head].include? args[:method]
-          raise Exceptions::EXCEPTIONS_MAP[code].new(self, code)
+          raise exception_with_response
         else
-          follow_redirection(request, result, & block)
+          follow_redirection(&block)
         end
       elsif code == 303
-        args[:method] = :get
-        args.delete :payload
-        follow_redirection(request, result, & block)
-      elsif Exceptions::EXCEPTIONS_MAP[code]
-        raise Exceptions::EXCEPTIONS_MAP[code].new(self, code)
+        follow_get_redirection(&block)
       else
-        raise RequestFailed.new(self, code)
+        raise exception_with_response
       end
     end
 
     def to_i
-      code
+      warn('warning: calling Response#to_i is not recommended')
+      super
     end
 
     def description
       "#{code} #{STATUSES[code]} | #{(headers[:content_type] || '').gsub(/;.*$/, '')} #{size} bytes\n"
     end
 
-    # Follow a redirection
-    def follow_redirection request = nil, result = nil, & block
-      new_args = @args.dup
+    # Follow a redirection response by making a new HTTP request to the
+    # redirection target.
+    def follow_redirection(&block)
+      _follow_redirection(@args.dup, &block)
+    end
 
-      url = headers[:location]
-      if url !~ /^http/
-        url = URI.parse(request.url).merge(url).to_s
-      end
-      new_args[:url] = url
-      if request
-        if request.max_redirects == 0
-          raise MaxRedirectsReached
-        end
-        new_args[:password] = request.password
-        new_args[:user] = request.user
-        new_args[:headers] = request.headers
-        new_args[:max_redirects] = request.max_redirects - 1
-
-        # TODO: figure out what to do with original :cookie, :cookies values
-        new_args[:headers]['Cookie'] = HTTP::Cookie.cookie_value(
-          cookie_jar.cookies(new_args.fetch(:url)))
-      end
+    # Follow a redirection response, but change the HTTP method to GET and drop
+    # the payload from the original request.
+    def follow_get_redirection(&block)
+      new_args = @args.dup
+      new_args[:method] = :get
+      new_args.delete(:payload)
 
-      Request.execute(new_args, &block)
+      _follow_redirection(new_args, &block)
     end
 
+    # Convert headers hash into canonical form.
+    #
+    # Header names will be converted to lowercase symbols with underscores
+    # instead of hyphens.
+    #
+    # Headers specified multiple times will be joined by comma and space,
+    # except for Set-Cookie, which will always be an array.
+    #
+    # Per RFC 2616, if a server sends multiple headers with the same key, they
+    # MUST be able to be joined into a single header by a comma. However,
+    # Set-Cookie (RFC 6265) cannot because commas are valid within cookie
+    # definitions. The newer RFC 7230 notes (3.2.2) that Set-Cookie should be
+    # handled as a special case.
+    #
+    # http://tools.ietf.org/html/rfc2616#section-4.2
+    # http://tools.ietf.org/html/rfc7230#section-3.2.2
+    # http://tools.ietf.org/html/rfc6265
+    #
+    # @param headers [Hash]
+    # @return [Hash]
+    #
     def self.beautify_headers(headers)
       headers.inject({}) do |out, (key, value)|
-        out[key.gsub(/-/, '_').downcase.to_sym] = %w{ set-cookie }.include?(key.downcase) ? value : value.first
+        key_sym = key.gsub(/-/, '_').downcase.to_sym
+
+        # Handle Set-Cookie specially since it cannot be joined by comma.
+        if key.downcase == 'set-cookie'
+          out[key_sym] = value
+        else
+          out[key_sym] = value.join(', ')
+        end
+
         out
       end
     end
 
     private
 
-    # Parse a cookie value and return its content in an Hash
-    def parse_cookie cookie_content
-      out = {}
-      CGI::Cookie::parse(cookie_content).each do |key, cookie|
-        unless ['expires', 'path'].include? key
-          out[CGI::escape(key)] = cookie.value[0] ? (CGI::escape(cookie.value[0]) || '') : ''
-        end
+    # Follow a redirection
+    #
+    # @param new_args [Hash] Start with this hash of arguments for the
+    #   redirection request. The hash will be mutated, so be sure to dup any
+    #   existing hash that should not be modified.
+    #
+    def _follow_redirection(new_args, &block)
+
+      # parse location header and merge into existing URL
+      url = headers[:location]
+
+      # handle relative redirects
+      unless url.start_with?('http')
+        url = URI.parse(request.url).merge(url).to_s
+      end
+      new_args[:url] = url
+
+      if request.max_redirects <= 0
+        raise exception_with_response
+      end
+      new_args[:password] = request.password
+      new_args[:user] = request.user
+      new_args[:headers] = request.headers
+      new_args[:max_redirects] = request.max_redirects - 1
+
+      # TODO: figure out what to do with original :cookie, :cookies values
+      new_args[:headers]['Cookie'] = HTTP::Cookie.cookie_value(
+        cookie_jar.cookies(new_args.fetch(:url)))
+
+
+      # prepare new request
+      new_req = Request.new(new_args)
+
+      # append self to redirection history
+      new_req.redirection_history = history + [self]
+
+      # execute redirected request
+      new_req.execute(&block)
+    end
+
+    def exception_with_response
+      begin
+        klass = Exceptions::EXCEPTIONS_MAP.fetch(code)
+      rescue KeyError
+        raise RequestFailed.new(self, code)
       end
-      out
+
+      raise klass.new(self, code)
     end
   end
-
 end