rest-client 2.0.0.rc2 → 2.0.0.rc3

data/Rakefile

@@ -127,6 +127,6 @@ Rake::RDocTask.new do |t|
   t.title    = "rest-client, fetch RESTful resources effortlessly"
   t.options << '--line-numbers' << '--inline-source' << '-A cattr_accessor=object'
   t.options << '--charset' << 'utf-8'
-  t.rdoc_files.include('README.rdoc')
+  t.rdoc_files.include('README.md')
   t.rdoc_files.include('lib/*.rb')
 end

data/history.md

@@ -2,7 +2,8 @@
 
 This release is largely API compatible, but makes several breaking changes.
 
-- Drop support for Ruby 1.9.2
+- Drop support for Ruby 1.9
+- Allow mime-types as new as 3.x (requires ruby 2.0)
 - Respect Content-Type charset header provided by server. Previously,
   rest-client would not override the string encoding chosen by Net::HTTP. Now
   responses that specify a charset will yield a body string in that encoding.
@@ -20,8 +21,8 @@ This release is largely API compatible, but makes several breaking changes.
   inherits from `ExceptionWithResponse`. Previously, HTTP 304, 401, and 404
   inherited directly from `ExceptionWithResponse` rather than from
   `RequestFailed`. Now _all_ HTTP status code exceptions inherit from both.
-- Rename `:timeout` to `:read_timeout`. When `:timeout` is passed, now set both
-  `:read_timeout` and `:open_timeout`.
+- Rename the `:timeout` request option to `:read_timeout`. When `:timeout` is
+  passed, now set both `:read_timeout` and `:open_timeout`.
 - Change default HTTP Accept header to `*/*`
 - Use a more descriptive User-Agent header by default
 - Drop RC4-MD5 from default cipher list
@@ -37,12 +38,30 @@ This release is largely API compatible, but makes several breaking changes.
   - `Response#to_i` will now behave like `String#to_i` instead of returning the
     HTTP response code, which was very surprising behavior.
   - `Response#body` and `#to_s` will now return a true `String` object rather
-    than self. Previously there was no easy way to get the true `String` response
-    instead of the Frankenstein response string object with AbstractResponse
-    mixed in.
+    than self. Previously there was no easy way to get the true `String`
+    response instead of the Frankenstein response string object with
+    AbstractResponse mixed in.
+  - Response objects no longer accept an extra request args hash, but instead
+    access request args directly from the request object, which reduces
+    confusion and duplication.
 - Handle multiple HTTP response headers with the same name (except for
   Set-Cookie, which is special) by joining the values with a comma space,
   compliant with RFC 7230
+- Rewrite cookie support to be much smarter and to use cookie jars consistently
+  for requests, responses, and redirection (resolving long-standing complaints
+  about the previously broken behavior):
+  - The `:cookies` option may now be a Hash of Strings, an Array of
+    HTTP::Cookie objects, or a full HTTP::CookieJar.
+  - Add `RestClient::Request#cookie_jar` and reimplement `Request#cookies` to
+    be a wrapper around the cookie jar.
+  - Still support passing the `:cookies` option in the headers hash, but now
+    raise ArgumentError if that option is also passed to `Request#initialize`.
+  - Warn if both `:cookies` and a `Cookie` header are supplied.
+  - Use the `Request#cookie_jar` as the basis for `Response#cookie_jar`,
+    creating a copy of the jar and adding any newly received cookies.
+  - When following redirection, also use this same strategy so that cookies
+    from the original request are carried through in a standards-compliant way
+    by the cookie jar.
 - Don't set basic auth header if explicit `Authorization` header is specified
 - Add `:proxy` option to requests, which can be used for thread-safe
   per-request proxy configuration, overriding `RestClient.proxy`
@@ -54,16 +73,29 @@ This release is largely API compatible, but makes several breaking changes.
   treat any object that responds to `.read` as a streaming payload and pass it
   through to `.body_stream=` on the Net:HTTP object. This massively reduces the
   memory required for large file uploads.
-- Remove `RestClient::MaxRedirectsReached` in favor of the normal
-  `ExceptionWithResponse` subclasses. This makes the response accessible on the
-  exception object as `.response`, making it possible for callers to tell what
-  has actually happened when the redirect limit is reached.
-- When following HTTP redirection, store a list of each previous response on
-  the response object as `.history`. This makes it possible to access the
-  original response headers and body before the redirection was followed.
+- Changes to redirection behavior:
+  - Remove `RestClient::MaxRedirectsReached` in favor of the normal
+    `ExceptionWithResponse` subclasses. This makes the response accessible on
+    the exception object as `.response`, making it possible for callers to tell
+    what has actually happened when the redirect limit is reached.
+  - When following HTTP redirection, store a list of each previous response on
+    the response object as `.history`. This makes it possible to access the
+    original response headers and body before the redirection was followed.
+  - Follow redirection consistently, regardless of whether the HTTP method was
+    passed as a symbol or string. Under the hood rest-client now normalizes the
+    HTTP request method to a lowercase string.
 - Add `:before_execution_proc` option to `RestClient::Request`. This makes it
   possible to add procs like `RestClient.add_before_execution_proc` to a single
   request without global state.
+- Run tests on Travis's beta OS X support.
+- Make `Request#transmit` a private method, along with a few others.
+- Refactor URI parsing to happen earlier, in Request initialization.
+- When adding URL params, handle URLs that already contain params.
+- Add a few more exception classes for obscure HTTP status codes.
+- Multipart: use a much more robust multipart boundary with greater entropy.
+- Make `RestClient::Payload::Base#inspect` stop pretending to be a String.
+- Add `Request#redacted_uri` and `Request#redacted_url` to display the URI
+  with any password redacted.
 
 # 2.0.0.rc1
 

data/lib/restclient.rb

@@ -13,6 +13,7 @@ require File.dirname(__FILE__) + '/restclient/abstract_response'
 require File.dirname(__FILE__) + '/restclient/response'
 require File.dirname(__FILE__) + '/restclient/raw_response'
 require File.dirname(__FILE__) + '/restclient/resource'
+require File.dirname(__FILE__) + '/restclient/params_array'
 require File.dirname(__FILE__) + '/restclient/payload'
 require File.dirname(__FILE__) + '/restclient/windows'
 
@@ -93,8 +94,9 @@ module RestClient
   # A global proxy URL to use for all requests. This can be overridden on a
   # per-request basis by passing `:proxy` to RestClient::Request.
   def self.proxy
-    @proxy
+    @proxy ||= nil
   end
+
   def self.proxy=(value)
     @proxy = value
     @proxy_set = true
@@ -106,7 +108,7 @@ module RestClient
   # @return [Boolean]
   #
   def self.proxy_set?
-    !!@proxy_set
+    @proxy_set ||= false
   end
 
   # Setup the log for RestClient calls.

data/lib/restclient/abstract_response.rb

@@ -5,7 +5,7 @@ module RestClient
 
   module AbstractResponse
 
-    attr_reader :net_http_res, :args, :request
+    attr_reader :net_http_res, :request
 
     def inspect
       raise NotImplementedError.new('must override in subclass')
@@ -31,20 +31,28 @@ module RestClient
       @raw_headers ||= @net_http_res.to_hash
     end
 
-    def response_set_vars(net_http_res, args, request)
+    def response_set_vars(net_http_res, request)
       @net_http_res = net_http_res
-      @args = args
       @request = request
 
       # prime redirection history
       history
     end
 
-    # Hash of cookies extracted from response headers
+    # Hash of cookies extracted from response headers.
+    #
+    # NB: This will return only cookies whose domain matches this request, and
+    # may not even return all of those cookies if there are duplicate names.
+    # Use the full cookie_jar for more nuanced access.
+    #
+    # @see #cookie_jar
+    #
+    # @return [Hash]
+    #
     def cookies
       hash = {}
 
-      cookie_jar.cookies.each do |cookie|
+      cookie_jar.cookies(@request.uri).each do |cookie|
         hash[cookie.name] = cookie.value
       end
 
@@ -56,28 +64,40 @@ module RestClient
     # @return [HTTP::CookieJar]
     #
     def cookie_jar
-      return @cookie_jar if @cookie_jar
+      return @cookie_jar if defined?(@cookie_jar) && @cookie_jar
 
-      jar = HTTP::CookieJar.new
+      jar = @request.cookie_jar.dup
       headers.fetch(:set_cookie, []).each do |cookie|
-        jar.parse(cookie, @request.url)
+        jar.parse(cookie, @request.uri)
       end
 
       @cookie_jar = jar
     end
 
     # Return the default behavior corresponding to the response code:
-    # the response itself for code in 200..206, redirection for 301, 302 and 307 in get and head cases, redirection for 303 and an exception in other cases
+    #
+    # For 20x status codes: return the response itself
+    #
+    # For 30x status codes:
+    #   301, 302, 307: redirect GET / HEAD if there is a Location header
+    #   303: redirect, changing method to GET, if there is a Location header
+    #
+    # For all other responses, raise a response exception
+    #
     def return!(&block)
-      if (200..207).include? code
+      case code
+      when 200..207
         self
-      elsif [301, 302, 307].include? code
-        unless [:get, :head].include? args[:method]
-          raise exception_with_response
-        else
+      when 301, 302, 307
+        case request.method
+        when 'get', 'head'
+          check_max_redirects
           follow_redirection(&block)
+        else
+          raise exception_with_response
         end
-      elsif code == 303
+      when 303
+        check_max_redirects
         follow_get_redirection(&block)
       else
         raise exception_with_response
@@ -96,13 +116,13 @@ module RestClient
     # Follow a redirection response by making a new HTTP request to the
     # redirection target.
     def follow_redirection(&block)
-      _follow_redirection(@args.dup, &block)
+      _follow_redirection(request.args.dup, &block)
     end
 
     # Follow a redirection response, but change the HTTP method to GET and drop
     # the payload from the original request.
     def follow_get_redirection(&block)
-      new_args = @args.dup
+      new_args = request.args.dup
       new_args[:method] = :get
       new_args.delete(:payload)
 
@@ -132,7 +152,7 @@ module RestClient
     #
     def self.beautify_headers(headers)
       headers.inject({}) do |out, (key, value)|
-        key_sym = key.gsub(/-/, '_').downcase.to_sym
+        key_sym = key.tr('-', '_').downcase.to_sym
 
         # Handle Set-Cookie specially since it cannot be joined by comma.
         if key.downcase == 'set-cookie'
@@ -158,24 +178,24 @@ module RestClient
       # parse location header and merge into existing URL
       url = headers[:location]
 
+      # cannot follow redirection if there is no location header
+      unless url
+        raise exception_with_response
+      end
+
       # handle relative redirects
       unless url.start_with?('http')
         url = URI.parse(request.url).merge(url).to_s
       end
       new_args[:url] = url
 
-      if request.max_redirects <= 0
-        raise exception_with_response
-      end
       new_args[:password] = request.password
       new_args[:user] = request.user
       new_args[:headers] = request.headers
       new_args[:max_redirects] = request.max_redirects - 1
 
-      # TODO: figure out what to do with original :cookie, :cookies values
-      new_args[:headers]['Cookie'] = HTTP::Cookie.cookie_value(
-        cookie_jar.cookies(new_args.fetch(:url)))
-
+      # pass through our new cookie jar
+      new_args[:cookies] = cookie_jar
 
       # prepare new request
       new_req = Request.new(new_args)
@@ -187,6 +207,12 @@ module RestClient
       new_req.execute(&block)
     end
 
+    def check_max_redirects
+      if request.max_redirects <= 0
+        raise exception_with_response
+      end
+    end
+
     def exception_with_response
       begin
         klass = Exceptions::EXCEPTIONS_MAP.fetch(code)

data/lib/restclient/exceptions.rb

@@ -1,5 +1,19 @@
 module RestClient
 
+  # Hash of HTTP status code => message.
+  #
+  # 1xx: Informational - Request received, continuing process
+  # 2xx: Success - The action was successfully received, understood, and
+  #      accepted
+  # 3xx: Redirection - Further action must be taken in order to complete the
+  #      request
+  # 4xx: Client Error - The request contains bad syntax or cannot be fulfilled
+  # 5xx: Server Error - The server failed to fulfill an apparently valid
+  #      request
+  #
+  # @see
+  #   http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
+  #
   STATUSES = {100 => 'Continue',
               101 => 'Switching Protocols',
               102 => 'Processing', #WebDAV
@@ -12,6 +26,8 @@ module RestClient
               205 => 'Reset Content',
               206 => 'Partial Content',
               207 => 'Multi-Status', #WebDAV
+              208 => 'Already Reported', # RFC5842
+              226 => 'IM Used', # RFC3229
 
               300 => 'Multiple Choices',
               301 => 'Moved Permanently',
@@ -21,6 +37,7 @@ module RestClient
               305 => 'Use Proxy', # http/1.1
               306 => 'Switch Proxy', # no longer used
               307 => 'Temporary Redirect', # http/1.1
+              308 => 'Permanent Redirect', # RFC7538
 
               400 => 'Bad Request',
               401 => 'Unauthorized',
@@ -35,10 +52,10 @@ module RestClient
               410 => 'Gone',
               411 => 'Length Required',
               412 => 'Precondition Failed',
-              413 => 'Request Entity Too Large',
-              414 => 'Request-URI Too Long',
+              413 => 'Payload Too Large', # RFC7231 (renamed, see below)
+              414 => 'URI Too Long', # RFC7231 (renamed, see below)
               415 => 'Unsupported Media Type',
-              416 => 'Requested Range Not Satisfiable',
+              416 => 'Range Not Satisfiable', # RFC7233 (renamed, see below)
               417 => 'Expectation Failed',
               418 => 'I\'m A Teapot', #RFC2324
               421 => 'Too Many Connections From This IP',
@@ -61,11 +78,28 @@ module RestClient
               505 => 'HTTP Version Not Supported',
               506 => 'Variant Also Negotiates',
               507 => 'Insufficient Storage', #WebDAV
+              508 => 'Loop Detected', # RFC5842
               509 => 'Bandwidth Limit Exceeded', #Apache
               510 => 'Not Extended',
               511 => 'Network Authentication Required', # RFC6585
   }
 
+  STATUSES_COMPATIBILITY = {
+    # The RFCs all specify "Not Found", but "Resource Not Found" was used in
+    # earlier RestClient releases.
+    404 => ['ResourceNotFound'],
+
+    # HTTP 413 was renamed to "Payload Too Large" in RFC7231.
+    413 => ['RequestEntityTooLarge'],
+
+    # HTTP 414 was renamed to "URI Too Long" in RFC7231.
+    414 => ['RequestURITooLong'],
+
+    # HTTP 416 was renamed to "Range Not Satisfiable" in RFC7233.
+    416 => ['RequestedRangeNotSatisfiable'],
+  }
+
+
   # This is the base RestClient exception class. Rescue it if you want to
   # catch any exception that your request might raise
   # You can get the status code by e.http_code, or see anything about the
@@ -148,8 +182,13 @@ module RestClient
     Exceptions::EXCEPTIONS_MAP[code] = klass_constant
   end
 
-  # Backwards compatibility. "Not Found" is the actual text in the RFCs.
-  ResourceNotFound = NotFound
+  # Create HTTP status exception classes used for backwards compatibility
+  STATUSES_COMPATIBILITY.each_pair do |code, compat_list|
+    klass = Exceptions::EXCEPTIONS_MAP.fetch(code)
+    compat_list.each do |old_name|
+      const_set(old_name, klass)
+    end
+  end
 
   module Exceptions
     # We have to split the Exceptions module like we do here because the
@@ -197,7 +236,7 @@ module RestClient
   end
 
   class SSLCertificateNotVerified < Exception
-    def initialize(message)
+    def initialize(message = 'SSL certificate not verified')
       super nil, nil
       self.message = message
     end

data/lib/restclient/params_array.rb

@@ -0,0 +1,72 @@
+module RestClient
+
+  # The ParamsArray class is used to represent an ordered list of [key, value]
+  # pairs. Use this when you need to include a key multiple times or want
+  # explicit control over parameter ordering.
+  #
+  # Most of the request payload & parameter functions normally accept a Hash of
+  # keys => values, which does not allow for duplicated keys.
+  #
+  # @see RestClient::Utils.encode_query_string
+  # @see RestClient::Utils.flatten_params
+  #
+  class ParamsArray
+    include Enumerable
+
+    # @param array [Array<Array>] An array of parameter key,value pairs. These
+    #   pairs may be 2 element arrays [key, value] or single element hashes
+    #   {key => value}. They may also be single element arrays to represent a
+    #   key with no value.
+    #
+    # @example
+    #   >> ParamsArray.new([[:foo, 123], [:foo, 456], [:bar, 789]])
+    #   This will be encoded as "foo=123&foo=456&bar=789"
+    #
+    # @example
+    #   >> ParamsArray.new({foo: 123, bar: 456})
+    #   This is valid, but there's no reason not to just use the Hash directly
+    #   instead of a ParamsArray.
+    #
+    #
+    def initialize(array)
+      @array = process_input(array)
+    end
+
+    def each(*args, &blk)
+      @array.each(*args, &blk)
+    end
+
+    def empty?
+      @array.empty?
+    end
+
+    private
+
+    def process_input(array)
+      array.map {|v| process_pair(v) }
+    end
+
+    # A pair may be:
+    # - A single element hash, e.g. {foo: 'bar'}
+    # - A two element array, e.g. ['foo', 'bar']
+    # - A one element array, e.g. ['foo']
+    #
+    def process_pair(pair)
+      case pair
+      when Hash
+        if pair.length != 1
+          raise ArgumentError.new("Bad # of fields for pair: #{pair.inspect}")
+        end
+        pair.to_a.fetch(0)
+      when Array
+        if pair.length > 2
+          raise ArgumentError.new("Bad # of fields for pair: #{pair.inspect}")
+        end
+        [pair.fetch(0), pair[1]]
+      else
+        # recurse, converting any non-array to an array
+        process_pair(pair.to_a)
+      end
+    end
+  end
+end