httpclient 2.2.7 → 2.3.0

data/README.txt

@@ -12,6 +12,7 @@ See HTTPClient for documentation.
 * methods like GET/HEAD/POST/* via HTTP/1.1.
 * HTTPS(SSL), Cookies, proxy, authentication(Digest, NTLM, Basic), etc.
 * asynchronous HTTP request, streaming HTTP request.
+* debug mode CLI.
 
 * by contrast with net/http in standard distribution;
   * Cookies support
@@ -29,6 +30,16 @@ See HTTPClient for documentation.
   * Rather advanced HTTP/1.1 usage such as Range, deflate, etc.
     (of course you can set it in header by yourself)
 
+== httpclient command
+
+Usage: 1) % httpclient get https://www.google.co.jp/ q=ruby
+Usage: 2) % httpclient
+
+For 1) it issues a GET request to the given URI and shows the wiredump and
+the parsed result.  For 2) it invokes irb shell with the binding that has a
+HTTPClient as 'self'.  You can call HTTPClient instance methods like;
+
+  > get "https://www.google.co.jp/", :q => :ruby
 
 == Author
 
@@ -43,10 +54,11 @@ This program is copyrighted free software by NAKAMURA, Hiroshi.  You can
 redistribute it and/or modify it under the same terms of Ruby's license;
 either the dual license version in 2003, or any later version.
 
-httpclient/session.rb is based on http-access.rb in http-access/0.0.4.
-Some part of code in http-access.rb was recycled in http-access2.rb.
-Those part is copyrighted by Maehashi-san who made and distributed
-http-access/0.0.4. Many thanks to Maehashi-san.
+httpclient/session.rb is based on http-access.rb in http-access/0.0.4.  Some
+part of it is copyrighted by Maebashi-san who made and published
+http-access/0.0.4.  http-access/0.0.4 did not include license notice but when
+I asked Maebashi-san he agreed that I can redistribute it under the same terms
+of Ruby.  Many thanks to Maebashi-san.
 
 
 == Install
@@ -95,6 +107,37 @@ Thanks in advance.
 
 == Changes
 
+= Changes in 2.3.0 =
+
+  October 10, 2012 - version 2.3.0
+
+    * Features
+
+      * Added debug mode CLI.  bin/httpclient is installed as CLI.
+          Usage: 1) % httpclient get https://www.google.co.jp/ q=ruby
+          Usage: 2) %httpclient
+        For 1) it issues a GET request to the given URI and shows the wiredump
+        and the parsed result.  For 2) it invokes irb shell with the binding
+        that has a HTTPClient as 'self'.  You can call HTTPClient instance
+        methods like;
+          > get "https://www.google.co.jp/", :q => :ruby
+
+      * #119 Addressable gem support (only if it exists); should handle IRI
+        properly.
+
+    * Bug fixes
+
+      * #115 Cookies couldn't work properly if the path in an URI is ommited.
+      * #112, #117 Proper handling of sized IO (the IO object that responds to
+        :size) for chunked POST. HTTPClient did read till EOF even if the
+        given IO has :size method.
+      * Handle '303 See Other' properly.  RFC2616 says it should be redirected
+        with GET.
+      * #116 Fix "100-continue" support.  It was just ignored.
+      * #118 Support for boolean values when making POST/PUT requests with
+        multiipart/form Content-Type.
+      * #110 Allows leading dots in no_proxy hostname suffixes.
+
 = Changes in 2.2.7 =
 
   August 14, 2012 - version 2.2.7
@@ -102,7 +145,7 @@ Thanks in advance.
     * Bug fixes
 
       * Fix arity incompatibility introduced in 2.2.6.  It broke Webmock.
-	Thanks Andrew France for the report!
+        Thanks Andrew France for the report!
 
 = Changes in 2.2.6 =
 
@@ -111,29 +154,29 @@ Thanks in advance.
     * Bug fixes
 
       * Make get_content doesn't raise a BadResponseError for perfectly good
-	responses like 304 Not Modified. Thanks to Florian Hars.
+        responses like 304 Not Modified. Thanks to Florian Hars.
 
       * Add 'Content-Type: application/x-www-form-urlencoded' for the PUT
-	request that has urlencoded entity-body.
+        request that has urlencoded entity-body.
 
     * Features
 
       * Add HTTPClient::IncludeClient by Jonathan Rochkind, a mix-in for easily
-	adding a thread-safe lazily initialized class-level HTTPClient object
-	to your class.
+        adding a thread-safe lazily initialized class-level HTTPClient object
+        to your class.
 
       * Proxy DigestAuth support. Thanks to Alexander Kotov and Florian Hars.
 
       * Accept an array of strings (and IO-likes) as a query value
-	e.g. `{ x: 'a', y: [1,2,3] }` is encoded into `"x=a&y=1&y=2&y=3"`.
-	Thanks to Akinori MUSHA.
+        e.g. `{ x: 'a', y: [1,2,3] }` is encoded into `"x=a&y=1&y=2&y=3"`.
+        Thanks to Akinori MUSHA.
 
       * Allow body for DELETE method.
 
       * Allow :follow_redirect => true for HEAD request.
 
       * Fill request parameters request_method, request_uri and request_query
-	as part of response Message::Header.
+        as part of response Message::Header.
 
 = Changes in 2.2.5 =
 
@@ -143,8 +186,8 @@ Thanks in advance.
     
       * Added Magic encoding comment to hexdump.rb to avoid encoding error.
       * Add workaround for JRuby issue on Windows (JRUBY-6136)
-	On Windows, calling File#size fails with an Unknown error (20047).
-	This workaround uses File#lstat instead.
+        On Windows, calling File#size fails with an Unknown error (20047).
+        This workaround uses File#lstat instead.
       * Require open-uri only on ruby 1.9, since it is not needed on 1.8.
 
     * Features
@@ -153,11 +196,11 @@ Thanks in advance.
       * Dump more SSL certificate information under $DEBUG.
       * Add HTTPClient::SSLConfig#ssl_version property.
       * Add 'Accept: */*' header to request by default. Rails requies it.
-	It doesn't override given Accept header from API.
+        It doesn't override given Accept header from API.
       * Add HTTPClient::SSLConfig#set_default_paths. This method makes
-	HTTPClient instance to use OpenSSL's default trusted CA certificates.
+        HTTPClient instance to use OpenSSL's default trusted CA certificates.
       * Allow to set Date header manually.
-	ex. clent.get(uri, :header => {'Date' => Time.now.httpdate})
+        ex. clent.get(uri, :header => {'Date' => Time.now.httpdate})
 
 = Changes in 2.2.4 =
 
@@ -665,7 +708,7 @@ Thanks in advance.
       [,:] separated. ("ruby-lang.org:rubyist.net")
       No regexp. (give "ruby-lang.org", not "*.ruby-lang.org")
       If you want specify hot by IP address, give full address.
-	("192.168.1.1, 192.168.1.2")
+        ("192.168.1.1, 192.168.1.2")
 
   September 10, 2003 - version 2.0
     CamelCase to non_camel_case.

data/bin/httpclient

@@ -0,0 +1,65 @@
+#!/usr/bin/env ruby
+
+# httpclient shell command.
+#
+# Usage: 1) % httpclient get https://www.google.co.jp/ q=ruby
+# Usage: 2) % httpclient
+#
+# For 1) it issues a GET request to the given URI and shows the wiredump and
+# the parsed result.  For 2) it invokes irb shell with the binding that has a
+# HTTPClient as 'self'.  You can call HTTPClient instance methods like;
+# > get "https://www.google.co.jp/", :q => :ruby
+require 'httpclient'
+
+METHODS = ['head', 'get', 'post', 'put', 'delete', 'options', 'propfind', 'proppatch', 'trace']
+if ARGV.size >= 2 && METHODS.include?(ARGV[0])
+  client = HTTPClient.new
+  client.debug_dev = STDERR
+  $DEBUG = true
+  require 'pp'
+  pp client.send(*ARGV)
+  exit
+end
+
+require 'irb'
+require 'irb/completion'
+
+class Runner
+  def initialize
+    @httpclient = HTTPClient.new
+  end
+
+  def method_missing(msg, *a, &b)
+    debug, $DEBUG = $DEBUG, true
+    begin
+      @httpclient.send(msg, *a, &b)
+    ensure
+      $DEBUG = debug
+    end
+  end
+
+  def run
+    IRB.setup(nil)
+    ws = IRB::WorkSpace.new(binding)
+    irb = IRB::Irb.new(ws)
+    IRB.conf[:MAIN_CONTEXT] = irb.context
+
+    trap("SIGINT") do
+      irb.signal_handle
+    end
+
+    begin
+      catch(:IRB_EXIT) do
+        irb.eval_input
+      end
+    ensure
+      IRB.irb_at_exit
+    end
+  end
+
+  def to_s
+    'HTTPClient'
+  end
+end
+
+Runner.new.run

data/lib/http-access2.rb

@@ -5,9 +5,11 @@
 # redistribute it and/or modify it under the same terms of Ruby's license;
 # either the dual license version in 2003, or any later version.
 
-# http-access2.rb is based on http-access.rb in http-access/0.0.4.  Some part
-# of code in http-access.rb was recycled in http-access2.rb.  Those part is
-# copyrighted by Maehashi-san.
+# http-access2.rb is based on http-access.rb in http-access/0.0.4.  Some
+# part of it is copyrighted by Maebashi-san who made and published
+# http-access/0.0.4.  http-access/0.0.4 did not include license notice but
+# when I asked Maebashi-san he agreed that I can redistribute it under the
+# same terms of Ruby.  Many thanks to Maebashi-san.
 
 
 require 'httpclient'

data/lib/httpclient.rb

@@ -6,7 +6,6 @@
 # either the dual license version in 2003, or any later version.
 
 
-require 'uri'
 require 'stringio'
 require 'digest/sha1'
 
@@ -378,6 +377,7 @@ class HTTPClient
     proxy, agent_name, from = keyword_argument(args, :proxy, :agent_name, :from)
     @proxy = nil        # assigned later.
     @no_proxy = nil
+    @no_proxy_regexps = []
     @www_auth = WWWAuth.new
     @proxy_auth = ProxyAuth.new
     @request_filter = [@proxy_auth, @www_auth]
@@ -478,6 +478,17 @@ class HTTPClient
   # Calling this method resets all existing sessions.
   def no_proxy=(no_proxy)
     @no_proxy = no_proxy
+    @no_proxy_regexps.clear
+    if @no_proxy
+      @no_proxy.scan(/([^:,]+)(?::(\d+))?/) do |host, port|
+        if host[0] == ?.
+          regexp = /#{Regexp.quote(host)}\z/i
+        else
+          regexp = /(\A|\.)#{Regexp.quote(host)}\z/i
+        end
+        @no_proxy_regexps << [regexp, port]
+      end
+    end
     reset_all
   end
 
@@ -623,11 +634,11 @@ class HTTPClient
   #   Location: ../foo/
   # in HTTP header. (raises BadResponseError instead)
   def strict_redirect_uri_callback(uri, res)
-    newuri = URI.parse(res.header['location'][0])
+    newuri = urify(res.header['location'][0])
     if https?(uri) && !https?(newuri)
       raise BadResponseError.new("redirecting to non-https resource")
     end
-    unless newuri.is_a?(URI::HTTP)
+    if !http?(newuri) && !https?(newuri)
       raise BadResponseError.new("unexpected location: #{newuri}", res)
     end
     puts "redirect to: #{newuri}" if $DEBUG
@@ -640,8 +651,8 @@ class HTTPClient
   #   Location: ../foo/
   # in HTTP header.
   def default_redirect_uri_callback(uri, res)
-    newuri = URI.parse(res.header['location'][0])
-    unless newuri.is_a?(URI::HTTP)
+    newuri = urify(res.header['location'][0])
+    if !http?(newuri) && !https?(newuri)
       newuri = uri + newuri
       warn("could be a relative URI in location header which is not recommended")
       warn("'The field value consists of a single absolute URI' in HTTP spec")
@@ -933,7 +944,7 @@ private
     uri = urify(uri)
     if block
       filtered_block = proc { |r, str|
-        block.call(str) if HTTP::Status.successful?(r.status)
+        block.call(str) if r.ok?
       }
     end
     if HTTP::Message.file?(body)
@@ -943,7 +954,8 @@ private
     while retry_number < @follow_redirect_count
       body.pos = pos if pos
       res = do_request(method, uri, query, body, header, &filtered_block)
-      if HTTP::Status.redirect?(res.status)
+      if res.redirect?
+        method = :get if res.see_other? # See RFC2616 10.3.4
         uri = urify(@redirect_uri_callback.call(uri, res))
         retry_number += 1
       else
@@ -954,7 +966,7 @@ private
   end
 
   def success_content(res)
-    if HTTP::Status.successful?(res.status)
+    if res.ok?
       return res.content
     else
       raise BadResponseError.new("unexpected response: #{res.header.inspect}", res)
@@ -1041,13 +1053,11 @@ private
     if !@proxy or NO_PROXY_HOSTS.include?(uri.host)
       return true
     end
-    unless @no_proxy
-      return false
-    end
-    @no_proxy.scan(/([^:,]+)(?::(\d+))?/) do |host, port|
-      if /(\A|\.)#{Regexp.quote(host)}\z/i =~ uri.host &&
-          (!port || uri.port == port.to_i)
-        return true
+    @no_proxy_regexps.each do |regexp, port|
+      if !port || uri.port == port.to_i
+        if regexp =~ uri.host
+          return true
+        end
       end
     end
     false

data/lib/httpclient/cookie.rb

@@ -11,9 +11,9 @@
 #
 #    w3m homepage:  http://ei5nazha.yz.yamagata-u.ac.jp/~aito/w3m/eng/
 
-require 'uri'
 require 'time'
 require 'monitor'
+require 'httpclient/util'
 
 class WebAgent
 
@@ -133,7 +133,7 @@ class WebAgent
       domainname = url.host
       if (!domainname ||
 	  !domain_match(domainname, @domain) ||
-	  (@path && !head_match?(@path, url.path)) ||
+	  (@path && !head_match?(@path, url.path.empty? ? '/' : url.path)) ||
 	  (@secure && (url.scheme != 'https')) )
 	return false
       else
@@ -421,7 +421,7 @@ class WebAgent
             cookie = WebAgent::Cookie.new()
             @cookies << cookie
             col = line.chomp.split(/\t/)
-            cookie.url = URI.parse(col[0])
+            cookie.url = HTTPClient::Util.urify(col[0])
             cookie.name = col[1]
             cookie.value = col[2]
             if col[3].empty? or col[3] == '0'

data/lib/httpclient/http.rb

@@ -188,7 +188,7 @@ module HTTP
       end
 
       # Placeholder URI object for nil uri.
-      NIL_URI = URI.parse('http://nil-uri-given/')
+      NIL_URI = HTTPClient::Util.urify('http://nil-uri-given/')
       # Initialize this instance as a general request.
       def init_request(method, uri, query = nil)
         @is_request = true
@@ -542,7 +542,7 @@ module HTTP
           # bear in mind that server may not support it. at least ruby's CGI doesn't.
           @body = body
           remember_pos(@body)
-          @size = body.respond_to?(:size) ? body.size - body.pos : nil
+          @size = (body.size - body.pos) rescue body.size if body.respond_to?(:size)
         elsif boundary and Message.multiparam_query?(body)
           @body = build_query_multipart_str(body, boundary)
           @size = @body.size
@@ -554,27 +554,45 @@ module HTTP
 
       def remember_pos(io)
         # IO may not support it (ex. IO.pipe)
-        @positions[io] = io.pos if io.respond_to?(:pos)
+        @positions[io] = io.pos rescue nil if io.respond_to?(:pos)
       end
 
       def reset_pos(io)
-        io.pos = @positions[io] if @positions.key?(io)
+        io.pos = @positions[io] if @positions[io]
       end
 
       def dump_file(io, dev)
+        return dump_file_length(io, @size, dev) if @size
         buf = ''
         while !io.read(@chunk_size, buf).nil?
           dev << buf
         end
       end
 
+      def dump_file_length(io, size, dev)
+        buf = ''
+        while size > 0 && !(read = io.read(size > @chunk_size ? @chunk_size : size, buf)).nil?
+          dev << buf
+          size -= read.bytesize
+        end
+      end
+
       def dump_chunks(io, dev)
+        return dump_chunks_length(io, io.size, dev) if io.respond_to?(:size) && io.size
         buf = ''
         while !io.read(@chunk_size, buf).nil?
           dev << dump_chunk(buf)
         end
       end
 
+      def dump_chunks_length(io, size, dev)
+        buf = ''
+        while size > 0 && !(read = io.read(size > @chunk_size ? @chunk_size : size, buf)).nil?
+          dev << dump_chunk(buf)
+          size -= read.bytesize
+        end
+      end
+
       def dump_chunk(str)
         dump_chunk_size(str.bytesize) + (str + CRLF)
       end
@@ -600,18 +618,8 @@ module HTTP
           if Message.file?(part)
             @as_stream = true
             @body << part
-            if part.respond_to?(:lstat)
-              @size += part.lstat.size
-            elsif part.respond_to?(:size)
-              if sz = part.size
-                @size += sz
-              else
-                @size = nil
-              end
-            else
-              # use chunked upload
-              @size = nil
-            end
+            # use chunked upload
+            @size = nil
           elsif @body[-1].is_a?(String)
             @body[-1] += part.to_s
             @size += part.to_s.bytesize if @size
@@ -633,7 +641,6 @@ module HTTP
       def build_query_multipart_str(query, boundary)
         parts = Parts.new
         query.each do |attr, value|
-          value ||= ''
           headers = ["--#{boundary}"]
           if Message.file?(value)
             remember_pos(value)
@@ -659,6 +666,7 @@ module HTTP
             remember_pos(value) if Message.file?(value)
           else
             headers << %{Content-Disposition: form-data; name="#{attr}"}
+            value = value.to_s
           end
           parts.add(headers.join(CRLF) + CRLF + CRLF)
           parts.add(value)
@@ -1027,7 +1035,20 @@ module HTTP
         }
       end
     end
-  end
 
+    # Convenience method to return boolean of whether we had a successful request
+    def ok?
+      HTTP::Status.successful?(status)
+    end
+
+    def redirect?
+      HTTP::Status.redirect?(status)
+    end
+
+    # SEE_OTHER is a redirect, but it should sent as GET
+    def see_other?
+      status == HTTP::Status::SEE_OTHER
+    end
+  end
 
 end

data/lib/httpclient/session.rb

@@ -4,10 +4,12 @@
 # This program is copyrighted free software by NAKAMURA, Hiroshi.  You can
 # redistribute it and/or modify it under the same terms of Ruby's license;
 # either the dual license version in 2003, or any later version.
-
-# httpclient/session.rb is based on http-access.rb in http-access/0.0.4.
-# Some part of code in http-access.rb was recycled in httpclient.rb.
-# Those part is copyrighted by Maehashi-san.
+#
+# httpclient/session.rb is based on http-access.rb in http-access/0.0.4.  Some
+# part of it is copyrighted by Maebashi-san who made and published
+# http-access/0.0.4.  http-access/0.0.4 did not include license notice but when
+# I asked Maebashi-san he agreed that I can redistribute it under the same terms
+# of Ruby.  Many thanks to Maebashi-san.
 
 
 require 'socket'
@@ -749,10 +751,10 @@ class HTTPClient
           @socket = create_socket(site)
           if https?(@dest)
             if @socket.is_a?(LoopBackSocket)
-              connect_ssl_proxy(@socket, URI.parse(@dest.to_s)) if @proxy
+              connect_ssl_proxy(@socket, urify(@dest.to_s)) if @proxy
             else
               @socket = create_ssl_socket(@socket)
-              connect_ssl_proxy(@socket, URI.parse(@dest.to_s)) if @proxy
+              connect_ssl_proxy(@socket, urify(@dest.to_s)) if @proxy
               begin
                 @socket.ssl_connect(@dest.host)
               ensure
@@ -872,17 +874,17 @@ class HTTPClient
       timeout(@receive_timeout, ReceiveTimeoutError) do
         initial_line = nil
         begin
-          initial_line = @socket.gets("\n")
-          if initial_line.nil?
+          begin
+            initial_line = @socket.gets("\n")
+            if initial_line.nil?
+              close
+              raise KeepAliveDisconnected.new(self)
+            end
+          rescue Errno::ECONNABORTED, Errno::ECONNRESET, Errno::EPIPE, IOError
+            # JRuby can raise IOError instead of ECONNRESET for now
             close
             raise KeepAliveDisconnected.new(self)
           end
-        rescue Errno::ECONNABORTED, Errno::ECONNRESET, Errno::EPIPE, IOError
-          # JRuby can raise IOError instead of ECONNRESET for now
-          close
-          raise KeepAliveDisconnected.new(self)
-        end
-        begin
           if StatusParseRegexp !~ initial_line
             @version = '0.9'
             @status = nil

data/lib/httpclient/ssl_config.rb

@@ -82,8 +82,7 @@ class HTTPClient
       @verify_callback = nil
       @dest = nil
       @timeout = nil
-      # TODO: change to "SSLv3" in future versions to make harder to use SSLv2.
-      @ssl_version = "SSLv23"
+      @ssl_version = "SSLv3"
       @options = defined?(SSL::OP_ALL) ? SSL::OP_ALL | SSL::OP_NO_SSLv2 : nil
       # OpenSSL 0.9.8 default: "ALL:!ADH:!LOW:!EXP:!MD5:+SSLv2:@STRENGTH"
       @ciphers = "ALL:!aNULL:!eNULL:!SSLv2" # OpenSSL >1.0.0 default

data/lib/httpclient/util.rb

@@ -1,14 +1,11 @@
 # HTTPClient - HTTP client library.
-# Copyright (C) 2000-2009  NAKAMURA, Hiroshi  <nahi@ruby-lang.org>.
+# Copyright (C) 2000-2012  NAKAMURA, Hiroshi  <nahi@ruby-lang.org>.
 #
 # This program is copyrighted free software by NAKAMURA, Hiroshi.  You can
 # redistribute it and/or modify it under the same terms of Ruby's license;
 # either the dual license version in 2003, or any later version.
 
 
-require 'uri'
-
-
 unless ''.respond_to?(:bytesize)
   class String
     alias bytesize size
@@ -21,6 +18,37 @@ class HTTPClient
 
   # A module for common function.
   module Util
+
+    # URI abstraction; Addressable::URI or URI
+    require 'uri'
+    begin
+      require 'addressable/uri'
+      class AddressableURI < Addressable::URI
+        # Overwrites the original definition just for one line...
+        def authority
+          self.host && @authority ||= (begin
+            authority = ""
+            if self.userinfo != nil
+              authority << "#{self.userinfo}@"
+            end
+            authority << self.host
+            if self.port != self.default_port # ...HERE! Compares with default_port because self.port is not nil in this wrapper.
+              authority << ":#{self.port}"
+            end
+            authority
+          end)
+        end
+
+        # HTTPClient expects urify("http://foo/").port to be not nil but 80 like URI.
+        def port
+          super || default_port
+        end
+      end
+      AddressableEnabled = true
+    rescue LoadError
+      AddressableEnabled = false
+    end
+
     # Keyword argument helper.
     # args:: given arguments.
     # *field:: a list of arguments to be extracted.
@@ -79,10 +107,13 @@ class HTTPClient
         nil
       elsif uri.is_a?(URI)
         uri
+      elsif AddressableEnabled
+        AddressableURI.parse(uri.to_s)
       else
         URI.parse(uri.to_s)
       end
     end
+    module_function :urify
 
     # Returns true if the given 2 URIs have a part_of relationship.
     # * the same scheme
@@ -114,7 +145,11 @@ class HTTPClient
 
     # Checks if the given URI is https.
     def https?(uri)
-      uri.scheme.downcase == 'https'
+      uri.scheme && uri.scheme.downcase == 'https'
+    end
+
+    def http?(uri)
+      uri.scheme && uri.scheme.downcase == 'http'
     end
   end
 

data/lib/httpclient/version.rb

@@ -1,3 +1,3 @@
 class HTTPClient
-  VERSION = '2.2.7'
+  VERSION = '2.3.0'
 end

data/test/test_cookie.rb

@@ -4,6 +4,7 @@ require 'uri'
 require 'httpclient/cookie'
 
 class TestCookie < Test::Unit::TestCase
+  include HTTPClient::Util
 
   def setup()
     @c = WebAgent::Cookie.new()
@@ -20,7 +21,7 @@ class TestCookie < Test::Unit::TestCase
   end
 
   def test_match()
-    url = URI.parse('http://www.rubycolor.org/hoge/funi/#919191')
+    url = urify('http://www.rubycolor.org/hoge/funi/#919191')
 
     @c.domain = 'www.rubycolor.org'
     assert_equal(true, @c.match?(url))
@@ -51,7 +52,7 @@ class TestCookie < Test::Unit::TestCase
     @c.secure = true
     assert_equal(false, @c.match?(url))
 
-    url2 = URI.parse('https://www.rubycolor.org/hoge/funi/#919191')
+    url2 = urify('https://www.rubycolor.org/hoge/funi/#919191')
     @c.domain = 'www.rubycolor.org'
     @c.path = '/hoge'
     @c.secure = true
@@ -68,6 +69,11 @@ class TestCookie < Test::Unit::TestCase
 #    @c.port = [80,8080]
     assert_equal(true, @c.match?(url))
 
+    url_nopath = URI.parse('http://www.rubycolor.org')
+    @c.domain = 'www.rubycolor.org'
+    @c.path = '/'
+    assert_equal(true, @c.match?(url_nopath))
+
   end
 
   def test_head_match?()
@@ -123,6 +129,7 @@ class TestCookie < Test::Unit::TestCase
 end
 
 class TestCookieManager < Test::Unit::TestCase
+  include HTTPClient::Util
 
   def setup()
     @cm = WebAgent::CookieManager.new()
@@ -147,7 +154,7 @@ class TestCookieManager < Test::Unit::TestCase
   
   def test_parse()
     str = "inkid=n92b0ADOgACIgUb9lsjHqAAAHu2a; expires=Wed, 01-Dec-2010 00:00:00 GMT; path=/"
-    @cm.parse(str,URI.parse('http://www.test.jp'))
+    @cm.parse(str, urify('http://www.test.jp'))
     cookie = @cm.cookies[0]
     assert_instance_of(WebAgent::Cookie, cookie)
     assert_equal("inkid", cookie.name)
@@ -158,7 +165,7 @@ class TestCookieManager < Test::Unit::TestCase
 
   def test_parse2()
     str = "xmen=off,0,0,1; path=/; domain=.excite.co.jp; expires=Wednesday, 31-Dec-2037 12:00:00 GMT"
-    @cm.parse(str,URI.parse('http://www.excite.co.jp'))
+    @cm.parse(str, urify('http://www.excite.co.jp'))
     cookie = @cm.cookies[0]
     assert_instance_of(WebAgent::Cookie, cookie)
     assert_equal("xmen", cookie.name)
@@ -170,7 +177,7 @@ class TestCookieManager < Test::Unit::TestCase
 
   def test_parse3()
     str = "xmen=off,0,0,1; path=/; domain=.excite.co.jp; expires=Wednesday, 31-Dec-2037 12:00:00 GMT;Secure;HTTPOnly"
-    @cm.parse(str,URI.parse('http://www.excite.co.jp'))
+    @cm.parse(str, urify('http://www.excite.co.jp'))
     cookie = @cm.cookies[0]
     assert_instance_of(WebAgent::Cookie, cookie)
     assert_equal("xmen", cookie.name)
@@ -184,7 +191,7 @@ class TestCookieManager < Test::Unit::TestCase
   
   def test_parse_double_semicolon()
     str = "xmen=off,0,0,1;; path=\"/;;\"; domain=.excite.co.jp; expires=Wednesday, 31-Dec-2037 12:00:00 GMT"
-    @cm.parse(str,URI.parse('http://www.excite.co.jp'))
+    @cm.parse(str, urify('http://www.excite.co.jp'))
     cookie = @cm.cookies[0]
     assert_instance_of(WebAgent::Cookie, cookie)
     assert_equal("xmen", cookie.name)
@@ -218,7 +225,7 @@ class TestCookieManager < Test::Unit::TestCase
 
   def test_parse_expires
     str = "inkid=n92b0ADOgACIgUb9lsjHqAAAHu2a; expires=; path=/"
-    @cm.parse(str,URI.parse('http://www.test.jp'))
+    @cm.parse(str, urify('http://www.test.jp'))
     cookie = @cm.cookies[0]
     assert_equal("inkid", cookie.name)
     assert_equal("n92b0ADOgACIgUb9lsjHqAAAHu2a", cookie.value)
@@ -226,7 +233,7 @@ class TestCookieManager < Test::Unit::TestCase
     assert_equal("/", cookie.path)
     #
     str = "inkid=n92b0ADOgACIgUb9lsjHqAAAHu2a; path=/; expires="
-    @cm.parse(str,URI.parse('http://www.test.jp'))
+    @cm.parse(str, urify('http://www.test.jp'))
     cookie = @cm.cookies[0]
     assert_equal("inkid", cookie.name)
     assert_equal("n92b0ADOgACIgUb9lsjHqAAAHu2a", cookie.value)
@@ -234,7 +241,7 @@ class TestCookieManager < Test::Unit::TestCase
     assert_equal("/", cookie.path)
     #
     str = "inkid=n92b0ADOgACIgUb9lsjHqAAAHu2a; path=/; expires=\"\""
-    @cm.parse(str,URI.parse('http://www.test.jp'))
+    @cm.parse(str, urify('http://www.test.jp'))
     cookie = @cm.cookies[0]
     assert_equal("inkid", cookie.name)
     assert_equal("n92b0ADOgACIgUb9lsjHqAAAHu2a", cookie.value)
@@ -244,15 +251,15 @@ class TestCookieManager < Test::Unit::TestCase
 
   def test_find_cookie()
     str = "xmen=off,0,0,1; path=/; domain=.excite2.co.jp; expires=Wednesday, 31-Dec-2037 12:00:00 GMT"
-    @cm.parse(str, URI.parse("http://www.excite2.co.jp/"))
+    @cm.parse(str, urify("http://www.excite2.co.jp/"))
 
     str = "xmen=off,0,0,2; path=/; domain=.excite.co.jp; expires=Wednesday, 31-Dec-2037 12:00:00 GMT"
-    @cm.parse(str, URI.parse("http://www.excite.co.jp/"))
+    @cm.parse(str, urify("http://www.excite.co.jp/"))
 
     @cm.cookies[0].use = true
     @cm.cookies[1].use = true
 
-    url = URI.parse('http://www.excite.co.jp/hoge/funi/')
+    url = urify('http://www.excite.co.jp/hoge/funi/')
     cookie_str = @cm.find(url)
     assert_equal("xmen=off,0,0,2", cookie_str)
   end
@@ -331,7 +338,7 @@ EOF
   def test_not_saved_expired_cookies
     begin
       @cm.cookies_file = 'tmp_test.tmp'
-      uri = URI.parse('http://www.example.org')
+      uri = urify('http://www.example.org')
       @cm.parse("foo=1; path=/", uri)
       @cm.parse("bar=2; path=/; expires=", uri)
       @cm.parse("baz=3; path=/; expires=\"\"", uri)
@@ -349,7 +356,7 @@ EOF
     c = WebAgent::Cookie.new()
     c.name = "hoge"
     c.value = "funi"
-    c.url = URI.parse("http://www.inac.co.jp/hoge")
+    c.url = urify("http://www.inac.co.jp/hoge")
     @cm.add(c)
     c = @cm.cookies[0]
     assert_equal('hoge', c.name)
@@ -362,14 +369,14 @@ EOF
     c.name = "hoge"
     c.value = "funi"
     c.path = ''
-    c.url = URI.parse("http://www.inac.co.jp/hoge/hoge2/hoge3")
+    c.url = urify("http://www.inac.co.jp/hoge/hoge2/hoge3")
     @cm.add(c)
     #
     c = WebAgent::Cookie.new()
     c.name = "hoge"
     c.value = "funi"
     #c.path = '' NO path given -> same as URL
-    c.url = URI.parse("http://www.inac.co.jp/hoge/hoge2/hoge3")
+    c.url = urify("http://www.inac.co.jp/hoge/hoge2/hoge3")
     @cm.add(c)
     #
     c1, c2 = @cm.cookies

data/test/test_http-access2.rb

@@ -7,6 +7,7 @@ module HTTPAccess2
 
 class TestClient < Test::Unit::TestCase
   include Helper
+  include HTTPClient::Util
 
   def setup
     super
@@ -23,7 +24,7 @@ class TestClient < Test::Unit::TestCase
     escape_noproxy do
       @proxyio.string = ""
       @client = HTTPAccess2::Client.new(proxyurl)
-      assert_equal(URI.parse(proxyurl), @client.proxy)
+      assert_equal(urify(proxyurl), @client.proxy)
       assert_equal(200, @client.head(serverurl).status)
       assert(!@proxyio.string.empty?)
     end
@@ -114,14 +115,16 @@ class TestClient < Test::Unit::TestCase
   def test_proxy
     setup_proxyserver
     escape_noproxy do
-      assert_raises(URI::InvalidURIError) do
+      begin
        	@client.proxy = "http://"
+      rescue
+        assert_match(/InvalidURIError/, $!.class.to_s)
       end
       @client.proxy = ""
       assert_nil(@client.proxy)
       @client.proxy = "http://foo:1234"
-      assert_equal(URI.parse("http://foo:1234"), @client.proxy)
-      uri = URI.parse("http://bar:2345")
+      assert_equal(urify("http://foo:1234"), @client.proxy)
+      uri = urify("http://bar:2345")
       @client.proxy = uri
       assert_equal(uri, @client.proxy)
       #

data/test/test_httpclient.rb

@@ -4,6 +4,7 @@ require File.expand_path('helper', File.dirname(__FILE__))
 
 class TestHTTPClient < Test::Unit::TestCase
   include Helper
+  include HTTPClient::Util
 
   def setup
     super
@@ -20,7 +21,7 @@ class TestHTTPClient < Test::Unit::TestCase
     escape_noproxy do
       @proxyio.string = ""
       @client = HTTPClient.new(proxyurl)
-      assert_equal(URI.parse(proxyurl), @client.proxy)
+      assert_equal(urify(proxyurl), @client.proxy)
       assert_equal(200, @client.head(serverurl).status)
       assert(/accept/ =~ @proxyio.string)
     end
@@ -174,14 +175,16 @@ class TestHTTPClient < Test::Unit::TestCase
   def test_proxy
     setup_proxyserver
     escape_noproxy do
-      assert_raises(URI::InvalidURIError) do
+      begin
         @client.proxy = "http://"
+      rescue
+        assert_match(/InvalidURIError/, $!.class.to_s)
       end
       @client.proxy = ""
       assert_nil(@client.proxy)
       @client.proxy = "http://admin:admin@foo:1234"
-      assert_equal(URI.parse("http://admin:admin@foo:1234"), @client.proxy)
-      uri = URI.parse("http://bar:2345")
+      assert_equal(urify("http://admin:admin@foo:1234"), @client.proxy)
+      uri = urify("http://bar:2345")
       @client.proxy = uri
       assert_equal(uri, @client.proxy)
       #
@@ -220,7 +223,7 @@ class TestHTTPClient < Test::Unit::TestCase
       ENV['http_proxy'] = "http://admin:admin@foo:1234"
       ENV['NO_PROXY'] = "foobar"
       client = HTTPClient.new
-      assert_equal(URI.parse("http://admin:admin@foo:1234"), client.proxy)
+      assert_equal(urify("http://admin:admin@foo:1234"), client.proxy)
       assert_equal('foobar', client.no_proxy)
     end
   end
@@ -235,7 +238,7 @@ class TestHTTPClient < Test::Unit::TestCase
       assert_equal(nil, client.proxy)
       ENV['CGI_HTTP_PROXY'] = "http://admin:admin@foo:1234"
       client = HTTPClient.new
-      assert_equal(URI.parse("http://admin:admin@foo:1234"), client.proxy)
+      assert_equal(urify("http://admin:admin@foo:1234"), client.proxy)
     end
   end
 
@@ -298,6 +301,36 @@ class TestHTTPClient < Test::Unit::TestCase
     end
   end
 
+  def test_no_proxy_with_initial_dot
+    @client.debug_dev = str = ""
+    @client.test_loopback_http_response << "HTTP/1.0 200 OK\r\n\r\n"
+    @client.no_proxy = ''
+    @client.proxy = proxyurl
+    @client.head('http://www.foo.com')
+    assert(/CONNECT TO localhost/ =~ str, 'via proxy')
+    #
+    @client.debug_dev = str = ""
+    @client.test_loopback_http_response << "HTTP/1.0 200 OK\r\n\r\n"
+    @client.no_proxy = '.foo.com'
+    @client.proxy = proxyurl
+    @client.head('http://www.foo.com')
+    assert(/CONNECT TO www.foo.com/ =~ str, 'no proxy because .foo.com matches with www.foo.com')
+    #
+    @client.debug_dev = str = ""
+    @client.test_loopback_http_response << "HTTP/1.0 200 OK\r\n\r\n"
+    @client.no_proxy = '.foo.com'
+    @client.proxy = proxyurl
+    @client.head('http://foo.com')
+    assert(/CONNECT TO localhost/ =~ str, 'via proxy because .foo.com does not matche with foo.com')
+    #
+    @client.debug_dev = str = ""
+    @client.test_loopback_http_response << "HTTP/1.0 200 OK\r\n\r\n"
+    @client.no_proxy = 'foo.com'
+    @client.proxy = proxyurl
+    @client.head('http://foo.com')
+    assert(/CONNECT TO foo.com/ =~ str, 'no proxy because foo.com matches with foo.com')
+  end
+
   def test_cookie_update_while_authentication
     escape_noproxy do
       @client.test_loopback_http_response << <<EOS
@@ -397,18 +430,18 @@ EOS
 
   def test_request_uri_in_response
     @client.test_loopback_http_response << "HTTP/1.0 200 OK\ncontent-length: 100\n\nmessage body"
-    assert_equal(URI('http://google.com/'), @client.get('http://google.com/').header.request_uri)
+    assert_equal(urify('http://google.com/'), @client.get('http://google.com/').header.request_uri)
   end
 
   def test_request_uri_in_response_when_redirect
-    expected = URI(serverurl + 'hello')
+    expected = urify(serverurl + 'hello')
     assert_equal(expected, @client.get(serverurl + 'redirect1', :follow_redirect => true).header.request_uri)
     assert_equal(expected, @client.get(serverurl + 'redirect2', :follow_redirect => true).header.request_uri)
   end
 
   def test_redirect_non_https
     url = serverurl + 'redirect1'
-    https_url = URI.parse(url)
+    https_url = urify(url)
     https_url.scheme = 'https'
     #
     redirect_to_http = "HTTP/1.0 302 OK\nLocation: #{url}\n\n"
@@ -449,6 +482,10 @@ EOS
     end
   end
 
+  def test_redirect_see_other
+    assert_equal('hello', @client.post_content(serverurl + 'redirect_see_other'))
+  end
+
   def test_redirect_relative
     @client.test_loopback_http_response << "HTTP/1.0 302 OK\nLocation: hello\n\n"
     silent do
@@ -470,7 +507,7 @@ EOS
 
   def test_redirect_https_relative
     url = serverurl + 'redirect1'
-    https_url = URI.parse(url)
+    https_url = urify(url)
     https_url.scheme = 'https'
     @client.test_loopback_http_response << "HTTP/1.0 302 OK\nLocation: /foo\n\n"
     @client.test_loopback_http_response << "HTTP/1.0 200 OK\n\nhello"
@@ -604,7 +641,7 @@ EOS
   end
 
   def test_head_follow_redirect
-    expected = URI(serverurl + 'hello')
+    expected = urify(serverurl + 'hello')
     assert_equal(expected, @client.head(serverurl + 'hello', :follow_redirect => true).header.request_uri)
     assert_equal(expected, @client.head(serverurl + 'redirect1', :follow_redirect => true).header.request_uri)
     assert_equal(expected, @client.head(serverurl + 'redirect2', :follow_redirect => true).header.request_uri)
@@ -685,6 +722,22 @@ EOS
     assert_equal("post,--hello\r\nContent-Disposition: form-data; name=\"1\"\r\n\r\n2\r\n--hello\r\nContent-Disposition: form-data; name=\"3\"\r\n\r\n4\r\n--hello--\r\n\r\n", res.content)
   end
 
+  def test_post_with_custom_multipart_and_boolean_params
+    param = [['boolean_true', true]]
+    ext = { 'content-type' => 'multipart/form-data' }
+    assert_equal("post", @client.post(serverurl + 'servlet').content[0, 4], ext)
+    res = @client.post(serverurl + 'servlet', param, ext)
+    assert_match(/Content-Disposition: form-data; name="boolean_true"\r\n\r\ntrue\r\n/, res.content)
+    #
+    param = [['boolean_false', false]]
+    res = @client.post(serverurl + 'servlet', param, ext)
+    assert_match(/Content-Disposition: form-data; name="boolean_false"\r\n\r\nfalse\r\n/, res.content)
+    #
+    param = [['nil', nil]]
+    res = @client.post(serverurl + 'servlet', param, ext)
+    assert_match(/Content-Disposition: form-data; name="nil"\r\n\r\n\r\n/, res.content)
+  end
+
   def test_post_with_file
     STDOUT.sync = true
     File.open(__FILE__) do |file|
@@ -695,19 +748,6 @@ EOS
     end
   end
 
-  def test_post_with_file_without_size
-    STDOUT.sync = true
-    File.open(__FILE__) do |file|
-      def file.size
-        # Simulates some strange Windows behaviour
-        raise SystemCallError.new "Unknown Error (20047)"
-      end
-      assert_nothing_raised do
-        @client.post(serverurl + 'servlet', {1=>2, 3=>file})
-      end
-    end
-  end
-
   def test_post_with_io # streaming, but not chunked
     myio = StringIO.new("X" * (HTTP::Message::Body::DEFAULT_CHUNK_SIZE + 1))
     def myio.read(*args)
@@ -724,7 +764,7 @@ EOS
     assert_match(/\r\n2\r\n/m, res.content)
     assert_match(/\r\nContent-Disposition: form-data; name="3"; filename=""\r\n/m, res.content)
     assert_match(/\r\nContent-Length:/m, str.string)
-    assert_equal(3, myio.called)
+    assert_equal(2, myio.called, 'should just read 2 times, no need to extra read for detecting EOF')
   end
 
   def test_post_with_io_nosize # streaming + chunked post
@@ -741,6 +781,26 @@ EOS
     assert_match(/\r\nTransfer-Encoding: chunked\r\n/m, str.string)
   end
 
+  def test_post_with_sized_io
+    myio = StringIO.new("45")
+    def myio.size
+      1
+    end
+    @client.debug_dev = str = StringIO.new
+    res = @client.post(serverurl + 'servlet', myio)
+    assert_match(/\r\n4\n/, str.string, 'should send "4" not "45"')
+  end
+
+  def test_post_with_sized_io_chunked
+    myio = StringIO.new("45")
+    def myio.size
+      1
+    end
+    @client.debug_dev = str = StringIO.new
+    res = @client.post(serverurl + 'servlet', {:file => myio})
+    assert_match(/\r\n4\r\n/, str.string, 'should send "4" not "45"')
+  end
+
   def test_post_async
     param = {'1'=>'2', '3'=>'4'}
     conn = @client.post_async(serverurl + 'servlet', param)
@@ -1126,8 +1186,8 @@ EOS
     extend HTTPClient::Util
     assert_nil(urify(nil))
     uri = 'http://foo'
-    assert_equal(URI.parse(uri), urify(uri))
-    assert_equal(URI.parse(uri), urify(URI.parse(uri)))
+    assert_equal(urify(uri), urify(uri))
+    assert_equal(urify(uri), urify(urify(uri)))
   end
 
   def test_connection
@@ -1147,7 +1207,7 @@ EOS
       site.inspect
     end
     #
-    site = HTTPClient::Site.new(URI.parse('http://localhost:12345/foo'))
+    site = HTTPClient::Site.new(urify('http://localhost:12345/foo'))
     assert_equal('http', site.scheme)
     assert_equal('localhost', site.host)
     assert_equal(12345, site.port)
@@ -1157,9 +1217,9 @@ EOS
       site.inspect
     end
     #
-    site1 = HTTPClient::Site.new(URI.parse('http://localhost:12341/'))
-    site2 = HTTPClient::Site.new(URI.parse('http://localhost:12342/'))
-    site3 = HTTPClient::Site.new(URI.parse('http://localhost:12342/'))
+    site1 = HTTPClient::Site.new(urify('http://localhost:12341/'))
+    site2 = HTTPClient::Site.new(urify('http://localhost:12342/'))
+    site3 = HTTPClient::Site.new(urify('http://localhost:12342/'))
     assert(!(site1 == site2))
     h = { site1 => 'site1', site2 => 'site2' }
     h[site3] = 'site3'
@@ -1212,9 +1272,9 @@ EOS
   end
 
   def test_connect_request
-    req = HTTP::Message.new_connect_request(URI.parse('https://foo/bar'))
+    req = HTTP::Message.new_connect_request(urify('https://foo/bar'))
     assert_equal("CONNECT foo:443 HTTP/1.0\r\n\r\n", req.dump)
-    req = HTTP::Message.new_connect_request(URI.parse('https://example.com/'))
+    req = HTTP::Message.new_connect_request(urify('https://example.com/'))
     assert_equal("CONNECT example.com:443 HTTP/1.0\r\n\r\n", req.dump)
   end
 
@@ -1313,6 +1373,17 @@ EOS
     assert_equal('PART_NUMBER', res.cookies[1].name)
   end
 
+  def test_ok_response_success
+    res = HTTP::Message.new_response('response')
+    assert_equal(true, res.ok?)
+    res.status = 404
+    assert_equal(false, res.ok?)
+    res.status = 500
+    assert_equal(false, res.ok?)
+    res.status = 302
+    assert_equal(false, res.ok?)
+  end
+
   if !defined?(JRUBY_VERSION) and RUBY_VERSION < '1.9'
     def test_timeout_scheduler
       assert_equal('hello', @client.get_content(serverurl + 'hello'))
@@ -1483,6 +1554,16 @@ EOS
     end
   end
 
+  if RUBY_VERSION >= "1.9.3"
+    def test_continue
+      @client.debug_dev = str = ''
+      res = @client.get(serverurl + 'continue', :header => {:Expect => '100-continue'})
+      assert_equal(200, res.status)
+      assert_equal('done!', res.body)
+      assert_match(/Expect: 100-continue/, str)
+    end
+  end
+
 private
 
   def check_query_get(query)
@@ -1506,7 +1587,11 @@ private
       :DocumentRoot => File.dirname(File.expand_path(__FILE__))
     )
     @serverport = @server.config[:Port]
-    [:hello, :sleep, :servlet_redirect, :redirect1, :redirect2, :redirect3, :redirect_self, :relative_redirect, :chunked, :largebody, :status, :compressed, :charset].each do |sym|
+    [
+      :hello, :sleep, :servlet_redirect, :redirect1, :redirect2, :redirect3,
+      :redirect_self, :relative_redirect, :redirect_see_other, :chunked,
+      :largebody, :status, :compressed, :charset, :continue
+    ].each do |sym|
       @server.mount(
         "/#{sym}",
         WEBrick::HTTPServlet::ProcHandler.new(method("do_#{sym}").to_proc)
@@ -1560,6 +1645,14 @@ private
     res.set_redirect(WEBrick::HTTPStatus::Found, "hello")
   end
 
+  def do_redirect_see_other(req, res)
+    if req.request_method == 'POST'
+      res.set_redirect(WEBrick::HTTPStatus::SeeOther, serverurl + "redirect_see_other") # self
+    else
+      res.body = 'hello'
+    end
+  end
+
   def do_chunked(req, res)
     res.chunked = true
     piper, pipew = IO.pipe
@@ -1597,6 +1690,11 @@ private
     res.status = req.query['status'].to_i
   end
 
+  def do_continue(req, res)
+    req.continue
+    res.body = 'done!'
+  end
+
   class TestServlet < WEBrick::HTTPServlet::AbstractServlet
     def get_instance(*arg)
       self

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: httpclient
 version: !ruby/object:Gem::Version
-  version: 2.2.7
+  version: 2.3.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,14 +9,16 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-08-14 00:00:00.000000000 Z
+date: 2012-10-10 00:00:00.000000000 Z
 dependencies: []
 description: 
 email: nahi@ruby-lang.org
-executables: []
+executables:
+- httpclient
 extensions: []
 extra_rdoc_files: []
 files:
+- bin/httpclient
 - lib/httpclient/auth.rb
 - lib/httpclient/ssl_config.rb
 - lib/httpclient/http.rb
@@ -88,7 +90,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -2044284490282545774
+      hash: -478554732977301043
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -97,7 +99,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -2044284490282545774
+      hash: -478554732977301043
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.23