excon 0.30.0 → 0.31.0

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    excon (0.30.0)
+    excon (0.31.0)
 
 GEM
   remote: http://rubygems.org/

data/README.md

@@ -45,6 +45,41 @@ post_response = connection.post(:path => '/foo')
 delete_response = connection.delete(:path => '/bar')
 ```
 
+By default, each connection is non-persistent. This means that each request made against a connection behaves like a
+one-off request. Each request will establish a socket connection to the server, then close the socket once the request
+is complete.
+
+To use a persistent connection, use the `:persistent` option:
+
+```ruby
+connection = Excon.new('http://geemus.com', :persistent => true)
+```
+
+The initial request will establish a socket connection to the server and leave the socket open. Subsequent requests
+will reuse that socket. You may call `Connection#reset` at any time to close the underlying socket, and the next request
+will establish a new socket connection.
+
+You may also control persistence on a per-request basis by setting the `:persistent` option for each request.
+
+```ruby
+connection = Excon.new('http://geemus.com') # non-persistent by default
+connection.get # socket established, then closed
+connection.get(:persistent => true) # socket established, left open
+connection.get(:persistent => true) # socket reused
+connection.get # socket reused, then closed
+
+connection = Excon.new('http://geemus.com', :persistent => true)
+connection.get # socket established, left open
+connection.get(:persistent => false) # socket reused, then closed
+connection.get(:persistent => false) # socket established, then closed
+connection.get # socket established, left open
+connection.get # socket reused
+```
+
+Note that sending a request with `:persistent => false` to close the socket will also send `Connection: close` to inform
+the server the connection is no longer needed. `Connection#reset` will simply close our end of the socket.
+
+
 Options
 -------
 
@@ -131,13 +166,16 @@ Iterating in this way allows you to have more granular control over writes and t
 Pipelining Requests
 ------------------
 
-You can make use of HTTP pipelining to improve performance. Instead of the normal request/response cyle, pipelining sends a series of requests and then receives a series of responses. You can take advantage of this using the `requests` method, which takes an array of params where each is a hash like request would receive and returns an array of responses.
+You can make use of HTTP pipelining to improve performance. Instead of the normal request/response cycle, pipelining sends a series of requests and then receives a series of responses. You can take advantage of this using the `requests` method, which takes an array of params where each is a hash like request would receive and returns an array of responses.
 
 ```ruby
 connection = Excon.new('http://geemus.com/')
 connection.requests([{:method => :get}, {:method => :get}])
 ```
 
+By default, each call to `requests` will use a separate persistent socket connection. To make multiple `requests` calls
+using a single persistent connection, set `:persistent => true` when establishing the connection.
+
 Streaming Responses
 -------------------
 
@@ -307,6 +345,35 @@ Excon.defaults[:ssl_verify_peer] = false
 
 Either of these should allow you to work around the socket error and continue with your work.
 
+## Getting Help
+
+* [General Documentation](http://excon.io).
+* Ask specific questions on [Stack Overflow](http://stackoverflow.com/questions/tagged/excon).
+* Report bugs and discuss potential features in [Github issues](https://github.com/geemus/excon/issues).
+
+## Getting Involved
+
+New contributors are always welcome, when it doubt please ask questions. We strive to be an open and welcoming community. Please be nice to one another.
+
+### Coding
+
+* Pick a task:
+  * Offer feedback on open [pull requests](https://github.com/geemus/excon/pulls).
+  * Review open [issues](https://github.com/geemus/excon/issues) for things to help on.
+  * [Create an issue](https://github.com/geemus/excon/issues/new) to start a discussion on additions or features.
+* Fork the project, add your changes and tests to cover them in a topic branch.
+* Commit your changes and rebase against `geemus/excon` to ensure everything is up to date.
+* [Submit a pull request](https://github.com/geemus/excon/compare/).
+
+### Non-Coding
+
+* Offer feedback on open [issues](https://github.com/geemus/excon/issues).
+* Write and help edit [documentation](https://github.com/geemus/excon.github.com).
+* Translate [documentation](https://github.com/geemus/excon.github.com) in to other languages.
+* Organize or volunteer at events.
+* [Donate](https://www.gittip.com/geemus/)!
+* Discuss other ideas for contribution with [geemus](mailto:geemus+excon@gmail.com).
+
 Copyright
 ---------
 

data/changelog.txt

@@ -1,3 +1,17 @@
+0.31.0 12/16/2013
+=================
+
+test fixes for Bundler.require usage and Rack::Lint
+use production mode + dump errors for tests
+use Utils in Excon.stub
+add implementation of stuff from WEBrick directly to utils
+update test server to send connection close and process buffer after response
+add :persistent option, defaults true
+group HTTP errors by type
+patch to webrick to workaround intermitent test failures
+only use Open4 for 1.8.7
+update/expand getting help/getting involved in readme
+
 0.30.0 11/25/2013
 =================
 

data/excon.gemspec

@@ -13,8 +13,8 @@ Gem::Specification.new do |s|
   ## If your rubyforge_project name is different, then edit it and comment out
   ## the sub! line in the Rakefile
   s.name              = 'excon'
-  s.version           = '0.30.0'
-  s.date              = '2013-11-25'
+  s.version           = '0.31.0'
+  s.date              = '2013-12-16'
   s.rubyforge_project = 'excon'
 
   ## Make sure your summary is short. The description may be as long
@@ -125,6 +125,7 @@ Gem::Specification.new do |s|
     tests/middlewares/instrumentation_tests.rb
     tests/middlewares/mock_tests.rb
     tests/middlewares/redirect_follower_tests.rb
+    tests/pipeline_tests.rb
     tests/proxy_tests.rb
     tests/query_string_tests.rb
     tests/rackups/basic.rb
@@ -140,9 +141,10 @@ Gem::Specification.new do |s|
     tests/rackups/ssl_verify_peer.ru
     tests/rackups/thread_safety.ru
     tests/rackups/timeout.ru
+    tests/rackups/webrick_patch.rb
     tests/request_headers_tests.rb
     tests/request_method_tests.rb
-    tests/requests_tests.rb
+    tests/request_tests.rb
     tests/response_tests.rb
     tests/servers/bad.rb
     tests/servers/eof.rb

data/lib/excon.rb

@@ -10,7 +10,6 @@ require 'timeout'
 require 'uri'
 require 'zlib'
 require 'stringio'
-require 'webrick/httputils'
 
 BasicSocket.do_not_reverse_lookup = true
 
@@ -41,6 +40,7 @@ module Excon
         :mock               => false,
         :nonblock           => true,
         :omit_default_port  => false,
+        :persistent         => false,
         :read_timeout       => 60,
         :retry_limit        => DEFAULT_RETRY_LIMIT,
         :ssl_verify_peer    => true,
@@ -166,7 +166,7 @@ module Excon
         )
         if uri.user || uri.password
           request_params[:headers] ||= {}
-          user, pass = URI.decode_www_form_component(uri.user.to_s), URI.decode_www_form_component(uri.password.to_s)
+          user, pass = Utils.unescape_form(uri.user.to_s), Utils.unescape_form(uri.password.to_s)
           request_params[:headers]['Authorization'] ||= 'Basic ' << ['' << user << ':' << pass].pack('m').delete(Excon::CR_NL)
         end
       end

data/lib/excon/connection.rb

@@ -152,7 +152,7 @@ module Excon
           else
             datum[:headers]['TE'] = 'trailers, deflate, gzip'
           end
-          datum[:headers]['Connection'] = 'TE'
+          datum[:headers]['Connection'] = datum[:persistent] ? 'TE' : 'TE, close'
 
           # add headers to request
           datum[:headers].each do |key, values|
@@ -267,10 +267,14 @@ module Excon
       unless datum[:pipeline]
         datum = response(datum)
 
-        if key = datum[:response][:headers].keys.detect {|k| k.casecmp('Connection') == 0 }
-          if split_header_value(datum[:response][:headers][key]).any? {|t| t.casecmp('close') }
-            reset
+        if datum[:persistent]
+          if key = datum[:response][:headers].keys.detect {|k| k.casecmp('Connection') == 0 }
+            if split_header_value(datum[:response][:headers][key]).any? {|t| t.casecmp('close') }
+              reset
+            end
           end
+        else
+          reset
         end
 
         Excon::Response.new(datum[:response])
@@ -290,16 +294,23 @@ module Excon
     # Sends the supplied requests to the destination host using pipelining.
     #   @pipeline_params [Array<Hash>] pipeline_params An array of one or more optional params, override defaults set in Connection.new, see #request for details
     def requests(pipeline_params)
+      pipeline_params.each {|params| params.merge!(:pipeline => true, :persistent => true) }
+      pipeline_params.last.merge!(:persistent => @data[:persistent])
+
       responses = pipeline_params.map do |params|
-        request(params.merge!(:pipeline => true))
+        request(params)
       end.map do |datum|
         Excon::Response.new(response(datum)[:response])
       end
 
-      if key = responses.last[:headers].keys.detect {|k| k.casecmp('Connection') == 0 }
-        if split_header_value(responses.last[:headers][key]).any? {|t| t.casecmp('close') }
-          reset
+      if @data[:persistent]
+        if key = responses.last[:headers].keys.detect {|k| k.casecmp('Connection') == 0 }
+          if split_header_value(responses.last[:headers][key]).any? {|t| t.casecmp('close') }
+            reset
+          end
         end
+      else
+        reset
       end
 
       responses

data/lib/excon/constants.rb

@@ -1,6 +1,6 @@
 module Excon
 
-  VERSION = '0.30.0'
+  VERSION = '0.31.0'
 
   CR_NL = "\r\n"
 
@@ -46,6 +46,7 @@ module Excon
     :middlewares,
     :mock,
     :path,
+    :persistent,
     :pipeline,
     :query,
     :read_timeout,

data/lib/excon/errors.rb

@@ -35,47 +35,54 @@ module Excon
         @response = response
       end
     end
-
-    class Continue < HTTPStatusError; end                     # 100
-    class SwitchingProtocols < HTTPStatusError; end           # 101
-    class OK < HTTPStatusError; end                           # 200
-    class Created < HTTPStatusError; end                      # 201
-    class Accepted < HTTPStatusError; end                     # 202
-    class NonAuthoritativeInformation < HTTPStatusError; end  # 203
-    class NoContent < HTTPStatusError; end                    # 204
-    class ResetContent < HTTPStatusError; end                 # 205
-    class PartialContent < HTTPStatusError; end               # 206
-    class MultipleChoices < HTTPStatusError; end              # 300
-    class MovedPermanently < HTTPStatusError; end             # 301
-    class Found < HTTPStatusError; end                        # 302
-    class SeeOther < HTTPStatusError; end                     # 303
-    class NotModified < HTTPStatusError; end                  # 304
-    class UseProxy < HTTPStatusError; end                     # 305
-    class TemporaryRedirect < HTTPStatusError; end            # 307
-    class BadRequest < HTTPStatusError; end                   # 400
-    class Unauthorized < HTTPStatusError; end                 # 401
-    class PaymentRequired < HTTPStatusError; end              # 402
-    class Forbidden < HTTPStatusError; end                    # 403
-    class NotFound < HTTPStatusError; end                     # 404
-    class MethodNotAllowed < HTTPStatusError; end             # 405
-    class NotAcceptable < HTTPStatusError; end                # 406
-    class ProxyAuthenticationRequired < HTTPStatusError; end  # 407
-    class RequestTimeout < HTTPStatusError; end               # 408
-    class Conflict < HTTPStatusError; end                     # 409
-    class Gone < HTTPStatusError; end                         # 410
-    class LengthRequired < HTTPStatusError; end               # 411
-    class PreconditionFailed < HTTPStatusError; end           # 412
-    class RequestEntityTooLarge < HTTPStatusError; end        # 413
-    class RequestURITooLong < HTTPStatusError; end            # 414
-    class UnsupportedMediaType < HTTPStatusError; end         # 415
-    class RequestedRangeNotSatisfiable < HTTPStatusError; end # 416
-    class ExpectationFailed < HTTPStatusError; end            # 417
-    class UnprocessableEntity < HTTPStatusError; end          # 422
-    class InternalServerError < HTTPStatusError; end          # 500
-    class NotImplemented < HTTPStatusError; end               # 501
-    class BadGateway < HTTPStatusError; end                   # 502
-    class ServiceUnavailable < HTTPStatusError; end           # 503
-    class GatewayTimeout < HTTPStatusError; end               # 504
+    
+    # HTTP Error classes
+    class Informational < HTTPStatusError; end
+    class Success < HTTPStatusError; end
+    class Redirection < HTTPStatusError; end
+    class ClientError < HTTPStatusError; end
+    class ServerError < HTTPStatusError; end
+
+    class Continue < Informational; end                       # 100
+    class SwitchingProtocols < Informational; end             # 101
+    class OK < Success; end                                   # 200
+    class Created < Success; end                              # 201
+    class Accepted < Success; end                             # 202
+    class NonAuthoritativeInformation < Success; end          # 203
+    class NoContent < Success; end                            # 204
+    class ResetContent < Success; end                         # 205
+    class PartialContent < Success; end                       # 206
+    class MultipleChoices < Redirection; end                  # 300
+    class MovedPermanently < Redirection; end                 # 301
+    class Found < Redirection; end                            # 302
+    class SeeOther < Redirection; end                         # 303
+    class NotModified < Redirection; end                      # 304
+    class UseProxy < Redirection; end                         # 305
+    class TemporaryRedirect < Redirection; end                # 307
+    class BadRequest < ClientError; end                       # 400
+    class Unauthorized < ClientError; end                     # 401
+    class PaymentRequired < ClientError; end                  # 402
+    class Forbidden < ClientError; end                        # 403
+    class NotFound < ClientError; end                         # 404
+    class MethodNotAllowed < ClientError; end                 # 405
+    class NotAcceptable < ClientError; end                    # 406
+    class ProxyAuthenticationRequired < ClientError; end      # 407
+    class RequestTimeout < ClientError; end                   # 408
+    class Conflict < ClientError; end                         # 409
+    class Gone < ClientError; end                             # 410
+    class LengthRequired < ClientError; end                   # 411
+    class PreconditionFailed < ClientError; end               # 412
+    class RequestEntityTooLarge < ClientError; end            # 413
+    class RequestURITooLong < ClientError; end                # 414
+    class UnsupportedMediaType < ClientError; end             # 415
+    class RequestedRangeNotSatisfiable < ClientError; end     # 416
+    class ExpectationFailed < ClientError; end                # 417
+    class UnprocessableEntity < ClientError; end              # 422
+    class InternalServerError < ServerError; end              # 500
+    class NotImplemented < ServerError; end                   # 501
+    class BadGateway < ServerError; end                       # 502
+    class ServiceUnavailable < ServerError; end               # 503
+    class GatewayTimeout < ServerError; end                   # 504
 
     # Messages for nicer exceptions, from rfc2616
     def self.status_error(request, response)

data/lib/excon/utils.rb

@@ -2,6 +2,8 @@ module Excon
   module Utils
     extend self
 
+    ESCAPED = /%([0-9a-fA-F]{2})/
+
     def valid_connection_keys(params = {})
       Excon::VALID_CONNECTION_KEYS
     end
@@ -57,17 +59,27 @@ module Excon
     # Splits a header value +str+ according to HTTP specification.
     def split_header_value(str)
       return [] if str.nil?
-      WEBrick::HTTPUtils.split_header_value(str.strip)
+      str = str.strip
+      str.force_encoding('BINARY') if FORCE_ENC
+      str.scan(%r'\G((?:"(?:\\.|[^"])+?"|[^",]+)+)
+                    (?:,\s*|\Z)'xn).flatten
     end
 
     # Unescapes HTTP reserved and unwise characters in +str+
     def unescape_uri(str)
-      WEBrick::HTTPUtils.unescape(str)
+      str = str.dup
+      str.force_encoding('BINARY') if FORCE_ENC
+      str.gsub!(ESCAPED) { $1.hex.chr }
+      str
     end
 
     # Unescape form encoded values in +str+
     def unescape_form(str)
-      WEBrick::HTTPUtils.unescape_form(str)
+      str = str.dup
+      str.force_encoding('BINARY') if FORCE_ENC
+      str.gsub!(/\+/, ' ')
+      str.gsub!(ESCAPED) { $1.hex.chr }
+      str
     end
   end
 end

data/tests/authorization_header_tests.rb

@@ -1,15 +1,16 @@
 Shindo.tests('Excon basics (Authorization data redacted)') do
   with_rackup('basic_auth.ru') do
     cases = [
-             ['user & pass', 'http://user1:pass1@foo.com/', 'Basic dXNlcjE6cGFzczE='],
-             ['email & pass', 'http://foo%40bar.com:pass1@foo.com/', 'Basic Zm9vQGJhci5jb206cGFzczE='],
-             ['user no pass', 'http://three_user@foo.com/', 'Basic dGhyZWVfdXNlcjo='],
-             ['pass no user', 'http://:derppass@foo.com/', 'Basic OmRlcnBwYXNz']
-            ]
+      ['user & pass', 'http://user1:pass1@foo.com/', 'Basic dXNlcjE6cGFzczE='],
+      ['email & pass', 'http://foo%40bar.com:pass1@foo.com/', 'Basic Zm9vQGJhci5jb206cGFzczE='],
+      ['user no pass', 'http://three_user@foo.com/', 'Basic dGhyZWVfdXNlcjo='],
+      ['pass no user', 'http://:derppass@foo.com/', 'Basic OmRlcnBwYXNz']
+    ]
     cases.each do |desc,url,auth_header|
-      conn = Excon.new(url)
+      conn = nil
 
       test("authorization header concealed for #{desc}") do
+        conn = Excon.new(url)
         !conn.inspect.include?(auth_header)
       end
 

data/tests/basic_tests.rb

@@ -61,8 +61,7 @@ Shindo.tests('Excon basics (ssl file)',['focus']) do
 
     basic_tests('https://127.0.0.1:8443',
                 :client_key => File.join(File.dirname(__FILE__), 'data', 'excon.cert.key'),
-                :client_cert => File.join(File.dirname(__FILE__), 'data', 'excon.cert.crt'),
-                :reset_connection => RUBY_VERSION == '1.9.2'
+                :client_cert => File.join(File.dirname(__FILE__), 'data', 'excon.cert.crt')
                )
 
   end

data/tests/header_tests.rb

@@ -4,12 +4,15 @@ Shindo.tests('Excon response header support') do
   with_rackup('response_header.ru') do
 
     tests('Response#get_header') do
-      connection = Excon.new('http://foo.com:8080', :proxy => 'http://127.0.0.1:9292')
-      response = connection.request(:method => :get, :path => '/foo')
+      connection = nil
+      response = nil
 
       tests('with variable header capitalization') do
 
         tests('response.get_header("mixedcase-header")').returns('MixedCase') do
+          connection = Excon.new('http://foo.com:8080', :proxy => 'http://127.0.0.1:9292')
+          response = connection.request(:method => :get, :path => '/foo')
+
           response.get_header("mixedcase-header")
         end
 

data/tests/middlewares/canned_response_tests.rb

@@ -12,15 +12,12 @@ Shindo.tests("Excon support for middlewares that return canned responses") do
     end
   end
 
-  connection = Excon.new(
-    'http://some-host.com/some-path',
-    :method         => :get,
-    :middlewares    => [canned_response_middleware] + Excon.defaults[:middlewares],
-    :response_block => Proc.new { } # to force streaming
-  )
-
   tests('does not mutate the canned response body').returns("canned") do
-    connection.request
+    Excon.get(
+      'http://some-host.com/some-path',
+      :middlewares    => [canned_response_middleware] + Excon.defaults[:middlewares],
+      :response_block => Proc.new { } # to force streaming
+    )
     the_body
   end
 end