em-http-request 0.2.11 → 0.2.12

data/Changelog.md

@@ -0,0 +1,27 @@
+# Changelog
+
+## 0.2.12 / 2010-09-12
+
+- added headers callback (http.headers {|h| p h})
+- added .close method on client obj to terminate session (accepts message)
+
+- bugfix: report 0 for response status on 1.9 on timeouts
+- bugfix: handle bad Location host redirects
+- bugfix: reset host override on connect
+
+## 0.2.11 / 2010-08-16
+
+- all URIs are now normalized prior to dispatch (and on redirect)
+- default to direct proxy (instead of CONNECT handshake) - better performance
+  - specify :proxy => {:tunnel => true} if you need to force CONNECT route
+- MultiRequest accepts block syntax for dispatching parallel requests (see specs)
+- MockHttpRequest accepts block syntax (see Mock wiki page)
+
+
+- bugfix: nullbyte frame for websockets
+- bugfix: set @uri on DNS resolve failure
+- bugfix: handle bad hosts in absolute redirects
+- bugfix: reset seen content on redirects (doh!)
+- bugfix: invalid multibyte escape in websocket regex (1.9.2+)
+- bugfix: report etag and last_modified headers correctly
+

data/README.md

@@ -1,157 +1,157 @@
-EM-HTTP-Request
-===============
-
-Asynchronous HTTP client for Ruby, based on EventMachine runtime.
-
-- Ragel HTTP parser for speed & performance
-- Simple interface for single & parallel requests via deferred callbacks
-- Automatic gzip & deflate decoding
-- Basic-Auth & OAuth support
-- Custom timeout support
-- Stream response processing
-- Proxy support (with SSL Tunneling)
-- Auto-follow 3xx redirects with custom max depth
-- Bi-directional communication with web-socket services
-- [Native mocking support](http://wiki.github.com/igrigorik/em-http-request/mocking-httprequest) and through [Webmock](http://github.com/bblimke/webmock)
-
-Getting started
----------------
-
-      gem install em-http-request
-      irb:0> require 'em-http'
-
-Or checkout [screencast / demo](http://everburning.com/news/eventmachine-screencast-em-http-request/) of using EM-HTTP-Request.
-
-Libraries & Applications using em-http
---------------------------------------
-
-- [chirpstream](http://github.com/joshbuddy/chirpstream) - EM client for Twitters Chirpstream API
-- [RDaneel](http://github.com/hasmanydevelopers/RDaneel) - Ruby crawler which respects robots.txt
-- [rsolr-async](http://github.com/mwmitchell/rsolr-async) - An asynchronus connection adapter for RSolr
-- [PubSubHubbub](http://github.com/igrigorik/PubSubHubbub) - Asynchronous PubSubHubbub ruby client
-- and many others.. drop me a link if you want yours included!
-
-Simple client example
----------------------
-
-      EventMachine.run {
-        http = EventMachine::HttpRequest.new('http://127.0.0.1/').get :query => {'keyname' => 'value'}, :timeout => 10
-
-        http.callback {
-          p http.response_header.status
-          p http.response_header
-          p http.response
-
-          EventMachine.stop
-        }
-      }
-
-Multi-request example
----------------------
-
-Fire and wait for multiple requests to complete via the MultiRequest interface.
-
-      EventMachine.run {
-        multi = EventMachine::MultiRequest.new
-
-        # add multiple requests to the multi-handler
-        multi.add(EventMachine::HttpRequest.new('http://www.google.com/').get)
-        multi.add(EventMachine::HttpRequest.new('http://www.yahoo.com/').get)
-
-        multi.callback  {
-          p multi.responses[:succeeded]
-          p multi.responses[:failed]
-
-          EventMachine.stop
-        }
-      }
-
-Basic-Auth example
-------------------
-
-Full basic author support. For OAuth, check examples/oauth-tweet.rb file.
-
-      EventMachine.run {
-        http = EventMachine::HttpRequest.new('http://www.website.com/').get :head => {'authorization' => ['user', 'pass']}
-
-        http.errback { failed }
-        http.callback {
-          p http.response_header
-          EventMachine.stop
-        }
-      }
-
-
-POSTing data example
---------------------
-
-      EventMachine.run {
-        http1 = EventMachine::HttpRequest.new('http://www.website.com/').post :body => {"key1" => 1, "key2" => [2,3]}
-        http2 = EventMachine::HttpRequest.new('http://www.website.com/').post :body => "some data"
-
-        # ...
-      }
-
-Streaming body processing
--------------------------
-
-Allows you to consume an HTTP stream of content in real-time. Each time a new piece of content is pushed
-to the client, it is passed to the stream callback for you to operate on.
-
-      EventMachine.run {
-        http = EventMachine::HttpRequest.new('http://www.website.com/').get
-        http.stream { |chunk| print chunk }
-      }
-
-Streaming files from disk
--------------------------
-Allows you to efficiently stream a (large) file from disk via EventMachine's FileStream interface.
-
-      EventMachine.run {
-        http = EventMachine::HttpRequest.new('http://www.website.com/').post :file => 'largefile.txt'
-        http.callback { |chunk| puts "Upload finished!" }
-      }
-
-Proxy example
--------------
-
-Full transparent proxy support with support for SSL tunneling.
-
-      EventMachine.run {
-        http = EventMachine::HttpRequest.new('http://www.website.com/').get :proxy => {
-          :host => 'www.myproxy.com',
-          :port => 8080,
-          :authorization => ['username', 'password'] # authorization is optional
-      }
-
-Auto-follow 3xx redirects
--------------------------
-
-Specify the max depth of redirects to follow, default is 0.
-
-      EventMachine.run {
-        http = EventMachine::HttpRequest.new('http://www.google.com/').get :redirects => 1
-        http.callback { p http.last_effective_url }
-      }
-
-WebSocket example
------------------
-
-[Bi-directional communication with WebSockets](http://www.igvita.com/2009/12/22/ruby-websockets-tcp-for-the-browser/): simply pass in a ws:// resource and the client will negotiate the connection upgrade for you. On successful handshake the callback is invoked, and any incoming messages will be passed to the stream callback. The client can also send data to the server at will by calling the "send" method!
-
-      EventMachine.run {
-        http = EventMachine::HttpRequest.new("ws://yourservice.com/websocket").get :timeout => 0
-
-        http.errback { puts "oops" }
-        http.callback {
-          puts "WebSocket connected!"
-          http.send("Hello client")
-        }
-
-        http.stream { |msg|
-          puts "Recieved: #{msg}"
-          http.send "Pong: #{msg}"
-        }
-
-        http.disconnect { puts "oops, dropped connection?" }
-      }
+EM-HTTP-Request
+===============
+
+Asynchronous HTTP client for Ruby, based on EventMachine runtime.
+
+- Ragel HTTP parser for speed & performance
+- Simple interface for single & parallel requests via deferred callbacks
+- Automatic gzip & deflate decoding
+- Basic-Auth & OAuth support
+- Custom timeout support
+- Stream response processing
+- Proxy support (with SSL Tunneling)
+- Auto-follow 3xx redirects with custom max depth
+- Bi-directional communication with web-socket services
+- [Native mocking support](http://wiki.github.com/igrigorik/em-http-request/mocking-httprequest) and through [Webmock](http://github.com/bblimke/webmock)
+
+Getting started
+---------------
+
+      gem install em-http-request
+      irb:0> require 'em-http'
+
+Or checkout [screencast / demo](http://everburning.com/news/eventmachine-screencast-em-http-request/) of using EM-HTTP-Request.
+
+Libraries & Applications using em-http
+--------------------------------------
+
+- [chirpstream](http://github.com/joshbuddy/chirpstream) - EM client for Twitters Chirpstream API
+- [RDaneel](http://github.com/hasmanydevelopers/RDaneel) - Ruby crawler which respects robots.txt
+- [rsolr-async](http://github.com/mwmitchell/rsolr-async) - An asynchronus connection adapter for RSolr
+- [PubSubHubbub](http://github.com/igrigorik/PubSubHubbub) - Asynchronous PubSubHubbub ruby client
+- and many others.. drop me a link if you want yours included!
+
+Simple client example
+---------------------
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://127.0.0.1/').get :query => {'keyname' => 'value'}, :timeout => 10
+
+        http.callback {
+          p http.response_header.status
+          p http.response_header
+          p http.response
+
+          EventMachine.stop
+        }
+      }
+
+Multi-request example
+---------------------
+
+Fire and wait for multiple requests to complete via the MultiRequest interface.
+
+      EventMachine.run {
+        multi = EventMachine::MultiRequest.new
+
+        # add multiple requests to the multi-handler
+        multi.add(EventMachine::HttpRequest.new('http://www.google.com/').get)
+        multi.add(EventMachine::HttpRequest.new('http://www.yahoo.com/').get)
+
+        multi.callback  {
+          p multi.responses[:succeeded]
+          p multi.responses[:failed]
+
+          EventMachine.stop
+        }
+      }
+
+Basic-Auth example
+------------------
+
+Full basic author support. For OAuth, check examples/oauth-tweet.rb file.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.website.com/').get :head => {'authorization' => ['user', 'pass']}
+
+        http.errback { failed }
+        http.callback {
+          p http.response_header
+          EventMachine.stop
+        }
+      }
+
+
+POSTing data example
+--------------------
+
+      EventMachine.run {
+        http1 = EventMachine::HttpRequest.new('http://www.website.com/').post :body => {"key1" => 1, "key2" => [2,3]}
+        http2 = EventMachine::HttpRequest.new('http://www.website.com/').post :body => "some data"
+
+        # ...
+      }
+
+Streaming body processing
+-------------------------
+
+Allows you to consume an HTTP stream of content in real-time. Each time a new piece of content is pushed
+to the client, it is passed to the stream callback for you to operate on.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.website.com/').get
+        http.stream { |chunk| print chunk }
+      }
+
+Streaming files from disk
+-------------------------
+Allows you to efficiently stream a (large) file from disk via EventMachine's FileStream interface.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.website.com/').post :file => 'largefile.txt'
+        http.callback { |chunk| puts "Upload finished!" }
+      }
+
+Proxy example
+-------------
+
+Full transparent proxy support with support for SSL tunneling.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.website.com/').get :proxy => {
+          :host => 'www.myproxy.com',
+          :port => 8080,
+          :authorization => ['username', 'password'] # authorization is optional
+      }
+
+Auto-follow 3xx redirects
+-------------------------
+
+Specify the max depth of redirects to follow, default is 0.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.google.com/').get :redirects => 1
+        http.callback { p http.last_effective_url }
+      }
+
+WebSocket example
+-----------------
+
+[Bi-directional communication with WebSockets](http://www.igvita.com/2009/12/22/ruby-websockets-tcp-for-the-browser/): simply pass in a ws:// resource and the client will negotiate the connection upgrade for you. On successful handshake the callback is invoked, and any incoming messages will be passed to the stream callback. The client can also send data to the server at will by calling the "send" method!
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new("ws://yourservice.com/websocket").get :timeout => 0
+
+        http.errback { puts "oops" }
+        http.callback {
+          puts "WebSocket connected!"
+          http.send("Hello client")
+        }
+
+        http.stream { |msg|
+          puts "Recieved: #{msg}"
+          http.send "Pong: #{msg}"
+        }
+
+        http.disconnect { puts "oops, dropped connection?" }
+      }

data/Rakefile

@@ -1,110 +1,110 @@
-require 'rake'
-require 'rake/clean'
-require 'rake/rdoctask'
-require 'rake/gempackagetask'
-require 'fileutils'
-include FileUtils
-
-# copied from EventMachine.
-MAKE = ENV['MAKE'] || if RUBY_PLATFORM =~ /mswin/ # mingw uses make.
-  'nmake'
-else
-  'make'
-end
-
-# Default Rake task is compile
-task :default => :compile
-
-# RDoc
-Rake::RDocTask.new(:rdoc) do |task|
-  task.rdoc_dir = 'doc'
-  task.title    = 'EventMachine::HttpRequest'
-  task.options = %w(--title HttpRequest --main README --line-numbers)
-  task.rdoc_files.include(['lib/**/*.rb'])
-  task.rdoc_files.include(['README', 'LICENSE'])
-end
-
-# Rebuild parser Ragel
-task :ragel do
-  Dir.chdir "ext/http11_client" do
-    target = "http11_parser.c"
-    File.unlink target if File.exist? target
-    sh "ragel http11_parser.rl | rlgen-cd -G2 -o #{target}"
-    raise "Failed to build C source" unless File.exist? target
-  end
-end
-
-require 'spec'
-require 'spec/rake/spectask'
-Spec::Rake::SpecTask.new(:spec) do |t|
-  t.spec_opts ||= []
-  t.spec_opts << "--options" << "spec/spec.opts"
-  t.spec_files = FileList['spec/**/*_spec.rb']
-end
-
-def make(makedir)
-  Dir.chdir(makedir) { sh MAKE }
-end
-
-def extconf(dir)
-  Dir.chdir(dir) { ruby "extconf.rb" }
-end
-
-def setup_extension(dir, extension)
-  ext = "ext/#{dir}"
-  ext_so = "#{ext}/#{extension}.#{Config::CONFIG['DLEXT']}"
-  ext_files = FileList[
-    "#{ext}/*.c",
-    "#{ext}/*.h",
-    "#{ext}/extconf.rb",
-    "#{ext}/Makefile",
-    "lib"
-  ]
-
-  task "lib" do
-    directory "lib"
-  end
-
-  desc "Builds just the #{extension} extension"
-
-  mf = (extension + '_makefile').to_sym
-
-  task mf do |t|
-    extconf "#{ext}"
-  end
-
-  task extension.to_sym => [mf] do
-    make "#{ext}"
-    cp ext_so, "lib"
-  end
-end
-
-setup_extension("buffer", "em_buffer")
-setup_extension("http11_client", "http11_client")
-
-task :compile => [:em_buffer, :http11_client]
-
-CLEAN.include ['build/*', '**/*.o', '**/*.so', '**/*.a', '**/*.log', 'pkg']
-CLEAN.include ['ext/buffer/Makefile', 'lib/em_buffer.*', 'lib/http11_client.*']
-
-begin
-  require 'jeweler'
-  Jeweler::Tasks.new do |gemspec|
-    gemspec.name = "em-http-request"
-    gemspec.summary = "EventMachine based, async HTTP Request interface"
-    gemspec.description = gemspec.summary
-    gemspec.email = "ilya@igvita.com"
-    gemspec.homepage = "http://github.com/igrigorik/em-http-request"
-    gemspec.authors = ["Ilya Grigorik"]
-    gemspec.required_ruby_version = ">= 1.8.6"
-    gemspec.extensions = ["ext/buffer/extconf.rb" , "ext/http11_client/extconf.rb"]
-    gemspec.add_dependency('eventmachine', '>= 0.12.9')
-    gemspec.add_dependency('addressable', '>= 2.0.0')
-    gemspec.rubyforge_project = "em-http-request"
-    gemspec.files = FileList[`git ls-files`.split]
-  end
-
-  Jeweler::GemcutterTasks.new
-rescue LoadError
-  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
-end
+require 'rake'
+require 'rake/clean'
+require 'rake/rdoctask'
+require 'rake/gempackagetask'
+require 'fileutils'
+include FileUtils
+
+# copied from EventMachine.
+MAKE = ENV['MAKE'] || if RUBY_PLATFORM =~ /mswin/ # mingw uses make.
+  'nmake'
+else
+  'make'
+end
+
+# Default Rake task is compile
+task :default => :compile
+
+# RDoc
+Rake::RDocTask.new(:rdoc) do |task|
+  task.rdoc_dir = 'doc'
+  task.title    = 'EventMachine::HttpRequest'
+  task.options = %w(--title HttpRequest --main README --line-numbers)
+  task.rdoc_files.include(['lib/**/*.rb'])
+  task.rdoc_files.include(['README', 'LICENSE'])
+end
+
+# Rebuild parser Ragel
+task :ragel do
+  Dir.chdir "ext/http11_client" do
+    target = "http11_parser.c"
+    File.unlink target if File.exist? target
+    sh "ragel http11_parser.rl | rlgen-cd -G2 -o #{target}"
+    raise "Failed to build C source" unless File.exist? target
+  end
+end
+
+require 'spec'
+require 'spec/rake/spectask'
+Spec::Rake::SpecTask.new(:spec) do |t|
+  t.spec_opts ||= []
+  t.spec_opts << "--options" << "spec/spec.opts"
+  t.spec_files = FileList['spec/**/*_spec.rb']
+end
+
+def make(makedir)
+  Dir.chdir(makedir) { sh MAKE }
+end
+
+def extconf(dir)
+  Dir.chdir(dir) { ruby "extconf.rb" }
+end
+
+def setup_extension(dir, extension)
+  ext = "ext/#{dir}"
+  ext_so = "#{ext}/#{extension}.#{Config::MAKEFILE_CONFIG['DLEXT']}"
+  ext_files = FileList[
+    "#{ext}/*.c",
+    "#{ext}/*.h",
+    "#{ext}/extconf.rb",
+    "#{ext}/Makefile",
+    "lib"
+  ]
+
+  task "lib" do
+    directory "lib"
+  end
+
+  desc "Builds just the #{extension} extension"
+
+  mf = (extension + '_makefile').to_sym
+
+  task mf do |t|
+    extconf "#{ext}"
+  end
+
+  task extension.to_sym => [mf] do
+    make "#{ext}"
+    cp ext_so, "lib"
+  end
+end
+
+setup_extension("buffer", "em_buffer")
+setup_extension("http11_client", "http11_client")
+
+task :compile => [:em_buffer, :http11_client]
+
+CLEAN.include ['build/*', '**/*.o', '**/*.so', '**/*.a', '**/*.log', 'pkg']
+CLEAN.include ['ext/buffer/Makefile', 'lib/em_buffer.*', 'lib/http11_client.*']
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gemspec|
+    gemspec.name = "em-http-request"
+    gemspec.summary = "EventMachine based, async HTTP Request interface"
+    gemspec.description = gemspec.summary
+    gemspec.email = "ilya@igvita.com"
+    gemspec.homepage = "http://github.com/igrigorik/em-http-request"
+    gemspec.authors = ["Ilya Grigorik"]
+    gemspec.required_ruby_version = ">= 1.8.6"
+    gemspec.extensions = ["ext/buffer/extconf.rb" , "ext/http11_client/extconf.rb"]
+    gemspec.add_dependency('eventmachine', '>= 0.12.9')
+    gemspec.add_dependency('addressable', '>= 2.0.0')
+    gemspec.rubyforge_project = "em-http-request"
+    gemspec.files = FileList[`git ls-files`.split]
+  end
+
+  Jeweler::GemcutterTasks.new
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end

data/VERSION

@@ -1 +1 @@
-0.2.11
+0.2.12

data/em-http-request.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = %q{em-http-request}
-  s.version = "0.2.11"
+  s.version = "0.2.12"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Ilya Grigorik"]
-  s.date = %q{2010-08-16}
+  s.date = %q{2010-09-12}
   s.description = %q{EventMachine based, async HTTP Request interface}
   s.email = %q{ilya@igvita.com}
   s.extensions = ["ext/buffer/extconf.rb", "ext/http11_client/extconf.rb"]
@@ -19,6 +19,7 @@ Gem::Specification.new do |s|
   ]
   s.files = [
     ".gitignore",
+     "Changelog.md",
      "LICENSE",
      "README.md",
      "Rakefile",
@@ -62,7 +63,7 @@ Gem::Specification.new do |s|
   s.require_paths = ["lib"]
   s.required_ruby_version = Gem::Requirement.new(">= 1.8.6")
   s.rubyforge_project = %q{em-http-request}
-  s.rubygems_version = %q{1.3.6}
+  s.rubygems_version = %q{1.3.7}
   s.summary = %q{EventMachine based, async HTTP Request interface}
   s.test_files = [
     "spec/hash_spec.rb",
@@ -83,7 +84,7 @@ Gem::Specification.new do |s|
     current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
     s.specification_version = 3
 
-    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+    if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
       s.add_runtime_dependency(%q<eventmachine>, [">= 0.12.9"])
       s.add_runtime_dependency(%q<addressable>, [">= 2.0.0"])
     else

data/lib/em-http/client.rb

@@ -32,7 +32,7 @@ module EventMachine
 
     # HTTP response status as an integer
     def status
-      Integer(http_status) rescue nil
+      Integer(http_status) rescue 0
     end
 
     # Length of content as an integer, or nil if chunked/unspecified
@@ -265,6 +265,7 @@ module EventMachine
       # fail the connection directly
       dns_error == true ? fail(self) : unbind
     end
+    alias :close :on_error
 
     # assign a stream processing block
     def stream(&blk)
@@ -276,6 +277,11 @@ module EventMachine
       @disconnect = blk
     end
 
+    # assign a headers parse callback
+    def headers(&blk)
+      @headers = blk
+    end
+
     # raw data push from the client (WebSocket) should
     # only be invoked after handshake, otherwise it will
     # inject data into the header exchange
@@ -403,20 +409,24 @@ module EventMachine
 
     def unbind
       if (@state == :finished) && (@last_effective_url != @uri) && (@redirects < @options[:redirects])
-        # update uri to redirect location if we're allowed to traverse deeper
-        @uri = @last_effective_url
+        begin
+          # update uri to redirect location if we're allowed to traverse deeper
+          @uri = @last_effective_url
 
-        # keep track of the depth of requests we made in this session
-        @redirects += 1
+          # keep track of the depth of requests we made in this session
+          @redirects += 1
 
-        # swap current connection and reassign current handler
-        req = HttpOptions.new(@method, @uri, @options)
-        reconnect(req.host, req.port)
+          # swap current connection and reassign current handler
+          req = HttpOptions.new(@method, @uri, @options)
+          reconnect(req.host, req.port)
 
-        @response_header = HttpResponseHeader.new
-        @state = :response_header
-        @response = ''
-        @data.clear
+          @response_header = HttpResponseHeader.new
+          @state = :response_header
+          @response = ''
+          @data.clear
+        rescue EventMachine::ConnectionError => e
+          on_error(e.message, true)
+        end
       else
         if @state == :finished || (@state == :body && @bytes_remaining.nil?)
           succeed(self)
@@ -479,6 +489,10 @@ module EventMachine
     def parse_response_header
       return false unless parse_header(@response_header)
 
+      # invoke headers callback after full parse if one
+      # is specified by the user
+      @headers.call(@response_header) if @headers
+
       unless @response_header.http_status and @response_header.http_reason
         @state = :invalid
         on_error "no HTTP response"
@@ -674,3 +688,4 @@ module EventMachine
   end
 
 end
+

data/lib/em-http/http_options.rb

@@ -1,34 +1,34 @@
-class HttpOptions
-  attr_reader :uri, :method, :host, :port, :options
-
-  def initialize(method, uri, options)
-    uri.normalize!
-
-    @options = options
-    @method = method.to_s.upcase
-    @uri = uri
-
-    if proxy = options[:proxy]
-      @host = proxy[:host]
-      @port = proxy[:port]
-    else
-      # optional host for cases where you may have
-      # pre-resolved the host, or you need an override
-      @host = options[:host] || uri.host
-      @port = uri.port
-    end
-
-    @options[:timeout]    ||= 10  # default connect & inactivity timeouts
-    @options[:redirects]  ||= 0   # default number of redirects to follow
-
-    # Make sure the ports are set as Addressable::URI doesn't
-    # set the port if it isn't there
-    if uri.scheme == "https"
-      @uri.port ||= 443
-      @port     ||= 443
-    else
-      @uri.port ||= 80
-      @port     ||= 80
-    end
-  end
+class HttpOptions
+  attr_reader :uri, :method, :host, :port, :options
+
+  def initialize(method, uri, options)
+    uri.normalize!
+
+    @options = options
+    @method = method.to_s.upcase
+    @uri = uri
+
+    if proxy = options[:proxy]
+      @host = proxy[:host]
+      @port = proxy[:port]
+    else
+      # optional host for cases where you may have
+      # pre-resolved the host, or you need an override
+      @host = options.delete(:host) || uri.host
+      @port = uri.port
+    end
+
+    @options[:timeout]    ||= 10  # default connect & inactivity timeouts
+    @options[:redirects]  ||= 0   # default number of redirects to follow
+
+    # Make sure the ports are set as Addressable::URI doesn't
+    # set the port if it isn't there
+    if uri.scheme == "https"
+      @uri.port ||= 443
+      @port     ||= 443
+    else
+      @uri.port ||= 80
+      @port     ||= 80
+    end
+  end
 end

data/spec/request_spec.rb

@@ -68,17 +68,37 @@ describe EventMachine::HttpRequest do
     }
   end
 
-  it "should accept optional host override" do
-    EventMachine.run {
-      http = EventMachine::HttpRequest.new('http://google.com:8080/').get :host => '127.0.0.1'
+  context "host override" do
 
-      http.errback { failed }
-      http.callback {
-        http.response_header.status.should == 200
-        http.response.should match(/Hello/)
-        EventMachine.stop
+    it "should accept optional host" do
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://google.com:8080/').get :host => '127.0.0.1'
+
+        http.errback { failed }
+        http.callback {
+          http.response_header.status.should == 200
+          http.response.should match(/Hello/)
+          EventMachine.stop
+        }
       }
-    }
+    end
+
+    it "should reset host on redirect" do
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://127.0.0.1:8080/redirect').get :redirects => 1, :host => '127.0.0.1'
+
+        http.errback { failed }
+        http.callback {
+          http.response_header.status.should == 200
+          http.response_header["CONTENT_ENCODING"].should == "gzip"
+          http.response.should == "compressed"
+          http.last_effective_url.to_s.should == 'http://127.0.0.1:8080/gzip'
+          http.redirects.should == 1
+
+          EM.stop
+        }
+      }
+    end
   end
 
   it "should perform successfull GET with a URI passed as argument" do
@@ -246,7 +266,7 @@ describe EventMachine::HttpRequest do
     EventMachine.run {
 
       # digg.com uses chunked encoding
-      http = EventMachine::HttpRequest.new('http://digg.com/').get
+      http = EventMachine::HttpRequest.new('http://digg.com/news').get
 
       http.errback { failed }
       http.callback {
@@ -435,12 +455,11 @@ describe EventMachine::HttpRequest do
     end
 
     it "should fail gracefully on an invalid host in Location header" do
-      pending "validate tld's?"
       EventMachine.run {
         http = EventMachine::HttpRequest.new('http://127.0.0.1:8080/redirect/badhost').get :redirects => 1
         http.callback { failed }
         http.errback {
-          http.error.should == 'Location header format error'
+          http.error.should == 'unable to resolve server address'
           EM.stop
         }
       }
@@ -464,6 +483,49 @@ describe EventMachine::HttpRequest do
     }
   end
 
+  context "optional header callback" do
+    it "should optionally pass the response headers" do
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://127.0.0.1:8080/').get
+
+        http.errback { failed }
+        http.headers { |hash|
+          hash.should be_an_kind_of Hash
+          hash.should include 'CONNECTION'
+          hash.should include 'CONTENT_LENGTH'
+        }
+
+        http.callback {
+          http.response_header.status.should == 200
+          http.response.should match(/Hello/)
+          EventMachine.stop
+        }
+      }
+    end
+
+    it "should allow to terminate current connection from header callback" do
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://127.0.0.1:8080/').get
+
+        http.callback { failed }
+        http.headers { |hash|
+          hash.should be_an_kind_of Hash
+          hash.should include 'CONNECTION'
+          hash.should include 'CONTENT_LENGTH'
+
+          http.close('header callback terminated connection')
+        }
+
+        http.errback { |e|
+          http.response_header.status.should == 200
+          http.error.should == 'header callback terminated connection'
+          http.response.should == ''
+          EventMachine.stop
+        }
+      }
+    end
+  end
+
   it "should optionally pass the deflate-encoded response body progressively" do
     EventMachine.run {
       body = ''
@@ -657,8 +719,8 @@ describe EventMachine::HttpRequest do
       http = EventMachine::MockHttpRequest.new('http://www.google.ca/').get
       http.errback { fail }
       http.callback {
-        c1 = "PREF=ID=9454187d21c4a6a6:TM=1258403955:LM=1258403955:S=2-mf1n5oV5yAeT9-; expires=Wed, 16-Nov-2011 20:39:15 GMT; path=/; domain=.google.ca"
-        c2 = "NID=28=lvxxVdiBQkCetu_WFaUxLyB7qPlHXS5OdAGYTqge_laVlCKVN8VYYeVBh4bNZiK_Oan2gm8oP9GA-FrZfMPC3ZMHeNq37MG2JH8AIW9LYucU8brOeuggMEbLNNXuiWg4; expires=Tue, 18-May-2010 20:39:15 GMT; path=/; domain=.google.ca; HttpOnly"
+        c1 = "PREF=ID=11955ae9690fd292:TM=1281823106:LM=1281823106:S=wHdloFqGQ_OLSE92; expires=Mon, 13-Aug-2012 21:58:26 GMT; path=/; domain=.google.ca"
+        c2 = "NID=37=USTdOsxOSMbLjphkJ3S5Ueua3Yc23COXuK_pbztcHx7JoyhomwQySrvebCf3_u8eyrBiLWssVzaZcEOiKGEJbNdy8lRhnq_mfrdz693LaMjNPh__ccW4sgn1ZO6nQltE; expires=Sun, 13-Feb-2011 21:58:26 GMT; path=/; domain=.google.ca; HttpOnly"
         http.response_header.cookie.should == [c1, c2]
 
         EventMachine.stop

metadata

@@ -5,8 +5,8 @@ version: !ruby/object:Gem::Version
   segments: 
   - 0
   - 2
-  - 11
-  version: 0.2.11
+  - 12
+  version: 0.2.12
 platform: ruby
 authors: 
 - Ilya Grigorik
@@ -14,13 +14,14 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2010-08-16 00:00:00 -04:00
+date: 2010-09-12 00:00:00 -04:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
   name: eventmachine
   prerelease: false
   requirement: &id001 !ruby/object:Gem::Requirement 
+    none: false
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
@@ -35,6 +36,7 @@ dependencies:
   name: addressable
   prerelease: false
   requirement: &id002 !ruby/object:Gem::Requirement 
+    none: false
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
@@ -57,6 +59,7 @@ extra_rdoc_files:
 - README.md
 files: 
 - .gitignore
+- Changelog.md
 - LICENSE
 - README.md
 - Rakefile
@@ -104,6 +107,7 @@ rdoc_options:
 require_paths: 
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
@@ -113,6 +117,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       - 6
       version: 1.8.6
 required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
@@ -122,7 +127,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 
 rubyforge_project: em-http-request
-rubygems_version: 1.3.6
+rubygems_version: 1.3.7
 signing_key: 
 specification_version: 3
 summary: EventMachine based, async HTTP Request interface