em-http-request 0.2.10 → 0.2.14

data/Changelog.md

@@ -0,0 +1,36 @@
+# Changelog
+
+## 0.2.14 / 2010-10-06
+
+- bugfix: form-encode keys/values of ruby objects passed in as body
+
+## 0.2.13 / 2010-09-25
+
+- added SOCKS5 proxy support
+- bugfix: follow redirects on HEAD requests
+
+## 0.2.12 / 2010-09-12
+
+- added headers callback (http.headers {|h| p h})
+- added .close method on client obj to terminate session (accepts message)
+
+- bugfix: report 0 for response status on 1.9 on timeouts
+- bugfix: handle bad Location host redirects
+- bugfix: reset host override on connect
+
+## 0.2.11 / 2010-08-16
+
+- all URIs are now normalized prior to dispatch (and on redirect)
+- default to direct proxy (instead of CONNECT handshake) - better performance
+  - specify :proxy => {:tunnel => true} if you need to force CONNECT route
+- MultiRequest accepts block syntax for dispatching parallel requests (see specs)
+- MockHttpRequest accepts block syntax (see Mock wiki page)
+
+
+- bugfix: nullbyte frame for websockets
+- bugfix: set @uri on DNS resolve failure
+- bugfix: handle bad hosts in absolute redirects
+- bugfix: reset seen content on redirects (doh!)
+- bugfix: invalid multibyte escape in websocket regex (1.9.2+)
+- bugfix: report etag and last_modified headers correctly
+

data/LICENSE

@@ -1,58 +1,58 @@
-Ruby is copyrighted free software by Yukihiro Matsumoto <matz@netlab.co.jp>.
-You can redistribute it and/or modify it under either the terms of the GPL
-(see COPYING.txt file), or the conditions below:
-
-  1. You may make and give away verbatim copies of the source form of the
-     software without restriction, provided that you duplicate all of the
-     original copyright notices and associated disclaimers.
-
-  2. You may modify your copy of the software in any way, provided that
-     you do at least ONE of the following:
-
-       a) place your modifications in the Public Domain or otherwise
-          make them Freely Available, such as by posting said
-	  modifications to Usenet or an equivalent medium, or by allowing
-	  the author to include your modifications in the software.
-
-       b) use the modified software only within your corporation or
-          organization.
-
-       c) rename any non-standard executables so the names do not conflict
-	  with standard executables, which must also be provided.
-
-       d) make other distribution arrangements with the author.
-
-  3. You may distribute the software in object code or executable
-     form, provided that you do at least ONE of the following:
-
-       a) distribute the executables and library files of the software,
-	  together with instructions (in the manual page or equivalent)
-	  on where to get the original distribution.
-
-       b) accompany the distribution with the machine-readable source of
-	  the software.
-
-       c) give non-standard executables non-standard names, with
-          instructions on where to get the original software distribution.
-
-       d) make other distribution arrangements with the author.
-
-  4. You may modify and include the part of the software into any other
-     software (possibly commercial).  But some files in the distribution
-     are not written by the author, so that they are not under this terms.
-
-     They are gc.c(partly), utils.c(partly), regex.[ch], st.[ch] and some
-     files under the ./missing directory.  See each file for the copying
-     condition.
-
-  5. The scripts and library files supplied as input to or produced as 
-     output from the software do not automatically fall under the
-     copyright of the software, but belong to whomever generated them, 
-     and may be sold commercially, and may be aggregated with this
-     software.
-
-  6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
-     IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
-     WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
-     PURPOSE.
-
+Ruby is copyrighted free software by Yukihiro Matsumoto <matz@netlab.co.jp>.
+You can redistribute it and/or modify it under either the terms of the GPL
+(see COPYING.txt file), or the conditions below:
+
+  1. You may make and give away verbatim copies of the source form of the
+     software without restriction, provided that you duplicate all of the
+     original copyright notices and associated disclaimers.
+
+  2. You may modify your copy of the software in any way, provided that
+     you do at least ONE of the following:
+
+       a) place your modifications in the Public Domain or otherwise
+          make them Freely Available, such as by posting said
+	  modifications to Usenet or an equivalent medium, or by allowing
+	  the author to include your modifications in the software.
+
+       b) use the modified software only within your corporation or
+          organization.
+
+       c) rename any non-standard executables so the names do not conflict
+	  with standard executables, which must also be provided.
+
+       d) make other distribution arrangements with the author.
+
+  3. You may distribute the software in object code or executable
+     form, provided that you do at least ONE of the following:
+
+       a) distribute the executables and library files of the software,
+	  together with instructions (in the manual page or equivalent)
+	  on where to get the original distribution.
+
+       b) accompany the distribution with the machine-readable source of
+	  the software.
+
+       c) give non-standard executables non-standard names, with
+          instructions on where to get the original software distribution.
+
+       d) make other distribution arrangements with the author.
+
+  4. You may modify and include the part of the software into any other
+     software (possibly commercial).  But some files in the distribution
+     are not written by the author, so that they are not under this terms.
+
+     They are gc.c(partly), utils.c(partly), regex.[ch], st.[ch] and some
+     files under the ./missing directory.  See each file for the copying
+     condition.
+
+  5. The scripts and library files supplied as input to or produced as 
+     output from the software do not automatically fall under the
+     copyright of the software, but belong to whomever generated them, 
+     and may be sold commercially, and may be aggregated with this
+     software.
+
+  6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
+     IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
+     WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+     PURPOSE.
+

data/README.md

@@ -0,0 +1,168 @@
+EM-HTTP-Request
+===============
+
+Asynchronous HTTP client for Ruby, based on EventMachine runtime.
+
+- Ragel HTTP parser for speed & performance
+- Simple interface for single & parallel requests via deferred callbacks
+- Automatic gzip & deflate decoding
+- Basic-Auth & OAuth support
+- Custom timeout support
+- Stream response processing
+- Proxy support (with SSL Tunneling): CONNECT, direct & SOCKS5
+- Auto-follow 3xx redirects with custom max depth
+- Bi-directional communication with web-socket services
+- [Native mocking support](http://wiki.github.com/igrigorik/em-http-request/mocking-httprequest) and through [Webmock](http://github.com/bblimke/webmock)
+
+Getting started
+---------------
+
+      gem install em-http-request
+      irb:0> require 'em-http'
+
+Or checkout [screencast / demo](http://everburning.com/news/eventmachine-screencast-em-http-request/) of using EM-HTTP-Request.
+
+Libraries & Applications using em-http
+--------------------------------------
+
+- [chirpstream](http://github.com/joshbuddy/chirpstream) - EM client for Twitters Chirpstream API
+- [RDaneel](http://github.com/hasmanydevelopers/RDaneel) - Ruby crawler which respects robots.txt
+- [rsolr-async](http://github.com/mwmitchell/rsolr-async) - An asynchronus connection adapter for RSolr
+- [PubSubHubbub](http://github.com/igrigorik/PubSubHubbub) - Asynchronous PubSubHubbub ruby client
+- [Firering](http://github.com/EmmanuelOga/firering) - Eventmachine powered Campfire API
+- and many others.. drop me a link if you want yours included!
+
+Simple client example
+---------------------
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://127.0.0.1/').get :query => {'keyname' => 'value'}, :timeout => 10
+
+        http.callback {
+          p http.response_header.status
+          p http.response_header
+          p http.response
+
+          EventMachine.stop
+        }
+      }
+
+Multi-request example
+---------------------
+
+Fire and wait for multiple requests to complete via the MultiRequest interface.
+
+      EventMachine.run {
+        multi = EventMachine::MultiRequest.new
+
+        # add multiple requests to the multi-handler
+        multi.add(EventMachine::HttpRequest.new('http://www.google.com/').get)
+        multi.add(EventMachine::HttpRequest.new('http://www.yahoo.com/').get)
+
+        multi.callback  {
+          p multi.responses[:succeeded]
+          p multi.responses[:failed]
+
+          EventMachine.stop
+        }
+      }
+
+Basic-Auth example
+------------------
+
+Full basic author support. For OAuth, check examples/oauth-tweet.rb file.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.website.com/').get :head => {'authorization' => ['user', 'pass']}
+
+        http.errback { failed }
+        http.callback {
+          p http.response_header
+          EventMachine.stop
+        }
+      }
+
+
+POSTing data example
+--------------------
+
+      EventMachine.run {
+        http1 = EventMachine::HttpRequest.new('http://www.website.com/').post :body => {"key1" => 1, "key2" => [2,3]}
+        http2 = EventMachine::HttpRequest.new('http://www.website.com/').post :body => "some data"
+
+        # ...
+      }
+
+Streaming body processing
+-------------------------
+
+Allows you to consume an HTTP stream of content in real-time. Each time a new piece of content is pushed
+to the client, it is passed to the stream callback for you to operate on.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.website.com/').get
+        http.stream { |chunk| print chunk }
+      }
+
+Streaming files from disk
+-------------------------
+Allows you to efficiently stream a (large) file from disk via EventMachine's FileStream interface.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.website.com/').post :file => 'largefile.txt'
+        http.callback { |chunk| puts "Upload finished!" }
+      }
+
+Proxy example
+-------------
+Full transparent proxy support with support for SSL tunneling.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.website.com/').get :proxy => {
+          :host => 'www.myproxy.com',
+          :port => 8080,
+          :authorization => ['username', 'password'] # authorization is optional
+      }
+
+SOCKS5 Proxy example
+-------------
+Tunnel your requests via connect via SOCKS5 proxies (ssh -D port somehost).
+
+    EventMachine.run {
+      http = EventMachine::HttpRequest.new('http://www.website.com/').get :proxy => {
+        :host => 'www.myproxy.com',
+        :port => 8080,
+        :type => :socks
+    }
+
+Auto-follow 3xx redirects
+-------------------------
+
+Specify the max depth of redirects to follow, default is 0.
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new('http://www.google.com/').get :redirects => 1
+        http.callback { p http.last_effective_url }
+      }
+
+WebSocket example
+-----------------
+
+[Bi-directional communication with WebSockets](http://www.igvita.com/2009/12/22/ruby-websockets-tcp-for-the-browser/): simply pass in a ws:// resource and the client will negotiate the connection upgrade for you. On successful handshake the callback is invoked, and any incoming messages will be passed to the stream callback. The client can also send data to the server at will by calling the "send" method!
+
+      EventMachine.run {
+        http = EventMachine::HttpRequest.new("ws://yourservice.com/websocket").get :timeout => 0
+
+        http.errback { puts "oops" }
+        http.callback {
+          puts "WebSocket connected!"
+          http.send("Hello client")
+        }
+
+        http.stream { |msg|
+          puts "Recieved: #{msg}"
+          http.send "Pong: #{msg}"
+        }
+
+        http.disconnect { puts "oops, dropped connection?" }
+      }

data/Rakefile

@@ -19,9 +19,9 @@ task :default => :compile
 Rake::RDocTask.new(:rdoc) do |task|
   task.rdoc_dir = 'doc'
   task.title    = 'EventMachine::HttpRequest'
-  task.options = %w(--title HttpRequest --main README --line-numbers)
+  task.options = %w(--title HttpRequest --main README.md --line-numbers)
   task.rdoc_files.include(['lib/**/*.rb'])
-  task.rdoc_files.include(['README', 'LICENSE'])
+  task.rdoc_files.include(['README.md', 'LICENSE'])
 end
 
 # Rebuild parser Ragel
@@ -34,8 +34,12 @@ task :ragel do
   end
 end
 
-task :spec do
-  sh 'spec spec/*_spec.rb'
+require 'spec'
+require 'spec/rake/spectask'
+Spec::Rake::SpecTask.new(:spec) do |t|
+  t.spec_opts ||= []
+  t.spec_opts << "--options" << "spec/spec.opts"
+  t.spec_files = FileList['spec/**/*_spec.rb']
 end
 
 def make(makedir)
@@ -48,7 +52,7 @@ end
 
 def setup_extension(dir, extension)
   ext = "ext/#{dir}"
-  ext_so = "#{ext}/#{extension}.#{Config::CONFIG['DLEXT']}"
+  ext_so = "#{ext}/#{extension}.#{Config::MAKEFILE_CONFIG['DLEXT']}"
   ext_files = FileList[
     "#{ext}/*.c",
     "#{ext}/*.h",

data/VERSION

@@ -1 +1 @@
-0.2.10
+0.2.14

data/em-http-request.gemspec

@@ -5,28 +5,30 @@
 
 Gem::Specification.new do |s|
   s.name = %q{em-http-request}
-  s.version = "0.2.10"
+  s.version = "0.2.14"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Ilya Grigorik"]
-  s.date = %q{2010-07-07}
+  s.date = %q{2010-10-06}
   s.description = %q{EventMachine based, async HTTP Request interface}
   s.email = %q{ilya@igvita.com}
   s.extensions = ["ext/buffer/extconf.rb", "ext/http11_client/extconf.rb"]
   s.extra_rdoc_files = [
     "LICENSE",
-     "README.rdoc"
+     "README.md"
   ]
   s.files = [
     ".gitignore",
+     "Changelog.md",
      "LICENSE",
-     "README.rdoc",
+     "README.md",
      "Rakefile",
      "VERSION",
      "em-http-request.gemspec",
      "examples/fetch.rb",
      "examples/fibered-http.rb",
      "examples/oauth-tweet.rb",
+     "examples/socks5.rb",
      "examples/websocket-handler.rb",
      "examples/websocket-server.rb",
      "ext/buffer/em_buffer.c",
@@ -41,18 +43,18 @@ Gem::Specification.new do |s|
      "lib/em-http.rb",
      "lib/em-http/client.rb",
      "lib/em-http/core_ext/bytesize.rb",
-     "lib/em-http/core_ext/hash.rb",
      "lib/em-http/decoders.rb",
      "lib/em-http/http_options.rb",
      "lib/em-http/mock.rb",
      "lib/em-http/multi.rb",
      "lib/em-http/request.rb",
+     "spec/encoding_spec.rb",
      "spec/fixtures/google.ca",
-     "spec/hash_spec.rb",
      "spec/helper.rb",
      "spec/mock_spec.rb",
      "spec/multi_spec.rb",
      "spec/request_spec.rb",
+     "spec/spec.opts",
      "spec/stallion.rb",
      "spec/stub_server.rb"
   ]
@@ -61,10 +63,10 @@ Gem::Specification.new do |s|
   s.require_paths = ["lib"]
   s.required_ruby_version = Gem::Requirement.new(">= 1.8.6")
   s.rubyforge_project = %q{em-http-request}
-  s.rubygems_version = %q{1.3.6}
+  s.rubygems_version = %q{1.3.7}
   s.summary = %q{EventMachine based, async HTTP Request interface}
   s.test_files = [
-    "spec/hash_spec.rb",
+    "spec/encoding_spec.rb",
      "spec/helper.rb",
      "spec/mock_spec.rb",
      "spec/multi_spec.rb",
@@ -74,6 +76,7 @@ Gem::Specification.new do |s|
      "examples/fetch.rb",
      "examples/fibered-http.rb",
      "examples/oauth-tweet.rb",
+     "examples/socks5.rb",
      "examples/websocket-handler.rb",
      "examples/websocket-server.rb"
   ]
@@ -82,7 +85,7 @@ Gem::Specification.new do |s|
     current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
     s.specification_version = 3
 
-    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+    if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
       s.add_runtime_dependency(%q<eventmachine>, [">= 0.12.9"])
       s.add_runtime_dependency(%q<addressable>, [">= 2.0.0"])
     else

data/examples/fetch.rb

@@ -1,30 +1,30 @@
-require 'rubygems'
-require 'eventmachine'
-require '../lib/em-http'
-
-urls = ARGV
-if urls.size < 1
-  puts "Usage: #{$0} <url> <url> <...>"
-  exit
-end
-
-pending = urls.size
-
-EM.run do
-  urls.each do |url|
-    http = EM::HttpRequest.new(url).get
-    http.callback {
-      puts "#{url}\n#{http.response_header.status} - #{http.response.length} bytes\n"
-      puts http.response
-
-      pending -= 1
-      EM.stop if pending < 1
-    }
-    http.errback {
-      puts "#{url}\n" + http.error
-
-      pending -= 1
-      EM.stop if pending < 1
-    }
-  end
-end
+require 'rubygems'
+require 'eventmachine'
+require '../lib/em-http'
+
+urls = ARGV
+if urls.size < 1
+  puts "Usage: #{$0} <url> <url> <...>"
+  exit
+end
+
+pending = urls.size
+
+EM.run do
+  urls.each do |url|
+    http = EM::HttpRequest.new(url).get
+    http.callback {
+      puts "#{url}\n#{http.response_header.status} - #{http.response.length} bytes\n"
+      puts http.response
+
+      pending -= 1
+      EM.stop if pending < 1
+    }
+    http.errback {
+      puts "#{url}\n" + http.error
+
+      pending -= 1
+      EM.stop if pending < 1
+    }
+  end
+end

data/examples/fibered-http.rb

@@ -1,39 +1,39 @@
-require 'eventmachine'
-require 'em-http'
-require 'fiber'
-
-# Using Fibers in Ruby 1.9 to simulate blocking IO / IO scheduling
-# while using the async EventMachine API's
-
-def async_fetch(url)
-  f = Fiber.current
-  http = EventMachine::HttpRequest.new(url).get :timeout => 10
-
-  http.callback { f.resume(http) }
-  http.errback  { f.resume(http) }
-
-  return Fiber.yield
-end
-
-EventMachine.run do
-  Fiber.new{
-
-    puts "Setting up HTTP request #1"
-    data = async_fetch('http://www.google.com/')
-    puts "Fetched page #1: #{data.response_header.status}"
-
-    puts "Setting up HTTP request #2"
-    data = async_fetch('http://www.yahoo.com/')
-    puts "Fetched page #2: #{data.response_header.status}"
-
-    EventMachine.stop
-  }.resume
-end
-
-puts "Done"
-
-#  Setting up HTTP request #1
-#  Fetched page #1: 302
-#  Setting up HTTP request #2
-#  Fetched page #2: 200
+require 'eventmachine'
+require 'em-http'
+require 'fiber'
+
+# Using Fibers in Ruby 1.9 to simulate blocking IO / IO scheduling
+# while using the async EventMachine API's
+
+def async_fetch(url)
+  f = Fiber.current
+  http = EventMachine::HttpRequest.new(url).get :timeout => 10
+
+  http.callback { f.resume(http) }
+  http.errback  { f.resume(http) }
+
+  return Fiber.yield
+end
+
+EventMachine.run do
+  Fiber.new{
+
+    puts "Setting up HTTP request #1"
+    data = async_fetch('http://www.google.com/')
+    puts "Fetched page #1: #{data.response_header.status}"
+
+    puts "Setting up HTTP request #2"
+    data = async_fetch('http://www.yahoo.com/')
+    puts "Fetched page #2: #{data.response_header.status}"
+
+    EventMachine.stop
+  }.resume
+end
+
+puts "Done"
+
+#  Setting up HTTP request #1
+#  Fetched page #1: 302
+#  Setting up HTTP request #2
+#  Fetched page #2: 200
 #  Done