RubyGems - tap-http - Versions diffs - 0.0.1 → 0.1.0 - Mend

tap-http 0.0.1 → 0.1.0

Files changed (8) hide show

data/README CHANGED Viewed

@@ -1,15 +1,111 @@
-= {TapHttp}[http://tap.rubyforge.org/tap-http]
+= {TapHttp}[http://tap.rubyforge.org/projects/tap-http]
 A task library for submitting http requests using {Tap}[http://tap.rubyforge.org].
 == Description
+TapHttp provides modules to construct and submit HTTP requests from a hash
+that specifies the target url, headers, parameters, etc.  TapHttp is
+designed to work with a {Ubiquity}[http://labs.mozilla.com/2008/08/introducing-ubiquity/]
+command called {redirect-http}[http://gist.github.com/25932]; together
+they allow the capture and resubmission of web forms.
 * Lighthouse[http://bahuvrihi.lighthouseapp.com/projects/9908-tap-task-application/tickets]
 * Github[http://github.com/bahuvrihi/tap-http/tree/master]
 * {Google Group}[http://groups.google.com/group/ruby-on-tap]
 === Usage
+TapHttp submits http requests using the Tap::Http::Dispatch module.  Headers,
+parameters, and other configurations may be specified, but the only required
+field is :url.
+  include Tap::Http
+  res = Dispatch.submit_request(
+    :params => {'q' => 'tap-http'},
+    :url => 'http://www.google.com/search')
+  res.body[0,80]  # => "<!doctype html><head><title>tap-http - Google Search</title><style>body{backgrou"
+=== Getting Http Configurations
+More complicated http requests may be captured and resubmitted using a
+combination of tools that redirects web forms to a tap server and reformats
+the request as YAML. To do so:
+* Install {Firefox}[http://www.mozilla.com/en-US/firefox/]
+* Install {Ubiquity}[http://labs.mozilla.com/2008/08/introducing-ubiquity/]
+* Install {redirect-http}[http://gist.github.com/25932]
+Start a tap server from the command line (of course tap-http must be installed):
+  % tap server
+Now in the browser, go to a web form like {google}[http://www.google.com/] and
+invoke the redirection.
+* Bring up Ubiquity in Firefox by pressing 'option+space'
+* Enter the command: 'redirect-http http://localhost:8080/http_to_yaml'
+You should see a notice that the form is being redirected.  Fill out the form
+and submit as normal; the redirect command will send the form to the tap server
+instead of performing the original action.  The tap server returns a yaml file
+with the http configuration.
+  # Copy and paste into a configuration file. Multiple configs
+  # can be added to a single file to perform batch submission.
+  - headers:
+      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
+      Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
+      Accept-Encoding: gzip,deflate
+      Accept-Language: en-us,en;q=0.5
+      Connection: keep-alive
+      Host: www.google.com
+      Keep-Alive: "300"
+      Referer: http://www.google.com/
+      User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4
+    params:
+      aq: f
+      btnG: Google Search
+      hl: en
+      oq: ""
+      q: tap-http
+    request_method: GET
+    url: http://www.google.com/search
+    version: "1.1"
+Save the file as 'request.yml' and resubmit the form using the Tap::Http::Request
+task.
+  % rap load requests.yml --:i request --+ dump --no-audit
+    I[10:51:40]               load request.yml
+    I[10:51:40]                GET http://www.google.com/search
+    I[10:51:41]                    OK
+  # date: 2008-11-25 10:51:41
+  ---
+  tap/http/request (2772040):
+  - - !ruby/object:Net::HTTPOK
+      body: !binary |
+        H4sIAAAAAAAC/6xabXPbNhL+3l/B0BeN1FAUJfktoihf07pppmkm06TXu0lz
+        HZAEScQgQZOQZVfhf79dgBRJS4ndmRvPSAC42F3sPvsCyssnoQjkXU6NRKZ8
+        tUwoCVdLySSnK0nycSJlboyNl0LEnBrvKCmCZDnRz5elvIMvX4R3W58EV3Eh
+        1lm4OIqiyA0EF8XiyHEc...
+Note the result is encoded as gzip, as per the parameters. As with all tasks,
+the request results could be passed into a workflow. Alternatively, the
+configuration could be used to submit the request using Dispatch.
+=== Bugs/Known Issues
+The Tap::Http::Helpers#parse_cgi_request (used in parsing redirected requests
+into a YAML file) is currently untested because I can't figure a way to setup
+the ENV variables in a standard way.  Of course I could set them up myself, but
+I can't be sure I'm setting up a realistic test environment.
+The capture procedure seems to work in practice, but please report bugs and let
+me know if you know a way to setup a CGI environment for testing!
 == Installation
 TapHttp is available as a gem on RubyForge[http://rubyforge.org/projects/tap].  Use:

data/cgi/echo.rb CHANGED Viewed

@@ -1,7 +1,7 @@
 #!/usr/local/bin/ruby
 ####################################
-# Echos back the HTTP header and parameters as YAML.
+# Echos back the HTTP header and parameters as YAML.
 #
 # Copyright (c) 2008, Regents of the University of Colorado
 # Developer: Simon Chiang, Biomolecular Structure Program
@@ -14,8 +14,8 @@ require 'tap/http/helpers'
 cgi = CGI.new
 cgi.out("text/plain") do
-  begin
-    request = Tap::Http::Helpers.parse_cgi_request(cgi)
+  begin
+    request = Tap::Http::Helpers.parse_cgi_request(cgi)
     request[:headers].to_yaml + request[:params].to_yaml
   rescue
     "Error: #{$!.message}\n"  +

data/cgi/http_to_yaml.rb CHANGED Viewed

@@ -88,14 +88,14 @@ begin
   # format output
   #
-  help = cgi.a(Tap::Http::Helpers::HELP_URL) { "help" }
-  how_to_get_cookies = cgi.a(Tap::Http::Helpers::COOKIES_HELP_URL) { "how to get cookies" }
+  # help = cgi.a(Tap::Http::Helpers::HELP_URL) { "help" }
+  # how_to_get_cookies = cgi.a(Tap::Http::Helpers::COOKIES_HELP_URL) { "how to get cookies" }
+  #
+  # If you need cookies, see #{how_to_get_cookies} or the #{help}.
   cgi.out('text/plain') do
 %Q{# Copy and paste into a configuration file. Multiple configs
 # can be added to a single file to perform batch submission.
-#
-# If you need cookies, see #{how_to_get_cookies} or the #{help}.
 - #{config.to_yaml[5..-1].gsub(/\n/, "\n  ")}
 }
   end

data/lib/tap/http/dispatch.rb CHANGED Viewed

@@ -1,53 +1,60 @@
 require 'tap/http/helpers'
 require 'net/http'
-#module Net
-#  class HTTP
-# attr_reader :socket
-# end
-# class BufferedIO
-#include Prosperity::Acts::Monitorable
-#private
-#def rbuf_fill
-#  tick_monitor
-#  timeout(@read_timeout) {
-#    @rbuf << @io.sysread(1024)
-#  }
-#end
-#end
-#end
 module Tap
   module Http
     # Dispatch provides methods for constructing and submitting get and post
-    # HTTP requests.
+    # HTTP requests from a configuration hash.
+    #
+    #   res = Tap::Http::Dispatch.submit_request(
+    #     :url => "http://tap.rubyforge.org",
+    #     :version => '1.1',
+    #     :request_method => 'GET',
+    #     :headers => {},
+    #     :params => {}
+    #   )
+    #   res.inspect           # => "#<Net::HTTPOK 200 OK readbody=true>"
+    #   res.body =~ /Tap/     # => true
+    #
+    # Headers and parameters take the form:
+    #
+    #   { 'single' => 'value',
+    #     'multiple' => ['value one', 'value two']}
+    #
     module Dispatch
-      REQUEST_KEYS = [:url, :request_method, :headers, :params, :redirection_limit]
       module_function
-      # Constructs and submits a request to the url using the headers and parameters.
-      # Returns the response from the submission.
+      DEFAULT_CONFIG = {
+        :request_method => 'GET',
+        :version => '1.1',
+        :params => {},
+        :headers => {},
+        :redirection_limit => nil
+      }
+      # Constructs and submits a request to the url using the request configuration.
+      # A url must be specified in the configuration, but other configurations are
+      # optional; if unspecified, the values in DEFAULT_CONFIG will be used.  A
+      # block may be given to receive the Net::HTTP and request just prior to
+      # submission.
       #
-      #   res = submit_request("http://www.google.com/search", {:request_method => 'get'}, {:q => 'tap rubyforge'})
-      #   # => <Net::HTTPOK 200 OK readbody=true>
+      # Returns the response from the submission.
       #
-      # Notes:
-      # - A request method must be specified in the headers; currently only get and
-      #   post are supported.  See construct_post for supported post content types.
-      # - A url or a url (as would result from URI.parse(url)) can be provided to
-      #   submit request.  The Net::HTTP object performing the submission is passed
-      #   to the block, if given, before the request is made.
-      def submit_request(config) # :yields: http
+      def submit_request(config)
+        symbolized = DEFAULT_CONFIG.dup
+        config.each_pair do |key, value|
+          symbolized[key.to_sym] = value
+        end
+        config = symbolized
+        request_method = (config[:request_method]).to_s
         url_or_uri = config[:url]
-        params = config[:params] || {}
-        headers = headerize_keys( config[:headers] || {})
-        request_method = (config[:request_method] || 'GET').to_s
+        version = config[:version]
+        params = config[:params]
+        headers = headerize_keys(config[:headers])
+        raise ArgumentError, "no url specified" unless url_or_uri
         uri = url_or_uri.kind_of?(URI) ? url_or_uri : URI.parse(url_or_uri)
         uri.path = "/" if uri.path.empty?
@@ -56,22 +63,20 @@ module Tap
         when /^get$/i then construct_get(uri, headers, params)
         when /^post$/i then construct_post(uri, headers, params)
         else
-          raise ArgumentError.new("Missing or unsupported request_method: #{request_method}")
+          raise ArgumentError, "unsupported request method: #{request_method}"
         end
         # set the http version
-        # if version = config[:http_version]
-        #   version_method = "version_#{version.to_s.gsub(".", "_")}".to_sym
-        #   if Object::Net::HTTP.respond_to?(version_method)
-        #     Object::Net::HTTP.send(version_method)
-        #   else
-        #     raise ArgumentError.new("unsupported http_version: #{version}")
-        #   end
-        # end
+        version_method = "version_#{version.to_s.gsub(".", "_")}".to_sym
+        if ::Net::HTTP.respond_to?(version_method)
+          ::Net::HTTP.send(version_method)
+        else
+          raise ArgumentError, "unsupported HTTP version: #{version}"
+        end
         # submit the request
-        res = Object::Net::HTTP.new(uri.host, uri.port).start do |http|
-          yield(http) if block_given?
+        res = ::Net::HTTP.new(uri.host, uri.port).start do |http|
+          yield(http, request) if block_given?
           http.request(request)
         end
@@ -80,120 +85,175 @@ module Tap
         redirection_limit ? fetch_redirection(res, redirection_limit) : res
       end
-      # Constructs a post query.  The 'Content-Type' header determines the format of the body
-      # content.  If the content type is 'multipart/form-data', then the parameters will be
-      # formatted using the boundary in the content-type header, if provided, or a randomly
-      # generated boundary.
+      # Constructs a Net::HTTP::Post query, setting headers and parameters.
       #
-      # Headers for the request are set in this method.  If uri contains query parameters,
-      # they will be included in the request URI.
+      # ==== Supported Content Types:
       #
-      # Supported content-types:
-      # -  application/x-www-form-urlencoded (the default)
-      # -  multipart/form-data
+      # - application/x-www-form-urlencoded (the default)
+      # - multipart/form-data
+      #
+      # The multipart/form-data content type may specify a boundary.  If no
+      # boundary is specified, a randomly generated boundary will be used
+      # to delimit the parameters.
+      #
+      #   post = construct_post(
+      #     URI.parse('http://some.url/'),
+      #     {:content_type => 'multipart/form-data; boundary=1234'},
+      #     {:key => 'value'})
+      #
+      #   post.body
+      #   # => %Q{--1234\r
+      #   # Content-Disposition: form-data; name="key"\r
+      #   # \r
+      #   # value\r
+      #   # --1234--\r
+      #   # }
+      #
+      # (Note the carriage returns are required in multipart content)
+      #
+      # The content-length header is determined automatically from the
+      # formatted request body; manually specified content-length headers
+      # will be overridden.
       #
       def construct_post(uri, headers, params)
-        req = Object::Net::HTTP::Post.new( URI.encode("#{uri.path}#{construct_query(uri)}") )
+        req = ::Net::HTTP::Post.new( URI.encode("#{uri.path}#{format_query(uri)}") )
         headers = headerize_keys(headers)
         content_type = headers['Content-Type']
         case content_type
-        when /multipart\/form-data/i
+        when nil, /^application\/x-www-form-urlencoded$/i
+          req.body = format_www_form_urlencoded(params)
+          headers['Content-Type'] ||= "application/x-www-form-urlencoded"
+          headers['Content-Length'] = req.body.length
+        when /^multipart\/form-data(;\s*boundary=(.*))?$/i
           # extract the boundary if it exists
-          content_type =~ /boundary=(.*)/i
-          boundary = $1 || rand.to_s[2..20]
+          boundary = $2 || rand.to_s[2..20]
-          req.body = format_multipart_form_data(boundary, params)
+          req.body = format_multipart_form_data(params, boundary)
           headers['Content-Type'] = "multipart/form-data; boundary=#{boundary}"
           headers['Content-Length'] = req.body.length
         else
-          req.body = format_www_form_urlencoded(params)
-          headers['Content-Type'] = "application/x-www-form-urlencoded"
-          headers['Content-Length'] = req.body.length
+          raise ArgumentError, "unsupported Content-Type for POST: #{content_type}"
         end
         headers.each_pair { |key, value| req[key] = value }
         req
       end
-      # Constructs a get query.  All parameters in uri and params are added to the
-      # request URI.  Headers for the request are set in this method.
+      # Constructs a Net::HTTP::Get query.  All parameters in uri and params are
+      # encoded and added to the request URI.
+      #
+      #   get = construct_get(URI.parse('http://some.url/path'), {}, {:key => 'value'})
+      #   get.path            # => "/path?key=value"
+      #
       def construct_get(uri, headers, params)
-        req = Object::Net::HTTP::Get.new( URI.encode("#{uri.path}#{construct_query(uri, params)}") )
+        req = ::Net::HTTP::Get.new( URI.encode("#{uri.path}#{format_query(uri, params)}") )
         headerize_keys(headers).each_pair { |key, value| req[key] = value }
         req
       end
-      # Checks the type of the response; if it is a redirection, get the redirection,
-      # otherwise return the response.
+      # Checks the type of the response; if it is a redirection, fetches the
+      # redirection.  Otherwise return the response.
       #
       # Notes:
-      # - The redirections will only recurse for the input redirection limit (default 10)
-      # - Responses that are not Net::HTTPRedirection or Net::HTTPSuccess raise an error.
+      # - Fetch will recurse up to the input redirection limit (default 10)
+      # - Responses that are not Net::HTTPRedirection or Net::HTTPSuccess
+      #   raise an error.
       def fetch_redirection(res, limit=10)
-        raise ArgumentError, 'Could not follow all the redirections.' if limit == 0
+        raise 'exceeded the redirection limit' if limit < 1
         case res
-        when Object::Net::HTTPRedirection
-          redirect = Object::Net::HTTP.get_response( URI.parse(res['location']) )
+        when ::Net::HTTPRedirection
+          redirect = ::Net::HTTP.get_response( URI.parse(res['location']) )
           fetch_redirection(redirect, limit - 1)
-        when Object::Net::HTTPSuccess then res
-        else raise StandardError, res.error!
+        when ::Net::HTTPSuccess
+          res
+        else
+          raise StandardError, res.error!
         end
       end
-      # Normalizes the header keys to a titleized, dasherized string.
-      #   'some_header' => 'Some-Header'
-      #   :some_header => 'Some-Header'
-      #   'some header' => 'Some-Header'
-      def headerize_keys(headers)
+      # Converts the keys of a hash to headers.  See Helpers#headerize.
+      #
+      #   headerize_keys('some_header' => 'value')   # => {'Some-Header' => 'value'}
+      #
+      def headerize_keys(hash)
         result = {}
-        headers.each_pair do |key, value|
+        hash.each_pair do |key, value|
           result[Helpers.headerize(key)] = value
         end
         result
       end
-      # Returns a URI query constructed from the query in uri and the input parameters.
+      # Constructs a URI query string from the uri and the input parameters.
+      # Multiple values for a parameter may be specified using an array.
       # The query is not encoded, so you may need to URI.encode it later.
-      def construct_query(uri, params={})
+      #
+      #   format_query(URI.parse('http://some.url/path'), {:key => 'value'})
+      #   # => "?key=value"
+      #
+      #   format_query(URI.parse('http://some.url/path?one=1'), {:two => '2'})
+      #   # => "?one=1&two=2"
+      #
+      def format_query(uri, params={})
         query = []
+        query << uri.query if uri.query
         params.each_pair do |key, values|
-          values = values.kind_of?(Array) ? values : [values]
+          values = [values] unless values.kind_of?(Array)
           values.each { |value| query << "#{key}=#{value}" }
         end
-        query << uri.query if uri.query
         "#{query.empty? ? '' : '?'}#{query.join('&')}"
       end
+      # Formats params as 'application/x-www-form-urlencoded' for use as the
+      # body of a post request.  Multiple values for a parameter may be
+      # specified using an array.  The result is obviously URI encoded.
+      #
+      #   format_www_form_urlencoded(:key => 'value with spaces')
+      #   # => "key=value%20with%20spaces"
+      #
       def format_www_form_urlencoded(params={})
         query = []
         params.each_pair do |key, values|
-          values = values.kind_of?(Array) ? values : [values]
+          values = [values] unless values.kind_of?(Array)
           values.each { |value| query << "#{key}=#{value}" }
         end
         URI.encode( query.join('&') )
       end
-      # Returns a post body formatted as 'multipart/form-data'.  Special formatting occures
-      # if value in one of the key-value parameter pairs is an Array or Hash.
-      #
-      # Array values are treated as a multiple inputs for a single key; each array value
-      # is assigned to the key. Hash values are treated as files, with all file-related
-      # headers specified in the hash.
-      #
-      #--
-      # Example:
-      #   "--1234", {:key => 'value'} =>
-      #   --1234
-      #   Content-Disposition: form-data; name="key"
-      #
-      #   value
-      #   --1234--
-      def format_multipart_form_data(boundary, params)
+      # Formats params as 'multipart/form-data' using the specified boundary,
+      # for use as the body of a post request.  Multiple values for a parameter
+      # may be specified using an array.  All newlines include a carriage
+      # return for proper formatting.
+      #
+      #   format_multipart_form_data(:key => 'value')
+      #   # => %Q{--1234567890\r
+      #   # Content-Disposition: form-data; name="key"\r
+      #   # \r
+      #   # value\r
+      #   # --1234567890--\r
+      #   # }
+      #
+      # To specify a file, use a hash of file-related headers.
+      #
+      #   format_multipart_form_data(:key => {
+      #     'Content-Type' => 'text/plain',
+      #     'Filename' => "path/to/file.txt"}
+      #   )
+      #   # => %Q{--1234567890\r
+      #   # Content-Disposition: form-data; name="key"; filename="path/to/file.txt"\r
+      #   # Content-Type: text/plain\r
+      #   # \r
+      #   # \r
+      #   # --1234567890--\r
+      #   # }
+      #
+      def format_multipart_form_data(params, boundary="1234567890")
         body = []
         params.each_pair do |key, values|
-          values = values.kind_of?(Array) ? values : [values]
+          values = [values] unless values.kind_of?(Array)
           values.each do |value|
             body << case value

data/lib/tap/http/helpers.rb CHANGED Viewed

@@ -1,36 +1,55 @@
 autoload(:WEBrick, 'webrick')
-autoload(:Zlib, 'zlib')
-autoload(:StringIO, 'stringio')
+autoload(:Zlib, 'zlib')
+autoload(:StringIO, 'stringio')
 module Tap
   module Http
     module Helpers
       module_function
-      # Parses a WEBrick::HTTPRequest from the input socket.
+      # Parses a WEBrick::HTTPRequest from the input socket into a hash that
+      # may be resubmitted by Dispatch.  Sockets can be any kind of IO (File,
+      # StringIO, etc..) and should be positioned such that the next line is
+      # the start of an HTTP request.  Strings used as sockets are converted
+      # into StringIO objects.
       #
-      # Notes:
-      # - socket can be any kind of IO (ex File, StringIO)
-      # - the socket should be in a position such that the next line
-      #    is the start of an HTTP request, and the request should
-      #    correctly formatted (see below)
+      #   parse_http_request("GET /path HTTP/1.1\n")
+      #   # => {
+      #   # :request_method => "GET",
+      #   # :url => "/path",
+      #   # :version => "1.1",
+      #   # :headers => {},
+      #   # :params => {},
+      #   # }
       #
-      # == WEBrick parsing of HTTP format
-      # WEBrick will parse headers then the body of a request, and
-      # currently (1.8.6) considers an empty line as a break between
-      # them.  In general header parsing is forgiving with end-line
-      # characters (ie "\r\n" and "\n" are both acceptable) but parsing
-      # of multipart/form data IS NOT.
+      # If splat_values is specified, single-value headers and parameters
+      # will be hashed as single values.  Otherwise, all header and parameter
+      # values will be arrays.
+      #
+      #   str = "GET /path?one=a&one=b&two=c HTTP/1.1\n"
+      #   req = parse_http_request(str)
+      #   req[:params]     # => {'one' => ['a', 'b'], 'two' => 'c'}
+      #
+      #   req = parse_http_request(str, false)
+      #   req[:params]     # => {'one' => ['a', 'b'], 'two' => ['c']}
+      #
+      # ==== WEBrick parsing of HTTP format
+      #
+      # WEBrick will parse headers then the body of a request, and currently
+      # (1.8.6) considers an empty line as a break between the headers and
+      # body.  In general header parsing is forgiving with end-line
+      # characters (ie "\r\n" and "\n" are both acceptable) but parsing of
+      # multipart/form data IS NOT.
       #
-      # Multipart/form data REQUIRES that the end-line characters
-      # are "\r\n".  The boundary is always started with "--" and the last
-      # boundary completed with "--".  As always, the content-length
-      # must be correct.
+      # Multipart/form data REQUIRES that the end-line characters are "\r\n".
+      # A boundary is always started with "--" and the last boundary completed
+      # with "--".  As always, the content-length must be correct.
       #
       #   # Notice an empty line between the last header
       #   # (in this case 'Content-Length') and the body.
       #   msg = <<-_end_of_message_
       #   POST /path HTTP/1.1
+      #   Host: localhost:8080
       #   Content-Type: multipart/form-data; boundary=1234567890
       #   Content-Length: 158
       #
@@ -48,82 +67,103 @@ module Tap
       #   # ensure the end of line characters are correct...
       #   socket = StringIO.new msg.gsub(/\n/, "\r\n")
       #
-      #   req = Tap::Net.parse_http_request(socket)
-      #   req.header   # => {"content-type" => ["multipart/form-data; boundary=1234567890"], "content-length" => ["158"]}
-      #   req.query     # => {"one" => "value one", "two" => "value two"}
+      #   Tap::Net.parse_http_request(socket)
+      #   # => {
+      #   # :request_method => "POST",
+      #   # :url => "http://localhost:8080/path",
+      #   # :version => "HTTP/1.1",
+      #   # :headers => {
+      #   #   "Host" => "localhost:8080",
+      #   #   "Content-Type" => "multipart/form-data; boundary=1234567890",
+      #   #   "Content-Length" => "158"},
+      #   # :params => {
+      #   #   "one" => "value one",
+      #   #   "two" => "value two"}}
       #
-      def parse_http_request(socket, parse_yaml=false)
+      #--
+      # TODO: check if there are other headers to capture from
+      # a multipart/form file.  Currently only
+      # 'Filename' and 'Content-Type' are added
+      def parse_http_request(socket, splat_values=true)
         socket = StringIO.new(socket) if socket.kind_of?(String)
         req = WEBrick::HTTPRequest.new(WEBrick::Config::HTTP)
         req.parse(socket)
-        parse_webrick_request(req, parse_yaml)
+        parse_webrick_request(req, splat_values)
       end
-      #
-      # TODO -- test with cookies, HTTPS?
-      #
-      def parse_webrick_request(req, parse_yaml=false)
+      # Parses a WEBrick::HTTPRequest, with the same activity as
+      # parse_http_request.
+      def parse_webrick_request(req, splat_values=true)
         headers = {}
         req.header.each_pair do |key, values|
-          headers[headerize(key)] = collect(values) do |value|
-            objectify(value, false)
-          end
-        end
+          headers[headerize(key)] = splat_values ? splat(values) : values
+        end if req.header
         params = {}
-        req.query.each_pair do |key, values|
-          params[key] = collect(values.to_ary) do |value|
-            # TODO - special for multipart file data?
-            objectify(value, parse_yaml)
+        req.query.each_pair do |key, value|
+          # no sense for how robust this is...
+          # In tests value is (always?) a WEBrick::HTTPUtils::FormData. Each
+          # data is likewise a FormData.  If FormData is a file, it has a
+          # filename and you have to try [] to get the content-type.
+          # Senseless.  No wonder WEBrick has no documentation, who could
+          # write it?
+          values = []
+          value.each_data do |data|
+            values << if data.filename
+              {'Filename' => data.filename, 'Content-Type' => data['Content-Type']}
+            else
+              data.to_s
+            end
           end
-        end
-        url = File.join("http://", headers['Host'], req.path_info)
+          params[key] = splat_values ? splat(values) : values
+        end if req.query
-        { :url => url,
-          :http_version => req.http_version.to_s,
-          :request_method => req.request_method,
+        { :url => headers['Host'] ? File.join("http://", headers['Host'], req.path_info) : req.path_info,
+          :request_method => req.request_method,
+          :version => req.http_version.to_s,
           :headers => headers,
           :params => params}
       end
-      def parse_cgi_request(cgi, parse_yaml=false)
+      # Parses the input CGI into a hash that may be resubmitted by Dispatch.
+      # To work properly, the standard CGI environmental variables must be
+      # set in ENV.
+      #
+      def parse_cgi_request(cgi, splat_values=true)
         headers = {}
         ENV.each_pair do |key, values|
           key = case key
+          when "HTTP_VERSION" then next
           when /^HTTP_(.*)/ then $1
           when 'CONTENT_TYPE' then key
           else next
           end
-          headers[headerize(key)] = collect(values) do |value|
-            objectify(value, false)
-          end
+          headers[headerize(key)] = splat_values ? splat(values) : values
         end
         params = {}
         cgi.params.each_pair do |key, values|
-          params[key] = collect(values) do |value|
-            if value.respond_to?(:read)
-              value = if value.original_filename.empty?
-                value.read
-              else
-                {'Filename' => value.original_filename, 'Content-Type' => value.content_type}
-              end
+          values = values.collect do |value|
+            case
+            when !value.respond_to?(:read)
+              value
+            when value.original_filename.empty?
+              value.read
+            else
+              {'Filename' => value.original_filename, 'Content-Type' => value.content_type}
             end
-            objectify(value, parse_yaml)
           end
+          params[key] = splat_values ? splat(values) : values
         end
-        url = File.join("http://", headers['Host'], ENV['PATH_INFO'])
-        { :url => url,
-          :http_version => ENV['SERVER_PROTOCOL'],  # right or no?
-          :request_method => ENV['REQUEST_METHOD'],
+        { :url => File.join("http://", headers['Host'], ENV['PATH_INFO']),
+          :request_method => ENV['REQUEST_METHOD'],
+          :version => ENV['HTTP_VERSION'] =~ /^HTTP\/(.*)$/ ? $1 : ENV['HTTP_VERSION'],
           :headers => headers,
           :params => params}
       end
@@ -139,22 +179,9 @@ module Tap
         else File.join(base, action)
         end
       end
-      HELP_URL = "http://"
-      COOKIES_HELP_URL = "http://"
-      CGI_VARIABLES = %w{
-        AUTH_TYPE               HTTP_HOST          REMOTE_IDENT
-        CONTENT_LENGTH          HTTP_NEGOTIATE     REMOTE_USER
-        CONTENT_TYPE            HTTP_PRAGMA        REQUEST_METHOD
-        GATEWAY_INTERFACE       HTTP_REFERER       SCRIPT_NAME
-        HTTP_ACCEPT             HTTP_USER_AGENT    SERVER_NAME
-        HTTP_ACCEPT_CHARSET     PATH_INFO          SERVER_PORT
-        HTTP_ACCEPT_ENCODING    PATH_TRANSLATED    SERVER_PROTOCOL
-        HTTP_ACCEPT_LANGUAGE    QUERY_STRING       SERVER_SOFTWARE
-        HTTP_CACHE_CONTROL      REMOTE_ADDR
-        HTTP_FROM               REMOTE_HOST}
-      # Headerizes an underscored string.
+      # Headerizes an underscored string.  The input is be converted to
+      # a string using to_s.
       #
       #   headerize('SOME_STRING')   # => 'Some-String'
       #   headerize('some string')   # => 'Some-String'
@@ -166,64 +193,34 @@ module Tap
           $1.upcase + $2.downcase
         end.join("-")
       end
-      def collect(array)
-        array = [array] unless array.kind_of?(Array)
-        array.collect! do |value|
-          yield(value)
-        end
-        case array.length
-        when 0 then nil
-        when 1 then array.first
-        else array
-        end
-      end
-      def objectify(str, parse_yaml=false)
-        return str unless str.kind_of?(String)
-        case str
-        when /^\d+(\.\d+)?$/ then YAML.load(str)
-        when /^\s*$/ then nil
-        when /^---\s*\n/
-          parse_yaml ? YAML.load(str) : str
-        else str
-        end
-      end
-      # Performs a deep merge of two hashes where hash values for
-      # corresponding keys are merged.
-      def deep_merge(a,b)
-        result = {}
-        a.each_pair do |key, value|
-          result[key] = case value
-          when Hash then value.dup
-          # when Array then value.dup
-          else value
-          end
-        end
+      # Returns the first member of arrays length <= 1, or the array in all
+      # other cases.  Splat is useful to simplify hashes of http headers
+      # and parameters that may have multiple values, but typically only
+      # have one.
+      #
+      #   splat([])                  # => nil
+      #   splat([:one])              # => :one
+      #   splat([:one, :two])        # => [:one, :two]
+      #
+      def splat(array)
+        return array unless array.kind_of?(Array)
-        b.each_pair do |key, value|
-          case value
-          when Hash  then (result[key.to_sym] ||= {}).merge!(value)
-          #when Array then (result[key.to_sym] ||= []).concat(value)
-          else result[key.to_sym] = value
-          end
+        case array.length
+        when 0 then nil
+        when 1 then array.first
+        else array
         end
-        result
-      end
-      # Inflates (ie unzips) a gzip string, as may be returned by requests
-      # that accept 'gzip' and 'deflate' content encoding.
-      #
-      #--
-      # Helpers.inflate(res.body) if res['content-encoding'] == 'gzip'
-      #
-      def inflate(str)
-        Zlib::GzipReader.new( StringIO.new( str ) ).read
+      end
+      # Inflates (ie unzips) a gzip string, as may be returned by requests
+      # that accept 'gzip' and 'deflate' content encoding.
+      #
+      #--
+      # Helpers.inflate(res.body) if res['content-encoding'] == 'gzip'
+      #
+      def inflate(str)
+        Zlib::GzipReader.new( StringIO.new( str ) ).read
       end
     end
   end

data/lib/tap/http/request.rb CHANGED Viewed

@@ -4,7 +4,7 @@ require 'thread'
 module Tap
   module Http
-    # ::manifest submits an http request
+    # :startdoc::manifest submits an http request
     #
     # Request is a base class for submitting HTTP requests from a request
     # hash.  Multiple requests may be submitted on individual threads, up

data/lib/tap/test/http_test.rb CHANGED Viewed

@@ -11,8 +11,7 @@ module Tap
     #
     module HttpTest
-      # The test-specific web server.  All request to the server
-      # are echoed back.
+      # Server echos back all requests.
       class Server
         include Singleton
         include WEBrick
@@ -144,150 +143,6 @@ module Tap
         end
         URI.encode(query.join('&'))
       end
-      module RequestLibrary
-        def get_request
-          msg = <<-_end_of_message_
-GET /path?str=value&int=123&yaml=---+%0A-+a%0A-+b%0A-+c%0A&float=1.23 HTTP/1.1
-Host: www.example.com
-Keep-Alive: 300
-Connection: keep-alive
-          _end_of_message_
-        end
-        def _get_request
-          { :url => "http://www.example.com/path",
-            :http_version => '1.1',
-            :request_method => 'GET',
-            :headers => {
-              "Host" => "www.example.com",
-              "Keep-Alive" => 300,
-              "Connection" => 'keep-alive'},
-            :params => {
-              'str' => 'value',
-              'int' => 123,
-              'float' => 1.23,
-              'yaml' => ['a', 'b', 'c']}
-            }
-        end
-        def get_request_with_multiple_values
-          msg = <<-_end_of_message_
-GET /path?one=value&one=123&one=---+%0A-+a%0A-+b%0A-+c%0A&two=1.23 HTTP/1.1
-Host: www.example.com
-Keep-Alive: 300
-Connection: keep-alive
-          _end_of_message_
-        end
-        def _get_request_with_multiple_values
-          { :url => "http://www.example.com/path",
-            :http_version => '1.1',
-            :request_method => 'GET',
-            :headers => {
-              "Host" => "www.example.com",
-              "Keep-Alive" => 300,
-              "Connection" => 'keep-alive'},
-            :params => {
-              'one' => ['value', 123, ['a', 'b', 'c']],
-              'two' => 1.23}
-            }
-        end
-        def multipart_request
-          msg = <<-_end_of_message_
-POST /path HTTP/1.1
-Host: www.example.com
-Content-Type: multipart/form-data; boundary=1234567890
-Content-Length: 305
---1234567890
-Content-Disposition: form-data; name="str"
-string value
---1234567890
-Content-Disposition: form-data; name="int"
-123
---1234567890
-Content-Disposition: form-data; name="float"
-1.23
---1234567890
-Content-Disposition: form-data; name="yaml"
----
-- a
-- b
-- c
---1234567890--
-_end_of_message_
-          msg.gsub(/\n/, "\r\n")
-        end
-        def _multipart_request
-          { :url => "http://www.example.com/path",
-            :http_version => '1.1',
-            :request_method => 'POST',
-            :headers => {
-              "Host" => 'www.example.com',
-              "Content-Type" => "multipart/form-data; boundary=1234567890",
-              "Content-Length" => 305},
-            :params => {
-              'str' => 'string value',
-              'int' => 123,
-              'float' => 1.23,
-              'yaml' => ['a', 'b', 'c']}
-            }
-        end
-        def multipart_request_with_multiple_values
-          msg = <<-_end_of_message_
-POST /path HTTP/1.1
-Host: www.example.com
-Content-Type: multipart/form-data; boundary=1234567890
-Content-Length: 302
---1234567890
-Content-Disposition: form-data; name="one"
-string value
---1234567890
-Content-Disposition: form-data; name="one"
-123
---1234567890
-Content-Disposition: form-data; name="two"
-1.23
---1234567890
-Content-Disposition: form-data; name="one"
----
-- a
-- b
-- c
---1234567890--
-_end_of_message_
-          msg.gsub(/\n/, "\r\n")
-        end
-        def _multipart_request_with_multiple_values
-          { :url => "http://www.example.com/path",
-            :http_version => '1.1',
-            :request_method => 'POST',
-            :headers => {
-              "Host" => 'www.example.com',
-              "Content-Type" => "multipart/form-data; boundary=1234567890",
-              "Content-Length" => 302},
-            :params => {
-              'one' => ['string value', 123, ['a', 'b', 'c']],
-              'two' => 1.23}
-            }
-        end
-      end
     end
   end
 end

metadata CHANGED Viewed

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: tap-http
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.1.0
 platform: ruby
 authors:
 - Simon Chiang
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
-date: 2008-11-18 00:00:00 -07:00
+date: 2008-11-25 00:00:00 -07:00
 default_executable:
 dependencies:
 - !ruby/object:Gem::Dependency
@@ -43,7 +43,7 @@ files:
 - README
 - MIT-LICENSE
 has_rdoc: true
-homepage: http://rubyforge.org/projects/tap
+homepage: http://tap.rubyforge.org/projects/tap-http
 post_install_message:
 rdoc_options: []
@@ -64,7 +64,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubyforge_project: tap
-rubygems_version: 1.3.0
+rubygems_version: 1.3.1
 signing_key:
 specification_version: 2
 summary: A task library for submitting http requests using Tap.