RubyGems - tap-http - Versions diffs - 0.0.1 → 0.1.0 - Mend

tap-http 0.0.1 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (8) hide show

data/README CHANGED Viewed

@@ -1,15 +1,111 @@
-= {TapHttp}[http://tap.rubyforge.org/tap-http]
+= {TapHttp}[http://tap.rubyforge.org/projects/tap-http]
 A task library for submitting http requests using {Tap}[http://tap.rubyforge.org].
 == Description
+TapHttp provides modules to construct and submit HTTP requests from a hash
+that specifies the target url, headers, parameters, etc.  TapHttp is
+designed to work with a {Ubiquity}[http://labs.mozilla.com/2008/08/introducing-ubiquity/]
+command called {redirect-http}[http://gist.github.com/25932]; together
+they allow the capture and resubmission of web forms.
 * Lighthouse[http://bahuvrihi.lighthouseapp.com/projects/9908-tap-task-application/tickets]
 * Github[http://github.com/bahuvrihi/tap-http/tree/master]
 * {Google Group}[http://groups.google.com/group/ruby-on-tap]
 === Usage
+TapHttp submits http requests using the Tap::Http::Dispatch module.  Headers,
+parameters, and other configurations may be specified, but the only required
+field is :url.
+  include Tap::Http
+  res = Dispatch.submit_request(
+    :params => {'q' => 'tap-http'},
+    :url => 'http://www.google.com/search')
+  res.body[0,80]  # => "<!doctype html><head><title>tap-http - Google Search</title><style>body{backgrou"
+=== Getting Http Configurations
+More complicated http requests may be captured and resubmitted using a
+combination of tools that redirects web forms to a tap server and reformats
+the request as YAML. To do so:
+* Install {Firefox}[http://www.mozilla.com/en-US/firefox/]
+* Install {Ubiquity}[http://labs.mozilla.com/2008/08/introducing-ubiquity/]
+* Install {redirect-http}[http://gist.github.com/25932]
+Start a tap server from the command line (of course tap-http must be installed):
+  % tap server
+Now in the browser, go to a web form like {google}[http://www.google.com/] and
+invoke the redirection.
+* Bring up Ubiquity in Firefox by pressing 'option+space'
+* Enter the command: 'redirect-http http://localhost:8080/http_to_yaml'
+You should see a notice that the form is being redirected.  Fill out the form
+and submit as normal; the redirect command will send the form to the tap server
+instead of performing the original action.  The tap server returns a yaml file
+with the http configuration.
+  # Copy and paste into a configuration file. Multiple configs
+  # can be added to a single file to perform batch submission.
+  - headers:
+      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
+      Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
+      Accept-Encoding: gzip,deflate
+      Accept-Language: en-us,en;q=0.5
+      Connection: keep-alive
+      Host: www.google.com
+      Keep-Alive: "300"
+      Referer: http://www.google.com/
+      User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4
+    params:
+      aq: f
+      btnG: Google Search
+      hl: en
+      oq: ""
+      q: tap-http
+    request_method: GET
+    url: http://www.google.com/search
+    version: "1.1"
+Save the file as 'request.yml' and resubmit the form using the Tap::Http::Request
+task.
+  % rap load requests.yml --:i request --+ dump --no-audit
+    I[10:51:40]               load request.yml
+    I[10:51:40]                GET http://www.google.com/search
+    I[10:51:41]                    OK
+  # date: 2008-11-25 10:51:41
+  ---
+  tap/http/request (2772040):
+  - - !ruby/object:Net::HTTPOK
+      body: !binary |
+        H4sIAAAAAAAC/6xabXPbNhL+3l/B0BeN1FAUJfktoihf07pppmkm06TXu0lz
+        HZAEScQgQZOQZVfhf79dgBRJS4ndmRvPSAC42F3sPvsCyssnoQjkXU6NRKZ8
+        tUwoCVdLySSnK0nycSJlboyNl0LEnBrvKCmCZDnRz5elvIMvX4R3W58EV3Eh
+        1lm4OIqiyA0EF8XiyHEc...
+Note the result is encoded as gzip, as per the parameters. As with all tasks,
+the request results could be passed into a workflow. Alternatively, the
+configuration could be used to submit the request using Dispatch.
+=== Bugs/Known Issues
+The Tap::Http::Helpers#parse_cgi_request (used in parsing redirected requests
+into a YAML file) is currently untested because I can't figure a way to setup
+the ENV variables in a standard way.  Of course I could set them up myself, but
+I can't be sure I'm setting up a realistic test environment.
+The capture procedure seems to work in practice, but please report bugs and let
+me know if you know a way to setup a CGI environment for testing!
 == Installation
 TapHttp is available as a gem on RubyForge[http://rubyforge.org/projects/tap].  Use:

data/cgi/echo.rb CHANGED Viewed

@@ -1,7 +1,7 @@
 #!/usr/local/bin/ruby
 ####################################
-# Echos back the HTTP header and parameters as YAML.
+# Echos back the HTTP header and parameters as YAML.
 #
 # Copyright (c) 2008, Regents of the University of Colorado
 # Developer: Simon Chiang, Biomolecular Structure Program
@@ -14,8 +14,8 @@ require 'tap/http/helpers'
 cgi = CGI.new
 cgi.out("text/plain") do
-  begin
-    request = Tap::Http::Helpers.parse_cgi_request(cgi)
+  begin
+    request = Tap::Http::Helpers.parse_cgi_request(cgi)
     request[:headers].to_yaml + request[:params].to_yaml
   rescue
     "Error: #{$!.message}\n"  +

data/cgi/http_to_yaml.rb CHANGED Viewed

@@ -88,14 +88,14 @@ begin
   # format output
   #
-  help = cgi.a(Tap::Http::Helpers::HELP_URL) { "help" }
-  how_to_get_cookies = cgi.a(Tap::Http::Helpers::COOKIES_HELP_URL) { "how to get cookies" }
+  # help = cgi.a(Tap::Http::Helpers::HELP_URL) { "help" }
+  # how_to_get_cookies = cgi.a(Tap::Http::Helpers::COOKIES_HELP_URL) { "how to get cookies" }
+  #
+  # If you need cookies, see #{how_to_get_cookies} or the #{help}.
   cgi.out('text/plain') do
 %Q{# Copy and paste into a configuration file. Multiple configs
 # can be added to a single file to perform batch submission.
-#
-# If you need cookies, see #{how_to_get_cookies} or the #{help}.
 - #{config.to_yaml[5..-1].gsub(/\n/, "\n  ")}
 }
   end

data/lib/tap/http/dispatch.rb CHANGED Viewed

@@ -1,53 +1,60 @@
 require 'tap/http/helpers'
 require 'net/http'
-#module Net
-#  class HTTP
-# attr_reader :socket
-# end
-# class BufferedIO
-#include Prosperity::Acts::Monitorable
-#private
-#def rbuf_fill
-#  tick_monitor
-#  timeout(@read_timeout) {
-#    @rbuf << @io.sysread(1024)
-#  }
-#end
-#end
-#end
 module Tap
   module Http
     # Dispatch provides methods for constructing and submitting get and post
-    # HTTP requests.
+    # HTTP requests from a configuration hash.
+    #
+    #   res = Tap::Http::Dispatch.submit_request(
+    #     :url => "http://tap.rubyforge.org",
+    #     :version => '1.1',
+    #     :request_method => 'GET',
+    #     :headers => {},
+    #     :params => {}
+    #   )
+    #   res.inspect           # => "#<Net::HTTPOK 200 OK readbody=true>"
+    #   res.body =~ /Tap/     # => true
+    #
+    # Headers and parameters take the form:
+    #
+    #   { 'single' => 'value',
+    #     'multiple' => ['value one', 'value two']}
+    #
     module Dispatch
-      REQUEST_KEYS = [:url, :request_method, :headers, :params, :redirection_limit]
       module_function
-      # Constructs and submits a request to the url using the headers and parameters.
-      # Returns the response from the submission.
+      DEFAULT_CONFIG = {
+        :request_method => 'GET',
+        :version => '1.1',
+        :params => {},
+        :headers => {},
+        :redirection_limit => nil
+      }
+      # Constructs and submits a request to the url using the request configuration.
+      # A url must be specified in the configuration, but other configurations are
+      # optional; if unspecified, the values in DEFAULT_CONFIG will be used.  A
+      # block may be given to receive the Net::HTTP and request just prior to
+      # submission.
       #
-      #   res = submit_request("http://www.google.com/search", {:request_method => 'get'}, {:q => 'tap rubyforge'})
-      #   # => <Net::HTTPOK 200 OK readbody=true>
+      # Returns the response from the submission.
       #
-      # Notes:
-      # - A request method must be specified in the headers; currently only get and
-      #   post are supported.  See construct_post for supported post content types.
-      # - A url or a url (as would result from URI.parse(url)) can be provided to
-      #   submit request.  The Net::HTTP object performing the submission is passed
-      #   to the block, if given, before the request is made.
-      def submit_request(config) # :yields: http
+      def submit_request(config)
+        symbolized = DEFAULT_CONFIG.dup
+        config.each_pair do |key, value|
+          symbolized[key.to_sym] = value
+        end
+        config = symbolized
+        request_method = (config[:request_method]).to_s
         url_or_uri = config[:url]
-        params = config[:params] || {}
-        headers = headerize_keys( config[:headers] || {})
-        request_method = (config[:request_method] || 'GET').to_s
+        version = config[:version]
+        params = config[:params]
+        headers = headerize_keys(config[:headers])
+        raise ArgumentError, "no url specified" unless url_or_uri
         uri = url_or_uri.kind_of?(URI) ? url_or_uri : URI.parse(url_or_uri)
         uri.path = "/" if uri.path.empty?
@@ -56,22 +63,20 @@ module Tap
         when /^get$/i then construct_get(uri, headers, params)
         when /^post$/i then construct_post(uri, headers, params)
         else
-          raise ArgumentError.new("Missing or unsupported request_method: #{request_method}")
+          raise ArgumentError, "unsupported request method: #{request_method}"
         end
         # set the http version
-        # if version = config[:http_version]
-        #   version_method = "version_#{version.to_s.gsub(".", "_")}".to_sym
-        #   if Object::Net::HTTP.respond_to?(version_method)
-        #     Object::Net::HTTP.send(version_method)
-        #   else
-        #     raise ArgumentError.new("unsupported http_version: #{version}")
-        #   end
-        # end
+        version_method = "version_#{version.to_s.gsub(".", "_")}".to_sym
+        if ::Net::HTTP.respond_to?(version_method)
+          ::Net::HTTP.send(version_method)
+        else
+          raise ArgumentError, "unsupported HTTP version: #{version}"
+        end
         # submit the request
-        res = Object::Net::HTTP.new(uri.host, uri.port).start do |http|
-          yield(http) if block_given?
+        res = ::Net::HTTP.new(uri.host, uri.port).start do |http|
+          yield(http, request) if block_given?
           http.request(request)
         end
@@ -80,120 +85,175 @@ module Tap
         redirection_limit ? fetch_redirection(res, redirection_limit) : res
       end
-      # Constructs a post query.  The 'Content-Type' header determines the format of the body
-      # content.  If the content type is 'multipart/form-data', then the parameters will be
-      # formatted using the boundary in the content-type header, if provided, or a randomly
-      # generated boundary.
+      # Constructs a Net::HTTP::Post query, setting headers and parameters.
       #
-      # Headers for the request are set in this method.  If uri contains query parameters,
-      # they will be included in the request URI.
+      # ==== Supported Content Types:
       #
-      # Supported content-types:
-      # -  application/x-www-form-urlencoded (the default)
-      # -  multipart/form-data
+      # - application/x-www-form-urlencoded (the default)
+      # - multipart/form-data
+      #
+      # The multipart/form-data content type may specify a boundary.  If no
+      # boundary is specified, a randomly generated boundary will be used
+      # to delimit the parameters.
+      #
+      #   post = construct_post(
+      #     URI.parse('http://some.url/'),
+      #     {:content_type => 'multipart/form-data; boundary=1234'},
+      #     {:key => 'value'})
+      #
+      #   post.body
+      #   # => %Q{--1234\r
+      #   # Content-Disposition: form-data; name="key"\r
+      #   # \r
+      #   # value\r
+      #   # --1234--\r
+      #   # }
+      #
+      # (Note the carriage returns are required in multipart content)
+      #
+      # The content-length header is determined automatically from the
+      # formatted request body; manually specified content-length headers
+      # will be overridden.
       #
       def construct_post(uri, headers, params)
-        req = Object::Net::HTTP::Post.new( URI.encode("#{uri.path}#{construct_query(uri)}") )
+        req = ::Net::HTTP::Post.new( URI.encode("#{uri.path}#{format_query(uri)}") )
         headers = headerize_keys(headers)
         content_type = headers['Content-Type']
         case content_type
-        when /multipart\/form-data/i
+        when nil, /^application\/x-www-form-urlencoded$/i
+          req.body = format_www_form_urlencoded(params)
+          headers['Content-Type'] ||= "application/x-www-form-urlencoded"
+          headers['Content-Length'] = req.body.length
+        when /^multipart\/form-data(;\s*boundary=(.*))?$/i
           # extract the boundary if it exists
-          content_type =~ /boundary=(.*)/i
-          boundary = $1 || rand.to_s[2..20]
+          boundary = $2 || rand.to_s[2..20]
-          req.body = format_multipart_form_data(boundary, params)
+          req.body = format_multipart_form_data(params, boundary)
           headers['Content-Type'] = "multipart/form-data; boundary=#{boundary}"
           headers['Content-Length'] = req.body.length
         else
-          req.body = format_www_form_urlencoded(params)
-          headers['Content-Type'] = "application/x-www-form-urlencoded"
-          headers['Content-Length'] = req.body.length
+          raise ArgumentError, "unsupported Content-Type for POST: #{content_type}"
         end
         headers.each_pair { |key, value| req[key] = value }
         req
       end
-      # Constructs a get query.  All parameters in uri and params are added to the
-      # request URI.  Headers for the request are set in this method.
+      # Constructs a Net::HTTP::Get query.  All parameters in uri and params are
+      # encoded and added to the request URI.
+      #
+      #   get = construct_get(URI.parse('http://some.url/path'), {}, {:key => 'value'})
+      #   get.path            # => "/path?key=value"
+      #
       def construct_get(uri, headers, params)
-        req = Object::Net::HTTP::Get.new( URI.encode("#{uri.path}#{construct_query(uri, params)}") )
+        req = ::Net::HTTP::Get.new( URI.encode("#{uri.path}#{format_query(uri, params)}") )
         headerize_keys(headers).each_pair { |key, value| req[key] = value }
         req
       end
-      # Checks the type of the response; if it is a redirection, get the redirection,
-      # otherwise return the response.
+      # Checks the type of the response; if it is a redirection, fetches the
+      # redirection.  Otherwise return the response.
       #
       # Notes:
-      # - The redirections will only recurse for the input redirection limit (default 10)
-      # - Responses that are not Net::HTTPRedirection or Net::HTTPSuccess raise an error.
+      # - Fetch will recurse up to the input redirection limit (default 10)
+      # - Responses that are not Net::HTTPRedirection or Net::HTTPSuccess
+      #   raise an error.
       def fetch_redirection(res, limit=10)
-        raise ArgumentError, 'Could not follow all the redirections.' if limit == 0
+        raise 'exceeded the redirection limit' if limit < 1
         case res
-        when Object::Net::HTTPRedirection
-          redirect = Object::Net::HTTP.get_response( URI.parse(res['location']) )
+        when ::Net::HTTPRedirection
+          redirect = ::Net::HTTP.get_response( URI.parse(res['location']) )
           fetch_redirection(redirect, limit - 1)
-        when Object::Net::HTTPSuccess then res
-        else raise StandardError, res.error!
+        when ::Net::HTTPSuccess
+          res
+        else
+          raise StandardError, res.error!
         end
       end
-      # Normalizes the header keys to a titleized, dasherized string.
-      #   'some_header' => 'Some-Header'
-      #   :some_header => 'Some-Header'
-      #   'some header' => 'Some-Header'
-      def headerize_keys(headers)
+      # Converts the keys of a hash to headers.  See Helpers#headerize.
+      #
+      #   headerize_keys('some_header' => 'value')   # => {'Some-Header' => 'value'}
+      #
+      def headerize_keys(hash)
         result = {}
-        headers.each_pair do |key, value|
+        hash.each_pair do |key, value|
           result[Helpers.headerize(key)] = value
         end
         result
       end
-      # Returns a URI query constructed from the query in uri and the input parameters.
+      # Constructs a URI query string from the uri and the input parameters.
+      # Multiple values for a parameter may be specified using an array.
       # The query is not encoded, so you may need to URI.encode it later.
-      def construct_query(uri, params={})
+      #
+      #   format_query(URI.parse('http://some.url/path'), {:key => 'value'})
+      #   # => "?key=value"
+      #
+      #   format_query(URI.parse('http://some.url/path?one=1'), {:two => '2'})
+      #   # => "?one=1&two=2"
+      #
+      def format_query(uri, params={})
         query = []
+        query << uri.query if uri.query
         params.each_pair do |key, values|
-          values = values.kind_of?(Array) ? values : [values]
+          values = [values] unless values.kind_of?(Array)
           values.each { |value| query << "#{key}=#{value}" }
         end
-        query << uri.query if uri.query
         "#{query.empty? ? '' : '?'}#{query.join('&')}"
       end
+      # Formats params as 'application/x-www-form-urlencoded' for use as the
+      # body of a post request.  Multiple values for a parameter may be
+      # specified using an array.  The result is obviously URI encoded.
+      #
+      #   format_www_form_urlencoded(:key => 'value with spaces')
+      #   # => "key=value%20with%20spaces"
+      #
       def format_www_form_urlencoded(params={})
         query = []
         params.each_pair do |key, values|
-          values = values.kind_of?(Array) ? values : [values]
+          values = [values] unless values.kind_of?(Array)
           values.each { |value| query << "#{key}=#{value}" }
         end
         URI.encode( query.join('&') )
       end
-      # Returns a post body formatted as 'multipart/form-data'.  Special formatting occures
-      # if value in one of the key-value parameter pairs is an Array or Hash.
-      #
-      # Array values are treated as a multiple inputs for a single key; each array value
-      # is assigned to the key. Hash values are treated as files, with all file-related
-      # headers specified in the hash.
-      #
-      #--
-      # Example:
-      #   "--1234", {:key => 'value'} =>
-      #   --1234
-      #   Content-Disposition: form-data; name="key"
-      #
-      #   value
-      #   --1234--
-      def format_multipart_form_data(boundary, params)
+      # Formats params as 'multipart/form-data' using the specified boundary,
+      # for use as the body of a post request.  Multiple values for a parameter
+      # may be specified using an array.  All newlines include a carriage
+      # return for proper formatting.
+      #
+      #   format_multipart_form_data(:key => 'value')
+      #   # => %Q{--1234567890\r
+      #   # Content-Disposition: form-data; name="key"\r
+      #   # \r
+      #   # value\r
+      #   # --1234567890--\r
+      #   # }
+      #
+      # To specify a file, use a hash of file-related headers.
+      #
+      #   format_multipart_form_data(:key => {
+      #     'Content-Type' => 'text/plain',
+      #     'Filename' => "path/to/file.txt"}
+      #   )
+      #   # => %Q{--1234567890\r
+      #   # Content-Disposition: form-data; name="key"; filename="path/to/file.txt"\r
+      #   # Content-Type: text/plain\r
+      #   # \r
+      #   # \r
+      #   # --1234567890--\r
+      #   # }
+      #
+      def format_multipart_form_data(params, boundary="1234567890")
         body = []
         params.each_pair do |key, values|
-          values = values.kind_of?(Array) ? values : [values]
+          values = [values] unless values.kind_of?(Array)
           values.each do |value|
             body << case value

data/lib/tap/http/helpers.rb CHANGED Viewed

@@ -1,36 +1,55 @@
 autoload(:WEBrick, 'webrick')
-autoload(:Zlib, 'zlib')
-autoload(:StringIO, 'stringio')
+autoload(:Zlib, 'zlib')
+autoload(:StringIO, 'stringio')
 module Tap
   module Http
     module Helpers
       module_function
-      # Parses a WEBrick::HTTPRequest from the input socket.
+      # Parses a WEBrick::HTTPRequest from the input socket into a hash that
+      # may be resubmitted by Dispatch.  Sockets can be any kind of IO (File,
+      # StringIO, etc..) and should be positioned such that the next line is
+      # the start of an HTTP request.  Strings used as sockets are converted
+      # into StringIO objects.
       #
-      # Notes:
-      # - socket can be any kind of IO (ex File, StringIO)
-      # - the socket should be in a position such that the next line
-      #    is the start of an HTTP request, and the request should
-      #    correctly formatted (see below)
+      #   parse_http_request("GET /path HTTP/1.1\n")
+      #   # => {
+      #   # :request_method => "GET",
+      #   # :url => "/path",
+      #   # :version => "1.1",
+      #   # :headers => {},
+      #   # :params => {},
+      #   # }
       #
-      # == WEBrick parsing of HTTP format
-      # WEBrick will parse headers then the body of a request, and
-      # currently (1.8.6) considers an empty line as a break between
-      # them.  In general header parsing is forgiving with end-line
-      # characters (ie "\r\n" and "\n" are both acceptable) but parsing
-      # of multipart/form data IS NOT.
+      # If splat_values is specified, single-value headers and parameters
+      # will be hashed as single values.  Otherwise, all header and parameter
+      # values will be arrays.
+      #
+      #   str = "GET /path?one=a&one=b&two=c HTTP/1.1\n"
+      #   req = parse_http_request(str)
+      #   req[:params]     # => {'one' => ['a', 'b'], 'two' => 'c'}
+      #
+      #   req = parse_http_request(str, false)
+      #   req[:params]     # => {'one' => ['a', 'b'], 'two' => ['c']}
+      #
+      # ==== WEBrick parsing of HTTP format
+      #
+      # WEBrick will parse headers then the body of a request, and currently
+      # (1.8.6) considers an empty line as a break between the headers and
+      # body.  In general header parsing is forgiving with end-line
+      # characters (ie "\r\n" and "\n" are both acceptable) but parsing of
+      # multipart/form data IS NOT.
       #
-      # Multipart/form data REQUIRES that the end-line characters
-      # are "\r\n".  The boundary is always started with "--" and the last
-      # boundary completed with "--".  As always, the content-length
-      # must be correct.
+      # Multipart/form data REQUIRES that the end-line characters are "\r\n".
+      # A boundary is always started with "--" and the last boundary completed
+      # with "--".  As always, the content-length must be correct.
       #
       #   # Notice an empty line between the last header
       #   # (in this case 'Content-Length') and the body.
       #   msg = <<-_end_of_message_
       #   POST /path HTTP/1.1
+      #   Host: localhost:8080
       #   Content-Type: multipart/form-data; boundary=1234567890
       #   Content-Length: 158
       #
@@ -48,82 +67,103 @@ module Tap
       #   # ensure the end of line characters are correct...
       #   socket = StringIO.new msg.gsub(/\n/, "\r\n")
       #
-      #   req = Tap::Net.parse_http_request(socket)
-      #   req.header   # => {"content-type" => ["multipart/form-data; boundary=1234567890"], "content-length" => ["158"]}
-      #   req.query     # => {"one" => "value one", "two" => "value two"}
+      #   Tap::Net.parse_http_request(socket)
+      #   # => {
+      #   # :request_method => "POST",
+      #   # :url => "http://localhost:8080/path",
+      #   # :version => "HTTP/1.1",
+      #   # :headers => {
+      #   #   "Host" => "localhost:8080",
+      #   #   "Content-Type" => "multipart/form-data; boundary=1234567890",
+      #   #   "Content-Length" => "158"},
+      #   # :params => {
+      #   #   "one" => "value one",
+      #   #   "two" => "value two"}}
       #
-      def parse_http_request(socket, parse_yaml=false)
+      #--
+      # TODO: check if there are other headers to capture from
+      # a multipart/form file.  Currently only
+      # 'Filename' and 'Content-Type' are added
+      def parse_http_request(socket, splat_values=true)
         socket = StringIO.new(socket) if socket.kind_of?(String)
         req = WEBrick::HTTPRequest.new(WEBrick::Config::HTTP)
         req.parse(socket)
-        parse_webrick_request(req, parse_yaml)
+        parse_webrick_request(req, splat_values)
       end
-      #
-      # TODO -- test with cookies, HTTPS?
-      #
-      def parse_webrick_request(req, parse_yaml=false)
+      # Parses a WEBrick::HTTPRequest, with the same activity as
+      # parse_http_request.
+      def parse_webrick_request(req, splat_values=true)
         headers = {}
         req.header.each_pair do |key, values|
-          headers[headerize(key)] = collect(values) do |value|
-            objectify(value, false)
-          end
-        end
+          headers[headerize(key)] = splat_values ? splat(values) : values
+        end if req.header
         params = {}
-        req.query.each_pair do |key, values|
-          params[key] = collect(values.to_ary) do |value|
-            # TODO - special for multipart file data?
-            objectify(value, parse_yaml)
+        req.query.each_pair do |key, value|
+          # no sense for how robust this is...
+          # In tests value is (always?) a WEBrick::HTTPUtils::FormData. Each
+          # data is likewise a FormData.  If FormData is a file, it has a
+          # filename and you have to try [] to get the content-type.
+          # Senseless.  No wonder WEBrick has no documentation, who could
+          # write it?
+          values = []
+          value.each_data do |data|
+            values << if data.filename
+              {'Filename' => data.filename, 'Content-Type' => data['Content-Type']}
+            else
+              data.to_s
+            end
           end
-        end
-        url = File.join("http://", headers['Host'], req.path_info)
+          params[key] = splat_values ? splat(values) : values
+        end if req.query
-        { :url => url,
-          :http_version => req.http_version.to_s,
-          :request_method => req.request_method,
+        { :url => headers['Host'] ? File.join("http://", headers['Host'], req.path_info) : req.path_info,
+          :request_method => req.request_method,
+          :version => req.http_version.to_s,
           :headers => headers,
           :params => params}
       end
-      def parse_cgi_request(cgi, parse_yaml=false)
+      # Parses the input CGI into a hash that may be resubmitted by Dispatch.
+      # To work properly, the standard CGI environmental variables must be
+      # set in ENV.
+      #
+      def parse_cgi_request(cgi, splat_values=true)
         headers = {}
         ENV.each_pair do |key, values|
           key = case key
+          when "HTTP_VERSION" then next
           when /^HTTP_(.*)/ then $1
           when 'CONTENT_TYPE' then key
           else next
           end
-          headers[headerize(key)] = collect(values) do |value|
-            objectify(value, false)
-          end
+          headers[headerize(key)] = splat_values ? splat(values) : values
         end
         params = {}
         cgi.params.each_pair do |key, values|
-          params[key] = collect(values) do |value|
-            if value.respond_to?(:read)
-              value = if value.original_filename.empty?
-                value.read
-              else
-                {'Filename' => value.original_filename, 'Content-Type' => value.content_type}
-              end
+          values = values.collect do |value|
+            case
+            when !value.respond_to?(:read)
+              value
+            when value.original_filename.empty?
+              value.read
+            else
+              {'Filename' => value.original_filename, 'Content-Type' => value.content_type}
             end
-            objectify(value, parse_yaml)
           end
+          params[key] = splat_values ? splat(values) : values
         end
-        url = File.join("http://", headers['Host'], ENV['PATH_INFO'])
-        { :url => url,
-          :http_version => ENV['SERVER_PROTOCOL'],  # right or no?
-          :request_method => ENV['REQUEST_METHOD'],
+        { :url => File.join("http://", headers['Host'], ENV['PATH_INFO']),
+          :request_method => ENV['REQUEST_METHOD'],
+          :version => ENV['HTTP_VERSION'] =~ /^HTTP\/(.*)$/ ? $1 : ENV['HTTP_VERSION'],
           :headers => headers,
           :params => params}
       end
@@ -139,22 +179,9 @@ module Tap
         else File.join(base, action)
         end
       end
-      HELP_URL = "http://"
-      COOKIES_HELP_URL = "http://"
-      CGI_VARIABLES = %w{
-        AUTH_TYPE               HTTP_HOST          REMOTE_IDENT
-        CONTENT_LENGTH          HTTP_NEGOTIATE     REMOTE_USER
-        CONTENT_TYPE            HTTP_PRAGMA        REQUEST_METHOD
-        GATEWAY_INTERFACE       HTTP_REFERER       SCRIPT_NAME
-        HTTP_ACCEPT             HTTP_USER_AGENT    SERVER_NAME
-        HTTP_ACCEPT_CHARSET     PATH_INFO          SERVER_PORT
-        HTTP_ACCEPT_ENCODING    PATH_TRANSLATED    SERVER_PROTOCOL
-        HTTP_ACCEPT_LANGUAGE    QUERY_STRING       SERVER_SOFTWARE
-        HTTP_CACHE_CONTROL      REMOTE_ADDR
-        HTTP_FROM               REMOTE_HOST}
-      # Headerizes an underscored string.
+      # Headerizes an underscored string.  The input is be converted to
+      # a string using to_s.
       #
       #   headerize('SOME_STRING')   # => 'Some-String'
       #   headerize('some string')   # => 'Some-String'
@@ -166,64 +193,34 @@ module Tap
           $1.upcase + $2.downcase
         end.join("-")
       end
-      def collect(array)
-        array = [array] unless array.kind_of?(Array)
-        array.collect! do |value|
-          yield(value)
-        end
-        case array.length
-        when 0 then nil
-        when 1 then array.first
-        else array
-        end
-      end
-      def objectify(str, parse_yaml=false)
-        return str unless str.kind_of?(String)
-        case str
-        when /^\d+(\.\d+)?$/ then YAML.load(str)
-        when /^\s*$/ then nil
-        when /^---\s*\n/
-          parse_yaml ? YAML.load(str) : str
-        else str
-        end
-      end
-      # Performs a deep merge of two hashes where hash values for
-      # corresponding keys are merged.
-      def deep_merge(a,b)
-        result = {}
-        a.each_pair do |key, value|
-          result[key] = case value
-          when Hash then value.dup
-          # when Array then value.dup
-          else value
-          end
-        end
+      # Returns the first member of arrays length <= 1, or the array in all
+      # other cases.  Splat is useful to simplify hashes of http headers
+      # and parameters that may have multiple values, but typically only
+      # have one.
+      #
+      #   splat([])                  # => nil
+      #   splat([:one])              # => :one
+      #   splat([:one, :two])        # => [:one, :two]
+      #
+      def splat(array)
+        return array unless array.kind_of?(Array)
-        b.each_pair do |key, value|
-          case value
-          when Hash  then (result[key.to_sym] ||= {}).merge!(value)
-          #when Array then (result[key.to_sym] ||= []).concat(value)
-          else result[key.to_sym] = value
-          end
+        case array.length
+        when 0 then nil
+        when 1 then array.first
+        else array
         end
-        result
-      end
-      # Inflates (ie unzips) a gzip string, as may be returned by requests
-      # that accept 'gzip' and 'deflate' content encoding.
-      #
-      #--
-      # Helpers.inflate(res.body) if res['content-encoding'] == 'gzip'
-      #
-      def inflate(str)
-        Zlib::GzipReader.new( StringIO.new( str ) ).read
+      end
+      # Inflates (ie unzips) a gzip string, as may be returned by requests
+      # that accept 'gzip' and 'deflate' content encoding.
+      #
+      #--
+      # Helpers.inflate(res.body) if res['content-encoding'] == 'gzip'
+      #
+      def inflate(str)
+        Zlib::GzipReader.new( StringIO.new( str ) ).read
       end
     end
   end

data/lib/tap/http/request.rb CHANGED Viewed

@@ -4,7 +4,7 @@ require 'thread'
 module Tap
   module Http
-    # ::manifest submits an http request
+    # :startdoc::manifest submits an http request
     #
     # Request is a base class for submitting HTTP requests from a request
     # hash.  Multiple requests may be submitted on individual threads, up

data/lib/tap/test/http_test.rb CHANGED Viewed

@@ -11,8 +11,7 @@ module Tap
     #
     module HttpTest
-      # The test-specific web server.  All request to the server
-      # are echoed back.
+      # Server echos back all requests.
       class Server
         include Singleton
         include WEBrick
@@ -144,150 +143,6 @@ module Tap
         end
         URI.encode(query.join('&'))
       end
-      module RequestLibrary
-        def get_request
-          msg = <<-_end_of_message_
-GET /path?str=value&int=123&yaml=---+%0A-+a%0A-+b%0A-+c%0A&float=1.23 HTTP/1.1
-Host: www.example.com
-Keep-Alive: 300
-Connection: keep-alive
-          _end_of_message_
-        end
-        def _get_request
-          { :url => "http://www.example.com/path",
-            :http_version => '1.1',
-            :request_method => 'GET',
-            :headers => {
-              "Host" => "www.example.com",
-              "Keep-Alive" => 300,
-              "Connection" => 'keep-alive'},
-            :params => {
-              'str' => 'value',
-              'int' => 123,
-              'float' => 1.23,
-              'yaml' => ['a', 'b', 'c']}
-            }
-        end
-        def get_request_with_multiple_values
-          msg = <<-_end_of_message_
-GET /path?one=value&one=123&one=---+%0A-+a%0A-+b%0A-+c%0A&two=1.23 HTTP/1.1
-Host: www.example.com
-Keep-Alive: 300
-Connection: keep-alive
-          _end_of_message_
-        end
-        def _get_request_with_multiple_values
-          { :url => "http://www.example.com/path",
-            :http_version => '1.1',
-            :request_method => 'GET',
-            :headers => {
-              "Host" => "www.example.com",
-              "Keep-Alive" => 300,
-              "Connection" => 'keep-alive'},
-            :params => {
-              'one' => ['value', 123, ['a', 'b', 'c']],
-              'two' => 1.23}
-            }
-        end
-        def multipart_request
-          msg = <<-_end_of_message_
-POST /path HTTP/1.1
-Host: www.example.com
-Content-Type: multipart/form-data; boundary=1234567890
-Content-Length: 305
---1234567890
-Content-Disposition: form-data; name="str"
-string value
---1234567890
-Content-Disposition: form-data; name="int"
-123
---1234567890
-Content-Disposition: form-data; name="float"
-1.23
---1234567890
-Content-Disposition: form-data; name="yaml"
----
-- a
-- b
-- c
---1234567890--
-_end_of_message_
-          msg.gsub(/\n/, "\r\n")
-        end
-        def _multipart_request
-          { :url => "http://www.example.com/path",
-            :http_version => '1.1',
-            :request_method => 'POST',
-            :headers => {
-              "Host" => 'www.example.com',
-              "Content-Type" => "multipart/form-data; boundary=1234567890",
-              "Content-Length" => 305},
-            :params => {
-              'str' => 'string value',
-              'int' => 123,
-              'float' => 1.23,
-              'yaml' => ['a', 'b', 'c']}
-            }
-        end
-        def multipart_request_with_multiple_values
-          msg = <<-_end_of_message_
-POST /path HTTP/1.1
-Host: www.example.com
-Content-Type: multipart/form-data; boundary=1234567890
-Content-Length: 302
---1234567890
-Content-Disposition: form-data; name="one"
-string value
---1234567890
-Content-Disposition: form-data; name="one"
-123
---1234567890
-Content-Disposition: form-data; name="two"
-1.23
---1234567890
-Content-Disposition: form-data; name="one"
----
-- a
-- b
-- c
---1234567890--
-_end_of_message_
-          msg.gsub(/\n/, "\r\n")
-        end
-        def _multipart_request_with_multiple_values
-          { :url => "http://www.example.com/path",
-            :http_version => '1.1',
-            :request_method => 'POST',
-            :headers => {
-              "Host" => 'www.example.com',
-              "Content-Type" => "multipart/form-data; boundary=1234567890",
-              "Content-Length" => 302},
-            :params => {
-              'one' => ['string value', 123, ['a', 'b', 'c']],
-              'two' => 1.23}
-            }
-        end
-      end
     end
   end
 end

metadata CHANGED Viewed

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: tap-http
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.1.0
 platform: ruby
 authors:
 - Simon Chiang
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
-date: 2008-11-18 00:00:00 -07:00
+date: 2008-11-25 00:00:00 -07:00
 default_executable:
 dependencies:
 - !ruby/object:Gem::Dependency
@@ -43,7 +43,7 @@ files:
 - README
 - MIT-LICENSE
 has_rdoc: true
-homepage: http://rubyforge.org/projects/tap
+homepage: http://tap.rubyforge.org/projects/tap-http
 post_install_message:
 rdoc_options: []
@@ -64,7 +64,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubyforge_project: tap
-rubygems_version: 1.3.0
+rubygems_version: 1.3.1
 signing_key:
 specification_version: 2
 summary: A task library for submitting http requests using Tap.