RubyGems - tap-mechanize - Versions diffs - 0.5.0 - Mend

tap-mechanize 0.5.0

Files changed (21) hide show

data/History +41 -0
data/MIT-LICENSE +19 -0
data/README +58 -0
data/lib/tap/mechanize/agent.rb +101 -0
data/lib/tap/mechanize/capture.rb +194 -0
data/lib/tap/mechanize/get.rb +20 -0
data/lib/tap/mechanize/request.rb +33 -0
data/lib/tap/mechanize/submit.rb +30 -0
data/lib/tap/mechanize/test.rb +59 -0
data/lib/tap/mechanize/test/echo_server.rb +20 -0
data/lib/tap/mechanize/test/mock_server.rb +31 -0
data/lib/tap/mechanize/utils.rb +296 -0
data/tap.yml +1 -0
data/views/tap/mechanize/capture/http.erb +31 -0
data/views/tap/mechanize/capture/index.erb +11 -0
data/views/tap/mechanize/capture/redirect.css +28 -0
data/views/tap/mechanize/capture/redirect.js +184 -0
data/views/tap/mechanize/capture/redirect_http.erb +15 -0
data/views/tap/mechanize/capture/test.erb +108 -0
data/views/tap/mechanize/capture/tutorial.erb +57 -0
metadata +108 -0

data/History ADDED Viewed

@@ -0,0 +1,41 @@
+== 0.5.0 / 2009-03-23
+Total refactoring of the entire namespace.  TapHttp is now
+known as TapMechanize.  Obviously not backwards compatible.
+* updated redirect-http to allow for returns within
+  the existing onSubmit
+* Request now returns Page rather than page content
+  by default
+* refactored HttpTest to Mechanize::Test
+* added Tap::Mechanize::Agent
+== 0.4.0 / 2009-03-17
+Significant rewrite using an updated version of TapServer
+and Mechanize.  TapHttp now serves its own redirection
+commands via TapUbiquity.
+== 0.3.1 / 2009-02-19
+Improved handling of multipart/form data.
+* http_to_yaml returns config file as an attachment.
+== 0.3.0 / 2009-02-19
+Rework of cgi scripts a tap controllers.  Nearly complete
+internal refactoring (this is not a backwards compatible
+release).
+* Reworked CGI scripts as Tap Controllers
+* Reworked Dispatch as the Request module
+* Added Get and Submit tasks
+* Reworked HttpTest to allow specification of a
+  Rack application
+== 0.2.1 / 2009-02-17
+Minor updates to utilize Tap 0.12.0
+* Added History

data/MIT-LICENSE ADDED Viewed

@@ -0,0 +1,19 @@
+Copyright (c) 2008-2009, Regents of the University of Colorado.
+Permission is hereby granted, free of charge, to any person obtaining a copy of this
+software and associated documentation files (the "Software"), to deal in the Software
+without restriction, including without limitation the rights to use, copy, modify, merge,
+publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
+to whom the Software is furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in all copies or
+substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.

data/README ADDED Viewed

@@ -0,0 +1,58 @@
+= {Tap Mechanize}[http://tap.rubyforge.org/projects/tap-mechanize]
+A task library for submitting http requests using Tap[http://tap.rubyforge.org] and Mechanize[http://mechanize.rubyforge.org]
+== Description
+Tap::Mechanize provides tasks and controllers to automate interaction with
+websites, and in particular with web applications. Tap::Mechanize provides a
+{Ubiquity}[http://labs.mozilla.com/2008/08/introducing-ubiquity/] command to
+redirect and capture HTTP requests as YAML. The resulting request
+configuration files can be edited and resubmitted using a Request task.
+Multi-step actions, file uploads, and actions that require the use of HTTPS
+may all be automated in this way.
+* Lighthouse[http://bahuvrihi.lighthouseapp.com/projects/9908-tap-task-application/tickets]
+* Github[http://github.com/bahuvrihi/tap-mechanize/tree/master]
+* {Google Group}[http://groups.google.com/group/ruby-on-tap]
+=== Usage
+Tap::Mechanize submits HTTP requests using the Tap::Mechanize::Submit task.
+Headers and parameters may be specified, but only a uri is required.
+  r = Tap::Mechanize::Submit.new
+  r.process(:uri => 'http://www.google.com')[0, 80]
+  # => "<html><head><meta http-equiv=\"content-type\" content=\"text/html; charset=ISO-8859"
+=== Capturing Web Forms
+HTTP requests from web forms may be captured and resubmitted using a
+combination of tools. First start a tap server from the command line:
+  % tap server
+Now open a browser and work through the tutorial: http://localhost:8080/capture/tutorial
+The current capture scripts redirect via the onsubmit and onclick events in a
+web page. This works in most cases, but is known to fail in some circumstances
+(ex Java Server Pages). A workaround exists using the {Live HTTP
+Headers}[https://addons.mozilla.org/en-US/firefox/addon/3829] addon. Start a
+web server and see http://localhost:8080/capture/http.
+== Installation
+Tap::Mechanize is available as a gem on
+RubyForge[http://rubyforge.org/projects/tap]. Use:
+  % gem install tap-mechanize
+Note this package was once called tap-http.
+== Info
+Copyright (c) 2008-2009, Regents of the University of Colorado.
+Developer:: {Simon Chiang}[http://bahuvrihi.wordpress.com], {Biomolecular Structure Program}[http://biomol.uchsc.edu/], {Hansen Lab}[http://hsc-proteomics.uchsc.edu/hansenlab/]
+Support:: CU Denver School of Medicine Deans Academic Enrichment Fund
+Licence:: {MIT-Style}[link:files/MIT-LICENSE.html]

data/lib/tap/mechanize/agent.rb ADDED Viewed

@@ -0,0 +1,101 @@
+require 'mechanize'
+module Tap
+  module Mechanize
+    # A Configurable version of the WWW::Mechanize class.
+    class Agent < ::WWW::Mechanize
+      include Configurable
+      DEFAULT_ATTRIBUTES = DEFAULT_ATTRIBUTES.dup
+      DEFAULT_ATTRIBUTES[nil] = {:reader => false, :writer => false}
+      config :ca_file, nil
+      config :cert, nil
+      config :conditional_requests, true, &c.switch
+      config :follow_meta_refresh, false, &c.switch
+      config :keep_alive, true, &c.switch
+      config :keep_alive_time, 300, &c.integer_or_nil
+      config :key, nil
+      config :open_timeout, nil
+      config :pass, nil
+      config :read_timeout, nil
+      config :redirect_ok, true, &c.switch
+      config :redirection_limit, 20, &c.integer_or_nil
+      config :user_agent, "WWW-Mechanize/0.9.2 (http://rubyforge.org/projects/mechanize/)"
+      config :verify_callback, nil
+      config :watch_for_set, nil
+      def initialize(config={})
+        super()
+        initialize_config(config)
+      end
+      # Fetches the specified request.  Request must be a hash at least
+      # specifying a url; these are the allowed keys and defaults:
+      #
+      # url:: nil, must be specified
+      # request_method:: get
+      # params:: {}
+      # headers:: {}
+      #
+      def fetch(request)
+        request = symbolize(request)
+        # note get handles nil request methods (ie '') and has to
+        # reassign uri to url to make mechanize happy
+        case request[:request_method].to_s
+        when /get/i, '' then get(request.merge(:url => request[:uri]))
+        when /post/i    then submit(prepare_form(request), nil)
+        when /head/i    then head(nil, nil, request)
+        when /put/i     then put(nil, nil, request)
+        when /delete/i  then delete(nil, nil, request)
+        else raise "unsupported request method: #{request.inspect}"
+        end
+      end
+      protected
+      # taken from ActiveSupport
+      def symbolize(hash) # :nodoc:
+        hash.inject({}) do |options, (key, value)|
+          options[(key.to_sym rescue key) || key] = value
+          options
+        end
+      end
+      # A subclass of Hash used exclusively as the node for a form.
+      # Evidently a search method is necessary.
+      class FormNode < Hash # :nodoc:
+        def search(*args); []; end
+      end
+      # prepares a Mechanize::Form for the request.  this method is patterned
+      # after what happens in Mechanize#post and hopefully will not be
+      # necessary in the future.
+      def prepare_form(request) # :nodoc:
+        node = FormNode.new
+        node['method'] = request[:request_method]
+        node['action'] = request[:uri]
+        node['enctype'] = request[:headers] ? request[:headers]['Content-Type'] : nil
+        form = Form.new(node)
+        request[:params].each_pair do |key, value|
+          case value
+          when IO
+            form.enctype = 'multipart/form-data'
+            upload = Form::FileUpload.new(key.to_s, ::File.basename(value.path))
+            upload.file_data = value.read
+            form.file_uploads << upload
+          when Array
+            fields = value.collect {|val| Form::Field.new(key.to_s, val)}
+            form.fields.concat fields
+          else
+            form.fields << Form::Field.new(key.to_s, value)
+          end
+        end
+        form
+      end
+    end
+  end
+end

data/lib/tap/mechanize/capture.rb ADDED Viewed

@@ -0,0 +1,194 @@
+require 'tap/controller'
+require 'tap/mechanize/utils'
+require 'tap/ubiquity/utils'
+module Tap
+  module Mechanize
+    # :startdoc::controller
+    # :startdoc::ubiquity
+    class Capture < Tap::Controller
+      include Tap::Mechanize::Utils
+      include Tap::Ubiquity::Utils
+      PREFIX = '__redirect_http_'
+      REDIRECT_PARAMETER = '__redirect_http_original_action'
+      # Brings up the tutorial.
+      def index
+        render 'index.erb', :locals => {:captures => persistence.index }
+      end
+      def create(id, keep_content=true)
+        persistence.create(id) do |io|
+          io << YAML.dump([parse_request(keep_content)])
+        end
+        download(id)
+      end
+      def show(id)
+        response['Content-Type'] = "text/plain"
+        persistence.read(id)
+      end
+      def download(id)
+        path = persistence.path(id)
+        filename = id
+        filename += ".yml" if File.extname(id) == ""
+        response['Content-Type'] = "text/plain"
+        response['Content-Disposition'] = "attachment; filename=#{filename};"
+        persistence.read(id)
+      end
+      def update(id="request", keep_content=true)
+        path = persistence.path(id)
+        requests = File.exists?(path) ? YAML.load_file(path) : []
+        requests << parse_request(keep_content)
+        persistence.update(id) do |io|
+          io << YAML.dump(requests)
+        end
+        download(id)
+      end
+      def destroy(id)
+        persistence.destroy(id)
+        redirect uri(:index)
+      end
+      # Brings up a tutorial teaching how to capture and resubmit forms.
+      def tutorial
+        serve js_injection(:redirect_http) do |link|
+          content = render 'tutorial.erb'
+          content + link
+        end
+      end
+      def test
+        render 'test.erb'
+      end
+      # Say is the target of the tutorial.
+      def say
+        "<pre>#{request.params['words']}</pre>"
+      end
+      # Returns the redirection script.
+      def redirect_http
+        css = render 'redirect.css'
+        script = render 'redirect.js', :locals => {
+          :redirect_parameter => REDIRECT_PARAMETER,
+          :redirect_action => uri(:update),
+        }
+        content = render 'redirect_http.erb', :locals => {
+          :css => css,
+          :script => script
+        }
+        if request.get?
+          response['Content-Type'] = 'text/plain'
+          %Q{
+<div id="#{prefix}">
+#{content}
+</div>}
+        else
+          response['Content-Type'] = 'text/javascript'
+        %Q{
+if(current = document.getElementById("#{prefix}")) {
+  RedirectHttp.revert();
+} else {
+  var div = document.createElement("div");
+  div.id = "#{prefix}";
+  div.innerHTML = #{content.to_json};
+  document.body.insertBefore(div, document.body.firstChild);
+  RedirectHttp.redirect();
+}
+}
+        end
+      end
+      # Parses HTTP request
+      def http
+        if request.get?
+          render 'http.erb'
+        else
+          keep_content = request.params['keep_content'] == "true"
+          hash = {}
+          parse_http_request(request.params['http'], keep_content).each_pair do |key, value|
+            hash[key.to_s] = value
+          end
+          # remove extranous data
+          hash.delete('headers')
+          hash.delete('version')
+          response['Content-Type'] = "text/plain"
+          YAML.dump(hash)
+        end
+      end
+      protected
+      # helper for rendering... saves specification of
+      # :locals => {:prefix => PREFIX}
+      def prefix # :nodoc:
+        PREFIX
+      end
+      def capture_overloaded_parameters # :nodoc:
+        # perform the actions of Rack::Request::POST, but capture
+        # overloaded parameter names
+        env = request.env
+        env["rack.request.form_input"] = env["rack.input"]
+        unless env["rack.request.form_hash"] = parse_multipart(env)
+          env["rack.request.form_vars"] = env["rack.input"].read
+          env["rack.request.form_hash"] = Rack::Utils.parse_query(env["rack.request.form_vars"])
+          env["rack.input"].rewind if env["rack.input"].respond_to?(:rewind)
+        end
+      end
+      # helper to parse the request into a request hash for
+      # use by a Tap::Mechanize::Submit task
+      def parse_request(keep_content=true) # :nodoc:
+        if keep_content.kind_of?(String)
+          keep_content = keep_content =~ /true/i
+        end
+        capture_overloaded_parameters
+        hash = {}
+        parse_rack_request(request, keep_content).each_pair do |key, value|
+          hash[key.to_s] = value
+        end
+        action = hash['params'].delete(REDIRECT_PARAMETER)
+        hash['uri'] = case action
+        when /^http/
+          # action is an href already
+          action
+        when /^\//
+          # make action relative to host
+          action, query = action.split('?', 2)
+          uri = URI.parse(hash['headers']['Referer'].to_s)
+          uri.path = action
+          uri.query = query.to_s.gsub(/\s/, "+")
+          uri.to_s
+        else
+          # make action relative to Referer
+          base = File.dirname(hash['headers']['Referer'].to_s)
+          File.join(base, action)
+        end
+        # remove extranous data
+        hash.delete('headers')
+        hash.delete('version')
+        hash
+      end
+    end
+  end
+end

data/lib/tap/mechanize/get.rb ADDED Viewed

@@ -0,0 +1,20 @@
+require 'tap/mechanize/request'
+module Tap
+  module Mechanize
+    # :startdoc::manifest gets the uri
+    #
+    # Submits an Http request to the specified uri and returns the message
+    # body.
+    #
+    #   % tap run -- get http://tap.rubyforge.org --: dump
+    #
+    class Get < Request
+      # Gets the uri and returns the page content.
+      def process(uri)
+        super(:uri => uri).content
+      end
+    end
+  end
+end

data/lib/tap/mechanize/request.rb ADDED Viewed

@@ -0,0 +1,33 @@
+require 'tap/mechanize/agent'
+module Tap
+  module Mechanize
+    class Request < Tap::Task
+      nest :mechanize, Agent        # the mechanize agent
+      # Returns the mechanize agent.
+      #--
+      # Overrides the default reader to ensure the agent log is set.
+      def mechanize
+        @mechanize ||= begin
+          agent = Agent.new
+          agent.log = app.logger
+          agent
+        end
+      end
+      # Submits each request in order and returns the final Page.
+      # Returns nil if no requests are specified.
+      def process(requests)
+        unless requests.kind_of?(Array)
+          requests = [requests]
+        end
+        requests.inject(nil) do |last_page, request|
+          mechanize.fetch(request)
+        end
+      end
+    end
+  end
+end