romanbsd-tarantula 0.1.8

data/CHANGELOG

@@ -0,0 +1,47 @@
+v0.1.8 Add timeouts for crawls to help really long builds [Rob Sanheim]
+
+v0.1.7 Minor clean up [Rob Sanheim]
+
+v0.1.6 
+	* add testing for all Rails versions 2.0.2 and up
+	* various clean up and housekeeping tasks; 
+	* start Ruby 1.9 work (but we need Hpricot)
+	* show 50 chars of URL, not 30
+	* ensure that ActiveRecord gets loaded correctly for the crawler, so that it can rescue RecordNotFound exceptions
+	[Rob Sanheim]
+
+v0.1.5 Initial implementation of updated look-and-feel [Erik Yowell] [Jason Rudolph]
+
+v0.1.4 Bugfix: Include look-and-feel files when building the gem #16 [Jason Rudolph]
+
+v0.1.3 Update list of known static file types (e.g., PDFs) to prevent false reports of 404s for links to files that exist in RAILS_ROOT/public [Aaron Bedra]
+
+v0.1.2 Remove dependency on Facets gem [Aaron Bedra]
+
+v0.1.1 Bugfix: Add ability to handle anchor tags that lack an href attribute #13 [Kevin Gisi]
+
+v0.1.0 
+* Improve the generated test template to include inline documentation and make the simple case simple [Jason Rudolph]
+* Update README to better serve first-time users [Jason Rudolph]
+* Update development dependencies declarations [Jason Rudolph]
+* Internal refactorings [Aaron Bedra]
+** Convert test suite to micronaut
+** Replace Echoe with Jeweler for gem management
+** Remove unused code
+    
+v0.0.8.1
+* Fix numerous installation and initial setup issues
+* Enhance rake tasks to support use of Tarantula in a continuous integration environment
+** Use "rake tarantula:test" to run headless with build-friendly exit codes
+** Use "rake tarantula:report" to open the Tarantula report in your browser
+* Update README
+** Provide better installation and setup documentation
+** Include example of adding a custom attack handler
+* Simplify design to address concerns about hard-to-read fonts
+
+v0.0.5 
+* Make sure we don't include Relevance::Tarantula into Object - will cause issues with Rails dependencies and is a bad idea in general
+* Update Rakefile for development dependencies
+* Other small clean up tasks
+
+v0.0.1 Tarantula becomes a gem. [Aaron Bedra]

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008-2009 Relevance, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,162 @@
+= Tarantula
+
+== DESCRIPTION
+
+Tarantula is a big fuzzy spider. It crawls your Rails application, fuzzing data to see what breaks.
+
+== Usage
+
+=== Installation
+
+The latest and greatest version is always available on GitHub. (See the rakefile for dependencies, or
+just let Rubygems handle it.)
+
+    gem install relevance-tarantula --source http://gems.github.com
+
+You can also grab it from RubyForge, where we will push stable releases but may not be as bleeding edge
+as the GitHub gem.
+
+    gem install tarantula
+
+=== Project Setup
+
+To set up Tarantula into your application, add the following line into either config/environment.rb or
+config/environments/test.rb (preferred). This assumes that you have Rails 2.1 or higher installed.
+
+    config.gem 'relevance-tarantula', :source => "http://gems.github.com", :lib => 'relevance/tarantula'
+
+Since Rails doesn't (yet) support automatically loading rake tasks that live inside gems, you will need
+to update your Rakefile to load Tarantula's rake tasks. The simplest approach is to start by vendoring 
+Tarantula into your Rails app.
+
+    mkdir -p vendor/gems
+    cd vendor/gems
+    gem unpack relevance-tarantula
+
+You can then add the following line into your Rakefile, which will allow your application to discover 
+Tarantula's rake tasks.
+
+  load File.join(RAILS_ROOT, Dir["vendor/gems/relevance-tarantula-*/tasks/*.rake"])
+
+=== Crawling Your App
+
+Use the included rake task to create a Rails integration test that will allow Tarantula to crawl your
+app.
+
+   #!sh
+   rake tarantula:setup
+
+Take a moment to familiarize yourself with the generated test. If parts of your application require
+login, update the test to make sure Tarantula can access those parts of your app.
+
+   require "relevance/tarantula"
+   
+   class TarantulaTest < ActionController::IntegrationTest
+     # Load enough test data to ensure that there's a link to every page in your
+     # application. Doing so allows Tarantula to follow those links and crawl 
+     # every page.  For many applications, you can load a decent data set by
+     # loading all fixtures.
+     fixtures :all
+   
+     def test_tarantula
+       # If your application requires users to log in before accessing certain 
+       # pages, uncomment the lines below and update them to allow this test to
+       # log in to your application.  Doing so allows Tarantula to crawl the 
+       # pages that are only accessible to logged-in users.
+       # 
+       #   post '/session', :login => 'quentin', :password => 'monkey'
+       #   follow_redirect!
+       
+       tarantula_crawl(self)
+     end
+   end
+
+If you want to set custom options, you can get access to the crawler and set properties before running
+it. For example, this would turn on HTMLTidy.
+
+   def test_tarantula
+     post '/session', :login => 'kilgore', :password => 'trout'
+     assert_response :redirect
+     assert_redirected_to '/'
+     follow_redirect!
+
+     t = tarantula_crawler(self)
+     t.handlers << Relevance::Tarantula::TidyHandler.new
+     t.crawl '/'
+   end
+
+Now it's time to turn Tarantula loose on your app. Assuming your project is at /work/project/:
+
+   #!sh
+   cd /work/project
+   rake tarantula:test
+
+== Verbose Mode
+
+If you run the test using the steps shown above, Tarantula will produce a report in tmp/tarantula. You
+can also set VERBOSE=true to see more detail as the test runs.
+
+For more options, please see the test suite.
+
+== Allowed Errors
+
+If, for example, a 404 is an appropriate response for some URLs, you can tell Tarantula to allow 404s
+for URLs matching a given regex:
+
+     t = tarantula_crawler(self)
+     t.allow_404_for %r{/users/\d+/}
+
+== Custom Attack Handlers
+
+You can specify the attack strings that Tarantula throws at your application.
+
+    def test_tarantula
+      t = tarantula_crawler(self)
+
+      Relevance::Tarantula::AttackFormSubmission.attacks << { 
+        :name => :xss,
+        :input => "<script>gotcha!</script>",
+        :output => "<script>gotcha!</script>",
+      }
+
+      Relevance::Tarantula::AttackFormSubmission.attacks << { 
+        :name => :sql_injection,
+        :input => "a'; DROP TABLE posts;",
+      }
+
+      t.handlers << Relevance::Tarantula::AttackHandler.new
+      t.fuzzers << Relevance::Tarantula::AttackFormSubmission
+      t.times_to_crawl = 2
+      t.crawl "/posts"
+    end
+
+This example adds custom attacks for both SQL injection and XSS. It also tells Tarantula to crawl the
+app 2 times. This is important for XSS attacks because the results won't appear until the second time
+Tarantula performs the crawl.
+
+== Timeout
+
+You can specify a timeout for each specific crawl that Tarantula runs.  For example:
+
+  def test_tarantula
+    t = tarantula_crawler(self)
+    t.times_to_crawl = 2
+    t.crawl_timeout = 5.minutes
+    t.crawl "/"
+  end
+
+The above will crawl your app twice, and each specific crawl will timeout if it takes longer then 5 minutes.  You may need a timeout to keep the tarantula test time reasonable if your app is large or just happens to have a large amount of 'never-ending' links, such as with an any sort of "auto-admin" interface.
+
+== Bugs/Requests
+
+Please submit your bug reports, patches, or feature requests at Lighthouse:
+
+http://relevance.lighthouseapp.com/projects/17868-tarantula/overview
+
+You can view the continuous integration results for Tarantula, including results against all supported versions of Rails, on RunCodeRun here:
+
+http://runcoderun.com/relevance/tarantula
+
+== License
+
+Tarantula is released under the MIT license.

data/Rakefile

@@ -0,0 +1,69 @@
+require 'rake'
+require 'rake/testtask'
+require 'rake/rdoctask'
+gem "spicycode-micronaut", ">= 0.2.4"
+require 'micronaut'
+require 'micronaut/rake_task'
+
+begin
+  require 'jeweler'
+  files = ["CHANGELOG", "MIT-LICENSE", "Rakefile", "README.rdoc", "VERSION.yml"]
+  files << Dir["examples/**/*", "laf/**/*", "lib/**/*", "tasks/**/*", "template/**/*",
+               "vendor/**/*"]
+  
+  Jeweler::Tasks.new do |s|
+    s.name = "tarantula"
+    s.summary = "A big hairy fuzzy spider that crawls your site, wreaking havoc"
+    s.description = "A big hairy fuzzy spider that crawls your site, wreaking havoc"
+    s.homepage = "http://github.com/relevance/tarantula"
+    s.email = "opensource@thinkrelevance.com"
+    s.authors = ["Relevance, Inc."]
+    s.require_paths = ["lib"]
+    s.files = files.flatten
+    s.add_dependency 'htmlentities'
+    s.add_dependency 'hpricot'
+    s.rubyforge_project = 'thinkrelevance'
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end
+
+desc 'Generate documentation for the tarantula plugin.'
+Rake::RDocTask.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'Tarantula'
+  rdoc.options << '--line-numbers' << '--inline-source'
+  rdoc.rdoc_files.include('README.rdoc')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+desc "Run all micronaut examples"
+Micronaut::RakeTask.new :examples do |t|
+  t.pattern = "examples/**/*_example.rb"
+end
+
+namespace :examples do
+  desc "Run all micronaut examples using rcov"
+  Micronaut::RakeTask.new :coverage do |t|
+    t.pattern = "examples/**/*_example.rb"
+    t.rcov = true
+    t.rcov_opts = %[--exclude "gems/*,/Library/Ruby/*,config/*" --text-summary  --sort coverage --no-validator-links]
+  end
+  
+  RAILS_VERSIONS = %w[2.0.2 2.1.0 2.1.1 2.2.2 2.3.1 2.3.2]
+  
+  desc "Run exmaples with multiple versions of rails"
+  task :multi_rails do
+    RAILS_VERSIONS.each do |rails_version|
+      puts
+      sh "RAILS_VERSION='#{rails_version}' rake examples"
+    end
+  end
+  
+end
+
+if ENV["RUN_CODE_RUN"]
+  task :default => "examples:multi_rails"
+else
+  task :default => "examples"
+end

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:patch: 8
+:major: 0
+:minor: 1

data/examples/example_helper.rb

@@ -0,0 +1,48 @@
+lib_path = File.expand_path(File.dirname(__FILE__) + "/../lib")
+$LOAD_PATH.unshift lib_path unless $LOAD_PATH.include?(lib_path)
+
+gem "spicycode-micronaut", ">= 0.2.4"
+gem "log_buddy"
+gem "mocha"
+if rails_version = ENV['RAILS_VERSION']
+  gem "rails", rails_version
+end
+require "rails/version"
+if Rails::VERSION::STRING < "2.3.1" && RUBY_VERSION >= "1.9.1"
+  puts "Tarantula requires Rails 2.3.1 or higher for Ruby 1.9 support"
+  exit(1)
+end
+puts "==== Testing with Rails #{Rails::VERSION::STRING} ===="
+gem 'actionpack'
+gem 'activerecord'
+gem 'activesupport'
+
+require 'ostruct'
+require 'active_support'
+require 'action_controller'
+require 'active_record'
+require 'relevance/tarantula'
+require 'micronaut'
+require 'mocha'
+
+def test_output_dir
+  File.join(File.dirname(__FILE__), "..", "tmp", "test_output")
+end
+
+# TODO change puts/print to use a single method for logging, which will then make the stubbing cleaner
+def stub_puts_and_print(obj)
+  obj.stubs(:puts)
+  obj.stubs(:print)
+end
+
+def not_in_editor?
+  ['TM_MODE', 'EMACS', 'VIM'].all? { |k| !ENV.has_key?(k) }
+end
+
+Micronaut.configure do |c|
+  c.alias_example_to :fit, :focused => true
+  c.alias_example_to :xit, :disabled => true
+  c.mock_with :mocha
+  c.color_enabled = not_in_editor?
+  c.filter_run :focused => true
+end

data/examples/relevance/core_extensions/ellipsize_example.rb

@@ -0,0 +1,19 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "../..", "example_helper.rb"))
+
+describe "Relevance::CoreExtensions::Object#ellipsize" do
+  it "converts nil to empty string" do
+    nil.ellipsize.should == ""
+  end
+  
+  it "doesn't touch short strings" do
+    "hello".ellipsize.should == "hello"
+  end
+  
+  it "calls inspect on non-strings" do
+    [1,2,3].ellipsize.should == "[1, 2, 3]"
+  end
+
+  it "shortens long strings and adds ..." do
+    "long-string".ellipsize(5).should == "long-..."
+  end
+end

data/examples/relevance/core_extensions/file_example.rb

@@ -0,0 +1,8 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "../..", "example_helper.rb"))
+require 'relevance/core_extensions/file'
+
+describe "Relevance::CoreExtensions::File#extension" do
+  it "should return the extension without the leading dot" do
+    File.extension("foo.bar").should == "bar"
+  end
+end

data/examples/relevance/core_extensions/response_example.rb

@@ -0,0 +1,29 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "../..", "example_helper.rb"))
+require 'relevance/core_extensions/file'
+
+describe "Relevance::CoreExtensions::Response#html?" do
+  before do
+    @response = OpenStruct.new
+    @response.extend(Relevance::CoreExtensions::Response)
+  end         
+  
+  it "should be html if the content-type is 'text/html'" do
+    @response.content_type = "text/html"
+    @response.should be_html
+    @response.content_type = "text/html;charset=iso-8859-2"
+    @response.should be_html
+  end
+
+  it "should not be html if the content-type isn't an html type" do
+    @response.content_type = "text/plain"
+    @response.should_not be_html
+  end
+
+  # better ideas welcome, but be careful not to  
+  # castrate tarantula for proxies that don't set the content-type
+  it "should pretend we have html if the content-type is nil" do
+    @response.content_type = nil
+    @response.should be_html
+  end
+  
+end

data/examples/relevance/core_extensions/test_case_example.rb

@@ -0,0 +1,20 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "../..", "example_helper.rb"))
+require 'relevance/core_extensions/test_case'
+
+describe "TestCase extensions" do
+  pending "can create the crawler" do
+    Relevance::Tarantula::RailsIntegrationProxy.stubs(:rails_root).returns("STUB_RAILS_ROOT")
+    Relevance::Tarantula::Crawler.any_instance.stubs(:rails_root).returns("STUB_RAILS_ROOT")
+    tarantula_crawler(stub_everything)
+  end
+
+  pending "can crawl" do
+    (crawler = mock).expects(:crawl).with("/foo")
+    expects(:tarantula_crawler).returns(crawler)
+    tarantula_crawl(:integration_test_stub, :url => "/foo")
+  end
+  
+  it "should get mixed into ActionController::IntegrationTest" do
+    ActionController::IntegrationTest.ancestors.should include(Relevance::CoreExtensions::TestCaseExtensions)
+  end
+end

data/examples/relevance/tarantula/attack_form_submission_example.rb

@@ -0,0 +1,79 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb"))
+
+describe "Relevance::Tarantula::AttackFormSubmission" do
+  
+  # TODO: add more from field types to this example form as needed
+  before do
+    @tag = Hpricot(<<END)
+<form action="/session" method="post">
+  <input id="email" name="email" size="30" type="text" />
+  <textarea id="comment" name="comment"value="1" />
+  <input name="commit" type="submit" value="Postit" />
+  <input name="secret" type="hidden" value="secret" />
+  <select id="foo_opened_on_1i" name="foo[opened_on(1i)]">
+    <option value="2003">2003</option>
+    <option value="2004">2004</option>
+  </select> 
+</form>
+END
+    @form = Relevance::Tarantula::Form.new(@tag.at('form'))
+    @fs = Relevance::Tarantula::AttackFormSubmission.new(@form, Relevance::Tarantula::Attack.new({:name => 'foo_name', :input => 'foo_code', :output => 'foo_code'}))
+  end
+  
+  it "can mutate text areas" do
+    @fs.mutate_text_areas(@form).should == {"comment" => "foo_code"}
+  end
+  
+  it "can mutate selects" do
+    Hpricot::Elements.any_instance.stubs(:rand).returns(stub(:[] => "2006-stub"))
+    @fs.mutate_selects(@form).should == {"foo[opened_on(1i)]" => "2006-stub"}
+  end
+  
+  it "can mutate inputs" do
+    @fs.mutate_inputs(@form).should == {"commit"=>"foo_code", "secret"=>"foo_code", "email"=>"foo_code"}
+  end
+
+  it "has a signature based on action,  fields, and attack name" do
+    @fs.signature.should == ['/session', [
+      "comment", 
+      "commit", 
+      "email", 
+      "foo[opened_on(1i)]", 
+      "secret"],
+      "foo_name"
+    ]
+  end
+  
+  it "has a friendly to_s" do
+    @fs.to_s.should =~ %r{^/session post}
+  end
+  
+  it "processes all its attacks" do
+    Relevance::Tarantula::AttackFormSubmission.stubs(:attacks).returns([
+      Relevance::Tarantula::Attack.new({:name => 'foo_name1', :input => 'foo_input', :output => 'foo_output'}),
+      Relevance::Tarantula::Attack.new({:name => 'foo_name2', :input => 'foo_input', :output => 'foo_output'}),
+    ])
+    Relevance::Tarantula::AttackFormSubmission.mutate(@form).size.should == 2
+  end
+  
+  it "maps hash attacks to Attack instances" do
+    Relevance::Tarantula::AttackFormSubmission.instance_variable_set("@attacks", [{ :name => "attack name"}])
+    Relevance::Tarantula::AttackFormSubmission.attacks.should == [Relevance::Tarantula::Attack.new({:name => "attack name"})]
+  end
+end
+
+describe "Relevance::Tarantula::AttackFormSubmission for a crummy form" do
+  before do
+    @tag = Hpricot(<<END)
+<form action="/session" method="post">
+  <input value="no_name" />
+</form>
+END
+    @form = Relevance::Tarantula::Form.new(@tag.at('form'))
+    @fs = Relevance::Tarantula::AttackFormSubmission.new(@form, {:name => 'foo_name', :input => 'foo_code', :output => 'foo_code'})
+  end
+  
+  it "ignores unnamed inputs" do
+    @fs.mutate_inputs(@form).should == {}
+  end
+end

data/examples/relevance/tarantula/attack_handler_example.rb

@@ -0,0 +1,29 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb"))
+
+describe "Relevance::Tarantula::AttackHandler" do
+  before do
+    @handler = Relevance::Tarantula::AttackHandler.new
+    attack = Relevance::Tarantula::Attack.new({:name => 'foo_name', :input => 'foo_code', :output => '<bad>'})
+    @handler.stubs(:attacks).returns([attack])
+  end
+  
+  it "lets safe documents through" do
+    result = @handler.handle(Relevance::Tarantula::Result.new(:response => stub(:html? => true, :body => '<a href="/foo">good</a>')))
+    result.should == nil
+  end
+  
+  it "detects the supplied code" do
+    result = @handler.handle(Relevance::Tarantula::Result.new(:response => stub(:html? => true, :body => '<a href="/foo"><bad></a>')))
+    result.success.should == false
+  end
+end
+
+describe "Attacks without an output specified" do
+  it "never matches anything" do
+    handler = Relevance::Tarantula::AttackHandler.new
+    attack = Relevance::Tarantula::Attack.new({:name => 'foo_name', :input => 'foo_code'})
+    Relevance::Tarantula::AttackFormSubmission.stubs(:attacks).returns([attack])
+    result = handler.handle(Relevance::Tarantula::Result.new(:response => stub(:html? => true, :body => '<a href="/foo">good</a>')))
+    result.should == nil
+  end
+end