tarantula 0.0.8.1 → 0.1.4

data/README.rdoc

@@ -6,33 +6,86 @@ Tarantula is a big fuzzy spider. It crawls your Rails application, fuzzing data
 
 == Usage
 
-   #!sh
-   rake tarantula:setup
+=== Installation
+
+The latest and greatest version is always available on GitHub. (See the rakefile for dependencies, or
+just let Rubygems handle it.)
+
+    gem install relevance-tarantula --source http://gems.github.com
+
+You can also grab it from RubyForge, where we will push stable releases but may not be as bleeding edge
+as the GitHub gem.
+
+    gem install tarantula
+
+=== Project Setup
+
+To set up Tarantula into your application, add the following line into either config/environment.rb or
+config/environments/test.rb (preferred). This assumes that you have Rails 2.1 or higher installed.
 
-Creates a Rails integration test that looks like this, filling in your own auth params. You will probably want to include all fixtures.
+    config.gem 'relevance-tarantula', :source => "http://gems.github.com", :lib => 'relevance/tarantula'
+
+Since Rails doesn't (yet) support automatically loading rake tasks that live inside gems, you will need
+to update your Rakefile to load Tarantula's rake tasks. The simplest approach is to start by vendoring 
+Tarantula into your Rails app.
+
+    mkdir -p vendor/gems
+    cd vendor/gems
+    gem unpack relevance-tarantula
+
+You can then add the following line into your Rakefile, substituting the proper version of
+relevance-tarantula in the path.
 
-   require 'relevance/tarantula'
+    load File.join(RAILS_ROOT, "vendor/gems/relevance-tarantula-0.0.8.1/tasks/tarantula_tasks.rake")
+
+=== Crawling Your App
+
+Use the included rake task to create a Rails integration test that will allow Tarantula to crawl your
+app.
 
-   # in your test
-   def test_with_login
-       post '/sessions/create', :password => 'your-pass'
-       follow_redirect!
+   #!sh
+   rake tarantula:setup
+
+Take a moment to familiarize yourself with the generated test. If parts of your application require
+login, update the test to make sure Tarantula can access those parts of your app.
+
+   require "relevance/tarantula"
+   
+   class TarantulaTest < ActionController::IntegrationTest
+     # Load enough test data to ensure that there's a link to every page in your
+     # application. Doing so allows Tarantula to follow those links and crawl 
+     # every page.  For many applications, you can load a decent data set by
+     # loading all fixtures.
+     fixtures :all
+   
+     def test_tarantula
+       # If your application requires users to log in before accessing certain 
+       # pages, uncomment the lines below and update them to allow this test to
+       # log in to your application.  Doing so allows Tarantula to crawl the 
+       # pages that are only accessible to logged-in users.
+       # 
+       #   post '/session', :login => 'quentin', :password => 'monkey'
+       #   follow_redirect!
+       
        tarantula_crawl(self)
+     end
    end
 
-If you want to set custom options, you can get access to the crawler and set properties before running it. For example, this would turn on HTMLTidy.
+If you want to set custom options, you can get access to the crawler and set properties before running
+it. For example, this would turn on HTMLTidy.
+
+   def test_tarantula
+     post '/session', :login => 'kilgore', :password => 'trout'
+     assert_response :redirect
+     assert_redirected_to '/'
+     follow_redirect!
 
-   def test_with_login
-       post '/sessions/create', :password => 'your-pass'
-       assert_response :redirect
-       assert_redirected_to '/'
-       follow_redirect!
-       t = tarantula_crawler(self)
-       t.handlers << Relevance::Tarantula::TidyHandler.new
-       t.crawl '/'
+     t = tarantula_crawler(self)
+     t.handlers << Relevance::Tarantula::TidyHandler.new
+     t.crawl '/'
    end
 
-Assuming your project is at /work/project/:
+Now it's time to turn Tarantula loose on your app. Assuming your project is at /work/project/:
 
    #!sh
    cd /work/project
@@ -40,14 +93,15 @@ Assuming your project is at /work/project/:
 
 == Verbose Mode
 
-If you run the test you will get a report in tmp/tarantula. You can also set VERBOSE=true to see more detail as the test runs.
+If you run the test using the steps shown above, Tarantula will produce a report in tmp/tarantula. You
+can also set VERBOSE=true to see more detail as the test runs.
 
-For more options see the test suite.
+For more options, please see the test suite.
 
 == Allowed Errors
 
-If, for example, a 404 is an appropriate response for some URLs, you can
-tell Tarantula to allow 404s for URLs matching a regexp:
+If, for example, a 404 is an appropriate response for some URLs, you can tell Tarantula to allow 404s
+for URLs matching a given regex:
 
      t = tarantula_crawler(self)
      t.allow_404_for %r{/users/\d+/}
@@ -76,38 +130,13 @@ You can specify the attack strings that Tarantula throws at your application.
       t.crawl "/posts"
     end
 
-This example adds custom attacks for both SQL injection and XSS.  It also tells tarantula to crawl the app 2 times.  This
-is important for XSS attacks because the results won't appear until the second time tarantula performs the crawl.
-
-== Install
-
-See the rakefile for dependencies, or just let Rubygems handle it.
-
-The latest and greatest gem will always be available from Github:
-
-    gem install relevance-tarantula --source http://gems.github.com
-
-To setup tarantula in your application add the following line into either config/environment.rb or config/environments/test.rb (preferred).
-This assumes that you have Rails 2.1 or higher installed.
-
-    config.gem 'relevance-tarantula', :source => "http://gems.github.com", :lib => 'relevance/tarantula'
-
-Since rails doesn't (yet) support loading rake tasks that live inside gems you will need to update your Rakefile.  This assumes that you have vendored tarantula.  Simply run
-	
-	cd vendor/gems
-    gem unpack relevance-tarantula
-
-You can then add the following line into your Rakefile, substituting the proper version of relevance-tarantula in the path.
-
-    load File.join(RAILS_ROOT, "vendor/gems/relevance-tarantula-0.0.8.1/tasks/tarantula_tasks.rake")
-
-You can also grab it from Rubyforge, where we will push stable releases but may not be as bleeding edge as the Github gem.
-
-    gem install tarantula
+This example adds custom attacks for both SQL injection and XSS. It also tells Tarantula to crawl the
+app 2 times. This is important for XSS attacks because the results won't appear until the second time
+Tarantula performs the crawl.
 
 == Bugs/Requests
 
-Please submit your bug reports, patches or feature requests at Lighthouse:
+Please submit your bug reports, patches, or feature requests at Lighthouse:
 
 http://relevance.lighthouseapp.com/projects/17868-tarantula/overview
 

data/Rakefile

@@ -1,27 +1,30 @@
 require 'rake'
 require 'rake/testtask'
 require 'rake/rdoctask'
+require 'rcov/rcovtask'
 require 'rubygems'
-
-gem 'echoe', '~> 3.0.1'
-require 'echoe'
+gem "spicycode-micronaut", ">= 0.2.0"
+require 'micronaut'
+require 'micronaut/rake_task'
 require 'lib/relevance/tarantula.rb'
 
-echoe = Echoe.new('tarantula') do |p|
-  p.rubyforge_name = 'thinkrelevance'
-  p.author = ["Relevance"]
-  p.email = 'opensource@thinkrelevance.com'
-  p.version = Relevance::Tarantula::VERSION
-  p.summary = "A big hairy fuzzy spider that crawls your site, wreaking havoc"
-  p.description = "A big hairy fuzzy spider that crawls your site, wreaking havoc"
-  p.url = "http://github.com/relevance/tarantula"
-  p.rdoc_pattern = /^(lib|bin)|txt|rdoc|CHANGELOG|MIT-LICENSE$/
-  rdoc_template = `allison --path`.strip << ".rb"
-  p.rdoc_template = rdoc_template
-  p.test_pattern = 'test/**/*_test.rb'
-  p.manifest_name = 'manifest.txt'
-  p.dependencies = ['htmlentities', 'hpricot', 'facets >=2.4.3', 'actionpack', 'activesupport']
-  p.development_dependencies = ['ruby-debug', 'test-spec', 'mocha']
+begin
+  require 'jeweler'
+  files = ["CHANGELOG", "MIT-LICENSE", "Rakefile", "README.rdoc", "VERSION.yml"]
+  files << Dir["examples/**/*", "laf/**/*", "lib/**/*", "tasks/**/*", "template/**/*"]
+  
+  Jeweler::Tasks.new do |s|
+    s.name = "tarantula"
+    s.summary = "A big hairy fuzzy spider that crawls your site, wreaking havoc"
+    s.description = "A big hairy fuzzy spider that crawls your site, wreaking havoc"
+    s.homepage = "http://github.com/relevance/tarantula"
+    s.email = "opensource@thinkrelevance.com"
+    s.authors = ["Relevance, Inc."]
+    s.require_paths = ["lib"]
+    s.files = files.flatten
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
 end
 
 desc 'Generate documentation for the tarantula plugin.'
@@ -33,32 +36,18 @@ Rake::RDocTask.new(:rdoc) do |rdoc|
   rdoc.rdoc_files.include('lib/**/*.rb')
 end
 
-begin
-  require 'rcov'
-  require "rcov/rcovtask"
-
-  namespace :coverage do
-    rcov_output = ENV["CC_BUILD_ARTIFACTS"] || 'tmp/coverage'
-    rcov_exclusions = %w{ /Library/Ruby/* }.join(',')
-
-    desc "Delete aggregate coverage data."
-    task(:clean) { rm_f "rcov_tmp" }
-
-    Rcov::RcovTask.new(:unit => :clean) do |t|
-      t.test_files = FileList['test/**/*_test.rb']
-      t.rcov_opts = ["--sort coverage", "--aggregate 'rcov_tmp'", "--html", "--rails", "--exclude '#{rcov_exclusions}'"]
-      t.output_dir = rcov_output + '/unit'
-    end
+desc "Run all micronaut examples"
+Micronaut::RakeTask.new :examples do |t|
+  t.pattern = "examples/**/*_example.rb"
+end
 
-    desc "Generate and open coverage report"
-    task(:all => [:unit]) do
-      system("open #{rcov_output}/unit/index.html") if PLATFORM['darwin']
-    end
-  end
-rescue LoadError
-  if RUBY_PLATFORM =~ /java/
-    puts 'running in jruby - rcov tasks not available'
-  else
-    puts 'sudo gem install rcov # if you want the rcov tasks'
+namespace :examples do
+  desc "Run all micronaut examples using rcov"
+  Micronaut::RakeTask.new :coverage do |t|
+    t.pattern = "examples/**/*_example.rb"
+    t.rcov = true
+    t.rcov_opts = %[--exclude "gems/*,/Library/Ruby/*,config/*" --text-summary  --sort coverage --no-validator-links]
   end
 end
+
+task :default => "examples"

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:patch: 4
+:major: 0
+:minor: 1

data/examples/example_helper.rb

@@ -0,0 +1,50 @@
+lib_path = File.expand_path(File.dirname(__FILE__) + "/../lib")
+$LOAD_PATH.unshift lib_path unless $LOAD_PATH.include?(lib_path)
+
+require 'rubygems'
+gem "spicycode-micronaut", ">= 0.2.0"
+gem "log_buddy"
+gem "mocha"
+gem 'ruby-debug'
+gem 'test-spec'
+gem 'actionpack'
+gem 'activerecord'
+gem 'activesupport'
+
+require 'ostruct'
+require 'ruby-debug'
+require 'activerecord'
+require 'relevance/tarantula'
+require 'micronaut'
+require 'mocha'
+
+# needed for html-scanner, grr
+require 'active_support'
+require 'action_controller'
+
+def test_output_dir
+  File.join(File.dirname(__FILE__), "..", "tmp", "test_output")
+end
+
+# TODO change puts/print to use a single method for logging, which will then make the stubbing cleaner
+def stub_puts_and_print(obj)
+  obj.stubs(:puts)
+  obj.stubs(:print)
+end
+
+def not_in_editor?
+  ['TM_MODE', 'EMACS', 'VIM'].all? { |k| !ENV.has_key?(k) }
+end
+
+def in_runcoderun?
+  ENV["RUN_CODE_RUN"]
+end
+
+Micronaut.configure do |c|
+  c.formatter = :documentation if in_runcoderun?
+  c.alias_example_to :fit, :focused => true
+  c.alias_example_to :xit, :disabled => true
+  c.mock_with :mocha
+  c.color_enabled = not_in_editor?
+  c.filter_run :focused => true
+end

data/{test/relevance/core_extensions/ellipsize_test.rb → examples/relevance/core_extensions/ellipsize_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "../..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "../..", "example_helper.rb")
 
 describe "Relevance::CoreExtensions::Object#ellipsize" do
   it "converts nil to empty string" do

data/{test/relevance/core_extensions/file_test.rb → examples/relevance/core_extensions/file_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "../..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "../..", "example_helper.rb")
 require 'relevance/core_extensions/file'
 
 describe "Relevance::CoreExtensions::File#extension" do

data/{test/relevance/core_extensions/response_test.rb → examples/relevance/core_extensions/response_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "../..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "../..", "example_helper.rb")
 require 'relevance/core_extensions/file'
 
 describe "Relevance::CoreExtensions::Response#html?" do
@@ -9,21 +9,21 @@ describe "Relevance::CoreExtensions::Response#html?" do
   
   it "should be html if the content-type is 'text/html'" do
     @response.content_type = "text/html"
-    @response.should.be.html
+    @response.should be_html
     @response.content_type = "text/html;charset=iso-8859-2"
-    @response.should.be.html
+    @response.should be_html
   end
 
   it "should not be html if the content-type isn't an html type" do
     @response.content_type = "text/plain"
-    @response.should.not.be.html
+    @response.should_not be_html
   end
 
   # better ideas welcome, but be careful not to  
   # castrate tarantula for proxies that don't set the content-type
   it "should pretend we have html if the content-type is nil" do
     @response.content_type = nil
-    @response.should.be.html
+    @response.should be_html
   end
   
 end

data/{test/relevance/core_extensions/test_case_test.rb → examples/relevance/core_extensions/test_case_example.rb}

@@ -1,14 +1,14 @@
-require File.join(File.dirname(__FILE__), "../..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "../..", "example_helper.rb")
 require 'relevance/core_extensions/test_case'
 
 describe "TestCase extensions" do
-  it "can create the crawler" do
+  pending "can create the crawler" do
     Relevance::Tarantula::RailsIntegrationProxy.stubs(:rails_root).returns("STUB_RAILS_ROOT")
     Relevance::Tarantula::Crawler.any_instance.stubs(:rails_root).returns("STUB_RAILS_ROOT")
     tarantula_crawler(stub_everything)
   end
 
-  it "can crawl" do
+  pending "can crawl" do
     (crawler = mock).expects(:crawl).with("/foo")
     expects(:tarantula_crawler).returns(crawler)
     tarantula_crawl(:integration_test_stub, :url => "/foo")

data/{test/relevance/tarantula/attack_form_submission_test.rb → examples/relevance/tarantula/attack_form_submission_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "..", "..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb")
 
 describe "Relevance::Tarantula::AttackFormSubmission" do
   

data/{test/relevance/tarantula/attack_handler_test.rb → examples/relevance/tarantula/attack_handler_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "..", "..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb")
 
 describe "Relevance::Tarantula::AttackHandler" do
   before do

data/{test/relevance/tarantula/crawler_test.rb → examples/relevance/tarantula/crawler_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "..", "..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb")
 
 describe 'Relevance::Tarantula::Crawler#transform_url' do
   before {@crawler = Relevance::Tarantula::Crawler.new}
@@ -67,7 +67,7 @@ describe 'Relevance::Tarantula::Crawler#crawl' do
     crawler = Relevance::Tarantula::Crawler.new
     crawler.expects(:do_crawl).raises(RuntimeError)
     crawler.expects(:report_results)
-    lambda {crawler.crawl('/')}.should.raise(RuntimeError)
+    lambda {crawler.crawl('/')}.should raise_error(RuntimeError)
   end
 end
 
@@ -291,6 +291,6 @@ describe "allow_nnn_for" do
   
   it "chains up to super for method_missing" do
     crawler = Relevance::Tarantula::Crawler.new
-    lambda{crawler.foo}.should.raise(NoMethodError)
+    lambda{crawler.foo}.should raise_error(NoMethodError)
   end
 end

data/{test/relevance/tarantula/form_test.rb → examples/relevance/tarantula/form_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "..", "..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb")
 
 describe "Relevance::Tarantula::Form large example" do
   before do

data/{test/relevance/tarantula/form_submission_test.rb → examples/relevance/tarantula/form_submission_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "..", "..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb")
 
 describe "Relevance::Tarantula::FormSubmission" do
   

data/{test/relevance/tarantula/html_document_handler_test.rb → examples/relevance/tarantula/html_document_handler_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "..", "..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb")
 
 describe "Relevance::Tarantula::HtmlDocumentHandler" do
   

data/{test/relevance/tarantula/html_report_helper_test.rb → examples/relevance/tarantula/html_report_helper_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "..", "..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb")
 
 module HtmlReportHelperSpec
   # Is there an idiom for this?

data/{test/relevance/tarantula/html_reporter_test.rb → examples/relevance/tarantula/html_reporter_example.rb}

@@ -1,4 +1,4 @@
-require File.join(File.dirname(__FILE__), "..", "..", "test_helper.rb")
+require File.join(File.dirname(__FILE__), "..", "..", "example_helper.rb")
 
 describe "Relevance::Tarantula::HtmlReporter file output" do
 
@@ -55,7 +55,7 @@ blah blah blah
     stub_puts_and_print(reporter)
     (@success_results + @fail_results).each {|r| reporter.report(r)}
     reporter.finish_report(@test_name)
-    File.should.exist @index
+    File.exist?(@index).should be_true
   end 
   
   it "creates a final report with links to detailed reports in subdirs" do
@@ -76,7 +76,7 @@ blah blah blah
     stub_puts_and_print(reporter)
     (@success_results + @fail_results).each {|r| reporter.report(r)}
     reporter.finish_report(@test_name)    
-    File.should.exist @detail
+    File.exist?(@detail).should be_true
   end
 
 end