panmind-tarantula 0.3.3

data/CHANGELOG

@@ -0,0 +1,49 @@
+v0.2.0 Implementation of updated look-and-feel [Jason Rudolph]
+
+v0.1.8 Add timeouts for crawls to help really long builds [Rob Sanheim]
+
+v0.1.7 Minor clean up [Rob Sanheim]
+
+v0.1.6 
+	* add testing for all Rails versions 2.0.2 and up
+	* various clean up and housekeeping tasks; 
+	* start Ruby 1.9 work (but we need Hpricot)
+	* show 50 chars of URL, not 30
+	* ensure that ActiveRecord gets loaded correctly for the crawler, so that it can rescue RecordNotFound exceptions
+	[Rob Sanheim]
+
+v0.1.5 Initial implementation of updated look-and-feel [Erik Yowell] [Jason Rudolph]
+
+v0.1.4 Bugfix: Include look-and-feel files when building the gem #16 [Jason Rudolph]
+
+v0.1.3 Update list of known static file types (e.g., PDFs) to prevent false reports of 404s for links to files that exist in RAILS_ROOT/public [Aaron Bedra]
+
+v0.1.2 Remove dependency on Facets gem [Aaron Bedra]
+
+v0.1.1 Bugfix: Add ability to handle anchor tags that lack an href attribute #13 [Kevin Gisi]
+
+v0.1.0 
+* Improve the generated test template to include inline documentation and make the simple case simple [Jason Rudolph]
+* Update README to better serve first-time users [Jason Rudolph]
+* Update development dependencies declarations [Jason Rudolph]
+* Internal refactorings [Aaron Bedra]
+** Convert test suite to micronaut
+** Replace Echoe with Jeweler for gem management
+** Remove unused code
+    
+v0.0.8.1
+* Fix numerous installation and initial setup issues
+* Enhance rake tasks to support use of Tarantula in a continuous integration environment
+** Use "rake tarantula:test" to run headless with build-friendly exit codes
+** Use "rake tarantula:report" to open the Tarantula report in your browser
+* Update README
+** Provide better installation and setup documentation
+** Include example of adding a custom attack handler
+* Simplify design to address concerns about hard-to-read fonts
+
+v0.0.5 
+* Make sure we don't include Relevance::Tarantula into Object - will cause issues with Rails dependencies and is a bad idea in general
+* Update Rakefile for development dependencies
+* Other small clean up tasks
+
+v0.0.1 Tarantula becomes a gem. [Aaron Bedra]

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008-2009 Relevance, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,171 @@
+= Tarantula
+
+== Purpose of this fork
+
+ * Add panmind/ssl_helper interaction support - if the crawler receives
+   a `302` response redirecting to the very same requested URL, assume
+   it is a SSL requirement and use `https!` to set the SSL request hdrs;
+
+ * Add panmind/jquery-ajax-nav support - if the crawler receives a `302`
+   response with a `#` or `#!` in it, assume it is an XHR requirement
+   thus use `xhr` to re-send the request.
+
+== DESCRIPTION
+
+Tarantula is a big fuzzy spider. It crawls your Rails application, fuzzing data to see what breaks.
+
+== Usage
+
+=== Installation
+
+The latest and greatest version is always available on GitHub. (See the rakefile for dependencies, or
+just let RubyGems handle it.)
+
+    gem install tarantula
+
+You can also grab it from RubyForge, where we will push stable releases but may not be as bleeding edge
+as the GitHub gem.
+
+    gem install tarantula
+
+=== Project Setup
+
+To set up Tarantula into your application, add the following line into either config/environment.rb or
+config/environments/test.rb (preferred). This assumes that you have Rails 2.1 or higher installed.
+
+    config.gem 'tarantula', :lib => 'relevance/tarantula'
+
+Since Rails doesn't (yet) support automatically loading rake tasks that live inside gems, you will need
+to update your Rakefile to load Tarantula's rake tasks. The simplest approach is to start by vendoring 
+Tarantula into your Rails app.
+
+    mkdir -p vendor/gems
+    cd vendor/gems
+    gem unpack tarantula
+
+You can then add the following line into your Rakefile, which will allow your application to discover 
+Tarantula's rake tasks.
+
+  load File.join(RAILS_ROOT, Dir["vendor/gems/tarantula-*/tasks/*.rake"])
+
+=== Crawling Your App
+
+Use the included rake task to create a Rails integration test that will allow Tarantula to crawl your
+app.
+
+   #!sh
+   rake tarantula:setup
+
+Take a moment to familiarize yourself with the generated test. If parts of your application require
+login, update the test to make sure Tarantula can access those parts of your app.
+
+   require "relevance/tarantula"
+   
+   class TarantulaTest < ActionController::IntegrationTest
+     # Load enough test data to ensure that there's a link to every page in your
+     # application. Doing so allows Tarantula to follow those links and crawl 
+     # every page.  For many applications, you can load a decent data set by
+     # loading all fixtures.
+     fixtures :all
+   
+     def test_tarantula
+       # If your application requires users to log in before accessing certain 
+       # pages, uncomment the lines below and update them to allow this test to
+       # log in to your application.  Doing so allows Tarantula to crawl the 
+       # pages that are only accessible to logged-in users.
+       # 
+       #   post '/session', :login => 'quentin', :password => 'monkey'
+       #   follow_redirect!
+       
+       tarantula_crawl(self)
+     end
+   end
+
+If you want to set custom options, you can get access to the crawler and set properties before running
+it. For example, this would turn on HTMLTidy.
+
+   def test_tarantula
+     post '/session', :login => 'kilgore', :password => 'trout'
+     assert_response :redirect
+     assert_redirected_to '/'
+     follow_redirect!
+
+     t = tarantula_crawler(self)
+     t.handlers << Relevance::Tarantula::TidyHandler.new
+     t.crawl '/'
+   end
+
+Now it's time to turn Tarantula loose on your app. Assuming your project is at /work/project/:
+
+   #!sh
+   cd /work/project
+   rake tarantula:test
+
+== Verbose Mode
+
+If you run the test using the steps shown above, Tarantula will produce a report in tmp/tarantula. You
+can also set VERBOSE=true to see more detail as the test runs.
+
+For more options, please see the test suite.
+
+== Allowed Errors
+
+If, for example, a 404 is an appropriate response for some URLs, you can tell Tarantula to allow 404s
+for URLs matching a given regex:
+
+     t = tarantula_crawler(self)
+     t.allow_404_for %r{/users/\d+/}
+
+== Testing for Common Attacks
+
+You can specify the attack strings that Tarantula throws at your application.
+
+    def test_tarantula
+      t = tarantula_crawler(self)
+
+      Relevance::Tarantula::FormSubmission.attacks << { 
+        :name => :xss,
+        :input => "<script>gotcha!</script>",
+        :output => "<script>gotcha!</script>",
+      }
+
+      Relevance::Tarantula::FormSubmission.attacks << { 
+        :name => :sql_injection,
+        :input => "a'; DROP TABLE posts;",
+      }
+
+      t.handlers << Relevance::Tarantula::AttackHandler.new
+      t.times_to_crawl = 2
+      t.crawl "/posts"
+    end
+
+This example adds custom attacks for both SQL injection and XSS. It also tells Tarantula to crawl the
+app 2 times. This is important for XSS attacks because the results won't appear until the second time
+Tarantula performs the crawl.
+
+== Timeout
+
+You can specify a timeout for each specific crawl that Tarantula runs.  For example:
+
+  def test_tarantula
+    t = tarantula_crawler(self)
+    t.times_to_crawl = 2
+    t.crawl_timeout = 5.minutes
+    t.crawl "/"
+  end
+
+The above will crawl your app twice, and each specific crawl will timeout if it takes longer then 5 minutes.  You may need a timeout to keep the tarantula test time reasonable if your app is large or just happens to have a large amount of 'never-ending' links, such as with an any sort of "auto-admin" interface.
+
+== Bugs/Requests
+
+Please submit your bug reports, patches, or feature requests at Lighthouse:
+
+http://relevance.lighthouseapp.com/projects/17868-tarantula/overview
+
+You can view the continuous integration results for Tarantula, including results against all supported versions of Rails, on RunCodeRun here:
+
+http://runcoderun.com/relevance/tarantula
+
+== License
+
+Tarantula is released under the MIT license.

data/Rakefile

@@ -0,0 +1,83 @@
+require 'rake'
+require 'rake/testtask'
+require 'rake/rdoctask'
+require 'micronaut'
+require 'micronaut/rake_task'
+
+begin
+  require 'jeweler'
+  files = ["CHANGELOG", "MIT-LICENSE", "Rakefile", "README.rdoc", "VERSION.yml"]
+  files << Dir["examples/**/*", "laf/**/*", "lib/**/*", "tasks/**/*", "template/**/*"]
+  
+  Jeweler::Tasks.new do |s|
+    s.name = "panmind-tarantula"
+    s.summary = "A big hairy fuzzy spider that crawls your site, wreaking havoc"
+    s.description = "A big hairy fuzzy spider that crawls your site, wreaking havoc"
+    s.homepage = "http://github.com/Panmind/tarantula"
+    s.email = "opensource@thinkrelevance.com"
+    s.authors = ["Relevance, Inc."]
+    s.require_paths = ["lib"]
+    s.files = files.flatten
+    s.add_dependency 'htmlentities', '>= 4.2.0'
+    s.add_dependency 'hpricot', '>= 0.8.1'
+    s.add_development_dependency 'micronaut'
+    s.add_development_dependency 'log_buddy'
+  end
+  Jeweler::GemcutterTasks.new
+rescue LoadError
+  puts "Jeweler not available. Install it with: gem install jeweler"
+end
+
+desc "Run all micronaut examples"
+Micronaut::RakeTask.new :examples do |t|
+  t.pattern = "examples/**/*_example.rb"
+end
+
+desc "Run all micronaut examples using rcov"
+Micronaut::RakeTask.new :rcov do |t|
+  t.pattern = "examples/**/*_example.rb"
+  t.rcov = true
+  t.rcov_opts = %[--exclude "gems/*,/Library/Ruby/*,config/*" --text-summary  --sort coverage]
+end
+
+namespace :examples do
+  
+  RAILS_VERSIONS = %w[2.3.2 2.3.4]
+  
+  unless RUBY_VERSION =~ /^1\.9\./
+    RAILS_VERSIONS.unshift(*%w[2.0.2 2.1.0 2.1.1 2.2.2 2.3.3])
+    RAILS_VERSIONS.sort!
+  end
+  
+  desc "Run examples with multiple versions of rails"
+  task :multi_rails do
+    RAILS_VERSIONS.each do |rails_version|
+      puts
+      sh "RAILS_VERSION='#{rails_version}' rake examples"
+    end
+  end
+  
+end
+
+if ENV["RUN_CODE_RUN"]
+  task :default => [:check_dependencies, "examples:multi_rails"]
+else
+  task :default => [:check_dependencies, :examples]
+end
+
+begin
+  %w{sdoc sdoc-helpers rdiscount}.each { |name| gem name }
+  require 'sdoc_helpers'
+rescue LoadError => ex
+  puts "sdoc support not enabled:"
+  puts ex.inspect
+end
+
+require 'rake/rdoctask'
+Rake::RDocTask.new do |rdoc|
+  version = File.exist?('VERSION') ? File.read('VERSION') : ''
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title = "panmind-tarantula #{version}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:major: 0
+:minor: 3
+:patch: 3

data/examples/example_helper.rb

@@ -0,0 +1,57 @@
+require 'rubygems'
+lib_path = File.expand_path(File.dirname(__FILE__) + "/../lib")
+$LOAD_PATH.unshift lib_path unless $LOAD_PATH.include?(lib_path)
+require 'rubygems'
+gem "micronaut", ">= 0.2.4"
+gem "log_buddy"
+gem "mocha"
+if rails_version = ENV['RAILS_VERSION']
+  gem "rails", rails_version
+end
+require "rails/version"
+if Rails::VERSION::STRING < "2.3.1" && RUBY_VERSION >= "1.9.1"
+  puts "Tarantula requires Rails 2.3.1 or higher for Ruby 1.9 support"
+  exit(1)
+end
+puts "==== Testing with Rails #{Rails::VERSION::STRING} ===="
+gem 'actionpack'
+gem 'activerecord'
+gem 'activesupport'
+
+require 'ostruct'
+require 'active_support'
+require 'action_controller'
+require 'active_record'
+require 'relevance/tarantula'
+require 'micronaut'
+require 'mocha'
+
+def test_output_dir
+  File.join(File.dirname(__FILE__), "..", "tmp", "test_output")
+end
+
+# TODO change puts/print to use a single method for logging, which will then make the stubbing cleaner
+def stub_puts_and_print(obj)
+  obj.stubs(:puts)
+  obj.stubs(:print)
+end
+
+def make_link(link, crawler=Relevance::Tarantula::Crawler.new, referrer=nil)
+  Relevance::Tarantula::Link.new(link, crawler, referrer)
+end
+
+def make_form(form, crawler=Relevance::Tarantula::Crawler.new, referrer=nil)
+  Relevance::Tarantula::Form.new(form, crawler, referrer)
+end
+
+def not_in_editor?
+  ['TM_MODE', 'EMACS', 'VIM'].all? { |k| !ENV.has_key?(k) }
+end
+
+Micronaut.configure do |c|
+  c.alias_example_to :fit, :focused => true
+  c.alias_example_to :xit, :disabled => true
+  c.mock_with :mocha
+  c.color_enabled = not_in_editor?
+  c.filter_run :focused => true
+end

data/examples/relevance/core_extensions/ellipsize_example.rb

@@ -0,0 +1,19 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "../..", "example_helper.rb"))
+
+describe "Relevance::CoreExtensions::Object#ellipsize" do
+  it "converts nil to empty string" do
+    nil.ellipsize.should == ""
+  end
+  
+  it "doesn't touch short strings" do
+    "hello".ellipsize.should == "hello"
+  end
+  
+  it "calls inspect on non-strings" do
+    [1,2,3].ellipsize.should == "[1, 2, 3]"
+  end
+
+  it "shortens long strings and adds ..." do
+    "long-string".ellipsize(5).should == "long-..."
+  end
+end

data/examples/relevance/core_extensions/file_example.rb

@@ -0,0 +1,8 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "../..", "example_helper.rb"))
+require 'relevance/core_extensions/file'
+
+describe "Relevance::CoreExtensions::File#extension" do
+  it "should return the extension without the leading dot" do
+    File.extension("foo.bar").should == "bar"
+  end
+end

data/examples/relevance/core_extensions/response_example.rb

@@ -0,0 +1,29 @@
+require File.expand_path(File.join(File.dirname(__FILE__), "../..", "example_helper.rb"))
+require 'relevance/core_extensions/file'
+
+describe "Relevance::CoreExtensions::Response#html?" do
+  before do
+    @response = OpenStruct.new
+    @response.extend(Relevance::CoreExtensions::Response)
+  end         
+  
+  it "should be html if the content-type is 'text/html'" do
+    @response.content_type = "text/html"
+    @response.should be_html
+    @response.content_type = "text/html;charset=iso-8859-2"
+    @response.should be_html
+  end
+
+  it "should not be html if the content-type isn't an html type" do
+    @response.content_type = "text/plain"
+    @response.should_not be_html
+  end
+
+  # better ideas welcome, but be careful not to  
+  # castrate tarantula for proxies that don't set the content-type
+  it "should pretend we have html if the content-type is nil" do
+    @response.content_type = nil
+    @response.should be_html
+  end
+  
+end