chaser 0.0.1

data/History.txt

@@ -0,0 +1,109 @@
+History for Chaser:
+
+* Coming soon
+
+History for Heckle:
+
+=== 1.4.3 / 2009-06-23
+
+* 2 minor enhancements:
+
+  * Added autotest/heckle plugin
+  * Skipping testing on 1.9
+
+=== 1.4.2 / 2009-02-08
+
+* 2 bug fixes:
+
+  * Fixed Ruby2Ruby dependency and Ruby2Ruby references (name changed).
+    Reported by David Chelimsky
+  * Fix bug #11435 where [:iter, [:call], ...] would cause an endless
+    loop.  Reported by Thomas Preymesser.
+
+=== 1.4.1 / 2007-06-05
+
+* 3 bug fixes:
+
+  * Add zentest as a heckle dependency. Closes #10996
+  * Fixed heckling of call with blocks.
+  * Fix test_unit_heckler's test_pass? so it returns the result of the
+    run rather than ARGV.clear
+
+=== 1.4.0 / 2007-05-18
+
+* 2 major enhancements:
+
+  * Method calls are now heckled (by removal).
+  * Assignments are now heckled (by value changing).
+
+* 3 minor enhancements:
+
+  * Added --focus to feel the Eye of Sauron (specify unit tests to run).
+  * Specify nodes to be included/excluded in heckle with -n/-x.
+  * Test only assignments with --assignments
+
+=== 1.3.0 / 2007-02-12
+
+* 1 major enhancement:
+
+  * Unified diffs for mutatated methods
+
+* 4 minor enhancements:
+
+  * Now returns exit status 1 if failed.
+  * Added a simple report at the end.
+  * Runs are now sorted by method.
+  * Autodetects rails and changes test_pattern accordingly.
+
+* 2 bug fixes:
+
+  * Aborts when an unknown method is supplied.
+  * Escapes slashes in random regexps.
+
+=== 1.2.0 / 2007-01-15
+
+* 2 major enhancements:
+
+  * Timeout for tests set dynamically and overridable with -T
+  * Class method support with "self.method_name"
+
+* 3 minor enhancements:
+
+  * -b allows heckling of branches only
+  * Restructured class heirarchy and got rid of Base and others.
+  * Revamped the tests and reduced size by 60%.
+
+* 1 bug fix:
+
+  * Fixed the infinite loop caused by syntax errors
+
+=== 1.1.1 / 2006-12-20
+
+* 3 bug fixes:
+
+  * Load tests properly when supplying method name.
+  * Make sure random symbols have at least one character.
+  * Removed all extra warnings from the unit tests. Consolidated and cleaned.
+
+=== 1.1.0 / 2006-12-19
+
+* 12 major enhancements:
+
+  * Able to roll back original method after processing.
+  * Can mutate numeric literals.
+  * Can mutate strings.
+  * Can mutate a node at a time.
+  * Can mutate if/unless
+  * Decoupled from Test::Unit
+  * Cleaner output
+  * Can mutate true and false.
+  * Can mutate while and until.
+  * Can mutate regexes, ranges, symbols
+  * Can run against entire classes
+  * Command line options!
+
+=== 1.0.0 / 2006-10-22
+
+* 1 major enhancement
+
+  * Birthday!

data/Manifest.txt

@@ -0,0 +1,10 @@
+chaser.gemspec
+History.txt
+Manifest.txt
+README.txt
+Rakefile
+bin/chaser
+lib/chaser.rb
+lib/test_unit_chaser.rb
+test/fixtures/chased.rb
+test/test_chaser.rb

data/README.txt

@@ -0,0 +1,47 @@
+= chaser
+
+* Web site coming real soon now
+
+== DESCRIPTION:
+
+Chaser is unit test sadism(tm), like Seattlerb's Heckle. It's more or less mutation testing, except that rather than mutating every line of code it can get its claws into, it merely modifies the return value of targeted methods. If the unit tests don't notice the modified return values, or the program going haywire as a result of using those modified return values, then they aren't doing their jobs properly.
+
+Unit test sadism is a trademark of Ryan Davis and Kevin Clark, and is used without permission.
+
+== FEATURES/PROBLEMS:
+
+* It only mutates the return values of methods.
+* It should be able to mutate class methods, and work in ruby 1.9 or Windows.
+
+== REQUIREMENTS:
+
+* None!
+
+== INSTALL:
+
+* sudo gem install agrimm-chaser
+
+== LICENSE:
+
+(The MIT License)
+
+Copyright (c) 2006-2009 Ryan Davis and Kevin Clark and Andrew Grimm
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,22 @@
+task :default => [:test]
+
+begin
+  require 'jeweler'
+  require File.dirname(__FILE__) + "/lib/chaser.rb"
+  Jeweler::Tasks.new do |gemspec|
+    gemspec.name = "chaser"
+    gemspec.summary = "Unit test sadism, with less masochism"
+    gemspec.description = "Lightweight mutation testing in any flavor of ruby"
+    gemspec.email = "andrew.j.grimm@gmail.com"
+    gemspec.authors = ["Andrew Grimm", "Ryan Davis", "Eric Hodel", "Kevin Clark"]
+    gemspec.add_dependency('test-unit') #Ruby 1.9 doesn't have full test-unit in the standard library.
+    gemspec.version = Chaser::VERSION
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end
+
+task :test do
+  ruby "test/test_chaser.rb"
+end
+

data/bin/chaser

@@ -0,0 +1,55 @@
+#!/usr/local/bin/ruby
+
+$:.unshift File.join(File.dirname(__FILE__), *%w[.. lib])
+require 'test_unit_chaser'
+require 'optparse'
+
+force = false
+
+opts = OptionParser.new do |opts|
+  opts.banner = "Usage: #{File.basename($0)} class_name [method_name]"
+  opts.on("-v", "--verbose", "Loudly explain chaser run") do |opt|
+    TestUnitChaser.debug = true
+  end
+
+  opts.on("-V", "--version", "Prints Chaser's version number") do |opt|
+    puts "Chaser #{Chaser::VERSION}"
+    exit 0
+  end
+
+  opts.on("-t", "--tests TEST_PATTERN",
+          "Location of tests (glob)") do |pattern|
+    TestUnitChaser.test_pattern = pattern
+  end
+
+  opts.on("-F", "--force", "Ignore initial test failures") do |opt|
+    force = true
+  end
+
+  opts.on("-T", "--timeout SECONDS", "The maximum time for a test run in seconds",
+                                     "Used to catch infinite loops") do |timeout|
+    Chaser.timeout = timeout.to_i
+    puts "Setting timeout at #{timeout} seconds."
+  end
+
+  opts.on("-h", "--help", "Show this message") do |opt|
+    puts opts
+    exit 0
+  end
+end
+
+looks_like_rails = test ?f, 'config/environment.rb'
+TestUnitChaser.test_pattern = "test/**/*.rb" if looks_like_rails
+
+opts.parse!
+
+impl = ARGV.shift
+meth = ARGV.shift
+
+unless impl then
+  puts opts
+  exit 1
+end
+
+exit TestUnitChaser.validate(impl, meth, force)
+

data/lib/chaser.rb

@@ -0,0 +1,329 @@
+require 'rubygems'
+require 'timeout'
+
+class String # :nodoc:
+  def to_class
+    split(/::/).inject(Object) { |klass, name| klass.const_get(name) }
+  end
+end
+
+##
+# Test Unit Sadism
+
+class Chaser
+
+  class Timeout < Timeout::Error; end
+
+  ##
+  # The version of Chaser you are using.
+
+  VERSION = '0.0.1'
+
+  ##
+  # Is this platform MS Windows-like?
+
+  WINDOZE = RUBY_PLATFORM =~ /mswin/
+
+  ##
+  # Path to the bit bucket.
+
+  NULL_PATH = WINDOZE ? 'NUL:' : '/dev/null'
+
+  ##
+  # Class being chased
+
+  attr_accessor :klass
+
+  ##
+  # Name of class being chased
+
+  attr_accessor :klass_name
+
+  ##
+  # Method being chased
+
+  attr_accessor :method
+
+  ##
+  # Name of method being chased
+
+  attr_accessor :method_name
+
+  ##
+  # The original version of the method being chased
+
+  attr_reader :old_method
+
+  @@debug = false
+  @@guess_timeout = true
+  @@timeout = 60 # default to something longer (can be overridden by runners)
+
+  def self.debug
+    @@debug
+  end
+
+  def self.debug=(value)
+    @@debug = value
+  end
+
+  def self.timeout=(value)
+    @@timeout = value
+    @@guess_timeout = false # We've set the timeout, don't guess
+  end
+
+  def self.guess_timeout?
+    @@guess_timeout
+  end
+
+  ##
+  # Creates a new Chaser that will chase +klass_name+ and +method_name+,
+  # sending results to +reporter+.
+
+  def initialize(klass_name = nil, method_name = nil, reporter = Reporter.new)
+    @klass_name = klass_name
+    @method_name = method_name.intern if method_name
+
+    @klass = klass_name.to_class
+
+    @method = nil
+    @reporter = reporter
+
+    @mutated = false
+
+    @failure = false
+  end
+
+  ##
+  # Overwrite test_pass? for your own Chaser runner.
+
+  def tests_pass?
+    raise NotImplementedError
+  end
+
+  def run_tests
+    if tests_pass? then
+      record_passing_mutation
+    else
+      @reporter.report_test_failures
+    end
+  end
+
+  ############################################################
+  ### Running the script
+
+  def validate
+    @reporter.method_loaded(klass_name, method_name)
+
+    begin
+      modify_method
+      timeout(@@timeout, Chaser::Timeout) { run_tests }
+    rescue Chaser::Timeout
+      @reporter.warning "Your tests timed out. Chaser may have caused an infinite loop."
+    rescue Interrupt
+      @reporter.warning 'Mutation canceled, hit ^C again to exit'
+      sleep 2
+    end
+
+    unmodify_method # in case we're validating again. we should clean up.
+
+    if @failure
+      @reporter.report_failure
+      false
+    else
+      @reporter.no_surviving_mutant
+      true
+    end
+  end
+
+  def record_passing_mutation
+    @failure = true
+  end
+
+  def unmodify_instance_method
+    chaser = self
+    @mutated = false
+    @klass.send(:define_method, @method_name) do |*args|
+      chaser.old_method.bind(self).call(*args)
+    end
+  end
+
+  def unmodify_class_method
+    chaser = self
+    @mutated = false
+    aliasing_class(@method_name).send(:define_method, clean_method_name) do |*args|
+      chaser.old_method.bind(self).call(*args)
+    end
+  end
+
+  def modify_instance_method
+    chaser = self
+    @mutated = true
+    @old_method = @klass.instance_method(@method_name)
+    @klass.send(:define_method, @method_name) do |*args|
+      chaser.mutate_value(chaser.old_method.bind(self).call(*args))
+    end
+  end
+
+  def modify_class_method
+    chaser = self
+    @mutated = true
+    @old_method = aliasing_class(@method_name).instance_method(clean_method_name)
+    aliasing_class(@method_name).send(:define_method, clean_method_name) do |*args|
+      chaser.mutate_value(chaser.old_method.bind(self).call(*args))
+    end
+  end
+
+  def modify_method
+    if method_name.to_s =~ /self\./
+      modify_class_method
+    else
+      modify_instance_method
+    end
+  end
+
+  def unmodify_method
+    if method_name.to_s =~ /self\./ #TODO fix duplication. Give the test a name
+      unmodify_class_method
+    else
+      unmodify_instance_method
+    end
+  end
+
+
+  ##
+  # Replaces the value with a random value.
+
+  def mutate_value(value)
+    case value
+    when Fixnum, Float, Bignum
+      value + rand_number
+    when String
+      rand_string
+    when Symbol
+      rand_symbol
+    when Regexp
+      Regexp.new(Regexp.escape(rand_string.gsub(/\//, '\\/')))
+    when Range
+      rand_range
+    when NilClass, FalseClass
+      rand_number
+    when TrueClass
+      false
+    else
+      nil
+    end
+  end
+
+  ############################################################
+  ### Convenience methods
+
+  def aliasing_class(method_name)
+    method_name.to_s =~ /self\./ ? class << @klass; self; end : @klass
+  end
+
+  def clean_method_name
+    method_name.to_s.gsub(/self\./, '')
+  end
+
+  ##
+  # Returns a random Fixnum.
+
+  def rand_number
+    (rand(100) + 1)*((-1)**rand(2))
+  end
+
+  ##
+  # Returns a random String
+
+  def rand_string
+    size = rand(50)
+    str = ""
+    size.times { str << rand(126).chr }
+    str
+  end
+
+  ##
+  # Returns a random Symbol
+
+  def rand_symbol
+    letters = ('a'..'z').to_a + ('A'..'Z').to_a
+    str = ""
+    (rand(50) + 1).times { str << letters[rand(letters.size)] }
+    :"#{str}"
+  end
+
+  ##
+  # Returns a random Range
+
+  def rand_range
+    min = rand(50)
+    max = min + rand(50)
+    min..max
+  end
+
+  ##
+  # Suppresses output on $stdout and $stderr.
+
+  def silence_stream
+    return yield if @@debug
+
+    begin
+      dead = File.open(Chaser::NULL_PATH, "w")
+
+      $stdout.flush
+      $stderr.flush
+
+      oldstdout = $stdout.dup
+      oldstderr = $stderr.dup
+
+      $stdout.reopen(dead)
+      $stderr.reopen(dead)
+
+      result = yield
+
+    ensure
+      $stdout.flush
+      $stderr.flush
+
+      $stdout.reopen(oldstdout)
+      $stderr.reopen(oldstderr)
+      result
+    end
+  end
+
+  class Reporter
+    def method_loaded(klass_name, method_name)
+      info "#{klass_name}\##{method_name} loaded"
+    end
+
+    def warning(message)
+      puts "!" * 70
+      puts "!!! #{message}"
+      puts "!" * 70
+      puts
+    end
+
+    def info(message)
+      puts "*"*70
+      puts "***  #{message}"
+      puts "*"*70
+      puts
+    end
+
+    def report_failure
+      puts
+      puts "The affected method didn't cause test failures."
+      puts
+    end
+
+    def no_surviving_mutant
+      puts "The mutant didn't survive. Cool!\n\n"
+    end
+
+    def report_test_failures
+      puts "Tests failed -- this is good" if Chaser.debug
+    end
+  end
+
+end
+

data/lib/test_unit_chaser.rb

@@ -0,0 +1,105 @@
+#!/usr/bin/env ruby
+
+require 'test/unit/autorunner'
+require 'test/unit/testcase'
+require 'chaser'
+
+$: << 'lib' << 'test'
+
+# Make sure test/unit doesn't swallow our timeout
+begin
+  Test::Unit::TestCase::PASSTHROUGH_EXCEPTIONS << Chaser::Timeout
+rescue NameError
+  # ignore
+end
+
+class TestUnitChaser < Chaser
+
+  @@test_pattern = 'test/test_*.rb'
+  @@tests_loaded = false
+
+  def self.test_pattern=(value)
+    @@test_pattern = value
+  end
+
+  def self.load_test_files
+    @@tests_loaded = true
+    Dir.glob(@@test_pattern).each {|test| require test}
+  end
+
+  def self.validate(klass_name, method_name = nil, force = false)
+    load_test_files
+    klass = klass_name.to_class
+
+    # Does the method exist?
+    klass_methods = klass.singleton_methods(false).collect {|meth| "self.#{meth}"}
+    if method_name
+      if method_name =~ /self\./
+        abort "Unknown method: #{klass_name}.#{method_name.gsub('self.', '')}" unless klass_methods.include? method_name
+      else
+        abort "Unknown method: #{klass_name}##{method_name}" unless klass.instance_methods(false).include? method_name
+      end
+    end
+
+    initial_time = Time.now
+
+    chaser = self.new(klass_name)
+
+    passed = chaser.tests_pass?
+
+    unless force or passed then
+      abort "Initial run of tests failed... fix and run chaser again"
+    end
+
+    if self.guess_timeout? then
+      running_time = Time.now - initial_time
+      adjusted_timeout = (running_time * 2 < 5) ? 5 : (running_time * 2).ceil
+      self.timeout = adjusted_timeout
+    end
+
+    puts "Timeout set to #{adjusted_timeout} seconds."
+
+    if passed then
+      puts "Initial tests pass. Let's rumble."
+    else
+      puts "Initial tests failed but you forced things. Let's rumble."
+    end
+    puts
+
+    methods = method_name ? Array(method_name) : klass.instance_methods(false) + klass_methods
+
+    counts = Hash.new(0)
+    methods.sort.each do |method_name|
+      result = self.new(klass_name, method_name).validate
+      counts[result] += 1
+    end
+    all_good = counts[false] == 0
+
+    puts "Chaser Results:"
+    puts
+    puts "Passed    : %3d" % counts[true]
+    puts "Failed    : %3d" % counts[false]
+    puts
+
+    if all_good then
+      puts "All chasing was thwarted! YAY!!!"
+    else
+      puts "Improve the tests and try again."
+    end
+
+    all_good
+  end
+
+  def initialize(klass_name=nil, method_name=nil)
+    super
+    self.class.load_test_files unless @@tests_loaded
+  end
+
+  def tests_pass?
+    silence_stream do
+      result = Test::Unit::AutoRunner.run
+      ARGV.clear
+      result
+    end
+  end
+end

data/test/fixtures/chased.rb

@@ -0,0 +1,13 @@
+class Chased
+  def add(a,b)
+    a + b
+  end
+
+  def say_hello
+    "G'day!"
+  end
+
+  def self.static_method
+    "Zap!"
+  end
+end

data/test/test_chaser.rb

@@ -0,0 +1,76 @@
+$:.unshift(File.dirname(__FILE__) + '/fixtures')
+$:.unshift(File.dirname(__FILE__) + '/../lib')
+
+require 'test/unit/testcase'
+require 'test/unit' if $0 == __FILE__
+require 'test_unit_chaser'
+require 'chased'
+
+class TestChaser < Chaser
+  def rand(*args)
+    5
+  end
+
+  def rand_string
+    "l33t h4x0r"
+  end
+
+  def rand_number(*args)
+    5
+  end
+
+  def rand_symbol
+    :"l33t h4x0r"
+  end
+end
+
+class ChaserTestCase < Test::Unit::TestCase
+  unless defined? Mini then
+    undef_method :default_test
+    alias :refute_equal :assert_not_equal
+  end
+
+  def setup
+  end
+
+  def teardown
+    @chaser.unmodify_method if defined?(@chaser) && @chaser
+  end
+
+  def test_unmodified_behaves_as_expected
+    chased = Chased.new
+    assert_equal 5, chased.add(2,3), "Unmodified version should equal 5"
+  end
+
+  def test_modify_and_unmodify_instance_method
+    @chaser = TestChaser.new("Chased", "add")
+    chased = Chased.new
+    assert_equal 5, chased.add(2,3), "method has been modified before it should have been"
+    @chaser.modify_method
+    assert_equal 10, chased.add(2,3), "method hasn't been modified"
+    @chaser.unmodify_method
+    assert_equal 5, chased.add(2,3), "method should be back to normal, but it isn't"
+  end
+
+  def test_modify_and_unmodify_string
+    @chaser = TestChaser.new("Chased", "say_hello")
+    chased = Chased.new
+    assert_equal "G'day!", chased.say_hello, "method has been modified before it should have been"
+    @chaser.modify_method
+    assert_equal "l33t h4x0r", chased.say_hello, "method hasn't been modified"
+    @chaser.unmodify_method
+    assert_equal "G'day!", chased.say_hello, "method should be back to normal, but it isn't"
+  end
+
+  def test_modify_and_unmodify_class_method
+    @chaser = TestChaser.new("Chased", "self.static_method")
+    assert_equal "Zap!", Chased.static_method, "class method has been modified before it should have been"
+    @chaser.modify_method
+    assert_equal "l33t h4x0r", Chased.static_method, "class method hasn't been modified"
+    @chaser.unmodify_method
+    assert_equal "Zap!", Chased.static_method, "class method should be back to normal, but it isn't"
+  end
+
+end
+
+

metadata

@@ -0,0 +1,67 @@
+--- !ruby/object:Gem::Specification 
+name: chaser
+version: !ruby/object:Gem::Version 
+  version: 0.0.1
+platform: ruby
+authors: 
+- Andrew Grimm
+- Ryan Davis
+- Eric Hodel
+- Kevin Clark
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2009-11-07 00:00:00 +11:00
+default_executable: chaser
+dependencies: []
+
+description: Lightweight mutation testing in any flavor of ruby
+email: andrew.j.grimm@gmail.com
+executables: 
+- chaser
+extensions: []
+
+extra_rdoc_files: 
+- README.txt
+files: 
+- History.txt
+- Manifest.txt
+- README.txt
+- Rakefile
+- bin/chaser
+- lib/chaser.rb
+- lib/test_unit_chaser.rb
+- test/fixtures/chased.rb
+- test/test_chaser.rb
+has_rdoc: true
+homepage: 
+licenses: []
+
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+  version: 
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+  version: 
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.5
+signing_key: 
+specification_version: 3
+summary: Unit test sadism, with less masochism
+test_files: 
+- test/fixtures/chased.rb
+- test/test_chaser.rb