mutant 0.3.0.beta22 → 0.3.0.rc1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 22c3af1fd28ad61a9e6968e66e5208272a9508cd
-  data.tar.gz: f8ac63722e87ff6c4899d3ba3e2d75ee6e76579c
+  metadata.gz: bd04beb38330a250653e83799b208a0eb92a8656
+  data.tar.gz: 4b6512fbb3f2e31fcbd90411a1ea1c0c9f10c261
 SHA512:
-  metadata.gz: 636bfb2b5e7bd64d40ef70427c739484cf1db6b1eb3cf17c02b5846cc566e882174ade582b04973034b0f795b6e01a56e169a910283af54cd1d0a7096e1d5d9e
-  data.tar.gz: 035c70cca872d8a52c9a012e0d18a4e9027495860659d9d2d93c783542fe8fe38b646eca9e489db7029ac751fecbfc01dc18e438d3c797b871be3c645e646ec3
+  metadata.gz: 8b11f4a25768e953373c1ce9a854051c9ffa28758a139d369568089a520eb90893585242dcc4586c276e112b886157083ac0165fcb0c83e94fcf768a97c613c8
+  data.tar.gz: 8d358b32f4c915ab85779f43106e607576250fa8f8badf1a94ff23cfc84845e919411bba73a6380b869ce7f8525eeca4174e38a313d864c81c3e636e31eb0ea7

data/.travis.yml

@@ -2,13 +2,12 @@ language: ruby
 before_install: gem install bundler
 bundler_args: --without yard guard benchmarks
 script: "bundle exec rake ci"
-rvm:
-  - 1.9.3
-  - 2.0.0
-  - ruby-head
-  - rbx-19mode
 matrix:
   include:
+    - rvm: 1.9.3
+    - rvm: 2.0.0
+    - rvm: ruby-head
+    - rvm: rbx-19mode
     - rvm: jruby-19mode
       env: JRUBY_OPTS="$JRUBY_OPTS --debug"
     - rvm: jruby-head

data/Gemfile.devtools

@@ -36,7 +36,7 @@ group :metrics do
   gem 'reek',      '~> 1.3.1',  git: 'https://github.com/troessner/reek.git'
   gem 'rubocop',   '~> 0.10.0', git: 'https://github.com/bbatsov/rubocop.git'
   gem 'simplecov', '~> 0.7.1'
-  gem 'yardstick', '~> 0.9.6'
+  gem 'yardstick', '~> 0.9.6', git: 'https://github.com/dkubb/yardstick.git'
 
   platforms :ruby_19, :ruby_20 do
     gem 'yard-spellcheck', '~> 0.1.5'

data/README.md

@@ -17,10 +17,7 @@ any ruby engine that supports POSIX-fork(2) semantics.
 
 Only rspec2 is supported currently. This is subject to change.
 
-It is easy to write a mutation killer for other test/spec frameworks than rspec2.
-Just create your own Mutant::Killer subclass, and make sure I get a PR!
-
-See this [ASCII-Cast](http://ascii.io/a/1707) for mutant in action! (v0.2.1)
+It is easy to write a mutation killer/strategy for other test/spec frameworks than rspec2.
 
 Projects using Mutant
 ---------------------
@@ -62,7 +59,7 @@ emits around 3-6 mutations.
 
 Currently mutant covers the majority of ruby's complex nodes that often occur in method bodies.
 
-A some stats from the [axiom](https://github.com/dkubb/axiom) library:
+Some stats from the [axiom](https://github.com/dkubb/axiom) library:
 
 ```
 Subjects:  417       # Amount of subjects being mutated (currently only methods)
@@ -84,63 +81,38 @@ the Generic handler altogether.
 Examples
 --------
 
-CLI will be simplified in the next releases, but currently stick with this:
-
 ```
 cd virtus
-# Run mutant on virtus namespace (that uses the dm-2 style spec layout)
-mutant --rspec-dm2 '::Virtus*'
+# Run mutant on virtus namespace
+mutant --rspec '::Virtus*'
 # Run mutant on specific virtus class
-mutant --rspec-dm2 ::Virtus::Attribute
+mutant --rspec ::Virtus::Attribute
 # Run mutant on specific virtus class method
-mutant --rspec-dm2 ::Virtus::Attribute.build
+mutant --rspec ::Virtus::Attribute.build
 # Run mutant on specific virtus instance method
-mutant --rspec-dm2 ::Virtus::Attribute#name
+mutant --rspec ::Virtus::Attribute#name
 ```
 
-Strategies
-----------
-
-Mutation testing is slow. The key to making it fast is selecting the correct set of tests to run.
-Mutant currently supports the following built-in strategies for selecting tests/specs.
-
-### --rspec-dm2
-
-This strategy is the *fastest* but requires discipline in spec file naming.
-
-The following specs are executed to kill a mutation on:
-```
-Public instance  methods: spec/unit/#{namespace}/#{class_name}/#{method_name}_spec.rb
-Public singleton methods: spec/unit/#{namespace}/#{class_name}/class_methods/#{method_name}_spec.rb
-Private instance  methods: spec/unit/#{namespace}/#{class_name}/*_spec.rb
-Private singleton methods: spec/unit/#{namespace}/#{class_name}/class_methods/*_spec.rb
-```
-
-#### Expansions:
-
-Symbolic operator-like methods are expanded, e.g. ```Foo#<<``` is expanded to:
-```
-spec/unit/foo/left_shift_operator_spec.rb
-````
-
-The full list of expansions can be found here:
-
-https://github.com/mbj/mutant/blob/master/lib/mutant/constants.rb
-
-### --rspec-unit
+Subjects:
+---------
 
-This strategy executes all specs under ``./spec/unit`` for each mutation.
+Mutant currently mutates code in instance and singleton methods. It is planned to support mutation
+of constant definitions and domain specific languages, DSL probably as plugins.
 
-### --rspec-integration
+Test-Selection
+--------------
 
-This strategy executes all specs under ``./spec/integration`` for each mutation.
+Mutation testing is slow. The key to making it fast is selecting the correct set of tests to run.
+Mutant currently supports the following built-in strategy for selecting tests/specs:
 
-### --rspec-full
+Mutant uses the "longest rspec example group descriptions prefix match" to select the tests to run.
 
-This strategy executes all specs under ``./spec`` for each mutation.
+Example for a subject like `Foo::Bar#baz` it will run all example groups with description prefixes in
+`Foo::Bar#baz`, `Foo::Bar` and `Foo`. The order is important, so if mutant finds example groups in the
+current prefix level, these example groups *must* kill the mutation.
 
-In the future, we plan on allowing explicit selections on the specs to be run, as well as support for other test frameworks.
-Custom project specific strategies are also on the roadmap.
+This test selection strategy is compatible with the old `--rspec-dm2` and `--rspec-unit` strategy.
+The old flags where removed.  It allows to define very fine grained specs, or coarse grained - as you like.
 
 Alternatives
 ------------
@@ -156,7 +128,7 @@ Your options:
 
 * GitHub Issues https://github.com/mbj/mutant/issues
 * Ping me on https://twitter.com/_m_b_j_
-* #mutant channel on freenode, I hang around on CET daytimes. (nick mbj) 
+* #mutant channel on freenode, I hang around on CET daytimes. (nick mbj)
   You'll also find others [ROM](https://github.com/rom-rb) team members here that can answer questions.
 
 Credits

data/config/flay.yml

@@ -1,3 +1,3 @@
 ---
 threshold: 16
-total_score: 764
+total_score: 738

data/config/flog.yml

@@ -1,2 +1,2 @@
 ---
-threshold: 20.5
+threshold: 20.0

data/config/reek.yml

@@ -25,11 +25,13 @@ FeatureEnvy:
   enabled: true
   exclude:
   - Mutant::CLI#parse
+  - Mutant::CLI#add_strategies
   - Mutant::Matcher::Method::Instance#match?
   - Mutant::Matcher::Method::Singleton#receiver?
   - Mutant::Mutation::Evil#success?
   - Mutant::Mutation::Neutral#success?
   - Mutant::Reporter::CLI#subject_results
+  - Mutant::Strategy::Rspec#options
 IrresponsibleModule:
   enabled: true
   exclude: []
@@ -78,6 +80,8 @@ TooManyStatements:
   exclude:
   - Mutant#self.singleton_subclass_instance
   - Mutant::CLI#parse
+  - Mutant::CLI#add_options
+  - Mutant::CLI#add_strategies
   - Mutant::Killer::Rspec#run
   - Mutant::Reporter::CLI#colorized_diff
   - Mutant::Reporter::CLI::Printer::Config::Runner#run
@@ -96,8 +100,7 @@ UncommunicativeMethodName:
   accept: []
 UncommunicativeModuleName:
   enabled: true
-  exclude:
-  - Mutant::Strategy::Rspec::DM2
+  exclude: []
   reject:
   - !ruby/regexp /^.$/
   - !ruby/regexp /[0-9]$/
@@ -128,4 +131,8 @@ UtilityFunction:
   - Mutant::Mutation::Evil#success?
   - Mutant::Mutation::Neutral#success?
   - Mutant::NodeHelpers#s
+  - Mutant::Strategy::Rspec#configuration
+  - Mutant::Strategy::Rspec#options
+  - Mutant::Strategy::Rspec#world
+
   max_helper_calls: 0

data/lib/mutant.rb

@@ -19,6 +19,7 @@ require 'diff/lcs/hunk'
 require 'rspec'
 require 'anima'
 require 'concord'
+require 'rspec'
 
 # Library namespace
 module Mutant
@@ -26,6 +27,7 @@ module Mutant
   EMPTY_STRING = ''.freeze
 end # Mutant
 
+require 'mutant/version'
 require 'mutant/cache'
 require 'mutant/node_helpers'
 require 'mutant/singleton_methods'
@@ -73,6 +75,7 @@ require 'mutant/mutator/node/send/binary'
 require 'mutant/mutator/node/when'
 require 'mutant/mutator/node/define'
 require 'mutant/mutator/node/mlhs'
+require 'mutant/mutator/node/nthref'
 require 'mutant/mutator/node/masgn'
 require 'mutant/mutator/node/return'
 require 'mutant/mutator/node/block'
@@ -103,11 +106,7 @@ require 'mutant/killer/forking'
 require 'mutant/killer/forked'
 require 'mutant/strategy'
 require 'mutant/strategy/static'
-require 'mutant/strategy/method_expansion'
 require 'mutant/strategy/rspec'
-require 'mutant/strategy/rspec/dm2'
-require 'mutant/strategy/rspec/dm2/lookup'
-require 'mutant/strategy/rspec/dm2/lookup/method'
 require 'mutant/runner'
 require 'mutant/runner/config'
 require 'mutant/runner/subject'

data/lib/mutant/cli.rb

@@ -59,6 +59,7 @@ module Mutant
     def config
       Config.new(
         :cache     => @cache,
+        :zombie    => @zombie,
         :debug     => debug?,
         :matcher   => matcher,
         :filter    => filter,
@@ -154,26 +155,6 @@ module Mutant
       @filters << klass.new(filter)
     end
 
-    # Set fail fast
-    #
-    # @api private
-    #
-    # @return [undefined]
-    #
-    def set_fail_fast
-      @fail_fast = true
-    end
-
-    # Set debug mode
-    #
-    # @api private
-    #
-    # @return [undefined]
-    #
-    def set_debug
-      @debug = true
-    end
-
     # Set strategy
     #
     # @param [Strategy] strategy
@@ -204,9 +185,8 @@ module Mutant
         builder.separator ''
         builder.separator 'Strategies:'
 
-        builder.on('--zombie', 'Run mutant zombified')
-
         add_strategies(builder)
+        add_environmental_options(builder)
         add_options(builder)
       end
 
@@ -237,25 +217,48 @@ module Mutant
 
     # Add strategies
     #
-    # @param [Object]
+    # @param [Object] opts
     #
     # @return [undefined]
     #
     # @api private
     #
     def add_strategies(opts)
-      opts.on('--rspec-unit', 'executes all specs under ./spec/unit') do
-        set_strategy Strategy::Rspec::Unit
-      end.on('--rspec-full', 'executes all specs under ./spec') do
-        set_strategy Strategy::Rspec::Full
-      end.on('--rspec-dm2', 'executes spec/unit/$nesting/$method_spec.rb') do
-        set_strategy Strategy::Rspec::DM2
+      opts.separator ''
+      opts.separator 'Strategies:'
+
+      opts.on('--static-success', 'does succeed on all mutations') do
+        set_strategy Strategy::Static::Success.new
+      end
+      opts.on('--static-fail', 'does fail on all mutations') do
+        set_strategy Strategy::Static::Fail.new
+      end
+      opts.on('--rspec', 'kills mutations with rspec') do
+        set_strategy Strategy::Rspec.new
+      end
+    end
+
+    # Add environmental options
+    #
+    # @param [Object] opts
+    #
+    # @return [undefined]
+    #
+    # @api private
+    #
+    def add_environmental_options(opts)
+      opts.on('--zombie', 'Run mutant zombified') do
+        @zombie = true
+      end.on('-I', 'Add directory to $LOAD_PATH') do |directory|
+        $LOAD_PATH << directory
+      end.on('-r', '--require NAME', 'Require file with NAME') do |name|
+        require name
       end
     end
 
     # Add options
     #
-    # @param [Object]
+    # @param [Object] opts
     #
     # @return [undefined]
     #
@@ -265,14 +268,17 @@ module Mutant
       opts.separator ''
       opts.separator 'Options:'
 
-      opts.on('--code FILTER', 'Adds a code filter') do |filter|
-        add_filter Mutation::Filter::Code, filter
+      opts.on('--version', 'Print mutants version') do |name|
+        puts("mutant-#{Mutant::VERSION}")
+        Kernel.exit(0)
+      end.on('--code FILTER', 'Adds a code filter') do |filter|
+        add_filter(Mutation::Filter::Code, filter)
       end.on('--fail-fast', 'Fail fast') do
-        set_fail_fast
+        @fail_fast = true
       end.on('-d', '--debug', 'Enable debugging output') do
-        set_debug
+        @debug = true
       end.on_tail('-h', '--help', 'Show this message') do
-        puts opts
+        puts(opts)
         exit
       end
     end

data/lib/mutant/cli/classifier.rb

@@ -49,7 +49,7 @@ module Mutant
           .split(SCOPE_OPERATOR)
           .reduce(Object) do |parent, name|
             parent.const_get(name, nil)
-          end
+        end
       end
 
       # Return matchers for input

data/lib/mutant/config.rb

@@ -4,7 +4,8 @@ module Mutant
   # The configuration of a mutator run
   class Config
     include Adamantium::Flat, Anima.new(
-      :cache, :debug, :strategy, :matcher, :filter, :reporter, :fail_fast
+      :cache, :debug, :strategy, :matcher, :filter,
+      :reporter, :fail_fast, :zombie
     )
 
     # Enumerate subjects

data/lib/mutant/context/scope.rb

@@ -23,7 +23,7 @@ module Mutant
       #
       # @return [String]
       #
-      # @ai private
+      # @api private
       #
       def identification
         scope.name
@@ -89,6 +89,19 @@ module Mutant
         scope.name
       end
 
+      # Return match prefixes
+      #
+      # @return [Enumerable<String>]
+      #
+      # @api private
+      #
+      def match_prefixes
+        name_nesting.length.downto(1).map do |last|
+          name_nesting[0...last].join('::')
+        end
+      end
+      memoize :match_prefixes
+
       # Return scope wrapped by context
       #
       # @return [::Module|::Class]

data/lib/mutant/killer.rb

@@ -98,6 +98,16 @@ module Mutant
       @runtime = times.real
     end
 
+    # Return subject
+    #
+    # @return [Subject]
+    #
+    # @api private
+    #
+    def subject
+      mutation.subject
+    end
+
     # Run killer
     #
     # @return [true]

data/lib/mutant/killer/rspec.rb

@@ -19,36 +19,70 @@ module Mutant
       #
       def run
         mutation.insert
-        # TODO: replace with real streams from configuration
-        require 'stringio'
-        # Note: We assume interesting output from a failed rspec run is stderr.
-        rspec_err = StringIO.new
 
-        exit_code = ::RSpec::Core::Runner.run(cli_arguments, nil, rspec_err)
+        groups = example_groups
 
-        killed = !exit_code.zero?
+        unless groups
+          $stderr.puts "No rspec example groups found for: #{match_prefixes.join(', ')}"
+          return false
+        end
 
-        if killed and mutation.should_survive?
-          rspec_err.rewind
+        reporter = RSpec::Core::Reporter.new
 
-          puts "#{mutation.class} test failed."
-          puts 'RSpec stderr:'
-          puts rspec_err.read
+        example_groups.each do |group|
+          return true unless group.run(reporter)
         end
 
-        killed
+        false
+      end
+
+      # Return match prefixes
+      #
+      # @return [Enumerble<String>]
+      #
+      # @api private
+      #
+      def match_prefixes
+        subject.match_prefixes
+      end
+
+      # Return example groups
+      #
+      # @return [Array<RSpec::Example>]
+      #
+      # @api private
+      #
+      def example_groups
+        match_prefixes.each do |match_expression|
+          example_groups = find_with(match_expression)
+          return example_groups unless example_groups.empty?
+        end
+
+        nil
+      end
+
+      # Return example groups that match expression
+      #
+      # @param [String] match_expression
+      #
+      # @return [Enumerable<String>]
+      #
+      # @api private
+      #
+      def find_with(match_expression)
+        all_example_groups.select do |example_group|
+          example_group.description.start_with?(match_expression)
+        end
       end
 
-      # Return command line arguments
+      # Return all example groups
       #
-      # @return [Array]
+      # @return [Enumerable<RSpec::Example>]
       #
       # @api private
       #
-      def cli_arguments
-        %W(
-          --fail-fast
-        ) + strategy.spec_files(mutation.subject)
+      def all_example_groups
+        strategy.example_groups
       end
 
     end # Rspec