mutant 0.9.10 → 0.10.0

data/config/yardstick.yml

@@ -1,2 +0,0 @@
----
-threshold: 100

data/docs/commercial-support.md

@@ -1,14 +0,0 @@
-# Commercial Support
-
-Mutant offers only community support.
-
-Commercial license customers are entited to priority support via email.
-
-## Priority Support
-
-Covers 1 incident per quarter, with a max response time of 7 days.
-Scope is limited to mutant not the application or infrastructure.
-
-For support email [Markus Schirp](mailto:mbj@schirp-dso.com?subject=Mutant%20Support).
-Please email using the same domain as the roiginal license email or explain
-your connection to the license.

data/docs/concurrency.md

@@ -1,39 +0,0 @@
-Concurrency
-===========
-
-By default, mutant will test mutations in parallel by running up
-to one process for each core on your system. You can control the
-number of processes created using the `-j/--jobs` argument.
-
-Mutant forks a new process for each mutation to be tested to prevent side
-affects in your specs and the lack of thread safety in integrations from
-impacting the results.
-
-Database
---------
-
-If the code under test relies on a database, you may experience problems
-when running mutant because of conflicting data in the database. For
-example, if you have a test like this:
-
-```ruby
-resource = MyModel.create!(...)
-expect(MyModel.first.name).to eql(resource.name)
-```
-
-It might fail if some other test wrote a record to the `MyModel` table
-at the same time as this test was executed. (It would find the MyModel
-record created by the other test.) Most of these issues can be fixed
-by writing more specific tests. Here is a concurrent safe version of
-the same test:
-
-```
-resource = MyModel.create!(...)
-expect(MyModel.find_by_id(m.id).name).to eql(resource.name)
-```
-
-An alternative is to try wrapping each test into an enclosing transaction.
-
-Note that some databases, SQLite in particular, are not designed for
-concurrent access and will fail if used in this manner. If you are
-using SQLite, you should set the `--jobs` to 1.

data/docs/incremental.md

@@ -1,76 +0,0 @@
-# Incremental mutation testing
-
-Incremental mutation testing allows you to significantly speed up mutation
-analysis, via skipping over subjects that are not changed.
-
-This is an explicit trade off, often worth the downsides.
-
-## Motivation
-
-Despite all efforts in setting up a project for maximum mutation testing
-efficiency, running a full pass over all available subjects is not always
-desired. On building your branches on CI, and during local commit stage, its
-instead more efficient to scope the Mutant run to everything that was
-touched in the current branch / commit. The working set of subjects.
-
-For not having to manually keep track of the working set, the `--since` option
-allows you to specify a git reference point. Mutant will automatically subset all
-available subjects to the ones that where touched since the reference point.
-
-## Usage
-
-Use the `--since git-reference` flag to enable automatic filtering of the
-matched subjects to the ones that have a line changed since the reference
-point.
-
-Internally mutant will use the semantics of `git diff git-reference` to
-determine which subjects have changes. A subject is selected by this mechanism
-when `git diff` reports a hunk that overlaps with the current subjects line
-range.
-
-## Example
-
-On a branch, executing:
-
-```
-mutant --since master 'ProjectNamespace*'
-```
-
-Will run mutation testing against all subjects that have a direct source code
-since the `master` branch. Assuming your project has 100 subjects, and you
-touched 2 of them in your branch: Mutant will only select these 2 subjects for
-mutation testing.
-
-## Recommended use
-
-Use incremental for any project where a full pass does not fit within acceptable
-round trip times for the human mind.
-
-A small 200 loc gem, should probably never use it. Typically its the local per
-developer environment that should use incremental first. On CI its recommended
-to stay with the full pass as long as possible and switch to incremental mode
-when the CI cycle time gets too annoying.
-
-In addition, once using incremental its recommended to run a full pass as a
-nightly job.
-
-Incremental is also a good mode to retrofit mutation testing into a legacy
-project.
-
-Good selectors for reference points are `HEAD~1` (the previous commit) or
-`master` (the integration branch).
-
-## Limitations
-
-Mutant only triggers incremental subject selection for *direct* code changes.
-It'll currently not select subjects that where indirectly changed.
-
-Counter examples:
-
-* A change to a constant that results in a behavior change of a subject will
-  not trigger that subject to be selected.
-* A change to a subject A that causes another subject B to change in behavior,
-  will not select subject B.
-
-These limitations may be removed in future versions of mutant, work on more
-fine grained tracing is underway.

data/docs/known-problems.md

@@ -1,30 +0,0 @@
-Known Problems
-==============
-
-The Crash / Stuck Problem (MRI)
--------------------------------
-
-Mutations generated by mutant can cause MRI to enter VM states its not prepared for.
-All MRI versions > 1.9 and < 2.2.1 are affected by this depending on your compiler flags,
-compiler version, and OS scheduling behavior.
-
-This can have the following unintended effects:
-
-* MRI crashes with a segfault. Mutant kills each mutation in a dedicated fork to isolate
-  the mutations side effects when this fork terminates abnormally (segfault) mutant
-  counts the mutation as killed.
-
-* MRI crashes with a segfault and gets stuck when handling the segfault.
-  Depending on the number of active kill jobs mutant might appear to continue normally until
-  all workers are stuck into this state when it begins to hang.
-  Currently mutant must assume that your test suite simply not terminated yet as from the outside
-  (parent process) the difference between a long running test and a stuck MRI is not observable.
-  Its planned to implement a timeout enforced from the parent process, but ideally MRI simply gets fixed.
-
-References:
-
-* [MRI fix](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
-* [MRI backport to 2.2.1](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
-* [Mutant issue](https://github.com/mbj/mutant/issues/265)
-* [Upstream bug redmine](https://bugs.ruby-lang.org/issues/10460)
-* [Upstream bug github](https://github.com/ruby/ruby/pull/822)

data/docs/limitations.md

@@ -1,50 +0,0 @@
-Limitations
-===========
-
-Subject
--------
-
-Mutant cannot emit mutations for some subjects.
-
-* methods defined within a closure.  For example, methods defined using `module_eval`, `class_eval`,
-  `define_method`, or `define_singleton_method`:
-
-    ```ruby
-    class Example
-      class_eval do
-        def example1
-        end
-      end
-
-      module_eval do
-        def example2
-        end
-      end
-
-      define_method(:example3) do
-      end
-
-      define_singleton_method(:example4) do
-      end
-    end
-    ```
-
-* singleton methods not defined on a constant or `self`
-
-    ```ruby
-    class Foo
-      def self.bar; end   # ok
-      def Foo.baz; end    # ok
-
-      myself = self
-      def myself.qux; end # cannot mutate
-    end
-    ```
-
-* methods defined with eval:
-
-    ```ruby
-    class Foo
-      class_eval('def bar; end') # cannot mutate
-    end
-    ```

data/docs/mutant-minitest.md

@@ -1,149 +0,0 @@
-mutant-minitest
-===============
-
-Before starting with mutant its recommended to understand the
-[nomenclature](/docs/nomenclature.md).
-
-## Setup
-
-To add mutant to your minitest code base you need to:
-
-1. Add `mutant-minitest` as development dependency to your `Gemfile` or `.gemspec`
-
-   This may look like:
-
-   ```ruby
-   # A gemfile
-   gem 'mutant-minitest'
-   ```
-
-2. Add `require 'mutant/minitest/coverage'` to your test environment (example to your `test/test_helper.rb`)
-
-   Example:
-
-   ```ruby
-   require 'minitest/autorun'
-   require 'mutant/minitest/coverage'
-
-   class YourTestBaseClass < MiniTest::Test
-     # ...
-   ```
-
-3. Add `.cover` call sides to your test suite to mark them as eligible for killing mutations in subjects.
-
-   Example:
-
-   ```ruby
-   class YourLibrarySomeClassTest < YourTestBaseClass
-     cover 'YourLibrary::SomeClass*' # tells mutant which subjects this tests should cover
-     # ...
-   ```
-
-4. Run mutant against the minitest integration
-
-   ```sh
-   bundle exec mutant --include lib --require 'your_library.rb' --use minitest -- 'YourLibrary*'
-   ```
-
-## Run through example
-
-This uses [mbj/auom](https://github.com/mbj/auom) a small library that
-has 100% mutation coverage. Its tests execute very fast and do not have any IO
-so its a good playground example to interact with.
-
-All the setup described above is already done.
-
-```sh
-git clone https://github.com/mbj/auom
-cd auom
-bundle install # gemfile references mutant-minitest already
-bundle exec mutant --include lib --require auom --use minitest -- 'AUOM*'
-```
-
-This prints a report like:
-
-```sh
-Mutant environment:
-Matcher:         #<Mutant::Matcher::Config match_expressions: [AUOM*]>
-Integration:     Mutant::Integration::Minitest
-Jobs:            8
-Includes:        ["lib"]
-Requires:        ["auom"]
-Subjects:        23
-Mutations:       1003
-Results:         1003
-Kills:           1003
-Alive:           0
-Runtime:         9.68s
-Killtime:        3.80s
-Overhead:        154.30%
-Mutations/s:     103.67
-Coverage:        100.00%
-```
-
-Now lets try adding some redundant (or unspecified) code:
-
-```sh
-patch -p1 <<'PATCH'
---- a/lib/auom/unit.rb
-+++ b/lib/auom/unit.rb
-@@ -170,7 +170,7 @@ module AUOM
-     # TODO: Move defaults coercions etc to .build method
-     #
-     def self.new(scalar, numerators = nil, denominators = nil)
--      scalar = rational(scalar)
-+      scalar = rational(scalar) if true
-
-       scalar, numerators   = resolve([*numerators], scalar, :*)
-       scalar, denominators = resolve([*denominators], scalar, :/)
-PATCH
-```
-
-Running mutant again prints the following:
-
-```
-AUOM::Unit.new:/home/mrh-dev/auom/lib/auom/unit.rb:172
-- minitest:AUOMTest::ClassMethods::New#test_reduced_unit
-- minitest:AUOMTest::ClassMethods::New#test_normalized_denominator_scalar
-- minitest:AUOMTest::ClassMethods::New#test_normalized_numerator_unit
-- minitest:AUOMTest::ClassMethods::New#test_incompatible_scalar
-- minitest:AUOMTest::ClassMethods::New#test_integer
-- minitest:AUOMTest::ClassMethods::New#test_sorted_numerator
-- minitest:AUOMTest::ClassMethods::New#test_unknown_unit
-- minitest:AUOMTest::ClassMethods::New#test_rational
-- minitest:AUOMTest::ClassMethods::New#test_normalized_numerator_scalar
-- minitest:AUOMTest::ClassMethods::New#test_sorted_denominator
-- minitest:AUOMTest::ClassMethods::New#test_normalized_denominator_unit
-evil:AUOM::Unit.new:/home/mrh-dev/auom/lib/auom/unit.rb:172:cd9ee
-@@ -1,9 +1,7 @@
- def self.new(scalar, numerators = nil, denominators = nil)
--  if true
--    scalar = rational(scalar)
--  end
-+  scalar = rational(scalar)
-   scalar, numerators = resolve([*numerators], scalar, :*)
-   scalar, denominators = resolve([*denominators], scalar, :/)
-   super(scalar, *[numerators, denominators].map(&:sort)).freeze
- end
------------------------
-Mutant configuration:
-Matcher:         #<Mutant::Matcher::Config match_expressions: [AUOM*]>
-Integration:     Mutant::Integration::Minitest
-Jobs:            8
-Includes:        ["lib"]
-Requires:        ["auom"]
-Subjects:        23
-Mutations:       1009
-Results:         1009
-Kills:           1008
-Alive:           1
-Runtime:         9.38s
-Killtime:        3.47s
-Overhead:        170.06%
-Mutations/s:     107.60
-Coverage:        99.90%
-```
-
-This shows mutant detected the alive mutation. Which shows the conditional we deliberately added above is redundant.
-
-Feel free to also remove some tests. Or do other modifications to either test or code.

data/docs/mutant-rspec.md

@@ -1,130 +0,0 @@
-mutant-rspec
-============
-
-Before starting with mutant its recommended to understand the
-[nomenclature](/docs/nomenclature.md).
-
-## Setup
-
-To add mutant to your rspec code base you need to:
-
-1. Add `mutant-rspec` as development dependency to your `Gemfile` or `.gemspec`
-
-   This may look like:
-
-   ```ruby
-   # A gemfile
-   gem 'mutant-rspec'
-   ```
-
-2. Run mutant against the rspec integration via the `--use rspec` flag.
-
-## Run through example
-
-This uses [mbj/auom](https://github.com/mbj/auom) a small library that
-has 100% mutation coverage. Its tests execute very fast and do not have any IO
-so its a good playground example to interact with.
-
-All the setup described above is already done.
-
-```sh
-git clone https://github.com/mbj/auom
-cd auom
-bundle install # gemfile references mutant-rspec already
-bundle exec mutant --include lib --require auom --use rspec -- 'AUOM*'
-```
-
-This prints a report like:
-
-```sh
-Mutant environment:
-Matcher:         #<Mutant::Matcher::Config match_expressions: [AUOM*]>
-Integration:     Mutant::Integration::Rspec
-Jobs:            8
-Includes:        ["lib"]
-Requires:        ["auom"]
-Subjects:        23
-Mutations:       1003
-Results:         1003
-Kills:           1003
-Alive:           0
-Runtime:         51.52s
-Killtime:        200.13s
-Overhead:        -74.26%
-Mutations/s:     19.47
-Coverage:        100.00%
-```
-
-Now lets try adding some redundant (or unspecified) code:
-
-```sh
-patch -p1 <<'PATCH'
---- a/lib/auom/unit.rb
-+++ b/lib/auom/unit.rb
-@@ -170,7 +170,7 @@ module AUOM
-     # TODO: Move defaults coercions etc to .build method
-     #
-     def self.new(scalar, numerators = nil, denominators = nil)
--      scalar = rational(scalar)
-+      scalar = rational(scalar) if true
-
-       scalar, numerators   = resolve([*numerators], scalar, :*)
-       scalar, denominators = resolve([*denominators], scalar, :/)
-PATCH
-```
-
-Running mutant again prints the following:
-
-```sh
-evil:AUOM::Unit.new:/home/mrh-dev/example/auom/lib/auom/unit.rb:172:45e17
-@@ -1,9 +1,7 @@
- def self.new(scalar, numerators = nil, denominators = nil)
--  if true
--    scalar = rational(scalar)
--  end
-+  scalar = rational(scalar)
-   scalar, numerators = resolve([*numerators], scalar, :*)
-   scalar, denominators = resolve([*denominators], scalar, :/)
-   super(scalar, *[numerators, denominators].map(&:sort)).freeze
- end
------------------------
-Mutant configuration:
-Matcher:         #<Mutant::Matcher::Config match_expressions: [AUOM*]>
-Integration:     Mutant::Integration::Rspec
-Jobs:            8
-Includes:        ["lib"]
-Requires:        ["auom"]
-Subjects:        23
-Mutations:       1009
-Results:         1009
-Kills:           1008
-Alive:           1
-Runtime:         50.93s
-Killtime:        190.09s
-Overhead:        -73.21%
-Mutations/s:     19.81
-Coverage:        99.90%
-```
-
-This shows mutant detected the alive mutation. Which shows the conditional we deliberately added above is redundant.
-
-Feel free to also remove some tests. Or do other modifications to either test or code.
-
-Test-Selection
---------------
-
-Mutation testing is slow. The key to making it fast is selecting the correct
-set of tests to run.  Mutant currently supports the following built-in
-strategy for selecting tests/specs:
-
-Mutant uses the "longest rspec example group descriptions prefix match" to
-select the tests to run.
-
-Example for a subject like `Foo::Bar#baz` it will run all example groups with
-description prefixes in `Foo::Bar#baz`, `Foo::Bar` and `Foo`. The order is
-important, so if mutant finds example groups in the current prefix level,
-these example groups *must* kill the mutation.
-
-```sh
-RAILS_ENV=test bundle exec mutant -r ./config/environment --use rspec User
-```