mutant 0.9.9 → 0.9.14

data/docs/known-problems.md

@@ -1,30 +0,0 @@
-Known Problems
-==============
-
-The Crash / Stuck Problem (MRI)
--------------------------------
-
-Mutations generated by mutant can cause MRI to enter VM states its not prepared for.
-All MRI versions > 1.9 and < 2.2.1 are affected by this depending on your compiler flags,
-compiler version, and OS scheduling behavior.
-
-This can have the following unintended effects:
-
-* MRI crashes with a segfault. Mutant kills each mutation in a dedicated fork to isolate
-  the mutations side effects when this fork terminates abnormally (segfault) mutant
-  counts the mutation as killed.
-
-* MRI crashes with a segfault and gets stuck when handling the segfault.
-  Depending on the number of active kill jobs mutant might appear to continue normally until
-  all workers are stuck into this state when it begins to hang.
-  Currently mutant must assume that your test suite simply not terminated yet as from the outside
-  (parent process) the difference between a long running test and a stuck MRI is not observable.
-  Its planned to implement a timeout enforced from the parent process, but ideally MRI simply gets fixed.
-
-References:
-
-* [MRI fix](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
-* [MRI backport to 2.2.1](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
-* [Mutant issue](https://github.com/mbj/mutant/issues/265)
-* [Upstream bug redmine](https://bugs.ruby-lang.org/issues/10460)
-* [Upstream bug github](https://github.com/ruby/ruby/pull/822)

data/docs/limitations.md

@@ -1,50 +0,0 @@
-Limitations
-===========
-
-Subject
--------
-
-Mutant cannot emit mutations for some subjects.
-
-* methods defined within a closure.  For example, methods defined using `module_eval`, `class_eval`,
-  `define_method`, or `define_singleton_method`:
-
-    ```ruby
-    class Example
-      class_eval do
-        def example1
-        end
-      end
-
-      module_eval do
-        def example2
-        end
-      end
-
-      define_method(:example3) do
-      end
-
-      define_singleton_method(:example4) do
-      end
-    end
-    ```
-
-* singleton methods not defined on a constant or `self`
-
-    ```ruby
-    class Foo
-      def self.bar; end   # ok
-      def Foo.baz; end    # ok
-
-      myself = self
-      def myself.qux; end # cannot mutate
-    end
-    ```
-
-* methods defined with eval:
-
-    ```ruby
-    class Foo
-      class_eval('def bar; end') # cannot mutate
-    end
-    ```

data/docs/mutant-minitest.md

@@ -1,149 +0,0 @@
-mutant-minitest
-===============
-
-Before starting with mutant its recommended to understand the
-[nomenclature](/docs/nomenclature.md).
-
-## Setup
-
-To add mutant to your minitest code base you need to:
-
-1. Add `mutant-minitest` as development dependency to your `Gemfile` or `.gemspec`
-
-   This may look like:
-
-   ```ruby
-   # A gemfile
-   gem 'mutant-minitest'
-   ```
-
-2. Add `require 'mutant/minitest/coverage'` to your test environment (example to your `test/test_helper.rb`)
-
-   Example:
-
-   ```ruby
-   require 'minitest/autorun'
-   require 'mutant/minitest/coverage'
-
-   class YourTestBaseClass < MiniTest::Test
-     # ...
-   ```
-
-3. Add `.cover` call sides to your test suite to mark them as eligible for killing mutations in subjects.
-
-   Example:
-
-   ```ruby
-   class YourLibrarySomeClassTest < YourTestBaseClass
-     cover 'YourLibrary::SomeClass*' # tells mutant which subjects this tests should cover
-     # ...
-   ```
-
-4. Run mutant against the minitest integration
-
-   ```sh
-   bundle exec mutant --include lib --require 'your_library.rb' --use minitest -- 'YourLibrary*'
-   ```
-
-## Run through example
-
-This uses [mbj/auom](https://github.com/mbj/auom) a small library that
-has 100% mutation coverage. Its tests execute very fast and do not have any IO
-so its a good playground example to interact with.
-
-All the setup described above is already done.
-
-```sh
-git clone https://github.com/mbj/auom
-cd auom
-bundle install # gemfile references mutant-minitest already
-bundle exec mutant --include lib --require auom --use minitest -- 'AUOM*'
-```
-
-This prints a report like:
-
-```sh
-Mutant environment:
-Matcher:         #<Mutant::Matcher::Config match_expressions: [AUOM*]>
-Integration:     Mutant::Integration::Minitest
-Jobs:            8
-Includes:        ["lib"]
-Requires:        ["auom"]
-Subjects:        23
-Mutations:       1003
-Results:         1003
-Kills:           1003
-Alive:           0
-Runtime:         9.68s
-Killtime:        3.80s
-Overhead:        154.30%
-Mutations/s:     103.67
-Coverage:        100.00%
-```
-
-Now lets try adding some redundant (or unspecified) code:
-
-```sh
-patch -p1 <<'PATCH'
---- a/lib/auom/unit.rb
-+++ b/lib/auom/unit.rb
-@@ -170,7 +170,7 @@ module AUOM
-     # TODO: Move defaults coercions etc to .build method
-     #
-     def self.new(scalar, numerators = nil, denominators = nil)
--      scalar = rational(scalar)
-+      scalar = rational(scalar) if true
-
-       scalar, numerators   = resolve([*numerators], scalar, :*)
-       scalar, denominators = resolve([*denominators], scalar, :/)
-PATCH
-```
-
-Running mutant again prints the following:
-
-```
-AUOM::Unit.new:/home/mrh-dev/auom/lib/auom/unit.rb:172
-- minitest:AUOMTest::ClassMethods::New#test_reduced_unit
-- minitest:AUOMTest::ClassMethods::New#test_normalized_denominator_scalar
-- minitest:AUOMTest::ClassMethods::New#test_normalized_numerator_unit
-- minitest:AUOMTest::ClassMethods::New#test_incompatible_scalar
-- minitest:AUOMTest::ClassMethods::New#test_integer
-- minitest:AUOMTest::ClassMethods::New#test_sorted_numerator
-- minitest:AUOMTest::ClassMethods::New#test_unknown_unit
-- minitest:AUOMTest::ClassMethods::New#test_rational
-- minitest:AUOMTest::ClassMethods::New#test_normalized_numerator_scalar
-- minitest:AUOMTest::ClassMethods::New#test_sorted_denominator
-- minitest:AUOMTest::ClassMethods::New#test_normalized_denominator_unit
-evil:AUOM::Unit.new:/home/mrh-dev/auom/lib/auom/unit.rb:172:cd9ee
-@@ -1,9 +1,7 @@
- def self.new(scalar, numerators = nil, denominators = nil)
--  if true
--    scalar = rational(scalar)
--  end
-+  scalar = rational(scalar)
-   scalar, numerators = resolve([*numerators], scalar, :*)
-   scalar, denominators = resolve([*denominators], scalar, :/)
-   super(scalar, *[numerators, denominators].map(&:sort)).freeze
- end
------------------------
-Mutant configuration:
-Matcher:         #<Mutant::Matcher::Config match_expressions: [AUOM*]>
-Integration:     Mutant::Integration::Minitest
-Jobs:            8
-Includes:        ["lib"]
-Requires:        ["auom"]
-Subjects:        23
-Mutations:       1009
-Results:         1009
-Kills:           1008
-Alive:           1
-Runtime:         9.38s
-Killtime:        3.47s
-Overhead:        170.06%
-Mutations/s:     107.60
-Coverage:        99.90%
-```
-
-This shows mutant detected the alive mutation. Which shows the conditional we deliberately added above is redundant.
-
-Feel free to also remove some tests. Or do other modifications to either test or code.

data/docs/mutant-rspec.md

@@ -1,130 +0,0 @@
-mutant-rspec
-============
-
-Before starting with mutant its recommended to understand the
-[nomenclature](/docs/nomenclature.md).
-
-## Setup
-
-To add mutant to your rspec code base you need to:
-
-1. Add `mutant-rspec` as development dependency to your `Gemfile` or `.gemspec`
-
-   This may look like:
-
-   ```ruby
-   # A gemfile
-   gem 'mutant-rspec'
-   ```
-
-2. Run mutant against the rspec integration via the `--use rspec` flag.
-
-## Run through example
-
-This uses [mbj/auom](https://github.com/mbj/auom) a small library that
-has 100% mutation coverage. Its tests execute very fast and do not have any IO
-so its a good playground example to interact with.
-
-All the setup described above is already done.
-
-```sh
-git clone https://github.com/mbj/auom
-cd auom
-bundle install # gemfile references mutant-rspec already
-bundle exec mutant --include lib --require auom --use rspec -- 'AUOM*'
-```
-
-This prints a report like:
-
-```sh
-Mutant environment:
-Matcher:         #<Mutant::Matcher::Config match_expressions: [AUOM*]>
-Integration:     Mutant::Integration::Rspec
-Jobs:            8
-Includes:        ["lib"]
-Requires:        ["auom"]
-Subjects:        23
-Mutations:       1003
-Results:         1003
-Kills:           1003
-Alive:           0
-Runtime:         51.52s
-Killtime:        200.13s
-Overhead:        -74.26%
-Mutations/s:     19.47
-Coverage:        100.00%
-```
-
-Now lets try adding some redundant (or unspecified) code:
-
-```sh
-patch -p1 <<'PATCH'
---- a/lib/auom/unit.rb
-+++ b/lib/auom/unit.rb
-@@ -170,7 +170,7 @@ module AUOM
-     # TODO: Move defaults coercions etc to .build method
-     #
-     def self.new(scalar, numerators = nil, denominators = nil)
--      scalar = rational(scalar)
-+      scalar = rational(scalar) if true
-
-       scalar, numerators   = resolve([*numerators], scalar, :*)
-       scalar, denominators = resolve([*denominators], scalar, :/)
-PATCH
-```
-
-Running mutant again prints the following:
-
-```sh
-evil:AUOM::Unit.new:/home/mrh-dev/example/auom/lib/auom/unit.rb:172:45e17
-@@ -1,9 +1,7 @@
- def self.new(scalar, numerators = nil, denominators = nil)
--  if true
--    scalar = rational(scalar)
--  end
-+  scalar = rational(scalar)
-   scalar, numerators = resolve([*numerators], scalar, :*)
-   scalar, denominators = resolve([*denominators], scalar, :/)
-   super(scalar, *[numerators, denominators].map(&:sort)).freeze
- end
------------------------
-Mutant configuration:
-Matcher:         #<Mutant::Matcher::Config match_expressions: [AUOM*]>
-Integration:     Mutant::Integration::Rspec
-Jobs:            8
-Includes:        ["lib"]
-Requires:        ["auom"]
-Subjects:        23
-Mutations:       1009
-Results:         1009
-Kills:           1008
-Alive:           1
-Runtime:         50.93s
-Killtime:        190.09s
-Overhead:        -73.21%
-Mutations/s:     19.81
-Coverage:        99.90%
-```
-
-This shows mutant detected the alive mutation. Which shows the conditional we deliberately added above is redundant.
-
-Feel free to also remove some tests. Or do other modifications to either test or code.
-
-Test-Selection
---------------
-
-Mutation testing is slow. The key to making it fast is selecting the correct
-set of tests to run.  Mutant currently supports the following built-in
-strategy for selecting tests/specs:
-
-Mutant uses the "longest rspec example group descriptions prefix match" to
-select the tests to run.
-
-Example for a subject like `Foo::Bar#baz` it will run all example groups with
-description prefixes in `Foo::Bar#baz`, `Foo::Bar` and `Foo`. The order is
-important, so if mutant finds example groups in the current prefix level,
-these example groups *must* kill the mutation.
-
-```sh
-RAILS_ENV=test bundle exec mutant -r ./config/environment --use rspec User
-```

data/docs/nomenclature.md

@@ -1,82 +0,0 @@
-Nomenclature
-============
-
-The following explains several nouns you may experience in mutant's documentation.
-It's a good idea to familiarize yourself before moving on.
-
-## AST
-
-Acronym for [Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree)
-and the level of abstraction mutant operates on.
-
-## Subject
-
-An addressable piece of code to be targeted for mutation testing.
-
-Mutant currently supports the following subjects:
-
-* Instance methods
-* Singleton (class) methods
-
-Other subjects (constants, class bodies for DSLs, ...) are possible but aren't
-implemented in the OSS version.
-
-## Mutation operator
-
-A transformation applied to the AST of a subject. Mutant knows the following high level operator
-classes:
-
-* Semantic Reduction
-* Orthogonal Replacement
-* [Noop](#neutral-noop-tests)
-
-An exhaustive list can be found in the [mutant-meta](https://github.com/mbj/mutant/tree/master/meta)
-subdirectory of the source.
-
-## Mutation
-
-The result of applying a mutation operator to the AST of a subject. A mutation represents a
-hypothesis that ideally gets falsified by the tests.
-
-## Insertion
-
-The process of inserting a mutation into the runtime environment.
-Mutant currently supports insertion via dynamically created monkeypatches.
-
-Other insertion strategies (such as "boot time") are possible but aren't implemented
-in the OSS version.
-
-## Isolation
-
-The attempt to isolate the (side) effects of killing a mutation via an integration
-to prevent a mutation leaking into adjacent concurrent, or future mutations.
-
-Examples of sources for leaks are
-
-* Global variable writes
-* Thread local writes
-* DB State
-* File system
-
-Natively, mutant offers fork isolation. This works for any state within the executing
-Ruby process. For all state reachable via IO, it's the test author's responsibility to
-provide proper isolation.
-
-## Integration
-
-The method used to determine if a specific inserted mutation is covered by tests.
-
-Currently mutant supports integrations for:
-
-* [mutant-rspec](/docs/mutant-rspec.md) for [rspec](https://rspec.info)
-* [mutant-minitest](/docs/mutant-minitest.md) for [minitest](https://github.com/seattlerb/minitest)
-
-## Report
-
-Mutant currently provides two different reporters:
-
-* Progress (printed during mutation testing).
-* Summary (printed at the end of a finished analysis run)
-
-A reporter producing a machine readable report does not exist in the OSS version
-at the time of writing this documentation.

data/docs/reading-reports.md

@@ -1,74 +0,0 @@
-Reading Reports
-===============
-
-Mutation output is grouped by selection groups. Each group contains three sections:
-
-1. An identifier for the current group.
-
-   **Format**:
-
-   ```text
-   [SUBJECT EXPRESSION]:[SOURCE LOCATION]:[LINENO]
-   ```
-
-   **Example**:
-
-   ```text
-   Book#add_page:Book#add_page:/home/dev/mutant-examples/lib/book.rb:18
-   ```
-
-2. A list of specs that mutant ran to try to kill mutations for the current group.
-
-   **Format**:
-
-   ```text
-   - [INTEGRATION]:0:[SPEC LOCATION]:[SPEC DESCRIPTION]
-   - [INTEGRATION]:1:[SPEC LOCATION]:[SPEC DESCRIPTION]
-   ```
-
-   **Example**:
-
-   ```text
-   - rspec:0:./spec/unit/book_spec.rb:9/Book#add_page should return self
-   - rspec:1:./spec/unit/book_spec.rb:13/Book#add_page should add page to book
-   ```
-
-3. A list of unkilled mutations diffed against the original unparsed source
-
-   **Format**:
-
-   ```text
-   [MUTATION TYPE]:[SUBJECT EXPRESSION]:[SOURCE LOCATION]:[SOURCE LINENO]:[IDENTIFIER]
-   [DIFF]
-   -----------------------
-   ```
-
-   - `[MUTATION TYPE]` will be one of the following:
-      - `evil` - a mutation of your source was not killed by your tests
-      - `neutral` your original source was injected and one or more tests failed
-   - `[IDENTIFIER]` - Unique identifier for this mutation
-
-   **Example**:
-
-   ```diff
-   evil:Book#add_page:Book#add_page:/home/dev/mutant-examples/lib/book.rb:18:01f69
-   @@ -1,6 +1,6 @@
-    def add_page(page)
-   -  @pages << page
-   +  @pages
-      @index[page.number] = page
-      self
-    end
-   -----------------------
-   evil:Book#add_page:Book#add_page:/home/dev/mutant-examples/lib/book.rb:18:b1ff2
-   @@ -1,6 +1,6 @@
-    def add_page(page)
-   -  @pages << page
-   +  self
-      @index[page.number] = page
-      self
-    end
-   -----------------------
-   ```
-
-At this time no machine readable output exists in the opensourced versions of mutant.