mutant 0.8.24 → 0.9.0

data/README.md

@@ -2,8 +2,6 @@ mutant
 ======
 
 [![Build Status](https://circleci.com/gh/mbj/mutant.svg?style=shield&circle-token=1afd77e8f0f9d0a11fd8f15f5d7b10270f4665e2)](https://circleci.com/gh/mbj/mutant/tree/master)
-[![Code Climate](https://codeclimate.com/github/mbj/mutant.svg)](https://codeclimate.com/github/mbj/mutant)
-[![Inline docs](http://inch-ci.org/github/mbj/mutant.svg)](http://inch-ci.org/github/mbj/mutant)
 [![Gem Version](https://img.shields.io/gem/v/mutant.svg)](https://rubygems.org/gems/mutant)
 [![Slack Status](https://mutation-testing-slack.herokuapp.com/badge.svg)](https://mutation-testing.slack.com/messages/mutant)
 
@@ -14,7 +12,7 @@ Mutant is a mutation testing tool for Ruby. Mutation testing is a technique to v
 ## Why do I want it?
 
 Mutant adds to your toolbox: Detection of uncovered semantics in your code.
-Coverage becomes a meaninful metric!
+Coverage becomes a meaningful metric!
 
 On each detection of uncovered semantics you have the opportunity to:
 
@@ -25,172 +23,107 @@ On each detection of uncovered semantics you have the opportunity to:
 ## How Do I use it?
 
 * Start with reading the [nomenclature](/docs/nomenclature.md) documentation.
-* Than select and setup your [integration](/docs/nomenclature.md#interation), also make sure
+* Then select and setup your [integration](/docs/nomenclature.md#integration), also make sure
   you can reproduce the examples in the integration specific documentation.
-* Identify your preferred mutation testing strategy. Its recommended to start at the commit level,
+* Identify your preferred mutation testing strategy. It is recommended to start at the commit level,
   to test only the code you had been touching. See the [incremental](#only-mutating-changed-code)
   mutation testing documentation.
 
-Mutant uses a pure Ruby [parser](https://github.com/whitequark/parser) and an [unparser](https://github.com/mbj/unparser)
-to do its magic.
+## Licensing
 
-Topics
-------
+Mutant was recently transitioned commercial software, with a free usage plan for opensource projects.
 
-* [Nomenclature](/docs/nomenclature.md)
-* [Reading Reports](/docs/reading-reports.md)
-* [Known Problems](/docs/known-problems.md)
-* [Limitations](/docs/limitations.md)
-* [Concurrency](/docs/concurrency.md)
-* [Rspec Integration](/docs/mutant-rspec.md)
-* [Minitest Integration](/docs/mutant-minitest.md)
+Commercial projects have to acquire a license per developer, with unlimited repositories
+per developer.
 
-Sponsoring
-----------
+Opensource projects have to acquire their free license per repository.
 
-Mutant, as published in the opensource version, would not exist without the help
-of [contributors](https://github.com/mbj/mutant/graphs/contributors) spending lots
-of their private time.
+The license distribution happens through the `mutant-license` gem in mutants dependencies.
+This gem is dynamically generated per licensee and comes with a unique license gem source
+URL.
 
-Additionally, the following features where sponsored by organizations:
+After signup for a license the following has to be added to your `Gemfile` replacing `${key}`
+with the license key and `${plan}` with `com` for commercial or `oss` for opensource usage.
 
-* The `mutant-minitest` integration was sponsored by [Arkency](https://arkency.com/)
-* Mutant's initial concurrency support was sponsored by an undisclosed company that does
-  currently not wish to be listed here.
-
-If your organization is interested in sponsoring a feature, general maintainership or bugfixes, contact
-[Markus Schirp](mailto:mbj@schirp-dso.com).
-
-Mutation-Operators
-------------------
-
-Mutant supports a wide range of mutation operators. An exhaustive list can be found in the [mutant-meta](https://github.com/mbj/mutant/tree/master/meta).
-The `mutant-meta` is arranged to the AST-Node-Types of parser. Refer to parsers [AST documentation](https://github.com/whitequark/parser/blob/master/doc/AST_FORMAT.md) in doubt.
-
-There is no easy and universal way to count the number of mutation operators a tool supports.
-
-Neutral (noop) Tests
---------------------
-
-Mutant will also test the original, unmutated, version your code. This ensures that mutant is able to properly setup and run your tests.
-If an error occurs while mutant/rspec is running testing the original code, you will receive an error like the following:
-```
---- Neutral failure ---
-Original code was inserted unmutated. And the test did NOT PASS.
-Your tests do not pass initially or you found a bug in mutant / unparser.
-...
-Test Output:
-marshal data too short
+```ruby
+source 'https://${plan}:${key}@gem.mutant.dev' do
+  gem 'mutant-license'
+end
 ```
-Currently, troubleshooting these errors requires using a debugger and/or modyifying mutant to print out the error. You will want to rescue and inspect exceptions raised in this method: lib/mutant/integration/rspec.rb:call
-
-Only Mutating Changed Code
---------------------------
 
-Running mutant for the first time on an existing codebase can be a rather disheartening experience due to the large number of alive mutations found! Mutant has a setting that can help. Using the `--since` argument, mutant will only mutate code that has been modified. This allows you to introduce mutant into an existing code base without drowning in errors. Example usage that will mutate all code changed between master and the current branch:
-
-```
-bundle exec mutant --include lib --require virtus --since master --use rspec Virtus::Attribute#type
-```
+The mutant license gem contains metadata that allows mutant to verify licensed use.
 
-Note that this feature requires at least git `2.13.0`.
+For commercial licenses mutant checks the git commit author or the configured git email
+to be in the set of licensed developers.
 
-Presentations
--------------
+For opensource licenses mutant checks the git remotes against the license whitelist.
+This allows the project maintainer to sign up and not bother collaborators with the details.
 
-There are some presentations about mutant in the wild:
+There are, apart from initial license gem installation, no remote interactions for
+license validation.
 
-* [RailsConf 2014](http://railsconf.com/) / http://confreaks.com/videos/3333-railsconf-mutation-testing-with-mutant
-* [Wrocloverb 2014](http://wrocloverb.com/) / https://www.youtube.com/watch?v=rz-lFKEioLk
-* [eurucamp 2013](http://2013.eurucamp.org/) / FrOSCon-2013 http://slid.es/markusschirp/mutation-testing
-* [Cologne.rb](http://www.colognerb.de/topics/mutation-testing-mit-mutant) / https://github.com/DonSchado/colognerb-on-mutant/blob/master/mutation_testing_slides.pdf
+To inquire for license please contact [Markus Schirp](mailto:mbj@schirp-dso.com?subject=Mutant%20License).
 
-Planning a presentation?
-------------------------
+### Pricing
 
-Mutation testing lately (not only mutant) seems to attract some attention. So naturally
-people do talks about it at conferences, user groups or other chances. Thanks for that!
+Only relevant for commercial use.
 
-As I (the author @mbj) am not too happy with some of the facts being presented about
-mutant the last month.
+Mutant offers a subscription model under a monthly plan.
+Yearly prepayments with discounts are available.
 
-So if you plan to do a presentation: I offer to review your slides / talk - for free of course.
-My intention is NOT to change your bias pro / against this tool. Just to help to fix
-invalid statements about the tool.
+For higher volumes different arrangements can be negotiated.
 
-Also in many cases a conversation to the author should help you to improve the talk
-significantly. One of mutants biggest weaknesses is the bad documentation, but instead of
-assumptions based on the absence of docs, use the tool authors brain to fill the gaps.
+## Topics
 
-Hint, same applies to papers.
-
-Blog posts
-----------
-
-Sorted by recency:
-
-* [A deep dive into mutation testing and how the Mutant gem works][troessner]
-* [Keep calm and kill mutants (December, 2015)][itransition]
-* [How to write better code using mutation testing (November 2015)][blockscore]
-* [How good are your Ruby tests? Testing your tests with mutant (June 2015)][arkency1]
-* [Mutation testing and continuous integration (May 2015)][arkency2]
-* [Why I want to introduce mutation testing to the `rails_event_store` gem (April 2015)][arkency3]
-* [Mutation testing with mutant (April 2014)][sitepoint]
-* [Mutation testing with mutant (January 2013)][solnic]
-
-[troessner]: https://troessner.svbtle.com/kill-all-the-mutants-a-deep-dive-into-mutation-testing-and-how-the-mutant-gem-works
-[itransition]: https://github.com/maksar/mentat
-[blockscore]: https://blog.blockscore.com/how-to-write-better-code-using-mutation-testing/
-[sitepoint]: http://www.sitepoint.com/mutation-testing-mutant/
-[arkency1]: http://blog.arkency.com/2015/06/how-good-are-your-ruby-tests-testing-your-tests-with-mutant/
-[arkency2]: http://blog.arkency.com/2015/05/mutation-testing-and-continuous-integration/
-[arkency3]: http://blog.arkency.com/2015/04/why-i-want-to-introduce-mutation-testing-to-the-rails-event-store-gem/
-[solnic]: http://solnic.eu/2013/01/23/mutation-testing-with-mutant.html
-
-Support
--------
-
-I'm very happy to receive/answer feedback/questions and criticism.
-
-Your options:
-
-* [GitHub Issues](https://github.com/mbj/mutant/issues)
-* Ping me on [twitter](https://twitter.com/_m_b_j_)
+* [Nomenclature](/docs/nomenclature.md)
+* [Reading Reports](/docs/reading-reports.md)
+* [Known Problems](/docs/known-problems.md)
+* [Limitations](/docs/limitations.md)
+* [Concurrency](/docs/concurrency.md)
+* [Rspec Integration](/docs/mutant-rspec.md)
+* [Minitest Integration](/docs/mutant-minitest.md)
 
-There is also a mutation testing slack chat. Get an invite [here](https://mutation-testing-slack.herokuapp.com).
-For discussing this project, join [#mutant](https://mutation-testing.slack.com/messages/#mutant).
+## Communication
 
-Other Channels:
+Try the following:
 
-- [#cosmic-ray](https://mutation-testing.slack.com/messages/cosmic-ray): for discussing `cosmic-ray`, the python mutation testing tool.
-- [#devtools](https://mutation-testing.slack.com/messages/devtools): for discussing the `devtools` metagem.
-- [#general](https://mutation-testing.slack.com/messages/general): for general discussions about mutation testing.
-- [#mutagen](https://mutation-testing.slack.com/messages/mutagen): for discussing `mutagen`, the javascript mutation testing tool.
-- [#random](https://mutation-testing.slack.com/messages/random): for misc. off topic discussion.
-- [#stryker](https://mutation-testing.slack.com/messages/stryker): for discussing `stryker`, the javascript mutation testing tool.
-- [#wtf-dev](https://mutation-testing.slack.com/messages/wtf-dev): for sharing software development wtfs.
+* [Github Issues](https://github.com/mbj/mutant/issues)
+* [Release Announcement Mailing List](https://announce.mutant.dev/signup)
+* [Slack channel](mutation-testing.slack.com) request invite from [Markus Schirp](mailto:mbj@schirp-dso.com?subject=Mutation%20Testing%20Slack%20Channel%20Invite).
 
+## Sponsoring
 
-Credits
--------
+Mutant, as published in the opensource version, would not exist without the help
+of [contributors](https://github.com/mbj/mutant/graphs/contributors) spending lots
+of their private time.
 
-* [Markus Schirp (mbj)](https://github.com/mbj)
-* A gist, now removed, from [dkubb](https://github.com/dkubb) showing ideas.
-* Older abandoned [mutant](https://github.com/txus/mutant). For motivating me doing this one.
-* [heckle](https://github.com/seattlerb/heckle). For getting me into mutation testing.
+Additionally, the following features where sponsored by organizations:
 
-Contributing
--------------
+* The `mutant-minitest` integration was sponsored by [Arkency](https://arkency.com/)
+* Mutant's initial concurrency support was sponsored by an undisclosed company that does
+  currently not wish to be listed here.
 
-* Fork the project.
-* Make your feature addition or bug fix.
-* Add tests for it. This is important so I don't break it in a
-  future version unintentionally.
-* Commit, do not mess with Rakefile or version
-  (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
-* Send me a pull request. Bonus points for topic branches.
+### Legal
 
-License
--------
+Contents of this repository are maintained by:
 
-See LICENSE file.
+```
+Schirp DSO LTD
+Director: Markus Schirp
+Email: info@schirp-dso.com
+Vat-ID: MT24186727
+Registration: C80467
+
+Office address:
+2, Carob Lane,
+Sir Harry Luke Street
+Naxxar NXR 2209,
+Malta
+
+Registred Address
+Phoenix Business Centre,
+The Penthouse,
+Old Railway Track,
+Santa Venera SVR9022,
+Malta
+```

data/Rakefile

@@ -3,24 +3,3 @@
 require 'devtools'
 
 Devtools.init_rake_tasks
-
-Rake.application.load_imports
-
-task('metrics:mutant').clear
-namespace :metrics do
-  task mutant: :coverage do
-    arguments = %w[
-      bundle exec mutant
-      --include lib
-      --since HEAD~1
-      --require mutant
-      --use rspec
-      --zombie
-    ]
-    arguments.concat(%w[--jobs 4]) if ENV.key?('CIRCLECI')
-
-    arguments.concat(%w[-- Mutant*])
-
-    Kernel.system(*arguments) or fail 'Mutant task is not successful'
-  end
-end

data/bin/mutant

@@ -25,7 +25,6 @@ namespace =
       includes:         %w[
         mutant
         unparser
-        morpher
         adamantium
         equalizer
         anima
@@ -37,4 +36,10 @@ namespace =
     Mutant
   end
 
-Kernel.exit(namespace::CLI.run(ARGV))
+Kernel.exit(
+  namespace::CLI.run(
+    namespace::WORLD,
+    namespace::Config::DEFAULT,
+    ARGV
+  )
+)

data/config/reek.yml

@@ -61,6 +61,8 @@ detectors:
     - Mutant::Mutator
     - Mutant::Meta::Example::DSL
     max_ifs: 1
+  TooManyConstants:
+    enabled: false
   TooManyInstanceVariables:
     enabled: true
     exclude:
@@ -125,7 +127,6 @@ detectors:
     enabled: true
     exclude:
     - Mutant::AST::Sexp#s
-    - Mutant::Actor::Env#new_mailbox
     - Mutant::CLI#reporter
     - Mutant::Integration::Null#call
     - Mutant::Integration::Rspec#parse_example

data/config/rubocop.yml

@@ -94,10 +94,6 @@ PercentLiteralDelimiters:
 SymbolArray:
   Enabled: true
 
-# Align if/else blocks with the variable assignment
-EndAlignment:
-  EnforcedStyleAlignWith: variable
-
 # Prefer #kind_of? over #is_a?
 ClassCheck:
   EnforcedStyle: kind_of?
@@ -111,11 +107,9 @@ Metrics/AbcSize:
   Max: 21.02
 
 Metrics/BlockLength:
-  Enabled: false
-
-# Do not prefer lambda.call(...) over lambda.(...)
-LambdaCall:
-  Enabled: false
+  Exclude:
+  - 'spec/**/*.rb'
+  - 'mutant.gemspec'
 
 # Buggy cop, returns false positive for our code base
 NonLocalExitFromIterator:
@@ -196,6 +190,8 @@ Style/CommentedKeyword:
   Enabled: false
 Style/MixinGrouping:
   Enabled: false
+Style/RaiseArgs:
+  Enabled: false
 Style/RescueStandardError:
   Enabled: false
 Style/StderrPuts:

data/docs/incremental.md

@@ -0,0 +1,76 @@
+# Incremental mutation testing
+
+Incremental mutation testing allows you to significantly speed up mutation
+analysis, via skipping over subjects that are not changed.
+
+This is an explicit trade off, often worth the downsides.
+
+## Motivation
+
+Despite all efforts in setting up a project for maximum mutation testing
+efficiency, running a full pass over all available subjects is not always
+desired. On building your branches on CI, and during local commit stage, its
+instead more efficient to scope the Mutant run to everything that was
+touched in the current branch / commit. The working set of subjects.
+
+For not having to manually keep track of the working set, the `--since` option
+allows you to specify a git reference point. Mutant will automatically subset all
+available subjects to the ones that where touched since the reference point.
+
+## Usage
+
+Use the `--since git-reference` flag to enable automatic filtering of the
+matched subjects to the ones that have a line changed since the reference
+point.
+
+Internally mutant will use the semantics of `git diff git-reference` to
+determine which subjects have changes. A subject is selected by this mechanism
+when `git diff` reports a hunk that overlaps with the current subjects line
+range.
+
+## Example
+
+On a branch, executing:
+
+```
+mutant --since master 'ProjectNamespace*'
+```
+
+Will run mutation testing against all subjects that have a direct source code
+since the `master` branch. Assuming your project has 100 subjects, and you
+touched 2 of them in your branch: Mutant will only select these 2 subjects for
+mutation testing.
+
+## Recommended use
+
+Use incremental for any project where a full pass does not fit within acceptable
+round trip times for the human mind.
+
+A small 200 loc gem, should probably never use it. Typically its the local per
+developer environment that should use incremental first. On CI its recommended
+to stay with the full pass as long as possible and switch to incremental mode
+when the CI cycle time gets too annoying.
+
+In addition, once using incremental its recommended to run a full pass as a
+nightly job.
+
+Incremental is also a good mode to retrofit mutation testing into a legacy
+project.
+
+Good selectors for reference points are `HEAD~1` (the previous commit) or
+`master` (the integration branch).
+
+## Limitations
+
+Mutant only triggers incremental subject selection for *direct* code changes.
+It'll currently not select subjects that where indirectly changed.
+
+Counter examples:
+
+* A change to a constant that results in a behavior change of a subject will
+  not trigger that subject to be selected.
+* A change to a subject A that causes another subject B to change in behavior,
+  will not select subject B.
+
+These limitations may be removed in future versions of mutant, work on more
+fine grained tracing is underway.

data/docs/known-problems.md

@@ -1,20 +1,6 @@
 Known Problems
 ==============
 
-Mutations with Infinite Runtimes
----------------------------------
-
-Occasionally mutant will produce a mutation with an infinite runtime. When this happens
-mutant will look like it is running indefinitely without killing a remaining mutation. To
-avoid mutations like this, consider adding a timeout around your tests. For example, in
-RSpec you can add the following to your `spec_helper`:
-```ruby
-config.around(:each) do |example|
-  Timeout.timeout(5, &example)
-end
-```
-which will fail specs which run for longer than 5 seconds.
-
 The Crash / Stuck Problem (MRI)
 -------------------------------