mutant 0.8.6 → 0.8.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Changelog.md +4 -0
- data/README.md +82 -75
- data/lib/mutant/matcher/method.rb +1 -1
- data/lib/mutant/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9f0d162e90a523f4f62c1f3541b1157fb78d6080
|
4
|
+
data.tar.gz: fa8dcc8abc2f6960ebd74a53711bf8e87ec0f32c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 4c672d952620acf28cff4a26127a03facbc32c13bece89934b487433dc1fd2a918a76188fa0efe86588c7a480e8bf54383baa66eed869eeaf27752dce5ab313e
|
7
|
+
data.tar.gz: f3e533b36d8f52544523fbd81cda480a0653861106cf841420e11043a953c41a043e7a3b88fd8838629272d3bd960e76d22494bce9b7f014b10c67601d7a2c6e
|
data/Changelog.md
CHANGED
data/README.md
CHANGED
@@ -21,30 +21,6 @@ to do its magic.
|
|
21
21
|
|
22
22
|
Mutant does not have really good "getting started" documentation currently so please refer to presentations and blog posts below.
|
23
23
|
|
24
|
-
Mutation-Operators:
|
25
|
-
-------------------
|
26
|
-
|
27
|
-
Mutant supports a wide range of mutation operators. An exhaustive list can be found in the [mutant-meta](https://github.com/mbj/mutant/tree/master/meta).
|
28
|
-
The `mutant-meta` is arranged to the AST-Node-Types of parser. Refer to parsers [AST documentation](https://github.com/whitequark/parser/blob/master/doc/AST_FORMAT.md) in doubt.
|
29
|
-
|
30
|
-
There is no easy and universal way to count the number of mutation operators a tool supports.
|
31
|
-
|
32
|
-
Presentations
|
33
|
-
-------------
|
34
|
-
|
35
|
-
There are some presentations about mutant in the wild:
|
36
|
-
|
37
|
-
* [RailsConf 2014](http://railsconf.com/) / http://confreaks.com/videos/3333-railsconf-mutation-testing-with-mutant
|
38
|
-
* [Wrocloverb 2014](http://wrocloverb.com/) / https://www.youtube.com/watch?v=rz-lFKEioLk
|
39
|
-
* [eurucamp 2013](http://2013.eurucamp.org/) / FrOSCon-2013 http://slid.es/markusschirp/mutation-testing
|
40
|
-
* [Cologne.rb](http://www.colognerb.de/topics/mutation-testing-mit-mutant) / https://github.com/DonSchado/colognerb-on-mutant/blob/master/mutation_testing_slides.pdf
|
41
|
-
|
42
|
-
Blog-Posts
|
43
|
-
----------
|
44
|
-
|
45
|
-
* http://www.sitepoint.com/mutation-testing-mutant/
|
46
|
-
* http://solnic.eu/2013/01/23/mutation-testing-with-mutant.html
|
47
|
-
|
48
24
|
Installation
|
49
25
|
------------
|
50
26
|
|
@@ -55,35 +31,7 @@ It'll pull the `mutant` gem (in correct version), that contains the main engine.
|
|
55
31
|
gem install mutant-rspec
|
56
32
|
```
|
57
33
|
|
58
|
-
The minitest integration is still in the [works](https://github.com/mbj/mutant/pull/
|
59
|
-
|
60
|
-
The Crash / Stuck Problem (MRI)
|
61
|
-
-------------------------------
|
62
|
-
|
63
|
-
Mutations generated by mutant can cause MRI to enter VM states its not prepared for.
|
64
|
-
All MRI versions > 1.9 and < 2.2.1 are affected by this depending on your compiler flags,
|
65
|
-
compiler version, and OS scheduling behavior.
|
66
|
-
|
67
|
-
This can have the following unintended effects:
|
68
|
-
|
69
|
-
* MRI crashes with a segfault. Mutant kills each mutation in a dedicated fork to isolate
|
70
|
-
the mutations side effects when this fork terminates abnormally (segfault) mutant
|
71
|
-
counts the mutation as killed.
|
72
|
-
|
73
|
-
* MRI crashes with a segfault and gets stuck when handling the segfault.
|
74
|
-
Depending on the number of active kill jobs mutant might appear to continue normally until
|
75
|
-
all workers are stuck into this state when it begins to hang.
|
76
|
-
Currently mutant must assume that your test suite simply not terminated yet as from the outside
|
77
|
-
(parent process) the difference between a long running test and a stuck MRI is not observable.
|
78
|
-
Its planned to implement a timeout enforced from the parent process, but ideally MRI simply gets fixed.
|
79
|
-
|
80
|
-
References:
|
81
|
-
|
82
|
-
* [MRI fix](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
|
83
|
-
* [MRI backport to 2.2.1](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
|
84
|
-
* [Mutant issue](https://github.com/mbj/mutant/issues/265)
|
85
|
-
* [Upstream bug redmine](https://bugs.ruby-lang.org/issues/10460)
|
86
|
-
* [Upstream bug github](https://github.com/ruby/ruby/pull/822)
|
34
|
+
The minitest integration is still in the [works](https://github.com/mbj/mutant/pull/445).
|
87
35
|
|
88
36
|
Examples
|
89
37
|
--------
|
@@ -100,6 +48,39 @@ mutant --include lib --require virtus --use rspec Virtus::Attribute.build
|
|
100
48
|
mutant --include lib --require virtus --use rspec Virtus::Attribute#type
|
101
49
|
```
|
102
50
|
|
51
|
+
Configuration
|
52
|
+
-------------
|
53
|
+
|
54
|
+
Occasionally mutant will produce a mutation with an infinite runtime. When this happens
|
55
|
+
mutant will look like it is running indefinitely without killing a remaining mutation. To
|
56
|
+
avoid mutations like this, consider adding a timeout around your tests. For example, in
|
57
|
+
RSpec you can add the following to your `spec_helper`:
|
58
|
+
|
59
|
+
```ruby
|
60
|
+
config.around(:each) do |example|
|
61
|
+
Timeout.timeout(5_000, &example)
|
62
|
+
end
|
63
|
+
```
|
64
|
+
|
65
|
+
which will fail specs which run for longer than 5 seconds.
|
66
|
+
|
67
|
+
Rails
|
68
|
+
-------
|
69
|
+
|
70
|
+
To mutation test Rails models with rspec comment out ```require 'rspec/autorun'``` from your spec_helper.rb file. Having done so you should be able to use commands like the following:
|
71
|
+
|
72
|
+
```sh
|
73
|
+
RAILS_ENV=test bundle exec mutant -r ./config/environment --use rspec User
|
74
|
+
```
|
75
|
+
|
76
|
+
Mutation-Operators:
|
77
|
+
-------------------
|
78
|
+
|
79
|
+
Mutant supports a wide range of mutation operators. An exhaustive list can be found in the [mutant-meta](https://github.com/mbj/mutant/tree/master/meta).
|
80
|
+
The `mutant-meta` is arranged to the AST-Node-Types of parser. Refer to parsers [AST documentation](https://github.com/whitequark/parser/blob/master/doc/AST_FORMAT.md) in doubt.
|
81
|
+
|
82
|
+
There is no easy and universal way to count the number of mutation operators a tool supports.
|
83
|
+
|
103
84
|
Subjects
|
104
85
|
--------
|
105
86
|
|
@@ -191,11 +172,55 @@ Mutation output is grouped by selection groups. Each group contains three sectio
|
|
191
172
|
-----------------------
|
192
173
|
```
|
193
174
|
|
175
|
+
Presentations
|
176
|
+
-------------
|
177
|
+
|
178
|
+
There are some presentations about mutant in the wild:
|
179
|
+
|
180
|
+
* [RailsConf 2014](http://railsconf.com/) / http://confreaks.com/videos/3333-railsconf-mutation-testing-with-mutant
|
181
|
+
* [Wrocloverb 2014](http://wrocloverb.com/) / https://www.youtube.com/watch?v=rz-lFKEioLk
|
182
|
+
* [eurucamp 2013](http://2013.eurucamp.org/) / FrOSCon-2013 http://slid.es/markusschirp/mutation-testing
|
183
|
+
* [Cologne.rb](http://www.colognerb.de/topics/mutation-testing-mit-mutant) / https://github.com/DonSchado/colognerb-on-mutant/blob/master/mutation_testing_slides.pdf
|
184
|
+
|
185
|
+
Blog-Posts
|
186
|
+
----------
|
187
|
+
|
188
|
+
* http://www.sitepoint.com/mutation-testing-mutant/
|
189
|
+
* http://solnic.eu/2013/01/23/mutation-testing-with-mutant.html
|
190
|
+
|
191
|
+
The Crash / Stuck Problem (MRI)
|
192
|
+
-------------------------------
|
193
|
+
|
194
|
+
Mutations generated by mutant can cause MRI to enter VM states its not prepared for.
|
195
|
+
All MRI versions > 1.9 and < 2.2.1 are affected by this depending on your compiler flags,
|
196
|
+
compiler version, and OS scheduling behavior.
|
197
|
+
|
198
|
+
This can have the following unintended effects:
|
199
|
+
|
200
|
+
* MRI crashes with a segfault. Mutant kills each mutation in a dedicated fork to isolate
|
201
|
+
the mutations side effects when this fork terminates abnormally (segfault) mutant
|
202
|
+
counts the mutation as killed.
|
203
|
+
|
204
|
+
* MRI crashes with a segfault and gets stuck when handling the segfault.
|
205
|
+
Depending on the number of active kill jobs mutant might appear to continue normally until
|
206
|
+
all workers are stuck into this state when it begins to hang.
|
207
|
+
Currently mutant must assume that your test suite simply not terminated yet as from the outside
|
208
|
+
(parent process) the difference between a long running test and a stuck MRI is not observable.
|
209
|
+
Its planned to implement a timeout enforced from the parent process, but ideally MRI simply gets fixed.
|
210
|
+
|
211
|
+
References:
|
212
|
+
|
213
|
+
* [MRI fix](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
|
214
|
+
* [MRI backport to 2.2.1](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
|
215
|
+
* [Mutant issue](https://github.com/mbj/mutant/issues/265)
|
216
|
+
* [Upstream bug redmine](https://bugs.ruby-lang.org/issues/10460)
|
217
|
+
* [Upstream bug github](https://github.com/ruby/ruby/pull/822)
|
218
|
+
|
194
219
|
Planning a presentation?
|
195
220
|
------------------------
|
196
221
|
|
197
222
|
Mutation testing lately (not only mutant) seems to attract some attention. So naturally
|
198
|
-
people do talks about it at conferences, user groups or other chances.
|
223
|
+
people do talks about it at conferences, user groups or other chances. Thanks for that!
|
199
224
|
|
200
225
|
As I (the author @mbj) am not too happy with some of the facts being presented about
|
201
226
|
mutant the last month.
|
@@ -204,29 +229,12 @@ So if you plan to do a presentation: I offer to review your slides / talk - for
|
|
204
229
|
My intention is NOT to change your bias pro / against this tool. Just to help to fix
|
205
230
|
invalid statements about the tool.
|
206
231
|
|
207
|
-
Also in many cases a conversation to the author
|
232
|
+
Also in many cases a conversation to the author should help you to improve the talk
|
208
233
|
significantly. One of mutants biggest weaknesses is the bad documentation, but instead of
|
209
234
|
assumptions based on the absence of docs, use the tool authors brain to fill the gaps.
|
210
235
|
|
211
236
|
Hint, same applies to papers.
|
212
237
|
|
213
|
-
Rails
|
214
|
-
-------
|
215
|
-
|
216
|
-
Assuming you are using rspec, you can mutation test Rails models by adding the following lines to your Gemfile:
|
217
|
-
|
218
|
-
```ruby
|
219
|
-
group :test do
|
220
|
-
gem 'mutant-rspec'
|
221
|
-
end
|
222
|
-
```
|
223
|
-
|
224
|
-
Next, run bundle and comment out ```require 'rspec/autorun'``` from your spec_helper.rb file. Having done so you should be able to use commands like the following:
|
225
|
-
|
226
|
-
```sh
|
227
|
-
RAILS_ENV=test bundle exec mutant -r ./config/environment --use rspec User
|
228
|
-
```
|
229
|
-
|
230
238
|
Support
|
231
239
|
-------
|
232
240
|
|
@@ -237,9 +245,8 @@ Your options:
|
|
237
245
|
* [GitHub Issues](https://github.com/mbj/mutant/issues)
|
238
246
|
* Ping me on [twitter](https://twitter.com/_m_b_j_)
|
239
247
|
|
240
|
-
There is also
|
241
|
-
|
242
|
-
a 'Question: ' prefix in title.
|
248
|
+
There is also a mutant [slack chat](https://mutation-testing.slack.com/).
|
249
|
+
@mention [@\_m\_b\_j\_](https://twitter.com/_m_b_j_) on twitter for an invite.
|
243
250
|
|
244
251
|
Credits
|
245
252
|
-------
|
@@ -7,7 +7,7 @@ module Mutant
|
|
7
7
|
|
8
8
|
# Methods within rbx kernel directory are precompiled and their source
|
9
9
|
# cannot be accessed via reading source location. Same for methods created by eval.
|
10
|
-
BLACKLIST = %r{\
|
10
|
+
BLACKLIST = %r{\A(kernel/|\(eval\)\z)}.freeze
|
11
11
|
|
12
12
|
# Enumerate matches
|
13
13
|
#
|
data/lib/mutant/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: mutant
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.8.
|
4
|
+
version: 0.8.7
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Markus Schirp
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-10-
|
11
|
+
date: 2015-10-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: parser
|