mutineer 1.4.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +250 -0
- data/README.md +198 -27
- data/lib/mutineer/changed_lines.rb +24 -1
- data/lib/mutineer/cli.rb +112 -16
- data/lib/mutineer/config.rb +64 -13
- data/lib/mutineer/coverage_map.rb +161 -14
- data/lib/mutineer/daemon_backend.rb +3 -0
- data/lib/mutineer/daemon_client.rb +170 -13
- data/lib/mutineer/daemon_server.rb +2 -1
- data/lib/mutineer/external_backend.rb +2 -2
- data/lib/mutineer/isolation.rb +91 -6
- data/lib/mutineer/kill_channel.rb +284 -0
- data/lib/mutineer/kill_matrix.rb +88 -0
- data/lib/mutineer/minitest_integration/kill_recorder.rb +301 -0
- data/lib/mutineer/minitest_integration/outer_reporter.rb +126 -0
- data/lib/mutineer/minitest_integration/stop_at_first_failure.rb +14 -68
- data/lib/mutineer/minitest_integration.rb +27 -10
- data/lib/mutineer/mutant_id.rb +5 -4
- data/lib/mutineer/mutators/array_literal.rb +0 -7
- data/lib/mutineer/mutators/base.rb +63 -4
- data/lib/mutineer/mutators/chain_link.rb +0 -7
- data/lib/mutineer/mutators/collection_method.rb +1 -1
- data/lib/mutineer/mutators/condition_forcing.rb +0 -7
- data/lib/mutineer/mutators/operand_removal.rb +0 -7
- data/lib/mutineer/mutators/operator_assignment.rb +0 -7
- data/lib/mutineer/mutators/return_nil.rb +5 -11
- data/lib/mutineer/mutators/string_literal.rb +1 -1
- data/lib/mutineer/pairing.rb +222 -15
- data/lib/mutineer/parser.rb +12 -3
- data/lib/mutineer/project.rb +7 -0
- data/lib/mutineer/rails_worker_db.rb +1 -1
- data/lib/mutineer/reporter.rb +288 -52
- data/lib/mutineer/result.rb +37 -1
- data/lib/mutineer/runner.rb +133 -16
- data/lib/mutineer/statement_lines.rb +143 -0
- data/lib/mutineer/test_runners/minitest.rb +4 -2
- data/lib/mutineer/test_runners/rspec.rb +243 -3
- data/lib/mutineer/test_runners.rb +5 -3
- data/lib/mutineer/version.rb +1 -1
- data/lib/mutineer/worker_pool.rb +4 -2
- data/lib/mutineer.rb +1 -0
- metadata +8 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 50113a063a85737ad99476f85e78cb2ac015f267018d19ba6350ba36ff4ac6df
|
|
4
|
+
data.tar.gz: c12343a00d2df6d495859644c326bc650e33d013ddf009f2648c06a199aceb86
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a170b15c6883a4dcec6a2798c3b102a5f9c9ded1f46c166df98d0e5bdf5b891ca5b24892efe06245680b45a94da38638adc36ca9a48a48d2ac1c07b29126f173
|
|
7
|
+
data.tar.gz: a30272ff6afb48dd666d6945c6532d987ef335e0e78dc422420cf26cdbec639ee1c850118a10ba25e10a426a2644b3926e39698126eaad0646707a4f2be2e719
|
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,255 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [1.5.0] - 2026-10-06
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
- **`--timeout SECONDS` and `--capture-timeout SECONDS`**, also `timeout:` and
|
|
13
|
+
`capture_timeout:` in `.mutineer.yml`. They set the in-process per-mutant
|
|
14
|
+
time limit (default 10s) and the coverage-capture time limit (default 120s),
|
|
15
|
+
which were fixed. On a large Rails suite the clean run of the unmutated
|
|
16
|
+
tests took longer than 120s, so every run that offered the whole suite
|
|
17
|
+
stopped as not green, and a mutant on a line many tests cover took longer
|
|
18
|
+
than 10s and was scored `timeout`. `--daemon` keeps its own per-mutant
|
|
19
|
+
limit but uses `--capture-timeout` for its coverage capture;
|
|
20
|
+
`--test-command` uses neither. Either backend warns about a limit it was
|
|
21
|
+
given and does not use.
|
|
22
|
+
- **`--cache-dir DIR`**, also `cache_dir:` in `.mutineer.yml`, sets where the
|
|
23
|
+
coverage cache is written (default `.mutineer`, as before). Two runs from the
|
|
24
|
+
same project root shared `.mutineer/coverage.json` and could overwrite each
|
|
25
|
+
other's map; each can now have its own directory. `--daemon` uses it too.
|
|
26
|
+
A blank value (an unset CI variable) exits 2 rather than writing the cache
|
|
27
|
+
into the project root.
|
|
28
|
+
- **The README states how timeouts affect the score** (#62): a timed-out
|
|
29
|
+
mutant is left out of the score, and counts as a mutant with no verdict
|
|
30
|
+
under `--threshold`.
|
|
31
|
+
- **`--matrix` reports which tests kill each mutant** (`matrix: true` in
|
|
32
|
+
`.mutineer.yml`). Each mutant runs every test in its covering files instead
|
|
33
|
+
of stopping at the first failure, and the child sends each test's outcome to
|
|
34
|
+
the parent as it is recorded. The report names blind tests, which ran in a
|
|
35
|
+
complete row and killed no mutant, and redundant tests, whose every kill
|
|
36
|
+
another test also makes. Redundancy is judged one test at a time, so delete
|
|
37
|
+
redundant tests one at a time. The human report lists up to 20 of each, the
|
|
38
|
+
HTML report all of them, and the JSON report moves to schema `1.5` with a
|
|
39
|
+
`matrix` block that appears only with the flag. A test is its file and its
|
|
40
|
+
id (the example id under RSpec), so examples that share a description stay
|
|
41
|
+
apart and an example whose generated description changes with the mutant
|
|
42
|
+
stays one test. Verdicts, the score and the exit code do not change: each
|
|
43
|
+
mutant gets the verdict a run without the flag gives. After a serial test
|
|
44
|
+
fails, an exit, a crash or the timeout in a later test, class or group
|
|
45
|
+
leaves it `killed`; an end in the code that run still reaches (the failing
|
|
46
|
+
test's class wrapper or group hooks, the suite's own cleanup) and a failure
|
|
47
|
+
in a `parallelize_me!` class keep the exit status. A row is complete only
|
|
48
|
+
when the child's stream arrived in order and every test reported (under
|
|
49
|
+
Minitest the recorder saw every test, under RSpec every planned example),
|
|
50
|
+
so a caught `Interrupt` leaves it incomplete. The human report names up
|
|
51
|
+
to 20 incomplete rows and the HTML and JSON reports all of them; the human
|
|
52
|
+
and HTML reports warn not to delete a blind test until its rows are
|
|
53
|
+
complete.
|
|
54
|
+
Minitest and RSpec 3.3+ are supported, on the in-process backend only;
|
|
55
|
+
`--daemon`, `--test-command`, `--fail-fast` and `--dry-run` exit 2 with it.
|
|
56
|
+
`--no-matrix` and `--no-fail-fast` beat the matching `.mutineer.yml` key,
|
|
57
|
+
so a file that sets one can still run with a flag that conflicts with it.
|
|
58
|
+
- **`no_coverage[]` and `uncapturable[]` name each mutant.** Every entry in
|
|
59
|
+
the JSON report now has `operator`, `token` and `id`, as `ignored[]` has, so
|
|
60
|
+
mutants on one line can be told apart and an `id` can go in `ignore:`. The
|
|
61
|
+
entries are sorted by `(file, line, operator, id)`. `schema_version` is `1.6`.
|
|
62
|
+
|
|
63
|
+
- **The site and README explain "clean-room" and compare Mutineer with
|
|
64
|
+
Mutant**, and the landing page hero has a one-minute explainer video. It
|
|
65
|
+
plays at 55% volume while in view (not with reduced motion; muted until
|
|
66
|
+
the first click or key where the browser requires it), pauses when it scrolls
|
|
67
|
+
away or the tab is hidden, and has an Enlarge button.
|
|
68
|
+
|
|
69
|
+
### Changed
|
|
70
|
+
- **The human report has a `Timeout:` row**, and the score line lists timeouts
|
|
71
|
+
apart from errored mutants. Before, the `Errored:` row and the score line
|
|
72
|
+
added the two together, while the JSON report kept them apart. The HTML
|
|
73
|
+
summary now shows a `timeout` count apart from `errored` too.
|
|
74
|
+
- **A test file given as a source after `--test` exits 2.** `--test` takes one
|
|
75
|
+
file, so in `mutineer run app/x.rb --test spec/a_spec.rb spec/b_spec.rb` the
|
|
76
|
+
second spec became a source to mutate, and the run tested with one spec file
|
|
77
|
+
without a word. Now a source named `*_test.rb` or `*_spec.rb` inside a
|
|
78
|
+
`test/` or `spec/` directory, given with `--test`, is a usage error that says
|
|
79
|
+
to repeat `--test`. The usage line now reads `--test <test> [--test <test>...]`.
|
|
80
|
+
|
|
81
|
+
### Fixed
|
|
82
|
+
- **The RSpec runner keeps what gems configured before the run.** It called
|
|
83
|
+
`RSpec.reset`, which drops the whole configuration. Under `rails: true`, a
|
|
84
|
+
`--boot` file or `require:`, the app's gems load first, so a setting such as
|
|
85
|
+
rspec-retry's `verbose_retry` was gone, and a support file that set it
|
|
86
|
+
raised `NoMethodError`. The run stopped with "the unmutated suite is not
|
|
87
|
+
green". The runner now calls `RSpec.clear_examples`, which keeps the
|
|
88
|
+
configuration. Hooks that gems added now apply too: an example with
|
|
89
|
+
rspec-retry's `retry: 3` runs up to 3 times per mutant, as under `rspec`.
|
|
90
|
+
- **`--since` no longer scores a file that Git ignores** (#168). A file
|
|
91
|
+
outside the index is scored in full, so a new file is not missed. That
|
|
92
|
+
check also caught files that Git ignores on purpose (`.gitignore`,
|
|
93
|
+
`.git/info/exclude` or the global excludes file), such as generated code
|
|
94
|
+
inside a directory run. Those files now get no mutants under `--since`,
|
|
95
|
+
and a warning names each one. Other untracked files are still scored in
|
|
96
|
+
full, and so is a file when Git cannot tell, such as one beyond a symlink or
|
|
97
|
+
inside a submodule.
|
|
98
|
+
- **The human report escapes control characters from the source** (#163).
|
|
99
|
+
A surviving line that held a terminal control byte (for example ESC) was
|
|
100
|
+
printed as that byte, so it could change what the terminal showed. The
|
|
101
|
+
token, the replacement, the diff lines, method names, file paths and the
|
|
102
|
+
kill matrix's test names now print each control character except tab as
|
|
103
|
+
its Ruby escape (`\e`), and a byte that is not valid UTF-8 (a Latin-1
|
|
104
|
+
source) as `\xNN`. The JSON and HTML
|
|
105
|
+
reports already escaped text and do not change.
|
|
106
|
+
- **One edit on a line is scored once** (#159). Some opt-in operators emitted
|
|
107
|
+
two mutants that give the same source: `literal_mutation` on `0` (the
|
|
108
|
+
"change to 1" and "add 1" rules), `negation_removal` on `!!x` (either `!`
|
|
109
|
+
removed), and `chain_link` or `operand_removal` on a repeated part such as
|
|
110
|
+
`a.b.b.c` or `x || x`. Each copy had its own id, so one surviving edit
|
|
111
|
+
counted twice. Now only the first copy on a line runs. Copies that start
|
|
112
|
+
on different lines (a multi-line chain) both stay, so `--since` never
|
|
113
|
+
loses the edit. Ids are assigned before the copy is dropped, so every kept
|
|
114
|
+
mutant keeps its id, and the dropped copy's id simply stops appearing.
|
|
115
|
+
Default runs do not change. If you use these operators with `--baseline`,
|
|
116
|
+
regenerate the baseline after upgrading: the score can move with no code
|
|
117
|
+
change, and in a full (unscoped) comparison a dropped copy that survived
|
|
118
|
+
is listed as fixed. An ignore entry for one copy's id still ignores only
|
|
119
|
+
that copy; the other copy still runs.
|
|
120
|
+
- **`# mutineer:disable-line` inside a string no longer silences a line**
|
|
121
|
+
(#158). The marker was found with a text search, so a string such as
|
|
122
|
+
`"# mutineer:disable-line"` ignored every mutant on its line and left
|
|
123
|
+
them out of the score. Only a real `#` comment counts now. A heredoc or
|
|
124
|
+
regex that holds the text is ignored too.
|
|
125
|
+
- **`./lib/x.rb` and an absolute path pair with a test** (#104). Auto-pairing
|
|
126
|
+
found the test for `lib/calc.rb` but not for `./lib/calc.rb` or the
|
|
127
|
+
absolute path to the same file. It reported "no test found by convention"
|
|
128
|
+
and exited 2. A file argument inside the project is now made relative to
|
|
129
|
+
the project root first, so equivalent spellings (`./`, `..`, absolute)
|
|
130
|
+
pair with the same test and run once. A symlink keeps its own name, so it
|
|
131
|
+
stays a separate source from its target. Reports show that root-relative
|
|
132
|
+
path. A path outside the project stays as typed.
|
|
133
|
+
- **A mutant on another line of a multi-line statement runs its tests**
|
|
134
|
+
instead of being reported as `no_coverage`. Ruby counts one line of a
|
|
135
|
+
statement only. When the mutant's own line has no count, the runner now uses
|
|
136
|
+
the tests that ran the statement that holds the mutation. A heredoc body
|
|
137
|
+
belongs to its statement. This fallback skips a first or last line that also
|
|
138
|
+
holds other code, such as the `def` line, and the body of `x while c` and
|
|
139
|
+
`x until c`. It also gives no tests to a mutant in code of the statement
|
|
140
|
+
that runs only sometimes: a `when` or `in` condition, a `rescue` class
|
|
141
|
+
list, a parameter default, the rescue side of `x rescue y`, the value of
|
|
142
|
+
`||=` or `&&=`, the operand of `defined?`, the right side of `a ||` or
|
|
143
|
+
`a &&`, a branch of `x if c`, `x unless c`, a ternary or `else`, the
|
|
144
|
+
pattern of `v in p` or `v => p`, an
|
|
145
|
+
argument or block of `x&.m(...)`, and the value of `x&.m += v`. Those
|
|
146
|
+
mutants stay `no_coverage`, as before.
|
|
147
|
+
- **A survivor diff for a multi-line mutant applies with `git apply`** (#106).
|
|
148
|
+
The JSON `diff` header always read `@@ -N +N @@`, even when the mutant
|
|
149
|
+
removed two lines and added one, so `git apply` rejected the patch as
|
|
150
|
+
corrupt. The header now carries each side's line count (`@@ -3,2 +3 @@`).
|
|
151
|
+
A one-line diff keeps its old header. A mutant that empties a line now
|
|
152
|
+
shows that empty line as a `+` line. A side whose last line has no final
|
|
153
|
+
newline gets ``. Lines of a CRLF file now keep
|
|
154
|
+
their `\r`, so the patch applies to that file.
|
|
155
|
+
- **Split test files pair with their source** (#87). A source such as
|
|
156
|
+
`app/foo/bar.rb` now also uses `test/foo/bar_upsert_test.rb` and
|
|
157
|
+
`test/foo/bar_guards_test.rb`, together with `test/foo/bar_test.rb` when
|
|
158
|
+
that file exists. The same names under `test/lib/` pair with a `lib/`
|
|
159
|
+
source. Before, only the exact `bar_test.rb` name was tried, so a suite
|
|
160
|
+
split across `bar_*_test.rb` files was skipped and a gate could pass with
|
|
161
|
+
no mutants run. A spec the old rules already find is unchanged. The
|
|
162
|
+
split name is left for `user_session.rb` when that file exists, so
|
|
163
|
+
`user.rb` does not run `user_session_test.rb`. The same rule holds outside
|
|
164
|
+
`app/` and `lib/`: `src/user.rb` does not take `test/src/user_session_test.rb`
|
|
165
|
+
when `src/user_session.rb` exists. A longer source claims a split file
|
|
166
|
+
only when that source itself searches the directory the file is in.
|
|
167
|
+
A failed capture of `test/foo/bar_upsert_test.rb` can mark `app/foo/bar.rb`
|
|
168
|
+
uncapturable only when no successful capture covered that source and
|
|
169
|
+
`bar_upsert.rb` does not exist. A test outside that directory does not,
|
|
170
|
+
and neither does a source that only shares the test basename.
|
|
171
|
+
`user_session__test.rb` stays with `user.rb` when `user_session.rb`
|
|
172
|
+
exists, because that name is not the exact test or a split test for
|
|
173
|
+
`user_session`. A failed exact spec in the mirrored `spec/` or `spec/lib/` directory
|
|
174
|
+
marks an `app/` or `lib/` source the same way. A failed spec in another
|
|
175
|
+
directory does not. A project rooted at `/` still finds split tests
|
|
176
|
+
inside that root. A stored failure keeps the pairing from the moment it
|
|
177
|
+
was checked. Adding or deleting a longer source rebuilds the coverage
|
|
178
|
+
map, so that failure is not moved onto the shorter source.
|
|
179
|
+
- **A blank `operators` key no longer runs zero mutants and exits 0** (#167).
|
|
180
|
+
`operators:` with no value became an empty list. The run made no mutants
|
|
181
|
+
and the gate passed. A blank operator list is now an error, and so is a
|
|
182
|
+
list of only unknown operator names. An empty `require` or `ignore` list
|
|
183
|
+
is still valid. `--operators` on the command line still replaces the
|
|
184
|
+
file list, including a blank or unknown file list.
|
|
185
|
+
- **The daemon no longer runs the tool's Ruby settings inside the app** (#100).
|
|
186
|
+
Open3 keeps a variable when the spawn hash omits it, so a parent `RUBYOPT`
|
|
187
|
+
ran inside the app. The child now gets the environment Bundler saved before
|
|
188
|
+
it activated, passed with `unsetenv_others`. The app keeps its gem home,
|
|
189
|
+
excluded groups, Bundler config, and chruby path. An `RBENV_VERSION` or
|
|
190
|
+
`ASDF_RUBY_VERSION` already in the environment stays. That pin is often
|
|
191
|
+
the Ruby that started the tool. `.ruby-version` applies only when no pin
|
|
192
|
+
is set. The tool's `RUBYOPT` and `RUBYLIB` do not reach the child, including
|
|
193
|
+
a value Bundler had saved. A gem home, `BUNDLE_PATH`, or `BUNDLE_WITHOUT`
|
|
194
|
+
reaches the child only when Bundler saved the app's value. The same
|
|
195
|
+
rule covers `GEM_PATH` and `BUNDLER_VERSION`. An rbenv version
|
|
196
|
+
bin under `RBENV_ROOT`, or under `~/.rbenv` when that variable is unset,
|
|
197
|
+
leaves `PATH` only when that root has a shim directory. An asdf version
|
|
198
|
+
bin leaves only when an asdf shim directory exists, including
|
|
199
|
+
`ASDF_DATA_DIR/shims`. Without that shim, the bin stays and `bundle` can
|
|
200
|
+
still be found. The app Gemfile, an explicit Ruby pin, and Rails env are
|
|
201
|
+
applied after that.
|
|
202
|
+
- **The Action refuses to run when `output` and `baseline` are the same file** (#160).
|
|
203
|
+
That setup used to work, and a failing run copied its report onto the
|
|
204
|
+
baseline. The next run then treated the failed report as the baseline, so
|
|
205
|
+
the gate could pass. The Action now stops before the run and exits 2.
|
|
206
|
+
The baseline may come from the `baseline` input, from `--baseline` in
|
|
207
|
+
`extra-args`, or from `.mutineer.yml`. A numeric path such as `123` is
|
|
208
|
+
read the same way as the CLI. Write the report to a separate path. Copy
|
|
209
|
+
that file onto the baseline in a later step, and only when this step
|
|
210
|
+
exits 0. You can also refresh the baseline with the CLI, and keep the
|
|
211
|
+
file only when the run exits 0. The next release moves the `v1` tag, so
|
|
212
|
+
workflows on `@v1` get this refusal then.
|
|
213
|
+
- **A mutation no longer leaves a heredoc body behind as Ruby** (#134).
|
|
214
|
+
Replacing a node deletes the heredoc opener and not the body. The body
|
|
215
|
+
then parses as code, raises, and counts as a kill. A mutation that drops
|
|
216
|
+
the opener is skipped. A replacement that still contains the opener stays.
|
|
217
|
+
- **A bare `public`, `private` or `protected` ends `module_function` mode**,
|
|
218
|
+
so later `def`s in the module body are named as instance methods, not
|
|
219
|
+
singleton methods (#144).
|
|
220
|
+
- **Turning on reduced motion mid-visit finishes the landing page count-ups**,
|
|
221
|
+
so no number keeps changing after the visitor asks for less motion (#176).
|
|
222
|
+
- **`rake site:build` can no longer delete a directory such as `lib/` or
|
|
223
|
+
`.git`** (#162). The task removed the destination it was given, and only
|
|
224
|
+
refused the checkout, its parents and `docs/`. It now always builds into
|
|
225
|
+
`_site/` at the checkout root, and `rake "site:build[DEST]"` and
|
|
226
|
+
`SITE_BUILD_DEST` are gone.
|
|
227
|
+
- **Documentation matches the 1.4 behavior.** Config examples use supported keys,
|
|
228
|
+
agent loops check the exit code and a real score, and report totals explain
|
|
229
|
+
partial `--fail-fast` runs. The README lists every config key and its validation
|
|
230
|
+
rules. API README links work outside GitHub, repeated changelog headings have
|
|
231
|
+
unique anchors, and the website checks validate built links and anchors.
|
|
232
|
+
Mutant-id wording and the gem's Minitest/RSpec description are also corrected.
|
|
233
|
+
- **A mutant in a nested method counts once** (#157). A `def` inside
|
|
234
|
+
another method is a subject of its own, but most operators also mutated
|
|
235
|
+
its code on the outer method. The same edit then counted twice in the
|
|
236
|
+
score, with two ids. Every operator now skips a nested `def` when it
|
|
237
|
+
mutates the outer method. A `def` inside `class << obj`, where `obj` is
|
|
238
|
+
not `self`, is not a subject, so the outer method still mutates its code.
|
|
239
|
+
`array_literal`, `chain_link`, `condition_true`, `condition_false`,
|
|
240
|
+
`operand_removal`, `operator_assignment` and `return_nil` skipped that
|
|
241
|
+
code before. Their new mutants there can show as new survivors against a
|
|
242
|
+
baseline.
|
|
243
|
+
|
|
244
|
+
A mutant id includes an ordinal among the mutants with the same operator
|
|
245
|
+
and token in a subject. When the duplicates leave the outer method, the
|
|
246
|
+
ordinals of its own mutants change. An old duplicate id can then name a
|
|
247
|
+
different mutant on the outer method, and an `ignore:` entry or a
|
|
248
|
+
baseline entry with that id applies to that mutant with no warning. If
|
|
249
|
+
your code has nested methods, regenerate the `ignore:` entries and the
|
|
250
|
+
baseline for the outer methods.
|
|
251
|
+
|
|
252
|
+
Under `--strategy redefine` (the `--rails` default), a mutant in a nested
|
|
253
|
+
method can survive or error even when a test would catch it. `redefine`
|
|
254
|
+
loads only the inner method, and a call to the outer method defines the
|
|
255
|
+
original inner method again. Before, the copy of the edit on the outer
|
|
256
|
+
method was killed. Now each edit has one verdict, from the inner method.
|
|
257
|
+
|
|
9
258
|
## [1.4.0] - 2026-09-30
|
|
10
259
|
|
|
11
260
|
### Added
|
|
@@ -677,6 +926,7 @@ Rails hardening + CI batch (issues #8–#13), all verified Rails-free.
|
|
|
677
926
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
678
927
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
679
928
|
|
|
929
|
+
[1.5.0]: https://github.com/davidteren/mutineer/releases/tag/v1.5.0
|
|
680
930
|
[1.4.0]: https://github.com/davidteren/mutineer/releases/tag/v1.4.0
|
|
681
931
|
[1.3.0]: https://github.com/davidteren/mutineer/releases/tag/v1.3.0
|
|
682
932
|
[1.2.0]: https://github.com/davidteren/mutineer/releases/tag/v1.2.0
|
data/README.md
CHANGED
|
@@ -9,12 +9,19 @@ change at a time, runs your test suite (Minitest or RSpec) against each mutant,
|
|
|
9
9
|
ones your tests failed to catch — the gaps where your suite isn't actually
|
|
10
10
|
testing anything.
|
|
11
11
|
|
|
12
|
+
"Clean-room" means Mutineer was written from scratch. It contains no code from
|
|
13
|
+
[Mutant](https://github.com/mbj/mutant) or any other mutation-testing tool.
|
|
14
|
+
Mutineer's own code is licensed under MIT. See
|
|
15
|
+
[Mutineer and Mutant](https://github.com/davidteren/mutineer#mutineer-and-mutant)
|
|
16
|
+
for how the two tools differ.
|
|
17
|
+
|
|
12
18
|
- **Prism + stdlib only** — zero runtime dependencies (Ruby ≥ 3.4).
|
|
13
19
|
- **One mutation per mutant**, validity-checked by re-parsing.
|
|
14
20
|
- **Fork-isolated**, parallel execution (Linux + macOS).
|
|
15
21
|
- **Coverage-guided** — each mutant runs only the test files that cover its line.
|
|
16
22
|
- **Stops at the first failing test** — in-process runs (not `--daemon` or
|
|
17
|
-
`--test-command`) stop a mutant's test run at the first failure
|
|
23
|
+
`--test-command`) stop a mutant's test run at the first failure, unless
|
|
24
|
+
`--matrix` asks for every covering test.
|
|
18
25
|
|
|
19
26
|
📖 **[mutineer.github.io →](https://davidteren.github.io/mutineer/)** — overview, operators, and usage.
|
|
20
27
|
|
|
@@ -33,7 +40,7 @@ gem "mutineer", group: :test
|
|
|
33
40
|
## Usage
|
|
34
41
|
|
|
35
42
|
```sh
|
|
36
|
-
mutineer run <source...> --test <test
|
|
43
|
+
mutineer run <source...> --test <test> [--test <test>...] [options]
|
|
37
44
|
```
|
|
38
45
|
|
|
39
46
|
Mutate `lib/calculator.rb`, checking it against its test, and fail CI if the
|
|
@@ -47,24 +54,30 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
|
|
|
47
54
|
|
|
48
55
|
| Flag | Meaning |
|
|
49
56
|
|------|---------|
|
|
50
|
-
| `--test FILE` | Test file covering the sources (
|
|
57
|
+
| `--test FILE` | Test file covering the sources; one file per flag, so repeat it for each (`--test a_test.rb --test b_test.rb`) |
|
|
51
58
|
| `--operators LIST` | Comma-separated operator names (default: the Tier-1 set) |
|
|
52
59
|
| `--threshold FLOAT` | Exit 1 when the score is below FLOAT, or when nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted (default: 0 = off) |
|
|
53
60
|
| `--only NAME` | Restrict to one fully-qualified subject, e.g. `Calculator#add` |
|
|
54
61
|
| `--framework NAME` | `minitest` (default) or `rspec`; auto-detected as rspec when most `--test` files end in `_spec.rb` |
|
|
55
|
-
| `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`)
|
|
62
|
+
| `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`). Untracked files are scored in full unless Git ignores them. |
|
|
56
63
|
| `--no-since` | Disable diff scoping; a typed no beats a `.mutineer.yml` `since:` key |
|
|
57
|
-
| `--baseline FILE` | Compare against a prior `--format json` run; exit 1 on new survivors / score drop (score drop is skipped under `--since`, whose score covers a different denominator; see [CI](#ci-gating)) |
|
|
64
|
+
| `--baseline FILE` | Compare against a prior `--format json` run; exit 1 on new survivors / score drop (score drop is skipped under `--since`, whose score covers a different denominator; see [CI](https://github.com/davidteren/mutineer#ci-gating)) |
|
|
58
65
|
| `--baseline-epsilon FLOAT` | Score-drop tolerance for `--baseline` (default: 0) |
|
|
59
|
-
| `--jobs N` | Parallel worker count (default: processor count; `1`
|
|
66
|
+
| `--jobs N` | Parallel worker count (default: processor count); forced to `1` by `--test-command`, `--fail-fast`, or `--rails` without `--daemon` |
|
|
67
|
+
| `--boot FILE` | Require an app entry point once before forking; select at least one test file |
|
|
68
|
+
| `--rails` | Boot `config/environment` and reconnect ActiveRecord per fork; without `--daemon`, defaults to `redefine` and runs serially |
|
|
60
69
|
| `--verbose` | Surface the real error when a fork capture fails (alias `--debug`) |
|
|
61
70
|
| `--strategy NAME` | Mutation application: `reload` whole-file (default) or `redefine` surgical (`7a`/`7b` accepted as deprecated aliases) |
|
|
62
|
-
| `--
|
|
63
|
-
| `--
|
|
71
|
+
| `--timeout SECONDS` | Per-mutant time limit for in-process runs, in whole seconds (default: 10). A mutant whose tests run longer is a `timeout`; see [Timeouts and the score](https://github.com/davidteren/mutineer#timeouts-and-the-score). `--daemon` and `--test-command` keep their own limits |
|
|
72
|
+
| `--capture-timeout SECONDS` | Time limit for each in-process coverage-capture subprocess and for the clean run of the unmutated tests, in whole seconds (default: 120). A suite slower than this stops the run as not green. `--daemon` uses it for its coverage capture; `--test-command` ignores it and says so |
|
|
73
|
+
| `--cache-dir DIR` | Directory for the coverage cache (default: `.mutineer`). Give two runs from the same project root different directories so they do not share `coverage.json`. `--test-command` builds no cache, so it ignores it and says so |
|
|
74
|
+
| `--test-command CMD` | Run the suite as a subprocess in the app's own runtime (for apps on Ruby < 3.4); `CMD` must contain `%{files}`. See [Apps on Ruby < 3.4](https://github.com/davidteren/mutineer#apps-on-ruby--34) |
|
|
75
|
+
| `--daemon` | Boot the app once in a persistent daemon and fork per mutant, with per-worker DB isolation so `--jobs N` is safe under Rails (needs `--rails`/`--boot`; not with `--test-command`). See [the daemon backend](https://github.com/davidteren/mutineer#faster-parallel-safe-rails-the---daemon-backend) |
|
|
64
76
|
| `--format human\|json\|html` | Report format (default: human; `html` is a self-contained file) |
|
|
65
77
|
| `--output FILE` | Write the report to FILE instead of stdout |
|
|
66
78
|
| `--dry-run` | List candidate mutations without executing (honors suppression) |
|
|
67
|
-
| `--fail-fast` | Stop at the first surviving mutant |
|
|
79
|
+
| `--fail-fast` | Stop at the first surviving mutant (`--no-fail-fast` beats a `.mutineer.yml` `fail_fast:` key) |
|
|
80
|
+
| `--matrix` | Run every covering test for each mutant and report the blind and redundant tests; the JSON report also lists each mutant's killers. In-process only; exits 2 with `--daemon`, `--test-command`, `--fail-fast` or `--dry-run`. RSpec needs 3.3 or later. `--no-matrix` beats a `.mutineer.yml` `matrix:` key. See [Kill matrix](https://github.com/davidteren/mutineer#kill-matrix) |
|
|
68
81
|
| `--list-operators` | List available operators (default vs optional) and exit |
|
|
69
82
|
| `--version`, `--help` | Print version / usage and exit |
|
|
70
83
|
|
|
@@ -99,8 +112,8 @@ RAILS_ENV=test bundle exec mutineer run \
|
|
|
99
112
|
```
|
|
100
113
|
|
|
101
114
|
`--rails` boots `config/environment` once in the parent process (every mutant
|
|
102
|
-
then forks and inherits it), defaults `--strategy` to `redefine`
|
|
103
|
-
avoids reloading files into the app tree), and reconnects ActiveRecord in each
|
|
115
|
+
then forks and inherits it), defaults `--strategy` to `redefine` without `--daemon`
|
|
116
|
+
(surgical — it avoids reloading files into the app tree; `--daemon` keeps `reload`), and reconnects ActiveRecord in each
|
|
104
117
|
fork so the database connection is fork-safe. Use `--boot FILE` to boot a
|
|
105
118
|
different entry point. Boot mode requires at least one `--test` file and is
|
|
106
119
|
coverage-guided — each mutant runs only the test files that exercise its line
|
|
@@ -135,9 +148,10 @@ RAILS_ENV=test bundle exec mutineer run \
|
|
|
135
148
|
- **One backend at a time** — `--daemon` can't be combined with `--test-command`
|
|
136
149
|
(choose one), and it needs an app to boot (`--rails` or `--boot`).
|
|
137
150
|
|
|
138
|
-
Status: **SQLite**
|
|
139
|
-
|
|
140
|
-
|
|
151
|
+
Status: **SQLite** only (hermetic, CI-proven). Per-worker provisioning for
|
|
152
|
+
**Postgres** and other adapters is not supported: on those, `--daemon` scores
|
|
153
|
+
every mutant as `error`. Use `--daemon` with a SQLite test database, or drop
|
|
154
|
+
`--daemon` to run serially on other adapters.
|
|
141
155
|
|
|
142
156
|
### Apps on Ruby < 3.4
|
|
143
157
|
|
|
@@ -207,7 +221,23 @@ Tradeoffs — this path is correct but not free:
|
|
|
207
221
|
front (a "smoke check") if your unmutated suite isn't green.
|
|
208
222
|
- **Reload strategy only** (`--strategy redefine` is rejected on this path) and
|
|
209
223
|
**serial** (`--jobs` is forced to 1). For apps on Ruby ≥ 3.4, `--daemon` gives
|
|
210
|
-
safe parallelism instead (see [the daemon backend](#faster-parallel-safe-rails-the---daemon-backend)).
|
|
224
|
+
safe parallelism instead (see [the daemon backend](https://github.com/davidteren/mutineer#faster-parallel-safe-rails-the---daemon-backend)).
|
|
225
|
+
|
|
226
|
+
## Timeouts and the score
|
|
227
|
+
|
|
228
|
+
The mutation score is `killed / (killed + survived)`. A mutant whose tests run
|
|
229
|
+
past `--timeout` is a `timeout`. It is neither killed nor survived, so it is
|
|
230
|
+
left out of the score, like `no_coverage`, `uncapturable`, `errored`, skipped
|
|
231
|
+
and ignored mutants. A timeout is not counted as a kill, because a hang the
|
|
232
|
+
mutant caused and a suite that is just slow look the same. The human report
|
|
233
|
+
shows the count in its `Timeout:` row, and the JSON report in `summary.timeout`.
|
|
234
|
+
|
|
235
|
+
Under `--threshold`, a timeout is a mutant with no verdict. The run exits 1 when
|
|
236
|
+
more than one mutant produced no verdict and they exceed 10% of those attempted,
|
|
237
|
+
or when nothing could be scored and something broke. If a slow suite times out
|
|
238
|
+
mutants that its tests would catch, raise `--timeout`. A limit set below what
|
|
239
|
+
the suite needs does not raise the score: it turns those mutants into timeouts,
|
|
240
|
+
and once they pass 10% of the attempted mutants the run fails.
|
|
211
241
|
|
|
212
242
|
## Suppressing equivalent mutants
|
|
213
243
|
|
|
@@ -268,6 +298,77 @@ Ids are relative to the directory you run mutineer from. mutineer finds
|
|
|
268
298
|
(other than your home directory), it warns that the ignore ids will not match
|
|
269
299
|
and tells you which directory to run from.
|
|
270
300
|
|
|
301
|
+
## Kill matrix
|
|
302
|
+
|
|
303
|
+
`--matrix` reports which tests kill each mutant, and from that, which tests
|
|
304
|
+
add nothing to the suite.
|
|
305
|
+
|
|
306
|
+
```sh
|
|
307
|
+
mutineer run lib/calculator.rb --test test/calculator_test.rb --matrix
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
For each mutant, Mutineer runs every test in its covering files instead of
|
|
311
|
+
stopping at the first failure, and records which tests failed. Coverage is
|
|
312
|
+
recorded per test file, so that set also holds tests that never reach the
|
|
313
|
+
mutated line; they pass and count as having run. A mutant's row is complete
|
|
314
|
+
when its whole covering set ran and the suite returned normally (under
|
|
315
|
+
Minitest, an `Interrupt` that cut the run short does not count as returning
|
|
316
|
+
normally). The report
|
|
317
|
+
names two kinds of test:
|
|
318
|
+
|
|
319
|
+
- **Blind:** the test ran in at least one complete row and killed no mutant
|
|
320
|
+
in any row. It would still pass with this run's mutations of the code, so
|
|
321
|
+
it guards none of them. It can still check behavior outside this run's
|
|
322
|
+
mutants (code in other files, or edits no operator makes), so look at what
|
|
323
|
+
it asserts before you delete it, or give it an assertion that can fail.
|
|
324
|
+
- **Redundant:** every mutant the test kills, another test kills too. That
|
|
325
|
+
makes it a candidate for deletion, one test at a time: two redundant tests
|
|
326
|
+
can be the only killers of one mutant, so re-run after each deletion.
|
|
327
|
+
|
|
328
|
+
The answers cover this run's mutants only. A test of code outside the sources
|
|
329
|
+
you passed kills nothing here, so mutate the code a test exercises before you
|
|
330
|
+
call the test blind.
|
|
331
|
+
|
|
332
|
+
The matrix changes no verdict, score or exit code: each mutant gets the
|
|
333
|
+
verdict a run without `--matrix` gives. That run skips every later test,
|
|
334
|
+
test class and example group after the first failure, so when a later one
|
|
335
|
+
exits the process, crashes or runs into the time limit, the mutant is still
|
|
336
|
+
`killed`. The run without `--matrix` still runs the code around the failing
|
|
337
|
+
test (the rest of its Minitest class `run` wrapper, the `after(:all)` hooks of
|
|
338
|
+
its RSpec groups) and the suite's own cleanup (RSpec's `after(:suite)`), so an
|
|
339
|
+
end there keeps the exit status. A failure in a Minitest `parallelize_me!`
|
|
340
|
+
class also keeps it: those run after every serial class and cannot be stopped
|
|
341
|
+
once queued, while a failure in a serial class still stops the run.
|
|
342
|
+
|
|
343
|
+
Under RSpec, `--matrix` needs RSpec 3.3 or later, whose example ids tell
|
|
344
|
+
apart examples on one line. With an older RSpec every row is incomplete.
|
|
345
|
+
|
|
346
|
+
The human report lists up to 20 blind and 20 redundant tests, and the HTML
|
|
347
|
+
report lists them all. `--format json` adds a `matrix` block with every test
|
|
348
|
+
and each mutant's killers (see the
|
|
349
|
+
[JSON schema](https://davidteren.github.io/mutineer/json-schema.html#matrix)).
|
|
350
|
+
A test is its file and its id. The name is `CalculatorTest#test_add` under
|
|
351
|
+
Minitest, or the example's full description under RSpec, where the example id
|
|
352
|
+
(`./spec/calc_spec.rb[1:2]`) tells apart examples that share a description
|
|
353
|
+
and keeps an example whose generated description changes with the mutant as
|
|
354
|
+
one test.
|
|
355
|
+
|
|
356
|
+
Each mutant runs its whole covering set, so every mutant costs what a survivor
|
|
357
|
+
costs. In 1.4, stopping at the first failure cut a full run of rack's
|
|
358
|
+
`lib/rack/utils.rb` from 86 to 89 seconds down to 35 to 41, so expect a matrix
|
|
359
|
+
run to take 2.1 to 2.5 times as long as a normal one. A mutant that reaches the
|
|
360
|
+
per-mutant time limit (`--timeout`, 10 seconds by default) after a test already failed stays `killed`
|
|
361
|
+
with its row marked incomplete. The report names each incomplete row and warns
|
|
362
|
+
that a blind test may have killed one of those mutants, so do not delete a
|
|
363
|
+
blind test until its rows are complete. Raise `--timeout` (or `timeout:` in
|
|
364
|
+
`.mutineer.yml`) to get complete rows.
|
|
365
|
+
|
|
366
|
+
`--matrix` runs on the in-process backend only. It exits 2 with `--daemon`,
|
|
367
|
+
`--test-command`, `--fail-fast` or `--dry-run`, and the message says whether
|
|
368
|
+
each setting came from the command line or `.mutineer.yml`. To run one of
|
|
369
|
+
those with `matrix: true` in the file, pass `--no-matrix`; to run `--matrix`
|
|
370
|
+
with `fail_fast: true` in the file, pass `--no-fail-fast`.
|
|
371
|
+
|
|
271
372
|
## CI gating
|
|
272
373
|
|
|
273
374
|
Store a JSON run as a baseline, then fail the build only when a PR makes things
|
|
@@ -277,10 +378,15 @@ worse:
|
|
|
277
378
|
mutineer run app/ --baseline .mutineer/baseline.json # exit 1 on NEW survivors or a score drop
|
|
278
379
|
```
|
|
279
380
|
|
|
280
|
-
`--baseline` reports which survivors are new (by [mutant id](#mutant-ids)) and any score drop. It
|
|
381
|
+
`--baseline` reports which survivors are new (by [mutant id](https://github.com/davidteren/mutineer#mutant-ids)) and any score drop. It
|
|
281
382
|
combines with `--threshold` (the worse of the two sets the exit code). Pass a
|
|
282
383
|
directory (or several sources) to audit a whole layer in one boot — tests are
|
|
283
|
-
auto-paired by convention and the report breaks down per source.
|
|
384
|
+
auto-paired by convention and the report breaks down per source. A source
|
|
385
|
+
`app/foo/bar.rb` pairs with `test/foo/bar_test.rb` and with unclaimed
|
|
386
|
+
`test/foo/bar_*_test.rb` files in that directory, such as `bar_upsert_test.rb`.
|
|
387
|
+
It does not take `user_session_test.rb` when `user_session.rb` exists in the same directory.
|
|
388
|
+
It does not take `bar_upsert_guards_test.rb` when `bar_upsert.rb` exists.
|
|
389
|
+
A spec that already pairs is left as that one file.
|
|
284
390
|
|
|
285
391
|
### GitHub Action
|
|
286
392
|
|
|
@@ -316,17 +422,17 @@ the format.
|
|
|
316
422
|
|
|
317
423
|
## For AI agents & pipelines
|
|
318
424
|
|
|
319
|
-
Mutineer is built for programmatic use — versioned JSON, [mutant ids](#mutant-ids) that survive unrelated edits,
|
|
425
|
+
Mutineer is built for programmatic use — versioned JSON, [mutant ids](https://github.com/davidteren/mutineer#mutant-ids) that survive unrelated edits,
|
|
320
426
|
structured exit codes, and diff-scoped runs. See:
|
|
321
427
|
|
|
322
428
|
- **AI agents & CI recipes** — the agent inner-loop and CI-gate recipes (and how
|
|
323
429
|
to avoid infinite loops on equivalent mutants):
|
|
324
430
|
[rendered](https://davidteren.github.io/mutineer/agentic-coding.html) ·
|
|
325
|
-
[source](
|
|
431
|
+
[source](https://davidteren.github.io/mutineer/agentic-coding.md)
|
|
326
432
|
- **JSON schema reference** — the `--format json` schema and its versioning
|
|
327
433
|
contract:
|
|
328
434
|
[rendered](https://davidteren.github.io/mutineer/json-schema.html) ·
|
|
329
|
-
[source](
|
|
435
|
+
[source](https://davidteren.github.io/mutineer/json-schema.md)
|
|
330
436
|
- **Ruby API (YARD)** — class reference for the shipped gem:
|
|
331
437
|
[https://davidteren.github.io/mutineer/api/](https://davidteren.github.io/mutineer/api/)
|
|
332
438
|
|
|
@@ -335,11 +441,44 @@ structured exit codes, and diff-scoped runs. See:
|
|
|
335
441
|
Mutineer reads an optional `.mutineer.yml` from the project root (nearest one,
|
|
336
442
|
walking up). CLI flags override config; config overrides defaults.
|
|
337
443
|
|
|
338
|
-
Sources are positional CLI arguments and test files come from `--test
|
|
339
|
-
config file accepts these keys:
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
444
|
+
Sources are positional CLI arguments and test files come from `--test`. The
|
|
445
|
+
config file accepts these keys:
|
|
446
|
+
|
|
447
|
+
| Key | Value and purpose |
|
|
448
|
+
|-----|-------------------|
|
|
449
|
+
| `operators` | An operator name or list of names; defaults to the Tier-1 set |
|
|
450
|
+
| `threshold` | A number from 0 to 100; 0 turns the score gate off |
|
|
451
|
+
| `jobs` | A positive integer; the default is the processor count. `test_command`, `fail_fast`, or `--rails` without `--daemon` forces 1. |
|
|
452
|
+
| `only` | A fully-qualified subject name, such as `Calculator#add` |
|
|
453
|
+
| `require` | A path or list of extra files to load before mutating |
|
|
454
|
+
| `boot` | The app entry point to require once before forking |
|
|
455
|
+
| `rails` | `true` or `false`; enables the Rails boot defaults |
|
|
456
|
+
| `since` | A nonblank git ref, or `false` to disable diff scoping |
|
|
457
|
+
| `framework` | `minitest` or `rspec`; an explicit value is kept during test pairing |
|
|
458
|
+
| `verbose` | `true` or `false`; shows capture diagnostics |
|
|
459
|
+
| `ignore` | A mutant id or list of ids to suppress |
|
|
460
|
+
| `baseline` | The path to a prior JSON report |
|
|
461
|
+
| `fail_fast` | `true` or `false`; stops scheduling after the first survivor |
|
|
462
|
+
| `test_command` | The external-runtime suite command, including `%{files}`; see [Apps on Ruby < 3.4](https://github.com/davidteren/mutineer#apps-on-ruby--34) |
|
|
463
|
+
| `daemon` | `true` or `false`; uses the persistent app daemon with worker DB isolation |
|
|
464
|
+
| `timeout` | A positive integer; the per-mutant time limit in seconds (default 10) |
|
|
465
|
+
| `capture_timeout` | A positive integer; the coverage-capture time limit in seconds (default 120) |
|
|
466
|
+
| `cache_dir` | The coverage cache directory (default `.mutineer`) |
|
|
467
|
+
| `matrix` | `true` or `false`; runs every covering test for each mutant and adds the kill matrix to the report |
|
|
468
|
+
|
|
469
|
+
Invalid values for known scalar keys, and a blank `operators` list, exit 2
|
|
470
|
+
with a message naming the file and key. An unknown operator name warns and
|
|
471
|
+
is skipped. If none of the names are known, the run exits 2. An empty
|
|
472
|
+
`require` or `ignore` list is valid. Boolean keys take `true` or `false` (quoted forms also work), not `"yes"`.
|
|
473
|
+
`jobs` must be positive; a string value contains digits only. String values for
|
|
474
|
+
`threshold` and the CLI-only `--baseline-epsilon` use plain decimals such as `90`
|
|
475
|
+
or `0.5`, not `+2`, `1e2`, or `1_0`. String options such as `only` and `baseline`
|
|
476
|
+
cannot be null or boolean. A blank `since` is invalid; use `since: false` to turn
|
|
477
|
+
scoping off. Unknown keys warn and are ignored. Unknown operator names warn and are skipped, and the run exits 2 when none remain. `--operators` replaces a blank or unknown file list.
|
|
478
|
+
|
|
479
|
+
`format`, `strategy`, `output`, `baseline_epsilon`, and `dry_run` are CLI-only.
|
|
480
|
+
For JSON output, use `--format json`, not a `format:` config key. To select RSpec
|
|
481
|
+
in the file, add `framework: rspec`.
|
|
343
482
|
|
|
344
483
|
```yaml
|
|
345
484
|
# .mutineer.yml
|
|
@@ -351,8 +490,40 @@ require:
|
|
|
351
490
|
```
|
|
352
491
|
|
|
353
492
|
Coverage results are cached in `.mutineer/coverage.json` (digest-keyed; rebuilt
|
|
354
|
-
automatically when sources change). Add `.mutineer/` to your `.gitignore
|
|
493
|
+
automatically when sources change). Add `.mutineer/` to your `.gitignore`, and
|
|
494
|
+
the directory you set with `--cache-dir` if it is inside the project. Keep
|
|
495
|
+
`.mutineer/` ignored even then: `--test-command` writes lock files to
|
|
496
|
+
`.mutineer/` beside each source file.
|
|
497
|
+
|
|
498
|
+
## Mutineer and Mutant
|
|
499
|
+
|
|
500
|
+
[Mutant](https://github.com/mbj/mutant), by Markus Schirp, is the established
|
|
501
|
+
mutation-testing tool for Ruby. It has been in development since 2012 and is
|
|
502
|
+
the subject of published research. Mutineer is a separate tool, and it shares
|
|
503
|
+
no code with Mutant.
|
|
504
|
+
|
|
505
|
+
| Aspect | Mutineer | Mutant |
|
|
506
|
+
|---|---|---|
|
|
507
|
+
| License | MIT, for every use | Free for open source (`--usage opensource`); commercial use needs a paid subscription |
|
|
508
|
+
| Runtime dependencies | None (Prism + stdlib) | `parser`, `unparser`, `regexp_parser`, `sorbet-runtime` and others |
|
|
509
|
+
| Ruby | 3.4 and later to run Mutineer; an app on an older Ruby works through `--test-command` (see [Apps on Ruby < 3.4](https://github.com/davidteren/mutineer#apps-on-ruby--34)) | 3.3 and later |
|
|
510
|
+
| Test frameworks | Minitest and RSpec, in one gem | RSpec, Minitest and Test::Unit, one integration gem each |
|
|
511
|
+
| What you mutate | Files (`mutineer run lib/foo.rb --test test/foo_test.rb`), narrowed to one method with `--only 'Foo#bar'` | Subjects named by expression (`mutant run 'Foo#bar'`, `'Foo*'`) |
|
|
512
|
+
| Which tests run | The test files whose coverage reaches the mutated line | The tests that declare the subject (an RSpec description or a Minitest `cover`), or the tests that ran it in a per-test coverage recording |
|
|
513
|
+
| Operators | 20 (5 by default, 15 opt-in) | A larger set; the default `light` set applies almost all of it (`full` adds `#==` to `#eql?`) |
|
|
514
|
+
| CI gating | `--threshold`, `--baseline` deltas, `--since`, and a GitHub Action | Incremental mode (`--since`) and a recorded session history |
|
|
515
|
+
| Machine-readable output | Versioned JSON report schema | Session JSON schema |
|
|
516
|
+
|
|
517
|
+
Choose Mutineer when you want an MIT-licensed tool with no extra gems in your
|
|
518
|
+
bundle, file-based runs, and a JSON contract built for CI gates and AI agents.
|
|
519
|
+
Choose Mutant when you need its deeper operator set, Test::Unit support, or
|
|
520
|
+
subject expressions that select whole namespaces. The two tools do not
|
|
521
|
+
conflict, so you can run both on one project.
|
|
522
|
+
|
|
523
|
+
This comparison describes Mutant 0.17 (checked on 2026-10-06). Check
|
|
524
|
+
[Mutant's README](https://github.com/mbj/mutant#readme) for its current
|
|
525
|
+
license terms and features.
|
|
355
526
|
|
|
356
527
|
## License
|
|
357
528
|
|
|
358
|
-
MIT — see [LICENSE](LICENSE).
|
|
529
|
+
MIT — see [LICENSE](https://github.com/davidteren/mutineer/blob/main/LICENSE).
|