mutineer 1.4.0 → 1.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +475 -0
- data/README.md +224 -28
- data/lib/mutineer/changed_lines.rb +24 -1
- data/lib/mutineer/cli.rb +117 -20
- data/lib/mutineer/config.rb +64 -13
- data/lib/mutineer/coverage_map.rb +533 -100
- data/lib/mutineer/daemon_backend.rb +51 -33
- data/lib/mutineer/daemon_client.rb +212 -22
- data/lib/mutineer/daemon_server.rb +74 -27
- data/lib/mutineer/external_backend.rb +2 -2
- data/lib/mutineer/file_swap.rb +1 -1
- data/lib/mutineer/isolation.rb +92 -6
- data/lib/mutineer/job_plan.rb +352 -0
- data/lib/mutineer/kill_channel.rb +284 -0
- data/lib/mutineer/kill_matrix.rb +88 -0
- data/lib/mutineer/minitest_integration/kill_recorder.rb +301 -0
- data/lib/mutineer/minitest_integration/outer_reporter.rb +126 -0
- data/lib/mutineer/minitest_integration/stop_at_first_failure.rb +14 -68
- data/lib/mutineer/minitest_integration.rb +75 -11
- data/lib/mutineer/mutant_id.rb +5 -4
- data/lib/mutineer/mutators/array_literal.rb +0 -7
- data/lib/mutineer/mutators/base.rb +63 -4
- data/lib/mutineer/mutators/chain_link.rb +0 -7
- data/lib/mutineer/mutators/collection_method.rb +1 -1
- data/lib/mutineer/mutators/condition_forcing.rb +0 -7
- data/lib/mutineer/mutators/operand_removal.rb +0 -7
- data/lib/mutineer/mutators/operator_assignment.rb +0 -7
- data/lib/mutineer/mutators/return_nil.rb +5 -11
- data/lib/mutineer/mutators/string_literal.rb +1 -1
- data/lib/mutineer/orphan_guard.rb +28 -0
- data/lib/mutineer/pairing.rb +222 -15
- data/lib/mutineer/parser.rb +12 -3
- data/lib/mutineer/project.rb +344 -21
- data/lib/mutineer/rails_worker_db.rb +83 -9
- data/lib/mutineer/reporter.rb +306 -53
- data/lib/mutineer/result.rb +73 -5
- data/lib/mutineer/runner.rb +123 -242
- data/lib/mutineer/statement_lines.rb +183 -0
- data/lib/mutineer/subject.rb +6 -2
- data/lib/mutineer/test_runners/minitest.rb +4 -2
- data/lib/mutineer/test_runners/rspec.rb +282 -3
- data/lib/mutineer/test_runners.rb +5 -3
- data/lib/mutineer/version.rb +1 -1
- data/lib/mutineer/worker_pool.rb +4 -2
- data/lib/mutineer.rb +2 -0
- metadata +10 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: eeac45268c75073f522daa5d26dd65fe425ae66b604cfb7543c0b529ab71f2e6
|
|
4
|
+
data.tar.gz: 834755a3b106f02013ef75b9515f18e692d1c6ac219bee42b837da8a6f696fc7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: '092252eef7f756e5fa117784b7c3696777127b0f1f1b96825c38d99a9f8d362a23416ad2351685e62bd780563a2b7fa2eab0e926c10bee3e293293ff88e0d28d'
|
|
7
|
+
data.tar.gz: 02344ad3f678a639a37f0272577e50e5f189e585a9246be0b0f1451202a03c63cb57e013181e349ab7b4db04f36fb2333e2de7d864063c4da92d4394f5dce6b3
|
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,479 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [1.6.0] - 2026-10-08
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- **An `unplaceable` mutant status.** Under `--strategy redefine`, a mutant in
|
|
14
|
+
a method whose class or module has no constant name is not run. It is left
|
|
15
|
+
out of the score and, unlike `uncapturable`, out of `no_verdict`, so it does
|
|
16
|
+
not fail `--threshold`. The human report gives it an `Unplaceable:` row, the
|
|
17
|
+
HTML report a count, and the JSON report `summary.unplaceable` and an
|
|
18
|
+
`unplaceable[]` list; `schema_version` is `1.7`.
|
|
19
|
+
- **The agent skill installs with `gh skill` and `npx skills` (#125).** It
|
|
20
|
+
moved from `docs/skill.md` to `skills/mutineer/SKILL.md`, the layout of the
|
|
21
|
+
Agent Skills specification, so `gh skill install davidteren/mutineer
|
|
22
|
+
mutineer` finds it and `npx skills add davidteren/mutineer --skill mutineer`
|
|
23
|
+
installs one `SKILL.md` instead of the whole `docs/` site. The site build
|
|
24
|
+
copies that file to `skill.md`, so its URL still works. The skill's
|
|
25
|
+
description now says when to use it, and the README shows how to install it.
|
|
26
|
+
- **A `ran_at_load` mutant status** (#187). A mutant on a method-body line
|
|
27
|
+
that ran while the app booted or its class loaded gets this status when it
|
|
28
|
+
survives, or when only the load ran the line. A kill stays a kill. It is
|
|
29
|
+
left out of the score and out of `no_verdict`, so it does not fail
|
|
30
|
+
`--threshold`. The human report gives it a `Ran at load:` row that points
|
|
31
|
+
to `--test-command` to verify these mutants, the HTML report a count, and
|
|
32
|
+
the JSON report `summary.ran_at_load` and a `ran_at_load[]` list, within
|
|
33
|
+
schema `1.7`.
|
|
34
|
+
|
|
35
|
+
### Changed
|
|
36
|
+
|
|
37
|
+
- **Mutants no longer run under Coverage with `--boot` or `--rails` (#228).** Coverage
|
|
38
|
+
starts before the app boots so the coverage map can be built, and it used to
|
|
39
|
+
keep running in every mutant fork after that, though nothing read it. It is
|
|
40
|
+
now suspended once the map is built, from a fresh capture or from the cache.
|
|
41
|
+
Coverage that a host process started before the run is left running.
|
|
42
|
+
On a small app whose test calls tiny methods in a loop, the CPU time of a
|
|
43
|
+
whole in-process `--boot` run fell from 0.76s to 0.57s (median of five). On
|
|
44
|
+
the Rails fixture app the run is mostly boot, and the change is within noise
|
|
45
|
+
(1.61s before, 1.60s after). `--daemon` runs its mutants in worker daemons
|
|
46
|
+
that never start Coverage, so its time does not change (3.48s before, 3.44s
|
|
47
|
+
after); the map-building daemon now suspends Coverage after the map too.
|
|
48
|
+
- **A mutant runs its paired test files first, then the cheapest (#203).** The
|
|
49
|
+
covering test files used to run in the order the coverage map stored them,
|
|
50
|
+
so a mutant that one fast file kills could still score `timeout` when slow
|
|
51
|
+
files ran first. Now the files that pairing finds for the source run first,
|
|
52
|
+
then the rest from fastest to slowest by the time each file took during
|
|
53
|
+
coverage capture, less the fastest file's time (the startup cost each
|
|
54
|
+
capture pays). The order is fixed for a given cache. Costs compare in
|
|
55
|
+
doubling buckets, so files of close cost keep their path order when a
|
|
56
|
+
rebuilt cache measures them again; a file near a bucket edge can still move,
|
|
57
|
+
so keep `.mutineer/` between CI runs for the most stable `--baseline`. The
|
|
58
|
+
order applies to Minitest (after the seeded class shuffle, with
|
|
59
|
+
`parallelize_me!` classes still last) and to RSpec. `--daemon` runs every
|
|
60
|
+
covering test with no stop, so the order does not change its verdicts. The
|
|
61
|
+
coverage cache now saves these timings, so a cache from an earlier version
|
|
62
|
+
rebuilds once. A complete run gives the same verdicts and `--matrix` rows,
|
|
63
|
+
but the new order can change which mutants time out, so a `--baseline` gate
|
|
64
|
+
can see a one-time change on upgrade.
|
|
65
|
+
- **Job selection moved from `Mutineer::Runner` to a new `Mutineer::JobPlan`
|
|
66
|
+
module (#75).** The in-process runner, the `--daemon` backend and
|
|
67
|
+
`--dry-run` share it, so each file now requires what it uses and no
|
|
68
|
+
`require_relative` cycle exists in `lib/mutineer`. The moved methods are
|
|
69
|
+
`collect_jobs`, `filter_since`, `coverage_selection`, `ran_at_load?`,
|
|
70
|
+
`load_verdict`, `abort_if_unclean!`, `test_load_roots`, `source_dirs`,
|
|
71
|
+
`sweep_orphans`, `suppress_map`, `suppressed?` and `result_keys`. These were
|
|
72
|
+
internal methods, so `Runner` keeps no copies. Runs, reports and caches do
|
|
73
|
+
not change.
|
|
74
|
+
|
|
75
|
+
### Fixed
|
|
76
|
+
|
|
77
|
+
- **An endless method that only the tests call is no longer `no_coverage`
|
|
78
|
+
under `--rails`, `--boot` or `--daemon` (#209).** Ruby counts no line when
|
|
79
|
+
an endless method such as `def half(x) = x / 2` runs, only its `def` line
|
|
80
|
+
when it is defined. The boot defines it before any test runs, so the forked
|
|
81
|
+
coverage capture never saw that line run. Coverage now also
|
|
82
|
+
counts methods (`Coverage.start(lines: true, methods: true)`), and a test
|
|
83
|
+
that calls such a method covers its `def` line. Coverage caches from earlier
|
|
84
|
+
versions rebuild once.
|
|
85
|
+
- **`module_function :name` in a reopened module inside `class << self`
|
|
86
|
+
promotes the method from the earlier opening (#216).** Ruby treats both
|
|
87
|
+
openings as one module, but Mutineer matched them only within one body, so
|
|
88
|
+
it named the method `#<Class:App>::M#c` instead of `#<Class:App>::M.c`.
|
|
89
|
+
The two openings now match by the module's full name. A module whose name
|
|
90
|
+
Mutineer cannot be sure of still matches only within its own body: a path
|
|
91
|
+
such as `self::X` or `Foo::X` written directly inside `class << self`, a
|
|
92
|
+
name under an anonymous class, or a module in a `class << self` inside a
|
|
93
|
+
method or a `class_eval` block. A `module self::X` in a block, or a
|
|
94
|
+
`self::X = ...` in a `class_eval` block, matches only openings in the same
|
|
95
|
+
block. A `self::X` like these now also stays apart from a
|
|
96
|
+
module of the same name outside `class << self`, where a `module_function`
|
|
97
|
+
in that module used to promote its methods.
|
|
98
|
+
- **A `module self::X` or `class self::X` in a block is `unplaceable` under
|
|
99
|
+
`--strategy redefine` (#229).** In a block, `self` is not always the
|
|
100
|
+
enclosing class: in `Other.class_eval do` it is `Other`, and in
|
|
101
|
+
`Foo = Class.new do` it is the new class. Mutineer gave these modules a
|
|
102
|
+
known owner on the enclosing namespace, so redefine loaded the mutant onto
|
|
103
|
+
a module the tests never use, and the mutant could falsely survive or
|
|
104
|
+
score `error`. Such a module, everything nested in it, and a
|
|
105
|
+
`self::X = ...` in a block that does not build a class now have
|
|
106
|
+
their owner unknown: redefine reports their mutants as `unplaceable`, and
|
|
107
|
+
`--strategy reload` still runs them. In a `Data.define`, `Struct.new`,
|
|
108
|
+
`Class.new` or `Module.new` block, the module is now named under the
|
|
109
|
+
builder's constant (`Outer::Foo::X#c`, was `Outer::X#c`), so the subject
|
|
110
|
+
names and mutant ids of those methods change. `self::X` directly in a class
|
|
111
|
+
or module body is unchanged.
|
|
112
|
+
|
|
113
|
+
- **`--daemon` worker databases start as a copy of the test database
|
|
114
|
+
(#222).** Each worker slot's SQLite database used to start from
|
|
115
|
+
`db/schema.rb` alone. Rows the daemon wrote while it booted, in
|
|
116
|
+
initializers or `--require` files, were only in the base test database, so
|
|
117
|
+
a test that read them passed in-process but failed under `--daemon`. On its
|
|
118
|
+
first use, a worker database is now a copy of the base test database (schema
|
|
119
|
+
and rows, made with the SQLite online backup API). The worker still loads
|
|
120
|
+
`db/schema.rb` when the copy's schema differs from it (another schema
|
|
121
|
+
version, or another or no `schema_sha1` checksum in `ar_internal_metadata`),
|
|
122
|
+
for example when the test database is empty, out of date, or set up with a
|
|
123
|
+
plain `load "db/schema.rb"`. That load drops the
|
|
124
|
+
copied rows of the tables it defines.
|
|
125
|
+
- **Standalone coverage capture loads the `--require` files (#217).** A run
|
|
126
|
+
without `--boot` requires the sources and then each `--require` (or
|
|
127
|
+
`require:`) file before it forks the mutants, but the coverage capture
|
|
128
|
+
loaded only the sources. A source method that a `--require` file called
|
|
129
|
+
while it loaded was then not seen as run at load, so its mutant was a false
|
|
130
|
+
`survived` instead of `ran_at_load`. The capture and the clean-suite check
|
|
131
|
+
now load the `--require` files after the sources. With or without
|
|
132
|
+
`--boot`, the coverage cache rebuilds when one of them changes, is added,
|
|
133
|
+
is removed, or moves in the load order.
|
|
134
|
+
- **`--daemon` loads the `--require` files (#220).** The daemon booted the app
|
|
135
|
+
but never loaded the `--require` (or `require:`) files, so a test that
|
|
136
|
+
needs one failed under `--daemon` and passed in-process. A source method
|
|
137
|
+
that such a file called while it loaded was also not seen as run at load,
|
|
138
|
+
so its mutant could be a false `survived` instead of `ran_at_load`. The
|
|
139
|
+
daemon now loads the files after the app boots, as an in-process run does,
|
|
140
|
+
and the daemon's coverage cache rebuilds when one of them changes.
|
|
141
|
+
- **The release workflow publishes only stable tags** (#161). Its trigger,
|
|
142
|
+
`v*.*.*`, also matches a prerelease tag such as `v1.3.1-rc1`, and the only
|
|
143
|
+
guard checked that the tag matched `Mutineer::VERSION`. A prerelease tag
|
|
144
|
+
could then publish the gem and move the floating major tag (`v1`) that
|
|
145
|
+
Action users pin. The workflow now stops with a clear error before any
|
|
146
|
+
publish step unless the tag is a stable `vMAJOR.MINOR.PATCH`.
|
|
147
|
+
- **Coverage capture keeps to `capture_timeout` from start to finish.** Under
|
|
148
|
+
`--boot` and `--daemon`, a hung test file ignored the timeout and blocked
|
|
149
|
+
the run; it is now skipped as "timed out", like a standalone capture. A
|
|
150
|
+
process that a test started and left running could keep the result pipe
|
|
151
|
+
open and make a boot-mode capture wait for it to exit. Each capture now runs
|
|
152
|
+
in its own process group, and a timeout stops the whole group, not only the
|
|
153
|
+
test process. A capture whose test finished just before the deadline could
|
|
154
|
+
also lose its valid result and report "invalid coverage output"; it now
|
|
155
|
+
gets a short grace period to read the result. (#101, #129)
|
|
156
|
+
- **`--daemon` no longer waits forever on a daemon that stops answering**
|
|
157
|
+
(#101). A mutant whose verdict has not arrived 30 seconds after its own
|
|
158
|
+
timeout is scored `error`, and the daemon is killed and respawned, as after
|
|
159
|
+
a crash. A boot that has not finished after 600 seconds ends the run with a
|
|
160
|
+
message that says so (it is not retried by a second daemon), and shutdown no longer waits on a stuck daemon.
|
|
161
|
+
|
|
162
|
+
- **A method in a `Data.define`, `Struct.new`, `Class.new` or `Module.new`
|
|
163
|
+
block belongs to the class or module the block builds.** In
|
|
164
|
+
`class App; Argo = Data.define(:url) do def self.load = ...; end; end` the
|
|
165
|
+
subject is now `App::Argo.load`, not `App.load`; at the top level it was
|
|
166
|
+
`.load`, with no owner. Under `--strategy redefine` (the `--rails` default)
|
|
167
|
+
the mutated method was loaded into the wrong class, so the real method never
|
|
168
|
+
changed and every such mutant falsely survived. The class or module is
|
|
169
|
+
named after the constant it is assigned to with `=`, `||=` or `&&=`, through
|
|
170
|
+
parentheses or `begin`, resolved as Ruby resolves the assignment: `X` in the
|
|
171
|
+
current namespace, `::X` and any path at the top level from `Object`,
|
|
172
|
+
`self::X` under the current class. When it cannot be named statically (no
|
|
173
|
+
constant, a relative path such as `User::Permission` inside a namespace, or
|
|
174
|
+
a constant assigned in `class << self`), redefine reports its mutants with
|
|
175
|
+
a new status, `unplaceable`, rather than load them onto the wrong class;
|
|
176
|
+
reload still runs them. `module_function :name` in a `Module.new` block
|
|
177
|
+
promotes that module's method. Mutant ids for these subjects change:
|
|
178
|
+
regenerate `ignore:` entries and baselines that name them.
|
|
179
|
+
- **The automated release PR gets its required CI checks without a token
|
|
180
|
+
secret (#85).** `release-pr.yml` pushes the release branch with the default
|
|
181
|
+
`GITHUB_TOKEN`, which starts no other workflow, so the four required checks
|
|
182
|
+
never ran and the PR stayed blocked until a person closed and reopened it.
|
|
183
|
+
The job now dispatches `ci.yml` on the release branch (CI gained a
|
|
184
|
+
`workflow_dispatch` trigger), and those runs report the required checks on
|
|
185
|
+
the PR's head commit. With a `RELEASE_PR_TOKEN` secret set, the push
|
|
186
|
+
triggers CI as before and no dispatch is made.
|
|
187
|
+
- **Code that runs while the app boots or a class loads no longer gives a false
|
|
188
|
+
`survived` or `no_coverage`** (#187). A class body such as
|
|
189
|
+
`ALL = [price(3)].freeze` runs `price` before any mutant is applied. Under
|
|
190
|
+
`--strategy redefine` (the `--rails` default) the load never runs again, so
|
|
191
|
+
a test that checks `ALL` saw the original value and the mutant falsely
|
|
192
|
+
survived. Under `--rails` or `--boot` no test was credited with lines that
|
|
193
|
+
ran during boot, so those mutants were falsely `no_coverage`. Mutineer now
|
|
194
|
+
reads the lines that ran at load (from the boot, or from requiring the
|
|
195
|
+
sources in standalone capture) and reports these mutants as `ran_at_load`,
|
|
196
|
+
under both strategies and with `--daemon`: reload runs the file's class body
|
|
197
|
+
again but not an initializer or another file's code, so a survivor there is
|
|
198
|
+
not trusted either. Under `--boot` with redefine, a
|
|
199
|
+
lazily loaded class (Zeitwerk, `autoload`) is loaded before the boot lines
|
|
200
|
+
are read, so its class body counts as load too. A baseline survivor that is
|
|
201
|
+
now `ran_at_load` is listed as fixed, not as a regression. Standalone
|
|
202
|
+
coverage caches from earlier versions rebuild once. A one-line or endless
|
|
203
|
+
`def` keeps its body on the `def` line, which Ruby counts when the method is
|
|
204
|
+
defined, so Mutineer reads the method's call count at load for it instead
|
|
205
|
+
(#209).
|
|
206
|
+
- **A boot file that prints to stdout no longer breaks the daemon handshake
|
|
207
|
+
(#102).** Output from `puts`, `STDOUT`, `$stdout`, a direct fd 1 write, or a
|
|
208
|
+
subprocess reached the JSON channel before the ready message, so the daemon
|
|
209
|
+
failed with "daemon exited before the handshake". The daemon now keeps a
|
|
210
|
+
private copy of its original stdout for the protocol and points the app's
|
|
211
|
+
stdout at stderr, where that output stays visible.
|
|
212
|
+
- **A class or module opened inside `class << self` belongs to the singleton
|
|
213
|
+
class (#208).** In `class App; class << self; class Q; def q1 ...` the
|
|
214
|
+
subject was `App::Q.q1`, a class method of a constant Ruby never defines:
|
|
215
|
+
`Q` is `App.singleton_class::Q`, and `q1` is an instance method. It is now
|
|
216
|
+
`#<Class:App>::Q#q1` with no known owner, as is a class or module opened in
|
|
217
|
+
a builder block there, and anything nested in either. Inside a builder
|
|
218
|
+
block, `class << self` opens the built class's singleton class, so a class
|
|
219
|
+
or constant there is named under it, for example `#<Class:App::P>::W`. Under
|
|
220
|
+
`--strategy redefine` loading the mutant raised `NameError` and scored
|
|
221
|
+
`error`; redefine now reports these mutants as `unplaceable`, and reload
|
|
222
|
+
still runs them. A compact `class Foo::X` and a top-level `class ::X` or
|
|
223
|
+
`::X = Class.new do` there are named as written (`::X` as `X`, now with
|
|
224
|
+
instance methods) and are owner-unknown too: redefine reopens them without
|
|
225
|
+
the singleton class, so constants their bodies look up through it would not
|
|
226
|
+
resolve. `module_function :name` in a body whose owner is unknown now
|
|
227
|
+
promotes that body's own methods, wherever it is: a `Foo::X = Module.new`
|
|
228
|
+
block's `c` is `Foo::X.c`, not `Foo::X#c`, and a def in an anonymous
|
|
229
|
+
`Class.new` block is no longer promoted by a `module_function` of the module
|
|
230
|
+
around it. Mutant ids for these subjects change: regenerate `ignore:`
|
|
231
|
+
entries and baselines that name them.
|
|
232
|
+
|
|
233
|
+
## [1.5.0] - 2026-10-06
|
|
234
|
+
|
|
235
|
+
### Added
|
|
236
|
+
- **`--timeout SECONDS` and `--capture-timeout SECONDS`**, also `timeout:` and
|
|
237
|
+
`capture_timeout:` in `.mutineer.yml`. They set the in-process per-mutant
|
|
238
|
+
time limit (default 10s) and the coverage-capture time limit (default 120s),
|
|
239
|
+
which were fixed. On a large Rails suite the clean run of the unmutated
|
|
240
|
+
tests took longer than 120s, so every run that offered the whole suite
|
|
241
|
+
stopped as not green, and a mutant on a line many tests cover took longer
|
|
242
|
+
than 10s and was scored `timeout`. `--daemon` keeps its own per-mutant
|
|
243
|
+
limit but uses `--capture-timeout` for its coverage capture;
|
|
244
|
+
`--test-command` uses neither. Either backend warns about a limit it was
|
|
245
|
+
given and does not use.
|
|
246
|
+
- **`--cache-dir DIR`**, also `cache_dir:` in `.mutineer.yml`, sets where the
|
|
247
|
+
coverage cache is written (default `.mutineer`, as before). Two runs from the
|
|
248
|
+
same project root shared `.mutineer/coverage.json` and could overwrite each
|
|
249
|
+
other's map; each can now have its own directory. `--daemon` uses it too.
|
|
250
|
+
A blank value (an unset CI variable) exits 2 rather than writing the cache
|
|
251
|
+
into the project root.
|
|
252
|
+
- **The README states how timeouts affect the score** (#62): a timed-out
|
|
253
|
+
mutant is left out of the score, and counts as a mutant with no verdict
|
|
254
|
+
under `--threshold`.
|
|
255
|
+
- **`--matrix` reports which tests kill each mutant** (`matrix: true` in
|
|
256
|
+
`.mutineer.yml`). Each mutant runs every test in its covering files instead
|
|
257
|
+
of stopping at the first failure, and the child sends each test's outcome to
|
|
258
|
+
the parent as it is recorded. The report names blind tests, which ran in a
|
|
259
|
+
complete row and killed no mutant, and redundant tests, whose every kill
|
|
260
|
+
another test also makes. Redundancy is judged one test at a time, so delete
|
|
261
|
+
redundant tests one at a time. The human report lists up to 20 of each, the
|
|
262
|
+
HTML report all of them, and the JSON report moves to schema `1.5` with a
|
|
263
|
+
`matrix` block that appears only with the flag. A test is its file and its
|
|
264
|
+
id (the example id under RSpec), so examples that share a description stay
|
|
265
|
+
apart and an example whose generated description changes with the mutant
|
|
266
|
+
stays one test. Verdicts, the score and the exit code do not change: each
|
|
267
|
+
mutant gets the verdict a run without the flag gives. After a serial test
|
|
268
|
+
fails, an exit, a crash or the timeout in a later test, class or group
|
|
269
|
+
leaves it `killed`; an end in the code that run still reaches (the failing
|
|
270
|
+
test's class wrapper or group hooks, the suite's own cleanup) and a failure
|
|
271
|
+
in a `parallelize_me!` class keep the exit status. A row is complete only
|
|
272
|
+
when the child's stream arrived in order and every test reported (under
|
|
273
|
+
Minitest the recorder saw every test, under RSpec every planned example),
|
|
274
|
+
so a caught `Interrupt` leaves it incomplete. The human report names up
|
|
275
|
+
to 20 incomplete rows and the HTML and JSON reports all of them; the human
|
|
276
|
+
and HTML reports warn not to delete a blind test until its rows are
|
|
277
|
+
complete.
|
|
278
|
+
Minitest and RSpec 3.3+ are supported, on the in-process backend only;
|
|
279
|
+
`--daemon`, `--test-command`, `--fail-fast` and `--dry-run` exit 2 with it.
|
|
280
|
+
`--no-matrix` and `--no-fail-fast` beat the matching `.mutineer.yml` key,
|
|
281
|
+
so a file that sets one can still run with a flag that conflicts with it.
|
|
282
|
+
- **`no_coverage[]` and `uncapturable[]` name each mutant.** Every entry in
|
|
283
|
+
the JSON report now has `operator`, `token` and `id`, as `ignored[]` has, so
|
|
284
|
+
mutants on one line can be told apart and an `id` can go in `ignore:`. The
|
|
285
|
+
entries are sorted by `(file, line, operator, id)`. `schema_version` is `1.6`.
|
|
286
|
+
|
|
287
|
+
- **The site and README explain "clean-room" and compare Mutineer with
|
|
288
|
+
Mutant**, and the landing page hero has a one-minute explainer video. It
|
|
289
|
+
plays at 55% volume while in view (not with reduced motion; muted until
|
|
290
|
+
the first click or key where the browser requires it), pauses when it scrolls
|
|
291
|
+
away or the tab is hidden, and has an Enlarge button.
|
|
292
|
+
|
|
293
|
+
### Changed
|
|
294
|
+
- **The human report has a `Timeout:` row**, and the score line lists timeouts
|
|
295
|
+
apart from errored mutants. Before, the `Errored:` row and the score line
|
|
296
|
+
added the two together, while the JSON report kept them apart. The HTML
|
|
297
|
+
summary now shows a `timeout` count apart from `errored` too.
|
|
298
|
+
- **A test file given as a source after `--test` exits 2.** `--test` takes one
|
|
299
|
+
file, so in `mutineer run app/x.rb --test spec/a_spec.rb spec/b_spec.rb` the
|
|
300
|
+
second spec became a source to mutate, and the run tested with one spec file
|
|
301
|
+
without a word. Now a source named `*_test.rb` or `*_spec.rb` inside a
|
|
302
|
+
`test/` or `spec/` directory, given with `--test`, is a usage error that says
|
|
303
|
+
to repeat `--test`. The usage line now reads `--test <test> [--test <test>...]`.
|
|
304
|
+
|
|
305
|
+
### Fixed
|
|
306
|
+
- **The RSpec runner keeps what gems configured before the run.** It called
|
|
307
|
+
`RSpec.reset`, which drops the whole configuration. Under `rails: true`, a
|
|
308
|
+
`--boot` file or `require:`, the app's gems load first, so a setting such as
|
|
309
|
+
rspec-retry's `verbose_retry` was gone, and a support file that set it
|
|
310
|
+
raised `NoMethodError`. The run stopped with "the unmutated suite is not
|
|
311
|
+
green". The runner now calls `RSpec.clear_examples`, which keeps the
|
|
312
|
+
configuration. Hooks that gems added now apply too: an example with
|
|
313
|
+
rspec-retry's `retry: 3` runs up to 3 times per mutant, as under `rspec`.
|
|
314
|
+
- **`--since` no longer scores a file that Git ignores** (#168). A file
|
|
315
|
+
outside the index is scored in full, so a new file is not missed. That
|
|
316
|
+
check also caught files that Git ignores on purpose (`.gitignore`,
|
|
317
|
+
`.git/info/exclude` or the global excludes file), such as generated code
|
|
318
|
+
inside a directory run. Those files now get no mutants under `--since`,
|
|
319
|
+
and a warning names each one. Other untracked files are still scored in
|
|
320
|
+
full, and so is a file when Git cannot tell, such as one beyond a symlink or
|
|
321
|
+
inside a submodule.
|
|
322
|
+
- **The human report escapes control characters from the source** (#163).
|
|
323
|
+
A surviving line that held a terminal control byte (for example ESC) was
|
|
324
|
+
printed as that byte, so it could change what the terminal showed. The
|
|
325
|
+
token, the replacement, the diff lines, method names, file paths and the
|
|
326
|
+
kill matrix's test names now print each control character except tab as
|
|
327
|
+
its Ruby escape (`\e`), and a byte that is not valid UTF-8 (a Latin-1
|
|
328
|
+
source) as `\xNN`. The JSON and HTML
|
|
329
|
+
reports already escaped text and do not change.
|
|
330
|
+
- **One edit on a line is scored once** (#159). Some opt-in operators emitted
|
|
331
|
+
two mutants that give the same source: `literal_mutation` on `0` (the
|
|
332
|
+
"change to 1" and "add 1" rules), `negation_removal` on `!!x` (either `!`
|
|
333
|
+
removed), and `chain_link` or `operand_removal` on a repeated part such as
|
|
334
|
+
`a.b.b.c` or `x || x`. Each copy had its own id, so one surviving edit
|
|
335
|
+
counted twice. Now only the first copy on a line runs. Copies that start
|
|
336
|
+
on different lines (a multi-line chain) both stay, so `--since` never
|
|
337
|
+
loses the edit. Ids are assigned before the copy is dropped, so every kept
|
|
338
|
+
mutant keeps its id, and the dropped copy's id simply stops appearing.
|
|
339
|
+
Default runs do not change. If you use these operators with `--baseline`,
|
|
340
|
+
regenerate the baseline after upgrading: the score can move with no code
|
|
341
|
+
change, and in a full (unscoped) comparison a dropped copy that survived
|
|
342
|
+
is listed as fixed. An ignore entry for one copy's id still ignores only
|
|
343
|
+
that copy; the other copy still runs.
|
|
344
|
+
- **`# mutineer:disable-line` inside a string no longer silences a line**
|
|
345
|
+
(#158). The marker was found with a text search, so a string such as
|
|
346
|
+
`"# mutineer:disable-line"` ignored every mutant on its line and left
|
|
347
|
+
them out of the score. Only a real `#` comment counts now. A heredoc or
|
|
348
|
+
regex that holds the text is ignored too.
|
|
349
|
+
- **`./lib/x.rb` and an absolute path pair with a test** (#104). Auto-pairing
|
|
350
|
+
found the test for `lib/calc.rb` but not for `./lib/calc.rb` or the
|
|
351
|
+
absolute path to the same file. It reported "no test found by convention"
|
|
352
|
+
and exited 2. A file argument inside the project is now made relative to
|
|
353
|
+
the project root first, so equivalent spellings (`./`, `..`, absolute)
|
|
354
|
+
pair with the same test and run once. A symlink keeps its own name, so it
|
|
355
|
+
stays a separate source from its target. Reports show that root-relative
|
|
356
|
+
path. A path outside the project stays as typed.
|
|
357
|
+
- **A mutant on another line of a multi-line statement runs its tests**
|
|
358
|
+
instead of being reported as `no_coverage`. Ruby counts one line of a
|
|
359
|
+
statement only. When the mutant's own line has no count, the runner now uses
|
|
360
|
+
the tests that ran the statement that holds the mutation. A heredoc body
|
|
361
|
+
belongs to its statement. This fallback skips a first or last line that also
|
|
362
|
+
holds other code, such as the `def` line, and the body of `x while c` and
|
|
363
|
+
`x until c`. It also gives no tests to a mutant in code of the statement
|
|
364
|
+
that runs only sometimes: a `when` or `in` condition, a `rescue` class
|
|
365
|
+
list, a parameter default, the rescue side of `x rescue y`, the value of
|
|
366
|
+
`||=` or `&&=`, the operand of `defined?`, the right side of `a ||` or
|
|
367
|
+
`a &&`, a branch of `x if c`, `x unless c`, a ternary or `else`, the
|
|
368
|
+
pattern of `v in p` or `v => p`, an
|
|
369
|
+
argument or block of `x&.m(...)`, and the value of `x&.m += v`. Those
|
|
370
|
+
mutants stay `no_coverage`, as before.
|
|
371
|
+
- **A survivor diff for a multi-line mutant applies with `git apply`** (#106).
|
|
372
|
+
The JSON `diff` header always read `@@ -N +N @@`, even when the mutant
|
|
373
|
+
removed two lines and added one, so `git apply` rejected the patch as
|
|
374
|
+
corrupt. The header now carries each side's line count (`@@ -3,2 +3 @@`).
|
|
375
|
+
A one-line diff keeps its old header. A mutant that empties a line now
|
|
376
|
+
shows that empty line as a `+` line. A side whose last line has no final
|
|
377
|
+
newline gets ``. Lines of a CRLF file now keep
|
|
378
|
+
their `\r`, so the patch applies to that file.
|
|
379
|
+
- **Split test files pair with their source** (#87). A source such as
|
|
380
|
+
`app/foo/bar.rb` now also uses `test/foo/bar_upsert_test.rb` and
|
|
381
|
+
`test/foo/bar_guards_test.rb`, together with `test/foo/bar_test.rb` when
|
|
382
|
+
that file exists. The same names under `test/lib/` pair with a `lib/`
|
|
383
|
+
source. Before, only the exact `bar_test.rb` name was tried, so a suite
|
|
384
|
+
split across `bar_*_test.rb` files was skipped and a gate could pass with
|
|
385
|
+
no mutants run. A spec the old rules already find is unchanged. The
|
|
386
|
+
split name is left for `user_session.rb` when that file exists, so
|
|
387
|
+
`user.rb` does not run `user_session_test.rb`. The same rule holds outside
|
|
388
|
+
`app/` and `lib/`: `src/user.rb` does not take `test/src/user_session_test.rb`
|
|
389
|
+
when `src/user_session.rb` exists. A longer source claims a split file
|
|
390
|
+
only when that source itself searches the directory the file is in.
|
|
391
|
+
A failed capture of `test/foo/bar_upsert_test.rb` can mark `app/foo/bar.rb`
|
|
392
|
+
uncapturable only when no successful capture covered that source and
|
|
393
|
+
`bar_upsert.rb` does not exist. A test outside that directory does not,
|
|
394
|
+
and neither does a source that only shares the test basename.
|
|
395
|
+
`user_session__test.rb` stays with `user.rb` when `user_session.rb`
|
|
396
|
+
exists, because that name is not the exact test or a split test for
|
|
397
|
+
`user_session`. A failed exact spec in the mirrored `spec/` or `spec/lib/` directory
|
|
398
|
+
marks an `app/` or `lib/` source the same way. A failed spec in another
|
|
399
|
+
directory does not. A project rooted at `/` still finds split tests
|
|
400
|
+
inside that root. A stored failure keeps the pairing from the moment it
|
|
401
|
+
was checked. Adding or deleting a longer source rebuilds the coverage
|
|
402
|
+
map, so that failure is not moved onto the shorter source.
|
|
403
|
+
- **A blank `operators` key no longer runs zero mutants and exits 0** (#167).
|
|
404
|
+
`operators:` with no value became an empty list. The run made no mutants
|
|
405
|
+
and the gate passed. A blank operator list is now an error, and so is a
|
|
406
|
+
list of only unknown operator names. An empty `require` or `ignore` list
|
|
407
|
+
is still valid. `--operators` on the command line still replaces the
|
|
408
|
+
file list, including a blank or unknown file list.
|
|
409
|
+
- **The daemon no longer runs the tool's Ruby settings inside the app** (#100).
|
|
410
|
+
Open3 keeps a variable when the spawn hash omits it, so a parent `RUBYOPT`
|
|
411
|
+
ran inside the app. The child now gets the environment Bundler saved before
|
|
412
|
+
it activated, passed with `unsetenv_others`. The app keeps its gem home,
|
|
413
|
+
excluded groups, Bundler config, and chruby path. An `RBENV_VERSION` or
|
|
414
|
+
`ASDF_RUBY_VERSION` already in the environment stays. That pin is often
|
|
415
|
+
the Ruby that started the tool. `.ruby-version` applies only when no pin
|
|
416
|
+
is set. The tool's `RUBYOPT` and `RUBYLIB` do not reach the child, including
|
|
417
|
+
a value Bundler had saved. A gem home, `BUNDLE_PATH`, or `BUNDLE_WITHOUT`
|
|
418
|
+
reaches the child only when Bundler saved the app's value. The same
|
|
419
|
+
rule covers `GEM_PATH` and `BUNDLER_VERSION`. An rbenv version
|
|
420
|
+
bin under `RBENV_ROOT`, or under `~/.rbenv` when that variable is unset,
|
|
421
|
+
leaves `PATH` only when that root has a shim directory. An asdf version
|
|
422
|
+
bin leaves only when an asdf shim directory exists, including
|
|
423
|
+
`ASDF_DATA_DIR/shims`. Without that shim, the bin stays and `bundle` can
|
|
424
|
+
still be found. The app Gemfile, an explicit Ruby pin, and Rails env are
|
|
425
|
+
applied after that.
|
|
426
|
+
- **The Action refuses to run when `output` and `baseline` are the same file** (#160).
|
|
427
|
+
That setup used to work, and a failing run copied its report onto the
|
|
428
|
+
baseline. The next run then treated the failed report as the baseline, so
|
|
429
|
+
the gate could pass. The Action now stops before the run and exits 2.
|
|
430
|
+
The baseline may come from the `baseline` input, from `--baseline` in
|
|
431
|
+
`extra-args`, or from `.mutineer.yml`. A numeric path such as `123` is
|
|
432
|
+
read the same way as the CLI. Write the report to a separate path. Copy
|
|
433
|
+
that file onto the baseline in a later step, and only when this step
|
|
434
|
+
exits 0. You can also refresh the baseline with the CLI, and keep the
|
|
435
|
+
file only when the run exits 0. The next release moves the `v1` tag, so
|
|
436
|
+
workflows on `@v1` get this refusal then.
|
|
437
|
+
- **A mutation no longer leaves a heredoc body behind as Ruby** (#134).
|
|
438
|
+
Replacing a node deletes the heredoc opener and not the body. The body
|
|
439
|
+
then parses as code, raises, and counts as a kill. A mutation that drops
|
|
440
|
+
the opener is skipped. A replacement that still contains the opener stays.
|
|
441
|
+
- **A bare `public`, `private` or `protected` ends `module_function` mode**,
|
|
442
|
+
so later `def`s in the module body are named as instance methods, not
|
|
443
|
+
singleton methods (#144).
|
|
444
|
+
- **Turning on reduced motion mid-visit finishes the landing page count-ups**,
|
|
445
|
+
so no number keeps changing after the visitor asks for less motion (#176).
|
|
446
|
+
- **`rake site:build` can no longer delete a directory such as `lib/` or
|
|
447
|
+
`.git`** (#162). The task removed the destination it was given, and only
|
|
448
|
+
refused the checkout, its parents and `docs/`. It now always builds into
|
|
449
|
+
`_site/` at the checkout root, and `rake "site:build[DEST]"` and
|
|
450
|
+
`SITE_BUILD_DEST` are gone.
|
|
451
|
+
- **Documentation matches the 1.4 behavior.** Config examples use supported keys,
|
|
452
|
+
agent loops check the exit code and a real score, and report totals explain
|
|
453
|
+
partial `--fail-fast` runs. The README lists every config key and its validation
|
|
454
|
+
rules. API README links work outside GitHub, repeated changelog headings have
|
|
455
|
+
unique anchors, and the website checks validate built links and anchors.
|
|
456
|
+
Mutant-id wording and the gem's Minitest/RSpec description are also corrected.
|
|
457
|
+
- **A mutant in a nested method counts once** (#157). A `def` inside
|
|
458
|
+
another method is a subject of its own, but most operators also mutated
|
|
459
|
+
its code on the outer method. The same edit then counted twice in the
|
|
460
|
+
score, with two ids. Every operator now skips a nested `def` when it
|
|
461
|
+
mutates the outer method. A `def` inside `class << obj`, where `obj` is
|
|
462
|
+
not `self`, is not a subject, so the outer method still mutates its code.
|
|
463
|
+
`array_literal`, `chain_link`, `condition_true`, `condition_false`,
|
|
464
|
+
`operand_removal`, `operator_assignment` and `return_nil` skipped that
|
|
465
|
+
code before. Their new mutants there can show as new survivors against a
|
|
466
|
+
baseline.
|
|
467
|
+
|
|
468
|
+
A mutant id includes an ordinal among the mutants with the same operator
|
|
469
|
+
and token in a subject. When the duplicates leave the outer method, the
|
|
470
|
+
ordinals of its own mutants change. An old duplicate id can then name a
|
|
471
|
+
different mutant on the outer method, and an `ignore:` entry or a
|
|
472
|
+
baseline entry with that id applies to that mutant with no warning. If
|
|
473
|
+
your code has nested methods, regenerate the `ignore:` entries and the
|
|
474
|
+
baseline for the outer methods.
|
|
475
|
+
|
|
476
|
+
Under `--strategy redefine` (the `--rails` default), a mutant in a nested
|
|
477
|
+
method can survive or error even when a test would catch it. `redefine`
|
|
478
|
+
loads only the inner method, and a call to the outer method defines the
|
|
479
|
+
original inner method again. Before, the copy of the edit on the outer
|
|
480
|
+
method was killed. Now each edit has one verdict, from the inner method.
|
|
481
|
+
|
|
9
482
|
## [1.4.0] - 2026-09-30
|
|
10
483
|
|
|
11
484
|
### Added
|
|
@@ -677,6 +1150,8 @@ Rails hardening + CI batch (issues #8–#13), all verified Rails-free.
|
|
|
677
1150
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
678
1151
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
679
1152
|
|
|
1153
|
+
[1.6.0]: https://github.com/davidteren/mutineer/releases/tag/v1.6.0
|
|
1154
|
+
[1.5.0]: https://github.com/davidteren/mutineer/releases/tag/v1.5.0
|
|
680
1155
|
[1.4.0]: https://github.com/davidteren/mutineer/releases/tag/v1.4.0
|
|
681
1156
|
[1.3.0]: https://github.com/davidteren/mutineer/releases/tag/v1.3.0
|
|
682
1157
|
[1.2.0]: https://github.com/davidteren/mutineer/releases/tag/v1.2.0
|