mutineer 1.6.0 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,6 +1,8 @@
1
1
  # Mutineer
2
2
 
3
3
  [![Gem Version](https://img.shields.io/gem/v/mutineer?logo=rubygems&color=e23b3b)](https://rubygems.org/gems/mutineer)
4
+ [![CI](https://github.com/davidteren/mutineer/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/davidteren/mutineer/actions/workflows/ci.yml)
5
+ [![mutation score at least 85%](https://img.shields.io/badge/mutation%20score-at%20least%2085%25-67286a)](https://github.com/davidteren/mutineer/actions/workflows/ci.yml)
4
6
  [![GitHub Marketplace](https://img.shields.io/badge/Marketplace-Mutineer%20Ruby-2da44e?logo=githubactions&logoColor=white)](https://github.com/marketplace/actions/mutineer-ruby)
5
7
  [![Socket](https://img.shields.io/badge/Socket-security%20report-0a66c2?logo=socket&logoColor=white)](https://socket.dev/rubygems/package/mutineer)
6
8
 
@@ -25,7 +27,202 @@ for how the two tools differ.
25
27
  first, then the rest from fastest to slowest in doubling steps (under 1 s,
26
28
  1 to 3 s, 3 to 7 s, and so on); files in one step run in path order.
27
29
 
28
- 📖 **[mutineer.github.io →](https://davidteren.github.io/mutineer/)** — overview, operators, and usage.
30
+ 📖 **[davidteren.github.io/mutineer →](https://davidteren.github.io/mutineer/)** — overview, operators, and usage.
31
+
32
+ New to mutation testing? Read
33
+ [What is mutation testing?](https://github.com/davidteren/mutineer#what-is-mutation-testing)
34
+ first. It takes five minutes and uses one small example.
35
+
36
+ Try it on one file and its test:
37
+
38
+ ```sh
39
+ gem install mutineer
40
+ mutineer run lib/calculator.rb --test test/calculator_test.rb
41
+ ```
42
+
43
+ ## Is it production-ready?
44
+
45
+ Read this section before you gate an app on Mutineer.
46
+ Use the Status column to decide.
47
+ Adopt a row whose Status says supported or serial only.
48
+ Not supported in the Parallelism column means `--daemon` cannot run that row.
49
+
50
+ Mutineer itself needs Ruby 3.4 or later.
51
+ In-process runs fork, so use Linux or macOS.
52
+ A plain Ruby project with no database is supported.
53
+ It runs in parallel on those systems.
54
+
55
+ <!-- contract:support-matrix -->
56
+ | Ruby | Database | Framework | Parallelism | Status |
57
+ | --- | --- | --- | --- | --- |
58
+ | 3.4 or later | SQLite file | Minitest | supported with `--daemon` | supported |
59
+ | 3.4 or later | SQLite file | RSpec | not supported | serial only |
60
+ | 3.4 or later | PostgreSQL | Minitest | supported with `--daemon` | supported |
61
+ | 3.4 or later | PostgreSQL | RSpec | not supported | serial only |
62
+ | 3.4 or later | MySQL 8 | Minitest | supported with `--daemon` | supported |
63
+ | 3.4 or later | MySQL 8 | RSpec | not supported | serial only |
64
+ | 3.4 or later | MariaDB | Minitest | not tested | not tested |
65
+ | 3.4 or later | SQLite in memory | Minitest | not supported | serial only |
66
+ | 3.4 or later | SQLite in memory | RSpec | not supported | serial only |
67
+ | 3.4 or later | more than one database | Minitest | one worker | serial only |
68
+ | 3.4 or later | more than one database | RSpec | not supported | serial only |
69
+
70
+ This matrix is for a Rails app on Ruby 3.4 or later.
71
+ `supported` means you can run that cell in parallel.
72
+ `serial only` means the safe run uses one worker.
73
+ `not supported` means `--daemon` cannot run that cell.
74
+ `not tested` means no CI job runs that cell.
75
+ SQLite file means a database file, not `:memory:`.
76
+ Minitest on that file can use `--daemon`.
77
+ RSpec on that file is serial only.
78
+ PostgreSQL with Minitest can use `--daemon`.
79
+ PostgreSQL with RSpec is serial only.
80
+ MySQL 8 with Minitest can use `--daemon`.
81
+ MariaDB is not tested.
82
+ MySQL 5.7 is not supported.
83
+ A `:memory:` database is serial only.
84
+ An app with more than one database runs one worker.
85
+ `--daemon` isolates only the primary connection.
86
+ `--rails` without `--daemon` is serial on every row.
87
+ A Postgres worker database is copied once, at the start of the run.
88
+ A MySQL 8 worker database is created once, at the start of the run, and the rows are copied then.
89
+ A test that commits rows outside a transaction leaves them in that worker database.
90
+ The next mutant on that worker sees those rows.
91
+ Rails parallel tests have the same ceiling.
92
+
93
+ Mutineer sets `PARALLEL_WORKERS=1`, so a `parallelize` call in `test_helper.rb` starts no Rails workers during a mutant run.
94
+ See [Rails parallel tests, Spring, Bootsnap, and Redis](https://github.com/davidteren/mutineer#rails-parallel-tests-spring-bootsnap-and-redis).
95
+
96
+ <!-- /contract:support-matrix -->
97
+
98
+ An app on an older Ruby uses `--test-command`.
99
+ Mutineer still runs on Ruby 3.4 or later.
100
+ That path is serial.
101
+ It does not narrow tests by coverage.
102
+ A crash in the command can count as a kill.
103
+
104
+ What a release will not break is in
105
+ [STABILITY.md](https://github.com/davidteren/mutineer/blob/main/STABILITY.md).
106
+
107
+ ## What is mutation testing?
108
+
109
+ Your tests pass. That shows the code does what the tests check. It does not
110
+ show that the tests check enough. Mutation testing finds out by changing your
111
+ code on purpose and seeing whether any test notices.
112
+
113
+ Each change is small, such as `>=` to `>` or `&&` to `||`. One changed copy of
114
+ your code is a **mutant**. Mutineer runs your tests against each mutant:
115
+
116
+ - When a test fails, the tests **killed** the mutant. Good: they noticed.
117
+ - When every test still passes, the mutant **survived**. Your tests ran that
118
+ code but did not check what it does.
119
+
120
+ ### A small example
121
+
122
+ ```ruby
123
+ # lib/discount.rb
124
+ class Discount
125
+ def rate(qty)
126
+ return 0.1 if qty >= 10
127
+ 0
128
+ end
129
+ end
130
+ ```
131
+
132
+ ```ruby
133
+ # test/discount_test.rb
134
+ require "minitest/autorun"
135
+ require_relative "../lib/discount"
136
+
137
+ class DiscountTest < Minitest::Test
138
+ def test_bulk_order_gets_ten_percent
139
+ assert_equal 0.1, Discount.new.rate(50)
140
+ end
141
+
142
+ def test_small_order_gets_nothing
143
+ assert_equal 0, Discount.new.rate(1)
144
+ end
145
+ end
146
+ ```
147
+
148
+ Both tests pass, and every line runs, so line coverage is 100%. Run Mutineer:
149
+
150
+ ```sh
151
+ mutineer run lib/discount.rb --test test/discount_test.rb
152
+ ```
153
+
154
+ Part of the report:
155
+
156
+ ```text
157
+ Mutation score: 50.0% (killed / (killed + survived); ...)
158
+
159
+ Surviving Mutants
160
+ -----------------
161
+
162
+ lib/discount.rb
163
+ Discount#rate (discount.rb:3)
164
+ Operator: comparison (>= -> >)
165
+ - return 0.1 if qty >= 10
166
+ + return 0.1 if qty > 10
167
+ ```
168
+
169
+ Mutineer changed `>=` to `>`, and both tests still passed. No test checks an
170
+ order of exactly 10, so a bug at that boundary would ship. Add that test:
171
+
172
+ ```ruby
173
+ def test_ten_is_the_first_bulk_quantity
174
+ assert_equal 0.1, Discount.new.rate(10)
175
+ end
176
+ ```
177
+
178
+ Run Mutineer again. Both mutants are killed, and the score is 100%.
179
+
180
+ ### Coverage and mutation testing
181
+
182
+ Line coverage tells you which lines ran during the tests. Mutation testing tells
183
+ you whether the tests would fail if those lines were wrong. A test with no
184
+ assertion still adds coverage, but it kills a mutant only when the change makes
185
+ the code raise an error. That gap is common in
186
+ tests written in a hurry, and in tests written by AI coding agents.
187
+
188
+ ### What each result means
189
+
190
+ | Report row | What happened | In the score? |
191
+ | --- | --- | --- |
192
+ | Killed | A test failed against the mutant. | Yes |
193
+ | Survived | Every test passed against the mutant. Look at it. | Yes |
194
+ | No coverage | No test runs that line. Write a test for it first. | No |
195
+ | Timeout | The tests ran past `--timeout`. | No |
196
+ | Errored | The test run crashed, so there is no verdict. | No |
197
+ | Ignored | You marked the mutant as equivalent (see below). | No |
198
+ | Skipped | The change made invalid Ruby, so it did not run. | No |
199
+
200
+ Uncapturable, Unplaceable, and Ran at load are rarer. The
201
+ [JSON schema](https://davidteren.github.io/mutineer/json-schema.html#summary)
202
+ explains each one.
203
+
204
+ The **mutation score** is `killed / (killed + survived)`. The other rows do not
205
+ change the score. When nothing can be scored, the score is empty (`null` in
206
+ JSON), not 0. Under `--threshold`, too many Timeout or Errored mutants still
207
+ fail the run; see [Timeouts and the score](https://github.com/davidteren/mutineer#timeouts-and-the-score).
208
+
209
+ ### What to do with a survivor
210
+
211
+ 1. Read the diff. Ask what behaviour the change breaks.
212
+ 2. Write or strengthen a test that fails under the change. Run again.
213
+ 3. Sometimes the change breaks nothing. For example, when `x` is a number,
214
+ swapping `x * 1` for `x / 1` gives the same result. That mutant is **equivalent**, and no test can
215
+ kill it. Suppress it with a reason (see
216
+ [Suppressing equivalent mutants](https://github.com/davidteren/mutineer#suppressing-equivalent-mutants)).
217
+
218
+ ### Where to start
219
+
220
+ Each mutant runs part of your suite again, so a run takes longer than one test
221
+ run. Start with one file and its test, as above. Use `--dry-run` to list the
222
+ mutants without running tests. When the report makes sense, widen the run to a
223
+ folder, then gate pull requests on the lines they change with `--since`. On a
224
+ Rails app, the [Rails rollout guide](https://davidteren.github.io/mutineer/rails.html)
225
+ walks through each stage.
29
226
 
30
227
  ## Install
31
228
 
@@ -43,8 +240,17 @@ gem "mutineer", group: :test
43
240
 
44
241
  ```sh
45
242
  mutineer run <source...> --test <test> [--test <test>...] [options]
243
+ mutineer init [--rails] [--force]
244
+ mutineer triage REPORT.json --reason TEXT (--id ID ... | --all)
245
+ mutineer migrate <source...> [--dry-run] [--operators LIST]
46
246
  ```
47
247
 
248
+ - `run` mutates the sources, runs the tests, and prints the report.
249
+ - `init` writes a commented `.mutineer.yml` and prints a first run command.
250
+ - `triage` prints `ignore:` entries for survivors in a JSON report. It does not
251
+ edit files. See [Suppressing equivalent mutants](https://github.com/davidteren/mutineer#suppressing-equivalent-mutants).
252
+ - `migrate` rewrites old ignore ids. See [Migrate old ignore ids](https://github.com/davidteren/mutineer#migrate-old-ignore-ids).
253
+
48
254
  Mutate `lib/calculator.rb`, checking it against its test, and fail CI if the
49
255
  mutation score drops below 90%:
50
256
 
@@ -52,6 +258,26 @@ mutation score drops below 90%:
52
258
  mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
53
259
  ```
54
260
 
261
+ ### Migrate old ignore ids
262
+
263
+ ```sh
264
+ mutineer migrate <source...> [--dry-run] [--operators LIST]
265
+ ```
266
+
267
+ `migrate` rewrites old-format ids in `.mutineer.yml` `ignore:` to the current
268
+ ids. It keeps comments and every other line. When one old id matches several
269
+ mutants, it writes every new id. That keeps the 1.x match. An id that matches
270
+ nothing stays, and the command exits 1. In an `{ id:, reason: }` entry, the
271
+ id changes and the reason stays. One entry holds one id, so an old id there
272
+ that now matches several mutants stays as written: the command names the new
273
+ ids, exits 1, and you split the entry by hand. A block scalar under `ignore:`
274
+ is not supported: the command exits 2 and does not change the file.
275
+ `--dry-run` prints each change and writes nothing.
276
+
277
+ Pass the same sources and `--operators` you use for `run`. The command runs
278
+ no tests. It does not rewrite a baseline. Generate a new baseline with
279
+ `--format json`.
280
+
55
281
  ### Options
56
282
 
57
283
  | Flag | Meaning |
@@ -61,7 +287,7 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
61
287
  | `--threshold FLOAT` | Exit 1 when the score is below FLOAT, or when nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted (default: 0 = off) |
62
288
  | `--only NAME` | Restrict to one fully-qualified subject, e.g. `Calculator#add` |
63
289
  | `--framework NAME` | `minitest` (default) or `rspec`; auto-detected as rspec when most `--test` files end in `_spec.rb` |
64
- | `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`). Untracked files are scored in full unless Git ignores them. |
290
+ | `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`). Untracked files are scored in full unless Git ignores them. The report lists only suppressed mutants on changed lines, too. |
65
291
  | `--no-since` | Disable diff scoping; a typed no beats a `.mutineer.yml` `since:` key |
66
292
  | `--baseline FILE` | Compare against a prior `--format json` run; exit 1 on new survivors / score drop (score drop is skipped under `--since`, whose score covers a different denominator; see [CI](https://github.com/davidteren/mutineer#ci-gating)) |
67
293
  | `--baseline-epsilon FLOAT` | Score-drop tolerance for `--baseline` (default: 0) |
@@ -73,12 +299,14 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
73
299
  | `--timeout SECONDS` | Per-mutant time limit for in-process runs, in whole seconds (default: 10). A mutant whose tests run longer is a `timeout`; see [Timeouts and the score](https://github.com/davidteren/mutineer#timeouts-and-the-score). `--daemon` and `--test-command` keep their own limits |
74
300
  | `--capture-timeout SECONDS` | Time limit for each in-process coverage-capture subprocess and for the clean run of the unmutated tests, in whole seconds (default: 120). A suite slower than this stops the run as not green. `--daemon` uses it for its coverage capture; `--test-command` ignores it and says so |
75
301
  | `--cache-dir DIR` | Directory for the coverage cache (default: `.mutineer`). Give two runs from the same project root different directories so they do not share `coverage.json`. `--test-command` builds no cache, so it ignores it and says so |
302
+ | `--coverage-from FILE` | Reuse the `coverage.json` of an earlier `--rails`/`--boot` run with the same tests over these sources or more, so parallel shards capture coverage once. The run skips the capture. It runs each test that covers one of its sources alone, because a mutant runs only those tests, then all tests once together, and writes its own cache to `--cache-dir`. A test that covers none of its sources does not run alone. A capture that does not fit (changed tests or sources, an owner file added or removed, a source it did not cover, a failed capture, another `PARALLEL_WORKERS`, a cache from before this flag) stops the run (exit 2). Under `--strategy redefine`, eager-load in both runs. It has no effect, and says so, without `--rails`/`--boot` or with `--daemon`, `--test-command` or `--dry-run` |
76
303
  | `--test-command CMD` | Run the suite as a subprocess in the app's own runtime (for apps on Ruby < 3.4); `CMD` must contain `%{files}`. See [Apps on Ruby < 3.4](https://github.com/davidteren/mutineer#apps-on-ruby--34) |
77
- | `--daemon` | Boot the app once in a persistent daemon and fork per mutant, with per-worker DB isolation so `--jobs N` is safe under Rails (needs `--rails`/`--boot`; not with `--test-command`). See [the daemon backend](https://github.com/davidteren/mutineer#faster-parallel-safe-rails-the---daemon-backend) |
304
+ | `--daemon` | Boot the app once in a persistent daemon and fork per mutant, with per-worker DB isolation so `--jobs N` is safe under Rails (needs `--rails`/`--boot`; not with `--test-command`). See [the daemon backend](https://github.com/davidteren/mutineer#faster-parallel-safe-rails-the---daemon-backend). `--no-daemon` beats a `.mutineer.yml` `daemon:` key. It keeps `--rails` serial, which is the 1.x default, and in 2.0 it opts out of the parallel default |
78
305
  | `--format human\|json\|html` | Report format (default: human; `html` is a self-contained file) |
79
306
  | `--output FILE` | Write the report to FILE instead of stdout |
80
307
  | `--dry-run` | List candidate mutations without executing (honors suppression) |
81
308
  | `--fail-fast` | Stop at the first surviving mutant (`--no-fail-fast` beats a `.mutineer.yml` `fail_fast:` key) |
309
+ | `--allow-empty` | A run with no mutants is expected. Without it, an empty run warns that Mutineer 2.0 will fail it. A `--since` run whose changes hold no mutants does not warn: it reports that the changes hold nothing to test. A `--since` run with no mutants even before scoping still warns. `--no-allow-empty` beats a `.mutineer.yml` `allow_empty:` key |
82
310
  | `--matrix` | Run every covering test for each mutant and report the blind and redundant tests; the JSON report also lists each mutant's killers. In-process only; exits 2 with `--daemon`, `--test-command`, `--fail-fast` or `--dry-run`. RSpec needs 3.3 or later. `--no-matrix` beats a `.mutineer.yml` `matrix:` key. See [Kill matrix](https://github.com/davidteren/mutineer#kill-matrix) |
83
311
  | `--list-operators` | List available operators (default vs optional) and exit |
84
312
  | `--version`, `--help` | Print version / usage and exit |
@@ -100,7 +328,7 @@ Run `mutineer --list-operators` to see them. Default (Tier 1): `arithmetic`,
100
328
  Available but off by default (Tier 2, enable via `--operators`): `return_nil`,
101
329
  `literal_mutation`, `condition_negation`, `string_literal`, `regex`,
102
330
  `collection_method`, `safe_navigation`, `range`, `negation_removal`, `chain_link`,
103
- `operand_removal`, `array_literal`, `condition_true`,
331
+ `call_to_receiver`, `operand_removal`, `array_literal`, `condition_true`,
104
332
  `condition_false`, `operator_assignment`.
105
333
 
106
334
  ## Rails apps
@@ -139,6 +367,50 @@ Add Mutineer to your Gemfile's test group:
139
367
  gem "mutineer", group: :test, require: false
140
368
  ```
141
369
 
370
+ Start from one command. It writes `.mutineer.yml` and prints the first run.
371
+ It does not replace a file you already wrote. Pass `--force` to replace one.
372
+
373
+ ```sh
374
+ RAILS_ENV=test bundle exec mutineer init --rails
375
+ ```
376
+
377
+ The path from that first report to a CI gate is the
378
+ [Rails rollout guide](https://davidteren.github.io/mutineer/rails.html).
379
+
380
+ ### Rails parallel tests, Spring, Bootsnap, and Redis
381
+
382
+ Mutineer sets `PARALLEL_WORKERS` to `1` for every run.
383
+ Rails then runs each mutant's tests in one process.
384
+ Your `parallelize(workers:)` call stays in `test_helper.rb`.
385
+ It does not start Rails workers during a mutant run.
386
+ If you already set `PARALLEL_WORKERS` to another value, Mutineer replaces it and prints one line.
387
+ With `--test-command`, a script your command runs can set `PARALLEL_WORKERS` again.
388
+ Its value wins, and the Rails workers start again.
389
+ Remove that line from the script, or write `PARALLEL_WORKERS=${PARALLEL_WORKERS:-4}` so it keeps the 1 Mutineer passes.
390
+
391
+ Mutineer does not run your `parallelize_setup` hooks.
392
+ Rails registers its own database hook in that same list.
393
+ Running the list would point the tests at `<base>-<n>` instead of Mutineer's database.
394
+
395
+ `--test-command` also sets `DISABLE_SPRING=1`.
396
+ Spring does not start for that command.
397
+ The `--daemon` path loads `config/environment` itself, so Spring does not start there either.
398
+ A wrapper that sets `DISABLE_SPRING` again wins, same as `PARALLEL_WORKERS`.
399
+
400
+ Bootsnap checks its compile cache by file path, size, and modification time in whole seconds.
401
+ When the size and the time match, it uses the cached code without reading the file content.
402
+ The in-process and `--daemon` paths load each mutant from a new file, so the cache cannot mix them up.
403
+ `--test-command` rewrites the source file in place.
404
+ Two mutants of the same size, written in the same second, can then load the first one's cached code.
405
+ To rule that out, set `DISABLE_BOOTSNAP_COMPILE_CACHE=1` on the Mutineer command.
406
+ Bootsnap keeps its cache in `tmp/cache/bootsnap`, or under `$BOOTSNAP_CACHE_DIR` when that is set.
407
+ If a run uses code that no longer matches your files, delete that folder and run again.
408
+
409
+ `redis-client` (which `redis` 5 uses) opens a new connection after a fork on its own.
410
+ You do not set anything in Mutineer for this.
411
+ `redis` 4.x raises `Redis::InheritedError` instead.
412
+ Open that connection after boot, or reconnect it in an after-fork hook.
413
+
142
414
  ### Faster, parallel-safe Rails (the `--daemon` backend)
143
415
 
144
416
  `--rails` boots your app once but runs mutants **serially** — parallel `--jobs`
@@ -169,10 +441,26 @@ RAILS_ENV=test bundle exec mutineer run \
169
441
  - **One backend at a time** — `--daemon` can't be combined with `--test-command`
170
442
  (choose one), and it needs an app to boot (`--rails` or `--boot`).
171
443
 
172
- Status: **SQLite** only (hermetic, CI-proven). Per-worker provisioning for
173
- **Postgres** and other adapters is not supported: on those, `--daemon` scores
174
- every mutant as `error`. Use `--daemon` with a SQLite test database, or drop
175
- `--daemon` to run serially on other adapters.
444
+ Status: **SQLite**, **PostgreSQL**, and **MySQL 8** (Minitest). Each Postgres worker uses a
445
+ database named `<base>-mutineer-<N>` (shortened when that would pass 63 bytes).
446
+ Mutineer copies the test database into those names before the workers start,
447
+ and leaves them in place when the run ends. Drop one with
448
+ `DROP DATABASE "myapp_test-mutineer-0";` (use your test database name and slot).
449
+ Each MySQL 8 worker uses a database named `<base>-mutineer-<N>` (shortened when that would pass 64 bytes).
450
+ Mutineer creates that database, loads `schema.rb` or `structure.sql`, and copies rows before the workers start.
451
+ It does not copy the database again after an error or a timeout.
452
+ The worker databases stay in place when the run ends. The next run drops and rebuilds them.
453
+ Triggers in the schema do not fire on the copied rows: in each worker database, Mutineer drops them for the copy and creates them again.
454
+ The database user needs CREATE, DROP, TRIGGER, and the table and row privileges a schema load and a row copy use (such as ALTER, INDEX, REFERENCES, and INSERT) on the worker databases, plus SELECT on the test database.
455
+ A trigger whose DEFINER is another account also needs SET_USER_ID.
456
+ An app on `structure.sql` also needs the `mysql` client on `PATH`.
457
+ Drop one with ``DROP DATABASE `myapp_test-mutineer-0`;`` (MySQL quotes the name with backticks).
458
+ A second run on the same test database stops with `another Mutineer run is using <base>` until the first run ends.
459
+ MySQL runs at most 256 workers.
460
+ `mysql2` and `trilogy` are supported.
461
+ MariaDB is not tested.
462
+ MySQL 5.7 is not supported.
463
+ A `:memory:` database is not supported on `--daemon`.
176
464
 
177
465
  ### Apps on Ruby < 3.4
178
466
 
@@ -201,7 +489,9 @@ RAILS_ENV=test mutineer run app/models/order.rb \
201
489
  prepends rbenv/asdf shims when a pin was scrubbed. Do not rely on
202
490
  `RBENV_VERSION=…` on the Mutineer command for the suite; use `.ruby-version`
203
491
  or a wrapper. Don't put `KEY=val` prefixes *inside* `--test-command` (no shell;
204
- that would be treated as the program name).
492
+ that would be treated as the program name). Mutineer also sets
493
+ `PARALLEL_WORKERS=1` and `DISABLE_SPRING=1` in the child and replaces a value
494
+ you set (see [Rails parallel tests, Spring, Bootsnap, and Redis](https://github.com/davidteren/mutineer#rails-parallel-tests-spring-bootsnap-and-redis)).
205
495
 
206
496
  #### Under a version manager (rbenv / asdf / chruby)
207
497
 
@@ -235,10 +525,13 @@ Mutineer also surfaces a targeted smoke-check message when Bundler prints
235
525
  Tradeoffs — this path is correct but not free:
236
526
 
237
527
  - **Slower:** your app re-boots for every mutant (no shared boot yet).
238
- - **No coverage narrowing:** every mutant runs the *full* `--test` set, so the
239
- score is an **upper bound and not comparable to an in-process (`--rails`)
240
- score** — uncovered mutants count as survivors, and an infrastructure failure
241
- is scored as a kill. Mutineer prints this caveat on every run and aborts up
528
+ - **No coverage narrowing:** every mutant runs the full `--test` set.
529
+ The run is serial.
530
+ A crash in the command can count as a kill.
531
+ Uncovered mutants count as survivors.
532
+ Do not compare a `--test-command` score with an in-process score.
533
+ Exit codes 126 and 127 (the command could not start) are scored `error`.
534
+ Mutineer prints this caveat on every run and aborts up
242
535
  front (a "smoke check") if your unmutated suite isn't green.
243
536
  - **Reload strategy only** (`--strategy redefine` is rejected on this path) and
244
537
  **serial** (`--jobs` is forced to 1). For apps on Ruby ≥ 3.4, `--daemon` gives
@@ -265,9 +558,12 @@ and once they pass 10% of the attempted mutants the run fails.
265
558
  Some mutants are equivalent (behaviour-identical) and survive forever — keeping a
266
559
  file off 100%. Suppress them so the score and `--threshold` gate stay meaningful:
267
560
 
268
- - **Inline:** `some_line # mutineer:disable-line` (or scope it: `# mutineer:disable-line comparison`). Put a reason after `--`: `# mutineer:disable-line comparison -- the test checks only 20`.
269
- - **Config:** a `.mutineer.yml` `ignore:` list of mutant ids. Each survivor's
270
- `id` is printed in the JSON report, so copy it straight into `ignore:`.
561
+ - **Inline:** `some_line # mutineer:disable-line` (or scope it: `# mutineer:disable-line comparison`). Put a reason after `--`: `# mutineer:disable-line comparison -- the test checks only 20`. The reason is stored on the ignored mutant and shown in the JSON report and the HTML report. A marker with no reason stores none.
562
+ - **Config:** a `.mutineer.yml` `ignore:` list. A bare mutant id still works. A mapping is `id` plus an optional `reason`. Mutineer 1.7 is the first version that reads a mapping. An older version matches no mutant to a mapping and does not warn, so the mutant it names is not ignored. Add a mapping only after every gate that reads the file runs 1.7 or later. In the Action, the `version:` input sets the version, or your `Gemfile.lock` does when you set `use-bundler: true`. In your own CI job, your `Gemfile.lock` sets it. The reason shows in the same reports as an inline reason. Each survivor's `id` is printed in the JSON report.
563
+
564
+ `mutineer triage REPORT.json --reason TEXT --all` prints one mapping entry per survivor. `--id ID` prints one entry. Repeat `--id` for several. The command does not edit files. Paste the lines under `ignore:`. They are mappings, so every gate that reads the file must run 1.7 or later (see **Config** above).
565
+
566
+ Built-in skips are listed on the [equivalent skips](https://davidteren.github.io/mutineer/equivalent-skips.html) page. There is no switch to turn them off.
271
567
 
272
568
  Suppressed mutants are excluded from the score (so 100% becomes reachable).
273
569
 
@@ -300,14 +596,18 @@ working until 2.0, with a warning:
300
596
  hid mutants you did not mean to ignore. The warning says so. Keep only the ids for the mutant you meant to
301
597
  ignore, not all of them. The list covers only the sources and operators in
302
598
  that run, so run over every source with every operator set you use (for
303
- example your Tier-2 `--operators`) for the full list.
599
+ example your Tier-2 `--operators`) for the full list. Or run
600
+ `mutineer migrate` on those sources. It writes every new id. Delete any id
601
+ you did not mean to ignore. See [Migrate old ignore ids](https://github.com/davidteren/mutineer#migrate-old-ignore-ids).
304
602
  - **`--baseline`** An old baseline still matches: a survivor matches a stored
305
603
  one with the same old id in the same file. A stored file that is an absolute
306
604
  path outside the project root (a baseline written on another machine)
307
605
  matches on the old id alone. The run tells you to
308
- regenerate it. Regenerate it with `--format json`, but only after every gate
309
- that reads it runs 1.3 or later (the Action's `version:` pin, your CI
310
- `Gemfile.lock`). An older version treats every new-format survivor as new.
606
+ generate a new baseline with `--format json`. `migrate` does not rewrite a
607
+ baseline. Generate the new file only after every gate
608
+ that reads it runs 1.3 or later. In the Action, the `version:` input sets
609
+ the version, or your `Gemfile.lock` does when you set `use-bundler: true`.
610
+ In your own CI job, your `Gemfile.lock` sets it. An older version treats every new-format survivor as new.
311
611
 
312
612
  The JSON report's `summary.id_format` is `2` for the new format.
313
613
  `summary.legacy_id_matches.ignore` counts the old-format ignore entries a run
@@ -490,6 +790,7 @@ config file accepts these keys:
490
790
  | `capture_timeout` | A positive integer; the coverage-capture time limit in seconds (default 120) |
491
791
  | `cache_dir` | The coverage cache directory (default `.mutineer`) |
492
792
  | `matrix` | `true` or `false`; runs every covering test for each mutant and adds the kill matrix to the report |
793
+ | `allow_empty` | `true` or `false`; a run with no mutants is expected, so it does not warn |
493
794
 
494
795
  Invalid values for known scalar keys, and a blank `operators` list, exit 2
495
796
  with a message naming the file and key. An unknown operator name warns and
@@ -499,9 +800,9 @@ is skipped. If none of the names are known, the run exits 2. An empty
499
800
  `threshold` and the CLI-only `--baseline-epsilon` use plain decimals such as `90`
500
801
  or `0.5`, not `+2`, `1e2`, or `1_0`. String options such as `only` and `baseline`
501
802
  cannot be null or boolean. A blank `since` is invalid; use `since: false` to turn
502
- scoping off. Unknown keys warn and are ignored. Unknown operator names warn and are skipped, and the run exits 2 when none remain. `--operators` replaces a blank or unknown file list.
803
+ scoping off. Unknown keys warn and are ignored. Unknown operator names warn and are skipped, and the run exits 2 when none remain. Both warnings suggest the closest valid name, and both become errors in Mutineer 2.0. `--operators` replaces a blank or unknown file list.
503
804
 
504
- `format`, `strategy`, `output`, `baseline_epsilon`, and `dry_run` are CLI-only.
805
+ `format`, `strategy`, `output`, `baseline_epsilon`, `dry_run`, and `coverage_from` are CLI-only.
505
806
  For JSON output, use `--format json`, not a `format:` config key. To select RSpec
506
807
  in the file, add `framework: rspec`.
507
808
 
@@ -535,7 +836,7 @@ no code with Mutant.
535
836
  | Test frameworks | Minitest and RSpec, in one gem | RSpec, Minitest and Test::Unit, one integration gem each |
536
837
  | What you mutate | Files (`mutineer run lib/foo.rb --test test/foo_test.rb`), narrowed to one method with `--only 'Foo#bar'` | Subjects named by expression (`mutant run 'Foo#bar'`, `'Foo*'`) |
537
838
  | Which tests run | The test files whose coverage reaches the mutated line | The tests that declare the subject (an RSpec description or a Minitest `cover`), or the tests that ran it in a per-test coverage recording |
538
- | Operators | 20 (5 by default, 15 opt-in) | A larger set; the default `light` set applies almost all of it (`full` adds `#==` to `#eql?`) |
839
+ | Operators | 21 (5 by default, 16 opt-in) | A larger set; the default `light` set applies almost all of it (`full` adds `#==` to `#eql?`) |
539
840
  | CI gating | `--threshold`, `--baseline` deltas, `--since`, and a GitHub Action | Incremental mode (`--since`) and a recorded session history |
540
841
  | Machine-readable output | Versioned JSON report schema | Session JSON schema |
541
842
 
@@ -0,0 +1,53 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Mutineer
4
+ # Lets a test helper start Coverage in a forked mutant child (#248).
5
+ #
6
+ # Once the coverage map is built, the parent suspends Coverage so mutant
7
+ # children do not pay for it (#228). A suspended Coverage reports
8
+ # `running? == false`, so a test helper that starts it when it is off
9
+ # (SimpleCov's `SimpleCov.start`) calls `Coverage.start`, which raises
10
+ # "coverage measurement is already setup" while Coverage is suspended. Every
11
+ # mutant then errors before a test runs.
12
+ #
13
+ # In the child, `Coverage.start` and `Coverage.setup` first stop the
14
+ # suspended Coverage, then run as usual. Stopping builds and drops the whole
15
+ # result, which walks the heap, so only a child whose helper starts Coverage
16
+ # pays for it. Only files loaded after that start are measured. The parent's
17
+ # Coverage is untouched.
18
+ #
19
+ # Stdlib-only, so the app-side daemon can load it.
20
+ module ChildCoverage
21
+ # Prepended to Coverage's singleton class in the child: stops a suspended
22
+ # Coverage before a start or setup.
23
+ module StopBeforeStart
24
+ # Stops a suspended Coverage, then sets it up.
25
+ #
26
+ # @return [nil]
27
+ def setup(...)
28
+ result(stop: true, clear: true) if state == :suspended
29
+ super
30
+ end
31
+
32
+ # Stops a suspended Coverage, then starts it.
33
+ #
34
+ # @return [nil]
35
+ def start(...)
36
+ result(stop: true, clear: true) if state == :suspended
37
+ super
38
+ end
39
+ end
40
+
41
+ # Makes a later `Coverage.start` or `Coverage.setup` work when Coverage is
42
+ # suspended. A running or idle Coverage is left alone.
43
+ #
44
+ # Call it only in a forked child, before any test loads.
45
+ #
46
+ # @return [void]
47
+ def self.allow_start
48
+ return unless defined?(::Coverage) && ::Coverage.state == :suspended
49
+
50
+ ::Coverage.singleton_class.prepend(StopBeforeStart)
51
+ end
52
+ end
53
+ end