mutineer 1.5.0 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,6 +1,8 @@
1
1
  # Mutineer
2
2
 
3
3
  [![Gem Version](https://img.shields.io/gem/v/mutineer?logo=rubygems&color=e23b3b)](https://rubygems.org/gems/mutineer)
4
+ [![CI](https://github.com/davidteren/mutineer/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/davidteren/mutineer/actions/workflows/ci.yml)
5
+ [![mutation score at least 85%](https://img.shields.io/badge/mutation%20score-at%20least%2085%25-67286a)](https://github.com/davidteren/mutineer/actions/workflows/ci.yml)
4
6
  [![GitHub Marketplace](https://img.shields.io/badge/Marketplace-Mutineer%20Ruby-2da44e?logo=githubactions&logoColor=white)](https://github.com/marketplace/actions/mutineer-ruby)
5
7
  [![Socket](https://img.shields.io/badge/Socket-security%20report-0a66c2?logo=socket&logoColor=white)](https://socket.dev/rubygems/package/mutineer)
6
8
 
@@ -21,9 +23,206 @@ for how the two tools differ.
21
23
  - **Coverage-guided** — each mutant runs only the test files that cover its line.
22
24
  - **Stops at the first failing test** — in-process runs (not `--daemon` or
23
25
  `--test-command`) stop a mutant's test run at the first failure, unless
24
- `--matrix` asks for every covering test.
26
+ `--matrix` asks for every covering test. The source's paired test files run
27
+ first, then the rest from fastest to slowest in doubling steps (under 1 s,
28
+ 1 to 3 s, 3 to 7 s, and so on); files in one step run in path order.
25
29
 
26
- 📖 **[mutineer.github.io →](https://davidteren.github.io/mutineer/)** — overview, operators, and usage.
30
+ 📖 **[davidteren.github.io/mutineer →](https://davidteren.github.io/mutineer/)** — overview, operators, and usage.
31
+
32
+ New to mutation testing? Read
33
+ [What is mutation testing?](https://github.com/davidteren/mutineer#what-is-mutation-testing)
34
+ first. It takes five minutes and uses one small example.
35
+
36
+ Try it on one file and its test:
37
+
38
+ ```sh
39
+ gem install mutineer
40
+ mutineer run lib/calculator.rb --test test/calculator_test.rb
41
+ ```
42
+
43
+ ## Is it production-ready?
44
+
45
+ Read this section before you gate an app on Mutineer.
46
+ Use the Status column to decide.
47
+ Adopt a row whose Status says supported or serial only.
48
+ Not supported in the Parallelism column means `--daemon` cannot run that row.
49
+
50
+ Mutineer itself needs Ruby 3.4 or later.
51
+ In-process runs fork, so use Linux or macOS.
52
+ A plain Ruby project with no database is supported.
53
+ It runs in parallel on those systems.
54
+
55
+ <!-- contract:support-matrix -->
56
+ | Ruby | Database | Framework | Parallelism | Status |
57
+ | --- | --- | --- | --- | --- |
58
+ | 3.4 or later | SQLite file | Minitest | supported with `--daemon` | supported |
59
+ | 3.4 or later | SQLite file | RSpec | not supported | serial only |
60
+ | 3.4 or later | PostgreSQL | Minitest | supported with `--daemon` | supported |
61
+ | 3.4 or later | PostgreSQL | RSpec | not supported | serial only |
62
+ | 3.4 or later | MySQL 8 | Minitest | supported with `--daemon` | supported |
63
+ | 3.4 or later | MySQL 8 | RSpec | not supported | serial only |
64
+ | 3.4 or later | MariaDB | Minitest | not tested | not tested |
65
+ | 3.4 or later | SQLite in memory | Minitest | not supported | serial only |
66
+ | 3.4 or later | SQLite in memory | RSpec | not supported | serial only |
67
+ | 3.4 or later | more than one database | Minitest | one worker | serial only |
68
+ | 3.4 or later | more than one database | RSpec | not supported | serial only |
69
+
70
+ This matrix is for a Rails app on Ruby 3.4 or later.
71
+ `supported` means you can run that cell in parallel.
72
+ `serial only` means the safe run uses one worker.
73
+ `not supported` means `--daemon` cannot run that cell.
74
+ `not tested` means no CI job runs that cell.
75
+ SQLite file means a database file, not `:memory:`.
76
+ Minitest on that file can use `--daemon`.
77
+ RSpec on that file is serial only.
78
+ PostgreSQL with Minitest can use `--daemon`.
79
+ PostgreSQL with RSpec is serial only.
80
+ MySQL 8 with Minitest can use `--daemon`.
81
+ MariaDB is not tested.
82
+ MySQL 5.7 is not supported.
83
+ A `:memory:` database is serial only.
84
+ An app with more than one database runs one worker.
85
+ `--daemon` isolates only the primary connection.
86
+ `--rails` without `--daemon` is serial on every row.
87
+ A Postgres worker database is copied once, at the start of the run.
88
+ A MySQL 8 worker database is created once, at the start of the run, and the rows are copied then.
89
+ A test that commits rows outside a transaction leaves them in that worker database.
90
+ The next mutant on that worker sees those rows.
91
+ Rails parallel tests have the same ceiling.
92
+
93
+ Mutineer sets `PARALLEL_WORKERS=1`, so a `parallelize` call in `test_helper.rb` starts no Rails workers during a mutant run.
94
+ See [Rails parallel tests, Spring, Bootsnap, and Redis](https://github.com/davidteren/mutineer#rails-parallel-tests-spring-bootsnap-and-redis).
95
+
96
+ <!-- /contract:support-matrix -->
97
+
98
+ An app on an older Ruby uses `--test-command`.
99
+ Mutineer still runs on Ruby 3.4 or later.
100
+ That path is serial.
101
+ It does not narrow tests by coverage.
102
+ A crash in the command can count as a kill.
103
+
104
+ What a release will not break is in
105
+ [STABILITY.md](https://github.com/davidteren/mutineer/blob/main/STABILITY.md).
106
+
107
+ ## What is mutation testing?
108
+
109
+ Your tests pass. That shows the code does what the tests check. It does not
110
+ show that the tests check enough. Mutation testing finds out by changing your
111
+ code on purpose and seeing whether any test notices.
112
+
113
+ Each change is small, such as `>=` to `>` or `&&` to `||`. One changed copy of
114
+ your code is a **mutant**. Mutineer runs your tests against each mutant:
115
+
116
+ - When a test fails, the tests **killed** the mutant. Good: they noticed.
117
+ - When every test still passes, the mutant **survived**. Your tests ran that
118
+ code but did not check what it does.
119
+
120
+ ### A small example
121
+
122
+ ```ruby
123
+ # lib/discount.rb
124
+ class Discount
125
+ def rate(qty)
126
+ return 0.1 if qty >= 10
127
+ 0
128
+ end
129
+ end
130
+ ```
131
+
132
+ ```ruby
133
+ # test/discount_test.rb
134
+ require "minitest/autorun"
135
+ require_relative "../lib/discount"
136
+
137
+ class DiscountTest < Minitest::Test
138
+ def test_bulk_order_gets_ten_percent
139
+ assert_equal 0.1, Discount.new.rate(50)
140
+ end
141
+
142
+ def test_small_order_gets_nothing
143
+ assert_equal 0, Discount.new.rate(1)
144
+ end
145
+ end
146
+ ```
147
+
148
+ Both tests pass, and every line runs, so line coverage is 100%. Run Mutineer:
149
+
150
+ ```sh
151
+ mutineer run lib/discount.rb --test test/discount_test.rb
152
+ ```
153
+
154
+ Part of the report:
155
+
156
+ ```text
157
+ Mutation score: 50.0% (killed / (killed + survived); ...)
158
+
159
+ Surviving Mutants
160
+ -----------------
161
+
162
+ lib/discount.rb
163
+ Discount#rate (discount.rb:3)
164
+ Operator: comparison (>= -> >)
165
+ - return 0.1 if qty >= 10
166
+ + return 0.1 if qty > 10
167
+ ```
168
+
169
+ Mutineer changed `>=` to `>`, and both tests still passed. No test checks an
170
+ order of exactly 10, so a bug at that boundary would ship. Add that test:
171
+
172
+ ```ruby
173
+ def test_ten_is_the_first_bulk_quantity
174
+ assert_equal 0.1, Discount.new.rate(10)
175
+ end
176
+ ```
177
+
178
+ Run Mutineer again. Both mutants are killed, and the score is 100%.
179
+
180
+ ### Coverage and mutation testing
181
+
182
+ Line coverage tells you which lines ran during the tests. Mutation testing tells
183
+ you whether the tests would fail if those lines were wrong. A test with no
184
+ assertion still adds coverage, but it kills a mutant only when the change makes
185
+ the code raise an error. That gap is common in
186
+ tests written in a hurry, and in tests written by AI coding agents.
187
+
188
+ ### What each result means
189
+
190
+ | Report row | What happened | In the score? |
191
+ | --- | --- | --- |
192
+ | Killed | A test failed against the mutant. | Yes |
193
+ | Survived | Every test passed against the mutant. Look at it. | Yes |
194
+ | No coverage | No test runs that line. Write a test for it first. | No |
195
+ | Timeout | The tests ran past `--timeout`. | No |
196
+ | Errored | The test run crashed, so there is no verdict. | No |
197
+ | Ignored | You marked the mutant as equivalent (see below). | No |
198
+ | Skipped | The change made invalid Ruby, so it did not run. | No |
199
+
200
+ Uncapturable, Unplaceable, and Ran at load are rarer. The
201
+ [JSON schema](https://davidteren.github.io/mutineer/json-schema.html#summary)
202
+ explains each one.
203
+
204
+ The **mutation score** is `killed / (killed + survived)`. The other rows do not
205
+ change the score. When nothing can be scored, the score is empty (`null` in
206
+ JSON), not 0. Under `--threshold`, too many Timeout or Errored mutants still
207
+ fail the run; see [Timeouts and the score](https://github.com/davidteren/mutineer#timeouts-and-the-score).
208
+
209
+ ### What to do with a survivor
210
+
211
+ 1. Read the diff. Ask what behaviour the change breaks.
212
+ 2. Write or strengthen a test that fails under the change. Run again.
213
+ 3. Sometimes the change breaks nothing. For example, when `x` is a number,
214
+ swapping `x * 1` for `x / 1` gives the same result. That mutant is **equivalent**, and no test can
215
+ kill it. Suppress it with a reason (see
216
+ [Suppressing equivalent mutants](https://github.com/davidteren/mutineer#suppressing-equivalent-mutants)).
217
+
218
+ ### Where to start
219
+
220
+ Each mutant runs part of your suite again, so a run takes longer than one test
221
+ run. Start with one file and its test, as above. Use `--dry-run` to list the
222
+ mutants without running tests. When the report makes sense, widen the run to a
223
+ folder, then gate pull requests on the lines they change with `--since`. On a
224
+ Rails app, the [Rails rollout guide](https://davidteren.github.io/mutineer/rails.html)
225
+ walks through each stage.
27
226
 
28
227
  ## Install
29
228
 
@@ -41,8 +240,17 @@ gem "mutineer", group: :test
41
240
 
42
241
  ```sh
43
242
  mutineer run <source...> --test <test> [--test <test>...] [options]
243
+ mutineer init [--rails] [--force]
244
+ mutineer triage REPORT.json --reason TEXT (--id ID ... | --all)
245
+ mutineer migrate <source...> [--dry-run] [--operators LIST]
44
246
  ```
45
247
 
248
+ - `run` mutates the sources, runs the tests, and prints the report.
249
+ - `init` writes a commented `.mutineer.yml` and prints a first run command.
250
+ - `triage` prints `ignore:` entries for survivors in a JSON report. It does not
251
+ edit files. See [Suppressing equivalent mutants](https://github.com/davidteren/mutineer#suppressing-equivalent-mutants).
252
+ - `migrate` rewrites old ignore ids. See [Migrate old ignore ids](https://github.com/davidteren/mutineer#migrate-old-ignore-ids).
253
+
46
254
  Mutate `lib/calculator.rb`, checking it against its test, and fail CI if the
47
255
  mutation score drops below 90%:
48
256
 
@@ -50,6 +258,26 @@ mutation score drops below 90%:
50
258
  mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
51
259
  ```
52
260
 
261
+ ### Migrate old ignore ids
262
+
263
+ ```sh
264
+ mutineer migrate <source...> [--dry-run] [--operators LIST]
265
+ ```
266
+
267
+ `migrate` rewrites old-format ids in `.mutineer.yml` `ignore:` to the current
268
+ ids. It keeps comments and every other line. When one old id matches several
269
+ mutants, it writes every new id. That keeps the 1.x match. An id that matches
270
+ nothing stays, and the command exits 1. In an `{ id:, reason: }` entry, the
271
+ id changes and the reason stays. One entry holds one id, so an old id there
272
+ that now matches several mutants stays as written: the command names the new
273
+ ids, exits 1, and you split the entry by hand. A block scalar under `ignore:`
274
+ is not supported: the command exits 2 and does not change the file.
275
+ `--dry-run` prints each change and writes nothing.
276
+
277
+ Pass the same sources and `--operators` you use for `run`. The command runs
278
+ no tests. It does not rewrite a baseline. Generate a new baseline with
279
+ `--format json`.
280
+
53
281
  ### Options
54
282
 
55
283
  | Flag | Meaning |
@@ -59,7 +287,7 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
59
287
  | `--threshold FLOAT` | Exit 1 when the score is below FLOAT, or when nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted (default: 0 = off) |
60
288
  | `--only NAME` | Restrict to one fully-qualified subject, e.g. `Calculator#add` |
61
289
  | `--framework NAME` | `minitest` (default) or `rspec`; auto-detected as rspec when most `--test` files end in `_spec.rb` |
62
- | `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`). Untracked files are scored in full unless Git ignores them. |
290
+ | `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`). Untracked files are scored in full unless Git ignores them. The report lists only suppressed mutants on changed lines, too. |
63
291
  | `--no-since` | Disable diff scoping; a typed no beats a `.mutineer.yml` `since:` key |
64
292
  | `--baseline FILE` | Compare against a prior `--format json` run; exit 1 on new survivors / score drop (score drop is skipped under `--since`, whose score covers a different denominator; see [CI](https://github.com/davidteren/mutineer#ci-gating)) |
65
293
  | `--baseline-epsilon FLOAT` | Score-drop tolerance for `--baseline` (default: 0) |
@@ -71,12 +299,14 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
71
299
  | `--timeout SECONDS` | Per-mutant time limit for in-process runs, in whole seconds (default: 10). A mutant whose tests run longer is a `timeout`; see [Timeouts and the score](https://github.com/davidteren/mutineer#timeouts-and-the-score). `--daemon` and `--test-command` keep their own limits |
72
300
  | `--capture-timeout SECONDS` | Time limit for each in-process coverage-capture subprocess and for the clean run of the unmutated tests, in whole seconds (default: 120). A suite slower than this stops the run as not green. `--daemon` uses it for its coverage capture; `--test-command` ignores it and says so |
73
301
  | `--cache-dir DIR` | Directory for the coverage cache (default: `.mutineer`). Give two runs from the same project root different directories so they do not share `coverage.json`. `--test-command` builds no cache, so it ignores it and says so |
302
+ | `--coverage-from FILE` | Reuse the `coverage.json` of an earlier `--rails`/`--boot` run with the same tests over these sources or more, so parallel shards capture coverage once. The run skips the capture. It runs each test that covers one of its sources alone, because a mutant runs only those tests, then all tests once together, and writes its own cache to `--cache-dir`. A test that covers none of its sources does not run alone. A capture that does not fit (changed tests or sources, an owner file added or removed, a source it did not cover, a failed capture, another `PARALLEL_WORKERS`, a cache from before this flag) stops the run (exit 2). Under `--strategy redefine`, eager-load in both runs. It has no effect, and says so, without `--rails`/`--boot` or with `--daemon`, `--test-command` or `--dry-run` |
74
303
  | `--test-command CMD` | Run the suite as a subprocess in the app's own runtime (for apps on Ruby < 3.4); `CMD` must contain `%{files}`. See [Apps on Ruby < 3.4](https://github.com/davidteren/mutineer#apps-on-ruby--34) |
75
- | `--daemon` | Boot the app once in a persistent daemon and fork per mutant, with per-worker DB isolation so `--jobs N` is safe under Rails (needs `--rails`/`--boot`; not with `--test-command`). See [the daemon backend](https://github.com/davidteren/mutineer#faster-parallel-safe-rails-the---daemon-backend) |
304
+ | `--daemon` | Boot the app once in a persistent daemon and fork per mutant, with per-worker DB isolation so `--jobs N` is safe under Rails (needs `--rails`/`--boot`; not with `--test-command`). See [the daemon backend](https://github.com/davidteren/mutineer#faster-parallel-safe-rails-the---daemon-backend). `--no-daemon` beats a `.mutineer.yml` `daemon:` key. It keeps `--rails` serial, which is the 1.x default, and in 2.0 it opts out of the parallel default |
76
305
  | `--format human\|json\|html` | Report format (default: human; `html` is a self-contained file) |
77
306
  | `--output FILE` | Write the report to FILE instead of stdout |
78
307
  | `--dry-run` | List candidate mutations without executing (honors suppression) |
79
308
  | `--fail-fast` | Stop at the first surviving mutant (`--no-fail-fast` beats a `.mutineer.yml` `fail_fast:` key) |
309
+ | `--allow-empty` | A run with no mutants is expected. Without it, an empty run warns that Mutineer 2.0 will fail it. A `--since` run whose changes hold no mutants does not warn: it reports that the changes hold nothing to test. A `--since` run with no mutants even before scoping still warns. `--no-allow-empty` beats a `.mutineer.yml` `allow_empty:` key |
80
310
  | `--matrix` | Run every covering test for each mutant and report the blind and redundant tests; the JSON report also lists each mutant's killers. In-process only; exits 2 with `--daemon`, `--test-command`, `--fail-fast` or `--dry-run`. RSpec needs 3.3 or later. `--no-matrix` beats a `.mutineer.yml` `matrix:` key. See [Kill matrix](https://github.com/davidteren/mutineer#kill-matrix) |
81
311
  | `--list-operators` | List available operators (default vs optional) and exit |
82
312
  | `--version`, `--help` | Print version / usage and exit |
@@ -98,7 +328,7 @@ Run `mutineer --list-operators` to see them. Default (Tier 1): `arithmetic`,
98
328
  Available but off by default (Tier 2, enable via `--operators`): `return_nil`,
99
329
  `literal_mutation`, `condition_negation`, `string_literal`, `regex`,
100
330
  `collection_method`, `safe_navigation`, `range`, `negation_removal`, `chain_link`,
101
- `operand_removal`, `array_literal`, `condition_true`,
331
+ `call_to_receiver`, `operand_removal`, `array_literal`, `condition_true`,
102
332
  `condition_false`, `operator_assignment`.
103
333
 
104
334
  ## Rails apps
@@ -119,12 +349,68 @@ different entry point. Boot mode requires at least one `--test` file and is
119
349
  coverage-guided — each mutant runs only the test files that exercise its line
120
350
  (coverage is captured by forking the booted app, then cached).
121
351
 
352
+ Some code runs while the app boots or a class loads, for example a class body
353
+ that calls a method to build a constant, a `to_prepare` initializer, or a
354
+ `--require` file that calls a source method. That run
355
+ happens before the mutant is applied, and the forked test does not repeat it.
356
+ A mutant on such a line is `ran_at_load`, not `survived` or `no_coverage`. It is
357
+ left out of the score and does not fail `--threshold`; a kill still counts.
358
+ This holds under both strategies: `reload` runs the mutated file's class body
359
+ again, but not an initializer or another file's code, so a survivor on such a
360
+ line is not trusted there either.
361
+ Run those mutants with `--test-command`, which boots a fresh process per mutant,
362
+ to get a verdict.
363
+
122
364
  Add Mutineer to your Gemfile's test group:
123
365
 
124
366
  ```ruby
125
367
  gem "mutineer", group: :test, require: false
126
368
  ```
127
369
 
370
+ Start from one command. It writes `.mutineer.yml` and prints the first run.
371
+ It does not replace a file you already wrote. Pass `--force` to replace one.
372
+
373
+ ```sh
374
+ RAILS_ENV=test bundle exec mutineer init --rails
375
+ ```
376
+
377
+ The path from that first report to a CI gate is the
378
+ [Rails rollout guide](https://davidteren.github.io/mutineer/rails.html).
379
+
380
+ ### Rails parallel tests, Spring, Bootsnap, and Redis
381
+
382
+ Mutineer sets `PARALLEL_WORKERS` to `1` for every run.
383
+ Rails then runs each mutant's tests in one process.
384
+ Your `parallelize(workers:)` call stays in `test_helper.rb`.
385
+ It does not start Rails workers during a mutant run.
386
+ If you already set `PARALLEL_WORKERS` to another value, Mutineer replaces it and prints one line.
387
+ With `--test-command`, a script your command runs can set `PARALLEL_WORKERS` again.
388
+ Its value wins, and the Rails workers start again.
389
+ Remove that line from the script, or write `PARALLEL_WORKERS=${PARALLEL_WORKERS:-4}` so it keeps the 1 Mutineer passes.
390
+
391
+ Mutineer does not run your `parallelize_setup` hooks.
392
+ Rails registers its own database hook in that same list.
393
+ Running the list would point the tests at `<base>-<n>` instead of Mutineer's database.
394
+
395
+ `--test-command` also sets `DISABLE_SPRING=1`.
396
+ Spring does not start for that command.
397
+ The `--daemon` path loads `config/environment` itself, so Spring does not start there either.
398
+ A wrapper that sets `DISABLE_SPRING` again wins, same as `PARALLEL_WORKERS`.
399
+
400
+ Bootsnap checks its compile cache by file path, size, and modification time in whole seconds.
401
+ When the size and the time match, it uses the cached code without reading the file content.
402
+ The in-process and `--daemon` paths load each mutant from a new file, so the cache cannot mix them up.
403
+ `--test-command` rewrites the source file in place.
404
+ Two mutants of the same size, written in the same second, can then load the first one's cached code.
405
+ To rule that out, set `DISABLE_BOOTSNAP_COMPILE_CACHE=1` on the Mutineer command.
406
+ Bootsnap keeps its cache in `tmp/cache/bootsnap`, or under `$BOOTSNAP_CACHE_DIR` when that is set.
407
+ If a run uses code that no longer matches your files, delete that folder and run again.
408
+
409
+ `redis-client` (which `redis` 5 uses) opens a new connection after a fork on its own.
410
+ You do not set anything in Mutineer for this.
411
+ `redis` 4.x raises `Redis::InheritedError` instead.
412
+ Open that connection after boot, or reconnect it in an after-fork hook.
413
+
128
414
  ### Faster, parallel-safe Rails (the `--daemon` backend)
129
415
 
130
416
  `--rails` boots your app once but runs mutants **serially** — parallel `--jobs`
@@ -144,14 +430,37 @@ RAILS_ENV=test bundle exec mutineer run \
144
430
  a mutant on an uncovered line is `no_coverage`, so the score stays comparable to
145
431
  the in-process `--rails` score.
146
432
  - **Safe `--jobs N`** — each worker routes to its own copy of the test database, so
147
- parallel verdicts equal serial (no fixture cross-talk).
433
+ parallel verdicts equal serial (no fixture cross-talk). On first use, a
434
+ worker's database is a copy of the test database after the app boots, so rows
435
+ written by initializers or `--require` files are there, as in-process. When
436
+ the copy's schema differs from `db/schema.rb` (another schema version or
437
+ `schema_sha1` checksum, or no stored checksum, as in an empty or out-of-date
438
+ test database, or one set up with a plain `load "db/schema.rb"`), the
439
+ worker loads `db/schema.rb` over it, which drops the copied rows of the
440
+ tables it defines.
148
441
  - **One backend at a time** — `--daemon` can't be combined with `--test-command`
149
442
  (choose one), and it needs an app to boot (`--rails` or `--boot`).
150
443
 
151
- Status: **SQLite** only (hermetic, CI-proven). Per-worker provisioning for
152
- **Postgres** and other adapters is not supported: on those, `--daemon` scores
153
- every mutant as `error`. Use `--daemon` with a SQLite test database, or drop
154
- `--daemon` to run serially on other adapters.
444
+ Status: **SQLite**, **PostgreSQL**, and **MySQL 8** (Minitest). Each Postgres worker uses a
445
+ database named `<base>-mutineer-<N>` (shortened when that would pass 63 bytes).
446
+ Mutineer copies the test database into those names before the workers start,
447
+ and leaves them in place when the run ends. Drop one with
448
+ `DROP DATABASE "myapp_test-mutineer-0";` (use your test database name and slot).
449
+ Each MySQL 8 worker uses a database named `<base>-mutineer-<N>` (shortened when that would pass 64 bytes).
450
+ Mutineer creates that database, loads `schema.rb` or `structure.sql`, and copies rows before the workers start.
451
+ It does not copy the database again after an error or a timeout.
452
+ The worker databases stay in place when the run ends. The next run drops and rebuilds them.
453
+ Triggers in the schema do not fire on the copied rows: in each worker database, Mutineer drops them for the copy and creates them again.
454
+ The database user needs CREATE, DROP, TRIGGER, and the table and row privileges a schema load and a row copy use (such as ALTER, INDEX, REFERENCES, and INSERT) on the worker databases, plus SELECT on the test database.
455
+ A trigger whose DEFINER is another account also needs SET_USER_ID.
456
+ An app on `structure.sql` also needs the `mysql` client on `PATH`.
457
+ Drop one with ``DROP DATABASE `myapp_test-mutineer-0`;`` (MySQL quotes the name with backticks).
458
+ A second run on the same test database stops with `another Mutineer run is using <base>` until the first run ends.
459
+ MySQL runs at most 256 workers.
460
+ `mysql2` and `trilogy` are supported.
461
+ MariaDB is not tested.
462
+ MySQL 5.7 is not supported.
463
+ A `:memory:` database is not supported on `--daemon`.
155
464
 
156
465
  ### Apps on Ruby < 3.4
157
466
 
@@ -180,7 +489,9 @@ RAILS_ENV=test mutineer run app/models/order.rb \
180
489
  prepends rbenv/asdf shims when a pin was scrubbed. Do not rely on
181
490
  `RBENV_VERSION=…` on the Mutineer command for the suite; use `.ruby-version`
182
491
  or a wrapper. Don't put `KEY=val` prefixes *inside* `--test-command` (no shell;
183
- that would be treated as the program name).
492
+ that would be treated as the program name). Mutineer also sets
493
+ `PARALLEL_WORKERS=1` and `DISABLE_SPRING=1` in the child and replaces a value
494
+ you set (see [Rails parallel tests, Spring, Bootsnap, and Redis](https://github.com/davidteren/mutineer#rails-parallel-tests-spring-bootsnap-and-redis)).
184
495
 
185
496
  #### Under a version manager (rbenv / asdf / chruby)
186
497
 
@@ -214,10 +525,13 @@ Mutineer also surfaces a targeted smoke-check message when Bundler prints
214
525
  Tradeoffs — this path is correct but not free:
215
526
 
216
527
  - **Slower:** your app re-boots for every mutant (no shared boot yet).
217
- - **No coverage narrowing:** every mutant runs the *full* `--test` set, so the
218
- score is an **upper bound and not comparable to an in-process (`--rails`)
219
- score** — uncovered mutants count as survivors, and an infrastructure failure
220
- is scored as a kill. Mutineer prints this caveat on every run and aborts up
528
+ - **No coverage narrowing:** every mutant runs the full `--test` set.
529
+ The run is serial.
530
+ A crash in the command can count as a kill.
531
+ Uncovered mutants count as survivors.
532
+ Do not compare a `--test-command` score with an in-process score.
533
+ Exit codes 126 and 127 (the command could not start) are scored `error`.
534
+ Mutineer prints this caveat on every run and aborts up
221
535
  front (a "smoke check") if your unmutated suite isn't green.
222
536
  - **Reload strategy only** (`--strategy redefine` is rejected on this path) and
223
537
  **serial** (`--jobs` is forced to 1). For apps on Ruby ≥ 3.4, `--daemon` gives
@@ -227,7 +541,7 @@ Tradeoffs — this path is correct but not free:
227
541
 
228
542
  The mutation score is `killed / (killed + survived)`. A mutant whose tests run
229
543
  past `--timeout` is a `timeout`. It is neither killed nor survived, so it is
230
- left out of the score, like `no_coverage`, `uncapturable`, `errored`, skipped
544
+ left out of the score, like `no_coverage`, `uncapturable`, `unplaceable`, `ran_at_load`, `errored`, skipped
231
545
  and ignored mutants. A timeout is not counted as a kill, because a hang the
232
546
  mutant caused and a suite that is just slow look the same. The human report
233
547
  shows the count in its `Timeout:` row, and the JSON report in `summary.timeout`.
@@ -244,9 +558,12 @@ and once they pass 10% of the attempted mutants the run fails.
244
558
  Some mutants are equivalent (behaviour-identical) and survive forever — keeping a
245
559
  file off 100%. Suppress them so the score and `--threshold` gate stay meaningful:
246
560
 
247
- - **Inline:** `some_line # mutineer:disable-line` (or scope it: `# mutineer:disable-line comparison`). Put a reason after `--`: `# mutineer:disable-line comparison -- the test checks only 20`.
248
- - **Config:** a `.mutineer.yml` `ignore:` list of mutant ids. Each survivor's
249
- `id` is printed in the JSON report, so copy it straight into `ignore:`.
561
+ - **Inline:** `some_line # mutineer:disable-line` (or scope it: `# mutineer:disable-line comparison`). Put a reason after `--`: `# mutineer:disable-line comparison -- the test checks only 20`. The reason is stored on the ignored mutant and shown in the JSON report and the HTML report. A marker with no reason stores none.
562
+ - **Config:** a `.mutineer.yml` `ignore:` list. A bare mutant id still works. A mapping is `id` plus an optional `reason`. Mutineer 1.7 is the first version that reads a mapping. An older version matches no mutant to a mapping and does not warn, so the mutant it names is not ignored. Add a mapping only after every gate that reads the file runs 1.7 or later. In the Action, the `version:` input sets the version, or your `Gemfile.lock` does when you set `use-bundler: true`. In your own CI job, your `Gemfile.lock` sets it. The reason shows in the same reports as an inline reason. Each survivor's `id` is printed in the JSON report.
563
+
564
+ `mutineer triage REPORT.json --reason TEXT --all` prints one mapping entry per survivor. `--id ID` prints one entry. Repeat `--id` for several. The command does not edit files. Paste the lines under `ignore:`. They are mappings, so every gate that reads the file must run 1.7 or later (see **Config** above).
565
+
566
+ Built-in skips are listed on the [equivalent skips](https://davidteren.github.io/mutineer/equivalent-skips.html) page. There is no switch to turn them off.
250
567
 
251
568
  Suppressed mutants are excluded from the score (so 100% becomes reachable).
252
569
 
@@ -279,14 +596,18 @@ working until 2.0, with a warning:
279
596
  hid mutants you did not mean to ignore. The warning says so. Keep only the ids for the mutant you meant to
280
597
  ignore, not all of them. The list covers only the sources and operators in
281
598
  that run, so run over every source with every operator set you use (for
282
- example your Tier-2 `--operators`) for the full list.
599
+ example your Tier-2 `--operators`) for the full list. Or run
600
+ `mutineer migrate` on those sources. It writes every new id. Delete any id
601
+ you did not mean to ignore. See [Migrate old ignore ids](https://github.com/davidteren/mutineer#migrate-old-ignore-ids).
283
602
  - **`--baseline`** An old baseline still matches: a survivor matches a stored
284
603
  one with the same old id in the same file. A stored file that is an absolute
285
604
  path outside the project root (a baseline written on another machine)
286
605
  matches on the old id alone. The run tells you to
287
- regenerate it. Regenerate it with `--format json`, but only after every gate
288
- that reads it runs 1.3 or later (the Action's `version:` pin, your CI
289
- `Gemfile.lock`). An older version treats every new-format survivor as new.
606
+ generate a new baseline with `--format json`. `migrate` does not rewrite a
607
+ baseline. Generate the new file only after every gate
608
+ that reads it runs 1.3 or later. In the Action, the `version:` input sets
609
+ the version, or your `Gemfile.lock` does when you set `use-bundler: true`.
610
+ In your own CI job, your `Gemfile.lock` sets it. An older version treats every new-format survivor as new.
290
611
 
291
612
  The JSON report's `summary.id_format` is `2` for the new format.
292
613
  `summary.legacy_id_matches.ignore` counts the old-format ignore entries a run
@@ -435,6 +756,10 @@ structured exit codes, and diff-scoped runs. See:
435
756
  [source](https://davidteren.github.io/mutineer/json-schema.md)
436
757
  - **Ruby API (YARD)** — class reference for the shipped gem:
437
758
  [https://davidteren.github.io/mutineer/api/](https://davidteren.github.io/mutineer/api/)
759
+ - **Agent skill** ([`skills/mutineer/SKILL.md`](https://github.com/davidteren/mutineer/blob/main/skills/mutineer/SKILL.md)): a short
760
+ card for coding agents with install, the agent loop, and exit codes. Install it
761
+ with `gh skill install davidteren/mutineer mutineer` or
762
+ `npx skills add davidteren/mutineer --skill mutineer`.
438
763
 
439
764
  ## Configuration
440
765
 
@@ -465,6 +790,7 @@ config file accepts these keys:
465
790
  | `capture_timeout` | A positive integer; the coverage-capture time limit in seconds (default 120) |
466
791
  | `cache_dir` | The coverage cache directory (default `.mutineer`) |
467
792
  | `matrix` | `true` or `false`; runs every covering test for each mutant and adds the kill matrix to the report |
793
+ | `allow_empty` | `true` or `false`; a run with no mutants is expected, so it does not warn |
468
794
 
469
795
  Invalid values for known scalar keys, and a blank `operators` list, exit 2
470
796
  with a message naming the file and key. An unknown operator name warns and
@@ -474,9 +800,9 @@ is skipped. If none of the names are known, the run exits 2. An empty
474
800
  `threshold` and the CLI-only `--baseline-epsilon` use plain decimals such as `90`
475
801
  or `0.5`, not `+2`, `1e2`, or `1_0`. String options such as `only` and `baseline`
476
802
  cannot be null or boolean. A blank `since` is invalid; use `since: false` to turn
477
- scoping off. Unknown keys warn and are ignored. Unknown operator names warn and are skipped, and the run exits 2 when none remain. `--operators` replaces a blank or unknown file list.
803
+ scoping off. Unknown keys warn and are ignored. Unknown operator names warn and are skipped, and the run exits 2 when none remain. Both warnings suggest the closest valid name, and both become errors in Mutineer 2.0. `--operators` replaces a blank or unknown file list.
478
804
 
479
- `format`, `strategy`, `output`, `baseline_epsilon`, and `dry_run` are CLI-only.
805
+ `format`, `strategy`, `output`, `baseline_epsilon`, `dry_run`, and `coverage_from` are CLI-only.
480
806
  For JSON output, use `--format json`, not a `format:` config key. To select RSpec
481
807
  in the file, add `framework: rspec`.
482
808
 
@@ -510,7 +836,7 @@ no code with Mutant.
510
836
  | Test frameworks | Minitest and RSpec, in one gem | RSpec, Minitest and Test::Unit, one integration gem each |
511
837
  | What you mutate | Files (`mutineer run lib/foo.rb --test test/foo_test.rb`), narrowed to one method with `--only 'Foo#bar'` | Subjects named by expression (`mutant run 'Foo#bar'`, `'Foo*'`) |
512
838
  | Which tests run | The test files whose coverage reaches the mutated line | The tests that declare the subject (an RSpec description or a Minitest `cover`), or the tests that ran it in a per-test coverage recording |
513
- | Operators | 20 (5 by default, 15 opt-in) | A larger set; the default `light` set applies almost all of it (`full` adds `#==` to `#eql?`) |
839
+ | Operators | 21 (5 by default, 16 opt-in) | A larger set; the default `light` set applies almost all of it (`full` adds `#==` to `#eql?`) |
514
840
  | CI gating | `--threshold`, `--baseline` deltas, `--since`, and a GitHub Action | Incremental mode (`--since`) and a recorded session history |
515
841
  | Machine-readable output | Versioned JSON report schema | Session JSON schema |
516
842
 
@@ -0,0 +1,53 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Mutineer
4
+ # Lets a test helper start Coverage in a forked mutant child (#248).
5
+ #
6
+ # Once the coverage map is built, the parent suspends Coverage so mutant
7
+ # children do not pay for it (#228). A suspended Coverage reports
8
+ # `running? == false`, so a test helper that starts it when it is off
9
+ # (SimpleCov's `SimpleCov.start`) calls `Coverage.start`, which raises
10
+ # "coverage measurement is already setup" while Coverage is suspended. Every
11
+ # mutant then errors before a test runs.
12
+ #
13
+ # In the child, `Coverage.start` and `Coverage.setup` first stop the
14
+ # suspended Coverage, then run as usual. Stopping builds and drops the whole
15
+ # result, which walks the heap, so only a child whose helper starts Coverage
16
+ # pays for it. Only files loaded after that start are measured. The parent's
17
+ # Coverage is untouched.
18
+ #
19
+ # Stdlib-only, so the app-side daemon can load it.
20
+ module ChildCoverage
21
+ # Prepended to Coverage's singleton class in the child: stops a suspended
22
+ # Coverage before a start or setup.
23
+ module StopBeforeStart
24
+ # Stops a suspended Coverage, then sets it up.
25
+ #
26
+ # @return [nil]
27
+ def setup(...)
28
+ result(stop: true, clear: true) if state == :suspended
29
+ super
30
+ end
31
+
32
+ # Stops a suspended Coverage, then starts it.
33
+ #
34
+ # @return [nil]
35
+ def start(...)
36
+ result(stop: true, clear: true) if state == :suspended
37
+ super
38
+ end
39
+ end
40
+
41
+ # Makes a later `Coverage.start` or `Coverage.setup` work when Coverage is
42
+ # suspended. A running or idle Coverage is left alone.
43
+ #
44
+ # Call it only in a forked child, before any test loads.
45
+ #
46
+ # @return [void]
47
+ def self.allow_start
48
+ return unless defined?(::Coverage) && ::Coverage.state == :suspended
49
+
50
+ ::Coverage.singleton_class.prepend(StopBeforeStart)
51
+ end
52
+ end
53
+ end