mutation_tester 1.5.0 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (31) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +12 -0
  3. data/Gemfile.lock +1 -1
  4. data/docs/ci.md +31 -0
  5. data/docs/execution-runners.md +23 -0
  6. data/docs/json-schema.md +44 -0
  7. data/examples/github_actions/ai_mutation_gate.yml +1 -1
  8. data/examples/github_actions/mutation_test.yml +1 -1
  9. data/examples/github_actions/redundant_tests.yml +77 -0
  10. data/exe/mutation_test +11 -1
  11. data/lib/mutation_tester/batch_runner.rb +3 -2
  12. data/lib/mutation_tester/configuration.rb +3 -1
  13. data/lib/mutation_tester/core.rb +37 -9
  14. data/lib/mutation_tester/fork_runner/worker.rb +4 -2
  15. data/lib/mutation_tester/fork_runner.rb +5 -4
  16. data/lib/mutation_tester/minitest_fail_fast.rb +3 -5
  17. data/lib/mutation_tester/minitest_load_hook.rb +35 -0
  18. data/lib/mutation_tester/mutation_runner.rb +50 -37
  19. data/lib/mutation_tester/progress_display.rb +134 -30
  20. data/lib/mutation_tester/reporters/base_reporter.rb +2 -1
  21. data/lib/mutation_tester/reporters/batch_json_reporter.rb +2 -1
  22. data/lib/mutation_tester/reporters/console_reporter.rb +1 -0
  23. data/lib/mutation_tester/reporters/json_reporter.rb +7 -0
  24. data/lib/mutation_tester/test_command.rb +60 -30
  25. data/lib/mutation_tester/test_recorder/minitest_hook.rb +55 -0
  26. data/lib/mutation_tester/test_recorder/rspec_hook.rb +62 -0
  27. data/lib/mutation_tester/test_recorder.rb +75 -0
  28. data/lib/mutation_tester/version.rb +1 -1
  29. data/lib/mutation_tester.rb +1 -0
  30. data/readme.md +421 -538
  31. metadata +7 -2
data/readme.md CHANGED
@@ -2,29 +2,60 @@
2
2
 
3
3
  [![CI](https://github.com/Oxyconit/mutation_tester/actions/workflows/ci.yml/badge.svg)](https://github.com/Oxyconit/mutation_tester/actions/workflows/ci.yml)
4
4
 
5
- Simple mutation testing framework for Ruby applications with RSpec or Minitest. Improve your AI workflow and test
6
- quality by identifying weak spots in your test suite through code mutations.
5
+ Simple mutation testing framework for Ruby applications with RSpec or Minitest. It mutates your code, runs your tests
6
+ against each mutant, and reports every change your tests failed to detect, so you (or your AI agent) know exactly which
7
+ test gaps to close.
8
+
9
+ ## Quick Start
7
10
 
8
11
  ```bash
9
- # Install, then mutation-test a file (its spec is found by convention):
12
+ # Install
10
13
  bundle add mutation_tester
14
+
15
+ # Test a file: its spec is found by convention (lib/X.rb -> spec/X_spec.rb)
11
16
  bundle exec mutation_test lib/calculator.rb
17
+
18
+ # Test several files in one aggregated run
19
+ bundle exec mutation_test lib/calculator.rb lib/parser.rb
20
+
21
+ # Test exactly what you have staged in git (the "test what I changed" flow)
22
+ bundle exec mutation_test --staged
23
+
24
+ # Minitest layout under test/
25
+ bundle exec mutation_test --spec-glob 'test/{name}_test.rb' lib/calculator.rb
26
+
27
+ # Or point at the test file explicitly
28
+ bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb
12
29
  ```
13
30
 
14
- See [Getting Started](#getting-started) for the full quick start.
31
+ The gem will:
32
+
33
+ - ✅ Run your original tests to make sure they pass
34
+ - ✅ Generate mutations of your code
35
+ - ✅ Run tests against each mutation
36
+ - ✅ Generate reports showing which mutations survived
37
+
38
+ Press Ctrl+C at any time to stop early: the run cleans up its worker processes and temporary workspaces, prints a single
39
+ interruption line (no backtrace), and exits with status 130.
15
40
 
16
41
  ## Table of Contents
17
42
 
18
43
  - [Features](#features)
19
44
  - [Requirements and Compatibility](#requirements-and-compatibility)
20
45
  - [Installation](#installation)
21
- - [Getting Started](#getting-started)
22
46
  - [Usage](#usage)
47
+ - [Choosing what to test](#choosing-what-to-test)
48
+ - [Mapping sources to specs](#mapping-sources-to-specs)
49
+ - [CLI options](#cli-options)
50
+ - [Exit codes](#exit-codes)
51
+ - [Running with rake](#running-with-rake)
52
+ - [Programmatic usage](#programmatic-usage)
23
53
  - [Configuration](#configuration)
24
54
  - [Execution model](#execution-model)
25
55
  - [Mutation types](#mutation-types)
26
56
  - [Equivalent mutants](#equivalent-mutants)
27
57
  - [Reports and output](#reports-and-output)
58
+ - [Finding redundant tests](#finding-redundant-tests)
28
59
  - [Pre-push hook](#pre-push-hook)
29
60
  - [CI/CD integration](#cicd-integration)
30
61
  - [Troubleshooting](#troubleshooting)
@@ -33,8 +64,8 @@ See [Getting Started](#getting-started) for the full quick start.
33
64
  - [License](#license)
34
65
 
35
66
  Reference material lives under [`docs/`](docs): the full
36
- [mutation catalog](docs/mutation-types.md), the [JSON report schema](docs/json-schema.md),
37
- and the [CI/CD recipes](docs/ci.md).
67
+ [mutation catalog](docs/mutation-types.md), the [execution runner internals](docs/execution-runners.md),
68
+ the [JSON report schema](docs/json-schema.md), and the [CI/CD recipes](docs/ci.md).
38
69
 
39
70
  ## Features
40
71
 
@@ -74,9 +105,6 @@ Notes:
74
105
 
75
106
  ## Installation
76
107
 
77
- MutationTester requires Ruby >= 3.0 (see [Requirements and Compatibility](#requirements-and-compatibility)). There are
78
- two supported ways to install it.
79
-
80
108
  ### In your project's bundle (recommended)
81
109
 
82
110
  Add it to your application's `Gemfile` and install in one step:
@@ -85,336 +113,149 @@ Add it to your application's `Gemfile` and install in one step:
85
113
  bundle add mutation_tester
86
114
  ```
87
115
 
88
- (this writes `gem "mutation_tester"` into your `Gemfile` and runs `bundle install`).
89
-
90
116
  Then run it through Bundler so it uses your project's locked dependency versions:
91
117
 
92
118
  ```bash
93
- bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb
119
+ bundle exec mutation_test lib/calculator.rb
94
120
  ```
95
121
 
96
122
  ### As a global gem
97
123
 
98
- Install it once, system-wide:
124
+ Install it once, system-wide, and run `mutation_test` directly (no `bundle exec`):
99
125
 
100
126
  ```bash
101
127
  gem install mutation_tester
128
+ mutation_test lib/calculator.rb
102
129
  ```
103
130
 
104
- Then run the `mutation_test` command directly (no `bundle exec`):
105
-
106
- ```bash
107
- mutation_test app/models/user.rb spec/models/user_spec.rb
108
- ```
109
-
110
- This is convenient for a project that does not list the gem in its `Gemfile`.
111
- The CLI then cannot load itself from that project bundle, so it loads the
112
- globally installed gem *outside* the bundle and prints one line to stderr:
131
+ This is convenient for a project that does not list the gem in its `Gemfile`. The CLI then cannot load itself from that
132
+ project bundle, so it loads the globally installed gem *outside* the bundle and prints one line to stderr:
113
133
 
114
134
  ```
115
135
  mutation_tester loaded outside the project bundle
116
136
  ```
117
137
 
118
- In this fallback the gem and its own dependencies (parser, unparser, parallel,
119
- rainbow) come from the global install, so their versions may differ from your
120
- project's `Gemfile.lock`. Your own tests are unaffected: when your project has a
121
- `Gemfile`, each mutant still runs through `bundle exec`, in your project's
122
- environment. Running in a directory with no `Gemfile` at all works too and
123
- prints no notice. If your project *does* list `mutation_tester`, prefer
124
- `bundle exec mutation_test ...`, which runs fully inside your bundle with no
125
- notice.
126
-
127
- ## Getting Started
128
-
129
- Install the gem, then point `mutation_test` at one or more source files. Each
130
- file is mapped to its spec by convention (`lib/X.rb` -> `spec/X_spec.rb`,
131
- override with `--spec-glob`):
132
-
133
- ```bash
134
- # Install
135
- bundle add mutation_tester
136
-
137
- # Test a file: the spec is found by convention (lib/X.rb -> spec/X_spec.rb)
138
- bundle exec mutation_test lib/calculator.rb
139
-
140
- # Test several files in one aggregated run
141
- bundle exec mutation_test lib/calculator.rb lib/parser.rb
142
-
143
- # Test exactly what you have staged in git (the "test what I changed" flow)
144
- bundle exec mutation_test --staged
145
-
146
- # Minitest layout under test/
147
- bundle exec mutation_test --spec-glob 'test/{name}_test.rb' lib/calculator.rb
148
-
149
- # Or point at the test file explicitly: exactly two arguments, the second a test file
150
- bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb
151
- bundle exec mutation_test lib/calculator.rb test/calculator_test.rb
152
-
153
- # Or with rake (non-Rails: first add `require 'mutation_tester/rake_task'` to your Rakefile)
154
- bundle exec rake "mutation_test[app/models/user.rb,spec/models/user_spec.rb]"
155
- ```
156
-
157
- That's it! 🎉 The gem will:
158
-
159
- - ✅ Run your original tests to make sure they pass
160
- - ✅ Generate mutations of your code
161
- - ✅ Run tests against each mutation
162
- - ✅ Generate reports showing which mutations survived
163
-
164
- Press Ctrl+C at any time to stop early: the run cleans up its worker processes and temporary workspaces, prints a single
165
- interruption line (no backtrace), and exits with status 130.
166
-
167
- See [Usage](#usage) for the full command reference and [Configuration](#configuration) to tune it.
138
+ In this fallback the gem and its own dependencies (parser, unparser, parallel, rainbow) come from the global install, so
139
+ their versions may differ from your project's `Gemfile.lock`. Your own tests are unaffected: when your project has a
140
+ `Gemfile`, each mutant still runs through `bundle exec`, in your project's environment. Running in a directory with no
141
+ `Gemfile` at all works too and prints no notice. If your project *does* list `mutation_tester`, prefer
142
+ `bundle exec mutation_test ...`, which runs fully inside your bundle with no notice.
168
143
 
169
144
  ## Usage
170
145
 
171
- `mutation_test` is the main CLI executable for running mutation tests. Pass one
172
- or more source files (specs mapped by convention), the git staging area, an
173
- explicit pair, or a glob:
174
-
175
146
  ```bash
176
- # Main interface: one or more source files, specs mapped by convention
147
+ # One or more source files, specs mapped by convention (the main interface)
177
148
  mutation_test [OPTIONS] FILE...
178
149
 
179
- # The files currently staged in git (see File lists and --staged below)
150
+ # The files currently staged in git
180
151
  mutation_test [OPTIONS] --staged
181
152
 
182
153
  # Explicit pair: exactly two arguments where the second is a test file
183
154
  mutation_test [OPTIONS] SOURCE_FILE TEST_FILE
184
155
 
185
- # Many files in one run: select sources with a glob (see Batch mode below)
156
+ # Many files in one run: select sources with a glob
186
157
  mutation_test [OPTIONS] --glob 'lib/**/*.rb'
187
158
  ```
188
159
 
189
- **Arguments:**
160
+ ### Choosing what to test
190
161
 
191
- - `FILE...` - Ruby source files to mutate; each is mapped to its spec by convention (`lib/X.rb` -> `spec/X_spec.rb`,
192
- override with `--spec-glob`). Files that cannot be mutated are reported as skipped with a reason (
193
- see [File lists and --staged](#file-lists-and---staged-test-what-you-changed)).
194
- - `SOURCE_FILE TEST_FILE` - Explicit pair: with exactly two arguments where the second is recognized as a test file (
195
- `*_spec.rb`, `*.spec.rb`, `*_test.rb`, `test_*.rb`, or a file requiring minitest), the second is used as the test file
196
- directly.
162
+ **File list (`FILE...`)** is the main interface. Each source file is mapped to its spec by convention (`lib/X.rb` ->
163
+ `spec/X_spec.rb`; see [Mapping sources to specs](#mapping-sources-to-specs) to override), and the whole list runs as one
164
+ aggregated batch: every file is processed, the summary shows one `PASS`/`FAIL` line per file, reports land in per-file
165
+ subdirectories under `--output-dir`, and the exit code reflects the whole run.
197
166
 
198
- ### CLI options
199
-
200
- | Flag | Description |
201
- |---|---|
202
- | `-p, --parallel N` | Run with N parallel processes (default: auto, derived from the CPU core count with a cap of 8; `-p 1` forces serial execution). |
203
- | `--runner MODE` | Mutant execution runner: `auto` (default) tries `in_memory` first (`Process.fork` available and a passing unmutated-source probe), then falls back to `fork`, then `spawn`, announcing every step down on stderr with its reason; `fork` (preloaded environment, on platforms with `Process.fork`), `spawn` (one full process per mutant) and `in_memory` (mutations applied in child-process memory, zero file writes per mutant) force the specific mode. See [Execution runners](#execution-runners-fork-spawn-in-memory). |
204
- | `--staged` | Mutation-test the files staged in git (`git diff --cached --name-only`; files staged as deleted are ignored), mapping each to its spec like a positional `FILE` list. Cannot be combined with positional arguments or `--glob`. See [File lists and --staged](#file-lists-and---staged-test-what-you-changed). |
205
- | `--glob PATTERN` | Batch mode: mutation-test every source file matching `PATTERN`, mapping each to its spec by convention (see [Batch mode](#batch-mode-run-many-files-in-one-command)). |
206
- | `--spec-glob TEMPLATE` | Spec-mapping template with a `{name}` placeholder (default: `spec/{name}_spec.rb`). Requires a positional `FILE` list, `--staged`, or `--glob`. |
207
- | `--spec-map RULE` | Spec-mapping rule `'PATTERN=>REPLACEMENT'`: a regular expression applied to the whole source path to build the whole spec path, for layouts a `{name}` template cannot express (Rails `app/` -> `test/`, engines, Packwerk packs). Repeatable, first matching rule wins, a source matching no rule falls back to `--spec-glob`. Requires a positional `FILE` list, `--staged`, or `--glob`. See [Mapping sources to specs](#mapping-sources-to-specs). |
208
- | `--minimum-score N` | Mutation score percentage a file must reach to pass (default: 80). Drives the `PASS`/`FAIL` verdict and the exit code. |
209
- | `--since REV` | Incremental batch mode: mutate only the files matched by `--glob` that changed since git revision `REV` (new files count as changed). Requires `--glob`. See [Incremental mode](#incremental-mode-mutate-only-what-changed). |
210
- | `--fail-fast` | Stop the run at the first surviving mutant and finish with a failing status. Works in single-file mode and with `--glob`. |
211
- | `--timeout-factor N` | Per-mutant timeout budget as `N` times the measured baseline test run, never below 5 s (default: 5, must be > 0). Ignored when `config.timeout` is set explicitly, which keeps a fixed budget. See [Configuration](#configuration). |
212
- | `--timeout-policy MODE` | Scoring policy for timed-out mutants: `killed` (default) counts a timeout as a kill; `separate` keeps timeouts out of the score entirely (`killed / (killed + survived)`) and reports them only as their own category in the console, JSON and HTML reports. |
213
- | `--worker-env NAME` | Set environment variable `NAME` to a distinct per-worker value before each parallel worker boots (`parallel_tests` `TEST_ENV_NUMBER` convention: worker 0 -> `""`, worker N -> `N+1`), so a `parallel_tests`-style `database.yml` selects a per-worker database. You provision the databases (e.g. `rake parallel:prepare`). A parallel `in_memory` run falls back to `fork` unless `--after-fork` is also given; a serial run (`-p 1`) decides mutants in a shadow workspace instead of mutating the checkout in place. See [Making parallelism work with Rails](#making-parallelism-work-with-rails). |
214
- | `--after-fork FILE` | Ruby file loaded inside each preloaded in-memory clone right after it forks and receives its per-worker environment (see `--worker-env`), so the app can re-establish per-worker state such as its database connection. With `--worker-env` set, this keeps the `in_memory` runner available in parallel runs. A clone whose after-fork file raises is dropped with a stderr warning and its worker falls back to file-based execution. See [Making parallelism work with Rails](#making-parallelism-work-with-rails). |
215
- | `--strict-equality` | Enable the opt-in strict-equality probes (`==` → `eql?` and `==` → `equal?`). Default off; expect noise on code that does not distinguish numeric types or object identity. See [Strict Equality Mutations](docs/mutation-types.md#strict-equality-mutations-opt-in). |
216
- | `-h, --help` | Show help message. |
217
- | `-v, --version` | Show version. |
218
- | `--verbose` | Show a per-mutation warning for every skipped mutation (quiet by default; the "Generated N mutations, skipped M" summary always prints when mutants are dropped). |
219
- | `--no-progress` | Disable progress display. |
220
- | `--no-test-selection` | Disable two-phase test selection and always run the full test file for every mutant. See [Test selection](#test-selection-fast-kill-with-full-file-confirmation). |
221
- | `--json` | Machine mode: print ONLY the JSON report to stdout (banner, progress and colours go to stderr). A single file prints the per-file report; a multi-file run (`FILE` list with more than one file, `--staged`, `--glob`) prints one aggregate envelope with a condensed `survivors` list. See [Reports and output](#reports-and-output). |
222
- | `--reporters LIST` | Comma-separated reporters to run: `console`, `html`, `json` (default: `console,html,json`). An unknown name errors and exits 1. |
223
- | `--output-dir PATH` | Directory for the generated report files (default: `tmp/mutation_reports`). In batch mode each file writes to its own subdirectory under this path. |
167
+ **`--staged`** reads the file list from the git staging area (`git diff --cached --name-only`), so it is the natural
168
+ "test what I changed" flow:
224
169
 
225
170
  ```bash
226
- # Choose which reporters run and where their files land
227
- bundle exec mutation_test --reporters json,html --output-dir build/mutation \
228
- app/models/user.rb spec/models/user_spec.rb
171
+ bundle exec mutation_test --staged
172
+ # equivalent, from the repository root:
173
+ bundle exec mutation_test $(git diff --cached --name-only)
229
174
  ```
230
175
 
231
- ### Exit codes (single-file mode)
232
-
233
- - `0` - the run passed (score met the threshold, or `fail_on_threshold` is disabled).
234
- - `1` - the mutation score is below the threshold, or the input is unusable (missing
235
- file, unknown reporter, source with a syntax error).
236
- - `2` - a usage error (conflicting flags; see the batch sections below).
237
- - `3` - the run aborted or degraded before reaching a verdict: the shadow workspace
238
- was unreliable (the unmutated source failed there, or the workspace copy of the
239
- source turned out not to be the code the tests execute), or every mutant ended as
240
- `error`/`stillborn` so nothing was scored.
241
- This signals an infrastructure or runner problem, not a test-quality gap, so CI
242
- hooks can distinguish it from a genuine threshold failure.
243
- - `130` - interrupted with Ctrl+C.
244
-
245
- Batch modes (`FILE...` lists, `--staged`, `--glob`) keep the exit codes documented in
246
- their sections below (`0`/`1`/`2`); a degraded file is named explicitly in the batch
247
- summary instead of being blamed on the threshold.
248
-
249
- ### Running with rake
250
-
251
- You can also run mutation tests through rake tasks.
252
-
253
- In a **non-Rails** project, require the tasks from your `Rakefile`:
254
-
255
- ```ruby
256
- # Rakefile
257
- require 'mutation_tester/rake_task'
258
- ```
176
+ Files staged as deleted are ignored. Outside a git repository, or when nothing is staged, the CLI prints a readable
177
+ error instead of running. `--staged` cannot be combined with positional arguments or `--glob`.
259
178
 
260
- In a **Rails** app the tasks load automatically through the gem's railtie, so no
261
- Rakefile change is needed.
179
+ **Explicit pair (`SOURCE_FILE TEST_FILE`)**: with exactly two arguments where the second is recognized as a test file
180
+ (`*_spec.rb`, `*.spec.rb`, `*_test.rb`, `test_*.rb`, or a file requiring minitest), the second is used as the test file
181
+ directly. Two source files enter list mode instead. An RSpec test file with an unconventional name (no `_spec.rb`
182
+ suffix) is not recognized, so that pair is treated as a file list; rename the test or use the conventional layout to get
183
+ the explicit pair.
262
184
 
263
- Then run either task. Rake takes the file arguments inside brackets (not
264
- space-separated), so quote the invocation for your shell:
185
+ **`--glob PATTERN`** mutation-tests every source file the pattern matches in a single run, so you do not need to script
186
+ a loop around `mutation_test`:
265
187
 
266
188
  ```bash
267
- # Top-level task
268
- bundle exec rake "mutation_test[app/models/user.rb,spec/models/user_spec.rb]"
269
-
270
- # Namespaced task
271
- bundle exec rake "mutation:test[app/models/user.rb,spec/models/user_spec.rb]"
189
+ # Minitest project, JSON report per file, custom output directory
190
+ bundle exec mutation_test --glob 'lib/**/*.rb' --spec-glob 'test/{name}_test.rb' \
191
+ --reporters json --output-dir build/mutation
272
192
  ```
273
193
 
274
- ### Programmatic usage
275
-
276
- You can also run the gem directly from Ruby with `MutationTester.run(source, test)`:
277
-
278
- ```ruby
279
- # RSpec
280
- MutationTester.run('examples/calculator.rb', 'examples/calculator_spec.rb')
194
+ **`--since REV`** narrows a `--glob` batch to the files that changed since a git revision, which is how you keep
195
+ mutation testing affordable on pull requests:
281
196
 
282
- # Minitest
283
- MutationTester.run('examples/calculator.rb', 'examples/calculator_minitest.rb')
197
+ ```bash
198
+ bundle exec mutation_test --glob 'lib/**/*.rb' --since origin/main
284
199
  ```
285
200
 
286
- ### File lists and --staged: test what you changed
287
-
288
- Passing one or more source files is the main interface. Each file is mapped to
289
- its spec by convention (`lib/X.rb` -> `spec/X_spec.rb`, override with
290
- `--spec-glob`), and the whole list runs as one aggregated batch: every file is
291
- processed, the summary shows one `PASS`/`FAIL` line per file, reports land in
292
- per-file subdirectories, and the exit code reflects the whole run.
201
+ A matched file counts as changed when `git diff --name-only REV` lists it; new files the revision does not know about
202
+ (committed or still untracked) also count as changed. Unchanged matched files are reported as
203
+ `SKIPPED (unchanged since REV)` and never mutated. When nothing changed, the run succeeds with a "Nothing to mutate"
204
+ message and exit code `0`, so a PR that does not touch your sources does not fail the gate.
293
205
 
294
- ```bash
295
- bundle exec mutation_test lib/calculator.rb lib/parser.rb
206
+ **`--fail-fast`** turns the run into a cheap gate: the run stops as soon as one mutant survives, the reports contain the
207
+ results obtained up to that point with a clear interruption notice, and the process finishes with exit code `1`. It
208
+ works in single-file mode, with `--glob` (the batch stops and remaining files are not run), and in parallel mode.
296
209
 
297
- # The natural "test what I changed" flow:
298
- bundle exec mutation_test --staged
299
- bundle exec mutation_test $(git diff --cached --name-only)
300
- ```
210
+ #### Batch behaviour and skipped files
301
211
 
302
- `--staged` reads the list from the git staging area (`git diff --cached
303
- --name-only`), so the two commands above are equivalent when run from the
304
- repository root. Files staged as deleted are ignored. Outside a git repository,
305
- or when nothing is staged, the CLI prints a readable error instead of running.
306
- `--staged` cannot be combined with positional arguments or `--glob`.
307
-
308
- When any mutant survives, the aggregate summary ends with a survivors section:
309
- one `file:line original -> mutated` line per surviving mutant. A surviving
310
- mutant is a change to your code that your tests do not detect, so each line is
311
- a concrete test gap to close. With zero survivors the section is absent.
312
-
313
- The list may contain anything a real `git diff` produces; unmutable entries are
314
- reported as `SKIPPED` with an explicit reason and never count as a success:
315
-
316
- - **file not found** - the path does not exist.
317
- - **not a Ruby source file** - e.g. a staged `.md` or config file.
318
- - **a test file, not a mutable source** - a test file passed directly
319
- (`*_spec.rb`, `*.spec.rb`, `*_test.rb`, `test_*.rb`, or minitest content).
320
- - **no matching spec file** - the convention (or `--spec-glob` / `--spec-map`)
321
- points at a spec that does not exist; the expected path is printed.
322
-
323
- Exit codes: `0` when at least one file was processed and every processed file
324
- met the threshold; `1` when any processed file was below threshold or when
325
- every listed file was skipped (nothing was actually mutation-tested); `2` for
326
- usage errors (flag conflicts, `--staged` outside a git repository).
327
-
328
- **Legacy pair heuristic:** exactly two arguments where the second is recognized
329
- as a test file (by the name patterns above or by minitest content) keep the
330
- original `SOURCE_FILE TEST_FILE` behavior. Two source files enter list mode. An
331
- RSpec test file with an unconventional name (no `_spec.rb` suffix) is not
332
- recognized, so that pair is treated as a file list; rename the test or use the
333
- conventional layout to get the explicit pair.
334
-
335
- ### Batch mode: run many files in one command
336
-
337
- `--glob PATTERN` mutation-tests every source file the pattern matches in a single
338
- run, so you no longer need to script a loop around `mutation_test` or depend on
339
- the Rails-only `rake mutation:test_models` task.
340
-
341
- Each matched source file is mapped to its spec by convention: `lib/X.rb` becomes
342
- `spec/X_spec.rb`. See [Mapping sources to specs](#mapping-sources-to-specs) for
343
- the two ways to override that convention.
344
-
345
- Behaviour:
346
-
347
- - **Every file is processed.** A file whose score is below the threshold does not
348
- abort the batch; the run continues to the next file (the `mutation:test_models`
349
- pattern).
350
- - **Reports never overwrite each other.** Each processed file writes its reporter
351
- output to its own subdirectory under `--output-dir` (a slug derived from the
352
- source path), so per-file HTML/JSON reports coexist.
353
- - **A console aggregate summary** is printed at the end: one line per processed
354
- file with its score and `PASS`/`FAIL` against the threshold, followed by a
355
- clearly separated `SKIPPED` list.
356
- - **A source file with no matching spec is `SKIPPED`**, reported explicitly and
357
- never counted as a success. A single skipped file next to processed ones does
358
- not by itself fail the run, but a run that skipped *every* matched file
359
- measured nothing and fails (see the exit codes below).
360
-
361
- Exit codes:
362
-
363
- - `0` - at least one file was mutation-tested and every processed file met the
364
- mutation score threshold. A `--since` run where nothing changed also exits `0`
365
- (see below).
366
- - `1` - at least one processed file was below threshold, the glob matched no
367
- source files at all, every matched file was skipped so nothing was actually
368
- mutation-tested, or `--fail-fast` stopped the run at a surviving mutant.
369
- - `2` - a usage error: `--spec-glob` or `--spec-map` with an explicit
370
- `SOURCE_FILE TEST_FILE` pair, a malformed `--spec-map` rule, `--since` given
371
- without `--glob`, `--staged` combined with positional arguments or `--glob`,
372
- or `--since`/`--staged` used outside a git repository (for `--since` also an
373
- unknown revision).
374
-
375
- The "every matched file was skipped" case is deliberate: a typo in
376
- `--spec-glob`/`--spec-map`, or a refactor that moves the test directory, would
377
- otherwise leave a green CI step that measured nothing.
212
+ All multi-file modes (`FILE...` lists, `--staged`, `--glob`) behave the same way:
378
213
 
379
- ```bash
380
- # Minitest project, JSON report per file, custom output directory
381
- bundle exec mutation_test --glob 'lib/**/*.rb' --spec-glob 'test/{name}_test.rb' \
382
- --reporters json --output-dir build/mutation
383
- ```
214
+ - **Every file is processed.** A file whose score is below the threshold does not abort the batch (unless
215
+ `--fail-fast`); the run continues to the next file.
216
+ - **Reports never overwrite each other.** Each processed file writes its reporter output to its own subdirectory under
217
+ `--output-dir` (a slug derived from the source path).
218
+ - **A console aggregate summary** is printed at the end: one line per processed file with its score and `PASS`/`FAIL`,
219
+ followed by a clearly separated `SKIPPED` list. When any mutant survives, the summary ends with a survivors section:
220
+ one `file:line original -> mutated` line per surviving mutant, each a concrete test gap to close.
221
+ - **Unmutable entries are `SKIPPED` with an explicit reason** and never count as a success:
222
+ - **file not found** - the path does not exist.
223
+ - **not a Ruby source file** - e.g. a staged `.md` or config file.
224
+ - **a test file, not a mutable source** - a test file passed directly.
225
+ - **no matching spec file** - the convention (or `--spec-glob` / `--spec-map`) points at a spec that does not exist;
226
+ the expected path is printed.
227
+ - **A run that skipped every file fails** (exit code `1`): a typo in `--spec-glob`/`--spec-map`, or a refactor that
228
+ moves the test directory, would otherwise leave a green CI step that measured nothing.
384
229
 
385
230
  ### Mapping sources to specs
386
231
 
387
- Every mode that takes more than an explicit `SOURCE_FILE TEST_FILE` pair (a
388
- positional `FILE` list, `--staged`, `--glob`) derives the test path from the
389
- source path. Two mechanisms do that, checked in this order:
232
+ Every mode except the explicit `SOURCE_FILE TEST_FILE` pair derives the test path from the source path. Two mechanisms
233
+ do that, checked in this order:
390
234
 
391
235
  1. `--spec-map 'PATTERN=>REPLACEMENT'` - regular-expression rules.
392
236
  2. `--spec-glob TEMPLATE` - a `{name}` template (default `spec/{name}_spec.rb`).
393
237
 
394
- **`--spec-glob TEMPLATE`** substitutes `{name}`, which is the source path with a
395
- leading `lib/` segment removed and the `.rb` extension stripped, subdirectories
396
- preserved (`lib/foo/bar.rb` -> `spec/foo/bar_spec.rb`). Because `{name}` is one
397
- value, a template can only add a prefix and a suffix around the source path. For
398
- a Minitest project laid out under `test/` that is enough:
238
+ **`--spec-glob TEMPLATE`** substitutes `{name}`, which is the source path with a leading `lib/` segment removed and the
239
+ `.rb` extension stripped, subdirectories preserved (`lib/foo/bar.rb` -> `spec/foo/bar_spec.rb`). Because `{name}` is one
240
+ value, a template can only add a prefix and a suffix around the source path. For a Minitest project laid out under
241
+ `test/` that is enough:
399
242
 
400
243
  ```bash
401
244
  bundle exec mutation_test --glob 'lib/**/*.rb' --spec-glob 'test/{name}_test.rb'
402
245
  ```
403
246
 
404
- **`--spec-map 'PATTERN=>REPLACEMENT'`** covers the layouts a template cannot
405
- express: those that substitute *inside* the path, after a variable-length
406
- prefix. `PATTERN` is a Ruby regular expression matched against the whole source
407
- path (a leading `./` removed); the first `=>` separates it from `REPLACEMENT`,
408
- which may use `\1`, `\2`, ... backreferences and produces the whole spec path.
409
- Only the first match in the path is replaced.
247
+ **`--spec-map 'PATTERN=>REPLACEMENT'`** covers the layouts a template cannot express: those that substitute *inside*
248
+ the path, after a variable-length prefix. `PATTERN` is a Ruby regular expression matched against the whole source path
249
+ (a leading `./` removed); the first `=>` separates it from `REPLACEMENT`, which may use `\1`, `\2`, ... backreferences
250
+ and produces the whole spec path. Only the first match in the path is replaced.
410
251
 
411
252
  - The flag is repeatable and the first matching rule wins.
412
- - A source that matches no rule falls back to `--spec-glob` (or the default
413
- convention), so one command can cover `app/` and `lib/` at once.
253
+ - A source that matches no rule falls back to `--spec-glob` (or the default convention), so one command can cover
254
+ `app/` and `lib/` at once.
414
255
  - Quote the rule in single quotes so the shell leaves the backslashes alone.
415
256
 
416
- Rails and Rails-shaped layouts, where the rule is "replace the `app/` segment
417
- with `test/`, keep whatever prefix comes before it":
257
+ Rails and Rails-shaped layouts, where the rule is "replace the `app/` segment with `test/`, keep whatever prefix comes
258
+ before it":
418
259
 
419
260
  ```bash
420
261
  # Plain Rails, Minitest: app/models/current.rb -> test/models/current_test.rb
@@ -438,43 +279,79 @@ bundle exec mutation_test --glob '{app,lib}/**/*.rb' \
438
279
  --spec-glob 'test/{name}_test.rb'
439
280
  ```
440
281
 
441
- When a rule produces a path that does not exist, the file is reported as
442
- `SKIPPED (no matching spec file)` with the expected path printed, and a run in
443
- which *every* file was skipped that way fails with exit code `1`.
282
+ When a rule produces a path that does not exist, the file is reported as `SKIPPED (no matching spec file)` with the
283
+ expected path printed.
284
+
285
+ ### CLI options
286
+
287
+ | Flag | Description |
288
+ |---|---|
289
+ | `-p, --parallel N` | Run with N parallel processes (default: auto from CPU cores, capped at 8; `-p 1` forces serial). See [Parallel execution](#parallel-execution-on-by-default). |
290
+ | `--runner MODE` | Mutant execution runner: `auto` (default), `in_memory`, `fork`, or `spawn`. See [Execution runners](#execution-runners-fork-spawn-in-memory). |
291
+ | `--staged` | Mutation-test the files staged in git. See [Choosing what to test](#choosing-what-to-test). |
292
+ | `--glob PATTERN` | Mutation-test every source file matching `PATTERN`. See [Choosing what to test](#choosing-what-to-test). |
293
+ | `--since REV` | With `--glob`: mutate only the files that changed since git revision `REV`. See [Choosing what to test](#choosing-what-to-test). |
294
+ | `--spec-glob TEMPLATE` | Spec-mapping template with a `{name}` placeholder (default: `spec/{name}_spec.rb`). See [Mapping sources to specs](#mapping-sources-to-specs). |
295
+ | `--spec-map RULE` | Spec-mapping regex rule `'PATTERN=>REPLACEMENT'`; repeatable, first match wins. See [Mapping sources to specs](#mapping-sources-to-specs). |
296
+ | `--minimum-score N` | Mutation score percentage a file must reach to pass (default: 80). Drives the `PASS`/`FAIL` verdict and the exit code. |
297
+ | `--fail-fast` | Stop the run at the first surviving mutant and finish with a failing status. |
298
+ | `--kill-matrix` | Audit mode: run every mutant against the full test file and record which tests kill it (`mutations[].killed_by`, `tests[]`). Slower; cannot be combined with `--fail-fast`. See [Finding redundant tests](#finding-redundant-tests). |
299
+ | `--timeout-factor N` | Per-mutant timeout budget as `N` times the measured baseline test run, never below 5 s (default: 5, must be > 0). Ignored when `config.timeout` is set explicitly. |
300
+ | `--timeout-policy MODE` | Scoring policy for timed-out mutants: `killed` (default) counts a timeout as a kill; `separate` keeps timeouts out of the score and reports them as their own category. |
301
+ | `--worker-env NAME` | Per-worker database isolation via the `parallel_tests` `TEST_ENV_NUMBER` convention. See [Making parallelism work with Rails](#making-parallelism-work-with-rails). |
302
+ | `--after-fork FILE` | Ruby file loaded inside each in-memory clone right after it forks, to re-establish per-worker state. See [Making parallelism work with Rails](#making-parallelism-work-with-rails). |
303
+ | `--strict-equality` | Enable the opt-in strict-equality probes (`==` to `eql?`/`equal?`). Default off; expect noise on code that does not distinguish numeric types or object identity. See [Strict Equality Mutations](docs/mutation-types.md#strict-equality-mutations-opt-in). |
304
+ | `--no-test-selection` | Disable two-phase test selection and always run the full test file for every mutant. See [Test selection](#test-selection-fast-kill-with-full-file-confirmation). |
305
+ | `--json` | Machine mode: print ONLY the JSON report to stdout (everything else goes to stderr). See [JSON report](#json-report-and-machine-readable-output). |
306
+ | `--reporters LIST` | Comma-separated reporters to run: `console`, `html`, `json` (default: all three). An unknown name errors and exits 1. |
307
+ | `--output-dir PATH` | Directory for the generated report files (default: `tmp/mutation_reports`). In batch mode each file writes to its own subdirectory. |
308
+ | `--verbose` | Show a per-mutation warning for every skipped mutation (quiet by default; the "Generated N mutations, skipped M" summary always prints when mutants are dropped). |
309
+ | `--no-progress` | Disable the live progress line (percentage, processed count, elapsed time, estimated remaining time, survived and timed out tallies, and an errored tally once a mutant errors). |
310
+ | `-h, --help` | Show help message. |
311
+ | `-v, --version` | Show version. |
444
312
 
445
- ### Incremental mode: mutate only what changed
313
+ ### Exit codes
446
314
 
447
- `--since REV` narrows a `--glob` batch to the files that changed since a git
448
- revision, which is how you keep mutation testing affordable on pull requests:
315
+ | Code | Meaning |
316
+ |---|---|
317
+ | `0` | The run passed: every processed file met the threshold (or `fail_on_threshold` is disabled). A `--since` run where nothing changed also exits `0`. |
318
+ | `1` | Below threshold, unusable input (missing file, unknown reporter, source with a syntax error), a glob that matched nothing, a batch where every file was skipped, or a `--fail-fast` stop. |
319
+ | `2` | A usage error: conflicting flags (`--spec-glob`/`--spec-map` with an explicit pair, `--staged` with positional arguments or `--glob`, `--since` without `--glob`), a malformed `--spec-map` rule, or `--since`/`--staged` outside a git repository (for `--since` also an unknown revision). |
320
+ | `3` | Single-file mode only: the run aborted or degraded before reaching a verdict. The shadow workspace was unreliable (the unmutated source failed there, or the workspace copy of the source turned out not to be the code the tests execute), or every mutant ended as `error`/`stillborn` so nothing was scored. This signals an infrastructure or runner problem, not a test-quality gap, so CI hooks can distinguish it from a genuine threshold failure. In batch modes a degraded file is named explicitly in the batch summary instead. |
321
+ | `130` | Interrupted with Ctrl+C. |
449
322
 
450
- ```bash
451
- bundle exec mutation_test --glob 'lib/**/*.rb' --since origin/main
452
- ```
323
+ ### Running with rake
453
324
 
454
- Behaviour:
325
+ In a **non-Rails** project, require the tasks from your `Rakefile`:
455
326
 
456
- - A matched file counts as **changed** when `git diff --name-only REV` lists it;
457
- new files the revision does not know about (committed or still untracked) also
458
- count as changed.
459
- - Unchanged matched files are reported as skipped in the batch summary
460
- (`SKIPPED (unchanged since REV)`), never mutated.
461
- - When nothing matched by the glob changed since `REV`, the run succeeds with a
462
- "Nothing to mutate" message and exit code `0`, so a PR that does not touch
463
- your sources does not fail the gate.
464
- - Outside a git repository (or without a `git` executable), or with a revision
465
- the repository does not know, the CLI prints a readable error and exits `2`
466
- before any mutation runs.
327
+ ```ruby
328
+ # Rakefile
329
+ require 'mutation_tester/rake_task'
330
+ ```
467
331
 
468
- ### Fail fast: stop at the first surviving mutant
332
+ In a **Rails** app the tasks load automatically through the gem's railtie, so no Rakefile change is needed.
469
333
 
470
- `--fail-fast` turns the run into a cheap gate: the run stops as soon as one
471
- mutant survives, the reports contain the results obtained up to that point with
472
- a clear interruption notice, and the process finishes with exit code `1`. It
473
- works in single-file mode, with `--glob` (the batch stops and remaining files
474
- are not run), and in parallel mode:
334
+ Then run either task. Rake takes the file arguments inside brackets (not space-separated), so quote the invocation for
335
+ your shell:
475
336
 
476
337
  ```bash
477
- bundle exec mutation_test --glob 'lib/**/*.rb' --since origin/main --fail-fast
338
+ # Top-level task
339
+ bundle exec rake "mutation_test[app/models/user.rb,spec/models/user_spec.rb]"
340
+
341
+ # Namespaced task
342
+ bundle exec rake "mutation:test[app/models/user.rb,spec/models/user_spec.rb]"
343
+ ```
344
+
345
+ ### Programmatic usage
346
+
347
+ Run the gem directly from Ruby with `MutationTester.run(source, test)`:
348
+
349
+ ```ruby
350
+ # RSpec
351
+ MutationTester.run('examples/calculator.rb', 'examples/calculator_spec.rb')
352
+
353
+ # Minitest
354
+ MutationTester.run('examples/calculator.rb', 'examples/calculator_minitest.rb')
478
355
  ```
479
356
 
480
357
  ## Configuration
@@ -529,6 +406,11 @@ MutationTester.configure do |config|
529
406
  # survived. Set to false to always run the full file. See Execution model.
530
407
  config.test_selection = true
531
408
 
409
+ # Kill matrix (audit mode): run every mutant against the full test file and
410
+ # record in the JSON report which tests kill it. Slower than a normal run.
411
+ # Equivalent to the --kill-matrix CLI flag. See Finding redundant tests.
412
+ config.kill_matrix = false
413
+
532
414
  # Hard deadline for the single baseline run of the whole suite (seconds).
533
415
  # Runs every example once, so it is looser than the per-mutant timeout above.
534
416
  # Set to nil to disable the baseline deadline.
@@ -572,14 +454,12 @@ end
572
454
 
573
455
  ## Execution model
574
456
 
575
- MutationTester runs in parallel by default and picks the fastest safe execution
576
- runner automatically. This section covers when to override those defaults, how
577
- the runners differ, and how two-phase test selection speeds up kills.
457
+ MutationTester runs in parallel by default and picks the fastest safe execution runner automatically. This section
458
+ covers when to override those defaults, how the runners differ, and how the gem speeds up kills.
578
459
 
579
460
  ### Parallel execution (on by default)
580
461
 
581
- By default, mutation_tester runs in **parallel**: the process count is derived
582
- from the number of CPU cores (`Etc.nprocessors`), capped at 8 and never below 1.
462
+ By default the process count is derived from the number of CPU cores (`Etc.nprocessors`), capped at 8 and never below 1.
583
463
  Force a specific count, or serial execution, when your tests need it:
584
464
 
585
465
  ```bash
@@ -593,48 +473,33 @@ bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb -p 1
593
473
  MUTATION_TESTER_PARALLEL_PROCESSES=4 bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb
594
474
  ```
595
475
 
596
- Precedence is: an explicit `--parallel/-p` flag overrides `MUTATION_TESTER_PARALLEL_PROCESSES`, which overrides the
476
+ Precedence: an explicit `--parallel/-p` flag overrides `MUTATION_TESTER_PARALLEL_PROCESSES`, which overrides the
597
477
  auto-derived core count; an explicit `config.parallel_processes` assignment in Ruby also replaces the auto default. An
598
- invalid value (less than 1, or non-numeric) falls back to 1 with a warning on stderr. `-p 1` forces serial execution.
599
-
600
- **⚠️ Force serial execution with `-p 1` if your tests share database state!**
601
-
602
- In parallel mode each mutant runs in an isolated shadow workspace. Every `.rb`
603
- file is a physical copy (non-Ruby files stay symlinks for speed), so mutations
604
- apply correctly even when a spec loads the source indirectly (e.g. via
605
- `spec_helper`), and `$LOAD_PATH` entries pointing into the project resolve
606
- inside the workspace, so a test file that reaches its source through
607
- `require "test_helper"` gets the mutated copy too. The parallel mutation score
608
- therefore matches serial.
478
+ invalid value (less than 1, or non-numeric) falls back to 1 with a warning on stderr.
609
479
 
610
- Before the first mutant, the run proves this in the workspace itself: the
611
- unmutated source must pass there, and the same suite must fail once that copy of
612
- the source is replaced by a `raise`. A run whose tests pass even then is aborted
613
- as an infrastructure failure (exit code `3`) rather than reported as a 0.0%
614
- score, because the mutated file is demonstrably not the code being executed.
480
+ **When to force serial execution with `-p 1`**: Rails apps whose tests share one test database, tests that share any
481
+ other state, the first time you try mutation testing (easier to debug), or machines with limited memory/CPU. Pure Ruby
482
+ classes, unit tests with mocks, and CI machines with many cores all benefit from the parallel default. To keep
483
+ parallelism on a Rails app instead of dropping to serial, see
484
+ [Making parallelism work with Rails](#making-parallelism-work-with-rails).
615
485
 
616
- ### When to use serial vs parallel execution
486
+ In parallel mode each mutant runs in an isolated shadow workspace. Every `.rb` file is a physical copy (non-Ruby files
487
+ stay symlinks for speed), so mutations apply correctly even when a spec loads the source indirectly (e.g. via
488
+ `spec_helper`), and `$LOAD_PATH` entries pointing into the project resolve inside the workspace, so a test file that
489
+ reaches its source through `require "test_helper"` gets the mutated copy too. The parallel mutation score therefore
490
+ matches serial.
617
491
 
618
- Use **parallel execution** (the default) for:
619
-
620
- - ⚡ **Pure Ruby classes** - No database, no shared state
621
- - ⚡ **Unit tests with mocks** - Fast and independent tests
622
- - ⚡ **Large codebases** - Significant time savings
623
- - ⚡ **CI/CD with powerful machines** - Make use of available resources
624
-
625
- Force **serial execution with `-p 1`** for:
626
-
627
- - ✅ **Rails applications with database** - Avoids conflicts
628
- - ✅ **Tests that share state** - No interference between test runs
629
- - ✅ **First time using mutation testing** - Easier to debug
630
- - ✅ **Limited system resources** - Less memory/CPU usage
492
+ Before the first mutant, the run proves this in the workspace itself: the unmutated source must pass there, and the
493
+ same suite must fail once that copy of the source is replaced by a `raise`. A run whose tests pass even then is aborted
494
+ as an infrastructure failure (exit code `3`) rather than reported as a 0.0% score, because the mutated file is
495
+ demonstrably not the code being executed.
631
496
 
632
497
  ### Making parallelism work with Rails
633
498
 
634
- Parallel workers get an isolated filesystem (each mutant runs in its own shadow workspace), but they share one
635
- **database** unless you give each worker its own. If your app is already set up for `parallel_tests` (a `database.yml`
636
- keyed on `TEST_ENV_NUMBER` and per-worker databases created with `rake parallel:prepare`), `--worker-env` bridges the
637
- gem to that setup so you can run parallel instead of serial.
499
+ Parallel workers get an isolated filesystem, but they share one **database** unless you give each worker its own. If
500
+ your app is already set up for `parallel_tests` (a `database.yml` keyed on `TEST_ENV_NUMBER` and per-worker databases
501
+ created with `rake parallel:prepare`), `--worker-env` bridges the gem to that setup so you can run parallel instead of
502
+ serial.
638
503
 
639
504
  **`--worker-env NAME`** sets the environment variable `NAME` to a distinct value in each worker before it boots its
640
505
  test environment, following the `parallel_tests` `TEST_ENV_NUMBER` convention:
@@ -712,39 +577,15 @@ model runs on serial `-p 1`. `--worker-env` is only useful once the databases ex
712
577
 
713
578
  ### Execution runners (fork, spawn, in-memory)
714
579
 
715
- Every mutant is executed by one of three runners, and `auto` (the default) picks
716
- the fastest safe one, announcing every fallback on stderr:
717
-
718
- - **in_memory** (default where supported): re-evaluates the mutated source in the
719
- memory of a fresh fork of a preloaded process, with zero file writes per mutant
720
- and no shadow workspaces. RSpec and Minitest; the fastest path. Mutations that only
721
- take effect at class-load time (constants consumed by macros, `validates`/`has_many`/
722
- `before_save`/`scope`/`attribute`, anything inside an `included do` block) cannot
723
- be observed by re-evaluating source in a preloaded process, so those mutants are
724
- routed automatically to the file-based path and the rest still run in memory (see
725
- below); the combined score matches a full `fork` run.
726
- - **fork**: preloads the environment once (RubyGems, Bundler, the test framework)
727
- and forks a fresh child per mutant. RSpec and Minitest on platforms with
728
- `Process.fork`; removes most of the fixed per-mutant boot cost.
729
- - **spawn**: starts one full process per mutant (`bundle exec rspec ...` or
730
- `bundle exec ruby test_file.rb`). Slower per mutant, but works everywhere
731
- (the only runner on platforms without `Process.fork`).
732
-
733
- `auto` tries `in_memory`, then `fork`, then `spawn`; every step down prints one
734
- stderr warning with its reason, so a fallback is never silent. All runners
735
- produce identical scores and per-mutant statuses and enforce the same hard
736
- per-mutant timeout (monotonic deadline plus a process-group kill).
737
-
738
- **Load-time mutants under in-memory (Rails).** The in-memory runner classifies each
739
- mutation by its AST context. A mutation inside a method body defined directly in a
740
- class/module is re-appliable in memory and runs there (fast). A mutation on a
741
- class/module-body statement (a constant, a `validates`/`has_many`/`before_save`/
742
- `scope`/`attribute` macro, or anything inside `included do ... end`) is decided
743
- file-based within the same run, because re-evaluating the source does not re-run
744
- those class-load registrations. This keeps in-memory speed for the common case
745
- while matching a full `fork` score on Rails concerns and models. When at least one
746
- mutant is routed this way, the run prints one stderr notice. It is automatic; you
747
- do not need to pick `--runner fork` for correctness on load-time code.
580
+ Every mutant is executed by one of three runners, and `auto` (the default) picks the fastest safe one, announcing every
581
+ fallback on stderr:
582
+
583
+ - **in_memory** (default where supported): re-evaluates the mutated source in the memory of a fresh fork of a preloaded
584
+ process, with zero file writes per mutant and no shadow workspaces. RSpec and Minitest; the fastest path.
585
+ - **fork**: preloads the environment once (RubyGems, Bundler, the test framework) and forks a fresh child per mutant.
586
+ RSpec and Minitest on platforms with `Process.fork`; removes most of the fixed per-mutant boot cost.
587
+ - **spawn**: starts one full process per mutant (`bundle exec rspec ...` or `bundle exec ruby test_file.rb`). Slower
588
+ per mutant, but works everywhere (the only runner on platforms without `Process.fork`).
748
589
 
749
590
  | Mode | Picked by `auto` when | Falls back to |
750
591
  |-------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|
@@ -752,84 +593,81 @@ do not need to pick `--runner fork` for correctness on load-time code.
752
593
  | `fork` | `Process.fork` is available, but in-memory is unavailable (each reason is printed) | `spawn`, with a stderr warning, when the helper process fails to preload the environment |
753
594
  | `spawn` | the platform has no `Process.fork` | nothing; it works everywhere |
754
595
 
755
- Force a specific runner (skipping the auto attempts) with the `--runner
756
- fork|spawn|in_memory` flag, the `MUTATION_TESTER_RUNNER` environment variable, or
757
- `config.runner`:
596
+ All runners produce identical scores and per-mutant statuses and enforce the same hard per-mutant timeout (monotonic
597
+ deadline plus a process-group kill).
598
+
599
+ **Load-time mutants under in-memory (Rails).** The in-memory runner classifies each mutation by its AST context. A
600
+ mutation inside a method body defined directly in a class/module is re-appliable in memory and runs there (fast). A
601
+ mutation on a class/module-body statement (a constant, a `validates`/`has_many`/`before_save`/`scope`/`attribute`
602
+ macro, or anything inside `included do ... end`) cannot be observed by re-evaluating source in a preloaded process, so
603
+ it is decided file-based within the same run. This keeps in-memory speed for the common case while matching a full
604
+ `fork` score on Rails concerns and models. When at least one mutant is routed this way, the run prints one stderr
605
+ notice. It is automatic; you do not need to pick `--runner fork` for correctness on load-time code.
606
+
607
+ Force a specific runner (skipping the auto attempts) with the `--runner fork|spawn|in_memory` flag, the
608
+ `MUTATION_TESTER_RUNNER` environment variable, or `config.runner`:
758
609
 
759
610
  ```bash
760
611
  mutation_test --runner spawn lib/calculator.rb spec/calculator_spec.rb
761
612
  ```
762
613
 
763
- Reach for `--runner spawn` when you want maximum isolation or are debugging a
764
- suspicious result from a preloaded runner, and `--runner fork` when your source
765
- is not cleanly re-evaluable in memory but you still want the preloaded-environment
766
- speed. For the full per-runner mechanics, when to force each one, and the complete
767
- fork and in-memory limitation lists (frozen classes, load-time `defined?` guards,
768
- worker-death fallback, `require_relative` idempotency), see
614
+ Reach for `--runner spawn` when you want maximum isolation or are debugging a suspicious result from a preloaded
615
+ runner, and `--runner fork` when your source is not cleanly re-evaluable in memory but you still want the
616
+ preloaded-environment speed. For the full per-runner mechanics and the complete fork and in-memory limitation lists
617
+ (frozen classes, load-time `defined?` guards, worker-death fallback, `require_relative` idempotency), see
769
618
  [docs/execution-runners.md](docs/execution-runners.md#execution-runners-fork-spawn-in-memory).
770
619
 
771
620
  ### Stopping a mutant at its first failing test
772
621
 
773
- A mutant only needs one failing test to be killed, so every mutant run stops at
774
- its first failure: RSpec mutant runs get `--fail-fast`, and Minitest mutant runs
775
- get a preloaded reporter that aborts the run the same way (both on the file-based
776
- runners and inside the preloaded fork worker). This never changes a verdict, only
777
- the work done to reach it: a run that stops early had already failed, and a run
778
- with no failure is unaffected and still executes every test.
779
-
780
- It matters most for a mutant that breaks something every test touches (a broken
781
- class body, a constant every example reads). Such a mutant used to pay the full
782
- test file once per mutant, which on a large test file can exceed the per-mutant
783
- deadline and turn a decided kill into a reported timeout. Adding tests to the file
784
- then made the score worse. The baseline run and the shadow sanity check are
622
+ A mutant only needs one failing test to be killed, so every mutant run stops at its first failure: RSpec mutant runs
623
+ get `--fail-fast`, and Minitest mutant runs get a preloaded reporter that aborts the run the same way (both on the
624
+ file-based runners and inside the preloaded fork worker). This never changes a verdict, only the work done to reach it:
625
+ a run that stops early had already failed, and a run with no failure is unaffected and still executes every test.
626
+
627
+ It matters most for a mutant that breaks something every test touches (a broken class body, a constant every example
628
+ reads). Such a mutant used to pay the full test file once per mutant, which on a large test file can exceed the
629
+ per-mutant deadline and turn a decided kill into a reported timeout. The baseline run and the shadow sanity check are
785
630
  unaffected: they are expected to pass, and a passing run runs every test.
786
631
 
632
+ The one exception is the opt-in `--kill-matrix` audit mode, which needs every failing test of every mutant and
633
+ therefore runs the full test file each time. See [Finding redundant tests](#finding-redundant-tests).
634
+
787
635
  ### Test selection (fast kill with full-file confirmation)
788
636
 
789
- For RSpec suites on the file-based runners, MutationTester runs each mutant in
790
- two phases: it first runs only the examples whose group matches the mutated
791
- method (`rspec spec_file -e '#foo' -e '.foo'`) to kill it fast, then confirms a
792
- passing subset against the full spec file before a mutant can be reported as
793
- survived, so selection never introduces false survivors. It degrades to the full
794
- file when the mutant is not inside a method, the spec has no matching group, or
795
- the suite is Minitest; the in-memory runner skips selection entirely (its
796
- examples are already loaded). Disable it with `--no-test-selection` or
797
- `config.test_selection = false`. See
798
- [docs/execution-runners.md](docs/execution-runners.md#test-selection-fast-kill-with-full-file-confirmation)
799
- for the full behavior.
637
+ For RSpec suites on the file-based runners, MutationTester runs each mutant in two phases: it first runs only the
638
+ examples whose group matches the mutated method (`rspec spec_file -e '#foo' -e '.foo'`) to kill it fast, then confirms
639
+ a passing subset against the full spec file before a mutant can be reported as survived, so selection never introduces
640
+ false survivors. It degrades to the full file when the mutant is not inside a method, the spec has no matching group,
641
+ or the suite is Minitest; the in-memory runner skips selection entirely (its examples are already loaded). Disable it
642
+ with `--no-test-selection` or `config.test_selection = false`. See
643
+ [docs/execution-runners.md](docs/execution-runners.md#test-selection-fast-kill-with-full-file-confirmation) for the
644
+ full behavior.
800
645
 
801
646
  ## Mutation types
802
647
 
803
- MutationTester generates several families of mutations, enabled by default:
804
- arithmetic, bitwise compound-assignment, comparison, logical, boolean, number,
805
- string, conditional, call-removal, nil-injection and argument mutations. A
806
- strict-equality family is opt-in. Enable or disable individual families through
807
- `config.mutation_types` (see [Configuration](#configuration)) or turn
808
- strict-equality on with `--strict-equality`.
648
+ MutationTester generates several families of mutations, enabled by default: arithmetic, bitwise compound-assignment,
649
+ comparison, logical, boolean, number, string, conditional, call-removal, nil-injection and argument mutations. A
650
+ strict-equality family is opt-in. Enable or disable individual families through `config.mutation_types`
651
+ (see [Configuration](#configuration)) or turn strict-equality on with `--strict-equality`.
809
652
 
810
- See [docs/mutation-types.md](docs/mutation-types.md) for the full catalog: every
811
- operator swap and structural mutation each family generates, its reported `type`,
812
- and the constructs each family intentionally leaves alone.
653
+ See [docs/mutation-types.md](docs/mutation-types.md) for the full catalog: every operator swap and structural mutation
654
+ each family generates, its reported `type`, and the constructs each family intentionally leaves alone.
813
655
 
814
656
  ## Equivalent mutants
815
657
 
816
- A mutation score of 100% is not always achievable, and a surviving mutation is
817
- not always a gap in your tests. Some mutations produce code that behaves
818
- **identically** to the original for every possible input, an *equivalent
819
- mutant*, and no test can ever kill it. For example, in a `max` implementation the
820
- original `a > b ? a : b` and the mutant `a >= b ? a : b` differ only when
821
- `a == b`, and both return the same value there, so the mutant survives no matter
822
- how thorough your tests are. Because equivalence is undecidable in the general
823
- case, treat survivors as *candidates* to review rather than guaranteed test gaps;
824
- once you confirm a survivor is equivalent, it is reasonable to accept a score
658
+ A mutation score of 100% is not always achievable, and a surviving mutation is not always a gap in your tests. Some
659
+ mutations produce code that behaves **identically** to the original for every possible input, an *equivalent mutant*,
660
+ and no test can ever kill it. For example, in a `max` implementation the original `a > b ? a : b` and the mutant
661
+ `a >= b ? a : b` differ only when `a == b`, and both return the same value there, so the mutant survives no matter how
662
+ thorough your tests are. Because equivalence is undecidable in the general case, treat survivors as *candidates* to
663
+ review rather than guaranteed test gaps; once you confirm a survivor is equivalent, it is reasonable to accept a score
825
664
  below 100%.
826
665
 
827
666
  ### Excluding a line with `# mutation_tester:disable`
828
667
 
829
- Once you have confirmed that a survivor is equivalent, annotate its line with a
830
- trailing `# mutation_tester:disable` comment (the same style as
831
- `# rubocop:disable`) so the mutator skips every mutation on that line and the
832
- excluded line drops out of the score and the report:
668
+ Once you have confirmed that a survivor is equivalent, annotate its line with a trailing `# mutation_tester:disable`
669
+ comment (the same style as `# rubocop:disable`) so the mutator skips every mutation on that line and the excluded line
670
+ drops out of the score and the report:
833
671
 
834
672
  ```ruby
835
673
  def max(a, b)
@@ -837,19 +675,24 @@ def max(a, b)
837
675
  end
838
676
  ```
839
677
 
840
- The marker is honoured only inside a real comment (never inside a string
841
- literal), and only on the line it sits on. See
842
- [docs/mutation-types.md](docs/mutation-types.md#excluding-a-line-with--mutation_testerdisable)
843
- for the console output and the current limits (no block ranges or per-type
844
- exclusion yet).
678
+ The marker is honoured only inside a real comment (never inside a string literal), and only on the line it sits on. See
679
+ [docs/mutation-types.md](docs/mutation-types.md#excluding-a-line-with--mutation_testerdisable) for the console output
680
+ and the current limits (no block ranges or per-type exclusion yet).
845
681
 
846
682
  ## Reports and output
847
683
 
684
+ Three reporters are available (`console`, `html`, `json`; all on by default). Choose which run with `--reporters` and
685
+ where their files land with `--output-dir`:
686
+
687
+ ```bash
688
+ bundle exec mutation_test --reporters json,html --output-dir build/mutation \
689
+ app/models/user.rb spec/models/user_spec.rb
690
+ ```
691
+
848
692
  ### Console report
849
693
 
850
- Surviving mutants are grouped by file and line (one header per location, all
851
- mutation variants listed under it) and each group shows a unified diff with a
852
- few lines of surrounding context:
694
+ Surviving mutants are grouped by file and line (one header per location, all mutation variants listed under it) and
695
+ each group shows a unified diff with a few lines of surrounding context:
853
696
 
854
697
  ```
855
698
  🧬 MUTATION TESTING REPORT
@@ -877,39 +720,28 @@ few lines of surrounding context:
877
720
  💡 Suggestion: Add tests to verify behavior for each of the 2 variants above
878
721
  ```
879
722
 
880
- When at least one mutant timed out, the summary also names the deadline those
881
- mutants were measured against and where it came from, so a genuine hang and a
882
- deadline calibrated from a slow test file are distinguishable at a glance:
723
+ When at least one mutant timed out, the summary also names the deadline those mutants were measured against and where
724
+ it came from, so a genuine hang and a deadline calibrated from a slow test file are distinguishable at a glance:
883
725
 
884
726
  ```
885
727
  Timeout: 3 ⏱️
886
728
  deadline: 6.50s (5x baseline 1.30s)
887
729
  ```
888
730
 
889
- With an explicit `config.timeout` / `--timeout` the same line reads
890
- `deadline: 30.00s (explicitly configured)`.
731
+ With an explicit `config.timeout` / `--timeout` the same line reads `deadline: 30.00s (explicitly configured)`.
891
732
 
892
733
  ### HTML report
893
734
 
894
- Beautiful interactive HTML report with:
895
-
896
- - Summary statistics
897
- - Mutation score visualization
898
- - Filterable mutation list
899
- - Survivors grouped by file and line, with all mutation variants under one card
900
- - Unified diffs with surrounding context for survived and timeout mutants
901
- - Detailed suggestions for improvements
735
+ Interactive HTML report with summary statistics, mutation score visualization, a filterable mutation list, survivors
736
+ grouped by file and line (all mutation variants under one card), unified diffs with surrounding context for survived
737
+ and timeout mutants, and detailed suggestions for improvements.
902
738
 
903
739
  ### JSON report and machine-readable output
904
740
 
905
- Machine-readable report for CI/CD integration and AI agents. Run with `--json`
906
- to get a single, clean JSON document on **stdout** and nothing else: the banner,
907
- progress spinner, colours and the "report saved" notice all go to **stderr**, so
908
- the stream is safe to pipe straight into `jq` or a parser. The process still
909
- exits `0` when the mutation score meets the configured threshold and `1` when it
910
- does not (a degraded single-file run exits `3`, see
911
- [Exit codes](#exit-codes-single-file-mode)), so the exit code remains a
912
- pass/fail signal.
741
+ Machine-readable report for CI/CD integration and AI agents. Run with `--json` to get a single, clean JSON document on
742
+ **stdout** and nothing else: the banner, progress spinner, colours and the "report saved" notice all go to **stderr**,
743
+ so the stream is safe to pipe straight into `jq` or a parser. The exit code remains the pass/fail signal
744
+ (see [Exit codes](#exit-codes)).
913
745
 
914
746
  ```bash
915
747
  bundle exec mutation_test --json examples/calculator.rb examples/calculator_spec.rb | jq .
@@ -918,83 +750,146 @@ bundle exec mutation_test --json examples/calculator.rb examples/calculator_spec
918
750
  bundle exec mutation_test --json lib/calculator.rb | jq .
919
751
  ```
920
752
 
921
- The same report is also written to `tmp/mutation_reports/mutation_report.json`
922
- (see `--output-dir`). A run that resolves exactly one source file, or an explicit
923
- `SOURCE_FILE TEST_FILE` pair, prints the plain per-file report; a multi-file run
924
- (a `FILE` list with more than one file, `--staged`, or `--glob`) prints one
925
- aggregate envelope with a condensed `survivors` array.
753
+ The same report is also written to `tmp/mutation_reports/mutation_report.json` (see `--output-dir`). A run that
754
+ resolves exactly one source file, or an explicit `SOURCE_FILE TEST_FILE` pair, prints the plain per-file report; a
755
+ multi-file run (a `FILE` list with more than one file, `--staged`, or `--glob`) prints one aggregate envelope with a
756
+ condensed `survivors` array.
757
+
758
+ Every **surviving** mutant carries `file_path`, `line`, `original` and `mutated`, which is a concrete, located test
759
+ gap: the worklist you can hand to an AI agent or a CI gate (see [CI/CD integration](#cicd-integration)).
760
+
761
+ Full field-by-field documentation of both shapes (the single-file report and the multi-file envelope), the
762
+ `schema_version` policy, and ready-to-use `jq` recipes live in [docs/json-schema.md](docs/json-schema.md).
763
+
764
+ ## Finding redundant tests
765
+
766
+ Mutation testing normally answers "which behavior is untested?". The opt-in **kill matrix** answers the opposite
767
+ question: "which tests could I delete without losing any protection?". A test is a redundancy candidate when removing
768
+ it leaves the set of killed mutants unchanged.
769
+
770
+ ```bash
771
+ # One file
772
+ bundle exec mutation_test --kill-matrix --json lib/calculator.rb > kill_matrix.json
773
+
774
+ # A whole directory (one aggregate JSON document)
775
+ bundle exec mutation_test --kill-matrix --json --glob 'lib/**/*.rb' > kill_matrix.json
776
+ ```
777
+
778
+ `--kill-matrix` (or `config.kill_matrix = true`) changes how every mutant is run and what the JSON report contains. It
779
+ works the same for RSpec and Minitest and on all three runners:
926
780
 
927
- Every **surviving** mutant carries `file_path`, `line`, `original` and `mutated`,
928
- which is a concrete, located test gap: the worklist you can hand to an AI agent
929
- or a CI gate (see [CI/CD integration](#cicd-integration)).
781
+ - Every mutant runs the **full** test file. The run does not stop at the first failing test and RSpec test selection
782
+ is off, because a run that stops early would name only one of the killers.
783
+ - `tests[]` lists every test of the unmutated baseline run (`id`, `name`, `line`, `status`), so a test that kills
784
+ nothing is still visible.
785
+ - `mutations[].killed_by` lists the `id` of every test that failed under that mutant.
930
786
 
931
- Full field-by-field documentation of both shapes (the single-file report and the
932
- multi-file envelope), the `schema_version` policy, and ready-to-use `jq` recipes
933
- live in [docs/json-schema.md](docs/json-schema.md).
787
+ Test ids are `TestClass#test_name` for Minitest and `spec/calculator_spec.rb[1:2:1]` for RSpec, which you can pass
788
+ straight to `rspec` from the project root to run that one example (RSpec older than 3.3 has no such ids and gets
789
+ `spec/calculator_spec.rb:12`, file and line, instead).
790
+
791
+ The analysis itself is a short `jq` program over the report. Both recipes work on a single-file report and on the
792
+ multi-file envelope.
793
+
794
+ ```bash
795
+ # 1. Tests that kill no mutant at all
796
+ jq -r '(.files // [.])[]
797
+ | [.mutations[].killed_by[]] as $killers
798
+ | .tests[] | select(.status == "passed" and (.id | IN($killers[]) | not))
799
+ | "\(.id)\t\(.name)"' kill_matrix.json
800
+
801
+ # 2. Tests whose every kill is shared: each mutant they kill is also killed by another test
802
+ jq -r '(.files // [.])[]
803
+ | [.mutations[] | select(.status == "killed") | .killed_by] as $kills
804
+ | .tests[] | .id as $id
805
+ | [$kills[] | select(index($id))] as $mine
806
+ | select(($mine | length) > 0 and all($mine[]; length > 1))
807
+ | "\(.id)\t\(.name)"' kill_matrix.json
808
+
809
+ # 3. Mutants whose killers are unknown (read the results with care when this is not 0)
810
+ jq '[(.files // [.])[] | .mutations[]
811
+ | select((.status == "killed" or .status == "timeout") and (.killed_by | length) == 0)] | length' kill_matrix.json
812
+ ```
813
+
814
+ Read the lists as candidates to review, not as a delete list:
815
+
816
+ - **Remove one test at a time and re-run.** Two tests from list 2 can cover for each other: each one is redundant
817
+ while the other exists, but deleting both lets their mutants survive.
818
+ - The result is relative to **one source file and the enabled mutation types**. A test that kills nothing here may
819
+ protect another file, or behavior no mutation operator changes (for example the exact wording of a message when the
820
+ `string` mutations are disabled).
821
+ - A `timeout` mutant always has an empty `killed_by`, and so does a `killed` mutant that failed without any test
822
+ failing (typically the mutated file no longer loads). Their killers are unknown, so a test must never be called
823
+ redundant because of them. Recipe 3 counts them; a higher `--timeout-factor` usually turns timeouts into real kills.
824
+ - A skipped test has `"status": "skipped"` in `tests[]` and is left out of the recipes.
825
+ - If the baseline passes but not a single test could be recorded (the test file defines no tests, or a plugin replaces
826
+ the framework's reporters), the run stops with an error instead of reporting an empty matrix.
827
+
828
+ The mode is an occasional audit, not a gate. Without the early stop a mutant that breaks something every test touches
829
+ pays for the whole test file, so expect a slower run and consider `--timeout-factor 10`. It cannot be combined with
830
+ `--fail-fast` (or `config.fail_fast`), which would stop the run at the first surviving mutant and leave the matrix
831
+ incomplete: the CLI rejects the pair as a usage error and a run configured from Ruby or rake fails before the baseline.
832
+
833
+ For a scheduled CI job that publishes the candidates, see
834
+ [`examples/github_actions/redundant_tests.yml`](examples/github_actions/redundant_tests.yml) and
835
+ [docs/ci.md](docs/ci.md#redundant-test-audit-scheduled-job).
934
836
 
935
837
  ## Pre-push hook
936
838
 
937
- Gate your pushes locally: run mutation testing on the file(s) you touched and
938
- block the push when the score is under your bar, the fast-feedback sibling of
939
- the CI gate below (see [CI/CD integration](#cicd-integration)).
839
+ Gate your pushes locally: run mutation testing on the file(s) you touched and block the push when the score is under
840
+ your bar, the fast-feedback sibling of the CI gate below.
940
841
 
941
- The gem ships a ready-to-copy hook at
942
- [`examples/hooks/pre-push`](examples/hooks/pre-push) (installed with the gem, so
943
- you have it offline). It runs `mutation_test --json`, reads the score with
944
- [`jq`](https://jqlang.github.io/jq/), and on a below-threshold run lists the
945
- surviving mutants (file, line, what changed) so you see which gaps to close.
946
- Install it with:
842
+ The gem ships a ready-to-copy hook at [`examples/hooks/pre-push`](examples/hooks/pre-push) (installed with the gem, so
843
+ you have it offline). It runs `mutation_test --json`, reads the score with [`jq`](https://jqlang.github.io/jq/), and on
844
+ a below-threshold run lists the surviving mutants (file, line, what changed) so you see which gaps to close. Install it
845
+ with:
947
846
 
948
847
  ```bash
949
848
  cp examples/hooks/pre-push .git/hooks/pre-push
950
849
  chmod +x .git/hooks/pre-push
951
850
  ```
952
851
 
953
- Then edit the `THRESHOLD` and the `SOURCE TEST` pair(s) at the top of the copied
954
- hook.
852
+ Then edit the `THRESHOLD` and the `SOURCE TEST` pair(s) at the top of the copied hook.
955
853
 
956
- Prefer to gate on your project's configured threshold rather than one written
957
- into the hook? `mutation_test` already exits non-zero when the score is below
958
- `config.minimum_score` (default 80, see [Configuration](#configuration)), so you
959
- can drop the `jq` comparison and let the exit code be the gate:
854
+ Prefer to gate on your project's configured threshold rather than one written into the hook? `mutation_test` already
855
+ exits non-zero when the score is below `config.minimum_score` (default 80), so you can drop the `jq` comparison and let
856
+ the exit code be the gate:
960
857
 
961
858
  ```sh
962
859
  bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb || exit 1
963
860
  ```
964
861
 
965
- `--minimum-score N` sets that threshold for a single run, which is how you start
966
- below 80 in an existing codebase and ratchet the number up over time:
862
+ `--minimum-score N` sets that threshold for a single run, which is how you start below 80 in an existing codebase and
863
+ ratchet the number up over time:
967
864
 
968
865
  ```sh
969
866
  bundle exec mutation_test --glob 'app/**/*.rb' \
970
867
  --spec-map '\Aapp/(.+)\.rb\z=>spec/\1_spec.rb' --minimum-score 60 || exit 1
971
868
  ```
972
869
 
973
- lefthook or overcommit users: call the shipped hook from your `pre-push` step
974
- instead of writing to `.git/hooks/`.
870
+ lefthook or overcommit users: call the shipped hook from your `pre-push` step instead of writing to `.git/hooks/`.
975
871
 
976
872
  ## CI/CD integration
977
873
 
978
- Run MutationTester as a CI quality gate: the CLI exits non-zero when the mutation
979
- score is below the threshold, so it fails the job with no extra configuration.
874
+ Run MutationTester as a CI quality gate: the CLI exits non-zero when the mutation score is below the threshold, so it
875
+ fails the job with no extra configuration.
980
876
 
981
- The gem ships ready-to-copy GitHub Actions workflows (installed alongside the
982
- gem, so you have them offline too):
877
+ The gem ships ready-to-copy GitHub Actions workflows (installed alongside the gem, so you have them offline too):
983
878
 
984
- - [`examples/github_actions/mutation_test.yml`](examples/github_actions/mutation_test.yml)
985
- is the maintained template. Add the gem to your bundle, copy it to
986
- `.github/workflows/`, edit the `EDIT:` lines, and it runs the gate, uploads the
987
- HTML/JSON reports from `tmp/mutation_reports/` as an artifact (even on
988
- failure), and fails the job below threshold. It also carries commented variants
989
- for parallel execution, several file pairs, and an incremental pull-request gate.
990
- - [`examples/github_actions/ai_mutation_gate.yml`](examples/github_actions/ai_mutation_gate.yml)
991
- is the AI gate: the same pass/fail gate, plus it writes the surviving-mutant
992
- worklist to the GitHub job summary and uploads `survivors.json` for an agent to
993
- turn into missing tests.
879
+ - [`examples/github_actions/mutation_test.yml`](examples/github_actions/mutation_test.yml) is the maintained template.
880
+ Add the gem to your bundle, copy it to `.github/workflows/`, edit the `EDIT:` lines, and it runs the gate, uploads
881
+ the HTML/JSON reports from `tmp/mutation_reports/` as an artifact (even on failure), and fails the job below
882
+ threshold. It also carries commented variants for parallel execution, several file pairs, and an incremental
883
+ pull-request gate.
884
+ - [`examples/github_actions/ai_mutation_gate.yml`](examples/github_actions/ai_mutation_gate.yml) is the AI gate: the
885
+ same pass/fail gate, plus it writes the surviving-mutant worklist to the GitHub job summary and uploads
886
+ `survivors.json` for an agent to turn into missing tests.
887
+ - [`examples/github_actions/redundant_tests.yml`](examples/github_actions/redundant_tests.yml) is a scheduled audit,
888
+ not a gate: it runs `--kill-matrix` and lists the tests that kill no mutant, or no mutant of their own, in the job
889
+ summary. See [Finding redundant tests](#finding-redundant-tests).
994
890
 
995
- See [docs/ci.md](docs/ci.md) for the full recipes: a 5-minute setup, minimal
996
- inline and pull-request workflows, machine mode as a gate and artifact, and the
997
- AI workflow.
891
+ See [docs/ci.md](docs/ci.md) for the full recipes: a 5-minute setup, minimal inline and pull-request workflows, machine
892
+ mode as a gate and artifact, and the AI workflow.
998
893
 
999
894
  ## Troubleshooting
1000
895
 
@@ -1010,18 +905,14 @@ This usually means:
1010
905
 
1011
906
  ### Database Conflicts in Parallel Mode
1012
907
 
1013
- If you see errors like "database is locked" or "record not found":
1014
-
1015
- **Solution**: Use serial execution (the default):
1016
-
1017
- ```bash
1018
- bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb
1019
- ```
908
+ If you see errors like "database is locked" or "record not found", your parallel workers are sharing one test database.
1020
909
 
1021
- Or if using environment variable, make sure it's not set or set to 1:
910
+ **Solution**: Force serial execution with `-p 1` (or `MUTATION_TESTER_PARALLEL_PROCESSES=1`), or keep parallelism by
911
+ giving each worker its own database with `--worker-env`
912
+ (see [Making parallelism work with Rails](#making-parallelism-work-with-rails)):
1022
913
 
1023
914
  ```bash
1024
- MUTATION_TESTER_PARALLEL_PROCESSES=1 bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb
915
+ bundle exec mutation_test app/models/user.rb spec/models/user_spec.rb -p 1
1025
916
  ```
1026
917
 
1027
918
  ### Slow Execution
@@ -1031,10 +922,11 @@ run.
1031
922
 
1032
923
  **Tips for faster execution**:
1033
924
 
1034
- - Test only critical files (don't test everything)
1035
- - Use parallel execution if your tests support it (`--parallel N`)
1036
- - Consider using faster test databases (SQLite in-memory for unit tests)
925
+ - Test only critical files (don't test everything), or only changed files (`--staged`, `--since`)
1037
926
  - Focus on high-value code (models, services, core logic)
927
+ - Keep the parallel and in-memory defaults working (fix the issues that force a fallback; every fallback is announced
928
+ on stderr with its reason)
929
+ - Consider using faster test databases (SQLite in-memory for unit tests)
1038
930
 
1039
931
  ### "No Mutations Generated"
1040
932
 
@@ -1057,32 +949,27 @@ If mutation testing stops at a `debugger` or `binding.pry` statement:
1057
949
 
1058
950
  ### Parser Version Warning on Newer Ruby (Supported Syntax Level)
1059
951
 
1060
- MutationTester parses your source with the `parser` gem. The newest published
1061
- `parser` line recognizes **Ruby 3.3 syntax**; there is not yet a release that
1062
- understands Ruby 3.4+/4.x syntax. So when you run on Ruby 3.4 or newer you may
1063
- see one line on stderr per run:
952
+ MutationTester parses your source with the `parser` gem. The newest published `parser` line recognizes **Ruby 3.3
953
+ syntax**; there is not yet a release that understands Ruby 3.4+/4.x syntax. So when you run on Ruby 3.4 or newer you
954
+ may see one line on stderr per run:
1064
955
 
1065
956
  ```
1066
957
  warning: parser/current is loading parser/ruby33, which recognizes 3.3.x-compliant syntax, but you are running 4.0.2.
1067
958
  ```
1068
959
 
1069
- This warning is **benign**. It only means the parser recognizes syntax up to
1070
- Ruby 3.3. The gem itself runs fine on Ruby 3.4+/4.x, and files written in Ruby
1071
- 3.3-and-earlier syntax are mutated normally.
960
+ This warning is **benign**. It only means the parser recognizes syntax up to Ruby 3.3. The gem itself runs fine on
961
+ Ruby 3.4+/4.x, and files written in Ruby 3.3-and-earlier syntax are mutated normally.
1072
962
 
1073
- The only real limitation is a source file that relies on syntax introduced
1074
- after Ruby 3.3. Such a file cannot be parsed, so it reports an explicit
1075
- `Failed to parse source file` and the run fails - it never fakes a passing
1076
- score. The warning is left in place on purpose as an honest signal; it is not
1077
- globally silenced.
963
+ The only real limitation is a source file that relies on syntax introduced after Ruby 3.3. Such a file cannot be
964
+ parsed, so it reports an explicit `Failed to parse source file` and the run fails - it never fakes a passing score. The
965
+ warning is left in place on purpose as an honest signal; it is not globally silenced.
1078
966
 
1079
967
  ## Development
1080
968
 
1081
969
  Run `bundle install`, then run `rake spec` to run the tests.
1082
970
 
1083
- The gem includes working examples for both RSpec and Minitest under the
1084
- `examples/` directory (a sample `Calculator` class with an RSpec spec and a
1085
- Minitest test). Run them directly with the CLI, or through the example rake tasks:
971
+ The gem includes working examples for both RSpec and Minitest under the `examples/` directory (a sample `Calculator`
972
+ class with an RSpec spec and a Minitest test). Run them directly with the CLI, or through the example rake tasks:
1086
973
 
1087
974
  ```bash
1088
975
  # Directly with the CLI
@@ -1101,12 +988,8 @@ rake example:minitest_parallel # Parallel execution
1101
988
  rake example
1102
989
  ```
1103
990
 
1104
- Each example will:
1105
-
1106
- - Run mutation tests on a sample Calculator class
1107
- - Generate console, HTML, and JSON reports
1108
- - Show mutation score and quality metrics
1109
- - Demonstrate the difference between serial and parallel execution
991
+ Each example runs mutation tests on the sample Calculator class, generates console, HTML, and JSON reports, and shows
992
+ the difference between serial and parallel execution.
1110
993
 
1111
994
  ## Contributing
1112
995