mutineer 1.8.1 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +733 -0
- data/README.md +85 -50
- data/lib/mutineer/changed_lines.rb +2 -1
- data/lib/mutineer/child_cause.rb +149 -0
- data/lib/mutineer/child_stdout.rb +12 -1
- data/lib/mutineer/cli.rb +540 -59
- data/lib/mutineer/command_redact.rb +38 -0
- data/lib/mutineer/concern_blocks.rb +95 -24
- data/lib/mutineer/config.rb +112 -6
- data/lib/mutineer/coverage_map.rb +1345 -156
- data/lib/mutineer/daemon_backend.rb +41 -18
- data/lib/mutineer/daemon_client.rb +104 -21
- data/lib/mutineer/daemon_server.rb +85 -14
- data/lib/mutineer/database_urls.rb +189 -0
- data/lib/mutineer/external_backend.rb +93 -6
- data/lib/mutineer/fs_path.rb +125 -0
- data/lib/mutineer/isolation.rb +380 -25
- data/lib/mutineer/job_plan.rb +352 -41
- data/lib/mutineer/minitest_integration/file_order.rb +56 -0
- data/lib/mutineer/minitest_integration/in_test_database_error.rb +243 -19
- data/lib/mutineer/minitest_integration.rb +26 -58
- data/lib/mutineer/mutant_id.rb +4 -1
- data/lib/mutineer/mutation.rb +10 -4
- data/lib/mutineer/mutator_registry.rb +6 -3
- data/lib/mutineer/mutators/collection_method.rb +195 -6
- data/lib/mutineer/mutators/literal_mutation.rb +33 -3
- data/lib/mutineer/mutators/range_literal.rb +78 -29
- data/lib/mutineer/mutators/rescue_body.rb +126 -0
- data/lib/mutineer/mutators/return_nil.rb +23 -6
- data/lib/mutineer/output_filter.rb +451 -0
- data/lib/mutineer/pairing.rb +463 -31
- data/lib/mutineer/parser.rb +53 -0
- data/lib/mutineer/project_path.rb +15 -8
- data/lib/mutineer/rails_worker_db.rb +366 -6
- data/lib/mutineer/reach_probe.rb +229 -0
- data/lib/mutineer/redaction.rb +301 -41
- data/lib/mutineer/reload_guards.rb +985 -0
- data/lib/mutineer/reporter.rb +467 -58
- data/lib/mutineer/result.rb +6 -7
- data/lib/mutineer/run_seed.rb +16 -0
- data/lib/mutineer/runner.rb +194 -55
- data/lib/mutineer/source_bytes.rb +187 -0
- data/lib/mutineer/statement_lines.rb +108 -14
- data/lib/mutineer/test_runners/rspec.rb +118 -10
- data/lib/mutineer/test_runners/rspec_seed.rb +151 -0
- data/lib/mutineer/test_runners.rb +3 -2
- data/lib/mutineer/triage.rb +55 -10
- data/lib/mutineer/version.rb +1 -1
- data/lib/mutineer.rb +1 -0
- data/skills/mutineer/SKILL.md +12 -4
- metadata +13 -1
data/README.md
CHANGED
|
@@ -19,6 +19,50 @@ Changes that still pass reveal possible test gaps. This is **mutation testing**.
|
|
|
19
19
|
|
|
20
20
|
📖 **[davidteren.github.io/mutineer →](https://davidteren.github.io/mutineer/)**: overview, operators, and usage.
|
|
21
21
|
|
|
22
|
+
## Quick start
|
|
23
|
+
|
|
24
|
+
Plain Ruby:
|
|
25
|
+
|
|
26
|
+
```sh
|
|
27
|
+
gem install mutineer
|
|
28
|
+
mutineer run lib/discount.rb --test test/discount_test.rb
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Rails: add to your `Gemfile` test group:
|
|
32
|
+
|
|
33
|
+
```ruby
|
|
34
|
+
gem "mutineer", group: :test, require: false
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Install, create `.mutineer.yml`, and check one model:
|
|
38
|
+
|
|
39
|
+
```sh
|
|
40
|
+
bundle install
|
|
41
|
+
RAILS_ENV=test bundle exec mutineer init --rails
|
|
42
|
+
RAILS_ENV=test bundle exec mutineer run app/models/order.rb --rails
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Without `--test`, `app/models/order.rb` pairs with `test/models/order_test.rb` or `spec/models/order_spec.rb`.
|
|
46
|
+
A concern at `app/models/concerns/storage/totaled.rb` pairs with `test/models/concerns/storage/totaled_test.rb` or an unclaimed split test such as `test/models/concerns/storage/totaled_bytes_test.rb`.
|
|
47
|
+
With no test there, it pairs with `test/models/storage/totaled_test.rb`, an unclaimed split test such as `test/models/storage/totaled_bytes_test.rb`, `test/models/storage/test_totaled.rb`, or `spec/models/storage/totaled_spec.rb`.
|
|
48
|
+
See [Quick start](https://github.com/davidteren/mutineer/blob/main/docs/quick-start.md).
|
|
49
|
+
|
|
50
|
+
## Documentation
|
|
51
|
+
|
|
52
|
+
| Guide | What it covers |
|
|
53
|
+
|-------|----------------|
|
|
54
|
+
| [Quick start](https://github.com/davidteren/mutineer/blob/main/docs/quick-start.md) | Install, run, fix a survivor |
|
|
55
|
+
| [First run to steady state](https://github.com/davidteren/mutineer/blob/main/docs/getting-started.md) | One file through changed-code CI |
|
|
56
|
+
| [Reading results](https://github.com/davidteren/mutineer/blob/main/docs/results.md) | Reports, survivors, suppression, mutant ids, kill matrix |
|
|
57
|
+
| [Set up Rails, the daemon, and databases](https://github.com/davidteren/mutineer/blob/main/docs/rails-setup.md) | Rails, daemon, databases, Spring, Bootsnap, Ruby < 3.4 |
|
|
58
|
+
| [Roll out on a Rails app](https://github.com/davidteren/mutineer/blob/main/docs/rails.md) | Four stages from model to CI gate |
|
|
59
|
+
| [CI: changed code and baselines](https://github.com/davidteren/mutineer/blob/main/docs/ci.md) | `--since`, `--baseline`, Action inputs, timeouts |
|
|
60
|
+
| [CLI reference](https://github.com/davidteren/mutineer/blob/main/docs/cli.md) | Flags, defaults, `.mutineer.yml` |
|
|
61
|
+
| [Comparison](https://github.com/davidteren/mutineer/blob/main/docs/comparison.md) | Mutineer, Mutant, other Ruby tools, fit-check prompt |
|
|
62
|
+
| [AI agents and CI](https://davidteren.github.io/mutineer/agentic-coding.html) | Agent loop and CI recipes |
|
|
63
|
+
| [JSON schema](https://davidteren.github.io/mutineer/json-schema.html) | Versioned report contract |
|
|
64
|
+
| [STABILITY.md](https://github.com/davidteren/mutineer/blob/main/STABILITY.md) | Release guarantees |
|
|
65
|
+
|
|
22
66
|
## Is it production-ready?
|
|
23
67
|
|
|
24
68
|
Check your database and framework before adding a CI gate.
|
|
@@ -56,9 +100,12 @@ job runs it. SQLite file means a database file, not `:memory:`.
|
|
|
56
100
|
- Postgres worker databases are copied once at the start; MySQL 8 worker databases are created then and the rows copied.
|
|
57
101
|
- A test that commits rows outside a transaction leaves them in that worker's
|
|
58
102
|
database, and the next mutant on that worker sees them. Rails parallel tests have the same limit.
|
|
59
|
-
- RSpec:
|
|
60
|
-
|
|
103
|
+
- RSpec: the first failed example with an Active Record connection or lock
|
|
104
|
+
error scores `errored`, on the same classes as Minitest
|
|
61
105
|
([#347](https://github.com/davidteren/mutineer/issues/347)).
|
|
106
|
+
On Rails 8, `ConnectionNotDefined` is a code error and still scores `killed`.
|
|
107
|
+
On Rails 7.2 that case raises `ConnectionNotEstablished` and scores `errored`.
|
|
108
|
+
Other statement errors still score `killed`.
|
|
62
109
|
|
|
63
110
|
Mutineer sets `PARALLEL_WORKERS=1`, so a `parallelize` call in `test_helper.rb` starts no Rails workers during a mutant run.
|
|
64
111
|
See [Rails parallel tests, Spring, Bootsnap, and Redis](https://github.com/davidteren/mutineer/blob/main/docs/rails-setup.md#rails-parallel-tests-spring-bootsnap-and-redis).
|
|
@@ -128,54 +175,35 @@ This gap is common in rushed tests and tests written by AI agents.
|
|
|
128
175
|
1. **Parse.** [Prism](https://github.com/ruby/prism), included in Ruby 3.4, parses source files. Mutineer uses only Prism and the standard library.
|
|
129
176
|
2. **Mutate.** Operators make one change per mutant. Mutineer parses each result again and skips invalid Ruby.
|
|
130
177
|
3. **Map coverage and check unchanged tests.** Each test file runs with coverage to map lines to tests.
|
|
131
|
-
`.mutineer/coverage.json` caches the map
|
|
132
|
-
|
|
178
|
+
`.mutineer/coverage.json` caches the map. A method-body edit with no added or removed line recaptures the tests that cover that source; an added or edited test file recaptures that test. A class-body edit, a load-time method, a `--require` file, or a project file the tests load recaptures every test. A fresh capture runs every
|
|
179
|
+
test alone; a failure stops the run. With a cached map, only the tests that cover the sources run again,
|
|
180
|
+
alone and together, because a mutant runs only those tests. In a standalone run, a line that runs only
|
|
181
|
+
while the source loads does not count as coverage of every test. The conventionally paired test still
|
|
182
|
+
covers that line. A `tests:` map entry does not count. With no conventionally paired test in the run,
|
|
183
|
+
every test still covers it.
|
|
133
184
|
4. **Run each mutant in a fork.** The parent loads code once, including the app with `--rails`.
|
|
185
|
+
A source that needs other files loaded first can use `--boot` or `boot:`, or be listed after the file that loads them.
|
|
186
|
+
Without a boot file, `require:` files load after the sources.
|
|
134
187
|
Children run in parallel on Linux and macOS.
|
|
135
188
|
Each loads the changed file (`reload`) or replaces the changed method (`redefine`).
|
|
136
189
|
Only covering test files run: paired tests first, then fastest to slowest. In-process runs stop at the first failure unless `--matrix` requests every covering test.
|
|
190
|
+
The clean check and each mutant run share one seed: Minitest uses `SEED` or `1`, and a random RSpec order uses seed `1` unless the suite sets one. See the [CLI reference](https://github.com/davidteren/mutineer/blob/main/docs/cli.md#test-order).
|
|
137
191
|
5. **Score.** A failing test kills the mutant; all tests passing means it survived.
|
|
138
192
|
Score: `killed / (killed + survived)`. In coverage-guided runs, timeouts, errors, and uncovered lines count as neither; `--test-command` has no coverage, so a mutant no test reaches survives.
|
|
139
193
|
|
|
140
194
|
| Alternative backend | Changes to step 4 |
|
|
141
195
|
|---|---|
|
|
142
196
|
| `--daemon` | Boots Rails once in a persistent process; isolates each worker's test database |
|
|
143
|
-
| `--test-command` | Runs tests in another Ruby subprocess; writes mutants to the real file, then restores it |
|
|
197
|
+
| `--test-command` | Runs tests in another Ruby subprocess; writes mutants to the real file, then restores it. Run the tests in a fixed order (`SEED=1 mutineer run ...` for Minitest, `--order defined` for RSpec, with no `--order rand` in `SPEC_OPTS`) |
|
|
144
198
|
|
|
145
199
|
| At a glance | |
|
|
146
200
|
|---|---|
|
|
147
201
|
| Parser | Prism, with no runtime gem dependencies |
|
|
148
202
|
| Ruby | 3.4 or later; older apps use `--test-command` |
|
|
149
|
-
| Operators |
|
|
203
|
+
| Operators | 22 (5 by default, 17 opt-in) |
|
|
150
204
|
| Isolation | One fork per mutant on Linux and macOS |
|
|
151
205
|
| Report formats | human, JSON (versioned schema), HTML |
|
|
152
206
|
|
|
153
|
-
## Quick start
|
|
154
|
-
|
|
155
|
-
Plain Ruby:
|
|
156
|
-
|
|
157
|
-
```sh
|
|
158
|
-
gem install mutineer
|
|
159
|
-
mutineer run lib/discount.rb --test test/discount_test.rb
|
|
160
|
-
```
|
|
161
|
-
|
|
162
|
-
Rails: add to your `Gemfile` test group:
|
|
163
|
-
|
|
164
|
-
```ruby
|
|
165
|
-
gem "mutineer", group: :test, require: false
|
|
166
|
-
```
|
|
167
|
-
|
|
168
|
-
Install, create `.mutineer.yml`, and check one model:
|
|
169
|
-
|
|
170
|
-
```sh
|
|
171
|
-
bundle install
|
|
172
|
-
RAILS_ENV=test bundle exec mutineer init --rails
|
|
173
|
-
RAILS_ENV=test bundle exec mutineer run app/models/order.rb --rails
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
Without `--test`, `app/models/order.rb` pairs with `test/models/order_test.rb` or `spec/models/order_spec.rb`.
|
|
177
|
-
See [Quick start](https://github.com/davidteren/mutineer/blob/main/docs/quick-start.md).
|
|
178
|
-
|
|
179
207
|
## Usage
|
|
180
208
|
|
|
181
209
|
```sh
|
|
@@ -183,23 +211,32 @@ mutineer run <source...> [--test <test>...] [options]
|
|
|
183
211
|
mutineer init [--rails] [--force]
|
|
184
212
|
mutineer triage REPORT.json --reason TEXT (--id ID ... | --all)
|
|
185
213
|
mutineer migrate <source...> [--dry-run] [--operators LIST]
|
|
214
|
+
mutineer config cache_dir|baseline [run options]
|
|
186
215
|
```
|
|
187
216
|
|
|
217
|
+
`mutineer triage REPORT.json --reason TEXT` prints `ignore:` entries and does not edit files.
|
|
218
|
+
`--id ID` selects one survivor. Repeat `--id` for several. `--all` selects every survivor.
|
|
219
|
+
A copy id selects that survivor and every id in its `copies`.
|
|
220
|
+
Paste every entry. An entry ignores only its own id.
|
|
221
|
+
|
|
222
|
+
`migrate --dry-run` lists `ignore:` entries that match no mutant in the sources and operators you pass. It writes nothing and exits 1 when any entry matches nothing. Run it over every source. `run` warns once for each such entry in that run, lists the ids under `summary.unmatched_ignores`, and still exits 0. `--since` does not warn about an entry whose mutant sits outside the diff. A run of some files warns only about entries those files and operators do not match. A source you passed that has no paired test is skipped, but its mutants still count for ignore matching, so `run` and `run --dry-run` agree; they stay out of the score and cannot fail `--threshold` (see [unpaired counts](https://github.com/davidteren/mutineer/blob/main/docs/json-schema.md)). `--only` skips the check. See [Mutant ids](https://github.com/davidteren/mutineer/blob/main/docs/results.md#mutant-ids).
|
|
223
|
+
|
|
188
224
|
### Options
|
|
189
225
|
|
|
190
226
|
Full flags, defaults, and `.mutineer.yml` keys: [CLI reference](https://github.com/davidteren/mutineer/blob/main/docs/cli.md).
|
|
191
227
|
|
|
192
228
|
| Flag | Meaning |
|
|
193
229
|
|------|---------|
|
|
194
|
-
| `--test
|
|
230
|
+
| `--test PATH` | Test file, directory, or quoted glob. Repeat the flag. Omit it to pair tests by convention and apply any `tests:` map. Unmatched sources are skipped and counted in the report. |
|
|
195
231
|
| `--threshold FLOAT` | Exit 1 when the score is below FLOAT, or when nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted (default: 0 = off) |
|
|
196
232
|
| `--since REF` | Mutate only lines changed since git `REF`, such as `origin/main` |
|
|
197
233
|
| `--baseline FILE` | Exit 1 for new survivors, or score drops on full runs, against a prior JSON report from a full run |
|
|
198
234
|
| `--rails` | Boot `config/environment` once; serial without `--daemon` |
|
|
235
|
+
| `--boot FILE` | Require this file once before the sources. An app entry point, or a gem's `lib/<name>.rb`. `require:` files load after this file |
|
|
199
236
|
| `--[no-]daemon` | Enable or disable daemon; isolated worker databases, Minitest only |
|
|
200
237
|
| `--jobs N` | Worker count; default: one per CPU |
|
|
201
238
|
| `--format human\|json\|html` | Report format; default: human |
|
|
202
|
-
| `--dry-run` | List mutants without running tests |
|
|
239
|
+
| `--dry-run` | List mutants without running tests. With `--format json`, print them with their ids |
|
|
203
240
|
|
|
204
241
|
### Exit codes
|
|
205
242
|
|
|
@@ -207,7 +244,7 @@ Full flags, defaults, and `.mutineer.yml` keys: [CLI reference](https://github.c
|
|
|
207
244
|
| Code | Meaning |
|
|
208
245
|
|------|---------|
|
|
209
246
|
| `0` | Score ≥ threshold (or no gate) **and** no baseline regression. |
|
|
210
|
-
| `1` | Score below `--threshold`, OR nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted, OR a `--baseline` regression, OR a runtime error, including a boot, source or `require:` file that raises while loading. |
|
|
247
|
+
| `1` | Score below `--threshold`, OR nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted, OR a `--baseline` regression, OR a runtime error, including a boot, source or `require:` file that raises while loading (except an in-process `--since` run with no mutant on a changed line, which checks that those files exist but does not load them). |
|
|
211
248
|
| `2` | Usage / invalid-flag error, including a mistyped flag, a bad path (such as a missing `--boot` or `require:` file), or an unreadable baseline. |
|
|
212
249
|
<!-- /contract:exit-codes -->
|
|
213
250
|
|
|
@@ -218,23 +255,21 @@ List with `mutineer --list-operators`. Enable specific operators with `--operato
|
|
|
218
255
|
| Group | Operators |
|
|
219
256
|
|---|---|
|
|
220
257
|
| Tier 1 (default) | `arithmetic`, `comparison`, `boolean_connector`, `boolean_literal`, `statement_removal` |
|
|
221
|
-
| Tier 2 (opt-in) | `return_nil`, `literal_mutation`, `condition_negation`, `string_literal`, `regex`, `collection_method`, `safe_navigation`, `range`, `negation_removal`, `chain_link`, `call_to_receiver`, `operand_removal`, `array_literal`, `condition_true`, `condition_false`, `operator_assignment` |
|
|
258
|
+
| Tier 2 (opt-in) | `return_nil`, `literal_mutation`, `condition_negation`, `string_literal`, `regex`, `collection_method`, `safe_navigation`, `range`, `negation_removal`, `chain_link`, `call_to_receiver`, `operand_removal`, `array_literal`, `condition_true`, `condition_false`, `operator_assignment`, `rescue_body` |
|
|
222
259
|
|
|
223
|
-
|
|
260
|
+
The default operators give no mutant to some methods, such as a method whose body is one `if` with a `rescue`.
|
|
261
|
+
The report counts these methods on its `No mutants:` line. `--dry-run --verbose` and the JSON report list them.
|
|
262
|
+
A method listed there is not proven tested. For a method with an `if`, `unless`, ternary or modifier condition,
|
|
263
|
+
add `condition_true` and `condition_false` to force its branches. They do not help an empty method or a `case`.
|
|
264
|
+
The `--operators` list replaces the default set, so name the default operators too:
|
|
224
265
|
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
| [CI: changed code and baselines](https://github.com/davidteren/mutineer/blob/main/docs/ci.md) | `--since`, `--baseline`, Action inputs, timeouts |
|
|
233
|
-
| [CLI reference](https://github.com/davidteren/mutineer/blob/main/docs/cli.md) | Flags, defaults, `.mutineer.yml` |
|
|
234
|
-
| [Comparison](https://github.com/davidteren/mutineer/blob/main/docs/comparison.md) | Mutineer, Mutant, other Ruby tools, fit-check prompt |
|
|
235
|
-
| [AI agents and CI](https://davidteren.github.io/mutineer/agentic-coding.html) | Agent loop and CI recipes |
|
|
236
|
-
| [JSON schema](https://davidteren.github.io/mutineer/json-schema.html) | Versioned report contract |
|
|
237
|
-
| [STABILITY.md](https://github.com/davidteren/mutineer/blob/main/STABILITY.md) | Release guarantees |
|
|
266
|
+
```sh
|
|
267
|
+
mutineer run lib --operators arithmetic,comparison,boolean_connector,boolean_literal,statement_removal,condition_true,condition_false
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
For a `rescue` clause, add `rescue_body`. It replaces each `rescue` body with `nil`, even a one-statement body,
|
|
271
|
+
and turns `x rescue y` into `x rescue nil`. `return_nil` also reaches a method with a def-level `rescue`:
|
|
272
|
+
it gives `nil` for the final expression of the body (or of the `else` clause when there is one) and of each `rescue` clause (#418). A final explicit `begin ... end` block gets the same treatment (#470).
|
|
238
273
|
|
|
239
274
|
## For AI agents & pipelines
|
|
240
275
|
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
require "set"
|
|
4
4
|
require "open3"
|
|
5
|
+
require_relative "fs_path"
|
|
5
6
|
|
|
6
7
|
module Mutineer
|
|
7
8
|
# Maps each source file to the set of NEW-side line numbers changed since a
|
|
@@ -50,7 +51,7 @@ module Mutineer
|
|
|
50
51
|
# @return [Hash<String, Set<Integer>>] absolute file path to changed lines.
|
|
51
52
|
def for(ref:, files:, project_root:, runner: method(:git_diff))
|
|
52
53
|
files.each_with_object({}) do |file, acc|
|
|
53
|
-
abs =
|
|
54
|
+
abs = FsPath.expand(file, project_root)
|
|
54
55
|
acc[abs] = parse(runner.call(ref, abs, project_root))
|
|
55
56
|
end
|
|
56
57
|
end
|
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "redaction"
|
|
4
|
+
|
|
5
|
+
module Mutineer
|
|
6
|
+
# Carries one line from a mutant's child to its parent: why the child
|
|
7
|
+
# raised. The parent puts it in the errored mutant's `details`, so the JSON
|
|
8
|
+
# report names the cause, not only stderr (#388, #417).
|
|
9
|
+
#
|
|
10
|
+
# The child writes the line to a pipe just before `exit!`; the parent reads
|
|
11
|
+
# it after the reap, without blocking, so a grandchild that still holds the
|
|
12
|
+
# write end cannot delay the result.
|
|
13
|
+
#
|
|
14
|
+
# The fork boundary {arm!}s the write end as process-local state so Minitest
|
|
15
|
+
# and RSpec can write a cause without a new runner argument (plan KTD3).
|
|
16
|
+
#
|
|
17
|
+
# Stdlib only: the app-side daemon loads this file without the rest of
|
|
18
|
+
# mutineer.
|
|
19
|
+
module ChildCause
|
|
20
|
+
# The longest line sent, in characters, including the `...` of a cut line.
|
|
21
|
+
LIMIT = 500
|
|
22
|
+
|
|
23
|
+
# Fixed text when `--matrix` ends early after a serial database error (KTD5).
|
|
24
|
+
MATRIX_EARLY_END =
|
|
25
|
+
"a test failed with a database error, then the child ended before the suite finished"
|
|
26
|
+
|
|
27
|
+
# Fixed text when RSpec reports an error outside examples (KTD5).
|
|
28
|
+
RSPEC_OUTSIDE_EXAMPLES =
|
|
29
|
+
"RSpec reported an error outside its examples, for example a spec file that raised while it loaded"
|
|
30
|
+
|
|
31
|
+
module_function
|
|
32
|
+
|
|
33
|
+
# Remembers the cause pipe for this process so suite code can write without
|
|
34
|
+
# holding the IO. Only {#write_error} / {#write_text} from the same pid succeed.
|
|
35
|
+
#
|
|
36
|
+
# @param io [IO] the pipe's write end, with `sync` on.
|
|
37
|
+
# @param owner [Integer] the pid of the mutant's child.
|
|
38
|
+
# @return [void]
|
|
39
|
+
def arm!(io, owner)
|
|
40
|
+
@armed_io = io
|
|
41
|
+
@armed_owner = owner
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# Clears {arm!}. Safe when nothing was armed.
|
|
45
|
+
#
|
|
46
|
+
# @return [void]
|
|
47
|
+
def disarm!
|
|
48
|
+
@armed_io = nil
|
|
49
|
+
@armed_owner = nil
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# The error as `Class: first non-blank message line`, or the class alone
|
|
53
|
+
# when the message is blank or is the class name. Redacted before the cut,
|
|
54
|
+
# so a cut cannot leave part of a password. Never raises.
|
|
55
|
+
#
|
|
56
|
+
# @param error [Exception]
|
|
57
|
+
# @return [String]
|
|
58
|
+
def line(error)
|
|
59
|
+
name = Redaction.text(error.class)
|
|
60
|
+
first = nil
|
|
61
|
+
Redaction.text(error.message).each_line("\n") do |l|
|
|
62
|
+
first = l.strip
|
|
63
|
+
break unless first.empty?
|
|
64
|
+
end
|
|
65
|
+
first = nil if first&.empty?
|
|
66
|
+
text = first.nil? || first == name ? name : "#{name}: #{first}"
|
|
67
|
+
text.length > LIMIT ? "#{text[0, LIMIT - 3]}..." : text
|
|
68
|
+
rescue StandardError
|
|
69
|
+
Redaction::FAILED
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# Write {line} for `error` to the pipe, but only from the mutant's child
|
|
73
|
+
# itself. A grandchild made with a bare `fork` (no block) runs the child's
|
|
74
|
+
# code after the fork, so its error can reach the same rescue; it writes
|
|
75
|
+
# nothing, so `details` never names another process's error. Never
|
|
76
|
+
# raises: a failed write only loses the cause, never the verdict.
|
|
77
|
+
#
|
|
78
|
+
# @param io [IO] the pipe's write end, with `sync` on (the child ends with `exit!`).
|
|
79
|
+
# @param error [Exception]
|
|
80
|
+
# @param owner [Integer] the pid of the mutant's child.
|
|
81
|
+
# @return [void]
|
|
82
|
+
def write(io, error, owner)
|
|
83
|
+
return unless Process.pid == owner
|
|
84
|
+
|
|
85
|
+
write_to(io, line(error))
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
# Write {line} for `error` to the armed pipe (#417).
|
|
89
|
+
#
|
|
90
|
+
# @param error [Exception]
|
|
91
|
+
# @return [void]
|
|
92
|
+
def write_error(error)
|
|
93
|
+
return unless armed_here?
|
|
94
|
+
|
|
95
|
+
write_to(@armed_io, line(error))
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
# Write a fixed or already-built cause text to the armed pipe (#417).
|
|
99
|
+
# Redacted and cut like {line}.
|
|
100
|
+
#
|
|
101
|
+
# @param text [String]
|
|
102
|
+
# @return [void]
|
|
103
|
+
def write_text(text)
|
|
104
|
+
return unless armed_here?
|
|
105
|
+
|
|
106
|
+
body = Redaction.text(text.to_s)
|
|
107
|
+
body = body.length > LIMIT ? "#{body[0, LIMIT - 3]}..." : body
|
|
108
|
+
write_to(@armed_io, body)
|
|
109
|
+
rescue StandardError
|
|
110
|
+
nil
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
# The last complete line in the pipe now, as UTF-8, or nil. Reads without
|
|
114
|
+
# blocking.
|
|
115
|
+
#
|
|
116
|
+
# @param io [IO] the pipe's read end.
|
|
117
|
+
# @return [String, nil]
|
|
118
|
+
def read(io)
|
|
119
|
+
buffer = +""
|
|
120
|
+
loop do
|
|
121
|
+
chunk = io.read_nonblock(65_536, exception: false)
|
|
122
|
+
break if chunk.nil? || chunk == :wait_readable
|
|
123
|
+
|
|
124
|
+
buffer << chunk
|
|
125
|
+
end
|
|
126
|
+
last = buffer.lines("\n").reverse.find { |l| l.end_with?("\n") }
|
|
127
|
+
last && Redaction.utf8(last.chomp("\n").force_encoding(Encoding::UTF_8))
|
|
128
|
+
rescue StandardError
|
|
129
|
+
nil
|
|
130
|
+
end
|
|
131
|
+
|
|
132
|
+
# @api private
|
|
133
|
+
# @return [Boolean]
|
|
134
|
+
def armed_here?
|
|
135
|
+
@armed_io && @armed_owner && Process.pid == @armed_owner
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
# @api private
|
|
139
|
+
# @param io [IO]
|
|
140
|
+
# @param text [String]
|
|
141
|
+
# @return [void]
|
|
142
|
+
def write_to(io, text)
|
|
143
|
+
io.write("#{text}\n")
|
|
144
|
+
rescue StandardError
|
|
145
|
+
nil
|
|
146
|
+
end
|
|
147
|
+
private_class_method :write_to
|
|
148
|
+
end
|
|
149
|
+
end
|
|
@@ -23,12 +23,23 @@ module Mutineer
|
|
|
23
23
|
# file in another mode (for example the "w+x" Tempfile of Minitest's
|
|
24
24
|
# `capture_subprocess_io`).
|
|
25
25
|
#
|
|
26
|
+
# The null device is binary. A text-mode one still transcodes writes to
|
|
27
|
+
# `Encoding.default_external` once `Encoding.default_internal` is set too.
|
|
28
|
+
# Rails sets both to UTF-8 (`config.encoding`). The reporter's print of a
|
|
29
|
+
# failure message that holds raw bytes (what `assert_response` does with a
|
|
30
|
+
# short body) then raises `Encoding::UndefinedConversionError`, after the
|
|
31
|
+
# test has already failed. The child's rescue scores that `error`, not
|
|
32
|
+
# `killed` (#434). Nothing reads the silenced bytes.
|
|
33
|
+
#
|
|
26
34
|
# Call it only in a child that exits after the tests run. Nothing restores
|
|
27
35
|
# the previous stdout.
|
|
28
36
|
#
|
|
29
37
|
# @return [void]
|
|
30
38
|
def self.silence
|
|
31
|
-
File.open(File::NULL, "
|
|
39
|
+
File.open(File::NULL, "wb") { |null| STDOUT.reopen(null) }
|
|
40
|
+
# `reopen` can keep a UTF-8 encoding the process already had. Binary
|
|
41
|
+
# mode drops it, so a later write of those bytes cannot raise.
|
|
42
|
+
STDOUT.binmode
|
|
32
43
|
$stdout = STDOUT
|
|
33
44
|
end
|
|
34
45
|
end
|