mutineer 1.0.1 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +93 -0
- data/README.md +10 -4
- data/lib/mutineer/child_stdout.rb +35 -0
- data/lib/mutineer/coverage_map.rb +86 -47
- data/lib/mutineer/daemon_client.rb +1 -1
- data/lib/mutineer/daemon_server.rb +2 -1
- data/lib/mutineer/file_swap.rb +2 -0
- data/lib/mutineer/isolation.rb +8 -2
- data/lib/mutineer/minitest_integration/stop_at_first_failure.rb +199 -0
- data/lib/mutineer/minitest_integration.rb +30 -14
- data/lib/mutineer/mutant_id.rb +1 -1
- data/lib/mutineer/mutator_registry.rb +22 -9
- data/lib/mutineer/mutators/chain_link.rb +139 -0
- data/lib/mutineer/mutators/negation_removal.rb +44 -0
- data/lib/mutineer/mutators/range_literal.rb +54 -0
- data/lib/mutineer/mutators/safe_navigation.rb +76 -0
- data/lib/mutineer/runner.rb +2 -1
- data/lib/mutineer/test_runners/minitest.rb +5 -1
- data/lib/mutineer/test_runners/rspec.rb +9 -11
- data/lib/mutineer/test_runners.rb +4 -2
- data/lib/mutineer/version.rb +1 -1
- data/lib/mutineer.rb +4 -0
- metadata +7 -15
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1ca70f2affdf60ddda53d3d38e23ddff4ba6cae27cd789501c7fe3060975f146
|
|
4
|
+
data.tar.gz: 94a9968fe182655e6d566e9edfc43f1f2e15b050c7690f8905e7be9f7d710b61
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8da96d70db7af241cda4ca44ecbe84522b0c0b68427a46c05e3178723c2eada7d957677c8b2cc7d2853d9593cba546c194f342862058dffdba55d03e8ef27021
|
|
7
|
+
data.tar.gz: 50481e9ca34d07530878f6ff0f25f192ab63c3200376a1a1ea5665341955a00c6bf2f9e7a4429d6cd8fd8d2119c4316c4c993741f219897f57ab493ca6c428b1
|
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,97 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [1.1.0] - 2026-09-28
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
- **Safe-navigation operator** (Tier-2, opt-in via `--operators`):
|
|
13
|
+
`safe_navigation` replaces `&.` with `.`. The mutant survives when no test
|
|
14
|
+
passes `nil` to the call.
|
|
15
|
+
- **Range operator** (Tier-2, opt-in via `--operators`): `range` replaces
|
|
16
|
+
`..` with `...` and `...` with `..`. The `..` -> `...` mutant survives when
|
|
17
|
+
no test checks the last element of the range. Endless ranges (`1..`) are
|
|
18
|
+
skipped, because `(1..)` and `(1...)` give the same result for slicing,
|
|
19
|
+
`include?`, `===` and pattern matching.
|
|
20
|
+
- **Negation-removal operator** (Tier-2, opt-in via `--operators`):
|
|
21
|
+
`negation_removal` removes the `!` from `!x` and the `not` from `not x`.
|
|
22
|
+
The mutant survives when no test depends on the negated value. The
|
|
23
|
+
explicit form `x.!` is skipped, because `x.` does not parse.
|
|
24
|
+
|
|
25
|
+
### Changed
|
|
26
|
+
- **Stderr of tests and specs is visible** in the in-process and `--daemon`
|
|
27
|
+
runs. Mutineer silences stdout once per child process and no longer hides
|
|
28
|
+
stderr, so its own child diagnostics always reach you. `--test-command`
|
|
29
|
+
runs still capture stderr with stdout and show it under `--verbose`.
|
|
30
|
+
- **A mutant's test run stops at the first failing test** — one failure
|
|
31
|
+
already kills the mutant, so the forked child does not run the tests that
|
|
32
|
+
remain. Killed mutants cost less time, and survived mutants cost the same.
|
|
33
|
+
Under Minitest, when the outer reporter of the run records a failure or an
|
|
34
|
+
error (a skip does not count), each remaining test and each remaining test
|
|
35
|
+
class returns before it starts. A skipped class does not start its
|
|
36
|
+
class-level hooks. The run does not unwind: a class that is running
|
|
37
|
+
finishes normally, so its `after_all` hooks and a class-level
|
|
38
|
+
`transaction { super; raise ActiveRecord::Rollback }` still run. RSpec runs
|
|
39
|
+
with `--fail-fast`. This applies to the in-process backend only: coverage
|
|
40
|
+
capture and the clean checks still run every test, and the `--daemon` and
|
|
41
|
+
`--test-command` backends do not change. The CLI `--fail-fast` flag keeps
|
|
42
|
+
its meaning. On rack's `lib/rack/utils.rb` (`--jobs 1`), a full run takes
|
|
43
|
+
about 35–41 s instead of about 86–89 s. With the same coverage map, the
|
|
44
|
+
verdicts are the same.
|
|
45
|
+
- **The mutant run uses a fixed Minitest seed** — with the stop, the test
|
|
46
|
+
order can decide the verdict, so the child runs Minitest with seed `1`
|
|
47
|
+
unless the environment sets `SEED`. The same code then gives the same
|
|
48
|
+
verdict on each run. Coverage capture and the clean checks keep the random
|
|
49
|
+
seed, so the clean check runs the tests in a random order while each mutant
|
|
50
|
+
run uses the fixed order. RSpec keeps its configured order: a suite configured with
|
|
51
|
+
`config.order = :random` can still get a different verdict on each run for
|
|
52
|
+
the case below. In an order-dependent Minitest suite, the fixed seed makes
|
|
53
|
+
a false `killed` happen on every run or on no run, not on some runs.
|
|
54
|
+
- **A mutant whose failing test runs before a hanging test is now `killed`,
|
|
55
|
+
not `timeout`** — the run stops at the failure, before the hang. The tests
|
|
56
|
+
did detect the mutation, so `killed` is the correct verdict. If the hanging
|
|
57
|
+
test runs first in the fixed order, the verdict stays `timeout`. Compared
|
|
58
|
+
with a baseline from an earlier version, the score usually goes up. In an
|
|
59
|
+
order-dependent suite it can also go down: a mutant that a random order
|
|
60
|
+
killed on some runs can survive on every run in the fixed order. A change
|
|
61
|
+
to the tests can change the fixed order, so a later run can move such a
|
|
62
|
+
mutant from `killed` to `timeout`, and a `--baseline` gate then reports a
|
|
63
|
+
score drop.
|
|
64
|
+
- **Some runs still run most tests** — Minitest `parallelize_me!`, and Rails
|
|
65
|
+
`parallelize` above its threshold (by default more than 50 tests in the
|
|
66
|
+
child, or at any test count when `PARALLEL_WORKERS` is 2 or more in the
|
|
67
|
+
environment), queue their tests before the first result comes back, so the
|
|
68
|
+
queued tests still run. The verdict is the same as before. Below the
|
|
69
|
+
Rails threshold, the tests run one after the other in the child, and the
|
|
70
|
+
stop works.
|
|
71
|
+
|
|
72
|
+
### Fixed
|
|
73
|
+
- **Tests that reopen `$stdout`** (Minitest's `capture_subprocess_io`,
|
|
74
|
+
RSpec's `to_stdout_from_any_process`) no longer make a green suite
|
|
75
|
+
"not green" or count as false kills.
|
|
76
|
+
- **Test or source files that print while they load** no longer make coverage
|
|
77
|
+
capture fail with `invalid coverage output`. The capture subprocess now
|
|
78
|
+
sends its result over a separate pipe, not over stdout.
|
|
79
|
+
- **Chain-link operator** (Tier-2, opt-in via `--operators`):
|
|
80
|
+
`chain_link` drops one call from a chain, with its arguments and block
|
|
81
|
+
(`user.account.name` -> `user.name`). The mutant survives when no test tells
|
|
82
|
+
the chain apart from the same chain without that step. Conversions and copies
|
|
83
|
+
(`to_s`, `to_a`, `dup`, `freeze`, ...) and `new` are never dropped.
|
|
84
|
+
|
|
85
|
+
## [1.0.2] - 2026-09-21
|
|
86
|
+
|
|
87
|
+
### Added
|
|
88
|
+
- **AI-readable docs wiring**: HTML pages with Markdown twins now advertise
|
|
89
|
+
`rel="alternate" type="text/markdown"`, `index.md` is the landing/CLI
|
|
90
|
+
essentials twin, `skill.md` is listed under Optional in `llms.txt`, and
|
|
91
|
+
`sitemap.xml` is generated from the same catalog as `llms.txt` (#91).
|
|
92
|
+
- **Single-source CLI contract**: exit codes and `--threshold` live in
|
|
93
|
+
`docs/fragments/contract.yml`; `rake docs:generate` writes `llms-full.txt`,
|
|
94
|
+
`json-schema.html`, and the marked copies so they cannot drift (#82).
|
|
95
|
+
- **YARD API on Pages**: the current gem's YARD HTML is published at
|
|
96
|
+
`/api/`, linked from the docs site, and `rake yard:pages:check` keeps it
|
|
97
|
+
from lagging the shipped sources. `documentation_uri` stays the Pages
|
|
98
|
+
root (#92).
|
|
99
|
+
|
|
9
100
|
## [1.0.1] - 2026-09-18
|
|
10
101
|
|
|
11
102
|
### Added
|
|
@@ -407,6 +498,8 @@ Rails hardening + CI batch (issues #8–#13), all verified Rails-free.
|
|
|
407
498
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
408
499
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
409
500
|
|
|
501
|
+
[1.1.0]: https://github.com/davidteren/mutineer/releases/tag/v1.1.0
|
|
502
|
+
[1.0.2]: https://github.com/davidteren/mutineer/releases/tag/v1.0.2
|
|
410
503
|
[1.0.1]: https://github.com/davidteren/mutineer/releases/tag/v1.0.1
|
|
411
504
|
[1.0.0]: https://github.com/davidteren/mutineer/releases/tag/v1.0.0
|
|
412
505
|
[0.11.4]: https://github.com/davidteren/mutineer/releases/tag/v0.11.4
|
data/README.md
CHANGED
|
@@ -13,6 +13,8 @@ testing anything.
|
|
|
13
13
|
- **One mutation per mutant**, validity-checked by re-parsing.
|
|
14
14
|
- **Fork-isolated**, parallel execution (Linux + macOS).
|
|
15
15
|
- **Coverage-guided** — each mutant runs only the test files that cover its line.
|
|
16
|
+
- **Stops at the first failing test** — in-process runs (not `--daemon` or
|
|
17
|
+
`--test-command`) stop a mutant's test run at the first failure.
|
|
16
18
|
|
|
17
19
|
📖 **[mutineer.github.io →](https://davidteren.github.io/mutineer/)** — overview, operators, and usage.
|
|
18
20
|
|
|
@@ -68,11 +70,13 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
|
|
|
68
70
|
|
|
69
71
|
### Exit codes
|
|
70
72
|
|
|
73
|
+
<!-- contract:exit-codes -->
|
|
71
74
|
| Code | Meaning |
|
|
72
75
|
|------|---------|
|
|
73
|
-
| `0` | Score ≥ threshold (or no
|
|
74
|
-
| `1` | Score below threshold
|
|
75
|
-
| `2` | Usage / invalid-flag error |
|
|
76
|
+
| `0` | Score ≥ threshold (or no gate) **and** no baseline regression. |
|
|
77
|
+
| `1` | Score below `--threshold`, OR nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted, OR a `--baseline` regression, OR a runtime error. |
|
|
78
|
+
| `2` | Usage / invalid-flag error (mistyped flag, bad path, unreadable baseline). |
|
|
79
|
+
<!-- /contract:exit-codes -->
|
|
76
80
|
|
|
77
81
|
### Operators
|
|
78
82
|
|
|
@@ -80,7 +84,7 @@ Run `mutineer --list-operators` to see them. Default (Tier 1): `arithmetic`,
|
|
|
80
84
|
`comparison`, `boolean_connector`, `boolean_literal`, `statement_removal`.
|
|
81
85
|
Available but off by default (Tier 2, enable via `--operators`): `return_nil`,
|
|
82
86
|
`literal_mutation`, `condition_negation`, `string_literal`, `regex`,
|
|
83
|
-
`collection_method`.
|
|
87
|
+
`collection_method`, `safe_navigation`, `range`, `negation_removal`, `chain_link`.
|
|
84
88
|
|
|
85
89
|
## Rails apps
|
|
86
90
|
|
|
@@ -273,6 +277,8 @@ structured exit codes, and diff-scoped runs. See:
|
|
|
273
277
|
contract:
|
|
274
278
|
[rendered](https://davidteren.github.io/mutineer/json-schema.html) ·
|
|
275
279
|
[source](docs/json-schema.md)
|
|
280
|
+
- **Ruby API (YARD)** — class reference for the shipped gem:
|
|
281
|
+
[https://davidteren.github.io/mutineer/api/](https://davidteren.github.io/mutineer/api/)
|
|
276
282
|
|
|
277
283
|
## Configuration
|
|
278
284
|
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Mutineer
|
|
4
|
+
# Silences stdout in a forked child that runs tests. Every fork boundary
|
|
5
|
+
# calls {.silence} once, right after `fork`, so the test runners themselves
|
|
6
|
+
# do not touch stdout. The parent reads each child's result from a separate
|
|
7
|
+
# pipe or from the exit status, never from stdout.
|
|
8
|
+
#
|
|
9
|
+
# Stderr stays open: it carries mutineer's own diagnostics from the child.
|
|
10
|
+
#
|
|
11
|
+
# Stdlib-only, so the app-side daemon can load it.
|
|
12
|
+
module ChildStdout
|
|
13
|
+
# Points fd 1 at File::NULL and makes `$stdout` the real STDOUT again.
|
|
14
|
+
#
|
|
15
|
+
# The reopen goes through STDOUT, not `$stdout`: the parent may have left
|
|
16
|
+
# `$stdout` as a StringIO, which cannot reopen. A test that calls
|
|
17
|
+
# `$stdout.reopen` (Minitest's `capture_subprocess_io`, RSpec's
|
|
18
|
+
# `to_stdout_from_any_process`) then gets a real IO. Child processes of the
|
|
19
|
+
# test inherit the silenced fd 1 too.
|
|
20
|
+
#
|
|
21
|
+
# The reopen takes an open IO, not a path. A path reopen checks the access
|
|
22
|
+
# mode of STDOUT and raises ArgumentError when a parent left STDOUT on a
|
|
23
|
+
# file in another mode (for example the "w+x" Tempfile of Minitest's
|
|
24
|
+
# `capture_subprocess_io`).
|
|
25
|
+
#
|
|
26
|
+
# Call it only in a child that exits after the tests run. Nothing restores
|
|
27
|
+
# the previous stdout.
|
|
28
|
+
#
|
|
29
|
+
# @return [void]
|
|
30
|
+
def self.silence
|
|
31
|
+
File.open(File::NULL, "w") { |null| STDOUT.reopen(null) }
|
|
32
|
+
$stdout = STDOUT
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
end
|
|
@@ -1,6 +1,5 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require "open3"
|
|
4
3
|
require "json"
|
|
5
4
|
require "digest"
|
|
6
5
|
require "fileutils"
|
|
@@ -9,6 +8,7 @@ require "coverage"
|
|
|
9
8
|
require "set"
|
|
10
9
|
require_relative "minitest_integration"
|
|
11
10
|
require_relative "test_runners"
|
|
11
|
+
require_relative "child_stdout"
|
|
12
12
|
|
|
13
13
|
module Mutineer
|
|
14
14
|
# Maps `(source_file, line) -> [test_files]` so each mutant runs only against
|
|
@@ -22,6 +22,10 @@ module Mutineer
|
|
|
22
22
|
# Seconds per coverage subprocess before the parent kills it.
|
|
23
23
|
DEFAULT_CAPTURE_TIMEOUT = 120
|
|
24
24
|
|
|
25
|
+
# File descriptor in a capture subprocess that carries the JSON result to
|
|
26
|
+
# the parent. Stdout stays free for test output, which goes to File::NULL.
|
|
27
|
+
RESULT_FD = 3
|
|
28
|
+
|
|
25
29
|
attr_reader :project_root, :failed_test_files, :failed_clean_tests, :phase_a_ran, :map
|
|
26
30
|
|
|
27
31
|
# Build a QUERY-ONLY map from data captured elsewhere (the daemon builds the
|
|
@@ -232,6 +236,7 @@ module Mutineer
|
|
|
232
236
|
rd.close
|
|
233
237
|
payload =
|
|
234
238
|
begin
|
|
239
|
+
ChildStdout.silence
|
|
235
240
|
# Fork-safety hook: the in-process path reconnects AR; the daemon
|
|
236
241
|
# routes to its worker DB. Nil (non-Rails) = no-op. Injected so this
|
|
237
242
|
# file needs neither Runner (Prism) nor Rails.
|
|
@@ -293,22 +298,8 @@ module Mutineer
|
|
|
293
298
|
# before any source is loaded. Returns the wrapped capture payload
|
|
294
299
|
# (`passed` + `coverage`), or nil when the subprocess failed (logged + skipped).
|
|
295
300
|
def capture(test_path)
|
|
296
|
-
out =
|
|
297
|
-
|
|
298
|
-
Open3.popen2(RbConfig.ruby, "-") do |stdin, stdout, wait_thr|
|
|
299
|
-
stdin.write(subprocess_script(test_path))
|
|
300
|
-
stdin.close
|
|
301
|
-
reader = Thread.new { out << stdout.read }
|
|
302
|
-
# Bound the subprocess with a wall clock: a hanging test file must not
|
|
303
|
-
# wedge the whole run before any per-mutant timeout.
|
|
304
|
-
unless wait_thr.join(@capture_timeout)
|
|
305
|
-
Process.kill(:KILL, wait_thr.pid) rescue nil # rubocop:disable Style/RescueModifier
|
|
306
|
-
reader.kill
|
|
307
|
-
return fail_test(test_path, "timed out after #{@capture_timeout}s")
|
|
308
|
-
end
|
|
309
|
-
reader.join
|
|
310
|
-
status = wait_thr.value
|
|
311
|
-
end
|
|
301
|
+
status, out = spawn_script(subprocess_script(test_path))
|
|
302
|
+
return fail_test(test_path, "timed out after #{@capture_timeout}s") unless status
|
|
312
303
|
return fail_test(test_path, "subprocess exited #{status.exitstatus}") unless status.success?
|
|
313
304
|
|
|
314
305
|
parsed = JSON.parse(out)
|
|
@@ -442,20 +433,72 @@ module Mutineer
|
|
|
442
433
|
# @param test_paths [Array<String>] test file paths.
|
|
443
434
|
# @return [Boolean]
|
|
444
435
|
def subprocess_clean_pass?(test_paths)
|
|
445
|
-
status =
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
436
|
+
status, = spawn_script(clean_check_script(test_paths), result: false)
|
|
437
|
+
status&.success? || false
|
|
438
|
+
end
|
|
439
|
+
|
|
440
|
+
# Runs `script` in a fresh `ruby -` that reads the script from stdin. The
|
|
441
|
+
# child's stdout goes to File::NULL, so test output never reaches the user
|
|
442
|
+
# or the result. With `result: true`, the child writes its result as one
|
|
443
|
+
# line to fd {RESULT_FD}, a pipe that only the script uses. The child's
|
|
444
|
+
# stderr is the parent's stderr, so warnings from the script reach the
|
|
445
|
+
# user. A wall clock of `@capture_timeout` bounds the whole call, so a hung
|
|
446
|
+
# test cannot wedge the run.
|
|
447
|
+
#
|
|
448
|
+
# The parent reads one line, not until EOF: a process that a test leaves
|
|
449
|
+
# running can inherit fd {RESULT_FD} (a `fork` without `exec` keeps it
|
|
450
|
+
# despite close-on-exec) and hold the pipe open long after the child exits.
|
|
451
|
+
# A clean check reports only through its exit status, so it gets no pipe.
|
|
452
|
+
#
|
|
453
|
+
# @api private
|
|
454
|
+
# @param script [String] Ruby script text.
|
|
455
|
+
# @param result [Boolean] whether to open the result pipe on fd {RESULT_FD}.
|
|
456
|
+
# @return [Array(Process::Status, String)] the exit status and the line the
|
|
457
|
+
# child wrote to fd {RESULT_FD} (`""` without one); `[nil, ""]` after a
|
|
458
|
+
# timeout.
|
|
459
|
+
def spawn_script(script, result: true)
|
|
460
|
+
deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + @capture_timeout
|
|
461
|
+
script_rd, script_wr = IO.pipe
|
|
462
|
+
result_rd, result_wr = IO.pipe if result
|
|
463
|
+
options = { in: script_rd, out: File::NULL }
|
|
464
|
+
options[RESULT_FD] = result_wr if result
|
|
465
|
+
pid = Process.spawn(RbConfig.ruby, "-", **options)
|
|
466
|
+
waiter = Process.detach(pid)
|
|
467
|
+
script_rd.close
|
|
468
|
+
result_wr&.close
|
|
469
|
+
reader = Thread.new { result_rd.gets.to_s } if result
|
|
470
|
+
script_wr.write(script)
|
|
471
|
+
script_wr.close
|
|
472
|
+
unless waiter.join(remaining(deadline))
|
|
473
|
+
Process.kill(:KILL, pid) rescue nil # rubocop:disable Style/RescueModifier
|
|
474
|
+
waiter.join
|
|
475
|
+
reader&.kill
|
|
476
|
+
return [nil, ""]
|
|
457
477
|
end
|
|
458
|
-
|
|
478
|
+
[waiter.value, reader&.join(remaining(deadline))&.value.to_s]
|
|
479
|
+
ensure
|
|
480
|
+
reader&.kill
|
|
481
|
+
[script_rd, script_wr, result_rd, result_wr].compact.each { |io| io.close unless io.closed? }
|
|
482
|
+
end
|
|
483
|
+
|
|
484
|
+
# Seconds left before `deadline`, never negative.
|
|
485
|
+
#
|
|
486
|
+
# @api private
|
|
487
|
+
# @param deadline [Float] a CLOCK_MONOTONIC time.
|
|
488
|
+
# @return [Float]
|
|
489
|
+
def remaining(deadline)
|
|
490
|
+
[deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC), 0].max
|
|
491
|
+
end
|
|
492
|
+
|
|
493
|
+
# Ruby source that opens the result channel in a {#spawn_script} child. The
|
|
494
|
+
# script runs it first, so no file that a test opens can take fd
|
|
495
|
+
# {RESULT_FD}. Close-on-exec keeps the fd out of the test's own
|
|
496
|
+
# subprocesses.
|
|
497
|
+
#
|
|
498
|
+
# @api private
|
|
499
|
+
# @return [String] Ruby script text.
|
|
500
|
+
def result_channel_expression
|
|
501
|
+
"_result = IO.new(#{RESULT_FD}, \"w\"); _result.close_on_exec = true"
|
|
459
502
|
end
|
|
460
503
|
|
|
461
504
|
# Runs test files in a fork of the booted parent and returns whether they passed.
|
|
@@ -473,6 +516,7 @@ module Mutineer
|
|
|
473
516
|
rd.close
|
|
474
517
|
Process.setpgid(0, 0) rescue nil # rubocop:disable Style/RescueModifier
|
|
475
518
|
begin
|
|
519
|
+
ChildStdout.silence
|
|
476
520
|
after_fork&.call
|
|
477
521
|
Coverage.result(clear: true, stop: false) if Coverage.running?
|
|
478
522
|
wr.write(Marshal.dump(TestRunners.for(@framework).run(abs_tests).zero?))
|
|
@@ -533,12 +577,10 @@ module Mutineer
|
|
|
533
577
|
loads = Array(test_paths).map { |t| "load #{absolute(t).inspect}" }.join("\n")
|
|
534
578
|
<<~RUBY
|
|
535
579
|
require "minitest"
|
|
536
|
-
require "stringio"
|
|
537
580
|
def Minitest.autorun; end
|
|
538
581
|
$LOAD_PATH.unshift(*#{abs_load_paths.inspect})
|
|
539
582
|
#{abs_source_paths.inspect}.each { |f| load f }
|
|
540
583
|
#{loads}
|
|
541
|
-
$stdout = StringIO.new
|
|
542
584
|
exit(Minitest.run([]) ? 0 : 1)
|
|
543
585
|
RUBY
|
|
544
586
|
end
|
|
@@ -561,7 +603,6 @@ module Mutineer
|
|
|
561
603
|
$LOAD_PATH.unshift(*#{abs_load_paths.inspect})
|
|
562
604
|
#{abs_source_paths.inspect}.each { |f| load f }
|
|
563
605
|
_sink = StringIO.new
|
|
564
|
-
$stdout = _sink
|
|
565
606
|
status = RSpec::Core::Runner.run(["--no-color", #{specs}], _sink, _sink)
|
|
566
607
|
exit(status.zero? ? 0 : 1)
|
|
567
608
|
RUBY
|
|
@@ -583,31 +624,31 @@ module Mutineer
|
|
|
583
624
|
# @return [String] Ruby script text.
|
|
584
625
|
def minitest_subprocess_script(test_path)
|
|
585
626
|
<<~RUBY
|
|
627
|
+
#{result_channel_expression}
|
|
586
628
|
require "coverage"
|
|
587
629
|
require "json"
|
|
588
|
-
require "stringio"
|
|
589
630
|
require "minitest"
|
|
590
631
|
def Minitest.autorun; end
|
|
591
632
|
Coverage.start(lines: true)
|
|
592
633
|
$LOAD_PATH.unshift(*#{abs_load_paths.inspect})
|
|
593
634
|
#{abs_source_paths.inspect}.each { |f| load f }
|
|
594
635
|
load #{absolute(test_path).inspect}
|
|
595
|
-
_orig = $stdout
|
|
596
|
-
$stdout = StringIO.new
|
|
597
636
|
_passed = Minitest.run([])
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
637
|
+
_result.puts JSON.generate("passed" => _passed == true, "coverage" => Coverage.result,
|
|
638
|
+
"loaded_files" => #{loaded_files_expression})
|
|
639
|
+
_result.close
|
|
601
640
|
RUBY
|
|
602
641
|
end
|
|
603
642
|
|
|
604
643
|
# Same coverage-JSON contract as the minitest path, but driven by RSpec:
|
|
605
644
|
# require rspec/core lazily, load the sources under Coverage, then run the
|
|
606
|
-
# one spec via RSpec::Core::Runner
|
|
607
|
-
#
|
|
608
|
-
# non-zero -> capture() records a skipped (incomplete-map)
|
|
645
|
+
# one spec via RSpec::Core::Runner. The JSON goes to the result channel (see
|
|
646
|
+
# {#spawn_script}), so spec output cannot corrupt it. A missing rspec makes
|
|
647
|
+
# the script exit non-zero -> capture() records a skipped (incomplete-map)
|
|
648
|
+
# test, with a hint.
|
|
609
649
|
def rspec_subprocess_script(test_path)
|
|
610
650
|
<<~RUBY
|
|
651
|
+
#{result_channel_expression}
|
|
611
652
|
require "coverage"
|
|
612
653
|
require "json"
|
|
613
654
|
require "stringio"
|
|
@@ -621,13 +662,11 @@ module Mutineer
|
|
|
621
662
|
Coverage.start(lines: true)
|
|
622
663
|
$LOAD_PATH.unshift(*#{abs_load_paths.inspect})
|
|
623
664
|
#{abs_source_paths.inspect}.each { |f| load f }
|
|
624
|
-
_orig = $stdout
|
|
625
665
|
_sink = StringIO.new
|
|
626
|
-
$stdout = _sink
|
|
627
666
|
_status = RSpec::Core::Runner.run(["--no-color", #{absolute(test_path).inspect}], _sink, _sink)
|
|
628
|
-
|
|
629
|
-
|
|
630
|
-
|
|
667
|
+
_result.puts JSON.generate("passed" => _status.zero?, "coverage" => Coverage.result,
|
|
668
|
+
"loaded_files" => #{loaded_files_expression})
|
|
669
|
+
_result.close
|
|
631
670
|
RUBY
|
|
632
671
|
end
|
|
633
672
|
|
|
@@ -56,7 +56,7 @@ module Mutineer
|
|
|
56
56
|
# this mutant. Never a wrong verdict, never a wedged run.
|
|
57
57
|
#
|
|
58
58
|
# @param id [Integer] request id (echoed back for ordering safety).
|
|
59
|
-
# @param payload [Hash]
|
|
59
|
+
# @param payload [Hash] mutated ruby under the "code" key, path under "source_file".
|
|
60
60
|
# @param tests [Array<String>] covering test file paths.
|
|
61
61
|
# @param timeout [Numeric] per-mutant wall-clock timeout (seconds).
|
|
62
62
|
# @param worker [Integer] worker slot; the daemon routes the fork to
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
require "json"
|
|
4
4
|
require "tempfile"
|
|
5
|
+
require_relative "child_stdout"
|
|
5
6
|
|
|
6
7
|
module Mutineer
|
|
7
8
|
# App-side daemon (persistent worker).
|
|
@@ -177,9 +178,9 @@ module Mutineer
|
|
|
177
178
|
# New process group so a per-fork timeout can SIGKILL the whole subtree,
|
|
178
179
|
# and silence the child's stdout so test output never corrupts the IPC pipe.
|
|
179
180
|
Process.setpgid(0, 0) rescue nil # rubocop:disable Style/RescueModifier
|
|
180
|
-
$stdout.reopen(File::NULL, "w")
|
|
181
181
|
code =
|
|
182
182
|
begin
|
|
183
|
+
ChildStdout.silence
|
|
183
184
|
# Route THIS fork at its own worker database before any test loads.
|
|
184
185
|
# A routing failure raises here and is scored `error`, never a false verdict.
|
|
185
186
|
@worker_db&.after_fork(worker, schema_for_fork)
|
data/lib/mutineer/file_swap.rb
CHANGED
|
@@ -7,6 +7,8 @@ module Mutineer
|
|
|
7
7
|
# Raised when another process already holds exclusive ownership of a source
|
|
8
8
|
# file. Aborting beats silently restoring (or capturing) the other run's mutant.
|
|
9
9
|
class ConcurrentRunError < StandardError
|
|
10
|
+
# @param path [String] the source file the other run already owns.
|
|
11
|
+
# @return [ConcurrentRunError]
|
|
10
12
|
def initialize(path)
|
|
11
13
|
super("another mutineer run owns #{path} — aborting to avoid corrupting the source file.")
|
|
12
14
|
end
|
data/lib/mutineer/isolation.rb
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "tempfile"
|
|
4
4
|
require_relative "result"
|
|
5
5
|
require_relative "parser"
|
|
6
|
+
require_relative "child_stdout"
|
|
6
7
|
|
|
7
8
|
module Mutineer
|
|
8
9
|
# Fork-based isolation for running one mutant. The block runs in a child
|
|
@@ -26,6 +27,9 @@ module Mutineer
|
|
|
26
27
|
# exit code) or any explicit `exit` is honoured; an unhandled exception
|
|
27
28
|
# becomes exit 2 with the cause written to STDERR.
|
|
28
29
|
#
|
|
30
|
+
# The child silences its stdout (see {ChildStdout.silence}) before the
|
|
31
|
+
# block runs, so test output never reaches the user. Stderr stays open.
|
|
32
|
+
#
|
|
29
33
|
# @param timeout [Integer] timeout in seconds.
|
|
30
34
|
# @yieldreturn [Integer] child exit status.
|
|
31
35
|
# @return [Mutineer::Result] result from the child process.
|
|
@@ -36,15 +40,17 @@ module Mutineer
|
|
|
36
40
|
Process.setpgid(0, 0) rescue nil # rubocop:disable Style/RescueModifier
|
|
37
41
|
code = 0
|
|
38
42
|
begin
|
|
43
|
+
ChildStdout.silence
|
|
39
44
|
result = yield
|
|
40
45
|
code = result.is_a?(Integer) ? result : 0
|
|
41
46
|
rescue SystemExit => e
|
|
42
47
|
code = e.status
|
|
43
48
|
rescue Exception => e # rubocop:disable Lint/RescueException
|
|
44
|
-
warn
|
|
49
|
+
# STDERR, not `warn`: a test may have left `$stderr` as a StringIO.
|
|
50
|
+
STDERR.puts "[mutineer-child] #{e.class}: #{e.message}"
|
|
45
51
|
code = 2
|
|
46
52
|
end
|
|
47
|
-
|
|
53
|
+
STDERR.flush
|
|
48
54
|
# exit! skips at_exit handlers — critical, since a child forked from
|
|
49
55
|
# inside our own Minitest suite would otherwise re-run the parent's
|
|
50
56
|
# at_exit autorun hook on the way out.
|