mutineer 0.2.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +53 -0
- data/README.md +6 -4
- data/lib/mutineer/changed_lines.rb +56 -0
- data/lib/mutineer/cli.rb +44 -1
- data/lib/mutineer/config.rb +16 -2
- data/lib/mutineer/coverage_map.rb +136 -14
- data/lib/mutineer/isolation.rb +32 -24
- data/lib/mutineer/minitest_integration.rb +11 -1
- data/lib/mutineer/project.rb +15 -1
- data/lib/mutineer/runner.rb +48 -21
- data/lib/mutineer/test_runners/minitest.rb +15 -0
- data/lib/mutineer/test_runners/rspec.rb +60 -0
- data/lib/mutineer/test_runners.rb +19 -0
- data/lib/mutineer/version.rb +1 -1
- data/lib/mutineer/worker_pool.rb +31 -26
- data/lib/mutineer.rb +2 -0
- metadata +5 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 88e7dbea9ba80b3fed23d634e30830372c89e2f42cf7403d8fa5860d80ba17dd
|
|
4
|
+
data.tar.gz: ac519c7e8c6e301e46242af5bd6a94672aa0a95f2d8dc46dfbd5e3b2611c366c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5e49d3e6fef7dddc74b93c6e1216a5740e556f0c9b1ba1d684cc28b61c3f464063669ad04468dc0d0882a1a141050d62c96e6f9995b8b0e541f38e94d9fbac7a
|
|
7
|
+
data.tar.gz: d70b8231caef83f5b55a9ee840cb77cdf16b8c93c8f227d7f278266db44584577eb17ca589eba677d3896dceae03b8e71a43f8b7747f7046060aa659d9b55d58
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,54 @@ All notable changes to this project are documented here. The format is based on
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/), and this project adheres to
|
|
5
5
|
[Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [0.6.1] - 2026-06-29
|
|
8
|
+
|
|
9
|
+
### Changed
|
|
10
|
+
- **Removed `eval` entirely** — the redefine strategy now `load`s the wrapped
|
|
11
|
+
method snippet from a tempfile instead of evaluating a string. Behavior is
|
|
12
|
+
identical (top-level load rebuilds the same `Module.nesting`), but the gem no
|
|
13
|
+
longer uses dynamic string execution, clearing supply-chain scanner flags.
|
|
14
|
+
Zero runtime dependencies unchanged.
|
|
15
|
+
|
|
16
|
+
## [0.6.0] - 2026-06-28
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
- **RSpec support** (#6) — `--framework rspec` (or auto-detected when most
|
|
20
|
+
`--test` files end in `_spec.rb`) runs RSpec suites instead of Minitest, via a
|
|
21
|
+
pluggable test-runner abstraction. Both frameworks are loaded lazily, so
|
|
22
|
+
Mutineer keeps zero runtime gem dependencies and works in an rspec-only
|
|
23
|
+
project; coverage selection works for both. `.mutineer.yml` accepts `framework:`.
|
|
24
|
+
|
|
25
|
+
### Fixed
|
|
26
|
+
- **Redefine strategy keeps compact `class A::B` as a single nesting wrapper**
|
|
27
|
+
(#5) — avoids a constant-resolution disagreement with the reload strategy.
|
|
28
|
+
|
|
29
|
+
## [0.5.0] - 2026-06-28
|
|
30
|
+
|
|
31
|
+
### Fixed
|
|
32
|
+
- **`class << self` methods are now discovered and mutated** (#3) — previously
|
|
33
|
+
they were treated as instance methods, so the redefine strategy mis-targeted
|
|
34
|
+
them. `class << other_obj` blocks are skipped (not representable).
|
|
35
|
+
- **Worker pool no longer deadlocks on large results** (#4) — pipes are drained
|
|
36
|
+
with `IO.select` and children reaped on EOF, so a result bigger than the OS
|
|
37
|
+
pipe buffer (~64KB) can't wedge the run.
|
|
38
|
+
|
|
39
|
+
## [0.4.0] - 2026-06-28
|
|
40
|
+
|
|
41
|
+
### Added
|
|
42
|
+
- **`--since <git-ref>`** (#2) — mutate only the lines changed since a git ref
|
|
43
|
+
(e.g. `--since origin/main`), so CI on a pull request mutation-tests just the
|
|
44
|
+
new/changed code. Composes with coverage selection; `--dry-run --since`
|
|
45
|
+
narrows the preview too. Unknown ref / not-a-git-repo exits 2.
|
|
46
|
+
|
|
47
|
+
## [0.3.0] - 2026-06-28
|
|
48
|
+
|
|
49
|
+
### Added
|
|
50
|
+
- **Coverage-guided test selection in boot mode** (#1) — `--rails`/`--boot` now
|
|
51
|
+
captures coverage by forking the booted app and runs only the test files that
|
|
52
|
+
cover each mutant's line (uncovered lines report `no_coverage`), instead of
|
|
53
|
+
running every `--test` file for every mutant. Cached like standalone mode.
|
|
54
|
+
|
|
7
55
|
## [0.2.0] - 2026-06-28
|
|
8
56
|
|
|
9
57
|
### Added
|
|
@@ -38,5 +86,10 @@ All notable changes to this project are documented here. The format is based on
|
|
|
38
86
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
39
87
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
40
88
|
|
|
89
|
+
[0.6.1]: https://github.com/davidteren/mutineer/releases/tag/v0.6.1
|
|
90
|
+
[0.6.0]: https://github.com/davidteren/mutineer/releases/tag/v0.6.0
|
|
91
|
+
[0.5.0]: https://github.com/davidteren/mutineer/releases/tag/v0.5.0
|
|
92
|
+
[0.4.0]: https://github.com/davidteren/mutineer/releases/tag/v0.4.0
|
|
93
|
+
[0.3.0]: https://github.com/davidteren/mutineer/releases/tag/v0.3.0
|
|
41
94
|
[0.2.0]: https://github.com/davidteren/mutineer/releases/tag/v0.2.0
|
|
42
95
|
[0.1.0]: https://github.com/davidteren/mutineer/releases/tag/v0.1.0
|
data/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Mutineer
|
|
2
2
|
|
|
3
3
|
A clean-room mutation-testing tool for Ruby. Mutineer mutates your source one
|
|
4
|
-
change at a time, runs your
|
|
4
|
+
change at a time, runs your test suite (Minitest or RSpec) against each mutant, and reports the
|
|
5
5
|
ones your tests failed to catch — the gaps where your suite isn't actually
|
|
6
6
|
testing anything.
|
|
7
7
|
|
|
@@ -45,6 +45,8 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
|
|
|
45
45
|
| `--operators LIST` | Comma-separated operator names (default: the Tier-1 set) |
|
|
46
46
|
| `--threshold FLOAT` | Exit 1 when the score is below FLOAT (default: 0 = off) |
|
|
47
47
|
| `--only NAME` | Restrict to one fully-qualified subject, e.g. `Calculator#add` |
|
|
48
|
+
| `--framework NAME` | `minitest` (default) or `rspec`; auto-detected as rspec when most `--test` files end in `_spec.rb` |
|
|
49
|
+
| `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`) — ideal for PR CI |
|
|
48
50
|
| `--jobs N` | Parallel worker count (default: processor count) |
|
|
49
51
|
| `--strategy NAME` | Mutation application: `reload` whole-file (default) or `redefine` surgical (`7a`/`7b` accepted as deprecated aliases) |
|
|
50
52
|
| `--format human\|json` | Report format (default: human) |
|
|
@@ -82,9 +84,9 @@ RAILS_ENV=test bundle exec mutineer run \
|
|
|
82
84
|
then forks and inherits it), defaults `--strategy` to `redefine` (surgical — it
|
|
83
85
|
avoids reloading files into the app tree), and reconnects ActiveRecord in each
|
|
84
86
|
fork so the database connection is fork-safe. Use `--boot FILE` to boot a
|
|
85
|
-
different entry point. Boot mode requires at least one `--test` file and
|
|
86
|
-
|
|
87
|
-
|
|
87
|
+
different entry point. Boot mode requires at least one `--test` file and is
|
|
88
|
+
coverage-guided — each mutant runs only the test files that exercise its line
|
|
89
|
+
(coverage is captured by forking the booted app, then cached).
|
|
88
90
|
|
|
89
91
|
Add Mutineer to your Gemfile's test group:
|
|
90
92
|
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "set"
|
|
4
|
+
require "open3"
|
|
5
|
+
|
|
6
|
+
module Mutineer
|
|
7
|
+
# Maps each source file to the set of NEW-side line numbers that changed since a
|
|
8
|
+
# given git ref, by parsing `git diff --unified=0`. Used to restrict mutations
|
|
9
|
+
# to only the diff (issue #2): on a PR you care whether the changed code is
|
|
10
|
+
# tested, so mutating just those lines is fast and actionable.
|
|
11
|
+
#
|
|
12
|
+
# git is an external tool, not a gem dependency — shelling out is fine. The pure
|
|
13
|
+
# `parse` carries the logic; `git_diff` is injectable so it stays testable
|
|
14
|
+
# without invoking git.
|
|
15
|
+
module ChangedLines
|
|
16
|
+
HUNK = /^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@/
|
|
17
|
+
|
|
18
|
+
module_function
|
|
19
|
+
|
|
20
|
+
# Parse `git diff --unified=0` output into the set of NEW-side line numbers
|
|
21
|
+
# added/modified. With --unified=0 each hunk's `+c,d` block is exactly the
|
|
22
|
+
# changed lines: c..c+d-1. `d` absent means 1 line; `d == 0` is a pure
|
|
23
|
+
# deletion and contributes nothing.
|
|
24
|
+
def parse(diff_text)
|
|
25
|
+
lines = Set.new
|
|
26
|
+
diff_text.each_line do |row|
|
|
27
|
+
m = HUNK.match(row) or next
|
|
28
|
+
start = m[1].to_i
|
|
29
|
+
count = m[2].nil? ? 1 : m[2].to_i
|
|
30
|
+
next if count.zero?
|
|
31
|
+
|
|
32
|
+
lines.merge(start...(start + count))
|
|
33
|
+
end
|
|
34
|
+
lines
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# { absolute_file_path => Set<Integer> } of changed new-side lines per file.
|
|
38
|
+
# `runner` is injected so tests can supply canned diff text per file.
|
|
39
|
+
def for(ref:, files:, project_root:, runner: method(:git_diff))
|
|
40
|
+
files.each_with_object({}) do |file, acc|
|
|
41
|
+
abs = File.expand_path(file, project_root)
|
|
42
|
+
acc[abs] = parse(runner.call(ref, abs, project_root))
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# stdout of `git -C <root> diff --unified=0 <ref> -- <file>`, "" on failure.
|
|
47
|
+
def git_diff(ref, abs_file, project_root)
|
|
48
|
+
out, _err, status = Open3.capture3(
|
|
49
|
+
"git", "-C", project_root, "diff", "--unified=0", ref, "--", abs_file
|
|
50
|
+
)
|
|
51
|
+
status.success? ? out : ""
|
|
52
|
+
rescue StandardError
|
|
53
|
+
""
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
data/lib/mutineer/cli.rb
CHANGED
|
@@ -2,10 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
require "optparse"
|
|
4
4
|
require "set"
|
|
5
|
+
require "open3"
|
|
5
6
|
require_relative "version"
|
|
6
7
|
require_relative "config"
|
|
7
8
|
require_relative "parser"
|
|
8
9
|
require_relative "project"
|
|
10
|
+
require_relative "changed_lines"
|
|
9
11
|
require_relative "runner"
|
|
10
12
|
require_relative "reporter"
|
|
11
13
|
require_relative "mutator_registry"
|
|
@@ -32,10 +34,12 @@ module Mutineer
|
|
|
32
34
|
--operators LIST Comma-separated operator names (default: Tier 1 set)
|
|
33
35
|
--threshold FLOAT Fail (exit 1) when score < FLOAT (default: 0 = off)
|
|
34
36
|
--only NAME Restrict to one fully-qualified subject
|
|
37
|
+
--since REF Only mutate lines changed since git REF (e.g. origin/main)
|
|
35
38
|
--jobs N Parallel worker count (default: processor count)
|
|
36
39
|
--strategy NAME reload (whole-file) or redefine (surgical); default: reload
|
|
40
|
+
--framework NAME minitest or rspec (default: auto-detect from --test names)
|
|
37
41
|
--boot FILE Require FILE once in the parent to boot the app env, then
|
|
38
|
-
fork per mutant (Rails apps; requires --test
|
|
42
|
+
fork per mutant (Rails apps; requires --test)
|
|
39
43
|
--rails Sugar for --boot config/environment --strategy redefine
|
|
40
44
|
--format human|json Report format (default: human)
|
|
41
45
|
--output FILE Write the report to FILE instead of stdout
|
|
@@ -71,11 +75,13 @@ module Mutineer
|
|
|
71
75
|
o.on("--list-operators") { show_operators = true }
|
|
72
76
|
o.on("--dry-run") { opts[:dry_run] = true }
|
|
73
77
|
o.on("--only NAME") { |v| opts[:only] = v; explicit << :only }
|
|
78
|
+
o.on("--since REF") { |v| opts[:since] = v; explicit << :since }
|
|
74
79
|
o.on("--test FILE") { |v| (opts[:tests] ||= []) << v }
|
|
75
80
|
o.on("--operators LIST") { |v| opts[:operators] = v.split(",").map(&:strip); explicit << :operators }
|
|
76
81
|
o.on("--threshold FLOAT") { |v| opts[:threshold] = v.to_f; explicit << :threshold }
|
|
77
82
|
o.on("--jobs N") { |v| opts[:jobs] = v; explicit << :jobs }
|
|
78
83
|
o.on("--strategy STRAT") { |v| opts[:strategy] = v; explicit << :strategy }
|
|
84
|
+
o.on("--framework NAME") { |v| opts[:framework] = v; explicit << :framework }
|
|
79
85
|
o.on("--boot FILE") { |v| opts[:boot] = v; explicit << :boot }
|
|
80
86
|
o.on("--rails") { opts[:rails] = true }
|
|
81
87
|
o.on("--format FORMAT") { |v| opts[:format] = v }
|
|
@@ -183,6 +189,11 @@ module Mutineer
|
|
|
183
189
|
exit 2
|
|
184
190
|
end
|
|
185
191
|
|
|
192
|
+
unless %w[minitest rspec].include?(config.framework)
|
|
193
|
+
warn %(mutineer: unknown framework "#{config.framework}". Expected: minitest, rspec)
|
|
194
|
+
exit 2
|
|
195
|
+
end
|
|
196
|
+
|
|
186
197
|
# Boot mode does no coverage selection — every mutant runs the given tests —
|
|
187
198
|
# so at least one --test file is mandatory (there is nothing to select from).
|
|
188
199
|
if config.boot && config.tests.empty?
|
|
@@ -190,10 +201,31 @@ module Mutineer
|
|
|
190
201
|
exit 2
|
|
191
202
|
end
|
|
192
203
|
|
|
204
|
+
validate_since!(config) if config.since
|
|
193
205
|
preflight_output!(config.output) if config.output
|
|
194
206
|
validate_paths!(config)
|
|
195
207
|
end
|
|
196
208
|
|
|
209
|
+
# --since needs a real git repo and a resolvable ref; either failure is a
|
|
210
|
+
# usage error (exit 2) so CI sees "bad invocation," not "tests too weak."
|
|
211
|
+
def self.validate_since!(config)
|
|
212
|
+
_out, _err, status = Open3.capture3(
|
|
213
|
+
"git", "-C", config.project_root, "rev-parse", "--verify", "--quiet",
|
|
214
|
+
"#{config.since}^{commit}"
|
|
215
|
+
)
|
|
216
|
+
return if status.success?
|
|
217
|
+
|
|
218
|
+
inside, = Open3.capture3(
|
|
219
|
+
"git", "-C", config.project_root, "rev-parse", "--is-inside-work-tree"
|
|
220
|
+
)
|
|
221
|
+
msg = inside.strip == "true" ? "unknown git ref: #{config.since}" : "--since requires a git repository"
|
|
222
|
+
warn "mutineer: #{msg}"
|
|
223
|
+
exit 2
|
|
224
|
+
rescue Errno::ENOENT
|
|
225
|
+
warn "mutineer: --since requires git on PATH"
|
|
226
|
+
exit 2
|
|
227
|
+
end
|
|
228
|
+
|
|
197
229
|
# R5: validate path existence up front so a typo is a clean usage error (exit
|
|
198
230
|
# 2), not an Errno::ENOENT backtrace from deep in the run. Flag checks run
|
|
199
231
|
# first so a bad flag still reports the flag, not the missing file.
|
|
@@ -234,6 +266,13 @@ module Mutineer
|
|
|
234
266
|
per_operator = Hash.new(0)
|
|
235
267
|
skipped = 0
|
|
236
268
|
|
|
269
|
+
# --since narrows the preview to changed lines too, so `--dry-run --since`
|
|
270
|
+
# shows exactly what a real `--since` run would mutate.
|
|
271
|
+
changed = if config.since
|
|
272
|
+
ChangedLines.for(ref: config.since, files: config.sources,
|
|
273
|
+
project_root: config.project_root)
|
|
274
|
+
end
|
|
275
|
+
|
|
237
276
|
Project.discover(config.sources, only: config.only).each do |subject|
|
|
238
277
|
source = (sources[subject.file] ||= Parser.parse_file(subject.file).source.source)
|
|
239
278
|
operator_classes.each do |klass|
|
|
@@ -242,6 +281,10 @@ module Mutineer
|
|
|
242
281
|
skipped += 1
|
|
243
282
|
next
|
|
244
283
|
end
|
|
284
|
+
if changed
|
|
285
|
+
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
|
286
|
+
next unless changed[File.expand_path(subject.file, config.project_root)]&.include?(line)
|
|
287
|
+
end
|
|
245
288
|
per_operator[mutation.operator] += 1
|
|
246
289
|
original = source.byteslice(mutation.start_offset...mutation.end_offset)
|
|
247
290
|
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
data/lib/mutineer/config.rb
CHANGED
|
@@ -24,12 +24,12 @@ module Mutineer
|
|
|
24
24
|
:sources, :tests, :operators, :threshold, :only, :dry_run,
|
|
25
25
|
:cache_dir, :project_root, :load_paths,
|
|
26
26
|
:jobs, :format, :output, :strategy, :require_paths,
|
|
27
|
-
:boot, :rails,
|
|
27
|
+
:boot, :rails, :since, :framework,
|
|
28
28
|
keyword_init: true
|
|
29
29
|
) do
|
|
30
30
|
CONFIG_FILE = ".mutineer.yml"
|
|
31
31
|
# Keys accepted in .mutineer.yml (R7). `require` maps to the :require_paths field.
|
|
32
|
-
KNOWN_KEYS = %w[operators jobs threshold only require boot rails].freeze
|
|
32
|
+
KNOWN_KEYS = %w[operators jobs threshold only require boot rails since framework].freeze
|
|
33
33
|
|
|
34
34
|
def initialize(**kwargs)
|
|
35
35
|
super
|
|
@@ -109,9 +109,22 @@ module Mutineer
|
|
|
109
109
|
config.boot ||= "config/environment"
|
|
110
110
|
config.strategy = "redefine" unless explicit.include?(:strategy)
|
|
111
111
|
end
|
|
112
|
+
|
|
113
|
+
# Auto-detect the framework only when neither CLI nor config file set it
|
|
114
|
+
# (explicit value, from either source, is already on config.framework and
|
|
115
|
+
# always wins). Default minitest unless the test files clearly look RSpec.
|
|
116
|
+
config.framework ||= detect_framework(config.tests)
|
|
112
117
|
config
|
|
113
118
|
end
|
|
114
119
|
|
|
120
|
+
# Pick rspec when a MAJORITY of the given test files end with _spec.rb;
|
|
121
|
+
# otherwise minitest. Empty/ambiguous -> minitest (the safe default).
|
|
122
|
+
def self.detect_framework(tests)
|
|
123
|
+
tests = Array(tests)
|
|
124
|
+
specs = tests.count { |t| t.to_s.end_with?("_spec.rb") }
|
|
125
|
+
specs > tests.length / 2.0 ? "rspec" : "minitest"
|
|
126
|
+
end
|
|
127
|
+
|
|
115
128
|
def self.field_for(known_key)
|
|
116
129
|
known_key == "require" ? :require_paths : known_key.to_sym
|
|
117
130
|
end
|
|
@@ -123,6 +136,7 @@ module Mutineer
|
|
|
123
136
|
when "threshold" then value.to_f
|
|
124
137
|
when "require" then Array(value).map(&:to_s)
|
|
125
138
|
when "boot" then value.to_s
|
|
139
|
+
when "framework" then value.to_s
|
|
126
140
|
when "rails" then value == true || value.to_s == "true"
|
|
127
141
|
else value
|
|
128
142
|
end
|
|
@@ -5,6 +5,9 @@ require "json"
|
|
|
5
5
|
require "digest"
|
|
6
6
|
require "fileutils"
|
|
7
7
|
require "rbconfig"
|
|
8
|
+
require "coverage"
|
|
9
|
+
require_relative "minitest_integration"
|
|
10
|
+
require_relative "test_runners"
|
|
8
11
|
|
|
9
12
|
module Mutineer
|
|
10
13
|
# Maps `(source_file, line) -> [test_files]` so each mutant runs only against
|
|
@@ -21,24 +24,52 @@ module Mutineer
|
|
|
21
24
|
|
|
22
25
|
def initialize(source_paths:, test_paths:, cache_dir: ".mutineer",
|
|
23
26
|
load_paths: ["lib"], project_root: Dir.pwd,
|
|
24
|
-
capture_timeout: DEFAULT_CAPTURE_TIMEOUT
|
|
27
|
+
capture_timeout: DEFAULT_CAPTURE_TIMEOUT, boot_path: nil,
|
|
28
|
+
framework: "minitest")
|
|
25
29
|
@source_paths = Array(source_paths)
|
|
26
30
|
@test_paths = Array(test_paths)
|
|
27
31
|
@cache_dir = cache_dir
|
|
28
32
|
@load_paths = Array(load_paths)
|
|
29
33
|
@project_root = project_root
|
|
30
34
|
@capture_timeout = capture_timeout
|
|
35
|
+
@boot_path = boot_path
|
|
36
|
+
@framework = framework || "minitest"
|
|
31
37
|
@map = {}
|
|
32
38
|
@failed_test_files = []
|
|
33
39
|
@phase_a_ran = false
|
|
34
40
|
end
|
|
35
41
|
|
|
36
|
-
# Phase A entry point: load the cached map when the content
|
|
37
|
-
# otherwise rebuild from subprocesses and overwrite the cache.
|
|
42
|
+
# Phase A entry point (standalone): load the cached map when the content
|
|
43
|
+
# digest matches, otherwise rebuild from subprocesses and overwrite the cache.
|
|
38
44
|
def build_or_load
|
|
39
45
|
warn_external_sources
|
|
40
|
-
|
|
46
|
+
cached_or { run_phase_a }
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Boot-mode Phase A: Coverage is already running in the parent (started before
|
|
50
|
+
# the app booted, so booted source lines are instrumented). A clean `ruby`
|
|
51
|
+
# subprocess has no booted env, so per-test coverage is captured by FORKING
|
|
52
|
+
# the booted parent instead. Inverts into the same map #tests_for reads, and
|
|
53
|
+
# reuses the digest cache (the digest mixes in the boot file so a boot cache
|
|
54
|
+
# never collides with a standalone one).
|
|
55
|
+
def build_via_fork(rails: false)
|
|
56
|
+
warn_external_sources
|
|
57
|
+
cached_or { run_phase_a_via_fork(rails: rails) }
|
|
58
|
+
end
|
|
41
59
|
|
|
60
|
+
# Phase B lookup: the test files that cover `file:line`, or [] when none do.
|
|
61
|
+
# ponytail: per-file granularity; upgrade to per-method when throughput
|
|
62
|
+
# warrants (requires Minitest method isolation + finer Coverage tracking).
|
|
63
|
+
def tests_for(file, line)
|
|
64
|
+
@map["#{relativize(file)}:#{line}"] || []
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
private
|
|
68
|
+
|
|
69
|
+
# Shared cache dance for both build paths: hit the digest-keyed cache, else
|
|
70
|
+
# yield to populate @map and persist it.
|
|
71
|
+
def cached_or
|
|
72
|
+
@digest = compute_digest
|
|
42
73
|
cached = read_cache
|
|
43
74
|
if cached && cached["digest"] == @digest
|
|
44
75
|
@map = cached["map"] || {}
|
|
@@ -47,20 +78,11 @@ module Mutineer
|
|
|
47
78
|
return self
|
|
48
79
|
end
|
|
49
80
|
|
|
50
|
-
|
|
81
|
+
yield
|
|
51
82
|
save
|
|
52
83
|
self
|
|
53
84
|
end
|
|
54
85
|
|
|
55
|
-
# Phase B lookup: the test files that cover `file:line`, or [] when none do.
|
|
56
|
-
# ponytail: per-file granularity; upgrade to per-method when throughput
|
|
57
|
-
# warrants (requires Minitest method isolation + finer Coverage tracking).
|
|
58
|
-
def tests_for(file, line)
|
|
59
|
-
@map["#{relativize(file)}:#{line}"] || []
|
|
60
|
-
end
|
|
61
|
-
|
|
62
|
-
private
|
|
63
|
-
|
|
64
86
|
def run_phase_a
|
|
65
87
|
@phase_a_ran = true
|
|
66
88
|
@map = {}
|
|
@@ -74,6 +96,65 @@ module Mutineer
|
|
|
74
96
|
end
|
|
75
97
|
end
|
|
76
98
|
|
|
99
|
+
# Boot-mode Phase A. For each test file, fork the booted parent; the child
|
|
100
|
+
# resets its Coverage delta, runs that ONE test, and marshals back the raw
|
|
101
|
+
# per-source coverage counts. record() inverts them exactly as the subprocess
|
|
102
|
+
# path does. ponytail: serial fork (one test at a time) — boot apps fork
|
|
103
|
+
# cheaply via COW and per-test isolation matters more than throughput here.
|
|
104
|
+
def run_phase_a_via_fork(rails:)
|
|
105
|
+
@phase_a_ran = true
|
|
106
|
+
@map = {}
|
|
107
|
+
@failed_test_files = []
|
|
108
|
+
abs_sources = abs_source_paths
|
|
109
|
+
|
|
110
|
+
@test_paths.each do |test_path|
|
|
111
|
+
coverage = fork_capture(absolute(test_path), abs_sources, rails)
|
|
112
|
+
next fail_test(test_path, "fork capture produced no result") unless coverage
|
|
113
|
+
|
|
114
|
+
record(coverage, test_path)
|
|
115
|
+
end
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
# Fork the booted parent, run one test under the inherited Coverage, and
|
|
119
|
+
# return its per-source counts hash (or nil on failure). Reuses the same
|
|
120
|
+
# fork + Marshal-over-pipe + hard-exit! discipline as WorkerPool/Isolation.
|
|
121
|
+
def fork_capture(abs_test, abs_sources, rails)
|
|
122
|
+
rd, wr = IO.pipe
|
|
123
|
+
pid = fork do
|
|
124
|
+
rd.close
|
|
125
|
+
payload =
|
|
126
|
+
begin
|
|
127
|
+
Runner.send(:reconnect_active_record) if rails
|
|
128
|
+
Coverage.result(clear: true, stop: false) # discard pre-test delta
|
|
129
|
+
TestRunners.for(@framework).run([abs_test])
|
|
130
|
+
# lines:true yields {file => {lines: [...]}}; reduce to the counts
|
|
131
|
+
# array record() expects, keeping only our source files.
|
|
132
|
+
Coverage.result(stop: false)
|
|
133
|
+
.select { |f, _| abs_sources.include?(f) }
|
|
134
|
+
.transform_values { |v| v.is_a?(Hash) ? v[:lines] : v }
|
|
135
|
+
rescue Exception # rubocop:disable Lint/RescueException
|
|
136
|
+
nil
|
|
137
|
+
end
|
|
138
|
+
begin
|
|
139
|
+
wr.write(Marshal.dump(payload))
|
|
140
|
+
rescue StandardError # rubocop:disable Lint/SuppressedException
|
|
141
|
+
# pipe gone; parent records "no result"
|
|
142
|
+
ensure
|
|
143
|
+
wr.close
|
|
144
|
+
exit!(0) # skip at_exit so the parent suite's autorun never re-fires here
|
|
145
|
+
end
|
|
146
|
+
end
|
|
147
|
+
wr.close
|
|
148
|
+
data = rd.read
|
|
149
|
+
rd.close
|
|
150
|
+
Process.waitpid(pid)
|
|
151
|
+
return nil if data.empty?
|
|
152
|
+
|
|
153
|
+
Marshal.load(data)
|
|
154
|
+
rescue StandardError
|
|
155
|
+
nil
|
|
156
|
+
end
|
|
157
|
+
|
|
77
158
|
# Spawns a fresh `ruby` reading an inline script from stdin. A fork would
|
|
78
159
|
# miss already-loaded app lines, so Coverage must start in a clean process
|
|
79
160
|
# before any source is loaded (KTD1/KTD2). Returns the parsed Coverage.result
|
|
@@ -110,6 +191,10 @@ module Mutineer
|
|
|
110
191
|
end
|
|
111
192
|
|
|
112
193
|
def subprocess_script(test_path)
|
|
194
|
+
@framework == "rspec" ? rspec_subprocess_script(test_path) : minitest_subprocess_script(test_path)
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
def minitest_subprocess_script(test_path)
|
|
113
198
|
<<~RUBY
|
|
114
199
|
require "coverage"
|
|
115
200
|
require "json"
|
|
@@ -128,6 +213,35 @@ module Mutineer
|
|
|
128
213
|
RUBY
|
|
129
214
|
end
|
|
130
215
|
|
|
216
|
+
# Same coverage-JSON contract as the minitest path, but driven by RSpec:
|
|
217
|
+
# require rspec/core lazily, load the sources under Coverage, then run the
|
|
218
|
+
# one spec via RSpec::Core::Runner with output silenced so only the JSON
|
|
219
|
+
# reaches stdout. A missing rspec makes `require` raise -> subprocess exits
|
|
220
|
+
# non-zero -> capture() records a skipped (incomplete-map) test, with a hint.
|
|
221
|
+
def rspec_subprocess_script(test_path)
|
|
222
|
+
<<~RUBY
|
|
223
|
+
require "coverage"
|
|
224
|
+
require "json"
|
|
225
|
+
require "stringio"
|
|
226
|
+
begin
|
|
227
|
+
require "rspec/core"
|
|
228
|
+
rescue LoadError
|
|
229
|
+
warn "[mutineer] framework 'rspec' requested but rspec is not available in the project"
|
|
230
|
+
exit 3
|
|
231
|
+
end
|
|
232
|
+
RSpec::Core::Runner.disable_autorun!
|
|
233
|
+
Coverage.start(lines: true)
|
|
234
|
+
$LOAD_PATH.unshift(*#{abs_load_paths.inspect})
|
|
235
|
+
#{abs_source_paths.inspect}.each { |f| load f }
|
|
236
|
+
_orig = $stdout
|
|
237
|
+
_sink = StringIO.new
|
|
238
|
+
$stdout = _sink
|
|
239
|
+
RSpec::Core::Runner.run(["--no-color", #{absolute(test_path).inspect}], _sink, _sink)
|
|
240
|
+
$stdout = _orig
|
|
241
|
+
puts Coverage.result.to_json
|
|
242
|
+
RUBY
|
|
243
|
+
end
|
|
244
|
+
|
|
131
245
|
# Records every source line with a non-zero execution count as covered by
|
|
132
246
|
# this test file. Coverage.result keys are absolute; relativize and drop any
|
|
133
247
|
# path outside the project (stdlib/gem files).
|
|
@@ -154,10 +268,18 @@ module Mutineer
|
|
|
154
268
|
d = Digest::SHA256.new
|
|
155
269
|
digest_group(d, "source", @source_paths)
|
|
156
270
|
digest_group(d, "test", @test_paths)
|
|
271
|
+
digest_group(d, "boot", [boot_digest_path]) if @boot_path
|
|
157
272
|
@load_paths.sort.each { |lp| d.update("loadpath\0#{lp}\0") }
|
|
273
|
+
d.update("framework\0#{@framework}\0")
|
|
158
274
|
d.hexdigest
|
|
159
275
|
end
|
|
160
276
|
|
|
277
|
+
# boot_path is a require-style path (e.g. "config/environment", no extension);
|
|
278
|
+
# resolve it to the real file for reading, appending ".rb" when needed.
|
|
279
|
+
def boot_digest_path
|
|
280
|
+
File.exist?(absolute(@boot_path)) ? @boot_path : "#{@boot_path}.rb"
|
|
281
|
+
end
|
|
282
|
+
|
|
161
283
|
def digest_group(digest, role, paths)
|
|
162
284
|
paths.sort.each do |p|
|
|
163
285
|
content = File.read(absolute(p))
|
data/lib/mutineer/isolation.rb
CHANGED
|
@@ -14,10 +14,10 @@ module Mutineer
|
|
|
14
14
|
# by the parent's monitor flag, not by status.signaled? (which is true for ANY
|
|
15
15
|
# signal death, e.g. SIGSEGV — it cannot tell our SIGKILL apart from the OS's).
|
|
16
16
|
#
|
|
17
|
-
# mutineer: the
|
|
17
|
+
# mutineer: the reload strategy this enables (whole-file `load`) re-executes the
|
|
18
18
|
# entire file — any top-level code runs again. Acceptable for POROs; document
|
|
19
|
-
# if users hit issues with initializers/callbacks.
|
|
20
|
-
#
|
|
19
|
+
# if users hit issues with initializers/callbacks. Alternative: the redefine
|
|
20
|
+
# strategy (surgical single-method redefinition).
|
|
21
21
|
class Isolation
|
|
22
22
|
DEFAULT_TIMEOUT = 10 # seconds
|
|
23
23
|
|
|
@@ -78,15 +78,13 @@ module Mutineer
|
|
|
78
78
|
end
|
|
79
79
|
end
|
|
80
80
|
|
|
81
|
-
#
|
|
82
|
-
# snippet,
|
|
83
|
-
#
|
|
81
|
+
# Redefine strategy: extract just the enclosing DefNode, apply the mutation to
|
|
82
|
+
# that snippet, wrap it in its real namespace, and `load` only that one method
|
|
83
|
+
# back into the running process. No file-level side effects re-run. Child-only.
|
|
84
84
|
#
|
|
85
|
-
#
|
|
86
|
-
#
|
|
87
|
-
#
|
|
88
|
-
# through singleton_class.class_eval (R17) would double-wrap and define on the
|
|
89
|
-
# wrong class.
|
|
85
|
+
# The snippet keeps its own `def self.x` for singletons, so the namespace
|
|
86
|
+
# wrapper redefines instance and singleton methods correctly without any
|
|
87
|
+
# special-casing.
|
|
90
88
|
def self.apply_surgical(mutation, subject, source)
|
|
91
89
|
loc = subject.def_node.location
|
|
92
90
|
def_start = loc.start_offset
|
|
@@ -97,9 +95,9 @@ module Mutineer
|
|
|
97
95
|
mutated_def = snippet.byteslice(0...rel_s) + mutation.replacement + snippet.byteslice(rel_e..)
|
|
98
96
|
|
|
99
97
|
# Rebuild the FULL namespace nesting textually so unqualified enclosing-
|
|
100
|
-
# namespace constants resolve exactly as
|
|
101
|
-
#
|
|
102
|
-
# NameError on such constants (C2 scope-collapse).
|
|
98
|
+
# namespace constants resolve exactly as the reload strategy would. A bare
|
|
99
|
+
# redefinition on the owner would collapse Module.nesting to [owner] and
|
|
100
|
+
# raise NameError on such constants (C2 scope-collapse).
|
|
103
101
|
keywords = nesting_keywords(subject.namespace)
|
|
104
102
|
prefix = keywords.map { |kw, name| "#{kw} #{name}" }.join("\n")
|
|
105
103
|
prefix += "\n" unless prefix.empty?
|
|
@@ -116,23 +114,33 @@ module Mutineer
|
|
|
116
114
|
target = subject.singleton ? owner.singleton_class : owner
|
|
117
115
|
vis = method_visibility(target, subject.name)
|
|
118
116
|
|
|
119
|
-
#
|
|
120
|
-
#
|
|
121
|
-
#
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
117
|
+
# Write the wrapped snippet to a tempfile and `load` it: `load` runs it at
|
|
118
|
+
# top level, so the textual class/module wrappers rebuild Module.nesting
|
|
119
|
+
# identically, with no dynamic string execution for scanners to flag. The
|
|
120
|
+
# input is the project's OWN source (the enclosing method, textually
|
|
121
|
+
# mutated), loaded only in this forked child.
|
|
122
|
+
Tempfile.create(["mutineer_surgical", ".rb"]) do |f|
|
|
123
|
+
f.write(wrapped)
|
|
124
|
+
f.flush
|
|
125
|
+
load f.path
|
|
126
|
+
end
|
|
125
127
|
|
|
126
128
|
target.send(vis, subject.name) if vis && vis != :public
|
|
127
129
|
end
|
|
128
130
|
|
|
129
|
-
# Resolve each
|
|
130
|
-
#
|
|
131
|
+
# Resolve each namespace ELEMENT to its live Module and pick the correct
|
|
132
|
+
# keyword (reopening a class with `module` — or vice versa — raises
|
|
131
133
|
# TypeError), so the textual wrapper matches the real definitions.
|
|
134
|
+
#
|
|
135
|
+
# #5: a compact element like "Foo::Bar" stays a SINGLE wrapper `class Foo::Bar`
|
|
136
|
+
# (nesting [Foo::Bar]), matching how a whole-file load (reload) sees it.
|
|
137
|
+
# Splitting it into `module Foo; class Bar` gave nesting [Foo::Bar, Foo], so an
|
|
138
|
+
# unqualified constant defined only in Foo would resolve under redefine but not
|
|
139
|
+
# reload — a strategy disagreement.
|
|
132
140
|
def self.nesting_keywords(namespace)
|
|
133
141
|
mod = Object
|
|
134
|
-
namespace.
|
|
135
|
-
mod = mod.const_get(name)
|
|
142
|
+
namespace.map do |name|
|
|
143
|
+
mod = mod.const_get(name) # const_get resolves a compact "Foo::Bar" too
|
|
136
144
|
[mod.is_a?(Class) ? "class" : "module", name]
|
|
137
145
|
end
|
|
138
146
|
end
|
|
@@ -1,6 +1,5 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require "minitest"
|
|
4
3
|
require "stringio"
|
|
5
4
|
|
|
6
5
|
module Mutineer
|
|
@@ -21,6 +20,17 @@ module Mutineer
|
|
|
21
20
|
# `test_files` is one path or an Array of paths (M3 coverage selection passes
|
|
22
21
|
# the covering subset); each is loaded before the single Minitest.run.
|
|
23
22
|
def self.run(test_files)
|
|
23
|
+
# minitest is required LAZILY (never at load time) so Mutineer keeps zero
|
|
24
|
+
# runtime gem deps and loads fine in an rspec-only project; a missing
|
|
25
|
+
# minitest raises a clear Mutineer error rather than a LoadError backtrace.
|
|
26
|
+
begin
|
|
27
|
+
require "minitest"
|
|
28
|
+
rescue LoadError
|
|
29
|
+
raise Mutineer::FrameworkUnavailable,
|
|
30
|
+
"minitest is not available — add minitest to the project under test, " \
|
|
31
|
+
"or use --framework rspec"
|
|
32
|
+
end
|
|
33
|
+
|
|
24
34
|
# Neutralise autorun so a test file's `require "minitest/autorun"`
|
|
25
35
|
# registers no at_exit hook.
|
|
26
36
|
def Minitest.autorun; end # rubocop:disable Lint/NestedMethodDefinition
|
data/lib/mutineer/project.rb
CHANGED
|
@@ -28,6 +28,7 @@ module Mutineer
|
|
|
28
28
|
@file = file
|
|
29
29
|
@namespace_stack = []
|
|
30
30
|
@subjects = []
|
|
31
|
+
@singleton_depth = 0
|
|
31
32
|
super()
|
|
32
33
|
end
|
|
33
34
|
|
|
@@ -43,12 +44,25 @@ module Mutineer
|
|
|
43
44
|
@namespace_stack.pop
|
|
44
45
|
end
|
|
45
46
|
|
|
47
|
+
# Methods inside `class << self` are class methods of the enclosing
|
|
48
|
+
# namespace, but their def nodes have no receiver — track the singleton
|
|
49
|
+
# context so they're recorded as singleton (so redefine targets the
|
|
50
|
+
# singleton_class, not instances). `class << some_other_obj` can't be
|
|
51
|
+
# represented against the namespace, so its defs are skipped (not recursed).
|
|
52
|
+
def visit_singleton_class_node(node)
|
|
53
|
+
return unless node.expression.is_a?(Prism::SelfNode)
|
|
54
|
+
|
|
55
|
+
@singleton_depth += 1
|
|
56
|
+
super
|
|
57
|
+
@singleton_depth -= 1
|
|
58
|
+
end
|
|
59
|
+
|
|
46
60
|
def visit_def_node(node)
|
|
47
61
|
@subjects << Subject.new(
|
|
48
62
|
file: @file,
|
|
49
63
|
namespace: @namespace_stack.dup,
|
|
50
64
|
name: node.name,
|
|
51
|
-
singleton: !node.receiver.nil?,
|
|
65
|
+
singleton: !node.receiver.nil? || @singleton_depth.positive?,
|
|
52
66
|
def_node: node
|
|
53
67
|
)
|
|
54
68
|
super
|
data/lib/mutineer/runner.rb
CHANGED
|
@@ -5,7 +5,9 @@ require_relative "project"
|
|
|
5
5
|
require_relative "result"
|
|
6
6
|
require_relative "isolation"
|
|
7
7
|
require_relative "minitest_integration"
|
|
8
|
+
require_relative "test_runners"
|
|
8
9
|
require_relative "coverage_map"
|
|
10
|
+
require_relative "changed_lines"
|
|
9
11
|
require_relative "mutator_registry"
|
|
10
12
|
require_relative "worker_pool"
|
|
11
13
|
|
|
@@ -38,29 +40,39 @@ module Mutineer
|
|
|
38
40
|
# parse that needs nothing loaded. Standalone mode requires the sources as
|
|
39
41
|
# before so their classes exist for the children to inherit.
|
|
40
42
|
if config.boot
|
|
43
|
+
# Coverage instruments only files loaded AFTER it starts. Start it BEFORE
|
|
44
|
+
# the boot require so the entire app loaded during boot is instrumented;
|
|
45
|
+
# forked children then measure each test's coverage delta against it.
|
|
46
|
+
require "coverage"
|
|
47
|
+
Coverage.start(lines: true) unless Coverage.running?
|
|
41
48
|
require File.expand_path(config.boot, config.project_root)
|
|
42
49
|
else
|
|
43
50
|
config.sources.each { |f| require File.expand_path(f, config.project_root) }
|
|
44
51
|
end
|
|
45
52
|
config.require_paths.each { |f| require File.expand_path(f, config.project_root) }
|
|
46
53
|
|
|
47
|
-
# Boot mode skips coverage selection entirely: every mutant runs the given
|
|
48
|
-
# --test files (precomputed absolute here, used directly by Runner.run).
|
|
49
54
|
if config.boot
|
|
50
|
-
coverage_map = nil
|
|
51
|
-
boot_tests = config.tests.map { |t| File.expand_path(t, config.project_root) }
|
|
52
55
|
# Rails/Minitest test files do `require "test_helper"`, which needs the
|
|
53
56
|
# test root on $LOAD_PATH (`bin/rails test` adds it). Prepend each test
|
|
54
57
|
# file's helper root here in the parent so loading them in the fork
|
|
55
|
-
# children
|
|
58
|
+
# children (both coverage capture and per-mutant) resolves.
|
|
59
|
+
boot_tests = config.tests.map { |t| File.expand_path(t, config.project_root) }
|
|
56
60
|
test_load_roots(boot_tests).each { |d| $LOAD_PATH.unshift(d) unless $LOAD_PATH.include?(d) }
|
|
61
|
+
|
|
62
|
+
# Boot mode now uses coverage selection too: capture each test's coverage
|
|
63
|
+
# by forking the booted parent, then select covering tests per mutant.
|
|
64
|
+
coverage_map = CoverageMap.new(
|
|
65
|
+
source_paths: config.sources, test_paths: config.tests,
|
|
66
|
+
cache_dir: config.cache_dir, project_root: config.project_root,
|
|
67
|
+
load_paths: config.load_paths, framework: config.framework,
|
|
68
|
+
boot_path: File.expand_path(config.boot, config.project_root)
|
|
69
|
+
).build_via_fork(rails: config.rails)
|
|
57
70
|
else
|
|
58
71
|
coverage_map = CoverageMap.new(
|
|
59
72
|
source_paths: config.sources, test_paths: config.tests,
|
|
60
73
|
cache_dir: config.cache_dir, project_root: config.project_root,
|
|
61
|
-
load_paths: config.load_paths
|
|
74
|
+
load_paths: config.load_paths, framework: config.framework
|
|
62
75
|
).build_or_load
|
|
63
|
-
boot_tests = nil
|
|
64
76
|
end
|
|
65
77
|
|
|
66
78
|
# Collect every (subject, mutation) up front so the pool can fan them out.
|
|
@@ -75,6 +87,8 @@ module Mutineer
|
|
|
75
87
|
end
|
|
76
88
|
end
|
|
77
89
|
|
|
90
|
+
jobs = filter_since(jobs, source_map, config) if config.since
|
|
91
|
+
|
|
78
92
|
# C3: 7a writes mutineer_mutant*.rb into each source dir (so require_relative
|
|
79
93
|
# resolves). A SIGKILL'd child skips the tempfile's ensure-unlink, orphaning
|
|
80
94
|
# it. `ensure` is unreliable vs SIGKILL, so the PARENT sweeps each source dir
|
|
@@ -86,10 +100,10 @@ module Mutineer
|
|
|
86
100
|
strategy = config.strategy
|
|
87
101
|
results =
|
|
88
102
|
begin
|
|
103
|
+
framework = config.framework
|
|
89
104
|
bare = WorkerPool.new(config.jobs).run(jobs) do |subject, mutation|
|
|
90
105
|
run(mutation, source_file: subject.file, coverage_map: coverage_map,
|
|
91
|
-
subject: subject, strategy: strategy,
|
|
92
|
-
test_files: boot_tests, rails: config.rails)
|
|
106
|
+
subject: subject, strategy: strategy, rails: config.rails, framework: framework)
|
|
93
107
|
end
|
|
94
108
|
# The bare Results carry only status (Subjects hold live AST nodes that
|
|
95
109
|
# do not marshal); reattach subject+mutation in the parent, in order.
|
|
@@ -101,6 +115,22 @@ module Mutineer
|
|
|
101
115
|
[AggregateResult.new(results), source_map]
|
|
102
116
|
end
|
|
103
117
|
|
|
118
|
+
# --since: keep only jobs whose mutation lands on a line changed since the git
|
|
119
|
+
# ref. Composes with coverage selection (it only narrows the job list; each
|
|
120
|
+
# surviving mutant still goes through Runner.run's coverage check). A file with
|
|
121
|
+
# no changed lines (absent from the diff) contributes no jobs. Line is computed
|
|
122
|
+
# exactly as Runner.run does, from the already-read source in source_map.
|
|
123
|
+
def self.filter_since(jobs, source_map, config)
|
|
124
|
+
changed = ChangedLines.for(ref: config.since, files: config.sources,
|
|
125
|
+
project_root: config.project_root)
|
|
126
|
+
jobs.select do |subject, mutation|
|
|
127
|
+
source = source_map[subject.file]
|
|
128
|
+
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
|
129
|
+
abs = File.expand_path(subject.file, config.project_root)
|
|
130
|
+
changed.fetch(abs, []).include?(line)
|
|
131
|
+
end
|
|
132
|
+
end
|
|
133
|
+
|
|
104
134
|
# For each test file, the directory to add to $LOAD_PATH so its
|
|
105
135
|
# `require "test_helper"` (or spec_helper) resolves: the nearest ancestor
|
|
106
136
|
# holding that helper, plus the file's own dir as a fallback.
|
|
@@ -131,24 +161,21 @@ module Mutineer
|
|
|
131
161
|
end
|
|
132
162
|
|
|
133
163
|
def self.run(mutation, source_file:, coverage_map: nil, subject: nil, strategy: "reload",
|
|
134
|
-
timeout: Isolation::DEFAULT_TIMEOUT,
|
|
164
|
+
timeout: Isolation::DEFAULT_TIMEOUT, rails: false, framework: "minitest")
|
|
135
165
|
source = File.read(source_file)
|
|
136
166
|
mutated = mutation.apply(source)
|
|
137
167
|
|
|
138
168
|
# Validity rule: a mutant that doesn't re-parse is skipped before forking.
|
|
139
169
|
return Result.skipped if Parser.parse_string(mutated).errors.any?
|
|
140
170
|
|
|
141
|
-
#
|
|
142
|
-
#
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
chosen = coverage_map.tests_for(source_file, line)
|
|
148
|
-
return Result.no_coverage if chosen.empty?
|
|
171
|
+
# Coverage selection (both standalone and boot mode): a mutation on a line
|
|
172
|
+
# no test exercises is :no_coverage (no fork); otherwise exactly the
|
|
173
|
+
# covering test files run in the child.
|
|
174
|
+
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
|
175
|
+
chosen = coverage_map.tests_for(source_file, line)
|
|
176
|
+
return Result.no_coverage if chosen.empty?
|
|
149
177
|
|
|
150
|
-
|
|
151
|
-
end
|
|
178
|
+
abs_tests = chosen.map { |t| File.expand_path(t, coverage_map.project_root) }
|
|
152
179
|
|
|
153
180
|
Isolation.run(timeout: timeout) do
|
|
154
181
|
# Forking inherits the parent's live DB connection; sharing one socket
|
|
@@ -159,7 +186,7 @@ module Mutineer
|
|
|
159
186
|
else
|
|
160
187
|
Isolation.apply_whole_file(mutated, source_file)
|
|
161
188
|
end
|
|
162
|
-
|
|
189
|
+
TestRunners.for(framework).run(abs_tests)
|
|
163
190
|
end
|
|
164
191
|
end
|
|
165
192
|
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../minitest_integration"
|
|
4
|
+
|
|
5
|
+
module Mutineer
|
|
6
|
+
module TestRunners
|
|
7
|
+
# Uniform wrapper over the existing MinitestIntegration impl so the runner
|
|
8
|
+
# selection (TestRunners.for) has one method shape across frameworks.
|
|
9
|
+
# MinitestIntegration stays the implementation — all its behaviour (autorun
|
|
10
|
+
# neutralisation, Runnable.reset, load, Minitest.run -> 0/1) is preserved.
|
|
11
|
+
module Minitest
|
|
12
|
+
def self.run(test_files) = MinitestIntegration.run(test_files)
|
|
13
|
+
end
|
|
14
|
+
end
|
|
15
|
+
end
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "stringio"
|
|
4
|
+
|
|
5
|
+
module Mutineer
|
|
6
|
+
# Raised when the target project asks for a framework whose gem isn't present.
|
|
7
|
+
# rspec is NOT a Mutineer dependency — it must come from the project's bundle.
|
|
8
|
+
class FrameworkUnavailable < StandardError; end
|
|
9
|
+
|
|
10
|
+
module TestRunners
|
|
11
|
+
# Child-process-only RSpec runner, mirroring MinitestIntegration's contract:
|
|
12
|
+
# run the given spec files and return 0 (all passed) / 1 (any failure).
|
|
13
|
+
#
|
|
14
|
+
# rspec-core is required LAZILY here, never at load time, so Mutineer keeps
|
|
15
|
+
# zero runtime gem deps; a missing rspec raises a clear Mutineer error.
|
|
16
|
+
#
|
|
17
|
+
# No `rescue` around the run itself: Isolation.run's fork block is the single
|
|
18
|
+
# exception boundary (any exception there -> exit 2), keeping the 0/1 contract.
|
|
19
|
+
module RSpec
|
|
20
|
+
# `spec_files` is one path or an Array of paths (coverage selection passes
|
|
21
|
+
# the covering subset). All are loaded+run in a single RSpec invocation.
|
|
22
|
+
def self.run(spec_files)
|
|
23
|
+
require_rspec!
|
|
24
|
+
|
|
25
|
+
# rspec/autorun (if a spec_helper required it) installs an at_exit run;
|
|
26
|
+
# neutralise it like Minitest.autorun so it never double-fires.
|
|
27
|
+
::RSpec::Core::Runner.disable_autorun!
|
|
28
|
+
|
|
29
|
+
# Drop any RSpec world/configuration inherited across runs in one process
|
|
30
|
+
# (e.g. successive forks of a booted parent) so examples never accumulate.
|
|
31
|
+
::RSpec.reset
|
|
32
|
+
|
|
33
|
+
# Silence RSpec's formatter output (and any stray puts/deprecations) so it
|
|
34
|
+
# never pollutes Mutineer's report streams. The formatter writes to the
|
|
35
|
+
# passed IO; $stdout/$stderr are redirected to catch everything else.
|
|
36
|
+
sink = StringIO.new
|
|
37
|
+
orig_out = $stdout
|
|
38
|
+
orig_err = $stderr
|
|
39
|
+
$stdout = sink
|
|
40
|
+
$stderr = sink
|
|
41
|
+
begin
|
|
42
|
+
status = ::RSpec::Core::Runner.run(["--no-color", *Array(spec_files)], sink, sink)
|
|
43
|
+
ensure
|
|
44
|
+
$stdout = orig_out
|
|
45
|
+
$stderr = orig_err
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
status.zero? ? 0 : 1
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def self.require_rspec!
|
|
52
|
+
require "rspec/core"
|
|
53
|
+
rescue LoadError
|
|
54
|
+
raise Mutineer::FrameworkUnavailable,
|
|
55
|
+
"framework 'rspec' requested but rspec is not available; " \
|
|
56
|
+
"add rspec to the project under test (its bundle), then retry"
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
end
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "test_runners/minitest"
|
|
4
|
+
require_relative "test_runners/rspec"
|
|
5
|
+
|
|
6
|
+
module Mutineer
|
|
7
|
+
# Picks the test-framework runner. Each runner responds to `.run(files) -> 0/1`
|
|
8
|
+
# (0 = all passed, 1 = any failure) and is called only inside a forked child.
|
|
9
|
+
module TestRunners
|
|
10
|
+
def self.for(framework)
|
|
11
|
+
case framework
|
|
12
|
+
when "rspec" then RSpec
|
|
13
|
+
when "minitest", nil, "" then Minitest
|
|
14
|
+
else
|
|
15
|
+
raise Mutineer::ConfigError, "unknown framework #{framework.inspect} (expected: minitest, rspec)"
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
data/lib/mutineer/version.rb
CHANGED
data/lib/mutineer/worker_pool.rb
CHANGED
|
@@ -10,7 +10,7 @@ module Mutineer
|
|
|
10
10
|
# SAME ORDER as `items` regardless of finish order, so verdicts are identical to
|
|
11
11
|
# a serial run (R4) and downstream output is stable.
|
|
12
12
|
#
|
|
13
|
-
# The block is
|
|
13
|
+
# The block is run inside the child via `yield(*items[i])`; whatever it
|
|
14
14
|
# returns (a Result) is the marshaled payload. Per-mutant timeout is handled one
|
|
15
15
|
# level down by Isolation (KTD2) — the pool adds no separate wall clock.
|
|
16
16
|
class WorkerPool
|
|
@@ -70,43 +70,48 @@ module Mutineer
|
|
|
70
70
|
return
|
|
71
71
|
end
|
|
72
72
|
wr.close
|
|
73
|
-
running[pid] = [idx, rd]
|
|
73
|
+
running[pid] = [idx, rd, +""] # idx, read end, accumulated bytes
|
|
74
74
|
end
|
|
75
75
|
end
|
|
76
76
|
|
|
77
|
-
#
|
|
78
|
-
#
|
|
79
|
-
#
|
|
77
|
+
# Drain pipes with IO.select and reap a child only on EOF (#4). The old code
|
|
78
|
+
# reaped first and read after — but a child whose payload exceeds the OS pipe
|
|
79
|
+
# buffer (~64KB) blocks on `write` before it can exit, so it was never reaped
|
|
80
|
+
# and the pool deadlocked. Reading concurrently keeps the pipe drained so the
|
|
81
|
+
# child can finish and exit; EOF means it closed its write end (done writing).
|
|
82
|
+
# We waitpid only OUR known pids (R6: never wait2(-1), which would steal the
|
|
83
|
+
# host suite's children).
|
|
80
84
|
def reap(results, running)
|
|
81
85
|
return if running.empty?
|
|
82
86
|
|
|
83
87
|
loop do
|
|
84
|
-
running.
|
|
85
|
-
|
|
86
|
-
|
|
88
|
+
rds = running.values.map { |v| v[1] }
|
|
89
|
+
ready, = IO.select(rds, nil, nil)
|
|
90
|
+
ready.each do |rd|
|
|
91
|
+
pid, (idx, _io, buf) = running.find { |_, v| v[1].equal?(rd) }
|
|
92
|
+
chunk = rd.read_nonblock(65_536, exception: false)
|
|
93
|
+
next if chunk == :wait_readable
|
|
87
94
|
|
|
88
|
-
|
|
95
|
+
if chunk.nil? # EOF: child closed wr; it is done writing and exiting
|
|
96
|
+
rd.close
|
|
97
|
+
Process.waitpid(pid) # reap the now-finished child (no zombie)
|
|
98
|
+
running.delete(pid)
|
|
99
|
+
results[idx] = decode(buf)
|
|
100
|
+
return
|
|
101
|
+
end
|
|
102
|
+
buf << chunk
|
|
89
103
|
end
|
|
90
|
-
sleep 0.005
|
|
91
104
|
end
|
|
92
105
|
end
|
|
93
106
|
|
|
94
|
-
def
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
# R6: a partial/garbage Marshal stream (dead worker) must not crash the
|
|
103
|
-
# pool — degrade to an error Result.
|
|
104
|
-
begin
|
|
105
|
-
Marshal.load(data)
|
|
106
|
-
rescue StandardError => e
|
|
107
|
-
Result.error("worker result unreadable: #{e.class}: #{e.message}")
|
|
108
|
-
end
|
|
109
|
-
end
|
|
107
|
+
def decode(data)
|
|
108
|
+
return Result.error("worker produced no result") if data.empty?
|
|
109
|
+
|
|
110
|
+
# R6: a partial/garbage Marshal stream (dead worker) must not crash the
|
|
111
|
+
# pool — degrade to an error Result.
|
|
112
|
+
Marshal.load(data)
|
|
113
|
+
rescue StandardError => e
|
|
114
|
+
Result.error("worker result unreadable: #{e.class}: #{e.message}")
|
|
110
115
|
end
|
|
111
116
|
end
|
|
112
117
|
end
|
data/lib/mutineer.rb
CHANGED
|
@@ -8,8 +8,10 @@ require_relative "mutineer/mutation"
|
|
|
8
8
|
require_relative "mutineer/project"
|
|
9
9
|
require_relative "mutineer/result"
|
|
10
10
|
require_relative "mutineer/coverage_map"
|
|
11
|
+
require_relative "mutineer/changed_lines"
|
|
11
12
|
require_relative "mutineer/isolation"
|
|
12
13
|
require_relative "mutineer/minitest_integration"
|
|
14
|
+
require_relative "mutineer/test_runners"
|
|
13
15
|
require_relative "mutineer/mutators/base"
|
|
14
16
|
require_relative "mutineer/mutators/arithmetic"
|
|
15
17
|
require_relative "mutineer/mutators/comparison"
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: mutineer
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.6.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- David Teren
|
|
@@ -52,6 +52,7 @@ files:
|
|
|
52
52
|
- README.md
|
|
53
53
|
- bin/mutineer
|
|
54
54
|
- lib/mutineer.rb
|
|
55
|
+
- lib/mutineer/changed_lines.rb
|
|
55
56
|
- lib/mutineer/cli.rb
|
|
56
57
|
- lib/mutineer/config.rb
|
|
57
58
|
- lib/mutineer/coverage_map.rb
|
|
@@ -74,6 +75,9 @@ files:
|
|
|
74
75
|
- lib/mutineer/result.rb
|
|
75
76
|
- lib/mutineer/runner.rb
|
|
76
77
|
- lib/mutineer/subject.rb
|
|
78
|
+
- lib/mutineer/test_runners.rb
|
|
79
|
+
- lib/mutineer/test_runners/minitest.rb
|
|
80
|
+
- lib/mutineer/test_runners/rspec.rb
|
|
77
81
|
- lib/mutineer/version.rb
|
|
78
82
|
- lib/mutineer/worker_pool.rb
|
|
79
83
|
homepage: https://github.com/davidteren/mutineer
|