mutineer 0.3.0 → 0.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +59 -0
- data/README.md +5 -1
- data/lib/mutineer/changed_lines.rb +56 -0
- data/lib/mutineer/cli.rb +62 -1
- data/lib/mutineer/config.rb +21 -2
- data/lib/mutineer/coverage_map.rb +39 -2
- data/lib/mutineer/isolation.rb +32 -24
- data/lib/mutineer/minitest_integration.rb +11 -1
- data/lib/mutineer/project.rb +15 -1
- data/lib/mutineer/runner.rb +42 -5
- data/lib/mutineer/test_runners/minitest.rb +15 -0
- data/lib/mutineer/test_runners/rspec.rb +60 -0
- data/lib/mutineer/test_runners.rb +19 -0
- data/lib/mutineer/version.rb +1 -1
- data/lib/mutineer/worker_pool.rb +31 -26
- data/lib/mutineer.rb +2 -0
- metadata +5 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b549b3640aa316aa715732f4e91f3be669b7ab8e7ad877b8207d0e31ac12b4aa
|
|
4
|
+
data.tar.gz: a626241007cf47b4fb0ef3c68dc1fba7f0c8679d0c41db516dca888893fc5f51
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: ba4d45a6ba52ead7da7f580452eb610bfc23b368501c496e80ffa56e2e364a62fcf896a1e9d37c50b1216727eb8a6a02386ba0917c85397ac61a3f2a9f41d694
|
|
7
|
+
data.tar.gz: 6be383459b367be9b72135b1672d57c63bd4e4a6c032c3301532f50a1bcb96dff2cc6ecf764979d6009fab9e4c1e4d327fbb1e81cf3652232cb38a2eb6a82558
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,60 @@ All notable changes to this project are documented here. The format is based on
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/), and this project adheres to
|
|
5
5
|
[Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [0.6.2] - 2026-06-29
|
|
8
|
+
|
|
9
|
+
### Fixed
|
|
10
|
+
- **`--rails` defaults `RAILS_ENV` to `test`** (#7) — an unset `RAILS_ENV` booted
|
|
11
|
+
development, where the suite isn't loaded, so every mutant was falsely reported
|
|
12
|
+
`no_coverage` (score N/A, exit 0). An explicit `RAILS_ENV` is respected.
|
|
13
|
+
- **`--rails` defaults to `--jobs 1`** (#12) — parallel mutant forks share one
|
|
14
|
+
database and deadlock on transactional fixtures; explicit `--jobs N` opts back
|
|
15
|
+
into parallelism.
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
- **Tier-2 operator discoverability** (#14) — the human-format run summary now
|
|
19
|
+
lists the available opt-in tier-2 operators and how to enable them.
|
|
20
|
+
|
|
21
|
+
## [0.6.1] - 2026-06-29
|
|
22
|
+
|
|
23
|
+
### Changed
|
|
24
|
+
- **Removed `eval` entirely** — the redefine strategy now `load`s the wrapped
|
|
25
|
+
method snippet from a tempfile instead of evaluating a string. Behavior is
|
|
26
|
+
identical (top-level load rebuilds the same `Module.nesting`), but the gem no
|
|
27
|
+
longer uses dynamic string execution, clearing supply-chain scanner flags.
|
|
28
|
+
Zero runtime dependencies unchanged.
|
|
29
|
+
|
|
30
|
+
## [0.6.0] - 2026-06-28
|
|
31
|
+
|
|
32
|
+
### Added
|
|
33
|
+
- **RSpec support** (#6) — `--framework rspec` (or auto-detected when most
|
|
34
|
+
`--test` files end in `_spec.rb`) runs RSpec suites instead of Minitest, via a
|
|
35
|
+
pluggable test-runner abstraction. Both frameworks are loaded lazily, so
|
|
36
|
+
Mutineer keeps zero runtime gem dependencies and works in an rspec-only
|
|
37
|
+
project; coverage selection works for both. `.mutineer.yml` accepts `framework:`.
|
|
38
|
+
|
|
39
|
+
### Fixed
|
|
40
|
+
- **Redefine strategy keeps compact `class A::B` as a single nesting wrapper**
|
|
41
|
+
(#5) — avoids a constant-resolution disagreement with the reload strategy.
|
|
42
|
+
|
|
43
|
+
## [0.5.0] - 2026-06-28
|
|
44
|
+
|
|
45
|
+
### Fixed
|
|
46
|
+
- **`class << self` methods are now discovered and mutated** (#3) — previously
|
|
47
|
+
they were treated as instance methods, so the redefine strategy mis-targeted
|
|
48
|
+
them. `class << other_obj` blocks are skipped (not representable).
|
|
49
|
+
- **Worker pool no longer deadlocks on large results** (#4) — pipes are drained
|
|
50
|
+
with `IO.select` and children reaped on EOF, so a result bigger than the OS
|
|
51
|
+
pipe buffer (~64KB) can't wedge the run.
|
|
52
|
+
|
|
53
|
+
## [0.4.0] - 2026-06-28
|
|
54
|
+
|
|
55
|
+
### Added
|
|
56
|
+
- **`--since <git-ref>`** (#2) — mutate only the lines changed since a git ref
|
|
57
|
+
(e.g. `--since origin/main`), so CI on a pull request mutation-tests just the
|
|
58
|
+
new/changed code. Composes with coverage selection; `--dry-run --since`
|
|
59
|
+
narrows the preview too. Unknown ref / not-a-git-repo exits 2.
|
|
60
|
+
|
|
7
61
|
## [0.3.0] - 2026-06-28
|
|
8
62
|
|
|
9
63
|
### Added
|
|
@@ -46,6 +100,11 @@ All notable changes to this project are documented here. The format is based on
|
|
|
46
100
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
47
101
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
48
102
|
|
|
103
|
+
[0.6.2]: https://github.com/davidteren/mutineer/releases/tag/v0.6.2
|
|
104
|
+
[0.6.1]: https://github.com/davidteren/mutineer/releases/tag/v0.6.1
|
|
105
|
+
[0.6.0]: https://github.com/davidteren/mutineer/releases/tag/v0.6.0
|
|
106
|
+
[0.5.0]: https://github.com/davidteren/mutineer/releases/tag/v0.5.0
|
|
107
|
+
[0.4.0]: https://github.com/davidteren/mutineer/releases/tag/v0.4.0
|
|
49
108
|
[0.3.0]: https://github.com/davidteren/mutineer/releases/tag/v0.3.0
|
|
50
109
|
[0.2.0]: https://github.com/davidteren/mutineer/releases/tag/v0.2.0
|
|
51
110
|
[0.1.0]: https://github.com/davidteren/mutineer/releases/tag/v0.1.0
|
data/README.md
CHANGED
|
@@ -1,7 +1,9 @@
|
|
|
1
1
|
# Mutineer
|
|
2
2
|
|
|
3
|
+
[](https://socket.dev/rubygems/package/mutineer/overview/0.6.2?platform=ruby)
|
|
4
|
+
|
|
3
5
|
A clean-room mutation-testing tool for Ruby. Mutineer mutates your source one
|
|
4
|
-
change at a time, runs your
|
|
6
|
+
change at a time, runs your test suite (Minitest or RSpec) against each mutant, and reports the
|
|
5
7
|
ones your tests failed to catch — the gaps where your suite isn't actually
|
|
6
8
|
testing anything.
|
|
7
9
|
|
|
@@ -45,6 +47,8 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
|
|
|
45
47
|
| `--operators LIST` | Comma-separated operator names (default: the Tier-1 set) |
|
|
46
48
|
| `--threshold FLOAT` | Exit 1 when the score is below FLOAT (default: 0 = off) |
|
|
47
49
|
| `--only NAME` | Restrict to one fully-qualified subject, e.g. `Calculator#add` |
|
|
50
|
+
| `--framework NAME` | `minitest` (default) or `rspec`; auto-detected as rspec when most `--test` files end in `_spec.rb` |
|
|
51
|
+
| `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`) — ideal for PR CI |
|
|
48
52
|
| `--jobs N` | Parallel worker count (default: processor count) |
|
|
49
53
|
| `--strategy NAME` | Mutation application: `reload` whole-file (default) or `redefine` surgical (`7a`/`7b` accepted as deprecated aliases) |
|
|
50
54
|
| `--format human\|json` | Report format (default: human) |
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "set"
|
|
4
|
+
require "open3"
|
|
5
|
+
|
|
6
|
+
module Mutineer
|
|
7
|
+
# Maps each source file to the set of NEW-side line numbers that changed since a
|
|
8
|
+
# given git ref, by parsing `git diff --unified=0`. Used to restrict mutations
|
|
9
|
+
# to only the diff (issue #2): on a PR you care whether the changed code is
|
|
10
|
+
# tested, so mutating just those lines is fast and actionable.
|
|
11
|
+
#
|
|
12
|
+
# git is an external tool, not a gem dependency — shelling out is fine. The pure
|
|
13
|
+
# `parse` carries the logic; `git_diff` is injectable so it stays testable
|
|
14
|
+
# without invoking git.
|
|
15
|
+
module ChangedLines
|
|
16
|
+
HUNK = /^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@/
|
|
17
|
+
|
|
18
|
+
module_function
|
|
19
|
+
|
|
20
|
+
# Parse `git diff --unified=0` output into the set of NEW-side line numbers
|
|
21
|
+
# added/modified. With --unified=0 each hunk's `+c,d` block is exactly the
|
|
22
|
+
# changed lines: c..c+d-1. `d` absent means 1 line; `d == 0` is a pure
|
|
23
|
+
# deletion and contributes nothing.
|
|
24
|
+
def parse(diff_text)
|
|
25
|
+
lines = Set.new
|
|
26
|
+
diff_text.each_line do |row|
|
|
27
|
+
m = HUNK.match(row) or next
|
|
28
|
+
start = m[1].to_i
|
|
29
|
+
count = m[2].nil? ? 1 : m[2].to_i
|
|
30
|
+
next if count.zero?
|
|
31
|
+
|
|
32
|
+
lines.merge(start...(start + count))
|
|
33
|
+
end
|
|
34
|
+
lines
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# { absolute_file_path => Set<Integer> } of changed new-side lines per file.
|
|
38
|
+
# `runner` is injected so tests can supply canned diff text per file.
|
|
39
|
+
def for(ref:, files:, project_root:, runner: method(:git_diff))
|
|
40
|
+
files.each_with_object({}) do |file, acc|
|
|
41
|
+
abs = File.expand_path(file, project_root)
|
|
42
|
+
acc[abs] = parse(runner.call(ref, abs, project_root))
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# stdout of `git -C <root> diff --unified=0 <ref> -- <file>`, "" on failure.
|
|
47
|
+
def git_diff(ref, abs_file, project_root)
|
|
48
|
+
out, _err, status = Open3.capture3(
|
|
49
|
+
"git", "-C", project_root, "diff", "--unified=0", ref, "--", abs_file
|
|
50
|
+
)
|
|
51
|
+
status.success? ? out : ""
|
|
52
|
+
rescue StandardError
|
|
53
|
+
""
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
data/lib/mutineer/cli.rb
CHANGED
|
@@ -2,10 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
require "optparse"
|
|
4
4
|
require "set"
|
|
5
|
+
require "open3"
|
|
5
6
|
require_relative "version"
|
|
6
7
|
require_relative "config"
|
|
7
8
|
require_relative "parser"
|
|
8
9
|
require_relative "project"
|
|
10
|
+
require_relative "changed_lines"
|
|
9
11
|
require_relative "runner"
|
|
10
12
|
require_relative "reporter"
|
|
11
13
|
require_relative "mutator_registry"
|
|
@@ -32,10 +34,12 @@ module Mutineer
|
|
|
32
34
|
--operators LIST Comma-separated operator names (default: Tier 1 set)
|
|
33
35
|
--threshold FLOAT Fail (exit 1) when score < FLOAT (default: 0 = off)
|
|
34
36
|
--only NAME Restrict to one fully-qualified subject
|
|
37
|
+
--since REF Only mutate lines changed since git REF (e.g. origin/main)
|
|
35
38
|
--jobs N Parallel worker count (default: processor count)
|
|
36
39
|
--strategy NAME reload (whole-file) or redefine (surgical); default: reload
|
|
40
|
+
--framework NAME minitest or rspec (default: auto-detect from --test names)
|
|
37
41
|
--boot FILE Require FILE once in the parent to boot the app env, then
|
|
38
|
-
fork per mutant (Rails apps; requires --test
|
|
42
|
+
fork per mutant (Rails apps; requires --test)
|
|
39
43
|
--rails Sugar for --boot config/environment --strategy redefine
|
|
40
44
|
--format human|json Report format (default: human)
|
|
41
45
|
--output FILE Write the report to FILE instead of stdout
|
|
@@ -71,11 +75,13 @@ module Mutineer
|
|
|
71
75
|
o.on("--list-operators") { show_operators = true }
|
|
72
76
|
o.on("--dry-run") { opts[:dry_run] = true }
|
|
73
77
|
o.on("--only NAME") { |v| opts[:only] = v; explicit << :only }
|
|
78
|
+
o.on("--since REF") { |v| opts[:since] = v; explicit << :since }
|
|
74
79
|
o.on("--test FILE") { |v| (opts[:tests] ||= []) << v }
|
|
75
80
|
o.on("--operators LIST") { |v| opts[:operators] = v.split(",").map(&:strip); explicit << :operators }
|
|
76
81
|
o.on("--threshold FLOAT") { |v| opts[:threshold] = v.to_f; explicit << :threshold }
|
|
77
82
|
o.on("--jobs N") { |v| opts[:jobs] = v; explicit << :jobs }
|
|
78
83
|
o.on("--strategy STRAT") { |v| opts[:strategy] = v; explicit << :strategy }
|
|
84
|
+
o.on("--framework NAME") { |v| opts[:framework] = v; explicit << :framework }
|
|
79
85
|
o.on("--boot FILE") { |v| opts[:boot] = v; explicit << :boot }
|
|
80
86
|
o.on("--rails") { opts[:rails] = true }
|
|
81
87
|
o.on("--format FORMAT") { |v| opts[:format] = v }
|
|
@@ -183,6 +189,11 @@ module Mutineer
|
|
|
183
189
|
exit 2
|
|
184
190
|
end
|
|
185
191
|
|
|
192
|
+
unless %w[minitest rspec].include?(config.framework)
|
|
193
|
+
warn %(mutineer: unknown framework "#{config.framework}". Expected: minitest, rspec)
|
|
194
|
+
exit 2
|
|
195
|
+
end
|
|
196
|
+
|
|
186
197
|
# Boot mode does no coverage selection — every mutant runs the given tests —
|
|
187
198
|
# so at least one --test file is mandatory (there is nothing to select from).
|
|
188
199
|
if config.boot && config.tests.empty?
|
|
@@ -190,10 +201,31 @@ module Mutineer
|
|
|
190
201
|
exit 2
|
|
191
202
|
end
|
|
192
203
|
|
|
204
|
+
validate_since!(config) if config.since
|
|
193
205
|
preflight_output!(config.output) if config.output
|
|
194
206
|
validate_paths!(config)
|
|
195
207
|
end
|
|
196
208
|
|
|
209
|
+
# --since needs a real git repo and a resolvable ref; either failure is a
|
|
210
|
+
# usage error (exit 2) so CI sees "bad invocation," not "tests too weak."
|
|
211
|
+
def self.validate_since!(config)
|
|
212
|
+
_out, _err, status = Open3.capture3(
|
|
213
|
+
"git", "-C", config.project_root, "rev-parse", "--verify", "--quiet",
|
|
214
|
+
"#{config.since}^{commit}"
|
|
215
|
+
)
|
|
216
|
+
return if status.success?
|
|
217
|
+
|
|
218
|
+
inside, = Open3.capture3(
|
|
219
|
+
"git", "-C", config.project_root, "rev-parse", "--is-inside-work-tree"
|
|
220
|
+
)
|
|
221
|
+
msg = inside.strip == "true" ? "unknown git ref: #{config.since}" : "--since requires a git repository"
|
|
222
|
+
warn "mutineer: #{msg}"
|
|
223
|
+
exit 2
|
|
224
|
+
rescue Errno::ENOENT
|
|
225
|
+
warn "mutineer: --since requires git on PATH"
|
|
226
|
+
exit 2
|
|
227
|
+
end
|
|
228
|
+
|
|
197
229
|
# R5: validate path existence up front so a typo is a clean usage error (exit
|
|
198
230
|
# 2), not an Errno::ENOENT backtrace from deep in the run. Flag checks run
|
|
199
231
|
# first so a bad flag still reports the flag, not the missing file.
|
|
@@ -225,15 +257,40 @@ module Mutineer
|
|
|
225
257
|
reporter = Reporter.new(aggregate, source_map)
|
|
226
258
|
reporter.report(out: $stdout, err: $stderr, threshold: config.threshold,
|
|
227
259
|
format: config.format, output: config.output)
|
|
260
|
+
|
|
261
|
+
# #14: nudge toward the opt-in tier-2 operators (human report only — never
|
|
262
|
+
# pollute JSON output).
|
|
263
|
+
if config.format != "json" && (hint = tier2_hint(config.operators))
|
|
264
|
+
puts hint
|
|
265
|
+
end
|
|
266
|
+
|
|
228
267
|
exit reporter.exit_code(threshold: config.threshold)
|
|
229
268
|
end
|
|
230
269
|
|
|
270
|
+
# The tier-2 operators not in the active set, as a one-line hint (or nil when
|
|
271
|
+
# they're all already enabled). `active` nil means the default (Tier-1) set.
|
|
272
|
+
def self.tier2_hint(active)
|
|
273
|
+
active ||= MutatorRegistry::DEFAULT_NAMES
|
|
274
|
+
unused = MutatorRegistry::TIER2_NAMES - active
|
|
275
|
+
return if unused.empty?
|
|
276
|
+
|
|
277
|
+
"#{unused.size} tier-2 operators available (#{unused.join(', ')}) — " \
|
|
278
|
+
"enable with --operators <list>."
|
|
279
|
+
end
|
|
280
|
+
|
|
231
281
|
def self.dry_run(config)
|
|
232
282
|
operator_classes = MutatorRegistry.resolve(config.operators || MutatorRegistry::DEFAULT_NAMES)
|
|
233
283
|
sources = {}
|
|
234
284
|
per_operator = Hash.new(0)
|
|
235
285
|
skipped = 0
|
|
236
286
|
|
|
287
|
+
# --since narrows the preview to changed lines too, so `--dry-run --since`
|
|
288
|
+
# shows exactly what a real `--since` run would mutate.
|
|
289
|
+
changed = if config.since
|
|
290
|
+
ChangedLines.for(ref: config.since, files: config.sources,
|
|
291
|
+
project_root: config.project_root)
|
|
292
|
+
end
|
|
293
|
+
|
|
237
294
|
Project.discover(config.sources, only: config.only).each do |subject|
|
|
238
295
|
source = (sources[subject.file] ||= Parser.parse_file(subject.file).source.source)
|
|
239
296
|
operator_classes.each do |klass|
|
|
@@ -242,6 +299,10 @@ module Mutineer
|
|
|
242
299
|
skipped += 1
|
|
243
300
|
next
|
|
244
301
|
end
|
|
302
|
+
if changed
|
|
303
|
+
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
|
304
|
+
next unless changed[File.expand_path(subject.file, config.project_root)]&.include?(line)
|
|
305
|
+
end
|
|
245
306
|
per_operator[mutation.operator] += 1
|
|
246
307
|
original = source.byteslice(mutation.start_offset...mutation.end_offset)
|
|
247
308
|
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
data/lib/mutineer/config.rb
CHANGED
|
@@ -24,12 +24,12 @@ module Mutineer
|
|
|
24
24
|
:sources, :tests, :operators, :threshold, :only, :dry_run,
|
|
25
25
|
:cache_dir, :project_root, :load_paths,
|
|
26
26
|
:jobs, :format, :output, :strategy, :require_paths,
|
|
27
|
-
:boot, :rails,
|
|
27
|
+
:boot, :rails, :since, :framework,
|
|
28
28
|
keyword_init: true
|
|
29
29
|
) do
|
|
30
30
|
CONFIG_FILE = ".mutineer.yml"
|
|
31
31
|
# Keys accepted in .mutineer.yml (R7). `require` maps to the :require_paths field.
|
|
32
|
-
KNOWN_KEYS = %w[operators jobs threshold only require boot rails].freeze
|
|
32
|
+
KNOWN_KEYS = %w[operators jobs threshold only require boot rails since framework].freeze
|
|
33
33
|
|
|
34
34
|
def initialize(**kwargs)
|
|
35
35
|
super
|
|
@@ -108,10 +108,28 @@ module Mutineer
|
|
|
108
108
|
if config.rails
|
|
109
109
|
config.boot ||= "config/environment"
|
|
110
110
|
config.strategy = "redefine" unless explicit.include?(:strategy)
|
|
111
|
+
# #12: parallel mutant forks share one database; transactional-fixture
|
|
112
|
+
# setup/teardown across processes contends and deadlocks. Default to
|
|
113
|
+
# serial under --rails; an explicit --jobs N opts back into parallelism
|
|
114
|
+
# (with the per-worker DB-isolation that implies).
|
|
115
|
+
config.jobs = 1 unless explicit.include?(:jobs)
|
|
111
116
|
end
|
|
117
|
+
|
|
118
|
+
# Auto-detect the framework only when neither CLI nor config file set it
|
|
119
|
+
# (explicit value, from either source, is already on config.framework and
|
|
120
|
+
# always wins). Default minitest unless the test files clearly look RSpec.
|
|
121
|
+
config.framework ||= detect_framework(config.tests)
|
|
112
122
|
config
|
|
113
123
|
end
|
|
114
124
|
|
|
125
|
+
# Pick rspec when a MAJORITY of the given test files end with _spec.rb;
|
|
126
|
+
# otherwise minitest. Empty/ambiguous -> minitest (the safe default).
|
|
127
|
+
def self.detect_framework(tests)
|
|
128
|
+
tests = Array(tests)
|
|
129
|
+
specs = tests.count { |t| t.to_s.end_with?("_spec.rb") }
|
|
130
|
+
specs > tests.length / 2.0 ? "rspec" : "minitest"
|
|
131
|
+
end
|
|
132
|
+
|
|
115
133
|
def self.field_for(known_key)
|
|
116
134
|
known_key == "require" ? :require_paths : known_key.to_sym
|
|
117
135
|
end
|
|
@@ -123,6 +141,7 @@ module Mutineer
|
|
|
123
141
|
when "threshold" then value.to_f
|
|
124
142
|
when "require" then Array(value).map(&:to_s)
|
|
125
143
|
when "boot" then value.to_s
|
|
144
|
+
when "framework" then value.to_s
|
|
126
145
|
when "rails" then value == true || value.to_s == "true"
|
|
127
146
|
else value
|
|
128
147
|
end
|
|
@@ -7,6 +7,7 @@ require "fileutils"
|
|
|
7
7
|
require "rbconfig"
|
|
8
8
|
require "coverage"
|
|
9
9
|
require_relative "minitest_integration"
|
|
10
|
+
require_relative "test_runners"
|
|
10
11
|
|
|
11
12
|
module Mutineer
|
|
12
13
|
# Maps `(source_file, line) -> [test_files]` so each mutant runs only against
|
|
@@ -23,7 +24,8 @@ module Mutineer
|
|
|
23
24
|
|
|
24
25
|
def initialize(source_paths:, test_paths:, cache_dir: ".mutineer",
|
|
25
26
|
load_paths: ["lib"], project_root: Dir.pwd,
|
|
26
|
-
capture_timeout: DEFAULT_CAPTURE_TIMEOUT, boot_path: nil
|
|
27
|
+
capture_timeout: DEFAULT_CAPTURE_TIMEOUT, boot_path: nil,
|
|
28
|
+
framework: "minitest")
|
|
27
29
|
@source_paths = Array(source_paths)
|
|
28
30
|
@test_paths = Array(test_paths)
|
|
29
31
|
@cache_dir = cache_dir
|
|
@@ -31,6 +33,7 @@ module Mutineer
|
|
|
31
33
|
@project_root = project_root
|
|
32
34
|
@capture_timeout = capture_timeout
|
|
33
35
|
@boot_path = boot_path
|
|
36
|
+
@framework = framework || "minitest"
|
|
34
37
|
@map = {}
|
|
35
38
|
@failed_test_files = []
|
|
36
39
|
@phase_a_ran = false
|
|
@@ -123,7 +126,7 @@ module Mutineer
|
|
|
123
126
|
begin
|
|
124
127
|
Runner.send(:reconnect_active_record) if rails
|
|
125
128
|
Coverage.result(clear: true, stop: false) # discard pre-test delta
|
|
126
|
-
|
|
129
|
+
TestRunners.for(@framework).run([abs_test])
|
|
127
130
|
# lines:true yields {file => {lines: [...]}}; reduce to the counts
|
|
128
131
|
# array record() expects, keeping only our source files.
|
|
129
132
|
Coverage.result(stop: false)
|
|
@@ -188,6 +191,10 @@ module Mutineer
|
|
|
188
191
|
end
|
|
189
192
|
|
|
190
193
|
def subprocess_script(test_path)
|
|
194
|
+
@framework == "rspec" ? rspec_subprocess_script(test_path) : minitest_subprocess_script(test_path)
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
def minitest_subprocess_script(test_path)
|
|
191
198
|
<<~RUBY
|
|
192
199
|
require "coverage"
|
|
193
200
|
require "json"
|
|
@@ -206,6 +213,35 @@ module Mutineer
|
|
|
206
213
|
RUBY
|
|
207
214
|
end
|
|
208
215
|
|
|
216
|
+
# Same coverage-JSON contract as the minitest path, but driven by RSpec:
|
|
217
|
+
# require rspec/core lazily, load the sources under Coverage, then run the
|
|
218
|
+
# one spec via RSpec::Core::Runner with output silenced so only the JSON
|
|
219
|
+
# reaches stdout. A missing rspec makes `require` raise -> subprocess exits
|
|
220
|
+
# non-zero -> capture() records a skipped (incomplete-map) test, with a hint.
|
|
221
|
+
def rspec_subprocess_script(test_path)
|
|
222
|
+
<<~RUBY
|
|
223
|
+
require "coverage"
|
|
224
|
+
require "json"
|
|
225
|
+
require "stringio"
|
|
226
|
+
begin
|
|
227
|
+
require "rspec/core"
|
|
228
|
+
rescue LoadError
|
|
229
|
+
warn "[mutineer] framework 'rspec' requested but rspec is not available in the project"
|
|
230
|
+
exit 3
|
|
231
|
+
end
|
|
232
|
+
RSpec::Core::Runner.disable_autorun!
|
|
233
|
+
Coverage.start(lines: true)
|
|
234
|
+
$LOAD_PATH.unshift(*#{abs_load_paths.inspect})
|
|
235
|
+
#{abs_source_paths.inspect}.each { |f| load f }
|
|
236
|
+
_orig = $stdout
|
|
237
|
+
_sink = StringIO.new
|
|
238
|
+
$stdout = _sink
|
|
239
|
+
RSpec::Core::Runner.run(["--no-color", #{absolute(test_path).inspect}], _sink, _sink)
|
|
240
|
+
$stdout = _orig
|
|
241
|
+
puts Coverage.result.to_json
|
|
242
|
+
RUBY
|
|
243
|
+
end
|
|
244
|
+
|
|
209
245
|
# Records every source line with a non-zero execution count as covered by
|
|
210
246
|
# this test file. Coverage.result keys are absolute; relativize and drop any
|
|
211
247
|
# path outside the project (stdlib/gem files).
|
|
@@ -234,6 +270,7 @@ module Mutineer
|
|
|
234
270
|
digest_group(d, "test", @test_paths)
|
|
235
271
|
digest_group(d, "boot", [boot_digest_path]) if @boot_path
|
|
236
272
|
@load_paths.sort.each { |lp| d.update("loadpath\0#{lp}\0") }
|
|
273
|
+
d.update("framework\0#{@framework}\0")
|
|
237
274
|
d.hexdigest
|
|
238
275
|
end
|
|
239
276
|
|
data/lib/mutineer/isolation.rb
CHANGED
|
@@ -14,10 +14,10 @@ module Mutineer
|
|
|
14
14
|
# by the parent's monitor flag, not by status.signaled? (which is true for ANY
|
|
15
15
|
# signal death, e.g. SIGSEGV — it cannot tell our SIGKILL apart from the OS's).
|
|
16
16
|
#
|
|
17
|
-
# mutineer: the
|
|
17
|
+
# mutineer: the reload strategy this enables (whole-file `load`) re-executes the
|
|
18
18
|
# entire file — any top-level code runs again. Acceptable for POROs; document
|
|
19
|
-
# if users hit issues with initializers/callbacks.
|
|
20
|
-
#
|
|
19
|
+
# if users hit issues with initializers/callbacks. Alternative: the redefine
|
|
20
|
+
# strategy (surgical single-method redefinition).
|
|
21
21
|
class Isolation
|
|
22
22
|
DEFAULT_TIMEOUT = 10 # seconds
|
|
23
23
|
|
|
@@ -78,15 +78,13 @@ module Mutineer
|
|
|
78
78
|
end
|
|
79
79
|
end
|
|
80
80
|
|
|
81
|
-
#
|
|
82
|
-
# snippet,
|
|
83
|
-
#
|
|
81
|
+
# Redefine strategy: extract just the enclosing DefNode, apply the mutation to
|
|
82
|
+
# that snippet, wrap it in its real namespace, and `load` only that one method
|
|
83
|
+
# back into the running process. No file-level side effects re-run. Child-only.
|
|
84
84
|
#
|
|
85
|
-
#
|
|
86
|
-
#
|
|
87
|
-
#
|
|
88
|
-
# through singleton_class.class_eval (R17) would double-wrap and define on the
|
|
89
|
-
# wrong class.
|
|
85
|
+
# The snippet keeps its own `def self.x` for singletons, so the namespace
|
|
86
|
+
# wrapper redefines instance and singleton methods correctly without any
|
|
87
|
+
# special-casing.
|
|
90
88
|
def self.apply_surgical(mutation, subject, source)
|
|
91
89
|
loc = subject.def_node.location
|
|
92
90
|
def_start = loc.start_offset
|
|
@@ -97,9 +95,9 @@ module Mutineer
|
|
|
97
95
|
mutated_def = snippet.byteslice(0...rel_s) + mutation.replacement + snippet.byteslice(rel_e..)
|
|
98
96
|
|
|
99
97
|
# Rebuild the FULL namespace nesting textually so unqualified enclosing-
|
|
100
|
-
# namespace constants resolve exactly as
|
|
101
|
-
#
|
|
102
|
-
# NameError on such constants (C2 scope-collapse).
|
|
98
|
+
# namespace constants resolve exactly as the reload strategy would. A bare
|
|
99
|
+
# redefinition on the owner would collapse Module.nesting to [owner] and
|
|
100
|
+
# raise NameError on such constants (C2 scope-collapse).
|
|
103
101
|
keywords = nesting_keywords(subject.namespace)
|
|
104
102
|
prefix = keywords.map { |kw, name| "#{kw} #{name}" }.join("\n")
|
|
105
103
|
prefix += "\n" unless prefix.empty?
|
|
@@ -116,23 +114,33 @@ module Mutineer
|
|
|
116
114
|
target = subject.singleton ? owner.singleton_class : owner
|
|
117
115
|
vis = method_visibility(target, subject.name)
|
|
118
116
|
|
|
119
|
-
#
|
|
120
|
-
#
|
|
121
|
-
#
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
117
|
+
# Write the wrapped snippet to a tempfile and `load` it: `load` runs it at
|
|
118
|
+
# top level, so the textual class/module wrappers rebuild Module.nesting
|
|
119
|
+
# identically, with no dynamic string execution for scanners to flag. The
|
|
120
|
+
# input is the project's OWN source (the enclosing method, textually
|
|
121
|
+
# mutated), loaded only in this forked child.
|
|
122
|
+
Tempfile.create(["mutineer_surgical", ".rb"]) do |f|
|
|
123
|
+
f.write(wrapped)
|
|
124
|
+
f.flush
|
|
125
|
+
load f.path
|
|
126
|
+
end
|
|
125
127
|
|
|
126
128
|
target.send(vis, subject.name) if vis && vis != :public
|
|
127
129
|
end
|
|
128
130
|
|
|
129
|
-
# Resolve each
|
|
130
|
-
#
|
|
131
|
+
# Resolve each namespace ELEMENT to its live Module and pick the correct
|
|
132
|
+
# keyword (reopening a class with `module` — or vice versa — raises
|
|
131
133
|
# TypeError), so the textual wrapper matches the real definitions.
|
|
134
|
+
#
|
|
135
|
+
# #5: a compact element like "Foo::Bar" stays a SINGLE wrapper `class Foo::Bar`
|
|
136
|
+
# (nesting [Foo::Bar]), matching how a whole-file load (reload) sees it.
|
|
137
|
+
# Splitting it into `module Foo; class Bar` gave nesting [Foo::Bar, Foo], so an
|
|
138
|
+
# unqualified constant defined only in Foo would resolve under redefine but not
|
|
139
|
+
# reload — a strategy disagreement.
|
|
132
140
|
def self.nesting_keywords(namespace)
|
|
133
141
|
mod = Object
|
|
134
|
-
namespace.
|
|
135
|
-
mod = mod.const_get(name)
|
|
142
|
+
namespace.map do |name|
|
|
143
|
+
mod = mod.const_get(name) # const_get resolves a compact "Foo::Bar" too
|
|
136
144
|
[mod.is_a?(Class) ? "class" : "module", name]
|
|
137
145
|
end
|
|
138
146
|
end
|
|
@@ -1,6 +1,5 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require "minitest"
|
|
4
3
|
require "stringio"
|
|
5
4
|
|
|
6
5
|
module Mutineer
|
|
@@ -21,6 +20,17 @@ module Mutineer
|
|
|
21
20
|
# `test_files` is one path or an Array of paths (M3 coverage selection passes
|
|
22
21
|
# the covering subset); each is loaded before the single Minitest.run.
|
|
23
22
|
def self.run(test_files)
|
|
23
|
+
# minitest is required LAZILY (never at load time) so Mutineer keeps zero
|
|
24
|
+
# runtime gem deps and loads fine in an rspec-only project; a missing
|
|
25
|
+
# minitest raises a clear Mutineer error rather than a LoadError backtrace.
|
|
26
|
+
begin
|
|
27
|
+
require "minitest"
|
|
28
|
+
rescue LoadError
|
|
29
|
+
raise Mutineer::FrameworkUnavailable,
|
|
30
|
+
"minitest is not available — add minitest to the project under test, " \
|
|
31
|
+
"or use --framework rspec"
|
|
32
|
+
end
|
|
33
|
+
|
|
24
34
|
# Neutralise autorun so a test file's `require "minitest/autorun"`
|
|
25
35
|
# registers no at_exit hook.
|
|
26
36
|
def Minitest.autorun; end # rubocop:disable Lint/NestedMethodDefinition
|
data/lib/mutineer/project.rb
CHANGED
|
@@ -28,6 +28,7 @@ module Mutineer
|
|
|
28
28
|
@file = file
|
|
29
29
|
@namespace_stack = []
|
|
30
30
|
@subjects = []
|
|
31
|
+
@singleton_depth = 0
|
|
31
32
|
super()
|
|
32
33
|
end
|
|
33
34
|
|
|
@@ -43,12 +44,25 @@ module Mutineer
|
|
|
43
44
|
@namespace_stack.pop
|
|
44
45
|
end
|
|
45
46
|
|
|
47
|
+
# Methods inside `class << self` are class methods of the enclosing
|
|
48
|
+
# namespace, but their def nodes have no receiver — track the singleton
|
|
49
|
+
# context so they're recorded as singleton (so redefine targets the
|
|
50
|
+
# singleton_class, not instances). `class << some_other_obj` can't be
|
|
51
|
+
# represented against the namespace, so its defs are skipped (not recursed).
|
|
52
|
+
def visit_singleton_class_node(node)
|
|
53
|
+
return unless node.expression.is_a?(Prism::SelfNode)
|
|
54
|
+
|
|
55
|
+
@singleton_depth += 1
|
|
56
|
+
super
|
|
57
|
+
@singleton_depth -= 1
|
|
58
|
+
end
|
|
59
|
+
|
|
46
60
|
def visit_def_node(node)
|
|
47
61
|
@subjects << Subject.new(
|
|
48
62
|
file: @file,
|
|
49
63
|
namespace: @namespace_stack.dup,
|
|
50
64
|
name: node.name,
|
|
51
|
-
singleton: !node.receiver.nil?,
|
|
65
|
+
singleton: !node.receiver.nil? || @singleton_depth.positive?,
|
|
52
66
|
def_node: node
|
|
53
67
|
)
|
|
54
68
|
super
|
data/lib/mutineer/runner.rb
CHANGED
|
@@ -5,7 +5,9 @@ require_relative "project"
|
|
|
5
5
|
require_relative "result"
|
|
6
6
|
require_relative "isolation"
|
|
7
7
|
require_relative "minitest_integration"
|
|
8
|
+
require_relative "test_runners"
|
|
8
9
|
require_relative "coverage_map"
|
|
10
|
+
require_relative "changed_lines"
|
|
9
11
|
require_relative "mutator_registry"
|
|
10
12
|
require_relative "worker_pool"
|
|
11
13
|
|
|
@@ -38,6 +40,11 @@ module Mutineer
|
|
|
38
40
|
# parse that needs nothing loaded. Standalone mode requires the sources as
|
|
39
41
|
# before so their classes exist for the children to inherit.
|
|
40
42
|
if config.boot
|
|
43
|
+
# #7: under --rails an unset RAILS_ENV boots development, where the test
|
|
44
|
+
# suite isn't loaded — coverage comes back empty and EVERY mutant is
|
|
45
|
+
# falsely reported no_coverage (score N/A, exit 0). Default it to test.
|
|
46
|
+
ensure_rails_env(config)
|
|
47
|
+
|
|
41
48
|
# Coverage instruments only files loaded AFTER it starts. Start it BEFORE
|
|
42
49
|
# the boot require so the entire app loaded during boot is instrumented;
|
|
43
50
|
# forked children then measure each test's coverage delta against it.
|
|
@@ -62,14 +69,14 @@ module Mutineer
|
|
|
62
69
|
coverage_map = CoverageMap.new(
|
|
63
70
|
source_paths: config.sources, test_paths: config.tests,
|
|
64
71
|
cache_dir: config.cache_dir, project_root: config.project_root,
|
|
65
|
-
load_paths: config.load_paths,
|
|
72
|
+
load_paths: config.load_paths, framework: config.framework,
|
|
66
73
|
boot_path: File.expand_path(config.boot, config.project_root)
|
|
67
74
|
).build_via_fork(rails: config.rails)
|
|
68
75
|
else
|
|
69
76
|
coverage_map = CoverageMap.new(
|
|
70
77
|
source_paths: config.sources, test_paths: config.tests,
|
|
71
78
|
cache_dir: config.cache_dir, project_root: config.project_root,
|
|
72
|
-
load_paths: config.load_paths
|
|
79
|
+
load_paths: config.load_paths, framework: config.framework
|
|
73
80
|
).build_or_load
|
|
74
81
|
end
|
|
75
82
|
|
|
@@ -85,6 +92,8 @@ module Mutineer
|
|
|
85
92
|
end
|
|
86
93
|
end
|
|
87
94
|
|
|
95
|
+
jobs = filter_since(jobs, source_map, config) if config.since
|
|
96
|
+
|
|
88
97
|
# C3: 7a writes mutineer_mutant*.rb into each source dir (so require_relative
|
|
89
98
|
# resolves). A SIGKILL'd child skips the tempfile's ensure-unlink, orphaning
|
|
90
99
|
# it. `ensure` is unreliable vs SIGKILL, so the PARENT sweeps each source dir
|
|
@@ -96,9 +105,10 @@ module Mutineer
|
|
|
96
105
|
strategy = config.strategy
|
|
97
106
|
results =
|
|
98
107
|
begin
|
|
108
|
+
framework = config.framework
|
|
99
109
|
bare = WorkerPool.new(config.jobs).run(jobs) do |subject, mutation|
|
|
100
110
|
run(mutation, source_file: subject.file, coverage_map: coverage_map,
|
|
101
|
-
subject: subject, strategy: strategy, rails: config.rails)
|
|
111
|
+
subject: subject, strategy: strategy, rails: config.rails, framework: framework)
|
|
102
112
|
end
|
|
103
113
|
# The bare Results carry only status (Subjects hold live AST nodes that
|
|
104
114
|
# do not marshal); reattach subject+mutation in the parent, in order.
|
|
@@ -110,6 +120,22 @@ module Mutineer
|
|
|
110
120
|
[AggregateResult.new(results), source_map]
|
|
111
121
|
end
|
|
112
122
|
|
|
123
|
+
# --since: keep only jobs whose mutation lands on a line changed since the git
|
|
124
|
+
# ref. Composes with coverage selection (it only narrows the job list; each
|
|
125
|
+
# surviving mutant still goes through Runner.run's coverage check). A file with
|
|
126
|
+
# no changed lines (absent from the diff) contributes no jobs. Line is computed
|
|
127
|
+
# exactly as Runner.run does, from the already-read source in source_map.
|
|
128
|
+
def self.filter_since(jobs, source_map, config)
|
|
129
|
+
changed = ChangedLines.for(ref: config.since, files: config.sources,
|
|
130
|
+
project_root: config.project_root)
|
|
131
|
+
jobs.select do |subject, mutation|
|
|
132
|
+
source = source_map[subject.file]
|
|
133
|
+
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
|
134
|
+
abs = File.expand_path(subject.file, config.project_root)
|
|
135
|
+
changed.fetch(abs, []).include?(line)
|
|
136
|
+
end
|
|
137
|
+
end
|
|
138
|
+
|
|
113
139
|
# For each test file, the directory to add to $LOAD_PATH so its
|
|
114
140
|
# `require "test_helper"` (or spec_helper) resolves: the nearest ancestor
|
|
115
141
|
# holding that helper, plus the file's own dir as a fallback.
|
|
@@ -131,6 +157,17 @@ module Mutineer
|
|
|
131
157
|
end.uniq
|
|
132
158
|
end
|
|
133
159
|
|
|
160
|
+
# #7: when --rails is on and RAILS_ENV is unset, default it to "test" (and
|
|
161
|
+
# say so) before the app boots — otherwise it boots development and nothing
|
|
162
|
+
# is measured. An explicitly-set RAILS_ENV is always respected.
|
|
163
|
+
def self.ensure_rails_env(config)
|
|
164
|
+
return unless config.rails
|
|
165
|
+
return unless ENV["RAILS_ENV"].nil? || ENV["RAILS_ENV"].empty?
|
|
166
|
+
|
|
167
|
+
ENV["RAILS_ENV"] = "test"
|
|
168
|
+
warn "[mutineer] RAILS_ENV was unset; defaulting to 'test' for --rails."
|
|
169
|
+
end
|
|
170
|
+
|
|
134
171
|
def self.sweep_orphans(dirs)
|
|
135
172
|
dirs.each do |dir|
|
|
136
173
|
Dir.glob(File.join(dir, "mutineer_mutant*.rb")).each do |f|
|
|
@@ -140,7 +177,7 @@ module Mutineer
|
|
|
140
177
|
end
|
|
141
178
|
|
|
142
179
|
def self.run(mutation, source_file:, coverage_map: nil, subject: nil, strategy: "reload",
|
|
143
|
-
timeout: Isolation::DEFAULT_TIMEOUT, rails: false)
|
|
180
|
+
timeout: Isolation::DEFAULT_TIMEOUT, rails: false, framework: "minitest")
|
|
144
181
|
source = File.read(source_file)
|
|
145
182
|
mutated = mutation.apply(source)
|
|
146
183
|
|
|
@@ -165,7 +202,7 @@ module Mutineer
|
|
|
165
202
|
else
|
|
166
203
|
Isolation.apply_whole_file(mutated, source_file)
|
|
167
204
|
end
|
|
168
|
-
|
|
205
|
+
TestRunners.for(framework).run(abs_tests)
|
|
169
206
|
end
|
|
170
207
|
end
|
|
171
208
|
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../minitest_integration"
|
|
4
|
+
|
|
5
|
+
module Mutineer
|
|
6
|
+
module TestRunners
|
|
7
|
+
# Uniform wrapper over the existing MinitestIntegration impl so the runner
|
|
8
|
+
# selection (TestRunners.for) has one method shape across frameworks.
|
|
9
|
+
# MinitestIntegration stays the implementation — all its behaviour (autorun
|
|
10
|
+
# neutralisation, Runnable.reset, load, Minitest.run -> 0/1) is preserved.
|
|
11
|
+
module Minitest
|
|
12
|
+
def self.run(test_files) = MinitestIntegration.run(test_files)
|
|
13
|
+
end
|
|
14
|
+
end
|
|
15
|
+
end
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "stringio"
|
|
4
|
+
|
|
5
|
+
module Mutineer
|
|
6
|
+
# Raised when the target project asks for a framework whose gem isn't present.
|
|
7
|
+
# rspec is NOT a Mutineer dependency — it must come from the project's bundle.
|
|
8
|
+
class FrameworkUnavailable < StandardError; end
|
|
9
|
+
|
|
10
|
+
module TestRunners
|
|
11
|
+
# Child-process-only RSpec runner, mirroring MinitestIntegration's contract:
|
|
12
|
+
# run the given spec files and return 0 (all passed) / 1 (any failure).
|
|
13
|
+
#
|
|
14
|
+
# rspec-core is required LAZILY here, never at load time, so Mutineer keeps
|
|
15
|
+
# zero runtime gem deps; a missing rspec raises a clear Mutineer error.
|
|
16
|
+
#
|
|
17
|
+
# No `rescue` around the run itself: Isolation.run's fork block is the single
|
|
18
|
+
# exception boundary (any exception there -> exit 2), keeping the 0/1 contract.
|
|
19
|
+
module RSpec
|
|
20
|
+
# `spec_files` is one path or an Array of paths (coverage selection passes
|
|
21
|
+
# the covering subset). All are loaded+run in a single RSpec invocation.
|
|
22
|
+
def self.run(spec_files)
|
|
23
|
+
require_rspec!
|
|
24
|
+
|
|
25
|
+
# rspec/autorun (if a spec_helper required it) installs an at_exit run;
|
|
26
|
+
# neutralise it like Minitest.autorun so it never double-fires.
|
|
27
|
+
::RSpec::Core::Runner.disable_autorun!
|
|
28
|
+
|
|
29
|
+
# Drop any RSpec world/configuration inherited across runs in one process
|
|
30
|
+
# (e.g. successive forks of a booted parent) so examples never accumulate.
|
|
31
|
+
::RSpec.reset
|
|
32
|
+
|
|
33
|
+
# Silence RSpec's formatter output (and any stray puts/deprecations) so it
|
|
34
|
+
# never pollutes Mutineer's report streams. The formatter writes to the
|
|
35
|
+
# passed IO; $stdout/$stderr are redirected to catch everything else.
|
|
36
|
+
sink = StringIO.new
|
|
37
|
+
orig_out = $stdout
|
|
38
|
+
orig_err = $stderr
|
|
39
|
+
$stdout = sink
|
|
40
|
+
$stderr = sink
|
|
41
|
+
begin
|
|
42
|
+
status = ::RSpec::Core::Runner.run(["--no-color", *Array(spec_files)], sink, sink)
|
|
43
|
+
ensure
|
|
44
|
+
$stdout = orig_out
|
|
45
|
+
$stderr = orig_err
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
status.zero? ? 0 : 1
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def self.require_rspec!
|
|
52
|
+
require "rspec/core"
|
|
53
|
+
rescue LoadError
|
|
54
|
+
raise Mutineer::FrameworkUnavailable,
|
|
55
|
+
"framework 'rspec' requested but rspec is not available; " \
|
|
56
|
+
"add rspec to the project under test (its bundle), then retry"
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
end
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "test_runners/minitest"
|
|
4
|
+
require_relative "test_runners/rspec"
|
|
5
|
+
|
|
6
|
+
module Mutineer
|
|
7
|
+
# Picks the test-framework runner. Each runner responds to `.run(files) -> 0/1`
|
|
8
|
+
# (0 = all passed, 1 = any failure) and is called only inside a forked child.
|
|
9
|
+
module TestRunners
|
|
10
|
+
def self.for(framework)
|
|
11
|
+
case framework
|
|
12
|
+
when "rspec" then RSpec
|
|
13
|
+
when "minitest", nil, "" then Minitest
|
|
14
|
+
else
|
|
15
|
+
raise Mutineer::ConfigError, "unknown framework #{framework.inspect} (expected: minitest, rspec)"
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
data/lib/mutineer/version.rb
CHANGED
data/lib/mutineer/worker_pool.rb
CHANGED
|
@@ -10,7 +10,7 @@ module Mutineer
|
|
|
10
10
|
# SAME ORDER as `items` regardless of finish order, so verdicts are identical to
|
|
11
11
|
# a serial run (R4) and downstream output is stable.
|
|
12
12
|
#
|
|
13
|
-
# The block is
|
|
13
|
+
# The block is run inside the child via `yield(*items[i])`; whatever it
|
|
14
14
|
# returns (a Result) is the marshaled payload. Per-mutant timeout is handled one
|
|
15
15
|
# level down by Isolation (KTD2) — the pool adds no separate wall clock.
|
|
16
16
|
class WorkerPool
|
|
@@ -70,43 +70,48 @@ module Mutineer
|
|
|
70
70
|
return
|
|
71
71
|
end
|
|
72
72
|
wr.close
|
|
73
|
-
running[pid] = [idx, rd]
|
|
73
|
+
running[pid] = [idx, rd, +""] # idx, read end, accumulated bytes
|
|
74
74
|
end
|
|
75
75
|
end
|
|
76
76
|
|
|
77
|
-
#
|
|
78
|
-
#
|
|
79
|
-
#
|
|
77
|
+
# Drain pipes with IO.select and reap a child only on EOF (#4). The old code
|
|
78
|
+
# reaped first and read after — but a child whose payload exceeds the OS pipe
|
|
79
|
+
# buffer (~64KB) blocks on `write` before it can exit, so it was never reaped
|
|
80
|
+
# and the pool deadlocked. Reading concurrently keeps the pipe drained so the
|
|
81
|
+
# child can finish and exit; EOF means it closed its write end (done writing).
|
|
82
|
+
# We waitpid only OUR known pids (R6: never wait2(-1), which would steal the
|
|
83
|
+
# host suite's children).
|
|
80
84
|
def reap(results, running)
|
|
81
85
|
return if running.empty?
|
|
82
86
|
|
|
83
87
|
loop do
|
|
84
|
-
running.
|
|
85
|
-
|
|
86
|
-
|
|
88
|
+
rds = running.values.map { |v| v[1] }
|
|
89
|
+
ready, = IO.select(rds, nil, nil)
|
|
90
|
+
ready.each do |rd|
|
|
91
|
+
pid, (idx, _io, buf) = running.find { |_, v| v[1].equal?(rd) }
|
|
92
|
+
chunk = rd.read_nonblock(65_536, exception: false)
|
|
93
|
+
next if chunk == :wait_readable
|
|
87
94
|
|
|
88
|
-
|
|
95
|
+
if chunk.nil? # EOF: child closed wr; it is done writing and exiting
|
|
96
|
+
rd.close
|
|
97
|
+
Process.waitpid(pid) # reap the now-finished child (no zombie)
|
|
98
|
+
running.delete(pid)
|
|
99
|
+
results[idx] = decode(buf)
|
|
100
|
+
return
|
|
101
|
+
end
|
|
102
|
+
buf << chunk
|
|
89
103
|
end
|
|
90
|
-
sleep 0.005
|
|
91
104
|
end
|
|
92
105
|
end
|
|
93
106
|
|
|
94
|
-
def
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
# R6: a partial/garbage Marshal stream (dead worker) must not crash the
|
|
103
|
-
# pool — degrade to an error Result.
|
|
104
|
-
begin
|
|
105
|
-
Marshal.load(data)
|
|
106
|
-
rescue StandardError => e
|
|
107
|
-
Result.error("worker result unreadable: #{e.class}: #{e.message}")
|
|
108
|
-
end
|
|
109
|
-
end
|
|
107
|
+
def decode(data)
|
|
108
|
+
return Result.error("worker produced no result") if data.empty?
|
|
109
|
+
|
|
110
|
+
# R6: a partial/garbage Marshal stream (dead worker) must not crash the
|
|
111
|
+
# pool — degrade to an error Result.
|
|
112
|
+
Marshal.load(data)
|
|
113
|
+
rescue StandardError => e
|
|
114
|
+
Result.error("worker result unreadable: #{e.class}: #{e.message}")
|
|
110
115
|
end
|
|
111
116
|
end
|
|
112
117
|
end
|
data/lib/mutineer.rb
CHANGED
|
@@ -8,8 +8,10 @@ require_relative "mutineer/mutation"
|
|
|
8
8
|
require_relative "mutineer/project"
|
|
9
9
|
require_relative "mutineer/result"
|
|
10
10
|
require_relative "mutineer/coverage_map"
|
|
11
|
+
require_relative "mutineer/changed_lines"
|
|
11
12
|
require_relative "mutineer/isolation"
|
|
12
13
|
require_relative "mutineer/minitest_integration"
|
|
14
|
+
require_relative "mutineer/test_runners"
|
|
13
15
|
require_relative "mutineer/mutators/base"
|
|
14
16
|
require_relative "mutineer/mutators/arithmetic"
|
|
15
17
|
require_relative "mutineer/mutators/comparison"
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: mutineer
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.6.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- David Teren
|
|
@@ -52,6 +52,7 @@ files:
|
|
|
52
52
|
- README.md
|
|
53
53
|
- bin/mutineer
|
|
54
54
|
- lib/mutineer.rb
|
|
55
|
+
- lib/mutineer/changed_lines.rb
|
|
55
56
|
- lib/mutineer/cli.rb
|
|
56
57
|
- lib/mutineer/config.rb
|
|
57
58
|
- lib/mutineer/coverage_map.rb
|
|
@@ -74,6 +75,9 @@ files:
|
|
|
74
75
|
- lib/mutineer/result.rb
|
|
75
76
|
- lib/mutineer/runner.rb
|
|
76
77
|
- lib/mutineer/subject.rb
|
|
78
|
+
- lib/mutineer/test_runners.rb
|
|
79
|
+
- lib/mutineer/test_runners/minitest.rb
|
|
80
|
+
- lib/mutineer/test_runners/rspec.rb
|
|
77
81
|
- lib/mutineer/version.rb
|
|
78
82
|
- lib/mutineer/worker_pool.rb
|
|
79
83
|
homepage: https://github.com/davidteren/mutineer
|