mutaruby 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0fb14367f23f77c599ae74ba8f36f1d5e0e71c73ab8fa8267378569a3a359655
4
- data.tar.gz: 036b55cb7754d44886e7056f0d63dda5725a0caf7037dc8eff50d49bf7df73fb
3
+ metadata.gz: 9b140915c577e60f56e81a1bec2d530a3791a82794bcb1e68de429f53fd5074d
4
+ data.tar.gz: 55cd843abdd670e427fc4a76b891cd75503623b160bc30dce514c90308966de4
5
5
  SHA512:
6
- metadata.gz: 718044deb537125a73dfc0864aeea698594711196ecf12ba16655fac80e6fa23e6c5ef2ef478d360b270da7f3264e7fccc7d8f5bff2bf88e2863cbe61e937d8c
7
- data.tar.gz: 7fb23d15489c6dae51343866c850e7e4a9ccf049f966e8521fa20fbe92a3eaca1f52bde4014a5387151d1aa36febbc44c92c20708e83fbf22e573dbddbd09f9e
6
+ metadata.gz: 1e6660338016f557eb61693a675c7fb672c8ed3f04bff6858412936ed470f4479278e9bc86da140bedfd7434141ba9699395d6bf9fa84831d18d14e4cd1c472f
7
+ data.tar.gz: 3b5aed66653a95656ef93303e76d981065cf68ed88116e3a4ffdedb097f706052189acb3b70457c1f06c617d49e135f3b3e530d9f7556e878b0dd5ea1a8c6e7f
data/CHANGELOG.md CHANGED
@@ -3,6 +3,37 @@
3
3
  All notable changes to this project are documented here. The format follows
4
4
  [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses [Semantic Versioning](https://semver.org/).
5
5
 
6
+ ## [0.3.0] - 2026-10-12
7
+
8
+ ### Added
9
+
10
+ - `--coverage`: coverage-based test selection. The baseline run records, with Ruby's Coverage API and the test
11
+ framework's per-test hooks, which tests execute which lines; each mutant then runs only those tests (Minitest and
12
+ RSpec), and falls back to the whole command for load-time code, unreached lines and other test commands. The
13
+ summary and the JSON report show how many mutants ran a subset.
14
+ - `--subject`: select methods by their place in the code (`Foo#bar`, `Foo.bar`, `Foo#*`, `Foo.*`, `Foo` with its
15
+ nested namespaces, `Foo*`).
16
+ - Eight operators: `block-removal`, `method-swap`, `element-removal`, `regexp-never-match`, `yield-removal`,
17
+ `super-removal`, `when-removal`, `rescue-removal`.
18
+ - `--unparser`: render rebuilt nodes (`element-removal`) with the MIT-licensed `unparser` gem, an optional extra.
19
+ Text splicing, with its self-check, stays the default.
20
+
21
+ ### Fixed
22
+
23
+ - An interrupted run starts no new mutant and waits for its workers to finish before it removes their sandboxes.
24
+ - A test run interrupted while mutaruby waits for it is stopped (with its process tree) before the interrupt goes
25
+ on, so its output file and sandbox are free to be removed.
26
+ - On Windows, a forcibly stopped run is given time to exit for good before its sandbox is removed (the directory
27
+ stays locked until then). A sandbox that still cannot be removed no longer hides the real error (Ctrl-C, a
28
+ failing baseline); removal is also retried when the directory is not yet empty.
29
+ - With `--coverage`, Ruby processes started by the tests themselves (a test that runs another suite) are neither
30
+ recorded nor filtered, and launcher processes (`bundle exec`, `rake`) do not overwrite the map.
31
+
32
+ ### Changed
33
+
34
+ - The README's comparison credits mutant as the mature reference implementation and states that mutaruby is an
35
+ independent, clean-room implementation that shares no code with it.
36
+
6
37
  ## [0.2.0] - 2026-10-12
7
38
 
8
39
  ### Added
@@ -71,6 +102,7 @@ First release.
71
102
  reason, ambiguity and staleness checks, `--strict-ignore` and `--suggest-ignore`.
72
103
  - Human, JSON and JUnit XML reports, a mutation score and `--fail-under` for CI.
73
104
 
105
+ [0.3.0]: https://github.com/Atlas-Authorization/mutaruby/releases/tag/v0.3.0
74
106
  [0.2.0]: https://github.com/Atlas-Authorization/mutaruby/releases/tag/v0.2.0
75
107
  [0.1.1]: https://github.com/Atlas-Authorization/mutaruby/releases/tag/v0.1.1
76
108
  [0.1.0]: https://github.com/Atlas-Authorization/mutaruby/releases/tag/v0.1.0
data/README.md CHANGED
@@ -5,10 +5,10 @@
5
5
 
6
6
  Free, MIT-licensed mutation testing for Ruby.
7
7
 
8
- mutaruby is mutation-tested with itself. In version 0.2.0, all 3,812 mutants of its own code are accounted for:
9
- 3,806 are killed by its tests (521 of them by timeout), and the other 6 are true equivalents, each listed in
10
- [`.mutaruby-ignore.yml`](.mutaruby-ignore.yml) with a reason you can check. `bundle exec rake mutate` reproduces the
11
- run; it fails on any surviving mutant or stale ignore entry.
8
+ mutaruby is mutation-tested with itself. In version 0.3.0, all 4,968 mutants of its own code are accounted for:
9
+ 4,962 are killed by its tests (573 of them by timeout), and the other 6 are true equivalents, each listed in
10
+ [`.mutaruby-ignore.yml`](.mutaruby-ignore.yml) with a reason you can check. `bundle exec rake mutate` (or the sharded
11
+ `Self-mutation` workflow) reproduces the run; it fails on any surviving mutant or stale ignore entry.
12
12
 
13
13
  mutaruby makes small, deliberate changes to your code, one at a time, and runs your test command against each change.
14
14
  If your tests still pass, the change (the *mutant*) **survived**, and that points to code your tests do not really check.
@@ -43,6 +43,7 @@ The survivor above tells you that no test checks a total of exactly 100.
43
43
  - [Quickstart](#quickstart)
44
44
  - [CLI reference](#cli-reference)
45
45
  - [Mutation operators](#mutation-operators)
46
+ - [Coverage-based test selection](#coverage-based-test-selection)
46
47
  - [Equivalent mutants: the ignore file](#equivalent-mutants-the-ignore-file)
47
48
  - [Using it in CI](#using-it-in-ci)
48
49
  - [Output formats](#output-formats)
@@ -138,6 +139,8 @@ Positional arguments are files or globs relative to the project root. The defaul
138
139
  | `-t`, `--test COMMAND` | **Required.** Shell command that runs your tests: run with `/bin/sh -c` on Linux and macOS, with `cmd.exe` on Windows. Exit 0 means the mutant survived, non-zero means killed. |
139
140
  | `-C`, `--root DIR` | Project root (default: the current directory). This is the directory copied into each sandbox, and the command runs there. |
140
141
  | `-j`, `--jobs N` | Parallel workers (default: the number of CPUs). Each worker has its own copy of the project. |
142
+ | `--coverage` | Run, for each mutant, only the tests that execute its lines (Minitest and RSpec). See [Coverage-based test selection](#coverage-based-test-selection). |
143
+ | `--unparser` | Render rebuilt nodes (the `element-removal` operator) with the [`unparser`](https://github.com/mbj/unparser) gem instead of from the source text. Needs `gem install unparser`. |
141
144
  | `--timeout SECONDS` | Per-mutant time limit. A mutant that exceeds it (an infinite loop, say) is killed together with every process it started and counted as **killed, by timeout**. The default is `baseline time x factor + 5` seconds. |
142
145
  | `--timeout-factor F` | The factor for the default timeout (default: 3). |
143
146
  | `--copy-exclude GLOB` | Do not copy matching paths into the sandboxes (repeatable). `.git` and `coverage` are never copied. Matches against the relative path and against the base name, e.g. `--copy-exclude node_modules --copy-exclude 'log/*'`. |
@@ -150,6 +153,7 @@ The command gets these environment variables: `MUTARUBY=1`, and while a mutant r
150
153
  | Option | Meaning |
151
154
  |---|---|
152
155
  | `--since REF` | Only mutate files changed since a git ref: committed, staged and unstaged changes plus new untracked files (`git diff --name-only REF` and `git ls-files --others --exclude-standard`), intersected with the file arguments. |
156
+ | `--subject PATTERN` | Only mutants in these methods (repeatable): `Foo#bar` (instance method), `Foo.bar` (singleton method), `Foo#*` / `Foo.*` (all of one kind), `Foo` or `Foo::Bar` (every method of that namespace and of the namespaces nested in it), `Foo*` (a glob over namespaces). |
153
157
  | `--method PATTERN` | Only mutants inside matching methods (repeatable). The pattern is a glob matched against the qualified name (`Pricing#discount` for an instance method, `Pricing.build` for a singleton method) or the bare name (`discount`). `Pricing#*` selects every instance method of `Pricing`. |
154
158
  | `--exclude-method PATTERN` | Skip mutants inside matching methods (repeatable). |
155
159
  | `--line SPEC` | Only mutants starting on these lines (repeatable). `SPEC` is `[FILE:]LINE` or `[FILE:]FROM-TO`, e.g. `lib/a.rb:10-20` or `42`. |
@@ -237,10 +241,56 @@ mutant's, are dropped before anything runs. `--list-operators` prints this list.
237
241
  | `op-assign` | swaps compound assignments: `+=` and `-=`, `*=` and `/=`, `\|=` and `&=`, `<<=` and `>>=` | `i += 1` to `i -= 1` |
238
242
  | `or-assign` | `\|\|=` becomes `=` | `@x \|\|= load` to `@x = load` |
239
243
  | `and-assign` | `&&=` becomes `=` | `a &&= b` to `a = b` |
244
+ | `block-removal` | a call loses its block | `list.map { \|x\| f(x) }` to `list.map` |
245
+ | `method-swap` | a method becomes its counterpart: `first`/`last`, `min`/`max`, `min_by`/`max_by`, `any?`/`all?`, `select`/`reject` (and `filter`), `upcase`/`downcase`, `floor`/`ceil`, `succ`/`pred`, `keys`/`values`, `start_with?`/`end_with?`, `take`/`drop`, `take_while`/`drop_while`, `push`/`unshift`, `pop`/`shift` | `items.first` to `items.last` |
246
+ | `element-removal` | one element of a bracketed array or braced hash removed (literals with 2 to 6 elements) | `[a, b]` to `[b]` and `[a]` |
247
+ | `regexp-never-match` | a regexp literal becomes one that matches nothing | `/\A\d+\z/` to `/(?!)/` |
248
+ | `yield-removal` | `yield` becomes `nil` | `yield(item)` to `nil` |
249
+ | `super-removal` | `super` becomes `nil` | `super(name)` to `nil` |
250
+ | `when-removal` | one `when` branch of a `case` removed | |
251
+ | `rescue-removal` | one `rescue` clause removed | |
240
252
 
241
253
  Strings inside interpolations, heredocs, `%w[]` arrays and the arguments of `require`-style declarations are not
242
254
  mutated, and neither are symbols used as hash keys (`a: 1`).
243
255
 
256
+ ## Coverage-based test selection
257
+
258
+ Most mutants are reached by only a few tests, yet by default every mutant runs the whole test command. With
259
+ `--coverage`, mutaruby first finds out which tests execute which lines, then runs each mutant against just those
260
+ tests:
261
+
262
+ ```sh
263
+ mutaruby -t "bundle exec rspec" --coverage
264
+ mutaruby -t "bundle exec rake test" --coverage
265
+ ```
266
+
267
+ How it works:
268
+
269
+ 1. The baseline run (which has to pass anyway) is made with a small probe loaded into the test process
270
+ (`RUBYLIB` plus `RUBYOPT=-rmutaruby_probe`). The probe uses Ruby's `Coverage` API and the test framework's
271
+ per-test hooks to record, for every test, the lines of the project it executes. Tests are identified as
272
+ `Class#test_name` for Minitest and by the example id (`./spec/a_spec.rb[1:2]`) for RSpec.
273
+ 2. For each mutant, the tests that execute any of its lines are written to a file, and the run loads the probe in
274
+ selection mode, which tells Minitest or RSpec to run only those tests. The test command itself is unchanged.
275
+ 3. A mutant runs the **whole** command instead when one of its lines ran while the code was being loaded (a class
276
+ body, a constant, a method definition line), when no test reaches its lines, or when the baseline recorded no
277
+ Minitest or RSpec tests at all (mutaruby then prints a warning).
278
+
279
+ What it saves depends on the suite. Measured on GitHub Actions (`script/benchmark-coverage.rb`, 4 workers), with the
280
+ same verdicts both ways:
281
+
282
+ | Project | Mutants | Whole test command | `--coverage` |
283
+ |---|---|---|---|
284
+ | mutaruby's own unit suite (a 1/25 sample of its mutants) | 199 | 3,788 s | 2,276 s (1.7x faster) |
285
+ | `test/fixtures/calc` (5 tests, Minitest or RSpec) | 43 | 13 s | 13 s |
286
+
287
+ A small suite gains nothing: starting Ruby costs more than its tests. The larger and slower the suite, and the fewer
288
+ tests reach each line, the bigger the win.
289
+
290
+ The summary shows how many mutants ran a subset, and the JSON report has, per mutant, the number of tests it ran
291
+ (`tests`, or `null` for the whole command). Selection assumes that tests are independent of each other: a test that
292
+ only passes after another one has run can make a mutant look killed.
293
+
244
294
  ## Equivalent mutants: the ignore file
245
295
 
246
296
  `.mutaruby-ignore.yml` (or the file given to `--ignore-file`) is a YAML list. Each entry names one **mutation site** and
@@ -342,17 +392,21 @@ survivors as failed tests.
342
392
 
343
393
  ## How it compares with mutant
344
394
 
345
- [mutant](https://github.com/mbj/mutant) is the established mutation tester for Ruby, and it is more mature and more
346
- capable than mutaruby: it has many more operators, semantic knowledge of Ruby's core library, selection of only the
347
- tests that cover each subject, and years of use on large code bases. It is commercially licensed (free for open
348
- source projects).
395
+ [mutant](https://github.com/mbj/mutant) is the mature reference implementation of mutation testing for Ruby, and the
396
+ tool mutaruby is most often compared with. It is more capable: many more operators, semantic knowledge of Ruby's core
397
+ library, precise subject and test selection, and years of use on large code bases. It is distributed under its own
398
+ commercial licence (free for open source projects).
399
+
400
+ **mutaruby shares no code with mutant.** It is an independent, clean-room implementation, written from general,
401
+ publicly known ideas about mutation testing (operators, coverage-based test selection, equivalent-mutant handling),
402
+ and it is MIT-licensed. Its only runtime dependency is `parser`; `unparser`, an MIT-licensed gem, is an optional
403
+ extra for `--unparser`.
349
404
 
350
405
  mutaruby aims to be **MIT-licensed and simple**:
351
406
 
352
407
  - It works with **any test command** (rspec, minitest, rake, a Makefile target), because the oracle is just an exit
353
- status. There is no framework integration to configure, and no per-test selection either, so it is slower than
354
- mutant on big suites unless you point the command at the relevant tests.
355
- - It is small enough to read in an afternoon, and it has no dependency besides `parser`.
408
+ status. With `--coverage` it also runs only the tests that reach each mutant, for Minitest and RSpec suites.
409
+ - It is small enough to read in an afternoon.
356
410
  - It runs each mutant in a **separate process** in a **copy of the project**, so it cannot leave your working tree
357
411
  modified.
358
412
 
@@ -419,7 +473,8 @@ mutaruby works the same way on Windows, with these differences:
419
473
 
420
474
  - On Windows, processes that a test run leaves running after it exits normally are not stopped (see
421
475
  [Windows notes](#windows-notes)).
422
- - The test command is run as is for every mutant; mutaruby does not work out which tests cover which code.
476
+ - Per-test selection (`--coverage`) understands Minitest and RSpec; any other command runs in full for every mutant.
477
+ Selection assumes the tests do not depend on each other's order or side effects.
423
478
  - The parser grammar is chosen to match the running Ruby, up to the newest grammar the installed `parser` gem knows.
424
479
  Files that `parser` cannot read (for example syntax newer than it supports) are skipped with a warning.
425
480
  - Mutants are syntactic. Some are equivalent by nature; that is what the ignore file is for.
data/lib/mutaruby/cli.rb CHANGED
@@ -71,6 +71,8 @@ module Mutaruby
71
71
  op.on('-j', '--jobs N', Integer, "Parallel workers (default: #{Etc.nprocessors})") { |v| o.jobs = v }
72
72
  op.on('--timeout SECONDS', Float, 'Per-mutant timeout (default: baseline time x factor + 5s)') { |v| o.timeout = v }
73
73
  op.on('--timeout-factor F', Float, 'Multiplier on the baseline time for the default timeout (default: 3)') { |v| o.timeout_factor = v }
74
+ op.on('--coverage', 'Run only the tests that cover each mutant (Minitest and RSpec; mapped in the baseline run)') { o.coverage = true }
75
+ op.on('--unparser', 'Render rebuilt nodes with the unparser gem (element-removal; needs gem install unparser)') { o.renderer = :unparser }
74
76
  op.on('--copy-exclude GLOB', 'Do not copy matching paths into the sandboxes (repeatable; .git and coverage are always skipped)') { |v| o.copy_excludes += [v] }
75
77
  op.separator ''
76
78
  op.separator 'Selection:'
@@ -79,6 +81,7 @@ module Mutaruby
79
81
  op.on('--exclude-method PATTERN', 'Skip mutants inside matching methods (repeatable)') { |v| o.exclude_methods += [v] }
80
82
  op.on('--line SPEC', 'Only mutants on these lines: [FILE:]LINE or [FILE:]FROM-TO (repeatable)') { |v| o.lines += [Selection.parse_line_spec(v)] }
81
83
  op.on('--exclude-line SPEC', 'Skip mutants on these lines (repeatable)') { |v| o.exclude_lines += [Selection.parse_line_spec(v)] }
84
+ op.on('--subject PATTERN', 'Only mutants in these methods: "Foo#bar", "Foo.bar", "Foo#*", "Foo" (with nested namespaces), "Foo*" (repeatable)') { |v| o.subjects += [v] }
82
85
  op.on('--operators LIST', Array, 'Only these operators (comma-separated; globs allowed, e.g. condition-*)') { |v| @include_ops += v }
83
86
  op.on('--exclude-operators LIST', Array, 'Skip these operators (comma-separated; globs allowed)') { |v| @exclude_ops += v }
84
87
  op.on('--shard I/N', 'Run only shard I of N (1-based) of the selected mutants') { |v| o.shard = Selection.parse_shard(v) }
@@ -52,11 +52,14 @@ module Mutaruby
52
52
  chdir: dir, in: File::NULL, out: out, err: out, **@platform.spawn_options)
53
53
  waiter = Process.detach(pid)
54
54
  @lock.synchronize { @live[pid] = waiter }
55
- timed_out = waiter.join(timeout).nil?
56
- # A timed-out run, and (where the platform allows) anything a finished run left behind, must not outlive it.
57
- kill_group(pid, waiter) if timed_out || @platform.stop_leftovers?
58
- st = waiter.value
59
- @lock.synchronize { @live.delete(pid) }
55
+ begin
56
+ timed_out = wait(pid, waiter, timeout)
57
+ # A timed-out run, and (where the platform allows) anything a finished run left behind, must not outlive it.
58
+ kill_group(pid, waiter) if timed_out || @platform.stop_leftovers?
59
+ st = waiter.value
60
+ ensure
61
+ @lock.synchronize { @live.delete(pid) }
62
+ end
60
63
  status = if timed_out then :timeout
61
64
  elsif st.success? then :passed
62
65
  else :failed
@@ -64,6 +67,15 @@ module Mutaruby
64
67
  Result.new(status: status, exitstatus: st.exitstatus, seconds: monotonic - started)
65
68
  end
66
69
 
70
+ # True when the run timed out. Interrupted while waiting (Ctrl-C), the run is stopped before the interrupt goes on:
71
+ # its output file, its sandbox and its process group must not outlive it.
72
+ def wait(pid, waiter, timeout)
73
+ waiter.join(timeout).nil?
74
+ rescue Interrupt
75
+ kill_group(pid, waiter)
76
+ raise
77
+ end
78
+
67
79
  # Bundler exports the ORIGINAL Gemfile path to child processes. Left as is, `bundle exec` in the sandbox would
68
80
  # resolve the Gemfile (and a `gemspec` line's lib/ directory) from the user's checkout and test the unmutated code.
69
81
  # The per-mutant variables are always set or cleared explicitly, so a mutaruby run nested inside another one's
@@ -86,6 +98,7 @@ module Mutaruby
86
98
  deadline = monotonic + @kill_grace
87
99
  sleep POLL while monotonic < deadline && @platform.alive?(pid, waiter)
88
100
  @platform.stop_tree(pid, graceful: false)
101
+ @platform.settle(waiter, @kill_grace)
89
102
  end
90
103
 
91
104
  def tail(bytes)
@@ -0,0 +1,51 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'json'
4
+
5
+ module Mutaruby
6
+ # Which tests execute which lines of the project, recorded in the baseline run by the probe
7
+ # (lib/mutaruby/probe/mutaruby_probe.rb) through Ruby's Coverage API. Used to run, for each mutant, only the tests
8
+ # that reach its lines.
9
+ class CoverageMap
10
+ PROBE_DIR = File.expand_path('probe', __dir__).freeze
11
+
12
+ # Environment that loads the probe into the test process, plus `extra` (the probe's own variables). RUBYLIB is a
13
+ # path list (spaces are fine there); RUBYOPT names only the feature, so no path has to survive its splitting.
14
+ def self.probe_env(extra)
15
+ lib = [PROBE_DIR, ENV.fetch('RUBYLIB', '')].reject(&:empty?).join(File::PATH_SEPARATOR)
16
+ opt = [ENV.fetch('RUBYOPT', ''), '-rmutaruby_probe'].reject(&:empty?).join(' ')
17
+ { 'RUBYLIB' => lib, 'RUBYOPT' => opt }.merge(extra)
18
+ end
19
+
20
+ def self.load(path)
21
+ new(JSON.parse(File.read(path, encoding: 'UTF-8')))
22
+ end
23
+
24
+ attr_reader :framework
25
+
26
+ # data: { "framework" => "minitest" | "rspec" | nil, "tests" => { id => { file => [line, ...] } },
27
+ # "load" => { file => [line, ...] } }
28
+ def initialize(data)
29
+ @framework = data['framework']
30
+ @tests = Hash.new { |h, k| h[k] = [] }
31
+ data['tests'].each do |id, files|
32
+ files.each { |file, lines| lines.each { |l| @tests[[file, l]] << id } }
33
+ end
34
+ @load = data['load'].flat_map { |file, lines| lines.map { |l| [file, l] } }
35
+ end
36
+
37
+ # True when the run recorded tests of a supported framework.
38
+ def usable?
39
+ !@framework.nil? && @tests.any?
40
+ end
41
+
42
+ # The ids of the tests that execute any of `lines` of `file`, sorted; nil when the whole suite has to run: a line
43
+ # that ran while the code was loaded (a class body, a method definition), or a line no test reaches.
44
+ def tests_for(file, lines)
45
+ return nil if lines.any? { |l| @load.include?([file, l]) }
46
+
47
+ ids = lines.flat_map { |l| @tests.fetch([file, l], []) }.uniq.sort
48
+ ids.empty? ? nil : ids
49
+ end
50
+ end
51
+ end
@@ -3,6 +3,7 @@
3
3
  require_relative 'parsing'
4
4
  require_relative 'mutation'
5
5
  require_relative 'splicer'
6
+ require_relative 'render'
6
7
 
7
8
  module Mutaruby
8
9
  # Walks a file's AST (the `parser` gem) and produces every mutant of it. Each mutant is ONE splice of the original
@@ -46,7 +47,24 @@ module Mutaruby
46
47
  'raise-removal' => 'raise X -> nil (also fail)',
47
48
  'op-assign' => 'swap compound assignment (+= <-> -=, *= <-> /=, |= <-> &=, <<= <-> >>=)',
48
49
  'or-assign' => 'a ||= b -> a = b',
49
- 'and-assign' => 'a &&= b -> a = b'
50
+ 'and-assign' => 'a &&= b -> a = b',
51
+ 'block-removal' => 'list.map { |x| f(x) } -> list.map (the call without its block)',
52
+ 'method-swap' => 'swap a method for its counterpart (first <-> last, min <-> max, select <-> reject, ...)',
53
+ 'element-removal' => '[a, b] -> [b], [a]; { k: 1, j: 2 } -> { j: 2 }, { k: 1 } (one element removed at a time)',
54
+ 'regexp-never-match' => '/pattern/ -> /(?!)/ (a regexp that matches nothing)',
55
+ 'yield-removal' => 'yield x -> nil',
56
+ 'super-removal' => 'super / super(x) -> nil',
57
+ 'when-removal' => 'one `when` branch of a case removed',
58
+ 'rescue-removal' => 'one `rescue` clause removed'
59
+ }.freeze
60
+
61
+ # method-swap pairs: each method and the one it is replaced with.
62
+ METHOD_SWAPS = {
63
+ first: :last, last: :first, min: :max, max: :min, min_by: :max_by, max_by: :min_by, any?: :all?, all?: :any?,
64
+ select: :reject, reject: :select, filter: :reject, upcase: :downcase, downcase: :upcase, floor: :ceil,
65
+ ceil: :floor, succ: :pred, pred: :succ, keys: :values, values: :keys, start_with?: :end_with?,
66
+ end_with?: :start_with?, take: :drop, drop: :take, take_while: :drop_while, drop_while: :take_while,
67
+ push: :unshift, unshift: :push, pop: :shift, shift: :pop
50
68
  }.freeze
51
69
 
52
70
  SWAPS = {
@@ -75,8 +93,11 @@ module Mutaruby
75
93
 
76
94
  # path: file path for messages; rel: path relative to the project root (stored on the mutants).
77
95
  # operators: nil (all) or a list of operator ids.
78
- def initialize(source, path:, rel: path, operators: nil)
96
+ # renderer: :text (default) builds replacements from the source text; any other value (:unparser) renders the
97
+ # operators that rebuild a node (element-removal) with the `unparser` gem instead, which must be installed.
98
+ def initialize(source, path:, rel: path, operators: nil, renderer: :text)
79
99
  @src = source
100
+ @renderer = renderer
80
101
  @path = path
81
102
  @rel = rel
82
103
  @operators = operators
@@ -226,10 +247,8 @@ module Mutaruby
226
247
  when :true then add(n.loc.expression, 'false', 'literal-boolean')
227
248
  when :false then add(n.loc.expression, 'true', 'literal-boolean')
228
249
  when :nil then add(n.loc.expression, 'true', 'literal-nil')
229
- when :array
230
- add(n.loc.expression, '[]', 'literal-array') if n.children.any? && n.loc.begin&.source == '['
231
- when :hash
232
- add(n.loc.expression, '{}', 'literal-hash') if n.children.any? && n.loc.begin&.source == '{'
250
+ when :array then visit_collection(n, '[', '[]', 'literal-array')
251
+ when :hash then visit_collection(n, '{', '{}', 'literal-hash')
233
252
  when :irange, :erange
234
253
  # an endless range (a..) means the same with either operator
235
254
  add(n.loc.operator, n.type == :irange ? '...' : '..', 'range-swap') unless n.children[1].nil?
@@ -242,6 +261,39 @@ module Mutaruby
242
261
  when :and_asgn then add(n.loc.operator, '=', 'and-assign')
243
262
  when :begin, :kwbegin then visit_body(n)
244
263
  when :def, :defs then visit_def(n)
264
+ when :block, :numblock then add(n.loc.expression, src_of(n.children.first), 'block-removal')
265
+ when :regexp then add(n.loc.expression, '/(?!)/', 'regexp-never-match')
266
+ when :yield then add(n.loc.expression, 'nil', 'yield-removal')
267
+ when :super, :zsuper then add(n.loc.expression, 'nil', 'super-removal')
268
+ when :case then remove_clauses(n.children[1...-1], 'when-removal')
269
+ when :rescue then remove_clauses(n.children[1...-1], 'rescue-removal')
270
+ end
271
+ end
272
+
273
+ # A bracketed array / braced hash: emptied, and each element removed in turn.
274
+ def visit_collection(n, open, empty, op)
275
+ return unless n.loc.begin&.source == open
276
+
277
+ add(n.loc.expression, empty, op) if n.children.any?
278
+ remove_elements(n) if n.children.size.between?(2, MAX_ARGS)
279
+ end
280
+
281
+ # Each clause (a `when`, a `rescue`) deleted in turn; the validity check drops what would not parse.
282
+ def remove_clauses(clauses, op)
283
+ clauses.each { |c| add(c.loc.expression, '', op) }
284
+ end
285
+
286
+ # [a, b, c] -> [b, c], [a, c], [a, b] (and the same for the pairs of a braced hash).
287
+ def remove_elements(n)
288
+ kids = n.children
289
+ kids.each_index do |i|
290
+ rest = kids.reject.with_index { |_, j| j == i }
291
+ if @renderer == :text
292
+ add(kids.first.loc.expression.join(kids.last.loc.expression), rest.map { |k| src_of(k) }.join(', '),
293
+ 'element-removal')
294
+ else
295
+ add(n.loc.expression, Render.unparse(src_of(n), i), 'element-removal')
296
+ end
245
297
  end
246
298
  end
247
299
 
@@ -250,6 +302,7 @@ module Mutaruby
250
302
  SWAPS.each do |op, table|
251
303
  add(n.loc.selector, table[meth].to_s, op) if table.key?(meth) && args.size == 1
252
304
  end
305
+ add(n.loc.selector, METHOD_SWAPS[meth].to_s, 'method-swap') if recv && METHOD_SWAPS.key?(meth)
253
306
  add(n.loc.expression, src_of(recv), 'negation-removal') if meth == :!
254
307
  add(n.loc.expression, src_of(recv), 'unary-minus-removal') if meth == :-@
255
308
  add(n.loc.dot, '.', 'safe-navigation') if n.type == :csend
@@ -298,12 +351,12 @@ module Mutaruby
298
351
  end
299
352
 
300
353
  def visit_int(n)
301
- v = n.children.first
354
+ v, = n.children
302
355
  [v + 1, v - 1, 0, 1].each { |x| add(n.loc.expression, x.to_s, 'literal-int') }
303
356
  end
304
357
 
305
358
  def visit_float(n)
306
- v = n.children.first
359
+ v, = n.children
307
360
  add(n.loc.expression, '0.0', 'literal-float') unless v.zero?
308
361
  add(n.loc.expression, (v + 1.0).to_s, 'literal-float') if v.finite?
309
362
  end
@@ -317,7 +370,8 @@ module Mutaruby
317
370
  close = n.loc.end if open
318
371
  return unless close
319
372
 
320
- fill = n.children.first.empty? ? 'mutaruby' : ''
373
+ text, = n.children
374
+ fill = text.empty? ? 'mutaruby' : ''
321
375
  add(n.loc.expression, "#{open.source}#{fill}#{close.source}", 'literal-string')
322
376
  end
323
377
 
@@ -20,6 +20,9 @@ module Mutaruby
20
20
  # After a run ends, whatever is left of its process group is stopped too.
21
21
  def stop_leftovers? = true
22
22
 
23
+ # Nothing to wait for: the group is gone once signalled with SIGKILL.
24
+ def settle(_waiter, _seconds); end
25
+
23
26
  # graceful: SIGTERM, otherwise SIGKILL, to the process group led by pid.
24
27
  def stop_tree(pid, graceful:)
25
28
  Process.kill(graceful ? 'TERM' : 'KILL', -pid)
@@ -57,6 +60,12 @@ module Mutaruby
57
60
  # already have reused its id for an unrelated process.
58
61
  def stop_leftovers? = false
59
62
 
63
+ # After taskkill /F, give the process up to `seconds` to exit for good: until then Windows keeps its files and
64
+ # working directory (the sandbox) locked.
65
+ def settle(waiter, seconds)
66
+ waiter.join(seconds)
67
+ end
68
+
60
69
  def stop_tree(pid, graceful:)
61
70
  @system.call('taskkill', '/T', *(graceful ? [] : ['/F']), '/PID', pid.to_s)
62
71
  end
@@ -0,0 +1,169 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Loaded into the user's test process by mutaruby (RUBYLIB=<this directory>, RUBYOPT=-rmutaruby_probe) when
4
+ # coverage-based test selection is on. It depends on nothing from mutaruby itself and does nothing unless one of its
5
+ # environment variables is set:
6
+ #
7
+ # MUTARUBY_COVERAGE_OUT=FILE record, for every test, the lines of the project it executes (Ruby's Coverage API),
8
+ # and write them to FILE as JSON when the process exits
9
+ # MUTARUBY_COVERAGE_ROOT=DIR the project directory; only files below it are recorded
10
+ # MUTARUBY_SELECT=FILE run only the tests whose ids FILE lists, one per line
11
+ #
12
+ # The test command may start several Ruby processes (bundle exec -> rake -> ruby running the tests). Each loads the
13
+ # probe, but only the process that loads a test framework records or selects: launchers write nothing. That process
14
+ # then removes the variables, so Ruby processes started by the tests themselves run normally.
15
+ #
16
+ # Test ids: "Class#test_method" for Minitest, the example id ("./spec/a_spec.rb[1:2]") for RSpec.
17
+ module MutarubyProbe
18
+ class << self
19
+ def setup
20
+ @out = ENV.fetch('MUTARUBY_COVERAGE_OUT', '')
21
+ @select = ENV.fetch('MUTARUBY_SELECT', '')
22
+ @installed = []
23
+ return if @out.empty? && @select.empty?
24
+
25
+ @select.empty? ? start_recording : load_selection
26
+ Object.prepend(RequireHook)
27
+ end
28
+
29
+ def recording? = !@out.empty?
30
+
31
+ # --- recording -------------------------------------------------------------------------------------------------
32
+
33
+ def start_recording
34
+ require 'coverage'
35
+ require 'json'
36
+ root = ENV.fetch('MUTARUBY_COVERAGE_ROOT')
37
+ @root = "#{File.realpath(root)}/"
38
+ @tests = Hash.new { |h, id| h[id] = {} }
39
+ @load = {}
40
+ Coverage.start(lines: true)
41
+ at_exit { write if @installed.any? }
42
+ end
43
+
44
+ # Lines run before a test starts (loading code, defining methods) count as load-time lines.
45
+ def test_started(id)
46
+ collect(@load)
47
+ @current = id
48
+ end
49
+
50
+ def test_finished
51
+ collect(@tests[@current])
52
+ end
53
+
54
+ def collect(into)
55
+ Coverage.result(stop: false, clear: true).each do |path, data|
56
+ rel = relative(path)
57
+ next unless rel
58
+
59
+ lines = data[:lines]
60
+ hit = lines.each_index.select { |i| lines[i].to_i.positive? }.map { |i| i + 1 }
61
+ into[rel] = (into.fetch(rel, []) | hit).sort
62
+ end
63
+ end
64
+
65
+ # Path relative to the project, or nil for files outside it. Coverage reports the path a file was loaded from; it
66
+ # is compared by its real path.
67
+ def relative(path)
68
+ real = real_path(path)
69
+ real.delete_prefix(@root) if real.start_with?(@root)
70
+ end
71
+
72
+ # Ruby records required files by their real path already; a file deleted since it was loaded keeps that path.
73
+ def real_path(path)
74
+ File.realpath(path)
75
+ rescue SystemCallError
76
+ path
77
+ end
78
+
79
+ # Lines run after the last test (at_exit and after_run hooks) count as load-time lines too.
80
+ def write
81
+ collect(@load)
82
+ File.write(@out, JSON.generate(framework: @installed.first, tests: @tests, load: @load))
83
+ end
84
+
85
+ # --- selection -------------------------------------------------------------------------------------------------
86
+
87
+ def load_selection
88
+ @selected = File.readlines(@select, chomp: true)
89
+ end
90
+
91
+ def selected?(id) = @selected.include?(id)
92
+
93
+ # --- frameworks ------------------------------------------------------------------------------------------------
94
+
95
+ def install_hooks
96
+ install_minitest if !@installed.include?('minitest') && defined?(::Minitest::Test)
97
+ install_rspec if !@installed.include?('rspec') && defined?(::RSpec) && ::RSpec.respond_to?(:configure)
98
+ end
99
+
100
+ # The first framework found: this is the test process. Its children must not record or select.
101
+ def claim
102
+ %w[MUTARUBY_COVERAGE_OUT MUTARUBY_COVERAGE_ROOT MUTARUBY_SELECT].each { |k| ENV.delete(k) }
103
+ end
104
+
105
+ def install_minitest
106
+ claim
107
+ @installed << 'minitest'
108
+ if recording?
109
+ ::Minitest::Test.prepend(MinitestRecording)
110
+ else
111
+ ::Minitest::Test.singleton_class.prepend(MinitestSelection)
112
+ end
113
+ end
114
+
115
+ def install_rspec
116
+ claim
117
+ @installed << 'rspec'
118
+ if recording?
119
+ ::RSpec.configure do |c|
120
+ c.around do |ex| # (an example hook)
121
+ MutarubyProbe.test_started(ex.id)
122
+ ex.run
123
+ ensure
124
+ MutarubyProbe.test_finished
125
+ end
126
+ end
127
+ else
128
+ ::RSpec.configure do |c|
129
+ c.define_derived_metadata do |meta|
130
+ meta[:mutaruby_selected] = true if MutarubyProbe.selected?("#{meta[:rerun_file_path]}[#{meta[:scoped_id]}]")
131
+ end
132
+ c.filter_run_including(mutaruby_selected: true)
133
+ end
134
+ end
135
+ end
136
+ end
137
+
138
+ # Re-checks for test frameworks after every require: they are loaded after this file.
139
+ module RequireHook
140
+ private
141
+
142
+ def require(name)
143
+ super
144
+ ensure
145
+ MutarubyProbe.install_hooks
146
+ end
147
+ end
148
+
149
+ module MinitestRecording
150
+ def before_setup
151
+ MutarubyProbe.test_started("#{self.class}##{name}")
152
+ super
153
+ end
154
+
155
+ def after_teardown
156
+ super
157
+ ensure
158
+ MutarubyProbe.test_finished
159
+ end
160
+ end
161
+
162
+ module MinitestSelection
163
+ def runnable_methods
164
+ super.select { |m| MutarubyProbe.selected?("#{name}##{m}") }
165
+ end
166
+ end
167
+ end
168
+
169
+ MutarubyProbe.setup
@@ -0,0 +1,28 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'error'
4
+
5
+ module Mutaruby
6
+ # Renders rebuilt nodes back to Ruby source with the MIT-licensed `unparser` gem (optional: `gem install unparser`).
7
+ # Text splicing stays the default; this path is used, with --unparser, for operators that rebuild a node. The
8
+ # rendered text is still spliced over the node's own range, so the splice self-check applies as usual.
9
+ module Render
10
+ class Unavailable < Error; end
11
+
12
+ module_function
13
+
14
+ def load!
15
+ require 'unparser'
16
+ rescue LoadError
17
+ raise Unavailable, '--unparser needs the unparser gem (gem install unparser)'
18
+ end
19
+
20
+ # `source` (an array or hash literal) without its element number `index`, rendered by unparser. The node is
21
+ # parsed again by unparser itself, so its AST is in the form unparser expects.
22
+ def unparse(source, index)
23
+ load!
24
+ node = Unparser.parse(source)
25
+ Unparser.unparse(node.updated(nil, node.children.reject.with_index { |_, j| j == index }))
26
+ end
27
+ end
28
+ end
@@ -43,6 +43,10 @@ module Mutaruby
43
43
  io.puts " survived: #{report.count(:survived)}"
44
44
  io.puts " equivalent: #{report.count(:equivalent)} (ignore file)"
45
45
  io.puts " score: #{score_text(report)} = killed / (mutants - equivalent)"
46
+ if report.coverage
47
+ subset = report.outcomes.count(&:tests)
48
+ io.puts " coverage: #{report.coverage}: #{subset} of #{report.outcomes.size} mutants ran only the tests covering them"
49
+ end
46
50
  io.puts format(' time: %.1fs (baseline %.2fs, timeout %ss, jobs %d)', report.seconds,
47
51
  report.baseline_seconds, report.timeout, report.jobs)
48
52
  end
@@ -50,12 +54,12 @@ module Mutaruby
50
54
  def json_data(report)
51
55
  {
52
56
  tool: 'mutaruby', version: VERSION, command: report.command,
53
- score: report.score&.round(4), complete: report.complete,
57
+ score: report.score&.round(4), complete: report.complete, coverage: report.coverage,
54
58
  counts: { mutants: report.outcomes.size, killed: report.killed, timeout: report.count(:timeout),
55
59
  survived: report.count(:survived), equivalent: report.count(:equivalent) },
56
60
  files: report.files, baseline_seconds: secs(report.baseline_seconds), timeout: report.timeout,
57
61
  mutants: report.outcomes.map do |o|
58
- o.mutation.to_h.merge(status: o.status, seconds: secs(o.seconds), ignore_reason: o.ignore&.reason)
62
+ o.mutation.to_h.merge(status: o.status, seconds: secs(o.seconds), ignore_reason: o.ignore&.reason, tests: o.tests)
59
63
  end,
60
64
  stale_ignores: report.stale.map { |e| e.to_h.compact.except(:index) },
61
65
  warnings: report.warnings
@@ -4,19 +4,21 @@ require_relative 'ignore_file'
4
4
  require_relative 'sandbox'
5
5
  require_relative 'command'
6
6
  require_relative 'selection'
7
+ require_relative 'coverage_map'
7
8
 
8
9
  module Mutaruby
9
10
  class BaselineFailed < Error; end
10
11
 
11
12
  # Generates the mutants, checks the baseline, runs every mutant and collects the verdicts.
12
13
  class Runner
13
- Outcome = Struct.new(:mutation, :status, :seconds, :ignore, keyword_init: true) do
14
+ # tests: how many tests the mutant ran with coverage-based selection; nil when it ran the whole command.
15
+ Outcome = Struct.new(:mutation, :status, :seconds, :ignore, :tests, keyword_init: true) do
14
16
  # :killed, :timeout (also a kill), :survived, :equivalent (survived, listed in the ignore file)
15
17
  def killed? = %i[killed timeout].include?(status)
16
18
  end
17
19
 
18
20
  Report = Struct.new(:root, :command, :files, :outcomes, :sources, :stale, :warnings, :baseline_seconds, :timeout,
19
- :jobs, :seconds, :complete, keyword_init: true) do
21
+ :jobs, :seconds, :complete, :coverage, keyword_init: true) do
20
22
  def count(status) = outcomes.count { |o| o.status == status }
21
23
  def killed = count(:killed) + count(:timeout)
22
24
 
@@ -29,10 +31,11 @@ module Mutaruby
29
31
 
30
32
  Options = Struct.new(:root, :command, :patterns, :since, :jobs, :timeout, :timeout_factor, :methods,
31
33
  :exclude_methods, :lines, :exclude_lines, :shard, :ignore_file, :operators, :copy_excludes,
32
- :progress, keyword_init: true) do
34
+ :progress, :subjects, :coverage, :renderer, keyword_init: true) do
33
35
  def self.build(**kw)
34
36
  new(root: '.', patterns: [], jobs: 1, timeout_factor: 3.0, methods: [], exclude_methods: [], lines: [],
35
- exclude_lines: [], copy_excludes: [], progress: false, **kw)
37
+ exclude_lines: [], copy_excludes: [], progress: false, subjects: [], coverage: false, renderer: :text,
38
+ **kw)
36
39
  end
37
40
  end
38
41
 
@@ -52,7 +55,7 @@ module Mutaruby
52
55
  sources = files.to_h { |f| [f, File.binread(File.join(@root, f)).force_encoding(Encoding::UTF_8)] }
53
56
  warnings = []
54
57
  all = files.flat_map do |f|
55
- g = Generator.new(sources[f], path: File.join(@root, f), rel: f, operators: @o.operators)
58
+ g = Generator.new(sources[f], path: File.join(@root, f), rel: f, operators: @o.operators, renderer: @o.renderer)
56
59
  ms = g.mutations
57
60
  warnings.concat(g.errors)
58
61
  ms
@@ -73,28 +76,45 @@ module Mutaruby
73
76
  p = plan
74
77
  first = Sandbox.new(@root, excludes: @o.copy_excludes)
75
78
  sandboxes << first
76
- base = cmd.run(first.dir, capture: true)
79
+ coverage_out = first.scratch('coverage.json')
80
+ base_env = @o.coverage ? CoverageMap.probe_env('MUTARUBY_COVERAGE_OUT' => coverage_out,
81
+ 'MUTARUBY_COVERAGE_ROOT' => first.dir) : {}
82
+ base = cmd.run(first.dir, capture: true, env: base_env)
77
83
  unless base.passed?
78
84
  raise BaselineFailed, "the test command fails on the UNMUTATED code (exit #{base.exitstatus}); every mutant " \
79
85
  "would count as killed, so nothing would be measured.\ncommand: #{@o.command}\n" \
80
86
  "--- output (tail) ---\n#{base.output}"
81
87
  end
82
88
  timeout = @o.timeout || ((base.seconds * @o.timeout_factor) + 5).ceil
89
+ p[:coverage] = coverage_map(coverage_out, p[:warnings]) if @o.coverage
83
90
  outcomes = execute(p, cmd, sandboxes, timeout)
84
91
  matched = outcomes.select { |o| o.status == :equivalent }.map(&:ignore)
85
92
  stale = p[:ignore] && p[:complete] ? p[:ignore].stale(matched, p[:files]) : []
86
93
  Report.new(root: @root, command: @o.command, files: p[:files], outcomes: outcomes, sources: p[:sources],
87
94
  stale: stale, warnings: p[:warnings], baseline_seconds: base.seconds, timeout: timeout,
88
- jobs: jobs, seconds: monotonic - t0, complete: p[:complete])
95
+ jobs: jobs, seconds: monotonic - t0, complete: p[:complete], coverage: p[:coverage]&.framework)
89
96
  rescue Interrupt
97
+ # Workers see @stop and start no new mutant; the runs in flight are killed; then the workers finish, before
98
+ # their sandboxes go.
99
+ @stop = true
90
100
  cmd.kill_all
101
+ @threads&.each(&:join)
91
102
  raise
92
103
  ensure
93
- sandboxes.each(&:cleanup)
104
+ remove(sandboxes, $!)
94
105
  end
95
106
 
96
107
  private
97
108
 
109
+ # Removes the sandboxes. When the run is already failing (an interrupt, a failing baseline), a sandbox that cannot
110
+ # be removed is left behind rather than replacing that error: on Windows a process that is still exiting keeps its
111
+ # directory locked for a moment.
112
+ def remove(sandboxes, failing)
113
+ sandboxes.each(&:cleanup)
114
+ rescue SystemCallError
115
+ raise unless failing
116
+ end
117
+
98
118
  def jobs
99
119
  [@o.jobs, 1].max
100
120
  end
@@ -102,7 +122,17 @@ module Mutaruby
102
122
  def selection
103
123
  Selection.new(root: @root, patterns: @o.patterns, since: @o.since, methods: @o.methods,
104
124
  exclude_methods: @o.exclude_methods, lines: @o.lines, exclude_lines: @o.exclude_lines,
105
- shard: @o.shard)
125
+ shard: @o.shard, subjects: @o.subjects)
126
+ end
127
+
128
+ # The coverage map recorded by the baseline run, or nil (with a warning) when no supported test framework ran.
129
+ def coverage_map(path, warnings)
130
+ map = File.exist?(path) ? CoverageMap.load(path) : nil
131
+ return map if map&.usable?
132
+
133
+ warnings << 'coverage: no Minitest or RSpec tests were recorded in the baseline run; every mutant runs the ' \
134
+ 'whole test command'
135
+ nil
106
136
  end
107
137
 
108
138
  def load_ignore
@@ -123,9 +153,9 @@ module Mutaruby
123
153
  sandboxes << Sandbox.new(@root, excludes: @o.copy_excludes) while sandboxes.size < n
124
154
  results = []
125
155
  lock = Mutex.new
126
- threads = sandboxes.map do |box|
156
+ @threads = sandboxes.map do |box|
127
157
  Thread.new do
128
- while (m = next_mutation(queue))
158
+ while !@stop && (m = next_mutation(queue))
129
159
  o = evaluate(m, box, plan, cmd, timeout)
130
160
  lock.synchronize do
131
161
  results << o
@@ -134,33 +164,33 @@ module Mutaruby
134
164
  end
135
165
  end
136
166
  end
137
- begin
138
- threads.each(&:join)
139
- rescue Interrupt
140
- threads.each(&:kill)
141
- raise
142
- end
167
+ @threads.each(&:join)
143
168
  @out.puts if @o.progress && results.any?
144
169
  results.sort_by { |o| o.mutation.id }
145
170
  end
146
171
 
147
172
  def next_mutation(queue)
148
- queue.pop(true)
173
+ queue.deq(true)
149
174
  rescue ThreadError
150
175
  nil
151
176
  end
152
177
 
153
178
  def evaluate(m, box, plan, cmd, timeout)
154
- r = box.with_file(m.file, m.text, plan[:sources][m.file]) do
155
- cmd.run(box.dir, timeout: timeout, env: { 'MUTARUBY_MUTANT_ID' => m.id.to_s, 'MUTARUBY_FILE' => m.file })
179
+ env = { 'MUTARUBY_MUTANT_ID' => m.id.to_s, 'MUTARUBY_FILE' => m.file }
180
+ tests = plan[:coverage]&.tests_for(m.file, m.lineno..(m.lineno + m.orig.count("\n")))
181
+ if tests
182
+ list = box.scratch('tests.txt')
183
+ File.write(list, tests.map { |t| "#{t}\n" }.join)
184
+ env.merge!(CoverageMap.probe_env('MUTARUBY_SELECT' => list))
156
185
  end
186
+ r = box.with_file(m.file, m.text, plan[:sources][m.file]) { cmd.run(box.dir, timeout: timeout, env: env) }
157
187
  entry = plan[:ignore]&.match(m)
158
188
  status = if r.timeout? then :timeout
159
189
  elsif !r.passed? then :killed
160
190
  elsif entry then :equivalent
161
191
  else :survived
162
192
  end
163
- Outcome.new(mutation: m, status: status, seconds: r.seconds, ignore: entry)
193
+ Outcome.new(mutation: m, status: status, seconds: r.seconds, ignore: entry, tests: tests&.size)
164
194
  end
165
195
 
166
196
  def progress(outcome)
@@ -17,7 +17,7 @@ module Mutaruby
17
17
  # process that is still exiting, or a virus scanner, can hold it for a moment).
18
18
  RETRIES = 10
19
19
  RETRY_DELAY = 0.1
20
- LOCKED = [Errno::EACCES, Errno::EBUSY, Errno::EPERM].freeze
20
+ LOCKED = [Errno::EACCES, Errno::EBUSY, Errno::EPERM, Errno::ENOTEMPTY].freeze
21
21
 
22
22
  # Runs the block, retrying while it raises one of the LOCKED errors.
23
23
  def self.retrying(tries: RETRIES, delay: RETRY_DELAY)
@@ -51,6 +51,11 @@ module Mutaruby
51
51
  File.join(@dir, rel)
52
52
  end
53
53
 
54
+ # A path for mutaruby's own files: next to the project copy, not inside it.
55
+ def scratch(name)
56
+ File.join(@base, name)
57
+ end
58
+
54
59
  # Writes `text` to `rel` inside the sandbox, yields, and restores the original bytes afterwards (also on error).
55
60
  def with_file(rel, text, original)
56
61
  target = path(rel)
@@ -34,7 +34,7 @@ module Mutaruby
34
34
 
35
35
  # lines / exclude_lines: LineSpec objects (see parse_line_spec). shard: [i, n] or nil.
36
36
  def initialize(root:, patterns: [], since: nil, methods: [], exclude_methods: [], lines: [], exclude_lines: [],
37
- shard: nil)
37
+ shard: nil, subjects: [])
38
38
  @root = File.expand_path(root)
39
39
  @patterns = patterns.empty? ? DEFAULT_PATTERNS : patterns
40
40
  @since = since
@@ -43,11 +43,13 @@ module Mutaruby
43
43
  @lines = lines
44
44
  @exclude_lines = exclude_lines
45
45
  @shard = shard
46
+ @subjects = subjects
46
47
  end
47
48
 
48
49
  # True when every mutant of each selected file is run (so stale ignore entries can be detected).
49
50
  def complete?
50
- @methods.empty? && @exclude_methods.empty? && @lines.empty? && @exclude_lines.empty? && @shard.nil?
51
+ @methods.empty? && @exclude_methods.empty? && @lines.empty? && @exclude_lines.empty? && @shard.nil? &&
52
+ @subjects.empty?
51
53
  end
52
54
 
53
55
  # Sorted paths of the matching files, relative to the root.
@@ -64,7 +66,8 @@ module Mutaruby
64
66
 
65
67
  def filter(mutations)
66
68
  kept = mutations.select do |m|
67
- (@methods.empty? || @methods.any? { |p| method_match?(p, m.method) }) &&
69
+ (@subjects.empty? || @subjects.any? { |s| subject_match?(s, m.method) }) &&
70
+ (@methods.empty? || @methods.any? { |p| method_match?(p, m.method) }) &&
68
71
  @exclude_methods.none? { |p| method_match?(p, m.method) } &&
69
72
  (@lines.empty? || @lines.any? { |s| s.cover?(m) }) &&
70
73
  @exclude_lines.none? { |s| s.cover?(m) }
@@ -83,6 +86,21 @@ module Mutaruby
83
86
  name ? File.fnmatch?(pattern, name) || File.fnmatch?(pattern, name.split(/[#.]/).last) : false
84
87
  end
85
88
 
89
+ # A subject names methods by their place in the code:
90
+ # "Foo#bar", "Foo.bar" one method (instance, singleton); globs allowed: "Foo#*", "Foo.*", "Foo#to_*"
91
+ # "Foo", "Foo::Bar" every method of that namespace and of the namespaces nested in it
92
+ # "Foo*" a glob over namespaces ("Foo*" matches Foo, FooBar and Foo::Baz)
93
+ def subject_match?(subject, name)
94
+ name ? subject_name_match?(subject, name) : false
95
+ end
96
+
97
+ def subject_name_match?(subject, name)
98
+ return File.fnmatch?(subject, name) if subject.match?(/[#.]/)
99
+
100
+ namespace = name.split(/[#.]/).first
101
+ namespace.start_with?("#{subject}::") || File.fnmatch?(subject, namespace)
102
+ end
103
+
86
104
  # Files changed relative to a git ref: committed, staged and unstaged edits, plus new untracked files.
87
105
  def changed_files
88
106
  git('diff', '--name-only', '--relative', @since, '--') + git('ls-files', '--others', '--exclude-standard')
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Mutaruby
4
- VERSION = '0.2.0'
4
+ VERSION = '0.3.0'
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: mutaruby
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Atlas Authorization
@@ -46,12 +46,15 @@ files:
46
46
  - lib/mutaruby.rb
47
47
  - lib/mutaruby/cli.rb
48
48
  - lib/mutaruby/command.rb
49
+ - lib/mutaruby/coverage_map.rb
49
50
  - lib/mutaruby/error.rb
50
51
  - lib/mutaruby/generator.rb
51
52
  - lib/mutaruby/ignore_file.rb
52
53
  - lib/mutaruby/mutation.rb
53
54
  - lib/mutaruby/parsing.rb
54
55
  - lib/mutaruby/platform.rb
56
+ - lib/mutaruby/probe/mutaruby_probe.rb
57
+ - lib/mutaruby/render.rb
55
58
  - lib/mutaruby/reporters.rb
56
59
  - lib/mutaruby/runner.rb
57
60
  - lib/mutaruby/sandbox.rb