mutineer 1.8.1 → 1.8.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +122 -0
- data/README.md +1 -1
- data/lib/mutineer/cli.rb +4 -1
- data/lib/mutineer/concern_blocks.rb +95 -24
- data/lib/mutineer/daemon_backend.rb +1 -1
- data/lib/mutineer/daemon_client.rb +55 -20
- data/lib/mutineer/daemon_server.rb +10 -9
- data/lib/mutineer/external_backend.rb +46 -5
- data/lib/mutineer/isolation.rb +6 -5
- data/lib/mutineer/job_plan.rb +3 -2
- data/lib/mutineer/mutation.rb +4 -3
- data/lib/mutineer/parser.rb +39 -0
- data/lib/mutineer/rails_worker_db.rb +10 -0
- data/lib/mutineer/redaction.rb +45 -17
- data/lib/mutineer/reload_guards.rb +564 -0
- data/lib/mutineer/reporter.rb +2 -1
- data/lib/mutineer/runner.rb +20 -5
- data/lib/mutineer/version.rb +1 -1
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 5b57ed76ead8c85effe91bba09f7b6570c6f7ae01ce0cfd6c32a608f5d2cbc0a
|
|
4
|
+
data.tar.gz: 24927a0b60b867e538591012c592a2e0accba0e2293e5add0f76bcc9c170601b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 44574f7ed61cbc3605f1fabb44dc73fd079f5879cb3a044cb8c24e855a29bc99c9bb258d4a58ffd0a67f52554654e547d90f87d239be637962483d75da5f7c0e
|
|
7
|
+
data.tar.gz: ba1701c0808e6a166402df1bfe7be18f938b970a2ee61da9bc66ee749fa8205c947eb872cabfe1f0ae962f885ffa79491d52e5718b80ac5cdf113ab4bcda1436
|
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,127 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [1.8.2] - 2026-11-10
|
|
10
|
+
|
|
11
|
+
### Fixed
|
|
12
|
+
|
|
13
|
+
- **Redaction no longer hides a password inside other words (#341).** A
|
|
14
|
+
form of the database password (raw, decoded or escaped) longer than 3
|
|
15
|
+
characters, made only of ASCII letters, digits and underscores, is now
|
|
16
|
+
hidden only as a whole word, like a short one. With
|
|
17
|
+
password `test`, the backtrace path `user_test.rb` stays readable. Such a
|
|
18
|
+
password inside a longer word is no longer hidden. The word `test` on its
|
|
19
|
+
own is still hidden, also right after an escape such as `%40` or `\n`, a
|
|
20
|
+
colour code, or a `-p` flag. That escape rule now covers short forms
|
|
21
|
+
too. A longer form with any
|
|
22
|
+
other character is still hidden wherever it appears.
|
|
23
|
+
- **Redaction keeps the host after a password with an `@` (#341).** In
|
|
24
|
+
`postgres://u:p@ss@h/db and p@ss`, only `p@ss` is hidden now. Before, the
|
|
25
|
+
host and the rest of the line were hidden too. One known limit: in a URL
|
|
26
|
+
other than DATABASE_URL, a password that holds an `@`, a host, a `/`, `:`,
|
|
27
|
+
`?` or `#`, and a space, shows the part after that `@` when the URL's own
|
|
28
|
+
host has nothing after it, or has an `@` in its path or query. #341 tracks
|
|
29
|
+
it.
|
|
30
|
+
- **Daemon shutdown no longer waits a second (#341).** A process the app
|
|
31
|
+
started could keep the daemon's error output open. Each quit or respawn of
|
|
32
|
+
`--daemon` then waited one second. Mutineer now stops waiting once the
|
|
33
|
+
daemon has exited and its output is read.
|
|
34
|
+
- **One line that Prism 2 rejects no longer skips every mutant in the file
|
|
35
|
+
(#392).** Prism 2.0 and later report an operator write to a bare constant
|
|
36
|
+
inside a method (`T += 1`, `T ||= 1`, `T &&= 1`) as a dynamic constant
|
|
37
|
+
assignment. Ruby's `parse.y` does the same, but Ruby 3.4 and 4.0 load such
|
|
38
|
+
a file with their own Prism 1. Mutineer kept a mutant only when the mutated
|
|
39
|
+
file parsed with no errors, so on Prism 2 every mutant of that file was
|
|
40
|
+
counted as `skipped (invalid)`. When Prism reports an error, Mutineer now
|
|
41
|
+
asks the running Ruby to compile the mutant, and skips it only when Ruby
|
|
42
|
+
cannot. This holds in the dry run, in process, with `--test-command`, on
|
|
43
|
+
`--daemon`, and for the redefine strategy's method snippet. A project gets
|
|
44
|
+
Prism 2 when its bundle loads it, for example
|
|
45
|
+
through minitest 6.1. On Prism 2, the mutants of such a file that were
|
|
46
|
+
skipped now get a verdict, so the score, the counts, and the JSON report
|
|
47
|
+
can change for those projects. Prism 1 runs do not change.
|
|
48
|
+
- **`--test-command` warns about a random test order (#338).** Mutineer
|
|
49
|
+
sets no seed for your command, so the clean run and each mutant run can
|
|
50
|
+
run the tests in different orders. An order-dependent suite can then fail
|
|
51
|
+
a mutant run for a reason the mutant did not cause, and that mutant
|
|
52
|
+
scores a false `killed`. When the clean run prints a random seed
|
|
53
|
+
(Minitest `Run options: --seed N`, RSpec `Randomized with seed N`) and
|
|
54
|
+
Mutineer finds no fixed seed, it now prints one warning with the fix for
|
|
55
|
+
that runner: put `SEED=1` before the `mutineer` command for Minitest, or
|
|
56
|
+
add `--order defined` to an RSpec command and keep `--order rand` out of
|
|
57
|
+
`SPEC_OPTS`, which RSpec reads after the command. The docs for
|
|
58
|
+
`--test-command` say the same. Mutineer does not set the seed itself, so scores do not
|
|
59
|
+
change. It cannot see a seed set in a wrapper script or `.rspec`, or a
|
|
60
|
+
Minitest `test_order`, so the warning can print when the order is
|
|
61
|
+
already fixed. #368 tracks these limits.
|
|
62
|
+
- **Reload places a hook-block mutant before the rest of its file runs
|
|
63
|
+
(#348).** Under `--strategy reload`, a placeable mutated method from a
|
|
64
|
+
concern's `included` or `prepended` block reached the classes that include
|
|
65
|
+
or prepend the concern only after the whole mutant file loaded. Methods
|
|
66
|
+
that reload cannot place still score `unplaceable`. A later line in the same
|
|
67
|
+
file that called the method while the file loaded, such as
|
|
68
|
+
`TOTAL = Host.new.doubled(2)`, still used the original method. Mutineer
|
|
69
|
+
now places the mutated method when the load reaches the hook call, before
|
|
70
|
+
the rest of the file runs, in-process and on `--daemon`. Such a mutant can
|
|
71
|
+
now be killed where it used to survive or score `ran_at_load`, so the
|
|
72
|
+
score of an affected file can change. A mutant that raises in such a
|
|
73
|
+
load-time call now scores `error`. The alias and kept-method checks
|
|
74
|
+
still score `error`. The kept-method check runs after the load, so a
|
|
75
|
+
method handle the file holds only while it loads does not count.
|
|
76
|
+
- **The Action creates the cache folder when `output` points into it
|
|
77
|
+
(#340).** On a fresh checkout, `output: .mutineer/pr.json` exited 2,
|
|
78
|
+
because `.mutineer/` did not exist yet. The CLI already created that
|
|
79
|
+
folder (#270), but the Action checks the path itself before it runs the
|
|
80
|
+
CLI. The Action now creates the cache folder when it is the missing parent
|
|
81
|
+
of `output`: `.mutineer` by default, the `cache_dir` from `.mutineer.yml`,
|
|
82
|
+
or a `--cache-dir` in `extra-args` (or a prefix of it down to `--cac`, such as
|
|
83
|
+
`--cache`). Any other missing folder still exits 2.
|
|
84
|
+
- **Reload runs a Rails class body a second time without errors (#378,
|
|
85
|
+
#384).** Reload loads the mutant file over the class that is already
|
|
86
|
+
defined, so its body runs again. A model that declares an `enum`, and a
|
|
87
|
+
class that calls `skip_callback` (a controller's `skip_before_action` or
|
|
88
|
+
`allow_unauthenticated_access` calls it), raised on that second run, and
|
|
89
|
+
every mutant in the file scored `error`. While the mutant file loads, an
|
|
90
|
+
enum the class had before the load now skips Rails' conflict check, and
|
|
91
|
+
`skip_callback` does not raise for a callback the first run already
|
|
92
|
+
removed: one named by a method the class has, such as a callback from the
|
|
93
|
+
parent class or an included concern. A skip that names no such method or
|
|
94
|
+
parent callback still raises. Both checks run as usual while the tests
|
|
95
|
+
run. This holds on `--daemon` and with `--strategy reload`.
|
|
96
|
+
- **Reload no longer runs validators and block callbacks twice (#390).**
|
|
97
|
+
The second run of a class body added a second copy of each `validates`,
|
|
98
|
+
each `validate` block, and each callback written as a block or a lambda,
|
|
99
|
+
such as `before_save do ... end`. Rails removes a repeated callback only
|
|
100
|
+
when it names a method. A test that counts errors or side effects then
|
|
101
|
+
failed for every mutant in the file, so those mutants scored `killed`
|
|
102
|
+
falsely. After the mutant file loads, Mutineer now keeps one copy of each:
|
|
103
|
+
the new one, in the place of the old one, so the order of callbacks does
|
|
104
|
+
not change. Entries from an included concern stay, because the reload does
|
|
105
|
+
not run the concern's block again. When Mutineer cannot tell which old
|
|
106
|
+
entry a new one repeats, it keeps both, as before (#413). This holds on
|
|
107
|
+
`--daemon` and with `--strategy reload`. With eager loading on, a mutant
|
|
108
|
+
of a parent class file still adds a callback back to each subclass that
|
|
109
|
+
skipped it (#414).
|
|
110
|
+
**Score change:** a Rails model with such a test can now show survivors
|
|
111
|
+
that were false kills before, so its score can go down.
|
|
112
|
+
- **Concern mutants load when the test app loads lazily (#383).** With
|
|
113
|
+
`eager_load` off (the Rails test default), the mutant's load was the first
|
|
114
|
+
use of the concern, so Rails loaded the original file inside it. The
|
|
115
|
+
original `included` block then came back, and the mutant's block raised
|
|
116
|
+
`MultipleIncludedBlocks`. Mutineer now loads the original file first, through
|
|
117
|
+
the app's autoloader, so the mutant's block replaces it.
|
|
118
|
+
- **The pending-migration check no longer trips `shared_databases` (#385).**
|
|
119
|
+
`rails/test_help` checks every database for pending migrations when the
|
|
120
|
+
tests load, shared ones too. A parallel `--daemon` run then reported the
|
|
121
|
+
shared database as used and scored every mutant `error`. Mutant forks of a
|
|
122
|
+
run with `shared_databases` now turn that check off. The clean check, which
|
|
123
|
+
runs before any mutant, still makes it.
|
|
124
|
+
- **No Rails deprecation warning from concern mutants (#387).** Placing a
|
|
125
|
+
mutant inside an `included` block looked at every module. Rails 8.2 keeps a
|
|
126
|
+
deprecated constant as a module that warns on every call, so each such
|
|
127
|
+
mutant printed `LoadInterlockAwareMonitor is deprecated`. Mutineer now skips
|
|
128
|
+
that module.
|
|
129
|
+
|
|
9
130
|
## [1.8.1] - 2026-11-02
|
|
10
131
|
|
|
11
132
|
### Fixed
|
|
@@ -1861,6 +1982,7 @@ Rails hardening + CI batch (issues #8–#13), all verified Rails-free.
|
|
|
1861
1982
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
1862
1983
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
1863
1984
|
|
|
1985
|
+
[1.8.2]: https://github.com/davidteren/mutineer/releases/tag/v1.8.2
|
|
1864
1986
|
[1.8.1]: https://github.com/davidteren/mutineer/releases/tag/v1.8.1
|
|
1865
1987
|
[1.8.0]: https://github.com/davidteren/mutineer/releases/tag/v1.8.0
|
|
1866
1988
|
[1.7.0]: https://github.com/davidteren/mutineer/releases/tag/v1.7.0
|
data/README.md
CHANGED
|
@@ -140,7 +140,7 @@ This gap is common in rushed tests and tests written by AI agents.
|
|
|
140
140
|
| Alternative backend | Changes to step 4 |
|
|
141
141
|
|---|---|
|
|
142
142
|
| `--daemon` | Boots Rails once in a persistent process; isolates each worker's test database |
|
|
143
|
-
| `--test-command` | Runs tests in another Ruby subprocess; writes mutants to the real file, then restores it |
|
|
143
|
+
| `--test-command` | Runs tests in another Ruby subprocess; writes mutants to the real file, then restores it. Run the tests in a fixed order (`SEED=1 mutineer run ...` for Minitest, `--order defined` for RSpec, with no `--order rand` in `SPEC_OPTS`) |
|
|
144
144
|
|
|
145
145
|
| At a glance | |
|
|
146
146
|
|---|---|
|
data/lib/mutineer/cli.rb
CHANGED
|
@@ -83,7 +83,10 @@ module Mutineer
|
|
|
83
83
|
--test-command CMD Run the target suite in the app's own runtime as a
|
|
84
84
|
subprocess (for apps on Ruby < 3.4). CMD must contain
|
|
85
85
|
%{files}. Scrubs Mutineer Ruby PATH pins; set RAILS_ENV
|
|
86
|
-
on the mutineer command (not as KEY=val inside CMD)
|
|
86
|
+
on the mutineer command (not as KEY=val inside CMD).
|
|
87
|
+
CMD must run tests in a fixed order: put SEED=1
|
|
88
|
+
before the mutineer command for Minitest, or add
|
|
89
|
+
--order defined to an RSpec CMD
|
|
87
90
|
--daemon Boot the app ONCE in a persistent daemon and fork per
|
|
88
91
|
mutant, with per-worker DB isolation so --jobs N is safe
|
|
89
92
|
under Rails (needs --rails/--boot; not with --test-command).
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "reload_guards"
|
|
4
|
+
|
|
3
5
|
module Mutineer
|
|
4
6
|
# Lets the reload strategy load a concern file a second time (#256).
|
|
5
7
|
#
|
|
@@ -15,10 +17,13 @@ module Mutineer
|
|
|
15
17
|
# ActiveSupport does not run the block again in a class that already includes
|
|
16
18
|
# the concern, so that class keeps what the original block defined. For a def
|
|
17
19
|
# in the block, the mutant file also passes the mutated def to
|
|
18
|
-
# {.store_hook_def}, just before the `included` or `prepended` call
|
|
19
|
-
# {.
|
|
20
|
-
#
|
|
21
|
-
#
|
|
20
|
+
# {.store_hook_def}, just before the `included` or `prepended` call. Under
|
|
21
|
+
# {.load_mutant} with a `hook_def`, {.store_hook_def} then calls
|
|
22
|
+
# {.place_hook_def} at once, which loads only that def into each class that
|
|
23
|
+
# has the method (#297). It runs before the rest of the file, so a later
|
|
24
|
+
# line of the file that calls the method while the file loads gets the
|
|
25
|
+
# mutant (#348). Nothing else in the block runs again. Discovery decides
|
|
26
|
+
# which defs can be placed this way (see {Subject#unplaceable?}).
|
|
22
27
|
#
|
|
23
28
|
# Call it only in a forked child that loads the mutant. Without
|
|
24
29
|
# ActiveSupport, {.replace} and {.forget} do nothing; {.store_hook_def} and
|
|
@@ -46,23 +51,49 @@ module Mutineer
|
|
|
46
51
|
end
|
|
47
52
|
end
|
|
48
53
|
|
|
49
|
-
# Loads the mutant file at `path` over `source_file`, through {.replace}
|
|
50
|
-
#
|
|
51
|
-
#
|
|
52
|
-
#
|
|
54
|
+
# Loads the mutant file at `path` over `source_file`, through {.replace}
|
|
55
|
+
# inside {ReloadGuards.during_load}, so Rails macros in the class body can
|
|
56
|
+
# run a second time (#378, #383, #384). The load assigns each of the file's
|
|
57
|
+
# constants a second time, so Ruby would warn twice per constant, per
|
|
58
|
+
# mutant (#364). Warnings are off for the load only, and back on before
|
|
59
|
+
# the tests run.
|
|
60
|
+
#
|
|
61
|
+
# With a `hook_def`, the file's {.store_hook_def} call places the mutated
|
|
62
|
+
# def during the load (#348). A load that never reached that call raises.
|
|
63
|
+
# When a handle to the original is live at placement, the kept-handle
|
|
64
|
+
# check runs again after the load, so a handle the file holds only while
|
|
65
|
+
# it loads does not count. Placement leaves no other way to reach the
|
|
66
|
+
# original, so no handle seen then means none after the load.
|
|
53
67
|
#
|
|
54
68
|
# @param source_file [String] the file the mutant replaces.
|
|
55
69
|
# @param path [String] the mutant file to load.
|
|
70
|
+
# @param hook_def [Hash{Symbol => Object}, nil] the original def's `line`,
|
|
71
|
+
# `name` and `singleton`, from {JobPlan.hook_mutant}.
|
|
72
|
+
# @raise [RuntimeError] from {.place_hook_def}, so the child scores `error`.
|
|
56
73
|
# @return [Object] what `load` returns.
|
|
57
|
-
def self.load_mutant(source_file, path)
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
$VERBOSE =
|
|
74
|
+
def self.load_mutant(source_file, path, hook_def: nil)
|
|
75
|
+
@hook_target = hook_def && { source_file: source_file, **hook_def }
|
|
76
|
+
@hook_def = nil
|
|
77
|
+
@handle_at_placement = false
|
|
78
|
+
loaded = ReloadGuards.during_load(source_file, path) do
|
|
79
|
+
replace(source_file) do
|
|
80
|
+
verbose, $VERBOSE = $VERBOSE, nil
|
|
81
|
+
begin
|
|
82
|
+
load path
|
|
83
|
+
ensure
|
|
84
|
+
$VERBOSE = verbose
|
|
85
|
+
end
|
|
64
86
|
end
|
|
65
87
|
end
|
|
88
|
+
if @hook_target
|
|
89
|
+
# Still set: the load did not reach store_hook_def, so this raises.
|
|
90
|
+
place_hook_def(**@hook_target)
|
|
91
|
+
elsif @handle_at_placement
|
|
92
|
+
refuse_retained_hook_def(source_file: source_file, line: hook_def[:line], name: hook_def[:name])
|
|
93
|
+
end
|
|
94
|
+
loaded
|
|
95
|
+
ensure
|
|
96
|
+
@hook_target = nil
|
|
66
97
|
end
|
|
67
98
|
|
|
68
99
|
# Removes every stored `included` and `prepended` block that the source
|
|
@@ -93,13 +124,23 @@ module Mutineer
|
|
|
93
124
|
|
|
94
125
|
# Keeps the mutated hook-block def for {.place_hook_def}. The mutant file
|
|
95
126
|
# calls it in the concern's body, just before the hook call, so the block has
|
|
96
|
-
# the same constant scope as the hook block (#297).
|
|
127
|
+
# the same constant scope as the hook block (#297). Under {.load_mutant}
|
|
128
|
+
# with a `hook_def`, it places the def now, before the rest of the file
|
|
129
|
+
# runs (#348).
|
|
97
130
|
#
|
|
98
|
-
# @yield the mutated def, run
|
|
131
|
+
# @yield the mutated def, run with `class_eval` in each owner.
|
|
132
|
+
# @raise [RuntimeError] from {.place_hook_def}.
|
|
99
133
|
# @return [nil]
|
|
100
134
|
def self.store_hook_def(&block)
|
|
101
135
|
@hook_def = block
|
|
102
|
-
|
|
136
|
+
return unless @hook_target
|
|
137
|
+
|
|
138
|
+
target = @hook_target
|
|
139
|
+
# A handle may be one the file drops before its load ends: load_mutant checks again then.
|
|
140
|
+
@handle_at_placement = method_handle_at?(real_path(target[:source_file]), target[:line], target[:name])
|
|
141
|
+
place_hook_def(**target, check_retained: false)
|
|
142
|
+
# Cleared only after placement, so a raise the file rescues still errors after the load.
|
|
143
|
+
@hook_target = nil
|
|
103
144
|
end
|
|
104
145
|
|
|
105
146
|
# Takes the def {.store_hook_def} kept, clears it, and runs it with
|
|
@@ -112,12 +153,14 @@ module Mutineer
|
|
|
112
153
|
# @param line [Integer] the original def's first line.
|
|
113
154
|
# @param name [Symbol] the method name.
|
|
114
155
|
# @param singleton [Boolean] true for a class method.
|
|
156
|
+
# @param check_retained [Boolean] false when the caller checks for a kept
|
|
157
|
+
# method handle itself, after the load ({.load_mutant}).
|
|
115
158
|
# @raise [RuntimeError] when no def was kept (the mutant file did not reach
|
|
116
159
|
# its hook call), or when a module keeps the original under another name
|
|
117
160
|
# (see {.aliases}) or in a live method handle; the child then scores the
|
|
118
161
|
# mutant `error`.
|
|
119
162
|
# @return [Integer] how many owners got the mutated def.
|
|
120
|
-
def self.place_hook_def(source_file:, line:, name:, singleton:)
|
|
163
|
+
def self.place_hook_def(source_file:, line:, name:, singleton:, check_retained: true)
|
|
121
164
|
block = @hook_def
|
|
122
165
|
@hook_def = nil
|
|
123
166
|
where = "mutineer: could not place the mutant of #{name} (#{source_file}:#{line})"
|
|
@@ -127,14 +170,11 @@ module Mutineer
|
|
|
127
170
|
end
|
|
128
171
|
|
|
129
172
|
# Check before owner and alias lookup create temporary UnboundMethods.
|
|
130
|
-
|
|
131
|
-
raise "#{where}: the original is retained in a Method or UnboundMethod object, " \
|
|
132
|
-
"so the mutant scores error."
|
|
133
|
-
end
|
|
173
|
+
refuse_retained_hook_def(source_file: source_file, line: line, name: name) if check_retained
|
|
134
174
|
|
|
135
175
|
# ponytail: one ObjectSpace walk per mutant (about 11 ms with Rails loaded);
|
|
136
176
|
# index modules by method name if a large app makes it slow.
|
|
137
|
-
modules = ObjectSpace.each_object(Module).
|
|
177
|
+
modules = ObjectSpace.each_object(Module).reject { |mod| deprecation_proxy?(mod) }
|
|
138
178
|
found = owners(modules, source_file: source_file, line: line, name: name, singleton: singleton)
|
|
139
179
|
mod, other = aliases(modules, source_file: source_file, line: line, name: name, singleton: singleton).first
|
|
140
180
|
if mod
|
|
@@ -213,6 +253,21 @@ module Mutineer
|
|
|
213
253
|
method.original_name == original_name && def_line == line && !file.nil? && real_path(file) == path
|
|
214
254
|
end
|
|
215
255
|
|
|
256
|
+
# Raises when a live Method or UnboundMethod keeps the original definition
|
|
257
|
+
# (see {.retained_hook_def?}), so the child scores the mutant `error`.
|
|
258
|
+
#
|
|
259
|
+
# @param source_file [String] the file the mutant replaces.
|
|
260
|
+
# @param line [Integer] the original def's first line.
|
|
261
|
+
# @param name [Symbol] the original method name.
|
|
262
|
+
# @raise [RuntimeError] when such a handle is live.
|
|
263
|
+
# @return [nil]
|
|
264
|
+
def self.refuse_retained_hook_def(source_file:, line:, name:)
|
|
265
|
+
return unless retained_hook_def?(source_file: source_file, line: line, name: name)
|
|
266
|
+
|
|
267
|
+
raise "mutineer: could not place the mutant of #{name} (#{source_file}:#{line}): the original is " \
|
|
268
|
+
"retained in a Method or UnboundMethod object, so the mutant scores error."
|
|
269
|
+
end
|
|
270
|
+
|
|
216
271
|
# True when a live Method or UnboundMethod keeps the original definition.
|
|
217
272
|
# A wrapper can call that handle after the named method was replaced, so
|
|
218
273
|
# loading another def cannot make the wrapper test the mutant. A matching
|
|
@@ -248,6 +303,22 @@ module Mutineer
|
|
|
248
303
|
end
|
|
249
304
|
end
|
|
250
305
|
|
|
306
|
+
# True for Rails' `DeprecatedConstantProxy`: a Module that forwards every
|
|
307
|
+
# call to its target and prints a deprecation warning, so asking it for
|
|
308
|
+
# methods would print one per mutant (#387). `Kernel#class` reads its real
|
|
309
|
+
# class without forwarding.
|
|
310
|
+
#
|
|
311
|
+
# @param mod [Module] a module from the ObjectSpace walk.
|
|
312
|
+
# @return [Boolean]
|
|
313
|
+
def self.deprecation_proxy?(mod)
|
|
314
|
+
return false unless defined?(::ActiveSupport::Deprecation::DeprecatedConstantProxy)
|
|
315
|
+
|
|
316
|
+
(KERNEL_CLASS.bind_call(mod) <= ::ActiveSupport::Deprecation::DeprecatedConstantProxy) == true
|
|
317
|
+
end
|
|
318
|
+
|
|
319
|
+
# `Kernel#class`, unbound, so a proxy that redefines `class` cannot answer.
|
|
320
|
+
KERNEL_CLASS = ::Kernel.instance_method(:class)
|
|
321
|
+
|
|
251
322
|
# The file's real path, so a symlinked or relative path matches the path
|
|
252
323
|
# Ruby recorded when it loaded the file.
|
|
253
324
|
#
|
|
@@ -425,7 +425,7 @@ module Mutineer
|
|
|
425
425
|
# (fallback, not narrowed).
|
|
426
426
|
sel = coverage_map && !coverage_map.map.empty? && JobPlan.coverage_selection(subject.file, mutation, subject, source, coverage_map)
|
|
427
427
|
r =
|
|
428
|
-
if Parser.
|
|
428
|
+
if Parser.invalid?(mutated)
|
|
429
429
|
Result.skipped
|
|
430
430
|
elsif sel && sel[0] == :verdict
|
|
431
431
|
sel[1]
|
|
@@ -44,9 +44,13 @@ module Mutineer
|
|
|
44
44
|
# Seconds the daemon gets to boot the app and answer the handshake. A boot
|
|
45
45
|
# that never returns ends the run instead of hanging it (#101).
|
|
46
46
|
BOOT_TIMEOUT = 600
|
|
47
|
-
#
|
|
48
|
-
#
|
|
47
|
+
# Most seconds close_io waits for the stderr drain to finish. The drain
|
|
48
|
+
# ends at EOF, or once the daemon has exited and the pipe is empty, so
|
|
49
|
+
# only a grandchild that keeps writing makes close_io wait this long.
|
|
49
50
|
DRAIN_GRACE = 1
|
|
51
|
+
# Seconds the stderr drain waits for output before it checks whether the
|
|
52
|
+
# daemon has exited. A grandchild that holds the pipe open never sends EOF.
|
|
53
|
+
DRAIN_POLL = 0.1
|
|
50
54
|
# Bundler's marker for a variable that was unset before it activated.
|
|
51
55
|
BUNDLER_UNSET = "BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL"
|
|
52
56
|
|
|
@@ -349,10 +353,10 @@ module Mutineer
|
|
|
349
353
|
"-r", DAEMON_PATH, "-e", "Mutineer::DaemonServer.run",
|
|
350
354
|
unsetenv_others: true, chdir: @app_root
|
|
351
355
|
)
|
|
352
|
-
# Replies are UTF-8 JSON
|
|
353
|
-
#
|
|
356
|
+
# Replies are UTF-8 JSON: with LANG unset the pipe would read as
|
|
357
|
+
# US-ASCII (#303). The stderr drain reads bytes and marks each line
|
|
358
|
+
# UTF-8 itself (write_redacted).
|
|
354
359
|
@stdout.set_encoding(Encoding::UTF_8)
|
|
355
|
-
@stderr.set_encoding(Encoding::UTF_8)
|
|
356
360
|
# Drain daemon stderr to the tool's stderr so child/boot errors are visible,
|
|
357
361
|
# redacted line by line (#288: the app's output goes through here too).
|
|
358
362
|
# Tracked (not fire-and-forget) so close_io can reclaim it on quit/respawn.
|
|
@@ -419,29 +423,60 @@ module Mutineer
|
|
|
419
423
|
# A line is the unit the daemon and the app write, so a password is never
|
|
420
424
|
# split across two redaction calls. The line break is kept outside the
|
|
421
425
|
# redaction, so a line redaction hides as a whole still ends its line.
|
|
422
|
-
#
|
|
423
|
-
#
|
|
424
|
-
#
|
|
425
|
-
#
|
|
426
|
-
#
|
|
426
|
+
# The drain ends at EOF, or once the daemon has exited and the pipe is
|
|
427
|
+
# empty: a grandchild that keeps the pipe open would otherwise hold it
|
|
428
|
+
# until close_io's DRAIN_GRACE (#341). Either way it writes a last line
|
|
429
|
+
# that has no newline. After a write fails (EPIPE from a tool stderr that
|
|
430
|
+
# went away), the drain keeps reading and drops the lines, so the daemon
|
|
431
|
+
# never blocks on a full pipe. The outer rescue swallows the benign
|
|
432
|
+
# EBADF/IOError raised when close_io closes the pipe under an in-flight
|
|
433
|
+
# read, so the thread never ends with an error that close_io's join would
|
|
434
|
+
# re-raise.
|
|
427
435
|
#
|
|
428
436
|
# @param io [IO] the daemon's stderr pipe.
|
|
429
437
|
# @return [nil]
|
|
430
438
|
def drain_stderr(io)
|
|
431
439
|
writable = true
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
@
|
|
437
|
-
|
|
438
|
-
|
|
440
|
+
pending = "".b
|
|
441
|
+
loop do
|
|
442
|
+
# Read the exit before the pipe: everything the daemon wrote is in the
|
|
443
|
+
# pipe by the time it has exited, so an empty read after it is the end.
|
|
444
|
+
exited = @wait_thr && !@wait_thr.alive?
|
|
445
|
+
chunk = io.read_nonblock(4096, exception: false)
|
|
446
|
+
break if chunk.nil?
|
|
447
|
+
|
|
448
|
+
if chunk == :wait_readable
|
|
449
|
+
# After the exit, one more quiet poll before stopping, so a mutant
|
|
450
|
+
# child's last line, written just after the daemon died, still shows.
|
|
451
|
+
ready = io.wait_readable(DRAIN_POLL)
|
|
452
|
+
break if exited && !ready
|
|
453
|
+
|
|
454
|
+
next
|
|
455
|
+
end
|
|
456
|
+
pending << chunk.b
|
|
457
|
+
while (cut = pending.index("\n"))
|
|
458
|
+
line = pending.slice!(0..cut)
|
|
459
|
+
writable &&= write_redacted(line)
|
|
460
|
+
end
|
|
439
461
|
end
|
|
462
|
+
write_redacted(pending) if writable && !pending.empty?
|
|
440
463
|
nil
|
|
441
464
|
rescue StandardError
|
|
442
465
|
nil
|
|
443
466
|
end
|
|
444
467
|
|
|
468
|
+
# Write one line of daemon stderr to the tool's stderr, redacted.
|
|
469
|
+
#
|
|
470
|
+
# @param line [String] raw bytes, with or without the trailing newline.
|
|
471
|
+
# @return [Boolean] false when the write failed.
|
|
472
|
+
def write_redacted(line)
|
|
473
|
+
body = line.chomp("\n").force_encoding(Encoding::UTF_8).scrub # an invalid byte must not stop the drain
|
|
474
|
+
@errio.write(Redaction.text(body), line.end_with?("\n") ? "\n" : "")
|
|
475
|
+
true
|
|
476
|
+
rescue StandardError
|
|
477
|
+
false
|
|
478
|
+
end
|
|
479
|
+
|
|
445
480
|
# Close the IPC pipes, stop the stderr-drain thread, and reap the daemon so a
|
|
446
481
|
# respawn or quit leaves no leaked fd, thread, or zombie.
|
|
447
482
|
#
|
|
@@ -454,9 +489,9 @@ module Mutineer
|
|
|
454
489
|
rescue Errno::ESRCH
|
|
455
490
|
nil # it exited between the check and the kill
|
|
456
491
|
end
|
|
457
|
-
# The drain
|
|
458
|
-
#
|
|
459
|
-
#
|
|
492
|
+
# The drain ends on its own once the daemon has exited and the pipe is
|
|
493
|
+
# empty, writing a boot error with no final newline. Wait for that, then
|
|
494
|
+
# stop it BEFORE closing its fd (avoids an EBADF mid-read).
|
|
460
495
|
@drain&.join(DRAIN_GRACE)
|
|
461
496
|
@drain&.kill
|
|
462
497
|
[@stdin, @stdout, @stderr].each { |io| io&.close rescue nil } # rubocop:disable Style/RescueModifier
|
|
@@ -360,7 +360,10 @@ module Mutineer
|
|
|
360
360
|
apply_payload(req["payload"])
|
|
361
361
|
# A parallel worker records each pool that runs SQL, so a test
|
|
362
362
|
# that closes its shared connection is still caught below.
|
|
363
|
-
|
|
363
|
+
if @worker_db && @shared_databases&.any?
|
|
364
|
+
@worker_db.skip_schema_check!
|
|
365
|
+
@worker_db.watch_sql
|
|
366
|
+
end
|
|
364
367
|
result = run_tests(Array(req["tests"]))
|
|
365
368
|
check_shared_databases!
|
|
366
369
|
result
|
|
@@ -473,10 +476,12 @@ module Mutineer
|
|
|
473
476
|
# removes orphans and never touches a live file. Same path for reload
|
|
474
477
|
# (whole file) and redefine (wrapped snippet). The backend sends the whole
|
|
475
478
|
# file, so the concern blocks it stores replace the forgotten ones (#256).
|
|
476
|
-
# With a `"hook_def"`, the
|
|
477
|
-
#
|
|
478
|
-
#
|
|
479
|
+
# With a `"hook_def"`, the load places the mutated hook-block def into each
|
|
480
|
+
# class that has the method, before the rest of the file runs; it raises
|
|
481
|
+
# when none was stored, so the mutant scores `error` (#297, #348).
|
|
479
482
|
def apply_payload(payload)
|
|
483
|
+
hook = payload["hook_def"]
|
|
484
|
+
hook &&= { line: hook.fetch("line"), name: hook.fetch("name").to_sym, singleton: hook.fetch("singleton") }
|
|
480
485
|
dir = File.dirname(File.expand_path(payload.fetch("source_file")))
|
|
481
486
|
Tempfile.create([".mutineer_daemon", ".rb"], dir) do |f|
|
|
482
487
|
# Held until this block ends, including SIGKILL (the kernel drops it).
|
|
@@ -484,12 +489,8 @@ module Mutineer
|
|
|
484
489
|
f.flock(File::LOCK_EX) rescue nil # rubocop:disable Style/RescueModifier
|
|
485
490
|
f.write(payload.fetch("code"))
|
|
486
491
|
f.flush
|
|
487
|
-
ConcernBlocks.load_mutant(payload.fetch("source_file"), f.path)
|
|
492
|
+
ConcernBlocks.load_mutant(payload.fetch("source_file"), f.path, hook_def: hook)
|
|
488
493
|
end
|
|
489
|
-
hook = payload["hook_def"] or return
|
|
490
|
-
|
|
491
|
-
ConcernBlocks.place_hook_def(source_file: payload.fetch("source_file"), line: hook.fetch("line"),
|
|
492
|
-
name: hook.fetch("name").to_sym, singleton: hook.fetch("singleton"))
|
|
493
494
|
end
|
|
494
495
|
|
|
495
496
|
# Load the covering test files and run them; 0 = all passed (survived),
|
|
@@ -61,6 +61,17 @@ module Mutineer
|
|
|
61
61
|
RSPEC_OUTSIDE_ERROR = /errors? occurred outside of examples/
|
|
62
62
|
# ANSI colour codes, removed before matching so a coloured summary still counts.
|
|
63
63
|
ANSI_CODE = /\e\[[\d;]*m/
|
|
64
|
+
# The seed line Minitest prints before every run. Captures the seed.
|
|
65
|
+
MINITEST_SEED = /^Run options:.*--seed (\d+)/
|
|
66
|
+
# The seed line RSpec prints when it shuffles the tests. Captures the seed.
|
|
67
|
+
RSPEC_SEED = /^Randomized with seed (\d+)/
|
|
68
|
+
# A Minitest seed given on the command or in TESTOPTS (`SEED=N`, `--seed N`
|
|
69
|
+
# `-s N` or `-sN`). Minitest prints its seed line for a fixed seed too, so
|
|
70
|
+
# the line alone cannot tell.
|
|
71
|
+
MINITEST_SEED_GIVEN = /\bSEED=|--seed\b|(?:\A|\s)-s(?![A-Za-z_])/
|
|
72
|
+
# An RSpec seed given on the command or in SPEC_OPTS (`--seed N` or
|
|
73
|
+
# `--order rand:N`). RSpec ignores SEED.
|
|
74
|
+
RSPEC_SEED_GIVEN = /--seed\b|\brand(?:om)?:\d/
|
|
64
75
|
|
|
65
76
|
# PATH entries that pin a concrete Ruby under a version manager (ahead of
|
|
66
77
|
# shims). Optional trailing slash; normal bins are left alone.
|
|
@@ -286,20 +297,50 @@ module Mutineer
|
|
|
286
297
|
output.b.gsub(ANSI_CODE, "")
|
|
287
298
|
end
|
|
288
299
|
|
|
300
|
+
# The runner and seed of a random test order the output shows, or nil.
|
|
301
|
+
# Nil also when Mutineer sees a fixed seed: for Minitest, SEED, TESTOPTS
|
|
302
|
+
# or the command sets one; for RSpec, SPEC_OPTS or the command does. RSpec
|
|
303
|
+
# `--order defined` prints no seed line. Each runner is checked on its
|
|
304
|
+
# own, so a seeded Minitest suite does not hide an unseeded RSpec suite
|
|
305
|
+
# that the same command runs. Ceiling: a seed set in a wrapper script or
|
|
306
|
+
# `.rspec`, or Minitest's `test_order`, is not seen, so the warning still
|
|
307
|
+
# prints; a seed-like argument meant for a wrapper hides it (#338, limits
|
|
308
|
+
# tracked in #368).
|
|
309
|
+
#
|
|
310
|
+
# @api private
|
|
311
|
+
# @param command [String] the --test-command template.
|
|
312
|
+
# @param text [String] output from {summary_text}.
|
|
313
|
+
# @return [Array(Symbol, String), nil] `[:minitest | :rspec, seed]`.
|
|
314
|
+
def self.random_order(command, text)
|
|
315
|
+
minitest = text[MINITEST_SEED, 1]
|
|
316
|
+
if minitest && ENV["SEED"].to_s.empty? &&
|
|
317
|
+
[command, ENV["TESTOPTS"]].none? { |s| s.to_s.match?(MINITEST_SEED_GIVEN) }
|
|
318
|
+
return [:minitest, minitest]
|
|
319
|
+
end
|
|
320
|
+
|
|
321
|
+
rspec = text[RSPEC_SEED, 1]
|
|
322
|
+
[:rspec, rspec] if rspec && [command, ENV["SPEC_OPTS"]].none? { |s| s.to_s.match?(RSPEC_SEED_GIVEN) }
|
|
323
|
+
end
|
|
324
|
+
|
|
289
325
|
# Pre-flight: run the command once against the UNMUTATED tree. Green (exit 0)
|
|
290
|
-
# returns the elapsed seconds (used to calibrate the per-mutant timeout)
|
|
326
|
+
# returns the elapsed seconds (used to calibrate the per-mutant timeout),
|
|
291
327
|
# whether the output has a Minitest or RSpec summary line (used to turn on
|
|
292
|
-
# the summary check in {run})
|
|
293
|
-
#
|
|
328
|
+
# the summary check in {run}), and the runner and seed of a random test
|
|
329
|
+
# order (see {random_order}); anything else raises SmokeCheckError so the run
|
|
330
|
+
# aborts before scoring.
|
|
294
331
|
#
|
|
295
332
|
# @param command [String] the --test-command template.
|
|
296
333
|
# @param files [Array<String>] test file paths.
|
|
297
334
|
# @param timeout [Numeric] ceiling for the calibration run.
|
|
298
|
-
# @return [Array(Float, Boolean
|
|
335
|
+
# @return [Array(Float, Boolean, Array(Symbol, String), nil)]
|
|
336
|
+
# `[elapsed_seconds, summary_seen, random_order]`.
|
|
299
337
|
# @raise [Mutineer::SmokeCheckError] when the clean suite is not green.
|
|
300
338
|
def self.smoke_check!(command, files, timeout: SMOKE_TIMEOUT)
|
|
301
339
|
kind, code, output, elapsed = spawn_capture(command, files, timeout)
|
|
302
|
-
|
|
340
|
+
if kind == :exited && code&.zero?
|
|
341
|
+
text = summary_text(output)
|
|
342
|
+
return [elapsed, summary_line?(text), random_order(command, text)]
|
|
343
|
+
end
|
|
303
344
|
|
|
304
345
|
reason =
|
|
305
346
|
if kind == :timeout then "did not finish within #{timeout}s"
|
data/lib/mutineer/isolation.rb
CHANGED
|
@@ -193,8 +193,8 @@ module Mutineer
|
|
|
193
193
|
# A concern's stored `included` and `prepended` blocks from the source file
|
|
194
194
|
# are forgotten around the load, so ActiveSupport accepts the blocks the
|
|
195
195
|
# temp file stores (#256). For a def in such a block, the caller passes the
|
|
196
|
-
# text from {JobPlan.hook_mutant} and
|
|
197
|
-
#
|
|
196
|
+
# text and `hook_def` from {JobPlan.hook_mutant}, and the load places the
|
|
197
|
+
# mutated def (#297, #348).
|
|
198
198
|
#
|
|
199
199
|
# {ConcernBlocks.load_mutant} turns Ruby's warnings off for the load, so
|
|
200
200
|
# the constants it assigns again do not warn (#364).
|
|
@@ -202,9 +202,10 @@ module Mutineer
|
|
|
202
202
|
# @api private
|
|
203
203
|
# @param mutated [String] mutated source text.
|
|
204
204
|
# @param source_file [String] original source file path.
|
|
205
|
+
# @param hook_def [Hash{Symbol => Object}, nil] see {ConcernBlocks.load_mutant}.
|
|
205
206
|
# @return [Object] whatever `load` returns.
|
|
206
|
-
def self.apply_whole_file(mutated, source_file)
|
|
207
|
-
write_beside(mutated, source_file) { |path| ConcernBlocks.load_mutant(source_file, path) }
|
|
207
|
+
def self.apply_whole_file(mutated, source_file, hook_def: nil)
|
|
208
|
+
write_beside(mutated, source_file) { |path| ConcernBlocks.load_mutant(source_file, path, hook_def: hook_def) }
|
|
208
209
|
end
|
|
209
210
|
|
|
210
211
|
# Writes `code` to a hidden `.mutineer_mutant` file beside `source_file`,
|
|
@@ -273,7 +274,7 @@ module Mutineer
|
|
|
273
274
|
# running the ORIGINAL method (C2 false-survived). Raise -> the fork
|
|
274
275
|
# block aborts before any test runs -> Result.error, never a bogus
|
|
275
276
|
# `survived`.
|
|
276
|
-
raise "surgical snippet failed to reparse" if Parser.
|
|
277
|
+
raise "surgical snippet failed to reparse" if Parser.invalid?(wrapped)
|
|
277
278
|
|
|
278
279
|
# Preserve original visibility — class/module bodies define methods
|
|
279
280
|
# public, but 7a's `load` would re-apply the file's private/protected
|