evilution 0.22.1 → 0.22.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.beads/interactions.jsonl +3 -0
- data/CHANGELOG.md +26 -0
- data/README.md +6 -1
- data/docs/isolation.md +115 -0
- data/lib/evilution/cli.rb +3 -0
- data/lib/evilution/config.rb +17 -2
- data/lib/evilution/integration/rspec.rb +11 -0
- data/lib/evilution/mcp/mutate_tool.rb +5 -1
- data/lib/evilution/rails_detector.rb +57 -0
- data/lib/evilution/runner.rb +95 -9
- data/lib/evilution/version.rb +1 -1
- metadata +4 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 05657264e506e5a6b5041d68b5eb46bfcbbf9a17780caeb3c44feaf5f5d18794
|
|
4
|
+
data.tar.gz: a7e6333fc26d21799b3807d3fbc2758ae54e00de2c7f1028e5ad30cf8777e8d2
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8d791eba8e795dcaa34c6ff3cc82a4b9653cfc282fcff1f7a75911bdfbd192689d21324ad793077700644a497c8320b2fb77e874bff63cd98c1a788390a27309
|
|
7
|
+
data.tar.gz: 5ee22113b3b0343c60c08ca4c4c4160492617827376c783fbe0052d5a5a59ab37199fc16de349502c6c7d26f2c64814abf7ad31228e325dcdc1be86e1a7930d7
|
data/.beads/interactions.jsonl
CHANGED
|
@@ -14,3 +14,6 @@
|
|
|
14
14
|
{"id":"int-3a6ffc92","kind":"field_change","created_at":"2026-04-10T10:26:29.483210031Z","actor":"Denis Kiselev","issue_id":"EV-sgsb","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged via PR #656"}}
|
|
15
15
|
{"id":"int-1df69312","kind":"field_change","created_at":"2026-04-10T10:40:18.391377715Z","actor":"Denis Kiselev","issue_id":"EV-1l8w","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged"}}
|
|
16
16
|
{"id":"int-427bdc14","kind":"field_change","created_at":"2026-04-10T12:40:12.974701482Z","actor":"Denis Kiselev","issue_id":"EV-k6cz","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged as PR #659 — capture error_class and error_backtrace in MutationResult, thread through isolators + runner + JSON reporter, log in --verbose"}}
|
|
17
|
+
{"id":"int-d3431bcd","kind":"field_change","created_at":"2026-04-12T02:57:43.902279367Z","actor":"Denis Kiselev","issue_id":"EV-86l6","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
|
|
18
|
+
{"id":"int-f409d79d","kind":"field_change","created_at":"2026-04-12T02:57:44.180309214Z","actor":"Denis Kiselev","issue_id":"EV-o28o","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
|
|
19
|
+
{"id":"int-ba5d5d3e","kind":"field_change","created_at":"2026-04-12T03:42:58.408103757Z","actor":"Denis Kiselev","issue_id":"EV-1fq8","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Closed"}}
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,31 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.22.3] - 2026-04-12
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
|
|
7
|
+
- **`LoadError: cannot load such file -- spec_helper`** — projects with `--require spec_helper` in `.rspec` failed on every mutation because `spec/` was not on `$LOAD_PATH`; RSpec's CLI normally adds it, but evilution calls `RSpec::Core::Runner.run` directly, bypassing the CLI; now adds `spec/` to `$LOAD_PATH` in both `ensure_framework_loaded` and `baseline_runner` (#669)
|
|
8
|
+
|
|
9
|
+
## [0.22.2] - 2026-04-12
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- **Rails-aware isolation auto-selection** — `isolation: :auto` (the default) now detects Rails projects by walking up from target files looking for `config/application.rb`; when Rails is detected, isolation resolves to `:fork` instead of `:in_process`, preventing indefinite hangs caused by Rails' `Thread.handle_interrupt(Exception => :never)` masking `Timeout.timeout`'s `Thread#raise` (#662, #663, PR #665)
|
|
14
|
+
- **Parent-process preload for fork isolation** — new `--preload FILE` / `--no-preload` CLI flags and `preload:` config key; when fork isolation is active on a Rails project, evilution auto-detects and preloads `spec/rails_helper.rb` or `test/test_helper.rb` in the parent process so forked children inherit the loaded framework via copy-on-write, eliminating per-mutation Rails boot cost (#662, PR #665)
|
|
15
|
+
- **`Evilution::RailsDetector` module** — lightweight filesystem-only detection of Rails roots with memoized cache and thread-safe mutex; used by both isolation resolution and preload auto-detection (#662, PR #665)
|
|
16
|
+
- **Isolation strategy documentation** — `docs/isolation.md` explains the three strategies, the `handle_interrupt` hazard, and preload configuration (#662, PR #665)
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
|
|
20
|
+
- **`run_mutations_parallel` ignoring `--isolation`** — parallel mode hardcoded `Isolation::InProcess.new` for its worker isolator, silently overriding `config.isolation`; now uses `build_isolator` so `--jobs N --isolation fork` correctly uses fork per-mutation inside workers (#663, PR #665)
|
|
21
|
+
- **Rails detection failing for auto-resolved targets** — `detected_rails_root` used `config.target_files` which is empty in git-changed or source-glob modes; now uses memoized `resolve_target_files` so Rails is detected regardless of how targets were specified (PR #665)
|
|
22
|
+
- **`SyntaxError` in preload file escaping rescue** — `perform_preload` rescued `LoadError, StandardError` but `SyntaxError` is a `ScriptError` (not `StandardError`); now rescues `ScriptError, StandardError` (PR #665)
|
|
23
|
+
- **Isolator built eagerly before targets resolved** — `build_isolator` ran in `initialize` before `resolve_target_files`; now lazy-initialized on first use, ensuring Rails detection has resolved targets available (PR #665)
|
|
24
|
+
|
|
25
|
+
### Changed
|
|
26
|
+
|
|
27
|
+
- **Isolation warning for explicit `in_process` under Rails** — when a user explicitly passes `--isolation in_process` on a detected Rails project, evilution emits a one-shot stderr warning about the `handle_interrupt` hang hazard and proceeds; suppressed by `--quiet` (#662, PR #665)
|
|
28
|
+
|
|
3
29
|
## [0.22.1] - 2026-04-10
|
|
4
30
|
|
|
5
31
|
### Added
|
data/README.md
CHANGED
|
@@ -64,6 +64,9 @@ evilution [command] [options] [files...]
|
|
|
64
64
|
| `--incremental` | Boolean | false | Cache killed/timeout results; skip unchanged mutations on re-runs. |
|
|
65
65
|
| `--save-session` | Boolean | false | Persist results as timestamped JSON under `.evilution/results/`. |
|
|
66
66
|
| `--no-progress` | Boolean | _(enabled)_ | Disable the TTY progress bar. |
|
|
67
|
+
| `--isolation MODE` | String | `auto` | Isolation strategy: `auto`, `fork`, or `in_process`. `auto` selects `fork` for Rails projects. See [docs/isolation.md](docs/isolation.md). |
|
|
68
|
+
| `--preload FILE` | String | _(auto)_ | File to require in parent before forking workers (e.g. `spec/rails_helper.rb`). Auto-detected for Rails. |
|
|
69
|
+
| `--no-preload` | Boolean | _(enabled)_ | Disable parent-process preload. |
|
|
67
70
|
| `--skip-heredoc-literals` | Boolean | false | Skip all string literal mutations inside heredocs. |
|
|
68
71
|
| `--show-disabled` | Boolean | false | Report mutations skipped by `# evilution:disable` comments. |
|
|
69
72
|
| `--baseline-session PATH` | String | _(none)_ | Saved session file for HTML report comparison. |
|
|
@@ -90,6 +93,8 @@ Creates `.evilution.yml`:
|
|
|
90
93
|
# integration: rspec # test framework: rspec, minitest
|
|
91
94
|
# suggest_tests: false # concrete test code in suggestions (matches integration)
|
|
92
95
|
# save_session: false # persist results under .evilution/results/
|
|
96
|
+
# isolation: auto # auto | fork | in_process (auto selects fork for Rails)
|
|
97
|
+
# preload: null # path to preload before forking; false to disable; auto-detects for Rails
|
|
93
98
|
# skip_heredoc_literals: false # skip all string literal mutations inside heredocs
|
|
94
99
|
# show_disabled: false # report mutations skipped by disable comments
|
|
95
100
|
# baseline_session: null # path to session file for HTML comparison
|
|
@@ -405,7 +410,7 @@ Tests 4 paths (InProcess isolation, Fork isolation, mutation generation + stripp
|
|
|
405
410
|
2. **Extract** — Methods are identified as mutation subjects
|
|
406
411
|
3. **Filter** — Disable comments, Sorbet `sig` blocks, and AST ignore patterns exclude mutations before execution
|
|
407
412
|
4. **Mutate** — 72 operators produce text replacements at precise byte offsets (source-level surgery, no AST unparsing); heredoc literal text is skipped by default
|
|
408
|
-
5. **Isolate** — Mutations are applied to temporary file copies (never modifying originals); load-path redirection ensures `require` resolves the mutated copy. Default isolation is in-process; `--isolation fork`
|
|
413
|
+
5. **Isolate** — Mutations are applied to temporary file copies (never modifying originals); load-path redirection ensures `require` resolves the mutated copy. Default isolation is in-process for plain Ruby projects and fork for Rails projects (auto-detected); `--isolation fork` forces forked child processes. Both sequential and parallel (`--jobs N`) modes respect the configured isolation strategy
|
|
409
414
|
6. **Test** — The configured test framework (RSpec or Minitest) executes against the mutated source
|
|
410
415
|
7. **Collect** — Source strings and AST nodes are released after use to minimize memory retention
|
|
411
416
|
8. **Report** — Results aggregated into text, JSON, or HTML, including efficiency metrics and peak memory usage
|
data/docs/isolation.md
ADDED
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
# Isolation strategies
|
|
2
|
+
|
|
3
|
+
Evilution runs every mutant inside an isolated environment so a misbehaving
|
|
4
|
+
mutant cannot corrupt the runner's state or the surrounding test suite. The
|
|
5
|
+
`--isolation` flag (and the `isolation:` config key) selects which strategy
|
|
6
|
+
is used.
|
|
7
|
+
|
|
8
|
+
## Strategies
|
|
9
|
+
|
|
10
|
+
| Strategy | What it does | When to use |
|
|
11
|
+
| ------------ | ---------------------------------------------------------------------- | ---------------------------------------------------------- |
|
|
12
|
+
| `auto` | Default. Picks `fork` for Rails projects, `in_process` otherwise. | Leave this on unless you have a specific reason to change. |
|
|
13
|
+
| `fork` | Forks a fresh child process per mutant. Parent `SIGKILL`s on timeout. | Rails / ActiveRecord projects; any code that uses mutexes, monitors, or async-interrupt masks. |
|
|
14
|
+
| `in_process` | Runs the mutant inside the runner process under `Timeout.timeout`. | Pure-Ruby libraries that do not use async-interrupt masks. |
|
|
15
|
+
|
|
16
|
+
## Why fork is the default for Rails projects
|
|
17
|
+
|
|
18
|
+
Rails wraps every ActiveRecord transaction in
|
|
19
|
+
`Thread.handle_interrupt(Exception => :never)` to guarantee that a transaction
|
|
20
|
+
either commits cleanly or rolls back cleanly — it must never exit the
|
|
21
|
+
transaction block with a half-delivered exception. The mask is
|
|
22
|
+
[load-bearing for transaction correctness][rails-handle-interrupt].
|
|
23
|
+
|
|
24
|
+
That mask interacts badly with `Timeout.timeout`. The Timeout gem schedules a
|
|
25
|
+
timer thread which fires `Thread#raise Timeout::Error` at the main thread when
|
|
26
|
+
the deadline hits. Ruby receives the `raise`, inspects the interrupt mask, sees
|
|
27
|
+
`Exception => :never`, and **queues the exception for later delivery** — "later"
|
|
28
|
+
meaning "when the masked block exits". If the mutant is stuck in an infinite
|
|
29
|
+
loop inside the transaction, the block never exits, the queued exception is
|
|
30
|
+
never delivered, and the runner hangs indefinitely.
|
|
31
|
+
|
|
32
|
+
This is not a bug in Timeout or in Rails. Each component is correct in
|
|
33
|
+
isolation. They simply do not compose for in-process timeout-based
|
|
34
|
+
cancellation. The only primitive that can escape a masked-interrupt section
|
|
35
|
+
in the same thread is an out-of-band kernel signal — `SIGKILL` from another
|
|
36
|
+
process. That is exactly what the `fork` strategy provides.
|
|
37
|
+
|
|
38
|
+
Because of this, `auto` resolves to `fork` whenever the target files live
|
|
39
|
+
under a detected Rails root (i.e. a directory containing
|
|
40
|
+
`config/application.rb`). If you explicitly pass `--isolation in_process` on
|
|
41
|
+
a Rails project, evilution emits a warning naming the hazard and proceeds
|
|
42
|
+
anyway — sometimes you know the code under test never enters a masked
|
|
43
|
+
section, and that is your call to make.
|
|
44
|
+
|
|
45
|
+
The same hazard applies to any Ruby code that uses
|
|
46
|
+
`Thread.handle_interrupt(... => :never)`: `Mutex#synchronize`, `Monitor#synchronize`,
|
|
47
|
+
`Queue`, `ActiveSupport::Notifications::Fanout` listeners, and custom cleanup
|
|
48
|
+
blocks that wrap "must complete" sections. If your target code can touch any
|
|
49
|
+
of those, prefer `--isolation fork`.
|
|
50
|
+
|
|
51
|
+
## Parent-process preload
|
|
52
|
+
|
|
53
|
+
Fork isolation's one downside is that every child pays the cost of loading
|
|
54
|
+
its test framework from scratch. For a Rails app, that is 5–15 seconds of
|
|
55
|
+
`require "rails/all"` per mutant — multiplied by hundreds of mutants, the
|
|
56
|
+
run takes hours.
|
|
57
|
+
|
|
58
|
+
To avoid that cost, evilution preloads a bootstrap file in the **parent**
|
|
59
|
+
process before the mutation loop begins. Once Rails is loaded in the parent,
|
|
60
|
+
every forked child inherits the loaded state via copy-on-write, and the child
|
|
61
|
+
runs its test near-instantly against an already-loaded framework.
|
|
62
|
+
|
|
63
|
+
### Automatic preload
|
|
64
|
+
|
|
65
|
+
For Rails projects (auto-detected via `config/application.rb`), evilution
|
|
66
|
+
automatically looks for these files in order and preloads the first one it
|
|
67
|
+
finds:
|
|
68
|
+
|
|
69
|
+
1. `spec/rails_helper.rb` (RSpec)
|
|
70
|
+
2. `test/test_helper.rb` (Minitest)
|
|
71
|
+
|
|
72
|
+
No configuration needed.
|
|
73
|
+
|
|
74
|
+
### Explicit preload
|
|
75
|
+
|
|
76
|
+
Override the auto-detected path with the `--preload` flag or the `preload:`
|
|
77
|
+
config key:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
bundle exec evilution run app/models/user.rb --preload config/evilution_boot.rb
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
```yaml
|
|
84
|
+
# .evilution.yml
|
|
85
|
+
preload: config/evilution_boot.rb
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
The path is resolved relative to the working directory. Preload failures
|
|
89
|
+
(the file does not exist, or it raises during `require`) are fatal —
|
|
90
|
+
evilution aborts with an `Evilution::ConfigError` that includes the original
|
|
91
|
+
exception.
|
|
92
|
+
|
|
93
|
+
### Disabling preload
|
|
94
|
+
|
|
95
|
+
Pass `--no-preload` on the CLI or set `preload: false` in `.evilution.yml`:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
bundle exec evilution run app/models/user.rb --no-preload
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
The MCP tool (`evilution-mutate`) disables preload unconditionally, because
|
|
102
|
+
the MCP server is a long-lived process that handles runs from different
|
|
103
|
+
projects — preloading one project's Rails stack into a shared process would
|
|
104
|
+
poison subsequent runs.
|
|
105
|
+
|
|
106
|
+
## Related flags
|
|
107
|
+
|
|
108
|
+
- `--timeout N` sets the per-mutation time limit. Under `fork`, this drives
|
|
109
|
+
SIGKILL. Under `in_process`, this drives `Timeout.timeout` and is subject
|
|
110
|
+
to the interrupt-mask hazard described above.
|
|
111
|
+
- `--jobs N` runs N workers in parallel. The parallel pool respects the
|
|
112
|
+
configured isolation strategy, so `--jobs 4 --isolation fork` uses fork
|
|
113
|
+
isolation per-mutation inside each worker.
|
|
114
|
+
|
|
115
|
+
[rails-handle-interrupt]: https://github.com/rails/rails/blob/main/activesupport/lib/active_support/concurrency/thread_monitor.rb
|
data/lib/evilution/cli.rb
CHANGED
|
@@ -249,6 +249,9 @@ class Evilution::CLI
|
|
|
249
249
|
opts.on("--incremental", "Cache killed/timeout results; skip re-running them on unchanged files") { @options[:incremental] = true }
|
|
250
250
|
opts.on("--integration NAME", "Test integration: rspec, minitest (default: rspec)") { |i| @options[:integration] = i }
|
|
251
251
|
opts.on("--isolation STRATEGY", "Isolation: auto, fork, in_process (default: auto)") { |s| @options[:isolation] = s }
|
|
252
|
+
opts.on("--preload FILE", "Preload FILE in the parent process before forking " \
|
|
253
|
+
"(default: auto-detect spec/rails_helper.rb for Rails projects)") { |f| @options[:preload] = f }
|
|
254
|
+
opts.on("--no-preload", "Disable parent-process preload even for Rails projects") { @options[:preload] = false }
|
|
252
255
|
opts.on("--stdin", "Read target file paths from stdin (one per line)") { @options[:stdin] = true }
|
|
253
256
|
opts.on("--suggest-tests", "Generate concrete test code in suggestions (RSpec or Minitest)") { @options[:suggest_tests] = true }
|
|
254
257
|
opts.on("--no-progress", "Disable progress bar") { @options[:progress] = false }
|
data/lib/evilution/config.rb
CHANGED
|
@@ -26,7 +26,8 @@ class Evilution::Config
|
|
|
26
26
|
ignore_patterns: [],
|
|
27
27
|
show_disabled: false,
|
|
28
28
|
baseline_session: nil,
|
|
29
|
-
skip_heredoc_literals: false
|
|
29
|
+
skip_heredoc_literals: false,
|
|
30
|
+
preload: nil
|
|
30
31
|
}.freeze
|
|
31
32
|
|
|
32
33
|
attr_reader :target_files, :timeout, :format,
|
|
@@ -34,7 +35,7 @@ class Evilution::Config
|
|
|
34
35
|
:jobs, :fail_fast, :baseline, :isolation, :incremental, :suggest_tests,
|
|
35
36
|
:progress, :save_session, :line_ranges, :spec_files, :hooks,
|
|
36
37
|
:ignore_patterns, :show_disabled, :baseline_session,
|
|
37
|
-
:skip_heredoc_literals
|
|
38
|
+
:skip_heredoc_literals, :preload
|
|
38
39
|
|
|
39
40
|
def initialize(**options)
|
|
40
41
|
file_options = options.delete(:skip_config_file) ? {} : load_config_file
|
|
@@ -140,6 +141,11 @@ class Evilution::Config
|
|
|
140
141
|
# Skip all string literal mutations inside heredocs (default: false)
|
|
141
142
|
# skip_heredoc_literals: false
|
|
142
143
|
|
|
144
|
+
# Preload file required in the parent process before forking workers.
|
|
145
|
+
# For Rails projects, spec/rails_helper.rb or test/test_helper.rb is
|
|
146
|
+
# auto-detected when isolation resolves to :fork. Set to false to disable.
|
|
147
|
+
# preload: spec/rails_helper.rb # or test/test_helper.rb
|
|
148
|
+
|
|
143
149
|
# Hooks: Ruby files returning a Proc, keyed by lifecycle event
|
|
144
150
|
# hooks:
|
|
145
151
|
# worker_process_start: config/evilution_hooks/worker_start.rb
|
|
@@ -190,6 +196,15 @@ class Evilution::Config
|
|
|
190
196
|
@baseline_session = merged[:baseline_session]
|
|
191
197
|
@skip_heredoc_literals = merged[:skip_heredoc_literals]
|
|
192
198
|
@hooks = validate_hooks(merged[:hooks])
|
|
199
|
+
@preload = validate_preload(merged[:preload])
|
|
200
|
+
end
|
|
201
|
+
|
|
202
|
+
def validate_preload(value)
|
|
203
|
+
return nil if value.nil?
|
|
204
|
+
return false if value == false
|
|
205
|
+
return value if value.is_a?(String)
|
|
206
|
+
|
|
207
|
+
raise Evilution::ConfigError, "preload must be nil, false, or a String path, got #{value.inspect}"
|
|
193
208
|
end
|
|
194
209
|
|
|
195
210
|
def validate_integration(value)
|
|
@@ -12,6 +12,8 @@ class Evilution::Integration::RSpec < Evilution::Integration::Base
|
|
|
12
12
|
def self.baseline_runner
|
|
13
13
|
lambda { |spec_file|
|
|
14
14
|
require "rspec/core"
|
|
15
|
+
spec_dir = File.expand_path("spec")
|
|
16
|
+
$LOAD_PATH.unshift(spec_dir) unless $LOAD_PATH.include?(spec_dir)
|
|
15
17
|
::RSpec.reset
|
|
16
18
|
status = ::RSpec::Core::Runner.run(
|
|
17
19
|
["--format", "progress", "--no-color", "--order", "defined", spec_file]
|
|
@@ -43,6 +45,7 @@ class Evilution::Integration::RSpec < Evilution::Integration::Base
|
|
|
43
45
|
|
|
44
46
|
fire_hook(:setup_integration_pre, integration: :rspec)
|
|
45
47
|
require "rspec/core"
|
|
48
|
+
add_spec_load_path
|
|
46
49
|
Evilution::Integration::CrashDetector.register_with_rspec
|
|
47
50
|
@rspec_loaded = true
|
|
48
51
|
fire_hook(:setup_integration_post, integration: :rspec)
|
|
@@ -163,4 +166,12 @@ class Evilution::Integration::RSpec < Evilution::Integration::Base
|
|
|
163
166
|
warn "[evilution] No matching spec found for #{file_path}, running full suite. " \
|
|
164
167
|
"Use --spec to specify the spec file."
|
|
165
168
|
end
|
|
169
|
+
|
|
170
|
+
# RSpec's CLI adds spec/ to $LOAD_PATH so that `--require spec_helper`
|
|
171
|
+
# (commonly in .rspec) resolves. We call Runner.run directly, bypassing
|
|
172
|
+
# the CLI, so we must replicate this.
|
|
173
|
+
def add_spec_load_path
|
|
174
|
+
spec_dir = File.expand_path("spec")
|
|
175
|
+
$LOAD_PATH.unshift(spec_dir) unless $LOAD_PATH.include?(spec_dir)
|
|
176
|
+
end
|
|
166
177
|
end
|
|
@@ -111,7 +111,11 @@ class Evilution::MCP::MutateTool < MCP::Tool
|
|
|
111
111
|
end
|
|
112
112
|
|
|
113
113
|
def build_config_opts(files, line_ranges, target, timeout, jobs, fail_fast, spec, suggest_tests)
|
|
114
|
-
|
|
114
|
+
# Preload is disabled for MCP invocations: `require`-ing Rails into the
|
|
115
|
+
# long-lived MCP server would poison subsequent runs against other
|
|
116
|
+
# projects. MCP users who want the speedup should use the CLI.
|
|
117
|
+
opts = { target_files: files, line_ranges: line_ranges, format: :json, quiet: true, skip_config_file: true,
|
|
118
|
+
preload: false }
|
|
115
119
|
opts[:target] = target if target
|
|
116
120
|
opts[:timeout] = timeout if timeout
|
|
117
121
|
opts[:jobs] = jobs if jobs
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Evilution::RailsDetector
|
|
4
|
+
MARKER = File.join("config", "application.rb").freeze
|
|
5
|
+
|
|
6
|
+
@cache = {}
|
|
7
|
+
@mutex = Mutex.new
|
|
8
|
+
|
|
9
|
+
class << self
|
|
10
|
+
def rails_root_for(path)
|
|
11
|
+
return nil if path.nil?
|
|
12
|
+
|
|
13
|
+
dir = starting_dir(path)
|
|
14
|
+
return nil if dir.nil?
|
|
15
|
+
|
|
16
|
+
@mutex.synchronize do
|
|
17
|
+
return @cache[dir] if @cache.key?(dir)
|
|
18
|
+
|
|
19
|
+
@cache[dir] = walk_up(dir)
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def rails_root_for_any(paths)
|
|
24
|
+
Array(paths).each do |path|
|
|
25
|
+
root = rails_root_for(path)
|
|
26
|
+
return root if root
|
|
27
|
+
end
|
|
28
|
+
nil
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def reset_cache!
|
|
32
|
+
@mutex.synchronize { @cache.clear }
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
private
|
|
36
|
+
|
|
37
|
+
def starting_dir(path)
|
|
38
|
+
return File.expand_path(path) if File.directory?(path)
|
|
39
|
+
return File.expand_path(File.dirname(path)) if File.file?(path)
|
|
40
|
+
|
|
41
|
+
parent = File.expand_path(File.dirname(path))
|
|
42
|
+
File.directory?(parent) ? parent : nil
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def walk_up(dir)
|
|
46
|
+
current = dir
|
|
47
|
+
loop do
|
|
48
|
+
return current if File.file?(File.join(current, MARKER))
|
|
49
|
+
|
|
50
|
+
parent = File.dirname(current)
|
|
51
|
+
return nil if parent == current
|
|
52
|
+
|
|
53
|
+
current = parent
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
data/lib/evilution/runner.rb
CHANGED
|
@@ -25,6 +25,7 @@ require_relative "ast/pattern/filter"
|
|
|
25
25
|
require_relative "temp_dir_tracker"
|
|
26
26
|
require_relative "disable_comment"
|
|
27
27
|
require_relative "ast/sorbet_sig_detector"
|
|
28
|
+
require_relative "rails_detector"
|
|
28
29
|
|
|
29
30
|
class Evilution::Runner
|
|
30
31
|
INTEGRATIONS = {
|
|
@@ -32,6 +33,11 @@ class Evilution::Runner
|
|
|
32
33
|
minitest: Evilution::Integration::Minitest
|
|
33
34
|
}.freeze
|
|
34
35
|
|
|
36
|
+
PRELOAD_CANDIDATES = [
|
|
37
|
+
File.join("spec", "rails_helper.rb"),
|
|
38
|
+
File.join("test", "test_helper.rb")
|
|
39
|
+
].freeze
|
|
40
|
+
|
|
35
41
|
attr_reader :config
|
|
36
42
|
|
|
37
43
|
def initialize(config: Evilution::Config.new, on_result: nil, hooks: nil)
|
|
@@ -40,7 +46,6 @@ class Evilution::Runner
|
|
|
40
46
|
@hooks = hooks
|
|
41
47
|
@parser = Evilution::AST::Parser.new
|
|
42
48
|
@registry = Evilution::Mutator::Registry.default
|
|
43
|
-
@isolator = build_isolator
|
|
44
49
|
@cache = config.incremental? ? Evilution::Cache.new : nil
|
|
45
50
|
@disable_detector = Evilution::DisableComment.new
|
|
46
51
|
@disabled_ranges_cache = {}
|
|
@@ -55,6 +60,9 @@ class Evilution::Runner
|
|
|
55
60
|
subjects = parse_and_filter_subjects
|
|
56
61
|
log_memory("after parse_subjects", "#{subjects.length} subjects")
|
|
57
62
|
|
|
63
|
+
perform_preload
|
|
64
|
+
log_memory("after preload") if rails_root_detected?
|
|
65
|
+
|
|
58
66
|
baseline_result = run_baseline(subjects)
|
|
59
67
|
|
|
60
68
|
mutations, skipped_count, disabled_mutations = generate_mutations(subjects)
|
|
@@ -89,7 +97,11 @@ class Evilution::Runner
|
|
|
89
97
|
|
|
90
98
|
private
|
|
91
99
|
|
|
92
|
-
attr_reader :parser, :registry, :
|
|
100
|
+
attr_reader :parser, :registry, :cache, :on_result, :hooks, :disable_detector, :sig_detector
|
|
101
|
+
|
|
102
|
+
def isolator
|
|
103
|
+
@isolator ||= build_isolator
|
|
104
|
+
end
|
|
93
105
|
|
|
94
106
|
def parse_subjects
|
|
95
107
|
files = resolve_target_files
|
|
@@ -97,10 +109,13 @@ class Evilution::Runner
|
|
|
97
109
|
end
|
|
98
110
|
|
|
99
111
|
def resolve_target_files
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
112
|
+
@resolve_target_files ||= if source_glob_target?
|
|
113
|
+
resolve_source_glob
|
|
114
|
+
elsif !config.target_files.empty?
|
|
115
|
+
config.target_files
|
|
116
|
+
else
|
|
117
|
+
Evilution::Git::ChangedFiles.new.call
|
|
118
|
+
end
|
|
104
119
|
end
|
|
105
120
|
|
|
106
121
|
def source_glob_target?
|
|
@@ -340,7 +355,7 @@ class Evilution::Runner
|
|
|
340
355
|
def run_mutations_parallel(mutations, baseline_result = nil)
|
|
341
356
|
integration = build_integration
|
|
342
357
|
pool = Evilution::Parallel::Pool.new(size: config.jobs, hooks: @hooks, item_timeout: config.timeout ? config.timeout * 2 : nil)
|
|
343
|
-
worker_isolator =
|
|
358
|
+
worker_isolator = build_isolator
|
|
344
359
|
spec_resolver = baseline_result&.failed? ? build_neutralization_resolver : nil
|
|
345
360
|
state = { results: [], survived_count: 0, truncated: false, completed: 0 }
|
|
346
361
|
|
|
@@ -483,9 +498,80 @@ class Evilution::Runner
|
|
|
483
498
|
end
|
|
484
499
|
|
|
485
500
|
def resolve_isolation
|
|
486
|
-
|
|
501
|
+
case config.isolation
|
|
502
|
+
when :fork
|
|
503
|
+
:fork
|
|
504
|
+
when :in_process
|
|
505
|
+
warn_in_process_under_rails if rails_root_detected?
|
|
506
|
+
:in_process
|
|
507
|
+
else # :auto
|
|
508
|
+
rails_root_detected? ? :fork : :in_process
|
|
509
|
+
end
|
|
510
|
+
end
|
|
511
|
+
|
|
512
|
+
def rails_root_detected?
|
|
513
|
+
return @rails_root_detected if defined?(@rails_root_detected)
|
|
514
|
+
|
|
515
|
+
@rails_root_detected = !detected_rails_root.nil?
|
|
516
|
+
end
|
|
517
|
+
|
|
518
|
+
def detected_rails_root
|
|
519
|
+
return @detected_rails_root if defined?(@detected_rails_root)
|
|
520
|
+
|
|
521
|
+
@detected_rails_root = Evilution::RailsDetector.rails_root_for_any(resolve_target_files)
|
|
522
|
+
end
|
|
523
|
+
|
|
524
|
+
def perform_preload
|
|
525
|
+
return if config.preload == false
|
|
526
|
+
return unless resolve_isolation == :fork
|
|
487
527
|
|
|
488
|
-
|
|
528
|
+
path = resolve_preload_path
|
|
529
|
+
return unless path
|
|
530
|
+
|
|
531
|
+
require File.expand_path(path)
|
|
532
|
+
rescue ScriptError, StandardError => e
|
|
533
|
+
raise Evilution::ConfigError.new(
|
|
534
|
+
"failed to preload #{path.inspect}: #{e.class}: #{e.message}",
|
|
535
|
+
file: path
|
|
536
|
+
)
|
|
537
|
+
end
|
|
538
|
+
|
|
539
|
+
def resolve_preload_path
|
|
540
|
+
if config.preload.is_a?(String)
|
|
541
|
+
unless File.file?(config.preload)
|
|
542
|
+
raise Evilution::ConfigError.new(
|
|
543
|
+
"preload file not found: #{config.preload.inspect}",
|
|
544
|
+
file: config.preload
|
|
545
|
+
)
|
|
546
|
+
end
|
|
547
|
+
return config.preload
|
|
548
|
+
end
|
|
549
|
+
|
|
550
|
+
root = detected_rails_root
|
|
551
|
+
return nil unless root
|
|
552
|
+
|
|
553
|
+
PRELOAD_CANDIDATES.each do |rel|
|
|
554
|
+
abs = File.join(root, rel)
|
|
555
|
+
return abs if File.file?(abs)
|
|
556
|
+
end
|
|
557
|
+
nil
|
|
558
|
+
end
|
|
559
|
+
|
|
560
|
+
# When the user explicitly requests InProcess on a Rails project, warn once
|
|
561
|
+
# per run. Rails wraps ActiveRecord transactions in
|
|
562
|
+
# Thread.handle_interrupt(Exception => :never), which defers Timeout's
|
|
563
|
+
# Thread#raise indefinitely — making InProcess unable to kill runaway mutants.
|
|
564
|
+
def warn_in_process_under_rails
|
|
565
|
+
return if config.quiet
|
|
566
|
+
return if @warned_in_process_under_rails
|
|
567
|
+
|
|
568
|
+
@warned_in_process_under_rails = true
|
|
569
|
+
$stderr.write(
|
|
570
|
+
"[evilution] warning: --isolation in_process is unsafe on Rails projects. " \
|
|
571
|
+
"ActiveRecord wraps transactions in Thread.handle_interrupt(Exception => :never), " \
|
|
572
|
+
"which swallows Timeout.timeout and can cause evilution to hang indefinitely on " \
|
|
573
|
+
"mutants that introduce infinite loops. Use --isolation fork for reliable interruption.\n"
|
|
574
|
+
)
|
|
489
575
|
end
|
|
490
576
|
|
|
491
577
|
def resolve_integration_class
|
data/lib/evilution/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: evilution
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.22.
|
|
4
|
+
version: 0.22.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Denis Kiselev
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-04-
|
|
11
|
+
date: 2026-04-12 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: diff-lcs
|
|
@@ -81,6 +81,7 @@ files:
|
|
|
81
81
|
- comparison_results/operator_classification.md
|
|
82
82
|
- comparison_results/operator_prioritization.md
|
|
83
83
|
- docs/ast_pattern_syntax.md
|
|
84
|
+
- docs/isolation.md
|
|
84
85
|
- docs/mutation_density_benchmark.md
|
|
85
86
|
- exe/evilution
|
|
86
87
|
- lib/evilution.rb
|
|
@@ -210,6 +211,7 @@ files:
|
|
|
210
211
|
- lib/evilution/parallel.rb
|
|
211
212
|
- lib/evilution/parallel/pool.rb
|
|
212
213
|
- lib/evilution/parallel/work_queue.rb
|
|
214
|
+
- lib/evilution/rails_detector.rb
|
|
213
215
|
- lib/evilution/related_spec_heuristic.rb
|
|
214
216
|
- lib/evilution/reporter.rb
|
|
215
217
|
- lib/evilution/reporter/cli.rb
|