evilution 0.22.0 → 0.22.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.beads/interactions.jsonl +6 -0
- data/.claude/settings.json +5 -0
- data/CHANGELOG.md +38 -0
- data/README.md +20 -4
- data/docs/isolation.md +115 -0
- data/lib/evilution/cli.rb +3 -0
- data/lib/evilution/config.rb +17 -2
- data/lib/evilution/integration/base.rb +37 -11
- data/lib/evilution/isolation/fork.rb +11 -3
- data/lib/evilution/isolation/in_process.rb +12 -3
- data/lib/evilution/mcp/mutate_tool.rb +5 -1
- data/lib/evilution/mutator/operator/symbol_literal.rb +9 -0
- data/lib/evilution/rails_detector.rb +57 -0
- data/lib/evilution/reporter/cli.rb +19 -0
- data/lib/evilution/reporter/html.rb +10 -1
- data/lib/evilution/reporter/json.rb +12 -1
- data/lib/evilution/result/mutation_result.rb +9 -2
- data/lib/evilution/runner.rb +120 -14
- data/lib/evilution/version.rb +1 -1
- data/script/memory_check +5 -5
- metadata +5 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2a90c882fed4185cf1514efbe1c560e05555efa19a749201e81d9287a65fefef
|
|
4
|
+
data.tar.gz: e5c02c5da957d3f9a647fd592b91c4ef82925020d5bb73bd0cde480c1435b26e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: db0a7c3ac58823326a8fc9d23b41dc928d8b1e45f7230c0c18dcdab133a9ec355d25f9303180043d41e9926387d90335920196b9d8651d0040eedb2982fd3b8c
|
|
7
|
+
data.tar.gz: 4a5b241452977ddd620ff4a082013b5ebcb2da3860579349ac58a8ac908d15a09862496e5418038eee52327aa46ccd1aac88ce21bb7d7cd2b7f3f931f2750e0f
|
data/.beads/interactions.jsonl
CHANGED
|
@@ -10,3 +10,9 @@
|
|
|
10
10
|
{"id":"int-0f073191","kind":"field_change","created_at":"2026-04-09T13:03:11.468115004Z","actor":"Denis Kiselev","issue_id":"EV-85","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
|
|
11
11
|
{"id":"int-91a9616c","kind":"field_change","created_at":"2026-04-09T13:04:05.459458165Z","actor":"Denis Kiselev","issue_id":"EV-69","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Closed"}}
|
|
12
12
|
{"id":"int-b4fe2b7b","kind":"field_change","created_at":"2026-04-09T13:42:59.996305852Z","actor":"Denis Kiselev","issue_id":"EV-277","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
|
|
13
|
+
{"id":"int-6671aef1","kind":"field_change","created_at":"2026-04-10T09:05:06.955840839Z","actor":"Denis Kiselev","issue_id":"EV-z42m","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged via PR #655"}}
|
|
14
|
+
{"id":"int-3a6ffc92","kind":"field_change","created_at":"2026-04-10T10:26:29.483210031Z","actor":"Denis Kiselev","issue_id":"EV-sgsb","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged via PR #656"}}
|
|
15
|
+
{"id":"int-1df69312","kind":"field_change","created_at":"2026-04-10T10:40:18.391377715Z","actor":"Denis Kiselev","issue_id":"EV-1l8w","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged"}}
|
|
16
|
+
{"id":"int-427bdc14","kind":"field_change","created_at":"2026-04-10T12:40:12.974701482Z","actor":"Denis Kiselev","issue_id":"EV-k6cz","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged as PR #659 — capture error_class and error_backtrace in MutationResult, thread through isolators + runner + JSON reporter, log in --verbose"}}
|
|
17
|
+
{"id":"int-d3431bcd","kind":"field_change","created_at":"2026-04-12T02:57:43.902279367Z","actor":"Denis Kiselev","issue_id":"EV-86l6","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
|
|
18
|
+
{"id":"int-f409d79d","kind":"field_change","created_at":"2026-04-12T02:57:44.180309214Z","actor":"Denis Kiselev","issue_id":"EV-o28o","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,43 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.22.2] - 2026-04-12
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- **Rails-aware isolation auto-selection** — `isolation: :auto` (the default) now detects Rails projects by walking up from target files looking for `config/application.rb`; when Rails is detected, isolation resolves to `:fork` instead of `:in_process`, preventing indefinite hangs caused by Rails' `Thread.handle_interrupt(Exception => :never)` masking `Timeout.timeout`'s `Thread#raise` (#662, #663, PR #665)
|
|
8
|
+
- **Parent-process preload for fork isolation** — new `--preload FILE` / `--no-preload` CLI flags and `preload:` config key; when fork isolation is active on a Rails project, evilution auto-detects and preloads `spec/rails_helper.rb` or `test/test_helper.rb` in the parent process so forked children inherit the loaded framework via copy-on-write, eliminating per-mutation Rails boot cost (#662, PR #665)
|
|
9
|
+
- **`Evilution::RailsDetector` module** — lightweight filesystem-only detection of Rails roots with memoized cache and thread-safe mutex; used by both isolation resolution and preload auto-detection (#662, PR #665)
|
|
10
|
+
- **Isolation strategy documentation** — `docs/isolation.md` explains the three strategies, the `handle_interrupt` hazard, and preload configuration (#662, PR #665)
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **`run_mutations_parallel` ignoring `--isolation`** — parallel mode hardcoded `Isolation::InProcess.new` for its worker isolator, silently overriding `config.isolation`; now uses `build_isolator` so `--jobs N --isolation fork` correctly uses fork per-mutation inside workers (#663, PR #665)
|
|
15
|
+
- **Rails detection failing for auto-resolved targets** — `detected_rails_root` used `config.target_files` which is empty in git-changed or source-glob modes; now uses memoized `resolve_target_files` so Rails is detected regardless of how targets were specified (PR #665)
|
|
16
|
+
- **`SyntaxError` in preload file escaping rescue** — `perform_preload` rescued `LoadError, StandardError` but `SyntaxError` is a `ScriptError` (not `StandardError`); now rescues `ScriptError, StandardError` (PR #665)
|
|
17
|
+
- **Isolator built eagerly before targets resolved** — `build_isolator` ran in `initialize` before `resolve_target_files`; now lazy-initialized on first use, ensuring Rails detection has resolved targets available (PR #665)
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
|
|
21
|
+
- **Isolation warning for explicit `in_process` under Rails** — when a user explicitly passes `--isolation in_process` on a detected Rails project, evilution emits a one-shot stderr warning about the `handle_interrupt` hang hazard and proceeds; suppressed by `--quiet` (#662, PR #665)
|
|
22
|
+
|
|
23
|
+
## [0.22.1] - 2026-04-10
|
|
24
|
+
|
|
25
|
+
### Added
|
|
26
|
+
|
|
27
|
+
- **Error class and backtrace capture** — `MutationResult` now stores `error_class` and `error_backtrace` alongside `error_message`; the backtrace array is duplicated and frozen to keep results immutable; both fields are threaded through `Isolation::Fork` (Marshal-safe across the IPC pipe), `Isolation::InProcess`, and the runner's `compact_result` / `rebuild_results` path (#648, PR #659)
|
|
28
|
+
- **Verbose error diagnostics** — `--verbose` now logs error class, message, and the first 5 backtrace lines for errored mutations (previously `--verbose` only showed memory/GC stats, leaving errors invisible) (#648, PR #659)
|
|
29
|
+
- **Error details in JSON reports** — JSON reporter output includes `error_class` and `error_backtrace` fields under `errors[]` entries when present, so downstream tools (CI, MCP consumers) can surface failure causes without re-running (#648, PR #659)
|
|
30
|
+
|
|
31
|
+
### Fixed
|
|
32
|
+
|
|
33
|
+
- **Silent load-time crashes in `Isolation::Fork`** — mutations that raised non-`SyntaxError` script errors at load time (e.g. `NoMethodError: super called outside of method`) escaped `Integration::Base`'s narrow rescue and either surfaced cryptically or went silent under fork isolation; both isolators now rescue `ScriptError, StandardError` as a safety net and report them as `:error` status with full class and backtrace (#646, PR #656)
|
|
34
|
+
- **`symbol_literal` operator breaking keyword arguments** — mutating symbols in label form (`foo:` inside hash literals or keyword arguments) produced invalid Ruby source; the operator now detects label-form symbols via Prism's `closing_loc` and skips them, only mutating standalone symbol literals (`:foo`) (#647, PR #657)
|
|
35
|
+
- **Syntax errors in mutated source crashing in-process runs** — `Integration::Base#apply_mutation` now captures `SyntaxError` during `require`/`load` and returns a structured error result instead of propagating the exception up through `call`; error results include the error class and backtrace for diagnosis (#644, #645, PR #653, PR #655)
|
|
36
|
+
|
|
37
|
+
### Changed
|
|
38
|
+
|
|
39
|
+
- **Integration::Base refactor** — `apply_mutation` split into `apply_via_require` and `apply_via_load` helpers; rescue scope moved from `#call` to `#apply_mutation` so load-time errors return a result hash while abstract-method `NotImplementedError`s still propagate as intended
|
|
40
|
+
|
|
3
41
|
## [0.22.0] - 2026-04-09
|
|
4
42
|
|
|
5
43
|
### Added
|
data/README.md
CHANGED
|
@@ -56,7 +56,7 @@ evilution [command] [options] [files...]
|
|
|
56
56
|
| `-j`, `--jobs N` | Integer | 1 | Number of parallel workers. Uses demand-driven work distribution with pipe-based IPC. |
|
|
57
57
|
| `--no-baseline` | Boolean | _(enabled)_ | Skip baseline test suite check. By default, a baseline run detects pre-existing failures and marks those mutations as `neutral`. |
|
|
58
58
|
| `--fail-fast [N]` | Integer | _(none)_ | Stop after N surviving mutants (default 1 if no value given). |
|
|
59
|
-
| `-v`, `--verbose` | Boolean | false | Verbose output with RSS memory and GC stats per phase and per mutation. |
|
|
59
|
+
| `-v`, `--verbose` | Boolean | false | Verbose output with RSS memory and GC stats per phase and per mutation; also prints error class, message, and first 5 backtrace lines for errored mutations. |
|
|
60
60
|
| `--suggest-tests` | Boolean | false | Generate concrete test code in suggestions (RSpec or Minitest, based on `--integration`). |
|
|
61
61
|
| `-q`, `--quiet` | Boolean | false | Suppress output. |
|
|
62
62
|
| `--stdin` | Boolean | false | Read target file paths from stdin (one per line). |
|
|
@@ -64,6 +64,9 @@ evilution [command] [options] [files...]
|
|
|
64
64
|
| `--incremental` | Boolean | false | Cache killed/timeout results; skip unchanged mutations on re-runs. |
|
|
65
65
|
| `--save-session` | Boolean | false | Persist results as timestamped JSON under `.evilution/results/`. |
|
|
66
66
|
| `--no-progress` | Boolean | _(enabled)_ | Disable the TTY progress bar. |
|
|
67
|
+
| `--isolation MODE` | String | `auto` | Isolation strategy: `auto`, `fork`, or `in_process`. `auto` selects `fork` for Rails projects. See [docs/isolation.md](docs/isolation.md). |
|
|
68
|
+
| `--preload FILE` | String | _(auto)_ | File to require in parent before forking workers (e.g. `spec/rails_helper.rb`). Auto-detected for Rails. |
|
|
69
|
+
| `--no-preload` | Boolean | _(enabled)_ | Disable parent-process preload. |
|
|
67
70
|
| `--skip-heredoc-literals` | Boolean | false | Skip all string literal mutations inside heredocs. |
|
|
68
71
|
| `--show-disabled` | Boolean | false | Report mutations skipped by `# evilution:disable` comments. |
|
|
69
72
|
| `--baseline-session PATH` | String | _(none)_ | Saved session file for HTML report comparison. |
|
|
@@ -90,6 +93,8 @@ Creates `.evilution.yml`:
|
|
|
90
93
|
# integration: rspec # test framework: rspec, minitest
|
|
91
94
|
# suggest_tests: false # concrete test code in suggestions (matches integration)
|
|
92
95
|
# save_session: false # persist results under .evilution/results/
|
|
96
|
+
# isolation: auto # auto | fork | in_process (auto selects fork for Rails)
|
|
97
|
+
# preload: null # path to preload before forking; false to disable; auto-detects for Rails
|
|
93
98
|
# skip_heredoc_literals: false # skip all string literal mutations inside heredocs
|
|
94
99
|
# show_disabled: false # report mutations skipped by disable comments
|
|
95
100
|
# baseline_session: null # path to session file for HTML comparison
|
|
@@ -163,7 +168,14 @@ Use `--format json` for machine-readable output. Schema:
|
|
|
163
168
|
],
|
|
164
169
|
"killed": ["... same shape as survived entries ..."],
|
|
165
170
|
"timed_out": ["... same shape as survived entries ..."],
|
|
166
|
-
"errors": [
|
|
171
|
+
"errors": [
|
|
172
|
+
{
|
|
173
|
+
"... same shape as survived entries, plus: ...": "",
|
|
174
|
+
"error_message": "string (optional) — error message from the failing mutation",
|
|
175
|
+
"error_class": "string (optional) — exception class name (e.g. 'SyntaxError', 'NoMethodError')",
|
|
176
|
+
"error_backtrace": ["string (optional) — first 5 backtrace lines from the exception"]
|
|
177
|
+
}
|
|
178
|
+
]
|
|
167
179
|
}
|
|
168
180
|
```
|
|
169
181
|
|
|
@@ -364,7 +376,11 @@ For each entry in `survived[]`:
|
|
|
364
376
|
4. Write a test that would fail if the mutation were applied
|
|
365
377
|
5. Re-run evilution on just that file to verify the mutant is now killed
|
|
366
378
|
|
|
367
|
-
### 7.
|
|
379
|
+
### 7. Diagnosing errored mutations
|
|
380
|
+
|
|
381
|
+
Entries in the JSON `errors[]` array represent mutations that raised an exception (syntax error, load failure, or runtime crash) rather than producing a test outcome. Each entry includes `error_class`, `error_message`, and the first 5 `error_backtrace` lines. Use these fields to decide whether the error is a bug in the mutation operator (file an issue), a load-time problem in the mutated source (often `NoMethodError: super called outside of method` or constant-redefinition issues), or a genuine crash that the original tests should have caught. Run with `--verbose` to stream the same error details to stderr during the run.
|
|
382
|
+
|
|
383
|
+
### 8. CI gate
|
|
368
384
|
|
|
369
385
|
```bash
|
|
370
386
|
bundle exec evilution run lib/ --format json --min-score 0.8 --quiet
|
|
@@ -394,7 +410,7 @@ Tests 4 paths (InProcess isolation, Fork isolation, mutation generation + stripp
|
|
|
394
410
|
2. **Extract** — Methods are identified as mutation subjects
|
|
395
411
|
3. **Filter** — Disable comments, Sorbet `sig` blocks, and AST ignore patterns exclude mutations before execution
|
|
396
412
|
4. **Mutate** — 72 operators produce text replacements at precise byte offsets (source-level surgery, no AST unparsing); heredoc literal text is skipped by default
|
|
397
|
-
5. **Isolate** — Mutations are applied to temporary file copies (never modifying originals); load-path redirection ensures `require` resolves the mutated copy. Default isolation is in-process; `--isolation fork`
|
|
413
|
+
5. **Isolate** — Mutations are applied to temporary file copies (never modifying originals); load-path redirection ensures `require` resolves the mutated copy. Default isolation is in-process for plain Ruby projects and fork for Rails projects (auto-detected); `--isolation fork` forces forked child processes. Both sequential and parallel (`--jobs N`) modes respect the configured isolation strategy
|
|
398
414
|
6. **Test** — The configured test framework (RSpec or Minitest) executes against the mutated source
|
|
399
415
|
7. **Collect** — Source strings and AST nodes are released after use to minimize memory retention
|
|
400
416
|
8. **Report** — Results aggregated into text, JSON, or HTML, including efficiency metrics and peak memory usage
|
data/docs/isolation.md
ADDED
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
# Isolation strategies
|
|
2
|
+
|
|
3
|
+
Evilution runs every mutant inside an isolated environment so a misbehaving
|
|
4
|
+
mutant cannot corrupt the runner's state or the surrounding test suite. The
|
|
5
|
+
`--isolation` flag (and the `isolation:` config key) selects which strategy
|
|
6
|
+
is used.
|
|
7
|
+
|
|
8
|
+
## Strategies
|
|
9
|
+
|
|
10
|
+
| Strategy | What it does | When to use |
|
|
11
|
+
| ------------ | ---------------------------------------------------------------------- | ---------------------------------------------------------- |
|
|
12
|
+
| `auto` | Default. Picks `fork` for Rails projects, `in_process` otherwise. | Leave this on unless you have a specific reason to change. |
|
|
13
|
+
| `fork` | Forks a fresh child process per mutant. Parent `SIGKILL`s on timeout. | Rails / ActiveRecord projects; any code that uses mutexes, monitors, or async-interrupt masks. |
|
|
14
|
+
| `in_process` | Runs the mutant inside the runner process under `Timeout.timeout`. | Pure-Ruby libraries that do not use async-interrupt masks. |
|
|
15
|
+
|
|
16
|
+
## Why fork is the default for Rails projects
|
|
17
|
+
|
|
18
|
+
Rails wraps every ActiveRecord transaction in
|
|
19
|
+
`Thread.handle_interrupt(Exception => :never)` to guarantee that a transaction
|
|
20
|
+
either commits cleanly or rolls back cleanly — it must never exit the
|
|
21
|
+
transaction block with a half-delivered exception. The mask is
|
|
22
|
+
[load-bearing for transaction correctness][rails-handle-interrupt].
|
|
23
|
+
|
|
24
|
+
That mask interacts badly with `Timeout.timeout`. The Timeout gem schedules a
|
|
25
|
+
timer thread which fires `Thread#raise Timeout::Error` at the main thread when
|
|
26
|
+
the deadline hits. Ruby receives the `raise`, inspects the interrupt mask, sees
|
|
27
|
+
`Exception => :never`, and **queues the exception for later delivery** — "later"
|
|
28
|
+
meaning "when the masked block exits". If the mutant is stuck in an infinite
|
|
29
|
+
loop inside the transaction, the block never exits, the queued exception is
|
|
30
|
+
never delivered, and the runner hangs indefinitely.
|
|
31
|
+
|
|
32
|
+
This is not a bug in Timeout or in Rails. Each component is correct in
|
|
33
|
+
isolation. They simply do not compose for in-process timeout-based
|
|
34
|
+
cancellation. The only primitive that can escape a masked-interrupt section
|
|
35
|
+
in the same thread is an out-of-band kernel signal — `SIGKILL` from another
|
|
36
|
+
process. That is exactly what the `fork` strategy provides.
|
|
37
|
+
|
|
38
|
+
Because of this, `auto` resolves to `fork` whenever the target files live
|
|
39
|
+
under a detected Rails root (i.e. a directory containing
|
|
40
|
+
`config/application.rb`). If you explicitly pass `--isolation in_process` on
|
|
41
|
+
a Rails project, evilution emits a warning naming the hazard and proceeds
|
|
42
|
+
anyway — sometimes you know the code under test never enters a masked
|
|
43
|
+
section, and that is your call to make.
|
|
44
|
+
|
|
45
|
+
The same hazard applies to any Ruby code that uses
|
|
46
|
+
`Thread.handle_interrupt(... => :never)`: `Mutex#synchronize`, `Monitor#synchronize`,
|
|
47
|
+
`Queue`, `ActiveSupport::Notifications::Fanout` listeners, and custom cleanup
|
|
48
|
+
blocks that wrap "must complete" sections. If your target code can touch any
|
|
49
|
+
of those, prefer `--isolation fork`.
|
|
50
|
+
|
|
51
|
+
## Parent-process preload
|
|
52
|
+
|
|
53
|
+
Fork isolation's one downside is that every child pays the cost of loading
|
|
54
|
+
its test framework from scratch. For a Rails app, that is 5–15 seconds of
|
|
55
|
+
`require "rails/all"` per mutant — multiplied by hundreds of mutants, the
|
|
56
|
+
run takes hours.
|
|
57
|
+
|
|
58
|
+
To avoid that cost, evilution preloads a bootstrap file in the **parent**
|
|
59
|
+
process before the mutation loop begins. Once Rails is loaded in the parent,
|
|
60
|
+
every forked child inherits the loaded state via copy-on-write, and the child
|
|
61
|
+
runs its test near-instantly against an already-loaded framework.
|
|
62
|
+
|
|
63
|
+
### Automatic preload
|
|
64
|
+
|
|
65
|
+
For Rails projects (auto-detected via `config/application.rb`), evilution
|
|
66
|
+
automatically looks for these files in order and preloads the first one it
|
|
67
|
+
finds:
|
|
68
|
+
|
|
69
|
+
1. `spec/rails_helper.rb` (RSpec)
|
|
70
|
+
2. `test/test_helper.rb` (Minitest)
|
|
71
|
+
|
|
72
|
+
No configuration needed.
|
|
73
|
+
|
|
74
|
+
### Explicit preload
|
|
75
|
+
|
|
76
|
+
Override the auto-detected path with the `--preload` flag or the `preload:`
|
|
77
|
+
config key:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
bundle exec evilution run app/models/user.rb --preload config/evilution_boot.rb
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
```yaml
|
|
84
|
+
# .evilution.yml
|
|
85
|
+
preload: config/evilution_boot.rb
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
The path is resolved relative to the working directory. Preload failures
|
|
89
|
+
(the file does not exist, or it raises during `require`) are fatal —
|
|
90
|
+
evilution aborts with an `Evilution::ConfigError` that includes the original
|
|
91
|
+
exception.
|
|
92
|
+
|
|
93
|
+
### Disabling preload
|
|
94
|
+
|
|
95
|
+
Pass `--no-preload` on the CLI or set `preload: false` in `.evilution.yml`:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
bundle exec evilution run app/models/user.rb --no-preload
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
The MCP tool (`evilution-mutate`) disables preload unconditionally, because
|
|
102
|
+
the MCP server is a long-lived process that handles runs from different
|
|
103
|
+
projects — preloading one project's Rails stack into a shared process would
|
|
104
|
+
poison subsequent runs.
|
|
105
|
+
|
|
106
|
+
## Related flags
|
|
107
|
+
|
|
108
|
+
- `--timeout N` sets the per-mutation time limit. Under `fork`, this drives
|
|
109
|
+
SIGKILL. Under `in_process`, this drives `Timeout.timeout` and is subject
|
|
110
|
+
to the interrupt-mask hazard described above.
|
|
111
|
+
- `--jobs N` runs N workers in parallel. The parallel pool respects the
|
|
112
|
+
configured isolation strategy, so `--jobs 4 --isolation fork` uses fork
|
|
113
|
+
isolation per-mutation inside each worker.
|
|
114
|
+
|
|
115
|
+
[rails-handle-interrupt]: https://github.com/rails/rails/blob/main/activesupport/lib/active_support/concurrency/thread_monitor.rb
|
data/lib/evilution/cli.rb
CHANGED
|
@@ -249,6 +249,9 @@ class Evilution::CLI
|
|
|
249
249
|
opts.on("--incremental", "Cache killed/timeout results; skip re-running them on unchanged files") { @options[:incremental] = true }
|
|
250
250
|
opts.on("--integration NAME", "Test integration: rspec, minitest (default: rspec)") { |i| @options[:integration] = i }
|
|
251
251
|
opts.on("--isolation STRATEGY", "Isolation: auto, fork, in_process (default: auto)") { |s| @options[:isolation] = s }
|
|
252
|
+
opts.on("--preload FILE", "Preload FILE in the parent process before forking " \
|
|
253
|
+
"(default: auto-detect spec/rails_helper.rb for Rails projects)") { |f| @options[:preload] = f }
|
|
254
|
+
opts.on("--no-preload", "Disable parent-process preload even for Rails projects") { @options[:preload] = false }
|
|
252
255
|
opts.on("--stdin", "Read target file paths from stdin (one per line)") { @options[:stdin] = true }
|
|
253
256
|
opts.on("--suggest-tests", "Generate concrete test code in suggestions (RSpec or Minitest)") { @options[:suggest_tests] = true }
|
|
254
257
|
opts.on("--no-progress", "Disable progress bar") { @options[:progress] = false }
|
data/lib/evilution/config.rb
CHANGED
|
@@ -26,7 +26,8 @@ class Evilution::Config
|
|
|
26
26
|
ignore_patterns: [],
|
|
27
27
|
show_disabled: false,
|
|
28
28
|
baseline_session: nil,
|
|
29
|
-
skip_heredoc_literals: false
|
|
29
|
+
skip_heredoc_literals: false,
|
|
30
|
+
preload: nil
|
|
30
31
|
}.freeze
|
|
31
32
|
|
|
32
33
|
attr_reader :target_files, :timeout, :format,
|
|
@@ -34,7 +35,7 @@ class Evilution::Config
|
|
|
34
35
|
:jobs, :fail_fast, :baseline, :isolation, :incremental, :suggest_tests,
|
|
35
36
|
:progress, :save_session, :line_ranges, :spec_files, :hooks,
|
|
36
37
|
:ignore_patterns, :show_disabled, :baseline_session,
|
|
37
|
-
:skip_heredoc_literals
|
|
38
|
+
:skip_heredoc_literals, :preload
|
|
38
39
|
|
|
39
40
|
def initialize(**options)
|
|
40
41
|
file_options = options.delete(:skip_config_file) ? {} : load_config_file
|
|
@@ -140,6 +141,11 @@ class Evilution::Config
|
|
|
140
141
|
# Skip all string literal mutations inside heredocs (default: false)
|
|
141
142
|
# skip_heredoc_literals: false
|
|
142
143
|
|
|
144
|
+
# Preload file required in the parent process before forking workers.
|
|
145
|
+
# For Rails projects, spec/rails_helper.rb or test/test_helper.rb is
|
|
146
|
+
# auto-detected when isolation resolves to :fork. Set to false to disable.
|
|
147
|
+
# preload: spec/rails_helper.rb # or test/test_helper.rb
|
|
148
|
+
|
|
143
149
|
# Hooks: Ruby files returning a Proc, keyed by lifecycle event
|
|
144
150
|
# hooks:
|
|
145
151
|
# worker_process_start: config/evilution_hooks/worker_start.rb
|
|
@@ -190,6 +196,15 @@ class Evilution::Config
|
|
|
190
196
|
@baseline_session = merged[:baseline_session]
|
|
191
197
|
@skip_heredoc_literals = merged[:skip_heredoc_literals]
|
|
192
198
|
@hooks = validate_hooks(merged[:hooks])
|
|
199
|
+
@preload = validate_preload(merged[:preload])
|
|
200
|
+
end
|
|
201
|
+
|
|
202
|
+
def validate_preload(value)
|
|
203
|
+
return nil if value.nil?
|
|
204
|
+
return false if value == false
|
|
205
|
+
return value if value.is_a?(String)
|
|
206
|
+
|
|
207
|
+
raise Evilution::ConfigError, "preload must be nil, false, or a String path, got #{value.inspect}"
|
|
193
208
|
end
|
|
194
209
|
|
|
195
210
|
def validate_integration(value)
|
|
@@ -22,7 +22,9 @@ class Evilution::Integration::Base
|
|
|
22
22
|
@temp_dir = nil
|
|
23
23
|
ensure_framework_loaded
|
|
24
24
|
fire_hook(:mutation_insert_pre, mutation: mutation, file_path: mutation.file_path)
|
|
25
|
-
apply_mutation(mutation)
|
|
25
|
+
load_error = apply_mutation(mutation)
|
|
26
|
+
return load_error if load_error
|
|
27
|
+
|
|
26
28
|
fire_hook(:mutation_insert_post, mutation: mutation, file_path: mutation.file_path)
|
|
27
29
|
run_tests(mutation)
|
|
28
30
|
ensure
|
|
@@ -58,18 +60,42 @@ class Evilution::Integration::Base
|
|
|
58
60
|
subpath = resolve_require_subpath(mutation.file_path)
|
|
59
61
|
|
|
60
62
|
if subpath
|
|
61
|
-
|
|
62
|
-
FileUtils.mkdir_p(File.dirname(dest))
|
|
63
|
-
File.write(dest, mutation.mutated_source)
|
|
64
|
-
$LOAD_PATH.unshift(@temp_dir)
|
|
65
|
-
displace_loaded_feature(mutation.file_path)
|
|
63
|
+
apply_via_require(mutation, subpath)
|
|
66
64
|
else
|
|
67
|
-
|
|
68
|
-
dest = File.join(@temp_dir, absolute)
|
|
69
|
-
FileUtils.mkdir_p(File.dirname(dest))
|
|
70
|
-
File.write(dest, mutation.mutated_source)
|
|
71
|
-
load(dest)
|
|
65
|
+
apply_via_load(mutation)
|
|
72
66
|
end
|
|
67
|
+
nil
|
|
68
|
+
rescue SyntaxError => e
|
|
69
|
+
{
|
|
70
|
+
passed: false,
|
|
71
|
+
error: "syntax error in mutated source: #{e.message}",
|
|
72
|
+
error_class: e.class.name,
|
|
73
|
+
error_backtrace: Array(e.backtrace).first(5)
|
|
74
|
+
}
|
|
75
|
+
rescue ScriptError, StandardError => e
|
|
76
|
+
{
|
|
77
|
+
passed: false,
|
|
78
|
+
error: "#{e.class}: #{e.message}",
|
|
79
|
+
error_class: e.class.name,
|
|
80
|
+
error_backtrace: Array(e.backtrace).first(5)
|
|
81
|
+
}
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def apply_via_require(mutation, subpath)
|
|
85
|
+
dest = File.join(@temp_dir, subpath)
|
|
86
|
+
FileUtils.mkdir_p(File.dirname(dest))
|
|
87
|
+
File.write(dest, mutation.mutated_source)
|
|
88
|
+
$LOAD_PATH.unshift(@temp_dir)
|
|
89
|
+
displace_loaded_feature(mutation.file_path)
|
|
90
|
+
require(subpath.delete_suffix(".rb"))
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
def apply_via_load(mutation)
|
|
94
|
+
absolute = File.expand_path(mutation.file_path)
|
|
95
|
+
dest = File.join(@temp_dir, absolute)
|
|
96
|
+
FileUtils.mkdir_p(File.dirname(dest))
|
|
97
|
+
File.write(dest, mutation.mutated_source)
|
|
98
|
+
load(dest)
|
|
73
99
|
end
|
|
74
100
|
|
|
75
101
|
def restore_original(_mutation)
|
|
@@ -57,8 +57,13 @@ class Evilution::Isolation::Fork
|
|
|
57
57
|
def execute_in_child(mutation, test_command)
|
|
58
58
|
result = test_command.call(mutation)
|
|
59
59
|
{ child_rss_kb: Evilution::Memory.rss_kb }.merge(result)
|
|
60
|
-
rescue StandardError => e
|
|
61
|
-
{
|
|
60
|
+
rescue ScriptError, StandardError => e
|
|
61
|
+
{
|
|
62
|
+
passed: false,
|
|
63
|
+
error: e.message,
|
|
64
|
+
error_class: e.class.name,
|
|
65
|
+
error_backtrace: Array(e.backtrace).first(5)
|
|
66
|
+
}
|
|
62
67
|
end
|
|
63
68
|
|
|
64
69
|
def wait_for_result(pid, read_io, timeout)
|
|
@@ -107,7 +112,10 @@ class Evilution::Isolation::Fork
|
|
|
107
112
|
duration: duration,
|
|
108
113
|
test_command: result[:test_command],
|
|
109
114
|
child_rss_kb: result[:child_rss_kb],
|
|
110
|
-
parent_rss_kb: parent_rss_kb
|
|
115
|
+
parent_rss_kb: parent_rss_kb,
|
|
116
|
+
error_message: result[:error],
|
|
117
|
+
error_class: result[:error_class],
|
|
118
|
+
error_backtrace: result[:error_backtrace]
|
|
111
119
|
)
|
|
112
120
|
end
|
|
113
121
|
end
|
|
@@ -34,8 +34,14 @@ class Evilution::Isolation::InProcess
|
|
|
34
34
|
{ timeout: false }.merge(result)
|
|
35
35
|
rescue Timeout::Error
|
|
36
36
|
{ timeout: true }
|
|
37
|
-
rescue StandardError => e
|
|
38
|
-
{
|
|
37
|
+
rescue ScriptError, StandardError => e
|
|
38
|
+
{
|
|
39
|
+
timeout: false,
|
|
40
|
+
passed: false,
|
|
41
|
+
error: e.message,
|
|
42
|
+
error_class: e.class.name,
|
|
43
|
+
error_backtrace: Array(e.backtrace).first(5)
|
|
44
|
+
}
|
|
39
45
|
end
|
|
40
46
|
|
|
41
47
|
def suppress_output
|
|
@@ -74,7 +80,10 @@ class Evilution::Isolation::InProcess
|
|
|
74
80
|
test_command: result[:test_command],
|
|
75
81
|
child_rss_kb: rss_after,
|
|
76
82
|
memory_delta_kb: memory_delta_kb,
|
|
77
|
-
parent_rss_kb: rss_before
|
|
83
|
+
parent_rss_kb: rss_before,
|
|
84
|
+
error_message: result[:error],
|
|
85
|
+
error_class: result[:error_class],
|
|
86
|
+
error_backtrace: result[:error_backtrace]
|
|
78
87
|
)
|
|
79
88
|
end
|
|
80
89
|
end
|
|
@@ -111,7 +111,11 @@ class Evilution::MCP::MutateTool < MCP::Tool
|
|
|
111
111
|
end
|
|
112
112
|
|
|
113
113
|
def build_config_opts(files, line_ranges, target, timeout, jobs, fail_fast, spec, suggest_tests)
|
|
114
|
-
|
|
114
|
+
# Preload is disabled for MCP invocations: `require`-ing Rails into the
|
|
115
|
+
# long-lived MCP server would poison subsequent runs against other
|
|
116
|
+
# projects. MCP users who want the speedup should use the CLI.
|
|
117
|
+
opts = { target_files: files, line_ranges: line_ranges, format: :json, quiet: true, skip_config_file: true,
|
|
118
|
+
preload: false }
|
|
115
119
|
opts[:target] = target if target
|
|
116
120
|
opts[:timeout] = timeout if timeout
|
|
117
121
|
opts[:jobs] = jobs if jobs
|
|
@@ -4,6 +4,8 @@ require_relative "../operator"
|
|
|
4
4
|
|
|
5
5
|
class Evilution::Mutator::Operator::SymbolLiteral < Evilution::Mutator::Base
|
|
6
6
|
def visit_symbol_node(node)
|
|
7
|
+
return super if label_form?(node)
|
|
8
|
+
|
|
7
9
|
add_mutation(
|
|
8
10
|
offset: node.location.start_offset,
|
|
9
11
|
length: node.location.length,
|
|
@@ -20,4 +22,11 @@ class Evilution::Mutator::Operator::SymbolLiteral < Evilution::Mutator::Base
|
|
|
20
22
|
|
|
21
23
|
super
|
|
22
24
|
end
|
|
25
|
+
|
|
26
|
+
private
|
|
27
|
+
|
|
28
|
+
def label_form?(node)
|
|
29
|
+
closing = node.closing_loc
|
|
30
|
+
!closing.nil? && closing.slice == ":"
|
|
31
|
+
end
|
|
23
32
|
end
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Evilution::RailsDetector
|
|
4
|
+
MARKER = File.join("config", "application.rb").freeze
|
|
5
|
+
|
|
6
|
+
@cache = {}
|
|
7
|
+
@mutex = Mutex.new
|
|
8
|
+
|
|
9
|
+
class << self
|
|
10
|
+
def rails_root_for(path)
|
|
11
|
+
return nil if path.nil?
|
|
12
|
+
|
|
13
|
+
dir = starting_dir(path)
|
|
14
|
+
return nil if dir.nil?
|
|
15
|
+
|
|
16
|
+
@mutex.synchronize do
|
|
17
|
+
return @cache[dir] if @cache.key?(dir)
|
|
18
|
+
|
|
19
|
+
@cache[dir] = walk_up(dir)
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def rails_root_for_any(paths)
|
|
24
|
+
Array(paths).each do |path|
|
|
25
|
+
root = rails_root_for(path)
|
|
26
|
+
return root if root
|
|
27
|
+
end
|
|
28
|
+
nil
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def reset_cache!
|
|
32
|
+
@mutex.synchronize { @cache.clear }
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
private
|
|
36
|
+
|
|
37
|
+
def starting_dir(path)
|
|
38
|
+
return File.expand_path(path) if File.directory?(path)
|
|
39
|
+
return File.expand_path(File.dirname(path)) if File.file?(path)
|
|
40
|
+
|
|
41
|
+
parent = File.expand_path(File.dirname(path))
|
|
42
|
+
File.directory?(parent) ? parent : nil
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def walk_up(dir)
|
|
46
|
+
current = dir
|
|
47
|
+
loop do
|
|
48
|
+
return current if File.file?(File.join(current, MARKER))
|
|
49
|
+
|
|
50
|
+
parent = File.dirname(current)
|
|
51
|
+
return nil if parent == current
|
|
52
|
+
|
|
53
|
+
current = parent
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
@@ -19,6 +19,7 @@ class Evilution::Reporter::CLI
|
|
|
19
19
|
append_survived(lines, summary)
|
|
20
20
|
append_neutral(lines, summary)
|
|
21
21
|
append_equivalent(lines, summary)
|
|
22
|
+
append_errors(lines, summary)
|
|
22
23
|
append_disabled(lines, summary)
|
|
23
24
|
lines << ""
|
|
24
25
|
lines << "[TRUNCATED] Stopped early due to --fail-fast" if summary.truncated?
|
|
@@ -54,6 +55,24 @@ class Evilution::Reporter::CLI
|
|
|
54
55
|
summary.equivalent_results.each { |result| lines << format_neutral(result) }
|
|
55
56
|
end
|
|
56
57
|
|
|
58
|
+
def append_errors(lines, summary)
|
|
59
|
+
errored = summary.results.select(&:error?)
|
|
60
|
+
return if errored.empty?
|
|
61
|
+
|
|
62
|
+
lines << ""
|
|
63
|
+
lines << "Errored mutations:"
|
|
64
|
+
errored.each { |result| lines << format_error(result) }
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
def format_error(result)
|
|
68
|
+
mutation = result.mutation
|
|
69
|
+
header = " #{mutation.operator_name}: #{mutation.file_path}:#{mutation.line}"
|
|
70
|
+
return header unless result.error_message
|
|
71
|
+
|
|
72
|
+
indented = result.error_message.lines.map { |line| " #{line.chomp}" }.join("\n")
|
|
73
|
+
"#{header}\n#{indented}"
|
|
74
|
+
end
|
|
75
|
+
|
|
57
76
|
def append_disabled(lines, summary)
|
|
58
77
|
return unless summary.disabled_mutations.any?
|
|
59
78
|
|
|
@@ -145,8 +145,10 @@ class Evilution::Reporter::HTML
|
|
|
145
145
|
def build_map_entry(result)
|
|
146
146
|
mutation = result.mutation
|
|
147
147
|
status = result.status.to_s
|
|
148
|
+
title_text = normalize_title(result.error_message)
|
|
149
|
+
title_attr = title_text ? %( title="#{h(title_text)}") : ""
|
|
148
150
|
<<~HTML.chomp
|
|
149
|
-
<div class="map-line #{status}">
|
|
151
|
+
<div class="map-line #{status}"#{title_attr}>
|
|
150
152
|
<span class="line-number">line #{mutation.line}</span>
|
|
151
153
|
<span class="operator">#{h(mutation.operator_name)}</span>
|
|
152
154
|
<span class="status-badge #{status}">#{status}</span>
|
|
@@ -154,6 +156,13 @@ class Evilution::Reporter::HTML
|
|
|
154
156
|
HTML
|
|
155
157
|
end
|
|
156
158
|
|
|
159
|
+
def normalize_title(message)
|
|
160
|
+
return nil if message.nil?
|
|
161
|
+
|
|
162
|
+
normalized = message.gsub(/\s+/, " ").strip
|
|
163
|
+
normalized.empty? ? nil : normalized
|
|
164
|
+
end
|
|
165
|
+
|
|
157
166
|
def build_survived_details(survived)
|
|
158
167
|
return "" if survived.empty?
|
|
159
168
|
|
|
@@ -76,10 +76,21 @@ class Evilution::Reporter::JSON
|
|
|
76
76
|
}
|
|
77
77
|
detail[:suggestion] = @suggestion.suggestion_for(mutation) if result.status == :survived
|
|
78
78
|
detail[:test_command] = result.test_command if result.test_command
|
|
79
|
+
append_memory_fields(detail, result)
|
|
80
|
+
append_error_fields(detail, result)
|
|
81
|
+
detail
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def append_memory_fields(detail, result)
|
|
79
85
|
detail[:parent_rss_kb] = result.parent_rss_kb if result.parent_rss_kb
|
|
80
86
|
detail[:child_rss_kb] = result.child_rss_kb if result.child_rss_kb
|
|
81
87
|
detail[:memory_delta_kb] = result.memory_delta_kb if result.memory_delta_kb
|
|
82
|
-
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def append_error_fields(detail, result)
|
|
91
|
+
detail[:error_message] = result.error_message if result.error_message
|
|
92
|
+
detail[:error_class] = result.error_class if result.error_class
|
|
93
|
+
detail[:error_backtrace] = result.error_backtrace if result.error_backtrace
|
|
83
94
|
end
|
|
84
95
|
|
|
85
96
|
def build_coverage_gaps(summary)
|
|
@@ -6,11 +6,15 @@ class Evilution::Result::MutationResult
|
|
|
6
6
|
STATUSES = %i[killed survived timeout error neutral equivalent].freeze
|
|
7
7
|
|
|
8
8
|
attr_reader :mutation, :status, :duration, :killing_test, :test_command,
|
|
9
|
-
:child_rss_kb, :memory_delta_kb, :parent_rss_kb
|
|
9
|
+
:child_rss_kb, :memory_delta_kb, :parent_rss_kb,
|
|
10
|
+
:error_message, :error_class, :error_backtrace
|
|
10
11
|
|
|
12
|
+
# rubocop:disable Metrics/ParameterLists
|
|
11
13
|
def initialize(mutation:, status:, duration: 0.0, killing_test: nil,
|
|
12
14
|
test_command: nil, child_rss_kb: nil, memory_delta_kb: nil,
|
|
13
|
-
parent_rss_kb: nil
|
|
15
|
+
parent_rss_kb: nil, error_message: nil, error_class: nil,
|
|
16
|
+
error_backtrace: nil)
|
|
17
|
+
# rubocop:enable Metrics/ParameterLists
|
|
14
18
|
raise ArgumentError, "invalid status: #{status}" unless STATUSES.include?(status)
|
|
15
19
|
|
|
16
20
|
@mutation = mutation
|
|
@@ -21,6 +25,9 @@ class Evilution::Result::MutationResult
|
|
|
21
25
|
@child_rss_kb = child_rss_kb
|
|
22
26
|
@memory_delta_kb = memory_delta_kb
|
|
23
27
|
@parent_rss_kb = parent_rss_kb
|
|
28
|
+
@error_message = error_message
|
|
29
|
+
@error_class = error_class
|
|
30
|
+
@error_backtrace = error_backtrace.nil? ? nil : error_backtrace.dup.freeze
|
|
24
31
|
freeze
|
|
25
32
|
end
|
|
26
33
|
|
data/lib/evilution/runner.rb
CHANGED
|
@@ -25,6 +25,7 @@ require_relative "ast/pattern/filter"
|
|
|
25
25
|
require_relative "temp_dir_tracker"
|
|
26
26
|
require_relative "disable_comment"
|
|
27
27
|
require_relative "ast/sorbet_sig_detector"
|
|
28
|
+
require_relative "rails_detector"
|
|
28
29
|
|
|
29
30
|
class Evilution::Runner
|
|
30
31
|
INTEGRATIONS = {
|
|
@@ -32,6 +33,11 @@ class Evilution::Runner
|
|
|
32
33
|
minitest: Evilution::Integration::Minitest
|
|
33
34
|
}.freeze
|
|
34
35
|
|
|
36
|
+
PRELOAD_CANDIDATES = [
|
|
37
|
+
File.join("spec", "rails_helper.rb"),
|
|
38
|
+
File.join("test", "test_helper.rb")
|
|
39
|
+
].freeze
|
|
40
|
+
|
|
35
41
|
attr_reader :config
|
|
36
42
|
|
|
37
43
|
def initialize(config: Evilution::Config.new, on_result: nil, hooks: nil)
|
|
@@ -40,7 +46,6 @@ class Evilution::Runner
|
|
|
40
46
|
@hooks = hooks
|
|
41
47
|
@parser = Evilution::AST::Parser.new
|
|
42
48
|
@registry = Evilution::Mutator::Registry.default
|
|
43
|
-
@isolator = build_isolator
|
|
44
49
|
@cache = config.incremental? ? Evilution::Cache.new : nil
|
|
45
50
|
@disable_detector = Evilution::DisableComment.new
|
|
46
51
|
@disabled_ranges_cache = {}
|
|
@@ -55,6 +60,9 @@ class Evilution::Runner
|
|
|
55
60
|
subjects = parse_and_filter_subjects
|
|
56
61
|
log_memory("after parse_subjects", "#{subjects.length} subjects")
|
|
57
62
|
|
|
63
|
+
perform_preload
|
|
64
|
+
log_memory("after preload") if rails_root_detected?
|
|
65
|
+
|
|
58
66
|
baseline_result = run_baseline(subjects)
|
|
59
67
|
|
|
60
68
|
mutations, skipped_count, disabled_mutations = generate_mutations(subjects)
|
|
@@ -89,7 +97,11 @@ class Evilution::Runner
|
|
|
89
97
|
|
|
90
98
|
private
|
|
91
99
|
|
|
92
|
-
attr_reader :parser, :registry, :
|
|
100
|
+
attr_reader :parser, :registry, :cache, :on_result, :hooks, :disable_detector, :sig_detector
|
|
101
|
+
|
|
102
|
+
def isolator
|
|
103
|
+
@isolator ||= build_isolator
|
|
104
|
+
end
|
|
93
105
|
|
|
94
106
|
def parse_subjects
|
|
95
107
|
files = resolve_target_files
|
|
@@ -97,10 +109,13 @@ class Evilution::Runner
|
|
|
97
109
|
end
|
|
98
110
|
|
|
99
111
|
def resolve_target_files
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
112
|
+
@resolve_target_files ||= if source_glob_target?
|
|
113
|
+
resolve_source_glob
|
|
114
|
+
elsif !config.target_files.empty?
|
|
115
|
+
config.target_files
|
|
116
|
+
else
|
|
117
|
+
Evilution::Git::ChangedFiles.new.call
|
|
118
|
+
end
|
|
104
119
|
end
|
|
105
120
|
|
|
106
121
|
def source_glob_target?
|
|
@@ -340,7 +355,7 @@ class Evilution::Runner
|
|
|
340
355
|
def run_mutations_parallel(mutations, baseline_result = nil)
|
|
341
356
|
integration = build_integration
|
|
342
357
|
pool = Evilution::Parallel::Pool.new(size: config.jobs, hooks: @hooks, item_timeout: config.timeout ? config.timeout * 2 : nil)
|
|
343
|
-
worker_isolator =
|
|
358
|
+
worker_isolator = build_isolator
|
|
344
359
|
spec_resolver = baseline_result&.failed? ? build_neutralization_resolver : nil
|
|
345
360
|
state = { results: [], survived_count: 0, truncated: false, completed: 0 }
|
|
346
361
|
|
|
@@ -411,7 +426,10 @@ class Evilution::Runner
|
|
|
411
426
|
test_command: result.test_command,
|
|
412
427
|
child_rss_kb: result.child_rss_kb,
|
|
413
428
|
memory_delta_kb: result.memory_delta_kb,
|
|
414
|
-
parent_rss_kb: result.parent_rss_kb
|
|
429
|
+
parent_rss_kb: result.parent_rss_kb,
|
|
430
|
+
error_message: result.error_message,
|
|
431
|
+
error_class: result.error_class,
|
|
432
|
+
error_backtrace: result.error_backtrace
|
|
415
433
|
)
|
|
416
434
|
end
|
|
417
435
|
|
|
@@ -423,7 +441,10 @@ class Evilution::Runner
|
|
|
423
441
|
test_command: result.test_command,
|
|
424
442
|
child_rss_kb: result.child_rss_kb,
|
|
425
443
|
memory_delta_kb: result.memory_delta_kb,
|
|
426
|
-
parent_rss_kb: result.parent_rss_kb
|
|
444
|
+
parent_rss_kb: result.parent_rss_kb,
|
|
445
|
+
error_message: result.error_message,
|
|
446
|
+
error_class: result.error_class,
|
|
447
|
+
error_backtrace: result.error_backtrace
|
|
427
448
|
}
|
|
428
449
|
end
|
|
429
450
|
|
|
@@ -437,7 +458,10 @@ class Evilution::Runner
|
|
|
437
458
|
test_command: data[:test_command],
|
|
438
459
|
child_rss_kb: data[:child_rss_kb],
|
|
439
460
|
memory_delta_kb: data[:memory_delta_kb],
|
|
440
|
-
parent_rss_kb: data[:parent_rss_kb]
|
|
461
|
+
parent_rss_kb: data[:parent_rss_kb],
|
|
462
|
+
error_message: data[:error_message],
|
|
463
|
+
error_class: data[:error_class],
|
|
464
|
+
error_backtrace: data[:error_backtrace]
|
|
441
465
|
)
|
|
442
466
|
end
|
|
443
467
|
end
|
|
@@ -474,9 +498,80 @@ class Evilution::Runner
|
|
|
474
498
|
end
|
|
475
499
|
|
|
476
500
|
def resolve_isolation
|
|
477
|
-
|
|
501
|
+
case config.isolation
|
|
502
|
+
when :fork
|
|
503
|
+
:fork
|
|
504
|
+
when :in_process
|
|
505
|
+
warn_in_process_under_rails if rails_root_detected?
|
|
506
|
+
:in_process
|
|
507
|
+
else # :auto
|
|
508
|
+
rails_root_detected? ? :fork : :in_process
|
|
509
|
+
end
|
|
510
|
+
end
|
|
511
|
+
|
|
512
|
+
def rails_root_detected?
|
|
513
|
+
return @rails_root_detected if defined?(@rails_root_detected)
|
|
478
514
|
|
|
479
|
-
|
|
515
|
+
@rails_root_detected = !detected_rails_root.nil?
|
|
516
|
+
end
|
|
517
|
+
|
|
518
|
+
def detected_rails_root
|
|
519
|
+
return @detected_rails_root if defined?(@detected_rails_root)
|
|
520
|
+
|
|
521
|
+
@detected_rails_root = Evilution::RailsDetector.rails_root_for_any(resolve_target_files)
|
|
522
|
+
end
|
|
523
|
+
|
|
524
|
+
def perform_preload
|
|
525
|
+
return if config.preload == false
|
|
526
|
+
return unless resolve_isolation == :fork
|
|
527
|
+
|
|
528
|
+
path = resolve_preload_path
|
|
529
|
+
return unless path
|
|
530
|
+
|
|
531
|
+
require File.expand_path(path)
|
|
532
|
+
rescue ScriptError, StandardError => e
|
|
533
|
+
raise Evilution::ConfigError.new(
|
|
534
|
+
"failed to preload #{path.inspect}: #{e.class}: #{e.message}",
|
|
535
|
+
file: path
|
|
536
|
+
)
|
|
537
|
+
end
|
|
538
|
+
|
|
539
|
+
def resolve_preload_path
|
|
540
|
+
if config.preload.is_a?(String)
|
|
541
|
+
unless File.file?(config.preload)
|
|
542
|
+
raise Evilution::ConfigError.new(
|
|
543
|
+
"preload file not found: #{config.preload.inspect}",
|
|
544
|
+
file: config.preload
|
|
545
|
+
)
|
|
546
|
+
end
|
|
547
|
+
return config.preload
|
|
548
|
+
end
|
|
549
|
+
|
|
550
|
+
root = detected_rails_root
|
|
551
|
+
return nil unless root
|
|
552
|
+
|
|
553
|
+
PRELOAD_CANDIDATES.each do |rel|
|
|
554
|
+
abs = File.join(root, rel)
|
|
555
|
+
return abs if File.file?(abs)
|
|
556
|
+
end
|
|
557
|
+
nil
|
|
558
|
+
end
|
|
559
|
+
|
|
560
|
+
# When the user explicitly requests InProcess on a Rails project, warn once
|
|
561
|
+
# per run. Rails wraps ActiveRecord transactions in
|
|
562
|
+
# Thread.handle_interrupt(Exception => :never), which defers Timeout's
|
|
563
|
+
# Thread#raise indefinitely — making InProcess unable to kill runaway mutants.
|
|
564
|
+
def warn_in_process_under_rails
|
|
565
|
+
return if config.quiet
|
|
566
|
+
return if @warned_in_process_under_rails
|
|
567
|
+
|
|
568
|
+
@warned_in_process_under_rails = true
|
|
569
|
+
$stderr.write(
|
|
570
|
+
"[evilution] warning: --isolation in_process is unsafe on Rails projects. " \
|
|
571
|
+
"ActiveRecord wraps transactions in Thread.handle_interrupt(Exception => :never), " \
|
|
572
|
+
"which swallows Timeout.timeout and can cause evilution to hang indefinitely on " \
|
|
573
|
+
"mutants that introduce infinite loops. Use --isolation fork for reliable interruption.\n"
|
|
574
|
+
)
|
|
480
575
|
end
|
|
481
576
|
|
|
482
577
|
def resolve_integration_class
|
|
@@ -562,9 +657,20 @@ class Evilution::Runner
|
|
|
562
657
|
|
|
563
658
|
parts << gc_stats_string
|
|
564
659
|
|
|
565
|
-
|
|
660
|
+
$stderr.write("[verbose] #{result.mutation}: #{parts.join(", ")}\n") unless parts.empty?
|
|
566
661
|
|
|
567
|
-
|
|
662
|
+
log_mutation_error(result) if result.error?
|
|
663
|
+
end
|
|
664
|
+
|
|
665
|
+
def log_mutation_error(result)
|
|
666
|
+
header = "[verbose] #{result.mutation}: error"
|
|
667
|
+
header += " #{result.error_class}" if result.error_class
|
|
668
|
+
header += ": #{result.error_message}" if result.error_message
|
|
669
|
+
$stderr.write("#{header}\n")
|
|
670
|
+
|
|
671
|
+
Array(result.error_backtrace).first(5).each do |line|
|
|
672
|
+
$stderr.write("[verbose] #{line}\n")
|
|
673
|
+
end
|
|
568
674
|
end
|
|
569
675
|
|
|
570
676
|
def gc_stats_string
|
data/lib/evilution/version.rb
CHANGED
data/script/memory_check
CHANGED
|
@@ -104,12 +104,12 @@ complex_mutations = complex_subjects.flat_map { |s| complex_registry.mutations_f
|
|
|
104
104
|
|
|
105
105
|
integration = Evilution::Integration::RSpec.new(test_files: [COMPLEX_FIXTURE_SPEC])
|
|
106
106
|
|
|
107
|
-
|
|
107
|
+
# Budget is generous: per-mutation require() adds the file to $LOADED_FEATURES and
|
|
108
|
+
# accumulates constant-redefinition warnings. Local runs land around 20-25 MB; CI
|
|
109
|
+
# varies up to ~30 MB depending on Ruby build and GC pressure, so we leave headroom.
|
|
110
|
+
all_passed &= run_check("RSpec integration per-mutation (Config)", iterations: 20, max_growth_kb: 40_960) do
|
|
108
111
|
mutation = complex_mutations.sample
|
|
109
|
-
|
|
110
|
-
raise "RSpec integration memory check failed: #{result[:error]}" if result[:error]
|
|
111
|
-
|
|
112
|
-
result
|
|
112
|
+
integration.call(mutation)
|
|
113
113
|
end
|
|
114
114
|
|
|
115
115
|
puts all_passed ? "All memory checks passed." : "Some memory checks failed!"
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: evilution
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.22.
|
|
4
|
+
version: 0.22.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Denis Kiselev
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-04-
|
|
11
|
+
date: 2026-04-12 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: diff-lcs
|
|
@@ -70,6 +70,7 @@ files:
|
|
|
70
70
|
- ".claude/prompts/architect.md"
|
|
71
71
|
- ".claude/prompts/devops.md"
|
|
72
72
|
- ".claude/prompts/tests.md"
|
|
73
|
+
- ".claude/settings.json"
|
|
73
74
|
- CHANGELOG.md
|
|
74
75
|
- CODE_OF_CONDUCT.md
|
|
75
76
|
- LICENSE.txt
|
|
@@ -80,6 +81,7 @@ files:
|
|
|
80
81
|
- comparison_results/operator_classification.md
|
|
81
82
|
- comparison_results/operator_prioritization.md
|
|
82
83
|
- docs/ast_pattern_syntax.md
|
|
84
|
+
- docs/isolation.md
|
|
83
85
|
- docs/mutation_density_benchmark.md
|
|
84
86
|
- exe/evilution
|
|
85
87
|
- lib/evilution.rb
|
|
@@ -209,6 +211,7 @@ files:
|
|
|
209
211
|
- lib/evilution/parallel.rb
|
|
210
212
|
- lib/evilution/parallel/pool.rb
|
|
211
213
|
- lib/evilution/parallel/work_queue.rb
|
|
214
|
+
- lib/evilution/rails_detector.rb
|
|
212
215
|
- lib/evilution/related_spec_heuristic.rb
|
|
213
216
|
- lib/evilution/reporter.rb
|
|
214
217
|
- lib/evilution/reporter/cli.rb
|