evilution 0.33.0 → 0.35.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.beads/interactions.jsonl +16 -0
- data/.rubocop_todo.yml +2 -1
- data/CHANGELOG.md +26 -0
- data/README.md +39 -10
- data/assets/logo/evilution-mark-1024.png +0 -0
- data/assets/logo/evilution-mark-64.png +0 -0
- data/assets/logo/evilution-mark-transparent-1024.png +0 -0
- data/assets/logo/evilution-mark-transparent.svg +53 -0
- data/assets/logo/evilution-mark.svg +70 -0
- data/docs/isolation.md +32 -2
- data/docs/migration-from-mutant.md +226 -0
- data/lib/evilution/cli/parser/options_builder.rb +17 -0
- data/lib/evilution/config/builders/spec_resolver.rb +3 -1
- data/lib/evilution/config/validators/example_targeting_strategy.rb +22 -0
- data/lib/evilution/config.rb +16 -2
- data/lib/evilution/coverage/digest.rb +16 -0
- data/lib/evilution/coverage/map.rb +64 -0
- data/lib/evilution/coverage/map_builder.rb +82 -0
- data/lib/evilution/coverage/map_store.rb +87 -0
- data/lib/evilution/coverage/recorder.rb +85 -0
- data/lib/evilution/coverage.rb +8 -0
- data/lib/evilution/coverage_example_filter.rb +41 -0
- data/lib/evilution/gem_detector.rb +7 -0
- data/lib/evilution/integration/minitest.rb +14 -3
- data/lib/evilution/integration/rspec/unresolved_spec_warner.rb +17 -3
- data/lib/evilution/integration/test_unit/test_file_resolver.rb +14 -3
- data/lib/evilution/isolation/fork.rb +38 -76
- data/lib/evilution/parallel/work_queue/dispatcher/deadline_tracker.rb +63 -0
- data/lib/evilution/parallel/work_queue/dispatcher.rb +7 -34
- data/lib/evilution/parallel/work_queue/worker.rb +41 -51
- data/lib/evilution/process_supervisor.rb +259 -0
- data/lib/evilution/runner/baseline_runner.rb +52 -0
- data/lib/evilution/runner/canary.rb +31 -4
- data/lib/evilution/runner/isolation_resolver.rb +120 -12
- data/lib/evilution/runner.rb +3 -2
- data/lib/evilution/spec_resolver.rb +93 -1
- data/lib/evilution/spec_selector.rb +14 -4
- data/lib/evilution/version.rb +1 -1
- data/lib/evilution.rb +1 -0
- data/scripts/canary_manifest.yml +47 -0
- data/scripts/compare_targeting +277 -0
- data/scripts/compare_targeting.example.yml +24 -0
- metadata +21 -3
- data/lib/evilution/parallel/work_queue/worker_registry.rb +0 -47
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 492fc3dd9f7676eaf9c263b1342602250561a382afe2aa9f5871daa04ce842a7
|
|
4
|
+
data.tar.gz: 8982ff01289997d8abaa3d5cb122de065ef0ebfcc1c043767e423d97a7d714f6
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 6653347d505820a813fa673d60013806738b1e37e9f268c4c9c429c531644998a6e9d27bac2dfe93613eaa4aca6b8a257ed44867d9a1d92d4f4c5a27a16f93fa
|
|
7
|
+
data.tar.gz: d6da79725f113a9e11ac33da8e835ecf58db554db34efdd5b74a015b190bc4717864718f04462993801a9b44d606ff53de303a32dbcb7bf7fe15d5b4d71cb6b8
|
data/.beads/interactions.jsonl
CHANGED
|
@@ -421,3 +421,19 @@
|
|
|
421
421
|
{"id":"int-cdc02b29","kind":"field_change","created_at":"2026-06-06T15:19:26.153490037Z","actor":"Denis Kiselev","issue_id":"EV-dlnn","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged via PR #1344. Flaky /tmp glob replaced with parent-side sandbox capture (spy Dir.mktmpdir, assert the evilution-run dir gone after timeout). Deterministic, fork.rb untouched."}}
|
|
422
422
|
{"id":"int-af4ca762","kind":"field_change","created_at":"2026-06-06T17:23:17.168065946Z","actor":"Denis Kiselev","issue_id":"EV-dwqw","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged via PR #1345. Real carrier was RSpec.configuration @preferred_options (color_mode getter reads it first); inner Runner.run --no-color force-merges color_mode=:off in place. StateGuard::ConfigurationState snapshots @preferred_options by dup + stream ivars and restores them; in-process example re-enabled. Visual dots green, suite 4881 green."}}
|
|
423
423
|
{"id":"int-f7253919","kind":"field_change","created_at":"2026-06-07T04:14:56.864785049Z","actor":"Denis Kiselev","issue_id":"EV-z7f5","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged PR #1347 (master). opt1+2+3 + Copilot review fixes. Empirical 4-repro re-run remains under EV-rxob canary harness (the dep)."}}
|
|
424
|
+
{"id":"int-41c70ee3","kind":"field_change","created_at":"2026-06-15T07:26:55.682158325Z","actor":"Denis Kiselev","issue_id":"EV-9f3b","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged PR #1358: ProcessSupervisor unit + specs"}}
|
|
425
|
+
{"id":"int-9f3581f4","kind":"field_change","created_at":"2026-06-15T08:27:50.649632334Z","actor":"Denis Kiselev","issue_id":"EV-3aw3","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged PR #1359: Isolation::Fork routed through ProcessSupervisor"}}
|
|
426
|
+
{"id":"int-9abff5aa","kind":"field_change","created_at":"2026-06-15T09:18:55.659697892Z","actor":"Denis Kiselev","issue_id":"EV-dg69","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged PR #1360: outer path (Worker/Runner trap) routed through ProcessSupervisor; WorkerRegistry removed"}}
|
|
427
|
+
{"id":"int-42f490de","kind":"field_change","created_at":"2026-06-15T11:28:14.219616586Z","actor":"Denis Kiselev","issue_id":"EV-7a91","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged PR #1361: worker INT/TERM handler reaps inner children, kill_and_reap_all + reset_for_child!, cross-path lifecycle stress spec"}}
|
|
428
|
+
{"id":"int-68e95598","kind":"field_change","created_at":"2026-06-15T11:28:31.205819027Z","actor":"Denis Kiselev","issue_id":"EV-5rrh","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"All 4 Track A steps merged: ProcessSupervisor owns spawn/register/signal-safe registry/group-kill/reap. Inner (Fork) + outer (Worker) paths routed through it; WorkerRegistry removed; Runner trap + worker trap reap on interrupt; cross-path stress spec proves no orphan/zombie."}}
|
|
429
|
+
{"id":"int-0bc92dbf","kind":"field_change","created_at":"2026-06-15T11:49:47.527861306Z","actor":"Denis Kiselev","issue_id":"EV-1q84","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged: Coverage::Map + Recorder + MapBuilder (forked full-suite per-example coverage map)"}}
|
|
430
|
+
{"id":"int-207eba53","kind":"field_change","created_at":"2026-06-15T12:01:00.790784019Z","actor":"Denis Kiselev","issue_id":"EV-k1xr","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged: Coverage::MapStore + Digest (per-file digest cache, prune-to-fresh partial invalidation, corrupt rebuild)"}}
|
|
431
|
+
{"id":"int-1e19ca1d","kind":"field_change","created_at":"2026-06-15T13:01:49.951898163Z","actor":"Denis Kiselev","issue_id":"EV-lqqk","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged PR #1364: CoverageExampleFilter + example_targeting_strategy config/CLI + build_example_filter wiring + executed-lines accuracy fix"}}
|
|
432
|
+
{"id":"int-1e5b072e","kind":"field_change","created_at":"2026-06-15T13:37:22.697876987Z","actor":"Denis Kiselev","issue_id":"EV-51d4","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged PR #1365: scripts/compare_targeting 3-mode comparison harness (lost_kills gate + wall-time ratios) + example manifest"}}
|
|
433
|
+
{"id":"int-c50dc4d6","kind":"field_change","created_at":"2026-06-15T17:19:15.781802557Z","actor":"Denis Kiselev","issue_id":"EV-7uui","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged PR #1368: direction C — per-resolved-file coverage (absolute locations, resolved-spec capture, accuracy-first empty->lexical). factory_bot 16 lost kills -> 0"}}
|
|
434
|
+
{"id":"int-2e23cd44","kind":"field_change","created_at":"2026-06-16T07:17:22.764949718Z","actor":"Denis Kiselev","issue_id":"EV-5hk5","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged via PR #1373 (master)"}}
|
|
435
|
+
{"id":"int-8e319322","kind":"field_change","created_at":"2026-06-16T07:17:22.312793419Z","actor":"Denis Kiselev","issue_id":"EV-bi41","extra":{"field":"status","new_value":"in_progress","old_value":"open"}}
|
|
436
|
+
{"id":"int-93c675ea","kind":"field_change","created_at":"2026-06-16T07:54:16.348995872Z","actor":"Denis Kiselev","issue_id":"EV-bi41","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged via PR #1374 (master)"}}
|
|
437
|
+
{"id":"int-555de29c","kind":"field_change","created_at":"2026-06-16T07:54:17.438977358Z","actor":"Denis Kiselev","issue_id":"EV-z03y","extra":{"field":"status","new_value":"in_progress","old_value":"open"}}
|
|
438
|
+
{"id":"int-616ccd27","kind":"field_change","created_at":"2026-06-16T09:31:30.217250542Z","actor":"Denis Kiselev","issue_id":"EV-z03y","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Merged via PR #1375 (master). Dep on EV-rxob is provenance only; code fix shipped."}}
|
|
439
|
+
{"id":"int-6d2f9b35","kind":"field_change","created_at":"2026-06-16T10:00:18.493013642Z","actor":"Denis Kiselev","issue_id":"EV-rxob","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"1.0 real-world validation complete. Rounds 2-4 ran 30+ gems across rspec/minitest/test-unit; no crash/memory blowup. All findings filed+fixed+merged: EV-z7f5/#1347, EV-gl1e/#1329, EV-52hf/#1341, EV-5hk5/#1373, EV-bi41/#1374, EV-z03y/#1375. Round-4 OOB re-verify confirmed fixes compound (state_machines 0->0.922, factory_bot/webmock/http real OOB scores, errors=0). Residual resolution-coverage gaps (behavior-named/flat-prefixed/non-std helper) tracked low-pri: EV-ajby/#1376, EV-6rbd/#1377, EV-no5u/#1378."}}
|
data/.rubocop_todo.yml
CHANGED
|
@@ -12,9 +12,10 @@ Metrics/ClassLength:
|
|
|
12
12
|
- "lib/evilution/isolation/fork.rb"
|
|
13
13
|
- "lib/evilution/integration/minitest.rb"
|
|
14
14
|
- "lib/evilution/mcp/mutate_tool.rb"
|
|
15
|
-
- "lib/evilution/parallel/work_queue/dispatcher.rb"
|
|
16
15
|
- "lib/evilution/runner.rb"
|
|
17
16
|
- "lib/evilution/runner/isolation_resolver.rb"
|
|
17
|
+
- "lib/evilution/cli/parser/options_builder.rb"
|
|
18
|
+
- "lib/evilution/spec_resolver.rb"
|
|
18
19
|
|
|
19
20
|
Metrics/MethodLength:
|
|
20
21
|
Exclude:
|
data/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,32 @@
|
|
|
2
2
|
|
|
3
3
|
Versioning policy: see [docs/versioning.md](docs/versioning.md).
|
|
4
4
|
|
|
5
|
+
## [0.35.0] - 2026-06-24
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- **Test::Unit now scores out-of-the-box: a dedicated canary spec plus a Test::Unit spec resolver** — `--integration test-unit` shared minitest's runtime but not its file conventions, so two things silently broke. (1) The proof-of-life canary always wrote either an RSpec `_spec.rb` or a minitest `_test.rb` and only ever generated RSpec/minitest source; under `test-unit` the synthetic mutation loaded nothing, scored `:error`, and aborted the run before any real mutation ran. The canary now emits a `Test::Unit::TestCase` subclass into a `_test.rb` file for `test-unit` (and keeps the `_test.rb`/`_spec.rb` split keyed on the integration). (2) The config spec-resolver builder fell through to the default `spec/`/`_spec.rb` resolver for `test-unit`, which resolved nothing for a `test/`-rooted suite; it now reuses minitest's `test/` + `_test.rb` resolver since test-unit gems mirror that layout (PR #1386)
|
|
10
|
+
- **Auto-preload finds namespaced gem test helpers (`test/<gem>/helper.rb`)** — gems that namespace their suite keep the bootstrap at `test/<gem>/helper.rb` (e.g. ruby/csv: `test/csv/helper.rb`) rather than a flat `test/test_helper.rb`. The conventional preload chain missed it, so autodetect fell back to the bare gem entry point (`lib/<gem>.rb`) — which never sets up the test framework — and the canary aborted. The gem-helper search now also looks for the namespaced form, deriving the directory from the gem name (dotted form for dash-named gems plus the flat form): `test/<gem>/helper.rb`. `Evilution::GemDetector.gem_name` is exposed publicly so callers can build these conventional paths (PR #1386)
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **`SpecResolver` resolves flat `test_`-prefixed Test::Unit/minitest files (`test/test_connection_pool_timed_stack.rb`)** — some minitest/Test::Unit gems flatten a nested source's path into a single `test_`-prefixed file at the test root (`lib/connection_pool/timed_stack.rb` → `test/test_connection_pool_timed_stack.rb`, with no `test/connection_pool/` subdir), so the resolver found nothing and scored `:unresolved`. The resolver now derives a flat candidate from the **full** lib-relative path (every segment joined with `_`) — deliberately not from namespace-dropped variants, which would yield bare-basename forms (`test_timed_stack.rb`) that collide across namespaces and are already covered by the existing `test_<name>.rb` convention. Top-level sources produce nothing here, and the flat candidate ranks below the mirrored layouts so a 1:1 file always wins. Minitest suffix only (PR #1385)
|
|
15
|
+
- **Unresolved-mutation warnings now name concrete recovery paths for behaviour-named spec layouts** — `SpecResolver` intentionally does not guess for **behaviour-named** layouts (specs named by behaviour, not by lib path, so nothing ties a source file to a spec — e.g. aasm's `lib/aasm/base.rb`, exercised through the public API by `spec/unit/{event,callbacks,guard,api}_spec.rb`). A fuzzy match there would report a misleading score, so every mutation stays `:unresolved` (a coverage gap, `errors=0`). The minitest, RSpec, and Test::Unit unresolved warnings now spell out the explicit opt-ins — name the covering specs via `--spec`/`--spec-dir`, or run the whole suite per mutation with `--fallback-full-suite` — so a behaviour-named layout reads as a fixable resolution choice rather than a silent 0%. Documented in the README "Unresolved mutations — behaviour-named spec layouts" section (PR #1384)
|
|
16
|
+
|
|
17
|
+
## [0.34.0] - 2026-06-16
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
- **Coverage-based example targeting (`--example-targeting coverage` / `example_targeting_strategy: coverage`)** — the default `lexical` strategy picks per-mutation examples by name-grepping example bodies for the mutated method/class token, which over-selects (any example that merely *mentions* the name) and under-selects (examples that exercise the code without naming it). The new `coverage` strategy instead runs exactly the examples that **execute the mutated line**, derived from a cached full-suite line-coverage map (`Coverage` stdlib), and falls back to `lexical` for any file the map has not built yet. Enable it with `--example-targeting coverage` (CLI) or `example_targeting_strategy: coverage` (`.evilution.yml`); the YAML key accepts `lexical` or `coverage`. The CLI flag additionally accepts `full_file` to run every resolved example (equivalent to `example_targeting: false` in `.evilution.yml`). See [docs/superpowers/specs/2026-06-07-coverage-based-targeting-design.md](docs/superpowers/specs/2026-06-07-coverage-based-targeting-design.md) (PRs #1364/#1365)
|
|
22
|
+
- **Auto spec-resolution now expands dir-grouped test layouts (`test/unit/<class>/*_test.rb`)** — `Evilution::SpecResolver` resolved a source file only to a single mirror test file, so projects that group a class's tests in a *directory* named after the source (e.g. `state_machines`: `lib/state_machines/branch.rb` → `test/unit/branch/*_test.rb`, 51 files) resolved nothing and scored 0.0 out of the box. The resolver now also matches a grouped directory and expands it into its test files (`*_test.rb` plus the `test_*.rb` prefix convention), ranked below the deterministic file mirror so a 1:1 spec still wins. `SpecSelector` consumes the resulting list and keeps working with custom resolvers that implement only the older `#call` contract. Repro that went 0.0 → 0.92 out of the box: state_machines (EV-bi41, PR #1374, GH #1371)
|
|
23
|
+
|
|
24
|
+
### Fixed
|
|
25
|
+
|
|
26
|
+
- **Non-Rails gems now preload and score out-of-the-box instead of erroring on every mutation** — auto-preload fired only for Rails projects, so a plain rspec/minitest **gem** run with the default `auto` isolation got no parent-process preload (its library and test framework were never loaded), and every mutation errored with `0 examples loaded` / a `NameError` (e.g. `undefined method 'delegate'`) — a 0.0 score with a cryptic cause. `auto` isolation now resolves to `fork` when a `*.gemspec` is detected (not just a Rails root), and the non-Rails autodetect chain now prefers the conventional test helper (`spec/spec_helper.rb` → `test/test_helper.rb` → `test/helper.rb`) — which loads the library *and* the suite's framework/support setup so example groups register — falling back to the gem's `lib/<gem>.rb` entry. When a gem is detected but no conventional helper exists, evilution prints an actionable warning naming the locations it searched and the `--preload` escape hatch, instead of a silent 0%. Out-of-box repros: factory_bot, webmock, http (442 errors → real scores) (EV-z03y, PR #1375, GH #1372)
|
|
27
|
+
- **Parent-process preload puts the test root on `$LOAD_PATH` for minitest/Test::Unit projects** — `IsolationResolver` only ever added `spec/` to `$LOAD_PATH` before requiring the preload file, even for `--integration minitest`. A `test/test_helper.rb` that does a non-relative `require "support/..."` (relying on the runner's `-Itest`) then failed preload with a `LoadError`. Preload is now integration-aware: rspec stays `spec/`-only (so a bare `require "support/foo"` still resolves from `spec/` in projects that have both dirs), while minitest/Test::Unit routes through `Integration::Loading::TestLoadPath` — the same policy the per-mutation test load uses — putting the test root on `$LOAD_PATH`. Repro: httprb/http (`require "support/capture_warning"`) now resolves without `RUBYOPT=-Itest` (EV-5hk5, PR #1373, GH #1370)
|
|
28
|
+
- **Worker and per-mutation child process lifecycle consolidated into `ProcessSupervisor`** — process-group creation (`setpgid`), signal forwarding (SIGINT/SIGTERM → worker groups), and child reaping were spread across `Isolation::Fork`, the parallel worker, and a separate `WorkerRegistry`. They are now centralized in a single `Evilution::ProcessSupervisor`, which removes `WorkerRegistry`, tolerates benign `setpgid` races (`ESRCH`) silently, warns rather than raises on unexpected `EPERM`, and ensures sandbox cleanup on failure — closing process/FD-leak windows on interrupted or worker-recycling runs (PRs #1358 fixes GH #1337, #1360 fixes GH #1339)
|
|
29
|
+
- **`WorkQueue::Dispatcher` per-worker timeout tracking extracted into `DeadlineTracker`** — the per-worker deadline bookkeeping that kills and recycles only a stuck worker (rather than aborting the whole run) is now an isolated, separately-tested `DeadlineTracker` collaborator, simplifying the dispatcher and hardening timeout management under load (PR #1369)
|
|
30
|
+
|
|
5
31
|
## [0.33.0] - 2026-06-07
|
|
6
32
|
|
|
7
33
|
### Added
|
data/README.md
CHANGED
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="https://raw.githubusercontent.com/marinazzio/evilution/refs/heads/master/assets/logo/evilution-mark-transparent.svg" width="160" height="160" alt="Evilution mascot">
|
|
3
|
+
</p>
|
|
4
|
+
|
|
1
5
|
[](https://badge.fury.io/rb/evilution)
|
|
2
6
|
|
|
3
7
|
# Evilution — Mutation Testing for Ruby
|
|
@@ -59,6 +63,13 @@ The first command writes a sibling `Gemfile.local.lock`. Decide whether to commi
|
|
|
59
63
|
|
|
60
64
|
The evilution gemspec already declares `prism >= 1.5, < 2`, so adding the `gem "prism"` line above is only necessary on stacks that also pin prism in `Gemfile.lock`.
|
|
61
65
|
|
|
66
|
+
## Migrating from `mutant`
|
|
67
|
+
|
|
68
|
+
Coming from the commercially-licensed [`mutant`](https://github.com/mbj/mutant) gem? See
|
|
69
|
+
[docs/migration-from-mutant.md](docs/migration-from-mutant.md) for a command/config
|
|
70
|
+
mapping, output-terminology differences, and how to diff the two tools' results with
|
|
71
|
+
`evilution compare` (it ingests mutant session JSON directly).
|
|
72
|
+
|
|
62
73
|
## Command Reference
|
|
63
74
|
|
|
64
75
|
```
|
|
@@ -94,10 +105,11 @@ Every command, subcommand, and flag listed in this section is part of evilution'
|
|
|
94
105
|
| `-f`, `--format FORMAT` | String | `text` | Output format: `text`, `json`, or `html`. |
|
|
95
106
|
| `--target EXPR` | String | _(none)_ | Only mutate matching methods. Supports method name (`Foo::Bar#calculate`), class (`Foo`), namespace wildcards (`Foo::Bar*`), method-type selectors (`Foo#`, `Foo.`), descendants (`descendants:Foo`), and source globs (`source:lib/**/*.rb`). |
|
|
96
107
|
| `--min-score FLOAT` | Float | 0.0 | Minimum mutation score (0.0–1.0) to pass. |
|
|
97
|
-
| `--spec FILES` | Array | _(none)_ | Spec files to run (comma-separated). Defaults to auto-detection via `SpecResolver
|
|
108
|
+
| `--spec FILES` | Array | _(none)_ | Spec files to run (comma-separated). Defaults to auto-detection via `SpecResolver`, which also resolves non-mirrored (`spec/unit`, `test/unit`), dir-grouped (`test/unit/<class>/*_test.rb`), and flat `test_`-prefixed (`test/test_connection_pool_timed_stack.rb`) layouts. |
|
|
98
109
|
| `--spec-dir DIR` | String | _(none)_ | Include all `*_spec.rb` files in DIR recursively. Composable with `--spec`. |
|
|
99
110
|
| `--spec-pattern GLOB` | String | _(none)_ | Restrict resolved spec candidates to files matching GLOB (e.g. `spec/models/**/*_spec.rb`). |
|
|
100
111
|
| `--no-example-targeting` | Boolean | _(enabled)_ | Disable per-mutation example targeting (always run every example in the resolved spec file). Example targeting scans each example body for symbols from the mutated method and runs only the matching subset. |
|
|
112
|
+
| `--example-targeting MODE` | String | `lexical` | How targeting picks examples: `lexical` (name-grep example bodies for the mutated method/class), `coverage` (run only the examples that actually execute the mutated line, from a cached line-coverage map), or `full_file` (run all resolved examples). |
|
|
101
113
|
| `--example-targeting-fallback MODE` | String | `full_file` | Behavior when no example matches: `full_file` (run the whole spec file) or `unresolved` (skip the mutation as `:unresolved`). |
|
|
102
114
|
| `-j`, `--jobs N` | Integer | 1 | Number of parallel workers. Uses demand-driven work distribution with pipe-based IPC. |
|
|
103
115
|
| `--no-baseline` | Boolean | _(enabled)_ | Skip baseline test suite check. By default, a baseline run detects pre-existing failures and marks those mutations as `neutral`. |
|
|
@@ -113,8 +125,8 @@ Every command, subcommand, and flag listed in this section is part of evilution'
|
|
|
113
125
|
| `--no-progress` | Boolean | _(enabled)_ | Disable the TTY progress bar. |
|
|
114
126
|
| `--quiet-children` | Boolean | false | Redirect each forked worker's stdout/stderr to per-pid files under `tmp/evilution_children/<pid>.{out,err}` so noisy app initializers (Datadog, Bullet, etc.) don't merge with parent output. Trade-off: live worker errors only appear in the side files, not the terminal — `tail -f tmp/evilution_children/*.err` to watch them. |
|
|
115
127
|
| `--quiet-children-dir DIR` | String | `tmp/evilution_children` | Override the directory used by `--quiet-children`. |
|
|
116
|
-
| `--isolation MODE` | String | `auto` | Isolation strategy: `auto`, `fork`, or `in_process`. `auto` selects `fork` for Rails projects. See [docs/isolation.md](docs/isolation.md). |
|
|
117
|
-
| `--preload FILE` | String | _(auto)_ | File to require in parent before forking workers. Auto-detect chain for Rails projects: `spec/rails_helper.rb` → `spec/spec_helper.rb` → `test/test_helper.rb`.
|
|
128
|
+
| `--isolation MODE` | String | `auto` | Isolation strategy: `auto`, `fork`, or `in_process`. `auto` selects `fork` for Rails projects and packaged gems (`*.gemspec`), `in_process` otherwise. See [docs/isolation.md](docs/isolation.md). |
|
|
129
|
+
| `--preload FILE` | String | _(auto)_ | File to require in parent before forking workers. Auto-detect chain for Rails projects: `spec/rails_helper.rb` → `spec/spec_helper.rb` → `test/test_helper.rb`. For non-Rails gems: `spec/spec_helper.rb` → `test/test_helper.rb` → `test/helper.rb`, falling back to the gem entry `lib/<gem>.rb` (with a warning naming the searched paths). Pass `--no-preload` to opt out. |
|
|
118
130
|
| `--no-preload` | Boolean | _(enabled)_ | Disable parent-process preload. |
|
|
119
131
|
| `--skip-heredoc-literals` | Boolean | false | Skip all string literal mutations inside heredocs. |
|
|
120
132
|
| `--show-disabled` | Boolean | false | Report mutations skipped by `# evilution:disable` comments. |
|
|
@@ -172,9 +184,9 @@ schema_version: 1 # opts into strict validation (rejects unknown keys
|
|
|
172
184
|
# integration: rspec # test framework: rspec, minitest, test_unit
|
|
173
185
|
# suggest_tests: false # concrete test code in suggestions (matches integration)
|
|
174
186
|
# save_session: false # persist results under .evilution/results/
|
|
175
|
-
# isolation: auto # auto | fork | in_process (auto selects fork for Rails)
|
|
187
|
+
# isolation: auto # auto | fork | in_process (auto selects fork for Rails + gems)
|
|
176
188
|
# canary: true # proof-of-life synthetic mutation at session start (false to skip)
|
|
177
|
-
# preload: null # path to preload before forking; false to disable; auto-detects for Rails
|
|
189
|
+
# preload: null # path to preload before forking; false to disable; auto-detects for Rails + gems
|
|
178
190
|
# skip_heredoc_literals: false # skip string literal mutations inside heredocs (recommended for Rails: heredoc SQL/templates rarely have test coverage)
|
|
179
191
|
# show_disabled: false # report mutations skipped by disable comments
|
|
180
192
|
# baseline_session: null # path to session file for HTML comparison
|
|
@@ -219,7 +231,7 @@ All keys recognised under `schema_version: 1`:
|
|
|
219
231
|
| `jobs` | Integer | `1` | Number of parallel workers. |
|
|
220
232
|
| `fail_fast` | Integer / null | `null` | Stop after N surviving mutants. `null` = disabled. |
|
|
221
233
|
| `baseline` | Boolean | `true` | Run baseline test suite to detect pre-existing failures (marked `:neutral`). |
|
|
222
|
-
| `isolation` | String | `auto` | Isolation strategy: `auto`, `fork`, `in_process`. `auto` selects `fork` for Rails projects.
|
|
234
|
+
| `isolation` | String | `auto` | Isolation strategy: `auto`, `fork`, `in_process`. `auto` selects `fork` for Rails projects and packaged gems (`*.gemspec`). |
|
|
223
235
|
| `incremental` | Boolean | `false` | Cache killed/timeout results across runs. |
|
|
224
236
|
| `suggest_tests` | Boolean | `false` | Generate concrete test code in survivor suggestions (matches `integration`). |
|
|
225
237
|
| `progress` | Boolean | `true` | TTY progress bar. |
|
|
@@ -232,10 +244,11 @@ All keys recognised under `schema_version: 1`:
|
|
|
232
244
|
| `skip_heredoc_literals` | Boolean | `false` | Skip string literal mutations inside heredocs. |
|
|
233
245
|
| `related_specs_heuristic` | Boolean | `false` | Append related request/integration/feature/system specs for `includes(...)` mutations. |
|
|
234
246
|
| `fallback_to_full_suite` | Boolean | `false` | When no matching spec resolves, run the entire suite instead of marking the mutation `:unresolved`. |
|
|
235
|
-
| `preload` | String / Boolean / null | `null` | File to preload in parent before forking. `false` to disable. `null` to auto-detect for Rails.
|
|
247
|
+
| `preload` | String / Boolean / null | `null` | File to preload in parent before forking. `false` to disable. `null` to auto-detect for Rails projects and packaged gems. |
|
|
236
248
|
| `spec_mappings` | Hash<String, String/Array> | `{}` | Custom mapping from source path to spec path(s). |
|
|
237
249
|
| `spec_pattern` | String / null | `null` | Glob restricting resolved spec candidates. |
|
|
238
250
|
| `example_targeting` | Boolean | `true` | Per-mutation example-level targeting. |
|
|
251
|
+
| `example_targeting_strategy` | String | `lexical` | How targeting picks examples: `lexical` (name-grep) or `coverage` (examples that execute the mutated line, from a cached coverage map). |
|
|
239
252
|
| `example_targeting_fallback` | String | `full_file` | When targeting finds no example: `full_file` or `unresolved`. |
|
|
240
253
|
| `example_targeting_cache` | Hash | `{ max_files: 50, max_blocks: 10000 }` | LRU cache bounds for the example-targeting AST parser. |
|
|
241
254
|
| `quiet_children` | Boolean | `false` | Redirect each worker's stdout/stderr to per-pid files under `quiet_children_dir`. |
|
|
@@ -375,7 +388,7 @@ Compatibility policy for the `1.x` gem line:
|
|
|
375
388
|
| `unresolved` | No spec file resolved for the mutated source — **coverage gap, not a failure**. Use `--fallback-full-suite` to run the full suite instead. | excluded |
|
|
376
389
|
| `unparseable` | Mutated source failed to parse (e.g. dangling heredoc opener after `method_body_replacement`). Short-circuited — never executed. | excluded |
|
|
377
390
|
|
|
378
|
-
Unresolved mutations indicate a missing test mapping — the file has no corresponding test file that the resolver could find (for example, an RSpec `_spec.rb` file or a Minitest `_test.rb` file, depending on configuration). The resolver searches
|
|
391
|
+
Unresolved mutations indicate a missing test mapping — the file has no corresponding test file that the resolver could find (for example, an RSpec `_spec.rb` file or a Minitest `_test.rb` file, depending on configuration). The resolver searches the `lib/`-mirrored path, common non-mirrored buckets (`spec/unit`, `spec/lib`, `test/unit`, `test/lib`), and the flat `test_`-prefixed Minitest/Test::Unit convention (`test/test_connection_pool_timed_stack.rb`), so a high unresolved rate usually means a genuinely missing or unconventionally-placed test; a run that leaves many mutations unresolved prints an unresolved-rate warning with a best-guess spec path per source file. They are reported separately so you can act on them (add a test, adjust test naming, pass `--spec`, or opt in to the full-suite fallback) without inflating the error count.
|
|
379
392
|
|
|
380
393
|
## Mutation Operators (74 total)
|
|
381
394
|
|
|
@@ -670,7 +683,7 @@ Use when you know which file was modified and want to verify its test coverage.
|
|
|
670
683
|
bundle exec evilution run lib/models/user.rb lib/models/account.rb lib/models/order.rb
|
|
671
684
|
```
|
|
672
685
|
|
|
673
|
-
Pass multiple file paths on a single invocation to amortise startup cost. The framework (Rails, Sorbet, etc.) and the `preload` chain (`spec/rails_helper.rb` → `spec/spec_helper.rb` → `test/test_helper.rb`) load **once** in the parent process. When `--isolation=fork` is selected (the default `--isolation=auto` resolves to `fork` on Rails projects), every subsequent mutation across all files forks from that warmed parent — materially faster than scripting a `for f in ...; do bundle exec evilution run "$f"; done` loop, which pays the bootstrap per file. With `--isolation=in_process` (default for non-Rails projects under `auto`), there is no per-mutation fork, but the parent-process boot still runs once instead of N times. Per-file paths and line numbers are preserved in the report (`survived[].file`, HTML grouping by source file).
|
|
686
|
+
Pass multiple file paths on a single invocation to amortise startup cost. The framework (Rails, Sorbet, etc.) and the `preload` chain (`spec/rails_helper.rb` → `spec/spec_helper.rb` → `test/test_helper.rb`) load **once** in the parent process. When `--isolation=fork` is selected (the default `--isolation=auto` resolves to `fork` on Rails projects and packaged gems), every subsequent mutation across all files forks from that warmed parent — materially faster than scripting a `for f in ...; do bundle exec evilution run "$f"; done` loop, which pays the bootstrap per file. With `--isolation=in_process` (default for non-Rails, non-gem projects under `auto`), there is no per-mutation fork, but the parent-process boot still runs once instead of N times. Per-file paths and line numbers are preserved in the report (`survived[].file`, HTML grouping by source file).
|
|
674
687
|
|
|
675
688
|
### 6. Fixing surviving mutants
|
|
676
689
|
|
|
@@ -685,6 +698,22 @@ For each entry in `survived[]`:
|
|
|
685
698
|
|
|
686
699
|
Entries in the JSON `errors[]` array represent mutations that raised an exception (syntax error, load failure, or runtime crash) rather than producing a test outcome. Each entry includes `error_class`, `error_message`, and the first 5 `error_backtrace` lines. Use these fields to decide whether the error is a bug in the mutation operator (file an issue), a load-time problem in the mutated source (often `NoMethodError: super called outside of method` or constant-redefinition issues), or a genuine crash that the original tests should have caught. Run with `--verbose` to stream the same error details to stderr during the run.
|
|
687
700
|
|
|
701
|
+
### Unresolved mutations — behaviour-named spec layouts
|
|
702
|
+
|
|
703
|
+
`SpecResolver` auto-detects mirrored (`spec/foo_spec.rb`), non-mirrored (`spec/unit`, `spec/lib`, `test/unit`), and dir-grouped (`test/unit/<class>/*_test.rb`) layouts. It deliberately does **not** guess for **behaviour-named** layouts — where specs are named by behaviour rather than by lib path, so no path or class name ties a source file to a spec. The canonical example is [aasm](https://github.com/aasm/aasm): `lib/aasm/base.rb` is the central DSL class, but it has no `base_spec.rb`; its behaviour is exercised through the public API by `spec/unit/{event,callbacks,guard,api}_spec.rb`, none of which name `AASM::Base`. Every mutation in such a file resolves to no spec and is reported `:unresolved` (a coverage gap, not a failure — `errors=0`).
|
|
704
|
+
|
|
705
|
+
This is intentional: a fuzzy content/grep match would pick a wrong-but-plausible spec subset and report a misleading score. `:unresolved` plus the warning is the honest signal. To get a real score, opt into one of the two explicit recovery paths the warning names:
|
|
706
|
+
|
|
707
|
+
```bash
|
|
708
|
+
# run a directory of specs you know covers the file (--spec-dir globs it)
|
|
709
|
+
bundle exec evilution mutate lib/aasm/base.rb --spec-dir spec/unit
|
|
710
|
+
# or name explicit files as a single comma-separated --spec value (no shell glob)
|
|
711
|
+
bundle exec evilution mutate lib/aasm/base.rb --spec spec/unit/event_spec.rb,spec/unit/callbacks_spec.rb
|
|
712
|
+
|
|
713
|
+
# or run the whole suite per mutation (slower, but correct-by-superset)
|
|
714
|
+
bundle exec evilution mutate lib/aasm/base.rb --fallback-full-suite
|
|
715
|
+
```
|
|
716
|
+
|
|
688
717
|
### Long Minitest fork runs — not a hang
|
|
689
718
|
|
|
690
719
|
Minitest projects under `--isolation=fork` re-bootstrap the test environment (`test_helper.rb`, plugins, runnable state) once per mutation. On constant-heavy files (e.g. Shopify/liquid's `lib/liquid/lexer.rb`, ~270 mutations) the wall-clock cost is dominated by that per-fork bootstrap and any mutations that hit a `--timeout` rather than killing the test fast. A single-worker run (`-j 1`) on a few hundred mutations can take 4+ minutes; combined with `--no-progress` and a non-TTY stderr (CI, redirected logs) the run looks silent the entire time.
|
|
@@ -773,7 +802,7 @@ Tests 4 paths (InProcess isolation, Fork isolation, mutation generation + stripp
|
|
|
773
802
|
2. **Extract** — Methods are identified as mutation subjects
|
|
774
803
|
3. **Filter** — Disable comments, Sorbet `sig` blocks, and AST ignore patterns exclude mutations before execution
|
|
775
804
|
4. **Mutate** — 74 operators produce text replacements at precise byte offsets (source-level surgery, no AST unparsing); heredoc literal text is skipped by default. Identical byte-mutations from different operators are deduplicated by `(file_path, mutated_source)` so the count is not inflated by overlap
|
|
776
|
-
5. **Isolate** — Mutations are applied to temporary file copies (never modifying originals); load-path redirection ensures `require` resolves the mutated copy. Default isolation is in-process for plain Ruby projects and fork for Rails projects (auto-detected); `--isolation fork` forces forked child processes. Both sequential and parallel (`--jobs N`) modes respect the configured isolation strategy
|
|
805
|
+
5. **Isolate** — Mutations are applied to temporary file copies (never modifying originals); load-path redirection ensures `require` resolves the mutated copy. Default isolation is in-process for plain Ruby projects (no gemspec) and fork for Rails projects and packaged gems (auto-detected); `--isolation fork` forces forked child processes. Both sequential and parallel (`--jobs N`) modes respect the configured isolation strategy
|
|
777
806
|
6. **Test** — The configured test framework (RSpec, Minitest, or Test::Unit) executes against the mutated source
|
|
778
807
|
7. **Collect** — Source strings and AST nodes are released after use to minimize memory retention
|
|
779
808
|
8. **Report** — Results aggregated into text, JSON, or HTML, including efficiency metrics and peak memory usage
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 256 256" width="256" height="256" role="img" aria-label="Evilution — evil ruby mutant mascot">
|
|
2
|
+
<title>Evilution</title>
|
|
3
|
+
<defs>
|
|
4
|
+
<filter id="glow" x="-60%" y="-60%" width="220%" height="220%">
|
|
5
|
+
<feGaussianBlur stdDeviation="2.4" result="b"/>
|
|
6
|
+
<feMerge><feMergeNode in="b"/><feMergeNode in="SourceGraphic"/></feMerge>
|
|
7
|
+
</filter>
|
|
8
|
+
</defs>
|
|
9
|
+
|
|
10
|
+
<!-- clawed arms (behind gem) -->
|
|
11
|
+
<g stroke="#470B13" stroke-width="2.5" stroke-linejoin="round">
|
|
12
|
+
<path d="M60 124 L40 132 L46 138 L36 142 L48 147 L40 153 L56 150 L64 136 Z" fill="#9A1826"/>
|
|
13
|
+
<path d="M196 124 L216 132 L210 138 L220 142 L208 147 L216 153 L200 150 L192 136 Z" fill="#7F1420"/>
|
|
14
|
+
</g>
|
|
15
|
+
|
|
16
|
+
<!-- faceted ruby body -->
|
|
17
|
+
<g stroke="#470B13" stroke-width="2.5" stroke-linejoin="round">
|
|
18
|
+
<polygon points="98,82 128,82 128,120 88,120" fill="#D5384A"/>
|
|
19
|
+
<polygon points="128,82 158,82 168,120 128,120" fill="#B91C2C"/>
|
|
20
|
+
<polygon points="54,120 88,120 98,82" fill="#E2566B"/>
|
|
21
|
+
<polygon points="168,120 202,120 158,82" fill="#911A28"/>
|
|
22
|
+
<polygon points="54,120 88,120 128,214" fill="#A81B2A"/>
|
|
23
|
+
<polygon points="88,120 128,120 128,214" fill="#911A28"/>
|
|
24
|
+
<polygon points="128,120 168,120 128,214" fill="#7C1320"/>
|
|
25
|
+
<polygon points="168,120 202,120 128,214" fill="#5E0F18"/>
|
|
26
|
+
</g>
|
|
27
|
+
<polygon points="98,82 158,82 202,120 128,214 54,120" fill="none" stroke="#3A0910" stroke-width="3" stroke-linejoin="round"/>
|
|
28
|
+
|
|
29
|
+
<!-- glowing mutant eyes -->
|
|
30
|
+
<g filter="url(#glow)">
|
|
31
|
+
<polygon points="100,99 123,106 118,117 96,110" fill="#22D3EE"/>
|
|
32
|
+
<polygon points="156,99 133,106 138,117 160,110" fill="#22D3EE"/>
|
|
33
|
+
<polygon points="110,105 118,107 116,113 108,111" fill="#0E1A2B"/>
|
|
34
|
+
<polygon points="146,105 138,107 140,113 148,111" fill="#0E1A2B"/>
|
|
35
|
+
</g>
|
|
36
|
+
|
|
37
|
+
<!-- fanged grin -->
|
|
38
|
+
<path d="M106 138 L150 138 L142 149 L134 144 L128 150 L122 144 L114 149 Z" fill="#1B060C"/>
|
|
39
|
+
<polygon points="118,138 125,138 122,148" fill="#EAF2F8"/>
|
|
40
|
+
<polygon points="131,138 138,138 134,148" fill="#EAF2F8"/>
|
|
41
|
+
|
|
42
|
+
<!-- mutation cracks leaking binary -->
|
|
43
|
+
<g filter="url(#glow)" fill="none" stroke="#67E8F9" stroke-width="1.7" stroke-linecap="round" opacity="0.92">
|
|
44
|
+
<polyline points="166,90 156,106 169,120 159,134"/>
|
|
45
|
+
<polyline points="82,150 92,160 86,172"/>
|
|
46
|
+
</g>
|
|
47
|
+
<g fill="#7FE9F7" font-family="monospace" font-size="11" font-weight="700" opacity="0.85">
|
|
48
|
+
<text x="170" y="128" transform="rotate(12 170 128)">1</text>
|
|
49
|
+
<text x="176" y="140" transform="rotate(12 176 140)">0</text>
|
|
50
|
+
<text x="168" y="150" transform="rotate(12 168 150)">1</text>
|
|
51
|
+
<text x="78" y="184" transform="rotate(-14 78 184)">0</text>
|
|
52
|
+
</g>
|
|
53
|
+
</svg>
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 256 256" width="256" height="256" role="img" aria-label="Evilution — evil ruby mutant mascot">
|
|
2
|
+
<title>Evilution</title>
|
|
3
|
+
<defs>
|
|
4
|
+
<radialGradient id="bg" cx="50%" cy="40%" r="78%">
|
|
5
|
+
<stop offset="0%" stop-color="#27364D"/>
|
|
6
|
+
<stop offset="100%" stop-color="#1E293B"/>
|
|
7
|
+
</radialGradient>
|
|
8
|
+
<pattern id="hex" width="56" height="100" patternUnits="userSpaceOnUse" patternTransform="scale(0.42)">
|
|
9
|
+
<path d="M28 66L0 50L0 16L28 0L56 16L56 50L28 66L28 100M28 0L28 34L0 50M28 34L56 50"
|
|
10
|
+
fill="none" stroke="#3B4D68" stroke-width="2" opacity="0.30"/>
|
|
11
|
+
</pattern>
|
|
12
|
+
<filter id="glow" x="-60%" y="-60%" width="220%" height="220%">
|
|
13
|
+
<feGaussianBlur stdDeviation="2.4" result="b"/>
|
|
14
|
+
<feMerge><feMergeNode in="b"/><feMergeNode in="SourceGraphic"/></feMerge>
|
|
15
|
+
</filter>
|
|
16
|
+
</defs>
|
|
17
|
+
|
|
18
|
+
<!-- backdrop -->
|
|
19
|
+
<rect width="256" height="256" rx="52" fill="url(#bg)"/>
|
|
20
|
+
<rect width="256" height="256" rx="52" fill="url(#hex)"/>
|
|
21
|
+
|
|
22
|
+
<!-- clawed arms (behind gem) -->
|
|
23
|
+
<g stroke="#470B13" stroke-width="2.5" stroke-linejoin="round">
|
|
24
|
+
<path d="M60 124 L40 132 L46 138 L36 142 L48 147 L40 153 L56 150 L64 136 Z" fill="#9A1826"/>
|
|
25
|
+
<path d="M196 124 L216 132 L210 138 L220 142 L208 147 L216 153 L200 150 L192 136 Z" fill="#7F1420"/>
|
|
26
|
+
</g>
|
|
27
|
+
|
|
28
|
+
<!-- faceted ruby body -->
|
|
29
|
+
<g stroke="#470B13" stroke-width="2.5" stroke-linejoin="round">
|
|
30
|
+
<!-- table (top) -->
|
|
31
|
+
<polygon points="98,82 128,82 128,120 88,120" fill="#D5384A"/>
|
|
32
|
+
<polygon points="128,82 158,82 168,120 128,120" fill="#B91C2C"/>
|
|
33
|
+
<!-- crown shoulders -->
|
|
34
|
+
<polygon points="54,120 88,120 98,82" fill="#E2566B"/>
|
|
35
|
+
<polygon points="168,120 202,120 158,82" fill="#911A28"/>
|
|
36
|
+
<!-- pavilion: main facet ridges run from the table corners (88,120 / 168,120)
|
|
37
|
+
down to the culet, continuing the crown's inclined edges -->
|
|
38
|
+
<polygon points="54,120 88,120 128,214" fill="#A81B2A"/>
|
|
39
|
+
<polygon points="88,120 128,120 128,214" fill="#911A28"/>
|
|
40
|
+
<polygon points="128,120 168,120 128,214" fill="#7C1320"/>
|
|
41
|
+
<polygon points="168,120 202,120 128,214" fill="#5E0F18"/>
|
|
42
|
+
</g>
|
|
43
|
+
<!-- crisp outer silhouette -->
|
|
44
|
+
<polygon points="98,82 158,82 202,120 128,214 54,120" fill="none" stroke="#3A0910" stroke-width="3" stroke-linejoin="round"/>
|
|
45
|
+
|
|
46
|
+
<!-- glowing mutant eyes -->
|
|
47
|
+
<g filter="url(#glow)">
|
|
48
|
+
<polygon points="100,99 123,106 118,117 96,110" fill="#22D3EE"/>
|
|
49
|
+
<polygon points="156,99 133,106 138,117 160,110" fill="#22D3EE"/>
|
|
50
|
+
<polygon points="110,105 118,107 116,113 108,111" fill="#0E1A2B"/>
|
|
51
|
+
<polygon points="146,105 138,107 140,113 148,111" fill="#0E1A2B"/>
|
|
52
|
+
</g>
|
|
53
|
+
|
|
54
|
+
<!-- fanged grin -->
|
|
55
|
+
<path d="M106 138 L150 138 L142 149 L134 144 L128 150 L122 144 L114 149 Z" fill="#1B060C"/>
|
|
56
|
+
<polygon points="118,138 125,138 122,148" fill="#EAF2F8"/>
|
|
57
|
+
<polygon points="131,138 138,138 134,148" fill="#EAF2F8"/>
|
|
58
|
+
|
|
59
|
+
<!-- mutation cracks leaking binary -->
|
|
60
|
+
<g filter="url(#glow)" fill="none" stroke="#67E8F9" stroke-width="1.7" stroke-linecap="round" opacity="0.92">
|
|
61
|
+
<polyline points="166,90 156,106 169,120 159,134"/>
|
|
62
|
+
<polyline points="82,150 92,160 86,172"/>
|
|
63
|
+
</g>
|
|
64
|
+
<g fill="#7FE9F7" font-family="monospace" font-size="11" font-weight="700" opacity="0.85">
|
|
65
|
+
<text x="170" y="128" transform="rotate(12 170 128)">1</text>
|
|
66
|
+
<text x="176" y="140" transform="rotate(12 176 140)">0</text>
|
|
67
|
+
<text x="168" y="150" transform="rotate(12 168 150)">1</text>
|
|
68
|
+
<text x="78" y="184" transform="rotate(-14 78 184)">0</text>
|
|
69
|
+
</g>
|
|
70
|
+
</svg>
|
data/docs/isolation.md
CHANGED
|
@@ -9,7 +9,7 @@ is used.
|
|
|
9
9
|
|
|
10
10
|
| Strategy | What it does | When to use |
|
|
11
11
|
| ------------ | ---------------------------------------------------------------------- | ---------------------------------------------------------- |
|
|
12
|
-
| `auto` | Default. Picks `fork` for Rails projects, `in_process` otherwise.
|
|
12
|
+
| `auto` | Default. Picks `fork` for Rails projects and packaged gems, `in_process` otherwise. | Leave this on unless you have a specific reason to change. |
|
|
13
13
|
| `fork` | Forks a fresh child process per mutant. Parent `SIGKILL`s on timeout. | Rails / ActiveRecord projects; any code that uses mutexes, monitors, or async-interrupt masks. |
|
|
14
14
|
| `in_process` | Runs the mutant inside the runner process under `Timeout.timeout`. | Pure-Ruby libraries that do not use async-interrupt masks. |
|
|
15
15
|
|
|
@@ -42,6 +42,15 @@ a Rails project, evilution emits a warning naming the hazard and proceeds
|
|
|
42
42
|
anyway — sometimes you know the code under test never enters a masked
|
|
43
43
|
section, and that is your call to make.
|
|
44
44
|
|
|
45
|
+
`auto` also resolves to `fork` when the target files live under a packaged
|
|
46
|
+
gem (a directory containing a `*.gemspec`). A gem's library and test
|
|
47
|
+
framework must be loaded in the parent before forking — see [Automatic
|
|
48
|
+
preload](#automatic-preload) — and `in_process` cannot preload them without
|
|
49
|
+
polluting the host process. A non-Rails gem run under the old `in_process`
|
|
50
|
+
default therefore produced 0 examples / 100% errors out of the box; defaulting
|
|
51
|
+
gems to `fork` lets auto-preload fire (EV-z03y, PR #1375). A plain non-Rails,
|
|
52
|
+
non-gem project (no gemspec) still defaults to `in_process`.
|
|
53
|
+
|
|
45
54
|
The same hazard applies to any Ruby code that uses
|
|
46
55
|
`Thread.handle_interrupt(... => :never)`: `Mutex#synchronize`, `Monitor#synchronize`,
|
|
47
56
|
`Queue`, `ActiveSupport::Notifications::Fanout` listeners, and custom cleanup
|
|
@@ -67,7 +76,28 @@ automatically looks for these files in order and preloads the first one it
|
|
|
67
76
|
finds:
|
|
68
77
|
|
|
69
78
|
1. `spec/rails_helper.rb` (RSpec)
|
|
70
|
-
2. `
|
|
79
|
+
2. `spec/spec_helper.rb` (RSpec)
|
|
80
|
+
3. `test/test_helper.rb` (Minitest)
|
|
81
|
+
|
|
82
|
+
For a packaged gem (auto-detected via `*.gemspec`, no Rails root), evilution
|
|
83
|
+
preloads the conventional test helper — which loads both the gem's library
|
|
84
|
+
and the suite's framework/support setup so example groups register — in this
|
|
85
|
+
order, falling back to the gem's library entry point (`lib/<gem>.rb`):
|
|
86
|
+
|
|
87
|
+
1. `spec/spec_helper.rb` (RSpec)
|
|
88
|
+
2. `test/test_helper.rb` (Minitest / Test::Unit)
|
|
89
|
+
3. `test/helper.rb` (flat-layout convention)
|
|
90
|
+
4. `test/<gem>/helper.rb` (namespaced suites, e.g. ruby/csv's `test/csv/helper.rb`; the namespace is derived from the gem name, with the dotted form for dash-named gems) (PR #1386)
|
|
91
|
+
|
|
92
|
+
When a gem is detected but none of those helpers exist, evilution prints a
|
|
93
|
+
warning naming the locations it looked in and pointing at `--preload`, so a
|
|
94
|
+
non-standard test layout reads as a fixable configuration issue rather than a
|
|
95
|
+
silent 0% (EV-z03y, PR #1375).
|
|
96
|
+
|
|
97
|
+
Minitest/Test::Unit helpers that `require "test_helper"` (or any non-relative
|
|
98
|
+
`require "support/..."`) work without `-Itest`: evilution puts the test root
|
|
99
|
+
on `$LOAD_PATH` for the preload just as the test runner would (EV-5hk5, PR
|
|
100
|
+
#1373).
|
|
71
101
|
|
|
72
102
|
No configuration needed.
|
|
73
103
|
|