evilution 0.30.2 → 0.30.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: af414a5edcfb3e25beea8ef2dea4235f7f9cadc1d8e086a526846ff456639235
-  data.tar.gz: a32bc1f4dcf1898dc8be020356ca746354384894994cab0dac5e8d0b25692ebb
+  metadata.gz: 2bd9eaca72cb973c6e11841144b8bae68d2a838aef1ff64c429e18b4090505f1
+  data.tar.gz: bbdedbc5ef66ca367deeec89a53399ce83a235616fe2ac179ed98bf880f8be9b
 SHA512:
-  metadata.gz: b65c3fd7ef73fccbf08a9d3dcf63c6e2160fa4ef3625139c7a8c4be137b52dcf55bd4b84da58d80862bf2f34cbf022bd2a1c937f474a292eb5368dde816fd0fd
-  data.tar.gz: 80b60c39df9dadb4193a7d65ff0fe22bfa3e395f658b13da53a8042699e7e9d869920a4ab127759d3e975cecdcd0821bc3bdfff6c2f482f6672383b2e924c3a4
+  metadata.gz: b7cf12d8381e05d66b1a2ffcf1c243e3e18d66f8f073de892d40ff139bb6952214ac5c6bdcff3ab026dd475601629790e7c4d5f86f8315a8baf989bcb5b4aa7b
+  data.tar.gz: 21a523a1f2753c5e70b9fb2d53262d28b169661b75c2bd2ca44ce5f9e5c6234591f6d8dcc935fea8c49a79689d7be5a2883cae6ff4517b711379b80b8d8315bf

data/.beads/interactions.jsonl

@@ -377,3 +377,13 @@
 {"id":"int-90ffe424","kind":"field_change","created_at":"2026-05-15T06:58:55.664447898Z","actor":"Denis Kiselev","issue_id":"EV-225l","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Canary SKIPPED 2026-05-15. test_helper.rb spawns its own redis-server (no env override); host has no redis-server binary. Sidekiq EV-mn3p already covers concurrency tier signal (100% PASS on client.rb). Resque revisit only justified if a resque-specific bug surfaces later. Artifact .artifacts/resque_resque.yml."}}
 {"id":"int-0648236b","kind":"field_change","created_at":"2026-05-15T07:00:59.017162278Z","actor":"Denis Kiselev","issue_id":"EV-ks6i","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Canary PASS 0.30.1+EV-5nxs. configuration.rb 40.69% upstream gap. body.rb 66.78% upstream gap. 0 errored mutations across both targets. No evilution bugs. Artifact .artifacts/mikel_mail.yml."}}
 {"id":"int-8a14f28f","kind":"field_change","created_at":"2026-05-15T08:24:58.326924498Z","actor":"Denis Kiselev","issue_id":"EV-5nxs","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}}
+{"id":"int-e084d525","kind":"field_change","created_at":"2026-05-15T09:44:03.368189125Z","actor":"Denis Kiselev","issue_id":"EV-9a6c","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Fixed PR #1251, shipped 0.30.2. method_body_replacement targets body.statements (not whole BeginNode) for method-level rescue/ensure; super-detection still scans full body. Copilot review addressed. Verified on redis-rb client.rb — 5 super-errors eliminated."}}
+{"id":"int-be59f27d","kind":"field_change","created_at":"2026-05-15T09:44:04.508238014Z","actor":"Denis Kiselev","issue_id":"EV-5rtm","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Canary PASS 0.30.2. pipeline.rb 100% (240/240), client.rb 76.67% (138k/42s/0err — 5 prior EV-9a6c errors eliminated), errors.rb 0% upstream gap. 3 evilution bugs surfaced + all fixed: EV-ju3o #1240 + EV-7u9c #1241 (0.30.1), EV-9a6c #1247 (0.30.2). Remaining survivors = redis-rb test debt. Artifact .artifacts/redis_redis-rb.yml."}}
+{"id":"int-3960db7a","kind":"field_change","created_at":"2026-05-15T10:18:04.635769183Z","actor":"Denis Kiselev","issue_id":"EV-xqv3","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Canary done 0.30.2. Baseline 1311/0/0. lexer.rb 1299 mutations 26.79% — real upstream coverage gap (abstract base class, subclass tests don't exercise base). 1 evilution bug surfaced EV-5nxs #1248 (autorun stub timing) — fixed PR #1249, shipped 0.30.2. Artifact .artifacts/rouge-ruby_rouge.yml."}}
+{"id":"int-b24804d2","kind":"field_change","created_at":"2026-05-15T10:19:29.488481997Z","actor":"Denis Kiselev","issue_id":"EV-rxob","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"20-repo stability matrix complete. All 20 sub-canaries closed. Results: 10 pass, 2 smoke_fail (pagy/kaminari — 0% score, follow-up EV-laxo investigates suspected reset_state vs class-reopen), 1 skipped (resque — host redis-server infra). 4 evilution bugs surfaced + all fixed/shipped: EV-ju3o #1240 + EV-7u9c #1241 (0.30.1), EV-9a6c #1247 + EV-5nxs #1248 (0.30.2). Aggregate .artifacts/SUMMARY.md. Open follow-up: EV-laxo (pagy/kaminari 0% root cause)."}}
+{"id":"int-8a9bcade","kind":"field_change","created_at":"2026-05-15T12:15:27.827753441Z","actor":"Denis Kiselev","issue_id":"EV-225l","extra":{"field":"status","new_value":"open","old_value":"closed"}}
+{"id":"int-434f659f","kind":"field_change","created_at":"2026-05-15T12:15:28.901913417Z","actor":"Denis Kiselev","issue_id":"EV-225l","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Canary PASS 0.30.2 (re-run, earlier skip was wrong). Baseline 285/286 (1 error = port-9736-hardcoded test, infra collateral). stat.rb 95.92%, job.rb 100% (451 muts, 1 equivalent). No evilution bugs. Ran against shared docker redis on 6381 with 3-line test_helper.rb patch (which-guard, backtick self-spawn, port). Artifact .artifacts/resque_resque.yml."}}
+{"id":"int-d1684261","kind":"field_change","created_at":"2026-05-15T13:01:27.370142624Z","actor":"Denis Kiselev","issue_id":"EV-laxo","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Investigation complete. Root causes found, both confirmed via instrumented iseq probing. pagy 0%: fork isolation clobbers eval'd mutation — spec require()s lazily-loaded target file not in $LOADED_FEATURES, reloads original from disk (EV-vxgl #1253 P1). kaminari 0%: kaminari runs on test-unit gem not Minitest (ActiveSupport::TestCase < Test::Unit::TestCase), test classes never Minitest::Runnable, 0 tests dispatch — plus evilution silently scores 0% instead of erroring (EV-5dxk #1254 P2). Original hypothesis (reset_state runnables clear) was wrong — disproven."}}
+{"id":"int-011584d4","kind":"field_change","created_at":"2026-05-15T15:55:27.437673072Z","actor":"Denis Kiselev","issue_id":"EV-vxgl","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Fixed PR (merged). MutationApplier#mark_feature_loaded registers realpath in $LOADED_FEATURES post-eval — spec require() no longer reloads original. pagy jobs=4: 0% -> 82.81%. Residual gap vs jobs=1 is separate (EV-wu8w in_process inflation)."}}
+{"id":"int-c44e7652","kind":"field_change","created_at":"2026-05-16T12:45:46.793709029Z","actor":"Denis Kiselev","issue_id":"EV-xfaj","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Fixed in PR #1260 — run_minitest counts test methods from Minitest::Runnable.runnables registry instead of an evictable reporter"}}
+{"id":"int-eea19850","kind":"field_change","created_at":"2026-05-16T14:22:49.671269232Z","actor":"Denis Kiselev","issue_id":"EV-wu8w","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Fixed in PR #1264 — run_minitest reads verdict from evilution's own per-run SummaryReporter attached after plugin init"}}

data/CHANGELOG.md

@@ -2,6 +2,21 @@
 
 Versioning policy: see [docs/versioning.md](docs/versioning.md).
 
+## [0.30.4] - 2026-05-16
+
+### Fixed
+
+- **Minitest integration no longer errors every mutation when the project test helper calls `Minitest::Reporters.use!`** — a regression shipped in 0.30.3. The EV-5dxk zero-tests guard read `summary.count` from a `SummaryReporter` that evilution adds to its `CompositeReporter`. But when the target's test helper uses the `minitest-reporters` gem (`Minitest::Reporters.use!` — an extremely common setup), that plugin **replaces** the composite's reporters during `Minitest.init_plugins`, evicting evilution's own `SummaryReporter`. `summary.count` then never advanced — it read 0 even when tests ran — so the guard false-fired and every mutation was reported as errored ("no Minitest tests executed"), collapsing the score to a meaningless 0/0. `run_minitest` now derives the dispatched test-method count from the runnable registry (`Minitest::Runnable.runnables`), which is immune to reporter plugins. Surfaced by the EV-7764 pagy stability canary (EV-xfaj, PR #1260, GH #1259)
+- **In-process isolation no longer false-kills genuinely-equivalent mutations and inflates the score** — `run_minitest` returned `passed: reporter.passed?`. When the target test helper calls `Minitest::Reporters.use!`, `init_plugins` replaces the composite's reporters with `minitest-reporters`' `DelegateReporter`, which delegates to a process-global reporter created once by `use!` and never reset between runs. Under in-process isolation one process runs every mutation in sequence, so that reporter's failures accumulate: `reporter.passed?` then reported `false` for every mutation after the first genuine kill, false-killing real survivors. On the pagy `request.rb` canary, in-process scored 98.44% against fork's correct 82.81%. `run_minitest` now attaches evilution's own fresh `SummaryReporter` to the composite **after** plugin init (so init_plugins cannot evict it) and reads the run's verdict from that per-run reporter; in-process and fork now converge (EV-wu8w, PR #1264, GH #1263)
+- **`MinitestCrashDetector` survives reporter-plugin eviction** — the same `Minitest::Reporters.use!` swap that evicted evilution's `SummaryReporter` also detached the `MinitestCrashDetector`, which was attached to the composite before `initialize_minitest_state`. With the detector evicted, `build_minitest_result`'s `detector.only_crashes?` path went dead on `minitest-reporters` projects: a crash-only mutation result lost its `test_crashed` / `error` / `error_class` crash diagnostics and returned a plain killed result. The detector is now attached after plugin init alongside the `SummaryReporter`, keeping it in the live composite (EV-8z2n, PR #1265, GH #1262)
+
+## [0.30.3] - 2026-05-16
+
+### Fixed
+
+- **Fork isolation no longer clobbers the eval'd mutation when the spec `require`s a lazily-loaded target file** — `MutationApplier` evals the mutated source straight into the VM, which does not register a `$LOADED_FEATURES` entry. When the project lazy-loads the target file (only the spec references it) and the spec then `require`s it, `require` found nothing loaded and re-read the **original** file from disk, clobbering the mutation before any test ran. Every mutation then silently survived. In-process runs masked it — the first mutation's spec-load populated `$LOADED_FEATURES` for the rest of the process — but under fork isolation each worker restarts from the same pre-`require` snapshot, so the whole file scored 0%. `MutationApplier` now registers the mutated file's canonical (`File.realpath`) path in `$LOADED_FEATURES` after applying the mutation, so a later `require`/`require_relative` is a no-op. Surfaced by the EV-7764 pagy stability canary: `gem/lib/pagy/classes/request.rb` went from 0% to 82.8% under `--jobs 4` (EV-vxgl, PR #1256, GH #1253)
+- **Minitest integration reports `:error` instead of a misleading 0% when zero test methods run** — if the dispatched Minitest run executed no test methods (the resolved spec registered no `Minitest::Runnable` suite — commonly because the project's tests use a different framework such as the `test-unit` gem, or `--spec` points at the wrong file), `reporter.passed?` returned `true` for the empty run and every mutation was scored *survived*, producing a meaningless 0%. `run_minitest` now reports the dispatched test-method count; a zero-count run yields an error-shaped result so `classify_status` maps it to `:error` and the high-error-rate warning fires, instead of silently inflating the denominator. The Minitest analogue of the EV-720r RSpec fix. Surfaced by the EV-9cd2 kaminari stability canary (kaminari runs on `test-unit`, not Minitest) (EV-5dxk, PR #1257, GH #1254)
+
 ## [0.30.2] - 2026-05-15
 
 ### Fixed

data/lib/evilution/integration/loading/mutation_applier.rb

@@ -59,6 +59,21 @@ class Evilution::Integration::Loading::MutationApplier
     @redefinition_recovery.call(mutation.original_source) do
       @source_evaluator.call(eval_target, mutation.file_path)
     end
+    mark_feature_loaded(mutation.file_path)
+  end
+
+  # The mutated source is eval'd straight into the VM — `eval` does not register
+  # a `$LOADED_FEATURES` entry. If the spec then `require`s the same file (common
+  # when the project lazy-loads it and only the test references it), `require`
+  # reloads the ORIGINAL from disk and clobbers the mutation, so every mutation
+  # silently survives. Under fork isolation each worker starts from the same
+  # pre-`require` snapshot, so the whole file scores 0%. Registering the
+  # canonical path makes a later `require`/`require_relative` a no-op.
+  def mark_feature_loaded(file_path)
+    absolute = File.realpath(File.expand_path(file_path))
+    $LOADED_FEATURES << absolute unless $LOADED_FEATURES.include?(absolute)
+  rescue Errno::ENOENT
+    nil
   end
 
   def failure_result(error, message)

data/lib/evilution/integration/minitest.rb

@@ -122,16 +122,37 @@ class Evilution::Integration::Minitest < Evilution::Integration::Base
     return unresolved_result(mutation) if files.nil?
 
     command = "ruby -Itest #{files.join(" ")}"
+    execute_minitest(mutation, files, command)
+  rescue StandardError => e
+    { passed: false, error: e.message, test_command: command }
+  end
 
+  def execute_minitest(mutation, files, command)
     files.each { |f| load(File.expand_path(f)) }
 
-    args = build_args(mutation)
     detector = reset_crash_detector
-    passed = run_minitest(args, detector)
+    run = run_minitest(build_args(mutation), detector)
 
-    build_minitest_result(passed, command, detector)
-  rescue StandardError => e
-    { passed: false, error: e.message, test_command: command }
+    return no_tests_ran_result(command) if run[:count].zero?
+
+    build_minitest_result(run[:passed], command, detector)
+  end
+
+  # Zero dispatched test methods means the run carries no signal — the result
+  # is neither survived nor killed. Most common cause: the project's tests use
+  # a framework other than Minitest (e.g. the test-unit gem, whose
+  # Test::Unit::TestCase classes are not Minitest::Runnable), or --spec points
+  # at a file that registers no Minitest suite. Report :error so the score is
+  # not silently inflated to 0% with every mutation marked survived.
+  def no_tests_ran_result(command)
+    {
+      passed: false,
+      error: "no Minitest tests executed (0 test methods ran) — the resolved " \
+             "spec registered no Minitest suite. Check --integration/--spec; " \
+             "the project may use a non-Minitest framework (e.g. test-unit).",
+      error_class: "Evilution::Error",
+      test_command: command
+    }
   end
 
   def unresolved_result(mutation)
@@ -157,15 +178,47 @@ class Evilution::Integration::Minitest < Evilution::Integration::Base
     options[:io] = out
 
     reporter = ::Minitest::CompositeReporter.new
-    reporter << ::Minitest::SummaryReporter.new(out, options)
-    reporter << detector
 
     self.class.initialize_minitest_state(reporter, options)
+    summary = attach_evilution_reporters(reporter, detector, out, options)
     reporter.start
     dispatch_minitest_suites(reporter, options)
     reporter.report
 
-    reporter.passed?
+    { passed: summary.passed?, count: minitest_method_count }
+  end
+
+  # Attach evilution's own reporters to the composite AFTER plugin init, and
+  # read the run's verdict from the SummaryReporter rather than reporter.passed?.
+  #
+  # A target test helper that calls Minitest::Reporters.use! makes
+  # init_plugins replace the composite's reporters with minitest-reporters'
+  # DelegateReporter, which delegates to a process-global reporter created
+  # once by use!. That global reporter is never reset between runs, so under
+  # in_process isolation — where one process runs every mutation in sequence
+  # — its failures accumulate: reporter.passed? would report false for every
+  # mutation after the first genuine kill, false-killing real survivors and
+  # inflating the score. Anything attached before init_plugins is likewise
+  # evicted — that silently disabled the MinitestCrashDetector, so
+  # build_minitest_result's only_crashes? path went dead and crashes were
+  # downgraded from :error to :killed. Attaching both reporters here, after
+  # init_plugins has run, keeps them in the live composite for the current
+  # run. Returns the SummaryReporter.
+  def attach_evilution_reporters(reporter, detector, out, options)
+    summary = ::Minitest::SummaryReporter.new(out, options)
+    reporter << summary
+    reporter << detector
+    summary
+  end
+
+  # Count dispatched test methods from the runnable registry, not a reporter.
+  # A project test helper that calls Minitest::Reporters.use! swaps the
+  # composite's reporters during init_plugins, evicting evilution's
+  # SummaryReporter — a reporter-based count then reads 0 even on a real run.
+  # The runnable registry is immune to reporter plugins. Must run after
+  # initialize_minitest_state: runnable_methods calls srand(Minitest.seed).
+  def minitest_method_count
+    ::Minitest::Runnable.runnables.sum { |r| r.runnable_methods.size }
   end
 
   def dispatch_minitest_suites(reporter, options)

data/lib/evilution/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Evilution
-  VERSION = "0.30.2"
+  VERSION = "0.30.4"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: evilution
 version: !ruby/object:Gem::Version
-  version: 0.30.2
+  version: 0.30.4
 platform: ruby
 authors:
 - Denis Kiselev
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-05-15 00:00:00.000000000 Z
+date: 2026-05-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: diff-lcs