mutation_tester 1.4.0 → 1.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: be945d45b6c4219df627808a62336abe51ce94b088d79db47e25c794cdf227bd
-  data.tar.gz: 80aaefd644319415bfb25cebab5eb28077f3b99c763d234d20cf6885f3c001d2
+  metadata.gz: 8391c9817c1f430ec2231982495e11ff1f6866c3ce11b8e67d9ab421894c44ab
+  data.tar.gz: c82152c72bf0c0faaa22807ade06134fc814dd0faddde7f3849100b29b2f54fa
 SHA512:
-  metadata.gz: 2ad47074139c21bb7cd6db12eb53d2dd8c350b85c24c43f2ce44ba25590da1b3a4aa3f2e9f5ea2394abbdb8779e50cb9b3bb94418a94b03a1148ef2bd186b1f0
-  data.tar.gz: 58078c8af01ec3155be934d6917a3b66b0779f70bc20b04a23d05f82c06e9198000156fc13a7f7eece21cff11582d6b142a5ffb4aaa11d3191c15ebe4c3d9fbb
+  metadata.gz: d663fe8bc725c6b99e50d0752c3c8452397794672ac2a79a64e1cc6da70b1076abca47bf5143050cc3afd405320ff584d17c587cca8fff47bed27a3f184ed543
+  data.tar.gz: f33e5cbcdf08e032015b358e2b027b7846af163d2d5bab5954abf13f6d06d48e3a584677ace35e97f9aee67e48faa96a5c5ffcc2704b9416984e4605bcecdff1

data/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## [1.4.1] - 2026-08-03
+
+- Fixed parallel runs reporting every mutant as survived (`Killed: 0`, score `0.0%`) on projects whose test file reaches its source through `$LOAD_PATH` (a Rails app run with `-Itest` in `RUBYOPT`, for example). A preloaded worker is started once in the real project root, so Ruby absolutizes every `-I`/`RUBYLIB` entry against that directory; the forked child then changed into the shadow workspace but kept resolving `require "test_helper"` back to the original tree, which loaded the unmutated source. Each job now rewrites the `$LOAD_PATH` entries of the mirrored project root into the workspace, so a preloaded worker resolves project code exactly like the `spawn` runner that starts inside the workspace. Only Minitest suites hit this in practice: RSpec re-adds `lib` and its default path at run time, after the child has changed directory. Serial runs and the `spawn` runner were never affected, which is why the same file scored 96.55% serially and 0.0% with `-p 8`.
+- The shadow-workspace sanity check now also proves that the workspace copy of the source is the code the tests execute: after the unmutated source passes there, the check replaces that copy with a `raise` and requires the run to fail. When it still passes, the run aborts as an infrastructure failure (exit code `3`) naming the likely causes, instead of reporting a complete-looking 0.0% score in which every mutant falsely survived.
+- The console summary now warns when a file with at least five scored mutants killed none of them, pointing at the runner rather than at test quality and suggesting a `--runner spawn` comparison.
+
 ## [1.4.0] - 2026-08-01
 
 - Minitest suites now use the same execution runners as RSpec instead of being pinned to `spawn`. The fork worker preloads `minitest` (disabling the `minitest/autorun` at-exit hook and driving `Minitest.run` itself, so the file still runs exactly once per mutant) and the in-memory runner preloads the test file once and re-evaluates each mutant in a fresh fork, so a Minitest project no longer pays a full interpreter, Bundler and framework boot per mutant. Preloaded workers are now keyed by framework, so a mixed-framework `--glob` run never hands a Minitest file to an RSpec-preloaded worker.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    mutation_tester (1.4.0)
+    mutation_tester (1.4.1)
       parallel (~> 1.20)
       parser (~> 3.3)
       rainbow (~> 3.0)

data/docs/execution-runners.md

@@ -110,6 +110,15 @@ can cross the per-mutant deadline and be reported as a `timeout` instead of a
   `spawn`, even when `--runner fork` is requested.
 - If the helper process fails to preload the environment, the run warns once
   and falls back to `spawn`.
+- The helper process is started once in the real project root, so Ruby has
+  already absolutized every `-I` / `RUBYLIB` entry against that directory before
+  any mutant runs. Changing directory into a shadow workspace cannot undo that,
+  so each job additionally rewrites the `$LOAD_PATH` entries that point into the
+  mirrored project root so they point into the workspace. Without it a Minitest
+  file reaching its source through `require "test_helper"` would load the
+  original, unmutated tree and every mutant would falsely survive. RSpec re-adds
+  `lib` and its default path at run time, after the child has changed directory,
+  so it resolves the workspace copy either way.
 
 ### Limitations of the in-memory runner
 

data/lib/mutation_tester/core.rb

@@ -132,13 +132,25 @@ module MutationTester
       return true if mutation_runner.in_memory_first?
 
       puts Rainbow("\n🩺 Verifying the shadow workspace with the unmutated source...").yellow
-      if mutation_runner.shadow_baseline_passes?
-        puts Rainbow('✓ Shadow workspace verified with the unmutated source').green
-        return true
+      case mutation_runner.shadow_workspace_check
+      when :ok
+        puts Rainbow('✓ Shadow workspace verified: the unmutated source passes and the workspace copy is what the tests execute').green
+        true
+      when :canary
+        report_unreachable_workspace_source
+      else
+        puts Rainbow('❌ The unmutated source fails inside the shadow workspace; the shadow environment is unreliable.').red
+        puts Rainbow('   Every mutant would falsely die there, so the run is aborted instead of reporting a misleading score.').red
+        false
       end
+    end
 
-      puts Rainbow('❌ The unmutated source fails inside the shadow workspace; the shadow environment is unreliable.').red
-      puts Rainbow('   Every mutant would falsely die there, so the run is aborted instead of reporting a misleading score.').red
+    def report_unreachable_workspace_source
+      puts Rainbow('❌ The tests still pass with the workspace copy of the source replaced by a raise.').red
+      puts Rainbow('   The mutated file is therefore not the code the tests execute, so every mutant would falsely').red
+      puts Rainbow('   survive; the run is aborted instead of reporting a misleading 0.0% score.').red
+      puts Rainbow('   Usual causes: the tests resolve this source outside the workspace (an absolute entry in').red
+      puts Rainbow('   $LOAD_PATH, a symlinked .rb file, a preloaded copy of the class), or they never load it at all.').red
       false
     end
 

data/lib/mutation_tester/fork_runner/worker.rb

@@ -26,6 +26,26 @@ end
 
 monotonic = lambda { Process.clock_gettime(Process::CLOCK_MONOTONIC) }
 
+resolved_path = lambda do |path|
+  File.realpath(path)
+rescue SystemCallError
+  path
+end
+
+mirror_load_path = lambda do |from, to|
+  next if from.nil? || to.nil? || from == to
+
+  roots = [from, resolved_path.call(from)].uniq.map { |root| root.chomp('/') }
+  $LOAD_PATH.map! do |entry|
+    path = File.expand_path(entry.to_s, from)
+    root = roots.find { |candidate| path == candidate || path.start_with?("#{candidate}/") }
+    next entry unless root
+
+    suffix = path.delete_prefix(root).delete_prefix('/')
+    suffix.empty? ? to : File.join(to, suffix)
+  end
+end
+
 current_child = nil
 preloaded = nil
 
@@ -50,6 +70,7 @@ supervise_child = lambda do |job, out, child_body|
     rescue Errno::EACCES, Errno::EPERM
     end
     Dir.chdir(job['chdir']) if job['chdir']
+    mirror_load_path.call(job['mirror_of'], job['chdir'])
     sink = File.open(job['log'] || File::NULL, 'w')
     sink.sync = true
     STDOUT.reopen(sink)

data/lib/mutation_tester/fork_runner.rb

@@ -153,7 +153,8 @@ module MutationTester
       @ready
     end
 
-    def execute(spec_file, timeout: nil, chdir: nil, capture: false, args: [], stop_on_first_failure: false)
+    def execute(spec_file, timeout: nil, chdir: nil, capture: false, args: [], stop_on_first_failure: false,
+                mirror_of: nil)
       log = capture ? Tempfile.new(['mutation_tester_fork', '.log']) : nil
       job = {
         spec: spec_file,
@@ -161,7 +162,8 @@ module MutationTester
         chdir: chdir,
         log: log&.path,
         args: args,
-        stop_on_first_failure: stop_on_first_failure
+        stop_on_first_failure: stop_on_first_failure,
+        mirror_of: mirror_of
       }
       @job_writer.puts(JSON.generate(job))
       status = await_result(timeout)['status']

data/lib/mutation_tester/mutation_runner.rb

@@ -4,6 +4,7 @@ require 'securerandom'
 
 module MutationTester
   class MutationRunner
+    CANARY_SOURCE = "raise 'mutation_tester canary: the workspace copy of this source was not executed'\n".freeze
     PARALLEL_INTERRUPT_LINE = "Parallel execution interrupted, exiting ...\n"
 
     class ParallelInterruptFilter
@@ -216,7 +217,7 @@ module MutationTester
         File.write(shadow_source, mutation[:code])
 
         outcome, phase = run_two_phase(mutation) do |example_filters|
-          run_specs_in_shadow(shadow_spec, shadow_root, example_filters: example_filters)
+          run_specs_in_shadow(shadow_spec, shadow_root, project_root, example_filters: example_filters)
         end
         apply_outcome(result, outcome, phase)
       ensure
@@ -240,6 +241,10 @@ module MutationTester
     end
 
     def shadow_baseline_passes?
+      shadow_workspace_check == :ok
+    end
+
+    def shadow_workspace_check
       project_root = find_project_root
 
       Dir.mktmpdir do |temp_dir|
@@ -254,12 +259,16 @@ module MutationTester
 
         File.unlink(shadow_source)
         File.write(shadow_source, @original_content)
+        return :baseline unless run_specs_in_shadow(shadow_spec, shadow_root, project_root).passed?
 
-        run_specs_in_shadow(shadow_spec, shadow_root).passed?
+        File.write(shadow_source, CANARY_SOURCE)
+        return :canary if run_specs_in_shadow(shadow_spec, shadow_root, project_root).passed?
+
+        :ok
       end
     rescue => e
       warn("[MutationTester] Shadow sanity check could not prepare the shadow workspace: #{e.message}")
-      false
+      :baseline
     end
 
     def shadow_copy_project(source, dest)
@@ -284,8 +293,9 @@ module MutationTester
       end
     end
 
-    def run_specs_in_shadow(spec_file, working_dir, example_filters: [])
-      test_command(spec_file, example_filters: example_filters).run(timeout: @config.effective_timeout, chdir: working_dir)
+    def run_specs_in_shadow(spec_file, working_dir, project_root, example_filters: [])
+      test_command(spec_file, example_filters: example_filters)
+        .run(timeout: @config.effective_timeout, chdir: working_dir, mirror_of: project_root)
     end
 
     def discoverable_project_root

data/lib/mutation_tester/reporters/console_reporter.rb

@@ -1,12 +1,15 @@
 module MutationTester
   module Reporters
     class ConsoleReporter < BaseReporter
+      NOTHING_KILLED_MIN_MUTANTS = 5
+
       def generate
         puts "\n" + Rainbow('=' * 80).bright
         puts Rainbow('🧬 MUTATION TESTING REPORT').bright.cyan
         puts Rainbow('=' * 80).bright
 
         print_summary
+        print_nothing_killed_warning
         print_survived_mutations if survived_count > 0
 
         puts "\n" + Rainbow('=' * 80).bright
@@ -30,6 +33,23 @@ module MutationTester
         puts "\n  " + progress_bar
       end
 
+      def print_nothing_killed_warning
+        return unless nothing_killed?
+
+        puts "\n#{Rainbow("⚠️  Not one of the #{scored_mutant_count} scored mutants was killed.").yellow}"
+        puts Rainbow('   A whole file that kills nothing is more often a runner problem (the mutated code never').yellow
+        puts Rainbow('   reached the tests) than a test-quality gap. Re-run with --runner spawn and compare before').yellow
+        puts Rainbow('   acting on this score.').yellow
+      end
+
+      def nothing_killed?
+        killed_count.zero? && timeout_count.zero? && scored_mutant_count >= NOTHING_KILLED_MIN_MUTANTS
+      end
+
+      def scored_mutant_count
+        killed_count + timeout_count + survived_count
+      end
+
       def print_timeout_deadline
         return unless timeout_count.positive?
 

data/lib/mutation_tester/test_command.rb

@@ -40,7 +40,7 @@ module MutationTester
       cmd
     end
 
-    def run(timeout: nil, chdir: nil, capture: false)
+    def run(timeout: nil, chdir: nil, capture: false, mirror_of: nil)
       if fork_execution?
         fork_runner = ForkRunner.acquire(use_bundle_exec: @use_bundle_exec, framework: @framework)
         if fork_runner
@@ -50,7 +50,8 @@ module MutationTester
             chdir: chdir || Dir.pwd,
             capture: capture,
             args: filter_args,
-            stop_on_first_failure: @stop_on_first_failure
+            stop_on_first_failure: @stop_on_first_failure,
+            mirror_of: mirror_of
           )
         end
       end

data/lib/mutation_tester/version.rb

@@ -1,3 +1,3 @@
 module MutationTester
-  VERSION = '1.4.0'.freeze
+  VERSION = '1.4.1'.freeze
 end

data/readme.md

@@ -232,7 +232,9 @@ bundle exec mutation_test --reporters json,html --output-dir build/mutation \
   file, unknown reporter, source with a syntax error).
 - `2` - a usage error (conflicting flags; see the batch sections below).
 - `3` - the run aborted or degraded before reaching a verdict: the shadow workspace
-  was unreliable, or every mutant ended as `error`/`stillborn` so nothing was scored.
+  was unreliable (the unmutated source failed there, or the workspace copy of the
+  source turned out not to be the code the tests execute), or every mutant ended as
+  `error`/`stillborn` so nothing was scored.
   This signals an infrastructure or runner problem, not a test-quality gap, so CI
   hooks can distinguish it from a genuine threshold failure.
 - `130` - interrupted with Ctrl+C.
@@ -523,7 +525,16 @@ invalid value (less than 1, or non-numeric) falls back to 1 with a warning on st
 In parallel mode each mutant runs in an isolated shadow workspace. Every `.rb`
 file is a physical copy (non-Ruby files stay symlinks for speed), so mutations
 apply correctly even when a spec loads the source indirectly (e.g. via
-`spec_helper`). The parallel mutation score therefore matches serial.
+`spec_helper`), and `$LOAD_PATH` entries pointing into the project resolve
+inside the workspace, so a test file that reaches its source through
+`require "test_helper"` gets the mutated copy too. The parallel mutation score
+therefore matches serial.
+
+Before the first mutant, the run proves this in the workspace itself: the
+unmutated source must pass there, and the same suite must fail once that copy of
+the source is replaced by a `raise`. A run whose tests pass even then is aborted
+as an infrastructure failure (exit code `3`) rather than reported as a 0.0%
+score, because the mutated file is demonstrably not the code being executed.
 
 ### When to use serial vs parallel execution
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: mutation_tester
 version: !ruby/object:Gem::Version
-  version: 1.4.0
+  version: 1.4.1
 platform: ruby
 authors:
 - Kamil Dzierbicki
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-08-01 00:00:00.000000000 Z
+date: 2026-08-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: parallel