mutation_tester 1.3.0 → 1.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 03a2b2364ebede84e6b56cfe4fba7d0450475262d59aa754abe0ce34ee1a83ba
-  data.tar.gz: 7439181f228ffbf2d1cf2e603ac21a91a46c764cd66f0ebfcc40dab30a2f2188
+  metadata.gz: 8391c9817c1f430ec2231982495e11ff1f6866c3ce11b8e67d9ab421894c44ab
+  data.tar.gz: c82152c72bf0c0faaa22807ade06134fc814dd0faddde7f3849100b29b2f54fa
 SHA512:
-  metadata.gz: 6be68e2928ca7ecc1a382e01c66cb5ab7d0afa35e5cf5b6008a62d0599fe399af17a8960a3f4e62025d40e640850c4fde423cff5aa2de69d34db2f6139661342
-  data.tar.gz: fb12cf8fd83bc37529e7e0fcdc8bb72bc902df0c846b05ad8fb856d8db8b382cf18fb2058b42f5efa2b161f573517b368d7aee92563c4115c7b3669a3d2e5ba9
+  metadata.gz: d663fe8bc725c6b99e50d0752c3c8452397794672ac2a79a64e1cc6da70b1076abca47bf5143050cc3afd405320ff584d17c587cca8fff47bed27a3f184ed543
+  data.tar.gz: f33e5cbcdf08e032015b358e2b027b7846af163d2d5bab5954abf13f6d06d48e3a584677ace35e97f9aee67e48faa96a5c5ffcc2704b9416984e4605bcecdff1

data/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## [1.4.1] - 2026-08-03
+
+- Fixed parallel runs reporting every mutant as survived (`Killed: 0`, score `0.0%`) on projects whose test file reaches its source through `$LOAD_PATH` (a Rails app run with `-Itest` in `RUBYOPT`, for example). A preloaded worker is started once in the real project root, so Ruby absolutizes every `-I`/`RUBYLIB` entry against that directory; the forked child then changed into the shadow workspace but kept resolving `require "test_helper"` back to the original tree, which loaded the unmutated source. Each job now rewrites the `$LOAD_PATH` entries of the mirrored project root into the workspace, so a preloaded worker resolves project code exactly like the `spawn` runner that starts inside the workspace. Only Minitest suites hit this in practice: RSpec re-adds `lib` and its default path at run time, after the child has changed directory. Serial runs and the `spawn` runner were never affected, which is why the same file scored 96.55% serially and 0.0% with `-p 8`.
+- The shadow-workspace sanity check now also proves that the workspace copy of the source is the code the tests execute: after the unmutated source passes there, the check replaces that copy with a `raise` and requires the run to fail. When it still passes, the run aborts as an infrastructure failure (exit code `3`) naming the likely causes, instead of reporting a complete-looking 0.0% score in which every mutant falsely survived.
+- The console summary now warns when a file with at least five scored mutants killed none of them, pointing at the runner rather than at test quality and suggesting a `--runner spawn` comparison.
+
+## [1.4.0] - 2026-08-01
+
+- Minitest suites now use the same execution runners as RSpec instead of being pinned to `spawn`. The fork worker preloads `minitest` (disabling the `minitest/autorun` at-exit hook and driving `Minitest.run` itself, so the file still runs exactly once per mutant) and the in-memory runner preloads the test file once and re-evaluates each mutant in a fresh fork, so a Minitest project no longer pays a full interpreter, Bundler and framework boot per mutant. Preloaded workers are now keyed by framework, so a mixed-framework `--glob` run never hands a Minitest file to an RSpec-preloaded worker.
+- Every mutant run now stops at its first failing test: RSpec runs get `--fail-fast` and Minitest runs get a preloaded reporter that aborts on the first non-passing result, on all three runners. This cannot change a verdict (a run that stops early has already failed, which is what makes a mutant killed), and only mutant runs opt in: the baseline run and the shadow sanity check still run the whole file. It removes the pathology where a mutant that breaks something every test touches (a class body that no longer loads, a constant every test reads) re-raised the same error once per test, crossed the calibrated deadline, and was reported as a `timeout` instead of a `killed` - a failure mode that got worse as tests were added to the file. Measured on a 15-mutant fixture with a 0.6 s boot and 12 tests: 22.2 s -> 6.5 s by default and 22.7 s -> 13.2 s with `--runner spawn`, with an unchanged score.
+- The console summary now names the deadline that timed-out mutants were measured against and where it came from (`deadline: 6.50s (5x baseline 1.30s)`, or `(explicitly configured)`), so a genuine hang and a deadline calibrated from a slow test file are no longer indistinguishable.
+- `--fail-fast` now stops a batch at the first file with a surviving mutant even when that file's own run completed. `Core#stopped_on_survivor?` reports the fail-fast stop, while `Core#interrupted?` keeps its narrower meaning (mutants were left unprocessed) for the reports and the interruption banner.
+
 ## [1.3.0] - 2026-07-15
 
 - Calibrated the per-mutant timeout against the measured baseline run: unless `config.timeout` is set explicitly, each mutant now gets `max(5s, timeout_factor * baseline duration)` (factor configurable via `config.timeout_factor` / `--timeout-factor N`, default 5) instead of a fixed 30 s, so a loaded machine no longer inflates the mutation score by killing healthy-but-slow runs as timeouts. An explicit `config.timeout` (including `nil` for no deadline) keeps today's fixed-budget behavior and disables calibration. Added the opt-in `config.timeout_policy = :separate` / `--timeout-policy separate`, which scores `killed / (killed + survived)` with timeouts excluded from the score and reported only as their own category; the default `:killed` policy and its output are unchanged.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    mutation_tester (1.3.0)
+    mutation_tester (1.4.1)
       parallel (~> 1.20)
       parser (~> 3.3)
       rainbow (~> 3.0)

data/docs/execution-runners.md

@@ -10,14 +10,17 @@ force a runner.
 Every mutant is executed by one of three runners:
 
 - **fork**: a helper process preloads the environment
-  once (RubyGems, Bundler and `rspec-core`, without loading the mutated file or
-  the specs), and each mutant runs in a fresh fork of that process. The fork
-  loads the spec only after the mutated source has been written, so every
-  mutant is visible and no state leaks between mutants. This removes most of
+  once (RubyGems, Bundler and the test framework: `rspec-core` for RSpec,
+  `minitest` for Minitest, without loading the mutated file or the tests), and
+  each mutant runs in a fresh fork of that process. The fork loads the test file
+  only after the mutated source has been written, so every mutant is visible and
+  no state leaks between mutants. For Minitest the worker disables the
+  `minitest/autorun` at-exit hook and drives `Minitest.run` itself, so the file
+  runs exactly once per mutant. This removes most of
   the fixed per-mutant boot cost, which matters on large suites and in CI.
-- **spawn**: each mutant starts a full new process (`bundle exec rspec ...`).
-  Slower per mutant, but works everywhere.
-- **in_memory** (default where supported): the helper process additionally preloads the spec
+- **spawn**: each mutant starts a full new process (`bundle exec rspec ...` or
+  `bundle exec ruby test_file.rb`). Slower per mutant, but works everywhere.
+- **in_memory** (default where supported): the helper process additionally preloads the test
   file and, through it, the original source, once per run. Each mutant then
   runs in a fresh fork that re-evaluates the mutated source in memory
   (redefining the loaded methods and class constants, with the
@@ -34,15 +37,16 @@ Selection is automatic (`auto`): the fastest safe path is tried first and every
 step down to a slower one prints a single stderr warning with its reason, so a
 fallback is never silent. The order is `in_memory` (RSpec with `Process.fork`
 available and a passing unmutated-source probe), then `fork`, then `spawn`.
+Both RSpec and Minitest suites use the same three runners.
 All runners produce identical scores and per-mutant statuses, and all enforce
 the same hard per-mutant timeout (monotonic deadline plus a process-group
 kill).
 
 | Mode        | Picked by `auto` when                                                                                                                                                   | Falls back to                                                                                                                              |
 |-------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|
-| `in_memory` | the suite is RSpec, the platform has `Process.fork`, the file has no load-time `defined?` guard, and re-applying the unmutated source in a probe child passes the suite | `fork`/`spawn` (whole run) with a stderr warning naming the reason; a single worker dying mid-run falls back only for its share of mutants; a mutant that raises while being applied falls back alone |
-| `fork`      | the suite is RSpec, `Process.fork` is available, but in-memory is unavailable (each reason is printed)                                                                  | `spawn`, with a stderr warning, when the helper process fails to preload the environment                                                   |
-| `spawn`     | the suite is Minitest, or the platform has no `Process.fork`                                                                                                            | nothing; it works everywhere                                                                                                               |
+| `in_memory` | the platform has `Process.fork`, the file has no load-time `defined?` guard, and re-applying the unmutated source in a probe child passes the suite | `fork`/`spawn` (whole run) with a stderr warning naming the reason; a single worker dying mid-run falls back only for its share of mutants; a mutant that raises while being applied falls back alone |
+| `fork`      | `Process.fork` is available, but in-memory is unavailable (each reason is printed)                                                                                      | `spawn`, with a stderr warning, when the helper process fails to preload the environment                                                   |
+| `spawn`     | the platform has no `Process.fork`                                                                                                                                      | nothing; it works everywhere                                                                                                               |
 
 Forcing a mode with `--runner fork|spawn|in_memory` skips the auto attempts and
 uses that mode directly (`in_memory` keeps its own documented safety fallbacks;
@@ -77,14 +81,44 @@ MutationTester.configure do |config|
 end
 ```
 
+### Stopping a mutant at its first failing test
+
+Every mutant run stops as soon as one test fails, on all three runners:
+
+- RSpec mutant runs are given `--fail-fast` (as a CLI argument on `spawn`, in the
+  runner arguments on `fork`, and in the preloaded configuration on `in_memory`).
+- Minitest mutant runs load `lib/mutation_tester/minitest_fail_fast.rb`, which
+  registers a Minitest plugin whose reporter raises `Interrupt` on the first
+  non-passing result. On `spawn` the file is preloaded with `ruby -r`, on the
+  preloaded runners the worker enables the same reporter per job.
+
+This cannot change a verdict. A run that stops early has already recorded a
+failure, which is exactly what makes a mutant killed, and a run without a failure
+is untouched and executes every test. Only the mutant runs opt in: the baseline
+run and the shadow-workspace sanity check are expected to pass and always run the
+whole file, so a failing baseline still reports every failure it finds.
+
+The pathological case it removes is a mutant that breaks something every test
+touches (a class body that no longer loads, a constant every test reads). Such a
+mutant used to re-raise the same error once per test, which on a large test file
+can cross the per-mutant deadline and be reported as a `timeout` instead of a
+`killed`, and which gets worse as tests are added to the file.
+
 ### Limitations of the fork runner
 
-- Minitest suites always use `spawn` (fork support for Minitest is a separate
-  decision after RSpec experience is collected).
 - Platforms without `Process.fork` (for example Windows or JRuby) always use
   `spawn`, even when `--runner fork` is requested.
 - If the helper process fails to preload the environment, the run warns once
   and falls back to `spawn`.
+- The helper process is started once in the real project root, so Ruby has
+  already absolutized every `-I` / `RUBYLIB` entry against that directory before
+  any mutant runs. Changing directory into a shadow workspace cannot undo that,
+  so each job additionally rewrites the `$LOAD_PATH` entries that point into the
+  mirrored project root so they point into the workspace. Without it a Minitest
+  file reaching its source through `require "test_helper"` would load the
+  original, unmutated tree and every mutant would falsely survive. RSpec re-adds
+  `lib` and its default path at run time, after the child has changed directory,
+  so it resolves the workspace copy either way.
 
 ### Limitations of the in-memory runner
 
@@ -92,8 +126,8 @@ The in-memory runner never fails silently: each case below falls back to
 file-based execution with a warning, and a mutant is marked `error` only when
 no fallback is possible.
 
-- RSpec only, and the file must be classic loadable code (classes/modules).
-  Minitest suites fall back to the file-based path with a warning.
+- The file must be classic loadable code (classes/modules) that survives being
+  evaluated a second time.
 - With `-p N` (N > 1) the run stays fully in memory: the environment, the
   original source and the specs are preloaded once, the preloaded process is
   forked into N pooled clones, and every parallel worker applies each mutant

data/lib/mutation_tester/batch_runner.rb

@@ -199,7 +199,7 @@ module MutationTester
         interrupted: core.interrupted?,
         degraded: core.infrastructure_failure?
       )
-      [entry, core.interrupted?]
+      [entry, core.stopped_on_survivor?]
     end
 
     def spec_path_for(source_file)

data/lib/mutation_tester/core.rb

@@ -44,10 +44,12 @@ module MutationTester
       ForkRunner.shutdown_all
     end
 
+    def stopped_on_survivor?
+      @config.fail_fast && @results.any? { |result| result[:status] == :survived }
+    end
+
     def interrupted?
-      @config.fail_fast &&
-        @results.size < @mutations.size &&
-        @results.any? { |result| result[:status] == :survived }
+      stopped_on_survivor? && @results.size < @mutations.size
     end
 
     def mutation_score
@@ -130,13 +132,25 @@ module MutationTester
       return true if mutation_runner.in_memory_first?
 
       puts Rainbow("\n🩺 Verifying the shadow workspace with the unmutated source...").yellow
-      if mutation_runner.shadow_baseline_passes?
-        puts Rainbow('✓ Shadow workspace verified with the unmutated source').green
-        return true
+      case mutation_runner.shadow_workspace_check
+      when :ok
+        puts Rainbow('✓ Shadow workspace verified: the unmutated source passes and the workspace copy is what the tests execute').green
+        true
+      when :canary
+        report_unreachable_workspace_source
+      else
+        puts Rainbow('❌ The unmutated source fails inside the shadow workspace; the shadow environment is unreliable.').red
+        puts Rainbow('   Every mutant would falsely die there, so the run is aborted instead of reporting a misleading score.').red
+        false
       end
+    end
 
-      puts Rainbow('❌ The unmutated source fails inside the shadow workspace; the shadow environment is unreliable.').red
-      puts Rainbow('   Every mutant would falsely die there, so the run is aborted instead of reporting a misleading score.').red
+    def report_unreachable_workspace_source
+      puts Rainbow('❌ The tests still pass with the workspace copy of the source replaced by a raise.').red
+      puts Rainbow('   The mutated file is therefore not the code the tests execute, so every mutant would falsely').red
+      puts Rainbow('   survive; the run is aborted instead of reporting a misleading 0.0% score.').red
+      puts Rainbow('   Usual causes: the tests resolve this source outside the workspace (an absolute entry in').red
+      puts Rainbow('   $LOAD_PATH, a symlinked .rb file, a preloaded copy of the class), or they never load it at all.').red
       false
     end
 

data/lib/mutation_tester/fork_runner/worker.rb

@@ -1,13 +1,22 @@
 # frozen_string_literal: true
 
 require 'json'
-require 'rspec/core'
 require_relative '../in_memory_loader'
 
 control = $stdout.dup
 control.sync = true
 STDOUT.reopen(File::NULL)
 
+framework = ARGV.shift == 'minitest' ? :minitest : :rspec
+
+if framework == :minitest
+  require 'minitest'
+  require_relative '../minitest_fail_fast'
+  Minitest.class_variable_set(:@@installed_at_exit, true)
+else
+  require 'rspec/core'
+end
+
 kill_group = lambda do |pid|
   begin
     Process.kill('KILL', -pid)
@@ -17,6 +26,26 @@ end
 
 monotonic = lambda { Process.clock_gettime(Process::CLOCK_MONOTONIC) }
 
+resolved_path = lambda do |path|
+  File.realpath(path)
+rescue SystemCallError
+  path
+end
+
+mirror_load_path = lambda do |from, to|
+  next if from.nil? || to.nil? || from == to
+
+  roots = [from, resolved_path.call(from)].uniq.map { |root| root.chomp('/') }
+  $LOAD_PATH.map! do |entry|
+    path = File.expand_path(entry.to_s, from)
+    root = roots.find { |candidate| path == candidate || path.start_with?("#{candidate}/") }
+    next entry unless root
+
+    suffix = path.delete_prefix(root).delete_prefix('/')
+    suffix.empty? ? to : File.join(to, suffix)
+  end
+end
+
 current_child = nil
 preloaded = nil
 
@@ -41,6 +70,7 @@ supervise_child = lambda do |job, out, child_body|
     rescue Errno::EACCES, Errno::EPERM
     end
     Dir.chdir(job['chdir']) if job['chdir']
+    mirror_load_path.call(job['mirror_of'], job['chdir'])
     sink = File.open(job['log'] || File::NULL, 'w')
     sink.sync = true
     STDOUT.reopen(sink)
@@ -91,10 +121,25 @@ supervise_child = lambda do |job, out, child_body|
   [timed_out, payload, reaped]
 end
 
+load_test_file = lambda do |path|
+  $PROGRAM_NAME = path
+  load(path)
+end
+
+run_test_file = lambda do |job|
+  if framework == :minitest
+    MutationTester::MinitestFailFast.enabled = job['stop_on_first_failure'] ? true : false
+    load_test_file.call(job['spec'])
+    Minitest.run(Array(job['args'])) ? 0 : 1
+  else
+    args = [job['spec'], *Array(job['args'])]
+    args << '--fail-fast' if job['stop_on_first_failure']
+    RSpec::Core::Runner.run(args, STDERR, STDOUT).to_i
+  end
+end
+
 run_job = lambda do |job, out|
-  timed_out, payload, reaped = supervise_child.call(job, out, lambda do
-    RSpec::Core::Runner.run([job['spec'], *Array(job['args'])], STDERR, STDOUT).to_i
-  end)
+  timed_out, payload, reaped = supervise_child.call(job, out, lambda { run_test_file.call(job) })
 
   status =
     if timed_out
@@ -111,16 +156,32 @@ end
 preload_specs = lambda do |request, out|
   begin
     Dir.chdir(request['chdir']) if request['chdir']
-    sink = File.open(File::NULL, 'w')
-    runner = RSpec::Core::Runner.new(RSpec::Core::ConfigurationOptions.new([request['spec']]))
-    runner.setup(sink, sink)
-    preloaded = runner
+    if framework == :minitest
+      MutationTester::MinitestFailFast.enabled = request['stop_on_first_failure'] ? true : false
+      load_test_file.call(request['spec'])
+      preloaded = true
+    else
+      sink = File.open(File::NULL, 'w')
+      options = [request['spec']]
+      options << '--fail-fast' if request['stop_on_first_failure']
+      runner = RSpec::Core::Runner.new(RSpec::Core::ConfigurationOptions.new(options))
+      runner.setup(sink, sink)
+      preloaded = runner
+    end
     out.puts(JSON.generate('event' => 'preloaded', 'status' => 'ok'))
   rescue ScriptError, StandardError => e
     out.puts(JSON.generate('event' => 'preloaded', 'status' => 'error', 'message' => "#{e.class}: #{e.message}"))
   end
 end
 
+run_preloaded_suite = lambda do
+  if framework == :minitest
+    Minitest.run([]) ? 0 : 1
+  else
+    preloaded.run_specs(RSpec.world.ordered_example_groups).to_i
+  end
+end
+
 run_in_memory_job = lambda do |job, out|
   request = job['in_memory']
 
@@ -132,7 +193,7 @@ run_in_memory_job = lambda do |job, out|
   timed_out, payload, reaped = supervise_child.call(job, out, lambda do
     begin
       MutationTester::InMemoryLoader.apply(request['source'], request['path'])
-      JSON.generate('code' => preloaded.run_specs(RSpec.world.ordered_example_groups).to_i)
+      JSON.generate('code' => run_preloaded_suite.call)
     rescue ScriptError, StandardError => e
       JSON.generate('error' => "#{e.class}: #{e.message}")
     end

data/lib/mutation_tester/fork_runner.rb

@@ -21,22 +21,22 @@ module MutationTester
         Process.respond_to?(:fork)
       end
 
-      def acquire(use_bundle_exec:)
+      def acquire(use_bundle_exec:, framework: :rspec)
         return nil unless available?
 
-        key = [Process.pid, use_bundle_exec]
+        key = [Process.pid, use_bundle_exec, framework]
         return registry[key] if registry.key?(key)
 
-        registry[key] = checkout_pooled(use_bundle_exec) || boot(use_bundle_exec)
+        registry[key] = checkout_pooled(pool_key(use_bundle_exec, framework)) || boot(use_bundle_exec, framework)
       end
 
-      def prepare_pool(count, use_bundle_exec:, env_for: nil)
+      def prepare_pool(count, use_bundle_exec:, framework: :rspec, env_for: nil)
         return unless available?
 
-        primary = acquire(use_bundle_exec: use_bundle_exec)
+        primary = acquire(use_bundle_exec: use_bundle_exec, framework: framework)
         return unless primary
 
-        refill_pool(use_bundle_exec, count, primary, env_for: env_for)
+        refill_pool(pool_key(use_bundle_exec, framework), count, primary, env_for: env_for)
       end
 
       def prepare_in_memory_pool(count, primary)
@@ -100,11 +100,15 @@ module MutationTester
 
       private
 
-      def checkout_pooled(use_bundle_exec)
+      def pool_key(use_bundle_exec, framework)
+        [use_bundle_exec, framework]
+      end
+
+      def checkout_pooled(key)
         number = parallel_worker_number
         return nil unless number
 
-        entry = pool[use_bundle_exec]
+        entry = pool[key]
         entry && entry[:runners][number]
       end
 
@@ -123,8 +127,8 @@ module MutationTester
         entry[:runners]
       end
 
-      def boot(use_bundle_exec)
-        runner = new(use_bundle_exec: use_bundle_exec)
+      def boot(use_bundle_exec, framework)
+        runner = new(use_bundle_exec: use_bundle_exec, framework: framework)
         return runner if runner.ready?
 
         runner.shutdown
@@ -133,8 +137,8 @@ module MutationTester
       end
     end
 
-    def initialize(use_bundle_exec:)
-      argv = ['ruby', WORKER_PATH]
+    def initialize(use_bundle_exec:, framework: :rspec)
+      argv = ['ruby', WORKER_PATH, framework.to_s]
       argv = ['bundle', 'exec', *argv] if use_bundle_exec
 
       job_reader, job_writer = IO.pipe
@@ -149,9 +153,18 @@ module MutationTester
       @ready
     end
 
-    def execute(spec_file, timeout: nil, chdir: nil, capture: false, args: [])
+    def execute(spec_file, timeout: nil, chdir: nil, capture: false, args: [], stop_on_first_failure: false,
+                mirror_of: nil)
       log = capture ? Tempfile.new(['mutation_tester_fork', '.log']) : nil
-      job = { spec: spec_file, timeout: timeout, chdir: chdir, log: log&.path, args: args }
+      job = {
+        spec: spec_file,
+        timeout: timeout,
+        chdir: chdir,
+        log: log&.path,
+        args: args,
+        stop_on_first_failure: stop_on_first_failure,
+        mirror_of: mirror_of
+      }
       @job_writer.puts(JSON.generate(job))
       status = await_result(timeout)['status']
       result = TestCommand::Result.new(status == 'pass', status == 'timeout')
@@ -164,8 +177,9 @@ module MutationTester
       log&.unlink
     end
 
-    def preload(spec_file, chdir: nil)
-      @job_writer.puts(JSON.generate(preload: { spec: spec_file, chdir: chdir }))
+    def preload(spec_file, chdir: nil, stop_on_first_failure: false)
+      request = { spec: spec_file, chdir: chdir, stop_on_first_failure: stop_on_first_failure }
+      @job_writer.puts(JSON.generate(preload: request))
       event = read_event(monotonic_time + BOOT_TIMEOUT)
       return [true, nil] if event.is_a?(Hash) && event['event'] == 'preloaded' && event['status'] == 'ok'
 

data/lib/mutation_tester/minitest_fail_fast.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module MutationTester
+  module MinitestFailFast
+    PLUGIN_NAME = :mutation_tester_fail_fast
+
+    class << self
+      attr_accessor :enabled
+    end
+
+    self.enabled = true
+  end
+end
+
+begin
+  require 'minitest'
+rescue LoadError
+  Kernel.warn '[MutationTester] minitest is not loadable here; mutant runs will execute every test instead of stopping at the first failure.'
+else
+  module MutationTester
+    module MinitestFailFast
+      class Reporter < Minitest::AbstractReporter
+        def record(result)
+          return unless MinitestFailFast.enabled
+          return if result.passed? || result.skipped?
+
+          raise Interrupt
+        end
+      end
+    end
+  end
+
+  module Minitest
+    def self.plugin_mutation_tester_fail_fast_init(_options)
+      reporter << MutationTester::MinitestFailFast::Reporter.new
+    end
+  end
+
+  unless Minitest.extensions.include?(MutationTester::MinitestFailFast::PLUGIN_NAME)
+    Minitest.register_plugin(MutationTester::MinitestFailFast::PLUGIN_NAME)
+  end
+end

data/lib/mutation_tester/mutation_runner.rb

@@ -4,6 +4,7 @@ require 'securerandom'
 
 module MutationTester
   class MutationRunner
+    CANARY_SOURCE = "raise 'mutation_tester canary: the workspace copy of this source was not executed'\n".freeze
     PARALLEL_INTERRUPT_LINE = "Parallel execution interrupted, exiting ...\n"
 
     class ParallelInterruptFilter
@@ -216,7 +217,7 @@ module MutationTester
         File.write(shadow_source, mutation[:code])
 
         outcome, phase = run_two_phase(mutation) do |example_filters|
-          run_specs_in_shadow(shadow_spec, shadow_root, example_filters: example_filters)
+          run_specs_in_shadow(shadow_spec, shadow_root, project_root, example_filters: example_filters)
         end
         apply_outcome(result, outcome, phase)
       ensure
@@ -240,6 +241,10 @@ module MutationTester
     end
 
     def shadow_baseline_passes?
+      shadow_workspace_check == :ok
+    end
+
+    def shadow_workspace_check
       project_root = find_project_root
 
       Dir.mktmpdir do |temp_dir|
@@ -254,12 +259,16 @@ module MutationTester
 
         File.unlink(shadow_source)
         File.write(shadow_source, @original_content)
+        return :baseline unless run_specs_in_shadow(shadow_spec, shadow_root, project_root).passed?
 
-        run_specs_in_shadow(shadow_spec, shadow_root).passed?
+        File.write(shadow_source, CANARY_SOURCE)
+        return :canary if run_specs_in_shadow(shadow_spec, shadow_root, project_root).passed?
+
+        :ok
       end
     rescue => e
       warn("[MutationTester] Shadow sanity check could not prepare the shadow workspace: #{e.message}")
-      false
+      :baseline
     end
 
     def shadow_copy_project(source, dest)
@@ -284,8 +293,9 @@ module MutationTester
       end
     end
 
-    def run_specs_in_shadow(spec_file, working_dir, example_filters: [])
-      test_command(spec_file, example_filters: example_filters).run(timeout: @config.effective_timeout, chdir: working_dir)
+    def run_specs_in_shadow(spec_file, working_dir, project_root, example_filters: [])
+      test_command(spec_file, example_filters: example_filters)
+        .run(timeout: @config.effective_timeout, chdir: working_dir, mirror_of: project_root)
     end
 
     def discoverable_project_root
@@ -390,19 +400,18 @@ module MutationTester
     end
 
     def prepare_in_memory_execution
-      return 'the in-memory runner supports RSpec suites only' unless detect_test_framework(@spec_file) == :rspec
       return 'Process.fork is not supported on this platform' unless ForkRunner.available?
       if InMemoryLoader.load_time_defined_guard?(@original_content)
         return 'the source file uses defined? at load time, so redefinition would silently skip the guarded code'
       end
 
-      runner = ForkRunner.new(use_bundle_exec: @use_bundle_exec)
+      runner = ForkRunner.new(use_bundle_exec: @use_bundle_exec, framework: detect_test_framework(@spec_file))
       unless runner.ready?
         runner.shutdown
         return 'the in-memory worker failed to preload the environment'
       end
 
-      preloaded, message = runner.preload(@spec_file, chdir: Dir.pwd)
+      preloaded, message = runner.preload(@spec_file, chdir: Dir.pwd, stop_on_first_failure: true)
       unless preloaded
         runner.shutdown
         return "the spec file could not be preloaded (#{message})"
@@ -521,6 +530,7 @@ module MutationTester
       ForkRunner.prepare_pool(
         [@config.parallel_processes, total].min,
         use_bundle_exec: @use_bundle_exec,
+        framework: detect_test_framework(@spec_file),
         env_for: worker_env_for
       )
     end
@@ -637,7 +647,8 @@ module MutationTester
         use_bundle_exec: @use_bundle_exec,
         runner: @config.runner,
         example_filters: example_filters,
-        worker_env_var: @config.worker_env_var
+        worker_env_var: @config.worker_env_var,
+        stop_on_first_failure: true
       )
     end
   end

data/lib/mutation_tester/reporters/console_reporter.rb

@@ -1,12 +1,15 @@
 module MutationTester
   module Reporters
     class ConsoleReporter < BaseReporter
+      NOTHING_KILLED_MIN_MUTANTS = 5
+
       def generate
         puts "\n" + Rainbow('=' * 80).bright
         puts Rainbow('🧬 MUTATION TESTING REPORT').bright.cyan
         puts Rainbow('=' * 80).bright
 
         print_summary
+        print_nothing_killed_warning
         print_survived_mutations if survived_count > 0
 
         puts "\n" + Rainbow('=' * 80).bright
@@ -20,6 +23,7 @@ module MutationTester
         puts "  #{Rainbow("Killed: " + killed_count.to_s).green} ✅"
         puts "  #{Rainbow("Survived: " + survived_count.to_s).red} ❌"
         puts "  #{Rainbow("Timeout: " + timeout_count.to_s).yellow} ⏱️"
+        print_timeout_deadline
         puts "  #{Rainbow("Stillborn: " + stillborn_count.to_s).yellow} 🧬"
         puts "  #{Rainbow("Errors: " + error_count.to_s).yellow} 💥"
         print_excluded_summary
@@ -29,6 +33,47 @@ module MutationTester
         puts "\n  " + progress_bar
       end
 
+      def print_nothing_killed_warning
+        return unless nothing_killed?
+
+        puts "\n#{Rainbow("⚠️  Not one of the #{scored_mutant_count} scored mutants was killed.").yellow}"
+        puts Rainbow('   A whole file that kills nothing is more often a runner problem (the mutated code never').yellow
+        puts Rainbow('   reached the tests) than a test-quality gap. Re-run with --runner spawn and compare before').yellow
+        puts Rainbow('   acting on this score.').yellow
+      end
+
+      def nothing_killed?
+        killed_count.zero? && timeout_count.zero? && scored_mutant_count >= NOTHING_KILLED_MIN_MUTANTS
+      end
+
+      def scored_mutant_count
+        killed_count + timeout_count + survived_count
+      end
+
+      def print_timeout_deadline
+        return unless timeout_count.positive?
+
+        puts "    #{Rainbow("deadline: " + timeout_deadline_description).yellow}"
+      end
+
+      def timeout_deadline_description
+        deadline = @config.effective_timeout
+        return 'none (no deadline configured)' if deadline.nil?
+
+        "#{format_seconds(deadline)} #{deadline_origin}"
+      end
+
+      def deadline_origin
+        baseline = @config.baseline_duration
+        return '(explicitly configured)' if @config.timeout_explicitly_set? || baseline.nil?
+
+        "(#{format('%g', @config.timeout_factor)}x baseline #{format_seconds(baseline)})"
+      end
+
+      def format_seconds(value)
+        "#{format('%.2f', value)}s"
+      end
+
       def print_excluded_summary
         count = excluded_line_count
         return unless count.positive?

data/lib/mutation_tester/test_command.rb

@@ -8,20 +8,23 @@ module MutationTester
     end
 
     POLL_INTERVAL = 0.05
+    MINITEST_FAIL_FAST_PATH = File.expand_path('minitest_fail_fast.rb', __dir__).freeze
 
     attr_reader :spec_file, :framework, :use_bundle_exec, :example_filters
 
-    def initialize(spec_file, use_bundle_exec:, framework: nil, runner: :spawn, example_filters: [], worker_env_var: nil)
+    def initialize(spec_file, use_bundle_exec:, framework: nil, runner: :spawn, example_filters: [], worker_env_var: nil,
+                   stop_on_first_failure: false)
       @spec_file = spec_file
       @framework = framework || self.class.detect_framework(spec_file)
       @use_bundle_exec = use_bundle_exec
       @runner = runner
       @example_filters = @framework == :rspec ? Array(example_filters) : []
       @worker_env_var = worker_env_var
+      @stop_on_first_failure = stop_on_first_failure
     end
 
     def argv
-      parts = [runner, spec_file, *filter_args]
+      parts = [runner, *interpreter_args, spec_file, *filter_args, *fail_fast_args]
       @use_bundle_exec ? ['bundle', 'exec', *parts] : parts
     end
 
@@ -37,10 +40,20 @@ module MutationTester
       cmd
     end
 
-    def run(timeout: nil, chdir: nil, capture: false)
+    def run(timeout: nil, chdir: nil, capture: false, mirror_of: nil)
       if fork_execution?
-        fork_runner = ForkRunner.acquire(use_bundle_exec: @use_bundle_exec)
-        return fork_runner.execute(spec_file, timeout: timeout, chdir: chdir || Dir.pwd, capture: capture, args: filter_args) if fork_runner
+        fork_runner = ForkRunner.acquire(use_bundle_exec: @use_bundle_exec, framework: @framework)
+        if fork_runner
+          return fork_runner.execute(
+            spec_file,
+            timeout: timeout,
+            chdir: chdir || Dir.pwd,
+            capture: capture,
+            args: filter_args,
+            stop_on_first_failure: @stop_on_first_failure,
+            mirror_of: mirror_of
+          )
+        end
       end
 
       return run_captured(timeout: timeout, chdir: chdir) if capture
@@ -53,7 +66,6 @@ module MutationTester
     end
 
     def fork_execution?
-      return false if @framework == :minitest
       return false if @runner == :spawn
 
       ForkRunner.available?
@@ -100,6 +112,18 @@ module MutationTester
       example_filters.flat_map { |filter| ['-e', filter] }
     end
 
+    def interpreter_args
+      return [] unless @stop_on_first_failure && @framework == :minitest
+
+      ['-r', MINITEST_FAIL_FAST_PATH]
+    end
+
+    def fail_fast_args
+      return [] unless @stop_on_first_failure && @framework == :rspec
+
+      ['--fail-fast']
+    end
+
     def run_captured(timeout:, chdir:)
       log = Tempfile.new(['mutation_tester_baseline', '.log'])
       spawn_options = { pgroup: true, %i[out err] => log.path }

data/lib/mutation_tester/version.rb

@@ -1,3 +1,3 @@
 module MutationTester
-  VERSION = '1.3.0'.freeze
+  VERSION = '1.4.1'.freeze
 end

data/readme.md

@@ -55,7 +55,7 @@ MutationTester keeps its requirements low so it drops into a wide range of proje
 |-------------------------|-----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | Ruby                    | `>= 3.0`        | Floor is Ruby 3.0. CI runs the suite on 3.0, 3.1, 3.2 and 3.3. The gem is developed on Ruby 4.0.2, and 4.x is supported. Ruby 4.x has no prebuilt binary on the GitHub-hosted runners yet, so it is verified on the development host rather than in the CI matrix. |
 | RSpec (your project)    | `3.x`           | The gem shells out to your project's own `rspec`, so any RSpec 3.x works. Both ends of the range are exercised by a real mutation run: the lowest 3.0.x line in the CI framework matrix, and 3.13.x in the example jobs.                                           |
-| Minitest (your project) | `5.x` and `6.x` | The gem shells out to your project's own `ruby test_file.rb`, so both the 5.x and 6.x lines work. Both are exercised by a real mutation run: 5.x in the example jobs, and 6.x in the CI framework matrix.                                                          |
+| Minitest (your project) | `5.x` and `6.x` | The gem runs your project's own test file (`ruby test_file.rb`, or the same file inside a preloaded fork worker), so both the 5.x and 6.x lines work. Both are exercised by a real mutation run: 5.x in the example jobs, and 6.x in the CI framework matrix.                                                          |
 
 Notes:
 
@@ -200,7 +200,7 @@ mutation_test [OPTIONS] --glob 'lib/**/*.rb'
 | Flag | Description |
 |---|---|
 | `-p, --parallel N` | Run with N parallel processes (default: auto, derived from the CPU core count with a cap of 8; `-p 1` forces serial execution). |
-| `--runner MODE` | Mutant execution runner: `auto` (default) tries `in_memory` first (RSpec with `Process.fork` available and a passing unmutated-source probe), then falls back to `fork`, then `spawn`, announcing every step down on stderr with its reason; `fork` (preloaded environment, RSpec on platforms with `Process.fork`), `spawn` (one full process per mutant) and `in_memory` (mutations applied in child-process memory, zero file writes per mutant, RSpec only) force the specific mode. See [Execution runners](#execution-runners-fork-spawn-in-memory). |
+| `--runner MODE` | Mutant execution runner: `auto` (default) tries `in_memory` first (`Process.fork` available and a passing unmutated-source probe), then falls back to `fork`, then `spawn`, announcing every step down on stderr with its reason; `fork` (preloaded environment, on platforms with `Process.fork`), `spawn` (one full process per mutant) and `in_memory` (mutations applied in child-process memory, zero file writes per mutant) force the specific mode. See [Execution runners](#execution-runners-fork-spawn-in-memory). |
 | `--staged` | Mutation-test the files staged in git (`git diff --cached --name-only`; files staged as deleted are ignored), mapping each to its spec like a positional `FILE` list. Cannot be combined with positional arguments or `--glob`. See [File lists and --staged](#file-lists-and---staged-test-what-you-changed). |
 | `--glob PATTERN` | Batch mode: mutation-test every source file matching `PATTERN`, mapping each to its spec by convention (see [Batch mode](#batch-mode-run-many-files-in-one-command)). |
 | `--spec-glob TEMPLATE` | Spec-mapping template with a `{name}` placeholder (default: `spec/{name}_spec.rb`). Requires a positional `FILE` list, `--staged`, or `--glob`. |
@@ -232,7 +232,9 @@ bundle exec mutation_test --reporters json,html --output-dir build/mutation \
   file, unknown reporter, source with a syntax error).
 - `2` - a usage error (conflicting flags; see the batch sections below).
 - `3` - the run aborted or degraded before reaching a verdict: the shadow workspace
-  was unreliable, or every mutant ended as `error`/`stillborn` so nothing was scored.
+  was unreliable (the unmutated source failed there, or the workspace copy of the
+  source turned out not to be the code the tests execute), or every mutant ended as
+  `error`/`stillborn` so nothing was scored.
   This signals an infrastructure or runner problem, not a test-quality gap, so CI
   hooks can distinguish it from a genuine threshold failure.
 - `130` - interrupted with Ctrl+C.
@@ -523,7 +525,16 @@ invalid value (less than 1, or non-numeric) falls back to 1 with a warning on st
 In parallel mode each mutant runs in an isolated shadow workspace. Every `.rb`
 file is a physical copy (non-Ruby files stay symlinks for speed), so mutations
 apply correctly even when a spec loads the source indirectly (e.g. via
-`spec_helper`). The parallel mutation score therefore matches serial.
+`spec_helper`), and `$LOAD_PATH` entries pointing into the project resolve
+inside the workspace, so a test file that reaches its source through
+`require "test_helper"` gets the mutated copy too. The parallel mutation score
+therefore matches serial.
+
+Before the first mutant, the run proves this in the workspace itself: the
+unmutated source must pass there, and the same suite must fail once that copy of
+the source is replaced by a `raise`. A run whose tests pass even then is aborted
+as an infrastructure failure (exit code `3`) rather than reported as a 0.0%
+score, because the mutated file is demonstrably not the code being executed.
 
 ### When to use serial vs parallel execution
 
@@ -597,18 +608,18 @@ the fastest safe one, announcing every fallback on stderr:
 
 - **in_memory** (default where supported): re-evaluates the mutated source in the
   memory of a fresh fork of a preloaded process, with zero file writes per mutant
-  and no shadow workspaces. RSpec only; the fastest path. Mutations that only take
-  effect at class-load time (constants consumed by macros, `validates`/`has_many`/
+  and no shadow workspaces. RSpec and Minitest; the fastest path. Mutations that only
+  take effect at class-load time (constants consumed by macros, `validates`/`has_many`/
   `before_save`/`scope`/`attribute`, anything inside an `included do` block) cannot
   be observed by re-evaluating source in a preloaded process, so those mutants are
   routed automatically to the file-based path and the rest still run in memory (see
   below); the combined score matches a full `fork` run.
-- **fork**: preloads the environment once (RubyGems, Bundler, `rspec-core`) and
-  forks a fresh child per mutant. RSpec on platforms with `Process.fork`; removes
-  most of the fixed per-mutant boot cost.
-- **spawn**: starts one full process per mutant (`bundle exec rspec ...`). Slower
-  per mutant, but works everywhere (the only runner for Minitest and for
-  platforms without `Process.fork`).
+- **fork**: preloads the environment once (RubyGems, Bundler, the test framework)
+  and forks a fresh child per mutant. RSpec and Minitest on platforms with
+  `Process.fork`; removes most of the fixed per-mutant boot cost.
+- **spawn**: starts one full process per mutant (`bundle exec rspec ...` or
+  `bundle exec ruby test_file.rb`). Slower per mutant, but works everywhere
+  (the only runner on platforms without `Process.fork`).
 
 `auto` tries `in_memory`, then `fork`, then `spawn`; every step down prints one
 stderr warning with its reason, so a fallback is never silent. All runners
@@ -628,9 +639,9 @@ do not need to pick `--runner fork` for correctness on load-time code.
 
 | Mode        | Picked by `auto` when                                                                                                                                                   | Falls back to                                                                                                                              |
 |-------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|
-| `in_memory` | the suite is RSpec, the platform has `Process.fork`, the file has no load-time `defined?` guard, and re-applying the unmutated source in a probe child passes the suite | `fork`/`spawn` (whole run) with a stderr warning naming the reason; a single worker dying mid-run falls back only for its share of mutants; a mutant that raises while being applied falls back alone |
-| `fork`      | the suite is RSpec, `Process.fork` is available, but in-memory is unavailable (each reason is printed)                                                                  | `spawn`, with a stderr warning, when the helper process fails to preload the environment                                                   |
-| `spawn`     | the suite is Minitest, or the platform has no `Process.fork`                                                                                                            | nothing; it works everywhere                                                                                                               |
+| `in_memory` | the platform has `Process.fork`, the file has no load-time `defined?` guard, and re-applying the unmutated source in a probe child passes the suite | `fork`/`spawn` (whole run) with a stderr warning naming the reason; a single worker dying mid-run falls back only for its share of mutants; a mutant that raises while being applied falls back alone |
+| `fork`      | `Process.fork` is available, but in-memory is unavailable (each reason is printed)                                                                                      | `spawn`, with a stderr warning, when the helper process fails to preload the environment                                                   |
+| `spawn`     | the platform has no `Process.fork`                                                                                                                                      | nothing; it works everywhere                                                                                                               |
 
 Force a specific runner (skipping the auto attempts) with the `--runner
 fork|spawn|in_memory` flag, the `MUTATION_TESTER_RUNNER` environment variable, or
@@ -648,6 +659,22 @@ fork and in-memory limitation lists (frozen classes, load-time `defined?` guards
 worker-death fallback, `require_relative` idempotency), see
 [docs/execution-runners.md](docs/execution-runners.md#execution-runners-fork-spawn-in-memory).
 
+### Stopping a mutant at its first failing test
+
+A mutant only needs one failing test to be killed, so every mutant run stops at
+its first failure: RSpec mutant runs get `--fail-fast`, and Minitest mutant runs
+get a preloaded reporter that aborts the run the same way (both on the file-based
+runners and inside the preloaded fork worker). This never changes a verdict, only
+the work done to reach it: a run that stops early had already failed, and a run
+with no failure is unaffected and still executes every test.
+
+It matters most for a mutant that breaks something every test touches (a broken
+class body, a constant every example reads). Such a mutant used to pay the full
+test file once per mutant, which on a large test file can exceed the per-mutant
+deadline and turn a decided kill into a reported timeout. Adding tests to the file
+then made the score worse. The baseline run and the shadow sanity check are
+unaffected: they are expected to pass, and a passing run runs every test.
+
 ### Test selection (fast kill with full-file confirmation)
 
 For RSpec suites on the file-based runners, MutationTester runs each mutant in
@@ -741,6 +768,18 @@ few lines of surrounding context:
   💡 Suggestion: Add tests to verify behavior for each of the 2 variants above
 ```
 
+When at least one mutant timed out, the summary also names the deadline those
+mutants were measured against and where it came from, so a genuine hang and a
+deadline calibrated from a slow test file are distinguishable at a glance:
+
+```
+  Timeout: 3 ⏱️
+    deadline: 6.50s (5x baseline 1.30s)
+```
+
+With an explicit `config.timeout` / `--timeout` the same line reads
+`deadline: 30.00s (explicitly configured)`.
+
 ### HTML report
 
 Beautiful interactive HTML report with:

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: mutation_tester
 version: !ruby/object:Gem::Version
-  version: 1.3.0
+  version: 1.4.1
 platform: ruby
 authors:
 - Kamil Dzierbicki
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-07-15 00:00:00.000000000 Z
+date: 2026-08-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: parallel
@@ -169,6 +169,7 @@ files:
 - lib/mutation_tester/fork_runner/worker.rb
 - lib/mutation_tester/framework_detector.rb
 - lib/mutation_tester/in_memory_loader.rb
+- lib/mutation_tester/minitest_fail_fast.rb
 - lib/mutation_tester/mutation_runner.rb
 - lib/mutation_tester/mutator.rb
 - lib/mutation_tester/progress_display.rb