mutation_tester 1.2.0 → 1.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 20fff769b6b8b7efeb874208f5e3c25206ca92a2fc9f220bdd4fcb520eab944c
-  data.tar.gz: 3b630985b285958780d49d356cbe04f629ce50e1aca947d2f40d8bdddd274d42
+  metadata.gz: be945d45b6c4219df627808a62336abe51ce94b088d79db47e25c794cdf227bd
+  data.tar.gz: 80aaefd644319415bfb25cebab5eb28077f3b99c763d234d20cf6885f3c001d2
 SHA512:
-  metadata.gz: f40cf255d91681f4ae191b444ba8f99ba897ea746b2f157190d69bce091e90c4aaec9bde9c73fe69008de17916478ff40227a7e2439de687d4a576d72d664d0a
-  data.tar.gz: 9c8d46ca0d8a98dff41de40b61e8fdd60c68c55ed37303e02d08009968a0263bf932e0892af3e13af12810c27cb61f695f1d01a77e46f2dbc375735fe49b87a4
+  metadata.gz: 2ad47074139c21bb7cd6db12eb53d2dd8c350b85c24c43f2ce44ba25590da1b3a4aa3f2e9f5ea2394abbdb8779e50cb9b3bb94418a94b03a1148ef2bd186b1f0
+  data.tar.gz: 58078c8af01ec3155be934d6917a3b66b0779f70bc20b04a23d05f82c06e9198000156fc13a7f7eece21cff11582d6b142a5ffb4aaa11d3191c15ebe4c3d9fbb

data/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## [1.4.0] - 2026-08-01
+
+- Minitest suites now use the same execution runners as RSpec instead of being pinned to `spawn`. The fork worker preloads `minitest` (disabling the `minitest/autorun` at-exit hook and driving `Minitest.run` itself, so the file still runs exactly once per mutant) and the in-memory runner preloads the test file once and re-evaluates each mutant in a fresh fork, so a Minitest project no longer pays a full interpreter, Bundler and framework boot per mutant. Preloaded workers are now keyed by framework, so a mixed-framework `--glob` run never hands a Minitest file to an RSpec-preloaded worker.
+- Every mutant run now stops at its first failing test: RSpec runs get `--fail-fast` and Minitest runs get a preloaded reporter that aborts on the first non-passing result, on all three runners. This cannot change a verdict (a run that stops early has already failed, which is what makes a mutant killed), and only mutant runs opt in: the baseline run and the shadow sanity check still run the whole file. It removes the pathology where a mutant that breaks something every test touches (a class body that no longer loads, a constant every test reads) re-raised the same error once per test, crossed the calibrated deadline, and was reported as a `timeout` instead of a `killed` - a failure mode that got worse as tests were added to the file. Measured on a 15-mutant fixture with a 0.6 s boot and 12 tests: 22.2 s -> 6.5 s by default and 22.7 s -> 13.2 s with `--runner spawn`, with an unchanged score.
+- The console summary now names the deadline that timed-out mutants were measured against and where it came from (`deadline: 6.50s (5x baseline 1.30s)`, or `(explicitly configured)`), so a genuine hang and a deadline calibrated from a slow test file are no longer indistinguishable.
+- `--fail-fast` now stops a batch at the first file with a surviving mutant even when that file's own run completed. `Core#stopped_on_survivor?` reports the fail-fast stop, while `Core#interrupted?` keeps its narrower meaning (mutants were left unprocessed) for the reports and the interruption banner.
+
+## [1.3.0] - 2026-07-15
+
+- Calibrated the per-mutant timeout against the measured baseline run: unless `config.timeout` is set explicitly, each mutant now gets `max(5s, timeout_factor * baseline duration)` (factor configurable via `config.timeout_factor` / `--timeout-factor N`, default 5) instead of a fixed 30 s, so a loaded machine no longer inflates the mutation score by killing healthy-but-slow runs as timeouts. An explicit `config.timeout` (including `nil` for no deadline) keeps today's fixed-budget behavior and disables calibration. Added the opt-in `config.timeout_policy = :separate` / `--timeout-policy separate`, which scores `killed / (killed + survived)` with timeouts excluded from the score and reported only as their own category; the default `:killed` policy and its output are unchanged.
+- Runs that finish without a single scored mutant (every mutant errored or was stillborn) are now reported as an infrastructure/runner failure with the error/stillborn counts instead of a misleading "score 0.0% is below threshold" verdict. `Core` exposes `infrastructure_failure?` to distinguish such degraded runs (including the shadow-workspace abort) from a genuine threshold failure, and `mutation_test` maps them to the new exit code `3` in single-file mode (`0` pass, `1` threshold failure, `2` usage error, `130` interrupt). The batch summary names degraded files instead of blaming the threshold.
+
 ## [1.2.0] - 2026-07-14
 
 - Fixed the in-memory runner reporting false survivors for mutations that only take effect at class-load time (constants consumed by macros, `validates`/`has_many`/`before_save`/`scope`/`attribute`, and anything inside an `included do` block). The mutator now classifies each mutation by AST context, and the in-memory run routes load-time mutations to the file-based path while keeping method-body mutations in memory, so the default (`auto`) score matches a full `fork` run on Rails concerns and models. Measured on a real Rails concern the default score went from a misleading 1.63% to the correct 35.77%.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    mutation_tester (1.2.0)
+    mutation_tester (1.4.0)
       parallel (~> 1.20)
       parser (~> 3.3)
       rainbow (~> 3.0)

data/docs/execution-runners.md

@@ -10,14 +10,17 @@ force a runner.
 Every mutant is executed by one of three runners:
 
 - **fork**: a helper process preloads the environment
-  once (RubyGems, Bundler and `rspec-core`, without loading the mutated file or
-  the specs), and each mutant runs in a fresh fork of that process. The fork
-  loads the spec only after the mutated source has been written, so every
-  mutant is visible and no state leaks between mutants. This removes most of
+  once (RubyGems, Bundler and the test framework: `rspec-core` for RSpec,
+  `minitest` for Minitest, without loading the mutated file or the tests), and
+  each mutant runs in a fresh fork of that process. The fork loads the test file
+  only after the mutated source has been written, so every mutant is visible and
+  no state leaks between mutants. For Minitest the worker disables the
+  `minitest/autorun` at-exit hook and drives `Minitest.run` itself, so the file
+  runs exactly once per mutant. This removes most of
   the fixed per-mutant boot cost, which matters on large suites and in CI.
-- **spawn**: each mutant starts a full new process (`bundle exec rspec ...`).
-  Slower per mutant, but works everywhere.
-- **in_memory** (default where supported): the helper process additionally preloads the spec
+- **spawn**: each mutant starts a full new process (`bundle exec rspec ...` or
+  `bundle exec ruby test_file.rb`). Slower per mutant, but works everywhere.
+- **in_memory** (default where supported): the helper process additionally preloads the test
   file and, through it, the original source, once per run. Each mutant then
   runs in a fresh fork that re-evaluates the mutated source in memory
   (redefining the loaded methods and class constants, with the
@@ -34,15 +37,16 @@ Selection is automatic (`auto`): the fastest safe path is tried first and every
 step down to a slower one prints a single stderr warning with its reason, so a
 fallback is never silent. The order is `in_memory` (RSpec with `Process.fork`
 available and a passing unmutated-source probe), then `fork`, then `spawn`.
+Both RSpec and Minitest suites use the same three runners.
 All runners produce identical scores and per-mutant statuses, and all enforce
 the same hard per-mutant timeout (monotonic deadline plus a process-group
 kill).
 
 | Mode        | Picked by `auto` when                                                                                                                                                   | Falls back to                                                                                                                              |
 |-------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|
-| `in_memory` | the suite is RSpec, the platform has `Process.fork`, the file has no load-time `defined?` guard, and re-applying the unmutated source in a probe child passes the suite | `fork`/`spawn` (whole run) with a stderr warning naming the reason; a single worker dying mid-run falls back only for its share of mutants; a mutant that raises while being applied falls back alone |
-| `fork`      | the suite is RSpec, `Process.fork` is available, but in-memory is unavailable (each reason is printed)                                                                  | `spawn`, with a stderr warning, when the helper process fails to preload the environment                                                   |
-| `spawn`     | the suite is Minitest, or the platform has no `Process.fork`                                                                                                            | nothing; it works everywhere                                                                                                               |
+| `in_memory` | the platform has `Process.fork`, the file has no load-time `defined?` guard, and re-applying the unmutated source in a probe child passes the suite | `fork`/`spawn` (whole run) with a stderr warning naming the reason; a single worker dying mid-run falls back only for its share of mutants; a mutant that raises while being applied falls back alone |
+| `fork`      | `Process.fork` is available, but in-memory is unavailable (each reason is printed)                                                                                      | `spawn`, with a stderr warning, when the helper process fails to preload the environment                                                   |
+| `spawn`     | the platform has no `Process.fork`                                                                                                                                      | nothing; it works everywhere                                                                                                               |
 
 Forcing a mode with `--runner fork|spawn|in_memory` skips the auto attempts and
 uses that mode directly (`in_memory` keeps its own documented safety fallbacks;
@@ -77,10 +81,31 @@ MutationTester.configure do |config|
 end
 ```
 
+### Stopping a mutant at its first failing test
+
+Every mutant run stops as soon as one test fails, on all three runners:
+
+- RSpec mutant runs are given `--fail-fast` (as a CLI argument on `spawn`, in the
+  runner arguments on `fork`, and in the preloaded configuration on `in_memory`).
+- Minitest mutant runs load `lib/mutation_tester/minitest_fail_fast.rb`, which
+  registers a Minitest plugin whose reporter raises `Interrupt` on the first
+  non-passing result. On `spawn` the file is preloaded with `ruby -r`, on the
+  preloaded runners the worker enables the same reporter per job.
+
+This cannot change a verdict. A run that stops early has already recorded a
+failure, which is exactly what makes a mutant killed, and a run without a failure
+is untouched and executes every test. Only the mutant runs opt in: the baseline
+run and the shadow-workspace sanity check are expected to pass and always run the
+whole file, so a failing baseline still reports every failure it finds.
+
+The pathological case it removes is a mutant that breaks something every test
+touches (a class body that no longer loads, a constant every test reads). Such a
+mutant used to re-raise the same error once per test, which on a large test file
+can cross the per-mutant deadline and be reported as a `timeout` instead of a
+`killed`, and which gets worse as tests are added to the file.
+
 ### Limitations of the fork runner
 
-- Minitest suites always use `spawn` (fork support for Minitest is a separate
-  decision after RSpec experience is collected).
 - Platforms without `Process.fork` (for example Windows or JRuby) always use
   `spawn`, even when `--runner fork` is requested.
 - If the helper process fails to preload the environment, the run warns once
@@ -92,8 +117,8 @@ The in-memory runner never fails silently: each case below falls back to
 file-based execution with a warning, and a mutant is marked `error` only when
 no fallback is possible.
 
-- RSpec only, and the file must be classic loadable code (classes/modules).
-  Minitest suites fall back to the file-based path with a warning.
+- The file must be classic loadable code (classes/modules) that survives being
+  evaluated a second time.
 - With `-p N` (N > 1) the run stays fully in memory: the environment, the
   original source and the specs are preloaded once, the preloaded process is
   forked into N pooled clones, and every parallel worker applies each mutant

data/docs/json-schema.md

@@ -52,13 +52,13 @@ history of changes.
 | `metadata.source_file`         | string  | Absolute path of the mutated source file.                                                                                                                                                                                                                                                                                                       |
 | `metadata.spec_file`           | string  | Absolute path of the test file that was run.                                                                                                                                                                                                                                                                                                    |
 | `summary.total`                | integer | Total number of mutations produced (all statuses).                                                                                                                                                                                                                                                                                              |
-| `summary.killed`               | integer | Effective kills (a `timeout` counts as a kill). Kept for backward compatibility.                                                                                                                                                                                                                                                                |
+| `summary.killed`               | integer | Effective kills (a `timeout` counts as a kill; under the opt-in `timeout_policy: :separate` only real kills are counted). Kept for backward compatibility.                                                                                                                                                                                      |
 | `summary.survived`             | integer | Number of surviving mutants.                                                                                                                                                                                                                                                                                                                    |
-| `summary.mutation_score`       | number  | Percentage `(killed + timeout) / (killed + timeout + survived) * 100`, rounded to 2 decimals. `stillborn` and `error` are excluded from the denominator.                                                                                                                                                                                        |
+| `summary.mutation_score`       | number  | Percentage `(killed + timeout) / (killed + timeout + survived) * 100`, rounded to 2 decimals; under the opt-in `timeout_policy: :separate` it is `killed / (killed + survived) * 100` with timeouts excluded. `stillborn` and `error` are excluded from the denominator.                                                                        |
 | `summary.quality_rating`       | string  | Human label derived from the score (Excellent/Good/Fair/Poor/Critical).                                                                                                                                                                                                                                                                         |
 | `summary.categories.killed`    | integer | Mutants whose covering tests failed.                                                                                                                                                                                                                                                                                                            |
 | `summary.categories.survived`  | integer | Mutants whose covering tests still passed (a test gap candidate).                                                                                                                                                                                                                                                                               |
-| `summary.categories.timeout`   | integer | Mutants that exceeded the per-mutant deadline (counted as kills in the score).                                                                                                                                                                                                                                                                  |
+| `summary.categories.timeout`   | integer | Mutants that exceeded the per-mutant deadline (counted as kills in the score by default; excluded from the score under `timeout_policy: :separate`).                                                                                                                                                                                            |
 | `summary.categories.stillborn` | integer | Mutants whose code no longer parses. Never run; excluded from the score.                                                                                                                                                                                                                                                                        |
 | `summary.categories.error`     | integer | Mutants that hit a runner-side error. Excluded from the score.                                                                                                                                                                                                                                                                                  |
 | `mutations[].id`               | integer | Stable id of the mutation within this run.                                                                                                                                                                                                                                                                                                      |
@@ -144,7 +144,7 @@ the exit code keeps its meaning (`0` passed, `1` failed).
 | `summary.files`        | integer | Files that entered the run: processed plus skipped. Files a `--since` filter left out as unchanged are not counted; they are reported on stderr.                                                                                             |
 | `summary.processed`    | integer | Files that were actually mutation-tested.                                                                                                                                                                                                    |
 | `summary.skipped`      | array   | One entry per skipped file: `file` (the path as given) and `reason`, one of `file not found`, `not a Ruby source file`, `a test file, not a mutable source`, `no matching spec file`.                                                        |
-| `summary.score`        | number  | Aggregate mutation score over every mutant of every processed file, with the per-file formula: `(killed + timeout) / (killed + timeout + survived) * 100`, rounded to 2 decimals; `stillborn` and `error` are excluded from the denominator. |
+| `summary.score`        | number  | Aggregate mutation score over every mutant of every processed file, with the per-file formula: `(killed + timeout) / (killed + timeout + survived) * 100` (or `killed / (killed + survived) * 100` under `timeout_policy: :separate`), rounded to 2 decimals; `stillborn` and `error` are excluded from the denominator. |
 | `summary.passed`       | boolean | `true` exactly when the process exits `0`: the run matched/processed files and every processed file met the threshold.                                                                                                                       |
 | `summary.interrupted`  | boolean | `true` when `--fail-fast` stopped the batch at a surviving mutant, so remaining files were not run and the envelope is partial.                                                                                                              |
 | `survivors[].file`     | string  | Absolute path of the mutated source file, the same convention as `files[].metadata.source_file`, so a survivor joins its full per-file report by an exact string match on this value.                                                        |

data/exe/mutation_test

@@ -68,6 +68,14 @@ OptionParser.new do |opts|
     options[:fail_fast] = true
   end
 
+  opts.on('--timeout-factor N', Float, "Per-mutant timeout budget as N times the measured baseline test run, never below #{MutationTester::Configuration::CALIBRATED_TIMEOUT_FLOOR}s (default: #{MutationTester::Configuration::DEFAULT_TIMEOUT_FACTOR}; ignored when config.timeout is set explicitly, which keeps a fixed budget)") do |n|
+    options[:timeout_factor] = n
+  end
+
+  opts.on('--timeout-policy MODE', 'Scoring policy for timed-out mutants: killed (default) counts a timeout as a kill; separate keeps timeouts out of the score entirely (killed / (killed + survived)) and reports them only as their own category') do |mode|
+    options[:timeout_policy] = mode
+  end
+
   opts.on('--strict-equality', 'Enable strict equality probes (== to eql? and equal?); default off, expect noise on code that does not distinguish numeric types or object identity') do
     options[:strict_equality] = true
   end
@@ -151,6 +159,8 @@ apply_configuration = lambda do
     config.reporters = selected_reporters if selected_reporters
     config.output_dir = options[:output_dir] if options[:output_dir]
     config.test_selection = false if options[:no_test_selection]
+    config.timeout_factor = options[:timeout_factor] if options[:timeout_factor]
+    config.timeout_policy = options[:timeout_policy] if options[:timeout_policy]
     config.worker_env_var = options[:worker_env] if options.key?(:worker_env)
     config.fail_fast = options.fetch(:fail_fast, false)
     config.mutation_types[:strict_equality] = true if options[:strict_equality]
@@ -342,4 +352,5 @@ else
   success = guard_interrupt.call { core.run }
 end
 
-exit(success ? 0 : 1)
+exit 0 if success
+exit(core.infrastructure_failure? ? 3 : 1)

data/lib/mutation_tester/batch_runner.rb

@@ -20,10 +20,14 @@ module MutationTester
 
     SKIP_ORDER = %i[missing not_ruby test_file no_spec].freeze
 
-    ProcessedEntry = Struct.new(:source_file, :spec_file, :score, :passed, :output_dir, :results, :interrupted, keyword_init: true) do
+    ProcessedEntry = Struct.new(:source_file, :spec_file, :score, :passed, :output_dir, :results, :interrupted, :degraded, keyword_init: true) do
       def passed?
         passed
       end
+
+      def degraded?
+        !!degraded
+      end
     end
 
     SkippedEntry = Struct.new(:source_file, :expected_spec, :reason, keyword_init: true)
@@ -192,9 +196,10 @@ module MutationTester
         passed: passed,
         output_dir: file_config.output_dir,
         results: core.results,
-        interrupted: core.interrupted?
+        interrupted: core.interrupted?,
+        degraded: core.infrastructure_failure?
       )
-      [entry, core.interrupted?]
+      [entry, core.stopped_on_survivor?]
     end
 
     def spec_path_for(source_file)
@@ -247,12 +252,19 @@ module MutationTester
       end
 
       puts Rainbow('=' * 80).bright
+      failed = result.processed.reject(&:passed?)
+      degraded = failed.select(&:degraded?)
       if @files && result.processed.empty?
         puts Rainbow('❌ No files were mutation-tested: every listed file was skipped').red
       elsif result.success?
         puts Rainbow('✓ All processed files met the mutation score threshold').green
+      elsif degraded.size == failed.size
+        puts Rainbow('❌ The failing files produced no scored mutants; this indicates an infrastructure or runner problem, not a test-quality gap').red
       else
         puts Rainbow('❌ One or more files did not meet the mutation score threshold').red
+        unless degraded.empty?
+          puts Rainbow("⚠️ #{degraded.size} of the failing files produced no scored mutants (infrastructure or runner problem); see the per-file output above").yellow
+        end
       end
 
       print_survivors(result.survivors)

data/lib/mutation_tester/configuration.rb

@@ -3,7 +3,11 @@ require 'etc'
 module MutationTester
   class Configuration
     RUNNER_MODES = %i[auto fork spawn in_memory].freeze
+    TIMEOUT_POLICIES = %i[killed separate].freeze
     AUTO_PARALLEL_CAP = 8
+    DEFAULT_TIMEOUT = 30
+    DEFAULT_TIMEOUT_FACTOR = 5
+    CALIBRATED_TIMEOUT_FLOOR = 5
 
     def self.auto_parallel_processes
       [[Etc.nprocessors, AUTO_PARALLEL_CAP].min, 1].max
@@ -14,9 +18,9 @@ module MutationTester
       number <= 0 ? '' : (number + 1).to_s
     end
 
-    attr_reader :parallel_processes, :runner, :worker_env_var
+    attr_reader :parallel_processes, :runner, :worker_env_var, :timeout, :timeout_factor, :timeout_policy
 
-    attr_accessor :timeout,
+    attr_accessor :baseline_duration,
       :baseline_timeout,
       :mutation_types,
       :reporters,
@@ -33,7 +37,9 @@ module MutationTester
       self.parallel_processes = ENV['MUTATION_TESTER_PARALLEL_PROCESSES'] || self.class.auto_parallel_processes
       self.runner = ENV['MUTATION_TESTER_RUNNER'] || :auto
       self.worker_env_var = ENV['MUTATION_TESTER_WORKER_ENV']
-      @timeout = 30
+      @timeout = DEFAULT_TIMEOUT
+      @timeout_factor = DEFAULT_TIMEOUT_FACTOR
+      @timeout_policy = :killed
       @baseline_timeout = 300
       @mutation_types = {
         arithmetic: true,
@@ -73,6 +79,39 @@ module MutationTester
       @reporters = source.reporters.dup
     end
 
+    def timeout=(value)
+      @timeout_explicit = true
+      @timeout = value
+    end
+
+    def timeout_explicitly_set?
+      !!@timeout_explicit
+    end
+
+    def timeout_factor=(value)
+      factor = coerce_numeric(value)
+      unless factor.positive?
+        warn "[MutationTester] timeout_factor must be a number greater than 0; got #{value.inspect}, falling back to #{DEFAULT_TIMEOUT_FACTOR}."
+        factor = DEFAULT_TIMEOUT_FACTOR
+      end
+      @timeout_factor = factor
+    end
+
+    def timeout_policy=(value)
+      policy = value.to_s.strip.downcase.to_sym
+      unless TIMEOUT_POLICIES.include?(policy)
+        warn "[MutationTester] timeout_policy must be one of #{TIMEOUT_POLICIES.join(", ")}; got #{value.inspect}, falling back to killed."
+        policy = :killed
+      end
+      @timeout_policy = policy
+    end
+
+    def effective_timeout
+      return @timeout if timeout_explicitly_set? || @baseline_duration.nil?
+
+      [CALIBRATED_TIMEOUT_FLOOR, @timeout_factor * @baseline_duration].max
+    end
+
     def parallel_processes=(value)
       count = value.to_i
       if count < 1
@@ -101,5 +140,13 @@ module MutationTester
 
       { @worker_env_var => self.class.worker_env_value(index) }
     end
+
+    private
+
+    def coerce_numeric(value)
+      Float(value)
+    rescue ArgumentError, TypeError
+      0
+    end
   end
 end

data/lib/mutation_tester/core.rb

@@ -11,6 +11,7 @@ module MutationTester
       @mutations = []
       @results = []
       @parse_failed = false
+      @shadow_aborted = false
       MutationRunner.recover_in_place_backup(@source_file)
       @original_content = File.read(@source_file)
     end
@@ -27,11 +28,15 @@ module MutationTester
         return true
       end
 
-      return false unless shadow_environment_reliable?
+      unless shadow_environment_reliable?
+        @shadow_aborted = true
+        return false
+      end
 
       run_mutations
       generate_reports
       return report_interruption if interrupted?
+      return report_infrastructure_failure if infrastructure_failure?
 
       check_threshold
     ensure
@@ -39,14 +44,20 @@ module MutationTester
       ForkRunner.shutdown_all
     end
 
+    def stopped_on_survivor?
+      @config.fail_fast && @results.any? { |result| result[:status] == :survived }
+    end
+
     def interrupted?
-      @config.fail_fast &&
-        @results.size < @mutations.size &&
-        @results.any? { |result| result[:status] == :survived }
+      stopped_on_survivor? && @results.size < @mutations.size
     end
 
     def mutation_score
-      Reporters::BaseReporter.score(@results)
+      Reporters::BaseReporter.score(@results, policy: @config.timeout_policy)
+    end
+
+    def infrastructure_failure?
+      @shadow_aborted || (!@results.empty? && scored_count.zero?)
     end
 
     def threshold_met?
@@ -69,7 +80,9 @@ module MutationTester
     def run_original_tests
       puts Rainbow("\n🧪 Running original tests...").yellow
       command = test_command
+      baseline_started = monotonic_time
       result = command.run(timeout: @config.baseline_timeout, capture: true)
+      baseline_elapsed = monotonic_time - baseline_started
 
       if result.timed_out?
         puts Rainbow("❌ Original tests exceeded the baseline deadline of #{@config.baseline_timeout}s and were terminated.").red
@@ -85,9 +98,14 @@ module MutationTester
         return false
       end
       puts Rainbow('✓ Original tests passed').green
+      @config.baseline_duration = baseline_elapsed
       true
     end
 
+    def monotonic_time
+      Process.clock_gettime(Process::CLOCK_MONOTONIC)
+    end
+
     def replay_baseline_output(output)
       return if output.nil? || output.strip.empty?
 
@@ -178,6 +196,20 @@ module MutationTester
       false
     end
 
+    def scored_count
+      @results.count { |r| %i[killed timeout survived].include?(Reporters::BaseReporter.status_for(r)) }
+    end
+
+    def status_count(status)
+      @results.count { |r| Reporters::BaseReporter.status_for(r) == status }
+    end
+
+    def report_infrastructure_failure
+      puts Rainbow("\n❌ Run failed: none of the #{@results.size} mutants could be scored (#{status_count(:error)} error, #{status_count(:stillborn)} stillborn).").red
+      puts Rainbow('   This indicates an infrastructure or runner problem (test environment, workspace, or test command), not a test-quality gap.').red
+      false
+    end
+
     def check_threshold
       score = mutation_score
 

data/lib/mutation_tester/fork_runner/worker.rb

@@ -1,13 +1,22 @@
 # frozen_string_literal: true
 
 require 'json'
-require 'rspec/core'
 require_relative '../in_memory_loader'
 
 control = $stdout.dup
 control.sync = true
 STDOUT.reopen(File::NULL)
 
+framework = ARGV.shift == 'minitest' ? :minitest : :rspec
+
+if framework == :minitest
+  require 'minitest'
+  require_relative '../minitest_fail_fast'
+  Minitest.class_variable_set(:@@installed_at_exit, true)
+else
+  require 'rspec/core'
+end
+
 kill_group = lambda do |pid|
   begin
     Process.kill('KILL', -pid)
@@ -91,10 +100,25 @@ supervise_child = lambda do |job, out, child_body|
   [timed_out, payload, reaped]
 end
 
+load_test_file = lambda do |path|
+  $PROGRAM_NAME = path
+  load(path)
+end
+
+run_test_file = lambda do |job|
+  if framework == :minitest
+    MutationTester::MinitestFailFast.enabled = job['stop_on_first_failure'] ? true : false
+    load_test_file.call(job['spec'])
+    Minitest.run(Array(job['args'])) ? 0 : 1
+  else
+    args = [job['spec'], *Array(job['args'])]
+    args << '--fail-fast' if job['stop_on_first_failure']
+    RSpec::Core::Runner.run(args, STDERR, STDOUT).to_i
+  end
+end
+
 run_job = lambda do |job, out|
-  timed_out, payload, reaped = supervise_child.call(job, out, lambda do
-    RSpec::Core::Runner.run([job['spec'], *Array(job['args'])], STDERR, STDOUT).to_i
-  end)
+  timed_out, payload, reaped = supervise_child.call(job, out, lambda { run_test_file.call(job) })
 
   status =
     if timed_out
@@ -111,16 +135,32 @@ end
 preload_specs = lambda do |request, out|
   begin
     Dir.chdir(request['chdir']) if request['chdir']
-    sink = File.open(File::NULL, 'w')
-    runner = RSpec::Core::Runner.new(RSpec::Core::ConfigurationOptions.new([request['spec']]))
-    runner.setup(sink, sink)
-    preloaded = runner
+    if framework == :minitest
+      MutationTester::MinitestFailFast.enabled = request['stop_on_first_failure'] ? true : false
+      load_test_file.call(request['spec'])
+      preloaded = true
+    else
+      sink = File.open(File::NULL, 'w')
+      options = [request['spec']]
+      options << '--fail-fast' if request['stop_on_first_failure']
+      runner = RSpec::Core::Runner.new(RSpec::Core::ConfigurationOptions.new(options))
+      runner.setup(sink, sink)
+      preloaded = runner
+    end
     out.puts(JSON.generate('event' => 'preloaded', 'status' => 'ok'))
   rescue ScriptError, StandardError => e
     out.puts(JSON.generate('event' => 'preloaded', 'status' => 'error', 'message' => "#{e.class}: #{e.message}"))
   end
 end
 
+run_preloaded_suite = lambda do
+  if framework == :minitest
+    Minitest.run([]) ? 0 : 1
+  else
+    preloaded.run_specs(RSpec.world.ordered_example_groups).to_i
+  end
+end
+
 run_in_memory_job = lambda do |job, out|
   request = job['in_memory']
 
@@ -132,7 +172,7 @@ run_in_memory_job = lambda do |job, out|
   timed_out, payload, reaped = supervise_child.call(job, out, lambda do
     begin
       MutationTester::InMemoryLoader.apply(request['source'], request['path'])
-      JSON.generate('code' => preloaded.run_specs(RSpec.world.ordered_example_groups).to_i)
+      JSON.generate('code' => run_preloaded_suite.call)
     rescue ScriptError, StandardError => e
       JSON.generate('error' => "#{e.class}: #{e.message}")
     end

data/lib/mutation_tester/fork_runner.rb

@@ -21,22 +21,22 @@ module MutationTester
         Process.respond_to?(:fork)
       end
 
-      def acquire(use_bundle_exec:)
+      def acquire(use_bundle_exec:, framework: :rspec)
         return nil unless available?
 
-        key = [Process.pid, use_bundle_exec]
+        key = [Process.pid, use_bundle_exec, framework]
         return registry[key] if registry.key?(key)
 
-        registry[key] = checkout_pooled(use_bundle_exec) || boot(use_bundle_exec)
+        registry[key] = checkout_pooled(pool_key(use_bundle_exec, framework)) || boot(use_bundle_exec, framework)
       end
 
-      def prepare_pool(count, use_bundle_exec:, env_for: nil)
+      def prepare_pool(count, use_bundle_exec:, framework: :rspec, env_for: nil)
         return unless available?
 
-        primary = acquire(use_bundle_exec: use_bundle_exec)
+        primary = acquire(use_bundle_exec: use_bundle_exec, framework: framework)
         return unless primary
 
-        refill_pool(use_bundle_exec, count, primary, env_for: env_for)
+        refill_pool(pool_key(use_bundle_exec, framework), count, primary, env_for: env_for)
       end
 
       def prepare_in_memory_pool(count, primary)
@@ -100,11 +100,15 @@ module MutationTester
 
       private
 
-      def checkout_pooled(use_bundle_exec)
+      def pool_key(use_bundle_exec, framework)
+        [use_bundle_exec, framework]
+      end
+
+      def checkout_pooled(key)
         number = parallel_worker_number
         return nil unless number
 
-        entry = pool[use_bundle_exec]
+        entry = pool[key]
         entry && entry[:runners][number]
       end
 
@@ -123,8 +127,8 @@ module MutationTester
         entry[:runners]
       end
 
-      def boot(use_bundle_exec)
-        runner = new(use_bundle_exec: use_bundle_exec)
+      def boot(use_bundle_exec, framework)
+        runner = new(use_bundle_exec: use_bundle_exec, framework: framework)
         return runner if runner.ready?
 
         runner.shutdown
@@ -133,8 +137,8 @@ module MutationTester
       end
     end
 
-    def initialize(use_bundle_exec:)
-      argv = ['ruby', WORKER_PATH]
+    def initialize(use_bundle_exec:, framework: :rspec)
+      argv = ['ruby', WORKER_PATH, framework.to_s]
       argv = ['bundle', 'exec', *argv] if use_bundle_exec
 
       job_reader, job_writer = IO.pipe
@@ -149,9 +153,16 @@ module MutationTester
       @ready
     end
 
-    def execute(spec_file, timeout: nil, chdir: nil, capture: false, args: [])
+    def execute(spec_file, timeout: nil, chdir: nil, capture: false, args: [], stop_on_first_failure: false)
       log = capture ? Tempfile.new(['mutation_tester_fork', '.log']) : nil
-      job = { spec: spec_file, timeout: timeout, chdir: chdir, log: log&.path, args: args }
+      job = {
+        spec: spec_file,
+        timeout: timeout,
+        chdir: chdir,
+        log: log&.path,
+        args: args,
+        stop_on_first_failure: stop_on_first_failure
+      }
       @job_writer.puts(JSON.generate(job))
       status = await_result(timeout)['status']
       result = TestCommand::Result.new(status == 'pass', status == 'timeout')
@@ -164,8 +175,9 @@ module MutationTester
       log&.unlink
     end
 
-    def preload(spec_file, chdir: nil)
-      @job_writer.puts(JSON.generate(preload: { spec: spec_file, chdir: chdir }))
+    def preload(spec_file, chdir: nil, stop_on_first_failure: false)
+      request = { spec: spec_file, chdir: chdir, stop_on_first_failure: stop_on_first_failure }
+      @job_writer.puts(JSON.generate(preload: request))
       event = read_event(monotonic_time + BOOT_TIMEOUT)
       return [true, nil] if event.is_a?(Hash) && event['event'] == 'preloaded' && event['status'] == 'ok'
 

data/lib/mutation_tester/minitest_fail_fast.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module MutationTester
+  module MinitestFailFast
+    PLUGIN_NAME = :mutation_tester_fail_fast
+
+    class << self
+      attr_accessor :enabled
+    end
+
+    self.enabled = true
+  end
+end
+
+begin
+  require 'minitest'
+rescue LoadError
+  Kernel.warn '[MutationTester] minitest is not loadable here; mutant runs will execute every test instead of stopping at the first failure.'
+else
+  module MutationTester
+    module MinitestFailFast
+      class Reporter < Minitest::AbstractReporter
+        def record(result)
+          return unless MinitestFailFast.enabled
+          return if result.passed? || result.skipped?
+
+          raise Interrupt
+        end
+      end
+    end
+  end
+
+  module Minitest
+    def self.plugin_mutation_tester_fail_fast_init(_options)
+      reporter << MutationTester::MinitestFailFast::Reporter.new
+    end
+  end
+
+  unless Minitest.extensions.include?(MutationTester::MinitestFailFast::PLUGIN_NAME)
+    Minitest.register_plugin(MutationTester::MinitestFailFast::PLUGIN_NAME)
+  end
+end

data/lib/mutation_tester/mutation_runner.rb

@@ -285,7 +285,7 @@ module MutationTester
     end
 
     def run_specs_in_shadow(spec_file, working_dir, example_filters: [])
-      test_command(spec_file, example_filters: example_filters).run(timeout: @config.timeout, chdir: working_dir)
+      test_command(spec_file, example_filters: example_filters).run(timeout: @config.effective_timeout, chdir: working_dir)
     end
 
     def discoverable_project_root
@@ -325,7 +325,7 @@ module MutationTester
     end
 
     def run_specs_in_place(spec_file, example_filters: [])
-      test_command(spec_file, example_filters: example_filters).run(timeout: @config.timeout)
+      test_command(spec_file, example_filters: example_filters).run(timeout: @config.effective_timeout)
     end
 
     def run_mutation_in_memory(mutation, result, project_root = nil)
@@ -335,7 +335,7 @@ module MutationTester
       outcome = runner.execute_in_memory(
         source: mutation[:code],
         path: @source_file,
-        timeout: @config.timeout,
+        timeout: @config.effective_timeout,
         chdir: Dir.pwd
       )
 
@@ -390,19 +390,18 @@ module MutationTester
     end
 
     def prepare_in_memory_execution
-      return 'the in-memory runner supports RSpec suites only' unless detect_test_framework(@spec_file) == :rspec
       return 'Process.fork is not supported on this platform' unless ForkRunner.available?
       if InMemoryLoader.load_time_defined_guard?(@original_content)
         return 'the source file uses defined? at load time, so redefinition would silently skip the guarded code'
       end
 
-      runner = ForkRunner.new(use_bundle_exec: @use_bundle_exec)
+      runner = ForkRunner.new(use_bundle_exec: @use_bundle_exec, framework: detect_test_framework(@spec_file))
       unless runner.ready?
         runner.shutdown
         return 'the in-memory worker failed to preload the environment'
       end
 
-      preloaded, message = runner.preload(@spec_file, chdir: Dir.pwd)
+      preloaded, message = runner.preload(@spec_file, chdir: Dir.pwd, stop_on_first_failure: true)
       unless preloaded
         runner.shutdown
         return "the spec file could not be preloaded (#{message})"
@@ -419,7 +418,7 @@ module MutationTester
       outcome = runner.execute_in_memory(
         source: @original_content,
         path: @source_file,
-        timeout: @config.timeout,
+        timeout: @config.effective_timeout,
         chdir: Dir.pwd
       )
       return nil if outcome.status == 'pass'
@@ -521,6 +520,7 @@ module MutationTester
       ForkRunner.prepare_pool(
         [@config.parallel_processes, total].min,
         use_bundle_exec: @use_bundle_exec,
+        framework: detect_test_framework(@spec_file),
         env_for: worker_env_for
       )
     end
@@ -637,7 +637,8 @@ module MutationTester
         use_bundle_exec: @use_bundle_exec,
         runner: @config.runner,
         example_filters: example_filters,
-        worker_env_var: @config.worker_env_var
+        worker_env_var: @config.worker_env_var,
+        stop_on_first_failure: true
       )
     end
   end

data/lib/mutation_tester/reporters/base_reporter.rb

@@ -3,12 +3,14 @@ module MutationTester
     class BaseReporter
       attr_reader :results, :source_file, :spec_file, :config
 
-      def self.score(results)
-        effective_killed = results.count { |r| %i[killed timeout].include?(status_for(r)) }
-        scored = results.count { |r| %i[killed timeout survived].include?(status_for(r)) }
+      def self.score(results, policy: :killed)
+        killing_statuses = policy == :separate ? %i[killed] : %i[killed timeout]
+        scoring_statuses = killing_statuses + %i[survived]
+        kills = results.count { |r| killing_statuses.include?(status_for(r)) }
+        scored = results.count { |r| scoring_statuses.include?(status_for(r)) }
         return 0.0 if scored.zero?
 
-        (effective_killed.to_f / scored * 100).round(2)
+        (kills.to_f / scored * 100).round(2)
       end
 
       def self.status_for(result)
@@ -62,6 +64,8 @@ module MutationTester
       end
 
       def effective_killed_count
+        return killed_count if @config.timeout_policy == :separate
+
         killed_count + timeout_count
       end
 
@@ -70,7 +74,7 @@ module MutationTester
       end
 
       def mutation_score
-        self.class.score(@results)
+        self.class.score(@results, policy: @config.timeout_policy)
       end
 
       def quality_rating

data/lib/mutation_tester/reporters/batch_json_reporter.rb

@@ -53,7 +53,7 @@ module MutationTester
       end
 
       def aggregate_score
-        BaseReporter.score(@result.processed.flat_map { |entry| entry.results || [] })
+        BaseReporter.score(@result.processed.flat_map { |entry| entry.results || [] }, policy: @config.timeout_policy)
       end
 
       def file_reports

data/lib/mutation_tester/reporters/console_reporter.rb

@@ -20,6 +20,7 @@ module MutationTester
         puts "  #{Rainbow("Killed: " + killed_count.to_s).green} ✅"
         puts "  #{Rainbow("Survived: " + survived_count.to_s).red} ❌"
         puts "  #{Rainbow("Timeout: " + timeout_count.to_s).yellow} ⏱️"
+        print_timeout_deadline
         puts "  #{Rainbow("Stillborn: " + stillborn_count.to_s).yellow} 🧬"
         puts "  #{Rainbow("Errors: " + error_count.to_s).yellow} 💥"
         print_excluded_summary
@@ -29,6 +30,30 @@ module MutationTester
         puts "\n  " + progress_bar
       end
 
+      def print_timeout_deadline
+        return unless timeout_count.positive?
+
+        puts "    #{Rainbow("deadline: " + timeout_deadline_description).yellow}"
+      end
+
+      def timeout_deadline_description
+        deadline = @config.effective_timeout
+        return 'none (no deadline configured)' if deadline.nil?
+
+        "#{format_seconds(deadline)} #{deadline_origin}"
+      end
+
+      def deadline_origin
+        baseline = @config.baseline_duration
+        return '(explicitly configured)' if @config.timeout_explicitly_set? || baseline.nil?
+
+        "(#{format('%g', @config.timeout_factor)}x baseline #{format_seconds(baseline)})"
+      end
+
+      def format_seconds(value)
+        "#{format('%.2f', value)}s"
+      end
+
       def print_excluded_summary
         count = excluded_line_count
         return unless count.positive?

data/lib/mutation_tester/test_command.rb

@@ -8,20 +8,23 @@ module MutationTester
     end
 
     POLL_INTERVAL = 0.05
+    MINITEST_FAIL_FAST_PATH = File.expand_path('minitest_fail_fast.rb', __dir__).freeze
 
     attr_reader :spec_file, :framework, :use_bundle_exec, :example_filters
 
-    def initialize(spec_file, use_bundle_exec:, framework: nil, runner: :spawn, example_filters: [], worker_env_var: nil)
+    def initialize(spec_file, use_bundle_exec:, framework: nil, runner: :spawn, example_filters: [], worker_env_var: nil,
+                   stop_on_first_failure: false)
       @spec_file = spec_file
       @framework = framework || self.class.detect_framework(spec_file)
       @use_bundle_exec = use_bundle_exec
       @runner = runner
       @example_filters = @framework == :rspec ? Array(example_filters) : []
       @worker_env_var = worker_env_var
+      @stop_on_first_failure = stop_on_first_failure
     end
 
     def argv
-      parts = [runner, spec_file, *filter_args]
+      parts = [runner, *interpreter_args, spec_file, *filter_args, *fail_fast_args]
       @use_bundle_exec ? ['bundle', 'exec', *parts] : parts
     end
 
@@ -39,8 +42,17 @@ module MutationTester
 
     def run(timeout: nil, chdir: nil, capture: false)
       if fork_execution?
-        fork_runner = ForkRunner.acquire(use_bundle_exec: @use_bundle_exec)
-        return fork_runner.execute(spec_file, timeout: timeout, chdir: chdir || Dir.pwd, capture: capture, args: filter_args) if fork_runner
+        fork_runner = ForkRunner.acquire(use_bundle_exec: @use_bundle_exec, framework: @framework)
+        if fork_runner
+          return fork_runner.execute(
+            spec_file,
+            timeout: timeout,
+            chdir: chdir || Dir.pwd,
+            capture: capture,
+            args: filter_args,
+            stop_on_first_failure: @stop_on_first_failure
+          )
+        end
       end
 
       return run_captured(timeout: timeout, chdir: chdir) if capture
@@ -53,7 +65,6 @@ module MutationTester
     end
 
     def fork_execution?
-      return false if @framework == :minitest
       return false if @runner == :spawn
 
       ForkRunner.available?
@@ -100,6 +111,18 @@ module MutationTester
       example_filters.flat_map { |filter| ['-e', filter] }
     end
 
+    def interpreter_args
+      return [] unless @stop_on_first_failure && @framework == :minitest
+
+      ['-r', MINITEST_FAIL_FAST_PATH]
+    end
+
+    def fail_fast_args
+      return [] unless @stop_on_first_failure && @framework == :rspec
+
+      ['--fail-fast']
+    end
+
     def run_captured(timeout:, chdir:)
       log = Tempfile.new(['mutation_tester_baseline', '.log'])
       spawn_options = { pgroup: true, %i[out err] => log.path }

data/lib/mutation_tester/version.rb

@@ -1,3 +1,3 @@
 module MutationTester
-  VERSION = '1.2.0'.freeze
+  VERSION = '1.4.0'.freeze
 end

data/lib/tasks/mutation_tester.rake

@@ -45,7 +45,7 @@ namespace :mutation do
     end
 
     unless all_passed
-      puts Rainbow("\n❌ One or more files did not meet the mutation score threshold").red
+      puts Rainbow("\n❌ One or more files failed mutation testing (see the per-file output above)").red
       abort
     end
   end

data/readme.md

@@ -55,7 +55,7 @@ MutationTester keeps its requirements low so it drops into a wide range of proje
 |-------------------------|-----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | Ruby                    | `>= 3.0`        | Floor is Ruby 3.0. CI runs the suite on 3.0, 3.1, 3.2 and 3.3. The gem is developed on Ruby 4.0.2, and 4.x is supported. Ruby 4.x has no prebuilt binary on the GitHub-hosted runners yet, so it is verified on the development host rather than in the CI matrix. |
 | RSpec (your project)    | `3.x`           | The gem shells out to your project's own `rspec`, so any RSpec 3.x works. Both ends of the range are exercised by a real mutation run: the lowest 3.0.x line in the CI framework matrix, and 3.13.x in the example jobs.                                           |
-| Minitest (your project) | `5.x` and `6.x` | The gem shells out to your project's own `ruby test_file.rb`, so both the 5.x and 6.x lines work. Both are exercised by a real mutation run: 5.x in the example jobs, and 6.x in the CI framework matrix.                                                          |
+| Minitest (your project) | `5.x` and `6.x` | The gem runs your project's own test file (`ruby test_file.rb`, or the same file inside a preloaded fork worker), so both the 5.x and 6.x lines work. Both are exercised by a real mutation run: 5.x in the example jobs, and 6.x in the CI framework matrix.                                                          |
 
 Notes:
 
@@ -200,12 +200,14 @@ mutation_test [OPTIONS] --glob 'lib/**/*.rb'
 | Flag | Description |
 |---|---|
 | `-p, --parallel N` | Run with N parallel processes (default: auto, derived from the CPU core count with a cap of 8; `-p 1` forces serial execution). |
-| `--runner MODE` | Mutant execution runner: `auto` (default) tries `in_memory` first (RSpec with `Process.fork` available and a passing unmutated-source probe), then falls back to `fork`, then `spawn`, announcing every step down on stderr with its reason; `fork` (preloaded environment, RSpec on platforms with `Process.fork`), `spawn` (one full process per mutant) and `in_memory` (mutations applied in child-process memory, zero file writes per mutant, RSpec only) force the specific mode. See [Execution runners](#execution-runners-fork-spawn-in-memory). |
+| `--runner MODE` | Mutant execution runner: `auto` (default) tries `in_memory` first (`Process.fork` available and a passing unmutated-source probe), then falls back to `fork`, then `spawn`, announcing every step down on stderr with its reason; `fork` (preloaded environment, on platforms with `Process.fork`), `spawn` (one full process per mutant) and `in_memory` (mutations applied in child-process memory, zero file writes per mutant) force the specific mode. See [Execution runners](#execution-runners-fork-spawn-in-memory). |
 | `--staged` | Mutation-test the files staged in git (`git diff --cached --name-only`; files staged as deleted are ignored), mapping each to its spec like a positional `FILE` list. Cannot be combined with positional arguments or `--glob`. See [File lists and --staged](#file-lists-and---staged-test-what-you-changed). |
 | `--glob PATTERN` | Batch mode: mutation-test every source file matching `PATTERN`, mapping each to its spec by convention (see [Batch mode](#batch-mode-run-many-files-in-one-command)). |
 | `--spec-glob TEMPLATE` | Spec-mapping template with a `{name}` placeholder (default: `spec/{name}_spec.rb`). Requires a positional `FILE` list, `--staged`, or `--glob`. |
 | `--since REV` | Incremental batch mode: mutate only the files matched by `--glob` that changed since git revision `REV` (new files count as changed). Requires `--glob`. See [Incremental mode](#incremental-mode-mutate-only-what-changed). |
 | `--fail-fast` | Stop the run at the first surviving mutant and finish with a failing status. Works in single-file mode and with `--glob`. |
+| `--timeout-factor N` | Per-mutant timeout budget as `N` times the measured baseline test run, never below 5 s (default: 5, must be > 0). Ignored when `config.timeout` is set explicitly, which keeps a fixed budget. See [Configuration](#configuration). |
+| `--timeout-policy MODE` | Scoring policy for timed-out mutants: `killed` (default) counts a timeout as a kill; `separate` keeps timeouts out of the score entirely (`killed / (killed + survived)`) and reports them only as their own category in the console, JSON and HTML reports. |
 | `--worker-env NAME` | Set environment variable `NAME` to a distinct per-worker value before each parallel worker boots (`parallel_tests` `TEST_ENV_NUMBER` convention: worker 0 -> `""`, worker N -> `N+1`), so a `parallel_tests`-style `database.yml` selects a per-worker database. You provision the databases (e.g. `rake parallel:prepare`). Not supported by the `in_memory` runner (it falls back to `fork`). See [Making parallelism work with Rails](#making-parallelism-work-with-rails). |
 | `--strict-equality` | Enable the opt-in strict-equality probes (`==` → `eql?` and `==` → `equal?`). Default off; expect noise on code that does not distinguish numeric types or object identity. See [Strict Equality Mutations](docs/mutation-types.md#strict-equality-mutations-opt-in). |
 | `-h, --help` | Show help message. |
@@ -223,6 +225,22 @@ bundle exec mutation_test --reporters json,html --output-dir build/mutation \
   app/models/user.rb spec/models/user_spec.rb
 ```
 
+### Exit codes (single-file mode)
+
+- `0` - the run passed (score met the threshold, or `fail_on_threshold` is disabled).
+- `1` - the mutation score is below the threshold, or the input is unusable (missing
+  file, unknown reporter, source with a syntax error).
+- `2` - a usage error (conflicting flags; see the batch sections below).
+- `3` - the run aborted or degraded before reaching a verdict: the shadow workspace
+  was unreliable, or every mutant ended as `error`/`stillborn` so nothing was scored.
+  This signals an infrastructure or runner problem, not a test-quality gap, so CI
+  hooks can distinguish it from a genuine threshold failure.
+- `130` - interrupted with Ctrl+C.
+
+Batch modes (`FILE...` lists, `--staged`, `--glob`) keep the exit codes documented in
+their sections below (`0`/`1`/`2`); a degraded file is named explicitly in the batch
+summary instead of being blamed on the threshold.
+
 ### Running with rake
 
 You can also run mutation tests through rake tasks.
@@ -403,8 +421,24 @@ MutationTester.configure do |config|
   # config.parallel_processes = 1
   config.parallel_processes = 4
 
-  # Timeout for each test run (seconds)
-  config.timeout = 30
+  # Per-mutant timeout (seconds). Calibrated automatically by default:
+  # max(5, timeout_factor * measured baseline duration), so a loaded machine
+  # cannot inflate the score by turning healthy-but-slow runs into timeout
+  # kills; on code paths without a measured baseline the fixed default of 30
+  # applies. Setting config.timeout explicitly (a number, or nil for no
+  # deadline at all) disables calibration and keeps that fixed budget:
+  # config.timeout = 30
+
+  # Multiplier for the baseline-calibrated per-mutant timeout (> 0; an
+  # invalid value falls back to the default with a warning):
+  config.timeout_factor = 5
+
+  # Scoring policy for timed-out mutants. :killed (default) counts a timeout
+  # as a kill: (killed + timeout) / (killed + timeout + survived). :separate
+  # keeps timeouts out of the score entirely, killed / (killed + survived),
+  # and reports them only as their own category, so timeouts under load can
+  # never raise the score:
+  config.timeout_policy = :killed
 
   # Mutant execution runner: :auto (default), :fork, :spawn or :in_memory.
   # :auto picks the fastest safe path and announces any fallback on stderr;
@@ -563,18 +597,18 @@ the fastest safe one, announcing every fallback on stderr:
 
 - **in_memory** (default where supported): re-evaluates the mutated source in the
   memory of a fresh fork of a preloaded process, with zero file writes per mutant
-  and no shadow workspaces. RSpec only; the fastest path. Mutations that only take
-  effect at class-load time (constants consumed by macros, `validates`/`has_many`/
+  and no shadow workspaces. RSpec and Minitest; the fastest path. Mutations that only
+  take effect at class-load time (constants consumed by macros, `validates`/`has_many`/
   `before_save`/`scope`/`attribute`, anything inside an `included do` block) cannot
   be observed by re-evaluating source in a preloaded process, so those mutants are
   routed automatically to the file-based path and the rest still run in memory (see
   below); the combined score matches a full `fork` run.
-- **fork**: preloads the environment once (RubyGems, Bundler, `rspec-core`) and
-  forks a fresh child per mutant. RSpec on platforms with `Process.fork`; removes
-  most of the fixed per-mutant boot cost.
-- **spawn**: starts one full process per mutant (`bundle exec rspec ...`). Slower
-  per mutant, but works everywhere (the only runner for Minitest and for
-  platforms without `Process.fork`).
+- **fork**: preloads the environment once (RubyGems, Bundler, the test framework)
+  and forks a fresh child per mutant. RSpec and Minitest on platforms with
+  `Process.fork`; removes most of the fixed per-mutant boot cost.
+- **spawn**: starts one full process per mutant (`bundle exec rspec ...` or
+  `bundle exec ruby test_file.rb`). Slower per mutant, but works everywhere
+  (the only runner on platforms without `Process.fork`).
 
 `auto` tries `in_memory`, then `fork`, then `spawn`; every step down prints one
 stderr warning with its reason, so a fallback is never silent. All runners
@@ -594,9 +628,9 @@ do not need to pick `--runner fork` for correctness on load-time code.
 
 | Mode        | Picked by `auto` when                                                                                                                                                   | Falls back to                                                                                                                              |
 |-------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|
-| `in_memory` | the suite is RSpec, the platform has `Process.fork`, the file has no load-time `defined?` guard, and re-applying the unmutated source in a probe child passes the suite | `fork`/`spawn` (whole run) with a stderr warning naming the reason; a single worker dying mid-run falls back only for its share of mutants; a mutant that raises while being applied falls back alone |
-| `fork`      | the suite is RSpec, `Process.fork` is available, but in-memory is unavailable (each reason is printed)                                                                  | `spawn`, with a stderr warning, when the helper process fails to preload the environment                                                   |
-| `spawn`     | the suite is Minitest, or the platform has no `Process.fork`                                                                                                            | nothing; it works everywhere                                                                                                               |
+| `in_memory` | the platform has `Process.fork`, the file has no load-time `defined?` guard, and re-applying the unmutated source in a probe child passes the suite | `fork`/`spawn` (whole run) with a stderr warning naming the reason; a single worker dying mid-run falls back only for its share of mutants; a mutant that raises while being applied falls back alone |
+| `fork`      | `Process.fork` is available, but in-memory is unavailable (each reason is printed)                                                                                      | `spawn`, with a stderr warning, when the helper process fails to preload the environment                                                   |
+| `spawn`     | the platform has no `Process.fork`                                                                                                                                      | nothing; it works everywhere                                                                                                               |
 
 Force a specific runner (skipping the auto attempts) with the `--runner
 fork|spawn|in_memory` flag, the `MUTATION_TESTER_RUNNER` environment variable, or
@@ -614,6 +648,22 @@ fork and in-memory limitation lists (frozen classes, load-time `defined?` guards
 worker-death fallback, `require_relative` idempotency), see
 [docs/execution-runners.md](docs/execution-runners.md#execution-runners-fork-spawn-in-memory).
 
+### Stopping a mutant at its first failing test
+
+A mutant only needs one failing test to be killed, so every mutant run stops at
+its first failure: RSpec mutant runs get `--fail-fast`, and Minitest mutant runs
+get a preloaded reporter that aborts the run the same way (both on the file-based
+runners and inside the preloaded fork worker). This never changes a verdict, only
+the work done to reach it: a run that stops early had already failed, and a run
+with no failure is unaffected and still executes every test.
+
+It matters most for a mutant that breaks something every test touches (a broken
+class body, a constant every example reads). Such a mutant used to pay the full
+test file once per mutant, which on a large test file can exceed the per-mutant
+deadline and turn a decided kill into a reported timeout. Adding tests to the file
+then made the score worse. The baseline run and the shadow sanity check are
+unaffected: they are expected to pass, and a passing run runs every test.
+
 ### Test selection (fast kill with full-file confirmation)
 
 For RSpec suites on the file-based runners, MutationTester runs each mutant in
@@ -707,6 +757,18 @@ few lines of surrounding context:
   💡 Suggestion: Add tests to verify behavior for each of the 2 variants above
 ```
 
+When at least one mutant timed out, the summary also names the deadline those
+mutants were measured against and where it came from, so a genuine hang and a
+deadline calibrated from a slow test file are distinguishable at a glance:
+
+```
+  Timeout: 3 ⏱️
+    deadline: 6.50s (5x baseline 1.30s)
+```
+
+With an explicit `config.timeout` / `--timeout` the same line reads
+`deadline: 30.00s (explicitly configured)`.
+
 ### HTML report
 
 Beautiful interactive HTML report with:
@@ -725,7 +787,9 @@ to get a single, clean JSON document on **stdout** and nothing else: the banner,
 progress spinner, colours and the "report saved" notice all go to **stderr**, so
 the stream is safe to pipe straight into `jq` or a parser. The process still
 exits `0` when the mutation score meets the configured threshold and `1` when it
-does not, so the exit code remains a pass/fail signal.
+does not (a degraded single-file run exits `3`, see
+[Exit codes](#exit-codes-single-file-mode)), so the exit code remains a
+pass/fail signal.
 
 ```bash
 bundle exec mutation_test --json examples/calculator.rb examples/calculator_spec.rb | jq .

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: mutation_tester
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.4.0
 platform: ruby
 authors:
 - Kamil Dzierbicki
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-07-14 00:00:00.000000000 Z
+date: 2026-08-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: parallel
@@ -169,6 +169,7 @@ files:
 - lib/mutation_tester/fork_runner/worker.rb
 - lib/mutation_tester/framework_detector.rb
 - lib/mutation_tester/in_memory_loader.rb
+- lib/mutation_tester/minitest_fail_fast.rb
 - lib/mutation_tester/mutation_runner.rb
 - lib/mutation_tester/mutator.rb
 - lib/mutation_tester/progress_display.rb