mutation_tester 1.2.0 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 20fff769b6b8b7efeb874208f5e3c25206ca92a2fc9f220bdd4fcb520eab944c
4
- data.tar.gz: 3b630985b285958780d49d356cbe04f629ce50e1aca947d2f40d8bdddd274d42
3
+ metadata.gz: 03a2b2364ebede84e6b56cfe4fba7d0450475262d59aa754abe0ce34ee1a83ba
4
+ data.tar.gz: 7439181f228ffbf2d1cf2e603ac21a91a46c764cd66f0ebfcc40dab30a2f2188
5
5
  SHA512:
6
- metadata.gz: f40cf255d91681f4ae191b444ba8f99ba897ea746b2f157190d69bce091e90c4aaec9bde9c73fe69008de17916478ff40227a7e2439de687d4a576d72d664d0a
7
- data.tar.gz: 9c8d46ca0d8a98dff41de40b61e8fdd60c68c55ed37303e02d08009968a0263bf932e0892af3e13af12810c27cb61f695f1d01a77e46f2dbc375735fe49b87a4
6
+ metadata.gz: 6be68e2928ca7ecc1a382e01c66cb5ab7d0afa35e5cf5b6008a62d0599fe399af17a8960a3f4e62025d40e640850c4fde423cff5aa2de69d34db2f6139661342
7
+ data.tar.gz: fb12cf8fd83bc37529e7e0fcdc8bb72bc902df0c846b05ad8fb856d8db8b382cf18fb2058b42f5efa2b161f573517b368d7aee92563c4115c7b3669a3d2e5ba9
data/CHANGELOG.md CHANGED
@@ -1,5 +1,10 @@
1
1
  # Changelog
2
2
 
3
+ ## [1.3.0] - 2026-07-15
4
+
5
+ - Calibrated the per-mutant timeout against the measured baseline run: unless `config.timeout` is set explicitly, each mutant now gets `max(5s, timeout_factor * baseline duration)` (factor configurable via `config.timeout_factor` / `--timeout-factor N`, default 5) instead of a fixed 30 s, so a loaded machine no longer inflates the mutation score by killing healthy-but-slow runs as timeouts. An explicit `config.timeout` (including `nil` for no deadline) keeps today's fixed-budget behavior and disables calibration. Added the opt-in `config.timeout_policy = :separate` / `--timeout-policy separate`, which scores `killed / (killed + survived)` with timeouts excluded from the score and reported only as their own category; the default `:killed` policy and its output are unchanged.
6
+ - Runs that finish without a single scored mutant (every mutant errored or was stillborn) are now reported as an infrastructure/runner failure with the error/stillborn counts instead of a misleading "score 0.0% is below threshold" verdict. `Core` exposes `infrastructure_failure?` to distinguish such degraded runs (including the shadow-workspace abort) from a genuine threshold failure, and `mutation_test` maps them to the new exit code `3` in single-file mode (`0` pass, `1` threshold failure, `2` usage error, `130` interrupt). The batch summary names degraded files instead of blaming the threshold.
7
+
3
8
  ## [1.2.0] - 2026-07-14
4
9
 
5
10
  - Fixed the in-memory runner reporting false survivors for mutations that only take effect at class-load time (constants consumed by macros, `validates`/`has_many`/`before_save`/`scope`/`attribute`, and anything inside an `included do` block). The mutator now classifies each mutation by AST context, and the in-memory run routes load-time mutations to the file-based path while keeping method-body mutations in memory, so the default (`auto`) score matches a full `fork` run on Rails concerns and models. Measured on a real Rails concern the default score went from a misleading 1.63% to the correct 35.77%.
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- mutation_tester (1.2.0)
4
+ mutation_tester (1.3.0)
5
5
  parallel (~> 1.20)
6
6
  parser (~> 3.3)
7
7
  rainbow (~> 3.0)
data/docs/json-schema.md CHANGED
@@ -52,13 +52,13 @@ history of changes.
52
52
  | `metadata.source_file` | string | Absolute path of the mutated source file. |
53
53
  | `metadata.spec_file` | string | Absolute path of the test file that was run. |
54
54
  | `summary.total` | integer | Total number of mutations produced (all statuses). |
55
- | `summary.killed` | integer | Effective kills (a `timeout` counts as a kill). Kept for backward compatibility. |
55
+ | `summary.killed` | integer | Effective kills (a `timeout` counts as a kill; under the opt-in `timeout_policy: :separate` only real kills are counted). Kept for backward compatibility. |
56
56
  | `summary.survived` | integer | Number of surviving mutants. |
57
- | `summary.mutation_score` | number | Percentage `(killed + timeout) / (killed + timeout + survived) * 100`, rounded to 2 decimals. `stillborn` and `error` are excluded from the denominator. |
57
+ | `summary.mutation_score` | number | Percentage `(killed + timeout) / (killed + timeout + survived) * 100`, rounded to 2 decimals; under the opt-in `timeout_policy: :separate` it is `killed / (killed + survived) * 100` with timeouts excluded. `stillborn` and `error` are excluded from the denominator. |
58
58
  | `summary.quality_rating` | string | Human label derived from the score (Excellent/Good/Fair/Poor/Critical). |
59
59
  | `summary.categories.killed` | integer | Mutants whose covering tests failed. |
60
60
  | `summary.categories.survived` | integer | Mutants whose covering tests still passed (a test gap candidate). |
61
- | `summary.categories.timeout` | integer | Mutants that exceeded the per-mutant deadline (counted as kills in the score). |
61
+ | `summary.categories.timeout` | integer | Mutants that exceeded the per-mutant deadline (counted as kills in the score by default; excluded from the score under `timeout_policy: :separate`). |
62
62
  | `summary.categories.stillborn` | integer | Mutants whose code no longer parses. Never run; excluded from the score. |
63
63
  | `summary.categories.error` | integer | Mutants that hit a runner-side error. Excluded from the score. |
64
64
  | `mutations[].id` | integer | Stable id of the mutation within this run. |
@@ -144,7 +144,7 @@ the exit code keeps its meaning (`0` passed, `1` failed).
144
144
  | `summary.files` | integer | Files that entered the run: processed plus skipped. Files a `--since` filter left out as unchanged are not counted; they are reported on stderr. |
145
145
  | `summary.processed` | integer | Files that were actually mutation-tested. |
146
146
  | `summary.skipped` | array | One entry per skipped file: `file` (the path as given) and `reason`, one of `file not found`, `not a Ruby source file`, `a test file, not a mutable source`, `no matching spec file`. |
147
- | `summary.score` | number | Aggregate mutation score over every mutant of every processed file, with the per-file formula: `(killed + timeout) / (killed + timeout + survived) * 100`, rounded to 2 decimals; `stillborn` and `error` are excluded from the denominator. |
147
+ | `summary.score` | number | Aggregate mutation score over every mutant of every processed file, with the per-file formula: `(killed + timeout) / (killed + timeout + survived) * 100` (or `killed / (killed + survived) * 100` under `timeout_policy: :separate`), rounded to 2 decimals; `stillborn` and `error` are excluded from the denominator. |
148
148
  | `summary.passed` | boolean | `true` exactly when the process exits `0`: the run matched/processed files and every processed file met the threshold. |
149
149
  | `summary.interrupted` | boolean | `true` when `--fail-fast` stopped the batch at a surviving mutant, so remaining files were not run and the envelope is partial. |
150
150
  | `survivors[].file` | string | Absolute path of the mutated source file, the same convention as `files[].metadata.source_file`, so a survivor joins its full per-file report by an exact string match on this value. |
data/exe/mutation_test CHANGED
@@ -68,6 +68,14 @@ OptionParser.new do |opts|
68
68
  options[:fail_fast] = true
69
69
  end
70
70
 
71
+ opts.on('--timeout-factor N', Float, "Per-mutant timeout budget as N times the measured baseline test run, never below #{MutationTester::Configuration::CALIBRATED_TIMEOUT_FLOOR}s (default: #{MutationTester::Configuration::DEFAULT_TIMEOUT_FACTOR}; ignored when config.timeout is set explicitly, which keeps a fixed budget)") do |n|
72
+ options[:timeout_factor] = n
73
+ end
74
+
75
+ opts.on('--timeout-policy MODE', 'Scoring policy for timed-out mutants: killed (default) counts a timeout as a kill; separate keeps timeouts out of the score entirely (killed / (killed + survived)) and reports them only as their own category') do |mode|
76
+ options[:timeout_policy] = mode
77
+ end
78
+
71
79
  opts.on('--strict-equality', 'Enable strict equality probes (== to eql? and equal?); default off, expect noise on code that does not distinguish numeric types or object identity') do
72
80
  options[:strict_equality] = true
73
81
  end
@@ -151,6 +159,8 @@ apply_configuration = lambda do
151
159
  config.reporters = selected_reporters if selected_reporters
152
160
  config.output_dir = options[:output_dir] if options[:output_dir]
153
161
  config.test_selection = false if options[:no_test_selection]
162
+ config.timeout_factor = options[:timeout_factor] if options[:timeout_factor]
163
+ config.timeout_policy = options[:timeout_policy] if options[:timeout_policy]
154
164
  config.worker_env_var = options[:worker_env] if options.key?(:worker_env)
155
165
  config.fail_fast = options.fetch(:fail_fast, false)
156
166
  config.mutation_types[:strict_equality] = true if options[:strict_equality]
@@ -342,4 +352,5 @@ else
342
352
  success = guard_interrupt.call { core.run }
343
353
  end
344
354
 
345
- exit(success ? 0 : 1)
355
+ exit 0 if success
356
+ exit(core.infrastructure_failure? ? 3 : 1)
@@ -20,10 +20,14 @@ module MutationTester
20
20
 
21
21
  SKIP_ORDER = %i[missing not_ruby test_file no_spec].freeze
22
22
 
23
- ProcessedEntry = Struct.new(:source_file, :spec_file, :score, :passed, :output_dir, :results, :interrupted, keyword_init: true) do
23
+ ProcessedEntry = Struct.new(:source_file, :spec_file, :score, :passed, :output_dir, :results, :interrupted, :degraded, keyword_init: true) do
24
24
  def passed?
25
25
  passed
26
26
  end
27
+
28
+ def degraded?
29
+ !!degraded
30
+ end
27
31
  end
28
32
 
29
33
  SkippedEntry = Struct.new(:source_file, :expected_spec, :reason, keyword_init: true)
@@ -192,7 +196,8 @@ module MutationTester
192
196
  passed: passed,
193
197
  output_dir: file_config.output_dir,
194
198
  results: core.results,
195
- interrupted: core.interrupted?
199
+ interrupted: core.interrupted?,
200
+ degraded: core.infrastructure_failure?
196
201
  )
197
202
  [entry, core.interrupted?]
198
203
  end
@@ -247,12 +252,19 @@ module MutationTester
247
252
  end
248
253
 
249
254
  puts Rainbow('=' * 80).bright
255
+ failed = result.processed.reject(&:passed?)
256
+ degraded = failed.select(&:degraded?)
250
257
  if @files && result.processed.empty?
251
258
  puts Rainbow('❌ No files were mutation-tested: every listed file was skipped').red
252
259
  elsif result.success?
253
260
  puts Rainbow('✓ All processed files met the mutation score threshold').green
261
+ elsif degraded.size == failed.size
262
+ puts Rainbow('❌ The failing files produced no scored mutants; this indicates an infrastructure or runner problem, not a test-quality gap').red
254
263
  else
255
264
  puts Rainbow('❌ One or more files did not meet the mutation score threshold').red
265
+ unless degraded.empty?
266
+ puts Rainbow("⚠️ #{degraded.size} of the failing files produced no scored mutants (infrastructure or runner problem); see the per-file output above").yellow
267
+ end
256
268
  end
257
269
 
258
270
  print_survivors(result.survivors)
@@ -3,7 +3,11 @@ require 'etc'
3
3
  module MutationTester
4
4
  class Configuration
5
5
  RUNNER_MODES = %i[auto fork spawn in_memory].freeze
6
+ TIMEOUT_POLICIES = %i[killed separate].freeze
6
7
  AUTO_PARALLEL_CAP = 8
8
+ DEFAULT_TIMEOUT = 30
9
+ DEFAULT_TIMEOUT_FACTOR = 5
10
+ CALIBRATED_TIMEOUT_FLOOR = 5
7
11
 
8
12
  def self.auto_parallel_processes
9
13
  [[Etc.nprocessors, AUTO_PARALLEL_CAP].min, 1].max
@@ -14,9 +18,9 @@ module MutationTester
14
18
  number <= 0 ? '' : (number + 1).to_s
15
19
  end
16
20
 
17
- attr_reader :parallel_processes, :runner, :worker_env_var
21
+ attr_reader :parallel_processes, :runner, :worker_env_var, :timeout, :timeout_factor, :timeout_policy
18
22
 
19
- attr_accessor :timeout,
23
+ attr_accessor :baseline_duration,
20
24
  :baseline_timeout,
21
25
  :mutation_types,
22
26
  :reporters,
@@ -33,7 +37,9 @@ module MutationTester
33
37
  self.parallel_processes = ENV['MUTATION_TESTER_PARALLEL_PROCESSES'] || self.class.auto_parallel_processes
34
38
  self.runner = ENV['MUTATION_TESTER_RUNNER'] || :auto
35
39
  self.worker_env_var = ENV['MUTATION_TESTER_WORKER_ENV']
36
- @timeout = 30
40
+ @timeout = DEFAULT_TIMEOUT
41
+ @timeout_factor = DEFAULT_TIMEOUT_FACTOR
42
+ @timeout_policy = :killed
37
43
  @baseline_timeout = 300
38
44
  @mutation_types = {
39
45
  arithmetic: true,
@@ -73,6 +79,39 @@ module MutationTester
73
79
  @reporters = source.reporters.dup
74
80
  end
75
81
 
82
+ def timeout=(value)
83
+ @timeout_explicit = true
84
+ @timeout = value
85
+ end
86
+
87
+ def timeout_explicitly_set?
88
+ !!@timeout_explicit
89
+ end
90
+
91
+ def timeout_factor=(value)
92
+ factor = coerce_numeric(value)
93
+ unless factor.positive?
94
+ warn "[MutationTester] timeout_factor must be a number greater than 0; got #{value.inspect}, falling back to #{DEFAULT_TIMEOUT_FACTOR}."
95
+ factor = DEFAULT_TIMEOUT_FACTOR
96
+ end
97
+ @timeout_factor = factor
98
+ end
99
+
100
+ def timeout_policy=(value)
101
+ policy = value.to_s.strip.downcase.to_sym
102
+ unless TIMEOUT_POLICIES.include?(policy)
103
+ warn "[MutationTester] timeout_policy must be one of #{TIMEOUT_POLICIES.join(", ")}; got #{value.inspect}, falling back to killed."
104
+ policy = :killed
105
+ end
106
+ @timeout_policy = policy
107
+ end
108
+
109
+ def effective_timeout
110
+ return @timeout if timeout_explicitly_set? || @baseline_duration.nil?
111
+
112
+ [CALIBRATED_TIMEOUT_FLOOR, @timeout_factor * @baseline_duration].max
113
+ end
114
+
76
115
  def parallel_processes=(value)
77
116
  count = value.to_i
78
117
  if count < 1
@@ -101,5 +140,13 @@ module MutationTester
101
140
 
102
141
  { @worker_env_var => self.class.worker_env_value(index) }
103
142
  end
143
+
144
+ private
145
+
146
+ def coerce_numeric(value)
147
+ Float(value)
148
+ rescue ArgumentError, TypeError
149
+ 0
150
+ end
104
151
  end
105
152
  end
@@ -11,6 +11,7 @@ module MutationTester
11
11
  @mutations = []
12
12
  @results = []
13
13
  @parse_failed = false
14
+ @shadow_aborted = false
14
15
  MutationRunner.recover_in_place_backup(@source_file)
15
16
  @original_content = File.read(@source_file)
16
17
  end
@@ -27,11 +28,15 @@ module MutationTester
27
28
  return true
28
29
  end
29
30
 
30
- return false unless shadow_environment_reliable?
31
+ unless shadow_environment_reliable?
32
+ @shadow_aborted = true
33
+ return false
34
+ end
31
35
 
32
36
  run_mutations
33
37
  generate_reports
34
38
  return report_interruption if interrupted?
39
+ return report_infrastructure_failure if infrastructure_failure?
35
40
 
36
41
  check_threshold
37
42
  ensure
@@ -46,7 +51,11 @@ module MutationTester
46
51
  end
47
52
 
48
53
  def mutation_score
49
- Reporters::BaseReporter.score(@results)
54
+ Reporters::BaseReporter.score(@results, policy: @config.timeout_policy)
55
+ end
56
+
57
+ def infrastructure_failure?
58
+ @shadow_aborted || (!@results.empty? && scored_count.zero?)
50
59
  end
51
60
 
52
61
  def threshold_met?
@@ -69,7 +78,9 @@ module MutationTester
69
78
  def run_original_tests
70
79
  puts Rainbow("\n🧪 Running original tests...").yellow
71
80
  command = test_command
81
+ baseline_started = monotonic_time
72
82
  result = command.run(timeout: @config.baseline_timeout, capture: true)
83
+ baseline_elapsed = monotonic_time - baseline_started
73
84
 
74
85
  if result.timed_out?
75
86
  puts Rainbow("❌ Original tests exceeded the baseline deadline of #{@config.baseline_timeout}s and were terminated.").red
@@ -85,9 +96,14 @@ module MutationTester
85
96
  return false
86
97
  end
87
98
  puts Rainbow('✓ Original tests passed').green
99
+ @config.baseline_duration = baseline_elapsed
88
100
  true
89
101
  end
90
102
 
103
+ def monotonic_time
104
+ Process.clock_gettime(Process::CLOCK_MONOTONIC)
105
+ end
106
+
91
107
  def replay_baseline_output(output)
92
108
  return if output.nil? || output.strip.empty?
93
109
 
@@ -178,6 +194,20 @@ module MutationTester
178
194
  false
179
195
  end
180
196
 
197
+ def scored_count
198
+ @results.count { |r| %i[killed timeout survived].include?(Reporters::BaseReporter.status_for(r)) }
199
+ end
200
+
201
+ def status_count(status)
202
+ @results.count { |r| Reporters::BaseReporter.status_for(r) == status }
203
+ end
204
+
205
+ def report_infrastructure_failure
206
+ puts Rainbow("\n❌ Run failed: none of the #{@results.size} mutants could be scored (#{status_count(:error)} error, #{status_count(:stillborn)} stillborn).").red
207
+ puts Rainbow(' This indicates an infrastructure or runner problem (test environment, workspace, or test command), not a test-quality gap.').red
208
+ false
209
+ end
210
+
181
211
  def check_threshold
182
212
  score = mutation_score
183
213
 
@@ -285,7 +285,7 @@ module MutationTester
285
285
  end
286
286
 
287
287
  def run_specs_in_shadow(spec_file, working_dir, example_filters: [])
288
- test_command(spec_file, example_filters: example_filters).run(timeout: @config.timeout, chdir: working_dir)
288
+ test_command(spec_file, example_filters: example_filters).run(timeout: @config.effective_timeout, chdir: working_dir)
289
289
  end
290
290
 
291
291
  def discoverable_project_root
@@ -325,7 +325,7 @@ module MutationTester
325
325
  end
326
326
 
327
327
  def run_specs_in_place(spec_file, example_filters: [])
328
- test_command(spec_file, example_filters: example_filters).run(timeout: @config.timeout)
328
+ test_command(spec_file, example_filters: example_filters).run(timeout: @config.effective_timeout)
329
329
  end
330
330
 
331
331
  def run_mutation_in_memory(mutation, result, project_root = nil)
@@ -335,7 +335,7 @@ module MutationTester
335
335
  outcome = runner.execute_in_memory(
336
336
  source: mutation[:code],
337
337
  path: @source_file,
338
- timeout: @config.timeout,
338
+ timeout: @config.effective_timeout,
339
339
  chdir: Dir.pwd
340
340
  )
341
341
 
@@ -419,7 +419,7 @@ module MutationTester
419
419
  outcome = runner.execute_in_memory(
420
420
  source: @original_content,
421
421
  path: @source_file,
422
- timeout: @config.timeout,
422
+ timeout: @config.effective_timeout,
423
423
  chdir: Dir.pwd
424
424
  )
425
425
  return nil if outcome.status == 'pass'
@@ -3,12 +3,14 @@ module MutationTester
3
3
  class BaseReporter
4
4
  attr_reader :results, :source_file, :spec_file, :config
5
5
 
6
- def self.score(results)
7
- effective_killed = results.count { |r| %i[killed timeout].include?(status_for(r)) }
8
- scored = results.count { |r| %i[killed timeout survived].include?(status_for(r)) }
6
+ def self.score(results, policy: :killed)
7
+ killing_statuses = policy == :separate ? %i[killed] : %i[killed timeout]
8
+ scoring_statuses = killing_statuses + %i[survived]
9
+ kills = results.count { |r| killing_statuses.include?(status_for(r)) }
10
+ scored = results.count { |r| scoring_statuses.include?(status_for(r)) }
9
11
  return 0.0 if scored.zero?
10
12
 
11
- (effective_killed.to_f / scored * 100).round(2)
13
+ (kills.to_f / scored * 100).round(2)
12
14
  end
13
15
 
14
16
  def self.status_for(result)
@@ -62,6 +64,8 @@ module MutationTester
62
64
  end
63
65
 
64
66
  def effective_killed_count
67
+ return killed_count if @config.timeout_policy == :separate
68
+
65
69
  killed_count + timeout_count
66
70
  end
67
71
 
@@ -70,7 +74,7 @@ module MutationTester
70
74
  end
71
75
 
72
76
  def mutation_score
73
- self.class.score(@results)
77
+ self.class.score(@results, policy: @config.timeout_policy)
74
78
  end
75
79
 
76
80
  def quality_rating
@@ -53,7 +53,7 @@ module MutationTester
53
53
  end
54
54
 
55
55
  def aggregate_score
56
- BaseReporter.score(@result.processed.flat_map { |entry| entry.results || [] })
56
+ BaseReporter.score(@result.processed.flat_map { |entry| entry.results || [] }, policy: @config.timeout_policy)
57
57
  end
58
58
 
59
59
  def file_reports
@@ -1,3 +1,3 @@
1
1
  module MutationTester
2
- VERSION = '1.2.0'.freeze
2
+ VERSION = '1.3.0'.freeze
3
3
  end
@@ -45,7 +45,7 @@ namespace :mutation do
45
45
  end
46
46
 
47
47
  unless all_passed
48
- puts Rainbow("\n❌ One or more files did not meet the mutation score threshold").red
48
+ puts Rainbow("\n❌ One or more files failed mutation testing (see the per-file output above)").red
49
49
  abort
50
50
  end
51
51
  end
data/readme.md CHANGED
@@ -206,6 +206,8 @@ mutation_test [OPTIONS] --glob 'lib/**/*.rb'
206
206
  | `--spec-glob TEMPLATE` | Spec-mapping template with a `{name}` placeholder (default: `spec/{name}_spec.rb`). Requires a positional `FILE` list, `--staged`, or `--glob`. |
207
207
  | `--since REV` | Incremental batch mode: mutate only the files matched by `--glob` that changed since git revision `REV` (new files count as changed). Requires `--glob`. See [Incremental mode](#incremental-mode-mutate-only-what-changed). |
208
208
  | `--fail-fast` | Stop the run at the first surviving mutant and finish with a failing status. Works in single-file mode and with `--glob`. |
209
+ | `--timeout-factor N` | Per-mutant timeout budget as `N` times the measured baseline test run, never below 5 s (default: 5, must be > 0). Ignored when `config.timeout` is set explicitly, which keeps a fixed budget. See [Configuration](#configuration). |
210
+ | `--timeout-policy MODE` | Scoring policy for timed-out mutants: `killed` (default) counts a timeout as a kill; `separate` keeps timeouts out of the score entirely (`killed / (killed + survived)`) and reports them only as their own category in the console, JSON and HTML reports. |
209
211
  | `--worker-env NAME` | Set environment variable `NAME` to a distinct per-worker value before each parallel worker boots (`parallel_tests` `TEST_ENV_NUMBER` convention: worker 0 -> `""`, worker N -> `N+1`), so a `parallel_tests`-style `database.yml` selects a per-worker database. You provision the databases (e.g. `rake parallel:prepare`). Not supported by the `in_memory` runner (it falls back to `fork`). See [Making parallelism work with Rails](#making-parallelism-work-with-rails). |
210
212
  | `--strict-equality` | Enable the opt-in strict-equality probes (`==` → `eql?` and `==` → `equal?`). Default off; expect noise on code that does not distinguish numeric types or object identity. See [Strict Equality Mutations](docs/mutation-types.md#strict-equality-mutations-opt-in). |
211
213
  | `-h, --help` | Show help message. |
@@ -223,6 +225,22 @@ bundle exec mutation_test --reporters json,html --output-dir build/mutation \
223
225
  app/models/user.rb spec/models/user_spec.rb
224
226
  ```
225
227
 
228
+ ### Exit codes (single-file mode)
229
+
230
+ - `0` - the run passed (score met the threshold, or `fail_on_threshold` is disabled).
231
+ - `1` - the mutation score is below the threshold, or the input is unusable (missing
232
+ file, unknown reporter, source with a syntax error).
233
+ - `2` - a usage error (conflicting flags; see the batch sections below).
234
+ - `3` - the run aborted or degraded before reaching a verdict: the shadow workspace
235
+ was unreliable, or every mutant ended as `error`/`stillborn` so nothing was scored.
236
+ This signals an infrastructure or runner problem, not a test-quality gap, so CI
237
+ hooks can distinguish it from a genuine threshold failure.
238
+ - `130` - interrupted with Ctrl+C.
239
+
240
+ Batch modes (`FILE...` lists, `--staged`, `--glob`) keep the exit codes documented in
241
+ their sections below (`0`/`1`/`2`); a degraded file is named explicitly in the batch
242
+ summary instead of being blamed on the threshold.
243
+
226
244
  ### Running with rake
227
245
 
228
246
  You can also run mutation tests through rake tasks.
@@ -403,8 +421,24 @@ MutationTester.configure do |config|
403
421
  # config.parallel_processes = 1
404
422
  config.parallel_processes = 4
405
423
 
406
- # Timeout for each test run (seconds)
407
- config.timeout = 30
424
+ # Per-mutant timeout (seconds). Calibrated automatically by default:
425
+ # max(5, timeout_factor * measured baseline duration), so a loaded machine
426
+ # cannot inflate the score by turning healthy-but-slow runs into timeout
427
+ # kills; on code paths without a measured baseline the fixed default of 30
428
+ # applies. Setting config.timeout explicitly (a number, or nil for no
429
+ # deadline at all) disables calibration and keeps that fixed budget:
430
+ # config.timeout = 30
431
+
432
+ # Multiplier for the baseline-calibrated per-mutant timeout (> 0; an
433
+ # invalid value falls back to the default with a warning):
434
+ config.timeout_factor = 5
435
+
436
+ # Scoring policy for timed-out mutants. :killed (default) counts a timeout
437
+ # as a kill: (killed + timeout) / (killed + timeout + survived). :separate
438
+ # keeps timeouts out of the score entirely, killed / (killed + survived),
439
+ # and reports them only as their own category, so timeouts under load can
440
+ # never raise the score:
441
+ config.timeout_policy = :killed
408
442
 
409
443
  # Mutant execution runner: :auto (default), :fork, :spawn or :in_memory.
410
444
  # :auto picks the fastest safe path and announces any fallback on stderr;
@@ -725,7 +759,9 @@ to get a single, clean JSON document on **stdout** and nothing else: the banner,
725
759
  progress spinner, colours and the "report saved" notice all go to **stderr**, so
726
760
  the stream is safe to pipe straight into `jq` or a parser. The process still
727
761
  exits `0` when the mutation score meets the configured threshold and `1` when it
728
- does not, so the exit code remains a pass/fail signal.
762
+ does not (a degraded single-file run exits `3`, see
763
+ [Exit codes](#exit-codes-single-file-mode)), so the exit code remains a
764
+ pass/fail signal.
729
765
 
730
766
  ```bash
731
767
  bundle exec mutation_test --json examples/calculator.rb examples/calculator_spec.rb | jq .
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: mutation_tester
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.2.0
4
+ version: 1.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kamil Dzierbicki
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2026-07-14 00:00:00.000000000 Z
11
+ date: 2026-07-15 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: parallel