mutineer 1.8.1 → 1.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +733 -0
  3. data/README.md +85 -50
  4. data/lib/mutineer/changed_lines.rb +2 -1
  5. data/lib/mutineer/child_cause.rb +149 -0
  6. data/lib/mutineer/child_stdout.rb +12 -1
  7. data/lib/mutineer/cli.rb +540 -59
  8. data/lib/mutineer/command_redact.rb +38 -0
  9. data/lib/mutineer/concern_blocks.rb +95 -24
  10. data/lib/mutineer/config.rb +112 -6
  11. data/lib/mutineer/coverage_map.rb +1345 -156
  12. data/lib/mutineer/daemon_backend.rb +41 -18
  13. data/lib/mutineer/daemon_client.rb +104 -21
  14. data/lib/mutineer/daemon_server.rb +85 -14
  15. data/lib/mutineer/database_urls.rb +189 -0
  16. data/lib/mutineer/external_backend.rb +93 -6
  17. data/lib/mutineer/fs_path.rb +125 -0
  18. data/lib/mutineer/isolation.rb +380 -25
  19. data/lib/mutineer/job_plan.rb +352 -41
  20. data/lib/mutineer/minitest_integration/file_order.rb +56 -0
  21. data/lib/mutineer/minitest_integration/in_test_database_error.rb +243 -19
  22. data/lib/mutineer/minitest_integration.rb +26 -58
  23. data/lib/mutineer/mutant_id.rb +4 -1
  24. data/lib/mutineer/mutation.rb +10 -4
  25. data/lib/mutineer/mutator_registry.rb +6 -3
  26. data/lib/mutineer/mutators/collection_method.rb +195 -6
  27. data/lib/mutineer/mutators/literal_mutation.rb +33 -3
  28. data/lib/mutineer/mutators/range_literal.rb +78 -29
  29. data/lib/mutineer/mutators/rescue_body.rb +126 -0
  30. data/lib/mutineer/mutators/return_nil.rb +23 -6
  31. data/lib/mutineer/output_filter.rb +451 -0
  32. data/lib/mutineer/pairing.rb +463 -31
  33. data/lib/mutineer/parser.rb +53 -0
  34. data/lib/mutineer/project_path.rb +15 -8
  35. data/lib/mutineer/rails_worker_db.rb +366 -6
  36. data/lib/mutineer/reach_probe.rb +229 -0
  37. data/lib/mutineer/redaction.rb +301 -41
  38. data/lib/mutineer/reload_guards.rb +985 -0
  39. data/lib/mutineer/reporter.rb +467 -58
  40. data/lib/mutineer/result.rb +6 -7
  41. data/lib/mutineer/run_seed.rb +16 -0
  42. data/lib/mutineer/runner.rb +194 -55
  43. data/lib/mutineer/source_bytes.rb +187 -0
  44. data/lib/mutineer/statement_lines.rb +108 -14
  45. data/lib/mutineer/test_runners/rspec.rb +118 -10
  46. data/lib/mutineer/test_runners/rspec_seed.rb +151 -0
  47. data/lib/mutineer/test_runners.rb +3 -2
  48. data/lib/mutineer/triage.rb +55 -10
  49. data/lib/mutineer/version.rb +1 -1
  50. data/lib/mutineer.rb +1 -0
  51. data/skills/mutineer/SKILL.md +12 -4
  52. metadata +13 -1
@@ -27,31 +27,125 @@ module Mutineer
27
27
  # counts when the file loads, and `x if done; y(` counts when `y` does
28
28
  # not run.
29
29
  def self.for(def_node, source, offset, loose: false)
30
- return [] unless def_node.body
30
+ statement = statement_at(def_node, offset, conditional: !loose)
31
+ return [] unless statement
32
+
33
+ location = statement.location
34
+ lines = (location.start_line..last_line(statement)).to_a
35
+ return lines - [def_node.location.start_line] if loose
36
+
37
+ strict_lines(source, location, lines)
38
+ end
39
+
40
+ # Lines of the statement that {for} drops because they share the line with
41
+ # other code, when that drop leaves no line. Empty when {for} keeps a line,
42
+ # when no statement holds the offset, or when the code runs only sometimes
43
+ # (a branch, the right side of `&&`, the body of `x if c`): those lines can
44
+ # count without this code (#187, #254).
45
+ #
46
+ # A one-line block on a call that ran at load has no strict line of its own.
47
+ # The call's line can count without the block: `each` on an empty array
48
+ # still counts the line. {JobPlan.ran_at_load?} does not treat that as the
49
+ # block having run. It uses these lines only for a `define_method` or
50
+ # `define_singleton_method` call (#435).
51
+ #
52
+ # @param def_node [Prism::DefNode] the method that holds the position.
53
+ # @param source [String] the source text of the file.
54
+ # @param offset [Integer] byte offset of the position.
55
+ # @return [Array<Integer>] the dropped lines, or an empty array.
56
+ def self.shared_lines(def_node, source, offset)
57
+ statement = statement_at(def_node, offset, conditional: true)
58
+ return [] unless statement
59
+
60
+ location = statement.location
61
+ lines = (location.start_line..last_line(statement)).to_a
62
+ return [] unless strict_lines(source, location, lines.dup).empty?
63
+
64
+ lines
65
+ end
66
+
67
+ # The statement that holds `offset`, or nil when none does. With
68
+ # `conditional`, also nil when the code runs only sometimes ({sometimes?}
69
+ # or a modifier such as `x if c`).
70
+ #
71
+ # @param def_node [Prism::DefNode, nil] the method that holds the position.
72
+ # @param offset [Integer] byte offset of the position.
73
+ # @param conditional [Boolean] drop code that runs only sometimes.
74
+ # @return [Prism::Node, nil]
75
+ def self.statement_at(def_node, offset, conditional:)
76
+ return nil unless def_node&.body
31
77
 
32
78
  path = path_to(def_node.body, offset) || []
33
79
  index = path.rindex(&:newline?)
34
- return [] unless index
80
+ return nil unless index
81
+ return nil if conditional && conditional_code?(path, index)
35
82
 
36
- statement = path[index]
37
- unless loose
38
- # Code that runs only sometimes inside the statement: the tests that ran
39
- # the statement need not have run it.
40
- return [] if path[index..].each_cons(2).any? { |parent, child| sometimes?(parent, child) }
83
+ path[index]
84
+ end
85
+ private_class_method :statement_at
41
86
 
87
+ # Whether the statement at `index` runs only sometimes when its parent runs.
88
+ #
89
+ # @param path [Array<Prism::Node>] nodes from the method body down to the offset.
90
+ # @param index [Integer] index of the statement on `path`.
91
+ # @return [Boolean]
92
+ def self.conditional_code?(path, index)
93
+ # Code that runs only sometimes inside the statement: the tests that ran
94
+ # the statement need not have run it.
95
+ path[index..].each_cons(2).any? { |parent, child| sometimes?(parent, child) } ||
42
96
  # Ruby counts the line of `x while c` or `x if c` when it checks `c`,
43
97
  # also when `x` never runs. The path is the modifier, its body, statement.
44
- return [] if index >= 2 && modifier?(path[index - 2])
98
+ (index >= 2 && modifier?(path[index - 2]))
99
+ end
100
+ private_class_method :conditional_code?
101
+
102
+ # `lines` with a first or last line removed when that line holds other code.
103
+ # Ruby can count the line without this statement: the `def` line counts when
104
+ # the file loads, and `x if done; y(` counts when `y` does not run.
105
+ #
106
+ # @param source [String] the source text of the file.
107
+ # @param location [Prism::Location] the statement's location.
108
+ # @param lines [Array<Integer>] the statement's lines; modified in place.
109
+ # @return [Array<Integer>] `lines`.
110
+ def self.strict_lines(source, location, lines)
111
+ lines.delete(location.start_line) unless before(source, location).strip.empty?
112
+ rest = after(source, location).strip
113
+ lines.delete(location.end_line) unless rest.empty? || rest.start_with?("#")
114
+ lines
115
+ end
116
+ private_class_method :strict_lines
117
+
118
+ # Calls that store a block as a method. The block does not run when the
119
+ # call does, so a mutant in the call or in the block is the original method
120
+ # once that call has run at load (#435).
121
+ STORED_METHOD_CALLS = %i[define_method define_singleton_method].freeze
122
+
123
+ # Lines of the nearest {STORED_METHOD_CALLS} call that holds `offset`: the
124
+ # call's strict lines, or the line it shares with other code when strict
125
+ # mode drops every line. Empty when `offset` is not in such a call, or when
126
+ # that call runs only sometimes. A line here ran only if the caller recorded
127
+ # it; an empty `each` on the same line can count without this call (#286).
128
+ #
129
+ # @param def_node [Prism::DefNode] the method that holds the position.
130
+ # @param source [String] the source text of the file.
131
+ # @param offset [Integer] byte offset of the position.
132
+ # @return [Array<Integer>]
133
+ def self.stored_method_lines(def_node, source, offset)
134
+ return [] unless def_node&.body
135
+
136
+ path = path_to(def_node.body, offset)
137
+ call = path&.reverse_each&.find do |node|
138
+ node.is_a?(Prism::CallNode) && STORED_METHOD_CALLS.include?(node.name)
45
139
  end
140
+ return [] unless call
141
+
142
+ statement = statement_at(def_node, call.location.start_offset, conditional: true)
143
+ return [] unless statement
46
144
 
47
145
  location = statement.location
48
146
  lines = (location.start_line..last_line(statement)).to_a
49
- return lines - [def_node.location.start_line] if loose
50
-
51
- lines.delete(location.start_line) unless before(source, location).strip.empty?
52
- after = after(source, location).strip
53
- lines.delete(location.end_line) unless after.empty? || after.start_with?("#")
54
- lines
147
+ kept = strict_lines(source, location, lines.dup)
148
+ kept.empty? ? lines : kept
55
149
  end
56
150
 
57
151
  # Whether the code at `offset` runs each time the method runs: it is in
@@ -1,7 +1,11 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "stringio"
4
+ require_relative "../fs_path"
5
+ require_relative "../child_cause"
4
6
  require_relative "../kill_channel"
7
+ require_relative "rspec_seed"
8
+ require_relative "../minitest_integration/in_test_database_error"
5
9
 
6
10
  module Mutineer
7
11
  # Raised when the target project asks for a framework whose gem isn't present.
@@ -12,9 +16,18 @@ module Mutineer
12
16
  # Child-process-only RSpec runner.
13
17
  #
14
18
  # Mirrors MinitestIntegration's contract: run the given spec files and
15
- # return 0 (all passed), 1 (an example failed) or 2 (only an error
16
- # outside examples, such as a spec file that raised while loading).
19
+ # return 0 (all passed), 1 (an example failed) or 2 (no verdict). Exit 2
20
+ # is an error outside examples, such as a spec file that raised while
21
+ # loading, or a run whose first failed example failed with an Active
22
+ # Record connection or lock error (#347).
17
23
  module RSpec
24
+ # The sink text from the most recent {.run} (#360 clean-failure detail).
25
+ #
26
+ # @return [String, nil]
27
+ def self.last_output
28
+ @last_output
29
+ end
30
+
18
31
  # Runs the given RSpec files.
19
32
  #
20
33
  # @param spec_files [String, Array<String>] one file or many files.
@@ -30,6 +43,7 @@ module Mutineer
30
43
  end
31
44
 
32
45
  require_rspec!
46
+ RSpecSeed.install!
33
47
 
34
48
  ::RSpec::Core::Runner.disable_autorun!
35
49
  # Not RSpec.reset: it also drops the settings that gems added when Bundler
@@ -45,36 +59,109 @@ module Mutineer
45
59
  # silenced here: the fork boundary that calls this method has already
46
60
  # pointed stdout at File::NULL (see ChildStdout).
47
61
  sink = StringIO.new
48
- keep_file_order(spec_files) if stop_at_first_failure || record_to
62
+ # RSpec matches the spec path with a UTF-8 regexp. A binary path
63
+ # under LC_ALL=C raises before the example runs (#346).
64
+ files = Array(spec_files).map { |file| FsPath.read(file) }
65
+ # Clean checks pass no hook. They still use this file order, so they
66
+ # share it with the mutant run (#282).
67
+ keep_file_order(files)
49
68
  args = ["--no-color"]
50
69
  args << "--fail-fast" if stop_at_first_failure
51
70
  status = with_fail_fast_off(record_to) do
52
- ::RSpec::Core::Runner.run([*args, *Array(spec_files)], sink, sink)
71
+ ::RSpec::Core::Runner.run([*args, *files], sink, sink)
53
72
  end
73
+ @last_output = sink.string
54
74
  # Reached only when the suite returned: an example that exits the
55
75
  # process skips it, and the parent then keeps the row incomplete. A
56
76
  # return alone proves nothing: fail-fast turned on mid-run stops RSpec
57
77
  # early, so `end` needs a reported outcome for every planned example.
58
78
  KillChannel.write_end(record_to) if record_to && KillFormatter.last&.saw_every_example?
59
79
 
60
- status.zero? ? 0 : failure_code
80
+ return 0 if status.zero?
81
+
82
+ code = failure_code
83
+ # Fixed cause when RSpec failed outside examples (#417 / KTD5). Status 2
84
+ # is also an in-example database error (#347), so test the predicate,
85
+ # not the status. Write here so failure_code stays a pure status helper.
86
+ ChildCause.write_text(ChildCause::RSPEC_OUTSIDE_EXAMPLES) if outside_examples?
87
+ code
61
88
  ensure
62
89
  FileOrder.rank = nil
63
90
  end
64
91
 
65
92
  # The exit status of a failed run: 2 when no example failed and RSpec
66
- # recorded an error outside examples, else 1.
93
+ # recorded an error outside examples, or when the first failed example's
94
+ # own exception is an Active Record connection or lock error (#347).
95
+ # Otherwise 1.
96
+ #
97
+ # A later `after` hook is not the example's own exception. RSpec keeps
98
+ # the first one and wraps a later hook error around it, the same way
99
+ # Minitest records a teardown error after the setup or body error.
67
100
  #
68
101
  # @api private
69
102
  # @return [Integer] 1 or 2.
70
103
  def self.failure_code
104
+ return 2 if outside_examples?
105
+
106
+ if (error = first_failed_database_error)
107
+ MinitestIntegration::InTestDatabaseError.warn_database_error(error)
108
+ return 2
109
+ end
110
+
111
+ 1
112
+ end
113
+
114
+ # True when RSpec recorded an error outside examples and no example failed.
115
+ #
116
+ # Exit status 2 is also an in-example database error ({.failure_code}).
117
+ # A cause line for the outside-examples case has to test this predicate,
118
+ # not a status of 2. Pull request #443 writes that cause when the status
119
+ # is 2; after this change that condition would label a database error as
120
+ # an error outside examples (#347, #417).
121
+ #
122
+ # @api private
123
+ # @return [Boolean]
124
+ def self.outside_examples?
71
125
  world = ::RSpec.world
72
- outside_examples = world.respond_to?(:non_example_failure) && world.non_example_failure
73
- outside_examples && ::RSpec.configuration.reporter.failed_examples.empty? ? 2 : 1
126
+ outside = world.respond_to?(:non_example_failure) && world.non_example_failure
127
+ !!(outside && ::RSpec.configuration.reporter.failed_examples.empty?)
128
+ end
129
+
130
+ # The database error of the first failed example, in run order, or nil
131
+ # when that example failed for another reason.
132
+ #
133
+ # @api private
134
+ # @return [Exception, nil]
135
+ def self.first_failed_database_error
136
+ example = ::RSpec.configuration.reporter.failed_examples.first
137
+ return unless example
138
+
139
+ error = primary_exception(example.exception)
140
+ error if MinitestIntegration::InTestDatabaseError.database_exception?(error)
141
+ rescue StandardError
142
+ nil
143
+ end
144
+
145
+ # The exception that ended the example, before a later hook error.
146
+ # `RSpec::Core::MultipleExceptionError` keeps that first exception at
147
+ # the front of `all_exceptions` and appends each later hook error.
148
+ #
149
+ # @api private
150
+ # @param error [Exception, nil]
151
+ # @return [Exception, nil]
152
+ def self.primary_exception(error)
153
+ nested = ::RSpec::Core::MultipleExceptionError
154
+ return error.all_exceptions.first if error.is_a?(nested) && error.respond_to?(:all_exceptions)
155
+
156
+ error
157
+ rescue StandardError
158
+ error
74
159
  end
75
160
 
76
161
  # Runs the top-level example groups in the order of `spec_files` (#203),
77
162
  # after RSpec's own ordering, which then decides the order within a file.
163
+ # Every run uses it, including the clean check, so a mutant run and the
164
+ # clean check of the same files share that order (#282).
78
165
  #
79
166
  # @api private
80
167
  # @param spec_files [String, Array<String>] the files, in run order.
@@ -238,7 +325,8 @@ module Mutineer
238
325
  # @param notification [RSpec::Core::Notifications::FailedExampleNotification]
239
326
  # @return [void]
240
327
  def example_failed(notification)
241
- send_event(KillChannel::KILL, notification.example)
328
+ example = notification.example
329
+ send_event(example_event(example), example)
242
330
  example_finished
243
331
  @failed = true
244
332
  end
@@ -302,9 +390,29 @@ module Mutineer
302
390
  KillChannel.write_unskip(@channel)
303
391
  end
304
392
 
393
+ # {KillChannel::UNAVAILABLE} when the example's own exception is a
394
+ # database error, otherwise {KillChannel::KILL}. A fault here is a
395
+ # kill: recording nothing would hide a failure, and raising would
396
+ # change the verdict.
397
+ #
398
+ # @param example [RSpec::Core::Example] the failed example.
399
+ # @return [String]
400
+ def example_event(example)
401
+ error = example.exception if example.respond_to?(:exception)
402
+ error = Mutineer::TestRunners::RSpec.primary_exception(error)
403
+ if error && Mutineer::MinitestIntegration::InTestDatabaseError.database_exception?(error)
404
+ KillChannel::UNAVAILABLE
405
+ else
406
+ KillChannel::KILL
407
+ end
408
+ rescue StandardError
409
+ KillChannel::KILL
410
+ end
411
+
305
412
  # Writes one test line for `example`.
306
413
  #
307
- # @param event [String] {KillChannel::PASS} or {KillChannel::KILL}.
414
+ # @param event [String] {KillChannel::PASS}, {KillChannel::KILL}, or
415
+ # {KillChannel::UNAVAILABLE}.
308
416
  # @param example [RSpec::Core::Example] the example.
309
417
  # @return [void]
310
418
  def send_event(event, example)
@@ -0,0 +1,151 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative "../run_seed"
4
+
5
+ module Mutineer
6
+ module TestRunners
7
+ # Gives a random RSpec order the same seed on the clean check and the
8
+ # mutant runs (#282). A defined order is left as it is: assigning
9
+ # `config.seed` would switch the suite to random. A seed the suite sets
10
+ # (`--seed`, `--order rand:N`, `SPEC_OPTS`, `.rspec`, or `config.seed=`)
11
+ # wins. The hooks install before boot, `--require` files, and spec files
12
+ # load, so a `config.seed=` in any of those is visible. The seed is
13
+ # written after spec files load and before RSpec orders examples.
14
+ #
15
+ # A spec helper that only reads `config.seed` (the generated
16
+ # `Kernel.srand config.seed`) does not count as setting it. The read
17
+ # still sees RSpec's temporary seed; the examples then run under the
18
+ # pinned seed.
19
+ module RSpecSeed
20
+ # Installs the hooks. Idempotent. Requires `rspec/core`, which the
21
+ # caller has already loaded.
22
+ #
23
+ # @api private
24
+ # @return [void]
25
+ def self.install!
26
+ return if ::RSpec::Core::Configuration.ancestors.include?(Watch)
27
+
28
+ ::RSpec::Core::Configuration.prepend(Watch)
29
+ ::RSpec::Core::World.prepend(PinBeforeOrder)
30
+ end
31
+
32
+ # True while {pin!} is assigning the shared seed, so that assignment
33
+ # is not recorded as the suite's own seed.
34
+ #
35
+ # @api private
36
+ # @return [Boolean]
37
+ def self.pinning?
38
+ @pinning == true
39
+ end
40
+
41
+ # True when `value` is an order string that carries a seed, such as
42
+ # `"rand:12"` or `"random:12"`. RSpec stores `--seed N` as
43
+ # `"rand:N"`.
44
+ #
45
+ # @api private
46
+ # @param value [Object]
47
+ # @return [Boolean]
48
+ def self.embedded_seed?(value)
49
+ _order, seed = value.to_s.split(":", 2)
50
+ !seed.nil? && !seed.empty? && seed.match?(/\A\d+\z/)
51
+ end
52
+
53
+ # The seed named in the not-green message. `SPEC_OPTS` is what the
54
+ # parent can see. A `.rspec` file or `config.seed` is applied in the
55
+ # child and is not part of this string.
56
+ #
57
+ # @api private
58
+ # @return [String]
59
+ def self.advertised_seed
60
+ opts = ENV.fetch("SPEC_OPTS", "")
61
+ found = []
62
+ opts.scan(/--seed(?:=|\s+)(\d+)|--order(?:=|\s+)rand(?:om)?:(\d+)/) do |seed, order|
63
+ found << (seed || order)
64
+ end
65
+ found.last || RunSeed::PINNED_SEED.to_s
66
+ end
67
+
68
+ # Sets the shared seed when the loaded suite's order is random and the
69
+ # suite did not set a seed. Re-seeds Ruby's global generator so
70
+ # `rand` in examples matches, after a spec helper's `Kernel.srand`.
71
+ #
72
+ # @api private
73
+ # @param config [RSpec::Core::Configuration]
74
+ # @return [void]
75
+ def self.pin!(config)
76
+ return if config.instance_variable_get(:@mutineer_seed_explicit)
77
+ return unless random_order?(config)
78
+
79
+ @pinning = true
80
+ begin
81
+ config.seed = RunSeed::PINNED_SEED
82
+ Kernel.srand(config.seed)
83
+ ensure
84
+ @pinning = false
85
+ end
86
+ end
87
+
88
+ # True when the global ordering strategy is RSpec's random one.
89
+ #
90
+ # @api private
91
+ # @param config [RSpec::Core::Configuration]
92
+ # @return [Boolean]
93
+ def self.random_order?(config)
94
+ registry = config.ordering_registry
95
+ registry.fetch(:global).equal?(registry.fetch(:random))
96
+ end
97
+
98
+ # Prepended on `RSpec::Core::Configuration`. Records a seed the suite
99
+ # sets, from the command, `SPEC_OPTS`, `.rspec`, or the spec helper.
100
+ module Watch
101
+ # CLI, `SPEC_OPTS` and `.rspec` arrive here. `--seed` is passed as
102
+ # an order of `rand:N`.
103
+ #
104
+ # @param hash [Hash]
105
+ # @return [void]
106
+ def force(hash)
107
+ mark_seed_explicit if hash.key?(:seed) || RSpecSeed.embedded_seed?(hash[:order])
108
+ super
109
+ end
110
+
111
+ # A spec helper's `config.seed =`.
112
+ #
113
+ # @param value [Object]
114
+ # @return [void]
115
+ def seed=(value)
116
+ mark_seed_explicit unless RSpecSeed.pinning?
117
+ super
118
+ end
119
+
120
+ # A spec helper's `config.order = "rand:N"`.
121
+ #
122
+ # @param value [Object]
123
+ # @return [void]
124
+ def order=(value)
125
+ mark_seed_explicit if RSpecSeed.embedded_seed?(value)
126
+ super
127
+ end
128
+
129
+ # Records that this configuration object set its own seed.
130
+ #
131
+ # @api private
132
+ # @return [void]
133
+ def mark_seed_explicit
134
+ @mutineer_seed_explicit = true
135
+ end
136
+ end
137
+
138
+ # Prepended on `RSpec::Core::World`. Pins the seed after spec files
139
+ # load and before examples are ordered.
140
+ module PinBeforeOrder
141
+ # The groups RSpec is about to run.
142
+ #
143
+ # @return [Array<RSpec::Core::ExampleGroup>]
144
+ def ordered_example_groups
145
+ RSpecSeed.pin!(::RSpec.configuration)
146
+ super
147
+ end
148
+ end
149
+ end
150
+ end
151
+ end
@@ -9,8 +9,9 @@ module Mutineer
9
9
  # Each runner responds to
10
10
  # `.run(files, stop_at_first_failure: false, record_to: nil) -> 0/1/2`
11
11
  # (0 = all passed, 1 = any failure, 2 = no verdict: an error outside tests,
12
- # which the RSpec runner reports, or a Minitest run whose first failure was
13
- # an Active Record connection or lock error, #296) and is called only inside
12
+ # which the RSpec runner reports, or a run whose first failure was an Active
13
+ # Record connection or lock error, on Minitest and on RSpec, #296, #347) and
14
+ # is called only inside
14
15
  # a forked child.
15
16
  # Only the mutant path passes `stop_at_first_failure: true`; a `--matrix`
16
17
  # mutant run passes `record_to:` instead (a {KillChannel} pipe) and runs every
@@ -5,9 +5,10 @@ require "optparse"
5
5
  require "yaml"
6
6
 
7
7
  module Mutineer
8
- # Prints `.mutineer.yml` `ignore:` entries for survivors in a JSON report.
9
- # The command writes to stdout and does not edit files. The user pastes the
10
- # lines under `ignore:`.
8
+ # Prints `.mutineer.yml` `ignore:` entries for survivors in a JSON report,
9
+ # and for each copy dropped as a repeat of one of those survivors (#374).
10
+ # An entry still ignores only its own id. The command writes to stdout and
11
+ # does not edit files. The user pastes the lines under `ignore:`.
11
12
  module Triage
12
13
  # Usage line for a bad invocation.
13
14
  USAGE = "Usage: mutineer triage REPORT.json --reason TEXT (--id ID ... | --all)"
@@ -59,17 +60,35 @@ module Mutineer
59
60
  return 2
60
61
  end
61
62
 
62
- survivors = survivor_ids(report, err)
63
- return 2 if survivors.nil?
63
+ groups = survivor_groups(report, err)
64
+ return 2 if groups.nil?
64
65
 
65
- chosen = all ? survivors : ids
66
- missing = chosen.reject { |id| survivors.include?(id) }
66
+ by_id = groups.to_h
67
+ copy_owner = {}
68
+ groups.each { |id, copies| copies.each { |copy| copy_owner[copy] ||= id } }
69
+
70
+ chosen = all ? groups.map(&:first) : ids
71
+ missing = chosen.reject { |id| by_id.key?(id) || copy_owner.key?(id) }
67
72
  unless missing.empty?
68
- err.puts "mutineer: triage id is not a survivor: #{missing.join(', ')}"
73
+ err.puts "mutineer: triage id is not a survivor or one of its copies: #{missing.join(', ')}"
69
74
  return 2
70
75
  end
71
76
 
72
- entries = chosen.uniq.map { |id| { "id" => id, "reason" => text } }
77
+ seen_owners = {}
78
+ seen_entry_ids = {}
79
+ entries = []
80
+ chosen.uniq.each do |id|
81
+ owner = by_id.key?(id) ? id : copy_owner[id]
82
+ next if seen_owners[owner]
83
+
84
+ seen_owners[owner] = true
85
+ ([owner] + by_id[owner]).each do |entry_id|
86
+ next if seen_entry_ids[entry_id]
87
+
88
+ seen_entry_ids[entry_id] = true
89
+ entries << { "id" => entry_id, "reason" => text }
90
+ end
91
+ end
73
92
  if entries.empty?
74
93
  err.puts "mutineer: triage has nothing to paste under ignore:"
75
94
  return 0
@@ -87,6 +106,18 @@ module Mutineer
87
106
  # @param err [IO] usage errors.
88
107
  # @return [Array<String>, nil]
89
108
  def self.survivor_ids(path, err)
109
+ groups = survivor_groups(path, err)
110
+ groups&.map(&:first)
111
+ end
112
+
113
+ # Each survivor id and the dropped-copy ids listed on it, in report order.
114
+ # A `copies` value that is not a list counts as none. Nil when the file
115
+ # cannot be used, after a message on `err`.
116
+ #
117
+ # @param path [String] report path.
118
+ # @param err [IO] usage errors.
119
+ # @return [Array<Array(String, Array<String>)>, nil]
120
+ def self.survivor_groups(path, err)
90
121
  unless File.file?(path)
91
122
  err.puts "mutineer: triage report not found: #{path}"
92
123
  return nil
@@ -99,7 +130,7 @@ module Mutineer
99
130
  return nil
100
131
  end
101
132
 
102
- rows.filter_map { |row| row["id"] if row.is_a?(Hash) && row["id"].is_a?(String) }
133
+ rows.filter_map { |row| group_for(row) }
103
134
  rescue JSON::ParserError => e
104
135
  err.puts "mutineer: triage report is not JSON: #{e.message}"
105
136
  nil
@@ -107,5 +138,19 @@ module Mutineer
107
138
  err.puts "mutineer: triage report cannot be read: #{path}"
108
139
  nil
109
140
  end
141
+
142
+ # One survivor and its copy ids, or nil when the row has no id.
143
+ #
144
+ # @api private
145
+ # @param row [Object] one `survivors[]` element.
146
+ # @return [Array(String, Array<String>), nil]
147
+ def self.group_for(row)
148
+ return unless row.is_a?(Hash) && row["id"].is_a?(String) && !row["id"].empty?
149
+
150
+ raw = row["copies"]
151
+ copies = raw.is_a?(Array) ? raw : []
152
+ copies = copies.select { |id| id.is_a?(String) && !id.empty? && id != row["id"] }.uniq
153
+ [row["id"], copies]
154
+ end
110
155
  end
111
156
  end
@@ -2,5 +2,5 @@
2
2
 
3
3
  module Mutineer
4
4
  # Current Mutineer release version.
5
- VERSION = "1.8.1"
5
+ VERSION = "1.9.0"
6
6
  end
data/lib/mutineer.rb CHANGED
@@ -34,6 +34,7 @@ require_relative "mutineer/mutators/operand_removal"
34
34
  require_relative "mutineer/mutators/array_literal"
35
35
  require_relative "mutineer/mutators/condition_forcing"
36
36
  require_relative "mutineer/mutators/operator_assignment"
37
+ require_relative "mutineer/mutators/rescue_body"
37
38
  require_relative "mutineer/mutator_registry"
38
39
  require_relative "mutineer/worker_pool"
39
40
  require_relative "mutineer/progress"
@@ -18,7 +18,7 @@ gem install mutineer
18
18
  ## Run
19
19
 
20
20
  ```sh
21
- mutineer run <source...> [--test <test>...] [options] # no --test: pairs each source with its test by convention; unmatched sources are skipped; --dry-run with --rails/--boot needs --test
21
+ mutineer run <source...> [--test <test>...] [options] # no --test: pairs each source with its test by convention; unmatched sources are skipped and counted in the report; --dry-run with --rails/--boot needs --test
22
22
  ```
23
23
 
24
24
  Example — mutate `lib/calculator.rb`, gate CI below 90%:
@@ -27,6 +27,11 @@ Example — mutate `lib/calculator.rb`, gate CI below 90%:
27
27
  mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
28
28
  ```
29
29
 
30
+ A source that raises `NameError` (including `NoMethodError`) while it loads, with no boot file, needs `--boot` or `boot:` pointed at an app entry point or a gem's `lib/<name>.rb`.
31
+ That file loads before the sources.
32
+ Without a boot file, `require:` files load after the sources, so a source cannot use them while it loads.
33
+ Listing the entry file as an earlier source also loads it first.
34
+
30
35
  ## Agent loop
31
36
 
32
37
  1. Edit code and tests on a branch. Fetch `origin/main` before using it below.
@@ -48,7 +53,10 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
48
53
  Stop after a few unsuccessful attempts and ask a person to review the survivor.
49
54
  Failed attempts do not prove that a mutant is equivalent. To
50
55
  suppress it, `mutineer triage .mutineer/run.json --reason TEXT --id ID` prints an
51
- `ignore:` entry for `.mutineer.yml` (Mutineer 1.7 or later reads it). See
56
+ `ignore:` entry for `.mutineer.yml`, and one entry for each id in that
57
+ survivor's `copies`. Paste every entry. An entry ignores only its own id
58
+ (Mutineer 1.7 or later reads it; `copies` is on reports from the version that
59
+ lists them). See
52
60
  [Suppressing equivalent mutants](https://github.com/davidteren/mutineer/blob/main/docs/results.md#suppressing-equivalent-mutants).
53
61
 
54
62
  Keep `fail_fast` disabled (the default). If the active `.mutineer.yml` sets it to
@@ -59,7 +67,7 @@ and its score and exit code do not cover the full scope.
59
67
  Zero survivors alone is not success: an empty or fully suppressed scope, or a run
60
68
  with no usable verdicts, also has zero survivors. If the score is null, stop and
61
69
  report “no score”. Check `no_coverage[]` for test gaps and `no_verdict[]` for
62
- harness failures, and fix them before starting a new run. A positive threshold fails when nothing can be scored and something
70
+ harness failures, and fix them before starting a new run. Also check `summary.unpaired_mutants` (Mutineer 1.9.0 or later). Above 0, a source with no paired test was skipped. Its mutants are not in the score or in `no_verdict[]`, so they cannot fail `--threshold`. They count in the unrun-share warning on stderr, except under `--fail-fast`. `unpaired_sources[]` names each skipped source. Add a test for each entry whose `mutants` is above 0, then run again. `--test` replaces pairing, so if you pass it, list every test file, the new ones included. Under `--since`, an entry with `mutants` 0 has no mutant on a changed line. Also look for an entry whose `mutants` is `null`. That count failed, and `summary.unpaired_mutants` leaves it out. Report that run as incomplete. Do not accept it. A positive threshold fails when nothing can be scored and something
63
71
  broke, or when more than one mutant has no verdict and they exceed 10% of those
64
72
  attempted; it does not require zero errors. Set the threshold to your target.
65
73
 
@@ -71,7 +79,7 @@ An empty run (`summary.total` is 0) exits 0 in 1.x and prints a warning on stder
71
79
  | Code | Meaning |
72
80
  |------|---------|
73
81
  | `0` | Score ≥ threshold (or no gate) **and** no baseline regression. |
74
- | `1` | Score below `--threshold`, OR nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted, OR a `--baseline` regression, OR a runtime error, including a boot, source or `require:` file that raises while loading. |
82
+ | `1` | Score below `--threshold`, OR nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted, OR a `--baseline` regression, OR a runtime error, including a boot, source or `require:` file that raises while loading (except an in-process `--since` run with no mutant on a changed line, which checks that those files exist but does not load them). |
75
83
  | `2` | Usage / invalid-flag error, including a mistyped flag, a bad path (such as a missing `--boot` or `require:` file), or an unreadable baseline. |
76
84
  <!-- /contract:exit-codes -->
77
85