minitest-mergify 2026.10.8 → 2026.10.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1f8329ced091f90450e17fd2609fb2f0c4b2473a4c9b4a3dcb2894976ca7cca1
4
- data.tar.gz: ca3816579117094718c11e849cac1674eff03803feb7302c63533cd2710acc31
3
+ metadata.gz: 37a85f8873524463ff41b43d75a2f06d01733e2d4590448bfeacffbc1764aba6
4
+ data.tar.gz: 07703b9504fdac92fe1ceaf24308ee98c0fd682edcd2b443b064803102d51bc9
5
5
  SHA512:
6
- metadata.gz: d7a4608e8303cfe68b9b9a27fac19ec798ffc0deecd61987823903fba281f6d741a97c647e1ca936b05731282e7235b3c6855f78346e43d35d5332577e79a146
7
- data.tar.gz: ef2e7eabb0d85e518a066e94fabebf9a212afe436a40e8018c50d013f782e5747fafcb86a4b90f923960bc43f4c012a689fdcb251c740c58ef60fe448571c8be
6
+ metadata.gz: e5ce47f1dc7fe4ff13c894eed9f6f833ab069a4f78cb18f8e75468aa25b0c3c76ab666c98dfe3b2f24eed4fdf9fdef594f6ef1ae15e392c559b0fc9905e75dc3
7
+ data.tar.gz: e60b50fcb1952b2c8adb3f22dbfae0aba129818199dcb95881987dc9b31b8e7a22b47710d13463e405ca427abaec43da3526bf0ff5764fc3704aa34e0cf98076
data/README.md CHANGED
@@ -8,6 +8,7 @@ Minitest plugin for [Mergify Test Engine](https://docs.mergify.com/test-engine/)
8
8
  - **Test quarantine** — Quarantines failing tests so they don't block CI
9
9
  - **Test selection** — Runs first the tests that failed in a merge-queue bisection step, and stops once one fails again
10
10
  - **Flaky test detection** — Reruns new tests to catch flaky ones before they merge
11
+ - **Test retry** — On the default branch, runs a failing test again, within a time budget Mergify sets
11
12
 
12
13
  ## Installation
13
14
 
@@ -55,6 +56,7 @@ environment variable). To enable outside CI, set `MERGIFY_CI_ENABLE=true`.
55
56
  | `MERGIFY_TRACEPARENT` | W3C distributed trace context | — |
56
57
  | `MERGIFY_TEST_JOB_NAME` | Mergify test job name | — |
57
58
  | `MERGIFY_FAIL_FIRST_ENABLE` | Set to `false` to never run first what a merge-queue bisection step of this job failed before (see below) | `true` |
59
+ | `MERGIFY_TEST_RETRY_ENABLE` | Set to `false` to have this job retry no failing test, whatever the repository sets | `true` |
58
60
 
59
61
  ### Quarantine
60
62
 
@@ -105,6 +107,20 @@ without failing the job. The reruns go through Minitest itself, so they run
105
107
  where the test ran: in one process, in Minitest's threads, or in Rails'
106
108
  parallel workers.
107
109
 
110
+ ### Test retry
111
+
112
+ On a push to the repository's default branch, when a test fails, the plugin
113
+ runs it again, up to the number of retries the repository sets, within a time
114
+ budget for the whole run. If a retry passes, so does the test: Minitest's own
115
+ reporters see the retry that passed instead of the failure, and Mergify still
116
+ learns the test failed first. A quarantined test, and one Mergify classifies
117
+ as broken, are not retried; pull requests, merge-queue runs and other branches
118
+ never are. The retries go through Minitest itself, as flaky detection's reruns
119
+ do.
120
+
121
+ A job that sets `MERGIFY_TEST_RETRY_ENABLE=false` retries nothing, whatever
122
+ the repository sets.
123
+
108
124
  ### Test names
109
125
 
110
126
  A test in a class is named as Minitest names it, `UserTest#test_valid`. A test
data/THIRD_PARTY_LICENSES CHANGED
@@ -8,8 +8,8 @@ The source code of each component is available from the repository listed
8
8
  next to it, as the Mozilla Public License 2.0 requires for the components
9
9
  under it.
10
10
 
11
- Apache License 2.0 (230)
12
- MIT License (73)
11
+ Apache License 2.0 (233)
12
+ MIT License (74)
13
13
  Unicode License v3 (19)
14
14
  ISC License (5)
15
15
  BSD 3-Clause "New" or "Revised" License (4)
@@ -23,7 +23,7 @@ Apache License 2.0 (Apache-2.0)
23
23
 
24
24
  Used by:
25
25
  - cmov 0.5.4 (https://github.com/RustCrypto/utils)
26
- - ctutils 0.4.3 (https://github.com/RustCrypto/utils)
26
+ - ctutils 0.4.6 (https://github.com/RustCrypto/utils)
27
27
  - encode_unicode 1.0.0 (https://github.com/tormol/encode_unicode)
28
28
  - encoding_rs 0.8.42 (https://github.com/hsivonen/encoding_rs)
29
29
  - multiversion_no_op 1.0.0 (https://github.com/hsivonen/multiversion_no_op)
@@ -4882,6 +4882,7 @@ Apache License 2.0 (Apache-2.0)
4882
4882
  Used by:
4883
4883
  - ahash 0.8.12 (https://github.com/tkaitchuck/ahash)
4884
4884
  - async-channel 2.5.0 (https://github.com/smol-rs/async-channel)
4885
+ - async-compression 0.4.51 (https://github.com/Nullus157/async-compression)
4885
4886
  - async-executor 1.14.0 (https://github.com/smol-rs/async-executor)
4886
4887
  - async-io 2.6.0 (https://github.com/smol-rs/async-io)
4887
4888
  - async-lock 3.4.2 (https://github.com/smol-rs/async-lock)
@@ -4894,6 +4895,8 @@ Used by:
4894
4895
  - bitflags 2.13.1 (https://github.com/bitflags/bitflags)
4895
4896
  - blocking 1.7.0 (https://github.com/smol-rs/blocking)
4896
4897
  - cfg-if 1.0.4 (https://github.com/rust-lang/cfg-if)
4898
+ - compression-codecs 0.4.46 (https://github.com/Nullus157/async-compression)
4899
+ - compression-core 0.4.34 (https://github.com/Nullus157/async-compression)
4897
4900
  - concurrent-queue 2.5.0 (https://github.com/smol-rs/concurrent-queue)
4898
4901
  - core-foundation-sys 0.8.7 (https://github.com/servo/core-foundation-rs)
4899
4902
  - core-foundation 0.10.1 (https://github.com/servo/core-foundation-rs)
@@ -8791,7 +8794,7 @@ THIS SOFTWARE.
8791
8794
  ISC License (ISC)
8792
8795
 
8793
8796
  Used by:
8794
- - rustls-webpki 0.103.15 (https://github.com/rustls/webpki)
8797
+ - rustls-webpki 0.103.16 (https://github.com/rustls/webpki)
8795
8798
 
8796
8799
  Except as otherwise noted, this project is licensed under the following
8797
8800
  (ISC-style) terms:
@@ -9826,10 +9829,10 @@ SOFTWARE.
9826
9829
  MIT License (MIT)
9827
9830
 
9828
9831
  Used by:
9829
- - jsonschema-regex 0.60.0 (https://github.com/Stranger6667/jsonschema)
9830
- - jsonschema-value 0.60.0 (https://github.com/Stranger6667/jsonschema)
9831
- - jsonschema 0.60.0 (https://github.com/Stranger6667/jsonschema)
9832
- - referencing 0.60.0 (https://github.com/Stranger6667/jsonschema)
9832
+ - jsonschema-regex 0.60.1 (https://github.com/Stranger6667/jsonschema)
9833
+ - jsonschema-value 0.60.1 (https://github.com/Stranger6667/jsonschema)
9834
+ - jsonschema 0.60.1 (https://github.com/Stranger6667/jsonschema)
9835
+ - referencing 0.60.1 (https://github.com/Stranger6667/jsonschema)
9833
9836
 
9834
9837
  MIT License
9835
9838
 
@@ -10083,6 +10086,7 @@ USE OR OTHER DEALINGS IN THE SOFTWARE.
10083
10086
  MIT License (MIT)
10084
10087
 
10085
10088
  Used by:
10089
+ - tokio-util 0.7.19 (https://github.com/tokio-rs/tokio)
10086
10090
  - tokio 1.54.1 (https://github.com/tokio-rs/tokio)
10087
10091
 
10088
10092
  MIT License
@@ -7,6 +7,7 @@ require_relative 'test_run'
7
7
  require_relative 'names'
8
8
  require_relative 'recorder'
9
9
  require_relative 'reruns'
10
+ require_relative 'retries'
10
11
  require_relative 'selection'
11
12
 
12
13
  module Mergify
@@ -60,14 +61,21 @@ module Mergify
60
61
  start(reporter, test_run, options)
61
62
  end
62
63
 
63
- # The run's suites, as the selection runs them, then flaky detection's
64
- # reruns -- not after a stop Fail First made, whose whole point is to
65
- # end the job sooner.
64
+ # The run's suites, as the selection runs them, once the session
65
+ # prepared its reruns; then test retry's retries and flaky
66
+ # detection's reruns -- not after a stop Fail First made, whose whole
67
+ # point is to end the job sooner -- and last the failures test retry
68
+ # held back for flaky detection's reruns.
66
69
  def run_suites(reporter, &)
67
70
  return yield unless @selection
68
71
 
72
+ @reruns.prepare
69
73
  @selection.run(@recorder, &)
70
- @reruns.run(reporter, @recorder) unless @recorder.stopping?
74
+ unless @recorder.stopping?
75
+ @retries.settle(@recorder, @selection.names_to_run)
76
+ @reruns.run(reporter, @recorder)
77
+ end
78
+ @recorder.release_held(reporter)
71
79
  end
72
80
 
73
81
  private
@@ -79,7 +87,8 @@ module Mergify
79
87
  @selection = nil
80
88
  @selection = select(test_run, names, options)
81
89
  @reruns = @selection && Reruns.new(test_run, @selection)
82
- @recorder = Recorder.new(test_run, names, options[:io] || $stdout, @selection, @reruns)
90
+ @retries = @selection && Retries.new(test_run, @selection, @reruns)
91
+ @recorder = Recorder.new(test_run, names, options[:io] || $stdout, @selection, @reruns, @retries)
83
92
  @recorder.attach(reporter)
84
93
  install
85
94
  end
@@ -7,8 +7,8 @@ require_relative 'results'
7
7
  module Mergify
8
8
  module Minitest
9
9
  # Hands each finished test to the session, for the run's trace, its
10
- # verdict, the quarantine, Fail First and flaky detection, and prints what
11
- # the session says once the run is over.
10
+ # verdict, the quarantine, Fail First, flaky detection and test retry, and
11
+ # prints what the session says once the run is over.
12
12
  #
13
13
  # It wraps the reporter Minitest sends every result through, rather than
14
14
  # joining the reporters inside it: a plugin can replace those, and
@@ -28,16 +28,23 @@ module Mergify
28
28
  attr_reader :first_attempts
29
29
 
30
30
  # `selection` is the run's, when it asked Test Selection; nil when it
31
- # does not report. `reruns`, flaky detection's, takes the results of the
32
- # tests it reruns.
33
- def initialize(test_run, names, io, selection, reruns)
31
+ # does not report. `reruns`, the session's, takes the results of the
32
+ # tests flaky detection reruns, and `retries` those of the tests test
33
+ # retry retries.
34
+ # rubocop:disable-next Metrics/MethodLength,Metrics/ParameterLists
35
+ def initialize(test_run, names, io, selection, reruns, retries)
34
36
  @test_run = test_run
35
37
  @names = names
36
38
  @io = io
37
39
  @selection = selection
38
40
  @reruns = reruns
41
+ @retries = retries
39
42
  @fail_first = selection && test_run.fail_first
40
43
  @first_attempts = {}
44
+ # The results Minitest's own reporters get as they are: the failures
45
+ # test retry held back for flaky detection's reruns, once they are
46
+ # over.
47
+ @released = {}.compare_by_identity
41
48
  @lock = Mutex.new
42
49
  @changed = ConditionVariable.new
43
50
  @stopping = false
@@ -48,30 +55,35 @@ module Mergify
48
55
  reporter.singleton_class.prepend(hooks)
49
56
  end
50
57
 
51
- # Before Minitest's own reporters count the result: a failure the
58
+ # What Minitest's own reporters get of `result`, which `reporter`
59
+ # records, before they count it; nil for nothing. A failure the
52
60
  # quarantine absorbs reaches them as a skip, so it fails nothing, and
53
- # Mergify still sees it fail.
54
- def record(result)
55
- return unless @test_run.reports?
61
+ # Mergify still sees it fail. A rerun of flaky detection's never does. A
62
+ # failure test retry answers for is held back until it answered, then
63
+ # reaches them as the retry that passed, or as itself.
64
+ def pass_on(result, reporter)
65
+ return result if @released.delete(result) || !@test_run.reports?
66
+ return if rerun_taken?(result)
67
+ return retry_taken(result, reporter) if @retries&.retrying?(result.klass, result.name)
56
68
 
57
- named = named(result)
58
- status = Results.status_of(result)
59
- absorbed = trace(result, named, status)
60
- conclude(named.name, status, absorbed, result.time)
69
+ record(result, reporter)
61
70
  end
62
71
 
63
- # Whether `result` is a rerun's, which flaky detection takes as an
64
- # attempt and Minitest's own reporters never see.
65
- def rerun_taken?(result)
66
- return false unless @reruns&.rerun?(result.klass, result.name)
67
-
68
- @reruns.take(result, named(result).name)
72
+ # Whether Minitest is about to report a rerun or a retry of `method` of
73
+ # `klass` (a class, or what Rails' workers report its name as).
74
+ def rerun?(klass, method)
75
+ klass = klass.respond_to?(:name) ? klass.name : klass.to_s
76
+ @reruns&.rerun?(klass, method) || @retries&.retrying?(klass, method) || false
69
77
  end
70
78
 
71
- # Whether Minitest is about to report a rerun of `method` of `klass` (a
72
- # class, or what Rails' workers report its name as).
73
- def rerun?(klass, method)
74
- @reruns&.rerun?(klass.respond_to?(:name) ? klass.name : klass.to_s, method) || false
79
+ # Hand Minitest's own reporters, through `reporter`, the failures test
80
+ # retry held back for flaky detection's reruns, once they are over.
81
+ def release_held(reporter)
82
+ @retries&.answered&.each do |name, result|
83
+ conclude(name, Results.status_of(result), false, result.time)
84
+ @released[result] = true
85
+ reporter.record(result)
86
+ end
75
87
  end
76
88
 
77
89
  # Whether flaky detection found a new test flaky, in reruns Minitest's
@@ -106,6 +118,43 @@ module Mergify
106
118
 
107
119
  private
108
120
 
121
+ # A test's first result: its span, the quarantine, the session's
122
+ # reruns, then its final status, unless test retry holds its failure
123
+ # back. Answers the result Minitest's own reporters get, nil while held
124
+ # back.
125
+ def record(result, reporter)
126
+ named = named(result)
127
+ name = named.name
128
+ status = Results.status_of(result)
129
+ absorbed = trace(result, named, status)
130
+ @reruns&.first_attempt(name, status, result.time * 1000)
131
+ held = !absorbed && @retries&.hold?(result, name, reporter)
132
+ conclude(name, status, absorbed, result.time) unless held
133
+ recorded(name, status, result.time)
134
+ result unless held
135
+ end
136
+
137
+ # Whether `result` is a rerun's, which flaky detection takes as an
138
+ # attempt, and test retry too where it holds the test's failure back.
139
+ def rerun_taken?(result)
140
+ return false unless @reruns&.rerun?(result.klass, result.name)
141
+
142
+ name = named(result).name
143
+ return false unless @reruns.take(result, name)
144
+
145
+ @retries&.rerun_ended(name, result)
146
+ true
147
+ end
148
+
149
+ # A retry's result, which test retry takes: once it answered, the
150
+ # result Minitest's own reporters get, whose status is the test's.
151
+ def retry_taken(result, reporter)
152
+ name = named(result).name
153
+ released = @retries.take(result, name, reporter)
154
+ conclude(name, Results.status_of(released), false, released.time) if released
155
+ released
156
+ end
157
+
109
158
  # The test's span, as the quarantine accounts for it, and the failure
110
159
  # it absorbs: whether it did.
111
160
  def trace(result, named, status)
@@ -119,16 +168,20 @@ module Mergify
119
168
  @names.of(result.klass, result.name, result.source_location.first)
120
169
  end
121
170
 
122
- # The test's first attempt, as it ended, for flaky detection, and its
123
- # final status, for the verdict and Fail First: a failure the quarantine
124
- # absorbed is none.
171
+ # The test's final status, for the verdict and Fail First: a failure
172
+ # the quarantine absorbed is none.
125
173
  def conclude(name, status, absorbed, time)
126
174
  final = absorbed ? 'quarantined_failed' : status
127
175
  @test_run.record_test(name, final, time)
128
176
  stop = @fail_first&.unit_done(name, final == 'failed')
177
+ @lock.synchronize { @stopping ||= stop }
178
+ end
179
+
180
+ # The test's first attempt was recorded, after its final status unless
181
+ # test retry held it back.
182
+ def recorded(name, status, time)
129
183
  @lock.synchronize do
130
184
  @first_attempts[name] ||= [status, time * 1000]
131
- @stopping ||= stop
132
185
  @changed.broadcast
133
186
  end
134
187
  end
@@ -152,10 +205,8 @@ module Mergify
152
205
  end
153
206
 
154
207
  define_method(:record) do |result|
155
- next if recorder.rerun_taken?(result)
156
-
157
- recorder.record(result)
158
- super(result)
208
+ passed_on = recorder.pass_on(result, self)
209
+ super(passed_on) if passed_on
159
210
  end
160
211
 
161
212
  define_method(:passed?) do
@@ -5,22 +5,45 @@ require_relative 'results'
5
5
 
6
6
  module Mergify
7
7
  module Minitest
8
- # Flaky detection, once the run's tests are over: the session plans how
9
- # many times to rerun the tests it targets, from their first attempts, and
10
- # each rerun goes through Minitest's own dispatch, so it runs where the
11
- # test ran -- inline, in Minitest's threads, or in Rails' workers -- and
12
- # its result comes back to the reporter Minitest records results through.
13
- # There, each is an attempt for the session, which Minitest's own
14
- # reporters never see.
8
+ # The session's reruns for the run, and flaky detection's, once the run's
9
+ # tests are over: the session plans how many times to rerun the tests it
10
+ # targets, from their first attempts, and each rerun goes through
11
+ # Minitest's own dispatch, so it runs where the test ran -- inline, in
12
+ # Minitest's threads, or in Rails' workers -- and its result comes back to
13
+ # the reporter Minitest records results through. There, each is an
14
+ # attempt for the session, which Minitest's own reporters never see.
15
15
  class Reruns
16
+ # The session's reruns, once prepared: nil before, and when neither
17
+ # flaky detection nor test retry runs.
18
+ attr_reader :prepared
19
+
16
20
  def initialize(test_run, selection)
17
21
  @test_run = test_run
18
22
  @selection = selection
23
+ @prepared = nil
19
24
  @pending = Hash.new(0)
20
25
  @lock = Mutex.new
21
26
  @changed = ConditionVariable.new
22
27
  end
23
28
 
29
+ # Prepare the session's reruns before any test runs, for every test the
30
+ # run is to run, where test retry retries: it answers each failure as
31
+ # it comes. Flaky detection alone waits for the run's tests to be over.
32
+ def prepare
33
+ prepare_for(@selection.tests.map(&:name)) if @test_run.test_retry_runs?
34
+ end
35
+
36
+ # The first attempt at the test named `name` ended with `outcome` after
37
+ # `duration_ms`: the session takes it, once the reruns are prepared,
38
+ # where flaky detection targets the test or test retry answers for it,
39
+ # as it passed or failed.
40
+ def first_attempt(name, outcome, duration_ms)
41
+ return unless @prepared && %w[passed failed].include?(outcome)
42
+ return unless @prepared.tracks?(name) || @prepared.retry_answers_for?(name)
43
+
44
+ @test_run.record_attempt(name, outcome, duration_ms)
45
+ end
46
+
24
47
  # Rerun what flaky detection targets, once `recorder` recorded every test
25
48
  # the run was to run: a run in Minitest's threads or Rails' workers
26
49
  # hands its results back after its suites were handed over.
@@ -28,8 +51,8 @@ module Mergify
28
51
  return unless @test_run.flaky_detection_runs?
29
52
 
30
53
  recorder.wait_for(@selection.names_to_run)
31
- reruns = prepare(recorder.first_attempts)
32
- plan = reruns&.shared_repeats
54
+ prepare_after(recorder.first_attempts) unless @prepared
55
+ plan = @prepared&.shared_repeats
33
56
  rerun(plan, reporter) if plan
34
57
  end
35
58
 
@@ -55,18 +78,18 @@ module Mergify
55
78
 
56
79
  private
57
80
 
58
- # The session's reruns for every test the run ran, each known by its
59
- # former name too, with the first attempt of each test flaky detection
60
- # targets, as it passed or failed.
61
- def prepare(first_attempts)
81
+ # The session's reruns for the tests named `names`, each known by its
82
+ # former name too.
83
+ def prepare_for(names)
62
84
  former_names = @selection.tests.to_h { |test| [test.name, ["#{test.klass.name}.#{test.method_name}"]] }
63
- reruns = @test_run.prepare_reruns(first_attempts.keys, former_names)
64
- first_attempts.each do |name, (outcome, duration_ms)|
65
- next unless reruns&.tracks?(name) && %w[passed failed].include?(outcome)
85
+ @prepared = @test_run.prepare_reruns(names, former_names)
86
+ end
66
87
 
67
- @test_run.record_attempt(name, outcome, duration_ms)
68
- end
69
- reruns
88
+ # The session's reruns once the run's tests are over, for every test it
89
+ # ran, with the first attempt of each.
90
+ def prepare_after(first_attempts)
91
+ prepare_for(first_attempts.keys)
92
+ first_attempts.each { |name, (outcome, duration_ms)| first_attempt(name, outcome, duration_ms) }
70
93
  end
71
94
 
72
95
  # Each test of `plan` again, as many times as it says, round after
@@ -74,7 +97,7 @@ module Mergify
74
97
  def rerun(plan, reporter)
75
98
  tests = @selection.tests.select { |test| plan['tests'].include?(test.name) }
76
99
  expect(tests, plan['repeats'])
77
- plan['repeats'].times { tests.each { |test| dispatch(test, reporter) } }
100
+ plan['repeats'].times { tests.each { |test| test.dispatch(reporter) } }
78
101
  @lock.synchronize { @changed.wait(@lock) until @pending.values.all?(&:zero?) }
79
102
  end
80
103
 
@@ -82,16 +105,6 @@ module Mergify
82
105
  def expect(tests, repeats)
83
106
  @lock.synchronize { tests.each { |test| @pending[[test.klass.name, test.method_name]] += repeats } }
84
107
  end
85
-
86
- # As Minitest runs a test of a class: inline and reported, or handed to
87
- # its parallel executor, which reports it once it ran.
88
- def dispatch(test, reporter)
89
- if ::Minitest::Runnable.respond_to?(:run_suite)
90
- test.klass.run(test.klass, test.method_name, reporter)
91
- else
92
- test.klass.run_one_method(test.klass, test.method_name, reporter)
93
- end
94
- end
95
108
  end
96
109
  end
97
110
  end
@@ -0,0 +1,131 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'minitest'
4
+ require 'set'
5
+ require_relative 'results'
6
+
7
+ module Mergify
8
+ module Minitest
9
+ # Test retry: a test whose first attempt failed runs again, through
10
+ # Minitest's own dispatch, as long as the session grants it one more
11
+ # attempt, so it runs where it ran -- inline, in Minitest's threads, or in
12
+ # Rails' workers -- and each result comes back to the reporter Minitest
13
+ # records results through, in the main process, which asks the session
14
+ # for the next. Its failure is held back from Minitest's own reporters
15
+ # until retry answered: a retry that passed reaches them in its place,
16
+ # and the failure itself otherwise.
17
+ #
18
+ # Which failures retry answers for, how many retries a test gets and
19
+ # within which budget are the session's. A test flaky detection reruns is
20
+ # retry's to answer for with those reruns, once the suite is over: its
21
+ # failure is held back until then.
22
+ class Retries
23
+ def initialize(test_run, selection, reruns)
24
+ @test_run = test_run
25
+ @tests = selection.tests.to_h { |test| [test.name, test] }
26
+ @reruns = reruns
27
+ # The failures held back, by name, until retry answered.
28
+ @held = {}
29
+ # The tests retried, by class name and method, until retry answered.
30
+ @retrying = Set.new
31
+ # A passing rerun of flaky detection's, by name, for a failure held
32
+ # back until its reruns are over.
33
+ @passed = {}
34
+ @lock = Mutex.new
35
+ @changed = ConditionVariable.new
36
+ end
37
+
38
+ # The first attempt at the test named `name` ended with `result`,
39
+ # which the session took: whether retry holds it back, a failure it
40
+ # answers for. A test the session grants a retry runs again; one flaky
41
+ # detection reruns waits for its reruns.
42
+ def hold?(result, name, reporter)
43
+ prepared = @reruns.prepared
44
+ return false unless Results.status_of(result) == 'failed' && prepared&.retry_answers_for?(name)
45
+
46
+ if prepared.grant_retry(name, result.time * 1000)
47
+ retry_test(result, name, reporter)
48
+ elsif prepared.tracks?(name)
49
+ @lock.synchronize { @held[name] = result }
50
+ else
51
+ return false
52
+ end
53
+ true
54
+ end
55
+
56
+ # Whether Minitest is about to report a retry of `method` of the class
57
+ # named `klass`, which the reporter keeps from Minitest's own.
58
+ def retrying?(klass, method)
59
+ @lock.synchronize { @retrying.include?([klass, method]) }
60
+ end
61
+
62
+ # The result of a retry of the test named `name`, as an attempt: the
63
+ # result Minitest's own reporters get once retry answered, the one that
64
+ # passed or the failure held back; nil while the test runs once more.
65
+ def take(result, name, reporter)
66
+ status = Results.status_of(result)
67
+ duration_ms = result.time * 1000
68
+ @test_run.record_attempt(name, status, duration_ms)
69
+ return answer(result, name) unless status == 'failed' && @reruns.prepared.grant_retry(name, duration_ms)
70
+
71
+ @tests.fetch(name).dispatch(reporter)
72
+ nil
73
+ end
74
+
75
+ # A rerun of flaky detection's of the test named `name` ended with
76
+ # `result`.
77
+ def rerun_ended(name, result)
78
+ @lock.synchronize do
79
+ @passed[name] ||= result if @held.key?(name) && Results.status_of(result) == 'passed'
80
+ end
81
+ end
82
+
83
+ # Wait, once the run's suites were handed over, until `recorder`
84
+ # recorded every test named `names`, and every retry they were granted
85
+ # ended, where test retry answers for a failure of any of them:
86
+ # Minitest's threads and Rails' workers stop once the suites are over.
87
+ def settle(recorder, names)
88
+ prepared = @reruns.prepared
89
+ return unless prepared && names.any? { |name| prepared.retry_answers_for?(name) }
90
+
91
+ recorder.wait_for(names)
92
+ @lock.synchronize { @changed.wait(@lock) until @retrying.empty? }
93
+ end
94
+
95
+ # The failures held back for flaky detection's reruns, once they are
96
+ # over, by name: the result Minitest's own reporters get for each, a
97
+ # rerun that passed where it rescued the test, the failure otherwise.
98
+ def answered
99
+ held = @lock.synchronize { @held.dup.tap { @held.clear } }
100
+ held.to_h do |name, failure|
101
+ passed = @passed[name]
102
+ [name, passed && @reruns.prepared.rescued?(name) ? passed : failure]
103
+ end
104
+ end
105
+
106
+ private
107
+
108
+ # Hold back the failure `result` of the test named `name`, and run the
109
+ # test again.
110
+ def retry_test(result, name, reporter)
111
+ @lock.synchronize do
112
+ @held[name] = result
113
+ @retrying << [result.klass, result.name]
114
+ end
115
+ @tests.fetch(name).dispatch(reporter)
116
+ end
117
+
118
+ # Retry answered the failure of the test named `name`, its last retry
119
+ # ended with `result`: what Minitest's own reporters get, `result`
120
+ # where a retry rescued the test, the failure held back otherwise.
121
+ def answer(result, name)
122
+ held = @lock.synchronize do
123
+ @retrying.delete([result.klass, result.name])
124
+ @changed.broadcast
125
+ @held.delete(name)
126
+ end
127
+ @reruns.prepared.rescued?(name) ? result : held
128
+ end
129
+ end
130
+ end
131
+ end
@@ -15,7 +15,18 @@ module Mergify
15
15
  class Selection
16
16
  # A test the run collected: its class, its method, and the name Mergify
17
17
  # knows it by.
18
- Test = Struct.new(:klass, :method_name, :name)
18
+ Test = Struct.new(:klass, :method_name, :name) do
19
+ # Run it again, as Minitest runs a test of a class: inline and
20
+ # reported, or handed to its parallel executor, which reports it once
21
+ # it ran.
22
+ def dispatch(reporter)
23
+ if ::Minitest::Runnable.respond_to?(:run_suite)
24
+ klass.run(klass, method_name, reporter)
25
+ else
26
+ klass.run_one_method(klass, method_name, reporter)
27
+ end
28
+ end
29
+ end
19
30
 
20
31
  # Every test the run collected, as Minitest's filters left them.
21
32
  attr_reader :tests
@@ -19,8 +19,9 @@ module Mergify
19
19
  # every result and stops the run.
20
20
  fail_first: { unit: 'test', waits_for_all: true, workers_stop_the_run: false },
21
21
  # Flaky detection reruns a test after the suite, as many times as
22
- # every other one it reruns, and nothing retries a failure.
23
- reruns: { plans_repeats: true, retries_failures: false },
22
+ # every other one it reruns; test retry runs a failed one again as it
23
+ # fails.
24
+ reruns: { plans_repeats: true, retries_failures: true },
24
25
  framework: { name: 'minitest', version: ::Minitest::VERSION, language: 'ruby' }
25
26
  }.freeze
26
27
 
@@ -93,23 +94,30 @@ module Mergify
93
94
  @session.flaky_detection_runs?
94
95
  end
95
96
 
97
+ # Whether test retry retries in this run: the repository has it, and
98
+ # the run builds its default branch.
99
+ def test_retry_runs?
100
+ @session.test_retry_runs?
101
+ end
102
+
96
103
  # Whether flaky detection found a new test flaky, which fails the run.
97
104
  def flaky_detection_fails_the_run?
98
105
  @session.flaky_detection_fails_the_run?
99
106
  end
100
107
 
101
- # Prepare flaky detection for the tests the run ran, by name, each also
102
- # known by its former names (`former_names`, a list by name), which
103
- # Mergify's lists may still hold it under: the run's reruns, or nil
104
- # when nothing is to rerun. Which tests it reruns, and how many times, are the
105
- # session's.
108
+ # Prepare flaky detection and test retry for the tests named `names`,
109
+ # each also known by its former names (`former_names`, a list by name),
110
+ # which Mergify's lists may still hold it under: the run's reruns, or
111
+ # nil when neither runs. Which tests each reruns, and how many times,
112
+ # are the session's. No test opts out of test retry: Minitest has no
113
+ # annotation of Mergify's.
106
114
  def prepare_reruns(names, former_names)
107
115
  @session.state_former_names(former_names)
108
116
  @session.prepare_reruns(names)
109
117
  end
110
118
 
111
119
  # An attempt at a test, ended with `outcome` (`'passed'` or `'failed'`)
112
- # after `duration_ms`: its first one, then each rerun.
120
+ # after `duration_ms`: its first one, then each rerun or retry.
113
121
  def record_attempt(name, outcome, duration_ms)
114
122
  @session.record_attempt(name, outcome, duration_ms)
115
123
  end
@@ -4,6 +4,6 @@ module Mergify
4
4
  module Minitest
5
5
  # Placeholder: the release workflow stamps the real version from the git
6
6
  # tag at build time, as it does for every client in this repository.
7
- VERSION = '2026.10.8'
7
+ VERSION = '2026.10.9'
8
8
  end
9
9
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: minitest-mergify
3
3
  version: !ruby/object:Gem::Version
4
- version: 2026.10.8
4
+ version: 2026.10.9
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mergify
@@ -35,14 +35,14 @@ dependencies:
35
35
  requirements:
36
36
  - - '='
37
37
  - !ruby/object:Gem::Version
38
- version: 2026.10.8
38
+ version: 2026.10.9
39
39
  type: :runtime
40
40
  prerelease: false
41
41
  version_requirements: !ruby/object:Gem::Requirement
42
42
  requirements:
43
43
  - - '='
44
44
  - !ruby/object:Gem::Version
45
- version: 2026.10.8
45
+ version: 2026.10.9
46
46
  description: 'Minitest integration for Mergify Test Engine: the run reported to CI
47
47
  Insights, test quarantine, Fail First on merge-queue bisection steps, and flaky
48
48
  test detection.'
@@ -62,6 +62,7 @@ files:
62
62
  - lib/mergify/minitest/recorder.rb
63
63
  - lib/mergify/minitest/reruns.rb
64
64
  - lib/mergify/minitest/results.rb
65
+ - lib/mergify/minitest/retries.rb
65
66
  - lib/mergify/minitest/selection.rb
66
67
  - lib/mergify/minitest/test_run.rb
67
68
  - lib/mergify/minitest/version.rb