rspec-mergify 2026.10.8 → 2026.10.10

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e5715b89995827b7f5d3840e0c3c1d94288f511fedcac9f3bdd58433c1908228
-  data.tar.gz: d70082926c396a3ac69ae778f9024be9d3cde6570cd84ef498d1e489ae9164d2
+  metadata.gz: addf160bfa2fcbe5d11558f76b497bdae4342d27ad69b4caadc7659854dedfde
+  data.tar.gz: 96d6a5c574bd767230faf016689753e8ad2f8eac4e9e68c3644cff8fe9fba47e
 SHA512:
-  metadata.gz: 036d1307e6752c12f0c4a88a5f4625528a8ef6eb7d0247b23100127d42a1842c2a3ddadcffc2439f27bb73f49f71d478346fb51b4a1524725c6812f232e10168
-  data.tar.gz: d3e25b1c4dda733655435cd3e28016a30127c82ed04e8d5b8e07a8e0e7c7374d572491bf8559e3f6c9262c814abe5130e4b65781b0c93cd7f91f2a72f5496de8
+  metadata.gz: 74839e7f18bcec97b58f5801c4c651f61dfe3f522188442f10609937ef1de454c832ef4776f3c6b7146415770ca9d6554e6b3b138742345a27f9ca17850956b5
+  data.tar.gz: 447fa60ca2e3f431980cb8294ea39c718b5969d263d1eb6af262dda3fd26f29c6941fbdf7e86bca736bef607b0a85563d1b8a557425dada761230edaacbb0f69

data/README.md

@@ -6,6 +6,7 @@ RSpec plugin for [Mergify Test Engine](https://docs.mergify.com/test-engine/).
 
 - **Test tracing** — Sends OpenTelemetry traces for every test to Mergify's API
 - **Flaky test detection** — Intelligently reruns tests to detect flakiness with budget constraints
+- **Test retry** — On the default branch, runs a failing example again, once rspec-retry's tries failed, within a time budget Mergify sets
 - **Test quarantine** — Quarantines failing tests so they don't block CI
 - **Test selection** — Runs first the examples a merge-queue bisection step failed in the step above, and stops once one fails again
 
@@ -25,12 +26,12 @@ The gem runs on `mergify-ci-core`, which Bundler installs with it. That gem ship
 
 | Platform | Requirement |
 |---|---|
-| Linux x86_64 / aarch64 (glibc) | **glibc 2.30 or newer** — Debian 11+, Ubuntu 20.04+, RHEL 9+, Amazon Linux 2023 |
-| Linux x86_64 / aarch64 (musl) | Alpine; on Ruby 3.1 also `apk add libgcc` |
+| Linux x86_64 / aarch64 (glibc) | **glibc 2.28 or newer** — Debian 10+, Ubuntu 20.04+, RHEL 8+ and its rebuilds, Amazon Linux 2023 |
+| Linux x86_64 / aarch64 (musl) | Alpine |
 | macOS arm64 / x86_64 | — |
 | Windows x64 (UCRT) | — |
 
-Anywhere else, and on glibc older than 2.30 — RHEL, Rocky, AlmaLinux and Oracle Linux 8, Amazon Linux 2, CentOS 7 — the extension cannot load. The gem still installs and your suite still runs, but nothing is reported to Mergify. Pin the last pure-Ruby release on those systems:
+Anywhere else, and on glibc older than 2.28 — Amazon Linux 2, CentOS 7, Ubuntu 18.04 — the extension cannot load. The gem still installs and your suite still runs, but nothing is reported to Mergify. Pin the last pure-Ruby release on those systems:
 
 ```ruby
 gem 'rspec-mergify', '0.1.4'
@@ -53,6 +54,7 @@ The plugin activates automatically when running in CI (detected via the `CI` env
 | `MERGIFY_TRACEPARENT` | W3C distributed trace context | — |
 | `MERGIFY_TEST_JOB_NAME` | Mergify test job name | — |
 | `MERGIFY_FAIL_FIRST_ENABLE` | Set to `false` to never run first what a merge-queue bisection step of this job failed before | `true` |
+| `MERGIFY_TEST_RETRY_ENABLE` | Set to `false` to have this job retry no failing test, whatever the repository sets | `true` |
 
 The names these replace, `RSPEC_MERGIFY_ENABLE` and `RSPEC_MERGIFY_DEBUG`, are
 deprecated: they still work when the new name is not set, and the summary says
@@ -143,7 +145,7 @@ test.
 
 [parallel_tests](https://github.com/grosser/parallel_tests) and [turbo_tests](https://github.com/serpapi/turbo_tests) need no setup. Each worker is a separate RSpec process, and each reports to Mergify as its own session:
 
-- flaky detection sizes each worker's rerun budget from the tests that worker runs;
+- flaky detection sizes each worker's rerun budget from the tests that worker runs, and test retry gives each worker a budget of its own;
 - each worker prints its own Mergify report, so a report covers only that worker's tests.
 
 Suites split across CI jobs (knapsack, `circleci tests split`, Buildkite `parallelism`) behave the same way, one session per job.

data/THIRD_PARTY_LICENSES

@@ -8,8 +8,8 @@ The source code of each component is available from the repository listed
 next to it, as the Mozilla Public License 2.0 requires for the components
 under it.
 
-Apache License 2.0 (230)
-MIT License (73)
+Apache License 2.0 (229)
+MIT License (74)
 Unicode License v3 (19)
 ISC License (5)
 BSD 3-Clause "New" or "Revised" License (4)
@@ -23,7 +23,7 @@ Apache License 2.0 (Apache-2.0)
 
 Used by:
   - cmov 0.5.4 (https://github.com/RustCrypto/utils)
-  - ctutils 0.4.3 (https://github.com/RustCrypto/utils)
+  - ctutils 0.4.6 (https://github.com/RustCrypto/utils)
   - encode_unicode 1.0.0 (https://github.com/tormol/encode_unicode)
   - encoding_rs 0.8.42 (https://github.com/hsivonen/encoding_rs)
   - multiversion_no_op 1.0.0 (https://github.com/hsivonen/multiversion_no_op)
@@ -474,11 +474,7 @@ Used by:
   - windows-link 0.2.1 (https://github.com/microsoft/windows-rs)
   - windows-result 0.4.1 (https://github.com/microsoft/windows-rs)
   - windows-strings 0.5.1 (https://github.com/microsoft/windows-rs)
-  - windows-sys 0.52.0 (https://github.com/microsoft/windows-rs)
   - windows-sys 0.61.2 (https://github.com/microsoft/windows-rs)
-  - windows-targets 0.52.6 (https://github.com/microsoft/windows-rs)
-  - windows_x86_64_gnu 0.52.6 (https://github.com/microsoft/windows-rs)
-  - windows_x86_64_msvc 0.52.6 (https://github.com/microsoft/windows-rs)
 
                                  Apache License
                            Version 2.0, January 2004
@@ -2571,14 +2567,14 @@ limitations under the License.
 Apache License 2.0 (Apache-2.0)
 
 Used by:
-  - futures-channel 0.3.33 (https://github.com/rust-lang/futures-rs)
-  - futures-core 0.3.33 (https://github.com/rust-lang/futures-rs)
+  - futures-channel 0.3.34 (https://github.com/rust-lang/futures-rs)
+  - futures-core 0.3.34 (https://github.com/rust-lang/futures-rs)
   - futures-executor 0.3.33 (https://github.com/rust-lang/futures-rs)
-  - futures-io 0.3.33 (https://github.com/rust-lang/futures-rs)
-  - futures-macro 0.3.33 (https://github.com/rust-lang/futures-rs)
-  - futures-sink 0.3.33 (https://github.com/rust-lang/futures-rs)
-  - futures-task 0.3.33 (https://github.com/rust-lang/futures-rs)
-  - futures-util 0.3.33 (https://github.com/rust-lang/futures-rs)
+  - futures-io 0.3.34 (https://github.com/rust-lang/futures-rs)
+  - futures-macro 0.3.34 (https://github.com/rust-lang/futures-rs)
+  - futures-sink 0.3.34 (https://github.com/rust-lang/futures-rs)
+  - futures-task 0.3.34 (https://github.com/rust-lang/futures-rs)
+  - futures-util 0.3.34 (https://github.com/rust-lang/futures-rs)
   - futures 0.3.33 (https://github.com/rust-lang/futures-rs)
 
                               Apache License
@@ -4882,6 +4878,7 @@ Apache License 2.0 (Apache-2.0)
 Used by:
   - ahash 0.8.12 (https://github.com/tkaitchuck/ahash)
   - async-channel 2.5.0 (https://github.com/smol-rs/async-channel)
+  - async-compression 0.4.51 (https://github.com/Nullus157/async-compression)
   - async-executor 1.14.0 (https://github.com/smol-rs/async-executor)
   - async-io 2.6.0 (https://github.com/smol-rs/async-io)
   - async-lock 3.4.2 (https://github.com/smol-rs/async-lock)
@@ -4894,6 +4891,8 @@ Used by:
   - bitflags 2.13.1 (https://github.com/bitflags/bitflags)
   - blocking 1.7.0 (https://github.com/smol-rs/blocking)
   - cfg-if 1.0.4 (https://github.com/rust-lang/cfg-if)
+  - compression-codecs 0.4.46 (https://github.com/Nullus157/async-compression)
+  - compression-core 0.4.34 (https://github.com/Nullus157/async-compression)
   - concurrent-queue 2.5.0 (https://github.com/smol-rs/concurrent-queue)
   - core-foundation-sys 0.8.7 (https://github.com/servo/core-foundation-rs)
   - core-foundation 0.10.1 (https://github.com/servo/core-foundation-rs)
@@ -4951,7 +4950,7 @@ Used by:
   - secret-service 5.2.0 (https://github.com/hwchen/secret-service-rs.git)
   - security-framework-sys 2.17.0 (https://github.com/kornelski/rust-security-framework)
   - security-framework 3.7.0 (https://github.com/kornelski/rust-security-framework)
-  - self-replace 1.5.0 (https://github.com/mitsuhiko/self-replace)
+  - self-replace 1.5.1 (https://github.com/mitsuhiko/self-replace)
   - signal-hook-registry 1.4.8 (https://github.com/vorner/signal-hook)
   - smallvec 1.15.2 (https://github.com/servo/rust-smallvec)
   - socket2 0.6.5 (https://github.com/rust-lang/socket2)
@@ -8791,7 +8790,7 @@ THIS SOFTWARE.
 ISC License (ISC)
 
 Used by:
-  - rustls-webpki 0.103.15 (https://github.com/rustls/webpki)
+  - rustls-webpki 0.103.16 (https://github.com/rustls/webpki)
 
 Except as otherwise noted, this project is licensed under the following
 (ISC-style) terms:
@@ -9826,10 +9825,10 @@ SOFTWARE.
 MIT License (MIT)
 
 Used by:
-  - jsonschema-regex 0.60.0 (https://github.com/Stranger6667/jsonschema)
-  - jsonschema-value 0.60.0 (https://github.com/Stranger6667/jsonschema)
-  - jsonschema 0.60.0 (https://github.com/Stranger6667/jsonschema)
-  - referencing 0.60.0 (https://github.com/Stranger6667/jsonschema)
+  - jsonschema-regex 0.60.2 (https://github.com/Stranger6667/jsonschema)
+  - jsonschema-value 0.60.2 (https://github.com/Stranger6667/jsonschema)
+  - jsonschema 0.60.2 (https://github.com/Stranger6667/jsonschema)
+  - referencing 0.60.2 (https://github.com/Stranger6667/jsonschema)
 
 MIT License
 
@@ -10083,6 +10082,7 @@ USE OR OTHER DEALINGS IN THE SOFTWARE.
 MIT License (MIT)
 
 Used by:
+  - tokio-util 0.7.19 (https://github.com/tokio-rs/tokio)
   - tokio 1.54.1 (https://github.com/tokio-rs/tokio)
 
 MIT License

data/lib/mergify/rspec/configuration.rb

@@ -146,44 +146,87 @@ module Mergify
       # engine around `example.run`, and its outcome read off the exception it
       # left behind. Its first run is the example's first attempt, and the
       # tries a retry plugin makes after it, inside that run; flaky detection's
-      # reruns follow, each a try alone.
+      # reruns follow, or test retry's, each a try alone.
       def run_example(example, test_run)
         name = test_run.names.of(example)
         detector = test_run.flaky_detector
-        tries = run_first(example, name, test_run, detector)
-        return unless detector&.rerunning_test?(name)
+        retrier = test_run.test_retrier
+        retrier = nil unless retrier&.answers_for?(name)
+        tries = run_first(example, name, test_run, detector, retrier)
+        if detector&.rerunning_test?(name)
+          detect_flakiness(example, name, detector, retrier, tries)
+        elsif retrier
+          retry_failure(example, name, retrier, tries)
+        end
+      end
 
+      # Flaky detection reruns the example, within its budget. Where test
+      # retry answers for the example, the reruns answer its failure too.
+      def detect_flakiness(example, name, detector, retrier, tries)
         initial_exception = example.exception
         first_failure = tries.failure || initial_exception
-        first_failure = rerun(example, name, detector, first_failure) if detector.start_reruns(name)
+        first_failure = rerun(example, name, detector, retrier, first_failure) if detector.start_reruns(name)
 
         # Which attempt decides the result is the session's rule: in 'new'
         # mode any failure fails the test, the retry plugin's tries included,
         # so a new flaky test cannot be merged; in 'unhealthy' mode the reruns
-        # only learn, and the example's own result stands.
+        # only learn, and the example's own result stands, unless test retry
+        # says a rerun rescued it.
         final_exception = detector.rerun_failure_fails_the_test? ? first_failure : initial_exception
+        final_exception = nil if retrier&.rescued?(name)
         example.example.instance_variable_set(:@exception, final_exception)
       end
 
+      # Test retry runs the failed example again, each time the session grants
+      # one more attempt, until one passes: the example then passes, as a pass
+      # of a retry plugin's would. Otherwise its first run's failure stands.
+      # Each attempt is one try, as flaky detection's reruns are.
+      def retry_failure(example, name, retrier, tries)
+        failure = example.exception
+        return unless failure
+
+        without_retries(example) { run_retries(example, name, retrier, tries.last_ms) }
+        example.example.instance_variable_set(:@exception, retrier.rescued?(name) ? nil : failure)
+      end
+
+      # Each attempt the session grants the example, once the one before it
+      # failed after `attempt_ms`.
+      def run_retries(example, name, retrier, attempt_ms)
+        while retrier.grant?(name, attempt_ms)
+          reset_for_rerun(example)
+          attempt_ms = run_tries(example).last_ms
+          retrier.record(name, example.exception ? :failed : :passed, attempt_ms)
+        end
+      end
+
       # The first run of the example named `name`: its first attempt, timed by
       # the engine up to the second try where a retry plugin made one, then
       # each try after it, handed to the session once the first reached flaky
-      # detection. What the first attempt was is kept for its span.
-      def run_first(example, name, test_run, detector)
+      # detection and test retry. What the first attempt was is kept for its
+      # span.
+      def run_first(example, name, test_run, detector, retrier)
         detector&.attempt_started(name)
         tries = run_tries(example) { detector&.attempt_stopped(name) }
-        detector&.attempt_ended(name, first_failed?(example, tries) ? :failed : :passed)
+        first_attempt_ended(example, name, tries, detector, retrier)
         tries.later.each { |try| test_run.record_retry(name, try.status, try.duration_ms) }
         example.metadata[:mergify_first_attempt] = { try: tries.first, ended_at: tries.first_ended_at } if tries.first
         tries
       end
 
+      # The example's first attempt is over: flaky detection and test retry
+      # take it before the tries after it.
+      def first_attempt_ended(example, name, tries, detector, retrier)
+        status = first_failed?(example, tries) ? :failed : :passed
+        detector&.attempt_ended(name, status)
+        retrier&.record(name, status, tries.first_ms)
+      end
+
       # Run the example once, noting each try a retry plugin makes of it; the
       # block is called as a second one starts.
       def run_tries(example, &)
         tries = Tries.new(&)
         example.metadata[:mergify_tries] = tries
-        example.run
+        tries.time { example.run }
         tries
       ensure
         example.metadata.delete(:mergify_tries)
@@ -215,16 +258,16 @@ module Mergify
       end
 
       # Which attempt decides the result is the session's rule (see
-      # `run_example`): answers the first failure, the reruns' included.
+      # `detect_flakiness`): answers the first failure, the reruns' included.
       # Each rerun is one try: a retry plugin's tries would hide its failure
       # behind the one that passed.
-      def rerun(example, name, detector, first_failure)
+      def rerun(example, name, detector, retrier, first_failure)
         without_retries(example) do
           until example.metadata[:is_last_rerun]
             example.metadata[:is_last_rerun] = detector.last_rerun_for_test?(name)
             reset_for_rerun(example)
 
-            run_attempt(example, name, detector)
+            run_attempt(example, name, detector, retrier)
             first_failure ||= example.exception
           end
         end
@@ -260,13 +303,15 @@ module Mergify
       end
 
       # Runs what the around hook wraps -- the before and after hooks included
-      # -- timed by the session, and answers its outcome.
-      def run_attempt(example, name, detector)
+      # -- timed by the session, and answers its outcome; test retry, where it
+      # answers for the example, is told of it too.
+      def run_attempt(example, name, detector, retrier)
         detector.attempt_started(name)
-        example.run
+        attempt = run_tries(example)
         failed = example.exception || example.metadata.delete(:mergify_quarantined_failure)
         status = failed ? :failed : :passed
         detector.attempt_ended(name, status)
+        retrier&.record(name, status, attempt.last_ms)
         status
       end
 

data/lib/mergify/rspec/test_retry.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Mergify
+  module RSpec
+    # Test retry for one RSpec run, as its session prepared it.
+    #
+    # Which failures it answers for, whether an example runs once more, and
+    # whether a retry rescued it are the session's: on a push to the default
+    # branch, every failing example but the ones the quarantine holds and the
+    # ones Mergify classifies as broken, up to the repository's retries, within
+    # one budget for the run. What is left here is how RSpec runs an attempt,
+    # which `Configuration#retry_failure` does.
+    class TestRetrier
+      # `reruns` is the session's.
+      def initialize(reruns)
+        @reruns = reruns
+      end
+
+      # Whether a failure of the example named `name` is test retry's to
+      # answer for: false where it does not run.
+      def answers_for?(name)
+        @reruns.retry_answers_for?(name)
+      end
+
+      # An attempt at the example ended with `status` (`:passed` or
+      # `:failed`) after `duration_ms`: its first, a rerun of flaky
+      # detection's, or one of retry's. Its first reaches flaky detection
+      # through its phases, so test retry is told apart.
+      def record(name, status, duration_ms)
+        @reruns.record_retry_attempt(name, status.to_s, duration_ms)
+      end
+
+      # Whether the example, whose attempt just failed after `attempt_ms`,
+      # runs once more. Asked once that attempt is recorded, and once the
+      # retry plugin's tries of it are over.
+      def grant?(name, attempt_ms)
+        @reruns.grant_retry(name, attempt_ms)
+      end
+
+      # Whether an attempt after the example's first passed, rescuing it.
+      def rescued?(name)
+        @reruns.rescued?(name)
+      end
+    end
+  end
+end

data/lib/mergify/rspec/test_run.rb

@@ -3,6 +3,7 @@
 require_relative 'utils'
 require 'mergify/ci/core'
 require_relative 'flaky_detection'
+require_relative 'test_retry'
 require_relative 'names'
 require 'json'
 require 'rspec/core/version'
@@ -22,14 +23,14 @@ module Mergify
         # RSpec runs examples, in one process that can wait for every one put
         # first.
         fail_first: { unit: 'test', waits_for_all: true, workers_stop_the_run: false },
-        # RSpec reruns an example as it goes, and retries no failure, in one
-        # process that plans the run's budget once.
-        reruns: { plans_repeats: false, retries_failures: false },
+        # RSpec reruns an example as it goes, and retries a failed one in
+        # place, in one process that plans the run's budgets once.
+        reruns: { plans_repeats: false, retries_failures: true },
         framework: { name: 'rspec', version: ::RSpec::Core::Version::STRING, language: 'ruby' }
       }.freeze
 
       attr_reader :session, :repo_name, :test_run_id,
-                  :flaky_detector, :quarantined_tests,
+                  :flaky_detector, :test_retrier, :quarantined_tests,
                   :captured_session_verdict, :names
 
       # `doubles` has the session play Mergify in memory, answering and keeping
@@ -51,6 +52,7 @@ module Mergify
         @test_run_id = @session.run_id
         @reruns = nil
         @flaky_detector = nil
+        @test_retrier = nil
         @quarantined_tests = nil
         @captured_session_verdict = nil
         @examples_to_run = nil
@@ -150,15 +152,17 @@ module Mergify
         @session.summary(Utils.parallel_worker?)
       end
 
-      # Prepare flaky detection for the examples this run is about to run, by
-      # their names, each one Mergify's lists may still hold under its former
-      # names (`former_names`, a list by name). Which of them it reruns, and whether
-      # it runs at all, are the session's.
+      # Prepare flaky detection and test retry for the examples this run is
+      # about to run, by their names, each one Mergify's lists may still hold
+      # under its former names (`former_names`, a list by name). Which of them
+      # each reruns, and whether it runs at all, are the session's. No example
+      # opts out of test retry: RSpec's metadata has no key of Mergify's.
       def prepare_reruns(names, former_names)
         @session.state_former_names(former_names)
         @reruns = @session.prepare_reruns(names)
         detection = @reruns&.detection
         @flaky_detector = detection && FlakyDetector.new(@reruns, detection)
+        @test_retrier = @reruns && TestRetrier.new(@reruns)
       end
 
       # Account for the real outcome of the example known by `keys` against
@@ -184,8 +188,8 @@ module Mergify
       end
 
       # What a run that reports loads before its examples: the session loads
-      # the quarantine list for the branch they target, then flaky
-      # detection's context. Without a list (no branch, a failed fetch, no
+      # the quarantine list for the branch they target and the reruns'
+      # contexts, all at once. Without a list (no branch, a failed fetch, no
       # token) nothing is quarantined, and its block says why.
       def set_up
         require_relative 'quarantine'

data/lib/mergify/rspec/tries.rb

@@ -25,6 +25,15 @@ module Mergify
         @on_second = on_second
         @started_ms = nil
         @first_ended_at = nil
+        @run_ms = 0.0
+      end
+
+      # Run the block, the run of the example its tries are made in, timed.
+      def time
+        started_ms = now_ms
+        yield
+      ensure
+        @run_ms = now_ms - started_ms
       end
 
       def start
@@ -53,6 +62,18 @@ module Mergify
         @ended.drop(1)
       end
 
+      # How long the first try took; the whole run where no try started (a
+      # skipped example, an `around` hook that raised before running it).
+      def first_ms
+        first ? first.duration_ms : @run_ms
+      end
+
+      # How long the last try took, as `first_ms`: what the next attempt is
+      # expected to take.
+      def last_ms
+        @ended.empty? ? @run_ms : @ended.last.duration_ms
+      end
+
       # The first failure a try left on the example, the quarantine's aside.
       def failure
         @ended.find(&:exception)&.exception

data/lib/mergify/rspec/version.rb

@@ -8,6 +8,6 @@ module Mergify
     # which resolved against whatever repository happened to be the working
     # directory at load time -- the monorepo's namespaced tags here, the user's
     # own tags once the gem was installed.
-    VERSION = '2026.10.8'
+    VERSION = '2026.10.10'
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rspec-mergify
 version: !ruby/object:Gem::Version
-  version: 2026.10.8
+  version: 2026.10.10
 platform: ruby
 authors:
 - Mergify
@@ -29,14 +29,14 @@ dependencies:
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 2026.10.8
+        version: 2026.10.10
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 2026.10.8
+        version: 2026.10.10
 description: 'RSpec integration for Mergify Test Engine: OpenTelemetry tracing, flaky
   test detection, and test quarantine.'
 email:
@@ -56,6 +56,7 @@ files:
 - lib/mergify/rspec/formatter.rb
 - lib/mergify/rspec/names.rb
 - lib/mergify/rspec/quarantine.rb
+- lib/mergify/rspec/test_retry.rb
 - lib/mergify/rspec/test_run.rb
 - lib/mergify/rspec/tries.rb
 - lib/mergify/rspec/utils.rb