rspec-mergify 2026.10.5 → 2026.10.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +44 -33
- data/THIRD_PARTY_LICENSES +339 -58
- data/lib/mergify/rspec/ci_insights.rb +15 -324
- data/lib/mergify/rspec/configuration.rb +175 -101
- data/lib/mergify/rspec/failure_first.rb +150 -0
- data/lib/mergify/rspec/flaky_detection.rb +35 -196
- data/lib/mergify/rspec/formatter.rb +125 -157
- data/lib/mergify/rspec/native.rb +47 -20
- data/lib/mergify/rspec/quarantine.rb +16 -67
- data/lib/mergify/rspec/test_run.rb +207 -0
- data/lib/mergify/rspec/tries.rb +74 -0
- data/lib/mergify/rspec/utils.rb +20 -20
- data/lib/mergify/rspec/version.rb +1 -1
- data/lib/mergify/rspec.rb +1 -1
- data/lib/rspec_mergify.rb +6 -2
- metadata +8 -9
- data/lib/mergify/rspec/resources/rspec.rb +0 -26
- data/lib/mergify/rspec/session_verdict.rb +0 -86
- data/lib/mergify/rspec/test_selection.rb +0 -299
- data/lib/mergify/rspec/trace.rb +0 -143
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2c8b41d3c9e361bcea64889951b41d214d237d06a56ee0873d725f1106b9c21a
|
|
4
|
+
data.tar.gz: 86f2ccc52676008a737da28ae8877ec49b84f7dc4f484847f0f3d3c83f1496bf
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e9c95b28c61e098fc580b9a854f3bbc3d8f9dc94d5084bd98d5fd0fe3bcf7d090711c3e993651739501bfb5fa11ef091a92e5959cbb78098883bfad9b5ce8882
|
|
7
|
+
data.tar.gz: 0cb4388da6ad71ae24ca0315fb29dde8e7995c031b79f29c37a177da5c77df548ffe8216c5b8af5931f810eb39999eab5ad5145dc0c2b20022588629906c45c6
|
data/README.md
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
# rspec-mergify
|
|
2
2
|
|
|
3
|
-
RSpec plugin for [Mergify Test
|
|
3
|
+
RSpec plugin for [Mergify Test Engine](https://docs.mergify.com/test-engine/).
|
|
4
4
|
|
|
5
5
|
## Features
|
|
6
6
|
|
|
7
7
|
- **Test tracing** — Sends OpenTelemetry traces for every test to Mergify's API
|
|
8
8
|
- **Flaky test detection** — Intelligently reruns tests to detect flakiness with budget constraints
|
|
9
9
|
- **Test quarantine** — Quarantines failing tests so they don't block CI
|
|
10
|
-
- **Test selection** — Runs
|
|
10
|
+
- **Test selection** — Runs first the examples a merge-queue bisection step failed in the step above, and stops once one fails again
|
|
11
11
|
|
|
12
12
|
## Installation
|
|
13
13
|
|
|
@@ -40,7 +40,7 @@ gem 'rspec-mergify', '0.1.4'
|
|
|
40
40
|
|
|
41
41
|
Set the `MERGIFY_TOKEN` environment variable with your Mergify API token.
|
|
42
42
|
|
|
43
|
-
The plugin activates automatically when running in CI (detected via the `CI` environment variable). To enable outside CI, set `
|
|
43
|
+
The plugin activates automatically when running in CI (detected via the `CI` environment variable). To enable outside CI, set `MERGIFY_CI_ENABLE=true`.
|
|
44
44
|
|
|
45
45
|
### Environment Variables
|
|
46
46
|
|
|
@@ -48,49 +48,57 @@ The plugin activates automatically when running in CI (detected via the `CI` env
|
|
|
48
48
|
|---|---|---|
|
|
49
49
|
| `MERGIFY_TOKEN` | Mergify API authentication token | (required) |
|
|
50
50
|
| `MERGIFY_API_URL` | Mergify API endpoint | `https://api.mergify.com` |
|
|
51
|
-
| `
|
|
52
|
-
| `
|
|
51
|
+
| `MERGIFY_CI_ENABLE` | Force-enable outside CI | `false` |
|
|
52
|
+
| `MERGIFY_CI_DEBUG` | Print the session verdict and the spans instead of sending them, and upload nothing | `false` |
|
|
53
53
|
| `MERGIFY_TRACEPARENT` | W3C distributed trace context | — |
|
|
54
54
|
| `MERGIFY_TEST_JOB_NAME` | Mergify test job name | — |
|
|
55
55
|
| `MERGIFY_TEST_SELECTION_ENABLE` | Opt this job into test selection (see below) | `false` |
|
|
56
56
|
|
|
57
|
+
The names these replace, `RSPEC_MERGIFY_ENABLE` and `RSPEC_MERGIFY_DEBUG`, are
|
|
58
|
+
deprecated: they still work when the new name is not set, and the summary says
|
|
59
|
+
which one to switch to. Both take a boolean, like the variables that replace
|
|
60
|
+
them: `RSPEC_MERGIFY_DEBUG=false` leaves debug off.
|
|
61
|
+
|
|
57
62
|
### Test selection
|
|
58
63
|
|
|
59
|
-
When Mergify's merge queue
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
64
|
+
When Mergify's merge queue splits a failed batch, each bisection step only
|
|
65
|
+
needs to tell whether it is red. The gem asks Mergify whether the current run
|
|
66
|
+
is such a step and, if so, runs first the examples this job failed in the run
|
|
67
|
+
the step was split from (Fail First). The top-level groups holding them run
|
|
68
|
+
first, and inside each group they run before its other examples and child
|
|
69
|
+
groups — whatever the order (`--order rand`, `config.order`, your own
|
|
70
|
+
`register_ordering`, `order:` metadata), which still applies to everything
|
|
71
|
+
else. RSpec runs a group's whole subtree before the next group, and a group's
|
|
72
|
+
own examples before its child groups, so when they span several groups the
|
|
73
|
+
rest of each of those groups runs in between. If one of them fails again, the
|
|
74
|
+
step is red whatever the rest does, so the run stops once they have all run
|
|
75
|
+
(exit code 1, the remaining examples not run) and the queue can split again
|
|
76
|
+
without waiting for the suite. If none fails again, the rest of the suite runs
|
|
77
|
+
as usual. No example is ever skipped, so a green step still comes from a full
|
|
78
|
+
suite. A quarantined failure does not stop the run; a `--fail-fast` of your own
|
|
79
|
+
still stops at the first failure.
|
|
63
80
|
|
|
64
81
|
**This is off until you turn it on, per job.** Set
|
|
65
|
-
`MERGIFY_TEST_SELECTION_ENABLE=true` on the job you want
|
|
66
|
-
gem is not enough
|
|
67
|
-
|
|
68
|
-
|
|
82
|
+
`MERGIFY_TEST_SELECTION_ENABLE=true` on the job you want to run its failures
|
|
83
|
+
first. Installing the gem is not enough. Anything that is not a recognised yes
|
|
84
|
+
(unset, empty, `false`, unparsable) means no, and a job that has not opted in
|
|
85
|
+
never asks.
|
|
69
86
|
|
|
70
|
-
Once opted in
|
|
87
|
+
Once opted in:
|
|
71
88
|
|
|
72
|
-
- the
|
|
73
|
-
|
|
74
|
-
widen it;
|
|
89
|
+
- the order applies **after** RSpec's own filters — file and line arguments,
|
|
90
|
+
`--tag`, `-e`, `--only-failures` all still narrow the run;
|
|
75
91
|
- examples are matched by their id (`./spec/models/user_spec.rb[1:2]`), the
|
|
76
92
|
identity RSpec itself reruns failures by;
|
|
77
|
-
-
|
|
78
|
-
|
|
79
|
-
- if any example Mergify asks for is not collected here, the full suite runs;
|
|
80
|
-
- any error, timeout, or unrecognised answer runs the full suite;
|
|
81
|
-
- if Mergify refuses to pick a previous attempt (several runs of this job
|
|
82
|
-
report under one name), the run **fails** with Mergify's explanation: give
|
|
83
|
-
each run its own `MERGIFY_TEST_JOB_NAME`;
|
|
93
|
+
- any error, timeout, or unrecognised answer runs the full suite in its usual
|
|
94
|
+
order;
|
|
84
95
|
- at the end of the run the gem sends Mergify what it concluded (the counts
|
|
85
96
|
and the failing examples) — unless the run failed outside any example (a
|
|
86
|
-
failing `after(:context)` or suite hook), in which case
|
|
87
|
-
full suite.
|
|
97
|
+
failing `after(:context)` or suite hook), in which case the next bisection
|
|
98
|
+
step runs its full suite in its usual order.
|
|
88
99
|
|
|
89
100
|
Each parallel_tests worker, and each CI job handed a slice of the spec files,
|
|
90
|
-
|
|
91
|
-
collects. This needs the slices to be the same on both attempts, which is the
|
|
92
|
-
case for a split by file size or by a committed runtime log; Knapsack Pro's
|
|
93
|
-
Queue Mode hands files out dynamically and is not supported.
|
|
101
|
+
reports its own collection, by the examples it collects.
|
|
94
102
|
|
|
95
103
|
### Parallel runs
|
|
96
104
|
|
|
@@ -101,7 +109,7 @@ Queue Mode hands files out dynamically and is not supported.
|
|
|
101
109
|
|
|
102
110
|
Suites split across CI jobs (knapsack, `circleci tests split`, Buildkite `parallelism`) behave the same way, one session per job.
|
|
103
111
|
|
|
104
|
-
For detailed documentation, see the [official guide](https://docs.mergify.com/
|
|
112
|
+
For detailed documentation, see the [official guide](https://docs.mergify.com/test-engine/test-frameworks/rspec/).
|
|
105
113
|
|
|
106
114
|
## Development
|
|
107
115
|
|
|
@@ -120,13 +128,16 @@ bundle install
|
|
|
120
128
|
### Running Tests
|
|
121
129
|
|
|
122
130
|
```bash
|
|
123
|
-
bundle exec
|
|
131
|
+
bundle exec rake test
|
|
124
132
|
```
|
|
125
133
|
|
|
134
|
+
`rake test` compiles the extension in Cargo's dev profile first: running
|
|
135
|
+
`rspec` alone skips every binding example when the extension isn't built.
|
|
136
|
+
|
|
126
137
|
### Linting
|
|
127
138
|
|
|
128
139
|
```bash
|
|
129
|
-
bundle exec
|
|
140
|
+
bundle exec rake lint
|
|
130
141
|
```
|
|
131
142
|
|
|
132
143
|
## License
|