duel 0.1.7 → 0.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/AGENT_PROMPT.md +2 -2
- data/CHANGELOG.md +58 -1
- data/README.md +6 -1
- data/docs/ci.md +77 -76
- data/docs/configuration.md +3 -1
- data/docs/how-it-works.md +4 -1
- data/docs/plugins.md +7 -0
- data/lib/duel/cli.rb +51 -9
- data/lib/duel/comment.rb +235 -0
- data/lib/duel/config.rb +11 -2
- data/lib/duel/init.rb +116 -5
- data/lib/duel/runner.rb +20 -5
- data/lib/duel/summary.rb +11 -5
- data/lib/duel/templates/duel.gitlab-ci.yml.erb +8 -19
- data/lib/duel/templates/perf-comment.yml.erb +17 -42
- data/lib/duel/version.rb +1 -1
- data/lib/duel.rb +8 -0
- data/sig/duel/cli.rbs +11 -0
- data/sig/duel/comment.rbs +82 -0
- data/sig/duel/config.rbs +5 -0
- data/sig/duel/init.rbs +39 -1
- data/sig/duel/runner.rbs +3 -0
- data/sig/duel/summary.rbs +3 -1
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e153495685151ec755386e3fb34aeb21d3f9e3cb2015d4900b6d09f611939a25
|
|
4
|
+
data.tar.gz: 30c26bc5ca7ac5d36c1cb438ccfb4886fa4e9b4fb0637188b9a656dd15d9b231
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 3b902a527e27cce62d6ac90883410da63a3842c727b81656983031235baf44b27e7e28fe3b3de69ec9f7554215b94d74643af41d5f569c0567ef78e423625d70
|
|
7
|
+
data.tar.gz: 3e0255a630de657157d36087f12144d6dd2e0b556a06afd87d86e7ac0cd43fc7047522ec83431dd33369a96d083d546e2fa5608fb247970d5993cf0a2288174f
|
data/AGENT_PROMPT.md
CHANGED
|
@@ -201,8 +201,8 @@ at the API.
|
|
|
201
201
|
- `.github/workflows/perf-comment.yml` comments on a pull request
|
|
202
202
|
with Duel's report once the first has run (`c.comment` in
|
|
203
203
|
`.duel.rb` chooses when, and Duel also adds the report to the
|
|
204
|
-
job's summary). It runs none of the pull request's code
|
|
205
|
-
|
|
204
|
+
job's summary). It runs none of the pull request's code. Keep it as
|
|
205
|
+
it is.
|
|
206
206
|
- `duel init` writes the jobs for pushes (a comment on a pushed
|
|
207
207
|
commit, and a `duel/passed/` branch that marks the last commit
|
|
208
208
|
Duel passed) only where `.duel.rb` sets `c.fail_on =
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,62 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.1.8] - 2026-11-12
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- `duel comment`, which the Perf comment workflow now runs instead of
|
|
15
|
+
a shell script: it installs Duel from RubyGems (the newest release of
|
|
16
|
+
the same minor version) and comments with it, so a fix to how it
|
|
17
|
+
comments comes with a release of Duel, without a change to the
|
|
18
|
+
workflow. `duel init --check` shows how a project's workflows differ
|
|
19
|
+
from what the installed Duel writes.
|
|
20
|
+
- Once the Perf comment workflow posts a comment, it hides Duel's
|
|
21
|
+
earlier ones on the pull request as outdated (those not hidden
|
|
22
|
+
already), which GitHub collapses, so the newest is the one in view.
|
|
23
|
+
- `duel init` offers to create the label that `c.accept_label` names,
|
|
24
|
+
with `gh`, where it can (at a terminal, in a GitHub repository that
|
|
25
|
+
hasn't the label). Where it doesn't, its next steps say to.
|
|
26
|
+
- A plugin can say how much memory what it starts for each worker
|
|
27
|
+
takes at most (`def self.worker_memory`), and Duel counts at least
|
|
28
|
+
that much for each worker as it fits them in the memory available. A
|
|
29
|
+
server takes more as it's used than as it starts, when Duel measures
|
|
30
|
+
it.
|
|
31
|
+
- `duel comment --gitlab`, which the GitLab CI jobs' comment job now
|
|
32
|
+
runs instead of a shell script, as the GitHub workflow runs `duel
|
|
33
|
+
comment`: it trusts only what Duel said, comments only if the merge
|
|
34
|
+
request's head is still the run's, and comes from RubyGems, so a fix
|
|
35
|
+
comes with a release. In a merged results pipeline, the report names
|
|
36
|
+
the merge request's own commit, as on GitHub, and `duel init --gitlab`
|
|
37
|
+
offers to create the accept label with `glab`.
|
|
38
|
+
|
|
39
|
+
### Changed
|
|
40
|
+
|
|
41
|
+
- On a pull request's run on GitHub Actions, the report names the pull
|
|
42
|
+
request's own commit, "merged with `main`", instead of the commit
|
|
43
|
+
that merges it, which GitHub makes for the run and isn't in the pull
|
|
44
|
+
request.
|
|
45
|
+
- A run whose failures `c.accept_label` accepted says so in its
|
|
46
|
+
heading, under 🟡 instead of 🔻: "1 benchmark failed, accepted".
|
|
47
|
+
- The Perf comment workflow posts a report from a fork's pull request
|
|
48
|
+
as Markdown, like any other, instead of as text in a code block.
|
|
49
|
+
- A regression that `c.accept_label` accepts is commented on as a run
|
|
50
|
+
that passed would be. With the default `c.comment = :failures`, Duel
|
|
51
|
+
no longer posts the report again when the label is added, since the
|
|
52
|
+
last comment already showed it failing.
|
|
53
|
+
|
|
54
|
+
### Fixed
|
|
55
|
+
|
|
56
|
+
- On TruffleRuby, Duel fits the workers in the memory available as
|
|
57
|
+
they'll be once warm, not as they start: its workers grow some 2.9
|
|
58
|
+
times as its JIT compiles, so where 8 of each version seemed to fit,
|
|
59
|
+
they took 24 GB, more than a 16 GB CI runner has, and the runner
|
|
60
|
+
stopped answering.
|
|
61
|
+
|
|
62
|
+
To pick up the Perf comment workflow's changes, run `duel init --check`
|
|
63
|
+
(or see duel-rails's generator's template), and copy them into
|
|
64
|
+
`.github/workflows/perf-comment.yml`.
|
|
65
|
+
|
|
10
66
|
## [0.1.7] - 2026-11-10
|
|
11
67
|
|
|
12
68
|
### Fixed
|
|
@@ -706,7 +762,8 @@ needs Ruby 3.4 or later (or JRuby 10, or TruffleRuby 40).
|
|
|
706
762
|
Anything marked `@api private`, in lib/ and sig/, may change in any
|
|
707
763
|
release.
|
|
708
764
|
|
|
709
|
-
[Unreleased]: https://github.com/sferik/duel-ruby/compare/v0.1.
|
|
765
|
+
[Unreleased]: https://github.com/sferik/duel-ruby/compare/v0.1.8...HEAD
|
|
766
|
+
[0.1.8]: https://github.com/sferik/duel-ruby/compare/v0.1.7...v0.1.8
|
|
710
767
|
[0.1.7]: https://github.com/sferik/duel-ruby/compare/v0.1.6...v0.1.7
|
|
711
768
|
[0.1.6]: https://github.com/sferik/duel-ruby/compare/v0.1.5...v0.1.6
|
|
712
769
|
[0.1.5]: https://github.com/sferik/duel-ruby/compare/v0.1.4...v0.1.5
|
data/README.md
CHANGED
|
@@ -312,7 +312,12 @@ close to the limit to call doesn't fail the run, and is in the details.
|
|
|
312
312
|
A benchmark that the baseline can't run (a new one, say) is skipped, and
|
|
313
313
|
the heading counts it. A run that has nothing to compare, or no release
|
|
314
314
|
to compare against, says so under ⬜, and one that couldn't compare
|
|
315
|
-
says why under 🔻.
|
|
315
|
+
says why under 🔻. One whose failures the pull request's label accepted
|
|
316
|
+
(see [`c.accept_label`](docs/configuration.md#accept-a-regression-with-a-label))
|
|
317
|
+
says so under 🟡. On a pull request, GitHub Actions checks out a commit
|
|
318
|
+
that merges it with the branch it targets, which isn't in the pull
|
|
319
|
+
request, so the report names the pull request's own commit, "merged
|
|
320
|
+
with `main`".
|
|
316
321
|
|
|
317
322
|
The workflow that `duel init` writes posts this report as a comment on
|
|
318
323
|
the pull request when a benchmark fails, or when Duel couldn't compare.
|
data/docs/ci.md
CHANGED
|
@@ -193,7 +193,9 @@ check, because `duel` exits with a nonzero status, and the job's
|
|
|
193
193
|
summary has the report.
|
|
194
194
|
|
|
195
195
|
The workflow never edits or deletes a comment, so the pull request
|
|
196
|
-
keeps every report that it posted.
|
|
196
|
+
keeps every report that it posted. Once it posts one, it hides its
|
|
197
|
+
earlier ones as outdated, which GitHub collapses, so the newest is the
|
|
198
|
+
one in view. Each comment starts with a hidden
|
|
197
199
|
marker that records the run's status: `<!-- duel status=passed -->`,
|
|
198
200
|
`status=failed` (a run that couldn't compare too), or `status=neutral`
|
|
199
201
|
(nothing to compare). When a run's outcome wouldn't get a comment of
|
|
@@ -228,16 +230,14 @@ the pull request's number, `comment`, and `status`. A second workflow,
|
|
|
228
230
|
the Perf workflow completes (`workflow_run`), with your repository's
|
|
229
231
|
own copy of the workflow and a token that can write, so a pull request
|
|
230
232
|
from a fork gets a comment too. It checks nothing out and runs none of
|
|
231
|
-
the pull request's code.
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
`github-actions[bot]`, it can't mention anyone, link anywhere or show an
|
|
240
|
-
image. A report too long for a comment is cut short. When the Duel job fails before it compares, there's
|
|
233
|
+
the pull request's code. It installs Duel from RubyGems (the newest
|
|
234
|
+
0.1 release, 0.1.8 or later) and runs `duel comment`, so a fix to how it comments comes
|
|
235
|
+
with a release of Duel, without a change to the workflow. The pull
|
|
236
|
+
request's code ran in the Duel job, and could have written anything
|
|
237
|
+
into the artifact, so `duel comment` trusts only what Duel could have
|
|
238
|
+
said there: a number that names the pull request whose head the run
|
|
239
|
+
compared, and a `comment` and a `status` that Duel uses. The report's
|
|
240
|
+
Markdown, a fork's too, is the comment's body. When the Duel job fails before it compares, there's
|
|
241
241
|
no report, and no comment. GitHub runs a `workflow_run` workflow only
|
|
242
242
|
from your default branch, so the pull request that adds the workflow
|
|
243
243
|
gets no comment.
|
|
@@ -249,14 +249,15 @@ name: Perf comment
|
|
|
249
249
|
# Duel's report, as a new comment on the pull request, when Duel says to:
|
|
250
250
|
# always (post), or only when its last comment's status differs from the
|
|
251
251
|
# run's (change), so the latest comment is never stale. It never edits or
|
|
252
|
-
# deletes a comment
|
|
253
|
-
#
|
|
254
|
-
#
|
|
255
|
-
#
|
|
256
|
-
#
|
|
257
|
-
#
|
|
258
|
-
#
|
|
259
|
-
#
|
|
252
|
+
# deletes a comment, but hides Duel's earlier ones as outdated, so the
|
|
253
|
+
# newest is the one in view. It runs once the Perf workflow has, with
|
|
254
|
+
# this repository's own workflow and a token that can write, so a pull
|
|
255
|
+
# request from a fork gets a comment too. It checks nothing out and runs
|
|
256
|
+
# none of the pull request's code, only `duel comment` (see Duel's
|
|
257
|
+
# docs/ci.md), which trusts only what the Duel job could have said in
|
|
258
|
+
# what it kept: the pull request's number, if it names the pull request
|
|
259
|
+
# the run compared, and the comment and the status, if they're ones Duel
|
|
260
|
+
# says. Its report, a fork's too, is the comment, as Markdown.
|
|
260
261
|
#
|
|
261
262
|
# And when the label that accepts a regression (c.accept_label in
|
|
262
263
|
# .duel.rb) is added to a pull request, or taken off, it runs the Duel
|
|
@@ -294,46 +295,20 @@ jobs:
|
|
|
294
295
|
path: ${{ runner.temp }}/report
|
|
295
296
|
run-id: ${{ github.event.workflow_run.id }}
|
|
296
297
|
github-token: ${{ github.token }}
|
|
298
|
+
- uses: ruby/setup-ruby@v1
|
|
299
|
+
if: steps.report.outcome == 'success'
|
|
300
|
+
with:
|
|
301
|
+
ruby-version: ruby
|
|
302
|
+
# Duel's own code, from RubyGems (none of the pull request's), so a
|
|
303
|
+
# fix to how it comments comes with a release of Duel.
|
|
297
304
|
- name: Comment on the pull request
|
|
298
305
|
if: steps.report.outcome == 'success'
|
|
299
306
|
env:
|
|
300
307
|
GH_TOKEN: ${{ github.token }}
|
|
301
308
|
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
|
|
302
309
|
run: |
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
COMMENT=$(said comment 'post|change')
|
|
306
|
-
STATUS=$(said status 'passed|failed|neutral')
|
|
307
|
-
[ -n "$PR" ] && [ -n "$COMMENT" ] && [ -n "$STATUS" ] || exit 0
|
|
308
|
-
# (The pull request's head commit, and the repository it's from.)
|
|
309
|
-
head=$(gh api "repos/$GITHUB_REPOSITORY/pulls/$PR" --jq '"\(.head.sha) \(.head.repo.full_name)"')
|
|
310
|
-
[ "${head%% *}" = "$HEAD_SHA" ] || exit 0
|
|
311
|
-
# The report, as Markdown for a branch of this repository (whose
|
|
312
|
-
# author can push here anyway), but for a fork's, as text: its run
|
|
313
|
-
# ran the fork's code, which could have written anything in it. In
|
|
314
|
-
# a code block, fenced with more backticks than any run of them in
|
|
315
|
-
# the report, it can't mention anyone, link anywhere or show an
|
|
316
|
-
# image. It's cut short where a comment couldn't hold it, and loses
|
|
317
|
-
# any NUL bytes, which GitHub wouldn't take, and grep reads it as
|
|
318
|
-
# text whatever's in it, so it finds every run of backticks.
|
|
319
|
-
report() {
|
|
320
|
-
if [ "${head#* }" = "$GITHUB_REPOSITORY" ]; then cat "$RUNNER_TEMP/report/duel.md" && return; fi
|
|
321
|
-
text="$RUNNER_TEMP/report.txt"
|
|
322
|
-
LC_ALL=C tr -d '\000' < "$RUNNER_TEMP/report/duel.md" | head -c 60000 > "$text"
|
|
323
|
-
fence=$(LC_ALL=C grep -ao '``*' "$text" | awk '{ if (length > n) n = length } END { n = (n < 3) ? 3 : n + 1; while (n-- > 0) printf "`" }')
|
|
324
|
-
printf 'From a fork, so shown as text:\n\n%s\n' "$fence"
|
|
325
|
-
cat "$text"
|
|
326
|
-
printf '\n%s\n' "$fence"
|
|
327
|
-
}
|
|
328
|
-
marker="<!-- duel"
|
|
329
|
-
# The first line of the last comment Duel posted here: its marker,
|
|
330
|
-
# with its status (none in Duel 0.1.0's, which counts as another).
|
|
331
|
-
last=$(gh api --paginate "repos/$GITHUB_REPOSITORY/issues/$PR/comments" \
|
|
332
|
-
--jq ".[] | select(.user.login == \"github-actions[bot]\") | .body | split(\"\n\")[0] | rtrimstr(\"\r\") | select(startswith(\"$marker status=\") or . == \"$marker -->\")" | tail -n 1)
|
|
333
|
-
if [ "$COMMENT" = post ] || { [ -n "$last" ] && [ "$last" != "$marker status=$STATUS -->" ]; }; then
|
|
334
|
-
{ echo "$marker status=$STATUS -->"; report; } > "$RUNNER_TEMP/comment.md"
|
|
335
|
-
gh api --method POST "repos/$GITHUB_REPOSITORY/issues/$PR/comments" -F body=@"$RUNNER_TEMP/comment.md" > /dev/null
|
|
336
|
-
fi
|
|
310
|
+
gem install duel --version '~> 0.1.8' --no-document
|
|
311
|
+
duel comment --report "$RUNNER_TEMP/report" --head "$HEAD_SHA" --marker 'duel'
|
|
337
312
|
|
|
338
313
|
again:
|
|
339
314
|
name: Run Duel again
|
|
@@ -442,7 +417,10 @@ label is added to a pull request or taken off. For the label `.duel.rb`
|
|
|
442
417
|
names, it waits for the Perf run of the pull request's head commit to
|
|
443
418
|
finish. Then it runs the failed jobs again where the label was added
|
|
444
419
|
to a run that failed, or the whole run again where the label was
|
|
445
|
-
taken off one that passed.
|
|
420
|
+
taken off one that passed. The run the label accepts is commented on
|
|
421
|
+
as one that passed would be: with the default `c.comment = :failures`,
|
|
422
|
+
only where the last comment's status isn't its own, so the report that
|
|
423
|
+
failed isn't posted again. It checks nothing out and runs none of the
|
|
446
424
|
pull request's code, and its only permission is `actions: write`. The
|
|
447
425
|
workflows name the label as `.duel.rb` did when `duel init` wrote them,
|
|
448
426
|
so if you rename it, rename it in both workflows too.
|
|
@@ -740,6 +718,27 @@ that branch, which a branch of its own marks: `duel/passed/2-x-stable`.
|
|
|
740
718
|
A pull request into the branch is compared with the branch, whatever
|
|
741
719
|
the workflow guards.
|
|
742
720
|
|
|
721
|
+
## Keep the workflows up to date
|
|
722
|
+
|
|
723
|
+
A release of Duel can change the workflows that `duel init` writes. To
|
|
724
|
+
see how yours differ from what the installed Duel writes, run it with
|
|
725
|
+
`--check`, and the options you ran it with (such as `--badge`):
|
|
726
|
+
|
|
727
|
+
```sh
|
|
728
|
+
bundle exec duel init --check
|
|
729
|
+
```
|
|
730
|
+
|
|
731
|
+
It says whether each workflow is the same, differs (and shows how, as
|
|
732
|
+
`diff -u` does), or is missing, and writes nothing. It exits with 1
|
|
733
|
+
where one isn't the same. Copy in the changes you want, so that steps
|
|
734
|
+
you added to the Duel job (to install a database server, say) stay.
|
|
735
|
+
With `--gitlab`, it checks the GitLab CI jobs. In a Rails app, run
|
|
736
|
+
duel-rails's generator with `--check` instead:
|
|
737
|
+
|
|
738
|
+
```sh
|
|
739
|
+
bin/rails generate duel:install --check
|
|
740
|
+
```
|
|
741
|
+
|
|
743
742
|
## Run Duel on GitLab CI
|
|
744
743
|
|
|
745
744
|
`duel init --gitlab` writes GitLab CI jobs, in
|
|
@@ -784,9 +783,22 @@ log and its `duel.md` artifact have the report.
|
|
|
784
783
|
protected, so only your protected branches' pipelines have it, to
|
|
785
784
|
comment on a pushed commit and to mark a push passed.
|
|
786
785
|
|
|
787
|
-
The comment on a
|
|
788
|
-
|
|
789
|
-
|
|
786
|
+
The comment on a merge request comes from `duel comment --gitlab`,
|
|
787
|
+
installed from RubyGems in a `ruby:slim` image, as on GitHub, so a fix
|
|
788
|
+
to it comes with a release of Duel. It trusts only a `DUEL_COMMENT` and
|
|
789
|
+
a `DUEL_STATUS` that Duel says, and comments only if the merge
|
|
790
|
+
request's head is still the one the pipeline compared. In a merged
|
|
791
|
+
results pipeline, the report names the merge request's own commit,
|
|
792
|
+
"merged with `main`", as it does on GitHub.
|
|
793
|
+
|
|
794
|
+
Three things GitLab does differently. Duel can't collapse its earlier
|
|
795
|
+
comments there, because only the Developer role can resolve a thread,
|
|
796
|
+
and the merge request's token has the Reporter role. Adding the label
|
|
797
|
+
that accepts a regression doesn't run the pipeline again, as GitLab has
|
|
798
|
+
no pipeline for a label, so run it again yourself (or push). And a
|
|
799
|
+
merge request from a fork runs its pipeline in the fork, without your
|
|
800
|
+
variables, so it gets no comment. The comment on a pushed commit looks
|
|
801
|
+
only at the comments on the commit before it for its last status.
|
|
790
802
|
|
|
791
803
|
```yaml
|
|
792
804
|
# .gitlab/duel.gitlab-ci.yml
|
|
@@ -853,9 +865,11 @@ duel:
|
|
|
853
865
|
# The report, as a new comment on the merge request, when Duel says to:
|
|
854
866
|
# always (post), or only when its last comment's status differs from this
|
|
855
867
|
# run's (change), so the latest comment is never stale. It never edits or
|
|
856
|
-
# deletes a comment.
|
|
868
|
+
# deletes a comment. It runs `duel comment`, from RubyGems (none of the
|
|
869
|
+
# merge request's code), so a fix to how it comments comes with a release
|
|
870
|
+
# of Duel.
|
|
857
871
|
duel:comment:
|
|
858
|
-
image:
|
|
872
|
+
image: ruby:slim
|
|
859
873
|
stage: .post
|
|
860
874
|
needs: [duel]
|
|
861
875
|
rules:
|
|
@@ -864,23 +878,10 @@ duel:comment:
|
|
|
864
878
|
variables:
|
|
865
879
|
GIT_STRATEGY: none # it runs none of the code
|
|
866
880
|
script:
|
|
867
|
-
-
|
|
868
|
-
|
|
869
|
-
|
|
870
|
-
|
|
871
|
-
[ -f "$report" ] || exit 0
|
|
872
|
-
api="$CI_API_V4_URL/projects/$CI_PROJECT_ID"
|
|
873
|
-
token="PRIVATE-TOKEN: $DUEL_COMMENT_TOKEN"
|
|
874
|
-
marker="<!-- duel"
|
|
875
|
-
me=$(curl -sf -H "$token" "$CI_API_V4_URL/user" | jq .id)
|
|
876
|
-
# The first line of the last comment Duel posted here: its marker,
|
|
877
|
-
# with its status.
|
|
878
|
-
last=$(curl -sf -H "$token" "$api/merge_requests/$CI_MERGE_REQUEST_IID/notes?sort=desc&order_by=created_at&per_page=100" |
|
|
879
|
-
jq -r --argjson me "$me" --arg marker "$marker status=" '.[] | select(.author.id == $me) | .body | split("\n")[0] | rtrimstr("\r") | select(startswith($marker))' | head -n 1)
|
|
880
|
-
if [ "$DUEL_COMMENT" = post ] || { [ -n "$last" ] && [ "$last" != "$marker status=$DUEL_STATUS -->" ]; }; then
|
|
881
|
-
{ echo "$marker status=$DUEL_STATUS -->"; cat "$report"; } | jq -Rs '{body: .}' |
|
|
882
|
-
curl -sf -H "$token" -H "Content-Type: application/json" --data @- "$api/merge_requests/$CI_MERGE_REQUEST_IID/notes" > /dev/null
|
|
883
|
-
fi
|
|
881
|
+
- gem install duel --version '~> 0.1.8' --no-document
|
|
882
|
+
# (The merge request's head: in a merged results pipeline, not the
|
|
883
|
+
# commit that merges it.)
|
|
884
|
+
- duel comment --gitlab --report . --head "${CI_MERGE_REQUEST_SOURCE_BRANCH_SHA:-$CI_COMMIT_SHA}" --marker 'duel'
|
|
884
885
|
|
|
885
886
|
# The report, as a new comment on a pushed commit, when Duel says to, as
|
|
886
887
|
# for a merge request: only where a regression in a push passes (see
|
data/docs/configuration.md
CHANGED
|
@@ -139,7 +139,9 @@ end
|
|
|
139
139
|
On GitHub Actions, the [workflow](ci.md#accept-a-regression-with-a-label)
|
|
140
140
|
that `duel init` writes passes Duel the pull request's labels. With
|
|
141
141
|
that label, a regression passes, with a warning, and the report says
|
|
142
|
-
that the label accepted it.
|
|
142
|
+
that the label accepted it. It comments on that run as it would on one
|
|
143
|
+
that passed, so by default, after a comment on the run that failed, it
|
|
144
|
+
doesn't comment again. The labels are read when the workflow runs,
|
|
143
145
|
so the comment workflow runs the Duel job again when you add the label
|
|
144
146
|
to a pull request whose run failed, or take it off one whose run
|
|
145
147
|
passed. For a push, Duel reads the labels of the pull request that
|
data/docs/how-it-works.md
CHANGED
|
@@ -62,7 +62,10 @@ never on noise, so that it can gate every pull request:
|
|
|
62
62
|
Running out would make the machine swap, or stop a CI runner from
|
|
63
63
|
answering, so its job is lost. To find what they take, Duel starts
|
|
64
64
|
one worker of each version first, and allows each pair a quarter
|
|
65
|
-
more than that
|
|
65
|
+
more than that (on TruffleRuby, whose workers grow as its JIT warms
|
|
66
|
+
up, 2.5 times that), or what its plugins say each worker's resources
|
|
67
|
+
take at most, if that's more (a server takes more as it's used), and
|
|
68
|
+
1 GB for everything else. Its first line says
|
|
66
69
|
when it lowered either. A `--jobs` or `--processes` you give is
|
|
67
70
|
kept as it is.
|
|
68
71
|
- **Measured again before failing.** Duel measures whatever looks
|
data/docs/plugins.md
CHANGED
|
@@ -77,6 +77,13 @@ When you write a plugin, follow these guidelines:
|
|
|
77
77
|
- Set up what a whole version needs once, such as its bundle, with
|
|
78
78
|
`c.prepare(:version)`, not in each worker's `c.prepare(:worker)`. Run
|
|
79
79
|
its commands on the version's Ruby: `v.env` puts it first on `PATH`.
|
|
80
|
+
- If each worker's `c.prepare(:worker)` starts something that takes a
|
|
81
|
+
lot of memory once it's used, such as a database server, say how much
|
|
82
|
+
at most, in bytes, as the module's `worker_memory`, for example
|
|
83
|
+
`def self.worker_memory = 450 << 20`. Duel fits its workers in the
|
|
84
|
+
memory available by what one of each version takes as it starts, and
|
|
85
|
+
a server takes more as it's used, so Duel counts at least this much
|
|
86
|
+
for each worker. A version of Duel before 0.1.8 ignores it.
|
|
80
87
|
|
|
81
88
|
> [!IMPORTANT]
|
|
82
89
|
> A helper that's meant for setup, such as one that runs a query, must
|
data/lib/duel/cli.rb
CHANGED
|
@@ -10,11 +10,13 @@ require_relative "lock"
|
|
|
10
10
|
require_relative "summary"
|
|
11
11
|
require_relative "json_report"
|
|
12
12
|
require_relative "init"
|
|
13
|
+
require_relative "comment"
|
|
13
14
|
|
|
14
15
|
module Duel
|
|
15
|
-
# The `duel` command (and `duel init`: see Init
|
|
16
|
-
# rake task or a CI expression can't pass
|
|
17
|
-
# the environment (DUEL_BASE, DUEL_FILTER,
|
|
16
|
+
# The `duel` command (and `duel init` and `duel comment`: see Init and
|
|
17
|
+
# Comment). What a run behind a rake task or a CI expression can't pass
|
|
18
|
+
# as an option can be set in the environment (DUEL_BASE, DUEL_FILTER,
|
|
19
|
+
# DUEL_FORCE), and options win.
|
|
18
20
|
# On GitHub Actions (GITHUB_STEP_SUMMARY), it adds its report to the
|
|
19
21
|
# job's summary, in Markdown (see Summary), and tells the step's
|
|
20
22
|
# outputs (GITHUB_OUTPUT) what to do about the comment on a pull
|
|
@@ -24,8 +26,9 @@ module Duel
|
|
|
24
26
|
# outcome, but not for a command line that isn't valid.
|
|
25
27
|
# @api private
|
|
26
28
|
class CLI
|
|
27
|
-
USAGE = "Usage: duel [options]\n duel init [options]\n\n" \
|
|
28
|
-
"Fails if this tree is slower than a baseline at any benchmark. `duel init` sets Duel up in a project
|
|
29
|
+
USAGE = "Usage: duel [options]\n duel init [options]\n duel comment [options]\n\n" \
|
|
30
|
+
"Fails if this tree is slower than a baseline at any benchmark. `duel init` sets Duel up in a project, and\n" \
|
|
31
|
+
"`duel comment` is what its Perf comment workflow runs.\n\n"
|
|
29
32
|
|
|
30
33
|
# Where GitHub Actions has a step add to its job's summary.
|
|
31
34
|
STEP_SUMMARY = "GITHUB_STEP_SUMMARY"
|
|
@@ -73,6 +76,7 @@ module Duel
|
|
|
73
76
|
# warned (see WARNED).
|
|
74
77
|
def run
|
|
75
78
|
return Init.new(@argv.drop(1), out: @out, err: @err).run if @argv.first == "init"
|
|
79
|
+
return Comment.new(@argv.drop(1), env: @env, out: @out, err: @err).run if @argv.first == "comment"
|
|
76
80
|
|
|
77
81
|
settings = @settings = catch(:version) { parse } or return 0
|
|
78
82
|
config = @config = settings.config_path ? Duel.load_config(settings.config_path) : Config.new
|
|
@@ -171,7 +175,7 @@ module Duel
|
|
|
171
175
|
passed ? "passed" : "failed",
|
|
172
176
|
-> {
|
|
173
177
|
Summary.new(runner, results, settings: settings, baseline: described_baseline(config, settings, baseline),
|
|
174
|
-
candidate: described_candidate(config, settings), note: @note).markdown
|
|
178
|
+
candidate: described_candidate(config, settings), note: @note, accepted: @accepted).markdown
|
|
175
179
|
},
|
|
176
180
|
->(comment) {
|
|
177
181
|
JSONReport.new(runner, results, settings: settings, exit: status, comment: comment,
|
|
@@ -190,6 +194,7 @@ module Duel
|
|
|
190
194
|
if accepted?(config)
|
|
191
195
|
# (Set: see #accepted?.)
|
|
192
196
|
label = config.accept_label #: String
|
|
197
|
+
@accepted = true
|
|
193
198
|
return passing("accepted with the #{label} label", "Accepted with the #{Summary.code(label)} label.")
|
|
194
199
|
end
|
|
195
200
|
return 1 unless pull_requests_only?(config, settings)
|
|
@@ -303,14 +308,17 @@ module Duel
|
|
|
303
308
|
# JSON report's: as --comment says, or else the configuration, or the
|
|
304
309
|
# default where the configuration didn't load. It comments on a push
|
|
305
310
|
# only where a regression in one passes (Config#fail_on), so the
|
|
306
|
-
# comment says what the check doesn't.
|
|
311
|
+
# comment says what the check doesn't. A regression the label
|
|
312
|
+
# accepts (Config#accept_label) is commented on as a run that passed
|
|
313
|
+
# would be: whoever added the label knows, so by default it posts
|
|
314
|
+
# only where the last comment's status isn't its own.
|
|
307
315
|
def commenting(status)
|
|
308
316
|
# (Parsed, so set: see #run.)
|
|
309
317
|
settings = @settings #: Settings
|
|
310
318
|
config = @config || Config.new
|
|
311
319
|
return "none" if pushed? && !pull_requests_only?(config, settings)
|
|
312
320
|
|
|
313
|
-
CLI.comment(settings.comment || config.comment, status)
|
|
321
|
+
CLI.comment(settings.comment || config.comment, @accepted ? "passed" : status)
|
|
314
322
|
end
|
|
315
323
|
|
|
316
324
|
# Writes the reports of a run whose +status+ is the JSON report's
|
|
@@ -392,7 +400,41 @@ module Duel
|
|
|
392
400
|
return Summary.code(dir) unless found.success?
|
|
393
401
|
|
|
394
402
|
_, clean = git(dir, "diff", "--quiet", "HEAD", "--", *Baseline.new(config, settings).watched)
|
|
395
|
-
"#{head.chomp}
|
|
403
|
+
return "#{head.chomp} (with changes not committed)" unless clean.success?
|
|
404
|
+
|
|
405
|
+
merged(dir) || head.chomp
|
|
406
|
+
end
|
|
407
|
+
|
|
408
|
+
# On GitHub Actions, a pull request's run checks out the commit that
|
|
409
|
+
# merges it with the branch it targets, which isn't in the pull
|
|
410
|
+
# request, so its head is named instead, as merged (or nil, where HEAD
|
|
411
|
+
# is no such commit). So does a merge request's merged results
|
|
412
|
+
# pipeline, on GitLab CI.
|
|
413
|
+
def merged(dir)
|
|
414
|
+
pull = (gitlab? ? merge_request : pull_request) or return
|
|
415
|
+
sha, branch = pull.dig("head", "sha"), pull.dig("base", "ref")
|
|
416
|
+
return unless branch.instance_of?(String) && git(dir, "rev-parse", "HEAD^2").first.chomp == sha
|
|
417
|
+
|
|
418
|
+
"#{git(dir, "rev-parse", "--short", sha).first.chomp} (merged with #{Summary.code(branch)})"
|
|
419
|
+
end
|
|
420
|
+
|
|
421
|
+
# A GitLab merge request's merged results pipeline's head and the
|
|
422
|
+
# branch it targets, as #pull_request has a pull request's (each nil
|
|
423
|
+
# in a merge request's other pipelines, whose HEAD is its head).
|
|
424
|
+
def merge_request = {"head" => {"sha" => variable("CI_MERGE_REQUEST_SOURCE_BRANCH_SHA")},
|
|
425
|
+
"base" => {"ref" => variable("CI_MERGE_REQUEST_TARGET_BRANCH_NAME")}}
|
|
426
|
+
|
|
427
|
+
# The pull request a pull_request event's run is for, as GitHub
|
|
428
|
+
# Actions describes it (GITHUB_EVENT_PATH), or nil.
|
|
429
|
+
def pull_request
|
|
430
|
+
path = variable("GITHUB_EVENT_PATH") if event == "pull_request"
|
|
431
|
+
return unless path && File.file?(path)
|
|
432
|
+
|
|
433
|
+
payload = JSON.parse(File.read(path))
|
|
434
|
+
pull = payload["pull_request"] if payload.instance_of?(Hash)
|
|
435
|
+
pull if pull.instance_of?(Hash)
|
|
436
|
+
rescue JSON::ParserError
|
|
437
|
+
nil
|
|
396
438
|
end
|
|
397
439
|
|
|
398
440
|
# The baseline, for JSON: the ref that named it (the main branch,
|