duel 0.1.1 → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: da30e26518ff9b626552115d24595276b5f6f371cc4cd9c614b126666a065898
4
- data.tar.gz: d863cf940c61196483a7521ec8066385e4b2cc9071ec00c96bc6f794c266342c
3
+ metadata.gz: 3827f9485e70bb95c50b32ff9438042bd60cb26885d92f01fe6c1dc6de33ee72
4
+ data.tar.gz: 53e410c89687f6bcdec3c46be4d370409110ac134d90ef74ca8aaaad62394a7e
5
5
  SHA512:
6
- metadata.gz: 777b2dc46c93019d739aed99764ac70ecdf761e1309944cdacf449b735b6a931d965fa3fc3b37141874dfeb9b8074ce120f070e4e764ecc15f9b1edbe00617d0
7
- data.tar.gz: 2879233c357cfd504d0b872efd37495cdf64e115da6ed9917fe1b0fad6ad0b7f070e8c2ed9b1cdf5b4b7444a3d7dbfeaa3155fe1cac58b425890dca99ce4f19a
6
+ metadata.gz: 7710b0ab9cc84821d6e66af1d66e1005b98aa0e97a1f047e2a76d6c5d3a83defb413895e11476d067265b97d589dcbc2fb5bdf78550616da74cf92df0da2bd19
7
+ data.tar.gz: 6d9d0df5accaf420f0a4e554f21eb22e79e77f8865986af856759272c5358b6cacd22af27800827d6faf7360daeded8d58481a6ef3c22669279d13a85fe4ff83
data/CHANGELOG.md CHANGED
@@ -7,6 +7,137 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.1.3] - 2026-11-04
11
+
12
+ ### Added
13
+
14
+ - The workflow that `duel init` writes keeps a badge for the README
15
+ that turns red when Duel fails a push to the default branch. Its
16
+ Update the badge job writes `badge.svg` to a branch of its own,
17
+ `duel/badge`, green ("perf guarded") when Duel passed the push, and
18
+ red ("perf regressed") when the Duel job failed or found a
19
+ regression, even one that passed.
20
+ - A run measures for 30 minutes at most (`--time-limit SECONDS`), not
21
+ counting the builds. Past that, it starts no more measurements: a
22
+ benchmark it hasn't measured fails as an error, and one that looked
23
+ worse is judged on the measurements it has, so a run cut short never
24
+ passes a regression. The report (and the JSON report's `cut_short`)
25
+ says what it left undone. A benchmark that starts processes of its own
26
+ and sat near its limit could be measured again for hours before.
27
+
28
+ ### Changed
29
+
30
+ - The workers, their directories and sockets, and the resources that
31
+ prepare blocks get are named `base` and `head` (`base-1`, `head-1`),
32
+ not `baseline` and `candidate`: `Resource#name` (`w.name`, `v.name`)
33
+ says so, and `Duel::Testing.prepare` names its worker `head-1` by
34
+ default. The report still says baseline and candidate.
35
+ - A run removes its lock (`tmp/duel/lock`) when it ends, and
36
+ `tmp/duel` (and `tmp`, if Duel made it) when that leaves them empty, so
37
+ a project whose plugins build nothing to keep has nothing left there.
38
+ - When a run ends, the builds that prepare blocks made once
39
+ (`Resource#cached`) and the run used replace the others of their name:
40
+ a key's name is what's before its version and the hash it ends with
41
+ (`mysql`, of `mysql-0.1.2-3f9c2a1b7e5d4c6a`). A database server or a
42
+ plugin upgraded left its old build in `tmp/duel/cache` before, tens
43
+ of megabytes (82 MB for MySQL's) each time.
44
+
45
+ ### Fixed
46
+
47
+ - The candidate's workers could measure slower in every pair, with the
48
+ same instruction counts, on code that hadn't changed. Their names
49
+ (`candidate-1`, against `baseline-1`) were a byte longer, and so were
50
+ the paths that a plugin's servers put in their environment
51
+ (duel-postgresql's `DATABASE_URL`, `PGHOST` and
52
+ `DUEL_POSTGRESQL_URL`, say). Where a process's stack starts depends on
53
+ how long its arguments and environment are, and at some lengths, a
54
+ tight loop runs the same instructions in more cycles: on macOS,
55
+ server-free benchmarks of pg came out 1.2 to 1.55 times slower on the
56
+ candidate with the plugin, and the same as the baseline without it.
57
+ The two versions' names are now as long as each other.
58
+
59
+ - The workflow that `duel init` writes (and its GitLab CI jobs) no
60
+ longer moves the branch that marks the last commit Duel passed
61
+ backward. When two pushes ran at once and the older one finished
62
+ last, the branch went back to the older push, so the next push was
63
+ compared with it, and a regression already accepted counted again.
64
+ Now the branch only moves forward, or after a force push rewrites
65
+ its history.
66
+ - Where a branch named `duel` (or `duel/passed`) keeps the workflow that
67
+ `duel init` writes from making `duel/passed/main` or `duel/badge`, as
68
+ Git can't have both, the job's error now names that branch and says to
69
+ delete it, instead of GitHub's bare `Reference update failed`.
70
+ - On JRuby, peak memory (`peak_memory: true`) is the most of the JVM's
71
+ heap that a benchmark had in use, not resident memory, which barely
72
+ moves there, as the JVM takes memory for its heap ahead of time: a
73
+ benchmark that needed 24 MB more could pass. JRuby now has peak
74
+ memory on macOS too.
75
+
76
+ ## [0.1.2] - 2026-11-02
77
+
78
+ ### Added
79
+
80
+ - `c.fail_on = :pull_requests` (or `duel --fail-on pull_requests`) fails
81
+ a regression only in a pull request (or a merge queue), and outside
82
+ GitHub Actions. On GitHub Actions, a regression in a push, the
83
+ schedule, or a run by hand passes, with a warning, and the workflow
84
+ that `duel init` writes comments on a pushed commit with the report,
85
+ as `c.comment` says, which notifies the commit's author. The default,
86
+ `:every_run`, fails every run, as before, and comments on no commit.
87
+ - `c.push_base = :last_passed` compares a push with the last commit
88
+ that Duel passed, which a branch marks (`duel/passed/main` for
89
+ `main`, and each branch the workflow guards has its own), instead of
90
+ the commit before the push, so slowdowns too small for any one push to
91
+ fail add up until they fail one. The workflow that `duel init` writes
92
+ moves the branch when a push passes. The default, `:before`, compares
93
+ a push with the commit before it, as before.
94
+ - `c.accept_label` names a pull request label that accepts a regression:
95
+ with it, the regression passes, with a warning, and the report says
96
+ so. The workflow that `duel init` writes passes the pull request's
97
+ labels to `duel` (`DUEL_LABELS`), or for a push, the labels of the
98
+ pull request that merged it. By default, there's no such label.
99
+ - `duel init --gitlab` writes GitLab CI jobs to include from
100
+ `.gitlab-ci.yml`, instead of GitHub Actions workflows. They compare a
101
+ merge request with the branch it targets, comment on it, and with
102
+ project access tokens, comment on a pushed commit and mark a push
103
+ passed. On GitLab CI, `duel` reads a merge request's target and labels,
104
+ writes its decisions to `DUEL_OUTPUT` as variables, and where only a
105
+ merge request fails (`c.fail_on`), exits with status 3 for a
106
+ regression outside one, which the job lets fail, as a warning.
107
+ - The workflow that `duel init` writes runs for a merge queue
108
+ (`merge_group`), and compares its group with the queue's base, so the
109
+ Duel check can be required with a merge queue.
110
+
111
+ ### Changed
112
+
113
+ - To upgrade a project that ran `duel init` before 0.1.2, delete
114
+ `.github/workflows/perf.yml` (or `perf-<its path>.yml`, for a project
115
+ in a subdirectory) and run `duel init` again, then carry over any
116
+ edits of your own. It writes both workflows, and skips the files that
117
+ exist, such as `.duel.rb`. Kept, the old workflow comments on a pull
118
+ request itself, and with the new one, every comment comes twice.
119
+ - A pull request gets its comment from a second workflow that `duel init`
120
+ writes, `.github/workflows/perf-comment.yml`, which runs once the Perf
121
+ workflow has (`workflow_run`), with a token that can write, so a pull
122
+ request from a fork gets a comment too. It runs none of the pull
123
+ request's code, and trusts only what Duel could have said in the
124
+ report's artifact. The Perf workflow no longer has a Comment job, and
125
+ for a project in a subdirectory, it's named for the directory, as
126
+ `Perf (gems/parser)`.
127
+ - The workflow that `duel init` writes cancels a pull request's run
128
+ when a newer push to the pull request starts one. Every other run,
129
+ such as a push's to `main`, still runs to the end.
130
+ - A push whose commit before it isn't in the history now reports that
131
+ there's no commit to compare against, and passes, from `duel` itself.
132
+ The workflow no longer checks this in its Compare step.
133
+
134
+ ### Fixed
135
+
136
+ - On GitHub Actions, a pull request is compared with the branch that it
137
+ targets (`GITHUB_BASE_REF`), not the main branch. A pull request into
138
+ a stable branch, or into another pull request's branch, was compared
139
+ with `main`.
140
+
10
141
  ## [0.1.1] - 2026-10-26
11
142
 
12
143
  ### Added
@@ -357,6 +488,8 @@ needs Ruby 3.4 or later (or JRuby 10, or TruffleRuby 40).
357
488
  Anything marked `@api private`, in lib/ and sig/, may change in any
358
489
  release.
359
490
 
360
- [Unreleased]: https://github.com/sferik/duel-ruby/compare/v0.1.1...HEAD
491
+ [Unreleased]: https://github.com/sferik/duel-ruby/compare/v0.1.3...HEAD
492
+ [0.1.3]: https://github.com/sferik/duel-ruby/compare/v0.1.2...v0.1.3
493
+ [0.1.2]: https://github.com/sferik/duel-ruby/compare/v0.1.1...v0.1.2
361
494
  [0.1.1]: https://github.com/sferik/duel-ruby/compare/v0.1.0...v0.1.1
362
495
  [0.1.0]: https://github.com/sferik/duel-ruby/releases/tag/v0.1.0
data/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # ⚔️ Duel: Performance regression guards for Ruby
2
2
 
3
- [![Duel](https://raw.githubusercontent.com/sferik/duel-ruby/main/assets/badge.svg)](https://github.com/sferik/duel-ruby)
3
+ [![Duel](https://raw.githubusercontent.com/sferik/duel-ruby/duel/badge/badge.svg)](https://github.com/sferik/duel-ruby/actions/workflows/perf.yml)
4
4
  [![Test](https://github.com/sferik/duel-ruby/actions/workflows/test.yml/badge.svg)](https://github.com/sferik/duel-ruby/actions/workflows/test.yml)
5
5
  [![Lint](https://github.com/sferik/duel-ruby/actions/workflows/lint.yml/badge.svg)](https://github.com/sferik/duel-ruby/actions/workflows/lint.yml)
6
6
  [![Mutant](https://github.com/sferik/duel-ruby/actions/workflows/mutant.yml/badge.svg)](https://github.com/sferik/duel-ruby/actions/workflows/mutant.yml)
@@ -194,6 +194,16 @@ hang the run. If a worker takes more than a minute to load or to answer,
194
194
  Duel stops the worker and reports the benchmark as an error. To change
195
195
  the limit, use `--timeout`.
196
196
 
197
+ A run measures for 30 minutes at most, not counting the builds. Past
198
+ that, Duel starts no more measurements. A benchmark that it hasn't
199
+ measured yet fails as an error, and a benchmark that looked worse, but
200
+ that Duel hasn't measured again as often as it usually does, fails if it
201
+ was worse in every measurement so far. So a run that's cut short never
202
+ passes a regression. The report says what was left undone. To change
203
+ the limit, use `--time-limit SECONDS`. A run usually takes this long only
204
+ when some benchmark is slow to settle, such as one that starts processes
205
+ of its own, so making it quicker is the better fix.
206
+
197
207
  `duel` exits with one of the following statuses:
198
208
 
199
209
  | Status | Meaning |
@@ -201,6 +211,7 @@ the limit, use `--timeout`.
201
211
  | `0` | Nothing got slower, or there was nothing to compare. |
202
212
  | `1` | Something got slower, or a benchmark failed. |
203
213
  | `2` | Duel couldn't compare. |
214
+ | `3` | Something got slower outside a merge request on GitLab CI, where only a merge request fails ([`c.fail_on`](docs/configuration.md#fail-only-a-pull-request)). The job that `duel init --gitlab` writes lets it fail, as a warning. |
204
215
 
205
216
  ## Run Duel in CI
206
217
 
@@ -215,6 +226,10 @@ duel:
215
226
  GIT_DEPTH: 0 # the baseline comes from history
216
227
  ```
217
228
 
229
+ `duel init --gitlab` writes GitLab CI jobs for you, which compare a
230
+ merge request with the branch it targets and comment on it. See
231
+ [Run Duel on GitLab CI](docs/ci.md#run-duel-on-gitlab-ci).
232
+
218
233
  To run Duel from Rake, define a task with `Duel::RakeTask`. Any
219
234
  arguments after the task's name are passed to `duel`:
220
235
 
@@ -225,8 +240,10 @@ Duel::RakeTask.new(:perf_since_release, "--base", "last-release")
225
240
  ```
226
241
 
227
242
  `duel init` writes a GitHub Actions workflow,
228
- `.github/workflows/perf.yml`. It compares each pull request with the
229
- main branch, each push to `main` with the commit before it, and, on a
243
+ `.github/workflows/perf.yml`, and a second one,
244
+ `.github/workflows/perf-comment.yml`, that comments on a pull request,
245
+ a fork's too. It compares each pull request with the
246
+ branch that it targets, each push to `main` with the commit before it, and, on a
230
247
  schedule, your last release, to catch slow creep. It also comments on a
231
248
  pull request with Duel's report when a benchmark fails. For the
232
249
  workflow, and for a branch that upgrades Ruby, see
@@ -304,17 +321,35 @@ but nothing to compare, or never, set
304
321
  in `.duel.rb`. For more information, see
305
322
  [Comment on a pull request](docs/ci.md#comment-on-a-pull-request).
306
323
 
324
+ In a project where people push to `main` directly, set
325
+ [`c.fail_on = :pull_requests`](docs/configuration.md#fail-only-a-pull-request)
326
+ to pass a regression in a push, with a warning, and comment on the
327
+ pushed commit with the report instead. To compare each push with the
328
+ last commit that passed, instead of the commit before it, so that small
329
+ slowdowns add up until they fail a push, set
330
+ [`c.push_base = :last_passed`](docs/configuration.md#compare-a-push-with-the-last-commit-that-passed).
331
+ To accept a regression that's on purpose, label the pull request, as
332
+ [`c.accept_label`](docs/configuration.md#accept-a-regression-with-a-label)
333
+ says.
334
+
307
335
  ### Show that a project is guarded
308
336
 
309
337
  To show that Duel guards your project, add its badge to your README,
310
- next to your other badges:
338
+ next to your other badges. With the workflow that `duel init` writes,
339
+ the badge turns red when Duel fails a push to your default branch.
340
+ Replace `OWNER/REPO` with your repository's:
311
341
 
312
342
  ```markdown
313
- [![Duel](https://raw.githubusercontent.com/sferik/duel-ruby/main/assets/badge.svg)](https://github.com/sferik/duel-ruby)
343
+ [![Duel](https://raw.githubusercontent.com/OWNER/REPO/duel/badge/badge.svg)](https://github.com/sferik/duel-ruby)
314
344
  ```
315
345
 
316
- The badge links to Duel. It doesn't show whether a comparison passed:
317
- your CI's status does that.
346
+ See [Show whether the last push passed](docs/ci.md#show-whether-the-last-push-passed)
347
+ for how the workflow keeps it. Without the workflow, such as on GitLab
348
+ CI, use Duel's own badge, which is always green:
349
+
350
+ ```markdown
351
+ [![Duel](https://raw.githubusercontent.com/sferik/duel-ruby/main/assets/badge.svg)](https://github.com/sferik/duel-ruby)
352
+ ```
318
353
 
319
354
  ## Plugins
320
355
 
data/docs/benchmarks.md CHANGED
@@ -217,7 +217,8 @@ To change that, pass these options to `duel`:
217
217
  benchmark thread's.
218
218
  - `peak_memory: true`: also judges the benchmark's peak memory, which is
219
219
  the most resident memory that its measured work reached, above where
220
- it started. Duel takes each version's lowest peak from three fresh
220
+ it started, or on JRuby, the most of the JVM's heap that it had in
221
+ use. Duel takes each version's lowest peak from three fresh
221
222
  workers. Like every other measure, peak memory leaves out what
222
223
  `once` and `untimed` blocks leave behind.
223
224