duel 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. checksums.yaml +4 -4
  2. data/AGENT_PROMPT.md +51 -27
  3. data/CHANGELOG.md +247 -1
  4. data/README.md +30 -14
  5. data/docs/benchmarks.md +28 -24
  6. data/docs/ci.md +328 -362
  7. data/docs/configuration.md +40 -37
  8. data/docs/how-it-works.md +23 -10
  9. data/docs/json.md +21 -25
  10. data/docs/plugins.md +40 -22
  11. data/docs/troubleshooting.md +3 -2
  12. data/lib/duel/across.rb +18 -18
  13. data/lib/duel/badge.rb +174 -0
  14. data/lib/duel/baseline.rb +38 -15
  15. data/lib/duel/cli.rb +175 -107
  16. data/lib/duel/comment.rb +80 -71
  17. data/lib/duel/config.rb +145 -63
  18. data/lib/duel/context.rb +29 -24
  19. data/lib/duel/forge.rb +87 -0
  20. data/lib/duel/init.rb +44 -87
  21. data/lib/duel/json_report.rb +34 -25
  22. data/lib/duel/lane.rb +107 -61
  23. data/lib/duel/lock.rb +1 -1
  24. data/lib/duel/mark_passed.rb +132 -0
  25. data/lib/duel/metric.rb +28 -27
  26. data/lib/duel/native.rb +7 -5
  27. data/lib/duel/platform.rb +4 -0
  28. data/lib/duel/process_sample.rb +9 -6
  29. data/lib/duel/rake_task.rb +4 -5
  30. data/lib/duel/resource.rb +10 -11
  31. data/lib/duel/result.rb +22 -14
  32. data/lib/duel/rubies.rb +6 -2
  33. data/lib/duel/runner.rb +27 -15
  34. data/lib/duel/spawn.rb +29 -4
  35. data/lib/duel/stats.rb +14 -12
  36. data/lib/duel/summary.rb +31 -29
  37. data/lib/duel/templates/benchmark_duel.rb.erb +2 -2
  38. data/lib/duel/templates/duel.gitlab-ci.yml.erb +18 -54
  39. data/lib/duel/templates/duel.rb.erb +7 -7
  40. data/lib/duel/templates/perf-caller.yml.erb +22 -5
  41. data/lib/duel/templates/perf-comment.yml.erb +11 -11
  42. data/lib/duel/templates/perf-push-jobs.yml.erb +127 -0
  43. data/lib/duel/templates/perf-push.yml.erb +50 -0
  44. data/lib/duel/templates/perf.yml.erb +113 -335
  45. data/lib/duel/testing.rb +20 -25
  46. data/lib/duel/version.rb +1 -1
  47. data/lib/duel/worker.rb +16 -9
  48. data/lib/duel/worker_process.rb +6 -6
  49. data/lib/duel.rb +88 -33
  50. data/sig/duel/across.rbs +4 -0
  51. data/sig/duel/badge.rbs +51 -0
  52. data/sig/duel/baseline.rbs +7 -2
  53. data/sig/duel/cli.rbs +31 -20
  54. data/sig/duel/comment.rbs +14 -15
  55. data/sig/duel/config.rbs +71 -29
  56. data/sig/duel/context.rbs +4 -4
  57. data/sig/duel/forge.rbs +42 -0
  58. data/sig/duel/init.rbs +8 -26
  59. data/sig/duel/json_report.rbs +7 -6
  60. data/sig/duel/lane.rbs +29 -6
  61. data/sig/duel/mark_passed.rbs +39 -0
  62. data/sig/duel/metric.rbs +10 -10
  63. data/sig/duel/native.rbs +3 -0
  64. data/sig/duel/platform.rbs +1 -0
  65. data/sig/duel/process_sample.rbs +3 -0
  66. data/sig/duel/resource.rbs +3 -3
  67. data/sig/duel/result.rbs +12 -5
  68. data/sig/duel/rubies.rbs +3 -0
  69. data/sig/duel/runner.rbs +23 -16
  70. data/sig/duel/spawn.rbs +7 -0
  71. data/sig/duel/stats.rbs +4 -1
  72. data/sig/duel/summary.rbs +8 -0
  73. data/sig/duel/testing.rbs +2 -2
  74. data/sig/duel/worker.rbs +6 -5
  75. data/sig/duel/worker_process.rbs +3 -3
  76. data/sig/duel.rbs +50 -31
  77. metadata +9 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3914db6930c721c09f8b53b0b0e8b9f15e04fa2106d7a385a2f773d36545e2f1
4
- data.tar.gz: ab23ea38c226c83ba1da5145caee9548f48a22ffd76fb225e3ad05f5b0da5f27
3
+ metadata.gz: 4011a6478545bb287113c617c1ffe74165fad6c74043b9208663a555bb4a30a8
4
+ data.tar.gz: a5e698aac1d4042e66aa0c1e5e2b948548ab2bb56b998db81973ac1fa7a21367
5
5
  SHA512:
6
- metadata.gz: e2e23a16ee7d7d893ce503667b17f1fc677cbd91cda676401cb514aec820cf6bcea97294af7c3f9aceadc2b34e249b402ec751641bcc3dad4496aed58f1f764d
7
- data.tar.gz: ae3d97e628cce46263ef4a255c937ff7f53e26c2582ec999bed4ed030881e3c6b3dc69fad0dfe5bb59616821ed774ec9444a182712f03fecd4e7e8a9544f868e
6
+ metadata.gz: b5ec57f3d533abe47b097d7ebe08594f3615340e72b2ff94ffa0db3cb98501502176555a3da673c93a7219a7ecff63b1f1972c7dc6733977792fc3a044eec102
7
+ data.tar.gz: 8d0b7c39d106e4e3ca32a69c9cb375a7177f0cd48d98055cfbdbf284e3cada2a75c1a489c37dc560beb440d578eafdef1290fe4ece9251d81143eeeae941feb4
data/AGENT_PROMPT.md CHANGED
@@ -141,9 +141,10 @@ at the API.
141
141
  If the project pins its Ruby (`.ruby-version`, `.tool-versions`, or
142
142
  `mise.toml`), set `c.ruby = :version_file`, so that a branch that
143
143
  upgrades Ruby is compared on each version's own Ruby. `duel init`
144
- does this, and adds the steps in Duel's `docs/ci.md` that install
145
- the baseline's Ruby to the workflow, for a `.ruby-version`. For another version
146
- file, do both yourself (see step 6). Duel uses the Ruby that runs it
144
+ does this for a `.ruby-version`, and has the workflow set up the
145
+ baseline's Ruby too (`version-file: true`). For another version
146
+ file, set it yourself, and see step 6 for the workflow. Duel uses the
147
+ Ruby that runs it
147
148
  when that's the one asked for, so CI needs nothing more until a
148
149
  branch changes the Ruby.
149
150
 
@@ -191,34 +192,40 @@ at the API.
191
192
  your names can be picked out. Without `--force`, a tree with
192
193
  nothing changed since the main branch has nothing to compare.
193
194
 
194
- 6. **Check CI.** `duel init` wrote two GitHub Actions workflows.
195
- - `.github/workflows/perf.yml` runs Duel. Keep `fetch-depth: 0`:
196
- Duel's baseline comes from git history. Keep its check for a push
197
- with no commit before it in the history (a branch's first push, or
198
- a force push), which would otherwise fail the job. The Duel job
199
- runs the branch's code, so it only reads: don't give it, or the
200
- workflow, any write permission.
195
+ 6. **Check CI.** `duel init` wrote two GitHub Actions workflows, short
196
+ ones that call Duel's reusable workflows, at the tag of the Duel
197
+ release that wrote them, with what `.duel.rb` sets that they need.
198
+ - `.github/workflows/perf.yml` runs Duel. Its `duel` job runs the
199
+ branch's code, so it only reads (`contents: read`), and its `push`
200
+ job, which runs only for a push and none of the code, writes
201
+ (`contents: write`). Keep the permissions as `duel init` wrote
202
+ them: GitHub checks every job of Duel's workflows against them,
203
+ whether it runs or not.
201
204
  - `.github/workflows/perf-comment.yml` comments on a pull request
202
205
  with Duel's report once the first has run (`c.comment` in
203
206
  `.duel.rb` chooses when, and Duel also adds the report to the
204
207
  job's summary). It runs none of the pull request's code. Keep it as
205
208
  it is.
206
- - `duel init` writes the jobs for pushes (a comment on a pushed
207
- commit, and a `duel/passed/` branch that marks the last commit
208
- Duel passed) only where `.duel.rb` sets `c.fail_on =
209
- :pull_requests` or `c.push_base = :last_passed`, and the badge's
210
- job only with `--badge`. Each needs `contents: write`, so don't
211
- set those settings, or add those jobs, unless I ask.
209
+ - The workflow's jobs for pushes (a comment on a pushed commit,
210
+ and a `duel/passed/` branch that marks the last commit Duel
211
+ passed) run only where `.duel.rb` sets `c.fail_on =
212
+ :pull_requests` or `c.push_base = :last_passed`, and `duel init`
213
+ writes the badge's job only with `--badge`. Each writes to the
214
+ repository, so don't set those settings, or ask for the badge,
215
+ unless I ask.
212
216
 
213
217
  Check that `ruby-version` is one the project supports, unless it has
214
- a `.ruby-version`. With `c.ruby = :version_file` and another version
215
- file than `.ruby-version`, add the steps in Duel's `docs/ci.md` that
216
- install the baseline's Ruby, reading that file instead. If
217
- `.duel.rb` sets `c.main_branch` or `c.release_tags`, set those
218
- steps' `MAIN_BRANCH` or `RELEASE_TAGS` to match. Add the project's
219
- other CI setup that its tests need (system packages, services) to
220
- the Duel job as well. Keep the scheduled `last-release` run only if
221
- the project tags its releases (`v*` by default, or else set
218
+ a `.ruby-version`. Give the project's other CI setup that its tests
219
+ need (system packages, services) to the call as `setup:`, shell
220
+ commands that run before Ruby is set up (Duel's `docs/ci.md` lists
221
+ the inputs). Where that isn't enough, or with `c.ruby =
222
+ :version_file` and another version file than `.ruby-version`, delete
223
+ the two workflows and run `bundle exec duel init --standalone`,
224
+ which writes the whole ones (and leaves the files that are there).
225
+ Add the setup to its Duel job, and for the version file, the steps in
226
+ Duel's `docs/ci.md` that install the baseline's Ruby, reading that
227
+ file instead. Keep the scheduled `last-release` run only if the
228
+ project tags its releases (`v*` by default, or else set
222
229
  `c.release_tags`), or leave the schedule out. Until the first
223
230
  release is tagged, that run says there's nothing to compare against,
224
231
  and passes. Once it's pushed, run the workflow by hand: it compares
@@ -227,10 +234,14 @@ at the API.
227
234
 
228
235
  7. **Show the badge.** If the project's README already shows badges (CI
229
236
  status, gem version), add Duel's to that row, with exactly this
230
- markdown:
237
+ markdown, `OWNER/REPO` the project's repository, so that it links to
238
+ the Perf workflow's runs (for a project in a subdirectory of its
239
+ repository, `perf-<its path>.yml`, the workflow `duel init` wrote, in
240
+ place of `perf.yml`; on GitLab CI, the link is to
241
+ `https://gitlab.com/OWNER/REPO/-/pipelines`):
231
242
 
232
243
  ```markdown
233
- [![Duel](https://raw.githubusercontent.com/sferik/duel-ruby/main/assets/badge.svg)](https://github.com/sferik/duel-ruby)
244
+ [![Duel](https://raw.githubusercontent.com/sferik/duel-ruby/main/assets/badge.svg)](https://github.com/OWNER/REPO/actions/workflows/perf.yml)
234
245
  ```
235
246
 
236
247
  If the README has no badges, don't add a row just for Duel's. This
@@ -249,4 +260,17 @@ at the API.
249
260
  (`c.accept_label`, `duel: accept` in the `.duel.rb` that
250
261
  `duel init` writes), which the repository needs to have.
251
262
 
252
- Don't commit. Leave the changes for me to review.
263
+ 9. **Add the project to Duel's list.** If the project's repository is
264
+ public, open a pull request in Duel's repository
265
+ (https://github.com/sferik/duel-ruby) that adds the project to
266
+ `USERS.md`, the list of projects using Duel. Fork Duel's repository
267
+ (`gh repo fork sferik/duel-ruby --clone`), add one line for the
268
+ project to the list, in alphabetical order and in the format of the
269
+ others (a link to its repository, named by its owner and name, and
270
+ what the project is, in a few words), commit it on a branch of its
271
+ own, and open the pull request (`gh pr create --repo
272
+ sferik/duel-ruby`). Leave out a private repository. Tell me the pull
273
+ request's link.
274
+
275
+ Apart from that pull request, don't commit. Leave the changes for me to
276
+ review.
data/CHANGELOG.md CHANGED
@@ -7,6 +7,251 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.3.0] - 2026-11-20
11
+
12
+ ### Added
13
+
14
+ - `c.release_tags` and `c.benchmarks` take a list of globs too:
15
+ `c.release_tags = %w[v* my_gem-v*]` for a gem that renamed its tags
16
+ (a tag that matches any of them is a release), and `c.benchmarks =
17
+ %w[perf/**/*_duel.rb test/perf/*_duel.rb]` for benchmarks in two
18
+ places (each glob's directory is watched). One glob works as before.
19
+ - `Duel.find_program` finds a plugin's server program, as
20
+ duel-postgresql, duel-mysql, duel-redis and duel-memcached now all
21
+ do: a path (one with a `/` or a `\` in it, `~` expanded) as it is,
22
+ or a name in the first of the `PATH`'s directories that has it,
23
+ either one ending with `.exe` on Windows too. `Duel.path?` says which
24
+ it takes a server for, for the plugin's error that it isn't there.
25
+ - Duel's reusable Perf workflow runs on Windows (`runs-on:
26
+ windows-latest`): the Duel job's steps run in bash on every runner (on
27
+ Windows, not PowerShell), and paths it hands Ruby are Windows's there.
28
+ A new input, `setup-after-ruby`, runs shell commands once Ruby is set
29
+ up, before the bundle is installed, for what needs Ruby first: on
30
+ Windows, a package of Ruby's devkit, from `pacman`, that a gem's C
31
+ extension links. And `c.ruby = :version_file` finds a Ruby on
32
+ Windows, whose program is `ruby.exe` (or `jruby.exe`).
33
+ - Duel fails a run where a worker's environment turns on one of CRuby's
34
+ JITs (`RUBY_YJIT_ENABLE` or `RUBY_ZJIT_ENABLE`, as `1`, `true` or
35
+ `yes`) that the worker doesn't run, saying which version's workers,
36
+ which JIT, and on what Ruby. A Ruby without the JIT ignores its
37
+ variable, so `jit: zjit` on Ruby 3.4, or `jit: yjit` on JRuby,
38
+ measured without it, and nothing said so.
39
+ - ZJIT, the JIT compiler that's experimental in Ruby 4.0 and is due to
40
+ ship in Ruby 4.1, is taken for a JIT as YJIT is: where a worker has
41
+ it on (`RUBY_ZJIT_ENABLE=1`, or turned on as the app loads), Duel
42
+ runs each benchmark until it's steady before it measures it, for up
43
+ to ten seconds (or as `--warm` says). It measured from the first
44
+ sample before, while ZJIT was still compiling. Duel's reusable Perf
45
+ workflow takes the JIT to turn on as one input, `jit:` (`none`,
46
+ `yjit` or `zjit`, as Ruby runs one at most), in place of its `yjit:
47
+ true`.
48
+ - `duel comment --commit SHA` comments on a pushed commit, as Duel's
49
+ outputs say (`DUEL_COMMENT` and `DUEL_STATUS`), on GitHub or GitLab.
50
+ `duel mark-passed BRANCH` moves the branch that marks the last commit
51
+ Duel passed forward to a push, or makes it, and `duel badge BRANCH`
52
+ writes Duel's badge to its branch, on GitHub. The Comment on the
53
+ commit, Mark the commit passed and Update the badge jobs that
54
+ `duel init` writes (and Duel's reusable Perf workflow) run them,
55
+ installed from RubyGems as the comment workflow's `duel comment` is,
56
+ in place of the shell, `curl` and `jq` that they ran. So a fix to one
57
+ comes with a release of Duel, without a change to the workflow.
58
+
59
+ ### Changed
60
+
61
+ - A plugin's resource gives its server a port with `w.free_port`, in
62
+ place of `w.port`, which read as the resource's one port, though each
63
+ call is another: `port = w.free_port`, then that `port` for both the
64
+ server's option and its URL.
65
+ - `duel_process` takes the Ruby code it runs as `code:`, in place of
66
+ `ruby:`, which read as the Ruby to run it on, as `c.ruby`, `--ruby`
67
+ and `DUEL_RUBY` mean: `duel_process "boot: require the library",
68
+ code: %(require "my_gem")`.
69
+ - The settings that take a glob, a path, a name or a lambda
70
+ (`c.benchmarks`, `c.watch`, `c.release_tags`, `c.library`, `c.build`,
71
+ `c.main_branch` and `c.ruby`) fail as they're set, on a value they
72
+ don't take, saying what to give (`c.watch = :lib: give a path, or a
73
+ list of them`), as `c.comment` and the others do. A wrong one failed
74
+ deep in a run (`String#+: no implicit conversion of Array into
75
+ String`), or not at all. And `c.watch` takes one path too
76
+ (`c.watch = "app"`), as `c.benchmarks` and `c.release_tags` take one
77
+ glob.
78
+ - A metric judged like CPU time is `kind: :timed`, in place of
79
+ `kind: :ratio`, as `judge: :timed`, `Definition#timed?` and the
80
+ report's `timed` call what it measures: a metric's `measures` say
81
+ `{"wait" => {kind: :timed, unit: "µs"}}`. (The JSON report's `ratio`,
82
+ a timing's, is as it was.)
83
+ - A ratio metric's `sizes_samples: false` is `size_samples: false`
84
+ (don't size samples by it), as `calibrate: false` and `skip: true`
85
+ say what to do.
86
+ - A counted metric's slack is `per_iteration:` (how much more an
87
+ iteration still isn't more) and `per_sample:` (how much more a
88
+ sample, whatever its length), in place of `tolerance:` and
89
+ `allowance:`, whose names didn't say which was which: in a metric's
90
+ `measures`, `c.measure("retained_objects", per_sample: 32)`, and
91
+ `Duel::Metric.within?(more, n, per_iteration:, per_sample:)`.
92
+ - A plugin's resource fails with `w.fail!(message)`, in place of
93
+ `w.failed(message)`, which read like a question but raised
94
+ `Duel::Failed`: `Duel.find_program(server) || w.fail!("no #{server}")`.
95
+ - The RBS signatures of `c.comment=`, `c.fail_on=`, `c.push_base=` and
96
+ `c.accept_label=` take what each takes (`c.comment = :failure`, a
97
+ typo for `:failures`, fails Steep), as the other settings' do. They
98
+ took anything, and said nothing of what they return.
99
+ - The jobs of Duel's reusable Perf workflow for a push, which write to
100
+ the repository (Comment on the commit, Mark the commit passed and
101
+ Update the badge), are a reusable workflow of their own,
102
+ `.github/workflows/duel-push.yml`, so a Perf workflow that calls
103
+ Duel's grants `contents: read` to the Duel job's call, which runs the
104
+ branch's code, and `contents: write` to the push workflow's alone,
105
+ which it calls only for a push. Before, its one call granted
106
+ `contents: write`, for a pull request's run too, as GitHub checks
107
+ every job of a called workflow against what the caller grants,
108
+ whether it runs or not. `duel init` writes both calls, the second
109
+ passing on what the Duel job said (`comment`, `status`, `passed`, and
110
+ `result`, how its call ended). The `badge` input is the push
111
+ workflow's, not the Perf workflow's: a Perf workflow that passes it
112
+ to `duel.yml` fails, so run `duel init --force` (or
113
+ `duel init --check` to see the change).
114
+ - The jobs of Duel's workflows that write to the repository or comment
115
+ (Comment on the commit, Mark the commit passed, Update the badge, and
116
+ the Perf comment workflow's), reusable or as `duel init` writes them,
117
+ install exactly the release of Duel that wrote them
118
+ (`gem install duel --version '0.3.0'`), not the newest of its minor
119
+ version (`~> 0.2.0`), and every action Duel's workflows use is named
120
+ by its commit, with its release beside it
121
+ (`ruby/setup-ruby@f9c4492… # v1.342.0`), as a tag can be moved.
122
+ `docs/ci.md` shows how to name Duel's reusable workflows by commit
123
+ too.
124
+ - `duel init` writes short workflows that call Duel's reusable ones, at
125
+ the tag of the release that writes them, as `--reusable` did, which
126
+ is gone. `--standalone` writes the whole workflows, as `duel init`
127
+ did, to change as you like. `--check` compares the ones it would
128
+ write with the same options, so `duel init --check --standalone`
129
+ checks whole workflows that an earlier `duel init` wrote.
130
+ - The badge links to the project's Perf workflow's runs
131
+ (`https://github.com/OWNER/REPO/actions/workflows/perf.yml`, or for a
132
+ project in a subdirectory, `perf-<its path>.yml`), which say why it's
133
+ green or red, not to the repository itself: in the README, the guide,
134
+ AGENT_PROMPT.md's badge step, and the workflows that `duel init` and
135
+ `rails generate duel:install` write, whose comments give the badge's
136
+ Markdown. USERS.md, in Duel's repository, lists the projects using
137
+ Duel: a pull request adds one, and AGENT_PROMPT.md's last step has the
138
+ agent open one for the project it set Duel up in, if that's public.
139
+ - `Duel::Testing.measure`'s block is one iteration, which it runs `n:`
140
+ times, as a worker runs a benchmark's block that takes no `n`. It ran
141
+ the block once, and only told the metrics `n`, unless there was an
142
+ `around_iteration` hook (the configuration's, or its own), so a
143
+ plugin that added one changed how many times every test's block ran.
144
+ A test that did its `n` iterations in the block does one.
145
+ - A `Duel::RakeTask` exits with `duel`'s own status. It exited 1 for
146
+ status 2 (couldn't compare), and passed for status 3 (slower, but only
147
+ warned, which GitLab's `allow_failure: exit_codes: [3]` relies on), so
148
+ `rake duel` hid a slowdown that `duel` reported.
149
+ - Each of a run's settings can be set the same three ways: an option, a
150
+ variable, or `.duel.rb`, in that order of precedence. `c.jobs` sets
151
+ `--jobs`, and each option `.duel.rb` can set, and `--ruby`, has a
152
+ variable, `DUEL_` and its name (`DUEL_LIMIT`, `DUEL_TIME_LIMIT`,
153
+ `DUEL_RUBY`...), as `--base`, `--filter` and `--force` had.
154
+ `--ruby` takes `version_file` too.
155
+ - `c.library` takes a lambda, given the root of a version's tree, that
156
+ loads the version instead, which `c.load { |root| ... }` did:
157
+ `c.library = ->(root) { require File.join(root, "config", "boot") }`.
158
+ `c.load`, which hid `Kernel#load`, is gone.
159
+ - What a benchmark is judged on is one option, `judge:`, one or a list of
160
+ `:timed`, `:counted` and `:peak_memory`, named as a result's measures
161
+ are, and `[:timed, :counted]` by default. It replaces `only: :count`
162
+ (`judge: :counted`), `only: :timing` (`judge: :timed`) and
163
+ `peak_memory: true` (`:peak_memory` too, which needs `:timed`: with
164
+ counts alone, it was quietly ignored), so `only: :timing, peak_memory:
165
+ true` reads as `judge: %i[timed peak_memory]`. A metric sees it as
166
+ `sample.benchmark.judge`, with `timed?`, `counted?` and
167
+ `peak_memory?`, in place of `only` and `peak_memory`, and
168
+ `Duel::Testing.measure` and `Duel::Testing.sample` take `judge:` too.
169
+ - The JSON report's schema is 2:
170
+ - `status` is what the comparison found, and `exit` is `duel`'s exit
171
+ status, with `passed_anyway` saying why a run that failed passed
172
+ anyway (`"accept_label"`, or `"fail_on"`), which nothing said
173
+ before: a regression a label accepted was `"passed"`.
174
+ - A timing too close to its limit to settle is `"unsettled"`, as its
175
+ benchmark's verdict is, not `"undecided"`, in the JSON report, the
176
+ Markdown report and the terminal's.
177
+ - A count, and peak memory, has a `verdict` (`"more"`, `"fewer"` or
178
+ `"same"`) in place of `worse`.
179
+ - `unsettled` (a benchmark's `verdict` says so), `errors` (each
180
+ benchmark's `error` says so) and `comment` (the workflow's step
181
+ output says so) are gone.
182
+ - On CI, Duel finds the baseline itself, from the run's event: for a
183
+ push, the commit before it (on GitHub Actions, the event's `before`,
184
+ and on GitLab CI, `CI_COMMIT_BEFORE_SHA`), for a merge queue, its
185
+ base, and on a schedule, the last release. The workflows `duel init`
186
+ writes gave it as `DUEL_BASE`, which now only names another, as
187
+ `--base` does. (A `DUEL_BASE` that's the event's own is still taken
188
+ for it, as the workflows earlier versions wrote give it.) And `duel
189
+ base` prints the commit a run compares with, which the workflow's
190
+ step that installs the baseline's Ruby reads, once the project's Ruby
191
+ and bundle are set up, so it no longer finds the main branch and the
192
+ last release in shell of its own, which `c.main_branch` and
193
+ `c.release_tags` had to be copied into.
194
+ - The Perf workflow follows `.duel.rb`, rather than copying it. Its
195
+ jobs for pushes, the comment on a pushed commit and marking a push
196
+ passed, are always there (`duel init` wrote them only where `.duel.rb`
197
+ asked, and said to write the workflow again if that changed), and run
198
+ where Duel's outputs say to: its `passed` output is the branch that
199
+ marks the push passed (`duel/passed/main`), not `true`, so the job
200
+ moves the one Duel reads. Duel's reusable Perf workflow takes none of
201
+ `.duel.rb`'s settings: `accept-label`, `release-tags`,
202
+ `comment-on-pushes`, `mark-passed`, `passed-branch`, `badge-branch`
203
+ and `marker` are gone, as Duel reads `.duel.rb`, and the names it
204
+ needs come from `working-directory`. And on a schedule, Duel finds
205
+ the last change a label accepted since the last release itself, with
206
+ `gh` (`DUEL_ACCEPTED` still names one), which the workflow's labels
207
+ step did, with `c.accept_label` and `c.release_tags` copied into it.
208
+ - `--comment` and `--fail-on` are gone: when the workflow comments, and
209
+ which runs a regression fails, are `.duel.rb`'s to say (`c.comment`,
210
+ `c.fail_on`), as the workflow passes neither, and outside CI, they
211
+ changed nothing.
212
+ - `duel` fails on an argument that isn't an option (`duel v1.2.0`, say,
213
+ for `duel --base v1.2.0`), which it ignored.
214
+ - `duel clean` removes `tmp/duel` and the baseline's git worktree, as
215
+ `duel --clean` did, which took (and ignored) every other option, and
216
+ is gone. It takes `--config`.
217
+ - `duel comment` comments on a GitLab merge request where `GITLAB_CI`
218
+ says it runs on GitLab CI, without `--gitlab`, which the GitLab CI
219
+ jobs `duel init --gitlab` writes no longer pass. (It's still taken,
220
+ and does nothing, for the jobs that earlier versions wrote.)
221
+ - `--candidate-dir` is `--candidate`, as `--base` names the baseline,
222
+ and `DUEL_CANDIDATE` sets it.
223
+ - `DUEL_FORCE` is `1` or `true` (or `0` or `false`, for off), and
224
+ anything else fails. Any value forced before, `DUEL_FORCE=0` too.
225
+ - A `setup`, `teardown`, `before_sample` or `after_iterations` hook
226
+ takes no parameter, at any level, and one that takes one fails before
227
+ anything runs. The first three were given the benchmark if they took
228
+ a parameter, which nothing needed. Only `around_iteration` takes one:
229
+ its iteration.
230
+ - `c.accept_label` is `"duel: accept"` by default, the label that the
231
+ `.duel.rb` `duel init` writes named, so a project set up by hand
232
+ accepts a regression as one set up by `duel init` does. Set it to
233
+ `nil` for no such label. That `.duel.rb` shows the label, and the
234
+ build of a project with `ext/` (`bundle exec rake compile`, Duel's
235
+ default there already), commented out, as the defaults they are.
236
+
237
+ ### Fixed
238
+
239
+ - A process benchmark whose Ruby process of its own ended while it ran
240
+ a command (killed, say) leaves that command running no longer: what's
241
+ left of the Ruby process's group is killed as it's stopped, where the
242
+ command went on, orphaned, for as long as it ran (for ever, for one
243
+ that hung).
244
+ - The Perf workflow that `duel init --reusable` wrote granted
245
+ `contents: read` unless it turned on a job that writes (marking a
246
+ push passed, commenting on one, or the badge), but GitHub checks every
247
+ job of a workflow it calls against what the caller grants, whether
248
+ the job runs or not, so the run failed before it started. The jobs
249
+ that write are now Duel's push workflow's, which the Perf workflow
250
+ that `duel init` writes calls apart, with `contents: write`, and only
251
+ for a push, while the Duel job's call still only reads (see Changed).
252
+ Run `duel init --force` for a workflow that calls both
253
+ (`duel init --check` shows the change).
254
+
10
255
  ## [0.2.0] - 2026-11-16
11
256
 
12
257
  ### Added
@@ -906,7 +1151,8 @@ needs Ruby 3.4 or later (or JRuby 10, or TruffleRuby 40).
906
1151
  Anything marked `@api private`, in lib/ and sig/, may change in any
907
1152
  release.
908
1153
 
909
- [Unreleased]: https://github.com/sferik/duel-ruby/compare/v0.2.0...HEAD
1154
+ [Unreleased]: https://github.com/sferik/duel-ruby/compare/v0.3.0...HEAD
1155
+ [0.3.0]: https://github.com/sferik/duel-ruby/compare/v0.2.0...v0.3.0
910
1156
  [0.2.0]: https://github.com/sferik/duel-ruby/compare/v0.1.8...v0.2.0
911
1157
  [0.1.8]: https://github.com/sferik/duel-ruby/compare/v0.1.7...v0.1.8
912
1158
  [0.1.7]: https://github.com/sferik/duel-ruby/compare/v0.1.6...v0.1.7
data/README.md CHANGED
@@ -109,7 +109,7 @@ one on Linux or macOS.
109
109
  |---|---|
110
110
  | `.duel.rb` | Duel's configuration. It builds a project that has an `ext/` directory with `rake compile`, and runs each version on its own Ruby when the project has a `.ruby-version`. Without a gemspec to name the library, it names the only file in `lib/`, or leaves the library for you to set. See [Configure Duel](docs/configuration.md). |
111
111
  | `duel/<library>_duel.rb` | An example benchmark to replace with your own, with advice on writing benchmarks that tell a regression from noise. See [Write a benchmark](#write-a-benchmark). |
112
- | `.github/workflows/perf.yml` | The GitHub Actions workflow in [Run Duel in CI](docs/ci.md#catch-slow-creep). For a project in a subdirectory of its repository, the workflow is at the repository's root, named after the subdirectory, and runs in it. |
112
+ | `.github/workflows/perf.yml` | A short GitHub Actions workflow that calls Duel's reusable one (see [Call Duel's reusable workflows](docs/ci.md#call-duels-reusable-workflows)), with `perf-comment.yml`, which comments on a pull request. With `--standalone`, they're the whole workflows in [Run Duel in CI](docs/ci.md#catch-slow-creep). For a project in a subdirectory of its repository, the workflow is at the repository's root, named after the subdirectory, and runs in it. |
113
113
  | `Rakefile` | If there is one, `duel init` adds Duel's Rake task to it, as `rake duel`. See [Run Duel in CI](#run-duel-in-ci). |
114
114
  | `Gemfile` | Unless it mentions StackProf already, `duel init` adds `gem "stackprof", platforms: :mri` to its `development` group (or, without one, to its end), so Duel can show where a regression spent its extra time. StackProf runs on CRuby only, so the gem isn't installed on JRuby or TruffleRuby. |
115
115
 
@@ -118,9 +118,9 @@ one on Linux or macOS.
118
118
  use `--dry-run`. For a badge in your README that turns red when Duel
119
119
  fails a push, use `--badge` (see
120
120
  [Show that a project is guarded](#show-that-a-project-is-guarded)).
121
- To write short workflows that call Duel's reusable ones instead of
122
- copies of them, use `--reusable` (see
123
- [Call Duel's reusable workflows](docs/ci.md#call-duels-reusable-workflows)).
121
+ To write the whole workflows, to change as you like, rather than short
122
+ ones that call Duel's reusable workflows, use `--standalone` (see
123
+ [Run Duel in CI](docs/ci.md)).
124
124
  When it's done, `duel init` lists the next steps.
125
125
 
126
126
  For a Rails app, use
@@ -195,6 +195,7 @@ duel --base v1.4.0 # against any git ref
195
195
  duel --base last-release # against the newest release tag before HEAD
196
196
  duel --filter "^parse:" # only benchmarks whose names match (a regular expression)
197
197
  duel --base ../old # against a tree on disk, already built
198
+ duel --candidate ../new # a tree on disk, already built, against the baseline
198
199
  ```
199
200
 
200
201
  To see every option, such as `--processes`, `--timeout`, and `--limit`,
@@ -210,16 +211,23 @@ second run fails and names the process that's running the first. A run
210
211
  that crashes leaves nothing behind that blocks the next run.
211
212
 
212
213
  Where it's hard to pass options, such as behind a Rake task or in a CI
213
- expression, set these environment variables instead:
214
+ expression, set environment variables instead. Each is `DUEL_` and the
215
+ option's name, for `--base`, `--candidate`, `--filter`, `--ruby`, and
216
+ each option that `.duel.rb` can set too (see
217
+ [Set a run's settings](docs/configuration.md#set-a-runs-settings)):
214
218
 
215
219
  | Variable | Option |
216
220
  |---|---|
217
221
  | `DUEL_BASE` | `--base` |
222
+ | `DUEL_CANDIDATE` | `--candidate` |
218
223
  | `DUEL_FILTER` | `--filter` |
219
- | `DUEL_FORCE=1` | `--force` |
224
+ | `DUEL_RUBY` | `--ruby` |
225
+ | `DUEL_LIMIT`, `DUEL_CONFIDENCE`, `DUEL_JOBS`, `DUEL_PROCESSES`, `DUEL_TIMEOUT`, `DUEL_TIME_LIMIT`, `DUEL_WARM` | `--limit`, `--confidence`, `--jobs`, `--processes`, `--timeout`, `--time-limit`, `--warm` |
226
+ | `DUEL_FORCE=1` (or `true`) | `--force` (`0` or `false` is off, and anything else fails) |
220
227
 
221
- An option on the command line takes precedence over its variable, and
222
- an empty variable counts as unset.
228
+ An option on the command line takes precedence over its variable,
229
+ which takes precedence over `.duel.rb`, and an empty variable counts as
230
+ unset.
223
231
 
224
232
  For where the baseline comes from, how Duel checks it out and builds it,
225
233
  and how it compares a project in a subdirectory of its repository, see
@@ -269,7 +277,8 @@ merge request with the branch it targets and comment on it. See
269
277
  [Run Duel on GitLab CI](docs/ci.md#run-duel-on-gitlab-ci).
270
278
 
271
279
  To run Duel from Rake, define a task with `Duel::RakeTask`. Any
272
- arguments after the task's name are passed to `duel`:
280
+ arguments after the task's name are passed to `duel`, and the task
281
+ exits with `duel`'s status:
273
282
 
274
283
  ```ruby
275
284
  require "duel/rake_task"
@@ -380,20 +389,27 @@ says.
380
389
  To show that Duel guards your project, add its badge to your README,
381
390
  next to your other badges. With the job that `duel init --badge` adds to
382
391
  its workflow, the badge turns red when Duel fails a push to your default
383
- branch. Replace `OWNER/REPO` with your repository's:
392
+ branch, and links to your Perf workflow's runs. Replace `OWNER/REPO`
393
+ with your repository's:
384
394
 
385
395
  ```markdown
386
- [![Duel](https://raw.githubusercontent.com/OWNER/REPO/duel/badge/badge.svg)](https://github.com/sferik/duel-ruby)
396
+ [![Duel](https://raw.githubusercontent.com/OWNER/REPO/duel/badge/badge.svg)](https://github.com/OWNER/REPO/actions/workflows/perf.yml)
387
397
  ```
388
398
 
389
399
  See [Show whether the last push passed](docs/ci.md#show-whether-the-last-push-passed)
390
- for how the job keeps it. Without the job, such as on GitLab CI, use
391
- Duel's own badge, which is always green:
400
+ for how the job keeps it. Without the job, use Duel's own badge, which
401
+ is always green, linked to your Perf workflow's runs too:
392
402
 
393
403
  ```markdown
394
- [![Duel](https://raw.githubusercontent.com/sferik/duel-ruby/main/assets/badge.svg)](https://github.com/sferik/duel-ruby)
404
+ [![Duel](https://raw.githubusercontent.com/sferik/duel-ruby/main/assets/badge.svg)](https://github.com/OWNER/REPO/actions/workflows/perf.yml)
395
405
  ```
396
406
 
407
+ For a project in a subdirectory of its repository, the workflow is
408
+ `perf-<its path>.yml` (`perf-gems-parser.yml`, say). On GitLab CI, link
409
+ to its pipelines (`https://gitlab.com/OWNER/REPO/-/pipelines`). Projects
410
+ using Duel are listed in [USERS.md](USERS.md): add yours with a pull
411
+ request.
412
+
397
413
  ## Plugins
398
414
 
399
415
  A plugin packages what a server or a library needs, such as a server for
data/docs/benchmarks.md CHANGED
@@ -47,9 +47,9 @@ benchmark's.
47
47
 
48
48
  A file's hooks and a benchmark's own run in the benchmark's context, so
49
49
  they share its instance variables. Hooks in `.duel.rb` run where they're
50
- written, which suits a plugin's hooks. `setup`, `teardown`, and
51
- `before_sample` receive the benchmark, with its `name`, `only`, and
52
- `peak_memory`, if they take a parameter.
50
+ written, which suits a plugin's hooks. Only `around_iteration` takes
51
+ a parameter, its iteration: a hook of any other kind that takes one
52
+ fails, before anything runs.
53
53
 
54
54
  Inside a benchmark's own block, `once { ... }` runs its block the first
55
55
  time that the benchmark runs, and isn't timed, as in
@@ -82,7 +82,7 @@ a database's tables, give the hook in `.duel.rb`:
82
82
 
83
83
  ```ruby
84
84
  Duel.configure do |c|
85
- c.before_sample { |benchmark| MyDatabase.truncate_all } # not measured
85
+ c.before_sample { MyDatabase.truncate_all } # not measured
86
86
  end
87
87
  ```
88
88
 
@@ -235,24 +235,28 @@ By default, Duel judges each benchmark on its timings and its counts:
235
235
  own.
236
236
  - Counts: allocations, and any other exact counts.
237
237
 
238
- To change that, pass these options to `duel`:
239
-
240
- - `only: :count`: for work that's mostly waiting, too slow and light to
241
- time. Duel judges the benchmark on its counts alone, over a few
242
- iterations.
243
- - `only: :timing`: for work done by threads that the benchmark starts,
244
- which make its counts vary from run to run. Duel judges the benchmark
245
- on its timings alone, and takes the process's CPU time instead of the
246
- benchmark thread's.
247
- - `peak_memory: true`: also judges the benchmark's peak memory, which is
248
- the most resident memory that its measured work reached, above where
249
- it started, or on JRuby, the most of the JVM's heap that it had in
250
- use. Duel takes each version's lowest peak from three fresh
251
- workers. Like every other measure, peak memory leaves out what
252
- `once` and `untimed` blocks leave behind.
238
+ To change that, give `duel` what to judge, `judge:`, one of these or a
239
+ list of them (`judge: %i[timed counted]` by default):
240
+
241
+ - `:counted`: its counts. Alone, for work that's mostly waiting, too
242
+ slow and light to time, Duel judges the benchmark on its counts over a
243
+ few iterations.
244
+ - `:timed`: its timings. Alone, for work done by threads that the
245
+ benchmark starts, which make its counts vary from run to run, Duel
246
+ takes the process's CPU time instead of the benchmark thread's.
247
+ - `:peak_memory`: its peak memory too, with its timings: the most
248
+ resident memory that its measured work reached, above where it
249
+ started, or on JRuby, the most of the JVM's heap that it had in use.
250
+ Duel takes each version's lowest peak from three fresh workers. Like
251
+ every other measure, peak memory leaves out what `once` and `untimed`
252
+ blocks leave behind.
253
253
 
254
254
  ```ruby
255
- duel "server: 10 workers", only: :timing, peak_memory: true do |n|
255
+ duel "redis: PING", judge: :counted do
256
+ # ...
257
+ end
258
+
259
+ duel "server: 10 workers", judge: %i[timed peak_memory] do |n|
256
260
  # ...
257
261
  end
258
262
  ```
@@ -263,12 +267,12 @@ To guard how long your app takes to boot, or your library to load, use
263
267
  `duel_process`. Each sample starts fresh processes for each version,
264
268
  in that version's directory and with its environment, and the two
265
269
  versions take turns, as they do for every other benchmark. Give
266
- `duel_process` a command, or Ruby code to run with `ruby:`:
270
+ `duel_process` a command, or Ruby code to run with `code:`:
267
271
 
268
272
  ```ruby
269
273
  # duel/boot_duel.rb
270
274
  duel_process "boot: the app", "bin/rails", "runner", "1"
271
- duel_process "boot: require the library", ruby: %(require "my_gem")
275
+ duel_process "boot: require the library", code: %(require "my_gem")
272
276
  ```
273
277
 
274
278
  Duel runs a command the way that `Process.spawn` does, so a single
@@ -294,7 +298,7 @@ the most, gets 3, and an app that takes a second or more to boot gets
294
298
  yourself, pass `runs:`:
295
299
 
296
300
  ```ruby
297
- duel_process "boot: require the library", ruby: %(require "my_gem"), runs: 5
301
+ duel_process "boot: require the library", code: %(require "my_gem"), runs: 5
298
302
  ```
299
303
 
300
304
  Duel starts and measures the processes from a small Ruby process of its
@@ -315,5 +319,5 @@ makes the same code look slower.
315
319
  A process benchmark doesn't run in a worker, so nothing that runs there
316
320
  applies to it: hooks, from `.duel.rb` or its file, and the metrics that
317
321
  plugins add. `duel_process` doesn't take the options of `duel`
318
- (`only:`, `peak_memory:`, and the hooks, such as `around_iteration:`),
322
+ (`judge:`, and the hooks, such as `around_iteration:`),
319
323
  and raises an `ArgumentError` if it's given one.