duel 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 24277600f238129c16a52fb4115ccb6a6bd340602e7a38187622bfe1550adc1d
-  data.tar.gz: 489836bc05d935ef59e369149df8ec47e06279005691fd084ea9b587c489f175
+  metadata.gz: da30e26518ff9b626552115d24595276b5f6f371cc4cd9c614b126666a065898
+  data.tar.gz: d863cf940c61196483a7521ec8066385e4b2cc9071ec00c96bc6f794c266342c
 SHA512:
-  metadata.gz: 2ce8583f3c82f785cd25b95339c9fd7a751e42b7221d0a8b9af793351560945b8ca4d4001c7725db93c3a4d77506baf3cf5de37b5decd4c055f1ca17313de997
-  data.tar.gz: 81e190402ee6b2eb3705b41fece6ee45e12bc3a6f71645ac1467d7bbe5ed015195d9291fb4b651888b69b759fa4f109f88029e7b7c0272260c2326af11692b65
+  metadata.gz: 777b2dc46c93019d739aed99764ac70ecdf761e1309944cdacf449b735b6a931d965fa3fc3b37141874dfeb9b8074ce120f070e4e764ecc15f9b1edbe00617d0
+  data.tar.gz: 2879233c357cfd504d0b872efd37495cdf64e115da6ed9917fe1b0fad6ad0b7f070e8c2ed9b1cdf5b4b7444a3d7dbfeaa3155fe1cac58b425890dca99ce4f19a

data/AGENT_PROMPT.md

@@ -151,9 +151,9 @@ at the API.
    candidate's).
 
    Then do all three:
-   - Run `bundle exec duel --force`. With `lib/` unchanged, it compares
-     the code with itself and must pass. Run it three times: a noisy
-     benchmark can pass twice by luck. A benchmark that fails or
+   - Run `bundle exec duel --force`. With the code unchanged, it
+     compares the code with itself and must pass. Run it three times: a
+     noisy benchmark can pass twice by luck. A benchmark that fails or
      errors is broken. One reported unsettled in more than one run is
      noisy: fix it or drop it. (Across many benchmarks, one unsettled
      once is normal: it doesn't fail the run.) Benchmarks that run a
@@ -168,24 +168,28 @@ at the API.
      and rebuild the extension if you changed one.
    - Run `bundle exec duel --force --filter <pattern>` for a single
      benchmark, and for one area (`--filter "^parse:"`), to confirm
-     your names can be picked out. Without `--force`, an unchanged
-     `lib/` has nothing to compare.
+     your names can be picked out. Without `--force`, a tree with
+     nothing changed since the main branch has nothing to compare.
 
 6. **Check CI.** `duel init` wrote Duel's GitHub Actions
    workflow (`.github/workflows/perf.yml`). Keep `fetch-depth: 0`:
    Duel's baseline comes from git history. Keep its check for a push
    with no commit before it in the history (a branch's first push, or a
-   force push), which would otherwise fail the job. Keep its last step,
-   which comments on a pull request with Duel's report (Duel also adds
-   the report to the job's summary). Check that `ruby-version` is one
-   the project supports, unless it has a `.ruby-version`. With
-   `c.ruby = :version_file` and another version file than
-   `.ruby-version`, add the steps in Duel's `docs/ci.md` that install the
-   baseline's Ruby for a pull request. Add the project's other CI setup that its
-   tests need (system packages, services) to the workflow as well. Keep
-   the scheduled `last-release` run only if the project tags its
-   releases (`v*` by default, or else set `c.release_tags`), or leave
-   the schedule out. Until the first release is tagged, that run says
+   force push), which would otherwise fail the job. Keep its Comment
+   job, which comments on a pull request with Duel's report when a
+   benchmark fails (Duel also adds the report to the job's summary, and
+   `c.comment` in `.duel.rb` chooses when it comments), and keep
+   `pull-requests: write` on that job alone: the Duel job runs the
+   branch's code, so it only reads. Check that `ruby-version` is one the project supports, unless
+   it has a `.ruby-version`. With `c.ruby = :version_file` and another
+   version file than `.ruby-version`, add the steps in Duel's
+   `docs/ci.md` that install the baseline's Ruby, reading that file
+   instead. If `.duel.rb` sets `c.main_branch` or `c.release_tags`, set
+   those steps' `MAIN_BRANCH` or `RELEASE_TAGS` to match. Add the
+   project's other CI setup that its tests need (system packages,
+   services) to the Duel job as well. Keep the scheduled `last-release`
+   run only if the project tags its releases (`v*` by default, or else
+   set `c.release_tags`), or leave the schedule out. Until the first release is tagged, that run says
    there's nothing to compare against, and passes. Once it's pushed,
    run the workflow by hand: it compares the code with itself there,
    which shows that the benchmarks run on CI's machines (with their

data/CHANGELOG.md

@@ -7,6 +7,86 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.1.1] - 2026-10-26
+
+### Added
+
+- `c.comment` chooses when the workflow that `duel init` writes
+  comments on a pull request with the report: `:always`, `:results`
+  (when a run passed or failed, but not with nothing to compare),
+  `:failures` (when a benchmark failed or Duel couldn't compare, the
+  default), or `:never`. `duel --comment MODE` overrides it, and any
+  other value fails with the values it takes. On GitHub Actions, `duel`
+  sets the step outputs `comment` (`post`, `change`, or `none`) and
+  `status` (`passed`, `failed`, or `neutral`), whatever the outcome, and
+  the workflow's Comment job acts on them, since it runs none of the
+  branch's code and can't read `.duel.rb`. The JSON report's `comment`
+  key has the same decision.
+
+### Changed
+
+- The workflow that `duel init` writes no longer comments on every pull
+  request. By default, it comments only when a benchmark fails or Duel
+  couldn't compare. Set `c.comment = :always` for the old behavior.
+- The workflow never edits a comment. It used to update one comment on
+  every push, and now posts a new one each time it comments, so the pull
+  request keeps every report. Each comment's hidden marker records the
+  run's status (`<!-- duel status=failed -->`, say). When a run wouldn't
+  get a comment of its own, the workflow still posts one if its last
+  comment on the pull request had another status, so after a failure the
+  first run that passes says so, and the last comment is never stale. A
+  comment from an earlier version, whose marker has no status, counts
+  as one with another status.
+
+- A change to only the benchmarks or the configuration is compared. Duel
+  watches what decides how the code is measured, as well as the code
+  that `c.watch` names: the benchmarks' directory (`c.benchmarks` up to
+  its first wildcard, `duel/` by default), the configuration's file
+  (`.duel.rb`), and, with `c.ruby = :version_file`, the version files at
+  the project's root. Each version runs the candidate's benchmarks and
+  configuration, so such a change compares the code with itself, which
+  checks that they run and pass, where before Duel passed saying there
+  was nothing to compare. "Nothing to compare" messages, and the
+  candidate's `uncommitted_changes` in JSON, cover them too.
+- The Markdown report (`--markdown`, the job's summary and the pull
+  request comment) is shorter. Its heading starts with 🟢 when every
+  benchmark passed (`🟢 Duel: 100% of benchmarks passed`), 🔻 when one
+  failed or Duel couldn't compare (`🔻 Duel: 3 benchmarks failed`), and
+  ⬜ when there's nothing to compare. One line names the two commits,
+  bare, so GitHub links them, and how the baseline was chosen. Only the
+  benchmarks that failed are in view, in a table of what got worse and
+  by how much, or the error, and where the most extra CPU time went.
+  Everything else is collapsed, and the report no longer names the
+  plugins or explains how Duel decides outside the collapsed details.
+  The terminal and JSON reports are unchanged.
+
+### Security
+
+- The workflow that `duel init` writes no longer gives the job that runs
+  a pull request's code the `pull-requests: write` permission. The Duel
+  job only reads (`contents: read`, and its checkout keeps no
+  credentials), and keeps the report as an artifact for a day. A
+  Comment job of its own, which checks nothing out and runs none of the
+  pull request's code, downloads the report and comments with it, for a
+  pull request from the same repository.
+
+### Fixed
+
+- With `c.ruby = :version_file`, the workflow installs the bundle for
+  the baseline's Ruby too, when it differs from the candidate's. The
+  baseline's workers load that bundle, and gems installed for one Ruby
+  version don't load on another, so a branch that changed the Ruby's
+  minor version failed before it compared anything.
+
+- With `c.ruby = :version_file`, the workflow that `duel init` writes
+  installs the baseline's Ruby for every event that compares, not only
+  for a pull request. It finds the baseline that the Compare step uses
+  (where HEAD left the main branch for a pull request and a manual run,
+  the commit before a push, and the last release on a schedule, with
+  the release tags' glob in `RELEASE_TAGS`), and installs its Ruby only
+  when its `.ruby-version` names another Ruby than HEAD's. Its scripts
+  take the workflow's values from environment variables, not `${{ }}`.
+
 ## [0.1.0] - 2026-10-24
 
 The first release. Duel is open source under the MIT license, and
@@ -277,5 +357,6 @@ needs Ruby 3.4 or later (or JRuby 10, or TruffleRuby 40).
   Anything marked `@api private`, in lib/ and sig/, may change in any
   release.
 
-[Unreleased]: https://github.com/sferik/duel-ruby/compare/v0.1.0...HEAD
+[Unreleased]: https://github.com/sferik/duel-ruby/compare/v0.1.1...HEAD
+[0.1.1]: https://github.com/sferik/duel-ruby/compare/v0.1.0...v0.1.1
 [0.1.0]: https://github.com/sferik/duel-ruby/releases/tag/v0.1.0

data/README.md

@@ -227,9 +227,10 @@ Duel::RakeTask.new(:perf_since_release, "--base", "last-release")
 `duel init` writes a GitHub Actions workflow,
 `.github/workflows/perf.yml`. It compares each pull request with the
 main branch, each push to `main` with the commit before it, and, on a
-schedule, your last release, to catch slow creep. It also comments on
-each pull request with Duel's report. For the workflow, and for a branch
-that upgrades Ruby, see [Run Duel in CI](docs/ci.md).
+schedule, your last release, to catch slow creep. It also comments on a
+pull request with Duel's report when a benchmark fails. For the
+workflow, and for a branch that upgrades Ruby, see
+[Run Duel in CI](docs/ci.md).
 
 ### Report on a pull request
 
@@ -242,28 +243,27 @@ comment, use `--markdown`:
 duel --markdown tmp/duel.md
 ```
 
-The report starts with the verdict and the two versions that Duel
-compared, with each version's Ruby when they differ. A table lists the
-benchmarks that changed, regressions first, with each timing's ratio and
-its interval, and the counts that changed. Each regression is explained
-as it is in the terminal, with what its metrics said and where the extra
-CPU time went. Then come the benchmarks whose timings were too close to
-call, and the benchmarks that failed. The benchmarks that didn't change
-are collapsed, so the report stays short with hundreds of benchmarks:
+The report is short. In view, it has the verdict, the two commits that
+Duel compared (bare, so GitHub links them), and how the baseline was
+chosen. When a benchmark fails, a table shows what got worse and by how
+much, or the error, and the frame where the most extra CPU time went.
+Everything else is collapsed: each regression explained as it is in the
+terminal, every benchmark's timings, the timings too close to call, the
+verdicts across the board, each version's Ruby when they differ, and
+how Duel decides. So the report stays short with hundreds of
+benchmarks:
 
 ````markdown
-### Duel failed: 1 regression
+### 🔻 Duel: 1 benchmark failed
 
-- **Baseline:** `3f9c2a1`, where HEAD left `origin/main`
-- **Candidate:** `8e4d0b7`
+Compared 8e4d0b7 with 3f9c2a1 (where HEAD left `origin/main`).
 
-12 benchmarks in 14s. A timing is slower when a 99.0% confidence interval for its ratio, the candidate's to the baseline's, is above 1.02.
-
-| Benchmark | Verdict | Timings (ratio [interval]) | Counts an iteration |
-|---|---|---|---|
-| `parse: a large document` | **regressed** | instructions 1.071 [1.069, 1.073] slower<br>cpu 1.081 [1.062, 1.097] slower | allocations 5790 → 5791 |
+| Benchmark | Failure | Most extra CPU in |
+|---|---|---|
+| `parse: a large document` | instructions +7.1%, cpu +8.1%, allocations 5790 → 5791 | `MyParser::Lexer#next_token` +6.8% |
 
-#### Regressions
+<details>
+<summary>Details: 12 benchmarks in 14s</summary>
 
 **`parse: a large document`**: worse instructions, cpu, allocations in every measurement
 
@@ -276,17 +276,32 @@ where the extra CPU went (as a share of the baseline's: in each frame itself, an
    +0.4%   +0.4%  String#byteslice
 ```
 
-<details>
-<summary>11 benchmarks unchanged</summary>
+| Benchmark | Verdict | Timings (ratio [interval]) | Counts an iteration |
+|---|---|---|---|
+| `parse: a large document` | **regressed** | instructions 1.071 [1.069, 1.073] slower<br>cpu 1.081 [1.062, 1.097] slower | allocations 5790 → 5791 |
+| `parse: a short line` | same | instructions 1.000 [1.000, 1.000]<br>cpu 1.002 [0.992, 1.017] |  |
 
 ...
 
 </details>
 ````
 
+When every benchmark passes, the heading says `🟢 Duel: 100% of
+benchmarks passed`, and only the commits are in view. A timing too
+close to the limit to call doesn't fail the run, and is in the details.
+A benchmark that the baseline can't run (a new one, say) is skipped, and
+the heading counts it. A run that has nothing to compare, or no release
+to compare against, says so under ⬜, and one that couldn't compare
+says why under 🔻.
+
 The workflow that `duel init` writes posts this report as a comment on
-the pull request, and updates the same comment on every push. For more
-information, see
+the pull request when a benchmark fails, or when Duel couldn't compare.
+After such a comment, the first run that passes posts one more, so the
+last comment is never stale. It never edits a comment, so the pull
+request keeps every report. To comment on every run, on every result
+but nothing to compare, or never, set
+[`c.comment`](docs/configuration.md#choose-when-to-comment-on-a-pull-request)
+in `.duel.rb`. For more information, see
 [Comment on a pull request](docs/ci.md#comment-on-a-pull-request).
 
 ### Show that a project is guarded

data/docs/ci.md

@@ -12,8 +12,8 @@ example, a series of changes that are each 1.5% slower would each pass.
 To catch that kind of slow creep, also compare against your last release
 on a schedule. The slowdowns add up there until they're a regression.
 
-The following GitHub Actions workflow does both, and reports on each
-pull request. `duel init` writes it for you, as
+The following GitHub Actions workflow does both, and comments on a pull
+request when a benchmark fails. `duel init` writes it for you, as
 `.github/workflows/perf.yml`:
 
 ```yaml
@@ -28,18 +28,25 @@ on:
     - cron: "0 4 * * *"
   workflow_dispatch:
 
+# The Duel job runs the branch's code (its benchmarks too), so it only
+# reads. Commenting is the Comment job's, which runs none of it.
 permissions:
   contents: read
-  pull-requests: write # to comment on a pull request with the report
 
 jobs:
   duel:
     name: Duel
     runs-on: ubuntu-latest
+    # Whether the Comment job comments (post, change or none), and the
+    # run's status (passed, failed or neutral), from Duel.
+    outputs:
+      comment: ${{ steps.compare.outputs.comment }}
+      status: ${{ steps.compare.outputs.status }}
     steps:
       - uses: actions/checkout@v7
         with:
           fetch-depth: 0 # the baseline comes from history, tags included
+          persist-credentials: false
       # (And whatever else your tests need set up: packages, services.)
       - uses: ruby/setup-ruby@v1
         with:
@@ -47,9 +54,11 @@ jobs:
           bundler-cache: true
       # Against the main branch for a pull request, what was there before
       # for a push, and the last release on a schedule (slow creep). Duel
-      # adds its report to the job's summary, and writes it to duel.md for
-      # the comment.
+      # adds its report to the job's summary, writes it to duel.md for the
+      # comment, and sets the step's outputs to say whether to comment, as
+      # c.comment in .duel.rb says, whatever the outcome.
       - name: Compare
+        id: compare
         env:
           DUEL_BASE: ${{ github.event_name == 'schedule' && 'last-release' || github.event_name == 'push' && github.event.before || '' }}
           # (Run by hand, it compares anyway: with nothing changed, the code
@@ -63,23 +72,53 @@ jobs:
             exit 0
           fi
           bundle exec duel --markdown "$RUNNER_TEMP/duel.md"
-      # The report, as a comment on the pull request: one comment, updated
-      # on every push. (A pull request from a fork gets no comment, as its
-      # token can't write one: the job's summary has the report.)
+      # The report, for the Comment job.
+      - name: Keep the report
+        if: ${{ !cancelled() && github.event_name == 'pull_request' }}
+        uses: actions/upload-artifact@v7
+        with:
+          name: duel-report
+          path: ${{ runner.temp }}/duel.md
+          if-no-files-found: ignore # none when Duel didn't get to compare
+          retention-days: 1
+
+  # The report, as a new comment on the pull request, when Duel says to:
+  # always (post), or only when its last comment's status differs from
+  # this run's (change), so the latest comment is never stale. It never
+  # edits or deletes a comment. This job checks nothing out and runs none
+  # of the branch's code, so it alone can write. (A pull request from a
+  # fork gets no comment, as its token can't write one: the Duel job's
+  # summary has the report.)
+  comment:
+    name: Comment
+    needs: duel
+    if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && (needs.duel.outputs.comment == 'post' || needs.duel.outputs.comment == 'change') }}
+    runs-on: ubuntu-latest
+    permissions:
+      pull-requests: write # to comment on the pull request with the report
+    steps:
+      - name: Get the report
+        id: report
+        continue-on-error: true # there's none when the Duel job failed before comparing
+        uses: actions/download-artifact@v8
+        with:
+          name: duel-report
+          path: ${{ runner.temp }}/report
       - name: Comment on the pull request
-        if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }}
+        if: steps.report.outcome == 'success'
         env:
           GH_TOKEN: ${{ github.token }}
           PR: ${{ github.event.pull_request.number }}
+          COMMENT: ${{ needs.duel.outputs.comment }}
+          STATUS: ${{ needs.duel.outputs.status }}
         run: |
-          [ -f "$RUNNER_TEMP/duel.md" ] || exit 0
-          marker="<!-- duel -->"
-          { echo "$marker"; cat "$RUNNER_TEMP/duel.md"; } > "$RUNNER_TEMP/comment.md"
-          ids=$(gh api --paginate "repos/$GITHUB_REPOSITORY/issues/$PR/comments" \
-            --jq ".[] | select(.user.login == \"github-actions[bot]\" and (.body | startswith(\"$marker\"))) | .id")
-          if [ -n "$ids" ]; then
-            gh api --method PATCH "repos/$GITHUB_REPOSITORY/issues/comments/${ids%%$'\n'*}" -F body=@"$RUNNER_TEMP/comment.md" > /dev/null
-          else
+          marker="<!-- duel"
+          # The first line of the last comment Duel posted here: its marker,
+          # with its status (none in Duel 0.1.0's, which counts as another).
+          last=$(gh api --paginate "repos/$GITHUB_REPOSITORY/issues/$PR/comments" \
+            --jq ".[] | select(.user.login == \"github-actions[bot]\") | .body | split(\"\n\")[0] | rtrimstr(\"\r\") | select(startswith(\"$marker status=\") or . == \"$marker -->\")" | tail -n 1)
+          if [ "$COMMENT" = post ] || { [ -n "$last" ] && [ "$last" != "$marker status=$STATUS -->" ]; }; then
+            { echo "$marker status=$STATUS -->"; cat "$RUNNER_TEMP/report/duel.md"; } > "$RUNNER_TEMP/comment.md"
             gh api --method POST "repos/$GITHUB_REPOSITORY/issues/$PR/comments" -F body=@"$RUNNER_TEMP/comment.md" > /dev/null
           fi
 ```
@@ -95,17 +134,61 @@ The workflow compares the following:
 - When you run the workflow manually, your code against itself. This
   checks that the benchmarks run on the CI machines.
 
+Duel compares a change that touches your code (`lib/` or `ext/`, or what
+`c.watch` names), your benchmarks, `.duel.rb`, or, with
+`c.ruby = :version_file`, your version file. Each version runs your
+branch's benchmarks and configuration, so a change to only those compares
+the same code with itself, which checks that they run and pass. A change
+to only the workflow has nothing to compare: to check it, run the
+workflow manually.
+
 ## Comment on a pull request
 
 The workflow writes Duel's report
 ([Report on a pull request](../README.md#report-on-a-pull-request))
-to a file, and then comments on the pull request with it. It updates
-the same comment on every push, rather than adding one each time. It
-finds its comment by the hidden marker on the comment's first line, so
-give each workflow that comments a marker of its own. Commenting takes
-the `pull-requests: write` permission. A pull request from a fork gets
-no comment, because its token can't write one, but the job's summary
-still has the report.
+to a file, and comments on the pull request with it when
+[`c.comment`](configuration.md#choose-when-to-comment-on-a-pull-request)
+says to. By default, it comments when a benchmark fails, or when Duel
+couldn't compare. Whatever `c.comment` says, a regression fails the
+check, because `duel` exits with a nonzero status, and the job's
+summary has the report.
+
+The workflow never edits or deletes a comment, so the pull request
+keeps every report that it posted. Each comment starts with a hidden
+marker that records the run's status: `<!-- duel status=passed -->`,
+`status=failed` (a run that couldn't compare too), or `status=neutral`
+(nothing to compare). When a run's outcome wouldn't get a comment of
+its own, the workflow still posts one if it commented on the pull
+request before, and its last comment's status differs from this run's.
+With the default, after a comment that a benchmark failed, the first
+push that passes gets one comment saying so, and the pushes after it
+get none. So the last comment is never stale. A comment from Duel 0.1.0,
+whose marker has no status, counts as a comment with another status.
+
+The workflow finds its comments by the marker's start. For a project
+in a subdirectory of its repository, the marker names the directory,
+such as `<!-- duel gems/parser status=failed -->`, so each of a
+monorepo's workflows has comments of its own.
+
+Duel decides, because it knows the outcome and reads `.duel.rb`. On
+GitHub Actions, the Compare step's `duel` sets two step outputs, which
+the Duel job passes on as its own: `comment`, and `status` (`passed`,
+`failed`, or `neutral`). `comment` is one of the following:
+
+| `comment` | What the Comment job does |
+|---|---|
+| `post` | Posts a new comment. |
+| `change` | Posts a new comment only if the pull request has one from the workflow, and the last one's status isn't this run's. |
+| `none` | Nothing: the Comment job doesn't run. |
+
+The Duel job runs the pull request's code, its benchmarks included, so
+it only reads (`contents: read`), and it keeps the report as an
+artifact for a day. A job of its own, Comment, posts the report. It
+checks nothing out and runs none of the pull request's code, so it
+alone gets the `pull-requests: write` permission that commenting takes.
+When the Duel job fails before it compares, there's no report, and no
+comment. A pull request from a fork gets no comment either, because its
+token can't write one, but the Duel job's summary still has the report.
 
 ## Run each version on its own Ruby on CI
 
@@ -113,29 +196,70 @@ With [`c.ruby = :version_file`](configuration.md#run-each-version-on-its-own-rub
 each version runs on the Ruby that its own version file names. On CI, a
 setup step such as `ruby/setup-ruby` installs the Ruby that
 runs `duel` where no version manager looks. That's enough while both
-versions ask for that Ruby. A pull request that upgrades Ruby needs the
+versions ask for that Ruby. A branch that upgrades Ruby needs the
 baseline's Ruby installed too, where Duel finds it. For example, for a
 project with a `.ruby-version`, in the
-[GitHub Actions workflow](#catch-slow-creep), install the main branch's
-Ruby with a second `ruby/setup-ruby` step, and link it into
-`~/.rubies`, where Duel looks for Rubies the way that chruby does.
-`duel init` adds these steps for a project that has a `.ruby-version`:
+[GitHub Actions workflow](#catch-slow-creep), find the baseline that the
+Compare step compares with, for each event: where `HEAD` left the main
+branch for a pull request and a manual run, the commit before a push,
+and the last release on a schedule. When its `.ruby-version` names
+another Ruby than yours, install that Ruby with a second
+`ruby/setup-ruby` step, with the bundle for it (`bundler-cache: true`),
+and link it into `~/.rubies`, where Duel looks for Rubies the way that
+chruby does. The baseline's workers load the same bundle as the
+candidate's, and gems installed for one Ruby version don't load on
+another, so without that bundle they can't load the code to compare. Without a baseline, such as for a
+branch's first push, or before your first release, there's no Ruby to
+install. The step finds the main branch and the last release the way
+Duel does by default: if `.duel.rb` sets `c.main_branch` or
+`c.release_tags`, set the step's `MAIN_BRANCH` or `RELEASE_TAGS` to
+match. `duel init` adds these steps for a project that has a
+`.ruby-version`:
 
 ```yaml
       # (Before the step that sets up the candidate's Ruby.)
+      # Each version runs on the Ruby its own .ruby-version names (see
+      # .duel.rb). When the baseline's names another Ruby than this code's,
+      # that one is installed too, and linked where Duel finds it (as
+      # chruby would), before the project's own Ruby, which runs Duel. The
+      # baseline is the Compare step's: where HEAD left the main branch
+      # (for a pull request, and run by hand), the commit before a push, or
+      # on a schedule the last release, the newest tag before HEAD that
+      # RELEASE_TAGS matches. (If .duel.rb sets c.main_branch or
+      # c.release_tags, set MAIN_BRANCH or RELEASE_TAGS to match.)
       - name: Find the baseline's Ruby
         id: baseline
-        if: github.event_name == 'pull_request'
-        run: echo "ruby=$(git show "origin/$GITHUB_BASE_REF:.ruby-version")" >> "$GITHUB_OUTPUT"
+        env:
+          BEFORE: ${{ github.event.before }}
+          MAIN_BRANCH: origin/${{ github.event.repository.default_branch }}
+          RELEASE_TAGS: "v*"
+        run: |
+          case "$GITHUB_EVENT_NAME" in
+            push) base=$BEFORE ;;
+            schedule) base=$(git describe --tags --abbrev=0 --match "$RELEASE_TAGS" HEAD^ 2>/dev/null) || base= ;;
+            *) base=$(git merge-base HEAD "$MAIN_BRANCH" 2>/dev/null) || base= ;;
+          esac
+          ruby_of() { git show "$1:./.ruby-version" 2>/dev/null | awk 'NF { print $1; exit }'; }
+          ruby=
+          # (None without a baseline: a branch's first push, or no release yet.)
+          if [ -n "$base" ] && git cat-file -e "$base^{commit}" 2>/dev/null; then
+            ruby=$(ruby_of "$base")
+            [ "$ruby" = "$(ruby_of HEAD)" ] && ruby=
+          fi
+          echo "ruby=$ruby" >> "$GITHUB_OUTPUT"
       - uses: ruby/setup-ruby@v1
         id: baseline-ruby
-        if: github.event_name == 'pull_request'
+        if: steps.baseline.outputs.ruby != ''
         with:
           ruby-version: ${{ steps.baseline.outputs.ruby }}
+          # The bundle for this Ruby too, which the baseline's workers load.
+          # (Gems installed for one Ruby version don't load on another.)
+          bundler-cache: true
       - name: Link the baseline's Ruby where Duel finds it
-        if: github.event_name == 'pull_request'
+        if: steps.baseline.outputs.ruby != ''
+        env:
+          PREFIX: ${{ steps.baseline-ruby.outputs.ruby-prefix }}
         run: |
-          prefix="${{ steps.baseline-ruby.outputs.ruby-prefix }}"
           mkdir -p ~/.rubies
-          ln -sfn "$prefix" ~/.rubies/ruby-"$("$prefix/bin/ruby" -e 'print RUBY_VERSION')"
+          ln -sfn "$PREFIX" ~/.rubies/ruby-"$("$PREFIX/bin/ruby" -e 'print RUBY_VERSION')"
 ```

data/docs/configuration.md

@@ -2,19 +2,21 @@
 
 Most gems don't need any configuration: Duel's conventions cover them.
 Benchmarks are in `duel/**/*_duel.rb`, each version's `lib/` is first on
-the load path, a change is compared when it touches `lib/` or `ext/`,
-and releases are tagged `v*`. If your project differs, create a `.duel.rb` file
-at its root. Both the runner and every worker load it:
+the load path, a change is compared when it touches `lib/` or `ext/`
+(or the benchmarks, or `.duel.rb`), and releases are tagged `v*`. If
+your project differs, create a `.duel.rb` file at its root. Both the
+runner and every worker load it:
 
 ```ruby
 Duel.configure do |c|
   c.benchmarks = "perf/**/*_bench.rb"    # default: duel/**/*_duel.rb
-  c.watch = %w[lib ext]                  # what a change must touch to be compared
+  c.watch = %w[lib ext]                  # the code a change must touch to be compared
   c.library = "my_gem"                   # required from each version (default: the gemspec's)
   c.build = "make"                       # for both versions (default: rake compile, with ext/)
   c.main_branch = "origin/trunk"
   c.release_tags = "release-*"           # default: v*
   c.ruby = :version_file                 # each version's own Ruby (default: the runner's)
+  c.comment = :results                   # when CI comments on a pull request (default: :failures)
   c.use :redis                           # a plugin (see Use plugins)
 
   # How a worker loads one version of your code, instead of the library.
@@ -28,6 +30,66 @@ directory. The configuration also takes hooks, such as
 `c.before_sample`. See
 [Run code around a benchmark](benchmarks.md#run-code-around-a-benchmark).
 
+## Choose what a change must touch to be compared
+
+When nothing that Duel watches changed since the baseline, there's
+nothing to compare, and Duel passes without running the benchmarks.
+`c.watch` names your code: `lib/` and `ext/` by default. Duel also
+watches what decides how your code is measured:
+
+- The benchmarks' directory: the part of `c.benchmarks` before its first
+  wildcard, `duel/` by default, with its helpers and fixtures.
+- The configuration's file, `.duel.rb`.
+- With [`c.ruby = :version_file`](#run-each-version-on-its-own-ruby),
+  the version files at your project's root (`.ruby-version`,
+  `.tool-versions`, `mise.toml`, or `.mise.toml`).
+
+Each version runs your branch's benchmarks and configuration, so a
+change to only those compares your code with itself. That checks that
+the changed benchmarks run, and pass on identical code, instead of
+passing without running them. To compare a change to anything else, such
+as your CI workflow, add it to `c.watch`:
+
+```ruby
+Duel.configure do |c|
+  c.watch += %w[.github/workflows/perf.yml]
+end
+```
+
+## Choose when to comment on a pull request
+
+The [GitHub Actions workflow](ci.md#comment-on-a-pull-request) that
+`duel init` writes comments on a pull request with Duel's report.
+`c.comment` chooses when, and `duel --comment` overrides it:
+
+| Value | When the workflow comments |
+|---|---|
+| `:always` | On every run, whatever the outcome. |
+| `:results` | When the run passed or failed, but not when there was nothing to compare. |
+| `:failures` | When a benchmark failed, or Duel couldn't compare. This is the default. |
+| `:never` | Never. |
+
+```ruby
+Duel.configure do |c|
+  c.comment = :results
+end
+```
+
+The workflow never edits a comment: each one is new, so the pull
+request keeps them all. Once the workflow has commented on a pull
+request, it also comments on a run that it otherwise wouldn't, except
+with `:never`, when the run's status (passed, failed, or nothing to
+compare) differs from its last comment's. So with `:failures`, after a
+comment that a benchmark failed, the first run that passes posts a
+comment saying so, and the runs after it post nothing. The last comment
+is never stale.
+
+With `:never`, a regression still fails the check, because `duel` exits
+with a nonzero status, and the job's summary still has the report.
+
+`c.comment` takes only these four values: any other fails the run, and
+says which values it takes.
+
 ## Prepare each worker
 
 Each worker can get things of its own, such as a server, so that the

data/docs/how-it-works.md

@@ -169,8 +169,10 @@ that small. If Duel can't write next to your project, or something else
 already has that name, it uses `tmp/duel/base` instead. Whenever the two
 versions' paths differ in length, Duel says so.
 
-If nothing under `lib/` or `ext/` changed since the baseline, there's
-nothing to compare, and Duel says so. To compare anyway, use `--force`.
+If nothing under `lib/` or `ext/`, your benchmarks, or `.duel.rb`
+changed since the baseline, there's nothing to compare, and Duel says
+so. To compare anyway, use `--force`. See
+[Choose what a change must touch to be compared](configuration.md#choose-what-a-change-must-touch-to-be-compared).
 
 Duel also builds the candidate before every run. Builds are incremental,
 so a build that's already done costs little. A stale or missing build,