duel-rails 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +99 -0
- data/LICENSE +21 -0
- data/README.md +456 -0
- data/lib/duel/rails/bundle.rb +45 -0
- data/lib/duel/rails/counter.rb +122 -0
- data/lib/duel/rails/database.rb +54 -0
- data/lib/duel/rails/jobs.rb +63 -0
- data/lib/duel/rails/railtie.rb +19 -0
- data/lib/duel/rails/request.rb +125 -0
- data/lib/duel/rails/version.rb +7 -0
- data/lib/duel/rails.rb +152 -0
- data/lib/generators/duel/install/install_generator.rb +126 -0
- data/lib/generators/duel/install/templates/benchmark.rb.tt +27 -0
- data/lib/generators/duel/install/templates/boot_duel.rb.tt +9 -0
- data/lib/generators/duel/install/templates/duel.rb.tt +23 -0
- data/lib/generators/duel/install/templates/perf.yml.tt +109 -0
- data/lib/generators/duel/install/templates/requests_duel.rb.tt +13 -0
- data/sig/duel/rails.rbs +245 -0
- metadata +106 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 6bcc25c75b990e312e5fb3a458f5e3eace8b5dbbe02fcb8c50f0ca0df01d8f54
|
|
4
|
+
data.tar.gz: a6b29676404b6e35d14321d05f428d902d230d169f084abd65afd7104081fed1
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 85ed8cb50f379354f0ba9a5769688fbcc1608bb4ee8dfff8e7586e2f736ed4db193bb99d82c7d76a37b6177175cb010364e60e0e83e6a6aec4fc11539ec30767
|
|
7
|
+
data.tar.gz: a514a4ddd962ac4b5c5711ec829ccb14a95d9e46cd1be6fc4a289d0bae2518995e49216f57c15e0e6d321832951f611c49e5db4563a5af580475f573f1fae682
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/2.0.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.1.0] - 2026-10-24
|
|
11
|
+
|
|
12
|
+
The first release. duel-rails is a Duel plugin for Rails apps, open
|
|
13
|
+
source under the MIT license. It needs Duel 0.1, duel-activerecord 0.1,
|
|
14
|
+
Rails 7.2 or later, and Ruby 3.4 or later.
|
|
15
|
+
|
|
16
|
+
### Added
|
|
17
|
+
|
|
18
|
+
- `c.use :rails`: each version of the app boots in the
|
|
19
|
+
`benchmark` environment, from its own tree, and Duel compares a change
|
|
20
|
+
to `app/`, `config/`, `db/`, `lib/`, `Gemfile`, `Gemfile.lock`, or
|
|
21
|
+
`.ruby-version`.
|
|
22
|
+
- Each version's workers run its own bundle: its `Gemfile.lock`,
|
|
23
|
+
installed once for the run, before its workers start (Duel's
|
|
24
|
+
`c.prepare(:version)`), on the version's Ruby, and kept installed for
|
|
25
|
+
each lockfile and Ruby. Duel puts its own gems and its plugins' ahead
|
|
26
|
+
of the bundle.
|
|
27
|
+
- Each worker's own database (`database:`): a SQLite file, or a
|
|
28
|
+
PostgreSQL or MySQL server from duel-postgresql or duel-mysql. Each
|
|
29
|
+
worker loads its version's schema into it, then the fixtures
|
|
30
|
+
(`fixtures:`) and seeds (`seeds:`) asked for, with Ruby's random
|
|
31
|
+
numbers, and Faker's, seeded first (`seed:`). duel-rails uses
|
|
32
|
+
duel-activerecord, and the database server's plugin, with `c.use`, so
|
|
33
|
+
an app that uses one of them itself first, with options of its own,
|
|
34
|
+
keeps them.
|
|
35
|
+
- `duel_request`: a benchmark of a request, sent through the app's whole
|
|
36
|
+
Rack stack without a server, with `params:`, `json:`, and `headers:`,
|
|
37
|
+
and a block to change its Rack environment. It fails on a status
|
|
38
|
+
other than the one expected (`status:`, `200..399` by default). It
|
|
39
|
+
takes Duel's hooks (`setup:`, `before_sample:`, and the rest), as
|
|
40
|
+
`duel` does, and gets its file's hooks, as a benchmark that the file
|
|
41
|
+
defines with `duel` does.
|
|
42
|
+
- `Duel::Rails::ROLLBACK`, an `around_iteration` hook that rolls back
|
|
43
|
+
each iteration: for one benchmark
|
|
44
|
+
(`around_iteration: Duel::Rails::ROLLBACK`), a file, or every
|
|
45
|
+
benchmark (`c.around_iteration(&Duel::Rails::ROLLBACK)`). What a
|
|
46
|
+
request writes is rolled back with it, so pinning the connections and
|
|
47
|
+
rolling back aren't measured, and the rollback transaction doesn't
|
|
48
|
+
count in `active_record_transactions`. `rollback: false` sends the
|
|
49
|
+
request without it.
|
|
50
|
+
- With a server's database (`database: :postgresql`, `:mysql2`, or
|
|
51
|
+
`:trilogy`), a request benchmark reads the server's statistics only
|
|
52
|
+
as each sample starts and stops, not around every request, because
|
|
53
|
+
the server's plugin says its metrics are costly to read. What pinning
|
|
54
|
+
and rolling back add is learned in each worker and subtracted.
|
|
55
|
+
- Jobs that a benchmark enqueues are performed in an `after_iterations`
|
|
56
|
+
hook, and measured with it: after each request, before it's rolled
|
|
57
|
+
back, or after each sample's iterations otherwise. duel-rails's own
|
|
58
|
+
queue is Active Job's adapter while benchmarking, on Rails 7.2, 8.0,
|
|
59
|
+
and 8.1. `jobs: false` leaves jobs out.
|
|
60
|
+
- `reset: true` empties every table, and loads the fixtures and seeds
|
|
61
|
+
again, before every sample (Duel's `c.before_sample`), for benchmarks
|
|
62
|
+
that write without rolling back. `Duel::Rails.reset_data` does the
|
|
63
|
+
same from a file's `before_sample` block.
|
|
64
|
+
- The `rails_views`, `rails_cache_reads`, `rails_cache_misses`, and
|
|
65
|
+
`rails_cache_writes` metrics, from Rails's notifications, subscribed
|
|
66
|
+
to as each worker opens them and unsubscribed from as it closes them.
|
|
67
|
+
Where the views regressed, the views each version rendered most.
|
|
68
|
+
Active Record's metrics too, from duel-activerecord (`active_record:`,
|
|
69
|
+
`query_time:`).
|
|
70
|
+
- `rails generate duel:install`: `.duel.rb`, the `benchmark`
|
|
71
|
+
environment, benchmarks of the app's requests (one for each action,
|
|
72
|
+
however many paths reach it), `duel/boot_duel.rb`, a benchmark of the
|
|
73
|
+
app's boot (`duel_process "boot: the app", "bin/rails", "runner",
|
|
74
|
+
"1"`, whose processes run in the version's `RAILS_ENV`), a GitHub
|
|
75
|
+
Actions workflow, and, in the `Gemfile`'s `development` group,
|
|
76
|
+
`gem "stackprof", platforms: :mri` (for profiles of regressions,
|
|
77
|
+
unless the `Gemfile` has StackProf) and the database server's plugin.
|
|
78
|
+
The `benchmark` environment sets Active Job's queue adapter only in an
|
|
79
|
+
app with Active Job.
|
|
80
|
+
- The generated workflow writes Duel's report in Markdown
|
|
81
|
+
(`duel --markdown`) and comments on each pull request with it,
|
|
82
|
+
updating one comment on every push. Duel adds the report to the job's
|
|
83
|
+
summary too. The workflow asks for the `pull-requests: write`
|
|
84
|
+
permission, and an app in a subdirectory of its repository gets a
|
|
85
|
+
comment of its own.
|
|
86
|
+
- For an app with a `.ruby-version`, the generated `.duel.rb` sets
|
|
87
|
+
`c.ruby = :version_file`, so each version runs on its own Ruby, and the
|
|
88
|
+
generated workflow also installs the main branch's Ruby for a pull
|
|
89
|
+
request.
|
|
90
|
+
- An app in a subdirectory of its repository (a monorepo's): the
|
|
91
|
+
generator writes its workflow at the repository's root, named for the
|
|
92
|
+
app's directory, and run there (`working-directory`).
|
|
93
|
+
- `rake duel`, in the app.
|
|
94
|
+
- Mutation tested: mutant kills every change to the plugin's code, in
|
|
95
|
+
CI.
|
|
96
|
+
- RBS signatures in sig/, checked with Steep.
|
|
97
|
+
|
|
98
|
+
[Unreleased]: https://github.com/sferik/duel-ruby/compare/duel-rails-v0.1.0...HEAD
|
|
99
|
+
[0.1.0]: https://github.com/sferik/duel-ruby/releases/tag/duel-rails-v0.1.0
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Erik Berlin
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,456 @@
|
|
|
1
|
+
# duel-rails
|
|
2
|
+
|
|
3
|
+
[](https://github.com/sferik/duel-ruby)
|
|
4
|
+
[](https://github.com/sferik/duel-ruby/actions/workflows/plugin-rails.yml)
|
|
5
|
+
[](https://rubygems.org/gems/duel-rails)
|
|
6
|
+
|
|
7
|
+
A [Duel](https://github.com/sferik/duel-ruby) plugin for Rails apps. It
|
|
8
|
+
fails a pull request that makes your app's requests slower, or makes
|
|
9
|
+
them query the database, render views, or allocate more.
|
|
10
|
+
|
|
11
|
+
```sh
|
|
12
|
+
$ bin/rake duel
|
|
13
|
+
Comparing 3 benchmarks, 2 at a time: baseline /src/.8e2f1c4 against candidate /src/my_app (slower: a 99.0% interval above 1.02)
|
|
14
|
+
posts: index 2.7s n=4 cpu 1.004 [0.991, 1.018] same active_record_queries 1→2 rails_views 4→7 allocations 988→1530
|
|
15
|
+
...
|
|
16
|
+
REGRESSION posts: index: worse active_record_queries, rails_views, allocations in every measurement
|
|
17
|
+
active_record_queries: 1 query → 2 queries an iteration
|
|
18
|
+
active_record_queries (candidate): an iteration: 1× SELECT "posts".* FROM "posts"; 1× SELECT COUNT(*) FROM "posts"
|
|
19
|
+
rails_views (candidate): an iteration: 6× posts/_post.html.erb; 1× posts/index.html.erb; 1× layouts/application.html.erb
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
duel-rails sets Duel up so that each version of your app runs the way
|
|
23
|
+
production does, with everything of its own:
|
|
24
|
+
|
|
25
|
+
- **Booted like production.** Each version boots in a `benchmark`
|
|
26
|
+
environment that's based on your production environment, with eager
|
|
27
|
+
loading and YJIT, but without SSL, logging, or other databases.
|
|
28
|
+
- **Its own bundle and Ruby.** Each version runs its own `Gemfile.lock`,
|
|
29
|
+
on the Ruby that its own `.ruby-version` names, so a pull request that
|
|
30
|
+
upgrades a gem, Rails, or Ruby itself is compared too.
|
|
31
|
+
- **Its own database.** Each worker gets a database of its own, with its
|
|
32
|
+
version's schema and your fixtures or seeds loaded into it.
|
|
33
|
+
- **Requests, through the whole stack.** A benchmark sends a request
|
|
34
|
+
through your app's middleware, routing, controller, and views, without
|
|
35
|
+
a server, performs the jobs that the request enqueued, and rolls back
|
|
36
|
+
what it wrote. Only the request and its jobs are measured.
|
|
37
|
+
- **More than time.** Besides CPU time and allocations, each request is
|
|
38
|
+
judged on its queries (with
|
|
39
|
+
[duel-activerecord](https://github.com/sferik/duel-ruby/tree/main/plugins/activerecord)), the
|
|
40
|
+
views it rendered, and its cache reads, misses, and writes.
|
|
41
|
+
- **Boot time.** A benchmark of your app's boot guards how long a fresh
|
|
42
|
+
process takes to start.
|
|
43
|
+
|
|
44
|
+
## Install duel-rails
|
|
45
|
+
|
|
46
|
+
1. Add duel-rails to the `development` group of your `Gemfile`:
|
|
47
|
+
|
|
48
|
+
```ruby
|
|
49
|
+
# Gemfile
|
|
50
|
+
group :development do
|
|
51
|
+
gem "duel-rails"
|
|
52
|
+
end
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
2. Run `bundle install`.
|
|
56
|
+
|
|
57
|
+
3. Run the generator:
|
|
58
|
+
|
|
59
|
+
```sh
|
|
60
|
+
bin/rails generate duel:install
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
The generator makes the following files:
|
|
64
|
+
|
|
65
|
+
| File | What it is |
|
|
66
|
+
|---|---|
|
|
67
|
+
| `.duel.rb` | Duel's configuration: `c.use :rails`, with your database, and your fixtures if you have any. If your app has a `.ruby-version`, also `c.ruby = :version_file`. See [Its own Ruby](#its-own-ruby). |
|
|
68
|
+
| `config/environments/benchmark.rb` | The environment that each version boots in, based on production. |
|
|
69
|
+
| `duel/requests_duel.rb` | A benchmark of each of your app's first 10 routes that a GET reaches without parameters. |
|
|
70
|
+
| `duel/boot_duel.rb` | A benchmark of your app's boot. See [Benchmark your app's boot](#benchmark-your-apps-boot). |
|
|
71
|
+
| `.github/workflows/perf.yml` | A GitHub Actions workflow that runs Duel on pull requests, on pushes to `main`, and nightly against your last release, and comments on each pull request with Duel's report, updating one comment on every push. See [Report on a pull request](https://github.com/sferik/duel-ruby#report-on-a-pull-request). For an app in a subdirectory of its repository, the workflow is at the repository's root. See [An app in a monorepo](#an-app-in-a-monorepo). |
|
|
72
|
+
|
|
73
|
+
The generator also adds `gem "stackprof", platforms: :mri` to your
|
|
74
|
+
`Gemfile`'s `development` group, unless it has StackProf already, so
|
|
75
|
+
Duel can show where a regression spent its extra time. StackProf runs
|
|
76
|
+
on CRuby only, so the gem isn't installed on JRuby or TruffleRuby. If
|
|
77
|
+
your database is PostgreSQL or MySQL, the generator adds
|
|
78
|
+
[duel-postgresql](https://github.com/sferik/duel-ruby/tree/main/plugins/postgresql) or
|
|
79
|
+
[duel-mysql](https://github.com/sferik/duel-ruby/tree/main/plugins/mysql) too.
|
|
80
|
+
Run `bundle install` again.
|
|
81
|
+
|
|
82
|
+
4. Review the generated files, and commit them.
|
|
83
|
+
|
|
84
|
+
5. Run the benchmarks against your main branch:
|
|
85
|
+
|
|
86
|
+
```sh
|
|
87
|
+
bin/rake duel
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
To compare your code with itself, which checks that the setup works,
|
|
91
|
+
run `bin/rake duel DUEL_FORCE=1`.
|
|
92
|
+
|
|
93
|
+
The `rake duel` task comes with duel-rails. You can also run `bundle exec
|
|
94
|
+
duel`, with any of [Duel's options](https://github.com/sferik/duel-ruby#run-duel).
|
|
95
|
+
|
|
96
|
+
## Benchmark requests
|
|
97
|
+
|
|
98
|
+
Benchmarks go in files named `*_duel.rb` in `duel/`. To benchmark a
|
|
99
|
+
request, use `duel_request` with a name and the request's method and
|
|
100
|
+
path:
|
|
101
|
+
|
|
102
|
+
```ruby
|
|
103
|
+
# duel/requests_duel.rb
|
|
104
|
+
duel_request "posts: index", "GET /posts"
|
|
105
|
+
duel_request "posts: page 2", "GET /posts?page=2"
|
|
106
|
+
duel_request "posts: create", "POST /posts", params: {post: {title: "Hi"}}
|
|
107
|
+
duel_request "api: search", "POST /api/search", json: {q: "duel"}, headers: {"Accept" => "application/json"}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Each iteration sends the request once, through the whole Rack stack,
|
|
111
|
+
and then performs the jobs that the request enqueued. See
|
|
112
|
+
[Measure the jobs that a request enqueues](#measure-the-jobs-that-a-request-enqueues).
|
|
113
|
+
After each iteration, duel-rails rolls back what the request and its
|
|
114
|
+
jobs wrote to the database, so every request finds the same data.
|
|
115
|
+
|
|
116
|
+
duel-rails rolls back with an `around_iteration` hook,
|
|
117
|
+
`Duel::Rails::ROLLBACK`, so the rollback isn't measured. Before each
|
|
118
|
+
iteration, the hook pins each database connection that Active Record
|
|
119
|
+
writes through, in a transaction, and after the iteration it rolls the
|
|
120
|
+
transaction back, the way that Rails's transactional tests do. Duel
|
|
121
|
+
measures only the iteration: the request, its jobs, and the
|
|
122
|
+
transactions that they open, which become savepoints.
|
|
123
|
+
|
|
124
|
+
To leave the rollback out of what it measures, Duel pauses every metric
|
|
125
|
+
after each iteration, and resumes it before the next. A metric that
|
|
126
|
+
reads a server, such as duel-postgresql's statistics, is costly to read,
|
|
127
|
+
so Duel doesn't pause it: it reads the server only when a sample starts
|
|
128
|
+
and stops, and subtracts what pinning and rolling back add, which it
|
|
129
|
+
learns in each worker. Pausing the other metrics still takes a little
|
|
130
|
+
time per iteration. To measure a request without the rollback, and
|
|
131
|
+
without the pauses, pass `rollback: false`.
|
|
132
|
+
|
|
133
|
+
`duel_request` takes the following options:
|
|
134
|
+
|
|
135
|
+
| Option | Description |
|
|
136
|
+
|---|---|
|
|
137
|
+
| `params:` | Form parameters. For a GET, they go in the query string. |
|
|
138
|
+
| `json:` | A body, sent as JSON with a `Content-Type` of `application/json`. Use either `params:` or `json:`, not both. |
|
|
139
|
+
| `headers:` | Request headers, by name, for example `{"Accept" => "text/csv"}`. |
|
|
140
|
+
| `status:` | The status that the response must have: an Integer, or a Range. The default is `200..399`. Any other status fails the benchmark, because it's measuring an error page. |
|
|
141
|
+
| `rollback:` | `false` to keep what the request writes, and to run its iterations without pausing the metrics between them. The default is `true`. To start each sample with the same data anyway, see [Start every sample with the same data](#start-every-sample-with-the-same-data). |
|
|
142
|
+
| `only:`, `peak_memory:` | Duel's options for what the benchmark is judged on. See [Choose what a benchmark is judged on](https://github.com/sferik/duel-ruby/blob/main/docs/benchmarks.md#choose-what-a-benchmark-is-judged-on). |
|
|
143
|
+
| `setup:`, `teardown:`, `before_sample:`, `around_iteration:`, `after_iterations:` | The benchmark's own hooks, as `duel` takes them. See [Run code around a benchmark](https://github.com/sferik/duel-ruby/blob/main/docs/benchmarks.md#run-code-around-a-benchmark). An `around_iteration:` hook would replace the rollback, so it raises an `ArgumentError` unless you also pass `rollback: false`. To run a hook around the rollback, give it to the whole file instead. |
|
|
144
|
+
|
|
145
|
+
To change the request before it's sent, pass a block. The block
|
|
146
|
+
receives the request's Rack environment, and runs in the benchmark's
|
|
147
|
+
context, so it can use what the file's `setup` hooks set. For example,
|
|
148
|
+
to send a session cookie that a file's `setup` hook made, with a helper
|
|
149
|
+
of your own in `duel/duel_helper.rb`:
|
|
150
|
+
|
|
151
|
+
```ruby
|
|
152
|
+
setup { @cookie = session_cookie_for(User.find_by!(admin: true)) }
|
|
153
|
+
|
|
154
|
+
duel_request "admin: dashboard", "GET /admin" do |env|
|
|
155
|
+
env["HTTP_COOKIE"] = @cookie
|
|
156
|
+
end
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
`duel_request` is a `duel`, so the hooks of its file and of `.duel.rb`
|
|
160
|
+
apply to it, and you can mix it with plain benchmarks of your models and
|
|
161
|
+
services:
|
|
162
|
+
|
|
163
|
+
```ruby
|
|
164
|
+
duel "invoice: totals for 100 lines" do
|
|
165
|
+
once { @invoice = Invoice.includes(:lines).first }
|
|
166
|
+
@invoice.recalculate_totals
|
|
167
|
+
end
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
### Roll back any benchmark
|
|
171
|
+
|
|
172
|
+
A plain `duel` that writes to the database, such as one that creates a
|
|
173
|
+
record, can roll back each iteration the way that a request does. Give
|
|
174
|
+
`Duel::Rails::ROLLBACK` as an `around_iteration` hook, at any level:
|
|
175
|
+
|
|
176
|
+
```ruby
|
|
177
|
+
# For one benchmark
|
|
178
|
+
duel "posts: create", around_iteration: Duel::Rails::ROLLBACK do
|
|
179
|
+
Post.create!(title: "Hi")
|
|
180
|
+
end
|
|
181
|
+
|
|
182
|
+
# For every benchmark in a file, at its top
|
|
183
|
+
around_iteration(&Duel::Rails::ROLLBACK)
|
|
184
|
+
|
|
185
|
+
# For every benchmark, in .duel.rb
|
|
186
|
+
Duel.configure do |c|
|
|
187
|
+
c.use :rails, fixtures: :all
|
|
188
|
+
c.around_iteration(&Duel::Rails::ROLLBACK)
|
|
189
|
+
end
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
A `duel_request` already rolls back. Under a file's or `.duel.rb`'s
|
|
193
|
+
hook, its own rollback runs inside the outer one, which costs a little
|
|
194
|
+
more and changes nothing. A benchmark whose block takes the number of
|
|
195
|
+
iterations (`|n|`) can't run inside an `around_iteration` hook, so
|
|
196
|
+
don't give `.duel.rb` one if you have such benchmarks.
|
|
197
|
+
|
|
198
|
+
### Measure the jobs that a request enqueues
|
|
199
|
+
|
|
200
|
+
Some of a request's work runs later, in a job that it enqueues, such as
|
|
201
|
+
an email that it sends with `deliver_later`. So that a regression in
|
|
202
|
+
that work fails too, duel-rails performs the jobs that a benchmark
|
|
203
|
+
enqueues, and Duel measures them with the benchmark:
|
|
204
|
+
|
|
205
|
+
- In a request's benchmark, or any benchmark that's rolled back,
|
|
206
|
+
duel-rails performs the jobs after each iteration, before it rolls
|
|
207
|
+
back what the iteration wrote, so the jobs find its data.
|
|
208
|
+
- In any other benchmark, duel-rails performs the jobs after the
|
|
209
|
+
iterations of each sample.
|
|
210
|
+
|
|
211
|
+
duel-rails performs them in an `after_iterations` hook, which Duel
|
|
212
|
+
measures with the iterations. For more information, see
|
|
213
|
+
[Run code around a benchmark](https://github.com/sferik/duel-ruby/blob/main/docs/benchmarks.md#run-code-around-a-benchmark).
|
|
214
|
+
|
|
215
|
+
duel-rails performs the jobs in order, along with any jobs that they
|
|
216
|
+
enqueue, and a job that's scheduled for later too. It discards the jobs
|
|
217
|
+
that were enqueued before a sample, such as by your seeds.
|
|
218
|
+
|
|
219
|
+
While Duel benchmarks your app, duel-rails's own queue is Active Job's
|
|
220
|
+
queue adapter, in place of the one that
|
|
221
|
+
`config/environments/benchmark.rb` sets. A job class that sets its own
|
|
222
|
+
`queue_adapter` keeps that adapter, so duel-rails doesn't perform its
|
|
223
|
+
jobs.
|
|
224
|
+
|
|
225
|
+
duel-rails performs jobs on Rails 7.2, 8.0, and 8.1 alike. In each,
|
|
226
|
+
a job that's enqueued inside a transaction can wait for the transaction
|
|
227
|
+
to commit, as Active Job decides: Rails 7.2 asks the adapter, and
|
|
228
|
+
duel-rails's says to wait, as Active Job's own adapters do. Rails 8.0
|
|
229
|
+
and later ask the job, with its `enqueue_after_transaction_commit`. The
|
|
230
|
+
transaction that duel-rails rolls each request back in doesn't count:
|
|
231
|
+
a job that waits is enqueued when the request's own transaction
|
|
232
|
+
commits.
|
|
233
|
+
|
|
234
|
+
To leave jobs out, and measure only what the request does itself, pass
|
|
235
|
+
`jobs: false` to `c.use :rails`.
|
|
236
|
+
|
|
237
|
+
### Start every sample with the same data
|
|
238
|
+
|
|
239
|
+
A benchmark that writes to the database without rolling back, such as a
|
|
240
|
+
`duel_request` with `rollback: false` or a plain `duel` that creates
|
|
241
|
+
records, leaves its writes for the samples after it. To start every
|
|
242
|
+
sample with the data that each worker loaded, pass `reset: true`:
|
|
243
|
+
|
|
244
|
+
```ruby
|
|
245
|
+
# .duel.rb
|
|
246
|
+
Duel.configure do |c|
|
|
247
|
+
c.use :rails, fixtures: :all, reset: true
|
|
248
|
+
end
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
Before every sample of every benchmark, duel-rails then empties every
|
|
252
|
+
table, and loads your fixtures and seeds again. Duel doesn't measure
|
|
253
|
+
it, but it makes each run slower.
|
|
254
|
+
|
|
255
|
+
To reset only before the samples of one file's benchmarks, call
|
|
256
|
+
`Duel::Rails.reset_data` in the file's `before_sample` block:
|
|
257
|
+
|
|
258
|
+
```ruby
|
|
259
|
+
# duel/imports_duel.rb
|
|
260
|
+
before_sample { Duel::Rails.reset_data } # not measured
|
|
261
|
+
|
|
262
|
+
duel_request "imports: create", "POST /imports", params: {import: {rows: 100}}, rollback: false
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
### Benchmark your app's boot
|
|
266
|
+
|
|
267
|
+
The generator adds a benchmark of how long your app takes to boot:
|
|
268
|
+
|
|
269
|
+
```ruby
|
|
270
|
+
# duel/boot_duel.rb
|
|
271
|
+
duel_process "boot: the app", "bin/rails", "runner", "1"
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
Each sample runs `bin/rails runner` in a fresh process, for each
|
|
275
|
+
version, in turn. Duel judges it on its CPU time, including that of the
|
|
276
|
+
processes that it starts, and on its peak memory. For more information,
|
|
277
|
+
see Duel's [Benchmark a whole process](https://github.com/sferik/duel-ruby/blob/main/docs/benchmarks.md#benchmark-a-whole-process).
|
|
278
|
+
|
|
279
|
+
Each sample boots your app once for each version, so this benchmark
|
|
280
|
+
takes longer than a request's, from tens of seconds to a few minutes.
|
|
281
|
+
|
|
282
|
+
Each version's process runs with that version's bundle, in the
|
|
283
|
+
`benchmark` environment (`RAILS_ENV`). It doesn't get a database of its
|
|
284
|
+
own, so it measures your app's boot alone. If your app queries the
|
|
285
|
+
database as it boots, remove the benchmark.
|
|
286
|
+
|
|
287
|
+
## Each version's own
|
|
288
|
+
|
|
289
|
+
### The benchmark environment
|
|
290
|
+
|
|
291
|
+
Each version boots in the `benchmark` environment, from
|
|
292
|
+
`config/environments/benchmark.rb`. The generated file starts from your
|
|
293
|
+
production environment, and turns off what needs a deployment:
|
|
294
|
+
|
|
295
|
+
- SSL, host authorization, and forgery protection, because Duel sends
|
|
296
|
+
requests without a server in front.
|
|
297
|
+
- Logging.
|
|
298
|
+
- Other databases: the cache uses memory, and Active Job uses its test
|
|
299
|
+
adapter, so neither needs a database of its own.
|
|
300
|
+
|
|
301
|
+
Change it as your app needs, for example to stub a service that your
|
|
302
|
+
app calls over the network.
|
|
303
|
+
|
|
304
|
+
### Its own bundle
|
|
305
|
+
|
|
306
|
+
Each version runs its own `Gemfile` and `Gemfile.lock`. Before a
|
|
307
|
+
version's workers start, duel-rails installs its `Gemfile.lock`, on the
|
|
308
|
+
version's Ruby, once for each lockfile and Ruby, and keeps it installed
|
|
309
|
+
for later runs. So a pull request that upgrades a gem is measured with
|
|
310
|
+
the upgrade, and its baseline without it.
|
|
311
|
+
|
|
312
|
+
Duel loads itself, and every Duel plugin that the runner loaded,
|
|
313
|
+
duel-rails included, ahead of each version's bundle. That way, a
|
|
314
|
+
baseline from before you added Duel can still run, and both versions
|
|
315
|
+
are measured with the same Duel.
|
|
316
|
+
|
|
317
|
+
### Its own Ruby
|
|
318
|
+
|
|
319
|
+
If your app has a `.ruby-version`, the generated `.duel.rb` sets
|
|
320
|
+
`c.ruby = :version_file`. Each version then runs on the Ruby that its
|
|
321
|
+
own `.ruby-version` names, so a pull request that upgrades Ruby is
|
|
322
|
+
measured on the new Ruby, and its baseline on the old one. duel-rails
|
|
323
|
+
installs each version's bundle for its Ruby.
|
|
324
|
+
|
|
325
|
+
Each Ruby must be installed where Duel finds it: with rbenv, chruby,
|
|
326
|
+
mise, or asdf, or as the Ruby that runs Duel. For more information, see
|
|
327
|
+
Duel's [Run each version on its own Ruby](https://github.com/sferik/duel-ruby/blob/main/docs/configuration.md#run-each-version-on-its-own-ruby).
|
|
328
|
+
On GitHub Actions, the generated workflow installs your app's Ruby, and,
|
|
329
|
+
for a pull request, also the main branch's, so a pull request that
|
|
330
|
+
upgrades Ruby works there too.
|
|
331
|
+
|
|
332
|
+
To run both versions on the Ruby that runs Duel, remove
|
|
333
|
+
`c.ruby = :version_file`.
|
|
334
|
+
|
|
335
|
+
### Its own database
|
|
336
|
+
|
|
337
|
+
Each worker gets a database of its own, set in `DATABASE_URL`, which
|
|
338
|
+
Rails prefers to `config/database.yml`. Choose the database with
|
|
339
|
+
`database:`:
|
|
340
|
+
|
|
341
|
+
| `database:` | Each worker's database | Plugin |
|
|
342
|
+
|---|---|---|
|
|
343
|
+
| `:sqlite` (default) | A SQLite file in the worker's directory | None |
|
|
344
|
+
| `:postgresql` | A PostgreSQL server | [duel-postgresql](https://github.com/sferik/duel-ruby/tree/main/plugins/postgresql) |
|
|
345
|
+
| `:mysql2` or `:trilogy` | A MySQL server, through that adapter | [duel-mysql](https://github.com/sferik/duel-ruby/tree/main/plugins/mysql) |
|
|
346
|
+
| `nil` | None | None |
|
|
347
|
+
|
|
348
|
+
A database server's plugin also judges each request on what the server
|
|
349
|
+
did for it, by statement.
|
|
350
|
+
|
|
351
|
+
Each worker loads its version's schema into its database, from
|
|
352
|
+
`db/schema.rb` or `db/structure.sql`. So a pull request with a migration
|
|
353
|
+
is measured with its new schema, and its baseline with the old one.
|
|
354
|
+
|
|
355
|
+
### Its own data
|
|
356
|
+
|
|
357
|
+
Each worker then loads the data that you ask for. Its random numbers are
|
|
358
|
+
seeded first, and Faker's too if it's loaded, so both versions get the
|
|
359
|
+
same data:
|
|
360
|
+
|
|
361
|
+
```ruby
|
|
362
|
+
# .duel.rb
|
|
363
|
+
Duel.configure do |c|
|
|
364
|
+
c.use :rails, database: :postgresql, fixtures: :all, seeds: true
|
|
365
|
+
end
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
Load data that every benchmark shares here. For data that one benchmark
|
|
369
|
+
needs, use a `setup` hook in its file, or a `once` block in the
|
|
370
|
+
benchmark.
|
|
371
|
+
|
|
372
|
+
> [!IMPORTANT]
|
|
373
|
+
> Duel loads the benchmark files before each worker's data, so don't
|
|
374
|
+
> query your database at the top level of a benchmark file. Do it in a
|
|
375
|
+
> `setup` hook or a `once` block.
|
|
376
|
+
|
|
377
|
+
## What each request is judged on
|
|
378
|
+
|
|
379
|
+
Besides Duel's own metrics (CPU time, instructions, and allocations),
|
|
380
|
+
each request is judged on the following:
|
|
381
|
+
|
|
382
|
+
| Metric | What it is | A regression |
|
|
383
|
+
|---|---|---|
|
|
384
|
+
| `active_record_queries` and the rest | Queries, cached queries, transactions, and records. See [duel-activerecord](https://github.com/sferik/duel-ruby/tree/main/plugins/activerecord). | Any increase |
|
|
385
|
+
| `rails_views` | Templates, partials, and layouts rendered. Each of a collection's partials counts. | Any increase |
|
|
386
|
+
| `rails_cache_reads` | Keys read from the cache | Any increase |
|
|
387
|
+
| `rails_cache_misses` | Keys read that weren't in the cache | Any increase |
|
|
388
|
+
| `rails_cache_writes` | Keys written to the cache | Any increase |
|
|
389
|
+
|
|
390
|
+
A regression in views shows which views a sample rendered most, as in
|
|
391
|
+
the example at the top of this page.
|
|
392
|
+
|
|
393
|
+
The transaction that duel-rails rolls each request back in isn't
|
|
394
|
+
measured, so it doesn't count in `active_record_transactions`, and the
|
|
395
|
+
queries that a request's jobs make count as the request's.
|
|
396
|
+
|
|
397
|
+
## Options
|
|
398
|
+
|
|
399
|
+
`c.use :rails` takes the following options:
|
|
400
|
+
|
|
401
|
+
| Option | Description |
|
|
402
|
+
|---|---|
|
|
403
|
+
| `database:` | Each worker's database. See [Its own database](#its-own-database). The default is `:sqlite`. |
|
|
404
|
+
| `fixtures:` | Fixtures from `test/fixtures` to load into each worker's database, by name, or `:all`. The default is none. |
|
|
405
|
+
| `seeds:` | `true` to run `db/seeds.rb` in each worker, after the fixtures. |
|
|
406
|
+
| `seed:` | What random numbers are seeded with. The default is 42. |
|
|
407
|
+
| `reset:` | `true` to empty every table, and load the fixtures and seeds again, before every sample. See [Start every sample with the same data](#start-every-sample-with-the-same-data). |
|
|
408
|
+
| `jobs:` | `false` to leave out the jobs that benchmarks enqueue. See [Measure the jobs that a request enqueues](#measure-the-jobs-that-a-request-enqueues). |
|
|
409
|
+
| `environment:` | The environment that each version boots in, and its processes run in. The default is `"benchmark"`. |
|
|
410
|
+
| `bundle:` | `false` to run every version with the runner's bundle. |
|
|
411
|
+
| `active_record:` | `false` to leave out Active Record's metrics, for an app without Active Record. |
|
|
412
|
+
| `query_time:` | `true` to also judge how long queries took. See [duel-activerecord](https://github.com/sferik/duel-ruby/tree/main/plugins/activerecord). |
|
|
413
|
+
|
|
414
|
+
Any other option goes to the database server's plugin, for example
|
|
415
|
+
`args:` or `setup:` for duel-mysql.
|
|
416
|
+
|
|
417
|
+
duel-rails uses duel-activerecord, and the database server's plugin,
|
|
418
|
+
with `c.use`. To give one of them options of its own, use it yourself,
|
|
419
|
+
before `c.use :rails`. duel-rails then uses the plugin that you
|
|
420
|
+
configured:
|
|
421
|
+
|
|
422
|
+
```ruby
|
|
423
|
+
# .duel.rb
|
|
424
|
+
Duel.configure do |c|
|
|
425
|
+
c.use :activerecord, ignore: ->(payload) { payload[:sql].include?("pg_catalog") }
|
|
426
|
+
c.use :rails, database: :postgresql, fixtures: :all
|
|
427
|
+
end
|
|
428
|
+
```
|
|
429
|
+
|
|
430
|
+
duel-rails also tells Duel to compare a change to `app/`, `config/`,
|
|
431
|
+
`db/`, `lib/`, `Gemfile`, `Gemfile.lock`, or `.ruby-version`.
|
|
432
|
+
|
|
433
|
+
## An app in a monorepo
|
|
434
|
+
|
|
435
|
+
Your app can be in a subdirectory of its repository, such as
|
|
436
|
+
`apps/blog`. Run the generator and `bin/rake duel` in the app's
|
|
437
|
+
directory, as usual. Duel checks out the whole repository for the
|
|
438
|
+
baseline, and compares the same subdirectory of it. The paths that
|
|
439
|
+
duel-rails watches are in the app's directory.
|
|
440
|
+
|
|
441
|
+
The generator writes the workflow at the repository's root, because
|
|
442
|
+
that's where GitHub Actions looks for it, and names it for the app's
|
|
443
|
+
directory, such as `.github/workflows/perf-apps-blog.yml`, so that
|
|
444
|
+
each app can have its own. The workflow runs Duel, and sets up Ruby
|
|
445
|
+
and the bundle, in the app's directory (`working-directory`).
|
|
446
|
+
|
|
447
|
+
## YJIT
|
|
448
|
+
|
|
449
|
+
Rails 7.2 and later turn YJIT on as the app boots. When YJIT is on,
|
|
450
|
+
Duel warms each benchmark up for two seconds before it measures it, so
|
|
451
|
+
that YJIT has compiled your app's code first.
|
|
452
|
+
|
|
453
|
+
## Requirements
|
|
454
|
+
|
|
455
|
+
duel-rails needs Rails 7.2 or later, and Ruby 3.4 or later. It doesn't
|
|
456
|
+
run on JRuby, whose Rails needs JDBC adapters.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "bundler"
|
|
4
|
+
require "digest"
|
|
5
|
+
|
|
6
|
+
module Duel
|
|
7
|
+
module Rails
|
|
8
|
+
# Each version runs its own bundle: its Gemfile and its Gemfile.lock,
|
|
9
|
+
# installed once for the run (Config#prepare(:version)) on the version's
|
|
10
|
+
# Ruby, and kept installed for later runs (Resource#cached), so a
|
|
11
|
+
# change that upgrades a gem (Rails itself, say) is compared too.
|
|
12
|
+
#
|
|
13
|
+
# (Duel puts its own gems, and its plugins', this one too, ahead of
|
|
14
|
+
# the version's bundle in each worker: a version from before the app
|
|
15
|
+
# added Duel still has them, and both versions are measured with the
|
|
16
|
+
# same.)
|
|
17
|
+
module Bundle
|
|
18
|
+
# Gives each version its own bundle (see .environment).
|
|
19
|
+
def self.configure(config) = config.prepare(:version) { |v| environment(v) }
|
|
20
|
+
|
|
21
|
+
# The environment for the bundle of the version (a Resource for it),
|
|
22
|
+
# installed first, on its Ruby: nothing for a version without a
|
|
23
|
+
# Gemfile.lock.
|
|
24
|
+
def self.environment(version)
|
|
25
|
+
gemfile = File.join(version.root, "Gemfile")
|
|
26
|
+
lockfile = "#{gemfile}.lock"
|
|
27
|
+
return {} unless File.exist?(lockfile)
|
|
28
|
+
|
|
29
|
+
version.cached(key(lockfile, version.ruby)) do
|
|
30
|
+
version.run(unbundled.merge(version.env, "BUNDLE_GEMFILE" => gemfile), "bundle", "install", log: "bundle",
|
|
31
|
+
chdir: version.root, unsetenv_others: true)
|
|
32
|
+
end
|
|
33
|
+
{"BUNDLE_GEMFILE" => gemfile, "BUNDLE_LOCKFILE" => lockfile}
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# What a bundle installed is kept as: its lockfile's, for the Ruby
|
|
37
|
+
# (its path) it's installed for, which has gems of its own.
|
|
38
|
+
def self.key(lockfile, ruby) = "bundle-#{Digest::SHA256.hexdigest("#{File.read(lockfile)}\0#{ruby}")[0, 16]}"
|
|
39
|
+
|
|
40
|
+
# The runner's environment without Bundler's part (the bundle it
|
|
41
|
+
# runs in is the candidate's), for installing another.
|
|
42
|
+
def self.unbundled = Bundler.unbundled_env
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|