rails-minify 0.1.3 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 44f089252dbde289b9e5cfeb25c2613e91544e92a1235369ec7c4457992e477b
4
- data.tar.gz: c768e54a78cd0ee553f8732f019dc4eab9fff3c5c72d4ef9d30d23abc68b2994
3
+ metadata.gz: c682f7b5f509613697d35d78e163167b825525a03aae99fecb6ab430550f2429
4
+ data.tar.gz: dd258711680a2472583815a96af871f418d8fac5cf06f323c2dc7f102908e962
5
5
  SHA512:
6
- metadata.gz: da6b92c40d4f1fada0154fb362cd52c8a56e458b67e5d959e20084bb4e37f25a8be93d7d86bc065d6a71ac09dc4763c79fbada65a4f7a73c281c8b9fef511d1b
7
- data.tar.gz: 8cca5d4c9850e66798cdb80127907dbfc64cdf5643d1064a7ce01be6bfdd8f46b79bbf63e9e8c80775dd6c3453d9877537bd6951707cb434378049fdb0a92227
6
+ metadata.gz: 29a12e198162b3bd148f39d2f7be79e69aa32d471164120d1055a8c2960a97f7afee2a1dd2728b30dfe6e8f2d8f22b528a854420600f3ae1d3e325077265ce60
7
+ data.tar.gz: c2031ba5b96903be93f11d324afe736b18c476f420467e1a48503b4ac397ee24540812e7173819aa1ff6fd97e5025c7ac865616f7ad15d0db0498de29f2144eb
data/CHANGELOG.md CHANGED
@@ -1,5 +1,21 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.3.0 (2026-11-04)
4
+
5
+ - `config.rails_minify.sql_profile = 0.01` samples requests and logs each distinct query and call site with its calls, time and rows, plus the request's SQL and instantiation time; `bin/rails minify:sql:report LOG=...` ranks actions and queries by total time, and with `EXPLAIN=1` adds SQLite query plans and flags full table scans and sorts without an index.
6
+ - `config.rails_minify.async_queries = 0.01` samples requests for relations the controller leaves unloaded and the view then loads one after another, and logs one line per request; `bin/rails minify:async:report LOG=...` ranks actions by the estimated saving from starting those relations with `load_async`. `minify:doctor` also reports `load_async` without `config.active_record.async_query_executor`.
7
+ - A/B in production, under the same traffic: `rails-minify-ab COMMAND` runs the server as groups A and B on one port (`SO_REUSEPORT`) with half the workers each when `RAILS_MINIFY_AB` is set; `config.rails_minify.ab` changes group B's configuration; both groups' log lines are tagged; `minify:ab:report` compares the groups from a request log (with B−A and a 95% interval) and `minify:ab:memory` reports their memory from `/proc`.
8
+ - `bin/rails minify:doctor` reports production configuration that costs speed or memory, and never fails: a JIT turned on before boot (or JIT options without `--zjit-disable`/`--yjit-disable`), Falcon without `preload` or Puma with `preload_app! false`, a Falcon preload script without `Process.warmup` on async-service 0.25 and earlier, `WEB_CONCURRENCY` that `falcon.rb` doesn't read, and Sentry breadcrumbs on every notification. It reads the environment from `config/deploy.yml`, the `Dockerfile` and `bin/docker-entrypoint` too.
9
+
10
+ ## 0.2.0 (2026-10-20)
11
+
12
+ Narrowed to what held up in production (Japan Voyage, each part on half the workers under the same traffic; README, "What paid off in production").
13
+
14
+ - `config.rails_minify.enabled` turns on only strict mode (`ar_minimizer.slim`), enums and ZJIT after boot. ZJIT only in production and from Ruby 4.1 (4.0's ZJIT isn't meant for production); set `config.rails_minify.zjit` yourself otherwise.
15
+ - No longer turned on by `enabled`: `ar_minimizer.views`, `.load_time`, `.small_records`, `as_minimizer.unroll_callbacks`, `ap_minimizer.route_helpers`, `.views`, and the patches. Each still works when set on its own and is documented as experimental. Patches: `config.rails_minify.patches = true` for all, or a list.
16
+ - actionpack-minimizer is no longer a dependency; rails-minify loads it, and `minify:census` covers it, when it's in the Gemfile.
17
+ - Requires activerecord-minimizer and activesupport-minimizer 0.2.
18
+
3
19
  ## 0.1.3 (2026-10-18)
4
20
 
5
21
  - `config.rails_minify.zjit = true` turns ZJIT on after boot, as `config.yjit` does for YJIT, instead of `RUBY_ZJIT_ENABLE=1` compiling boot-only code too. Japan Voyage, per process: 24% fewer compiled methods, 18 MB less JIT memory, about 25 MB less RSS, the same speed.
data/README.md CHANGED
@@ -1,6 +1,7 @@
1
1
  # rails-minify
2
2
 
3
- Cuts what Rails (and common gems) build or redo on every request.
3
+ Fewer generated Active Record methods, ZJIT turned on after boot, and checks
4
+ for callback problems.
4
5
 
5
6
  ```ruby
6
7
  # Gemfile
@@ -10,14 +11,19 @@ gem "rails-minify"
10
11
  config.rails_minify.enabled = true
11
12
  ```
12
13
 
13
- `enabled` turns on every part below; each can still be set on its own
14
- (`config.ar_minimizer.enums = false`, `config.rails_minify.patches = [:omniauth]`).
14
+ `enabled` turns on what held up in production on a real app:
15
+
16
+ * strict mode (`config.ar_minimizer.slim`): only readers and writers per column;
17
+ * enum predicates and bang methods as plain methods (`config.ar_minimizer.enums`);
18
+ * ZJIT after boot (`config.rails_minify.zjit`), in production on Ruby 4.1+.
19
+
20
+ Each can still be set on its own (`config.ar_minimizer.enums = false`).
21
+ Everything else is [experimental](#experimental) and off unless you set it.
15
22
  Ruby 3.4+, Rails 8.0, 8.1 and main.
16
23
 
17
24
  Turn it on in tests too, so a call to a method strict mode removed fails in CI
18
25
  rather than in production. In development, strict mode alone is enough: the
19
- same errors show up while you work, and the compiling parts, which don't
20
- follow code reloads, stay off.
26
+ same errors show up while you work.
21
27
 
22
28
  ```ruby
23
29
  # config/environments/test.rb
@@ -27,7 +33,7 @@ config.rails_minify.enabled = true
27
33
  config.ar_minimizer.slim = true # strict mode, without the rest
28
34
  ```
29
35
 
30
- ## Strict by default
36
+ ## Strict mode
31
37
 
32
38
  Active Record defines, for every column, a reader, a writer and 19 more
33
39
  methods (`name?`, `name_changed?`, `saved_change_to_name?`, ...). With
@@ -46,50 +52,17 @@ rubocop-minify rewrites those calls in your code. For calls in gems
46
52
  `config.ar_minimizer.strict = false` keeps the removed methods working
47
53
  instead, generated the first time each is called.
48
54
 
49
- ## What's in it
50
-
51
- | part | does | gem |
52
- |---|---|---|
53
- | attribute methods | readers and writers only (strict) | activerecord-minimizer |
54
- | enums | predicates/bang methods as plain methods (1.3-1.6x) | activerecord-minimizer |
55
- | small records | no Hashes for records that never fill them (-22% per record) | activerecord-minimizer |
56
- | callbacks | chains as generated methods (`valid?` -33..45%) | activesupport-minimizer |
57
- | route helpers | `post_path(post)` as interpolation (3-14x) | actionpack-minimizer |
58
- | views | `render`/`link_to` without their machinery (2x per page of partials) | actionpack-minimizer |
59
- | patches | below | rails-minify |
60
- | rewrites | removed-method calls in Ruby and ERB, as RuboCop autocorrections | rubocop-minify (development) |
61
-
62
- ## Patches
63
-
64
- For parts of Rails and common gems that redo work on every request.
65
- `config.rails_minify.patches = [...]` picks them; all by default.
66
-
67
- * `:action_text_plain_text`: `ActionText::Content#to_plain_text` parses the
68
- HTML with Nokogiri, and looks up every attachment's record, on every call.
69
- It's computed once per distinct HTML (cache capped at 8 MB of HTML).
70
- Attachments that are Active Storage blobs, remote images or inline content
71
- are cached too: a blob without a caption contributes its filename, which a
72
- hit checks in one query (renamed or deleted: computed again). Other
73
- attachables aren't cached. In [Japan Voyage](https://japanvoyage.jp), 13% of all allocations on its
74
- pages came from it, and guides with images another 4.5%.
75
- * `:request_host`: `ActionDispatch::Request#host` runs a regexp substitution
76
- on every call, port or not, and Rails reads it several times per request.
77
- rails/rails#59104 (Rails 8.2) for 8.1: no substitution without a port, and
78
- the host kept while the raw host is the same String. Not installed on 8.2.
79
- * `:omniauth`: every OmniAuth strategy middleware dups itself and its
80
- options on every request before looking at the path. Requests that can't
81
- be OmniAuth's go straight to the app. Only when that's certain up front:
82
- a session is present, not in test mode, no `other_phase` or callable
83
- paths, and the strategy doesn't override how paths are matched. Not
84
- installed when OmniAuth already does this itself (it then has a public
85
- `Strategy#passthrough?`, as in the version proposed upstream).
86
-
87
55
  ## ZJIT after boot
88
56
 
89
57
  ```ruby
90
58
  config.rails_minify.zjit = true # instead of RUBY_ZJIT_ENABLE=1 / --zjit
91
59
  ```
92
60
 
61
+ `enabled` sets it in production on Ruby 4.1 and later; on 4.0, ZJIT isn't
62
+ meant for production yet, so set it yourself if you want it there. ZJIT
63
+ options still go in `RUBYOPT`, after `--zjit-disable` so they don't turn it
64
+ on at startup: `RUBYOPT="--zjit-disable --zjit-call-threshold=2"`.
65
+
93
66
  `RUBY_ZJIT_ENABLE=1` turns ZJIT on before anything loads, so it also compiles
94
67
  what runs only while booting: RubyGems, requiring and eager loading, and the
95
68
  minimizers' own compiling. Rails turns YJIT on after boot (`config.yjit`) for
@@ -100,16 +73,33 @@ On Japan Voyage, per process after the same 132 pages: 12,612 → 9,557
100
73
  compiled methods, JIT code 45.9 → 35.3 MB, ZJIT memory 82.9 → 64.6 MB,
101
74
  RSS about 25 MB less, the same CPU time per page.
102
75
 
103
- ## Numbers
104
-
105
- Japan Voyage, 132 production pages, with everything on (strict mode after
106
- rubocop-minify's rewrites, all patches): identical HTML, and
107
- 1,561,283 → 1,224,141 allocations per pass (−21.6%). The machine was too
108
- loaded for wall-clock times to mean much.
109
-
110
76
  ## Checks
111
77
 
112
- * `bin/rails minify:census`: how much of the app runs compiled, and why the rest doesn't.
78
+ * `bin/rails minify:doctor`: production configuration that costs speed or
79
+ memory. It only reports, and never fails. Each finding says where it was
80
+ found, why it matters (with the number from Japan Voyage) and the fix:
81
+ * a JIT turned on before boot (`RUBY_ZJIT_ENABLE`, `RUBY_YJIT_ENABLE`,
82
+ `--zjit`/`--yjit` in `RUBYOPT`), or ZJIT/YJIT options without
83
+ `--zjit-disable`/`--yjit-disable`, which turn it on at startup too;
84
+ * Falcon with more than one worker and no `preload`, or a
85
+ `config/preload.rb` it doesn't load; Puma with `workers` and
86
+ `preload_app! false`;
87
+ * a Falcon preload script without `Process.warmup`, while async-service
88
+ (0.25 and earlier) warms the heap up only after forking the workers
89
+ (socketry/async-service#13);
90
+ * `WEB_CONCURRENCY` set while `falcon.rb` sets no `count`: Falcon starts
91
+ one worker per CPU;
92
+ * Sentry's `breadcrumbs_logger` with `:active_support_logger`;
93
+ * `load_async` in `app/` without `config.active_record.async_query_executor`:
94
+ the query then runs in the foreground, as if `load_async` weren't there.
95
+
96
+ Doctor usually runs on your machine rather than the server, so it reads the
97
+ environment from `config/deploy.yml` (Kamal's `env.clear`), the
98
+ `Dockerfile`'s `ENV` lines and `bin/docker-entrypoint`'s `export`s as well
99
+ as its own. Run on Japan Voyage's history, it finds the three problems
100
+ fixed there: ZJIT turned on in the entrypoint, the preload file Falcon
101
+ didn't load (and `WEB_CONCURRENCY` it didn't read), and the heap warmed up
102
+ after the fork.
113
103
  * `bin/rails minify:check` (for CI; works whether or not the rest of
114
104
  rails-minify is on) fails on:
115
105
  * callbacks naming methods that don't exist;
@@ -121,7 +111,8 @@ loaded for wall-clock times to mean much.
121
111
  reported once, with the classes that inherit it;
122
112
  * model callbacks that call `save`, `update`, `touch`, ... on the record
123
113
  they run for, which runs the chain again;
124
- * an out-of-date manifest.
114
+ * an out-of-date manifest, when there is one (only the experimental
115
+ compiling parts use it).
125
116
 
126
117
  ```ruby
127
118
  config.rails_minify.callback_limit = 8
@@ -131,7 +122,224 @@ loaded for wall-clock times to mean much.
131
122
  config.rails_minify.allow_callback_loops = ["Trip#recalculate"] # guarded on purpose
132
123
  ```
133
124
 
134
- ## Things found that are app config, not patches
125
+ ## Where Active Record time goes
126
+
127
+ ```ruby
128
+ # config/environments/production.rb
129
+ config.rails_minify.sql_profile = 0.01 # 1% of requests; true for all
130
+ ```
131
+
132
+ For each sampled request, one log line per distinct query and call site
133
+ (calls, time, rows, the first line in the app that ran it, the SQL with its
134
+ values taken out) and one with the request's totals, SQL time against the
135
+ time spent turning rows into records. `bin/rails minify:sql:report
136
+ LOG=production.log` ranks actions and queries by total time; with
137
+ `EXPLAIN=1` it loads the app and adds each SELECT's SQLite query plan,
138
+ flagging full table scans and sorts without an index:
139
+
140
+ ```
141
+ 5. 5.2% of SQL time TravelDestination Pluck 2.30 ms per call, 1.0 calls per request, 7.2 rows per call
142
+ SELECT "travel_destinations"."id", ... WHERE "travel_destinations"."published" = TRUE AND "travel_destinations"."latitude" BETWEEN ? AND ? ...
143
+ at app/models/concerns/nearby_venues.rb:39 (SkiResortsController#show 12x, GolfCoursesController#show 12x)
144
+ plan: SEARCH travel_destinations USING INDEX index_travel_destinations_on_published (published=?)
145
+ ```
146
+
147
+ Many calls per request from one line is an N+1; a query on every page
148
+ (a layout or footer) is worth caching. A sort without an index only matters
149
+ when the query is slow; on a few rows it isn't. Run EXPLAIN against a
150
+ database with production's schema and, ideally, its data: plans depend on
151
+ both.
152
+
153
+ ## Queries load_async could start
154
+
155
+ A page's queries usually run one after another, each where the view first
156
+ needs it. But a relation the controller leaves unloaded in an instance
157
+ variable already has its SQL fixed when rendering starts, so all of them
158
+ could start at once with `load_async`, in the background, while the view
159
+ renders. This finds them in production traffic:
160
+
161
+ ```ruby
162
+ # config/environments/production.rb
163
+ config.rails_minify.async_queries = 0.01 # 1% of requests; true for all
164
+ ```
165
+
166
+ For each sampled request with such relations, one log line records which
167
+ the view loaded, how long each main query took and how far into rendering
168
+ it was first needed:
169
+
170
+ ```
171
+ [rails-minify] load_async SpotsController#index sql=28.0 @spots=Spot:11.0@2.0 @areas=Area:4.0@14.0 @news=Article:-
172
+ ```
173
+
174
+ `bin/rails minify:async:report LOG=production.log` adds them up per action,
175
+ with the estimated saving:
176
+
177
+ ```
178
+ SpotsController#index n 2 saving mean 8.0 ms, p90 10.0 ms (foreground SQL mean 26.5 ms)
179
+ @spots Spot loaded in 100% query 10.0 ms first used 1.5 ms into rendering
180
+ @areas Area loaded in 100% query 3.5 ms first used 13.0 ms into rendering
181
+ @news Article loaded in 50% query 6.0 ms first used 20.0 ms into rendering
182
+ ```
183
+
184
+ The fix is `@spots = Spot.where(...).load_async` in the controller. The
185
+ estimate assumes the database answers as fast in parallel; check the change
186
+ with the [A/B](#ab-in-production) before keeping it. `load_async` needs
187
+ `config.active_record.async_query_executor = :global_thread_pool` and a
188
+ database pool big enough for it (`minify:doctor` reports a missing
189
+ executor). Not counted: preloads (`includes`), which still run in the
190
+ foreground when the relation is used; relations the view only counts
191
+ (`size`, `any?`); and association proxies (`@user.posts`). A relation loaded
192
+ in only some requests costs a query nobody reads in the others.
193
+
194
+ Not for SQLite: the sqlite3 gem (2.10) holds the GVL while a query runs,
195
+ so a query in the background stops the view rendering in the foreground and
196
+ nothing overlaps (a 415 ms query and 136 ms of Ruby: 526 ms in two threads,
197
+ 551 ms one after the other). It needs a database the process waits on over
198
+ a socket (PostgreSQL, MySQL).
199
+
200
+ No production result yet.
201
+
202
+ ## What paid off in production
203
+
204
+ Each gem's README has benchmark numbers. In production, on
205
+ [Japan Voyage](https://japanvoyage.jp) (Falcon, 6 workers, ZJIT, SQLite),
206
+ each part ran for an hour on half the workers while the other half ran
207
+ without it, under the same traffic (about 5,000 requests per half):
208
+
209
+ | part | in production | in 0.2.0 |
210
+ |---|---|---|
211
+ | ZJIT after boot | HTML p50/p90/p99 75/131/231 → 70/126/208 ms, ~25 MB less per worker (an hour before against an hour after) | on with `enabled` (production, Ruby 4.1+) |
212
+ | enums | the same medians; that hour's lower mean and p99 came from a few slow requests, as it changes no allocations on these pages | on with `enabled`: harmless, no measured gain |
213
+ | strict | the same latency and memory; no calls to a removed method | on with `enabled`, so production runs what the tests run |
214
+ | small records | no difference (tens of records per request) | experimental |
215
+ | route helpers | no difference in view time | experimental |
216
+ | views (`render`/`link_to`) | no difference in view time | experimental |
217
+ | compiled call sites in views | ~22 MB more per worker, no faster | experimental |
218
+ | compiled call sites at load time | ~12% more CPU per request (not reproduced locally) | experimental |
219
+ | callbacks | no faster, ~20 MB more per worker | experimental |
220
+ | patches | no difference (view time 25.3 vs 26.8 ms) | experimental |
221
+
222
+ ZJIT after boot is the one part with a clear effect. What `enabled` turns on
223
+ is what Japan Voyage runs.
224
+
225
+ What the benchmarks speed up is per call: a method call, a Hash, a helper.
226
+ A page there spends ~30 ms in Active Record and ~30 ms in views, and those
227
+ savings were smaller than the difference between two halves of the same
228
+ workers. Parts that generate or compile code in each worker (callbacks,
229
+ compiled views) cost memory in every worker; most of these ran before the
230
+ app was preloaded, and would cost less with preloading.
231
+
232
+ Besides ZJIT after boot, what sped Japan Voyage up was outside this gem: a
233
+ preloading server warming the heap up before it forks, and one query fixed.
234
+ Together, HTML p50/p90/p99 went from 75/131/231 to 45/66/~105 ms.
235
+
236
+ ## A/B in production
237
+
238
+ How the numbers above were measured, packaged: the server runs twice on one
239
+ port, half the workers each, as groups A and B. The kernel spreads
240
+ connections over both (`SO_REUSEPORT`), so the groups see the same traffic,
241
+ and group B runs with one change. Comparing an hour before with an hour after
242
+ mixes the change up with the traffic; this doesn't.
243
+
244
+ ```ruby
245
+ # config/environments/production.rb: group B's change (any configuration)
246
+ config.rails_minify.ab = ->(config) { config.ar_minimizer.small_records = true }
247
+ ```
248
+
249
+ ```ruby
250
+ # config/falcon.rb: both groups bind the port, and count follows WEB_CONCURRENCY
251
+ service ["myapp", ENV["RAILS_MINIFY_GROUP"]].compact.join("-") do
252
+ include Falcon::Environment::Rack
253
+ preload "config/preload.rb"
254
+ count ENV.fetch("WEB_CONCURRENCY", Etc.nprocessors).to_i
255
+ endpoint do
256
+ Async::HTTP::Endpoint.parse("http://0.0.0.0:3000")
257
+ .with(protocol: Async::HTTP::Protocol::HTTP11, reuse_port: true)
258
+ end
259
+ end
260
+ ```
261
+
262
+ ```dockerfile
263
+ CMD ["bundle", "exec", "rails-minify-ab", "falcon", "host", "config/falcon.rb"]
264
+ ```
265
+
266
+ * `rails-minify-ab COMMAND` runs the command as it is. With
267
+ `RAILS_MINIFY_AB=1` it runs it twice instead, with `RAILS_MINIFY_GROUP=A`
268
+ and `=B` and `WEB_CONCURRENCY` split in half. It stops both on TERM/INT,
269
+ exiting 0 for a clean stop, and stops the other when one exits.
270
+ * Each group's log lines get a tag, after the request id (which is added if
271
+ `config.log_tags` doesn't have it): `[<request id>] [minify:B] ...`.
272
+ * `bin/rails minify:ab:report LOG=file` (or `LOG=-` for stdin) compares the
273
+ groups: requests and 5xx, latency p50/p90/p99 and mean, view, Active Record
274
+ and GC time, for all requests, HTML and the busiest actions, with B−A and a
275
+ 95% interval. For example,
276
+ `bin/kamal app logs --since 60m --skip-timestamps | bin/rails minify:ab:report LOG=-`.
277
+ * `bin/rails minify:ab:memory`, where the server runs
278
+ (`bin/kamal app exec --reuse 'bin/rails minify:ab:memory'`): RSS, PSS and
279
+ private memory per process and PSS per group, from `/proc`.
280
+ * `RailsMinify::AB.group` is the process's group (nil outside an A/B run),
281
+ for changes that aren't configuration.
282
+
283
+ Tried with Falcon only so far; Puma would need each cluster to bind with
284
+ `SO_REUSEPORT`. An hour shows a large difference; run another before believing a small
285
+ one.
286
+
287
+ ## Experimental
288
+
289
+ Each was faster in benchmarks and made no measurable difference in
290
+ production (above). They still work, each turned on by its own setting:
291
+
292
+ | part | setting | gem |
293
+ |---|---|---|
294
+ | small records: no Hashes for records that never fill them (-22% per record) | `config.ar_minimizer.small_records` | activerecord-minimizer |
295
+ | call sites compiled to what removed methods stood for | `config.ar_minimizer.views`, `.load_time` | activerecord-minimizer |
296
+ | callback chains as generated methods (`valid?` -33..45%) | `config.as_minimizer.unroll_callbacks` | activesupport-minimizer |
297
+ | route helpers as interpolation (3-14x) | `config.ap_minimizer.route_helpers` | actionpack-minimizer |
298
+ | `render`/`link_to` without their machinery (2x per page of partials) | `config.ap_minimizer.views` | actionpack-minimizer |
299
+ | patches | `config.rails_minify.patches` | rails-minify |
300
+
301
+ actionpack-minimizer is no longer a dependency: add `gem "actionpack-minimizer"`
302
+ to use it. `bin/rails minify:census` shows how much of the app the compiling
303
+ parts cover, and why the rest isn't (actionpack-minimizer's part when it's
304
+ installed).
305
+
306
+ ### Patches
307
+
308
+ For parts of Rails and common gems that redo work on every request. Off
309
+ unless `config.rails_minify.patches` is set: `true` for all of them, or a
310
+ list (`[:omniauth]`). In production they made no difference in view time;
311
+ each is on its way upstream.
312
+
313
+ * `:action_text_plain_text`: `ActionText::Content#to_plain_text` parses the
314
+ HTML with Nokogiri, and looks up every attachment's record, on every call.
315
+ It's computed once per distinct HTML (cache capped at 8 MB of HTML).
316
+ Attachments that are Active Storage blobs, remote images or inline content
317
+ are cached too: a blob without a caption contributes its filename, which a
318
+ hit checks in one query (renamed or deleted: computed again). Other
319
+ attachables aren't cached. In [Japan Voyage](https://japanvoyage.jp), 13% of all allocations on its
320
+ pages came from it, and guides with images another 4.5%.
321
+ * `:request_host`: `ActionDispatch::Request#host` runs a regexp substitution
322
+ on every call, port or not, and Rails reads it several times per request.
323
+ rails/rails#59104 (Rails 8.2) for 8.1: no substitution without a port, and
324
+ the host kept while the raw host is the same String. Not installed on 8.2.
325
+ * `:omniauth`: every OmniAuth strategy middleware dups itself and its
326
+ options on every request before looking at the path. Requests that can't
327
+ be OmniAuth's go straight to the app. Only when that's certain up front:
328
+ a session is present, not in test mode, no `other_phase` or callable
329
+ paths, and the strategy doesn't override how paths are matched. Not
330
+ installed when OmniAuth already does this itself (it then has a public
331
+ `Strategy#passthrough?`, as in the version proposed upstream).
332
+
333
+ ### Allocations
334
+
335
+ Japan Voyage, 132 production pages, with everything on (strict mode after
336
+ rubocop-minify's rewrites, all patches): identical HTML, and
337
+ 1,561,283 → 1,224,141 allocations per pass (−21.6%). The machine was too
338
+ loaded for wall-clock times to mean much.
339
+
340
+ ## Things found that are app config
341
+
342
+ `minify:doctor` checks for these.
135
343
 
136
344
  * sentry-rails' `breadcrumbs_logger = [:active_support_logger]` subscribes to
137
345
  every notification (each SQL query, render, instantiation) and takes
@@ -0,0 +1,56 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ # rails-minify-ab COMMAND...
5
+ #
6
+ # Runs COMMAND (the server, e.g. `bundle exec falcon host config/falcon.rb`).
7
+ # With RAILS_MINIFY_AB set, runs it twice instead, as groups A and B:
8
+ # RAILS_MINIFY_GROUP=A and =B, WEB_CONCURRENCY split in half. Both bind the
9
+ # same port, so the server has to set SO_REUSEPORT (Falcon: an endpoint
10
+ # `.with(reuse_port: true)`); the kernel then spreads connections over both,
11
+ # and the groups see the same traffic. Stops both when told to (TERM, INT)
12
+ # or when either exits.
13
+
14
+ require "etc"
15
+
16
+ abort "usage: rails-minify-ab COMMAND..." if ARGV.empty?
17
+ exec(*ARGV) if ENV["RAILS_MINIFY_AB"].to_s.empty?
18
+
19
+ workers = Integer(ENV.fetch("WEB_CONCURRENCY", Etc.nprocessors))
20
+ half = workers / 2
21
+ abort "rails-minify-ab: WEB_CONCURRENCY=#{workers}, needs at least 2 for two groups" if half.zero?
22
+
23
+ pids = { "A" => half, "B" => workers - half }.to_h do |group, count|
24
+ [Process.spawn({ "RAILS_MINIFY_GROUP" => group, "WEB_CONCURRENCY" => count.to_s }, *ARGV), group]
25
+ end
26
+
27
+ stopping = false
28
+ %w[TERM INT].each do |signal|
29
+ Signal.trap(signal) do
30
+ stopping = true
31
+ pids.each_key { Process.kill(signal, it) rescue nil }
32
+ end
33
+ end
34
+
35
+ # The first to exit, then the other: told to stop, exit with their own
36
+ # status (0 for a clean stop); one exited by itself is never a clean stop.
37
+ statuses = {}
38
+ pid, status = Process.wait2
39
+ statuses[pid] = status
40
+ (pids.keys - [pid]).each do |other|
41
+ Process.kill("TERM", other)
42
+ rescue Errno::ESRCH
43
+ end
44
+ (pids.keys - [pid]).each do |other|
45
+ statuses[other] = Process.wait2(other).last
46
+ rescue Errno::ECHILD
47
+ end
48
+
49
+ codes = statuses.values.compact.map { it.exitstatus || 128 + it.termsig.to_i }
50
+ if stopping
51
+ exit codes.find(&:nonzero?) || 0
52
+ else
53
+ first = status.exitstatus || 128 + status.termsig.to_i
54
+ warn "rails-minify-ab: group #{pids[pid]} exited (#{first}), stopped the other"
55
+ exit first.zero? ? 1 : first
56
+ end
@@ -0,0 +1,154 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RailsMinify
4
+ # Compares a change against the app without it, under the same traffic: the
5
+ # server runs twice on one port (SO_REUSEPORT), half the workers each, as
6
+ # groups A and B (exe/rails-minify-ab). RAILS_MINIFY_GROUP tells a process
7
+ # its group, config.rails_minify.ab changes group B's configuration, and a
8
+ # log tag per group lets AB::Report compare them. One hour is enough to see
9
+ # a large difference, not to rule one out: run it again before believing a
10
+ # small one.
11
+ module AB
12
+ GROUPS = %w[A B].freeze
13
+
14
+ def self.group
15
+ group = ENV["RAILS_MINIFY_GROUP"].to_s.upcase
16
+ group if GROUPS.include?(group)
17
+ end
18
+
19
+ def self.log_tag(group = self.group) = "minify:#{group}"
20
+
21
+ # Request log lines, as Rails writes them with config.log_tags, from any
22
+ # log the tags survive in (kamal app logs, docker logs, a log file).
23
+ class Report
24
+ Request = Data.define(:group, :action, :status, :total, :views, :db, :gc)
25
+
26
+ TAGS = /((?:\[[^\]]*\] )+)/
27
+ PROCESSING = /#{TAGS}Processing by (\S+) as (\S+)/
28
+ COMPLETED = /#{TAGS}Completed (\d{3}) .*? in (\d+)ms(?: \((.*)\))?/
29
+
30
+ attr_reader :requests
31
+
32
+ def initialize(lines)
33
+ actions = {}
34
+ @requests = []
35
+ lines.each do |line|
36
+ line = line.scrub
37
+ if (m = PROCESSING.match(line))
38
+ group, key = tags(m[1])
39
+ actions[key] = "#{m[2]} #{m[3]}" if group
40
+ elsif (m = COMPLETED.match(line))
41
+ group, key = tags(m[1])
42
+ next unless group
43
+
44
+ details = m[4].to_s
45
+ ms = ->(name) { details[/#{name}: ([\d.]+)ms/, 1].to_f }
46
+ @requests << Request.new(group, actions.delete(key) || "?", m[2].to_i, m[3].to_i,
47
+ ms.("Views"), ms.("ActiveRecord"), ms.("GC"))
48
+ end
49
+ end
50
+ end
51
+
52
+ def report(io = $stdout, actions: 10)
53
+ ok = @requests.select { it.status < 400 }
54
+ io.puts "requests: #{GROUPS.map { |g| "#{g} #{@requests.count { it.group == g }}" }.join(", ")}; " \
55
+ "5xx: #{GROUPS.map { |g| "#{g} #{@requests.count { it.group == g && it.status >= 500 }}" }.join(", ")}"
56
+ if @requests.empty?
57
+ io.puts "no tagged requests: is the log from processes started by rails-minify-ab, with RAILS_MINIFY_AB set?"
58
+ return
59
+ end
60
+
61
+ io.puts
62
+ section(io, "all", ok)
63
+ section(io, "html", ok.select { it.action.end_with?(" HTML") })
64
+ by_action = ok.select { it.action.end_with?(" HTML") }.group_by(&:action)
65
+ by_action.max_by(actions) { |_, rs| rs.size }.each { |action, rs| section(io, action, rs) }
66
+ io.puts
67
+ io.puts "B-A: difference of the means with a 95% interval. An interval that includes 0 is no difference " \
68
+ "this window can show; one that doesn't is worth another window (requests aren't independent, " \
69
+ "so the interval is optimistic)."
70
+ end
71
+
72
+ private
73
+
74
+ def tags(text)
75
+ list = text.scan(/\[([^\]]*)\]/).flatten
76
+ group = list.find { it.start_with?("minify:") }&.delete_prefix("minify:")
77
+ [group, (list - ["minify:#{group}"]).join(" ")] if GROUPS.include?(group)
78
+ end
79
+
80
+ def section(io, title, requests)
81
+ a, b = GROUPS.map { |g| requests.select { it.group == g } }
82
+ io.puts title
83
+ { "A" => a, "B" => b }.each { |g, rs| io.puts " #{g}: #{summary(rs)}" }
84
+ io.puts " B-A: #{difference(a, b)}" unless a.empty? || b.empty?
85
+ end
86
+
87
+ def summary(rs)
88
+ return "-" if rs.empty?
89
+
90
+ t = rs.map(&:total)
91
+ format("n %5d p50 %4d p90 %4d p99 %4d mean %6.1f views %5.1f AR %5.1f GC %4.2f ms",
92
+ t.size, pct(t, 50), pct(t, 90), pct(t, 99), mean(t), mean(rs.map(&:views)), mean(rs.map(&:db)), mean(rs.map(&:gc)))
93
+ end
94
+
95
+ def difference(a, b)
96
+ %i[total views db gc].map do |field|
97
+ x, y = a.map(&field), b.map(&field)
98
+ d = mean(y) - mean(x)
99
+ half = 1.96 * Math.sqrt(variance(x) / x.size + variance(y) / y.size)
100
+ label = { total: "mean", views: "views", db: "AR", gc: "GC" }[field]
101
+ format("%s %+.1f (%+.1f..%+.1f)", label, d, d - half, d + half)
102
+ end.join(" ")
103
+ end
104
+
105
+ def pct(xs, p)
106
+ xs = xs.sort
107
+ xs[[xs.size - 1, ((p / 100.0) * (xs.size - 1)).round].min]
108
+ end
109
+
110
+ def mean(xs) = xs.empty? ? 0.0 : xs.sum.to_f / xs.size
111
+
112
+ def variance(xs)
113
+ return 0.0 if xs.size < 2
114
+
115
+ m = mean(xs)
116
+ xs.sum { (it - m)**2 } / (xs.size - 1)
117
+ end
118
+ end
119
+
120
+ # Memory of the processes in each group, from /proc (Linux; run it where
121
+ # the server runs, e.g. bin/kamal app exec). PSS counts pages shared with
122
+ # other processes (the preloaded parent's) in part, so the groups' totals
123
+ # compare what each costs.
124
+ module Memory
125
+ Entry = Data.define(:pid, :group, :title, :rss, :pss, :private_dirty)
126
+
127
+ def self.processes(proc_dir = "/proc")
128
+ Dir["#{proc_dir}/[0-9]*"].filter_map do |dir|
129
+ environ = File.binread("#{dir}/environ") rescue next
130
+ group = environ.split("\0").find { it.start_with?("RAILS_MINIFY_GROUP=") }&.delete_prefix("RAILS_MINIFY_GROUP=")
131
+ next unless GROUPS.include?(group)
132
+
133
+ rollup = File.read("#{dir}/smaps_rollup") rescue next
134
+ kb = ->(name) { rollup[/^#{name}:\s+(\d+) kB/, 1].to_i }
135
+ title = (File.binread("#{dir}/cmdline").tr("\0", " ").strip rescue "")
136
+ Entry.new(File.basename(dir).to_i, group, title, kb.("Rss"), kb.("Pss"), kb.("Private_Dirty"))
137
+ end.sort_by { [it.group, it.pid] }
138
+ end
139
+
140
+ def self.report(io = $stdout, proc_dir: "/proc")
141
+ list = processes(proc_dir)
142
+ if list.empty?
143
+ io.puts "no processes with RAILS_MINIFY_GROUP in #{proc_dir}: run this where the server runs, with RAILS_MINIFY_AB set"
144
+ return
145
+ end
146
+
147
+ list.each do |p|
148
+ io.puts format("%-2s %7d rss %5d pss %5d private %5d MB %s", p.group, p.pid, p.rss / 1024, p.pss / 1024, p.private_dirty / 1024, p.title[0, 50])
149
+ end
150
+ io.puts GROUPS.map { |g| "#{g}: PSS #{list.select { it.group == g }.sum(&:pss) / 1024} MB" }.join(", ")
151
+ end
152
+ end
153
+ end
154
+ end