activerecord-precompiled 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 491ffe9057eef6ea61d917c9811cf90747a0e76f1dd438228e144fc2cc57dbde
4
+ data.tar.gz: 1a7435d366a3c7542f600e3fef0704b73a49bf894e4fe45ba1c444203be0c8b5
5
+ SHA512:
6
+ metadata.gz: 43ed76ec56fa4d2827c898b5805706f3c4376158e8fccc37904ce0163432d161129a5278ce017bbcab9346a0a6350d091bc718fc2137234df968e56e98760421
7
+ data.tar.gz: 792f076ccf90410f48604a188d3fb4c6d223641d2c38b7edca7b9fb98f85be5e1610bb1a2b061bb526ff81cec19289c427b4968ecc788fc239302e985f460261
data/CHANGELOG.md ADDED
@@ -0,0 +1,45 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0 (2026-10-19)
4
+
5
+ First release. An ahead-of-time compiler for ActiveRecord query chains on
6
+ SQLite: the build rewrites chains in app source into calls on precompiled
7
+ Sites that run ActiveRecord's own SQL directly on the SQLite handle, keeping
8
+ the original chain as the fallback.
9
+
10
+ - Chains rooted at a model constant: `all`, `where`/`where.not` (hash
11
+ conditions, belongs_to keys, Array values as IN lists), `order` (literals,
12
+ `Arel.sql("...")`), `group`, `limit`, `offset`, static scopes,
13
+ `includes`/`preload`; terminators `to_a`, `first`/`last`/`take` (and `!`),
14
+ `find`, `find_by`/`find_by!`, `count`, `exists?` (with conditions or an id),
15
+ `pluck`, `sum`/`maximum`/`minimum`/`average`, plus `each`/`map`/... with a block.
16
+ - Chains rooted at a collection association (`user.posts.where(...)`,
17
+ has_many :through, HABTM), one Site per owner model.
18
+ - Precompiled preloads, including through/HABTM associations; whole
19
+ `includes` trees run without the Preloader, mirroring Preloader::Batch
20
+ (rounds, deferral, batching of equal queries).
21
+ - friendly_id `friendly.find`, ar_lazy_preload's `auto_preload`, deterministic
22
+ encrypted attributes (including `extend_queries`).
23
+ - Fast-path queries are instrumented as `sql.active_record` (logs with
24
+ `filter_attributes`, APM, query counts; `ActiveRecordPrecompiled.active_record_events
25
+ = false` to skip) and as `sql.active_record_precompiled`, and use ActiveRecord's query
26
+ cache store when it is on.
27
+ - A schema guard: if the runtime table layout differs from the build's, those
28
+ sites use ActiveRecord.
29
+ - `ar_precompile:scopes` reports which scopes compile, and how to fix the rest.
30
+ - `ar_precompile:schema:dump` writes the production schema to
31
+ `config/ar_precompile_schema_cache.yml`; the build uses it for a SQLite
32
+ database whose file is missing or has no tables (and removes that file), so
33
+ it runs without the database (`docker build`). Migrations newer than the
34
+ dump run on its tables, rebuilt in memory, so the build sees the schema
35
+ production will have after the deploy.
36
+ The build switches to the dumps after loading the app, so databases that
37
+ models connect to as they load (`connects_to`) get theirs too.
38
+ - Loaded Sites keep one copy of each string, bind, preload plan and option
39
+ they share, and allocate caches on first use; a Site object has six
40
+ instance variables (64 bytes on Ruby 4.1, 80 on 4.0; `bench/memory.rb`).
41
+
42
+ Supports Ruby 3.4+, ActiveRecord 8.0, 8.1 and main (8.2; association keys as
43
+ ActiveRecord::Key, October 2026), the sqlite3 adapter;
44
+ JRuby 10 with Rails 8.1 and activerecord-jdbc master. The gem no longer
45
+ depends on the sqlite3 gem itself (the app's adapter brings its driver).
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Sampo Kuokkanen
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,284 @@
1
+ # activerecord-precompiled
2
+
3
+ [![CI](https://github.com/sampokuokkanen/activerecord-precompiled/actions/workflows/ci.yml/badge.svg)](https://github.com/sampokuokkanen/activerecord-precompiled/actions/workflows/ci.yml)
4
+
5
+ `assets:precompile` for your ActiveRecord queries, on SQLite. You write normal
6
+ ActiveRecord; at deploy time, on production only, the query chains in your
7
+ source are rewritten into precompiled SQL that runs straight on the `sqlite3`
8
+ handle: no Arel, no relation objects. Logs, APM and the query cache see them as
9
+ usual.
10
+
11
+ ```ruby
12
+ # what you write (and commit)
13
+ def self.named(name) = User.find_by(name: name)
14
+
15
+ # what runs in production after `rails ar_precompile:build`
16
+ def self.named(name) = ::ActiveRecordPrecompiled::Sites::S9802ca3c9c_0.call(name) { |__xs0| User.find_by(name: __xs0) }
17
+ BEGIN { ::ActiveRecordPrecompiled::Site.define(:S9802ca3c9c_0, model: "User", kind: :one, sql: "SELECT \"users\".* FROM \"users\" WHERE \"users\".\"name\" = ? LIMIT ?", binds: [[:arg, 0, "name"], [:val, 1, "LIMIT"]], schema: "e79cff1769c2ac6f") } # activerecord-precompiled v0.1.0
18
+ ```
19
+
20
+ Same idea as ruby-next's `nextify` (source-to-source at build time) and
21
+ bootsnap (do the expensive work once, ahead of time), applied to queries.
22
+
23
+ ## How it works
24
+
25
+ * **SQL comes from ActiveRecord itself.** For each chain, the build runs the real
26
+ ActiveRecord method with placeholder arguments and intercepts the query at
27
+ the adapter. The SQL is byte-for-byte what ActiveRecord would send.
28
+ * **Binds are cast like ActiveRecord does it**: `type_for_attribute(col).serialize`, then
29
+ `connection.type_cast`, so enums, booleans, times, and decimals behave the same.
30
+ * **Execution** goes through `with_raw_connection`, using the adapter's own
31
+ prepared statement pool. Locking, lazy transactions, and reconnects behave as
32
+ they do in ActiveRecord.
33
+ * **The original chain is kept as a fallback block.** It runs for `nil`,
34
+ Range, Relation, or record arguments (and arrays ActiveRecord wouldn't turn
35
+ into a plain IN list), inside `scoping`/`unscoped` blocks, on
36
+ non-SQLite connections, when `ActiveRecordPrecompiled.enabled = false`, and to raise
37
+ `RecordNotFound` for `find`/`find_by!`/`first!`/`take!` misses.
38
+ * **Line numbers are preserved**: site definitions go into one `BEGIN {}` line
39
+ at the end of the file.
40
+ * **Schema guard.** Each site records a fingerprint of its table's columns,
41
+ including their order. If the runtime schema differs, the site logs one
42
+ warning and uses ActiveRecord from then on.
43
+
44
+ ## Supported
45
+
46
+ Chains rooted at a model constant:
47
+ `all`, `where(col: v, ...)`, `where.not(col: v, ...)`, `order(literals)`, `group(:cols)`, `limit(int)`, `offset(int)`,
48
+ `includes(literals)`, `preload(literals)`, static scopes,
49
+ followed by `to_a`, `first`, `last`, `take`, `first!`, `take!`, `find`, `find_by`, `find_by!`, `count`,
50
+ `exists?`, `exists?(col: v, ...)`, `exists?(id)`, `pluck(:cols)`, `sum`/`maximum`/`minimum`/`average(:col)`, or one of `each`, `map`, `flat_map`, `filter_map`,
51
+ `each_with_index`, `each_with_object`, `index_by`, `group_by` with a block.
52
+ `order` also takes `Arel.sql("...")` with a plain string. Calculations return what ActiveRecord does
53
+ (the column's type; `average` of an integer or decimal column is a BigDecimal;
54
+ `sum` of nothing is 0). With `group`, `count` and the calculations return
55
+ ActiveRecord's Hash (keys cast with the columns' types, an Array key for
56
+ several columns); `having` is left to ActiveRecord. `exists?(id)` takes the
57
+ fast path for an Integer or String id; `false`, `nil`, Arrays, Hashes and
58
+ records mean something else to ActiveRecord and go to it.
59
+
60
+ `includes`/`preload` run the main query on the fast path and then preload,
61
+ as `Relation#load` does. Each association's query is precompiled
62
+ (ActiveRecord's SQL for `key = ?` and `key IN (?, ...)`, up to 100 keys) for
63
+ associations whose scope is a fixed `where`/`order`. For `has_many`/`has_one
64
+ :through` and `has_and_belongs_to_many` that is the two queries the Preloader
65
+ runs for them (owners to the through records, then to the targets), each
66
+ checked at runtime against the scope ActiveRecord builds for it.
67
+
68
+ When every association in an `includes` tree has one, the Site preloads the
69
+ tree without building the Preloader's objects or the association scopes. It
70
+ follows what `Preloader::Batch` does, in the same order: rounds of runnable
71
+ loaders, loaders deferred while another branch will still load their table,
72
+ loaders with equal queries loaded by one query (so, as with ActiveRecord, a
73
+ comment's `author` can be the very object that is its `post.user`), grouping
74
+ by owner, inverses and already-loaded targets. A through association takes
75
+ this path when its two inner preloads assign what they load (no conditions,
76
+ order or `distinct` on it). Otherwise the Preloader runs as usual and only
77
+ its queries are precompiled. A through query that ActiveRecord joins (a
78
+ `where` on the through association, `source_type`), nested through
79
+ associations, polymorphic ones and scopes that read the clock stay with
80
+ ActiveRecord. If includes would JOIN instead (conditions or order on the
81
+ included table), or with `pluck`, the chain is left to ActiveRecord.
82
+
83
+ A condition value may be an Array at runtime: `where(id: ids)` runs
84
+ ActiveRecord's `id IN (?, ...)` (`NOT IN` for `where.not`) for any length up
85
+ to 100, and a one-element array the scalar SQL, as ActiveRecord does. The
86
+ build checks the IN form against ActiveRecord for each argument. Empty arrays,
87
+ arrays with `nil`, ranges or records, and longer ones go to ActiveRecord.
88
+
89
+ Chains can also start at a collection association (`has_many`, `has_many
90
+ :through`, `has_and_belongs_to_many`): `user.posts.where(title: t).first`,
91
+ `@trip.trip_days.count`, or `photos.maximum(:position)` inside a model. The
92
+ receiver's class isn't known at build time, so each model with an association
93
+ of that name whose scopes are static gets its own Site, with the owner's key
94
+ as a bind; at runtime the CollectionProxy tells which owner it is. Records
95
+ point back to the owner (inverse) and take its strict_loading mode, as
96
+ ActiveRecord's AssociationRelation does. ActiveRecord runs the chain for a
97
+ new or strict-loading owner, an owner class the build didn't see, and, when
98
+ the terminator is called on the association itself (`posts.first`), when the
99
+ association is loaded or holds unsaved records, since ActiveRecord may answer
100
+ from memory then. `posts.to_a`/`posts.each` (which load the association into
101
+ the owner) are left alone.
102
+
103
+ A condition key can also be a `belongs_to` association: `where(site: site)`
104
+ binds `site.id` to `site_id`, as ActiveRecord does (an id works too).
105
+ Polymorphic associations are left to ActiveRecord.
106
+
107
+ `Model.friendly.find(id)` from [friendly_id](https://github.com/norman/friendly_id)
108
+ compiles too, for `:slugged` and `:history` setups with an integer primary key
109
+ and FriendlyId's own finders: the slug, old-slug, and primary-key lookups are
110
+ each precompiled and tried in FriendlyId's order. A miss runs the original
111
+ chain, so it raises FriendlyId's own `RecordNotFound`.
112
+
113
+ Deterministic encrypted attributes (`encrypts :email, deterministic: true`)
114
+ work as condition keys: the argument is encrypted at runtime with the
115
+ attribute's type, like ActiveRecord does. With `extend_queries`, the extra
116
+ values it adds (the clear text, previous schemes) are bound too. A value that
117
+ isn't a String, or an encryption setup that differs from the build's, goes to
118
+ ActiveRecord. Constants compared to an encrypted attribute aren't compiled,
119
+ since their ciphertext would be made with the build machine's keys.
120
+
121
+ With [ar_lazy_preload](https://github.com/DmitryTsepelev/ar_lazy_preload)'s
122
+ `auto_preload` on, fast-path records get the same lazy-preload context that
123
+ `Relation#load` gives them, so their associations still load in batches.
124
+
125
+ Scopes are compiled when their body is static: one chain of `all`/`where`/
126
+ `where.not`/`order`/`limit`/`offset`/`includes`/`preload` and other static scopes, with
127
+ literals or the scope's own parameters as values (ranges included). Scopes
128
+ Rails generates (enum values, `with_rich_text_*`, `with_attached_*`) count as
129
+ static too:
130
+
131
+ ```ruby
132
+ scope :adults, -> { where(age: 18..) } # compiled
133
+ scope :named, ->(n) { where(name: n) } # compiled
134
+ scope :born_since, ->(t) { where(born_at: t..) } # compiled
135
+ scope :recent, -> { where(created_at: 1.week.ago..) } # not: evaluated per call
136
+ ```
137
+
138
+ To see which scopes compile and how to fix the rest:
139
+
140
+ ```sh
141
+ bin/rails ar_precompile:scopes # or: bundle exec ar-precompile scopes
142
+ ```
143
+ ```
144
+ ✅ User.adults test/test_helper.rb:58
145
+ ❌ User.recent app/models/user.rb:12
146
+ `1.week.ago` in `created_at:` is computed when the scope runs; make it a parameter:
147
+ ->(x) { where(created_at: x..) } and pass the value from the caller
148
+ activerecord-precompiled: 14 of 19 scopes can be compiled (plus 6 enum scopes)
149
+ ```
150
+
151
+ Bare constants resolve the way Ruby does: inside `module Admin`, `User` means
152
+ `Admin::User` if it exists.
153
+
154
+ Everything else is left alone: non-static scopes, joins, `eager_load` (and `includes`
155
+ that would JOIN), string conditions, chains on a relation held in a variable,
156
+ models with `default_scope`, keys that are neither columns nor `belongs_to`
157
+ associations, and arguments that are ranges or heredocs. STI subclasses work.
158
+
159
+ ## Deploy
160
+
161
+ ```ruby
162
+ gem "activerecord-precompiled"
163
+ ```
164
+
165
+ Run once per release, after `db:migrate` and before the app boots:
166
+
167
+ ```sh
168
+ RAILS_ENV=production bin/rails ar_precompile:build # DRY_RUN=1 to preview
169
+ ```
170
+
171
+ or `bundle exec ar-precompile [--dry-run] [--force] [paths]` from the Rails root.
172
+ Paths default to `app` and `lib`; for the rake task, set `AR_PRECOMPILE_PATHS=app,lib/foo`.
173
+ It refuses to run outside production unless `FORCE=1` / `--force`, it prints every
174
+ skipped call site with the reason, and running it again changes nothing. It never
175
+ writes a file that doesn't parse. Never commit the rewritten files.
176
+
177
+ ### Building without the database (`docker build`)
178
+
179
+ The build needs the production schema (columns, in order, with their types),
180
+ never data. Dump it where the production database is, and commit the file:
181
+
182
+ ```sh
183
+ RAILS_ENV=production bin/rails ar_precompile:schema:dump # or: bundle exec ar-precompile schema-dump
184
+ # writes config/ar_precompile_schema_cache.yml (ActiveRecord's schema cache format)
185
+ ```
186
+
187
+ Then the build runs anywhere, the image build included:
188
+
189
+ ```dockerfile
190
+ RUN SECRET_KEY_BASE_DUMMY=1 RAILS_ENV=production bin/rails ar_precompile:build
191
+ ```
192
+
193
+ When a SQLite database's file is missing, the build uses its dump on an empty
194
+ in-memory database, so it writes no database file into the image. The output
195
+ is the same as a build against the production database. A database file with
196
+ no tables counts as missing, and the build removes it: that is what an
197
+ initializer or gem that connects while the app loads leaves behind, and the
198
+ app would otherwise start on it in production. When the dump is older than your newest
199
+ migrations, the build recreates its tables (columns in order, keys, indexes)
200
+ and runs those migrations on them, as the deploy will on production: SQLite
201
+ adds a column last, so the order comes out the same. Dump again once they
202
+ have run on production. A migration that can't run on empty tables (one that
203
+ needs data, or a view) leaves the dump as it is, with a warning; the schema
204
+ guard then notices at runtime, and the queries it changes run on ActiveRecord. Other databases (Solid Cache,
205
+ Queue, Cable) don't need a dump: without one, queries on them are left to
206
+ ActiveRecord. `AR_PRECOMPILE_SCHEMA=dump` uses the dumps even when the
207
+ database is there, and `=database` never uses them.
208
+
209
+ The dump is kept apart from ActiveRecord's own `db/schema_cache.yml` on
210
+ purpose: the app would load that one at boot, and the schema guard has to see
211
+ the real database.
212
+
213
+ ## Trade-offs
214
+
215
+ * Fast-path queries are instrumented like ActiveRecord's own: a
216
+ `sql.active_record` event with the same payload (`sql`, `name`, `binds` as
217
+ named attributes, `type_casted_binds`, `connection`, `row_count`, ...) and
218
+ `precompiled: true`. So they show up in the Rails log (with
219
+ `filter_attributes` applied), APM, query counts (`assert_queries_count`) and
220
+ the request's "ActiveRecord: ...ms (N queries)". The event costs about as
221
+ much as ActiveRecord's (a few microseconds; Rails always has a subscriber).
222
+ To skip it: `ActiveRecordPrecompiled.active_record_events = false`. They are also
223
+ published as `sql.active_record_precompiled` (payload `sql`, `name`, `binds`,
224
+ `row_count`), which costs nothing when nobody subscribes.
225
+ * With ActiveRecord's query cache on (as in every request), fast-path results
226
+ are cached in the connection's own cache store, so they live exactly as
227
+ long as ActiveRecord's (any write clears them; `uncached` skips them).
228
+ Entries aren't shared with uncompiled calls of the same query. A hit is
229
+ logged and counted as `CACHE`, as ActiveRecord's are.
230
+ * SQL is fixed at build time, so build against the production schema: after
231
+ `db:migrate` on the production database, or from its
232
+ [dump](#building-without-the-database-docker-build). Column order matters,
233
+ and it is the order of the database's own migration history; a database
234
+ loaded from `schema.rb` has the order of whichever database dumped it. The
235
+ schema guard catches a mismatch, but those sites then run on ActiveRecord.
236
+ * Works with `query_log_tags_enabled`, which turns prepared statements off: the
237
+ build turns them on just while capturing SQL, and the runtime always binds.
238
+ Fast-path queries don't carry the query-log comment.
239
+
240
+ ## Results on a real app (golf-buddy, 720 Ruby files)
241
+
242
+ * 407 call sites rewritten in 183 files (about 190 of them on associations).
243
+ * Every rewritten site with data in the dev database (172 of them) returned
244
+ results identical to ActiveRecord, preloaded associations included. Query
245
+ cache off on both sides, fast-path queries instrumented as
246
+ `sql.active_record`: median 1.98× faster, geomean 2.17× (2.33× and 2.61×
247
+ with `active_record_events = false`). Medians by kind: association
248
+ `count`/`exists?` 3.0–3.5×, `exists?` 2.6×, `friendly.find` 2.5×,
249
+ `count` 2.3×, single-record lookups about 1.8× (loading the record
250
+ dominates on wide models). Given five-element arrays instead, the 62 sites
251
+ with IN lists matched too.
252
+ * The app's test suite (4317 runs, same seed, one worker) gives the same
253
+ results before and after the rewrite: the same 3 failures, which are in the
254
+ clean baseline too.
255
+ * Whole requests (12 public pages, production mode, dev data, fragment cache
256
+ off, query cache on): 1.6% less time in total. Pages run 15–175 queries, of
257
+ which the fast path takes 1–19; venue pages (4–6 fast-path queries) are 4–6%
258
+ faster, the rest within noise. Most remaining queries are lazy association
259
+ loads (`venue.venue_photos`, `photo.image` inside loops), which no rewritten
260
+ call site covers.
261
+
262
+ ## Requirements
263
+
264
+ Ruby 3.4+, ActiveRecord 8.0–8.2 (8.2 tested against Rails main, see
265
+ `Gemfile.edge`; 8.0 with `Gemfile.rails80`), the `sqlite3` adapter (the
266
+ `sqlite3` gem, or on JRuby activerecord-jdbc's sqlite3 adapter). JRuby 10 works
267
+ with Rails 8.1 and activerecord-jdbc's master branch, which is the only
268
+ activerecord-jdbc for Rails 8 so far (`Gemfile.jruby`): the fast path runs the
269
+ precompiled SQL on its JDBC connection, about 1.3–2.2× faster than ActiveRecord
270
+ there.
271
+
272
+ The build checks that it can still capture queries before touching any file.
273
+ If a Rails change breaks that, it stops with `ActiveRecordPrecompiled::Capture::Broken`
274
+ instead of finishing without rewriting anything.
275
+
276
+ ## Development
277
+
278
+ ```sh
279
+ bundle exec rake # differential tests: every case runs through AR and the rewrite
280
+ bundle exec rake test:edge # the same against Rails main
281
+ BUNDLE_GEMFILE=Gemfile.rails80 bundle exec rake test # ActiveRecord 8.0
282
+ BUNDLE_GEMFILE=Gemfile.jruby bundle exec rake test # JRuby (see Gemfile.jruby)
283
+ bundle exec ruby bench/bench.rb
284
+ ```
data/exe/ar-precompile ADDED
@@ -0,0 +1,47 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ # Usage (from the Rails root):
5
+ # RAILS_ENV=production bundle exec ar-precompile [--dry-run] [--force] [paths...]
6
+ # bundle exec ar-precompile scopes # which scopes can be compiled, and why not
7
+ # RAILS_ENV=production bundle exec ar-precompile schema-dump # for builds without the database
8
+ require "optparse"
9
+
10
+ if ARGV.first == "scopes"
11
+ require File.expand_path("config/environment", Dir.pwd)
12
+ Rails.application.eager_load!
13
+ ActiveRecordPrecompiled::ScopeCheck.new(root: Rails.root).run(out: $stdout)
14
+ exit
15
+ end
16
+
17
+ if ARGV.first == "schema-dump"
18
+ require File.expand_path("config/environment", Dir.pwd)
19
+ unless Rails.env.production? || ARGV.include?("--force")
20
+ abort "activerecord-precompiled: the build needs the production schema; run this with " \
21
+ "RAILS_ENV=production where the production database is, or override with --force"
22
+ end
23
+ Rails.application.eager_load! # every database a model connects to
24
+ ActiveRecordPrecompiled::SchemaDump.dump(Rails.root, out: $stdout)
25
+ exit
26
+ end
27
+
28
+ options = { dry_run: false, force: false }
29
+ schema = ENV["AR_PRECOMPILE_SCHEMA"]
30
+ OptionParser.new do |o|
31
+ o.banner = "Usage: ar-precompile [--dry-run] [--force] [paths (default: app lib)]"
32
+ o.on("--dry-run", "Report what would be rewritten, write nothing") { options[:dry_run] = true }
33
+ o.on("--force", "Allow running outside RAILS_ENV=production") { options[:force] = true }
34
+ o.on("--schema MODE", "auto (a schema dump for a missing database), dump or database") { schema = it }
35
+ end.parse!
36
+
37
+ require File.expand_path("config/environment", Dir.pwd)
38
+ Rails.application.eager_load! # before: models connect to their databases (connects_to) as they load
39
+ ActiveRecordPrecompiled::SchemaDump.use(Rails.root, mode: schema, out: $stdout)
40
+
41
+ ActiveRecordPrecompiled::Build.new(
42
+ root: Rails.root,
43
+ paths: ARGV.empty? ? %w[app lib] : ARGV,
44
+ env: Rails.env,
45
+ out: $stdout,
46
+ **options,
47
+ ).run
@@ -0,0 +1,65 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordPrecompiled
4
+ # A chain on a collection association: `owner.posts.where(name: n).first`
5
+ # becomes `AssocSite.call(owner.posts, n) { |proxy, n| proxy.where(name: n).first }`.
6
+ # The build compiled one Site per model with such an association, with the
7
+ # owner's key as its first bind; the CollectionProxy says which owner and
8
+ # which key this call is for.
9
+ #
10
+ # The original chain runs when the proxy isn't one of those, when the owner
11
+ # is a new record or loads strictly, and, for a terminator called on the
12
+ # association itself (`posts.first`, `posts.count`), when the association is
13
+ # loaded or holds unsaved records: ActiveRecord may answer from memory then.
14
+ class AssocSite
15
+ def self.define(name, **options)
16
+ Sites.send(:remove_const, name) if Sites.const_defined?(name, false)
17
+ Sites.const_set(name, new(**options))
18
+ end
19
+
20
+ # `owners`: owner class name => [Site name, owner key attribute].
21
+ def initialize(association:, direct:, owners:)
22
+ @association = association
23
+ @direct = direct
24
+ @owners = owners
25
+ @by_class = {}.compare_by_identity
26
+ end
27
+
28
+ def call(proxy, *args, &fallback)
29
+ return fallback.call(proxy, *args) unless proxy.is_a?(ActiveRecord::Associations::CollectionProxy)
30
+
31
+ association = proxy.proxy_association
32
+ owner = association.owner
33
+ site, key_name = entry_for(owner.class)
34
+ return fallback.call(proxy, *args) unless site && association.reflection.name == @association
35
+ return fallback.call(proxy, *args) if owner.new_record? || owner.strict_loading? || association.reflection.strict_loading?
36
+ return fallback.call(proxy, *args) if @direct && (association.loaded? || !association.target.empty?)
37
+
38
+ key = owner._read_attribute(key_name)
39
+ return fallback.call(proxy, *args) unless key.is_a?(Integer) || key.is_a?(String)
40
+
41
+ result = site.call(key, *args) { fallback.call(proxy, *args) }
42
+ case result
43
+ when ActiveRecord::Base then loaded(association, result)
44
+ when Array then result.each { loaded(association, it) if it.is_a?(ActiveRecord::Base) }
45
+ end
46
+ result
47
+ end
48
+
49
+ private
50
+
51
+ # As AssociationRelation#exec_queries: the record points back to the
52
+ # owner and takes its strict_loading mode.
53
+ def loaded(association, record)
54
+ association.set_inverse_instance_from_queries(record)
55
+ association.set_strict_loading(record)
56
+ end
57
+
58
+ def entry_for(klass)
59
+ @by_class.fetch(klass) do
60
+ site, key = @owners[klass.name]
61
+ @by_class[klass] = site ? [Sites.const_get(site, false), key] : nil
62
+ end
63
+ end
64
+ end
65
+ end
@@ -0,0 +1,52 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordPrecompiled
4
+ # Rewrites application files in place. Meant to run once per release, on the
5
+ # production host (or in the production image build), after the app has
6
+ # booted and the database schema is in place. Never commit the output.
7
+ class Build
8
+ NotProduction = Class.new(StandardError)
9
+ Report = Data.define(:files_changed, :sites, :skipped)
10
+
11
+ def initialize(root:, paths: %w[app lib], env: ENV["RAILS_ENV"] || ENV["RACK_ENV"], dry_run: false, force: false, out: nil)
12
+ @root = root.to_s
13
+ @paths = paths
14
+ @env = env
15
+ @dry_run = dry_run
16
+ @force = force
17
+ @out = out
18
+ end
19
+
20
+ def run
21
+ unless @force || @env == "production"
22
+ raise NotProduction, "activerecord-precompiled rewrites source in place; run it with RAILS_ENV=production, " \
23
+ "or override with FORCE=1 (rake), --force (CLI) or force: true (Ruby)"
24
+ end
25
+
26
+ Capture.verify!
27
+
28
+ files_changed = sites = 0
29
+ skipped = []
30
+ files.each do |file|
31
+ relative = file.delete_prefix("#{@root}/")
32
+ result = Rewriter.new(path: relative).rewrite(File.read(file))
33
+ skipped.concat(result.skipped)
34
+ result.skipped.each { @out&.puts "activerecord-precompiled: skipped #{it}" }
35
+ next if result.sites.zero?
36
+
37
+ files_changed += 1
38
+ sites += result.sites
39
+ @out&.puts "activerecord-precompiled: #{relative} (#{result.sites} queries)"
40
+ File.write(file, result.source) unless @dry_run
41
+ end
42
+ @out&.puts "activerecord-precompiled: #{sites} queries in #{files_changed} files#{" (dry run)" if @dry_run}"
43
+ Report.new(files_changed, sites, skipped)
44
+ end
45
+
46
+ private
47
+
48
+ def files
49
+ @paths.flat_map { Dir.glob(File.join(@root, it, "**", "*.rb")) }.sort
50
+ end
51
+ end
52
+ end
@@ -0,0 +1,94 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordPrecompiled
4
+ # Build-time only: runs a real ActiveRecord terminator (`first`, `count`,
5
+ # `find`, ...) but intercepts the query at the adapter, so we get the exact
6
+ # SQL and binds ActiveRecord would send, without touching any data.
7
+ module Capture
8
+ Query = Data.define(:sql, :binds)
9
+
10
+ Broken = Class.new(StandardError)
11
+
12
+ # Every relation read (records, count, pluck, exists?, find_by_sql) goes
13
+ # through `select_all`, on 8.0/8.1 and on Rails main, where the private
14
+ # `select` it used to call is gone. Compile the SQL and binds the way
15
+ # `select_all` does and return an empty result instead of running it.
16
+ module AdapterHook
17
+ def select_all(arel, name = nil, binds = [], preparable: nil, **)
18
+ log = Thread.current[:__ar_precompiled_capture]
19
+ return super unless log
20
+
21
+ sql, binds, = to_sql_and_binds(arel_from_relation(arel), binds, preparable)
22
+ log << Query.new(sql, binds)
23
+ ActiveRecord::Result.empty
24
+ end
25
+ end
26
+
27
+ # Placeholder for a runtime argument. ActiveRecord keeps it as the
28
+ # `value_before_type_cast` of the bind it creates (possibly a dup, hence
29
+ # the index), which tells us which `?` belongs to which argument. It is a
30
+ # String so that types which validate input (enums) can be satisfied.
31
+ # `record_key` is set for a belongs_to key (`where(site: x)`): the runtime
32
+ # value may be a record, whose primary key ActiveRecord binds instead.
33
+ class Sentinel < String
34
+ attr_reader :index, :record_key
35
+
36
+ def self.for(type, index, record_key: nil)
37
+ sentinel = new(index, record_key:)
38
+ return sentinel if bindable?(sentinel, type)
39
+
40
+ label = type.respond_to?(:mapping, true) && type.send(:mapping).keys.first
41
+ label ? new(index, label.to_s, record_key:) : sentinel
42
+ end
43
+
44
+ # Would ActiveRecord turn this value into a plain `column = ?` bind?
45
+ def self.bindable?(value, type)
46
+ attribute = ActiveRecord::Relation::QueryAttribute.new("probe", value, type)
47
+ !attribute.nil? && !attribute.unboundable?
48
+ end
49
+
50
+ def initialize(index, text = "__ar_precompiled_arg_#{index}__", record_key: nil)
51
+ super(text)
52
+ @index = index
53
+ @record_key = record_key
54
+ freeze
55
+ end
56
+ end
57
+
58
+ # Proves the hook still sees queries on this Rails version. Without this a
59
+ # Rails change would make every chain "run 0 queries" and the build would
60
+ # succeed while rewriting nothing.
61
+ def self.verify!
62
+ pool = ActiveRecord::Base.connection_handler.connection_pool_list(:all)
63
+ .find { it.db_config.adapter.to_s.start_with?("sqlite") }
64
+ return unless pool
65
+
66
+ captured = pool.with_connection { |conn| queries { conn.select_all("SELECT 1 AS ar_precompiled_probe") } }
67
+ return if captured.any? { it.sql.include?("ar_precompiled_probe") }
68
+
69
+ raise Broken, "activerecord-precompiled cannot capture queries on ActiveRecord #{ActiveRecord.version}; " \
70
+ "nothing was rewritten. This ActiveRecord version is not supported yet."
71
+ end
72
+
73
+ def self.install!
74
+ ActiveRecord::ConnectionAdapters::SQLite3Adapter.prepend(AdapterHook) unless
75
+ ActiveRecord::ConnectionAdapters::SQLite3Adapter <= AdapterHook
76
+ end
77
+
78
+ # Yields, returning every query ActiveRecord tried to run. Errors raised
79
+ # because the result was empty (RecordNotFound) are expected.
80
+ def self.queries
81
+ install!
82
+ log = []
83
+ previous = Thread.current[:__ar_precompiled_capture]
84
+ Thread.current[:__ar_precompiled_capture] = log
85
+ begin
86
+ ActiveRecord::Base.uncached { yield }
87
+ rescue ActiveRecord::RecordNotFound
88
+ ensure
89
+ Thread.current[:__ar_precompiled_capture] = previous
90
+ end
91
+ log
92
+ end
93
+ end
94
+ end
@@ -0,0 +1,49 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordPrecompiled
4
+ # `Model.friendly.find(id)` from friendly_id, as precompiled Sites taken in
5
+ # the order FriendlyId::FinderMethods#find tries them:
6
+ #
7
+ # * a Numeric id is "unfriendly": find by primary key;
8
+ # * a String is looked up by slug, then (with :history) by old slug, then,
9
+ # if it parses as an integer, by primary key.
10
+ #
11
+ # Anything else, any step that can't take the fast path, and a miss (which
12
+ # raises FriendlyId's own RecordNotFound) run the original chain.
13
+ class FriendlySite
14
+ def self.define(name, **options)
15
+ Sites.send(:remove_const, name) if Sites.const_defined?(name, false)
16
+ Sites.const_set(name, new(**options))
17
+ end
18
+
19
+ def initialize(slug:, id:, history: nil)
20
+ @slug = Sites.const_get(slug, false)
21
+ @history = history && Sites.const_get(history, false)
22
+ @id = Sites.const_get(id, false)
23
+ end
24
+
25
+ def call(id, &fallback)
26
+ catch do |tag|
27
+ give_up = ->(*) { throw tag }
28
+ return @id.call(id, &give_up) if id.is_a?(Integer)
29
+
30
+ if id.is_a?(String)
31
+ record = @slug.call(id, &give_up) || @history&.call(id, &give_up)
32
+ return record if record
33
+ return @id.call(id, &give_up) if integer?(id)
34
+ end
35
+ end
36
+ fallback.call(id)
37
+ end
38
+
39
+ private
40
+
41
+ # FriendlyId's potential_primary_key? for an integer primary key.
42
+ def integer?(id)
43
+ Integer(id, 10)
44
+ true
45
+ rescue ArgumentError
46
+ false
47
+ end
48
+ end
49
+ end