prosopite 2.1.2 → 2.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,139 +1,110 @@
1
1
  # Prosopite ![CI](https://github.com/charkost/prosopite/actions/workflows/ci.yml/badge.svg) [![Gem Version](https://badge.fury.io/rb/prosopite.svg)](https://badge.fury.io/rb/prosopite)
2
2
 
3
- Prosopite is able to auto-detect Rails N+1 queries with zero false positives / false negatives.
3
+ <picture>
4
+ <source media="(prefers-reduced-motion: reduce)" srcset="docs/demo-still.png">
5
+ <img src="docs/demo.webp" width="840" alt="A Rails page runs 9 queries. Prosopite flags the N+1 in a footer and suggests Chair.includes(:legs) on line 2 of the view. After the fix, the page runs 2 queries.">
6
+ </picture>
4
7
 
5
- ```
6
- N+1 queries detected:
7
- SELECT `users`.* FROM `users` WHERE `users`.`id` = 20 LIMIT 1
8
- SELECT `users`.* FROM `users` WHERE `users`.`id` = 21 LIMIT 1
9
- SELECT `users`.* FROM `users` WHERE `users`.`id` = 22 LIMIT 1
10
- SELECT `users`.* FROM `users` WHERE `users`.`id` = 23 LIMIT 1
11
- SELECT `users`.* FROM `users` WHERE `users`.`id` = 24 LIMIT 1
12
- Call stack:
13
- app/controllers/thank_you_controller.rb:4:in `block in index'
14
- app/controllers/thank_you_controller.rb:3:in `each'
15
- app/controllers/thank_you_controller.rb:3:in `index':
16
- app/controllers/application_controller.rb:8:in `block in <class:ApplicationController>'
17
- ```
18
-
19
- The need for prosopite emerged after dealing with various false positives / negatives using the
20
- [bullet](https://github.com/flyerhzm/bullet) gem.
21
-
22
- ## Compared to Bullet
23
-
24
- Prosopite can auto-detect the following extra cases of N+1 queries:
8
+ Prosopite auto-detects N+1 queries in Rails apps, with zero false positives and false negatives. It finds every
9
+ query that repeats from the same place in your code, not only the lazy loading of associations, and suggests how
10
+ to fix it:
25
11
 
26
- #### N+1 queries after record creations (usually in tests)
27
-
28
- ```ruby
29
- FactoryBot.create_list(:leg, 10)
30
-
31
- Leg.last(10).each do |l|
32
- l.chair
33
- end
34
12
  ```
35
-
36
- #### Not triggered by ActiveRecord associations
37
-
38
- ```ruby
39
- Leg.last(4).each do |l|
40
- Chair.find(l.chair_id)
41
- end
13
+ N+1 queries detected (1.2ms):
14
+ SELECT "legs".* FROM "legs" WHERE "legs"."chair_id" = $1 ORDER BY "legs"."id" ASC LIMIT $2
15
+ SELECT "legs".* FROM "legs" WHERE "legs"."chair_id" = $1 ORDER BY "legs"."id" ASC LIMIT $2
16
+ SELECT "legs".* FROM "legs" WHERE "legs"."chair_id" = $1 ORDER BY "legs"."id" ASC LIMIT $2
17
+ Possible fix:
18
+ Chair.includes(:legs) where the chairs are loaded, at app/views/chairs/index.html.erb:2
19
+ Context:
20
+ GET /chairs
21
+ Call stack:
22
+ app/views/chairs/index.html.erb:3
23
+ app/views/chairs/index.html.erb:2
42
24
  ```
43
25
 
44
- #### First/last/pluck of collection associations
26
+ * [Quick start](#quick-start)
27
+ * [How it works](#how-it-works)
28
+ * [Usage](#usage): [development](#development), [tests](#tests), [jobs](#jobs), [production](#production),
29
+ [any code](#scanning-any-code)
30
+ * [Reports](#reports): [loggers](#loggers), [fix suggestions](#fix-suggestions),
31
+ [structured notifications](#structured-notifications), [CI](#ci)
32
+ * [Ignoring N+1 queries](#ignoring-n1-queries)
33
+ * [Configuration](#configuration)
34
+ * Compared to [Bullet](#compared-to-bullet) and [strict_loading](#compared-to-strict_loading)
45
35
 
46
- ```ruby
47
- Chair.last(20).each do |c|
48
- c.legs.first
49
- c.legs.last
50
- c.legs.pluck(:id)
51
- end
52
- ```
36
+ ## Quick start
53
37
 
54
- #### Changing the ActiveRecord class with #becomes
38
+ Add Prosopite to your Gemfile:
55
39
 
56
40
  ```ruby
57
- Chair.last(20).map{ |c| c.becomes(ArmChair) }.each do |ac|
58
- ac.legs.map(&:id)
59
- end
41
+ gem 'prosopite'
42
+ gem 'pg_query' # Optional, with PostgreSQL
60
43
  ```
61
44
 
62
- #### Mongoid models calling ActiveRecord
45
+ With PostgreSQL, [pg_query](https://github.com/pganalyze/pg_query) makes the fingerprints of queries more precise,
46
+ but it's a native extension that doesn't install everywhere, and Prosopite works without it.
63
47
 
64
- ```ruby
65
- class Leg::Design
66
- include Mongoid::Document
67
- ...
68
- field :cid, as: :chair_id, type: Integer
69
- ...
70
- def chair
71
- @chair ||= Chair.where(id: chair_id).first!
72
- end
73
- end
48
+ Then install it and set it up:
74
49
 
75
- Leg::Design.last(20) do |l|
76
- l.chair
77
- end
78
- ```
50
+ $ bundle install
51
+ $ bin/rails generate prosopite:install
79
52
 
80
- ## Why a new gem
53
+ The generator creates `config/initializers/prosopite.rb`, which shows N+1 queries on the pages in development and
54
+ raises for them in tests, and it scans each test. N+1 queries also go to the Rails log, unless another logger is set.
55
+ The sections below show how to set it up yourself.
81
56
 
82
- Creating a new gem makes more sense since bullet's core mechanism is completely
83
- different from prosopite's.
57
+ Prosopite needs Ruby 2.7+ and Rails 6.1+. The footer on pages needs Rails 7.1+, and the scanning of fibers Ruby 3.2+.
84
58
 
85
59
  ## How it works
86
60
 
87
- Prosopite monitors all SQL queries using the Active Support instrumentation
88
- and looks for the following pattern which is present in all N+1 query cases:
61
+ Prosopite subscribes to the SQL queries of Active Record. In a scan, e.g. of a request, a job or a test, it reports
62
+ the queries that run more than once from the same call stack and have the same fingerprint, i.e. that are the same
63
+ query apart from its values. Queries on different databases, e.g. on shards, never count together.
89
64
 
90
- More than one queries have the same call stack and the same query fingerprint.
65
+ ## Usage
91
66
 
92
- ## Installation
67
+ ### Development
93
68
 
94
- Add this line to your application's Gemfile:
69
+ The Rack middleware scans every request:
95
70
 
96
71
  ```ruby
97
- gem 'prosopite'
98
- ```
99
-
100
- If you're **not** using MySQL/MariaDB, you should also add:
72
+ # config/initializers/prosopite.rb
101
73
 
102
- ```ruby
103
- gem 'pg_query'
74
+ if Rails.env.development?
75
+ require 'prosopite/middleware/rack'
76
+ Rails.configuration.middleware.use(Prosopite::Middleware::Rack)
77
+ end
104
78
  ```
105
79
 
106
- And then execute:
107
-
108
- $ bundle install
109
-
110
- Or install it yourself as:
80
+ N+1 queries go to the Rails log, with the method and the path of the request as their context, e.g. `GET /chairs`.
81
+ In tests, the scan of each test covers its requests instead, see [Tests](#tests).
111
82
 
112
- $ gem install prosopite
83
+ To see the N+1 queries of a page on the page itself, in a footer and/or the browser console:
113
84
 
114
- ## Configuration
85
+ ```ruby
86
+ # config/environments/development.rb
115
87
 
116
- The preferred type of notifications can be configured with:
88
+ config.after_initialize do
89
+ Prosopite.add_footer = true
90
+ Prosopite.browser_console = true
91
+ end
92
+ ```
117
93
 
118
- * `Prosopite.min_n_queries`: Minimum number of N queries to report per N+1 case. Defaults to 2.
119
- * `Prosopite.raise = true`: Raise warnings as exceptions. Defaults to `false`.
120
- * `Prosopite.start_raise`: Raises warnings as exceptions from when this is called. Overrides `Proposite.raise`.
121
- * `Propsoite.stop_raise`: Disables raising warnings as exceptions if previously enabled with `Proposite.start_raise`.
122
- * `Prosopite.local_raise?`: Returns `true` if `Prosopite.start_raise` has been called previously.
123
- * `Prosopite.rails_logger = true`: Send warnings to the Rails log. Defaults to `false`.
124
- * `Prosopite.prosopite_logger = true`: Send warnings to `log/prosopite.log`. Defaults to `false`.
125
- * `Prosopite.stderr_logger = true`: Send warnings to STDERR. Defaults to `false`.
126
- * `Prosopite.backtrace_cleaner = my_custom_backtrace_cleaner`: use a different [ActiveSupport::BacktraceCleaner](https://api.rubyonrails.org/classes/ActiveSupport/BacktraceCleaner.html). Defaults to `Rails.backtrace_cleaner`.
127
- * `Prosopite.custom_logger = my_custom_logger`: Set a custom logger. See the following section for the details. Defaults to `false`.
128
- * `Prosopite.enabled = true`: Enables or disables the gem. Defaults to `true`.
94
+ ![The footer of Prosopite](docs/footer.png)
129
95
 
130
- ## Development Environment Usage
96
+ They also get the N+1 queries of later requests, like the ones of Turbo frames and streams or of other fetch and
97
+ XHR requests, which send them in an `x-prosopite-notifications` header. Requests that redirect, like form
98
+ submissions, can't show theirs, which go to the log only. The footer and the console need Rails 7.1+, show only when
99
+ `Prosopite.raise` is off, and follow your Content Security Policy: they use its nonce, if any, and leave out a style
100
+ or a script that it would block.
131
101
 
132
- Prosopite auto-detection can be enabled on all controllers:
102
+ Instead of the middleware, an `around_action` can scan the actions of controllers. Don't use a `before_action` and an
103
+ `after_action` for this: an exception skips the `after_action`, leaving the scan open into the next request.
133
104
 
134
105
  ```ruby
135
106
  class ApplicationController < ActionController::Base
136
- unless Rails.env.production?
107
+ if Rails.env.development?
137
108
  around_action :n_plus_one_detection
138
109
 
139
110
  def n_plus_one_detection
@@ -145,64 +116,99 @@ class ApplicationController < ActionController::Base
145
116
  end
146
117
  end
147
118
  ```
148
- And the preferred notification channel should be configured:
119
+
120
+ ### Tests
121
+
122
+ To make the tests with N+1 queries fail, raise for them and scan each test:
149
123
 
150
124
  ```ruby
151
- # config/environments/development.rb
125
+ # config/environments/test.rb
152
126
 
153
127
  config.after_initialize do
154
- Prosopite.rails_logger = true
128
+ Prosopite.raise = true
129
+ end
130
+ ```
131
+
132
+ ```ruby
133
+ # spec/rails_helper.rb
134
+
135
+ RSpec.configure do |config|
136
+ config.before { |example| Prosopite.scan(example.location) unless example.metadata[:prosopite] == false }
137
+ config.after { Prosopite.finish }
155
138
  end
156
139
  ```
140
+
141
+ ```ruby
142
+ # test/test_helper.rb
143
+
144
+ class ActiveSupport::TestCase
145
+ setup { Prosopite.scan("#{self.class}##{name}") }
146
+ teardown { Prosopite.finish }
147
+ end
157
148
  ```
158
- ## Test Environment Usage
159
149
 
160
- Tests with N+1 queries can be configured to fail with:
150
+ Each test is a scan of its own, with its location or name as the context, and specs can opt out with
151
+ `prosopite: false`. Don't scan the whole suite, as that would count the queries of different tests together.
152
+
153
+ The N+1 queries of [FactoryBot](https://github.com/thoughtbot/factory_bot) factories are usually only test setup.
154
+ To leave them out, pause the scans while factories run:
161
155
 
162
156
  ```ruby
163
157
  # config/environments/test.rb
164
158
 
165
159
  config.after_initialize do
166
- Prosopite.rails_logger = true
167
- Prosopite.raise = true
160
+ Prosopite.pause_during('factory_bot.run_factory')
168
161
  end
169
162
  ```
170
163
 
171
- And each test can be scanned with:
164
+ #### Assertions
172
165
 
173
- ```ruby
174
- # spec/spec_helper.rb
166
+ To check specific code for N+1 queries, or to check that it has some, use the RSpec matcher:
175
167
 
176
- config.before(:each) do
177
- Prosopite.scan
178
- end
168
+ ```ruby
169
+ # spec/rails_helper.rb
170
+ require 'prosopite/rspec'
179
171
 
180
- config.after(:each) do
181
- Prosopite.finish
182
- end
172
+ # in a spec
173
+ expect { get :index }.not_to have_n_plus_one_queries
183
174
  ```
184
175
 
185
- WARNING: scan/finish should run before/after **each** test and NOT before/after the whole suite.
176
+ or the Minitest assertions:
186
177
 
187
- ## Middleware
178
+ ```ruby
179
+ # test/test_helper.rb
180
+ require 'prosopite/minitest'
188
181
 
189
- ### Rack
182
+ # in a test
183
+ assert_no_n_plus_one_queries { get chairs_url }
184
+ assert_n_plus_one_queries { Chair.all.each { |chair| chair.legs.first } }
185
+ ```
186
+
187
+ The block runs in a scan of its own, so its N+1 queries are reported only by the assertion, even when each test
188
+ is scanned too. `Prosopite.n_plus_one_queries { ... }` returns them for any other use.
190
189
 
191
- Instead of using an `around_action` hook in a Rails Controller, you can also use the rack middleware instead
192
- implementing auto detect for all controllers.
190
+ #### Adopting Prosopite gradually
193
191
 
194
- Add the following line into your `config/initializers/prosopite.rb` file.
192
+ To start using Prosopite on an app that already has N+1 queries, record them in a todo file. Its N+1 queries are
193
+ then ignored, while new ones are reported:
195
194
 
196
195
  ```ruby
197
- unless Rails.env.production?
198
- require 'prosopite/middleware/rack'
199
- Rails.configuration.middleware.use(Prosopite::Middleware::Rack)
196
+ # config/environments/test.rb
197
+
198
+ config.after_initialize do
199
+ Prosopite.todo_file = Rails.root.join('.prosopite_todo.yml')
200
+ Prosopite.record_todo = ENV['PROSOPITE_RECORD_TODO'].present?
200
201
  end
201
202
  ```
202
203
 
203
- ### Sidekiq
204
- We also provide a middleware for sidekiq `6.5.0+` so that you can auto detect n+1 queries that may occur in a sidekiq job.
205
- You just need to add the following to your sidekiq initializer.
204
+ $ PROSOPITE_RECORD_TODO=1 bin/rails test
205
+
206
+ Entries match by query and file, so remove the ones of the N+1 queries you fix, or delete the file and record it
207
+ again. The assertions above don't use the todo file.
208
+
209
+ ### Jobs
210
+
211
+ The Sidekiq middleware, for Sidekiq 6.5+, scans every job, with its class, ID and queue as context:
206
212
 
207
213
  ```ruby
208
214
  Sidekiq.configure_server do |config|
@@ -215,99 +221,107 @@ Sidekiq.configure_server do |config|
215
221
  end
216
222
  ```
217
223
 
218
- For applications running sidekiq < `6.5.0` but want to add the snippet, you can guard the snippet with something like this and remove it once you upgrade sidekiq:
224
+ Other jobs can be scanned with a callback, e.g. for Active Job:
225
+
219
226
  ```ruby
220
- if Sidekiq::VERSION >= '6.5.0' && (Rails.env.development? || Rails.env.test?)
221
- .....
227
+ class ApplicationJob < ActiveJob::Base
228
+ around_perform do |job, block|
229
+ Prosopite.scan(job.class.name) { block.call }
230
+ end
222
231
  end
223
232
  ```
224
233
 
225
- ## Allow list
234
+ ### Production
226
235
 
227
- Ignore notifications for call stacks containing one or more substrings / regex:
236
+ To find N+1 queries in production with little overhead, scan a sample of the requests and jobs, and report only
237
+ the N+1 queries that take long enough to matter:
228
238
 
229
239
  ```ruby
230
- Prosopite.allow_stack_paths = ['substring_in_call_stack', /regex/]
231
- ```
240
+ # config/environments/production.rb
232
241
 
233
- Ignore notifications matching a specific SQL query:
242
+ require 'prosopite/middleware/rack'
234
243
 
235
- ```ruby
236
- Prosopite.ignore_queries = [/regex_match/, "SELECT * from EXACT_STRING_MATCH"]
244
+ Rails.application.configure do
245
+ config.middleware.use(Prosopite::Middleware::Rack)
246
+
247
+ config.after_initialize do
248
+ Prosopite.sample_rate = 0.01 # Scan 1% of the requests and jobs
249
+ Prosopite.min_duration_ms = 50
250
+ end
251
+ end
237
252
  ```
238
253
 
239
- ## Scanning code outside controllers or tests
254
+ ### Scanning any code
240
255
 
241
- All you have to do is to wrap the code with:
256
+ Wrap the code in a scan:
242
257
 
243
258
  ```ruby
244
259
  Prosopite.scan
245
- <code to scan>
260
+ # <code to scan>
246
261
  Prosopite.finish
247
262
  ```
248
263
 
249
- In block form the `Prosopite.finish` is called automatically for you at the end of the block:
264
+ In block form, `Prosopite.finish` runs at the end of the block, and the scan returns the result of the block:
250
265
 
251
266
  ```ruby
252
- Prosopite.scan do
253
- <code to scan>
267
+ my_object = Prosopite.scan do
268
+ MyObjectFactory.create(params)
254
269
  end
255
270
  ```
256
271
 
257
- The result of the code block is also returned by `Prosopite.scan`, so you can wrap calls as follows:
272
+ A context passed to `Prosopite.scan` is added to the notifications, to tell where they came from:
258
273
 
259
274
  ```ruby
260
- my_object = Prosopite.scan do
261
- MyObjectFactory.create(params)
275
+ Prosopite.scan("ImportJob #{import.id}") do
276
+ # <code to scan>
262
277
  end
263
278
  ```
264
279
 
265
- ## Pausing and resuming scans
280
+ ### Pausing scans
266
281
 
267
- Scans can be paused:
282
+ To leave out code that has N+1 queries, pause the scan:
268
283
 
269
284
  ```ruby
270
285
  Prosopite.scan
271
286
  # <code to scan>
272
287
  Prosopite.pause
273
- # <code that has n+1s>
288
+ # <code that has N+1 queries>
274
289
  Prosopite.resume
275
290
  # <code to scan>
276
291
  Prosopite.finish
277
292
  ```
278
293
 
279
- You can also pause items in a block, and the `Prosopite.resume` will be done
280
- for you automatically:
294
+ or pause it for a block, which can be nested:
281
295
 
282
296
  ```ruby
283
- Prosopite.scan
284
- # <code to scan>
285
-
286
297
  result = Prosopite.pause do
287
- # <code that has n+1s>
298
+ # <code that has N+1 queries>
288
299
  end
289
-
290
- Prosopite.finish
291
300
  ```
292
301
 
293
- Pauses can be ignored with `Prosopite.ignore_pauses = true` in case you want to remember their N+1 queries.
302
+ or pause it while the code instrumented as an Active Support event runs, e.g. the factories of FactoryBot:
294
303
 
295
- An example of when you might use this is if you are [testing Active Jobs inline](https://guides.rubyonrails.org/testing.html#testing-jobs),
296
- and don't want to run Prosopite on background job code, just foreground app code. In that case you could write an [Active Job callback](https://edgeguides.rubyonrails.org/active_job_basics.html#callbacks) that pauses the scan while the job is running.
304
+ ```ruby
305
+ Prosopite.pause_during('factory_bot.run_factory')
306
+ ```
297
307
 
298
- ## Local Raise
308
+ `Prosopite.finish` ends a scan even while it's paused, and reports the N+1 queries found before the pause, while
309
+ `Prosopite.scan` doesn't start another scan, so the pause applies to its code too. With
310
+ `Prosopite.ignore_pauses = true`, pauses don't apply, e.g. to report all the N+1 queries for a while.
299
311
 
300
- In some cases you may want to configure prosopite to not raise by default and only raise in certain scenarios.
301
- In this example we scan on all controllers but also provide an API to only raise on specific actions.
312
+ For example, when [testing Active Jobs inline](https://guides.rubyonrails.org/testing.html#testing-jobs), an
313
+ [Active Job callback](https://guides.rubyonrails.org/active_job_basics.html#callbacks) can pause the scan while a
314
+ job runs, to check only the code of the app.
302
315
 
303
- ```ruby
304
- Proposite.raise = false
305
- ```
316
+ ### Raising in specific places
317
+
318
+ To raise only for some code, e.g. some controller actions, instead of setting `Prosopite.raise`, call
319
+ `Prosopite.start_raise` and `Prosopite.stop_raise` around it. `Prosopite.local_raise?` tells whether it's on.
306
320
 
307
321
  ```ruby
308
322
  # app/controllers/application_controller.rb
309
323
  class ApplicationController < ActionController::Base
310
- def raise_on_n_plus_ones!(**options)
324
+ def self.raise_on_n_plus_ones!(**options)
311
325
  return if Rails.env.production?
312
326
 
313
327
  prepend_around_action(:_raise_on_n_plus_ones, **options)
@@ -317,11 +331,14 @@ class ApplicationController < ActionController::Base
317
331
  around_action :n_plus_one_detection
318
332
 
319
333
  def n_plus_one_detection
320
- ...
334
+ Prosopite.scan
335
+ yield
336
+ ensure
337
+ Prosopite.finish
321
338
  end
322
339
 
323
340
  def _raise_on_n_plus_ones
324
- Proposite.start_raise
341
+ Prosopite.start_raise
325
342
  yield
326
343
  ensure
327
344
  Prosopite.stop_raise
@@ -329,43 +346,249 @@ class ApplicationController < ActionController::Base
329
346
  end
330
347
  end
331
348
  ```
349
+
332
350
  ```ruby
333
351
  # app/controllers/books_controller.rb
334
352
  class BooksController < ApplicationController
335
353
  raise_on_n_plus_ones!(only: [:index])
336
354
 
337
355
  def index
338
- @books = Book.all.map(&:author) # This will raise N+1 errors
356
+ @books = Book.all.map(&:author) # This raises for the N+1 queries
339
357
  end
340
358
 
341
359
  def show
342
360
  @book = Book.find(params[:id])
343
- @book.reviews.map(&:author) # This will not raise N+1 errors
361
+ @book.reviews.map(&:author) # This only logs them
344
362
  end
345
363
  end
364
+ ```
365
+
366
+ ### Fibers and threads
367
+
368
+ On Ruby 3.2+, a scan also covers the fibers spawned while it runs, e.g. [Async](https://github.com/socketry/async)
369
+ tasks, unless they start a scan of their own. The queries of `ActionController::Live` actions count towards the scan
370
+ of their request too, while the ones of other threads don't. Pausing a scan pauses it in all of them.
371
+
372
+ ## Reports
373
+
374
+ ### Loggers
375
+
376
+ N+1 queries are reported to any of:
377
+
378
+ ```ruby
379
+ Prosopite.raise = true # Raise Prosopite::NPlusOneQueriesError
380
+ Prosopite.rails_logger = true # The Rails log, in red
381
+ Prosopite.stderr_logger = true # STDERR, in red
382
+ Prosopite.prosopite_logger = true # log/prosopite.log, or a path or a logger
383
+ Prosopite.custom_logger = logger # Any logger, e.g. Rails.logger for the Rails log without the red
384
+ ```
385
+
386
+ When no logger is set, they go to the Rails log, unless `Prosopite.rails_logger = false`.
387
+
388
+ The Rails backtrace cleaner leaves out the frames of gems, including engines. To see them, e.g. when developing an
389
+ engine, set another cleaner:
390
+
391
+ ```ruby
392
+ Prosopite.backtrace_cleaner = ActiveSupport::BacktraceCleaner.new
393
+ ```
394
+
395
+ To show fewer queries of each N+1, e.g. when they're many, and their count instead:
396
+
397
+ ```ruby
398
+ Prosopite.display_max_n_queries = 3
399
+ Prosopite.display_query_count = true
400
+ ```
401
+
402
+ ### Fix suggestions
403
+
404
+ Notifications suggest how to fix an N+1, from the associations whose queries look like its own, and show where the
405
+ query that loaded its records ran, like in the example at the top. If that query was built elsewhere, e.g.
406
+ `@chairs = Chair.order(:name)` in a controller, add the `includes` there. For `COUNT` queries, they suggest a counter
407
+ cache too.
408
+
409
+ Several associations can run the same query, so check which one your code uses. To turn them off, set
410
+ `Prosopite.suggest_fixes = false`.
411
+
412
+ ### Structured notifications
413
+
414
+ `Prosopite.finish` returns the N+1 queries it found. Each one is also published as an `n_plus_one_queries.prosopite`
415
+ event through [Active Support Instrumentation](https://guides.rubyonrails.org/active_support_instrumentation.html),
416
+ whether a logger is configured or not, with these keys:
417
+
418
+ | Key | Value |
419
+ | -------------- | ---------------------------------------------------------- |
420
+ | `:queries` | The SQL queries of the N+1 |
421
+ | `:call_stack` | The call stack, cleaned with `Prosopite.backtrace_cleaner` |
422
+ | `:duration_ms` | The total time of the queries in milliseconds |
423
+ | `:fixes` | Suggested fixes, see [Fix suggestions](#fix-suggestions) |
424
+ | `:fingerprint` | The fingerprint of the queries |
425
+ | `:context` | The context passed to `Prosopite.scan`, if any |
426
+
427
+ For example, to log them as JSON for Datadog, the ELK stack or similar:
428
+
429
+ ```ruby
430
+ Prosopite.rails_logger = false # Instead of the text ones
431
+
432
+ ActiveSupport::Notifications.subscribe('n_plus_one_queries.prosopite') do |event|
433
+ Rails.logger.warn(event.payload.to_json)
434
+ end
435
+ ```
436
+
437
+ ### CI
438
+
439
+ ```ruby
440
+ Prosopite.github_annotations = true # Show each N+1 on its line of code in GitHub Actions
441
+ Prosopite.json_report = 'tmp/prosopite.jsonl' # Add each N+1 to a file as a line of JSON, e.g. for coding agents
442
+ ```
443
+
444
+ ## Ignoring N+1 queries
445
+
446
+ Ignore the N+1 queries whose call stack contains one or more substrings or regexes, e.g. of specific files or
447
+ directories:
448
+
449
+ ```ruby
450
+ Prosopite.allow_stack_paths = ['app/models/legacy_report.rb', %r{lib/generated/}]
451
+ ```
452
+
453
+ Report only the N+1 queries whose call stack contains one of them, e.g. to fail tests for the N+1 queries of the app
454
+ code, but not of the test code itself. `allow_stack_paths` takes precedence:
455
+
456
+ ```ruby
457
+ Prosopite.focus_stack_paths = [Rails.root.join('app').to_s]
458
+ ```
459
+
460
+ Ignore the N+1 queries matching a regex, or equal to a string:
461
+
462
+ ```ruby
463
+ Prosopite.ignore_queries = [/regex_match/, 'SELECT * from EXACT_STRING_MATCH']
464
+ ```
465
+
466
+ Some gems run code through different files on the first and the later calls of a method, e.g. `sorbet-runtime`
467
+ (`call_validation.rb` and `call_validation_2_7.rb`), so the queries of an N+1 get different call stacks and aren't
468
+ counted together. Leave such frames out of the call stacks that Prosopite compares:
469
+
470
+ ```ruby
471
+ Prosopite.ignore_location_paths = ['sorbet-runtime', /observability/]
472
+ ```
473
+
474
+ The notifications still show the full call stack.
475
+
476
+ ## Configuration
477
+
478
+ | Reporting | Default | Description |
479
+ | ---------------------------- | ------- | ------------------------------------------------------------------------ |
480
+ | `Prosopite.raise` | `false` | Raise `Prosopite::NPlusOneQueriesError` |
481
+ | `Prosopite.rails_logger` | when no other logger is set | Log to the Rails log |
482
+ | `Prosopite.stderr_logger` | `false` | Log to STDERR |
483
+ | `Prosopite.prosopite_logger` | `false` | Log to `log/prosopite.log`, or to a path or a logger |
484
+ | `Prosopite.custom_logger` | none | Log to this logger |
485
+ | `Prosopite.add_footer` | `false` | Show them in a footer of pages, with the [Rack middleware](#development) |
486
+ | `Prosopite.browser_console` | `false` | Show them in the browser console, with the Rack middleware |
487
+ | `Prosopite.github_annotations` | `false` | Show them on the code in GitHub Actions |
488
+ | `Prosopite.json_report` | none | Add them to this file as lines of JSON |
489
+
490
+ | Messages | Default | Description |
491
+ | --------------------------------- | ------------------------- | ------------------------------------------ |
492
+ | `Prosopite.suggest_fixes` | `true` | Suggest fixes |
493
+ | `Prosopite.display_max_n_queries` | all | Show at most this many queries of each N+1 |
494
+ | `Prosopite.display_query_count` | `false` | Show the number of queries of each N+1 |
495
+ | `Prosopite.backtrace_cleaner` | `Rails.backtrace_cleaner` | Clean the call stacks |
496
+
497
+ | Scanning | Default | Description |
498
+ | --------------------------- | ------- | --------------------------------------------------------------- |
499
+ | `Prosopite.enabled` | `true` | Scan at all |
500
+ | `Prosopite.min_n_queries` | `2` | Report N+1s of at least this many queries |
501
+ | `Prosopite.min_duration_ms` | none | Report N+1s whose queries take at least this long in total |
502
+ | `Prosopite.sample_rate` | all | Run only this share of the scans, from 0 to 1 |
503
+ | `Prosopite.ignore_pauses` | `false` | Scan [paused code](#pausing-scans) too |
504
+
505
+ | Ignoring | Default | Description |
506
+ | --------------------------------- | ------- | --------------------------------------------------------------- |
507
+ | `Prosopite.allow_stack_paths` | none | Ignore N+1s from these places, see [Ignoring N+1 queries](#ignoring-n1-queries) |
508
+ | `Prosopite.focus_stack_paths` | none | Report only N+1s from these places |
509
+ | `Prosopite.ignore_queries` | none | Ignore these queries |
510
+ | `Prosopite.ignore_location_paths` | none | Leave these frames out of the call stacks that are compared |
511
+ | `Prosopite.todo_file` | none | Ignore the N+1s in this file, see [Adopting Prosopite gradually](#adopting-prosopite-gradually) |
512
+ | `Prosopite.record_todo` | `false` | Add the N+1s to the todo file instead of reporting them |
513
+
514
+ ## Compared to Bullet
515
+
516
+ Prosopite started after dealing with false positives and false negatives of [bullet](https://github.com/flyerhzm/bullet),
517
+ whose core mechanism is completely different: bullet tracks the loading of Active Record associations, while
518
+ Prosopite looks at the SQL queries themselves. This way it detects these N+1 queries too:
519
+
520
+ #### After creating records, usually in tests
521
+
522
+ ```ruby
523
+ FactoryBot.create_list(:leg, 10)
524
+
525
+ Leg.last(10).each do |l|
526
+ l.chair
527
+ end
528
+ ```
346
529
 
347
- ## Custom Logging Configuration
530
+ #### Not run by associations
348
531
 
349
- You can supply a custom logger with the `Prosopite.custom_logger` setting.
532
+ ```ruby
533
+ Leg.last(4).each do |l|
534
+ Chair.find(l.chair_id)
535
+ end
536
+ ```
350
537
 
351
- This is useful for circumstances where you don't want your logs to be
352
- highlighted with red, or you want logs sent to a custom location.
538
+ #### First, last and pluck of collection associations
353
539
 
354
- One common scenario is that you may be generating json logs and sending them to
355
- Datadog, ELK stack, or similar, and don't want to have to remove the default red
356
- escaping data from messages sent to the Rails logger, or want to tag them
357
- differently with your own custom logger.
540
+ ```ruby
541
+ Chair.last(20).each do |c|
542
+ c.legs.first
543
+ c.legs.last
544
+ c.legs.pluck(:id)
545
+ end
546
+ ```
547
+
548
+ #### After changing the class of records with becomes
358
549
 
359
550
  ```ruby
360
- # Turns off logging with red highlights, but still sends them to the Rails logger
361
- Prosopite.custom_logger = Rails.logger
551
+ Chair.last(20).map { |c| c.becomes(ArmChair) }.each do |ac|
552
+ ac.legs.map(&:id)
553
+ end
362
554
  ```
363
555
 
556
+ #### Of Mongoid models calling Active Record
557
+
364
558
  ```ruby
365
- # Use a completely custom logging instance
366
- Prosopite.custom_logger = MyLoggerClass.new
559
+ class Leg::Design
560
+ include Mongoid::Document
561
+ ...
562
+ field :cid, as: :chair_id, type: Integer
563
+ ...
564
+ def chair
565
+ @chair ||= Chair.where(id: chair_id).first!
566
+ end
567
+ end
568
+
569
+ Leg::Design.last(20).each do |l|
570
+ l.chair
571
+ end
367
572
  ```
368
573
 
574
+ On the other hand, bullet also detects eager loading that isn't used, and suggests counter caches for counting
575
+ associations. The first needs to track how associations are loaded and used, which queries alone can't show, and is
576
+ known for false positives. Prosopite suggests counter caches for N+1s of `COUNT` queries.
577
+
578
+ ## Compared to strict_loading
579
+
580
+ Rails' [`strict_loading`](https://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#method-i-strict_loading)
581
+ prevents the lazy loading of associations, which is a different trade-off:
582
+
583
+ * It raises on every lazy load of a strict record, even a single one that isn't part of an N+1, so enabling it on an
584
+ existing app means fixing all of them first. Its `n_plus_one_only` mode raises only for records loaded through a
585
+ `has_many` association of a strict record, so it misses the N+1 queries on the records of a plain query.
586
+ * It doesn't see N+1 queries that don't lazy load an association, like calling `Chair.find(leg.chair_id)`,
587
+ `chair.legs.count` or `chair.legs.exists?` in a loop.
588
+
589
+ Prosopite reports only the queries that actually repeat, whatever runs them, without changing your models.
590
+ The two can also be combined, e.g. with `strict_loading` on the models you have already fixed.
591
+
369
592
  ## Contributing
370
593
 
371
594
  Bug reports and pull requests are welcome on GitHub at https://github.com/charkost/prosopite.