n_plus_one_control 0.4.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b20b8f4269aac76f3da642b271b93ddee2adf508539bba6852e02225695de155
4
- data.tar.gz: e739f342b4d46cc451229f3a065cb0e2b83ee28c301d99045d0f094e74bc137b
3
+ metadata.gz: 394ec848692af46dd43c3a03a3315349c2aea19d174ff05ededa1ef825839ee0
4
+ data.tar.gz: 7e2a271fe8ae173b117d1fc514e33b6f08aa7c528e4ea476f5360591a28da6f8
5
5
  SHA512:
6
- metadata.gz: 74805b4ea497ff96bb882557ed8a14e04d32cf5ea5c62e9c65167647c880275aec9d0f8a405a298554e9cfecbb20049d99f630b977475a75122d847c7b27e1a9
7
- data.tar.gz: 19063ec1fb2a84edcfd0e38075a3858e239a438803ffb40e03916bc8eb5d49dda9e8b03a30a60788e4e00f9c5484811740448fe3241e64658a78ccf884619321
6
+ metadata.gz: 1f411b0c1539f517e0c4036570e274fe62327817722065551fce09307e15402c31185566343ddb9fd388324a71c5c8d7730fc938767397f5bbba60f103ffdccb
7
+ data.tar.gz: 21a4445b6dd7efa9dad9bbfe2ccf8708ae813e2e60ea424b60f7b00cf58bef1971d58b1392893f34e817a3834792b461536890ed38619a20e0f9bd770ab7ddfd
@@ -1,5 +1,19 @@
1
1
  ## master (unreleased)
2
2
 
3
+ ## 0.5.0 (2020-09-07)
4
+
5
+ - **Ruby 2.5+ is required**. ([@palkan][])
6
+
7
+ - Add support for multiple backtrace lines in verbose output. ([@palkan][])
8
+
9
+ Could be specified via `NPLUSONE_BACKTRACE` env var.
10
+
11
+ - Add `NPLUSONE_TRUNCATE` env var to truncate queries in verbose mode. ([@palkan][])
12
+
13
+ - Support passing default filter via `NPLUSONE_FILTER` env var. ([@palkan][])
14
+
15
+ - Add location tracing to SQLs in verbose mode. ([@palkan][])
16
+
3
17
  ## 0.4.1 (2020-09-04)
4
18
 
5
19
  - Enhance failure message by showing differences in table hits. ([@palkan][])
data/README.md CHANGED
@@ -1,4 +1,5 @@
1
- [![Gem Version](https://badge.fury.io/rb/n_plus_one_control.svg)](https://rubygems.org/gems/n_plus_one_control) [![Build Status](https://travis-ci.org/palkan/n_plus_one_control.svg?branch=master)](https://travis-ci.org/palkan/n_plus_one_control)
1
+ [![Gem Version](https://badge.fury.io/rb/n_plus_one_control.svg)](https://rubygems.org/gems/n_plus_one_control)
2
+ ![Build](https://github.com/palkan/n_plus_one_control/workflows/Build/badge.svg)
2
3
 
3
4
  # N + 1 Control
4
5
 
@@ -31,7 +32,7 @@ Add this line to your application's Gemfile:
31
32
 
32
33
  ```ruby
33
34
  group :test do
34
- gem 'n_plus_one_control'
35
+ gem "n_plus_one_control"
35
36
  end
36
37
  ```
37
38
 
@@ -47,8 +48,6 @@ First, add NPlusOneControl to your `spec_helper.rb`:
47
48
 
48
49
  ```ruby
49
50
  # spec_helper.rb
50
- ...
51
-
52
51
  require "n_plus_one_control/rspec"
53
52
  ```
54
53
 
@@ -86,10 +85,10 @@ Availables modifiers:
86
85
  ```ruby
87
86
  # You can specify the RegExp to filter queries.
88
87
  # By default, it only considers SELECT queries.
89
- expect { ... }.to perform_constant_number_of_queries.matching(/INSERT/)
88
+ expect { subject }.to perform_constant_number_of_queries.matching(/INSERT/)
90
89
 
91
90
  # You can also provide custom scale factors
92
- expect { ... }.to perform_constant_number_of_queries.with_scale_factors(10, 100)
91
+ expect { subject }.to perform_constant_number_of_queries.with_scale_factors(10, 100)
93
92
  ```
94
93
 
95
94
  #### Using scale factor in spec
@@ -97,7 +96,7 @@ expect { ... }.to perform_constant_number_of_queries.with_scale_factors(10, 100)
97
96
  Let's suppose your action accepts parameter, which can make impact on the number of returned records:
98
97
 
99
98
  ```ruby
100
- get :index, params: { per_page: 10 }
99
+ get :index, params: {per_page: 10}
101
100
  ```
102
101
 
103
102
  Then it is enough to just change `per_page` parameter between executions and do not recreate records in DB. For this purpose, you can use `current_scale` method in your example:
@@ -107,7 +106,7 @@ context "N+1", :n_plus_one do
107
106
  before { create_list :post, 3 }
108
107
 
109
108
  specify do
110
- expect { get :index, params: { per_page: current_scale } }.to perform_constant_number_of_queries
109
+ expect { get :index, params: {per_page: current_scale} }.to perform_constant_number_of_queries
111
110
  end
112
111
  end
113
112
  ```
@@ -118,8 +117,6 @@ First, add NPlusOneControl to your `test_helper.rb`:
118
117
 
119
118
  ```ruby
120
119
  # test_helper.rb
121
- ...
122
-
123
120
  require "n_plus_one_control/minitest"
124
121
  ```
125
122
 
@@ -153,6 +150,12 @@ assert_perform_constant_number_of_queries(
153
150
  end
154
151
  ```
155
152
 
153
+ It's possible to specify a filter via `NPLUSONE_FILTER` env var, e.g.:
154
+
155
+ ```ruby
156
+ NPLUSONE_FILTER = users bundle exec rake test
157
+ ```
158
+
156
159
  You can also specify `populate` as a test class instance method:
157
160
 
158
161
  ```ruby
@@ -165,6 +168,7 @@ def test_no_n_plus_one_error
165
168
  get :index
166
169
  end
167
170
  end
171
+
168
172
  ```
169
173
 
170
174
  As in RSpec, you can use `current_scale` factor instead of `populate` block:
@@ -172,7 +176,7 @@ As in RSpec, you can use `current_scale` factor instead of `populate` block:
172
176
  ```ruby
173
177
  def test_no_n_plus_one_error
174
178
  assert_perform_constant_number_of_queries do
175
- get :index, params: { per_page: current_scale }
179
+ get :index, params: {per_page: current_scale}
176
180
  end
177
181
  end
178
182
  ```
@@ -223,6 +227,7 @@ end
223
227
  ```
224
228
 
225
229
  If your `warmup` and testing procs are identical, you can use:
230
+
226
231
  ```ruby
227
232
  expext { get :index }.to perform_constant_number_of_queries.with_warming_up # RSpec only
228
233
  ```
@@ -254,7 +259,7 @@ NPlusOneControl.ignore = /^(BEGIN|COMMIT|SAVEPOINT|RELEASE)/
254
259
  # ActiveSupport notifications event to track queries.
255
260
  # We track ActiveRecord event by default,
256
261
  # but can also track rom-rb events ('sql.rom') as well.
257
- NPlusOneControl.event = 'sql.active_record'
262
+ NPlusOneControl.event = "sql.active_record"
258
263
 
259
264
  # configure transactional behavour for populate method
260
265
  # in case of use multiple database connections
@@ -268,6 +273,21 @@ NPlusOneControl::Executor.tap do |executor|
268
273
  connections.each(&:rollback_transaction)
269
274
  end
270
275
  end
276
+
277
+ # Provide a backtrace cleaner callable object used to filter SQL caller location to display in the verbose mode
278
+ # Set it to nil to disable tracing.
279
+ #
280
+ # In Rails apps, we use Rails.backtrace_cleaner by default.
281
+ NPlusOneControl.backtrace_cleaner = ->(locations_array) { do_some_filtering(locations_array) }
282
+
283
+ # You can also specify the number of backtrace lines to show.
284
+ # MOTE: It could be specified via NPLUSONE_BACKTRACE env var
285
+ NPlusOneControl.backtrace_length = 1
286
+
287
+ # Sometime queries could be too large to provide any meaningful insight.
288
+ # You can configure an output length limit for quries in verbose mode by setting the follwing option
289
+ # NOTE: It could be specified via NPLUSONE_TRUNCATE env var
290
+ NPlusOneControl.truncate_query_size = 100
271
291
  ```
272
292
 
273
293
  ## How does it work?
@@ -276,22 +296,29 @@ Take a look at our [Executor](https://github.com/palkan/n_plus_one_control/blob/
276
296
 
277
297
  ## What's next?
278
298
 
299
+ - More matchers.
300
+
279
301
  It may be useful to provide more matchers/assertions, for example:
280
302
 
281
303
  ```ruby
282
304
 
283
305
  # Actually, that means that it is N+1))
284
- assert_linear_number_of_queries { ... }
306
+ assert_linear_number_of_queries { some_code }
285
307
 
286
308
  # But we can tune it with `coef` and handle such cases as selecting in batches
287
309
  assert_linear_number_of_queries(coef: 0.1) do
288
- Post.find_in_batches { ... }
310
+ Post.find_in_batches { some_code }
289
311
  end
290
312
 
291
313
  # probably, also make sense to add another curve types
292
- assert_logarithmic_number_of_queries { ... }
314
+ assert_logarithmic_number_of_queries { some_code }
293
315
  ```
294
316
 
317
+ - Support custom non-SQL events.
318
+
319
+ N+1 problem is not a database specific: we can have N+1 Redis calls, N+1 HTTP external requests, etc.
320
+ We can make `n_plus_one_control` customizable to support these scenarios (technically, we need to make it possible to handle different payload in the event subscriber).
321
+
295
322
  If you want to discuss or implement any of these, feel free to open an [issue](https://github.com/palkan/n_plus_one_control/issues) or propose a [pull request](https://github.com/palkan/n_plus_one_control/pulls).
296
323
 
297
324
  ## Development
@@ -18,7 +18,10 @@ module NPlusOneControl
18
18
  }.freeze
19
19
 
20
20
  class << self
21
- attr_accessor :default_scale_factors, :verbose, :show_table_stats, :ignore, :event
21
+ attr_accessor :default_scale_factors, :verbose, :show_table_stats, :ignore, :event,
22
+ :backtrace_cleaner, :backtrace_length, :truncate_query_size
23
+
24
+ attr_reader :default_matching
22
25
 
23
26
  def failure_message(queries) # rubocop:disable Metrics/MethodLength
24
27
  msg = ["Expected to make the same number of queries, but got:\n"]
@@ -30,8 +33,8 @@ module NPlusOneControl
30
33
 
31
34
  if verbose
32
35
  queries.each do |(scale, data)|
33
- msg << " Queries for N=#{scale}\n"
34
- msg << data.map { |sql| " #{sql}\n" }.join.to_s
36
+ msg << "Queries for N=#{scale}\n"
37
+ msg << data.map { |sql| " #{truncate_query(sql)}\n" }.join.to_s
35
38
  end
36
39
  end
37
40
 
@@ -39,7 +42,7 @@ module NPlusOneControl
39
42
  end
40
43
 
41
44
  def table_usage_stats(runs) # rubocop:disable Metrics/MethodLength
42
- msg = ["\nUnmatched query numbers by tables:\n"]
45
+ msg = ["Unmatched query numbers by tables:\n"]
43
46
 
44
47
  before, after = runs.map do |queries|
45
48
  queries.group_by do |query|
@@ -58,6 +61,38 @@ module NPlusOneControl
58
61
 
59
62
  msg
60
63
  end
64
+
65
+ def default_matching=(val)
66
+ unless val
67
+ @default_matching = nil
68
+ return
69
+ end
70
+
71
+ @default_matching =
72
+ if val.is_a?(Regexp)
73
+ val
74
+ else
75
+ Regexp.new(val, Regexp::MULTILINE | Regexp::IGNORECASE)
76
+ end
77
+ end
78
+
79
+ private
80
+
81
+ def truncate_query(sql)
82
+ return sql unless truncate_query_size
83
+
84
+ # Only truncate query, leave tracing (if any) as is
85
+ parts = sql.split(/(\s+↳)/)
86
+
87
+ parts[0] =
88
+ if truncate_query_size < 4
89
+ "..."
90
+ else
91
+ parts[0][0..(truncate_query_size - 4)] + "..."
92
+ end
93
+
94
+ parts.join
95
+ end
61
96
  end
62
97
 
63
98
  # Scale factors to use.
@@ -65,7 +100,7 @@ module NPlusOneControl
65
100
  self.default_scale_factors = [2, 3]
66
101
 
67
102
  # Print performed queries if true
68
- self.verbose = ENV['NPLUSONE_VERBOSE'] == '1'
103
+ self.verbose = ENV["NPLUSONE_VERBOSE"] == "1"
69
104
 
70
105
  # Print table hits difference
71
106
  self.show_table_stats = true
@@ -76,5 +111,16 @@ module NPlusOneControl
76
111
  # ActiveSupport notifications event to track queries.
77
112
  # We track ActiveRecord event by default,
78
113
  # but can also track rom-rb events ('sql.rom') as well.
79
- self.event = 'sql.active_record'
114
+ self.event = "sql.active_record"
115
+
116
+ # Default query filtering applied if none provided explicitly
117
+ self.default_matching = ENV["NPLUSONE_FILTER"] || /^SELECT/i
118
+
119
+ # Truncate queries in verbose mode to fit the length
120
+ self.truncate_query_size = ENV["NPLUSONE_TRUNCATE"]&.to_i
121
+
122
+ # Define the number of backtrace lines to show
123
+ self.backtrace_length = ENV.fetch("NPLUSONE_BACKTRACE", 1).to_i
80
124
  end
125
+
126
+ require "n_plus_one_control/railtie" if defined?(Rails::Railtie)
@@ -19,10 +19,27 @@ module NPlusOneControl
19
19
  @queries
20
20
  end
21
21
 
22
- def callback(_name, _start, _finish, _message_id, values)
22
+ def callback(_name, _start, _finish, _message_id, values) # rubocop:disable Metrics/CyclomaticComplexity,Metrics/LineLength
23
23
  return if %w[CACHE SCHEMA].include? values[:name]
24
24
 
25
- @queries << values[:sql] if @pattern.nil? || (values[:sql] =~ @pattern)
25
+ return unless @pattern.nil? || (values[:sql] =~ @pattern)
26
+
27
+ query = values[:sql]
28
+
29
+ if NPlusOneControl.backtrace_cleaner && NPlusOneControl.verbose
30
+ source = extract_query_source_location(caller)
31
+
32
+ query = "#{query}\n ↳ #{source.join("\n")}" unless source.empty?
33
+ end
34
+
35
+ @queries << query
36
+ end
37
+
38
+ private
39
+
40
+ def extract_query_source_location(locations)
41
+ NPlusOneControl.backtrace_cleaner.call(locations.lazy)
42
+ .take(NPlusOneControl.backtrace_length).to_a
26
43
  end
27
44
  end
28
45
 
@@ -18,7 +18,7 @@ module NPlusOneControl
18
18
 
19
19
  @executor = NPlusOneControl::Executor.new(
20
20
  population: populate || population_method,
21
- matching: matching || /^SELECT/i,
21
+ matching: matching || NPlusOneControl.default_matching,
22
22
  scale_factors: scale_factors || NPlusOneControl.default_scale_factors
23
23
  )
24
24
 
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+
3
+ module NPlusOneControl # :nodoc:
4
+ class Railtie < ::Rails::Railtie # :nodoc:
5
+ initializer "n_plus_one_control.backtrace_cleaner" do
6
+ ActiveSupport.on_load(:active_record) do
7
+ NPlusOneControl.backtrace_cleaner = lambda do |locations|
8
+ ::Rails.backtrace_cleaner.clean(locations)
9
+ end
10
+ end
11
+ end
12
+ end
13
+ end
@@ -27,8 +27,7 @@
27
27
 
28
28
  warmup.call if warmup.present?
29
29
 
30
- # by default we're looking for select queries
31
- pattern = @pattern || /^SELECT/i
30
+ pattern = @pattern || NPlusOneControl.default_matching
32
31
 
33
32
  @matcher_execution_context.executor = NPlusOneControl::Executor.new(
34
33
  population: populate,
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module NPlusOneControl
4
- VERSION = "0.4.1"
4
+ VERSION = "0.5.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: n_plus_one_control
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.1
4
+ version: 0.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - palkan
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-09-04 00:00:00.000000000 Z
11
+ date: 2020-09-07 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -28,16 +28,16 @@ dependencies:
28
28
  name: rake
29
29
  requirement: !ruby/object:Gem::Requirement
30
30
  requirements:
31
- - - "~>"
31
+ - - ">="
32
32
  - !ruby/object:Gem::Version
33
- version: '10.0'
33
+ version: '13.0'
34
34
  type: :development
35
35
  prerelease: false
36
36
  version_requirements: !ruby/object:Gem::Requirement
37
37
  requirements:
38
- - - "~>"
38
+ - - ">="
39
39
  - !ruby/object:Gem::Version
40
- version: '10.0'
40
+ version: '13.0'
41
41
  - !ruby/object:Gem::Dependency
42
42
  name: rspec
43
43
  requirement: !ruby/object:Gem::Requirement
@@ -80,62 +80,6 @@ dependencies:
80
80
  - - "~>"
81
81
  - !ruby/object:Gem::Version
82
82
  version: 4.8.0
83
- - !ruby/object:Gem::Dependency
84
- name: rubocop
85
- requirement: !ruby/object:Gem::Requirement
86
- requirements:
87
- - - "~>"
88
- - !ruby/object:Gem::Version
89
- version: 0.61.0
90
- type: :development
91
- prerelease: false
92
- version_requirements: !ruby/object:Gem::Requirement
93
- requirements:
94
- - - "~>"
95
- - !ruby/object:Gem::Version
96
- version: 0.61.0
97
- - !ruby/object:Gem::Dependency
98
- name: activerecord
99
- requirement: !ruby/object:Gem::Requirement
100
- requirements:
101
- - - "~>"
102
- - !ruby/object:Gem::Version
103
- version: '5.1'
104
- type: :development
105
- prerelease: false
106
- version_requirements: !ruby/object:Gem::Requirement
107
- requirements:
108
- - - "~>"
109
- - !ruby/object:Gem::Version
110
- version: '5.1'
111
- - !ruby/object:Gem::Dependency
112
- name: sqlite3
113
- requirement: !ruby/object:Gem::Requirement
114
- requirements:
115
- - - "~>"
116
- - !ruby/object:Gem::Version
117
- version: 1.3.6
118
- type: :development
119
- prerelease: false
120
- version_requirements: !ruby/object:Gem::Requirement
121
- requirements:
122
- - - "~>"
123
- - !ruby/object:Gem::Version
124
- version: 1.3.6
125
- - !ruby/object:Gem::Dependency
126
- name: pry-byebug
127
- requirement: !ruby/object:Gem::Requirement
128
- requirements:
129
- - - ">="
130
- - !ruby/object:Gem::Version
131
- version: '0'
132
- type: :development
133
- prerelease: false
134
- version_requirements: !ruby/object:Gem::Requirement
135
- requirements:
136
- - - ">="
137
- - !ruby/object:Gem::Version
138
- version: '0'
139
83
  description: "\n RSpec and Minitest matchers to prevent N+1 queries problem.\n\n
140
84
  \ Evaluates code under consideration several times with different scale factors\n
141
85
  \ to make sure that the number of DB queries behaves as expected (i.e. O(1) instead
@@ -152,6 +96,7 @@ files:
152
96
  - lib/n_plus_one_control.rb
153
97
  - lib/n_plus_one_control/executor.rb
154
98
  - lib/n_plus_one_control/minitest.rb
99
+ - lib/n_plus_one_control/railtie.rb
155
100
  - lib/n_plus_one_control/rspec.rb
156
101
  - lib/n_plus_one_control/rspec/context.rb
157
102
  - lib/n_plus_one_control/rspec/dsl.rb
@@ -174,7 +119,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
174
119
  requirements:
175
120
  - - ">="
176
121
  - !ruby/object:Gem::Version
177
- version: 2.0.0
122
+ version: 2.5.0
178
123
  required_rubygems_version: !ruby/object:Gem::Requirement
179
124
  requirements:
180
125
  - - ">="