ar-query-matchers 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/README.md +91 -0
- data/lib/ar_query_matchers.rb +241 -0
- data/lib/ar_query_matchers/queries/create_counter.rb +31 -0
- data/lib/ar_query_matchers/queries/load_counter.rb +40 -0
- data/lib/ar_query_matchers/queries/model_name.rb +15 -0
- data/lib/ar_query_matchers/queries/query_counter.rb +95 -0
- data/lib/ar_query_matchers/queries/query_filter.rb +22 -0
- data/lib/ar_query_matchers/queries/table_name.rb +40 -0
- data/lib/ar_query_matchers/queries/update_counter.rb +30 -0
- data/lib/ar_query_matchers/version.rb +5 -0
- metadata +180 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: ac905a2afca1579c9f829cb80ec5ea9db151311b7970e3a35f90528d0d9524b0
|
4
|
+
data.tar.gz: 5f1413a65d8276a0b282070be3d86f8a910224e51ac8fff893d3882c62986f25
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 8c76ea6089d6e15a9ae2dcd9d07319756caa09d36d3645373a5362b04c677c67432305cd9598368231aaacd872604d73df20ca5c93c4d9e2bcd92ceea1860e90
|
7
|
+
data.tar.gz: ba4fb57beaf019f17ac41fbc66cb974bcb76616db0d0b5ce1a3028e87b66d8cf2f6b954342ee59ca52162debb6e4cf7478ae41266f0fad53c353437fea3849d8
|
data/README.md
ADDED
@@ -0,0 +1,91 @@
|
|
1
|
+
## AR Query Matchers
|
2
|
+
|
3
|
+
These RSpec matchers allows guarding against N+1 queries by specifying
|
4
|
+
exactly how many queries we expect each of our models to perform.
|
5
|
+
|
6
|
+
They also help us reason about the type of record interactions happening in a block of code.
|
7
|
+
|
8
|
+
This pattern is a based on how Rails itself tests queries:
|
9
|
+
https://github.com/rails/rails/blob/ac2bc00482c1cf47a57477edb6ab1426a3ba593c/activerecord/test/cases/test_case.rb#L104-L141
|
10
|
+
|
11
|
+
This module defines a few categories of matchers:
|
12
|
+
- **Create**: Which models are created during a block
|
13
|
+
- **Load**: Which models are fetched during a block
|
14
|
+
- **Update**: Which models are updated during a block
|
15
|
+
|
16
|
+
Each matcher category includes 3 assertions, for example, for the Load category, you could use the following assertions:
|
17
|
+
- **only_load_models**: Strict assertion, not other query is allowed.
|
18
|
+
- **not_load_models**: No models are allowed to be loaded.
|
19
|
+
- **load_models**: Inclusion, other models are allowed to be loaded if not specified in the assertion.
|
20
|
+
|
21
|
+
|
22
|
+
**For example:**
|
23
|
+
|
24
|
+
The following spec will pass only if there are exactly 4 SQL SELECTs that
|
25
|
+
load User records (and 1 for Address, 1 for Payroll) _and_ no other models
|
26
|
+
perform any SELECT queries.
|
27
|
+
```ruby
|
28
|
+
expect { some_code() }.to only_load_models(
|
29
|
+
'User' => 4,
|
30
|
+
'Address' => 1,
|
31
|
+
'Payroll' => 1,
|
32
|
+
)
|
33
|
+
```
|
34
|
+
|
35
|
+
The following spec will pass only if there are exactly no select queries.
|
36
|
+
```ruby
|
37
|
+
expect { some_code() }.to not_load_models
|
38
|
+
```
|
39
|
+
|
40
|
+
The following spec will pass only if there are exactly 4 SQL SELECTs that
|
41
|
+
load User records (and 1 for Address, 1 for Payroll).
|
42
|
+
```ruby
|
43
|
+
expect { some_code() }.to load_models(
|
44
|
+
'User' => 4,
|
45
|
+
'Address' => 1,
|
46
|
+
'Payroll' => 1,
|
47
|
+
)
|
48
|
+
```
|
49
|
+
|
50
|
+
This will show some helpful output on failure:
|
51
|
+
|
52
|
+
```
|
53
|
+
Expected to run queries to load models exactly {"Address"=>1, "Payroll"=>1, "User"=>4} queries, got {"Address"=>1, "Payroll"=>1, "User"=>5}
|
54
|
+
Expectations that differed:
|
55
|
+
User - expected: 4, got: 5 (+1)
|
56
|
+
|
57
|
+
Source lines:
|
58
|
+
User loaded from:
|
59
|
+
4 call: /spec/controllers/employees_controller_spec.rb:128:in 'block (4 levels) in <top (required)>'
|
60
|
+
1 call: /app/models/user.rb:299
|
61
|
+
```
|
62
|
+
|
63
|
+
### High Level Design:
|
64
|
+
The RSpec matcher delegates to the query counters, asserts expectations and formats error messages to provide meaningful failures.
|
65
|
+
|
66
|
+
|
67
|
+
The matchers are pretty simple, and delegate instrumentation into specialized QueryCounter classes.
|
68
|
+
The QueryCounters are different classes instruments a ruby block by listening on all sql, parsing the queries and returning structured data describing the interactions.
|
69
|
+
|
70
|
+
```
|
71
|
+
┌────────────────────────────────────────────────────────────────────────────────────────┐
|
72
|
+
┌─┤expect { Employee.create!() }.to only_create_models('Employee' => 1) │
|
73
|
+
│ └────────────────────────────────────────────────────────────────────────────────────────┘
|
74
|
+
└▶┌────────────────────────────────────────────────────────────────────────────────────────┐
|
75
|
+
┌─┤Queries::CreateCounter.instrument { Employee.create!() } => QueryStats │
|
76
|
+
│ └────────────────────────────────────────────────────────────────────────────────────────┘
|
77
|
+
└▶┌────────────────────────────────────────────────────────────────────────────────────────┐
|
78
|
+
│QueryCounter.new(CreateQueryFilter.new).instrument { Employee.create!() } => QueryStats │
|
79
|
+
└────────────────────────────────────────────────────────────────────────────────────────┘
|
80
|
+
```
|
81
|
+
|
82
|
+
For more information, see:
|
83
|
+
1. `ArQueryMatchers::Queries::QueryCounter`
|
84
|
+
2. `ArQueryMatchers::Queries::CreateCounter`
|
85
|
+
3. `ArQueryMatchers::Queries::LoadCounter`
|
86
|
+
4. `ArQueryMatchers::Queries::UpdateCounter`
|
87
|
+
|
88
|
+
### Known problems
|
89
|
+
- The Rails 4 `ActiveRecord::Base#pluck` method doesn't issue a
|
90
|
+
`Load` or `Exists` named query and therefore we don't capture the counts with
|
91
|
+
this tool. This may be fixed in Rails 5/6.
|
@@ -0,0 +1,241 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'ar_query_matchers/queries/create_counter'
|
4
|
+
require 'ar_query_matchers/queries/load_counter'
|
5
|
+
require 'ar_query_matchers/queries/update_counter'
|
6
|
+
|
7
|
+
module ArQueryMatchers
|
8
|
+
module ArQueryMatchers
|
9
|
+
module CreateModels
|
10
|
+
# The following will succeed:
|
11
|
+
# expect {
|
12
|
+
# WcRiskClass.create
|
13
|
+
# WcRiskClass.create
|
14
|
+
# Company.last
|
15
|
+
# }.to only_create_models(
|
16
|
+
# 'WcRiskClass' => 2,
|
17
|
+
# )
|
18
|
+
#
|
19
|
+
RSpec::Matchers.define(:only_create_models) do |expected = {}|
|
20
|
+
include MatcherConfiguration
|
21
|
+
include MatcherErrors
|
22
|
+
|
23
|
+
match do |block|
|
24
|
+
@query_stats = Queries::CreateCounter.instrument(&block)
|
25
|
+
expected == @query_stats.query_counts
|
26
|
+
end
|
27
|
+
|
28
|
+
def failure_text
|
29
|
+
expectation_failed_message('create')
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
# The following will not succeed because the code creates models:
|
34
|
+
#
|
35
|
+
# expect {
|
36
|
+
# WcRiskClass.create
|
37
|
+
# WcRiskClass.create
|
38
|
+
# Company.last
|
39
|
+
# }.to not_create_any_models
|
40
|
+
#
|
41
|
+
RSpec::Matchers.define(:not_create_any_models) do
|
42
|
+
include MatcherConfiguration
|
43
|
+
include MatcherErrors
|
44
|
+
|
45
|
+
match do |block|
|
46
|
+
@query_stats = Queries::CreateCounter.instrument(&block)
|
47
|
+
@query_stats.query_counts.empty?
|
48
|
+
end
|
49
|
+
|
50
|
+
def failure_text
|
51
|
+
no_queries_fail_message('create')
|
52
|
+
end
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
module LoadModels
|
57
|
+
# The following will fail because the call to `User` is not expected, even
|
58
|
+
# though the Payroll count is correct:
|
59
|
+
#
|
60
|
+
# expect {
|
61
|
+
# Payroll.count
|
62
|
+
# Payroll.count
|
63
|
+
# User.count
|
64
|
+
# }.to only_load_models(
|
65
|
+
# 'Payroll' => 2,
|
66
|
+
# )
|
67
|
+
#
|
68
|
+
# The following will succeed because the counts are exact:
|
69
|
+
#
|
70
|
+
# expect {
|
71
|
+
# Payroll.count
|
72
|
+
# Payroll.count
|
73
|
+
# User.count
|
74
|
+
# }.to only_load_models(
|
75
|
+
# 'Payroll' => 2,
|
76
|
+
# 'User' => 1,
|
77
|
+
# )
|
78
|
+
#
|
79
|
+
RSpec::Matchers.define(:only_load_models) do |expected = {}|
|
80
|
+
include MatcherConfiguration
|
81
|
+
include MatcherErrors
|
82
|
+
|
83
|
+
match do |block|
|
84
|
+
@query_stats = Queries::LoadCounter.instrument(&block)
|
85
|
+
expected == @query_stats.query_counts
|
86
|
+
end
|
87
|
+
|
88
|
+
def failure_text
|
89
|
+
expectation_failed_message('load')
|
90
|
+
end
|
91
|
+
end
|
92
|
+
|
93
|
+
RSpec::Matchers.define(:not_load_any_models) do
|
94
|
+
include MatcherConfiguration
|
95
|
+
include MatcherErrors
|
96
|
+
|
97
|
+
match do |block|
|
98
|
+
@query_stats = Queries::LoadCounter.instrument(&block)
|
99
|
+
@query_stats.query_counts.empty?
|
100
|
+
end
|
101
|
+
|
102
|
+
def failure_text
|
103
|
+
no_queries_fail_message('load')
|
104
|
+
end
|
105
|
+
end
|
106
|
+
|
107
|
+
# The following will succeed because `load_models` allows any value for
|
108
|
+
# models not specified:
|
109
|
+
#
|
110
|
+
# expect {
|
111
|
+
# Payroll.count
|
112
|
+
# Payroll.count
|
113
|
+
# User.count
|
114
|
+
# }.to load_models(
|
115
|
+
# 'Payroll' => 2,
|
116
|
+
# )
|
117
|
+
RSpec::Matchers.define(:load_models) do |expected = {}|
|
118
|
+
include MatcherConfiguration
|
119
|
+
include MatcherErrors
|
120
|
+
|
121
|
+
match do |block|
|
122
|
+
@query_stats = Queries::LoadCounter.instrument(&block)
|
123
|
+
expected.each do |model_name, expected_count|
|
124
|
+
expect(expected_count).to eq @query_stats.query_counts[model_name]
|
125
|
+
end
|
126
|
+
end
|
127
|
+
|
128
|
+
def failure_text
|
129
|
+
expectation_failed_message('load')
|
130
|
+
end
|
131
|
+
end
|
132
|
+
end
|
133
|
+
|
134
|
+
class UpdateModels
|
135
|
+
# The following will succeed:
|
136
|
+
#
|
137
|
+
# expect {
|
138
|
+
# WcRiskClass.last.update_attributes(id: 9999)
|
139
|
+
# }.to only_update_models(
|
140
|
+
# 'WcRiskClass' => 1,
|
141
|
+
# )
|
142
|
+
#
|
143
|
+
RSpec::Matchers.define(:only_update_models) do |expected = {}|
|
144
|
+
include MatcherConfiguration
|
145
|
+
include MatcherErrors
|
146
|
+
|
147
|
+
match do |block|
|
148
|
+
@query_stats = Queries::UpdateCounter.instrument(&block)
|
149
|
+
expected == @query_stats.query_counts
|
150
|
+
end
|
151
|
+
|
152
|
+
def failure_text
|
153
|
+
expectation_failed_message('update')
|
154
|
+
end
|
155
|
+
end
|
156
|
+
|
157
|
+
# The following will not succeed because the code updates models:
|
158
|
+
#
|
159
|
+
# expect {
|
160
|
+
# WcRiskClass.last.update_attributes(id: 9999)
|
161
|
+
# }.to not_update_any_models
|
162
|
+
#
|
163
|
+
RSpec::Matchers.define(:not_update_any_models) do
|
164
|
+
include MatcherConfiguration
|
165
|
+
include MatcherErrors
|
166
|
+
|
167
|
+
match do |block|
|
168
|
+
@query_stats = Queries::UpdateCounter.instrument(&block)
|
169
|
+
@query_stats.query_counts.empty?
|
170
|
+
end
|
171
|
+
|
172
|
+
def failure_text
|
173
|
+
no_queries_fail_message('update')
|
174
|
+
end
|
175
|
+
end
|
176
|
+
end
|
177
|
+
|
178
|
+
# Shared methods that are included in the matchers.
|
179
|
+
# They configure it and ensure we get consistent and human readable error messages
|
180
|
+
module MatcherConfiguration
|
181
|
+
def self.included(base)
|
182
|
+
if base.respond_to?(:failure_message)
|
183
|
+
base.failure_message do |_actual|
|
184
|
+
failure_text
|
185
|
+
end
|
186
|
+
else
|
187
|
+
base.failure_message_for_should do |_actual|
|
188
|
+
failure_text
|
189
|
+
end
|
190
|
+
end
|
191
|
+
end
|
192
|
+
|
193
|
+
def supports_block_expectations?
|
194
|
+
true
|
195
|
+
end
|
196
|
+
end
|
197
|
+
|
198
|
+
module MatcherErrors
|
199
|
+
# Show the difference between expected and actual values with one value
|
200
|
+
# per line. This is done by hand because as of this writing the author
|
201
|
+
# doesn't understand how RSpec does its nice hash diff printing.
|
202
|
+
def difference(keys)
|
203
|
+
max_key_length = keys.reduce(0) { |max, key| [max, key.size].max }
|
204
|
+
|
205
|
+
keys.map do |key|
|
206
|
+
left = expected.fetch(key, 0)
|
207
|
+
right = @query_stats.queries.fetch(key, {}).fetch(:count, 0)
|
208
|
+
|
209
|
+
diff = "#{'+' if right > left}#{right - left}"
|
210
|
+
|
211
|
+
"#{key.rjust(max_key_length, ' ')} – expected: #{left}, got: #{right} (#{diff})"
|
212
|
+
end.compact
|
213
|
+
end
|
214
|
+
|
215
|
+
def source_lines(keys)
|
216
|
+
line_frequency = @query_stats.query_lines_by_frequency
|
217
|
+
keys_with_source_lines = keys.select { |key| line_frequency[key].present? }
|
218
|
+
keys_with_source_lines.map do |key|
|
219
|
+
source_lines = line_frequency[key].sort_by(&:last).reverse # Most frequent on top
|
220
|
+
next if source_lines.blank?
|
221
|
+
|
222
|
+
[
|
223
|
+
" #{key}"
|
224
|
+
] + source_lines.map { |line, count| " #{count} #{'call'.pluralize(count)}: #{line}" } + [
|
225
|
+
''
|
226
|
+
]
|
227
|
+
end
|
228
|
+
end
|
229
|
+
|
230
|
+
def no_queries_fail_message(crud_operation)
|
231
|
+
"Expected ActiveRecord to not #{crud_operation} any records, got #{@query_stats.query_counts}\n\nWhere unexpected queries came from:\n\n#{source_lines(@query_stats.query_counts.keys).join("\n")}"
|
232
|
+
end
|
233
|
+
|
234
|
+
def expectation_failed_message(crud_operation)
|
235
|
+
all_model_names = expected.keys + @query_stats.queries.keys
|
236
|
+
model_names_with_wrong_count = all_model_names.reject { |key| expected[key] == @query_stats.queries[key][:count] }.uniq
|
237
|
+
"Expected ActiveRecord to #{crud_operation} #{expected}, got #{@query_stats.query_counts}\nExpectations that differed:\n#{difference(model_names_with_wrong_count).join("\n")}\n\nWhere unexpected queries came from:\n\n#{source_lines(model_names_with_wrong_count).join("\n")}"
|
238
|
+
end
|
239
|
+
end
|
240
|
+
end
|
241
|
+
end
|
@@ -0,0 +1,31 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative './query_counter'
|
4
|
+
require_relative './table_name'
|
5
|
+
require_relative './query_filter'
|
6
|
+
|
7
|
+
module ArQueryMatchers
|
8
|
+
module Queries
|
9
|
+
# A specialized QueryCounter for "creates".
|
10
|
+
# It uses a simple regex to identify and parse sql INSERT queries.
|
11
|
+
# For more information, see the QueryCounter class.
|
12
|
+
class CreateCounter
|
13
|
+
def self.instrument(&block)
|
14
|
+
QueryCounter.new(CreateQueryFilter.new).instrument(&block)
|
15
|
+
end
|
16
|
+
|
17
|
+
class CreateQueryFilter < QueryFilter
|
18
|
+
# Matches unnamed SQL operations like the following:
|
19
|
+
# "INSERT INTO `company_approval_details` ..."
|
20
|
+
TABLE_NAME_SQL_PATTERN = /INSERT INTO [`"](?<table_name>[^`"]+)[`"]/.freeze
|
21
|
+
|
22
|
+
def filter_map(_name, sql)
|
23
|
+
# for inserts, name is always 'SQL', we have to rely on pattern matching the query string.
|
24
|
+
select_from_table = sql.match(TABLE_NAME_SQL_PATTERN)
|
25
|
+
|
26
|
+
TableName.new(select_from_table[:table_name]) if select_from_table
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
@@ -0,0 +1,40 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative './query_counter'
|
4
|
+
require_relative './table_name'
|
5
|
+
require_relative './model_name'
|
6
|
+
require_relative './query_filter'
|
7
|
+
|
8
|
+
module ArQueryMatchers
|
9
|
+
module Queries
|
10
|
+
# A specialized QueryCounter for "loads".
|
11
|
+
# For more information, see the QueryCounter class.
|
12
|
+
class LoadCounter
|
13
|
+
def self.instrument(&block)
|
14
|
+
QueryCounter.new(LoadQueryFilter.new).instrument(&block)
|
15
|
+
end
|
16
|
+
|
17
|
+
class LoadQueryFilter < Queries::QueryFilter
|
18
|
+
# Matches named SQL operations like the following:
|
19
|
+
# 'User Load'
|
20
|
+
MODEL_LOAD_PATTERN = /\A(?<model_name>[\w:]+) (Load|Exists)\Z/.freeze
|
21
|
+
|
22
|
+
# Matches unnamed SQL operations like the following:
|
23
|
+
# "SELECT COUNT(*) FROM `users` ..."
|
24
|
+
MODEL_SQL_PATTERN = /SELECT .* FROM [`"](?<table_name>[^`"]+)[`"]/.freeze
|
25
|
+
|
26
|
+
def filter_map(name, sql)
|
27
|
+
# First check for a `SELECT * FROM` query that ActiveRecord has
|
28
|
+
# helpfully named for us in the payload
|
29
|
+
match = name.match(MODEL_LOAD_PATTERN)
|
30
|
+
return ModelName.new(match[:model_name]) if match
|
31
|
+
|
32
|
+
# Fall back to pattern-matching on the table name in a COUNT and looking
|
33
|
+
# up the table name from ActiveRecord's loaded descendants.
|
34
|
+
select_from_table = sql.match(MODEL_SQL_PATTERN)
|
35
|
+
TableName.new(select_from_table[:table_name]) if select_from_table
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ArQueryMatchers
|
4
|
+
module Queries
|
5
|
+
# An instance of this class is one of the values that could be returned from the QueryFilter#filter_map.
|
6
|
+
# its accepts a name of an ActiveRecord model, for example: 'Company'.
|
7
|
+
class ModelName
|
8
|
+
attr_reader(:model_name)
|
9
|
+
|
10
|
+
def initialize(model_name)
|
11
|
+
@model_name = model_name
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
@@ -0,0 +1,95 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ArQueryMatchers
|
4
|
+
module Queries
|
5
|
+
# The QueryCounter instruments a ruby block and collect stats on the
|
6
|
+
# SQL queries performed during its execution.
|
7
|
+
#
|
8
|
+
# It's "generic" meaning it requires an instance of a QueryFilter to operate.
|
9
|
+
# The QueryFilter is an interface that both filters SQL statements and maps them to an ActiveRecord model, sort of a Enumerator#filter_map.
|
10
|
+
#
|
11
|
+
# This class is meant to be wrapped by another class that provides a concrete QueryFilter implementation.
|
12
|
+
# For example, you could implement a SelectQueryFilter using it:
|
13
|
+
# class SelectQueryCounter
|
14
|
+
# class SelectFilter < Queries::QueryFilter
|
15
|
+
# def extract(_name, sql)
|
16
|
+
# select_from_table = sql.match(/SELECT .* FROM [`"](?<table_name>[^`"]+)[`"]/)
|
17
|
+
# Queries::TableName.new(select_from_table[:table_name]) if select_from_table
|
18
|
+
# end
|
19
|
+
# end
|
20
|
+
#
|
21
|
+
# def self.instrument(&block)
|
22
|
+
# QueryCounter.new(SelectFilter.new).instrument(&block)
|
23
|
+
# end
|
24
|
+
# end
|
25
|
+
#
|
26
|
+
# stats = SelectQueryCounter.instrument do
|
27
|
+
# Company.first
|
28
|
+
# Employee.last(100)
|
29
|
+
# User.find(1)
|
30
|
+
# User.find(2)
|
31
|
+
# end
|
32
|
+
#
|
33
|
+
# stats.query_counts == { 'Company' => 1, Employee => '1', 'User' => 2 }
|
34
|
+
class QueryCounter
|
35
|
+
class QueryStats
|
36
|
+
def initialize(queries)
|
37
|
+
@queries = queries
|
38
|
+
end
|
39
|
+
|
40
|
+
attr_reader(:queries)
|
41
|
+
|
42
|
+
# @return [Hash] of model name to query count, for example: { 'Company' => 5}
|
43
|
+
def query_counts
|
44
|
+
Hash[*queries.reduce({}) { |acc, (model_name, data)| acc.update model_name => data[:count] }.sort_by(&:first).flatten]
|
45
|
+
end
|
46
|
+
|
47
|
+
# @return [Hash] of line in the source code to its frequency
|
48
|
+
def query_lines_by_frequency
|
49
|
+
queries.reduce({}) do |lines, (model_name, data)|
|
50
|
+
frequencies = data[:lines].reduce(Hash.new { |h, k| h[k] = 0 }) do |freq, line|
|
51
|
+
freq.update line => freq[line] + 1
|
52
|
+
end
|
53
|
+
lines.update model_name => frequencies
|
54
|
+
end
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
def initialize(query_filter)
|
59
|
+
@query_filter = query_filter
|
60
|
+
end
|
61
|
+
|
62
|
+
# @param [block] block to instrument
|
63
|
+
# @return [QueryStats] stats about all the SQL queries executed during the block
|
64
|
+
def instrument(&block)
|
65
|
+
queries = Hash.new { |h, k| h[k] = { count: 0, lines: [] } }
|
66
|
+
ActiveSupport::Notifications.subscribed(to_proc(queries), 'sql.active_record', &block)
|
67
|
+
QueryStats.new(queries)
|
68
|
+
end
|
69
|
+
|
70
|
+
private
|
71
|
+
|
72
|
+
# The 'marginalia' gem adds a line from the backtrace to the SQL query in
|
73
|
+
# the form of a comment.
|
74
|
+
MARGINALIA_SQL_COMMENT_PATTERN = %r{/*line:(?<line>.*)'*/}.freeze
|
75
|
+
private_constant :MARGINALIA_SQL_COMMENT_PATTERN
|
76
|
+
|
77
|
+
def to_proc(queries)
|
78
|
+
lambda do |_name, _start, _finish, _message_id, payload|
|
79
|
+
return if payload[:cached]
|
80
|
+
|
81
|
+
# Given a `sql.active_record` event, figure out which model is being
|
82
|
+
# accessed. Some of the simpler queries have a :ame key that makes this
|
83
|
+
# really easy. Others require parsing the SQL by hand.
|
84
|
+
model_name = @query_filter.filter_map(payload[:name] || '', payload[:sql] || '')&.model_name
|
85
|
+
|
86
|
+
if model_name
|
87
|
+
comment = payload[:sql].match(MARGINALIA_SQL_COMMENT_PATTERN)
|
88
|
+
queries[model_name][:lines] << comment[:line] if comment
|
89
|
+
queries[model_name][:count] += 1
|
90
|
+
end
|
91
|
+
end
|
92
|
+
end
|
93
|
+
end
|
94
|
+
end
|
95
|
+
end
|
@@ -0,0 +1,22 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ArQueryMatchers
|
4
|
+
module Queries
|
5
|
+
# An instance of this interface must be provided to the QueryCounter class.
|
6
|
+
# it allows one to customize which queries it wants to capture.
|
7
|
+
class QueryFilter
|
8
|
+
# @param _name [String] the name of the ActiveRecord operation (this is sometimes garbage)
|
9
|
+
# For example: "User Load"
|
10
|
+
#
|
11
|
+
# @param _sql [String] the sql query that was executed
|
12
|
+
# For example: "SELECT `users`.* FROM `users` .."
|
13
|
+
#
|
14
|
+
# @return nil or an instance which responds to #model_name (see TableName or ModelName)
|
15
|
+
# By returning nil we omit the query
|
16
|
+
# By not returning nil, we are associating this query with a model_name.
|
17
|
+
def filter_map(_name, _sql)
|
18
|
+
raise NotImplementedError
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
@@ -0,0 +1,40 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ArQueryMatchers
|
4
|
+
module Queries
|
5
|
+
# An instance of this class is one of the values that could be returned from the QueryFilter#filter_map.
|
6
|
+
# its accepts a name of a table in the database, for example: 'companies'.
|
7
|
+
#
|
8
|
+
# #model_name would transform the table name ('companies') into the ActiveRecord model name ('Company').
|
9
|
+
# It relies on the class to be loaded in the global namespace, which should be the case if we issues a query through ActiveRecord.
|
10
|
+
class TableName
|
11
|
+
def initialize(table_name)
|
12
|
+
@table_name = table_name
|
13
|
+
end
|
14
|
+
|
15
|
+
def model_name
|
16
|
+
active_record_class_for_table(@table_name)&.name
|
17
|
+
end
|
18
|
+
|
19
|
+
private
|
20
|
+
|
21
|
+
# We recalculate this each time because new classes might get loaded between queries
|
22
|
+
def active_record_class_for_table(table_name)
|
23
|
+
# Retrieve all (known) subclasses of ActiveRecord::Base
|
24
|
+
klasses = ActiveRecord::Base.descendants.reject(&:abstract_class)
|
25
|
+
|
26
|
+
# Group them by their table_names
|
27
|
+
tables = klasses.each_with_object(Hash.new { |k, v| k[v] = [] }) do |klass, accumulator|
|
28
|
+
accumulator[klass.table_name] << klass
|
29
|
+
end
|
30
|
+
# Structure:
|
31
|
+
# { 'users' => [User, AtoUser],
|
32
|
+
# 'employees => [Employee, PandaFlows::StateFields] }
|
33
|
+
|
34
|
+
# Of all the models that share the same table name sort them by their
|
35
|
+
# relative ancestry and pick the one that all the rest inherit from
|
36
|
+
tables[table_name].min_by { |a, b| a.ancestors.include?(b) }
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
@@ -0,0 +1,30 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative './query_counter'
|
4
|
+
require_relative './table_name'
|
5
|
+
require_relative './query_filter'
|
6
|
+
|
7
|
+
module ArQueryMatchers
|
8
|
+
module Queries
|
9
|
+
# A specialized QueryCounter for "updates".
|
10
|
+
# It uses a simple regex to identify and parse sql UPDATE queries.
|
11
|
+
# For more information, see the QueryCounter class.
|
12
|
+
class UpdateCounter
|
13
|
+
def self.instrument(&block)
|
14
|
+
QueryCounter.new(UpdateQueryFilter.new).instrument(&block)
|
15
|
+
end
|
16
|
+
|
17
|
+
class UpdateQueryFilter < QueryFilter
|
18
|
+
# Matches unnamed SQL operations like the following:
|
19
|
+
# "UPDATE `bank_account_verifications` ..."
|
20
|
+
TABLE_NAME_SQL_PATTERN = /UPDATE [`"](?<table_name>[^`"]+)[`"]/.freeze
|
21
|
+
|
22
|
+
def filter_map(_name, sql)
|
23
|
+
# for updates, name is always 'SQL', we have to rely on pattern matching on the query string instead.
|
24
|
+
select_from_table = sql.match(TABLE_NAME_SQL_PATTERN)
|
25
|
+
TableName.new(select_from_table[:table_name]) if select_from_table
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
metadata
ADDED
@@ -0,0 +1,180 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: ar-query-matchers
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Matan Zruya
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2019-09-14 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: activerecord
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '4.0'
|
20
|
+
- - "<="
|
21
|
+
- !ruby/object:Gem::Version
|
22
|
+
version: '6.0'
|
23
|
+
type: :runtime
|
24
|
+
prerelease: false
|
25
|
+
version_requirements: !ruby/object:Gem::Requirement
|
26
|
+
requirements:
|
27
|
+
- - ">="
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: '4.0'
|
30
|
+
- - "<="
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '6.0'
|
33
|
+
- !ruby/object:Gem::Dependency
|
34
|
+
name: activesupport
|
35
|
+
requirement: !ruby/object:Gem::Requirement
|
36
|
+
requirements:
|
37
|
+
- - ">="
|
38
|
+
- !ruby/object:Gem::Version
|
39
|
+
version: '4.0'
|
40
|
+
- - "<="
|
41
|
+
- !ruby/object:Gem::Version
|
42
|
+
version: '6.0'
|
43
|
+
type: :runtime
|
44
|
+
prerelease: false
|
45
|
+
version_requirements: !ruby/object:Gem::Requirement
|
46
|
+
requirements:
|
47
|
+
- - ">="
|
48
|
+
- !ruby/object:Gem::Version
|
49
|
+
version: '4.0'
|
50
|
+
- - "<="
|
51
|
+
- !ruby/object:Gem::Version
|
52
|
+
version: '6.0'
|
53
|
+
- !ruby/object:Gem::Dependency
|
54
|
+
name: rspec
|
55
|
+
requirement: !ruby/object:Gem::Requirement
|
56
|
+
requirements:
|
57
|
+
- - "~>"
|
58
|
+
- !ruby/object:Gem::Version
|
59
|
+
version: '3.0'
|
60
|
+
type: :runtime
|
61
|
+
prerelease: false
|
62
|
+
version_requirements: !ruby/object:Gem::Requirement
|
63
|
+
requirements:
|
64
|
+
- - "~>"
|
65
|
+
- !ruby/object:Gem::Version
|
66
|
+
version: '3.0'
|
67
|
+
- !ruby/object:Gem::Dependency
|
68
|
+
name: bundler
|
69
|
+
requirement: !ruby/object:Gem::Requirement
|
70
|
+
requirements:
|
71
|
+
- - ">="
|
72
|
+
- !ruby/object:Gem::Version
|
73
|
+
version: '0'
|
74
|
+
type: :development
|
75
|
+
prerelease: false
|
76
|
+
version_requirements: !ruby/object:Gem::Requirement
|
77
|
+
requirements:
|
78
|
+
- - ">="
|
79
|
+
- !ruby/object:Gem::Version
|
80
|
+
version: '0'
|
81
|
+
- !ruby/object:Gem::Dependency
|
82
|
+
name: rake
|
83
|
+
requirement: !ruby/object:Gem::Requirement
|
84
|
+
requirements:
|
85
|
+
- - "~>"
|
86
|
+
- !ruby/object:Gem::Version
|
87
|
+
version: '10.0'
|
88
|
+
type: :development
|
89
|
+
prerelease: false
|
90
|
+
version_requirements: !ruby/object:Gem::Requirement
|
91
|
+
requirements:
|
92
|
+
- - "~>"
|
93
|
+
- !ruby/object:Gem::Version
|
94
|
+
version: '10.0'
|
95
|
+
- !ruby/object:Gem::Dependency
|
96
|
+
name: rspec
|
97
|
+
requirement: !ruby/object:Gem::Requirement
|
98
|
+
requirements:
|
99
|
+
- - "~>"
|
100
|
+
- !ruby/object:Gem::Version
|
101
|
+
version: '3.0'
|
102
|
+
type: :development
|
103
|
+
prerelease: false
|
104
|
+
version_requirements: !ruby/object:Gem::Requirement
|
105
|
+
requirements:
|
106
|
+
- - "~>"
|
107
|
+
- !ruby/object:Gem::Version
|
108
|
+
version: '3.0'
|
109
|
+
- !ruby/object:Gem::Dependency
|
110
|
+
name: rubocop
|
111
|
+
requirement: !ruby/object:Gem::Requirement
|
112
|
+
requirements:
|
113
|
+
- - ">="
|
114
|
+
- !ruby/object:Gem::Version
|
115
|
+
version: '0'
|
116
|
+
type: :development
|
117
|
+
prerelease: false
|
118
|
+
version_requirements: !ruby/object:Gem::Requirement
|
119
|
+
requirements:
|
120
|
+
- - ">="
|
121
|
+
- !ruby/object:Gem::Version
|
122
|
+
version: '0'
|
123
|
+
- !ruby/object:Gem::Dependency
|
124
|
+
name: sqlite3
|
125
|
+
requirement: !ruby/object:Gem::Requirement
|
126
|
+
requirements:
|
127
|
+
- - "~>"
|
128
|
+
- !ruby/object:Gem::Version
|
129
|
+
version: '1.4'
|
130
|
+
type: :development
|
131
|
+
prerelease: false
|
132
|
+
version_requirements: !ruby/object:Gem::Requirement
|
133
|
+
requirements:
|
134
|
+
- - "~>"
|
135
|
+
- !ruby/object:Gem::Version
|
136
|
+
version: '1.4'
|
137
|
+
description: ''
|
138
|
+
email:
|
139
|
+
- mzruya@gmail.com
|
140
|
+
executables: []
|
141
|
+
extensions: []
|
142
|
+
extra_rdoc_files: []
|
143
|
+
files:
|
144
|
+
- README.md
|
145
|
+
- lib/ar_query_matchers.rb
|
146
|
+
- lib/ar_query_matchers/queries/create_counter.rb
|
147
|
+
- lib/ar_query_matchers/queries/load_counter.rb
|
148
|
+
- lib/ar_query_matchers/queries/model_name.rb
|
149
|
+
- lib/ar_query_matchers/queries/query_counter.rb
|
150
|
+
- lib/ar_query_matchers/queries/query_filter.rb
|
151
|
+
- lib/ar_query_matchers/queries/table_name.rb
|
152
|
+
- lib/ar_query_matchers/queries/update_counter.rb
|
153
|
+
- lib/ar_query_matchers/version.rb
|
154
|
+
homepage: https://github.com/Gusto/ar-query-matchers
|
155
|
+
licenses:
|
156
|
+
- MIT
|
157
|
+
metadata:
|
158
|
+
homepage_uri: https://github.com/Gusto/ar-query-matchers
|
159
|
+
source_code_uri: https://github.com/Gusto/ar-query-matchers
|
160
|
+
changelog_uri: https://github.com/Gusto/ar-query-matchers
|
161
|
+
post_install_message:
|
162
|
+
rdoc_options: []
|
163
|
+
require_paths:
|
164
|
+
- lib
|
165
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
166
|
+
requirements:
|
167
|
+
- - ">="
|
168
|
+
- !ruby/object:Gem::Version
|
169
|
+
version: '0'
|
170
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
171
|
+
requirements:
|
172
|
+
- - ">="
|
173
|
+
- !ruby/object:Gem::Version
|
174
|
+
version: '0'
|
175
|
+
requirements: []
|
176
|
+
rubygems_version: 3.0.3
|
177
|
+
signing_key:
|
178
|
+
specification_version: 4
|
179
|
+
summary: ''
|
180
|
+
test_files: []
|