ar-query-matchers 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/README.md +91 -0
- data/lib/ar_query_matchers.rb +241 -0
- data/lib/ar_query_matchers/queries/create_counter.rb +31 -0
- data/lib/ar_query_matchers/queries/load_counter.rb +40 -0
- data/lib/ar_query_matchers/queries/model_name.rb +15 -0
- data/lib/ar_query_matchers/queries/query_counter.rb +95 -0
- data/lib/ar_query_matchers/queries/query_filter.rb +22 -0
- data/lib/ar_query_matchers/queries/table_name.rb +40 -0
- data/lib/ar_query_matchers/queries/update_counter.rb +30 -0
- data/lib/ar_query_matchers/version.rb +5 -0
- metadata +180 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: ac905a2afca1579c9f829cb80ec5ea9db151311b7970e3a35f90528d0d9524b0
|
4
|
+
data.tar.gz: 5f1413a65d8276a0b282070be3d86f8a910224e51ac8fff893d3882c62986f25
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 8c76ea6089d6e15a9ae2dcd9d07319756caa09d36d3645373a5362b04c677c67432305cd9598368231aaacd872604d73df20ca5c93c4d9e2bcd92ceea1860e90
|
7
|
+
data.tar.gz: ba4fb57beaf019f17ac41fbc66cb974bcb76616db0d0b5ce1a3028e87b66d8cf2f6b954342ee59ca52162debb6e4cf7478ae41266f0fad53c353437fea3849d8
|
data/README.md
ADDED
@@ -0,0 +1,91 @@
|
|
1
|
+
## AR Query Matchers
|
2
|
+
|
3
|
+
These RSpec matchers allows guarding against N+1 queries by specifying
|
4
|
+
exactly how many queries we expect each of our models to perform.
|
5
|
+
|
6
|
+
They also help us reason about the type of record interactions happening in a block of code.
|
7
|
+
|
8
|
+
This pattern is a based on how Rails itself tests queries:
|
9
|
+
https://github.com/rails/rails/blob/ac2bc00482c1cf47a57477edb6ab1426a3ba593c/activerecord/test/cases/test_case.rb#L104-L141
|
10
|
+
|
11
|
+
This module defines a few categories of matchers:
|
12
|
+
- **Create**: Which models are created during a block
|
13
|
+
- **Load**: Which models are fetched during a block
|
14
|
+
- **Update**: Which models are updated during a block
|
15
|
+
|
16
|
+
Each matcher category includes 3 assertions, for example, for the Load category, you could use the following assertions:
|
17
|
+
- **only_load_models**: Strict assertion, not other query is allowed.
|
18
|
+
- **not_load_models**: No models are allowed to be loaded.
|
19
|
+
- **load_models**: Inclusion, other models are allowed to be loaded if not specified in the assertion.
|
20
|
+
|
21
|
+
|
22
|
+
**For example:**
|
23
|
+
|
24
|
+
The following spec will pass only if there are exactly 4 SQL SELECTs that
|
25
|
+
load User records (and 1 for Address, 1 for Payroll) _and_ no other models
|
26
|
+
perform any SELECT queries.
|
27
|
+
```ruby
|
28
|
+
expect { some_code() }.to only_load_models(
|
29
|
+
'User' => 4,
|
30
|
+
'Address' => 1,
|
31
|
+
'Payroll' => 1,
|
32
|
+
)
|
33
|
+
```
|
34
|
+
|
35
|
+
The following spec will pass only if there are exactly no select queries.
|
36
|
+
```ruby
|
37
|
+
expect { some_code() }.to not_load_models
|
38
|
+
```
|
39
|
+
|
40
|
+
The following spec will pass only if there are exactly 4 SQL SELECTs that
|
41
|
+
load User records (and 1 for Address, 1 for Payroll).
|
42
|
+
```ruby
|
43
|
+
expect { some_code() }.to load_models(
|
44
|
+
'User' => 4,
|
45
|
+
'Address' => 1,
|
46
|
+
'Payroll' => 1,
|
47
|
+
)
|
48
|
+
```
|
49
|
+
|
50
|
+
This will show some helpful output on failure:
|
51
|
+
|
52
|
+
```
|
53
|
+
Expected to run queries to load models exactly {"Address"=>1, "Payroll"=>1, "User"=>4} queries, got {"Address"=>1, "Payroll"=>1, "User"=>5}
|
54
|
+
Expectations that differed:
|
55
|
+
User - expected: 4, got: 5 (+1)
|
56
|
+
|
57
|
+
Source lines:
|
58
|
+
User loaded from:
|
59
|
+
4 call: /spec/controllers/employees_controller_spec.rb:128:in 'block (4 levels) in <top (required)>'
|
60
|
+
1 call: /app/models/user.rb:299
|
61
|
+
```
|
62
|
+
|
63
|
+
### High Level Design:
|
64
|
+
The RSpec matcher delegates to the query counters, asserts expectations and formats error messages to provide meaningful failures.
|
65
|
+
|
66
|
+
|
67
|
+
The matchers are pretty simple, and delegate instrumentation into specialized QueryCounter classes.
|
68
|
+
The QueryCounters are different classes instruments a ruby block by listening on all sql, parsing the queries and returning structured data describing the interactions.
|
69
|
+
|
70
|
+
```
|
71
|
+
┌────────────────────────────────────────────────────────────────────────────────────────┐
|
72
|
+
┌─┤expect { Employee.create!() }.to only_create_models('Employee' => 1) │
|
73
|
+
│ └────────────────────────────────────────────────────────────────────────────────────────┘
|
74
|
+
└▶┌────────────────────────────────────────────────────────────────────────────────────────┐
|
75
|
+
┌─┤Queries::CreateCounter.instrument { Employee.create!() } => QueryStats │
|
76
|
+
│ └────────────────────────────────────────────────────────────────────────────────────────┘
|
77
|
+
└▶┌────────────────────────────────────────────────────────────────────────────────────────┐
|
78
|
+
│QueryCounter.new(CreateQueryFilter.new).instrument { Employee.create!() } => QueryStats │
|
79
|
+
└────────────────────────────────────────────────────────────────────────────────────────┘
|
80
|
+
```
|
81
|
+
|
82
|
+
For more information, see:
|
83
|
+
1. `ArQueryMatchers::Queries::QueryCounter`
|
84
|
+
2. `ArQueryMatchers::Queries::CreateCounter`
|
85
|
+
3. `ArQueryMatchers::Queries::LoadCounter`
|
86
|
+
4. `ArQueryMatchers::Queries::UpdateCounter`
|
87
|
+
|
88
|
+
### Known problems
|
89
|
+
- The Rails 4 `ActiveRecord::Base#pluck` method doesn't issue a
|
90
|
+
`Load` or `Exists` named query and therefore we don't capture the counts with
|
91
|
+
this tool. This may be fixed in Rails 5/6.
|
@@ -0,0 +1,241 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'ar_query_matchers/queries/create_counter'
|
4
|
+
require 'ar_query_matchers/queries/load_counter'
|
5
|
+
require 'ar_query_matchers/queries/update_counter'
|
6
|
+
|
7
|
+
module ArQueryMatchers
|
8
|
+
module ArQueryMatchers
|
9
|
+
module CreateModels
|
10
|
+
# The following will succeed:
|
11
|
+
# expect {
|
12
|
+
# WcRiskClass.create
|
13
|
+
# WcRiskClass.create
|
14
|
+
# Company.last
|
15
|
+
# }.to only_create_models(
|
16
|
+
# 'WcRiskClass' => 2,
|
17
|
+
# )
|
18
|
+
#
|
19
|
+
RSpec::Matchers.define(:only_create_models) do |expected = {}|
|
20
|
+
include MatcherConfiguration
|
21
|
+
include MatcherErrors
|
22
|
+
|
23
|
+
match do |block|
|
24
|
+
@query_stats = Queries::CreateCounter.instrument(&block)
|
25
|
+
expected == @query_stats.query_counts
|
26
|
+
end
|
27
|
+
|
28
|
+
def failure_text
|
29
|
+
expectation_failed_message('create')
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
# The following will not succeed because the code creates models:
|
34
|
+
#
|
35
|
+
# expect {
|
36
|
+
# WcRiskClass.create
|
37
|
+
# WcRiskClass.create
|
38
|
+
# Company.last
|
39
|
+
# }.to not_create_any_models
|
40
|
+
#
|
41
|
+
RSpec::Matchers.define(:not_create_any_models) do
|
42
|
+
include MatcherConfiguration
|
43
|
+
include MatcherErrors
|
44
|
+
|
45
|
+
match do |block|
|
46
|
+
@query_stats = Queries::CreateCounter.instrument(&block)
|
47
|
+
@query_stats.query_counts.empty?
|
48
|
+
end
|
49
|
+
|
50
|
+
def failure_text
|
51
|
+
no_queries_fail_message('create')
|
52
|
+
end
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
module LoadModels
|
57
|
+
# The following will fail because the call to `User` is not expected, even
|
58
|
+
# though the Payroll count is correct:
|
59
|
+
#
|
60
|
+
# expect {
|
61
|
+
# Payroll.count
|
62
|
+
# Payroll.count
|
63
|
+
# User.count
|
64
|
+
# }.to only_load_models(
|
65
|
+
# 'Payroll' => 2,
|
66
|
+
# )
|
67
|
+
#
|
68
|
+
# The following will succeed because the counts are exact:
|
69
|
+
#
|
70
|
+
# expect {
|
71
|
+
# Payroll.count
|
72
|
+
# Payroll.count
|
73
|
+
# User.count
|
74
|
+
# }.to only_load_models(
|
75
|
+
# 'Payroll' => 2,
|
76
|
+
# 'User' => 1,
|
77
|
+
# )
|
78
|
+
#
|
79
|
+
RSpec::Matchers.define(:only_load_models) do |expected = {}|
|
80
|
+
include MatcherConfiguration
|
81
|
+
include MatcherErrors
|
82
|
+
|
83
|
+
match do |block|
|
84
|
+
@query_stats = Queries::LoadCounter.instrument(&block)
|
85
|
+
expected == @query_stats.query_counts
|
86
|
+
end
|
87
|
+
|
88
|
+
def failure_text
|
89
|
+
expectation_failed_message('load')
|
90
|
+
end
|
91
|
+
end
|
92
|
+
|
93
|
+
RSpec::Matchers.define(:not_load_any_models) do
|
94
|
+
include MatcherConfiguration
|
95
|
+
include MatcherErrors
|
96
|
+
|
97
|
+
match do |block|
|
98
|
+
@query_stats = Queries::LoadCounter.instrument(&block)
|
99
|
+
@query_stats.query_counts.empty?
|
100
|
+
end
|
101
|
+
|
102
|
+
def failure_text
|
103
|
+
no_queries_fail_message('load')
|
104
|
+
end
|
105
|
+
end
|
106
|
+
|
107
|
+
# The following will succeed because `load_models` allows any value for
|
108
|
+
# models not specified:
|
109
|
+
#
|
110
|
+
# expect {
|
111
|
+
# Payroll.count
|
112
|
+
# Payroll.count
|
113
|
+
# User.count
|
114
|
+
# }.to load_models(
|
115
|
+
# 'Payroll' => 2,
|
116
|
+
# )
|
117
|
+
RSpec::Matchers.define(:load_models) do |expected = {}|
|
118
|
+
include MatcherConfiguration
|
119
|
+
include MatcherErrors
|
120
|
+
|
121
|
+
match do |block|
|
122
|
+
@query_stats = Queries::LoadCounter.instrument(&block)
|
123
|
+
expected.each do |model_name, expected_count|
|
124
|
+
expect(expected_count).to eq @query_stats.query_counts[model_name]
|
125
|
+
end
|
126
|
+
end
|
127
|
+
|
128
|
+
def failure_text
|
129
|
+
expectation_failed_message('load')
|
130
|
+
end
|
131
|
+
end
|
132
|
+
end
|
133
|
+
|
134
|
+
class UpdateModels
|
135
|
+
# The following will succeed:
|
136
|
+
#
|
137
|
+
# expect {
|
138
|
+
# WcRiskClass.last.update_attributes(id: 9999)
|
139
|
+
# }.to only_update_models(
|
140
|
+
# 'WcRiskClass' => 1,
|
141
|
+
# )
|
142
|
+
#
|
143
|
+
RSpec::Matchers.define(:only_update_models) do |expected = {}|
|
144
|
+
include MatcherConfiguration
|
145
|
+
include MatcherErrors
|
146
|
+
|
147
|
+
match do |block|
|
148
|
+
@query_stats = Queries::UpdateCounter.instrument(&block)
|
149
|
+
expected == @query_stats.query_counts
|
150
|
+
end
|
151
|
+
|
152
|
+
def failure_text
|
153
|
+
expectation_failed_message('update')
|
154
|
+
end
|
155
|
+
end
|
156
|
+
|
157
|
+
# The following will not succeed because the code updates models:
|
158
|
+
#
|
159
|
+
# expect {
|
160
|
+
# WcRiskClass.last.update_attributes(id: 9999)
|
161
|
+
# }.to not_update_any_models
|
162
|
+
#
|
163
|
+
RSpec::Matchers.define(:not_update_any_models) do
|
164
|
+
include MatcherConfiguration
|
165
|
+
include MatcherErrors
|
166
|
+
|
167
|
+
match do |block|
|
168
|
+
@query_stats = Queries::UpdateCounter.instrument(&block)
|
169
|
+
@query_stats.query_counts.empty?
|
170
|
+
end
|
171
|
+
|
172
|
+
def failure_text
|
173
|
+
no_queries_fail_message('update')
|
174
|
+
end
|
175
|
+
end
|
176
|
+
end
|
177
|
+
|
178
|
+
# Shared methods that are included in the matchers.
|
179
|
+
# They configure it and ensure we get consistent and human readable error messages
|
180
|
+
module MatcherConfiguration
|
181
|
+
def self.included(base)
|
182
|
+
if base.respond_to?(:failure_message)
|
183
|
+
base.failure_message do |_actual|
|
184
|
+
failure_text
|
185
|
+
end
|
186
|
+
else
|
187
|
+
base.failure_message_for_should do |_actual|
|
188
|
+
failure_text
|
189
|
+
end
|
190
|
+
end
|
191
|
+
end
|
192
|
+
|
193
|
+
def supports_block_expectations?
|
194
|
+
true
|
195
|
+
end
|
196
|
+
end
|
197
|
+
|
198
|
+
module MatcherErrors
|
199
|
+
# Show the difference between expected and actual values with one value
|
200
|
+
# per line. This is done by hand because as of this writing the author
|
201
|
+
# doesn't understand how RSpec does its nice hash diff printing.
|
202
|
+
def difference(keys)
|
203
|
+
max_key_length = keys.reduce(0) { |max, key| [max, key.size].max }
|
204
|
+
|
205
|
+
keys.map do |key|
|
206
|
+
left = expected.fetch(key, 0)
|
207
|
+
right = @query_stats.queries.fetch(key, {}).fetch(:count, 0)
|
208
|
+
|
209
|
+
diff = "#{'+' if right > left}#{right - left}"
|
210
|
+
|
211
|
+
"#{key.rjust(max_key_length, ' ')} – expected: #{left}, got: #{right} (#{diff})"
|
212
|
+
end.compact
|
213
|
+
end
|
214
|
+
|
215
|
+
def source_lines(keys)
|
216
|
+
line_frequency = @query_stats.query_lines_by_frequency
|
217
|
+
keys_with_source_lines = keys.select { |key| line_frequency[key].present? }
|
218
|
+
keys_with_source_lines.map do |key|
|
219
|
+
source_lines = line_frequency[key].sort_by(&:last).reverse # Most frequent on top
|
220
|
+
next if source_lines.blank?
|
221
|
+
|
222
|
+
[
|
223
|
+
" #{key}"
|
224
|
+
] + source_lines.map { |line, count| " #{count} #{'call'.pluralize(count)}: #{line}" } + [
|
225
|
+
''
|
226
|
+
]
|
227
|
+
end
|
228
|
+
end
|
229
|
+
|
230
|
+
def no_queries_fail_message(crud_operation)
|
231
|
+
"Expected ActiveRecord to not #{crud_operation} any records, got #{@query_stats.query_counts}\n\nWhere unexpected queries came from:\n\n#{source_lines(@query_stats.query_counts.keys).join("\n")}"
|
232
|
+
end
|
233
|
+
|
234
|
+
def expectation_failed_message(crud_operation)
|
235
|
+
all_model_names = expected.keys + @query_stats.queries.keys
|
236
|
+
model_names_with_wrong_count = all_model_names.reject { |key| expected[key] == @query_stats.queries[key][:count] }.uniq
|
237
|
+
"Expected ActiveRecord to #{crud_operation} #{expected}, got #{@query_stats.query_counts}\nExpectations that differed:\n#{difference(model_names_with_wrong_count).join("\n")}\n\nWhere unexpected queries came from:\n\n#{source_lines(model_names_with_wrong_count).join("\n")}"
|
238
|
+
end
|
239
|
+
end
|
240
|
+
end
|
241
|
+
end
|
@@ -0,0 +1,31 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative './query_counter'
|
4
|
+
require_relative './table_name'
|
5
|
+
require_relative './query_filter'
|
6
|
+
|
7
|
+
module ArQueryMatchers
|
8
|
+
module Queries
|
9
|
+
# A specialized QueryCounter for "creates".
|
10
|
+
# It uses a simple regex to identify and parse sql INSERT queries.
|
11
|
+
# For more information, see the QueryCounter class.
|
12
|
+
class CreateCounter
|
13
|
+
def self.instrument(&block)
|
14
|
+
QueryCounter.new(CreateQueryFilter.new).instrument(&block)
|
15
|
+
end
|
16
|
+
|
17
|
+
class CreateQueryFilter < QueryFilter
|
18
|
+
# Matches unnamed SQL operations like the following:
|
19
|
+
# "INSERT INTO `company_approval_details` ..."
|
20
|
+
TABLE_NAME_SQL_PATTERN = /INSERT INTO [`"](?<table_name>[^`"]+)[`"]/.freeze
|
21
|
+
|
22
|
+
def filter_map(_name, sql)
|
23
|
+
# for inserts, name is always 'SQL', we have to rely on pattern matching the query string.
|
24
|
+
select_from_table = sql.match(TABLE_NAME_SQL_PATTERN)
|
25
|
+
|
26
|
+
TableName.new(select_from_table[:table_name]) if select_from_table
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
@@ -0,0 +1,40 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative './query_counter'
|
4
|
+
require_relative './table_name'
|
5
|
+
require_relative './model_name'
|
6
|
+
require_relative './query_filter'
|
7
|
+
|
8
|
+
module ArQueryMatchers
|
9
|
+
module Queries
|
10
|
+
# A specialized QueryCounter for "loads".
|
11
|
+
# For more information, see the QueryCounter class.
|
12
|
+
class LoadCounter
|
13
|
+
def self.instrument(&block)
|
14
|
+
QueryCounter.new(LoadQueryFilter.new).instrument(&block)
|
15
|
+
end
|
16
|
+
|
17
|
+
class LoadQueryFilter < Queries::QueryFilter
|
18
|
+
# Matches named SQL operations like the following:
|
19
|
+
# 'User Load'
|
20
|
+
MODEL_LOAD_PATTERN = /\A(?<model_name>[\w:]+) (Load|Exists)\Z/.freeze
|
21
|
+
|
22
|
+
# Matches unnamed SQL operations like the following:
|
23
|
+
# "SELECT COUNT(*) FROM `users` ..."
|
24
|
+
MODEL_SQL_PATTERN = /SELECT .* FROM [`"](?<table_name>[^`"]+)[`"]/.freeze
|
25
|
+
|
26
|
+
def filter_map(name, sql)
|
27
|
+
# First check for a `SELECT * FROM` query that ActiveRecord has
|
28
|
+
# helpfully named for us in the payload
|
29
|
+
match = name.match(MODEL_LOAD_PATTERN)
|
30
|
+
return ModelName.new(match[:model_name]) if match
|
31
|
+
|
32
|
+
# Fall back to pattern-matching on the table name in a COUNT and looking
|
33
|
+
# up the table name from ActiveRecord's loaded descendants.
|
34
|
+
select_from_table = sql.match(MODEL_SQL_PATTERN)
|
35
|
+
TableName.new(select_from_table[:table_name]) if select_from_table
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ArQueryMatchers
|
4
|
+
module Queries
|
5
|
+
# An instance of this class is one of the values that could be returned from the QueryFilter#filter_map.
|
6
|
+
# its accepts a name of an ActiveRecord model, for example: 'Company'.
|
7
|
+
class ModelName
|
8
|
+
attr_reader(:model_name)
|
9
|
+
|
10
|
+
def initialize(model_name)
|
11
|
+
@model_name = model_name
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
@@ -0,0 +1,95 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ArQueryMatchers
|
4
|
+
module Queries
|
5
|
+
# The QueryCounter instruments a ruby block and collect stats on the
|
6
|
+
# SQL queries performed during its execution.
|
7
|
+
#
|
8
|
+
# It's "generic" meaning it requires an instance of a QueryFilter to operate.
|
9
|
+
# The QueryFilter is an interface that both filters SQL statements and maps them to an ActiveRecord model, sort of a Enumerator#filter_map.
|
10
|
+
#
|
11
|
+
# This class is meant to be wrapped by another class that provides a concrete QueryFilter implementation.
|
12
|
+
# For example, you could implement a SelectQueryFilter using it:
|
13
|
+
# class SelectQueryCounter
|
14
|
+
# class SelectFilter < Queries::QueryFilter
|
15
|
+
# def extract(_name, sql)
|
16
|
+
# select_from_table = sql.match(/SELECT .* FROM [`"](?<table_name>[^`"]+)[`"]/)
|
17
|
+
# Queries::TableName.new(select_from_table[:table_name]) if select_from_table
|
18
|
+
# end
|
19
|
+
# end
|
20
|
+
#
|
21
|
+
# def self.instrument(&block)
|
22
|
+
# QueryCounter.new(SelectFilter.new).instrument(&block)
|
23
|
+
# end
|
24
|
+
# end
|
25
|
+
#
|
26
|
+
# stats = SelectQueryCounter.instrument do
|
27
|
+
# Company.first
|
28
|
+
# Employee.last(100)
|
29
|
+
# User.find(1)
|
30
|
+
# User.find(2)
|
31
|
+
# end
|
32
|
+
#
|
33
|
+
# stats.query_counts == { 'Company' => 1, Employee => '1', 'User' => 2 }
|
34
|
+
class QueryCounter
|
35
|
+
class QueryStats
|
36
|
+
def initialize(queries)
|
37
|
+
@queries = queries
|
38
|
+
end
|
39
|
+
|
40
|
+
attr_reader(:queries)
|
41
|
+
|
42
|
+
# @return [Hash] of model name to query count, for example: { 'Company' => 5}
|
43
|
+
def query_counts
|
44
|
+
Hash[*queries.reduce({}) { |acc, (model_name, data)| acc.update model_name => data[:count] }.sort_by(&:first).flatten]
|
45
|
+
end
|
46
|
+
|
47
|
+
# @return [Hash] of line in the source code to its frequency
|
48
|
+
def query_lines_by_frequency
|
49
|
+
queries.reduce({}) do |lines, (model_name, data)|
|
50
|
+
frequencies = data[:lines].reduce(Hash.new { |h, k| h[k] = 0 }) do |freq, line|
|
51
|
+
freq.update line => freq[line] + 1
|
52
|
+
end
|
53
|
+
lines.update model_name => frequencies
|
54
|
+
end
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
def initialize(query_filter)
|
59
|
+
@query_filter = query_filter
|
60
|
+
end
|
61
|
+
|
62
|
+
# @param [block] block to instrument
|
63
|
+
# @return [QueryStats] stats about all the SQL queries executed during the block
|
64
|
+
def instrument(&block)
|
65
|
+
queries = Hash.new { |h, k| h[k] = { count: 0, lines: [] } }
|
66
|
+
ActiveSupport::Notifications.subscribed(to_proc(queries), 'sql.active_record', &block)
|
67
|
+
QueryStats.new(queries)
|
68
|
+
end
|
69
|
+
|
70
|
+
private
|
71
|
+
|
72
|
+
# The 'marginalia' gem adds a line from the backtrace to the SQL query in
|
73
|
+
# the form of a comment.
|
74
|
+
MARGINALIA_SQL_COMMENT_PATTERN = %r{/*line:(?<line>.*)'*/}.freeze
|
75
|
+
private_constant :MARGINALIA_SQL_COMMENT_PATTERN
|
76
|
+
|
77
|
+
def to_proc(queries)
|
78
|
+
lambda do |_name, _start, _finish, _message_id, payload|
|
79
|
+
return if payload[:cached]
|
80
|
+
|
81
|
+
# Given a `sql.active_record` event, figure out which model is being
|
82
|
+
# accessed. Some of the simpler queries have a :ame key that makes this
|
83
|
+
# really easy. Others require parsing the SQL by hand.
|
84
|
+
model_name = @query_filter.filter_map(payload[:name] || '', payload[:sql] || '')&.model_name
|
85
|
+
|
86
|
+
if model_name
|
87
|
+
comment = payload[:sql].match(MARGINALIA_SQL_COMMENT_PATTERN)
|
88
|
+
queries[model_name][:lines] << comment[:line] if comment
|
89
|
+
queries[model_name][:count] += 1
|
90
|
+
end
|
91
|
+
end
|
92
|
+
end
|
93
|
+
end
|
94
|
+
end
|
95
|
+
end
|
@@ -0,0 +1,22 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ArQueryMatchers
|
4
|
+
module Queries
|
5
|
+
# An instance of this interface must be provided to the QueryCounter class.
|
6
|
+
# it allows one to customize which queries it wants to capture.
|
7
|
+
class QueryFilter
|
8
|
+
# @param _name [String] the name of the ActiveRecord operation (this is sometimes garbage)
|
9
|
+
# For example: "User Load"
|
10
|
+
#
|
11
|
+
# @param _sql [String] the sql query that was executed
|
12
|
+
# For example: "SELECT `users`.* FROM `users` .."
|
13
|
+
#
|
14
|
+
# @return nil or an instance which responds to #model_name (see TableName or ModelName)
|
15
|
+
# By returning nil we omit the query
|
16
|
+
# By not returning nil, we are associating this query with a model_name.
|
17
|
+
def filter_map(_name, _sql)
|
18
|
+
raise NotImplementedError
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
@@ -0,0 +1,40 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ArQueryMatchers
|
4
|
+
module Queries
|
5
|
+
# An instance of this class is one of the values that could be returned from the QueryFilter#filter_map.
|
6
|
+
# its accepts a name of a table in the database, for example: 'companies'.
|
7
|
+
#
|
8
|
+
# #model_name would transform the table name ('companies') into the ActiveRecord model name ('Company').
|
9
|
+
# It relies on the class to be loaded in the global namespace, which should be the case if we issues a query through ActiveRecord.
|
10
|
+
class TableName
|
11
|
+
def initialize(table_name)
|
12
|
+
@table_name = table_name
|
13
|
+
end
|
14
|
+
|
15
|
+
def model_name
|
16
|
+
active_record_class_for_table(@table_name)&.name
|
17
|
+
end
|
18
|
+
|
19
|
+
private
|
20
|
+
|
21
|
+
# We recalculate this each time because new classes might get loaded between queries
|
22
|
+
def active_record_class_for_table(table_name)
|
23
|
+
# Retrieve all (known) subclasses of ActiveRecord::Base
|
24
|
+
klasses = ActiveRecord::Base.descendants.reject(&:abstract_class)
|
25
|
+
|
26
|
+
# Group them by their table_names
|
27
|
+
tables = klasses.each_with_object(Hash.new { |k, v| k[v] = [] }) do |klass, accumulator|
|
28
|
+
accumulator[klass.table_name] << klass
|
29
|
+
end
|
30
|
+
# Structure:
|
31
|
+
# { 'users' => [User, AtoUser],
|
32
|
+
# 'employees => [Employee, PandaFlows::StateFields] }
|
33
|
+
|
34
|
+
# Of all the models that share the same table name sort them by their
|
35
|
+
# relative ancestry and pick the one that all the rest inherit from
|
36
|
+
tables[table_name].min_by { |a, b| a.ancestors.include?(b) }
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
@@ -0,0 +1,30 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative './query_counter'
|
4
|
+
require_relative './table_name'
|
5
|
+
require_relative './query_filter'
|
6
|
+
|
7
|
+
module ArQueryMatchers
|
8
|
+
module Queries
|
9
|
+
# A specialized QueryCounter for "updates".
|
10
|
+
# It uses a simple regex to identify and parse sql UPDATE queries.
|
11
|
+
# For more information, see the QueryCounter class.
|
12
|
+
class UpdateCounter
|
13
|
+
def self.instrument(&block)
|
14
|
+
QueryCounter.new(UpdateQueryFilter.new).instrument(&block)
|
15
|
+
end
|
16
|
+
|
17
|
+
class UpdateQueryFilter < QueryFilter
|
18
|
+
# Matches unnamed SQL operations like the following:
|
19
|
+
# "UPDATE `bank_account_verifications` ..."
|
20
|
+
TABLE_NAME_SQL_PATTERN = /UPDATE [`"](?<table_name>[^`"]+)[`"]/.freeze
|
21
|
+
|
22
|
+
def filter_map(_name, sql)
|
23
|
+
# for updates, name is always 'SQL', we have to rely on pattern matching on the query string instead.
|
24
|
+
select_from_table = sql.match(TABLE_NAME_SQL_PATTERN)
|
25
|
+
TableName.new(select_from_table[:table_name]) if select_from_table
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
metadata
ADDED
@@ -0,0 +1,180 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: ar-query-matchers
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Matan Zruya
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2019-09-14 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: activerecord
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '4.0'
|
20
|
+
- - "<="
|
21
|
+
- !ruby/object:Gem::Version
|
22
|
+
version: '6.0'
|
23
|
+
type: :runtime
|
24
|
+
prerelease: false
|
25
|
+
version_requirements: !ruby/object:Gem::Requirement
|
26
|
+
requirements:
|
27
|
+
- - ">="
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: '4.0'
|
30
|
+
- - "<="
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '6.0'
|
33
|
+
- !ruby/object:Gem::Dependency
|
34
|
+
name: activesupport
|
35
|
+
requirement: !ruby/object:Gem::Requirement
|
36
|
+
requirements:
|
37
|
+
- - ">="
|
38
|
+
- !ruby/object:Gem::Version
|
39
|
+
version: '4.0'
|
40
|
+
- - "<="
|
41
|
+
- !ruby/object:Gem::Version
|
42
|
+
version: '6.0'
|
43
|
+
type: :runtime
|
44
|
+
prerelease: false
|
45
|
+
version_requirements: !ruby/object:Gem::Requirement
|
46
|
+
requirements:
|
47
|
+
- - ">="
|
48
|
+
- !ruby/object:Gem::Version
|
49
|
+
version: '4.0'
|
50
|
+
- - "<="
|
51
|
+
- !ruby/object:Gem::Version
|
52
|
+
version: '6.0'
|
53
|
+
- !ruby/object:Gem::Dependency
|
54
|
+
name: rspec
|
55
|
+
requirement: !ruby/object:Gem::Requirement
|
56
|
+
requirements:
|
57
|
+
- - "~>"
|
58
|
+
- !ruby/object:Gem::Version
|
59
|
+
version: '3.0'
|
60
|
+
type: :runtime
|
61
|
+
prerelease: false
|
62
|
+
version_requirements: !ruby/object:Gem::Requirement
|
63
|
+
requirements:
|
64
|
+
- - "~>"
|
65
|
+
- !ruby/object:Gem::Version
|
66
|
+
version: '3.0'
|
67
|
+
- !ruby/object:Gem::Dependency
|
68
|
+
name: bundler
|
69
|
+
requirement: !ruby/object:Gem::Requirement
|
70
|
+
requirements:
|
71
|
+
- - ">="
|
72
|
+
- !ruby/object:Gem::Version
|
73
|
+
version: '0'
|
74
|
+
type: :development
|
75
|
+
prerelease: false
|
76
|
+
version_requirements: !ruby/object:Gem::Requirement
|
77
|
+
requirements:
|
78
|
+
- - ">="
|
79
|
+
- !ruby/object:Gem::Version
|
80
|
+
version: '0'
|
81
|
+
- !ruby/object:Gem::Dependency
|
82
|
+
name: rake
|
83
|
+
requirement: !ruby/object:Gem::Requirement
|
84
|
+
requirements:
|
85
|
+
- - "~>"
|
86
|
+
- !ruby/object:Gem::Version
|
87
|
+
version: '10.0'
|
88
|
+
type: :development
|
89
|
+
prerelease: false
|
90
|
+
version_requirements: !ruby/object:Gem::Requirement
|
91
|
+
requirements:
|
92
|
+
- - "~>"
|
93
|
+
- !ruby/object:Gem::Version
|
94
|
+
version: '10.0'
|
95
|
+
- !ruby/object:Gem::Dependency
|
96
|
+
name: rspec
|
97
|
+
requirement: !ruby/object:Gem::Requirement
|
98
|
+
requirements:
|
99
|
+
- - "~>"
|
100
|
+
- !ruby/object:Gem::Version
|
101
|
+
version: '3.0'
|
102
|
+
type: :development
|
103
|
+
prerelease: false
|
104
|
+
version_requirements: !ruby/object:Gem::Requirement
|
105
|
+
requirements:
|
106
|
+
- - "~>"
|
107
|
+
- !ruby/object:Gem::Version
|
108
|
+
version: '3.0'
|
109
|
+
- !ruby/object:Gem::Dependency
|
110
|
+
name: rubocop
|
111
|
+
requirement: !ruby/object:Gem::Requirement
|
112
|
+
requirements:
|
113
|
+
- - ">="
|
114
|
+
- !ruby/object:Gem::Version
|
115
|
+
version: '0'
|
116
|
+
type: :development
|
117
|
+
prerelease: false
|
118
|
+
version_requirements: !ruby/object:Gem::Requirement
|
119
|
+
requirements:
|
120
|
+
- - ">="
|
121
|
+
- !ruby/object:Gem::Version
|
122
|
+
version: '0'
|
123
|
+
- !ruby/object:Gem::Dependency
|
124
|
+
name: sqlite3
|
125
|
+
requirement: !ruby/object:Gem::Requirement
|
126
|
+
requirements:
|
127
|
+
- - "~>"
|
128
|
+
- !ruby/object:Gem::Version
|
129
|
+
version: '1.4'
|
130
|
+
type: :development
|
131
|
+
prerelease: false
|
132
|
+
version_requirements: !ruby/object:Gem::Requirement
|
133
|
+
requirements:
|
134
|
+
- - "~>"
|
135
|
+
- !ruby/object:Gem::Version
|
136
|
+
version: '1.4'
|
137
|
+
description: ''
|
138
|
+
email:
|
139
|
+
- mzruya@gmail.com
|
140
|
+
executables: []
|
141
|
+
extensions: []
|
142
|
+
extra_rdoc_files: []
|
143
|
+
files:
|
144
|
+
- README.md
|
145
|
+
- lib/ar_query_matchers.rb
|
146
|
+
- lib/ar_query_matchers/queries/create_counter.rb
|
147
|
+
- lib/ar_query_matchers/queries/load_counter.rb
|
148
|
+
- lib/ar_query_matchers/queries/model_name.rb
|
149
|
+
- lib/ar_query_matchers/queries/query_counter.rb
|
150
|
+
- lib/ar_query_matchers/queries/query_filter.rb
|
151
|
+
- lib/ar_query_matchers/queries/table_name.rb
|
152
|
+
- lib/ar_query_matchers/queries/update_counter.rb
|
153
|
+
- lib/ar_query_matchers/version.rb
|
154
|
+
homepage: https://github.com/Gusto/ar-query-matchers
|
155
|
+
licenses:
|
156
|
+
- MIT
|
157
|
+
metadata:
|
158
|
+
homepage_uri: https://github.com/Gusto/ar-query-matchers
|
159
|
+
source_code_uri: https://github.com/Gusto/ar-query-matchers
|
160
|
+
changelog_uri: https://github.com/Gusto/ar-query-matchers
|
161
|
+
post_install_message:
|
162
|
+
rdoc_options: []
|
163
|
+
require_paths:
|
164
|
+
- lib
|
165
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
166
|
+
requirements:
|
167
|
+
- - ">="
|
168
|
+
- !ruby/object:Gem::Version
|
169
|
+
version: '0'
|
170
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
171
|
+
requirements:
|
172
|
+
- - ">="
|
173
|
+
- !ruby/object:Gem::Version
|
174
|
+
version: '0'
|
175
|
+
requirements: []
|
176
|
+
rubygems_version: 3.0.3
|
177
|
+
signing_key:
|
178
|
+
specification_version: 4
|
179
|
+
summary: ''
|
180
|
+
test_files: []
|