query_police 0.1.4.beta → 0.1.5.beta

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -1,16 +1,16 @@
1
1
  # QueryPolice
2
2
 
3
- It is a rule-based engine with custom rules to Analyze Active-Record relations using explain results to detect bad queries.
3
+ It is a rule-based engine with custom rules to Analyze Active-Record relations using explain results and detect bad queries.
4
4
 
5
5
  ## Installation
6
6
 
7
7
  Install the gem and add it to the application's Gemfile by executing:
8
8
 
9
- $ bundle add query_police
9
+ $ gem 'query_police', '~> 0.1.5.beta'
10
10
 
11
11
  If the bundler is not being used to manage dependencies, install the gem by executing:
12
12
 
13
- $ gem install query_police
13
+ $ gem install query_police --pre
14
14
 
15
15
  ---
16
16
 
@@ -20,320 +20,318 @@ If the bundler is not being used to manage dependencies, install the gem by exec
20
20
 
21
21
  Use `QueryPolice.analyse` to generate an analysis object for your Active-Record relation or query and then you can use pretty print on the object.
22
22
 
23
- ```
24
- analysis = QueryPolice.analyse(<active_record_relation>)
25
- puts analysis.pretty_analysis_for(<impact>)
23
+ ```ruby
24
+ analysis = QueryPolice.analyse(<query>)
25
+ puts analysis.pretty_analysis
26
26
  ```
27
27
 
28
28
  **Eg.**
29
- ```
29
+ ```ruby
30
30
  analysis = QueryPolice.analyse(
31
- User.joins('join orders on orders.user_id = users.id')
31
+ User.joins('join orders on orders.user_id = users.id') # analyse query using active record relation
32
32
  )
33
- puts analysis.pretty_analysis_for('negative')
34
33
  # or
35
- puts analysis.pretty_analysis({'negative' => true, 'positive' => true})
34
+ analysis = QueryPolice.analyse(
35
+ "select * from users join orders on orders.user_id = users.id" # analyse query using query string
36
+ )
37
+
38
+ puts analysis.pretty_analysis
36
39
  ```
37
40
  **Results**
38
41
 
42
+ **Note:** [Query debt significance](#significance)
43
+
44
+
39
45
  ```
40
- query_score: 330.0
46
+ query_debt: 330.0
41
47
 
42
48
  +----------------------------------------------------------------------------------------------------------------------------------+
43
49
  | orders |
44
50
  +------------+---------------------------------------------------------------------------------------------------------------------+
45
- | score | 200.0 |
51
+ | debt | 200.0 |
46
52
  +------------+---------------------------------------------------------------------------------------------------------------------+
47
53
  | column | type |
48
54
  | impact | negative |
49
- | tag_score | 100.0 |
55
+ | tag_debt | 100.0 |
50
56
  | message | Entire orders table is scanned to find matching rows, you have 0 possible keys to use. |
51
57
  | suggestion | Use index here. You can use index from possible key: absent or add new one to orders table as per the requirements. |
52
58
  +------------+---------------------------------------------------------------------------------------------------------------------+
53
59
  | column | possible_keys |
54
60
  | impact | negative |
55
- | tag_score | 50.0 |
61
+ | tag_debt | 50.0 |
56
62
  | message | There are no possible keys for orders table to be used, can result into full scan |
57
63
  | suggestion | Please add index keys for orders table |
58
64
  +------------+---------------------------------------------------------------------------------------------------------------------+
59
65
  | column | key |
60
66
  | impact | negative |
61
- | tag_score | 50.0 |
67
+ | tag_debt | 50.0 |
62
68
  | message | There is no index key used for orders table, and can result into full scan of the orders table |
63
69
  | suggestion | Please use index from possible_keys: absent or add new one to orders table as per the requirements. |
64
70
  +------------+---------------------------------------------------------------------------------------------------------------------+
65
71
  +------------------------------------------------------------------------------------+
66
72
  | users |
67
73
  +------------+-----------------------------------------------------------------------+
68
- | score | 130.0 |
74
+ | debt | 130.0 |
69
75
  +------------+-----------------------------------------------------------------------+
70
76
  | column | detailed#used_columns |
71
77
  | impact | negative |
72
- | tag_score | 130.0 |
78
+ | tag_debt | 130.0 |
73
79
  | message | You have selected 18 columns, You should not select too many columns. |
74
80
  | suggestion | Please only select required columns. |
75
81
  +------------+-----------------------------------------------------------------------+
76
82
  ```
77
83
 
78
- ### Add a logger for every query
79
-
80
- Add `QueryPolice.subscribe_logger` to your initial load file like `application.rb`
81
-
82
- You can make logger silence of error using `QueryPolice.subscribe_logger silent: true`.
83
-
84
- You can change logger config using `QueryPolice logger_config: <config>`, default logger_config `{'negative' => true}`, options `positive: <Boolean>, caution: <Boolean>`.
85
-
86
- ---
84
+ ### Analysis for a Impact
87
85
 
88
- ## How it works?
86
+ To print pretty analysis for different impacts
87
+ ```ruby
88
+ analysis = QueryPolice.analyse("select * from users")
89
+ puts analysis.pretty_analysis_for('positive') # impact negative, positive, caution
89
90
 
90
- 1. Query police converts the relation into SQL query
91
-
92
- 2. Query police generates an execution plan using EXPLAIN and EXPLAIN format=JSON based on the configuration.
93
-
94
- 3. Query police load rules from the config file.
95
-
96
- 4. Query police apply rules on the execution plan and generate a new analysis object.
97
-
98
- 5. Analysis object provides different methods to print the analysis in a more descriptive format.
99
-
100
-
101
- ## Execution plan
102
-
103
- We have 2 possible execution plans:-
104
-
105
- Normal - using `EXPLAIN`
106
-
107
- Detailed - using `EXPLAIN format=JSON`
108
-
109
- **NOTE:** By default Detailed execution plan is added in the final execution plan, you can remove that by `QueryPolice.detailed=false`
110
-
111
- ### Normal execution plan
112
-
113
- Generated using `EXPAIN <query>`
114
-
115
- **Result**
116
-
117
- | id | select_type | table | partitions | type. | possible_keys | key | key_len | ref | rows | filtered | Extra |
118
- |----|-------------|---------|------------|--------|---------------------------|---------------------|---------|-----------------------------|------|----------|--------------------------|
119
- | 1 | SIMPLE | profile | NULL. | index | fk_rails_249a7ebca1 | fk_rails_249a7ebca1 | 5 | NULL | 603 | 100.00 | Using where; Using index |
120
- | 1 | SIMPLE | users | NULL | eq_ref | PRIMARY,index_users_on_id | PRIMARY | 4 | development.profile.user_id |1 | 100.00 | NULL |
121
-
122
-
123
- The result for this is added as it is in the final execution plan
91
+ # puts
92
+ # +----------------------------------------------------------+
93
+ # | users |
94
+ # +-----------+----------------------------------------------+
95
+ # | debt | 330.0 |
96
+ # +-----------+----------------------------------------------+
97
+ # | column | select_type |
98
+ # | impact | positive |
99
+ # | tag_debt | 0 |
100
+ # | message | A simple query without subqueries or unions. |
101
+ # +-----------+----------------------------------------------+
102
+ ```
124
103
 
125
- **Eg.**
104
+ ### Analysis for Multiple Impacts
126
105
 
106
+ To print pretty analysis for multiple impacts, default: `{ 'negative' => true, 'caution' => true }`
107
+ ```ruby
108
+ analysis = QueryPolice.analyse("select * from users")
109
+ puts analysis.pretty_analysis({'negative' => true, 'positive' => true})
127
110
  ```
128
- {
129
- "profile" => {
130
- "id" => 1,
131
- "select_type" => "SIMPLE",
132
- "table" => "profile",
133
- "partitions" => nil,
134
- "type" => "index",
135
- "possible_keys" => "fk_rails_249a7ebca1",
136
- "key" => "fk_rails_249a7ebca1",
137
- "key_len" => "5",
138
- "ref" => nil,
139
- "rows" => 603,
140
- "filtered" => 100.0,
141
- "Extra" => "Using where; Using index"
142
- },
143
- "users" => {
144
- "id" => 1,
145
- "select_type" => "SIMPLE",
146
- "table" => "users",
147
- "partitions" => nil,
148
- "type" => "eq_ref",
149
- "possible_keys" => "PRIMARY,index_users_on_id",
150
- "key" => "PRIMARY",
151
- "key_len" => "4",
152
- "ref" => "development.profile.user_id",
153
- "rows" => 1,
154
- "filtered" => 100.0,
155
- "Extra" => nil
156
- }
157
- }
158
- ```
159
-
160
- ### Detailed execution plan
161
111
 
162
- Generated using `EXPAIN format=JSON <query>`
112
+ ### Query debt
163
113
 
164
- **Truncated Result**
114
+ To get the final debt of the query
115
+ ```ruby
116
+ analysis = QueryPolice.analyse("select * from users")
117
+ puts analysis.query_debt
165
118
 
119
+ # puts
120
+ # 100.0
166
121
  ```
167
- {
168
- "query_block": {
169
- "select_id": 1,
170
- "cost_info": {
171
- "query_cost": "850.20"
172
- },
173
- "nested_loop": [
174
- {
175
- "table": {
176
- "table_name": "profile",
177
- "access_type": "index",
178
- "key_length": "5",
179
- "cost_info": {
180
- "read_cost": "6.00",
181
- "eval_cost": "120.60",
182
- "prefix_cost": "126.60",
183
- "data_read_per_join": "183K"
184
- },
185
- "used_columns": [
186
- "user_id"
187
- ],
188
- "attached_condition": "(`development`.`profile`.`user_id` is not null)"
189
- }
190
- }
191
- ]
192
- }
193
- }
194
- ```
195
-
196
122
 
197
- The result for this is added in the flattened form to the final execution plan, where the `detailed#` prefix is added before each key.
123
+ #### Significance
124
+ Query debt signifies the quality of the query, high value represents a bad query.
125
+ - `0 - 199` - Good Query
126
+ - `200 - 499` - Potentially Bad query
127
+ - `>=500` - Bad query
198
128
 
199
- **Truncated Eg.**
129
+ ### Word wrap
200
130
 
201
- ```
202
- {
203
- "detailed#key_length" => "5",
204
- "detailed#rows_examined_per_scan" => 603,
205
- "detailed#rows_produced_per_join" => 603,
206
- "detailed#filtered" => "100.00",
207
- "detailed#using_index" => true,
208
- "detailed#cost_info#read_cost" => "6.00",
209
- "detailed#cost_info#eval_cost" => "120.60",
210
- "detailed#cost_info#prefix_cost" => "126.60",
211
- "detailed#cost_info#data_read_per_join" => "183K",
212
- "detailed#used_columns" => ["user_id"]
213
- ...
131
+ To change the word wrap width for the pretty analysis result, default value: `100`
132
+ ```ruby
133
+ analysis = QueryPolice.analyse("select * from users")
134
+ puts analysis.pretty_analysis_for('positive', {'wrap_width' => 40})
135
+ # or
136
+ puts analysis.pretty_analysis({'positive' => true, 'wrap_width' => 40})
137
+
138
+ # puts
139
+ # +--------------------------------------------------+
140
+ # | users |
141
+ # +-----------+--------------------------------------+
142
+ # | debt | 330.0 |
143
+ # +-----------+--------------------------------------+
144
+ # | column | select_type |
145
+ # | impact | positive |
146
+ # | tag_debt | 0 |
147
+ # | message | A simple query without subqueries or |
148
+ # | | unions. |
149
+ # +-----------+--------------------------------------+
150
+
151
+ puts analysis.pretty_analysis_for('positive', {'wrap_width' => 20})
152
+ # or
153
+ puts analysis.pretty_analysis({'positive' => true, 'wrap_width' => 20})
154
+
155
+ # puts
156
+ # +--------------------------------+
157
+ # | users |
158
+ # +-----------+--------------------+
159
+ # | debt | 330.0 |
160
+ # +-----------+--------------------+
161
+ # | column | select_type |
162
+ # | impact | positive |
163
+ # | tag_debt | 0 |
164
+ # | message | A simple query |
165
+ # | | without subqueries |
166
+ # | | or unions. |
167
+ # +-----------+--------------------+
168
+ ```
169
+
170
+ ### Skip footer
171
+
172
+ To skip the footer (added in [query police config](#analysis-footer)) after an analysis of a query
173
+ ```ruby
174
+ analysis = QueryPolice.analyse("select * from users")
175
+ puts analysis.pretty_analysis_for('positive', {'skip_footer' => true})
176
+ # or
177
+ puts analysis.pretty_analysis({'positive' => true, 'skip_footer' => true})
214
178
  ```
215
179
 
180
+ ### Analysis Footer
181
+ To define a footer text that will be added after an analysis for a query, by default there is no footer
182
+ ```ruby
183
+ QueryPolice.analysis_footer = 'Please check more details with this link...'
184
+ # or
185
+ QueryPolice.configure do |config|
186
+ config.analysis_footer = 'Please check more details with this link...'
187
+ end
188
+
189
+ # puts
190
+ # +----------------------------------------------------------+
191
+ # | users |
192
+ # +-----------+----------------------------------------------+
193
+ # | debt | 330.0 |
194
+ # +-----------+----------------------------------------------+
195
+ # | column | select_type |
196
+ # | impact | positive |
197
+ # | tag_debt | 0 |
198
+ # | message | A simple query without subqueries or unions. |
199
+ # +-----------+----------------------------------------------+
200
+ # Please check more details with this link...
201
+ ```
202
+
203
+ ### Custom rules path
204
+
205
+ To define custom rules path (More details about [how to define custom rules](#how-to-define-custom-rules))
206
+ ```ruby
207
+ QueryPolice.rules_path = 'path/to/rules/file.<json/yml>'
208
+ # or
209
+ QueryPolice.configure do |config|
210
+ config.rules_path = 'path/to/rules/file.<json/yml>'
211
+ end
212
+ ```
216
213
 
214
+ ### Verbosity
217
215
 
218
- ##### Flatten
219
-
220
- `{a: {b: 1}, c: 2}` is converted into `{a#b: 1, c: 2}`.
216
+ Verbosity defines which `EXPLAIN` result should be used for analysis. (More details about [EXPLAIN vs Detailed EXPLAIN](#execution-plan))
217
+ - `basic` - It uses only `EXPLAIN`(basic) result
218
+ - `detailed` - It uses both `EXPLAIN`(basic) and `EXPLAIN format=json`(detailed) results `(default value)`
219
+ ```ruby
220
+ QueryPolice.verbosity = "detailed"
221
+ # or
222
+ QueryPolice.configure do |config|
223
+ config.verbosity = "detailed"
224
+ end
225
+ ```
221
226
 
227
+ ### Disable Actions
222
228
 
223
- ## Analysis object
229
+ To disable actions that are performed on each query.
230
+ ```ruby
231
+ # Note: A logger action is already added, so query police will log pretty analysis after each query by default
232
+ QueryPolice.action_enabled = false
233
+ # or
234
+ QueryPolice.configure do |config|
235
+ config.action_enabled = false
236
+ end
237
+ ```
224
238
 
225
- Analysis object stores a detailed analysis report of a relation inside `:tables :summary attributes`.
239
+ ### Custom Actions
226
240
 
227
- #### Attributes
241
+ To add custom actions, by default a logger action is already added and enabled (More details about [Analysis Object](#analysis-object))
242
+ ```ruby
243
+ QueryPolice.add_action do |analysis| # analysis object for the query
244
+ puts analysis.tables
245
+ end
246
+ ```
228
247
 
229
- **tables [Hash] - detailed table analysis**
248
+ ### Logger config
230
249
 
231
- ```
232
- {
233
- 'users' => {
234
- 'id'=>1,
235
- 'name' => 'users', # table alias user in the execution plan
236
- 'score' => <float> # score for the table
237
- 'analysis' => {
238
- 'type' => { # attribute name
239
- 'value' => <string>, # raw value of attribute in execution plan
240
- 'tags' => {
241
- 'all' => { # tag based on the value of a attribute
242
- 'impact'=> <string>, # negative, positive, cautions
243
- 'warning'=> <string>, # Eg. 'warning to represent the issue'
244
- 'suggestions'=> <string> # Eg. 'some follow-up suggestions'
245
- 'score' => <float> # score for the tag
246
- }
247
- }
248
- }
249
- }
250
- }
251
- }
252
- ```
253
- **summary [Hash] - hash of analysis summary**
250
+ To change the logger options which will be used to generate analysis
251
+ ```ruby
252
+ QueryPolice.logger_options = {'negative' => true}
253
+ # or
254
+ QueryPolice.configure do |config|
255
+ config.logger_options = {'negative' => true}
256
+ end
254
257
 
255
- ```
256
- {
257
- 'cardinality' => {
258
- 'amount' => 10,
259
- 'warning' => 'warning to represent the issue',
260
- 'suggestions' => 'some follow up suggestions',
261
- 'score' => 100.0
262
- }
263
- }
258
+ # default logger_config: {'negative' => true, 'caution' => true}
259
+ # options negative: <Boolean>, positive: <Boolean>, caution: <Boolean>, wrap_width: <Integer>, skip_footer: <Boolean>
264
260
  ```
265
261
 
262
+ ---
266
263
 
264
+ ## How to define custom rules?
267
265
 
268
- ## How to define rules?
269
-
270
- Rules defined in the json file at config_path is applied to the execution plan. We have variety of option to define rules.
266
+ Rules defined in the `json/yaml` file at rules_path is applied to the execution plan. Query Police have variety of option to define rules.
271
267
 
272
268
  You can change this by `QueryPolice.rules_path=<path>` and define your own rules
273
269
 
274
270
  ### Rule Structure
271
+ **Note:** Check Query Police default rules defined at [rules.json](lib/query_police/rules.json) or examples at [examples/rules/](examples/rules/) for better clarity
275
272
 
276
273
  A basic rule structure -
277
- ```
274
+
275
+ JSON
276
+ ```json
278
277
  "<column_name>": {
279
- "description": <string>,
280
- "value_type": <string>,
281
- "delimiter": <string>,
278
+ "description": "<string>",
279
+ "value_type": "<string>",
280
+ "delimiter": "<string>",
282
281
  "rules": {
283
282
  "<rule>": {
284
- "amount": <integer>
285
- "impact": <string>,
286
- "message": <string>,
287
- "suggestion": <string>,
288
- "score": {
289
- "value": <integer>,
290
- "type": <string>
283
+ "amount": "<integer>",
284
+ "impact": "<string>",
285
+ "message": "<string>",
286
+ "suggestion": "<string>",
287
+ "debt": {
288
+ "value": "<integer>",
289
+ "type": "<string>"
291
290
  }
292
291
  }
293
292
  }
294
293
  }
295
294
  ```
296
- - `<column_name>` - attribute name in the final execution plan.
297
-
295
+ YAML
296
+ ```yaml
297
+ <column_name>:
298
+ description: <string>
299
+ value_type: <string>
300
+ delimiter: <string>
301
+ rules:
302
+ <rule>:
303
+ amount: <integer>
304
+ impact: <string>
305
+ message: <string>
306
+ suggestion: <string>
307
+ debt:
308
+ value: <integer>
309
+ type: <string>
310
+ ```
311
+ - `<column_name>` - attribute name in the final execution plan. (more details about [attributes](#attributes-for-rules))
298
312
  - `description` - description of the attribute
299
-
300
313
  - `value_type` - value type of the attribute
301
-
302
314
  - `delimiter` - delimiter to parse array type attribute values, if no delimiter is passed engine will consider value is already in array form.
303
-
304
315
  - `<rule>` - kind of rule for the attribute
305
-
306
316
  - `<tag>` - direct value match eg. ALL, SIMPLE
307
-
308
317
  - `absent` - when the value is missing
309
-
310
318
  - `threshold` - a greater than threshold check based on the amount set inside the rule.
311
-
312
319
  - `amount` - the amount of threshold that needs to check for
313
-
314
320
  - length for string
315
-
316
321
  - value for number
317
-
318
322
  - size for the array
319
-
320
323
  - `impact` - impact of the rule
321
-
322
324
  - `negative`
323
-
324
325
  - `postive`
325
-
326
326
  - `caution`
327
-
328
327
  - `message` - the message needs to provide the significance of the rule
329
-
330
328
  - `suggestion` - suggestion on how we can fix the issue
331
- - `score` - score-related config that will be affected to final query score
332
- - `value` - value that will be added to the query score
333
- - `type` - the type of scoring that will be added to the query score
329
+ - `debt` - debt-related config that will be affected to final query debt
330
+ - `value` - value that will be added to the query debt
331
+ - `type` - the type of scoring that will be added to the query debt
334
332
  - `base`- value
335
333
  - `relative` - value * (amount for that column in query)
336
- - `treshold_relative` - (value - (threshold amount)) * (amount for that column in query)
334
+ - `threshold_relative` - (value - (threshold amount)) * (amount for that column in query)
337
335
 
338
336
 
339
337
 
@@ -342,34 +340,22 @@ A basic rule structure -
342
340
  We can define dynamic messages and suggestions with variables provided by the engine.
343
341
 
344
342
  - `$amount` - the amount of the value
345
-
346
343
  - length for string
347
-
348
344
  - value for number
349
-
350
345
  - size for the array
351
-
352
346
  - `$column` - attribute name
353
-
354
347
  - `$impact` - impact for the rule
355
-
356
348
  - `$table` - table alias used in the plan
357
-
358
349
  - `$tag` - tag for which rule is applied
359
-
360
350
  - `$value` - original parsed value
361
-
362
351
  - `$<column_name>` - the value of that specific column in that table
363
-
364
352
  - `$amount_<column_name>` - amount of that specific column
365
353
 
366
-
367
-
368
354
  ### Rules Examples
369
355
 
370
356
  #### Basic rule example
371
-
372
- ```
357
+ File: [JSON](examples/rules/json/basic_rule.json) | [YAML](examples/rules/yaml/basic_rule.yml)
358
+ ```json
373
359
  "type": {
374
360
  "description": "Join used in the query for a specific table.",
375
361
  "value_type": "string",
@@ -383,11 +369,12 @@ We can define dynamic messages and suggestions with variables provided by the en
383
369
  "impact": "negative",
384
370
  "message": "Entire $table table is scanned to find matching rows, you have $amount_possible_keys possible keys to use.",
385
371
  "suggestion": "Use index here. You can use index from possible key: $possible_keys or add new one to $table table as per the requirements.",
386
- "score": {
372
+ "debt": {
387
373
  "value": 200,
388
374
  "type": "base"
389
375
  }
390
376
  }
377
+ }
391
378
  }
392
379
  ```
393
380
  For the above rule, dynamic message will be generated as-
@@ -401,8 +388,8 @@ Use index here. You can use index from possible key: ["PRIMARY", "user_email"] o
401
388
 
402
389
 
403
390
  #### Absent rule example
404
-
405
- ```
391
+ File: [JSON](examples/rules/json/absent_rule.json) | [YAML](examples/rules/yaml/absent_rule.yml)
392
+ ```json
406
393
  "key": {
407
394
  "description": "index key used for the table",
408
395
  "value_type": "string",
@@ -427,8 +414,8 @@ Please use index from possible_keys: ["PRIMARY", "user_email"] or add new one to
427
414
 
428
415
 
429
416
  #### Threshold rule example
430
-
431
- ```
417
+ File: [JSON](examples/rules/json/threshold_rule.json) | [YAML](examples/rules/yaml/threshold_rule.yml)
418
+ ```json
432
419
  "possible_keys": {
433
420
  "description": "Index keys possible for a specifc table",
434
421
  "value_type": "array",
@@ -454,8 +441,8 @@ Please check if there are extra indexes in users table.
454
441
 
455
442
 
456
443
  #### Complex Detailed rule example
457
-
458
- ```
444
+ File: [JSON](examples/rules/json/complex_detailed_rule.json) | [YAML](examples/rules/yaml/complex_detailed_rule.yml)
445
+ ```json
459
446
  "detailed#used_columns": {
460
447
  "description": "",
461
448
  "value_type": "array",
@@ -465,9 +452,9 @@ Please check if there are extra indexes in users table.
465
452
  "impact": "negative",
466
453
  "message": "You have selected $amount columns, You should not select too many columns.",
467
454
  "suggestion": "Please only select required columns.",
468
- "score": {
455
+ "debt": {
469
456
  "value": 10,
470
- "type": "treshold_relative"
457
+ "type": "threshold_relative"
471
458
  }
472
459
  }
473
460
  }
@@ -483,9 +470,9 @@ Please only select required columns.
483
470
  ```
484
471
 
485
472
 
486
- ### Summary
473
+ ### Summary in Analysis
487
474
 
488
- You can define similar rules for the summary. Current summary attribute supported -
475
+ You can define similar rules for the summary. Current only one attribute is supported in summary -
489
476
 
490
477
  - `cardinality` - cardinality based on all tables
491
478
 
@@ -493,7 +480,7 @@ You can define similar rules for the summary. Current summary attribute supporte
493
480
 
494
481
 
495
482
 
496
- ### Attributes
483
+ ### Attributes for Rules
497
484
 
498
485
  There are a lot of attributes for you to use based on the final execution plan.
499
486
 
@@ -509,6 +496,185 @@ To check more keys you can use `EXPLAIN format=JSON <query>`
509
496
 
510
497
  ---
511
498
 
499
+ ## How Query Police works?
500
+
501
+ 1. Query police converts the relation into SQL query
502
+
503
+ 2. Query police generates an execution plan using EXPLAIN and EXPLAIN format=JSON based on the configuration.
504
+
505
+ 3. Query police load rules from the config file.
506
+
507
+ 4. Query police apply rules on the execution plan and generate a new analysis object.
508
+
509
+ 5. Analysis object provides different methods to print the analysis in a more descriptive format.
510
+
511
+
512
+ ## Execution plan
513
+
514
+ We have 2 possible execution plans:-
515
+
516
+ Normal - using `EXPLAIN`
517
+
518
+ Detailed - using `EXPLAIN format=JSON`
519
+
520
+ **NOTE:** By default Detailed execution plan is added in the final execution plan, you can remove that by `QueryPolice.detailed=false`
521
+
522
+ ### Normal execution plan
523
+
524
+ Generated using `EXPAIN <query>`
525
+
526
+ **Result**
527
+
528
+ | id | select_type | table | partitions | type. | possible_keys | key | key_len | ref | rows | filtered | Extra |
529
+ |----|-------------|---------|------------|--------|---------------------------|---------------------|---------|-----------------------------|------|----------|--------------------------|
530
+ | 1 | SIMPLE | profile | NULL. | index | fk_rails_249a7ebca1 | fk_rails_249a7ebca1 | 5 | NULL | 603 | 100.00 | Using where; Using index |
531
+ | 1 | SIMPLE | users | NULL | eq_ref | PRIMARY,index_users_on_id | PRIMARY | 4 | development.profile.user_id |1 | 100.00 | NULL |
532
+
533
+
534
+ The result for this is added as it is in the final execution plan
535
+
536
+ **Eg.**
537
+
538
+ ```ruby
539
+ {
540
+ "profile" => {
541
+ "id" => 1,
542
+ "select_type" => "SIMPLE",
543
+ "table" => "profile",
544
+ "partitions" => nil,
545
+ "type" => "index",
546
+ "possible_keys" => "fk_rails_249a7ebca1",
547
+ "key" => "fk_rails_249a7ebca1",
548
+ "key_len" => "5",
549
+ "ref" => nil,
550
+ "rows" => 603,
551
+ "filtered" => 100.0,
552
+ "Extra" => "Using where; Using index"
553
+ },
554
+ "users" => {
555
+ "id" => 1,
556
+ "select_type" => "SIMPLE",
557
+ "table" => "users",
558
+ "partitions" => nil,
559
+ "type" => "eq_ref",
560
+ "possible_keys" => "PRIMARY,index_users_on_id",
561
+ "key" => "PRIMARY",
562
+ "key_len" => "4",
563
+ "ref" => "development.profile.user_id",
564
+ "rows" => 1,
565
+ "filtered" => 100.0,
566
+ "Extra" => nil
567
+ }
568
+ }
569
+ ```
570
+
571
+ ### Detailed execution plan
572
+
573
+ Generated using `EXPAIN format=JSON <query>`
574
+
575
+ **Truncated Result**
576
+
577
+ ```ruby
578
+ {
579
+ "query_block": {
580
+ "select_id": 1,
581
+ "cost_info": {
582
+ "query_cost": "850.20"
583
+ },
584
+ "nested_loop": [
585
+ {
586
+ "table": {
587
+ "table_name": "profile",
588
+ "access_type": "index",
589
+ "key_length": "5",
590
+ "cost_info": {
591
+ "read_cost": "6.00",
592
+ "eval_cost": "120.60",
593
+ "prefix_cost": "126.60",
594
+ "data_read_per_join": "183K"
595
+ },
596
+ "used_columns": [
597
+ "user_id"
598
+ ],
599
+ "attached_condition": "(`development`.`profile`.`user_id` is not null)"
600
+ }
601
+ }
602
+ ]
603
+ }
604
+ }
605
+ ```
606
+
607
+
608
+ The result for this is added in the flattened form to the final execution plan, where the `detailed#` prefix is added before each key.
609
+
610
+ **Truncated Eg.**
611
+
612
+ ```ruby
613
+ {
614
+ "detailed#key_length" => "5",
615
+ "detailed#rows_examined_per_scan" => 603,
616
+ "detailed#rows_produced_per_join" => 603,
617
+ "detailed#filtered" => "100.00",
618
+ "detailed#using_index" => true,
619
+ "detailed#cost_info#read_cost" => "6.00",
620
+ "detailed#cost_info#eval_cost" => "120.60",
621
+ "detailed#cost_info#prefix_cost" => "126.60",
622
+ "detailed#cost_info#data_read_per_join" => "183K",
623
+ "detailed#used_columns" => ["user_id"]
624
+ ...
625
+ ```
626
+
627
+
628
+
629
+ ##### Flatten
630
+
631
+ `{a: {b: 1}, c: 2}` is converted into `{a#b: 1, c: 2}`.
632
+
633
+ ---
634
+
635
+ ## Analysis object
636
+
637
+ Analysis object stores a detailed analysis report of a relation inside `:tables :summary attributes`.
638
+
639
+ #### Attributes
640
+
641
+ **tables [Hash] - detailed table analysis**
642
+
643
+ ```ruby
644
+ {
645
+ 'users' => {
646
+ 'id'=>1,
647
+ 'name' => 'users', # table alias user in the execution plan
648
+ 'debt' => <float> # debt for the table
649
+ 'analysis' => {
650
+ 'type' => { # attribute name
651
+ 'value' => <string>, # raw value of attribute in execution plan
652
+ 'tags' => {
653
+ 'all' => { # tag based on the value of a attribute
654
+ 'impact'=> <string>, # negative, positive, cautions
655
+ 'warning'=> <string>, # Eg. 'warning to represent the issue'
656
+ 'suggestions'=> <string> # Eg. 'some follow-up suggestions'
657
+ 'debt' => <float> # debt for the tag
658
+ }
659
+ }
660
+ }
661
+ }
662
+ }
663
+ }
664
+ ```
665
+ **summary [Hash] - hash of analysis summary**
666
+
667
+ ```ruby
668
+ {
669
+ 'cardinality' => {
670
+ 'amount' => 10,
671
+ 'warning' => 'warning to represent the issue',
672
+ 'suggestions' => 'some follow up suggestions',
673
+ 'debt' => 100.0
674
+ }
675
+ }
676
+ ```
677
+
512
678
  ## Development
513
679
 
514
680
  After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.