query_police 0.1.4.beta → 0.1.5.beta

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,16 +1,16 @@
1
1
  # QueryPolice
2
2
 
3
- It is a rule-based engine with custom rules to Analyze Active-Record relations using explain results to detect bad queries.
3
+ It is a rule-based engine with custom rules to Analyze Active-Record relations using explain results and detect bad queries.
4
4
 
5
5
  ## Installation
6
6
 
7
7
  Install the gem and add it to the application's Gemfile by executing:
8
8
 
9
- $ bundle add query_police
9
+ $ gem 'query_police', '~> 0.1.5.beta'
10
10
 
11
11
  If the bundler is not being used to manage dependencies, install the gem by executing:
12
12
 
13
- $ gem install query_police
13
+ $ gem install query_police --pre
14
14
 
15
15
  ---
16
16
 
@@ -20,320 +20,318 @@ If the bundler is not being used to manage dependencies, install the gem by exec
20
20
 
21
21
  Use `QueryPolice.analyse` to generate an analysis object for your Active-Record relation or query and then you can use pretty print on the object.
22
22
 
23
- ```
24
- analysis = QueryPolice.analyse(<active_record_relation>)
25
- puts analysis.pretty_analysis_for(<impact>)
23
+ ```ruby
24
+ analysis = QueryPolice.analyse(<query>)
25
+ puts analysis.pretty_analysis
26
26
  ```
27
27
 
28
28
  **Eg.**
29
- ```
29
+ ```ruby
30
30
  analysis = QueryPolice.analyse(
31
- User.joins('join orders on orders.user_id = users.id')
31
+ User.joins('join orders on orders.user_id = users.id') # analyse query using active record relation
32
32
  )
33
- puts analysis.pretty_analysis_for('negative')
34
33
  # or
35
- puts analysis.pretty_analysis({'negative' => true, 'positive' => true})
34
+ analysis = QueryPolice.analyse(
35
+ "select * from users join orders on orders.user_id = users.id" # analyse query using query string
36
+ )
37
+
38
+ puts analysis.pretty_analysis
36
39
  ```
37
40
  **Results**
38
41
 
42
+ **Note:** [Query debt significance](#significance)
43
+
44
+
39
45
  ```
40
- query_score: 330.0
46
+ query_debt: 330.0
41
47
 
42
48
  +----------------------------------------------------------------------------------------------------------------------------------+
43
49
  | orders |
44
50
  +------------+---------------------------------------------------------------------------------------------------------------------+
45
- | score | 200.0 |
51
+ | debt | 200.0 |
46
52
  +------------+---------------------------------------------------------------------------------------------------------------------+
47
53
  | column | type |
48
54
  | impact | negative |
49
- | tag_score | 100.0 |
55
+ | tag_debt | 100.0 |
50
56
  | message | Entire orders table is scanned to find matching rows, you have 0 possible keys to use. |
51
57
  | suggestion | Use index here. You can use index from possible key: absent or add new one to orders table as per the requirements. |
52
58
  +------------+---------------------------------------------------------------------------------------------------------------------+
53
59
  | column | possible_keys |
54
60
  | impact | negative |
55
- | tag_score | 50.0 |
61
+ | tag_debt | 50.0 |
56
62
  | message | There are no possible keys for orders table to be used, can result into full scan |
57
63
  | suggestion | Please add index keys for orders table |
58
64
  +------------+---------------------------------------------------------------------------------------------------------------------+
59
65
  | column | key |
60
66
  | impact | negative |
61
- | tag_score | 50.0 |
67
+ | tag_debt | 50.0 |
62
68
  | message | There is no index key used for orders table, and can result into full scan of the orders table |
63
69
  | suggestion | Please use index from possible_keys: absent or add new one to orders table as per the requirements. |
64
70
  +------------+---------------------------------------------------------------------------------------------------------------------+
65
71
  +------------------------------------------------------------------------------------+
66
72
  | users |
67
73
  +------------+-----------------------------------------------------------------------+
68
- | score | 130.0 |
74
+ | debt | 130.0 |
69
75
  +------------+-----------------------------------------------------------------------+
70
76
  | column | detailed#used_columns |
71
77
  | impact | negative |
72
- | tag_score | 130.0 |
78
+ | tag_debt | 130.0 |
73
79
  | message | You have selected 18 columns, You should not select too many columns. |
74
80
  | suggestion | Please only select required columns. |
75
81
  +------------+-----------------------------------------------------------------------+
76
82
  ```
77
83
 
78
- ### Add a logger for every query
79
-
80
- Add `QueryPolice.subscribe_logger` to your initial load file like `application.rb`
81
-
82
- You can make logger silence of error using `QueryPolice.subscribe_logger silent: true`.
83
-
84
- You can change logger config using `QueryPolice logger_config: <config>`, default logger_config `{'negative' => true}`, options `positive: <Boolean>, caution: <Boolean>`.
85
-
86
- ---
84
+ ### Analysis for a Impact
87
85
 
88
- ## How it works?
86
+ To print pretty analysis for different impacts
87
+ ```ruby
88
+ analysis = QueryPolice.analyse("select * from users")
89
+ puts analysis.pretty_analysis_for('positive') # impact negative, positive, caution
89
90
 
90
- 1. Query police converts the relation into SQL query
91
-
92
- 2. Query police generates an execution plan using EXPLAIN and EXPLAIN format=JSON based on the configuration.
93
-
94
- 3. Query police load rules from the config file.
95
-
96
- 4. Query police apply rules on the execution plan and generate a new analysis object.
97
-
98
- 5. Analysis object provides different methods to print the analysis in a more descriptive format.
99
-
100
-
101
- ## Execution plan
102
-
103
- We have 2 possible execution plans:-
104
-
105
- Normal - using `EXPLAIN`
106
-
107
- Detailed - using `EXPLAIN format=JSON`
108
-
109
- **NOTE:** By default Detailed execution plan is added in the final execution plan, you can remove that by `QueryPolice.detailed=false`
110
-
111
- ### Normal execution plan
112
-
113
- Generated using `EXPAIN <query>`
114
-
115
- **Result**
116
-
117
- | id | select_type | table | partitions | type. | possible_keys | key | key_len | ref | rows | filtered | Extra |
118
- |----|-------------|---------|------------|--------|---------------------------|---------------------|---------|-----------------------------|------|----------|--------------------------|
119
- | 1 | SIMPLE | profile | NULL. | index | fk_rails_249a7ebca1 | fk_rails_249a7ebca1 | 5 | NULL | 603 | 100.00 | Using where; Using index |
120
- | 1 | SIMPLE | users | NULL | eq_ref | PRIMARY,index_users_on_id | PRIMARY | 4 | development.profile.user_id |1 | 100.00 | NULL |
121
-
122
-
123
- The result for this is added as it is in the final execution plan
91
+ # puts
92
+ # +----------------------------------------------------------+
93
+ # | users |
94
+ # +-----------+----------------------------------------------+
95
+ # | debt | 330.0 |
96
+ # +-----------+----------------------------------------------+
97
+ # | column | select_type |
98
+ # | impact | positive |
99
+ # | tag_debt | 0 |
100
+ # | message | A simple query without subqueries or unions. |
101
+ # +-----------+----------------------------------------------+
102
+ ```
124
103
 
125
- **Eg.**
104
+ ### Analysis for Multiple Impacts
126
105
 
106
+ To print pretty analysis for multiple impacts, default: `{ 'negative' => true, 'caution' => true }`
107
+ ```ruby
108
+ analysis = QueryPolice.analyse("select * from users")
109
+ puts analysis.pretty_analysis({'negative' => true, 'positive' => true})
127
110
  ```
128
- {
129
- "profile" => {
130
- "id" => 1,
131
- "select_type" => "SIMPLE",
132
- "table" => "profile",
133
- "partitions" => nil,
134
- "type" => "index",
135
- "possible_keys" => "fk_rails_249a7ebca1",
136
- "key" => "fk_rails_249a7ebca1",
137
- "key_len" => "5",
138
- "ref" => nil,
139
- "rows" => 603,
140
- "filtered" => 100.0,
141
- "Extra" => "Using where; Using index"
142
- },
143
- "users" => {
144
- "id" => 1,
145
- "select_type" => "SIMPLE",
146
- "table" => "users",
147
- "partitions" => nil,
148
- "type" => "eq_ref",
149
- "possible_keys" => "PRIMARY,index_users_on_id",
150
- "key" => "PRIMARY",
151
- "key_len" => "4",
152
- "ref" => "development.profile.user_id",
153
- "rows" => 1,
154
- "filtered" => 100.0,
155
- "Extra" => nil
156
- }
157
- }
158
- ```
159
-
160
- ### Detailed execution plan
161
111
 
162
- Generated using `EXPAIN format=JSON <query>`
112
+ ### Query debt
163
113
 
164
- **Truncated Result**
114
+ To get the final debt of the query
115
+ ```ruby
116
+ analysis = QueryPolice.analyse("select * from users")
117
+ puts analysis.query_debt
165
118
 
119
+ # puts
120
+ # 100.0
166
121
  ```
167
- {
168
- "query_block": {
169
- "select_id": 1,
170
- "cost_info": {
171
- "query_cost": "850.20"
172
- },
173
- "nested_loop": [
174
- {
175
- "table": {
176
- "table_name": "profile",
177
- "access_type": "index",
178
- "key_length": "5",
179
- "cost_info": {
180
- "read_cost": "6.00",
181
- "eval_cost": "120.60",
182
- "prefix_cost": "126.60",
183
- "data_read_per_join": "183K"
184
- },
185
- "used_columns": [
186
- "user_id"
187
- ],
188
- "attached_condition": "(`development`.`profile`.`user_id` is not null)"
189
- }
190
- }
191
- ]
192
- }
193
- }
194
- ```
195
-
196
122
 
197
- The result for this is added in the flattened form to the final execution plan, where the `detailed#` prefix is added before each key.
123
+ #### Significance
124
+ Query debt signifies the quality of the query, high value represents a bad query.
125
+ - `0 - 199` - Good Query
126
+ - `200 - 499` - Potentially Bad query
127
+ - `>=500` - Bad query
198
128
 
199
- **Truncated Eg.**
129
+ ### Word wrap
200
130
 
201
- ```
202
- {
203
- "detailed#key_length" => "5",
204
- "detailed#rows_examined_per_scan" => 603,
205
- "detailed#rows_produced_per_join" => 603,
206
- "detailed#filtered" => "100.00",
207
- "detailed#using_index" => true,
208
- "detailed#cost_info#read_cost" => "6.00",
209
- "detailed#cost_info#eval_cost" => "120.60",
210
- "detailed#cost_info#prefix_cost" => "126.60",
211
- "detailed#cost_info#data_read_per_join" => "183K",
212
- "detailed#used_columns" => ["user_id"]
213
- ...
131
+ To change the word wrap width for the pretty analysis result, default value: `100`
132
+ ```ruby
133
+ analysis = QueryPolice.analyse("select * from users")
134
+ puts analysis.pretty_analysis_for('positive', {'wrap_width' => 40})
135
+ # or
136
+ puts analysis.pretty_analysis({'positive' => true, 'wrap_width' => 40})
137
+
138
+ # puts
139
+ # +--------------------------------------------------+
140
+ # | users |
141
+ # +-----------+--------------------------------------+
142
+ # | debt | 330.0 |
143
+ # +-----------+--------------------------------------+
144
+ # | column | select_type |
145
+ # | impact | positive |
146
+ # | tag_debt | 0 |
147
+ # | message | A simple query without subqueries or |
148
+ # | | unions. |
149
+ # +-----------+--------------------------------------+
150
+
151
+ puts analysis.pretty_analysis_for('positive', {'wrap_width' => 20})
152
+ # or
153
+ puts analysis.pretty_analysis({'positive' => true, 'wrap_width' => 20})
154
+
155
+ # puts
156
+ # +--------------------------------+
157
+ # | users |
158
+ # +-----------+--------------------+
159
+ # | debt | 330.0 |
160
+ # +-----------+--------------------+
161
+ # | column | select_type |
162
+ # | impact | positive |
163
+ # | tag_debt | 0 |
164
+ # | message | A simple query |
165
+ # | | without subqueries |
166
+ # | | or unions. |
167
+ # +-----------+--------------------+
168
+ ```
169
+
170
+ ### Skip footer
171
+
172
+ To skip the footer (added in [query police config](#analysis-footer)) after an analysis of a query
173
+ ```ruby
174
+ analysis = QueryPolice.analyse("select * from users")
175
+ puts analysis.pretty_analysis_for('positive', {'skip_footer' => true})
176
+ # or
177
+ puts analysis.pretty_analysis({'positive' => true, 'skip_footer' => true})
214
178
  ```
215
179
 
180
+ ### Analysis Footer
181
+ To define a footer text that will be added after an analysis for a query, by default there is no footer
182
+ ```ruby
183
+ QueryPolice.analysis_footer = 'Please check more details with this link...'
184
+ # or
185
+ QueryPolice.configure do |config|
186
+ config.analysis_footer = 'Please check more details with this link...'
187
+ end
188
+
189
+ # puts
190
+ # +----------------------------------------------------------+
191
+ # | users |
192
+ # +-----------+----------------------------------------------+
193
+ # | debt | 330.0 |
194
+ # +-----------+----------------------------------------------+
195
+ # | column | select_type |
196
+ # | impact | positive |
197
+ # | tag_debt | 0 |
198
+ # | message | A simple query without subqueries or unions. |
199
+ # +-----------+----------------------------------------------+
200
+ # Please check more details with this link...
201
+ ```
202
+
203
+ ### Custom rules path
204
+
205
+ To define custom rules path (More details about [how to define custom rules](#how-to-define-custom-rules))
206
+ ```ruby
207
+ QueryPolice.rules_path = 'path/to/rules/file.<json/yml>'
208
+ # or
209
+ QueryPolice.configure do |config|
210
+ config.rules_path = 'path/to/rules/file.<json/yml>'
211
+ end
212
+ ```
216
213
 
214
+ ### Verbosity
217
215
 
218
- ##### Flatten
219
-
220
- `{a: {b: 1}, c: 2}` is converted into `{a#b: 1, c: 2}`.
216
+ Verbosity defines which `EXPLAIN` result should be used for analysis. (More details about [EXPLAIN vs Detailed EXPLAIN](#execution-plan))
217
+ - `basic` - It uses only `EXPLAIN`(basic) result
218
+ - `detailed` - It uses both `EXPLAIN`(basic) and `EXPLAIN format=json`(detailed) results `(default value)`
219
+ ```ruby
220
+ QueryPolice.verbosity = "detailed"
221
+ # or
222
+ QueryPolice.configure do |config|
223
+ config.verbosity = "detailed"
224
+ end
225
+ ```
221
226
 
227
+ ### Disable Actions
222
228
 
223
- ## Analysis object
229
+ To disable actions that are performed on each query.
230
+ ```ruby
231
+ # Note: A logger action is already added, so query police will log pretty analysis after each query by default
232
+ QueryPolice.action_enabled = false
233
+ # or
234
+ QueryPolice.configure do |config|
235
+ config.action_enabled = false
236
+ end
237
+ ```
224
238
 
225
- Analysis object stores a detailed analysis report of a relation inside `:tables :summary attributes`.
239
+ ### Custom Actions
226
240
 
227
- #### Attributes
241
+ To add custom actions, by default a logger action is already added and enabled (More details about [Analysis Object](#analysis-object))
242
+ ```ruby
243
+ QueryPolice.add_action do |analysis| # analysis object for the query
244
+ puts analysis.tables
245
+ end
246
+ ```
228
247
 
229
- **tables [Hash] - detailed table analysis**
248
+ ### Logger config
230
249
 
231
- ```
232
- {
233
- 'users' => {
234
- 'id'=>1,
235
- 'name' => 'users', # table alias user in the execution plan
236
- 'score' => <float> # score for the table
237
- 'analysis' => {
238
- 'type' => { # attribute name
239
- 'value' => <string>, # raw value of attribute in execution plan
240
- 'tags' => {
241
- 'all' => { # tag based on the value of a attribute
242
- 'impact'=> <string>, # negative, positive, cautions
243
- 'warning'=> <string>, # Eg. 'warning to represent the issue'
244
- 'suggestions'=> <string> # Eg. 'some follow-up suggestions'
245
- 'score' => <float> # score for the tag
246
- }
247
- }
248
- }
249
- }
250
- }
251
- }
252
- ```
253
- **summary [Hash] - hash of analysis summary**
250
+ To change the logger options which will be used to generate analysis
251
+ ```ruby
252
+ QueryPolice.logger_options = {'negative' => true}
253
+ # or
254
+ QueryPolice.configure do |config|
255
+ config.logger_options = {'negative' => true}
256
+ end
254
257
 
255
- ```
256
- {
257
- 'cardinality' => {
258
- 'amount' => 10,
259
- 'warning' => 'warning to represent the issue',
260
- 'suggestions' => 'some follow up suggestions',
261
- 'score' => 100.0
262
- }
263
- }
258
+ # default logger_config: {'negative' => true, 'caution' => true}
259
+ # options negative: <Boolean>, positive: <Boolean>, caution: <Boolean>, wrap_width: <Integer>, skip_footer: <Boolean>
264
260
  ```
265
261
 
262
+ ---
266
263
 
264
+ ## How to define custom rules?
267
265
 
268
- ## How to define rules?
269
-
270
- Rules defined in the json file at config_path is applied to the execution plan. We have variety of option to define rules.
266
+ Rules defined in the `json/yaml` file at rules_path is applied to the execution plan. Query Police have variety of option to define rules.
271
267
 
272
268
  You can change this by `QueryPolice.rules_path=<path>` and define your own rules
273
269
 
274
270
  ### Rule Structure
271
+ **Note:** Check Query Police default rules defined at [rules.json](lib/query_police/rules.json) or examples at [examples/rules/](examples/rules/) for better clarity
275
272
 
276
273
  A basic rule structure -
277
- ```
274
+
275
+ JSON
276
+ ```json
278
277
  "<column_name>": {
279
- "description": <string>,
280
- "value_type": <string>,
281
- "delimiter": <string>,
278
+ "description": "<string>",
279
+ "value_type": "<string>",
280
+ "delimiter": "<string>",
282
281
  "rules": {
283
282
  "<rule>": {
284
- "amount": <integer>
285
- "impact": <string>,
286
- "message": <string>,
287
- "suggestion": <string>,
288
- "score": {
289
- "value": <integer>,
290
- "type": <string>
283
+ "amount": "<integer>",
284
+ "impact": "<string>",
285
+ "message": "<string>",
286
+ "suggestion": "<string>",
287
+ "debt": {
288
+ "value": "<integer>",
289
+ "type": "<string>"
291
290
  }
292
291
  }
293
292
  }
294
293
  }
295
294
  ```
296
- - `<column_name>` - attribute name in the final execution plan.
297
-
295
+ YAML
296
+ ```yaml
297
+ <column_name>:
298
+ description: <string>
299
+ value_type: <string>
300
+ delimiter: <string>
301
+ rules:
302
+ <rule>:
303
+ amount: <integer>
304
+ impact: <string>
305
+ message: <string>
306
+ suggestion: <string>
307
+ debt:
308
+ value: <integer>
309
+ type: <string>
310
+ ```
311
+ - `<column_name>` - attribute name in the final execution plan. (more details about [attributes](#attributes-for-rules))
298
312
  - `description` - description of the attribute
299
-
300
313
  - `value_type` - value type of the attribute
301
-
302
314
  - `delimiter` - delimiter to parse array type attribute values, if no delimiter is passed engine will consider value is already in array form.
303
-
304
315
  - `<rule>` - kind of rule for the attribute
305
-
306
316
  - `<tag>` - direct value match eg. ALL, SIMPLE
307
-
308
317
  - `absent` - when the value is missing
309
-
310
318
  - `threshold` - a greater than threshold check based on the amount set inside the rule.
311
-
312
319
  - `amount` - the amount of threshold that needs to check for
313
-
314
320
  - length for string
315
-
316
321
  - value for number
317
-
318
322
  - size for the array
319
-
320
323
  - `impact` - impact of the rule
321
-
322
324
  - `negative`
323
-
324
325
  - `postive`
325
-
326
326
  - `caution`
327
-
328
327
  - `message` - the message needs to provide the significance of the rule
329
-
330
328
  - `suggestion` - suggestion on how we can fix the issue
331
- - `score` - score-related config that will be affected to final query score
332
- - `value` - value that will be added to the query score
333
- - `type` - the type of scoring that will be added to the query score
329
+ - `debt` - debt-related config that will be affected to final query debt
330
+ - `value` - value that will be added to the query debt
331
+ - `type` - the type of scoring that will be added to the query debt
334
332
  - `base`- value
335
333
  - `relative` - value * (amount for that column in query)
336
- - `treshold_relative` - (value - (threshold amount)) * (amount for that column in query)
334
+ - `threshold_relative` - (value - (threshold amount)) * (amount for that column in query)
337
335
 
338
336
 
339
337
 
@@ -342,34 +340,22 @@ A basic rule structure -
342
340
  We can define dynamic messages and suggestions with variables provided by the engine.
343
341
 
344
342
  - `$amount` - the amount of the value
345
-
346
343
  - length for string
347
-
348
344
  - value for number
349
-
350
345
  - size for the array
351
-
352
346
  - `$column` - attribute name
353
-
354
347
  - `$impact` - impact for the rule
355
-
356
348
  - `$table` - table alias used in the plan
357
-
358
349
  - `$tag` - tag for which rule is applied
359
-
360
350
  - `$value` - original parsed value
361
-
362
351
  - `$<column_name>` - the value of that specific column in that table
363
-
364
352
  - `$amount_<column_name>` - amount of that specific column
365
353
 
366
-
367
-
368
354
  ### Rules Examples
369
355
 
370
356
  #### Basic rule example
371
-
372
- ```
357
+ File: [JSON](examples/rules/json/basic_rule.json) | [YAML](examples/rules/yaml/basic_rule.yml)
358
+ ```json
373
359
  "type": {
374
360
  "description": "Join used in the query for a specific table.",
375
361
  "value_type": "string",
@@ -383,11 +369,12 @@ We can define dynamic messages and suggestions with variables provided by the en
383
369
  "impact": "negative",
384
370
  "message": "Entire $table table is scanned to find matching rows, you have $amount_possible_keys possible keys to use.",
385
371
  "suggestion": "Use index here. You can use index from possible key: $possible_keys or add new one to $table table as per the requirements.",
386
- "score": {
372
+ "debt": {
387
373
  "value": 200,
388
374
  "type": "base"
389
375
  }
390
376
  }
377
+ }
391
378
  }
392
379
  ```
393
380
  For the above rule, dynamic message will be generated as-
@@ -401,8 +388,8 @@ Use index here. You can use index from possible key: ["PRIMARY", "user_email"] o
401
388
 
402
389
 
403
390
  #### Absent rule example
404
-
405
- ```
391
+ File: [JSON](examples/rules/json/absent_rule.json) | [YAML](examples/rules/yaml/absent_rule.yml)
392
+ ```json
406
393
  "key": {
407
394
  "description": "index key used for the table",
408
395
  "value_type": "string",
@@ -427,8 +414,8 @@ Please use index from possible_keys: ["PRIMARY", "user_email"] or add new one to
427
414
 
428
415
 
429
416
  #### Threshold rule example
430
-
431
- ```
417
+ File: [JSON](examples/rules/json/threshold_rule.json) | [YAML](examples/rules/yaml/threshold_rule.yml)
418
+ ```json
432
419
  "possible_keys": {
433
420
  "description": "Index keys possible for a specifc table",
434
421
  "value_type": "array",
@@ -454,8 +441,8 @@ Please check if there are extra indexes in users table.
454
441
 
455
442
 
456
443
  #### Complex Detailed rule example
457
-
458
- ```
444
+ File: [JSON](examples/rules/json/complex_detailed_rule.json) | [YAML](examples/rules/yaml/complex_detailed_rule.yml)
445
+ ```json
459
446
  "detailed#used_columns": {
460
447
  "description": "",
461
448
  "value_type": "array",
@@ -465,9 +452,9 @@ Please check if there are extra indexes in users table.
465
452
  "impact": "negative",
466
453
  "message": "You have selected $amount columns, You should not select too many columns.",
467
454
  "suggestion": "Please only select required columns.",
468
- "score": {
455
+ "debt": {
469
456
  "value": 10,
470
- "type": "treshold_relative"
457
+ "type": "threshold_relative"
471
458
  }
472
459
  }
473
460
  }
@@ -483,9 +470,9 @@ Please only select required columns.
483
470
  ```
484
471
 
485
472
 
486
- ### Summary
473
+ ### Summary in Analysis
487
474
 
488
- You can define similar rules for the summary. Current summary attribute supported -
475
+ You can define similar rules for the summary. Current only one attribute is supported in summary -
489
476
 
490
477
  - `cardinality` - cardinality based on all tables
491
478
 
@@ -493,7 +480,7 @@ You can define similar rules for the summary. Current summary attribute supporte
493
480
 
494
481
 
495
482
 
496
- ### Attributes
483
+ ### Attributes for Rules
497
484
 
498
485
  There are a lot of attributes for you to use based on the final execution plan.
499
486
 
@@ -509,6 +496,185 @@ To check more keys you can use `EXPLAIN format=JSON <query>`
509
496
 
510
497
  ---
511
498
 
499
+ ## How Query Police works?
500
+
501
+ 1. Query police converts the relation into SQL query
502
+
503
+ 2. Query police generates an execution plan using EXPLAIN and EXPLAIN format=JSON based on the configuration.
504
+
505
+ 3. Query police load rules from the config file.
506
+
507
+ 4. Query police apply rules on the execution plan and generate a new analysis object.
508
+
509
+ 5. Analysis object provides different methods to print the analysis in a more descriptive format.
510
+
511
+
512
+ ## Execution plan
513
+
514
+ We have 2 possible execution plans:-
515
+
516
+ Normal - using `EXPLAIN`
517
+
518
+ Detailed - using `EXPLAIN format=JSON`
519
+
520
+ **NOTE:** By default Detailed execution plan is added in the final execution plan, you can remove that by `QueryPolice.detailed=false`
521
+
522
+ ### Normal execution plan
523
+
524
+ Generated using `EXPAIN <query>`
525
+
526
+ **Result**
527
+
528
+ | id | select_type | table | partitions | type. | possible_keys | key | key_len | ref | rows | filtered | Extra |
529
+ |----|-------------|---------|------------|--------|---------------------------|---------------------|---------|-----------------------------|------|----------|--------------------------|
530
+ | 1 | SIMPLE | profile | NULL. | index | fk_rails_249a7ebca1 | fk_rails_249a7ebca1 | 5 | NULL | 603 | 100.00 | Using where; Using index |
531
+ | 1 | SIMPLE | users | NULL | eq_ref | PRIMARY,index_users_on_id | PRIMARY | 4 | development.profile.user_id |1 | 100.00 | NULL |
532
+
533
+
534
+ The result for this is added as it is in the final execution plan
535
+
536
+ **Eg.**
537
+
538
+ ```ruby
539
+ {
540
+ "profile" => {
541
+ "id" => 1,
542
+ "select_type" => "SIMPLE",
543
+ "table" => "profile",
544
+ "partitions" => nil,
545
+ "type" => "index",
546
+ "possible_keys" => "fk_rails_249a7ebca1",
547
+ "key" => "fk_rails_249a7ebca1",
548
+ "key_len" => "5",
549
+ "ref" => nil,
550
+ "rows" => 603,
551
+ "filtered" => 100.0,
552
+ "Extra" => "Using where; Using index"
553
+ },
554
+ "users" => {
555
+ "id" => 1,
556
+ "select_type" => "SIMPLE",
557
+ "table" => "users",
558
+ "partitions" => nil,
559
+ "type" => "eq_ref",
560
+ "possible_keys" => "PRIMARY,index_users_on_id",
561
+ "key" => "PRIMARY",
562
+ "key_len" => "4",
563
+ "ref" => "development.profile.user_id",
564
+ "rows" => 1,
565
+ "filtered" => 100.0,
566
+ "Extra" => nil
567
+ }
568
+ }
569
+ ```
570
+
571
+ ### Detailed execution plan
572
+
573
+ Generated using `EXPAIN format=JSON <query>`
574
+
575
+ **Truncated Result**
576
+
577
+ ```ruby
578
+ {
579
+ "query_block": {
580
+ "select_id": 1,
581
+ "cost_info": {
582
+ "query_cost": "850.20"
583
+ },
584
+ "nested_loop": [
585
+ {
586
+ "table": {
587
+ "table_name": "profile",
588
+ "access_type": "index",
589
+ "key_length": "5",
590
+ "cost_info": {
591
+ "read_cost": "6.00",
592
+ "eval_cost": "120.60",
593
+ "prefix_cost": "126.60",
594
+ "data_read_per_join": "183K"
595
+ },
596
+ "used_columns": [
597
+ "user_id"
598
+ ],
599
+ "attached_condition": "(`development`.`profile`.`user_id` is not null)"
600
+ }
601
+ }
602
+ ]
603
+ }
604
+ }
605
+ ```
606
+
607
+
608
+ The result for this is added in the flattened form to the final execution plan, where the `detailed#` prefix is added before each key.
609
+
610
+ **Truncated Eg.**
611
+
612
+ ```ruby
613
+ {
614
+ "detailed#key_length" => "5",
615
+ "detailed#rows_examined_per_scan" => 603,
616
+ "detailed#rows_produced_per_join" => 603,
617
+ "detailed#filtered" => "100.00",
618
+ "detailed#using_index" => true,
619
+ "detailed#cost_info#read_cost" => "6.00",
620
+ "detailed#cost_info#eval_cost" => "120.60",
621
+ "detailed#cost_info#prefix_cost" => "126.60",
622
+ "detailed#cost_info#data_read_per_join" => "183K",
623
+ "detailed#used_columns" => ["user_id"]
624
+ ...
625
+ ```
626
+
627
+
628
+
629
+ ##### Flatten
630
+
631
+ `{a: {b: 1}, c: 2}` is converted into `{a#b: 1, c: 2}`.
632
+
633
+ ---
634
+
635
+ ## Analysis object
636
+
637
+ Analysis object stores a detailed analysis report of a relation inside `:tables :summary attributes`.
638
+
639
+ #### Attributes
640
+
641
+ **tables [Hash] - detailed table analysis**
642
+
643
+ ```ruby
644
+ {
645
+ 'users' => {
646
+ 'id'=>1,
647
+ 'name' => 'users', # table alias user in the execution plan
648
+ 'debt' => <float> # debt for the table
649
+ 'analysis' => {
650
+ 'type' => { # attribute name
651
+ 'value' => <string>, # raw value of attribute in execution plan
652
+ 'tags' => {
653
+ 'all' => { # tag based on the value of a attribute
654
+ 'impact'=> <string>, # negative, positive, cautions
655
+ 'warning'=> <string>, # Eg. 'warning to represent the issue'
656
+ 'suggestions'=> <string> # Eg. 'some follow-up suggestions'
657
+ 'debt' => <float> # debt for the tag
658
+ }
659
+ }
660
+ }
661
+ }
662
+ }
663
+ }
664
+ ```
665
+ **summary [Hash] - hash of analysis summary**
666
+
667
+ ```ruby
668
+ {
669
+ 'cardinality' => {
670
+ 'amount' => 10,
671
+ 'warning' => 'warning to represent the issue',
672
+ 'suggestions' => 'some follow up suggestions',
673
+ 'debt' => 100.0
674
+ }
675
+ }
676
+ ```
677
+
512
678
  ## Development
513
679
 
514
680
  After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.