query_reviewer 0.1
Sign up to get free protection for your applications and to get access to all the features.
- data/MIT-LICENSE +20 -0
- data/README.md +118 -0
- data/Rakefile +24 -0
- data/lib/query_reviewer/array_extensions.rb +29 -0
- data/lib/query_reviewer/controller_extensions.rb +65 -0
- data/lib/query_reviewer/mysql_adapter_extensions.rb +90 -0
- data/lib/query_reviewer/mysql_analyzer.rb +62 -0
- data/lib/query_reviewer/query_warning.rb +17 -0
- data/lib/query_reviewer/rails.rb +33 -0
- data/lib/query_reviewer/sql_query.rb +130 -0
- data/lib/query_reviewer/sql_query_collection.rb +103 -0
- data/lib/query_reviewer/sql_sub_query.rb +45 -0
- data/lib/query_reviewer/tasks.rb +8 -0
- data/lib/query_reviewer/views/_box.html.erb +11 -0
- data/lib/query_reviewer/views/_box_ajax.js +34 -0
- data/lib/query_reviewer/views/_box_body.html.erb +73 -0
- data/lib/query_reviewer/views/_box_disabled.html.erb +2 -0
- data/lib/query_reviewer/views/_box_header.html.erb +1 -0
- data/lib/query_reviewer/views/_box_includes.html.erb +234 -0
- data/lib/query_reviewer/views/_explain.html.erb +30 -0
- data/lib/query_reviewer/views/_js_includes.html.erb +68 -0
- data/lib/query_reviewer/views/_js_includes_new.html.erb +68 -0
- data/lib/query_reviewer/views/_profile.html.erb +26 -0
- data/lib/query_reviewer/views/_query_sql.html.erb +8 -0
- data/lib/query_reviewer/views/_query_trace.html.erb +31 -0
- data/lib/query_reviewer/views/_query_with_warning.html.erb +54 -0
- data/lib/query_reviewer/views/_spectrum.html.erb +10 -0
- data/lib/query_reviewer/views/_warning_no_query.html.erb +8 -0
- data/lib/query_reviewer/views/query_review_box_helper.rb +98 -0
- data/lib/query_reviewer.rb +54 -0
- data/query_reviewer_defaults.yml +39 -0
- metadata +96 -0
data/MIT-LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2007 [name of plugin creator]
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,118 @@
|
|
1
|
+
# QueryReviewer #
|
2
|
+
|
3
|
+
## Introduction ##
|
4
|
+
|
5
|
+
QueryReviewer is an advanced SQL query analyzer. It accomplishes the following goals:
|
6
|
+
|
7
|
+
* View all EXPLAIN output for all SELECT queries to generate a page
|
8
|
+
* Rate a page's SQL usage into one of three categories: OK, WARNING, CRITICAL
|
9
|
+
* Attach meaningful warnings to individual queries, and collections of queries
|
10
|
+
* Display interactive summary on page
|
11
|
+
|
12
|
+
## This Fork ##
|
13
|
+
|
14
|
+
I use this utility for most of my rails projects. Still the best out there in my opinion for analyzing and understanding your generated SQL queries. I forked the original [query_reviewer](https://github.com/dsboulder/query_reviewer) and applied a collection of patches that have been made since the plugin was created. A list of the biggest additions below:
|
15
|
+
|
16
|
+
* Snazzed up the README into markdown for better readability
|
17
|
+
* Full compatibility for Rails 3 (including Railtie)
|
18
|
+
* Cleanup and move rake task to `lib/tasks` to fix deprecation warnings
|
19
|
+
* Added gemspec for use with Bundler (as a gem)
|
20
|
+
* Fixed missing tags and additional XHTML escaping
|
21
|
+
* Fix SQL escaping for better XHTML compatibility
|
22
|
+
* Fixes for deprecation warnings and for 1.9 compatiblity
|
23
|
+
* Converts templates to more modern foo.html.erb naming
|
24
|
+
|
25
|
+
Last commit to the main repository was on March 30th, 2009. This fork compiles a variety of patches that were made since that time along with additional work to support compatibility with 1.9 and Rails 3. **Also:** If anyone else creates generally useful enhancements to this utility please start by forking this and then issue me a pull request.
|
26
|
+
|
27
|
+
**Note:** This plugin should work for Rails 2.X and Rails 3. Support for Rails 3 has been confirmed in the latest revision (with fixed deprecation warnings).
|
28
|
+
|
29
|
+
## Installation ##
|
30
|
+
|
31
|
+
All you have to do is install it into your Rails 2 or 3 project.
|
32
|
+
|
33
|
+
Right now if you use bundler, simply add this to your Gemfile:
|
34
|
+
|
35
|
+
# Gemfile
|
36
|
+
gem "query_reviewer", :git => "git://github.com/nesquena/query_reviewer.git"
|
37
|
+
|
38
|
+
If you are not using bundler, you might want to [start using it](http://gembundler.com/rails23.html). You can also install this as a plugin:
|
39
|
+
|
40
|
+
script/plugin install git://github.com/nesquena/query_reviewer.git
|
41
|
+
|
42
|
+
In Rails 2, if the rake tasks are not loaded automatically (as a gem), you’ll need to add the following to your Rakefile:
|
43
|
+
|
44
|
+
# Rakefile
|
45
|
+
begin
|
46
|
+
require 'query_reviewer/tasks'
|
47
|
+
rescue LoadError
|
48
|
+
STDERR.puts "The query_reviewer gem could not be found!"
|
49
|
+
end
|
50
|
+
|
51
|
+
You can then run:
|
52
|
+
|
53
|
+
$ rake query_reviewer:setup
|
54
|
+
|
55
|
+
Which will create `config/query_reviewer.yml` in your application, see below for what these options mean.
|
56
|
+
If you don't create a config file, the gem will use the default in `vendor/plugins/query_reviewer`.
|
57
|
+
|
58
|
+
## Configuration ##
|
59
|
+
|
60
|
+
The configuration file allows you to set configuration parameters shared across all rails environment, as well as overriding those shared parameteres with environment-specific parameters (such as disabling analysis on production!)
|
61
|
+
|
62
|
+
* `enabled`: whether any output or query analysis is performed. Set this false in production!
|
63
|
+
* `inject_view`: controls whether the output automatically is injected before the </body> in HTML output.
|
64
|
+
* `profiling`: when enabled, runs the MySQL SET PROFILING=1 for queries longer than the `warn_duration_threshold` / 2.0
|
65
|
+
* `production_data`: whether the duration of a query should be taken into account
|
66
|
+
* `stack_trace_lines`: number of lines of call stack to include in the "short" version of the stack trace
|
67
|
+
* `trace_includes_vendor`: whether the "short" verison of the stack trace should include files in /vendor
|
68
|
+
* `trace_includes_lib`: whether the "short" verison of the stack trace should include files in /lib
|
69
|
+
* `warn_severity`: the severity of problem that merits "WARNING" status
|
70
|
+
* `critical_severity`: the severity of problem that merits "CRITICAL" status
|
71
|
+
* `warn_query_count`: the number of queries in a single request that merits "WARNING" status
|
72
|
+
* `critical_query_count`: the number of queries in a single request that merits "CRITICAL" status
|
73
|
+
* `warn_duration_threshold`: how long a query must take in seconds (float) before it's considered "WARNING"
|
74
|
+
* `critical_duration_threshold`: how long a query must take in seconds (float) before it's considered "CRITICIAL"
|
75
|
+
|
76
|
+
## Example ##
|
77
|
+
|
78
|
+
If you disable the inject_view option above, you'll need to manually put the analyzer's output into your view:
|
79
|
+
|
80
|
+
# view.html.haml
|
81
|
+
= query_review_output
|
82
|
+
|
83
|
+
and that will display the analyzer view!
|
84
|
+
|
85
|
+
## Resources ##
|
86
|
+
|
87
|
+
Random collection of resources that might be interesting related to this utility:
|
88
|
+
|
89
|
+
* <http://blog.purifyapp.com/2010/06/15/optimise-your-mysql/>
|
90
|
+
* <http://www.tatvartha.com/2009/09/rails-optimizing-database-indexes-using-query_analyzer-and-query_reviewer/>
|
91
|
+
* <http://www.geekskillz.com/articles/using-indexes-to-improve-rails-performance>
|
92
|
+
* <http://www.williambharding.com/blog/rails/rails-mysql-indexes-step-1-in-pitiful-to-prime-performance/>
|
93
|
+
* <http://guides.rubyonrails.org/performance_testing.html>
|
94
|
+
|
95
|
+
Other related gems that prove useful for database optimization:
|
96
|
+
|
97
|
+
* [bullet](https://github.com/flyerhzm/bullet)
|
98
|
+
* [slim-scrooge](https://github.com/sdsykes/slim_scrooge)
|
99
|
+
* [slim-attributes](https://github.com/sdsykes/slim-attributes)
|
100
|
+
|
101
|
+
## Alternatives ##
|
102
|
+
|
103
|
+
There have been other alternatives created since this was originally released. A few of the best are listed below. I for one still prefer this utility over the other options:
|
104
|
+
|
105
|
+
* [rack-bug](https://github.com/brynary/rack-bug)
|
106
|
+
* [rails-footnotes](https://github.com/josevalim/rails-footnotes)
|
107
|
+
* [newrelic-development](http://support.newrelic.com/kb/docs/developer-mode)
|
108
|
+
* [palmist](https://github.com/flyingmachine/palmist)
|
109
|
+
* [query_diet](https://github.com/makandra/query_diet)
|
110
|
+
* [query_trace](https://github.com/ntalbott/query_trace)
|
111
|
+
|
112
|
+
Know of a better alternative? Let me know!
|
113
|
+
|
114
|
+
## Acknowledgements ##
|
115
|
+
|
116
|
+
Created by Kongregate & David Stevenson. Refactorings and compilations of all fixes since was done by Nathan Esquenazi.
|
117
|
+
|
118
|
+
Copyright (c) 2007-2008 Kongregate & David Stevenson, released under the MIT license
|
data/Rakefile
ADDED
@@ -0,0 +1,24 @@
|
|
1
|
+
require 'rake'
|
2
|
+
require 'rake/testtask'
|
3
|
+
require 'rake/rdoctask'
|
4
|
+
require 'bundler'
|
5
|
+
Bundler::GemHelper.install_tasks
|
6
|
+
|
7
|
+
desc 'Default: run unit tests.'
|
8
|
+
task :default => :test
|
9
|
+
|
10
|
+
desc 'Test the query_reviewer plugin.'
|
11
|
+
Rake::TestTask.new(:test) do |t|
|
12
|
+
t.libs << 'lib'
|
13
|
+
t.pattern = 'test/**/*_test.rb'
|
14
|
+
t.verbose = true
|
15
|
+
end
|
16
|
+
|
17
|
+
desc 'Generate documentation for the query_reviewer plugin.'
|
18
|
+
Rake::RDocTask.new(:rdoc) do |rdoc|
|
19
|
+
rdoc.rdoc_dir = 'rdoc'
|
20
|
+
rdoc.title = 'QueryReviewer'
|
21
|
+
rdoc.options << '--line-numbers' << '--inline-source'
|
22
|
+
rdoc.rdoc_files.include('README')
|
23
|
+
rdoc.rdoc_files.include('lib/**/*.rb')
|
24
|
+
end
|
@@ -0,0 +1,29 @@
|
|
1
|
+
module QueryReviewer
|
2
|
+
module ArrayExtensions #taken from query_analyser plugin
|
3
|
+
protected
|
4
|
+
def qa_columnized_row(fields, sized)
|
5
|
+
row = []
|
6
|
+
fields.each_with_index do |f, i|
|
7
|
+
row << sprintf("%0-#{sized[i]}s", f.to_s)
|
8
|
+
end
|
9
|
+
row.join(' | ')
|
10
|
+
end
|
11
|
+
|
12
|
+
public
|
13
|
+
|
14
|
+
def qa_columnized
|
15
|
+
sized = {}
|
16
|
+
self.each do |row|
|
17
|
+
row.values.each_with_index do |value, i|
|
18
|
+
sized[i] = [sized[i].to_i, row.keys[i].length, value.to_s.length].max
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
table = []
|
23
|
+
table << qa_columnized_row(self.first.keys, sized)
|
24
|
+
table << '-' * table.first.length
|
25
|
+
self.each { |row| table << qa_columnized_row(row.values, sized) }
|
26
|
+
table.join("\n ") # Spaces added to work with format_log_entry
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
@@ -0,0 +1,65 @@
|
|
1
|
+
require "action_view"
|
2
|
+
require File.join(File.dirname(__FILE__), "views", "query_review_box_helper")
|
3
|
+
|
4
|
+
module QueryReviewer
|
5
|
+
module ControllerExtensions
|
6
|
+
class QueryViewBase < ActionView::Base
|
7
|
+
include QueryReviewer::Views::QueryReviewBoxHelper
|
8
|
+
end
|
9
|
+
|
10
|
+
def self.included(base)
|
11
|
+
if QueryReviewer::CONFIGURATION["inject_view"]
|
12
|
+
alias_name = defined?(Rails::Railtie) ? :process_action : :perform_action
|
13
|
+
base.alias_method_chain(alias_name, :query_review)
|
14
|
+
end
|
15
|
+
base.alias_method_chain :process, :query_review
|
16
|
+
base.helper_method :query_review_output
|
17
|
+
end
|
18
|
+
|
19
|
+
def query_review_output(ajax = false, total_time = nil)
|
20
|
+
faux_view = QueryViewBase.new([File.join(File.dirname(__FILE__), "views")], {}, self)
|
21
|
+
queries = Thread.current["queries"]
|
22
|
+
queries.analyze!
|
23
|
+
faux_view.instance_variable_set("@queries", queries)
|
24
|
+
faux_view.instance_variable_set("@total_time", total_time)
|
25
|
+
if ajax
|
26
|
+
js = faux_view.render(:partial => "/box_ajax.js")
|
27
|
+
else
|
28
|
+
html = faux_view.render(:partial => "/box")
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
def add_query_output_to_view(total_time)
|
33
|
+
if request.xhr?
|
34
|
+
if cookies["query_review_enabled"]
|
35
|
+
if !response.content_type || response.content_type.include?("text/html")
|
36
|
+
response.body += "<script type=\"text/javascript\">"+query_review_output(true, total_time)+"</script>"
|
37
|
+
elsif response.content_type && response.content_type.include?("text/javascript")
|
38
|
+
response.body += ";\n"+query_review_output(true, total_time)
|
39
|
+
end
|
40
|
+
end
|
41
|
+
else
|
42
|
+
if response.body.is_a?(String) && response.body.match(/<\/body>/i) && Thread.current["queries"]
|
43
|
+
idx = (response.body =~ /<\/body>/i)
|
44
|
+
html = query_review_output(false, total_time)
|
45
|
+
response.body = response.body.insert(idx, html)
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
def perform_action_with_query_review(*args)
|
51
|
+
Thread.current["query_reviewer_enabled"] = cookies["query_review_enabled"]
|
52
|
+
t1 = Time.now
|
53
|
+
r = defined?(Rails::Railtie) ? process_action_without_query_review(*args) : perform_action_without_query_review(*args)
|
54
|
+
t2 = Time.now
|
55
|
+
add_query_output_to_view(t2 - t1)
|
56
|
+
r
|
57
|
+
end
|
58
|
+
alias_method :process_action_with_query_review, :perform_action_with_query_review
|
59
|
+
|
60
|
+
def process_with_query_review(*args) #:nodoc:
|
61
|
+
Thread.current["queries"] = SqlQueryCollection.new
|
62
|
+
process_without_query_review(*args)
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
@@ -0,0 +1,90 @@
|
|
1
|
+
module QueryReviewer
|
2
|
+
module MysqlAdapterExtensions
|
3
|
+
def self.included(base)
|
4
|
+
base.alias_method_chain :select, :review
|
5
|
+
base.alias_method_chain :update, :review
|
6
|
+
base.alias_method_chain :insert, :review
|
7
|
+
base.alias_method_chain :delete, :review
|
8
|
+
end
|
9
|
+
|
10
|
+
def update_with_review(sql, *args)
|
11
|
+
t1 = Time.now
|
12
|
+
result = update_without_review(sql, *args)
|
13
|
+
t2 = Time.now
|
14
|
+
|
15
|
+
create_or_add_query_to_query_reviewer!(sql, nil, t2 - t1, nil, "UPDATE", result)
|
16
|
+
|
17
|
+
result
|
18
|
+
end
|
19
|
+
|
20
|
+
def insert_with_review(sql, *args)
|
21
|
+
t1 = Time.now
|
22
|
+
result = insert_without_review(sql, *args)
|
23
|
+
t2 = Time.now
|
24
|
+
|
25
|
+
create_or_add_query_to_query_reviewer!(sql, nil, t2 - t1, nil, "INSERT")
|
26
|
+
|
27
|
+
result
|
28
|
+
end
|
29
|
+
|
30
|
+
def delete_with_review(sql, *args)
|
31
|
+
t1 = Time.now
|
32
|
+
result = delete_without_review(sql, *args)
|
33
|
+
t2 = Time.now
|
34
|
+
|
35
|
+
create_or_add_query_to_query_reviewer!(sql, nil, t2 - t1, nil, "DELETE", result)
|
36
|
+
|
37
|
+
result
|
38
|
+
end
|
39
|
+
|
40
|
+
def select_with_review(sql, *args)
|
41
|
+
sql.gsub!(/^SELECT /i, "SELECT SQL_NO_CACHE ") if QueryReviewer::CONFIGURATION["disable_sql_cache"]
|
42
|
+
@logger.silence { execute("SET PROFILING=1") } if QueryReviewer::CONFIGURATION["profiling"]
|
43
|
+
t1 = Time.now
|
44
|
+
query_results = select_without_review(sql, *args)
|
45
|
+
t2 = Time.now
|
46
|
+
|
47
|
+
if @logger && sql =~ /^select/i && query_reviewer_enabled?
|
48
|
+
use_profiling = QueryReviewer::CONFIGURATION["profiling"]
|
49
|
+
use_profiling &&= (t2 - t1) >= QueryReviewer::CONFIGURATION["warn_duration_threshold"].to_f / 2.0 if QueryReviewer::CONFIGURATION["production_data"]
|
50
|
+
|
51
|
+
if use_profiling
|
52
|
+
t5 = Time.now
|
53
|
+
@logger.silence { execute("SET PROFILING=1") }
|
54
|
+
t3 = Time.now
|
55
|
+
select_without_review(sql, *args)
|
56
|
+
t4 = Time.now
|
57
|
+
profile = @logger.silence { select_without_review("SHOW PROFILE ALL", *args) }
|
58
|
+
@logger.silence { execute("SET PROFILING=0") }
|
59
|
+
t6 = Time.now
|
60
|
+
Thread.current["queries"].overhead_time += t6 - t5
|
61
|
+
else
|
62
|
+
profile = nil
|
63
|
+
end
|
64
|
+
|
65
|
+
cols = @logger.silence do
|
66
|
+
select_without_review("explain #{sql}", *args)
|
67
|
+
end
|
68
|
+
|
69
|
+
duration = t3 ? [t2 - t1, t4 - t3].min : t2 - t1
|
70
|
+
create_or_add_query_to_query_reviewer!(sql, cols, duration, profile)
|
71
|
+
|
72
|
+
#@logger.debug(format_log_entry("Analyzing #{name}\n", query.to_table)) if @logger.level <= Logger::INFO
|
73
|
+
end
|
74
|
+
query_results
|
75
|
+
end
|
76
|
+
|
77
|
+
def query_reviewer_enabled?
|
78
|
+
Thread.current["queries"] && Thread.current["queries"].respond_to?(:find_or_create_sql_query) && Thread.current["query_reviewer_enabled"]
|
79
|
+
end
|
80
|
+
|
81
|
+
def create_or_add_query_to_query_reviewer!(sql, cols, run_time, profile, command = "SELECT", affected_rows = 1)
|
82
|
+
if query_reviewer_enabled?
|
83
|
+
t1 = Time.now
|
84
|
+
Thread.current["queries"].find_or_create_sql_query(sql, cols, run_time, profile, command, affected_rows)
|
85
|
+
t2 = Time.now
|
86
|
+
Thread.current["queries"].overhead_time += t2 - t1
|
87
|
+
end
|
88
|
+
end
|
89
|
+
end
|
90
|
+
end
|
@@ -0,0 +1,62 @@
|
|
1
|
+
module QueryReviewer
|
2
|
+
module MysqlAnalyzer
|
3
|
+
def do_mysql_analysis!
|
4
|
+
analyze_select_type!
|
5
|
+
analyze_query_type!
|
6
|
+
analyze_key!
|
7
|
+
analyze_extras!
|
8
|
+
analyze_keylen!
|
9
|
+
end
|
10
|
+
|
11
|
+
def analyze_select_type!
|
12
|
+
if select_type.match /uncacheable subquery/
|
13
|
+
warn(:severity => 10, :field => "select_type", :desc => "Subquery must be run once for EVERY row in main query")
|
14
|
+
elsif select_type.match /dependent/
|
15
|
+
warn(:severity => 2, :field => "select_type", :desc => "Dependent subqueries can not be executed while the main query is running")
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
def analyze_query_type!
|
20
|
+
case query_type
|
21
|
+
when "system", "const", "eq_ref" then
|
22
|
+
praise("Yay")
|
23
|
+
when "ref", "ref_or_null", "range", "index_merge" then
|
24
|
+
praise("Not bad eh...")
|
25
|
+
when "unique_subquery", "index_subquery" then
|
26
|
+
#NOT SURE
|
27
|
+
when "index" then
|
28
|
+
warn(:severity => 8, :field => "query_type", :desc => "Full index tree scan (slightly faster than a full table scan)") unless !extra.include?("using where")
|
29
|
+
when "all" then
|
30
|
+
warn(:severity => 9, :field => "query_type", :desc => "Full table scan") unless !extra.include?("using where")
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
def analyze_key!
|
35
|
+
if self.key == "const"
|
36
|
+
praise "Way to go!"
|
37
|
+
elsif self.key.blank? && !self.extra.include?("select tables optimized away")
|
38
|
+
warn :severity => 6, :field => "key", :desc => "No index was used here. In this case, that meant scanning #{self.rows} rows."
|
39
|
+
end
|
40
|
+
end
|
41
|
+
|
42
|
+
def analyze_extras!
|
43
|
+
if self.extra.match(/range checked for each record/)
|
44
|
+
warn :severity => 4, :problem => "Range checked for each record", :desc => "MySQL found no good index to use, but found that some of indexes might be used after column values from preceding tables are known"
|
45
|
+
end
|
46
|
+
|
47
|
+
if self.extra.match(/using filesort/)
|
48
|
+
warn :severity => 2, :problem => "Using filesort", :desc => "MySQL must do an extra pass to find out how to retrieve the rows in sorted order."
|
49
|
+
end
|
50
|
+
|
51
|
+
if self.extra.match(/using temporary/)
|
52
|
+
warn :severity => 10, :problem => "Using temporary table", :desc => "To resolve the query, MySQL needs to create a temporary table to hold the result."
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
def analyze_keylen!
|
57
|
+
if self.key_len && !self.key_len.to_i.nil? && (self.key_len.to_i > QueryReviewer::CONFIGURATION["max_safe_key_length"])
|
58
|
+
warn :severity => 4, :problem => "Long key length (#{self.key_len.to_i})", :desc => "The key used for the index was rather long, potentially affecting indices in memory"
|
59
|
+
end
|
60
|
+
end
|
61
|
+
end
|
62
|
+
end
|
@@ -0,0 +1,17 @@
|
|
1
|
+
module QueryReviewer
|
2
|
+
class QueryWarning
|
3
|
+
attr_reader :query, :severity, :problem, :desc, :table, :id
|
4
|
+
|
5
|
+
cattr_accessor :next_id
|
6
|
+
self.next_id = 1
|
7
|
+
|
8
|
+
def initialize(options)
|
9
|
+
@query = options[:query]
|
10
|
+
@severity = options[:severity]
|
11
|
+
@problem = options[:problem]
|
12
|
+
@desc = options[:desc]
|
13
|
+
@table = options[:table]
|
14
|
+
@id = (self.class.next_id += 1)
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
@@ -0,0 +1,33 @@
|
|
1
|
+
require 'query_reviewer'
|
2
|
+
|
3
|
+
module QueryReviewer
|
4
|
+
def self.inject_reviewer
|
5
|
+
# Load adapters
|
6
|
+
ActiveRecord::Base
|
7
|
+
adapter_class = ActiveRecord::ConnectionAdapters::MysqlAdapter if defined? ActiveRecord::ConnectionAdapters::MysqlAdapter
|
8
|
+
adapter_class = ActiveRecord::ConnectionAdapters::Mysql2Adapter if defined? ActiveRecord::ConnectionAdapters::Mysql2Adapter
|
9
|
+
adapter_class.send(:include, QueryReviewer::MysqlAdapterExtensions) if adapter_class
|
10
|
+
# Load into controllers
|
11
|
+
ActionController::Base.send(:include, QueryReviewer::ControllerExtensions)
|
12
|
+
Array.send(:include, QueryReviewer::ArrayExtensions)
|
13
|
+
if ActionController::Base.respond_to?(:append_view_path)
|
14
|
+
ActionController::Base.append_view_path(File.dirname(__FILE__) + "/lib/query_reviewer/views")
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
if defined?(Rails::Railtie)
|
20
|
+
module QueryReviewer
|
21
|
+
class Railtie < Rails::Railtie
|
22
|
+
rake_tasks do
|
23
|
+
load File.dirname(__FILE__) + "/tasks.rb"
|
24
|
+
end
|
25
|
+
|
26
|
+
initializer "query_reviewer.initialize" do
|
27
|
+
QueryReviewer.inject_reviewer if QueryReviewer.enabled?
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
else # Rails 2
|
32
|
+
QueryReviewer.inject_reviewer
|
33
|
+
end
|
@@ -0,0 +1,130 @@
|
|
1
|
+
require "ostruct"
|
2
|
+
|
3
|
+
module QueryReviewer
|
4
|
+
# a single SQL SELECT query
|
5
|
+
class SqlQuery
|
6
|
+
attr_reader :sqls, :rows, :subqueries, :trace, :id, :command, :affected_rows, :profiles, :durations, :sanitized_sql
|
7
|
+
|
8
|
+
cattr_accessor :next_id
|
9
|
+
self.next_id = 1
|
10
|
+
|
11
|
+
def initialize(sql, rows, full_trace, duration = 0.0, profile = nil, command = "SELECT", affected_rows = 1, sanitized_sql = nil)
|
12
|
+
@trace = full_trace
|
13
|
+
@rows = rows
|
14
|
+
@sqls = [sql]
|
15
|
+
@sanitized_sql = sanitized_sql
|
16
|
+
@subqueries = rows ? rows.collect{|row| SqlSubQuery.new(self, row)} : []
|
17
|
+
@id = (self.class.next_id += 1)
|
18
|
+
@profiles = profile ? [profile.collect { |p| OpenStruct.new(p) }] : [nil]
|
19
|
+
@durations = [duration.to_f]
|
20
|
+
@warnings = []
|
21
|
+
@command = command
|
22
|
+
@affected_rows = affected_rows
|
23
|
+
end
|
24
|
+
|
25
|
+
def add(sql, duration, profile)
|
26
|
+
sql << sql
|
27
|
+
durations << duration
|
28
|
+
profiles << profile
|
29
|
+
end
|
30
|
+
|
31
|
+
def sql
|
32
|
+
sqls.first
|
33
|
+
end
|
34
|
+
|
35
|
+
def count
|
36
|
+
durations.size
|
37
|
+
end
|
38
|
+
|
39
|
+
def profile
|
40
|
+
profiles.first
|
41
|
+
end
|
42
|
+
|
43
|
+
def duration
|
44
|
+
durations.sum
|
45
|
+
end
|
46
|
+
|
47
|
+
def duration_stats
|
48
|
+
"TOTAL:#{'%.3f' % duration} AVG:#{'%.3f' % (durations.sum / durations.size)} MAX:#{'%.3f' % (durations.max)} MIN:#{'%.3f' % (durations.min)}"
|
49
|
+
end
|
50
|
+
|
51
|
+
def to_table
|
52
|
+
rows.qa_columnized
|
53
|
+
end
|
54
|
+
|
55
|
+
def warnings
|
56
|
+
self.subqueries.collect(&:warnings).flatten + @warnings
|
57
|
+
end
|
58
|
+
|
59
|
+
def has_warnings?
|
60
|
+
!self.warnings.empty?
|
61
|
+
end
|
62
|
+
|
63
|
+
def max_severity
|
64
|
+
self.warnings.empty? ? 0 : self.warnings.collect(&:severity).max
|
65
|
+
end
|
66
|
+
|
67
|
+
def table
|
68
|
+
@subqueries.first.table
|
69
|
+
end
|
70
|
+
|
71
|
+
def analyze!
|
72
|
+
self.subqueries.collect(&:analyze!)
|
73
|
+
if duration
|
74
|
+
if duration >= QueryReviewer::CONFIGURATION["critical_duration_threshold"]
|
75
|
+
warn(:problem => "Query took #{duration} seconds", :severity => 9)
|
76
|
+
elsif duration >= QueryReviewer::CONFIGURATION["warn_duration_threshold"]
|
77
|
+
warn(:problem => "Query took #{duration} seconds", :severity => QueryReviewer::CONFIGURATION["critical_severity"])
|
78
|
+
end
|
79
|
+
end
|
80
|
+
|
81
|
+
if affected_rows >= QueryReviewer::CONFIGURATION["critical_affected_rows"]
|
82
|
+
warn(:problem => "#{affected_rows} rows affected", :severity => 9, :description => "An UPDATE or DELETE query can be slow and lock tables if it affects many rows.")
|
83
|
+
elsif affected_rows >= QueryReviewer::CONFIGURATION["warn_affected_rows"]
|
84
|
+
warn(:problem => "#{affected_rows} rows affected", :severity => QueryReviewer::CONFIGURATION["critical_severity"], :description => "An UPDATE or DELETE query can be slow and lock tables if it affects many rows.")
|
85
|
+
end
|
86
|
+
end
|
87
|
+
|
88
|
+
def to_hash
|
89
|
+
@sql.hash
|
90
|
+
end
|
91
|
+
|
92
|
+
def relevant_trace
|
93
|
+
trace.collect(&:strip).select{|t| t.starts_with?(Rails.root.to_s) &&
|
94
|
+
(!t.starts_with?("#{Rails.root}/vendor") || QueryReviewer::CONFIGURATION["trace_includes_vendor"]) &&
|
95
|
+
(!t.starts_with?("#{Rails.root}/lib") || QueryReviewer::CONFIGURATION["trace_includes_lib"]) &&
|
96
|
+
!t.starts_with?("#{Rails.root}/vendor/plugins/query_reviewer") }
|
97
|
+
end
|
98
|
+
|
99
|
+
def full_trace
|
100
|
+
self.class.generate_full_trace(trace)
|
101
|
+
end
|
102
|
+
|
103
|
+
def warn(options)
|
104
|
+
options[:query] = self
|
105
|
+
options[:table] ||= self.table
|
106
|
+
@warnings << QueryWarning.new(options)
|
107
|
+
end
|
108
|
+
|
109
|
+
def select?
|
110
|
+
self.command == "SELECT"
|
111
|
+
end
|
112
|
+
|
113
|
+
def self.generate_full_trace(trace = Kernel.caller)
|
114
|
+
trace.collect(&:strip).select{|t| !t.starts_with?("#{Rails.root}/vendor/plugins/query_reviewer") }
|
115
|
+
end
|
116
|
+
|
117
|
+
def self.sanitize_strings_and_numbers_from_sql(sql)
|
118
|
+
new_sql = sql.clone
|
119
|
+
new_sql.gsub!(/\b\d+\b/, "N")
|
120
|
+
new_sql.gsub!(/\b0x[0-9A-Fa-f]+\b/, "N")
|
121
|
+
new_sql.gsub!(/''/, "'S'")
|
122
|
+
new_sql.gsub!(/""/, "\"S\"")
|
123
|
+
new_sql.gsub!(/\\'/, "")
|
124
|
+
new_sql.gsub!(/\\"/, "")
|
125
|
+
new_sql.gsub!(/'[^']+'/, "'S'")
|
126
|
+
new_sql.gsub!(/"[^"]+"/, "\"S\"")
|
127
|
+
new_sql
|
128
|
+
end
|
129
|
+
end
|
130
|
+
end
|