shiba 0.5.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4c19ddf8bb56062725650ff02bede1ee4863ec0de0c6ab56ca420496eb35cc5b
-  data.tar.gz: 8ddfbc998e013cae6ecb5fc0b3ad05e7e76c2f086317cca14699099ffbdb7064
+  metadata.gz: f700a4b773d7a1b3c474ac6437c13ea77743dd6a9b95a8bde1b703d9914f6c20
+  data.tar.gz: 208f0d44f7ba356ad15340cb9ef48bb4ccfaba7050151cd12bcca26cd08b6bab
 SHA512:
-  metadata.gz: e73819cb77cdc7efdaf521d76a64a2dc0ad3e27d97260cc45c2e4c59ac021fac6d12ad501fac2c299455963e6cd6c7650366e91ec6f067c873a1ff97e8d397c4
-  data.tar.gz: 302a916584abf9fd32c210498639ab63739d953b569d5b57313bcde0baa563d5fbf66bba1dcff28f16282c3fb8f9e18b56d6187d689f77a378bafc13323a7f49
+  metadata.gz: a6d464627f14abd14753473965b4935582095dd5ba4b9187760710bffbe75a40edb9392899608f1c3d86d7911b2c0b73eafa8e01adafcf6370b044b191ed3e8e
+  data.tar.gz: 2417ce5680cf5468fe72a4750e30c964a184c7f943c9ce1e37efe738bf812197a1a9b07511e1a2541ede2c0834aa6d0896515add9937b132ba8af9e2b49a6188

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    shiba (0.5.0)
+    shiba (0.6.0)
       activesupport
       mysql2
       pg

data/README.md

@@ -2,12 +2,9 @@
 
 # Shiba
 
-Shiba is a tool that helps catch poorly performing queries before they cause problems in production, including:
+Shiba is a tool (currently in alpha) that automatically reviews SQL queries before they cause problems in production. It uses production statistics for realistic query analysis. It catches missing indexes, overly broad indexes, and queries that return too much data.
 
-* Full table scans
-* Poorly performing indexes
-
-By default, it will pretty much only detect queries that miss indexes. As it's fed more information, it warns about advanced problems, such as queries that use indexes but are still very expensive. To help find such queries, Shiba monitors test runs for ActiveRecord queries. A warning and report are then generated
+![screenshot](https://shiba-sql.com/wp-content/uploads/2019/03/shiba-screenshot-1024x581.png)
 
 ## Installation
 
@@ -27,8 +24,7 @@ require 'shiba/setup'
 
 ## Usage
 
-A report will only be generated when problem queries are detected.
-To verify shiba is actually running, you can run your tests with SHIBA_DEBUG=true.
+To get started, try out shiba locally. To verify shiba is actually running, you can run your tests with SHIBA_DEBUG=true.
 
 ```ruby
 # Install
@@ -43,55 +39,15 @@ SHIBA_DEBUG=true ruby test/controllers/users_controller_test.rb
 # Report available at /tmp/shiba-explain.log-1550099512
 ```
 
-### Screenshot
-`open /tmp/shiba-explain.log-1550099512`
-![screenshot](/data/screenshot.png?raw=true)
-
+## Next steps
+* [Integrate with Github pull requests](#automatic-pull-request-reviews)
+* [Add production stats for realistic analysis](#going-beyond-table-scans)
+* [Preview queries from the developer console](#analyze-queries-from-the-developer-console)
+* [Read more about typical query problems](#typical-query-problems)
 
-## Typical query problems
-
-Here are some typical query problems Shiba can detect. We'll assume the following schema:
-
-```ruby
-create_table :users do |t|
-  t.string :name
-  t.string :email
-  # add an organization_id column with an index
-  t.references :organization, index: true
 
-  t.timestamps
-end
-```
 
-#### Full table scans
-
-The most simple case to detect are queries that don't utilize indexes. While it isn't a problem to scan small tables, often tables will grow large enough where this can become a serious issue.
-
-```ruby
-user = User.where(email: 'squirrel@example.com').limit(1)
-```
-
-Without an index, the database will read every row in the table until it finds one with an email address that matches. By adding an index, the database can perform a quick lookup for the record.
-
-#### Non selective indexes
-
-Another common case is queries that use an index, and work fine in the average case, but the distribution is non normal. These issues can be hard to track down and often impact large customers.
-
-```ruby
-users = User.where(organization_id: 1)
-users.size
-# => 75
-
-users = User.where(organization_id: 42)
-users.size
-# => 52,000
-```
-
-Normally a query like this would only become a problem as the app grows in popularity. Fixes include adding `limit` or `find_each`.
-
-With more data, Shiba can help detect this issue when it appears in a pull request.
-
-## Going beyond table scans
+### Going beyond table scans
 
 Without more information, Shiba acts as a simple missed index detector. To catch other problems that can bring down production (or at least cause some performance issues), Shiba requires general statistics about production data, such as the number of rows in a table and how unique columns are.
 
@@ -135,11 +91,18 @@ users:
       unique: false
 ```
 
-## Automatic pull request reviews
+### Automatic pull request reviews
+
+Shiba can automatically comment on Github pull requests when code changes appear to introduce a query issue. To do this, it will need the Github API token of a user that has access to the repo. Shiba's comments will appear to come from that user, so you'll likely want to setup a bot account on Github with repo access for this. The token can be generated on Github at https://github.com/settings/tokens.
+
+Once the token is ready, you can integrate Shiba on your CI server by following these steps:
+* [Travis CI](#travis-integration)
+* [CircleCI](#circleci-integration)
+* [Customized CI](#custom-ci-integration)
 
-Shiba can automatically comment on Github pull requests when code changes appear to introduce a query issue. The comments are similar to those in the query report dashboard. This guide will walk through setup on Travis CI, but other CI services should work in a similar fashion.
+#### Travis Integration
 
-Once Shiba is installed, the `shiba review` command needs to be run after the tests are finished. On Travis, this goes in an after_script setting:
+On Travis, add this to the after_script setting:
 
 ```yml
 # .travis.yml
@@ -147,12 +110,99 @@ after_script:
  - bundle exec shiba review --submit
  ```
  
-The `--submit` option tells Shiba to comment on the relevant PR when an issue is found. To do this, it will need the Github API token of a user that has access to the repo. Shiba's comments will appear to come from that user, so you'll likely want to setup a bot account on Github with repo access for this.
+Add the Github API token you've generated as an environment variable named `GITHUB_TOKEN` at https://travis-ci.com/{organization}/{repo}/settings.
  
-By default, the review script looks for an environment variable named  GITHUB_TOKEN that can be specified at https://travis-ci.com/{organization}/{repo}/settings. The token can be generated on Github at https://github.com/settings/tokens. If you have another environment variable name for your Github token, it can be manually configured using the `--token` flag.
+#### CircleCI Integration
  
-```yml
-# .travis.yml
-after_script:
- - bundle exec shiba review --token $MY_GITHUB_API_TOKEN --submit
- ```
+To integrate with CircleCI, add this after the the test run step in `.circleci/config.yml`.
+ 
+ ```yml
+# .circleci/config.yml
+- run:
+    name: Review SQL queries
+    command: bundle exec shiba review --submit
+```
+
+An environment variable named `GITHUB_TOKEN` will need to be configured on CircleCI under *Project settings > Environment Variables*
+
+#### Custom CI Integration
+
+To run on other servers, two steps are required:
+1. Ensure an environment variable named `CI` is set when the tests and shiba script are run.
+2. Run the `shiba review` command after tests are run, supplying the required arguments to `--submit, --token, --branch, and --pull-request`. For example:
+
+```bash
+CI=true
+export CI
+rake test
+bundle exec shiba review --submit --token $MY_GITHUB_TOKEN --branch $(git rev-parse HEAD) --pull-request $MY_PR_NUMBER
+```
+
+The `--submit` option tells Shiba to comment on the relevant PR when an issue is found. 
+
+
+### Analyze queries from the developer console
+
+For quick analysis, queries can be analyzed from the Rails console.
+```ruby
+# rails console
+[1] pry(main)> require 'shiba/console'
+=> true
+[2] pry(main)> shiba User.where(email: "squirrel@example.com")
+
+Severity: high
+----------------------------
+Fuzzed Data: Table sizes estimated as follows -- 100000: users
+Table Scan: The database reads 100% (100000) of the of the rows in **users**, skipping any indexes.
+Results: The database returns 100000 row(s) to the client.
+Estimated query time: 3.02s
+
+=> #<Shiba::Console::ExplainRecord:0x00007ffc154e6128>: 'SELECT `users`.* FROM `users` WHERE `users`.`email` = 'squirrel@example.com''. Call the 'help' method on this object for more info.
+[3] pry(main)> 
+```
+
+Raw query strings are also supported, e.g. `shiba "select * from users where users.email = 'squirrel@example.com'"`
+
+
+### Typical query problems
+
+Here are some typical query problems Shiba can detect. We'll assume the following schema:
+
+```ruby
+create_table :users do |t|
+  t.string :name
+  t.string :email
+  # add an organization_id column with an index
+  t.references :organization, index: true
+
+  t.timestamps
+end
+```
+
+#### Full table scans
+
+The most simple case to detect are queries that don't utilize indexes. While it isn't a problem to scan small tables, often tables will grow large enough where this can become a serious issue.
+
+```ruby
+user = User.where(email: 'squirrel@example.com').limit(1)
+```
+
+Without an index, the database will read every row in the table until it finds one with an email address that matches. By adding an index, the database can perform a quick lookup for the record.
+
+#### Non selective indexes
+
+Another common case is queries that use an index, and work fine in the average case, but the distribution is non normal. These issues can be hard to track down and often impact large customers.
+
+```ruby
+users = User.where(organization_id: 1)
+users.size
+# => 75
+
+users = User.where(organization_id: 42)
+users.size
+# => 52,000
+```
+
+Normally a query like this would only become a problem as the app grows in popularity. Fixes include adding `limit` or `find_each`.
+
+With more data, Shiba can help detect this issue when it appears in a pull request.

data/bin/review

@@ -2,166 +2,56 @@
 
 $LOAD_PATH << File.expand_path("../lib", File.dirname(__FILE__))
 require 'optionparser'
-require 'shiba/reviewer'
-require 'shiba/checker'
 require 'shiba/configure'
+require 'shiba/reviewer'
+require 'shiba/review/explain_diff'
+require 'shiba/review/cli'
+require 'json'
 
-options = {}
-parser = OptionParser.new do |opts|
-  opts.banner = "Review changes for query problems. Optionally submit the comments to a Github pull request."
-
-  opts.separator "Required:"
-
-  opts.on("-f","--file FILE", "The explain output log to compare with. Automatically configured when $CI environment variable is set") do |f|
-    options["file"] = f
-  end
-
-  opts.separator ""
-  opts.separator "Git diff options:"
-
-  opts.on("-b", "--branch GIT_BRANCH", "Compare to changes between origin/HEAD and BRANCH. Attempts to read from CI environment when not set.") do |b|
-    options["branch"] = b
-  end
-
-  opts.on("--staged", "Only check files that are staged for commit") do
-    options["staged"] = true
-  end
-
-  opts.on("--unstaged", "Only check files that are not staged for commit") do
-    options["unstaged"] = true
-  end
-
-  opts.separator ""
-  opts.separator "Github options:"
-
-  opts.on("--submit", "Submit comments to Github") do
-    options["submit"] = true
-  end
-
-  opts.on("-p", "--pull-request PR_ID", "The ID of the pull request to comment on. Attempts to read from CI environment when not set.") do |p|
-    options["pull_request"] = p
-  end
+cli = Shiba::Review::CLI.new
+cli.report_options("diff", "branch", "pull_request")
 
-  opts.on("-t", "--token TOKEN", "The Github API token to use for commenting. Defaults to $GITHUB_TOKEN.") do |t|
-    options["token"] = t
-  end
+if !cli.valid?
+  $stderr.puts cli.failure
+  exit 1
+end
 
-  opts.separator ""
-  opts.separator "Common options:"
+explain_diff = Shiba::Review::ExplainDiff.new(cli.options["file"], cli.options)
 
-  opts.on("--verbose", "Verbose/debug mode") do
-    options["verbose"] = true
-  end
+problems = if explain_diff.diff_requested_by_user?
+  result = explain_diff.result
 
-  opts.on_tail("-h", "--help", "Show this message") do
-    puts opts
-    exit
+  if result.message
+    $stderr.puts result.message
   end
 
-  opts.on_tail("--version", "Show version") do
-    require 'shiba/version'
-    puts Shiba::VERSION
+  if result.status == :pass
     exit
   end
-end
-parser.parse!
-
-# This is a noop since it's the default behavior. Ignore.
-if options["staged"] && options["unstaged"]
-  options.delete("staged")
-  options.delete("unstaged")
-end
-
-
-log = options["file"]
-
-if log.nil? && Shiba::Configure.ci?
-  log = options["file"] = File.join(Shiba.path, 'ci.json')
-  $stderr.puts "CI detected, setting file to #{log}" if options["verbose"]
-end
-
-if log.nil?
-  $stderr.puts "Provide an explain log, or run 'shiba explain' to generate one."
-  $stderr.puts ""
-  $stderr.puts parser
-  exit 1
-end
-
-if !File.exist?(log)
-  $stderr.puts "File not found: '#{log}'"
-  exit 1
-end
-
-pr_sha = ENV['TRAVIS_PULL_REQUEST_SHA'] || ENV['CIRCLE_BRANCH']
-
-if options["branch"] == nil && pr_sha && !pr_sha.empty?
-  options["branch"] = pr_sha
-end
-
-if options["token"] == nil
-  options["token"] = ENV['GITHUB_TOKEN']
-end
-
-# https://circleci.com/docs/2.0/env-vars/
-# This may be wrong for circle ci
-pr_id = ENV['TRAVIS_PULL_REQUEST'] || ENV['CIRCLE_PR_NUMBER']
-
-if options["pull_request"] == nil && pr_id && !pr_id.empty?
-  options["pull_request"] = pr_id
-end
 
-if options["verbose"]
-  $stderr.puts "DIFF: #{ENV['DIFF']}" if ENV['DIFF']
-  $stderr.puts "branch: #{options["branch"].inspect}" if options["branch"]
-  $stderr.puts "pull_request: #{options["pull_request"]}" if options["pull_request"]
+  explain_diff.problems
+else
+  explains = File.open(cli.options["file"]).each_line.map { |json| JSON.parse(json) }
+  bad = explains.select { |explain| explain["severity"] && explain["severity"] != 'none' }
+  bad.map { |explain| [ "#{explain["sql"]}:-2", explain ] }
 end
 
 repo_cmd = "git config --get remote.origin.url"
 repo_url = `#{repo_cmd}`.chomp
 
-if options["verbose"]
+if cli.options["verbose"]
   $stderr.puts "#{repo_cmd}\t#{repo_url}"
 end
 
-def require_option(parser, name)
-  $stderr.puts "Required: #{name}"
-  $stderr.puts ""
-  $stderr.puts parser
-  exit 1
-end
-
 if repo_url.empty?
   $stderr.puts "'#{Dir.pwd}' does not appear to be a git repo"
   exit 1
 end
 
-if options["submit"]
-  if (options["branch"].nil? || options["branch"].empty?) && (ENV['DIFF'].nil? || ENV['DIFF'].empty?)
-    require_option(parser, "branch")
-  end
-  require_option(parser, "token") if options["token"].nil?
-  require_option(parser, "pull_request") if options["pull_request"].nil?
-end
-
-if ENV['DIFF']
-  options['diff'] = ENV['DIFF']
-end
-
-# Check to see if the log overlaps with the git diff
-result = Shiba::Checker.new(options).run(log)
-
-if result.message
-  $stderr.puts result.message
-end
-
-if result.status == :pass
-  exit
-end
-
 # Generate comments for the problem queries
-reviewer = Shiba::Reviewer.new(repo_url, result.problems, options)
+reviewer = Shiba::Reviewer.new(repo_url, problems, cli.options)
 
-if !options["submit"] || options["verbose"]
+if !cli.options["submit"] || cli.options["verbose"]
   reviewer.comments.each do |c|
     puts "#{c[:path]}:#{c[:line]} (#{c[:position]})"
     puts c[:body]
@@ -169,7 +59,7 @@ if !options["submit"] || options["verbose"]
   end
 end
 
-if options["submit"]
+if cli.options["submit"]
   if reviewer.repo_host.empty? || reviewer.repo_path.empty?
     $stderr.puts "Invalid repo url '#{repo_url}' from git config --get remote.origin.url"
     exit 1

data/lib/shiba.rb

@@ -8,13 +8,17 @@ require "byebug" if ENV['SHIBA_DEBUG']
 module Shiba
   class Error < StandardError; end
   class ConfigError < StandardError; end
+  TEMPLATE_FILE = File.join(File.dirname(__dir__), 'lib/shiba/output/tags.yaml')
 
   def self.configure(options)
+    return false if @connection_hash
+
     configure_mysql_defaults(options)
 
     @connection_hash = options.select { |k, v| [ 'default_file', 'default_group', 'server', 'username', 'database', 'host', 'password', 'port'].include?(k) }
     @main_config = Configure.read_config_file(options['config'], "config/shiba.yml")
     @index_config = Configure.read_config_file(options['index'], "config/shiba_index.yml")
+    true
   end
 
   def self.configure_mysql_defaults(options)