querly 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: b38a9eb384827611a39b2d1bef7e1b8cb574f2c8
+  data.tar.gz: c02c0c2ebb9844465855e2691e02715b5c73b359
+SHA512:
+  metadata.gz: e9a5550cec1e87aad9dcfd0372af5a65ec82b46f685c8960c89b7a34316e64ffc2172136a4a238dfa3803349796df235d960ca4ca24df503f8895c4776a9631f
+  data.tar.gz: fa20ac97b8485b262e625e20a945719c254a7a13f4006e0e8513156567324e3b932c6d75e623b16062085026eb9ad8e4ad902adb169f786026588ebd64cfc729

data/.gitignore

@@ -0,0 +1,13 @@
+/.bundle/
+/.yardoc
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+/.idea
+/parser.output
+/lib/querly/pattern/parser.rb
+parser.output
+/Gemfile.lock

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 2.3.1
+before_install: gem install bundler -v 1.12.5

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in querly.gemspec
+gemspec

data/README.md

@@ -0,0 +1,143 @@
+# Querly - Query Method Calls from Ruby Programs
+
+[![Build Status](https://travis-ci.org/soutaro/querly.svg?branch=master)](https://travis-ci.org/soutaro/querly)
+
+Querly is a query language and tool to find out method calls from Ruby programs.
+You write simple query for method calls, and Querly finds out *wrong* pieces from your program.
+
+## Overview
+
+Your project should have many local rules:
+
+* Should not use `Customer#update_mail` but 30x faster `Customer.update_all_email` (Slower `#update_mail` is left just for existing code, but new code should not use it)
+* Should not use `root_url` without `locale:` parameter
+* Should not use `Net::HTTP` for Web API calls, but use `HTTPClient`
+
+These local rule violations will be found during code review.
+Reviewers will ask commiter to revise, commiter will fix, fine.
+Really?
+It is boring and time consuming.
+We need some automation!
+
+However, that rules cannot be the standard.
+They make sense only in your project.
+Okay, start writing a plug-in for RuboCop? (or other checking tools)
+
+Instead of writing RuboCop plug-in, just define a Querly rule in a few lines of YAML.
+
+```yml
+rules:
+  - id: my_project.use_faster_email_update
+    pattern: update_mail
+    message: When updating Customer#email, newly written code should use 30x faster Customer.update_all_email
+    justification:
+      - When you are editing old code (it should be refactored...)
+      - You are sure updating only small number of customers, and performance does not matter
+
+  - id: my_project.root_url_without_locale
+    pattern: "root_url(!locale: _)"
+    message: Links to top page should be with locale parameter
+
+  - id: my_project.net_http
+    pattern: Net::HTTP
+    message: Use HTTPClient to make HTTP request
+```
+
+Write down your local rules, and let Querly do boring checks.
+Focus on spec, design, UX, and other important things during code review!
+
+## Installation
+
+Install via RubyGems.
+
+    $ gem install querly
+
+Or you can put it in your Gemfile.
+
+```rb
+gem 'querly'
+```
+
+## Quick Start
+
+Copy the following YAML and paste as `querly.yml` in your project's repo.
+
+```yaml
+rules:
+  - id: sample.debug_print
+    pattern:
+      - p
+      - pp
+    message: Delete debug print
+```
+
+Run `querly` in the repo.
+
+```
+$ querly check .
+```
+
+If your code contains `p` or `pp` calls, querly will print warning messages.
+
+```
+./app/models/account.rb:44:10                  p(account.id)      Delete debug print
+./app/controllers/accounts_controller.rb:17:2  pp params: params  Delete debug print
+```
+
+## Configuration
+
+Visit our wiki pages for configuration and query language reference.
+
+* [Configuration](https://github.com/soutaro/querly/wiki/Configuration)
+* [Patterns](https://github.com/soutaro/querly/wiki/Patterns)
+
+Use `querly console` command to test patterns interactively.
+
+## Notes
+
+### Querly's analysis is syntactic
+
+The analysis is currently purely syntactic:
+
+```rb
+record.save(validate: false)
+```
+
+and
+
+```rb
+x = false
+record.save(validate: x)
+```
+
+will yield different results.
+This can be improved by doing very primitive data flow analysis, and I'm planning to do that.
+
+### Too many false positives!
+
+The analysis itself does not have very good precision.
+There will be many false positives, and *querly warning free code* does not make much sense.
+
+* TODO: support to ignore warnings through magic comments in code
+
+This is not to ensure *there is nothing wrong in the code*, but just tells you *code fragments you should review with special care*.
+I believe it still improves your software development productivity.
+
+### Incoming updates?
+
+The following is the list of updates which would make sense.
+
+* Support for importing rule sets, and provide some good default rules
+* Support for ignoring warnings
+* Improve analysis precision by intra procedural data flow analysis
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/soutaro/querly.
+

data/Rakefile

@@ -0,0 +1,18 @@
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList['test/**/*_test.rb']
+end
+
+task :default => :test
+task :build => :racc
+task :test => :racc
+
+rule /\.rb/ => ".y" do |t|
+  sh "racc", "-v", "-o", "#{t.name}", "#{t.source}"
+end
+
+task :racc => "lib/querly/pattern/parser.rb"

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "querly"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/exe/querly

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+
+$LOAD_PATH << File.join(__dir__, "../lib")
+
+require "querly"
+require "querly/cli"
+
+Querly::CLI.start(ARGV)

data/lib/querly.rb

@@ -0,0 +1,26 @@
+require 'pathname'
+require "yaml"
+require "rainbow"
+require "parser/current"
+require "set"
+require "open3"
+
+Parser::Builders::Default.emit_lambda = true
+
+require "querly/version"
+require 'querly/analyzer'
+require 'querly/rule'
+require 'querly/pattern/expr'
+require 'querly/pattern/argument'
+require 'querly/script'
+require 'querly/script_enumerator'
+require 'querly/node_pair'
+require "querly/pattern/parser"
+require 'querly/pattern/kind'
+require "querly/config"
+require "querly/tagging"
+require "querly/preprocessor"
+
+module Querly
+  # Your code goes here...
+end

data/lib/querly/analyzer.rb

@@ -0,0 +1,64 @@
+module Querly
+  class Analyzer
+    attr_reader :rules
+    attr_reader :scripts
+    attr_reader :taggings
+
+    def initialize(taggings:)
+      @rules = []
+      @scripts = []
+      @taggings = taggings
+    end
+
+    #
+    # yields(script, rule, node_pair)
+    #
+    def run
+      scripts.each do |script|
+        tagging = taggings.find {|tagging| tagging.applicable?(script) }
+
+        each_subnode script.root_pair do |node_pair|
+          rules.each do |rule|
+            if applicable_rule?(tagging, rule)
+              if rule.patterns.any? {|pattern| test_pair(node_pair, pattern) }
+                yield script, rule, node_pair
+              end
+            end
+          end
+        end
+      end
+    end
+
+    def find(pattern)
+      scripts.each do |script|
+        each_subnode script.root_pair do |node_pair|
+          if test_pair(node_pair, pattern)
+            yield script, node_pair
+          end
+        end
+      end
+    end
+
+    def test_pair(node_pair, pattern)
+      pattern.expr =~ node_pair && pattern.test_kind(node_pair)
+    end
+
+    def applicable_rule?(tagging, rule)
+      if tagging && !rule.tags.empty?
+        tagging.tags_set.any? {|set| set.subset?(rule.tags) }
+      else
+        true
+      end
+    end
+
+    def each_subnode(node_pair, &block)
+      return unless node_pair.node
+
+      yield node_pair
+
+      node_pair.children.each do |child|
+        each_subnode child, &block
+      end
+    end
+  end
+end

data/lib/querly/cli.rb

@@ -0,0 +1,81 @@
+require "thor"
+require "json"
+
+module Querly
+  class CLI < Thor
+    desc "check [paths]", "Check paths based on configuration"
+    option :config, default: "querly.yml"
+    option :format, default: "text", type: :string, enum: %w(text json)
+    def check(*paths)
+      require 'querly/cli/formatter'
+
+      formatter = case options[:format]
+                  when "text"
+                    Formatter::Text.new
+                  when "json"
+                    Formatter::JSON.new
+                  end
+      formatter.start
+
+      begin
+        unless config_path.file?
+          STDERR.puts <<-Message
+Configuration file #{config_path} does not look a file.
+Specify configuration file by --config option.
+          Message
+          exit 1
+        end
+
+        config = Config.new
+        begin
+          config.add_file config_path
+        rescue => exn
+          formatter.config_error config_path, exn
+          exit
+        end
+
+        analyzer = Analyzer.new(taggings: config.taggings)
+        analyzer.rules.concat config.rules
+
+        ScriptEnumerator.new(paths: paths.map {|path| Pathname(path) }, preprocessors: config.preprocessors).each do |path, script|
+          case script
+          when Script
+            analyzer.scripts << script
+            formatter.script_load script
+          when StandardError, LoadError
+            formatter.script_error path, script
+          end
+        end
+
+        analyzer.run do |script, rule, pair|
+          formatter.issue_found script, rule, pair
+        end
+      rescue => exn
+        formatter.fatal_error exn
+        exit 1
+      ensure
+        formatter.finish
+      end
+    end
+
+    desc "console [paths]", "Start console for given paths"
+    def console(*paths)
+      require 'querly/cli/console'
+      Console.new(paths: paths.map {|path| Pathname(path) }).start
+    end
+
+    desc "test", "Check configuration"
+    option :config, default: "querly.yml"
+    def test()
+      require "querly/cli/test"
+      Test.new(config_path: config_path).run
+    end
+
+    private
+
+    def config_path
+      [Pathname(options[:config]),
+       Pathname("querly.yaml")].compact.find(&:file?) || Pathname(options[:config])
+    end
+  end
+end

data/lib/querly/cli/console.rb

@@ -0,0 +1,110 @@
+require 'readline'
+
+module Querly
+  class CLI
+    class Console
+      attr_reader :paths
+
+      def initialize(paths:)
+        @paths = paths
+      end
+
+      def start
+        puts <<-Message
+Querly #{VERSION}, interactive console
+
+        Message
+
+        puts_commands
+
+        STDOUT.print "Loading..."
+        STDOUT.flush
+        reload!
+        STDOUT.puts " ready!"
+
+        loop
+      end
+
+      def reload!
+        @analyzer = nil
+        analyzer
+      end
+
+      def analyzer
+        return @analyzer if @analyzer
+
+        @analyzer = Analyzer.new(taggings: [])
+
+        ScriptEnumerator.new(paths: paths).each do |path, script|
+          case script
+          when Script
+            @analyzer.scripts << script
+          when StandardError
+            p script
+          end
+        end
+
+        @analyzer
+      end
+
+      def loop
+        while line = Readline.readline("> ", true)
+          case line
+          when "quit"
+            exit
+          when "reload!"
+            STDOUT.print "reloading..."
+            STDOUT.flush
+            reload!
+            STDOUT.puts " done"
+          when /^find (.+)/
+            begin
+              pattern = Pattern::Parser.parse($1)
+
+              count = 0
+
+              analyzer.find(pattern) do |script, pair|
+                path = script.path.to_s
+                line = pair.node.loc.first_line
+
+                while true
+                  parent = pair.parent
+
+                  if parent && parent.node.loc.first_line == line
+                    pair = pair.parent
+                  else
+                    break
+                  end
+                end
+
+                src = Rainbow(pair.node.loc.expression.source.split(/\n/).first).blue
+                col = pair.node.loc.column
+
+                puts "  #{path}:#{line}:#{col}\t#{src}"
+
+                count += 1
+              end
+
+              puts "#{count} results"
+            rescue => exn
+              STDOUT.puts exn.inspect
+              STDOUT.puts exn.backtrace
+            end
+          else
+            puts_commands
+          end
+        end
+      end
+
+      def puts_commands
+        puts <<-Message
+Commands:
+  - find PATTERN   Find PATTERN from given paths
+  - reload!        Reload program from paths
+  - quit
+
+        Message
+      end
+    end
+  end
+end