ffast 0.1.5 → 0.2.0

data/docs/shortcuts.md

@@ -0,0 +1,323 @@
+# Shortcuts
+
+Shortcuts are defined on a `Fastfile` inside any ruby project.
+
+!!!info "Use `~/Fastfile`"
+    You can also add one extra in your `$HOME` if you want to have something loaded always.
+
+By default, the command line interface does not load any `Fastfile` if the
+first param is not a shortcut. It should start with `.`.
+
+I'm building several researches and I'll make the examples open here to show
+several interesting cases in action.
+
+## List your fast shortcuts
+
+As the interface is very rudimentar, let's build a shortcut to print what
+shortcuts are available. This is a good one to your `$HOME/Fastfile`:
+
+```ruby
+# List all shortcut with comments
+Fast.shortcut :shortcuts do
+  fast_files.each do |file|
+    lines = File.readlines(file).map{|line|line.chomp.gsub(/\s*#/,'').strip}
+    result = capture_file('(send ... shortcut $(sym _', file)
+    result = [result] unless result.is_a?Array
+    result.each do |capture|
+      target = capture.loc.expression
+      puts "fast .#{target.source[1..-1].ljust(30)} # #{lines[target.line-2]}"
+    end
+  end
+end
+```
+
+And using it on `fast` project that loads both `~/Fastfile` and the Fastfile from the project:
+
+```
+fast .version       # Let's say you'd like to show the version that is over the version file
+fast .parser        # Simple shortcut that I used often to show how the expression parser works
+fast .bump_version  # Use `fast .bump_version` to rewrite the version file
+fast .shortcuts     # List all shortcut with comments
+```
+
+## Search for references
+
+I always miss bringing something simple as `grep keyword` where I can leave a simple string and it can
+search in all types of nodes and report interesting things about it.
+
+Let's consider a very flexible search that can target any code related to some
+keyword. Considering that we're talking about code indentifiers:
+
+
+```ruby
+# Search all references about some keyword or regular expression
+Fast.shortcut(:ref) do
+  require 'fast/cli'
+  Kernel.class_eval do
+    def matches_args? identifier
+      search = ARGV.last
+      regex = Regexp.new(search, Regexp::IGNORECASE)
+      case identifier
+      when Symbol, String
+        regex.match?(identifier) || identifier.to_s.include?(search)
+      when Astrolabe::Node
+        regex.match?(identifier.to_sexp)
+      end
+    end
+  end
+  pattern = <<~FAST
+    {
+      ({class def sym str} #matches_args?)'
+      ({const send} nil #matches_args?)'
+    }
+  FAST
+  Fast::Cli.run!([pattern, '.', '--parallel'])
+end
+```
+
+## Rails: Show validations from models
+
+If the shortcut does not define a block, it works as a holder for arguments from
+the command line.
+
+Let's say you always use `fast "(send nil {validate validates})" app/models` to
+check validations in the models. You can define a shortcut to hold the args and
+avoid retyping long lines:
+
+```ruby
+# Show validations from app/models
+Fast.shortcut(:validations, "(send nil {validate validates})", "app/models")
+```
+And you can reuse the search with the shortcut starting with a `.`:
+
+```
+fast .validations
+```
+And it will also accept params if you want to filter a specific file:
+
+```
+fast .validations app/models/user.rb
+```
+
+!!! info "Note that you can also use flags in the command line shortcuts"
+
+    Let's say you also want to use `fast --headless` you can add it to the params:
+
+    > Fast.shortcut(:validations, "(send nil {validate validates})", "app/models", "--headless")
+
+## Automated Refactor: Bump version
+
+Let's start with a [real usage](https://github.com/jonatas/fast/blob/master/Fastfile#L20-L34)
+to bump a new version of the gem.
+
+```ruby
+Fast.shortcut :bump_version do
+  rewrite_file('(casgn nil VERSION (str _)', 'lib/fast/version.rb') do |node|
+    target = node.children.last.loc.expression
+    pieces = target.source.split('.').map(&:to_i)
+    pieces.reverse.each_with_index do |fragment, i|
+      if fragment < 9
+        pieces[-(i + 1)] = fragment + 1
+        break
+      else
+        pieces[-(i + 1)] = 0
+      end
+    end
+    replace(target, "'#{pieces.join('.')}'")
+  end
+end
+```
+
+And then the change is done in the `lib/fast/version.rb`:
+
+```diff
+module Fast
+-  VERSION = '0.1.6'
++  VERSION = '0.1.7'
+end
+```
+
+## RSpec: Find unused shared contexts
+
+If you build shared contexts often, probably you can forget some left overs.
+
+The objective of the shortcut is find leftovers from shared contexts.
+
+First, the objective is capture all names of the `RSpec.shared_context` or
+ `shared_context` declared in the `spec/support` folder.
+
+```ruby
+Fast.capture_all('(block (send {nil,_} shared_context (str $_)))', Fast.ruby_files_from('spec/support'))
+```
+
+Then, we need to check all the specs and search for `include_context` usages to
+confirm if all defined contexts are being used:
+
+```ruby
+specs = Fast.ruby_files_from('spec').select{|f|f !~ %r{spec/support/}}
+Fast.search_all("(send nil include_context (str #register_usage)", specs)
+```
+
+Note that we created a new reference to `#register_usage` and we need to define the method too:
+
+
+```ruby
+@used = []
+def register_usage context_name
+	@used << context_name
+end
+```
+
+Wrapping up everything in a shortcut:
+
+```ruby
+# Show unused shared contexts
+Fast.shortcut(:unused_shared_contexts) do
+  puts "Checking shared contexts"
+  Kernel.class_eval do
+    @used = []
+    def register_usage context_name
+      @used << context_name
+    end
+    def show_report! defined_contexts
+      unused = defined_contexts.values.flatten - @used
+      if unused.any?
+        puts "Unused shared contexts", unused
+      else
+        puts "Good job! all the #{defined_contexts.size} contexts are used!"
+      end
+    end
+  end
+  specs = ruby_files_from('spec/').select{|f|f !~ %r{spec/support/}}
+  search_all("(send nil include_context (str #register_usage)", specs)
+  defined_contexts = capture_all('(block (send {nil,_} shared_context (str $_)))', ruby_files_from('spec'))
+  Kernel.public_send(:show_report!, defined_contexts)
+end
+```
+
+!!! faq "Why `#register_usage` is defined on the `Kernel`?"
+    Yes! note that the `#register_usage` was forced to be inside `Kernel`
+    because of the `shortcut` block that takes the `Fast` context to be easy
+    to access in the default functions. As I can define multiple shortcuts
+    I don't want to polute my Kernel module with other methods that are not useful.
+
+
+## RSpec: Remove unused let
+
+!!! hint "First shortcut with experiments"
+    If you're not familiar with automated experiments, you can read about it [here](/experiments).
+
+The current scenario is similar in terms of search with the previous one, but more advanced
+because we're going to introduce automated refactoring.
+
+The idea is simple, if it finds a `let` in a RSpec scenario that is not referenced, it tries to experimentally remove the `let` and run the tests:
+
+```ruby
+# Experimental remove `let` that are not referenced in the spec
+Fast.shortcut(:exp_remove_let) do
+  require 'fast/experiment'
+  Kernel.class_eval do
+    file = ARGV.last
+
+    defined_lets = Fast.capture_file('(block (send nil let (sym $_)))', file).uniq
+    @unreferenced= defined_lets.select do |identifier|
+      Fast.search_file("(send nil #{identifier})", file).empty?
+    end
+
+    def unreferenced_let?(identifier)
+      @unreferenced.include? identifier
+    end
+  end
+
+  experiment('RSpec/RemoveUnreferencedLet') do
+    lookup ARGV.last
+    search '(block (send nil let (sym #unreferenced_let?)))'
+    edit { |node| remove(node.loc.expression) }
+    policy { |new_file| system("bundle exec rspec --fail-fast #{new_file}") }
+  end.run
+end
+```
+
+And it will run with a single file from command line:
+
+```
+fast .exp_remove_let spec/my_file_spec.rb
+```
+
+## FactoryBot: Replace `create` with `build_stubbed`
+
+For performance reasons, if we can avoid touching the database the test will
+always be faster.
+
+```ruby
+# Experimental switch from `create` to `build_stubbed`
+Fast.shortcut(:exp_build_stubbed) do
+  require 'fast/experiment'
+  Fast.experiment('FactoryBot/UseBuildStubbed') do
+  lookup ARGV.last
+    search '(send nil create)'
+    edit { |node| replace(node.loc.selector, 'build_stubbed') }
+    policy { |new_file| system("bundle exec rspec --fail-fast #{new_file}") }
+  end.run
+end
+```
+## RSpec: Use `let_it_be` instead of `let`
+
+The `let_it_be` is a simple helper from
+[TestProf](https://test-prof.evilmartians.io/#/let_it_be) gem that can speed up
+the specs by caching some factories using like a `before_all` approach.
+
+This experiment hunts  for `let(...) { create(...) }` and switch the `let` to
+`let_it_be`:
+
+```ruby
+# Experimental replace `let(_)`  with `let_it_be` case it calls `create` inside the block
+Fast.shortcut(:exp_let_it_be) do
+  require 'fast/experiment'
+  Fast.experiment('FactoryBot/LetItBe') do
+    lookup ARGV.last
+    search '(block (send nil let (sym _)) (args) (send nil create))'
+    edit { |node| replace(node.children.first.loc.selector, 'let_it_be') }
+    policy { |new_file| system("bin/spring rspec --fail-fast #{new_file}") }
+  end.run
+end
+```
+
+## RSpec: Remove `before` or `after` blocks
+
+From time to time, we forget some left overs like `before` or `after` blocks
+that even removing from the code, the tests still passes. This experiment
+removes the before/after blocks and check if the test passes.
+
+```ruby
+# Experimental remove `before` or `after` blocks.
+Fast.shortcut(:exp_remove_before_after) do
+  require 'fast/experiment'
+  Fast.experiment('RSpec/RemoveBeforeAfter') do
+    lookup ARGV.last
+    search '(block (send nil {before after}))'
+    edit { |node| remove(node.loc.expression) }
+    policy { |new_file| system("bin/spring rspec --fail-fast #{new_file}") }
+  end.run
+end
+```
+
+## RSpec: Show message chains
+
+I often forget the syntax and need to search for message chains on specs, so I
+created an shortcut for it.
+
+```ruby
+# Show RSpec message chains
+Fast.shortcut(:message_chains, '^^(send nil receive_message_chain)', 'spec')
+```
+
+## RSpec: Show nested assertions
+
+I love to use nested assertions and I often need examples to refer to them:
+
+```ruby
+# Show RSpec nested assertions with .and
+Fast.shortcut(:nested_assertions, '^^(send ... and)', 'spec')
+```
+

data/docs/videos.md

@@ -0,0 +1,16 @@
+# Videos
+
+- [Ruby Kaigi TakeOut 2020: Grepping Ruby code like a boss](https://www.youtube.com/watch?v=YzcYXB4L2so&feature=youtu.be&t=11855)
+
+<iframe width="560" height="315" src="https://www.youtube.com/embed/YzcYXB4L2so?start=11855" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
+
+Also, similar livecoding session at [RubyConf Brazil 2019 (Portuguese)](https://www.eventials.com/locaweb/jonatas-paganini-live-coding-grepping-ruby-code-like-a-boss/#_=_).
+
+- Introduction to [inline code](https://www.youtube.com/watch?v=KQXglNLUv7o).
+<iframe width="560" height="315" src="https://www.youtube.com/embed/KQXglNLUv7o" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
+
+- [Making local variables inline](https://www.youtube.com/watch?v=JD44nhegCRs)
+<iframe width="560" height="315" src="https://www.youtube.com/embed/YN0s9kV1A2A" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
+
+- [Making methods inline](https://www.youtube.com/watch?v=JD44nhegCRs)
+<iframe width="560" height="315" src="https://www.youtube.com/embed/YN0s9kV1A2A" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

data/fast.gemspec

@@ -7,7 +7,7 @@ require 'fast/version'
 Gem::Specification.new do |spec|
   spec.name          = 'ffast'
   spec.version       = Fast::VERSION
-  spec.required_ruby_version = '>= 2.3'
+  spec.required_ruby_version = '>= 2.6'
   spec.authors       = ['Jônatas Davi Paganini']
   spec.email         = ['jonatasdp@gmail.com']
   spec.homepage      = 'https://jonatas.github.io/fast/'
@@ -26,18 +26,20 @@ Gem::Specification.new do |spec|
 
   spec.add_dependency 'astrolabe'
   spec.add_dependency 'coderay'
+  spec.add_dependency 'parallel'
   spec.add_dependency 'parser'
-  spec.add_dependency 'pry'
 
   spec.add_development_dependency 'bundler'
   spec.add_development_dependency 'guard'
   spec.add_development_dependency 'guard-livereload'
   spec.add_development_dependency 'guard-rspec'
-  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'pry'
+  spec.add_development_dependency 'git'
+  spec.add_development_dependency 'rake'
   spec.add_development_dependency 'rspec', '~> 3.0'
   spec.add_development_dependency 'rspec-its', '~> 1.2'
   spec.add_development_dependency 'rubocop'
   spec.add_development_dependency 'rubocop-performance'
   spec.add_development_dependency 'rubocop-rspec'
-  spec.add_development_dependency 'simplecov'
+  spec.add_development_dependency 'simplecov', '~> 0.10', '< 0.18'
 end

data/lib/fast.rb

@@ -50,7 +50,7 @@ module Fast
     |
     \?                    # maybe expression
     |
-    [\d\w_]+[\\!\?]?      # method names or numbers
+    [\d\w_]+[=\\!\?]?     # method names or numbers
     |
     \(|\)                 # parens `(` and `)` for tuples
     |
@@ -67,18 +67,86 @@ module Fast
     %\d                   # bind extra arguments to the expression
   /x.freeze
 
+  # Set some convention methods from file.
+  class Node < Astrolabe::Node
+    # @return [String] with path of the file or simply buffer name.
+    def buffer_name
+      expression.source_buffer.name
+    end
+
+    # @return [Parser::Source::Range] from the expression
+    def expression
+      location.expression
+    end
+
+    # @return [String] with the content of the #expression
+    def source
+      expression.source
+    end
+
+    # @return [Boolean] true if a file exists with the #buffer_name
+    def from_file?
+      File.exist?(buffer_name)
+    end
+
+    # @return [Array<String>] with authors from the current expression range
+    def blame_authors
+      `git blame -L #{expression.first_line},#{expression.last_line} #{buffer_name}`.lines.map do |line|
+        line.split('(')[1].split(/\d+/).first.strip
+      end
+    end
+
+    # @return [String] with the first element from #blame_authors
+    def author
+      blame_authors.first
+    end
+
+    # Search recursively into a node and its children using a pattern.
+    # @param [String] pattern
+    # @param [Array] *args extra arguments to interpolate in the pattern.
+    # @return [Array<Fast::Node>>] with files and results
+    def search(pattern, *args)
+      Fast.search(pattern, self, *args)
+    end
+
+    # Captures elements from search recursively
+    # @param [String] pattern
+    # @param [Array] *args extra arguments to interpolate in the pattern.
+    # @return [Array<Fast::Node>>] with files and results
+    def capture(pattern, *args)
+      Fast.capture(pattern, self, *args)
+    end
+  end
+
+  # Custom builder allow us to set a buffer name for each Node
+  class Builder < Astrolabe::Builder
+    attr_writer :buffer_name
+    # Generates {Node} from the given information.
+    #
+    # @return [Node] the generated node
+    def n(type, children, source_map)
+      Node.new(type, children, location: source_map, buffer_name: @buffer_name)
+    end
+  end
+
   class << self
-    # @return [Astrolabe::Node] from the parsed content
+    # @return [Fast::Node] from the parsed content
     # @example
     #   Fast.ast("1") # => s(:int, 1)
     #   Fast.ast("a.b") # => s(:send, s(:send, nil, :a), :b)
     def ast(content, buffer_name: '(string)')
       buffer = Parser::Source::Buffer.new(buffer_name)
       buffer.source = content
-      Parser::CurrentRuby.new(Astrolabe::Builder.new).parse(buffer)
+      Parser::CurrentRuby.new(builder_for(buffer_name)).parse(buffer)
     end
 
-    # @return [Astrolabe::Node] parsed from file content
+    def builder_for(buffer_name)
+      builder = Builder.new
+      builder.buffer_name = buffer_name
+      builder
+    end
+
+    # @return [Fast::Node] parsed from file content
     # caches the content based on the filename.
     # @example
     #   Fast.ast_from_file("example.rb") # => s(...)
@@ -96,73 +164,99 @@ module Fast
     end
 
     # Search with pattern directly on file
-    # @return [Array<Astrolabe::Node>] that matches the pattern
+    # @return [Array<Fast::Node>] that matches the pattern
     def search_file(pattern, file)
       node = ast_from_file(file)
-      return [] if node.nil?
+      return [] unless node
+
       search pattern, node
     end
 
     # Search with pattern on a directory or multiple files
     # @param [String] pattern
     # @param [Array<String>] *locations where to search. Default is '.'
-    # @return [Hash<String,Array<Astrolabe::Node>>] with files and results
-    def search_all(pattern, locations = ['.'])
-      search_pattern = method(:search_file).curry.call(pattern)
-      group_results(search_pattern, locations)
+    # @return [Hash<String,Array<Fast::Node>>] with files and results
+    def search_all(pattern, locations = ['.'], parallel: true, on_result: nil)
+      group_results(build_grouped_search(:search_file, pattern, on_result),
+                    locations, parallel: parallel)
     end
 
     # Capture with pattern on a directory or multiple files
     # @param [String] pattern
     # @param [Array<String>] locations where to search. Default is '.'
     # @return [Hash<String,Object>] with files and captures
-    def capture_all(pattern, locations = ['.'])
-      capture_pattern = method(:capture_file).curry.call(pattern)
-      group_results(capture_pattern, locations)
+    def capture_all(pattern, locations = ['.'], parallel: true, on_result: nil)
+      group_results(build_grouped_search(:capture_file, pattern, on_result),
+                    locations, parallel: parallel)
+    end
+
+    # @return [Proc] binding `pattern` argument from a given `method_name`.
+    # @param [Symbol] method_name with `:capture_file` or `:search_file`
+    # @param [String] pattern to match in a search to any file
+    # @param [Proc] on_result is a callback that can be notified soon it matches
+    def build_grouped_search(method_name, pattern, on_result)
+      search_pattern = method(method_name).curry.call(pattern)
+      proc do |file|
+        results = search_pattern.call(file)
+        next if results.nil? || results.empty?
+
+        on_result&.(file, results)
+        { file => results }
+      end
     end
 
-    def group_results(search_pattern, locations)
-      ruby_files_from(*locations)
-        .map { |f| { f => search_pattern.call(f) } }
-        .reject { |results| results.values.all?(&:empty?) }
-        .inject(&:merge!)
+    # Compact grouped results by file allowing parallel processing.
+    # @param [Proc] group_files allows to define a search that can be executed
+    # parallel or not.
+    # @param [Proc] on_result allows to define a callback for fast feedback
+    # while it process several locations in parallel.
+    # @param [Boolean] parallel runs the `group_files` in parallel
+    # @return [Hash[String, Array]] with files and results
+    def group_results(group_files, locations, parallel: true)
+      files = ruby_files_from(*locations)
+      if parallel
+        require 'parallel' unless defined?(Parallel)
+        Parallel.map(files, &group_files)
+      else
+        files.map(&group_files)
+      end.compact.inject(&:merge!)
     end
 
     # Capture elements from searches in files. Keep in mind you need to use `$`
     # in the pattern to make it work.
     # @return [Array<Object>] captured from the pattern matched in the file
     def capture_file(pattern, file)
-      capture pattern, ast_from_file(file)
+      node = ast_from_file(file)
+      return [] unless node
+
+      capture pattern, node
     end
 
     # Search recursively into a node and its children.
     # If the node matches with the pattern it returns the node,
     # otherwise it recursively collect possible children nodes
     # @yield node and capture if block given
-    def search(pattern, node)
-      if (match = match?(pattern, node))
+    def search(pattern, node, *args)
+      if (match = match?(pattern, node, *args))
         yield node, match if block_given?
         match != true ? [node, match] : [node]
       else
         node.each_child_node
-          .flat_map { |child| search(pattern, child) }
+          .flat_map { |child| search(pattern, child, *args) }
           .compact.flatten
       end
     end
 
-    # Return only captures from a search
+    # Only captures from a search
     # @return [Array<Object>] with all captured elements.
-    # @return [Object] with single element when single capture.
     def capture(pattern, node)
-      res = if (match = match?(pattern, node))
-              match == true ? node : match
-            else
-              node.each_child_node
-                .flat_map { |child| capture(pattern, child) }
-                .compact.flatten
-            end
-      res = [res] unless res.is_a?(Array)
-      res.one? ? res.first : res
+      if (match = match?(pattern, node))
+        match == true ? node : match
+      else
+        node.each_child_node
+          .flat_map { |child| capture(pattern, child) }
+          .compact.flatten
+      end
     end
 
     def expression(string)
@@ -200,21 +294,22 @@ module Fast
     # @param files can be file paths or directories.
     # When the argument is a folder, it recursively fetches all `.rb` files from it.
     def ruby_files_from(*files)
-      directories = files.select(&File.method(:directory?))
+      dir_filter = File.method(:directory?)
+      directories = files.select(&dir_filter)
 
       if directories.any?
         files -= directories
         files |= directories.flat_map { |dir| Dir["#{dir}/**/*.rb"] }
         files.uniq!
       end
-      files
+      files.reject(&dir_filter)
     end
 
     # Extracts a node pattern expression from a given node supressing identifiers and primitive types.
     # Useful to index abstract patterns or similar code structure.
     # @see https://jonatas.github.io/fast/similarity_tutorial/
     # @return [String] with an pattern to search from it.
-    # @param node [Astrolabe::Node]
+    # @param node [Fast::Node]
     # @example
     #   Fast.expression_from(Fast.ast('1')) # => '(int _)'
     #   Fast.expression_from(Fast.ast('a = 1')) # => '(lvasgn _ (int _))'
@@ -223,7 +318,7 @@ module Fast
       case node
       when Parser::AST::Node
         children_expression = node.children.map(&method(:expression_from)).join(' ')
-        "(#{node.type}#{' ' + children_expression if node.children.any?})"
+        "(#{node.type}#{" #{children_expression}" if node.children.any?})"
       when nil, 'nil'
         'nil'
       when Symbol, String, Numeric
@@ -278,14 +373,14 @@ module Fast
       when '{' then Any.new(parse_until_peek('}'))
       when '[' then All.new(parse_until_peek(']'))
       when /^"/ then FindString.new(token[1..-2])
-      when /^#\w/ then MethodCall.new(token[1..-1])
-      when /^\.\w[\w\d_]+\?/ then InstanceMethodCall.new(token[1..-1])
+      when /^#\w/ then MethodCall.new(token[1..])
+      when /^\.\w[\w\d_]+\?/ then InstanceMethodCall.new(token[1..])
       when '$' then Capture.new(parse)
       when '!' then (@tokens.any? ? Not.new(parse) : Find.new(token))
       when '?' then Maybe.new(parse)
       when '^' then Parent.new(parse)
       when '\\' then FindWithCapture.new(parse)
-      when /^%\d/ then FindFromArgument.new(token[1..-1])
+      when /^%\d/ then FindFromArgument.new(token[1..])
       else Find.new(token)
       end
     end