ffast 0.2.0 → 0.2.2

data/docs/git.md

@@ -0,0 +1,115 @@
+
+You can overload the AST node with extra methods to get information from Git.
+
+Let's start with some basic setup to reuse in the next examples:
+
+## Git require
+
+By default, this extension is not loaded in the fast environment, so you should
+require it.
+
+```ruby
+require 'fast/git'
+```
+
+
+Then it will work with any AST node.
+
+```ruby
+ast = Fast.ast_from_file('lib/fast.rb')
+```
+
+## Log
+
+First commit from git:
+
+```ruby
+ast.git_log.first.author.name # => "Jonatas Davi Paganini"
+```
+
+It uses [ruby-git](https://github.com/ruby-git/ruby-git#examples) gem, so all
+methods are available:
+
+```ruby
+ast.git_log.since(Time.mktime(2019)).entries.map(&:message)
+```
+
+Counting commits per year:
+
+```ruby
+ast.git_log.entries.group_by{|t|t.date.year}.transform_values(&:size)
+# => {2020=>4, 2019=>22, 2018=>4}
+```
+
+Counting commits per contributor:
+
+```ruby
+ast.git_log.entries.group_by{|t|t.author.name}.transform_values(&:size)
+# => {"Jônatas Davi Paganini"=>29, ...}
+```
+
+Selecting last commit message:
+
+```ruby
+ast.last_commit.message # => "Add node extensions for extracting info from git (#21)"
+```
+
+Remote git URL:
+
+```ruby
+ast.remote_url  # => "git@github.com:jonatas/fast.git"
+ast.project_url # => "https://github.com/jonatas/fast"
+```
+
+The `sha` from last commit:
+
+```ruby
+ast.sha # => "cd1c036b55ec1d41e5769ad73b282dd6429a90a6"
+```
+
+Pick a link from the files to master version:
+
+```ruby
+ast.link # => "https://github.com/jonatas/fast/blob/master/lib/fast.rb#L3-L776"
+```
+
+Getting permalink from current commit:
+
+```ruby
+ast.permalink # => "https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L3-L776"
+```
+
+## Markdown link
+
+Let's say you'd like to capture a list of class names that inherits the `Find`
+class:
+
+```ruby
+puts ast.capture("(class $(const nil _) (const nil Find)").map(&:md_link).join("\n* ")
+```
+
+It will output the following links:
+
+* [FindString](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L485)
+* [MethodCall](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L496)
+* [InstanceMethodCall](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L507)
+* [FindWithCapture](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L524)
+* [FindFromArgument](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L551)
+* [Capture](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L598)
+* [Parent](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L622)
+* [Any](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L636)
+* [All](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L647)
+* [Not](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L659)
+* [Maybe](https://github.com/jonatas/fast/blob/master/lib/fast.rb#L667)
+
+## Permalink
+
+If you need to get a permanent link to the code, use the `permalink` method:
+
+```ruby
+ast.search("(class (const nil _) (const nil Find)").map(&:permalink)
+# => ["https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L524-L541",
+#     "https://github.com/jonatas/fast/blob/cd1c036b55ec1d41e5769ad73b282dd6429a90a6/lib/fast.rb#L551-L571", ...]
+```
+
+

data/docs/ideas.md

@@ -21,16 +21,6 @@ run the code you have and not the code that is overloading the project.
 Easy pipe fast results to Neo4J. It would facilitate to explore more complex
 scenarios and combine data from other sources.
 
-## Git adapter
-
-Add extra tags to nodes with information from Git.
-
-* Revision
-* Author
-* Date
-
-Tag every node with the proper author.
-
 ## Ast Diff
 
 Allow to compare and return a summary of differences between two trees.

data/docs/index.md

@@ -1,5 +1,7 @@
 # Fast
 
+<center>![](assets/logo-small.png)</center>
+
 Fast is a "Find AST" tool to help you search in the code abstract syntax tree.
 
 Ruby allow us to do the same thing in a few ways then it's hard to check

data/docs/sql-support.md

@@ -0,0 +1,253 @@
+# SQL Support
+
+Fast is partially supporting SQL syntax. Behind the scenes it parses SQL using
+[pg_query](https://github.com/pganalyze/pg_query) and simplifies it to AST Nodes
+using the same interface. It's using Postgresql parser behind the scenes,
+but probably could be useful for other SQL similar diallects.
+
+
+!!! info "fast auto detects SQL files in the command line"
+
+    By default, this module is not included into the main library.
+    Fast can auto-detect file extensions and choose the sql path in case the
+    file relates to sql.
+
+    Use `fast --sql` in case you want to force the usage of the SQL parser.
+
+    ```
+
+    ```
+
+
+# Parsing a sql content
+
+```ruby
+require 'fast/sql'
+ast = Fast.parse_sql('select 1')
+# => s(:select_stmt,
+#     s(:target_list,
+#       s(:res_target,
+#         s(:val,
+#           s(:a_const,
+#             s(:val,
+#               s(:integer,
+#                 s(:ival, 1))))))))
+```
+
+## Why it's interesting to use AST for SQL?
+
+Both SQL are available and do the same thing:
+```sql
+select * from customers
+```
+or
+```sql
+table customers
+```
+
+they have exactly the same objective but written down in very different syntax.
+
+Give a try:
+
+```ruby
+Fast.parse_sql("select * from customers") == Fast.parse_sql("table customers") # => true
+```
+
+## Match
+
+Use `match?` with your node pattern to traverse the abstract syntax tree.
+
+```ruby
+ Fast.match?("(select_stmt ...)", ast) # => true
+```
+
+Use `$` to capture elements from the AST:
+
+```ruby
+ Fast.match?("(select_stmt $...)", ast)
+=> [s(:target_list,
+  s(:res_target,
+    s(:val,
+      s(:a_const,
+        s(:val,
+          s(:integer,
+            s(:ival, 1)))))))]
+
+```
+
+You can dig deeper into the AST specifying nodes:
+
+```ruby
+Fast.match?("(select_stmt (target_list (res_target (val ($...)))))", ast)
+# => [s(:a_const,
+#     s(:val,
+#       s(:integer,
+#         s(:ival, 1))))]
+```
+
+And ignoring node types or values using `_`. Check all [syntax](/syntax) options.
+
+```ruby
+Fast.match?("(select_stmt (_ (_ (val ($...)))))", ast)
+# => [s(:a_const,
+#     s(:val,
+#       s(:integer,
+#         s(:ival, 1))))]
+```
+
+## Search directly from the AST
+
+You can also search directly from nodes and keep digging:
+
+```ruby
+ast = Fast.parse_sql('select 1');
+ast.search('ival') # => [s(:ival, s(:ival, 1))]
+```
+
+Use first to return the node directly:
+
+```ruby
+ast.first('(ival (ival _))')  #=> s(:ival, s(:ival, 1))
+```
+
+Combine the `capture` method with `$`:
+
+```ruby
+ast.capture('(ival (ival $_))') # => [1]
+```
+
+
+# Examples
+
+Let's dive into a more complex example capturing fields and from clause of a
+condition. Let's start parsing the sql:
+
+## Capturing fields and where clause
+
+
+```ruby
+ast = Fast.parse_sql('select name from customer')
+#   => s(:select_stmt,
+#     s(:target_list,
+#       s(:res_target,
+#         s(:val,
+#           s(:column_ref,
+#             s(:fields,
+#               s(:string,
+#                 s(:str, "name"))))))),
+#     s(:from_clause,
+#       s(:range_var,
+#         s(:relname, "customer"),
+#         s(:inh, true),
+#         s(:relpersistence, "p"))))
+```
+
+Now, let's build the expression to get the fields and from_clause.
+
+```ruby
+ cols_and_from = "
+   (select_stmt
+     (target_list (res_target (val (column_ref (fields $...)))))
+     (from_clause (range_var $(relname _))))
+"
+```
+
+Now, we can use `Fast.capture` or `Fast.match?` to extract the values from the
+AST.
+
+```ruby
+Fast.capture(cols_and_from, ast)
+# => [s(:string,
+#     s(:str, "name")), s(:relname, "customer")]
+```
+
+## Search inside
+
+```ruby
+relname = Fast.parse_sql('select name from customer').search('relname').first
+# => s(:relname, "customer")
+```
+
+Find the location of a node.
+
+```ruby
+relname.location # => #<Parser::Source::Map:0x00007fd3bcb0b7f0
+#  @expression=#<Parser::Source::Range (sql) 17...25>,
+#  @node=s(:relname, "customer")>
+```
+
+The location can be useful to allow you to do refactorings and find specific
+delimitations of objects in the string.
+
+The attribute `expression` gives access to the source range.
+
+```ruby
+relname.location.expression
+# => #<Parser::Source::Range (sql) 17...25>
+```
+
+The `source_buffer` is shared and can be accessed through the expression.
+
+```ruby
+relname.location.expression.source_buffer
+# => #<Fast::SQL::SourceBuffer:0x00007fd3bc2a6420
+#    @name="(sql)",
+#    @source="select name from customer",
+#    @tokens=
+#     [<PgQuery::ScanToken: start: 0, end: 6, token: :SELECT, keyword_kind: :RESERVED_KEYWORD>,
+#      <PgQuery::ScanToken: start: 7, end: 11, token: :NAME_P, keyword_kind: :UNRESERVED_KEYWORD>,
+#      <PgQuery::ScanToken: start: 12, end: 16, token: :FROM, keyword_kind: :RESERVED_KEYWORD>,
+#      <PgQuery::ScanToken: start: 17, end: 25, token: :IDENT, keyword_kind: :NO_KEYWORD>]>
+```
+
+The tokens are useful to find the proper node location during the build but
+they're not available for all the nodes, so, it can be very handy as an extra
+reference.
+
+## Replace
+
+Replace fragments of your SQL based on AST can also be done with all the work
+inherited from Parser::TreeRewriter components.
+
+```ruby
+Fast.parse_sql('select 1').replace('ival', '2') # => "select 2"
+```
+
+The previous example is a syntax sugar for the following code:
+
+```ruby
+Fast.replace_sql('ival',
+  Fast.parse_sql('select 1'),
+  &->(node){ replace(node.location.expression, '2') }
+) # => "select 2"
+```
+
+The last argument is a proc that runs on the [parser tree rewriter](https://www.rubydoc.info/gems/parser/Parser/TreeRewriter
+) scope.
+
+Let's break down the previous code:
+
+```ruby
+ast = Fast.parse_sql("select 1")
+#  => s(:select_stmt,
+#    s(:target_list,
+#      s(:res_target,
+#        s(:val,
+#          s(:a_const,
+#            s(:ival,
+#              s(:ival, 1)))))))
+```
+
+The pattern is simply matching node type that is `ival` but it could be a complex expression
+like `(val (a_const (val (ival (ival _)))))`.
+
+Completing the example:
+
+```ruby
+ Fast.replace_sql("ival", ast, &-> (n) { replace(n.loc.expression, "3") })
+ # => "select 3"
+```
+
+`loc` is a shortcut for `location` attribute.
+
+

data/docs/videos.md

@@ -1,8 +1,8 @@
 # Videos
 
-- [Ruby Kaigi TakeOut 2020: Grepping Ruby code like a boss](https://www.youtube.com/watch?v=YzcYXB4L2so&feature=youtu.be&t=11855)
+- [Ruby Kaigi TakeOut 2020: Grepping Ruby code like a boss](https://www.youtube.com/watch?v=YczrZQC9aP8&feature=youtu.be&amp)
 
-<iframe width="560" height="315" src="https://www.youtube.com/embed/YzcYXB4L2so?start=11855" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
+<iframe width="560" height="315" src="https://www.youtube.com/embed/YczrZQC9aP8?" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
 
 Also, similar livecoding session at [RubyConf Brazil 2019 (Portuguese)](https://www.eventials.com/locaweb/jonatas-paganini-live-coding-grepping-ruby-code-like-a-boss/#_=_).
 

data/docs/walkthrough.md

@@ -0,0 +1,135 @@
+# Fast walkthrough
+
+!!! note "This is the main interactive tutorial we have on `fast`. If you're reading it on the web, please consider also try it in the command line: `fast .intro` in the terminal to get a rapid pace on reading and testing on your own computer."
+
+The objective here is give you some insights about how to use `ffast` gem in the
+command line.
+
+Let's start finding the main `fast.rb` file for the fast library:
+
+```
+$ gem which fast
+```
+
+And now, let's combine the previous expression that returns the path to the file
+and take a quick look into the methods `match?` in the file using a regular grep:
+
+```
+$ grep "def match\?" $(gem which fast)
+```
+
+Boring results, no? The code here is not easy to digest because we just see a
+fragment of the code block that we want.
+Let's make it a bit more advanced with `grep -rn` to file name and line number:
+
+```
+$ grep -rn "def match\?" $(gem which fast)
+```
+
+Still hard to understand the scope of the search.
+
+That's why fast exists! Now, let's take a look on how a method like this looks
+like from the AST perspective. Let's use `ruby-parse` for it:
+
+```
+$ ruby-parse -e "def match?(node); end"
+```
+
+Now, let's make the same search with `fast` node pattern:
+
+```
+fast "(def match?)" $(gem which fast)
+```
+
+Wow! in this case you got all the `match?` methods, but you'd like to go one level upper
+and understand the classes that implements the method with a single node as
+argument. Let's first use `^` to jump into the parent:
+
+```
+fast "^(def match?)" $(gem which fast)
+```
+
+As you can see it still prints some `match?` methods that are not the ones that
+we want, so, let's add a filter by the argument node `(args (arg node))`:
+
+```
+fast "(def match? (args (arg node)))" $(gem which fast)
+```
+
+Now, it looks closer to have some understanding of the scope, filtering only
+methods that have the name `match?` and receive `node` as a parameter.
+
+Now, let's do something different and find all methods that receives a `node` as
+an argument:
+
+```
+fast "(def _ (args (arg node)))" $(gem which fast)
+```
+
+Looks like almost all of them are the `match?` and we can also skip the `match?`
+methods negating the expression prefixing with `!`:
+
+```
+fast "(def !match? (args (arg node)))" $(gem which fast)
+```
+
+Let's move on and learn more about node pattern with the RuboCop project:
+
+```
+$ VISUAL=echo gem open rubocop
+```
+
+RuboCop contains `def_node_matcher` and `def_node_search`. Let's make a search
+for both method names wrapping the query with `{}` selector:
+
+```
+fast "(send nil {def_node_matcher def_node_search})" $(VISUAL=echo gem open rubocop)
+```
+
+As you can see, node pattern is widely adopted in the cops to target code.
+Rubocop contains a few projects with dedicated cops that can help you learn
+more.
+
+## How to automate refactor using AST
+
+Moving towards to the code automation, the next step after finding some target code
+is refactor and change the code behavior.
+
+Let's imagine that we already found some code that we want to edit or remove. If
+we get the AST we can also cherry-pick any fragment of the expression to be
+replaced. As you can imagine, RuboCop also benefits from automatic refactoring
+offering the `--autocorrect` option.
+
+All the hardcore algorithms are in the [parser](https://github.com/whitequark/parser)
+rewriter, but we can find a lot of great examples on RuboCop project searching
+for the `autocorrect` method.
+
+```
+fast "(def autocorrect)" $(VISUAL=echo gem open rubocop)
+```
+
+Look that most part of the methods are just returning a lambda with a
+corrector. Now, let's use the `--ast` to get familiar with the tree details for the
+implementation:
+
+```
+fast --ast "(def autocorrect)" $(VISUAL=echo gem open rubocop)/lib/rubocop/cop/style
+```
+
+As we can see, we have a `(send (lvar corrector))` that is the interface that we
+can get the most interesting calls to overwrite files:
+
+```
+fast "(send (lvar corrector)" $(VISUAL=echo gem open rubocop)
+```
+
+
+## That is all for now!
+
+I hope you enjoyed to learn by example searching with fast. If you like it,
+please [star the project](https://github.com/jonatas/fast/)!
+
+You can also build your own tutorials simply using markdown files like I did
+here, you can find this tutorial [here](https://github.com/jonatas/fast/tree/master/docs/walkthrough.md).
+
+

data/fast.gemspec

@@ -17,9 +17,27 @@ Gem::Specification.new do |spec|
   spec.license       = 'MIT'
 
   spec.files         = `git ls-files -z`.split("\x0").reject do |f|
-    f.match(%r{^(test|spec|features)/})
+    f.match(%r{^(test|spec|features|docs\/(assets|stylesheets))/})
   end
 
+  spec.post_install_message = <<~THANKS
+
+    ==========================================================
+    Yay! Thanks for installing
+
+       ___       __  ___
+      |__   /\  /__`  |
+      |    /~~\ .__/  |
+
+    To interactive learn about the gem in the terminal use:
+
+    fast .intro
+
+    More docs at: https://jonatas.github.io/fast/
+    ==========================================================
+
+  THANKS
+
   spec.bindir        = 'bin'
   spec.executables   = %w[fast fast-experiment]
   spec.require_paths = %w[lib experiments]
@@ -28,18 +46,19 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'coderay'
   spec.add_dependency 'parallel'
   spec.add_dependency 'parser'
+  spec.add_dependency 'pg_query'
 
   spec.add_development_dependency 'bundler'
+  spec.add_development_dependency 'git'
   spec.add_development_dependency 'guard'
   spec.add_development_dependency 'guard-livereload'
   spec.add_development_dependency 'guard-rspec'
   spec.add_development_dependency 'pry'
-  spec.add_development_dependency 'git'
   spec.add_development_dependency 'rake'
-  spec.add_development_dependency 'rspec', '~> 3.0'
-  spec.add_development_dependency 'rspec-its', '~> 1.2'
+  spec.add_development_dependency 'rspec'
+  spec.add_development_dependency 'rspec-its'
   spec.add_development_dependency 'rubocop'
   spec.add_development_dependency 'rubocop-performance'
   spec.add_development_dependency 'rubocop-rspec'
-  spec.add_development_dependency 'simplecov', '~> 0.10', '< 0.18'
+  spec.add_development_dependency 'simplecov'
 end