syntax_search 0.1.0 → 0.1.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7bc1e7685829016c797af5b036c68d0ba625a3a195c134486ca2852fda4aa339
-  data.tar.gz: be5bb2c996fe5a4f9762257be2ff93d028ac3c2faa8db1e27c239c56ad65a552
+  metadata.gz: 77f054eda8b4e25dcaf0306818ae17bb1ff3da9c52f2d606141ff44cd343c51d
+  data.tar.gz: 44f4347d5078d0250286e78a08ff730b02094aeb05897c9088963381a95e9071
 SHA512:
-  metadata.gz: ab335dbb7813fca47d939f9e88af8f669eddec5a7e773a846f09fe04082982bc59d79b1f8d7a056c3441e79226536161544aada44fd3eabea72d5363303cc616
-  data.tar.gz: 47c0ccd38ec581c3266001db6028d71d3fdec089e51edc603ca4d3518eae042753b1fb885e9c194dc697b7522ec5aca8ac23447ff58f79755d7f1291fdef26d6
+  metadata.gz: 6ccc7e2ed620689614bdfae8943b8c15b81b4567d3b42e2c734ac7974699ed231272ce52929eff9d49eb25edd843ae3f7fa2398d0552b77665be779a4412461f
+  data.tar.gz: 1e866b4aaafb534f946da5e00efb50536a4d20bf0e1527f8641fb298cca5cc6b7d0e1f744f63da443b9920500d7d4ddf0113ffa8bde2b4689cd6793538a12b6e

data/.github/workflows/check_changelog.yml

@@ -0,0 +1,13 @@
+name: Check Changelog
+
+on:
+ pull_request:
+  types: [opened, reopened, edited, synchronize]
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v1
+    - name: Check that CHANGELOG is touched
+      run: |
+        cat $GITHUB_EVENT_PATH | jq .pull_request.title |  grep -i '\[\(\(changelog skip\)\|\(ci skip\)\)\]' ||  git diff remotes/origin/${{ github.base_ref }} --name-only | grep CHANGELOG.md

data/CHANGELOG.md

@@ -0,0 +1,26 @@
+## HEAD (unreleased)
+
+## 0.1.5
+
+- Strip out heredocs in documents first (https://github.com/zombocom/syntax_search/pull/19)
+
+## 0.1.4
+
+- Parser gem replaced with Ripper (https://github.com/zombocom/syntax_search/pull/17)
+
+## 0.1.3
+
+- Internal refactor (https://github.com/zombocom/syntax_search/pull/13)
+
+## 0.1.2
+
+- Codeblocks in output are now indented with 4 spaces and "code fences" are removed (https://github.com/zombocom/syntax_search/pull/11)
+- "Unmatched end" and "missing end" not generate different error text instructions (https://github.com/zombocom/syntax_search/pull/10)
+
+## 0.1.1
+
+- Fire search on both unexpected end-of-input and unexected end (https://github.com/zombocom/syntax_search/pull/8)
+
+## 0.1.0
+
+- Initial release

data/Gemfile

@@ -7,3 +7,4 @@ gemspec
 
 gem "rake", "~> 12.0"
 gem "rspec", "~> 3.0"
+gem "stackprof"

data/Gemfile.lock

@@ -1,16 +1,12 @@
 PATH
   remote: .
   specs:
-    syntax_search (0.1.0)
-      parser
+    syntax_search (0.1.5)
 
 GEM
   remote: https://rubygems.org/
   specs:
-    ast (2.4.1)
     diff-lcs (1.4.4)
-    parser (2.7.2.0)
-      ast (~> 2.4.1)
     rake (12.3.3)
     rspec (3.10.0)
       rspec-core (~> 3.10.0)
@@ -25,6 +21,7 @@ GEM
       diff-lcs (>= 1.2.0, < 2.0)
       rspec-support (~> 3.10.0)
     rspec-support (3.10.0)
+    stackprof (0.2.16)
 
 PLATFORMS
   ruby
@@ -32,6 +29,7 @@ PLATFORMS
 DEPENDENCIES
   rake (~> 12.0)
   rspec (~> 3.0)
+  stackprof
   syntax_search!
 
 BUNDLED WITH

data/README.md

@@ -10,34 +10,31 @@ What happened? Likely you forgot a `def`, `do`, or maybe you deleted some code a
 
 What if I told you, that there was a library that helped find your missing `def`s and missing `do`s. What if instead of searching through hundreds of lines of source for the cause of your syntax error, there was a way to highlight just code in the file that contained syntax errors.
 
-```
-$ syntax_search <path/to/file.rb>
+    $ syntax_search path/to/file.rb
 
-SyntaxErrorSearch: A syntax error was detected
+    SyntaxSearch: Unmatched `end` detected
 
-This code has an unmatched `end` this is caused by either
-missing a syntax keyword (`def`,  `do`, etc.) or inclusion
-of an extra `end` line
+    This code has an unmatched `end`. Ensure that all `end` lines
+    in your code have a matching syntax keyword  (`def`,  `do`, etc.)
+    and that you don't have any extra `end` lines.
 
-file: path/to/file.rb
-simplified:
+    file: path/to/file.rb
+    simplified:
 
-  ```
-       1  require 'animals'
+       1  require 'zoo'
        2
-    ❯ 10  defdog
-    ❯ 15  end
-    ❯ 16
-      20  def cat
-      22  end
-  ```
-```
+       3  class Animal
+       4
+    ❯  5    defdog
+    ❯  7    end
+       8
+      12  end
 
 How much would you pay for such a library? A million, a billion, a trillion? Well friends, today is your lucky day because you can use this library today for free!
 
-## Installation
+## Installation in your codebase
 
-Add this line to your application's Gemfile:
+To automatically search syntax errors when they happen, add this to your Gemfile:
 
 ```ruby
 gem 'syntax_search', require: "syntax_search/auto"
@@ -47,10 +44,28 @@ And then execute:
 
     $ bundle install
 
-Or install it yourself as:
+If your application is not calling `Bundler.require` then you must manually add a require:
+
+```ruby
+require "syntax_search/auto"
+```
+
+If you're using rspec add this to your `.rspec` file:
+
+```
+--require syntax_search/auto
+```
+
+> This is needed because people can execute a single test file via `bundle exec rspec path/to/file_spec.rb` and if that file has a syntax error, it won't load `spec_helper.rb` to trigger any requires.
+
+## Install the CLI
+
+To get the CLI and manually search for syntax errors, install the gem:
 
     $ gem install syntax_search
 
+This gives you the CLI command `$ syntax_search` for more info run `$ syntax_search --help`.
+
 ## What does it do?
 
 When your code triggers a SyntaxError due to an "unexpected `end'" in a file, this library fires to narrow down your search to the most likely offending locations.
@@ -68,6 +83,10 @@ We know that source code that does not contain a syntax error can be parsed. We
 
 Since there can be multiple syntax errors in a document it's not good enough to check individual code blocks, we've got to check multiple at the same time. We will keep creating and adding new blocks to our search until we detect that our "frontier" (which contains all of our blocks) contains the syntax error. After this, we can stop our search and instead focus on filtering to find the smallest subset of blocks that contain the syntax error.
 
+Here's an example:
+
+![](assets/syntax_search.gif)
+
 ## How is source code broken up into smaller blocks?
 
 By definition source code with a syntax error in it cannot be parsed, so we have to guess how to chunk up the file into smaller pieces. Once we've split up the file we can safely rule out or zoom into a specific piece of code to determine the location of the syntax error. This libary uses indentation and empty lines to make guesses about what might be a "block" of code. Once we've got a chunk of code, we can test it.

data/lib/syntax_search.rb

@@ -2,17 +2,17 @@
 
 require_relative "syntax_search/version"
 
-require 'parser/current'
 require 'tmpdir'
 require 'stringio'
 require 'pathname'
+require 'ripper'
 
 module SyntaxErrorSearch
   class Error < StandardError; end
   SEARCH_SOURCE_ON_ERROR_DEFAULT = true
 
   def self.handle_error(e, search_source_on_error: SEARCH_SOURCE_ON_ERROR_DEFAULT)
-    raise e if !e.message.include?("expecting end-of-input")
+    raise e if !e.message.include?("end-of-input")
 
     filename = e.message.split(":").first
 
@@ -23,7 +23,7 @@ module SyntaxErrorSearch
       self.call(
         source: Pathname(filename).read,
         filename: filename,
-        terminal: true
+        terminal: true,
       )
     end
 
@@ -40,6 +40,8 @@ module SyntaxErrorSearch
       blocks: blocks,
       filename: filename,
       terminal: terminal,
+      code_lines: search.code_lines,
+      invalid_type: invalid_type(source),
       io: $stderr
     ).call
   end
@@ -78,6 +80,13 @@ module SyntaxErrorSearch
     end
   end
 
+  def self.invalid?(source)
+    source = source.join if source.is_a?(Array)
+    source = source.to_s
+
+    Ripper.new(source).tap(&:parse).error?
+  end
+
   # Returns truthy if a given input source is valid syntax
   #
   #   SyntaxErrorSearch.valid?(<<~EOM) # => true
@@ -113,24 +122,23 @@ module SyntaxErrorSearch
   # so passing a CodeLine in as an object or as an array
   # will convert it to it's code representation.
   def self.valid?(source)
-    source = source.join if source.is_a?(Array)
-    source = source.to_s
+    !invalid?(source)
+  end
 
-    # Parser writes to stderr even if you catch the error
-    stderr = $stderr
-    $stderr = StringIO.new
 
-    Parser::CurrentRuby.parse(source)
-    true
-  rescue Parser::SyntaxError
-    false
-  ensure
-    $stderr = stderr if stderr
+  def self.invalid_type(source)
+    WhoDisSyntaxError.new(source).call.error_symbol
   end
 end
 
 require_relative "syntax_search/code_line"
 require_relative "syntax_search/code_block"
 require_relative "syntax_search/code_frontier"
-require_relative "syntax_search/code_search"
 require_relative "syntax_search/display_invalid_blocks"
+require_relative "syntax_search/around_block_scan"
+require_relative "syntax_search/block_expand"
+require_relative "syntax_search/parse_blocks_from_indent_line"
+
+require_relative "syntax_search/code_search"
+require_relative "syntax_search/who_dis_syntax_error"
+require_relative "syntax_search/heredoc_block_parse"

data/lib/syntax_search/around_block_scan.rb

@@ -0,0 +1,91 @@
+# frozen_string_literal: true
+#
+module SyntaxErrorSearch
+  # This class is useful for exploring contents before and after
+  # a block
+  #
+  # It searches above and below the passed in block to match for
+  # whatever criteria you give it:
+  #
+  # Example:
+  #
+  #   def dog
+  #     puts "bark"
+  #     puts "bark"
+  #   end
+  #
+  #   scan = AroundBlockScan.new(
+  #     code_lines: code_lines
+  #     block: CodeBlock.new(lines: code_lines[1])
+  #   )
+  #
+  #   scan.scan_while { true }
+  #
+  #   puts scan.before_index # => 0
+  #   puts scan.after_index # => 3
+  #
+  # Contents can also be filtered using AroundBlockScan#skip
+  #
+  # To grab the next surrounding indentation use AroundBlockScan#scan_adjacent_indent
+  class AroundBlockScan
+    def initialize(code_lines: , block:)
+      @code_lines = code_lines
+      @orig_before_index = block.lines.first.index
+      @orig_after_index = block.lines.last.index
+      @skip_array = []
+      @after_array = []
+      @before_array = []
+    end
+
+    def skip(name)
+      @skip_array << name
+      self
+    end
+
+    def scan_while(&block)
+      @before_index = before_lines.reverse_each.take_while do |line|
+        next true if @skip_array.detect {|meth| line.send(meth) }
+
+        block.call(line)
+      end.reverse.first&.index
+
+      @after_index = after_lines.take_while do |line|
+        next true if @skip_array.detect {|meth| line.send(meth) }
+
+        block.call(line)
+      end.last&.index
+      self
+    end
+
+    def scan_adjacent_indent
+      before_indent = @code_lines[@orig_before_index.pred]&.indent || 0
+      after_indent = @code_lines[@orig_after_index.next]&.indent || 0
+
+      indent = [before_indent, after_indent].min
+      @before_index = before_index.pred if before_indent >= indent
+      @after_index = after_index.next if after_indent >= indent
+
+      self
+    end
+
+    def code_block
+      CodeBlock.new(lines: @code_lines[before_index..after_index])
+    end
+
+    def before_index
+      @before_index || @orig_before_index
+    end
+
+    def after_index
+      @after_index || @orig_after_index
+    end
+
+    private def before_lines
+      @code_lines[0...@orig_before_index]
+    end
+
+    private def after_lines
+      @code_lines[@orig_after_index.next..-1]
+    end
+  end
+end

data/lib/syntax_search/block_expand.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+module SyntaxErrorSearch
+  # This class is responsible for taking a code block that exists
+  # at a far indentaion and then iteratively increasing the block
+  # so that it captures everything within the same indentation block.
+  #
+  #   def dog
+  #     puts "bow"
+  #     puts "wow"
+  #   end
+  #
+  # block = BlockExpand.new(code_lines: code_lines)
+  #   .call(CodeBlock.new(lines: code_lines[1]))
+  #
+  # puts block.to_s
+  # # => puts "bow"
+  #      puts "wow"
+  #
+  #
+  # Once a code block has captured everything at a given indentation level
+  # then it will expand to capture surrounding indentation.
+  #
+  # block = BlockExpand.new(code_lines: code_lines)
+  #   .call(block)
+  #
+  # block.to_s
+  # # => def dog
+  #        puts "bow"
+  #        puts "wow"
+  #      end
+  #
+  class BlockExpand
+    def initialize(code_lines: )
+      @code_lines = code_lines
+    end
+
+    def call(block)
+      if (next_block = expand_neighbors(block, grab_empty: true))
+        return next_block
+      end
+
+      expand_indent(block)
+    end
+
+    def expand_indent(block)
+      block = AroundBlockScan.new(code_lines: @code_lines, block: block)
+        .scan_adjacent_indent
+        .code_block
+
+      # Handle if/else/end case
+      if (next_block = expand_neighbors(block, grab_empty: false))
+        return next_block
+      else
+        return block
+      end
+    end
+
+    def expand_neighbors(block, grab_empty: true)
+      scan = AroundBlockScan.new(code_lines: @code_lines, block: block)
+        .skip(:hidden?)
+        .scan_while {|line| line.not_empty? && line.indent >= block.current_indent }
+
+      # Slurp up empties
+      if grab_empty
+        scan = AroundBlockScan.new(code_lines: @code_lines, block: scan.code_block)
+          .scan_while {|line| line.empty? || line.hidden? }
+      end
+
+      new_block = scan.code_block
+
+      if block.lines == new_block.lines
+        return nil
+      else
+        return new_block
+      end
+    end
+  end
+end

data/lib/syntax_search/code_block.rb

@@ -3,11 +3,7 @@
 module SyntaxErrorSearch
   # Multiple lines form a singular CodeBlock
   #
-  # Source code is made of multiple CodeBlocks. A code block
-  # has a reference to the source code that created itself, this allows
-  # a code block to "expand" when needed
-  #
-  # The most important ability of a CodeBlock is this ability to expand:
+  # Source code is made of multiple CodeBlocks.
   #
   # Example:
   #
@@ -16,33 +12,35 @@ module SyntaxErrorSearch
   #     #     puts "foo"
   #     #   end
   #
-  #   code_block.expand_until_next_boundry
+  #   code_block.valid? # => true
+  #   code_block.in_valid? # => false
   #
-  #   code_block.to_s # =>
-  #     # class Foo
-  #     #   def foo
-  #     #     puts "foo"
-  #     #   end
-  #     # end
   #
   class CodeBlock
     attr_reader :lines
 
-    def initialize(code_lines: nil, lines: [])
+    def initialize(lines: [])
       @lines = Array(lines)
-      @code_lines = code_lines
+    end
+
+    def mark_invisible
+      @lines.map(&:mark_invisible)
     end
 
     def is_end?
       to_s.strip == "end"
     end
 
+    def hidden?
+      @lines.all?(&:hidden?)
+    end
+
     def starts_at
-      @lines.first&.line_number
+      @starts_at ||= @lines.first&.line_number
     end
 
-    def code_lines
-      @code_lines
+    def ends_at
+      @ends_at ||= @lines.last&.line_number
     end
 
     # This is used for frontier ordering, we are searching from
@@ -53,155 +51,8 @@ module SyntaxErrorSearch
       self.current_indent <=> other.current_indent
     end
 
-    # Only the lines that are not empty and visible
-    def visible_lines
-      @lines
-        .select(&:not_empty?)
-        .select(&:visible?)
-    end
-
-    # This method is used to expand a code block to capture it's calling context
-    def expand_until_next_boundry
-      expand_to_indent(next_indent)
-      self
-    end
-
-    # This method expands the given code block until it captures
-    # its nearest neighbors. This is used to expand a single line of code
-    # to its smallest likely block.
-    #
-    #   code_block.to_s # =>
-    #     #     puts "foo"
-    #   code_block.expand_until_neighbors
-    #
-    #   code_block.to_s # =>
-    #     #     puts "foo"
-    #     #     puts "bar"
-    #     #     puts "baz"
-    #
-    def expand_until_neighbors
-      expand_to_indent(current_indent)
-
-      expand_hidden_parner_line if self.to_s.strip == "end"
-      self
-    end
-
-    def expand_hidden_parner_line
-      index = @lines.first.index
-      indent = current_indent
-      partner_line  = code_lines.select {|line| line.index < index && line.indent == indent }.last
-
-      if partner_line&.hidden?
-        partner_line.mark_visible
-        @lines.prepend(partner_line)
-      end
-    end
-
-    # This method expands the existing code block up (before)
-    # and down (after). It will break on change in indentation
-    # and empty lines.
-    #
-    #   code_block.to_s # =>
-    #     #   def foo
-    #     #     puts "foo"
-    #     #   end
-    #
-    #   code_block.expand_to_indent(0)
-    #   code_block.to_s # =>
-    #     # class Foo
-    #     #   def foo
-    #     #     puts "foo"
-    #     #   end
-    #     # end
-    #
-    private def expand_to_indent(indent)
-      array = []
-      before_lines(skip_empty: false).each do |line|
-        if line.empty?
-          array.prepend(line)
-          break
-        end
-
-        if line.indent == indent
-          array.prepend(line)
-        else
-          break
-        end
-      end
-
-      array << @lines
-
-      after_lines(skip_empty: false).each do |line|
-        if line.empty?
-          array << line
-          break
-        end
-
-        if line.indent == indent
-          array << line
-        else
-          break
-        end
-      end
-
-      @lines = array.flatten
-    end
-
-    def next_indent
-      [
-        before_line&.indent || 0,
-        after_line&.indent || 0
-      ].max
-    end
-
     def current_indent
-      lines.detect(&:not_empty?)&.indent || 0
-    end
-
-    def before_line
-      before_lines.first
-    end
-
-    def after_line
-      after_lines.first
-    end
-
-    def before_lines(skip_empty: true)
-      index = @lines.first.index
-      lines = code_lines.select {|line| line.index < index }
-      lines.select!(&:not_empty?) if skip_empty
-      lines.select!(&:visible?)
-      lines.reverse!
-
-      lines
-    end
-
-    def after_lines(skip_empty: true)
-      index = @lines.last.index
-      lines = code_lines.select {|line| line.index > index }
-      lines.select!(&:not_empty?) if skip_empty
-      lines.select!(&:visible?)
-      lines
-    end
-
-    # Returns a code block of the source that does not include
-    # the current lines. This is useful for checking if a source
-    # with the given lines removed parses successfully. If so
-    #
-    # Then it's proof that the current block is invalid
-    def block_without
-      @block_without ||= CodeBlock.new(
-        source: @source,
-        lines: @source.code_lines - @lines
-      )
-    end
-
-    def document_valid_without?
-      block_without.valid?
-    end
-
-    def valid_without?
-      block_without.valid?
+      @current_indent ||= lines.select(&:not_empty?).map(&:indent).min || 0
     end
 
     def invalid?