dead_end 2.0.2 → 3.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ec9e05716b47eb5d96ac063126a59977d98b8cf3e6f09ba0a0065d425a3c15e5
-  data.tar.gz: c78047c6dd36c1c716e1970e9bec5887bbcaf1fcc0341cbb5c11405043a07105
+  metadata.gz: 72373898d38363c0ea0c3cec7fe2747399724ce55dae4eca1ce57b8f59dfc767
+  data.tar.gz: 9e99a5fddb8a054b839aef4fe1c2b87792bc9b4161bf5a6689156f2999dca935
 SHA512:
-  metadata.gz: 2eca38b35ef1371cae5fd104fa88dad33d44b17cb6bf29b01b5dea0c650d3d21c0dfbf2b1ac96f0a6f307e7d541cf2a2f7be062d59f1626b16c82f1a2f7d3a3e
-  data.tar.gz: 1a09171cb208465cf48ff5ac0346240d5bf155aebb886c27b6259742f325bc58a0b73ec9998fcbb1544440e6df7363fa7ab57a04dba9e7887207f30916e8c3a9
+  metadata.gz: 7312a010846453f222bbb8717738831f9a2e23572eace345ed7738f00429c126d1a4760548ad9e7552237aee057b5c5bae90b7a0fe197144c3a9dc9d3db2ab0b
+  data.tar.gz: 0db2b356ab237f01eda6dc5ebeac0156c317d94535c2b59d099f766e1716d5d6f6b429edcc65bf439dea1bc15e1866f4d66a41cd28d927a29a666f54f54ee804

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 ## HEAD (unreleased)
 
+## 3.0.0
+
+- [Breaking] Remove previously deprecated `require "dead_end/fyi"` interface (https://github.com/zombocom/dead_end/pull/94)
+- Fix double output bug (https://github.com/zombocom/dead_end/pull/99)
+- Fix bug causing poor results (fix #95, fix #88) (https://github.com/zombocom/dead_end/pull/96)
+- DeadEnd is now fired on EVERY syntax error (https://github.com/zombocom/dead_end/pull/94)
+- Output format changes:
+  - Parse errors emitted per-block rather than for the whole document (https://github.com/zombocom/dead_end/pull/94)
+  - The "banner" is now based on lexical analysis rather than parser regex (fix #68, fix #87) (https://github.com/zombocom/dead_end/pull/96)
+
 ## 2.0.2
 
 - Don't print terminal color codes when output is not tty (https://github.com/zombocom/dead_end/pull/91)

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    dead_end (2.0.2)
+    dead_end (3.0.0)
 
 GEM
   remote: https://rubygems.org/

data/README.md

@@ -2,19 +2,14 @@
 
 An error in your code forces you to stop. DeadEnd helps you find those errors to get you back on your way faster.
 
-    DeadEnd: Unmatched `end` detected
-
-    This code has an unmatched `end`. Ensure that all `end` lines
-    in your code have a matching syntax keyword  (`def`,  `do`, etc.)
-    and that you don't have any extra `end` lines.
-
-    file: path/to/dog.rb
-    simplified:
+```
+Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ?
 
-       3  class Dog
-    ❯  5    defbark
-    ❯  7    end
-      12  end
+  1  class Dog
+❯ 2    defbark
+❯ 4    end
+  5  end
+```
 
 ## Installation in your codebase
 
@@ -52,34 +47,99 @@ This gives you the CLI command `$ dead_end` for more info run `$ dead_end --help
 
 ## What syntax errors does it handle?
 
+Dead end will fire against all syntax errors and can isolate any syntax error. In addition, dead_end attempts to produce human readable descriptions of what needs to be done to resolve the issue. For example:
+
 - Missing `end`:
 
+<!--
 ```ruby
 class Dog
   def bark
     puts "bark"
-
-  def woof
-    puts "woof"
-  end
 end
-# => scratch.rb:8: syntax error, unexpected end-of-input, expecting `end'
 ```
+-->
 
-- Unexpected `end`
+```
+Unmatched keyword, missing `end' ?
 
+❯ 1  class Dog
+❯ 2    def bark
+❯ 4  end
+```
+
+- Missing keyword
+<!--
 ```ruby
 class Dog
   def speak
-    @sounds.each |sound| # Note the missing `do` here
+    @sounds.each |sound|
       puts sound
     end
   end
 end
-# => scratch.rb:7: syntax error, unexpected `end', expecting end-of-input
 ```
+-->
 
-As well as unmatched `|` and unmatched `}`. These errors can be time consuming to debug because Ruby often only tells you the last line in the file. The command `ruby -wc path/to/file.rb` can narrow it down a little bit, but this library does a better job.
+```
+Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ?
+
+  1  class Dog
+  2    def speak
+❯ 3      @sounds.each |sound|
+❯ 5      end
+  6    end
+  7  end
+```
+
+- Missing pair characters (like `{}`, `[]`, `()` , or `|<var>|`)
+<!--
+
+```ruby
+class Dog
+  def speak(sound
+    puts sound
+  end
+end
+```
+-->
+
+```
+Unmatched `(', missing `)' ?
+
+  1  class Dog
+❯ 2    def speak(sound
+❯ 4    end
+  5  end
+```
+
+- Any ambiguous or unknown errors will be annotated by the original ripper error output:
+
+<!--
+class Dog
+  def meals_last_month
+    puts 3 *
+  end
+end
+-->
+
+```
+syntax error, unexpected end-of-input
+
+  1  class Dog
+  2    def meals_last_month
+❯ 3      puts 3 *
+  4    end
+  5  end
+```
+
+## How is it better than `ruby -wc`?
+
+Ruby allows you to syntax check a file with warnings using `ruby -wc`. This emits a parser error instead of a human focused error. Ruby's parse errors attempt to narrow down the location and can tell you if there is a glaring indentation error involving `end`.
+
+The `dead_end` algorithm doesn't just guess at the location of syntax errors, it re-parses the document to prove that it captured them.
+
+This library focuses on the human side of syntax errors. It cares less about why the document could not be parsed (computer problem) and more on what the programmer needs (human problem) to fix the problem.
 
 ## Sounds cool, but why isn't this baked into Ruby directly?
 
@@ -105,6 +165,14 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
+### How to debug changes to output display
+
+You can see changes to output against a variety of invalid code by running specs and using the `DEBUG_DISPLAY=1` environment variable. For example:
+
+```
+$ DEBUG_DISPLAY=1 be rspec spec/ --format=failures
+```
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/zombocom/dead_end. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/zombocom/dead_end/blob/master/CODE_OF_CONDUCT.md).

data/exe/dead_end

@@ -1,81 +1,7 @@
 #!/usr/bin/env ruby
 
-require "pathname"
-require "optparse"
 require_relative "../lib/dead_end"
 
-options = {}
-options[:record_dir] = ENV["DEAD_END_RECORD_DIR"]
-
-parser = OptionParser.new do |opts|
-  opts.banner = <<~EOM
-    Usage: dead_end <file> [options]
-
-    Parses a ruby source file and searches for syntax error(s) such as
-    unexpected `end', expecting end-of-input.
-
-    Example:
-
-      $ dead_end dog.rb
-
-      # ...
-
-        ❯ 10  defdog
-        ❯ 15  end
-        ❯ 16
-
-    Env options:
-
-      DEAD_END_RECORD_DIR=<dir>
-
-      When enabled, records the steps used to search for a syntax error to the
-      given directory
-
-    Options:
-  EOM
-
-  opts.version = DeadEnd::VERSION
-
-  opts.on("--help", "Help - displays this message") do |v|
-    puts opts
-    exit
-  end
-
-  opts.on("--record <dir>", "When enabled, records the steps used to search for a syntax error to the given directory") do |v|
-    options[:record_dir] = v
-  end
-
-  opts.on("--terminal", "Enable terminal highlighting") do |v|
-    options[:terminal] = true
-  end
-
-  opts.on("--no-terminal", "Disable terminal highlighting") do |v|
-    options[:terminal] = false
-  end
-end
-parser.parse!
-
-file = ARGV[0]
-
-if file.nil? || file.empty?
-  # Display help if raw command
-  parser.parse! %w[--help]
-end
-
-file = Pathname(file)
-options[:record_dir] = "tmp" if ENV["DEBUG"]
-
-warn "Record dir: #{options[:record_dir]}" if options[:record_dir]
-
-display = DeadEnd.call(
-  source: file.read,
-  filename: file.expand_path,
-  terminal: options.fetch(:terminal, DeadEnd::DEFAULT_VALUE),
-  record_dir: options[:record_dir]
-)
-
-if display.document_ok?
-  exit(0)
-else
-  exit(1)
-end
+DeadEnd::Cli.new(
+  argv: ARGV
+).call

data/lib/dead_end/auto.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-require_relative "../dead_end/internals"
+require_relative "../dead_end"
 
 # Monkey patch kernel to ensure that all `require` calls call the same
 # method
@@ -33,23 +33,3 @@ module Kernel
     DeadEnd.handle_error(e)
   end
 end
-
-# I honestly have no idea why this Object delegation is needed
-# I keep staring at bootsnap and it doesn't have to do this
-# is there a bug in their implementation they haven't caught or
-# am I doing something different?
-class Object
-  private
-
-  def load(path, wrap = false)
-    Kernel.load(path, wrap)
-  rescue SyntaxError => e
-    DeadEnd.handle_error(e)
-  end
-
-  def require(path)
-    Kernel.require(path)
-  rescue SyntaxError => e
-    DeadEnd.handle_error(e)
-  end
-end

data/lib/dead_end/cli.rb

@@ -0,0 +1,118 @@
+# frozen_string_literal: true
+
+require "pathname"
+require "optparse"
+
+module DeadEnd
+  # All the logic of the exe/dead_end CLI in one handy spot
+  #
+  #   Cli.new(argv: ["--help"]).call
+  #   Cli.new(argv: ["<path/to/file>.rb"]).call
+  #   Cli.new(argv: ["<path/to/file>.rb", "--record=tmp"]).call
+  #   Cli.new(argv: ["<path/to/file>.rb", "--terminal"]).call
+  #
+  class Cli
+    attr_accessor :options, :file_name
+
+    # ARGV is Everything passed to the executable, does not include executable name
+    #
+    # All other intputs are dependency injection for testing
+    def initialize(argv:, exit_obj: Kernel, io: $stdout, env: ENV)
+      @options = {}
+      @parser = nil
+      options[:record_dir] = env["DEAD_END_RECORD_DIR"]
+      options[:record_dir] = "tmp" if env["DEBUG"]
+      options[:terminal] = DeadEnd::DEFAULT_VALUE
+
+      @io = io
+      @argv = argv
+      @file_name = argv[0]
+      @exit_obj = exit_obj
+    end
+
+    def call
+      if file_name.nil? || file_name.empty?
+        # Display help if raw command
+        parser.parse! %w[--help]
+      else
+        parse
+      end
+
+      # Needed for testing since we fake exit
+      return if options[:exit]
+
+      file = Pathname(file_name)
+
+      @io.puts "Record dir: #{options[:record_dir]}" if options[:record_dir]
+
+      display = DeadEnd.call(
+        io: @io,
+        source: file.read,
+        filename: file.expand_path,
+        terminal: options.fetch(:terminal, DeadEnd::DEFAULT_VALUE),
+        record_dir: options[:record_dir]
+      )
+
+      if display.document_ok?
+        @exit_obj.exit(0)
+      else
+        @exit_obj.exit(1)
+      end
+    end
+
+    def parse
+      parser.parse!(@argv)
+
+      self
+    end
+
+    def parser
+      @parser ||= OptionParser.new do |opts|
+        opts.banner = <<~EOM
+          Usage: dead_end <file> [options]
+
+          Parses a ruby source file and searches for syntax error(s) such as
+          unexpected `end', expecting end-of-input.
+
+          Example:
+
+            $ dead_end dog.rb
+
+            # ...
+
+              ❯ 10  defdog
+              ❯ 15  end
+
+          ENV options:
+
+            DEAD_END_RECORD_DIR=<dir>
+
+            Records the steps used to search for a syntax error
+            to the given directory
+
+          Options:
+        EOM
+
+        opts.version = DeadEnd::VERSION
+
+        opts.on("--help", "Help - displays this message") do |v|
+          @io.puts opts
+          options[:exit] = true
+          @exit_obj.exit
+        end
+
+        opts.on("--record <dir>", "Records the steps used to search for a syntax error to the given directory") do |v|
+          options[:record_dir] = v
+        end
+
+        opts.on("--terminal", "Enable terminal highlighting") do |v|
+          options[:terminal] = true
+        end
+
+        opts.on("--no-terminal", "Disable terminal highlighting") do |v|
+          options[:terminal] = false
+        end
+      end
+    end
+  end
+end

data/lib/dead_end/code_block.rb

@@ -70,8 +70,24 @@ module DeadEnd
     end
 
     def valid?
-      return @valid if @valid != UNSET
-      @valid = DeadEnd.valid?(to_s)
+      if @valid == UNSET
+        # Performance optimization
+        #
+        # If all the lines were previously hidden
+        # and we expand to capture additional empty
+        # lines then the result cannot be invalid
+        #
+        # That means there's no reason to re-check all
+        # lines with ripper (which is expensive).
+        # Benchmark in commit message
+        @valid = if lines.all? { |l| l.hidden? || l.empty? }
+          true
+        else
+          DeadEnd.valid?(lines.map(&:original).join)
+        end
+      else
+        @valid
+      end
     end
 
     def to_s

data/lib/dead_end/code_frontier.rb

@@ -54,18 +54,42 @@ module DeadEnd
       @code_lines = code_lines
       @frontier = []
       @unvisited_lines = @code_lines.sort_by(&:indent_index)
+      @has_run = false
+      @check_next = true
     end
 
     def count
       @frontier.count
     end
 
+    # Performance optimization
+    #
+    # Parsing with ripper is expensive
+    # If we know we don't have any blocks with invalid
+    # syntax, then we know we cannot have found
+    # the incorrect syntax yet.
+    #
+    # When an invalid block is added onto the frontier
+    # check document state
+    private def can_skip_check?
+      check_next = @check_next
+      @check_next = false
+
+      if check_next
+        false
+      else
+        true
+      end
+    end
+
     # Returns true if the document is valid with all lines
     # removed. By default it checks all blocks in present in
     # the frontier array, but can be used for arbitrary arrays
     # of codeblocks as well
-    def holds_all_syntax_errors?(block_array = @frontier)
-      without_lines = block_array.map do |block|
+    def holds_all_syntax_errors?(block_array = @frontier, can_cache: true)
+      return false if can_cache && can_skip_check?
+
+      without_lines = block_array.flat_map do |block|
         block.lines
       end
 
@@ -120,6 +144,8 @@ module DeadEnd
       @frontier.reject! { |b|
         b.starts_at >= block.starts_at && b.ends_at <= block.ends_at
       }
+
+      @check_next = true if block.invalid?
       @frontier << block
       @frontier.sort!
 
@@ -142,7 +168,7 @@ module DeadEnd
     # the smallest possible set of blocks that contain all the syntax errors
     def detect_invalid_blocks
       self.class.combination(@frontier.select(&:invalid?)).detect do |block_array|
-        holds_all_syntax_errors?(block_array)
+        holds_all_syntax_errors?(block_array, can_cache: false)
       end || []
     end
   end

data/lib/dead_end/code_search.rb

@@ -73,12 +73,13 @@ module DeadEnd
         puts "  block indent:      #{block.current_indent}"
       end
       @record_dir.join(filename).open(mode: "a") do |f|
-        display = DisplayInvalidBlocks.new(
-          blocks: block,
+        document = DisplayCodeWithLineNumbers.new(
+          lines: @code_lines.select(&:visible?),
           terminal: false,
-          code_lines: @code_lines
-        )
-        f.write(display.indent(display.code_with_lines))
+          highlight_lines: block.lines
+        ).call
+
+        f.write(document)
       end
     end
 

data/lib/dead_end/display_invalid_blocks.rb

@@ -1,6 +1,5 @@
 # frozen_string_literal: true
 
-require_relative "banner"
 require_relative "capture_code_context"
 require_relative "display_code_with_line_numbers"
 
@@ -9,18 +8,13 @@ module DeadEnd
   class DisplayInvalidBlocks
     attr_reader :filename
 
-    def initialize(code_lines:, blocks:, io: $stderr, filename: nil, terminal: DEFAULT_VALUE, invalid_obj: WhoDisSyntaxError::Null.new)
-      @terminal = terminal == DEFAULT_VALUE ? io.isatty : terminal
-
-      @filename = filename
+    def initialize(code_lines:, blocks:, io: $stderr, filename: nil, terminal: DEFAULT_VALUE)
       @io = io
-
       @blocks = Array(blocks)
-
-      @invalid_lines = @blocks.map(&:lines).flatten
+      @filename = filename
       @code_lines = code_lines
 
-      @invalid_obj = invalid_obj
+      @terminal = terminal == DEFAULT_VALUE ? io.isatty : terminal
     end
 
     def document_ok?
@@ -30,61 +24,58 @@ module DeadEnd
     def call
       if document_ok?
         @io.puts "Syntax OK"
-      else
-        found_invalid_blocks
+        return self
       end
-      self
-    end
 
-    private def no_invalid_blocks
-      @io.puts <<~EOM
-      EOM
-    end
-
-    private def found_invalid_blocks
-      @io.puts
-      if banner
-        @io.puts banner
+      if filename
+        @io.puts("--> #{filename}")
         @io.puts
       end
-      @io.puts("file: #{filename}") if filename
-      @io.puts <<~EOM
-        simplified:
-
-        #{indent(code_block)}
-      EOM
-    end
-
-    def banner
-      Banner.new(invalid_obj: @invalid_obj).call
-    end
+      @blocks.each do |block|
+        display_block(block)
+      end
 
-    def indent(string, with: "    ")
-      string.each_line.map { |l| with + l }.join
+      self
     end
 
-    def code_block
-      string = +""
-      string << code_with_context
-      string
-    end
+    private def display_block(block)
+      # Build explanation
+      explain = ExplainSyntax.new(
+        code_lines: block.lines
+      ).call
 
-    def code_with_context
+      # Enhance code output
+      # Also handles several ambiguious cases
       lines = CaptureCodeContext.new(
-        blocks: @blocks,
+        blocks: block,
         code_lines: @code_lines
       ).call
 
-      DisplayCodeWithLineNumbers.new(
+      # Build code output
+      document = DisplayCodeWithLineNumbers.new(
         lines: lines,
         terminal: @terminal,
-        highlight_lines: @invalid_lines
+        highlight_lines: block.lines
       ).call
+
+      # Output syntax error explanation
+      explain.errors.each do |e|
+        @io.puts e
+      end
+      @io.puts
+
+      # Output code
+      @io.puts(document)
     end
 
-    def code_with_lines
+    private def code_with_context
+      lines = CaptureCodeContext.new(
+        blocks: @blocks,
+        code_lines: @code_lines
+      ).call
+
       DisplayCodeWithLineNumbers.new(
-        lines: @code_lines.select(&:visible?),
+        lines: lines,
         terminal: @terminal,
         highlight_lines: @invalid_lines
       ).call