loxxy 0.0.4 → 0.0.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f5383d6eb1725f5c53bb343e46d763f18d9d69f6b010241dbc4a93c18f949a6f
-  data.tar.gz: 286f213ab01ebec8061a9f25c86c6f28be064891880e06942a0d6d86b4210fcb
+  metadata.gz: 74cd4aa3bd58cad64cad8ac452535052c761f166da883b06b593590310edd97d
+  data.tar.gz: 15af4b4a4c86c7b8e6a14bda16433a9d1663ab581eae78331d29adeed02313d3
 SHA512:
-  metadata.gz: 32d447c7dbe6593ad0002ad383a917561c82da22cbf70b4102d0eaa389cf92196e03ee14ab1cc160f530b1c96a1e259db8e7923e2de81322fc854898fa306d49
-  data.tar.gz: 245d4c9e255b7bb9d8602ab103367d7f83baf516db61f5a9192945ffa689027eb9f5f7a282e7052bca6b9e904cd6f6ed956d61ea949e02b7916f100033ebd9ec
+  metadata.gz: 0dbd9244c36913eed955b1ff92bdc63fd77cc6f7cda3d6b037f3ee87cdcc8d4450673f852426a7395d968711aaf9a3a539efeeaf5e2a0d46865f4f4fa7276c74
+  data.tar.gz: c8c65b58fa53d0cce81a3e7508317dfbeb5d8e10f883a0a011d278ca33ee647967bfd34e79fc4703ad7a5b4a3a88aece8d17ca0d0db107f0ec0fcb9ac12e4280

data/.rubocop.yml

@@ -1,6 +1,7 @@
 AllCops:
   Exclude:
     - 'exp/**/*'
+    - 'demo/**/*'
 
 Layout/ArgumentAlignment:
   Enabled: false
@@ -55,6 +56,9 @@ Layout/MultilineMethodCallBraceLayout:
 
 Layout/SpaceAroundOperators:
   Enabled: true
+  
+Layout/SpaceBeforeBrackets:
+  Enabled: true
 
 Layout/SpaceInsideParens:
   Enabled: true
@@ -70,6 +74,9 @@ Layout/TrailingEmptyLines:
 
 Layout/TrailingWhitespace:
   Enabled: true
+  
+Lint/AmbiguousAssignment:
+  Enabled: true
 
 Lint/DuplicateBranch:
   Enabled: true
@@ -163,6 +170,9 @@ Naming/BlockParameterName:
 
 Naming/MethodParameterName:
   Enabled: false
+  
+Naming/MethodName:
+  Enabled: false
 
 Naming/VariableName:
   Enabled: false
@@ -226,6 +236,9 @@ Style/GuardClause:
 
 Style/HashEachMethods:
   Enabled: true
+  
+Style/HashExcept:
+  Enabled: true
 
 Style/HashTransformKeys:
   Enabled: true

data/.yardopts

@@ -0,0 +1,6 @@
+--exclude examples --exclude features --exclude spec
+--no-private
+--markup markdown
+-
+CHANGELOG.md
+LICENSE.txt

data/CHANGELOG.md

@@ -1,3 +1,57 @@
+## [0.0.9] - 2021-01-07
+- AST node generation for comparison expression.
+
+## Changed
+- Class `AST::ASTBuilder` added `reduce_` methods for comparison operations.
+- File `grammar.rb`added name to comparison rules
+
+## [0.0.8] - 2021-01-07
+- AST node generation for arithmetic operations of number literals.
+
+## Changed
+- Class `AST::ASTBuilder` added `reduce_` methods for arithmetic operations.
+- File `grammar.rb`added name to arithmetic rules
+
+## Fixed
+- File `grammar.rb`: second rule for `factor` had a missing member in rhs.
+
+## [0.0.7] - 2021-01-06
+- Lox grammar reworked, initial AST classes created.
+
+## Added
+- Class `Parser` this one generates AST's (Abstract Syntax Tree)
+- Class `AST::ASTVisitor` draft initial implementation.
+- Class `AST::BinaryExpr` draft initial implementation.
+- Class `AST::LoxCompoundExpr` draft initial implementation.
+- Class `AST::LiteralExpr` draft initial implementation.
+- Class `AST::LoxNode` draft initial implementation.
+
+## Changed
+- File `spec_helper.rb`: removed Bundler dependency
+- Class `AST::ASTBuilder` added initial `reduce_` methods.
+- File `README.md` Removed example with AST generation since this is in flux.
+
+## [0.0.6] - 2021-01-03
+- First iteration of a parser with AST generation.
+
+## Added
+- Class `Parser` this one generates AST's (Abstract Syntax Tree)
+- Class `AST::ASTBuilder` default code to generate an AST.
+
+## Changed
+- File `spec_helper.rb`: removed Bundler dependency
+- File `README.md` Added example with AST visualization.
+
+## Fixed
+- File `grammar.rb` ensure that the constant `Grammar` is created once only.
+
+## [0.0.5] - 2021-01-02
+- Improved example in `README.md`, code re-styling to please `Rubocop` 1.7
+
+## Changed
+- Code re-styling to please `Rubocop` 1.7
+- File `README.md` Improved example with better parse tree visualization.
+
 ## [0.0.4] - 2021-01-01
 - A first parser implementation able to parse Lox source code.
 

data/README.md

@@ -1,7 +1,7 @@
 # Loxxy
 
-A Ruby implementation of the [Lox programming language](https://craftinginterpreters.com/the-lox-language.html),
-a simple language used in Bob Nystrom's online book [Crafting Interpreters](https://craftinginterpreters.com/).
+A Ruby implementation of the [Lox programming language](https://craftinginterpreters.com/the-lox-language.html ),
+a simple language used in Bob Nystrom's online book [Crafting Interpreters](https://craftinginterpreters.com/ ).
 
 ## Purpose of this project:
 - To deliver an open source example of a programming language fully implemented in Ruby  
@@ -9,17 +9,23 @@ a simple language used in Bob Nystrom's online book [Crafting Interpreters](http
 - The implementation should be mature enough to run (LoxLox)[https://github.com/benhoyt/loxlox],  
   a Lox interpreter written in Lox.
 
+## Current status
+The __loxxy__ gem hosts two distinct parsers classes (`RawParser` and `Parser`).
+- A `RawParser` instance is able to recognize valid Lox input and to generate
+  a concrete parse tree from it.
+- A `Parser` instance can also parse Lox source code but will generate an AST
+  (Abstract Syntax Tree) that will be used by the future tree-walking interpreter.
+  Currently it generates AST for arithmetic expressions with literal numbers only.
+
 ## Roadmap
 - [DONE] Scanner (tokenizer)
 - [DONE] Raw parser. It parses Lox programs and generates a parse tree.
-- [TODO] Tailored parser for generating AST (Abstract Syntax Tree)
+- [DONE] Tailored parser for generating AST (Abstract Syntax Tree)
+- [STARTED] Custom AST builder class  
+- [STARTED] Hierarchy classes for representing Lox expressions in AST  
 - [TODO] Interpreter or transpiler
 
-
-## Example
-At this, stage, the raw parser is able to recognize Lox input and to generate
-a concrete parse tree from it.
-
+## Example using RawParser class
 ```ruby
 require 'loxxy'
 
@@ -34,13 +40,35 @@ base_parser = Loxxy::FrontEnd::RawParser.new
 # Now parse the input into a concrete parse tree...
 ptree = base_parser.parse(lox_input)
 
-# Dump the parse tree contents...
-p ptree # Lengthy (unwieldy) output
+# Display the parse tree thanks to Rley formatters...
+visitor = Rley::ParseTreeVisitor.new(ptree)
+tree_formatter = Rley::Formatter::Asciitree.new($stdout)
+tree_formatter.render(visitor)
 ```
 
-Soon, the `loxxy`will host another, tailored parser, that will generate
-abstract syntax tree which much more convenient for further processing 
-(like implementing an interpreter).
+This is the output produced by the above example:
+```
+program
++-- declaration_plus
+|   +-- declaration
+|       +-- statement
+|           +-- printStmt
+|               +-- PRINT: 'print'
+|               +-- expression
+|               |   +-- assignment
+|               |       +-- logic_or
+|               |           +-- logic_and
+|               |               +-- equality
+|               |                   +-- comparison
+|               |                       +-- term
+|               |                           +-- factor
+|               |                               +-- unary
+|               |                                   +-- call
+|               |                                       +-- primary
+|               |                                           +-- STRING: '"Hello, world!"'
+|               +-- SEMICOLON: ';'
++-- EOF: ''
+```
 
 ## Installation
 
@@ -63,9 +91,9 @@ Or install it yourself as:
 TODO: Write usage instructions here
 
 ## Other Lox implementations in Ruby
-An impressive list of Lox implementations can be found [here](https://github.com/munificent/craftinginterpreters/wiki/Lox-implementations
+An impressive list of Lox implementations can be found [here](https://github.com/munificent/craftinginterpreters/wiki/Lox-implementations)
 
-For Ruby, ther is the [lox](https://github.com/rdodson41/ruby-lox) gem.
+For Ruby, there is the [lox](https://github.com/rdodson41/ruby-lox) gem.
 There are other Ruby-based projects as well:  
 - [SlowLox](https://github.com/ArminKleinert/SlowLox), described as a "1-to-1 conversion of JLox to Ruby"
 - [rulox](https://github.com/LevitatingBusinessMan/rulox)

data/lib/loxxy.rb

@@ -2,6 +2,7 @@
 
 require_relative 'loxxy/version'
 require_relative 'loxxy/front_end/raw_parser'
+require_relative 'loxxy/front_end/parser'
 
 module Loxxy
   class Error < StandardError; end

data/lib/loxxy/ast/ast_builder.rb

@@ -0,0 +1,157 @@
+# frozen_string_literal: true
+
+require_relative '../datatype/all_datatypes'
+require_relative 'lox_literal_expr'
+require_relative 'lox_binary_expr'
+
+module Loxxy
+  module Ast
+    # The purpose of ASTBuilder is to build piece by piece an AST
+    # (Abstract Syntax Tree) from a sequence of input tokens and
+    # visit events produced by walking over a GFGParsing object.
+    class ASTBuilder < Rley::ParseRep::ASTBaseBuilder
+      # Terminal2NodeClass = {
+      #   'FALSE' => Datatype::False,
+      #   'NIL' => Datatype::Nil,
+      #   'NUMBER' => Datatype::Number,
+      #   'STRING' => Datatype::LXString,
+      #   'TRUE' => Datatype::True
+      # }.freeze
+
+      attr_reader :strict
+
+      # Create a new AST builder instance.
+      # @param theTokens [Array<Rley::Lexical::Token>] The sequence of input tokens.
+      def initialize(theTokens)
+        super(theTokens)
+        @strict = false
+      end
+
+      protected
+
+      def terminal2node
+        Terminal2NodeClass
+      end
+
+      # Method override
+      def new_leaf_node(_production, _terminal, aTokenPosition, aToken)
+        Rley::PTree::TerminalNode.new(aToken, aTokenPosition)
+      end
+
+      # Factory method for creating a parent node object.
+      # @param aProduction [Production] Production rule
+      # @param aRange [Range] Range of tokens matched by the rule
+      # @param theTokens [Array] The input tokens
+      # @param theChildren [Array] Children nodes (one per rhs symbol)
+      def new_parent_node(aProduction, aRange, theTokens, theChildren)
+        mth_name = method_name(aProduction.name)
+        if respond_to?(mth_name, true)
+          node = send(mth_name, aProduction, aRange, theTokens, theChildren)
+        else
+          # Default action...
+          node = case aProduction.rhs.size
+                 when 0
+                   return_epsilon(aRange, theTokens, theChildren)
+                 when 1
+                   return_first_child(aRange, theTokens, theChildren)
+                 else
+                   if strict
+                     msg = "Don't know production '#{aProduction.name}'"
+                     raise StandardError, msg
+                   else
+                     node = Rley::PTree::NonTerminalNode.new(aProduction.lhs, aRange)
+                     theChildren&.reverse_each do |child|
+                       node.add_subnode(child) if child
+                     end
+
+                     node
+                   end
+                 end
+        end
+
+        node
+      end
+
+      # rule('lhs' => 'nonterm_i nonterm_k_plus')
+      def reduce_binary_operator(_production, _range, tokens, theChildren)
+        operand1 = theChildren[0]
+
+        # Second child is anray with couples [operator, operand2]
+        theChildren[1].each do |(operator, operand2)|
+          operand1 = LoxBinaryExpr.new(tokens[0].position, operator, operand1, operand2)
+        end
+
+        operand1
+      end
+
+      # rule('comparison' => 'term comparisonTest_plus')
+      def reduce_comparison_plus(production, range, tokens, theChildren)
+        reduce_binary_operator(production, range, tokens, theChildren)
+      end
+
+      # rule('comparisonTest_plus' => 'comparisonTest_plus comparisonTest term').as 'comparison_t_plus_more'
+      # TODO: is it meaningful to implement this rule?
+
+      # rule('comparisonTest_plus' => 'comparisonTest term')
+      def reduce_comparison_t_plus_end(_production, _range, _tokens, theChildren)
+        name2operators = {
+          'GREATER' => '>',
+          'GREATER_EQUAL' => '>=',
+          'LESS' =>  '<',
+          'LESS_EQUAL' => '<='
+        }
+        operator = name2operators[theChildren[0].symbol.name].to_sym
+        operand2 = theChildren[1]
+        [[operator, operand2]]
+      end
+
+      # rule('term' => 'factor additive_plus')
+      def reduce_term_additive(production, range, tokens, theChildren)
+        reduce_binary_operator(production, range, tokens, theChildren)
+      end
+
+      # rule('additive_star' => 'additive_star additionOp factor').as 'additionOp_expr'
+      def reduce_additive_plus_more(_production, _range, _tokens, theChildren)
+        result = theChildren[0]
+        operator = theChildren[1].symbol.name == 'MINUS' ? :- : :+
+        operand2 = theChildren[2]
+        result << [operator, operand2]
+      end
+
+      # rule('additive_plus' => 'additionOp factor')
+      def reduce_additive_plus_end(_production, _range, _tokens, theChildren)
+        operator = theChildren[0].symbol.name == 'MINUS' ? :- : :+
+        operand2 = theChildren[1]
+        [[operator, operand2]]
+      end
+
+      # rule('factor' => 'multiplicative_plus')
+      def reduce_factor_multiplicative(production, range, tokens, theChildren)
+        reduce_binary_operator(production, range, tokens, theChildren)
+      end
+
+      # rule('multiplicative_plus' => 'multiplicative_plus multOp unary')
+      def reduce_multiplicative_plus_more(_production, _range, _tokens, theChildren)
+        result = theChildren[0]
+        operator = theChildren[1].symbol.name == 'SLASH' ? :/ : :*
+        operand2 = theChildren[2]
+        result << [operator, operand2]
+      end
+
+      # rule('multiplicative_plus' => 'multOp unary')
+      def reduce_multiplicative_plus_end(_production, _range, _tokens, theChildren)
+        operator = theChildren[0].symbol.name == 'SLASH' ? :/ : :*
+        operand2 = theChildren[1]
+        [[operator, operand2]]
+      end
+
+      # rule('primary' => 'FALSE' | TRUE').as 'literal_expr'
+      def reduce_literal_expr(_production, _range, _tokens, theChildren)
+        first_child = theChildren.first
+        pos = first_child.token.position
+        literal = first_child.token.value
+        LoxLiteralExpr.new(pos, literal)
+      end
+    end # class
+  end # module
+end # module

data/lib/loxxy/ast/ast_visitor.rb

@@ -0,0 +1,125 @@
+# frozen_string_literal: true
+
+module Loxxy
+  module Ast
+    class ASTVisitor
+      # Link to the top node to visit
+      attr_reader(:top)
+
+      # List of objects that subscribed to the visit event notification.
+      attr_reader(:subscribers)
+
+      # attr_reader(:runtime)
+
+      # Build a visitor for the given top.
+      # @param aRoot [AST::LoxNode] the parse tree to visit.
+      def initialize(aTop)
+        raise StandardError if aTop.nil?
+
+        @top = aTop
+        @subscribers = []
+      end
+
+      # Add a subscriber for the visit event notifications.
+      # @param aSubscriber [Object]
+      def subscribe(aSubscriber)
+        subscribers << aSubscriber
+      end
+
+      # Remove the given object from the subscription list.
+      # The object won't be notified of visit events.
+      # @param aSubscriber [Object]
+      def unsubscribe(aSubscriber)
+        subscribers.delete_if { |entry| entry == aSubscriber }
+      end
+
+      # The signal to begin the visit of the top.
+      def start # (aRuntime)
+        # @runtime = aRuntime
+        top.accept(self)
+      end
+
+      # Visit event. The visitor is visiting the
+      # given terminal node containing a datatype object.
+      # @param aLiteralExpr [AST::LoxLiteralExpr] the leaf node to visit.
+      def visit_literal_expr(aLiteralExpr)
+        broadcast(:before_literal_expr, aLiteralExpr)
+        broadcast(:after_literal_expr, aLiteralExpr)
+      end
+
+=begin
+
+
+      def visit_compound_datum(aCompoundDatum)
+        broadcast(:before_compound_datum, aCompoundDatum)
+        traverse_children(aCompoundDatum)
+        broadcast(:after_compound_datum, aCompoundDatum)
+      end
+
+      # Visit event. The visitor is visiting the
+      # given empty list object.
+      # @param anEmptyList [SkmEmptyList] the empty list object to visit.
+      def visit_empty_list(anEmptyList)
+        broadcast(:before_empty_list, anEmptyList)
+        broadcast(:after_empty_list, anEmptyList)
+      end
+
+      def visit_pair(aPair)
+        broadcast(:before_pair, aPair)
+        traverse_car_cdr(aPair)
+        broadcast(:after_pair, aPair)
+      end
+=end
+=begin
+   # Visit event. The visitor is about to visit the given non terminal node.
+    # @param aNonTerminalNode [NonTerminalNode] the node to visit.
+    def visit_nonterminal(aNonTerminalNode)
+      if @traversal == :post_order
+        broadcast(:before_non_terminal, aNonTerminalNode)
+        traverse_subnodes(aNonTerminalNode)
+      else
+        traverse_subnodes(aNonTerminalNode)
+        broadcast(:before_non_terminal, aNonTerminalNode)
+      end
+      broadcast(:after_non_terminal, aNonTerminalNode)
+    end
+=end
+
+      private
+
+      def traverse_children(aParent)
+        children = aParent.children
+        broadcast(:before_children, aParent, children)
+
+        # Let's proceed with the visit of children
+        children.each { |a_child| a_child.accept(self) }
+
+        broadcast(:after_children, aParent, children)
+      end
+
+      def traverse_car_cdr(aPair)
+        if aPair.car
+          broadcast(:before_car, aPair, aPair.car)
+          aPair.car.accept(self)
+          broadcast(:after_car, aPair, aPair.car)
+        end
+        if aPair.cdr
+          broadcast(:before_cdr, aPair, aPair.cdr)
+          aPair.cdr.accept(self)
+          broadcast(:after_cdr, aPair, aPair.cdr)
+        end
+      end
+
+      # Send a notification to all subscribers.
+      # @param msg [Symbol] event to notify
+      # @param args [Array] arguments of the notification.
+      def broadcast(msg, *args)
+        subscribers.each do |subscr|
+          next unless subscr.respond_to?(msg) || subscr.respond_to?(:accept_all)
+
+          subscr.send(msg, runtime, *args)
+        end
+      end
+    end # class
+  end # module
+end # module