loxxy 0.0.3 → 0.0.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3f8733558204a56db1177c9c25e5fae5af1676c455b1e3a2f3145a70fae741a7
-  data.tar.gz: ac99f27c25e52703adea30eca1db61671a275355a4a3a00657f2f0359e7fe57f
+  metadata.gz: ed8bad2906a9ffd76c81a1fe14d89088f913f84996f0a6d98e7f104e0eccfdc9
+  data.tar.gz: 40ee48dc0f4d7e0ce1028820b9c5cf56d0c78d9853916f3bf9e5f32a7106887a
 SHA512:
-  metadata.gz: beb8fa74fab5d2799ac982a2585a4ba8ebce992ae2b9a98d9356fe9e0afdd8d2b08d371bd9a6940e7b8761710f876c5494697a47ba1b4a047d7a38babb0d1695
-  data.tar.gz: 64df1b2c919c47f8852bc2885e7dc2eac5bd55b5892f2f6477e56401a9e2d1cf01080c512d2592d07472ef8ff053ba26ac0b5c29278e9aefa57cea9f639c33ba
+  metadata.gz: 69c3c6e6b2a60c3f27284be820ac8f2687de0073c28719bd9414578ab37ac59de267dc1e69897966fef33d4ba725597587fe991b8e20ce6d40ec1587f3aa53c4
+  data.tar.gz: c296dcedcf97fe7426a59cfb2df2144685f2809c64becb18483e62a5aea536c2ca3ae2ab697b865c0945456e191a0a011f01123d34160acfa554761b8a2f917e

data/.rubocop.yml

@@ -1,6 +1,7 @@
 AllCops:
   Exclude:
     - 'exp/**/*'
+    - 'demo/**/*'
 
 Layout/ArgumentAlignment:
   Enabled: false
@@ -55,6 +56,9 @@ Layout/MultilineMethodCallBraceLayout:
 
 Layout/SpaceAroundOperators:
   Enabled: true
+  
+Layout/SpaceBeforeBrackets:
+  Enabled: true
 
 Layout/SpaceInsideParens:
   Enabled: true
@@ -70,6 +74,9 @@ Layout/TrailingEmptyLines:
 
 Layout/TrailingWhitespace:
   Enabled: true
+  
+Lint/AmbiguousAssignment:
+  Enabled: true
 
 Lint/DuplicateBranch:
   Enabled: true
@@ -163,6 +170,9 @@ Naming/BlockParameterName:
 
 Naming/MethodParameterName:
   Enabled: false
+  
+Naming/MethodName:
+  Enabled: false
 
 Naming/VariableName:
   Enabled: false
@@ -226,6 +236,9 @@ Style/GuardClause:
 
 Style/HashEachMethods:
   Enabled: true
+  
+Style/HashExcept:
+  Enabled: true
 
 Style/HashTransformKeys:
   Enabled: true

data/.yardopts

@@ -0,0 +1,6 @@
+--exclude examples --exclude features --exclude spec
+--no-private
+--markup markdown
+-
+CHANGELOG.md
+LICENSE.txt

data/CHANGELOG.md

@@ -1,3 +1,59 @@
+## [0.0.8] - 2021-01-07
+- AST node generation for arithmetic operations of number literals.
+
+## Changed
+- Class `AST::ASTBuilder` added `reduce_` methods for arithmetic operations.
+
+## Fixed
+- File `grammar.rb`: second rule for `factor` had a missing member in rhs.
+
+## [0.0.7] - 2021-01-06
+- Lox grammar reworked, initial AST classes created.
+
+## Added
+- Class `Parser` this one generates AST's (Abstract Syntax Tree)
+- Class `AST::ASTVisitor` draft initial implementation.
+- Class `AST::BinaryExpr` draft initial implementation.
+- Class `AST::LoxCompoundExpr` draft initial implementation.
+- Class `AST::LiteralExpr` draft initial implementation.
+- Class `AST::LoxNode` draft initial implementation.
+
+## Changed
+- File `spec_helper.rb`: removed Bundler dependency
+- Class `AST::ASTBuilder` added initial `reduce_` methods.
+- File `README.md` Removed example with AST generation since this is in flux.
+
+## [0.0.6] - 2021-01-03
+- First iteration of a parser with AST generation.
+
+## Added
+- Class `Parser` this one generates AST's (Abstract Syntax Tree)
+- Class `AST::ASTBuilder` default code to generate an AST.
+
+## Changed
+- File `spec_helper.rb`: removed Bundler dependency
+- File `README.md` Added example with AST visualization.
+
+## Fixed
+- File `grammar.rb` ensure that the constant `Grammar` is created once only.
+
+## [0.0.5] - 2021-01-02
+- Improved example in `README.md`, code re-styling to please `Rubocop` 1.7
+
+## Changed
+- Code re-styling to please `Rubocop` 1.7
+- File `README.md` Improved example with better parse tree visualization.
+
+## [0.0.4] - 2021-01-01
+- A first parser implementation able to parse Lox source code.
+
+## Added
+- Method `LXString::==` equality operator.
+
+## Changed
+- class `Parser` renamed to `RawParser`
+- File `README.md` Added an example showing the use of `RawParser` class.
+
 ## [0.0.3] - 2020-12-29
 - Scanner can recognize strings and nil tokens
 ### Added

data/README.md

@@ -1,7 +1,7 @@
 # Loxxy
 
-A Ruby implementation of the [Lox programming language](https://craftinginterpreters.com/the-lox-language.html),
-a simple language used in Bob Nystrom's online book [Crafting Interpreters](https://craftinginterpreters.com/).
+A Ruby implementation of the [Lox programming language](https://craftinginterpreters.com/the-lox-language.html ),
+a simple language used in Bob Nystrom's online book [Crafting Interpreters](https://craftinginterpreters.com/ ).
 
 ## Purpose of this project:
 - To deliver an open source example of a programming language fully implemented in Ruby  
@@ -9,12 +9,66 @@ a simple language used in Bob Nystrom's online book [Crafting Interpreters](http
 - The implementation should be mature enough to run (LoxLox)[https://github.com/benhoyt/loxlox],  
   a Lox interpreter written in Lox.
 
+## Current status
+The __loxxy__ gem hosts two distinct parsers classes (`RawParser` and `Parser`).
+- A `RawParser` instance is able to recognize valid Lox input and to generate
+  a concrete parse tree from it.
+- A `Parser` instance can also parse Lox source code but will generate an AST
+  (Abstract Syntax Tree) that will be used by the future tree-walking interpreter.
+  Currently it generates AST for arithmetic expressions with literal numbers only.
+
 ## Roadmap
 - [DONE] Scanner (tokenizer)
-- [STARTED] Recognizer
-- [TODO] Parser
+- [DONE] Raw parser. It parses Lox programs and generates a parse tree.
+- [DONE] Tailored parser for generating AST (Abstract Syntax Tree)
+- [STARTED] Custom AST builder class  
+- [STARTED] Hierarchy classes for representing Lox expressions in AST  
 - [TODO] Interpreter or transpiler
 
+## Example using RawParser class
+```ruby
+require 'loxxy'
+
+lox_input = <<-LOX_END
+  // Your first Lox program!
+  print "Hello, world!";
+LOX_END
+
+# Show that the raw parser accepts the above program
+base_parser = Loxxy::FrontEnd::RawParser.new
+
+# Now parse the input into a concrete parse tree...
+ptree = base_parser.parse(lox_input)
+
+# Display the parse tree thanks to Rley formatters...
+visitor = Rley::ParseTreeVisitor.new(ptree)
+tree_formatter = Rley::Formatter::Asciitree.new($stdout)
+tree_formatter.render(visitor)
+```
+
+This is the output produced by the above example:
+```
+program
++-- declaration_plus
+|   +-- declaration
+|       +-- statement
+|           +-- printStmt
+|               +-- PRINT: 'print'
+|               +-- expression
+|               |   +-- assignment
+|               |       +-- logic_or
+|               |           +-- logic_and
+|               |               +-- equality
+|               |                   +-- comparison
+|               |                       +-- term
+|               |                           +-- factor
+|               |                               +-- unary
+|               |                                   +-- call
+|               |                                       +-- primary
+|               |                                           +-- STRING: '"Hello, world!"'
+|               +-- SEMICOLON: ';'
++-- EOF: ''
+```
 
 ## Installation
 
@@ -37,9 +91,9 @@ Or install it yourself as:
 TODO: Write usage instructions here
 
 ## Other Lox implementations in Ruby
-An impressive list of Lox implementations can be found [here](https://github.com/munificent/craftinginterpreters/wiki/Lox-implementations
+An impressive list of Lox implementations can be found [here](https://github.com/munificent/craftinginterpreters/wiki/Lox-implementations)
 
-For Ruby, ther is the [lox](https://github.com/rdodson41/ruby-lox) gem.
+For Ruby, there is the [lox](https://github.com/rdodson41/ruby-lox) gem.
 There are other Ruby-based projects as well:  
 - [SlowLox](https://github.com/ArminKleinert/SlowLox), described as a "1-to-1 conversion of JLox to Ruby"
 - [rulox](https://github.com/LevitatingBusinessMan/rulox)

data/lib/loxxy.rb

@@ -1,6 +1,8 @@
 # frozen_string_literal: true
 
-require_relative './loxxy/version'
+require_relative 'loxxy/version'
+require_relative 'loxxy/front_end/raw_parser'
+require_relative 'loxxy/front_end/parser'
 
 module Loxxy
   class Error < StandardError; end

data/lib/loxxy/ast/ast_builder.rb

@@ -0,0 +1,138 @@
+# frozen_string_literal: true
+
+require_relative '../datatype/all_datatypes'
+require_relative 'lox_literal_expr'
+require_relative 'lox_binary_expr'
+
+module Loxxy
+  module Ast
+    # The purpose of ASTBuilder is to build piece by piece an AST
+    # (Abstract Syntax Tree) from a sequence of input tokens and
+    # visit events produced by walking over a GFGParsing object.
+    class ASTBuilder < Rley::ParseRep::ASTBaseBuilder
+      # Terminal2NodeClass = {
+      #   'FALSE' => Datatype::False,
+      #   'NIL' => Datatype::Nil,
+      #   'NUMBER' => Datatype::Number,
+      #   'STRING' => Datatype::LXString,
+      #   'TRUE' => Datatype::True
+      # }.freeze
+
+      attr_reader :strict
+
+      # Create a new AST builder instance.
+      # @param theTokens [Array<Rley::Lexical::Token>] The sequence of input tokens.
+      def initialize(theTokens)
+        super(theTokens)
+        @strict = false
+      end
+
+      protected
+
+      def terminal2node
+        Terminal2NodeClass
+      end
+
+      # Method override
+      def new_leaf_node(_production, _terminal, aTokenPosition, aToken)
+        Rley::PTree::TerminalNode.new(aToken, aTokenPosition)
+      end
+
+      # Factory method for creating a parent node object.
+      # @param aProduction [Production] Production rule
+      # @param aRange [Range] Range of tokens matched by the rule
+      # @param theTokens [Array] The input tokens
+      # @param theChildren [Array] Children nodes (one per rhs symbol)
+      def new_parent_node(aProduction, aRange, theTokens, theChildren)
+        mth_name = method_name(aProduction.name)
+        if respond_to?(mth_name, true)
+          node = send(mth_name, aProduction, aRange, theTokens, theChildren)
+        else
+          # Default action...
+          node = case aProduction.rhs.size
+                 when 0
+                   return_epsilon(aRange, theTokens, theChildren)
+                 when 1
+                   return_first_child(aRange, theTokens, theChildren)
+                 else
+                   if strict
+                     msg = "Don't know production '#{aProduction.name}'"
+                     raise StandardError, msg
+                   else
+                     node = Rley::PTree::NonTerminalNode.new(aProduction.lhs, aRange)
+                     theChildren&.reverse_each do |child|
+                       node.add_subnode(child) if child
+                     end
+
+                     node
+                   end
+                 end
+        end
+
+        node
+      end
+
+      # rule('term' => 'factor additive_plus')
+      def reduce_term_additive(_production, _range, tokens, theChildren)
+        operand1 = theChildren[0]
+
+        # Second child is anray with couples [operator, operand2]
+        theChildren[1].each do |(operator, operand2)|
+          operand1 = LoxBinaryExpr.new(tokens[0].position, operator, operand1, operand2)
+        end
+
+        operand1
+      end
+
+      # rule('additive_star' => 'additive_star additionOp factor').as 'additionOp_expr'
+      def reduce_additive_plus_more(_production, _range, _tokens, theChildren)
+        result = theChildren[0]
+        operator = theChildren[1].symbol.name == 'MINUS' ? :- : :+
+        operand2 = theChildren[2]
+        result << [operator, operand2]
+      end
+
+      # rule('additive_plus' => 'additionOp factor')
+      def reduce_additive_plus_end(_production, _range, _tokens, theChildren)
+        operator = theChildren[0].symbol.name == 'MINUS' ? :- : :+
+        operand2 = theChildren[1]
+        [[operator, operand2]]
+      end
+
+      # rule('factor' => 'multiplicative_plus')
+      def reduce_factor_multiplicative(_production, _range, tokens, theChildren)
+        operand1 = theChildren[0]
+
+        # Second child is anray with couples [operator, operand2]
+        theChildren[1].each do |(operator, operand2)|
+          operand1 = LoxBinaryExpr.new(tokens[0].position, operator, operand1, operand2)
+        end
+
+        operand1
+      end
+
+      # rule('multiplicative_plus' => 'multiplicative_plus multOp unary')
+      def reduce_multiplicative_plus_more(_production, _range, _tokens, theChildren)
+        result = theChildren[0]
+        operator = theChildren[1].symbol.name == 'SLASH' ? :/ : :*
+        operand2 = theChildren[2]
+        result << [operator, operand2]
+      end
+
+      # rule('multiplicative_plus' => 'multOp unary')
+      def reduce_multiplicative_plus_end(_production, _range, _tokens, theChildren)
+        operator = theChildren[0].symbol.name == 'SLASH' ? :/ : :*
+        operand2 = theChildren[1]
+        [[operator, operand2]]
+      end
+
+      # rule('primary' => 'FALSE' | TRUE').as 'literal_expr'
+      def reduce_literal_expr(_production, _range, _tokens, theChildren)
+        first_child = theChildren.first
+        pos = first_child.token.position
+        literal = first_child.token.value
+        LoxLiteralExpr.new(pos, literal)
+      end
+    end # class
+  end # module
+end # module

data/lib/loxxy/ast/ast_visitor.rb

@@ -0,0 +1,125 @@
+# frozen_string_literal: true
+
+module Loxxy
+  module Ast
+    class ASTVisitor
+      # Link to the top node to visit
+      attr_reader(:top)
+
+      # List of objects that subscribed to the visit event notification.
+      attr_reader(:subscribers)
+
+      # attr_reader(:runtime)
+
+      # Build a visitor for the given top.
+      # @param aRoot [AST::LoxNode] the parse tree to visit.
+      def initialize(aTop)
+        raise StandardError if aTop.nil?
+
+        @top = aTop
+        @subscribers = []
+      end
+
+      # Add a subscriber for the visit event notifications.
+      # @param aSubscriber [Object]
+      def subscribe(aSubscriber)
+        subscribers << aSubscriber
+      end
+
+      # Remove the given object from the subscription list.
+      # The object won't be notified of visit events.
+      # @param aSubscriber [Object]
+      def unsubscribe(aSubscriber)
+        subscribers.delete_if { |entry| entry == aSubscriber }
+      end
+
+      # The signal to begin the visit of the top.
+      def start # (aRuntime)
+        # @runtime = aRuntime
+        top.accept(self)
+      end
+
+      # Visit event. The visitor is visiting the
+      # given terminal node containing a datatype object.
+      # @param aLiteralExpr [AST::LoxLiteralExpr] the leaf node to visit.
+      def visit_literal_expr(aLiteralExpr)
+        broadcast(:before_literal_expr, aLiteralExpr)
+        broadcast(:after_literal_expr, aLiteralExpr)
+      end
+
+=begin
+
+
+      def visit_compound_datum(aCompoundDatum)
+        broadcast(:before_compound_datum, aCompoundDatum)
+        traverse_children(aCompoundDatum)
+        broadcast(:after_compound_datum, aCompoundDatum)
+      end
+
+      # Visit event. The visitor is visiting the
+      # given empty list object.
+      # @param anEmptyList [SkmEmptyList] the empty list object to visit.
+      def visit_empty_list(anEmptyList)
+        broadcast(:before_empty_list, anEmptyList)
+        broadcast(:after_empty_list, anEmptyList)
+      end
+
+      def visit_pair(aPair)
+        broadcast(:before_pair, aPair)
+        traverse_car_cdr(aPair)
+        broadcast(:after_pair, aPair)
+      end
+=end
+=begin
+   # Visit event. The visitor is about to visit the given non terminal node.
+    # @param aNonTerminalNode [NonTerminalNode] the node to visit.
+    def visit_nonterminal(aNonTerminalNode)
+      if @traversal == :post_order
+        broadcast(:before_non_terminal, aNonTerminalNode)
+        traverse_subnodes(aNonTerminalNode)
+      else
+        traverse_subnodes(aNonTerminalNode)
+        broadcast(:before_non_terminal, aNonTerminalNode)
+      end
+      broadcast(:after_non_terminal, aNonTerminalNode)
+    end
+=end
+
+      private
+
+      def traverse_children(aParent)
+        children = aParent.children
+        broadcast(:before_children, aParent, children)
+
+        # Let's proceed with the visit of children
+        children.each { |a_child| a_child.accept(self) }
+
+        broadcast(:after_children, aParent, children)
+      end
+
+      def traverse_car_cdr(aPair)
+        if aPair.car
+          broadcast(:before_car, aPair, aPair.car)
+          aPair.car.accept(self)
+          broadcast(:after_car, aPair, aPair.car)
+        end
+        if aPair.cdr
+          broadcast(:before_cdr, aPair, aPair.cdr)
+          aPair.cdr.accept(self)
+          broadcast(:after_cdr, aPair, aPair.cdr)
+        end
+      end
+
+      # Send a notification to all subscribers.
+      # @param msg [Symbol] event to notify
+      # @param args [Array] arguments of the notification.
+      def broadcast(msg, *args)
+        subscribers.each do |subscr|
+          next unless subscr.respond_to?(msg) || subscr.respond_to?(:accept_all)
+
+          subscr.send(msg, runtime, *args)
+        end
+      end
+    end # class
+  end # module
+end # module