rley 0.4.02 → 0.4.03

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: cd7c7f9e941ac7139080960dca0cb2a16665bcbd
4
- data.tar.gz: dd4e2a8dae9fd1958427beab6c63b0e9206834e7
3
+ metadata.gz: 2d13db61565b20ef542592a59bcd056d4dd2192e
4
+ data.tar.gz: a8d8c12a600d9621d6be682570cb165d526641ef
5
5
  SHA512:
6
- metadata.gz: 7611109ca8b7d99a090e88de7eea31cc08c22250d790c2befd3b3aaa9e222450ac900d10ab733d0ee72e831da0bf7a7ad07da6f6e33e20724879659a2e45d620
7
- data.tar.gz: f81cc80909844e4cbd50ec85ad803b08483e714f78734b234de13b544f891f77759cdb8df4349dea1c68a8de250d44ef860e0a8bf27dd6313da08f0cc3c49b34
6
+ metadata.gz: 679e3f7d6747264092f317521e1f8f9e241ad912de71280c0c6d68454110bd3ba75fdc6f1f4a9c76ac61622ed87fba86f2662e3f15b5cd9a0e2d571b340e2988
7
+ data.tar.gz: edb83ffb5fb673f27abef6c7ef10014fb8c6c312de093251f6ce509e2893c20cf5114cb77abf1b97c1406d9d22d474b0ab577829acb9d59f58117e43e96eaef2
data/CHANGELOG.md CHANGED
@@ -1,6 +1,8 @@
1
- ### 0.4.02 / 2017-04-09
2
- * [NEW] Module re-organization for clearer dependencies: Classes `Token` and `TokenRange` are moved to a separate module `Tokens`.
3
- * [CHANGE] Code, specs, examples and `README.md` adapted to reflect the module re-organization.
1
+ ### 0.4.03 / 2017-04-17
2
+ * [FIX] File `rley.rb` reference to obsolete `EarleyParser` class removed.
3
+ * [NEW] `BracketNotation` formatter class. Allows parse tree output in Labelled Bracket Notation.
4
+ * [CHANGE] Code refactoring in directory `examples\data_formats\JSON`. The demo command-line tool parses JSON and converts it into LBN (Labelled Bracket Notation). There are two diagrams (in SVG format) generated from the LBN output.
5
+
4
6
 
5
7
  ### 0.4.01 / 2016-12-21
6
8
  * [NEW] File `appveyor.yml`. Add AppVeyor CI to Github commits. AppVeyor complements Travis by running builds under Windows OS.
@@ -48,7 +50,7 @@
48
50
  ### 0.3.06 / 2016-11-06
49
51
  * [FIX] There were missing links to shared parse forest nodes for ambiguous parses.
50
52
  * [NEW] RSpec file `ambiguous_parse_spec.rb` added in order to test the parse forest building for an ambiguous parse.
51
- * [CHANGE] Attribute `ParseWalkerContext#nterm2start`: previous implementation assumed -wrongly- that for each non terminal there was only one start entry.
53
+ * [CHANGE] Attribute `ParseWalkerContext#nterm2start`: previous implementation assumed -wrongly- that for each non terminal there was only one start entry.
52
54
  Now this attribute uses nested hashes as data structure in order to disambiguate the mapping.
53
55
  * [CHANGE] Method `ParseWalkerFactory#visit_entry` updated to reflect change in the `ParseWalkerContext#nterm2start` attribute.
54
56
  * [CHANGE] Method `ParseWalkerFactory#visit_entry` now emits an event if an item entry is re-visited (previously, no such event were generated)
@@ -73,10 +75,10 @@
73
75
  ### 0.3.01 / 2016-10-23
74
76
  * [CHANGE] Method `ParseWalkerFactory#build_walker`. Signature change in order prevent direct dependency on `GFGParsing` class.
75
77
  * [CHANGE] Class `ParseForestBuilder`. Removal of `parsing` attribute, no direct dependency on `GFGParsing` class.
76
- * [CHANGE] Internal changed to `ParseForestFactory` class.
78
+ * [CHANGE] Internal changed to `ParseForestFactory` class.
77
79
 
78
80
  ### 0.3.00 / 2016-10-23
79
- * [CHANGE] Many new classes. The gem bundles a second parser that copes with ambiguous grammars.
81
+ * [CHANGE] Many new classes. The gem bundles a second parser that copes with ambiguous grammars.
80
82
 
81
83
 
82
84
  ### 0.2.15 / 2016-02-14
@@ -90,11 +92,11 @@
90
92
  * [CHANGED] method `Parsing#success?`. New implementation that relies on start symbol derivation.
91
93
  * [NEW] New method `Chart#start_symbol` added. Returns the start symbol of the grammar.
92
94
  * [NEW] New method `StateSet#ambiguities` added. Returns the parse sets that are ambiguous (= distinct derivation for same input tokens).
93
- * [FIX] In special cases the parsing didn't work correctly when there more than one
95
+ * [FIX] In special cases the parsing didn't work correctly when there more than one
94
96
  production rule for the start symbol of a grammar.
95
97
 
96
98
  ### 0.2.12 / 2015-11-20
97
- * [FIX] In special cases the parsing didn't work correctly when there more than one
99
+ * [FIX] In special cases the parsing didn't work correctly when there more than one
98
100
  production rule for the start symbol of a grammar.
99
101
 
100
102
  ### 0.2.11 / 2015-09-05
@@ -150,7 +152,7 @@ Version number bump: major re-design of the parse tree generation.
150
152
  * [FIX] Method `Parsing#parse_tree` now handles situations where there are multiple complete parse states for a non-terminal.
151
153
 
152
154
  ### 0.1.12 / 2014-12-22
153
- * [FIX] Fixed `Parsing#parse_tree`: code couldn't cope with parse state set containing more
155
+ * [FIX] Fixed `Parsing#parse_tree`: code couldn't cope with parse state set containing more
154
156
  than one parse state that expected the same symbol.
155
157
  * [NEW] Added one more parser example (for very basic arithmetic expression)
156
158
 
@@ -239,19 +241,19 @@ in method argument names between source code and documentation.
239
241
 
240
242
 
241
243
  ### 0.0.11 / 2014-11-16
242
- * [CHANGE] Usage of `GrammarBuilder`simplified: the call to method `GrammarBuilder#add_non_terminal` isn't necessary. Method is removed
244
+ * [CHANGE] Usage of `GrammarBuilder`simplified: the call to method `GrammarBuilder#add_non_terminal` isn't necessary. Method is removed
243
245
  * [CHANGE] Updated the `examples` folder accordingly.
244
246
 
245
247
  ### 0.0.10 / 2014-11-15
246
- * [NEW] New folder `examples` added with two examples of grammar creation
248
+ * [NEW] New folder `examples` added with two examples of grammar creation
247
249
 
248
250
  ### 0.0.09 / 2014-11-15
249
- * [NEW] New class `GrammarBuilder` added and tested, its purpose is
251
+ * [NEW] New class `GrammarBuilder` added and tested, its purpose is
250
252
  to simplify the construction of grammars.
251
253
 
252
254
  ### 0.0.08 / 2014-11-14
253
255
  * [CHANGE] `EarleyParser#parse` method: Initial API documentation.
254
- * [INFO] This version was committed to force Travis CI to execute a complete build
256
+ * [INFO] This version was committed to force Travis CI to execute a complete build
255
257
  failed because Travis couldn't connect to GitHub)
256
258
 
257
259
  ### 0.0.07 / 2014-11-14
@@ -280,4 +282,4 @@ failed because Travis couldn't connect to GitHub)
280
282
 
281
283
 
282
284
  ### 0.0.00 / 2014-11-07
283
- * [FEATURE] Initial public working version
285
+ * [FEATURE] Initial public working version
data/README.md CHANGED
@@ -1,4 +1,4 @@
1
- #[Rley](https://github.com/famished-tiger/Rley)
1
+ [Rley](https://github.com/famished-tiger/Rley)
2
2
 
3
3
  [![Linux Build Status](https://img.shields.io/travis/famished-tiger/Rley/master.svg?label=Linux%20build)](https://travis-ci.org/famished-tiger/Rley)
4
4
  [![Build status](https://ci.appveyor.com/api/projects/status/l5adgcbfo128rvo9?svg=true)](https://ci.appveyor.com/project/famished-tiger/rley)
@@ -28,7 +28,7 @@ encode efficiently all the possible parse trees that result from an ambiguous
28
28
  grammar.
29
29
 
30
30
  As another distinctive mark, __Rley__ is also the first Ruby implementation of a
31
- parsing library based on the new _Grammar Flow Graph_ approach (_TODO: add details_).
31
+ parsing library based on the new _Grammar Flow Graph_ approach [References on GFG](#references-on-gfg).
32
32
 
33
33
  ### What it can do?
34
34
  Maybe parsing algorithms and internal implementation details are of lesser
@@ -217,7 +217,7 @@ ask it to generate a message.
217
217
  puts result.failure_reason.message unless result.success?
218
218
  ```
219
219
 
220
- Re-running the example with the error, result in the error message:
220
+ Re-running the example with the error, results in the error message:
221
221
  ```
222
222
  Syntax error at or near token 2 >>>Mary<<<
223
223
  Expected one 'Verb', found a 'Proper-Noun' instead.
@@ -281,6 +281,15 @@ Here are a few other ones:
281
281
  ## Thanks to:
282
282
  * Professor Keshav Pingali, one of the creators of the Grammar Flow Graph parsing approach for his encouraging e-mail exchanges.
283
283
 
284
+ ## References on GFG
285
+ Since the __G__rammar __F__low __G__raph parsing approach is quite new, it has not yet taken a place in
286
+ standard parser textbooks. Here are a few references (and links) of papers on GFG:
287
+ - K. Pingali, G. Bilardi. [Parsing with Pictures](http://apps.cs.utexas.edu/tech_reports/reports/tr/TR-2102.pdf)
288
+ - K. Pingali, G. Bilardi. [A Graphical Model for Context-Free Grammar Parsing.](https://link.springer.com/chapter/10.1007/978-3-662-46663-6_1)
289
+ In : International Conference on Compiler Construction. Springer Berlin Heidelberg, 2015. p. 3-27.
290
+ - M. Fulbright. [An Evaluation of Two Approaches to Parsing](http://apps.cs.utexas.edu/tech_reports/reports/tr/TR-2199.pdf)
291
+
292
+
284
293
  Copyright
285
294
  ---------
286
295
  Copyright (c) 2014-2017, Dimitri Geshef.
@@ -6,7 +6,12 @@ parser = JSONParser.new
6
6
  # Parse the input file with name given in command-line
7
7
  if ARGV.empty?
8
8
  msg = <<-END_MSG
9
- Command-line symtax:
9
+ A demo utility that converts a JSON file into labelled square notation (LBN).
10
+ Use online tools (e.g. http://yohasebe.com/rsyntaxtree/) to visualize
11
+ parse trees from LBN output.
12
+
13
+ Command-line syntax:
14
+
10
15
  ruby #{__FILE__} filename
11
16
  where:
12
17
  filename is the name of a JSON file
@@ -27,7 +32,17 @@ unless result.success?
27
32
  exit(1)
28
33
  end
29
34
 
30
- # Generate a parse forest from the parse result
31
- pforest = result.parse_forest
32
- # puts pforest.ambiguous?
35
+ # Generate a parse tree from the parse result
36
+ ptree = result.parse_tree
37
+ require 'yaml'
38
+ File.open('json1.yml', 'w') {|f| YAML.dump(ptree, f)}
39
+
40
+ # Let's create a parse tree visitor
41
+ visitor = Rley::ParseTreeVisitor.new(ptree)
42
+
43
+ # Output the labelled bracket notation of the tree
44
+ use_notation = Rley::Formatter::BracketNotation.new($stdout)
45
+ use_notation.render(visitor)
46
+
47
+
33
48
  # End of file
@@ -4,27 +4,33 @@ require 'rley' # Load the gem
4
4
 
5
5
  ########################################
6
6
  # Define a grammar for JSON
7
+ # Original JSON grammar is available http://www.json.org/fatfree.html
8
+ # Official JSON grammar: http://rfc7159.net/rfc7159#rfc.section.2
9
+ # Names of grammar elements are based on the RFC 7159 documentation
7
10
  builder = Rley::Syntax::GrammarBuilder.new do
8
- add_terminals('KEYWORD') # For true, false, null keywords
9
- add_terminals('JSON_STRING', 'JSON_NUMBER')
10
- add_terminals('LACCOL', 'RACCOL') # For '{', '}' delimiters
11
- add_terminals('LBRACKET', 'RBRACKET') # For '[', ']' delimiters
12
- add_terminals('COLON', 'COMMA') # For ':', ',' separators
13
- rule 'json_text' => 'json_value'
14
- rule 'json_value' => 'json_object'
15
- rule 'json_value' => 'json_array'
16
- rule 'json_value' => 'JSON_STRING'
17
- rule 'json_value' => 'JSON_NUMBER'
18
- rule 'json_value' => 'KEYWORD'
19
- rule 'json_object' => %w(LACCOL json_pairs RACCOL)
20
- rule 'json_object' => %w(LACCOL RACCOL)
21
- rule 'json_pairs' => %w(json_pairs COMMA single_pair)
22
- rule 'json_pairs' => 'single_pair'
23
- rule 'single_pair' => %w(JSON_STRING COLON json_value)
24
- rule 'json_array' => %w(LBRACKET array_items RBRACKET)
25
- rule 'json_array' => %w(LBRACKET RBRACKET)
26
- rule 'array_items' => %w(array_items COMMA json_value)
27
- rule 'array_items' => %w(json_value)
11
+ add_terminals('false', 'null', 'true') # Literal names
12
+ add_terminals('string', 'number')
13
+ add_terminals('begin-object', 'end-object') # For '{', '}' delimiters
14
+ add_terminals('begin-array', 'end-array') # For '[', ']' delimiters
15
+ add_terminals('name-separator', 'value-separator') # For ':', ',' separators
16
+ rule 'JSON-text' => 'value'
17
+ rule 'value' => 'false'
18
+ rule 'value' => 'null'
19
+ rule 'value' => 'true'
20
+ rule 'value' => 'object'
21
+ rule 'value' => 'array'
22
+ rule 'value' => 'number'
23
+ rule 'value' => 'string'
24
+ rule 'object' => %w(begin-object member-list end-object)
25
+ rule 'object' => %w(begin-object end-object)
26
+ # Next rule is an example of a left recursive rule
27
+ rule 'member-list' => %w(member-list value-separator member)
28
+ rule 'member-list' => 'member'
29
+ rule 'member' => %w(string name-separator value)
30
+ rule 'array' => %w(begin-array array_items end-array)
31
+ rule 'array' => %w(begin-array end-array)
32
+ rule 'array_items' => %w(array_items value-separator value)
33
+ rule 'array_items' => %w(value)
28
34
  end
29
35
 
30
36
  # And now build the grammar...
@@ -11,12 +11,12 @@ class JSONLexer
11
11
  attr_reader(:name2symbol)
12
12
 
13
13
  @@lexeme2name = {
14
- '{' => 'LACCOL',
15
- '}' => 'RACCOL',
16
- '[' => 'LBRACKET',
17
- ']' => 'RBRACKET',
18
- ',' => 'COMMA',
19
- ':' => 'COLON'
14
+ '{' => 'begin-object',
15
+ '}' => 'end-object',
16
+ '[' => 'begin-array',
17
+ ']' => 'end-array',
18
+ ',' => 'value-separator',
19
+ ':' => 'name-separator'
20
20
  }
21
21
 
22
22
  class ScanError < StandardError ; end
@@ -53,14 +53,6 @@ private
53
53
  token_type = name2symbol[type_name]
54
54
  token = Rley::Tokens::Token.new(curr_ch, token_type)
55
55
 
56
- # LITERALS
57
- when '"' # Start string delimiter found
58
- value = scanner.scan(/([^"\\]|\\.)*/)
59
- end_delimiter = scanner.getch()
60
- raise ScanError.new('No closing quotes (") found') if end_delimiter.nil?
61
- token_type = name2symbol['JSON_STRING']
62
- token = Rley::Tokens::Token.new(value, token_type)
63
-
64
56
  when /[ftn]/ # First letter of keywords
65
57
  @scanner.pos = scanner.pos - 1 # Simulate putback
66
58
  keyw = scanner.scan(/false|true|null/)
@@ -68,18 +60,24 @@ private
68
60
  invalid_keyw = scanner.scan(/\w+/)
69
61
  raise ScanError.new("Invalid keyword: #{invalid_keyw}")
70
62
  else
71
- token_type = name2symbol['KEYWORD']
63
+ token_type = name2symbol[keyw]
72
64
  token = Rley::Tokens::Token.new(keyw, token_type)
73
65
  end
74
66
 
67
+ # LITERALS
68
+ when '"' # Start string delimiter found
69
+ value = scanner.scan(/([^"\\]|\\.)*/)
70
+ end_delimiter = scanner.getch()
71
+ raise ScanError.new('No closing quotes (") found') if end_delimiter.nil?
72
+ token_type = name2symbol['string']
73
+ token = Rley::Tokens::Token.new(value, token_type)
75
74
 
76
75
  when /[-0-9]/ # Start character of number literal found
77
76
  @scanner.pos = scanner.pos - 1 # Simulate putback
78
77
  value = scanner.scan(/-?[0-9]+(\.[0-9]+)?([eE][-+]?[0-9])?/)
79
- token_type = name2symbol['JSON_NUMBER']
78
+ token_type = name2symbol['number']
80
79
  token = Rley::Tokens::Token.new(value, token_type)
81
80
 
82
-
83
81
  else # Unknown token
84
82
  erroneous = curr_ch.nil? ? '' : curr_ch
85
83
  sequel = scanner.scan(/.{1,20}/)
data/lib/rley.rb CHANGED
@@ -5,10 +5,11 @@
5
5
  require_relative './rley/constants'
6
6
  require_relative './rley/syntax/grammar_builder'
7
7
  require_relative './rley/tokens/token'
8
- require_relative './rley/parser/earley_parser'
9
8
  require_relative './rley/parser/gfg_earley_parser'
10
9
  require_relative './rley/parse_tree_visitor'
11
10
  require_relative './rley/formatter/debug'
12
11
  require_relative './rley/formatter/json'
12
+ require_relative './rley/formatter/debug'
13
+ require_relative './rley/formatter/bracket_notation'
13
14
 
14
15
  # End of file
@@ -3,7 +3,7 @@
3
3
 
4
4
  module Rley # Module used as a namespace
5
5
  # The version number of the gem.
6
- Version = '0.4.02'.freeze
6
+ Version = '0.4.03'.freeze
7
7
 
8
8
  # Brief description of the gem.
9
9
  Description = "Ruby implementation of the Earley's parsing algorithm".freeze
@@ -0,0 +1,68 @@
1
+ require_relative 'base_formatter'
2
+
3
+
4
+ module Rley # This module is used as a namespace
5
+ # Namespace dedicated to parse tree formatters.
6
+ module Formatter
7
+ # A formatter class that generates the labelled bracket notation (LBN)
8
+ # representation of a parse tree.
9
+ # The output can be then fed to an application or library that is
10
+ # capable of displaying parse tree diagrams.
11
+ # For Ruby developers, there is RSyntaxTree by Yoichiro Hasebe.
12
+ # (accessible via: http://yohasebe.com/rsyntaxtree/)
13
+ class BracketNotation < BaseFormatter
14
+
15
+ # Constructor.
16
+ # @param anIO [IO] The output stream to which the rendered grammar
17
+ # is written.
18
+ def initialize(anIO)
19
+ super(anIO)
20
+ end
21
+
22
+
23
+ # Method called by a ParseTreeVisitor to which the formatter subscribed.
24
+ # Notification of a visit event: the visitor is about to visit
25
+ # a non-terminal node
26
+ # @param nonterm [NonTerminalNode]
27
+ def before_non_terminal(aNonTerm)
28
+ write("[#{aNonTerm.symbol.name} ")
29
+ end
30
+
31
+
32
+ # Method called by a ParseTreeVisitor to which the formatter subscribed.
33
+ # Notification of a visit event: the visitor is about to visit
34
+ # a terminal node
35
+ # @param _term [TerminalNode]
36
+ def before_terminal(aTerm)
37
+ write("[#{aTerm.symbol.name} ")
38
+ end
39
+
40
+ # Method called by a ParseTreeVisitor to which the formatter subscribed.
41
+ # Notification of a visit event: the visitor completed the visit of
42
+ # a terminal node.
43
+ # @param _term [TerminalNode]
44
+ def after_terminal(aTerm)
45
+ # Escape all opening and closing square brackets
46
+ escape_lbrackets = aTerm.token.lexeme.gsub(/\[/, "\\[")
47
+ escaped = escape_lbrackets.gsub(/\]/, "\\]")
48
+ write(escaped + ']')
49
+ end
50
+
51
+ # Method called by a ParseTreeVisitor to which the formatter subscribed.
52
+ # Notification of a visit event: the visitor completed the visit of
53
+ # a non-terminal node
54
+ # @param _nonterm [NonTerminalNode]
55
+ def after_non_terminal(_nonterm)
56
+ write(']')
57
+ end
58
+
59
+ private
60
+
61
+ def write(aText)
62
+ output.write(aText)
63
+ end
64
+ end # class
65
+ end # module
66
+ end # module
67
+
68
+ # End of file
@@ -7,7 +7,7 @@ module Rley # This module is used as a namespace
7
7
 
8
8
  # List of objects that subscribed to the visit event notification.
9
9
  attr_reader(:subscribers)
10
-
10
+
11
11
  # Indicates the kind of tree traversal to perform: :post_order, :pre-order
12
12
  attr_reader(:traversal)
13
13
 
@@ -52,7 +52,7 @@ module Rley # This module is used as a namespace
52
52
  broadcast(:before_non_terminal, aNonTerminalNode)
53
53
  traverse_subnodes(aNonTerminalNode)
54
54
  else
55
- traverse_subnodes(aNonTerminalNode)
55
+ traverse_subnodes(aNonTerminalNode)
56
56
  broadcast(:before_non_terminal, aNonTerminalNode)
57
57
  end
58
58
  broadcast(:after_non_terminal, aNonTerminalNode)
@@ -67,7 +67,7 @@ module Rley # This module is used as a namespace
67
67
  end
68
68
 
69
69
 
70
- # Visit event. The visitor has completed its visit of the given
70
+ # Visit event. The visitor has completed its visit of the given
71
71
  # non-terminal node.
72
72
  # @param aNonTerminalNode [NonTerminalNode] the node to visit.
73
73
  def end_visit_nonterminal(aNonTerminalNode)
@@ -81,17 +81,17 @@ module Rley # This module is used as a namespace
81
81
  end
82
82
 
83
83
  private
84
-
85
- # Visit event. The visitor is about to visit the subnodes of a non
84
+
85
+ # Visit event. The visitor is about to visit the subnodes of a non
86
86
  # terminal node.
87
87
  # @param aParentNode [NonTeminalNode] the (non-terminal) parent node.
88
88
  def traverse_subnodes(aParentNode)
89
89
  subnodes = aParentNode.subnodes
90
90
  broadcast(:before_subnodes, aParentNode, subnodes)
91
-
91
+
92
92
  # Let's proceed with the visit of subnodes
93
93
  subnodes.each { |a_node| a_node.accept(self) }
94
-
94
+
95
95
  broadcast(:after_subnodes, aParentNode, subnodes)
96
96
  end
97
97
 
@@ -98,11 +98,6 @@ module Rley # This module is used as a namespace
98
98
  return result
99
99
  end
100
100
 
101
- # An item with the dot in front of a terminal is called a shift item
102
- def shift_item?()
103
- return position.zero?
104
- end
105
-
106
101
  # Return true if this dotted item has a dot one place
107
102
  # to the right compared to the dotted item argument.
108
103
  def successor_of?(another)
@@ -114,11 +114,6 @@ module Rley # This module is used as a namespace
114
114
  create_token_node(anEntry, anIndex)
115
115
 
116
116
  when NilClass # Dot at the beginning of production
117
- if anEntry.vertex.dotted_item.production.empty?
118
- # Empty rhs => create an epsilon node ...
119
- # ... without changing current path
120
- create_epsilon_node(anEntry, anIndex)
121
- end
122
117
  curr_path.pop if curr_parent.kind_of?(SPPF::AlternativeNode)
123
118
  end
124
119
 
@@ -195,15 +190,6 @@ module Rley # This module is used as a namespace
195
190
  return candidate
196
191
  end
197
192
 
198
-
199
- def create_epsilon_node(anEntry, anIndex)
200
- new_node = PTree::EpsilonNode.new(anIndex)
201
- candidate = add_node_to_tree(new_node)
202
- entry2node[anEntry] = candidate
203
-
204
- return candidate
205
- end
206
-
207
193
  # Add the given node if not yet present in parse tree
208
194
  def add_node_to_tree(aNode)
209
195
  new_node = aNode
@@ -14,6 +14,7 @@ module Rley # This module is used as a namespace
14
14
  @range = Tokens::TokenRange.new(aRange)
15
15
  end
16
16
 
17
+ # Assign a value from given range to each undefined range bound
17
18
  def range=(aRange)
18
19
  range.assign(aRange)
19
20
  end
@@ -0,0 +1,96 @@
1
+ require_relative '../../spec_helper'
2
+ require 'stringio'
3
+
4
+ require_relative '../support/grammar_abc_helper'
5
+ require_relative '../../../lib/rley/tokens/token'
6
+ require_relative '../../../lib/rley/parser/gfg_earley_parser'
7
+ require_relative '../../../lib/rley/ptree/parse_tree'
8
+ require_relative '../../../lib/rley/parse_tree_visitor'
9
+ # Load the class under test
10
+ require_relative '../../../lib/rley/formatter/bracket_notation'
11
+
12
+ module Rley # Re-open the module to get rid of qualified names
13
+ module Formatter
14
+ describe BracketNotation do
15
+ include GrammarABCHelper # Mix-in module for grammar abc
16
+
17
+ # Factory method. Build a production with the given sequence
18
+ # of symbols as its rhs.
19
+ let(:grammar_abc) do
20
+ builder = grammar_abc_builder
21
+ builder.grammar
22
+ end
23
+
24
+ # Variables for the terminal symbols
25
+ let(:a_) { grammar_abc.name2symbol['a'] }
26
+ let(:b_) { grammar_abc.name2symbol['b'] }
27
+ let(:c_) { grammar_abc.name2symbol['c'] }
28
+
29
+ # Helper method that mimicks the output of a tokenizer
30
+ # for the language specified by gramma_abc
31
+ let(:grm_abc_tokens1) do
32
+ [
33
+ Tokens::Token.new('a', a_),
34
+ Tokens::Token.new('a', a_),
35
+ Tokens::Token.new('b', b_),
36
+ Tokens::Token.new('c', c_),
37
+ Tokens::Token.new('c', c_)
38
+ ]
39
+ end
40
+
41
+ # Factory method that builds a sample parse tree.
42
+ # Generated tree has the following structure:
43
+ # S[0,5]
44
+ # +- A[0,5]
45
+ # +- a[0,0]
46
+ # +- A[1,4]
47
+ # | +- a[1,1]
48
+ # | +- A[2,3]
49
+ # | | +- b[2,3]
50
+ # | +- c[3,4]
51
+ # +- c[4,5]
52
+ # Capital letters represent non-terminal nodes
53
+ let(:grm_abc_ptree1) do
54
+ parser = Parser::GFGEarleyParser.new(grammar_abc)
55
+ parse_result = parser.parse(grm_abc_tokens1)
56
+ parse_result.parse_tree
57
+ end
58
+
59
+ let(:destination) { StringIO.new('', 'w') }
60
+ subject { BracketNotation.new(destination) }
61
+
62
+ context 'Standard creation & initialization:' do
63
+ it 'should be initialized with an IO argument' do
64
+ expect { BracketNotation.new(StringIO.new('', 'w')) }.not_to raise_error
65
+ end
66
+
67
+ it 'should know its output destination' do
68
+ expect(subject.output).to eq(destination)
69
+ end
70
+ end # context
71
+
72
+
73
+ context 'Formatting events:' do
74
+ it 'should support visit events of a parse tree' do
75
+ visitor = Rley::ParseTreeVisitor.new(grm_abc_ptree1)
76
+ subject.render(visitor)
77
+ expectations = '[S [A [a a][A [a a][A [b b]][c c]][c c]]]'
78
+ expect(destination.string).to eq(expectations)
79
+ end
80
+
81
+ it 'should escape square brackets' do
82
+ f_node = double('fake-node')
83
+ f_token = double('fake-token')
84
+ expect(f_node).to receive(:token).and_return(f_token)
85
+ expect(f_token).to receive(:lexeme).and_return('[][]')
86
+
87
+ subject.after_terminal(f_node)
88
+ expectations = '\[\]\[\]]'
89
+ expect(destination.string).to eq(expectations)
90
+ end
91
+ end # context
92
+ end # describe
93
+ end # module
94
+ end # module
95
+
96
+ # End of file
@@ -110,6 +110,12 @@ module Rley # Open this namespace to avoid module qualifier prefixes
110
110
  shortcut = ShortcutEdge.new(subject, next_vertex)
111
111
  expect(subject.shortcut).to eq(shortcut)
112
112
  end
113
+
114
+ it 'should reject an invalid shortcut edge' do
115
+ err = StandardError
116
+ err_msg = 'Invalid shortcut argument'
117
+ expect { subject.shortcut = 'invalid'}.to raise_error(err, err_msg)
118
+ end
113
119
  end # context
114
120
  end # describe
115
121
  end # module
@@ -212,6 +212,64 @@ module Rley # Open this namespace to avoid module qualifier prefixes
212
212
  # Here we go...
213
213
  subject.start
214
214
  end
215
+
216
+ it 'should also visit in pre-order' do
217
+ # Reminder: parse tree structure is
218
+ # S[0,5]
219
+ # +- A[0,5]
220
+ # +- a[0,0]
221
+ # +- A[1,4]
222
+ # | +- a[1,1]
223
+ # | +- A[2,3]
224
+ # | | +- b[2,3]
225
+ # | +- c[3,4]
226
+ # +- c[4,5]
227
+ root = grm_abc_ptree1.root
228
+ # Here we defeat encapsulation for the good cause
229
+ subject.instance_variable_set(:@traversal, :pre_order)
230
+
231
+ children = root.subnodes
232
+ big_a_1 = children[0]
233
+ big_a_1_children = big_a_1.subnodes
234
+ big_a_2 = big_a_1_children[1]
235
+ big_a_2_children = big_a_2.subnodes
236
+ big_a_3 = big_a_2_children[1]
237
+ big_a_3_children = big_a_3.subnodes
238
+ expectations = [
239
+ [:before_ptree, [grm_abc_ptree1]],
240
+ # TODO: fix this test
241
+ #[:before_subnodes, [root, children]],
242
+ #[:before_non_terminal, [root]],
243
+
244
+ # [:before_non_terminal, [big_a_1]],
245
+ # [:before_subnodes, [big_a_1, big_a_1_children]],
246
+ # [:before_terminal, [big_a_1_children[0]]],
247
+ # [:after_terminal, [big_a_1_children[0]]],
248
+ # [:before_non_terminal, [big_a_2]],
249
+ # [:before_subnodes, [big_a_2, big_a_2_children]],
250
+ # [:before_terminal, [big_a_2_children[0]]],
251
+ # [:after_terminal, [big_a_2_children[0]]],
252
+ # [:before_non_terminal, [big_a_3]],
253
+ # [:before_subnodes, [big_a_3, big_a_3_children]],
254
+ # [:before_terminal, [big_a_3_children[0]]],
255
+ # [:after_terminal, [big_a_3_children[0]]],
256
+ # [:after_subnodes, [big_a_3, big_a_3_children]],
257
+ # [:before_terminal, [big_a_2_children[2]]],
258
+ # [:after_terminal, [big_a_2_children[2]]],
259
+ # [:after_subnodes, [big_a_2, big_a_2_children]],
260
+ # [:before_terminal, [big_a_1_children[2]]],
261
+ # [:after_terminal, [big_a_1_children[2]]],
262
+ # [:after_subnodes, [big_a_1, big_a_1_children]],
263
+ # [:after_subnodes, [root, children]],
264
+ # [:after_ptree, [grm_abc_ptree1]]
265
+ ]
266
+ expectations.each do |(msg, args)|
267
+ expect(listener1).to receive(msg).with(*args).ordered
268
+ end
269
+
270
+ # Here we go...
271
+ subject.start
272
+ end
215
273
  end # context
216
274
  end # describe
217
275
  end # module
@@ -31,6 +31,19 @@ module Rley # Open this namespace to avoid module qualifier prefixes
31
31
  subject.push_state(state2)
32
32
  expect(subject.states).to eq([state1, state2])
33
33
  end
34
+
35
+ it 'should ignore a second push of a state' do
36
+ expect(subject.states).to be_empty
37
+ subject.push_state(state1)
38
+ subject.push_state(state2)
39
+ expect(subject.states).to eq([state1, state2])
40
+
41
+ # One tries to push an already pushed state
42
+ expect(subject.push_state(state1)).to be_falsy
43
+
44
+ # ...It is not added
45
+ expect(subject.states).to eq([state1, state2])
46
+ end
34
47
 
35
48
  it 'should list the states expecting a given terminal' do
36
49
  # Case of no state
@@ -25,6 +25,17 @@ module Rley # Open this namespace to avoid module qualifier prefixes
25
25
  expect(subject.range).to eq(sample_range)
26
26
  end
27
27
  end # context
28
+
29
+ context 'Initialization:' do
30
+ it 'should assign undefined range bounds' do
31
+ partial_range = { low: 0 } # High bound left undefined
32
+ instance = ParseTreeNode.new(sample_symbol, partial_range)
33
+
34
+ another = { low: 1, high: 4 } # High bound is specified
35
+ instance.range = another
36
+ expect(instance.range).to eq({low: 0, high: 4})
37
+ end
38
+ end # context
28
39
  end # describe
29
40
  end # module
30
41
  end # module
@@ -55,6 +55,10 @@ module Rley # Open this namespace to avoid module qualifier prefixes
55
55
  it 'should have a string representation' do
56
56
  expect(subject.to_string(0)).to eq('VP[0, 3]')
57
57
  end
58
+
59
+ it 'should return a key value of itself' do
60
+ expect(subject.key).to eq('VP[0, 3]')
61
+ end
58
62
  end # context
59
63
  end # describe
60
64
  end # module
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rley
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.02
4
+ version: 0.4.03
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dimitri Geshef
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2017-04-09 00:00:00.000000000 Z
11
+ date: 2017-04-17 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -142,6 +142,7 @@ files:
142
142
  - lib/rley.rb
143
143
  - lib/rley/constants.rb
144
144
  - lib/rley/formatter/base_formatter.rb
145
+ - lib/rley/formatter/bracket_notation.rb
145
146
  - lib/rley/formatter/debug.rb
146
147
  - lib/rley/formatter/json.rb
147
148
  - lib/rley/gfg/call_edge.rb
@@ -201,6 +202,7 @@ files:
201
202
  - lib/rley/syntax/verbatim_symbol.rb
202
203
  - lib/rley/tokens/token.rb
203
204
  - lib/rley/tokens/token_range.rb
205
+ - spec/rley/formatter/bracket_notation_spec.rb
204
206
  - spec/rley/formatter/debug_spec.rb
205
207
  - spec/rley/formatter/json_spec.rb
206
208
  - spec/rley/gfg/call_edge_spec.rb
@@ -291,6 +293,7 @@ signing_key:
291
293
  specification_version: 4
292
294
  summary: Ruby implementation of the Earley's parsing algorithm
293
295
  test_files:
296
+ - spec/rley/formatter/bracket_notation_spec.rb
294
297
  - spec/rley/formatter/debug_spec.rb
295
298
  - spec/rley/formatter/json_spec.rb
296
299
  - spec/rley/gfg/call_edge_spec.rb