rley 0.8.00 → 0.8.05

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ee99e1b9803a447d18a3f82e91353659e211109a5fb99d48a834ddebf572998f
-  data.tar.gz: f4719d9fa301a0de2be143ca0aa5d0c261cf1562194a40b038c79b1faa1c04dc
+  metadata.gz: a94c23265bf2996856da4d4b1fc6f2749873626951d30bacec9f1ca7619d4d4f
+  data.tar.gz: fc6a6ae19388b532dc711915a9c33ffc455a602ea9e6a1ae13e53522bc0616d0
 SHA512:
-  metadata.gz: 9810a2d3884f86d2d437e1ba0911681f53895ff60ef500e4d19bd5d9429ff433f46c1dd57d6e185eb0bce3fcf869b1e4884acd09aaff72aee0323d00a781e763
-  data.tar.gz: e7d49393824bbbeaff70bc7d57137d1fe82f86a98bfda52fc9144dbafa3e5568e51ce7e9da8c15fc2f51438b29100801e8ea145f613165ba4369093f07bfad23
+  metadata.gz: 9b25f6a34e4e5603d4b88f5cf3220e998d9e0c38ffef204ebbf4a8eacaf57853c2bb9bc2bb5a3765440aaed41ee5a4883edadc533c6df0dc907b58457b085469
+  data.tar.gz: 57c3021f24eb5764fef2be6bdd58a9cda1c9b9343737c357d6c1e1862beceaf041ed9eee9fb625cf9acc9492bac0fcb6a6ac8cd9dc464dcf2096fccd5b51e6e9

data/.rubocop.yml

@@ -1,7 +1,6 @@
 AllCops:
   Exclude:
     - 'exp/**/*'
-    - 'demo/**/*'
 
 Gemspec/DateAssignment:
   Enabled: true
@@ -50,6 +49,9 @@ Layout/IndentationConsistency:
 Layout/HeredocIndentation:
   Enabled: false
 
+Layout/LineEndStringConcatenationIndentation:
+  Enabled: true
+
 Layout/MultilineHashBraceLayout:
   Enabled: true
 
@@ -78,9 +80,15 @@ Layout/TrailingEmptyLines:
 Layout/TrailingWhitespace:
   Enabled: true
 
+Lint/AmbiguousOperatorPrecedence:
+  Enabled: true
+
 Lint/AmbiguousAssignment:
   Enabled: true
 
+Lint/AmbiguousRange:
+  Enabled: true
+
 Lint/DeprecatedConstants:
   Enabled: true
 
@@ -96,6 +104,12 @@ Lint/EmptyBlock:
 Lint/EmptyClass:
   Enabled: false
 
+Lint/EmptyInPattern:
+  Enabled: true
+
+Lint/IncompatibleIoSelectWithFiberScheduler:
+  Enabled: true
+
 Lint/LambdaWithoutLiteralBlock:
   Enabled: true
 
@@ -117,6 +131,9 @@ Lint/RaiseException:
 Lint/RedundantDirGlobSort:
   Enabled: true
 
+Lint/RequireRelativeSelfPath:
+  Enabled: true
+
 Lint/RescueException:
   Enabled: true
 
@@ -192,6 +209,9 @@ Naming/ClassAndModuleCamelCase:
 Naming/BlockParameterName:
   Enabled: true
 
+Naming/InclusiveLanguage:
+  Enabled: true
+
 Naming/MethodParameterName:
   Enabled: false
 
@@ -201,8 +221,8 @@ Naming/MethodName:
 Naming/VariableName:
   Enabled: false
 
-Style/AccessorGrouping:
-  Enabled: false
+Security/IoMethods:
+  Enabled: true
 
 Style/Alias:
   Enabled: true
@@ -276,12 +296,18 @@ Style/HashTransformValues:
 Style/IfUnlessModifier:
   Enabled: false
 
+Style/InPatternThen:
+  Enabled: true
+
 Style/InverseMethods:
   Enabled: false
 
 Style/MissingRespondToMissing:
   Enabled: false
 
+Style/MultilineInPatternThen:
+  Enabled: true
+
 Style/NegatedIfElseCondition:
   Enabled: true
 
@@ -291,9 +317,18 @@ Style/Next:
 Style/NilLambda:
   Enabled: true
 
+Style/NumberedParameters:
+  Enabled: true
+
+Style/NumberedParametersLimit:
+  Enabled: true
+
 Style/NumericLiterals:
   Enabled: false
 
+Style/QuotedSymbols:
+  Enabled: true
+
 Style/RaiseArgs:
   Enabled: true
 
@@ -306,11 +341,17 @@ Style/RedundantReturn:
 Style/RedundantSelf:
   Enabled: true
 
+Style/RedundantSelfAssignmentBranch:
+  Enabled: true
+
 Style/RegexpLiteral:
   Enabled: false
 
 Style/PercentLiteralDelimiters:
   Enabled: false
+  
+Style/SelectByRegexp:
+  Enabled: true  
 
 Style/StderrPuts:
   Enabled: false
@@ -367,6 +408,9 @@ Lint/TopLevelReturnWithArgument:
 Lint/UnreachableLoop:
   Enabled: true
 
+Style/AccessorGrouping:
+  Enabled: true
+
 Style/ArrayCoercion:
   Enabled: true
 

data/CHANGELOG.md

@@ -1,3 +1,31 @@
+### 0.8.05 / 2021-10-04
+- Added example of generated scanner. Unused class removal, code re-styling to please `Rubocop`
+
+* [NEW] Folder `example\tokenizer` contains an example with a scanner that was generated.
+* [NEW] Class `Rley::Lexical::Literal` a specialization of Token class for literal tokens.
+* [CHANGE] File `.rubocp.yml` updated to take into account new 1.21 and 1.22 cops.
+* [DELETED] Class `Rley::Parser::ParserTracer` this class was no more in use.
+* [DELETED] Class `Rley::Syntax::Literal` this class was unused.
+* [DELETED] Class `Rley::Syntax::VerbatimSymbol` this class was unused.
+
+### 0.8.03 / 2021-08-27
+- Added missing method in `ASTBaseBuilder` class
+
+* [FIX] Added method `ParseRep::ASTBaseBuilder#reduce_children`.
+
+### 0.8.02 / 2021-08-23
+- Code restyling to please Rubocop 1.19.1
+
+* [CHANGE] File `.rubocp.yml` updated to take into account new cops.
+* [CHANGE] File `*.rb` files updated to fix Rubocop findings
+
+
+### 0.8.01 / 2021-08-22
+- Unused/redundant file removal. Fix in rule generation
+
+* [CHANGE] Removal of files in repository that were redundant/useless.
+* [FIX] The rule ordering was broken by the rules implicitly generated by Rlry
+
 ### 0.8.00 / 2021-08-15
 - New grammar builder that accepts ? * + modifiers
 
@@ -15,10 +43,10 @@
 - [FIX] Method `GFGParsing#nullable_rule`: issue with nullable productions having at least one member in their rhs.
 
 ### 0.7.04 / 2019-08-17
-- Rley recognizer is about 25% faster than previous version. Kudos to the people 
-behind the *magic_frozen_string_literal* gem. 
+- Rley recognizer is about 25% faster than previous version. Kudos to the people
+behind the *magic_frozen_string_literal* gem.
 - Code refactoring to use string frozen magic comments (as a consequence, Rley runs only on Rubies 2.3 or newer).  
-- Code restyling to please rubocop 0.7.40. 
+- Code restyling to please rubocop 0.7.40.
 - [CHANGE] Class `ParseEntrySet`: minor code optimization
 - [CHANGE] File `README.md` removed allusion to Ruby 2.0.x up to 2.2.x.
 - [CHANGE] File `README.md` added Ruby 2.6.x up as supported version.
@@ -103,7 +131,7 @@ behind the *magic_frozen_string_literal* gem.
 * [FIX] Code re-styling to remove most style offenses found by Rubocop 0.52.1
 
 ### 0.6.00 / 2018-02-25
-Version bump. Highlights: new programming interface through facade object, improved AST generation. 
+Version bump. Highlights: new programming interface through facade object, improved AST generation.
 * [NEW] Class `Rley::Engine`: Implementation of Facade design pattern to reach more convenient interface.
 * [NEW] Class `Rley::ParseRep::ASTBaseBuilder` Abstract class that simplifies the creation of custom AST (Abstract Syntax Tree)
 * [NEW] Module `Rley::ParseRep` hosts the classes for building parse representations (parse trees and forests)

data/examples/NLP/pico_en_demo.rb

@@ -26,8 +26,8 @@ engine.build_grammar do
   rule 'S' => 'NP VP'
   rule 'NP' => 'Proper-Noun'
   rule 'NP' => 'Determiner Noun'
-  rule 'NP' => 'Determiner Noun PP'  
-  rule 'VP' => 'Verb NP'  
+  rule 'NP' => 'Determiner Noun PP'
+  rule 'VP' => 'Verb NP'
   rule 'VP' => 'Verb NP PP'
   rule 'PP' => 'Preposition NP'
 end

data/examples/data_formats/JSON/README.md

@@ -0,0 +1,34 @@
+## Demo JSON parser
+
+### Source files
+This sample project consists of the following files:  
+- **cli_opt.rb** containing the utility class `CLIOptions` that retrieves the
+  command-line options.
+
+- **json_ast_builder.rb** containing the class `JSONASTBuilder` that creates
+  the abstract syntax trees (AST) representation for a given parse result.  
+
+- **json_ast_nodes.rb** containing the node classs for implementing the AST.
+The nodes are augmented with methods that allow the re-use of some formatters
+designed initially for CST (Concrete Syntax Tree). The nodes have a 'to_ruby'
+method that enable a conversion of the input JSON text into Ruby representation.
+
+- **JSON_demo.rb** a demo command-line program.  
+  It parses the JSON file specified in the command-line and outputs the parse tree
+  into different formats. For more details, run the command-line: `JSON_demo.rb --help`
+
+- **JSON_grammar.rb** implementing the class `JSONGrammar`.  
+  The grammar is a list of (production) rules that specifies the syntax of JSON data.  
+  A little examination of the grammar will reveal interesting grammar features such as:  
+    * Recursive rules (e.g. terminal in left-side of rule also appears in right-hand side)
+
+- **JSON_lexer.rb** implementing  the class `JSONLexer`.  
+  The purpose of the lexer is to break the JSON source text into a sequence of tokens.  
+
+- **JSON_parser.rb** implementing a `JSONParser` class.  
+  The parser processes the tokens stream from the lexer and delivers its results  
+
+
+- sample0x.json files of various degree of complexity.  
+
+- sample0x.svg files that are syntax tree diagrams obtained by feeding [RSyntaxTree](http://yohasebe.com/rsyntaxtree/) with the output of JSON_demo.rb. Remark: the website works best with not too complex parse trees.

data/examples/data_formats/JSON/sample01.json

@@ -0,0 +1,3 @@
+{ 
+  "name" : "John Doe"
+}

data/examples/data_formats/JSON/sample01.svg

@@ -0,0 +1,36 @@
+<?xml version="1.0" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" 
+ "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg width="469.5" height="502" version="1.1" xmlns="http://www.w3.org/2000/svg">
+<text style='fill: green; font-size: 16px;' x='296.53125' y='490'> John Doe</text>
+<text style='fill: green; font-size: 16px;' x='109.53125' y='424'> name</text>
+<text style='fill: green; font-size: 16px;' x='221.2421875' y='424'> :</text>
+<text style='fill: blue; font-size: 16px;' x='309.3828125' y='424'>string</text>
+<polygon style='fill: white; stroke: black; stroke-width:1;' points='361 467 297 467 329 431' />
+<text style='fill: blue; font-size: 16px;' x='109.796875' y='358'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='130' y1='401' x2='130' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='164.796875' y='358'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='223' y1='401' x2='223' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='309.75' y='358'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='329' y1='401' x2='329' y2='365' />
+<text style='fill: green; font-size: 16px;' x='46.5078125' y='292'> {</text>
+<text style='fill: blue; font-size: 16px;' x='198.40234375' y='292'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='130' y1='335' x2='229' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='223' y1='335' x2='229' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='329' y1='335' x2='229' y2='299' />
+<text style='fill: green; font-size: 16px;' x='412.4375' y='292'> }</text>
+<text style='fill: blue; font-size: 16px;' x='5.0' y='226'>begin-object</text>
+<line style='stroke:black; stroke-width:1;' x1='50' y1='269' x2='50' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='185.58984375' y='226'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='229' y1='269' x2='229' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='376.96875' y='226'>end-object</text>
+<line style='stroke:black; stroke-width:1;' x1='416' y1='269' x2='416' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='209.984375' y='160'>object</text>
+<line style='stroke:black; stroke-width:1;' x1='50' y1='203' x2='233' y2='167' />
+<line style='stroke:black; stroke-width:1;' x1='229' y1='203' x2='233' y2='167' />
+<line style='stroke:black; stroke-width:1;' x1='416' y1='203' x2='233' y2='167' />
+<text style='fill: blue; font-size: 16px;' x='213.25' y='94'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='233' y1='137' x2='233' y2='101' />
+<text style='fill: blue; font-size: 16px;' x='197.03125' y='28'>JSON-text</text>
+<line style='stroke:black; stroke-width:1;' x1='233' y1='71' x2='233' y2='35' />
+</svg>

data/examples/data_formats/JSON/sample02.json

@@ -0,0 +1,6 @@
+{
+    "id": 1,
+    "name": "A green door",
+    "price": 12.50,
+    "tags": ["home", "green"]
+}

data/examples/data_formats/JSON/sample02.svg

@@ -0,0 +1,128 @@
+<?xml version="1.0" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" 
+ "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg width="2060.15625" height="766" version="1.1" xmlns="http://www.w3.org/2000/svg">
+<text style='fill: green; font-size: 16px;' x='1597.078125' y='754'> home</text>
+<text style='fill: green; font-size: 16px;' x='320.75' y='688'> 1</text>
+<text style='fill: blue; font-size: 16px;' x='1597.265625' y='688'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='1617' y1='731' x2='1617' y2='695' />
+<text style='fill: green; font-size: 16px;' x='1825.0' y='688'> green</text>
+<text style='fill: green; font-size: 16px;' x='122.84375' y='622'> id</text>
+<text style='fill: green; font-size: 16px;' x='220.7109375' y='622'> :</text>
+<text style='fill: blue; font-size: 16px;' x='296.0' y='622'>number</text>
+<line style='stroke:black; stroke-width:1;' x1='325' y1='665' x2='325' y2='629' />
+<text style='fill: green; font-size: 16px;' x='685.109375' y='622'> A green door</text>
+<text style='fill: blue; font-size: 16px;' x='1597.6328125' y='622'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1617' y1='665' x2='1617' y2='629' />
+<text style='fill: green; font-size: 16px;' x='1750.75' y='622'> ,</text>
+<text style='fill: blue; font-size: 16px;' x='1825.984375' y='622'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='1846' y1='665' x2='1846' y2='629' />
+<text style='fill: blue; font-size: 16px;' x='109.53125' y='556'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='129' y1='599' x2='129' y2='563' />
+<text style='fill: blue; font-size: 16px;' x='164.265625' y='556'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='223' y1='599' x2='223' y2='563' />
+<text style='fill: blue; font-size: 16px;' x='305.0546875' y='556'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='325' y1='599' x2='325' y2='563' />
+<text style='fill: green; font-size: 16px;' x='498.109375' y='556'> name</text>
+<text style='fill: green; font-size: 16px;' x='609.8203125' y='556'> :</text>
+<text style='fill: blue; font-size: 16px;' x='712.2421875' y='556'>string</text>
+<polygon style='fill: white; stroke: black; stroke-width:1;' points='779 599 686 599 732 563' />
+<text style='fill: green; font-size: 16px;' x='1119.0078125' y='556'> 12.50</text>
+<text style='fill: green; font-size: 16px;' x='1538.2578125' y='556'> [</text>
+<text style='fill: blue; font-size: 16px;' x='1575.671875' y='556'>array-items</text>
+<line style='stroke:black; stroke-width:1;' x1='1617' y1='599' x2='1617' y2='563' />
+<text style='fill: blue; font-size: 16px;' x='1695.0' y='556'>value-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='1752' y1='599' x2='1752' y2='563' />
+<text style='fill: blue; font-size: 16px;' x='1826.3515625' y='556'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1846' y1='599' x2='1846' y2='563' />
+<text style='fill: green; font-size: 16px;' x='1914.8671875' y='556'> ]</text>
+<text style='fill: blue; font-size: 16px;' x='195.921875' y='490'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='129' y1='533' x2='227' y2='497' />
+<line style='stroke:black; stroke-width:1;' x1='223' y1='533' x2='227' y2='497' />
+<line style='stroke:black; stroke-width:1;' x1='325' y1='533' x2='227' y2='497' />
+<text style='fill: green; font-size: 16px;' x='423.859375' y='490'> ,</text>
+<text style='fill: blue; font-size: 16px;' x='498.375' y='490'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='518' y1='533' x2='518' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='553.375' y='490'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='612' y1='533' x2='612' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='712.609375' y='490'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='732' y1='533' x2='732' y2='497' />
+<text style='fill: green; font-size: 16px;' x='926.921875' y='490'> price</text>
+<text style='fill: green; font-size: 16px;' x='1035.2890625' y='490'> :</text>
+<text style='fill: blue; font-size: 16px;' x='1110.578125' y='490'>number</text>
+<line style='stroke:black; stroke-width:1;' x1='1139' y1='533' x2='1139' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='1499.15625' y='490'>begin-array</text>
+<line style='stroke:black; stroke-width:1;' x1='1541' y1='533' x2='1541' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='1690.03125' y='490'>array-items</text>
+<line style='stroke:black; stroke-width:1;' x1='1617' y1='533' x2='1731' y2='497' />
+<line style='stroke:black; stroke-width:1;' x1='1752' y1='533' x2='1731' y2='497' />
+<line style='stroke:black; stroke-width:1;' x1='1846' y1='533' x2='1731' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='1881.703125' y='490'>end-array</text>
+<line style='stroke:black; stroke-width:1;' x1='1917' y1='533' x2='1917' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='183.109375' y='424'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='227' y1='467' x2='227' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='368.109375' y='424'>value-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='426' y1='467' x2='426' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='594.12109375' y='424'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='518' y1='467' x2='625' y2='431' />
+<line style='stroke:black; stroke-width:1;' x1='612' y1='467' x2='625' y2='431' />
+<line style='stroke:black; stroke-width:1;' x1='732' y1='467' x2='625' y2='431' />
+<text style='fill: green; font-size: 16px;' x='849.859375' y='424'> ,</text>
+<text style='fill: blue; font-size: 16px;' x='924.109375' y='424'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='944' y1='467' x2='944' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='978.84375' y='424'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='1037' y1='467' x2='1037' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='1119.6328125' y='424'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1139' y1='467' x2='1139' y2='431' />
+<text style='fill: green; font-size: 16px;' x='1317.6875' y='424'> tags</text>
+<text style='fill: green; font-size: 16px;' x='1423.8671875' y='424'> :</text>
+<text style='fill: blue; font-size: 16px;' x='1709.4296875' y='424'>array</text>
+<line style='stroke:black; stroke-width:1;' x1='1541' y1='467' x2='1729' y2='431' />
+<line style='stroke:black; stroke-width:1;' x1='1731' y1='467' x2='1729' y2='431' />
+<line style='stroke:black; stroke-width:1;' x1='1917' y1='467' x2='1729' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='382.208984375' y='358'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='227' y1='401' x2='426' y2='365' />
+<line style='stroke:black; stroke-width:1;' x1='426' y1='401' x2='426' y2='365' />
+<line style='stroke:black; stroke-width:1;' x1='625' y1='401' x2='426' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='794.109375' y='358'>value-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='852' y1='401' x2='852' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='1010.5' y='358'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='944' y1='401' x2='1042' y2='365' />
+<line style='stroke:black; stroke-width:1;' x1='1037' y1='401' x2='1042' y2='365' />
+<line style='stroke:black; stroke-width:1;' x1='1139' y1='401' x2='1042' y2='365' />
+<text style='fill: green; font-size: 16px;' x='1238.4375' y='358'> ,</text>
+<text style='fill: blue; font-size: 16px;' x='1312.6875' y='358'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='1333' y1='401' x2='1333' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='1367.421875' y='358'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='1426' y1='401' x2='1426' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='1709.390625' y='358'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1729' y1='401' x2='1729' y2='365' />
+<text style='fill: green; font-size: 16px;' x='46.5078125' y='292'> {</text>
+<text style='fill: blue; font-size: 16px;' x='689.9482421875' y='292'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='426' y1='335' x2='734' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='852' y1='335' x2='734' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='1042' y1='335' x2='734' y2='299' />
+<text style='fill: blue; font-size: 16px;' x='1182.6875' y='292'>value-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='1240' y1='335' x2='1240' y2='299' />
+<text style='fill: blue; font-size: 16px;' x='1499.66796875' y='292'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='1333' y1='335' x2='1531' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='1426' y1='335' x2='1531' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='1729' y1='335' x2='1531' y2='299' />
+<text style='fill: green; font-size: 16px;' x='2003.09375' y='292'> }</text>
+<text style='fill: blue; font-size: 16px;' x='5.0' y='226'>begin-object</text>
+<line style='stroke:black; stroke-width:1;' x1='50' y1='269' x2='50' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='1088.40185546875' y='226'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='734' y1='269' x2='1132' y2='233' />
+<line style='stroke:black; stroke-width:1;' x1='1240' y1='269' x2='1132' y2='233' />
+<line style='stroke:black; stroke-width:1;' x1='1531' y1='269' x2='1132' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='1967.625' y='226'>end-object</text>
+<line style='stroke:black; stroke-width:1;' x1='2006' y1='269' x2='2006' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='1005.3125' y='160'>object</text>
+<line style='stroke:black; stroke-width:1;' x1='50' y1='203' x2='1028' y2='167' />
+<line style='stroke:black; stroke-width:1;' x1='1132' y1='203' x2='1028' y2='167' />
+<line style='stroke:black; stroke-width:1;' x1='2006' y1='203' x2='1028' y2='167' />
+<text style='fill: blue; font-size: 16px;' x='1008.578125' y='94'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1028' y1='137' x2='1028' y2='101' />
+<text style='fill: blue; font-size: 16px;' x='992.359375' y='28'>JSON-text</text>
+<line style='stroke:black; stroke-width:1;' x1='1028' y1='71' x2='1028' y2='35' />
+</svg>

data/examples/data_formats/JSON/sample03.json

@@ -0,0 +1,88 @@
+{"web-app": {
+  "servlet": [   
+    {
+      "servlet-name": "cofaxCDS",
+      "servlet-class": "org.cofax.cds.CDSServlet",
+      "init-param": {
+        "configGlossary:installationAt": "Philadelphia, PA",
+        "configGlossary:adminEmail": "ksm@pobox.com",
+        "configGlossary:poweredBy": "Cofax",
+        "configGlossary:poweredByIcon": "/images/cofax.gif",
+        "configGlossary:staticPath": "/content/static",
+        "templateProcessorClass": "org.cofax.WysiwygTemplate",
+        "templateLoaderClass": "org.cofax.FilesTemplateLoader",
+        "templatePath": "templates",
+        "templateOverridePath": "",
+        "defaultListTemplate": "listTemplate.htm",
+        "defaultFileTemplate": "articleTemplate.htm",
+        "useJSP": false,
+        "jspListTemplate": "listTemplate.jsp",
+        "jspFileTemplate": "articleTemplate.jsp",
+        "cachePackageTagsTrack": 200,
+        "cachePackageTagsStore": 200,
+        "cachePackageTagsRefresh": 60,
+        "cacheTemplatesTrack": 100,
+        "cacheTemplatesStore": 50,
+        "cacheTemplatesRefresh": 15,
+        "cachePagesTrack": 200,
+        "cachePagesStore": 100,
+        "cachePagesRefresh": 10,
+        "cachePagesDirtyRead": 10,
+        "searchEngineListTemplate": "forSearchEnginesList.htm",
+        "searchEngineFileTemplate": "forSearchEngines.htm",
+        "searchEngineRobotsDb": "WEB-INF/robots.db",
+        "useDataStore": true,
+        "dataStoreClass": "org.cofax.SqlDataStore",
+        "redirectionClass": "org.cofax.SqlRedirection",
+        "dataStoreName": "cofax",
+        "dataStoreDriver": "com.microsoft.jdbc.sqlserver.SQLServerDriver",
+        "dataStoreUrl": "jdbc:microsoft:sqlserver://LOCALHOST:1433;DatabaseName=goon",
+        "dataStoreUser": "sa",
+        "dataStorePassword": "dataStoreTestQuery",
+        "dataStoreTestQuery": "SET NOCOUNT ON;select test='test';",
+        "dataStoreLogFile": "/usr/local/tomcat/logs/datastore.log",
+        "dataStoreInitConns": 10,
+        "dataStoreMaxConns": 100,
+        "dataStoreConnUsageLimit": 100,
+        "dataStoreLogLevel": "debug",
+        "maxUrlLength": 500}},
+    {
+      "servlet-name": "cofaxEmail",
+      "servlet-class": "org.cofax.cds.EmailServlet",
+      "init-param": {
+      "mailHost": "mail1",
+      "mailHostOverride": "mail2"}},
+    {
+      "servlet-name": "cofaxAdmin",
+      "servlet-class": "org.cofax.cds.AdminServlet"},
+ 
+    {
+      "servlet-name": "fileServlet",
+      "servlet-class": "org.cofax.cds.FileServlet"},
+    {
+      "servlet-name": "cofaxTools",
+      "servlet-class": "org.cofax.cms.CofaxToolsServlet",
+      "init-param": {
+        "templatePath": "toolstemplates/",
+        "log": 1,
+        "logLocation": "/usr/local/tomcat/logs/CofaxTools.log",
+        "logMaxSize": "",
+        "dataLog": 1,
+        "dataLogLocation": "/usr/local/tomcat/logs/dataLog.log",
+        "dataLogMaxSize": "",
+        "removePageCache": "/content/admin/remove?cache=pages&id=",
+        "removeTemplateCache": "/content/admin/remove?cache=templates&id=",
+        "fileTransferFolder": "/usr/local/tomcat/webapps/content/fileTransferFolder",
+        "lookInContext": 1,
+        "adminGroupID": 4,
+        "betaServer": true}}],
+  "servlet-mapping": {
+    "cofaxCDS": "/",
+    "cofaxEmail": "/cofaxutil/aemail/*",
+    "cofaxAdmin": "/admin/*",
+    "fileServlet": "/static/*",
+    "cofaxTools": "/tools/*"},
+ 
+  "taglib": {
+    "taglib-uri": "cofax.tld",
+    "taglib-location": "/WEB-INF/tlds/cofax.tld"}}}

data/examples/general/calc_iter1/README.md

@@ -0,0 +1,26 @@
+## Demo calculator (iteration 1)
+
+### Source files
+This sample project consists of the following files:  
+- **calc_ast_builder.rb** containing the class `CalcASTBuilder` that creates
+  the abstract syntax trees (AST) representation for a given parse result.  
+
+- **calc_ast_nodes.rb** containing the node classes for implementing the AST.
+The nodes have a 'interpret' method that computes of the numeric value of the
+node.
+
+- **calc_demo.rb** a demo command-line program.  
+  It parses the expression in the command-line and outputs the numeric result
+  For more details, run the command-line: `calc_demo.rb`
+
+- **calc_grammar.rb** implementing the class `CalcGrammar`.  
+  The grammar is a list of (production) rules that specifies the syntax of math 
+  expression.   A little examination of the grammar will reveal interesting 
+  grammar features such as:  
+    * Recursive rules (e.g. terminal in left-side of rule also appears in right-hand side)
+
+- **calc_lexer.rb** implementing  the class `CalcLexer`.  
+  The purpose of the lexer is to break the math expression into a sequence of tokens.  
+
+- **calc_parser.rb** implementing a `CalcParser` class.  
+  The parser processes the tokens stream from the lexer and delivers its results  

data/examples/general/calc_iter2/README.md

@@ -0,0 +1,55 @@
+## Demo calculator (iteration 2)
+Compared to the first iteration, this calculator supports:
+- the exponentiation operator '**',
+- the unary '-' operator (sign change),
+- the PI and E constants,
+- and several functions from Ruby's Math module.
+
+
+It also prints out:
+- The Concrete Syntax Tree (CST), a complete but verbose parse tree representation
+- The Abstract Syntax Tree (AST), a customized parse tree representation that is more
+convenient for further processing (here calculation.).
+
+BUILT-IN OPERATORS AND FUNCTIONS
+---------------------------------
+Operators: `+`, `-`, `*`, `/`, `**`  
+Numeric constants: `PI` (= 3.14159...), `E` (= 2.71828...)
+Functions from Ruby's Math module:
+`sqrt` (square root), `cbrt` (cubic root),  
+`exp` (exponential), `ln` (natural or neperian logarithm), `log10` (logarithm in base 10)  
+Trigonometric functions: `sin`, `cos`, `tan`  
+Inverse trigonometric functions: `asin`, `acos`, `atan`  
+
+
+### Source files
+This sample project consists of the following files:  
+- **calc_ast_builder.rb** containing the class `CalcASTBuilder` that creates
+  the abstract syntax trees (AST) representation for a given parse result.  
+
+- **calc_ast_nodes.rb** containing the node classes for implementing the AST.
+The nodes have a 'interpret' method that computes of the numeric value of the
+node.
+
+- **calc_demo.rb** a demo command-line program.  
+  It parses the expression in the command-line and outputs the numeric result
+  For more details, run the command-line: `calc_demo.rb`
+
+- **calc_grammar.rb** implementing the class `CalcGrammar`.  
+  The grammar is a list of (production) rules that specifies the syntax of math
+  expression.   A little examination of the grammar will reveal interesting
+  grammar features such as:  
+    * Recursive rules (e.g. terminal in left-side of rule also appears in right-hand side)
+
+- **calc_lexer.rb** implementing  the class `CalcLexer`.  
+  The purpose of the lexer is to break the math expression into a sequence of tokens.  
+
+- **calc_parser.rb** implementing a `CalcParser` class.  
+  The parser processes the tokens stream from the lexer and delivers its results
+
+- **sample_result.txt** a sample text containing the output of calculator for the expression:
+2 * 3 + (1 + 3 ** 2). It illustrates the huge difference in size and nesting between the CST
+(Concrete Syntax Tree) and the AST (Abstract Syntax Tree) representations. The generation of CSTs
+comes out-of-the-box with **Rley**. Creating ASTs requires customization. Here, the class for the
+AST nodes of the AST are defined in the **calc_ast_nodes.rb** and the methods for assembling the
+AST from the grammar rules are implemented in the `CalcASTBuilder` class.