rley 0.8.03 → 0.8.05

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3e3f429e3129dc66ab35ca8bba695b0788d710c6e61033a086d7b3c7c40af7cf
-  data.tar.gz: eea709ac4554918cc690e27ebb8540418234b6b0706fd98390c6f8f5d603f868
+  metadata.gz: a94c23265bf2996856da4d4b1fc6f2749873626951d30bacec9f1ca7619d4d4f
+  data.tar.gz: fc6a6ae19388b532dc711915a9c33ffc455a602ea9e6a1ae13e53522bc0616d0
 SHA512:
-  metadata.gz: bf936e0f984789b7c509b9bbd5078c787e79caef32120f4409b23225fe7589d1bdddf5e74e1467401acffb329461b0812c3bb46a9da4b651ae10c94fcd4a52e1
-  data.tar.gz: 606e0e80bfedcd9915eff6b776a826b9032cef201febc07bfcbc1a027622606584bd83e9874a5f448637dd46f66592490c92138c874a07ae630d678e17b5982f
+  metadata.gz: 9b25f6a34e4e5603d4b88f5cf3220e998d9e0c38ffef204ebbf4a8eacaf57853c2bb9bc2bb5a3765440aaed41ee5a4883edadc533c6df0dc907b58457b085469
+  data.tar.gz: 57c3021f24eb5764fef2be6bdd58a9cda1c9b9343737c357d6c1e1862beceaf041ed9eee9fb625cf9acc9492bac0fcb6a6ac8cd9dc464dcf2096fccd5b51e6e9

data/.rubocop.yml

@@ -1,7 +1,6 @@
 AllCops:
   Exclude:
-    - 'examples/**/*'
-    - 'exp/**/*'    
+    - 'exp/**/*'
 
 Gemspec/DateAssignment:
   Enabled: true
@@ -81,6 +80,9 @@ Layout/TrailingEmptyLines:
 Layout/TrailingWhitespace:
   Enabled: true
 
+Lint/AmbiguousOperatorPrecedence:
+  Enabled: true
+
 Lint/AmbiguousAssignment:
   Enabled: true
 
@@ -105,6 +107,9 @@ Lint/EmptyClass:
 Lint/EmptyInPattern:
   Enabled: true
 
+Lint/IncompatibleIoSelectWithFiberScheduler:
+  Enabled: true
+
 Lint/LambdaWithoutLiteralBlock:
   Enabled: true
 
@@ -126,6 +131,9 @@ Lint/RaiseException:
 Lint/RedundantDirGlobSort:
   Enabled: true
 
+Lint/RequireRelativeSelfPath:
+  Enabled: true
+
 Lint/RescueException:
   Enabled: true
 
@@ -213,6 +221,9 @@ Naming/MethodName:
 Naming/VariableName:
   Enabled: false
 
+Security/IoMethods:
+  Enabled: true
+
 Style/Alias:
   Enabled: true
 
@@ -293,9 +304,9 @@ Style/InverseMethods:
 
 Style/MissingRespondToMissing:
   Enabled: false
-  
+
 Style/MultilineInPatternThen:
-  Enabled: true  
+  Enabled: true
 
 Style/NegatedIfElseCondition:
   Enabled: true
@@ -306,11 +317,17 @@ Style/Next:
 Style/NilLambda:
   Enabled: true
 
+Style/NumberedParameters:
+  Enabled: true
+
+Style/NumberedParametersLimit:
+  Enabled: true
+
 Style/NumericLiterals:
   Enabled: false
-  
+
 Style/QuotedSymbols:
-  Enabled: true  
+  Enabled: true
 
 Style/RaiseArgs:
   Enabled: true
@@ -323,15 +340,18 @@ Style/RedundantReturn:
 
 Style/RedundantSelf:
   Enabled: true
-  
+
 Style/RedundantSelfAssignmentBranch:
-  Enabled: true  
+  Enabled: true
 
 Style/RegexpLiteral:
   Enabled: false
 
 Style/PercentLiteralDelimiters:
   Enabled: false
+  
+Style/SelectByRegexp:
+  Enabled: true  
 
 Style/StderrPuts:
   Enabled: false

data/CHANGELOG.md

@@ -1,3 +1,13 @@
+### 0.8.05 / 2021-10-04
+- Added example of generated scanner. Unused class removal, code re-styling to please `Rubocop`
+
+* [NEW] Folder `example\tokenizer` contains an example with a scanner that was generated.
+* [NEW] Class `Rley::Lexical::Literal` a specialization of Token class for literal tokens.
+* [CHANGE] File `.rubocp.yml` updated to take into account new 1.21 and 1.22 cops.
+* [DELETED] Class `Rley::Parser::ParserTracer` this class was no more in use.
+* [DELETED] Class `Rley::Syntax::Literal` this class was unused.
+* [DELETED] Class `Rley::Syntax::VerbatimSymbol` this class was unused.
+
 ### 0.8.03 / 2021-08-27
 - Added missing method in `ASTBaseBuilder` class
 

data/examples/data_formats/JSON/README.md

@@ -0,0 +1,34 @@
+## Demo JSON parser
+
+### Source files
+This sample project consists of the following files:  
+- **cli_opt.rb** containing the utility class `CLIOptions` that retrieves the
+  command-line options.
+
+- **json_ast_builder.rb** containing the class `JSONASTBuilder` that creates
+  the abstract syntax trees (AST) representation for a given parse result.  
+
+- **json_ast_nodes.rb** containing the node classs for implementing the AST.
+The nodes are augmented with methods that allow the re-use of some formatters
+designed initially for CST (Concrete Syntax Tree). The nodes have a 'to_ruby'
+method that enable a conversion of the input JSON text into Ruby representation.
+
+- **JSON_demo.rb** a demo command-line program.  
+  It parses the JSON file specified in the command-line and outputs the parse tree
+  into different formats. For more details, run the command-line: `JSON_demo.rb --help`
+
+- **JSON_grammar.rb** implementing the class `JSONGrammar`.  
+  The grammar is a list of (production) rules that specifies the syntax of JSON data.  
+  A little examination of the grammar will reveal interesting grammar features such as:  
+    * Recursive rules (e.g. terminal in left-side of rule also appears in right-hand side)
+
+- **JSON_lexer.rb** implementing  the class `JSONLexer`.  
+  The purpose of the lexer is to break the JSON source text into a sequence of tokens.  
+
+- **JSON_parser.rb** implementing a `JSONParser` class.  
+  The parser processes the tokens stream from the lexer and delivers its results  
+
+
+- sample0x.json files of various degree of complexity.  
+
+- sample0x.svg files that are syntax tree diagrams obtained by feeding [RSyntaxTree](http://yohasebe.com/rsyntaxtree/) with the output of JSON_demo.rb. Remark: the website works best with not too complex parse trees.

data/examples/data_formats/JSON/sample01.json

@@ -0,0 +1,3 @@
+{ 
+  "name" : "John Doe"
+}

data/examples/data_formats/JSON/sample01.svg

@@ -0,0 +1,36 @@
+<?xml version="1.0" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" 
+ "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg width="469.5" height="502" version="1.1" xmlns="http://www.w3.org/2000/svg">
+<text style='fill: green; font-size: 16px;' x='296.53125' y='490'> John Doe</text>
+<text style='fill: green; font-size: 16px;' x='109.53125' y='424'> name</text>
+<text style='fill: green; font-size: 16px;' x='221.2421875' y='424'> :</text>
+<text style='fill: blue; font-size: 16px;' x='309.3828125' y='424'>string</text>
+<polygon style='fill: white; stroke: black; stroke-width:1;' points='361 467 297 467 329 431' />
+<text style='fill: blue; font-size: 16px;' x='109.796875' y='358'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='130' y1='401' x2='130' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='164.796875' y='358'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='223' y1='401' x2='223' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='309.75' y='358'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='329' y1='401' x2='329' y2='365' />
+<text style='fill: green; font-size: 16px;' x='46.5078125' y='292'> {</text>
+<text style='fill: blue; font-size: 16px;' x='198.40234375' y='292'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='130' y1='335' x2='229' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='223' y1='335' x2='229' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='329' y1='335' x2='229' y2='299' />
+<text style='fill: green; font-size: 16px;' x='412.4375' y='292'> }</text>
+<text style='fill: blue; font-size: 16px;' x='5.0' y='226'>begin-object</text>
+<line style='stroke:black; stroke-width:1;' x1='50' y1='269' x2='50' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='185.58984375' y='226'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='229' y1='269' x2='229' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='376.96875' y='226'>end-object</text>
+<line style='stroke:black; stroke-width:1;' x1='416' y1='269' x2='416' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='209.984375' y='160'>object</text>
+<line style='stroke:black; stroke-width:1;' x1='50' y1='203' x2='233' y2='167' />
+<line style='stroke:black; stroke-width:1;' x1='229' y1='203' x2='233' y2='167' />
+<line style='stroke:black; stroke-width:1;' x1='416' y1='203' x2='233' y2='167' />
+<text style='fill: blue; font-size: 16px;' x='213.25' y='94'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='233' y1='137' x2='233' y2='101' />
+<text style='fill: blue; font-size: 16px;' x='197.03125' y='28'>JSON-text</text>
+<line style='stroke:black; stroke-width:1;' x1='233' y1='71' x2='233' y2='35' />
+</svg>

data/examples/data_formats/JSON/sample02.json

@@ -0,0 +1,6 @@
+{
+    "id": 1,
+    "name": "A green door",
+    "price": 12.50,
+    "tags": ["home", "green"]
+}

data/examples/data_formats/JSON/sample02.svg

@@ -0,0 +1,128 @@
+<?xml version="1.0" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" 
+ "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg width="2060.15625" height="766" version="1.1" xmlns="http://www.w3.org/2000/svg">
+<text style='fill: green; font-size: 16px;' x='1597.078125' y='754'> home</text>
+<text style='fill: green; font-size: 16px;' x='320.75' y='688'> 1</text>
+<text style='fill: blue; font-size: 16px;' x='1597.265625' y='688'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='1617' y1='731' x2='1617' y2='695' />
+<text style='fill: green; font-size: 16px;' x='1825.0' y='688'> green</text>
+<text style='fill: green; font-size: 16px;' x='122.84375' y='622'> id</text>
+<text style='fill: green; font-size: 16px;' x='220.7109375' y='622'> :</text>
+<text style='fill: blue; font-size: 16px;' x='296.0' y='622'>number</text>
+<line style='stroke:black; stroke-width:1;' x1='325' y1='665' x2='325' y2='629' />
+<text style='fill: green; font-size: 16px;' x='685.109375' y='622'> A green door</text>
+<text style='fill: blue; font-size: 16px;' x='1597.6328125' y='622'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1617' y1='665' x2='1617' y2='629' />
+<text style='fill: green; font-size: 16px;' x='1750.75' y='622'> ,</text>
+<text style='fill: blue; font-size: 16px;' x='1825.984375' y='622'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='1846' y1='665' x2='1846' y2='629' />
+<text style='fill: blue; font-size: 16px;' x='109.53125' y='556'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='129' y1='599' x2='129' y2='563' />
+<text style='fill: blue; font-size: 16px;' x='164.265625' y='556'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='223' y1='599' x2='223' y2='563' />
+<text style='fill: blue; font-size: 16px;' x='305.0546875' y='556'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='325' y1='599' x2='325' y2='563' />
+<text style='fill: green; font-size: 16px;' x='498.109375' y='556'> name</text>
+<text style='fill: green; font-size: 16px;' x='609.8203125' y='556'> :</text>
+<text style='fill: blue; font-size: 16px;' x='712.2421875' y='556'>string</text>
+<polygon style='fill: white; stroke: black; stroke-width:1;' points='779 599 686 599 732 563' />
+<text style='fill: green; font-size: 16px;' x='1119.0078125' y='556'> 12.50</text>
+<text style='fill: green; font-size: 16px;' x='1538.2578125' y='556'> [</text>
+<text style='fill: blue; font-size: 16px;' x='1575.671875' y='556'>array-items</text>
+<line style='stroke:black; stroke-width:1;' x1='1617' y1='599' x2='1617' y2='563' />
+<text style='fill: blue; font-size: 16px;' x='1695.0' y='556'>value-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='1752' y1='599' x2='1752' y2='563' />
+<text style='fill: blue; font-size: 16px;' x='1826.3515625' y='556'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1846' y1='599' x2='1846' y2='563' />
+<text style='fill: green; font-size: 16px;' x='1914.8671875' y='556'> ]</text>
+<text style='fill: blue; font-size: 16px;' x='195.921875' y='490'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='129' y1='533' x2='227' y2='497' />
+<line style='stroke:black; stroke-width:1;' x1='223' y1='533' x2='227' y2='497' />
+<line style='stroke:black; stroke-width:1;' x1='325' y1='533' x2='227' y2='497' />
+<text style='fill: green; font-size: 16px;' x='423.859375' y='490'> ,</text>
+<text style='fill: blue; font-size: 16px;' x='498.375' y='490'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='518' y1='533' x2='518' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='553.375' y='490'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='612' y1='533' x2='612' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='712.609375' y='490'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='732' y1='533' x2='732' y2='497' />
+<text style='fill: green; font-size: 16px;' x='926.921875' y='490'> price</text>
+<text style='fill: green; font-size: 16px;' x='1035.2890625' y='490'> :</text>
+<text style='fill: blue; font-size: 16px;' x='1110.578125' y='490'>number</text>
+<line style='stroke:black; stroke-width:1;' x1='1139' y1='533' x2='1139' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='1499.15625' y='490'>begin-array</text>
+<line style='stroke:black; stroke-width:1;' x1='1541' y1='533' x2='1541' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='1690.03125' y='490'>array-items</text>
+<line style='stroke:black; stroke-width:1;' x1='1617' y1='533' x2='1731' y2='497' />
+<line style='stroke:black; stroke-width:1;' x1='1752' y1='533' x2='1731' y2='497' />
+<line style='stroke:black; stroke-width:1;' x1='1846' y1='533' x2='1731' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='1881.703125' y='490'>end-array</text>
+<line style='stroke:black; stroke-width:1;' x1='1917' y1='533' x2='1917' y2='497' />
+<text style='fill: blue; font-size: 16px;' x='183.109375' y='424'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='227' y1='467' x2='227' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='368.109375' y='424'>value-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='426' y1='467' x2='426' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='594.12109375' y='424'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='518' y1='467' x2='625' y2='431' />
+<line style='stroke:black; stroke-width:1;' x1='612' y1='467' x2='625' y2='431' />
+<line style='stroke:black; stroke-width:1;' x1='732' y1='467' x2='625' y2='431' />
+<text style='fill: green; font-size: 16px;' x='849.859375' y='424'> ,</text>
+<text style='fill: blue; font-size: 16px;' x='924.109375' y='424'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='944' y1='467' x2='944' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='978.84375' y='424'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='1037' y1='467' x2='1037' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='1119.6328125' y='424'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1139' y1='467' x2='1139' y2='431' />
+<text style='fill: green; font-size: 16px;' x='1317.6875' y='424'> tags</text>
+<text style='fill: green; font-size: 16px;' x='1423.8671875' y='424'> :</text>
+<text style='fill: blue; font-size: 16px;' x='1709.4296875' y='424'>array</text>
+<line style='stroke:black; stroke-width:1;' x1='1541' y1='467' x2='1729' y2='431' />
+<line style='stroke:black; stroke-width:1;' x1='1731' y1='467' x2='1729' y2='431' />
+<line style='stroke:black; stroke-width:1;' x1='1917' y1='467' x2='1729' y2='431' />
+<text style='fill: blue; font-size: 16px;' x='382.208984375' y='358'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='227' y1='401' x2='426' y2='365' />
+<line style='stroke:black; stroke-width:1;' x1='426' y1='401' x2='426' y2='365' />
+<line style='stroke:black; stroke-width:1;' x1='625' y1='401' x2='426' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='794.109375' y='358'>value-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='852' y1='401' x2='852' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='1010.5' y='358'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='944' y1='401' x2='1042' y2='365' />
+<line style='stroke:black; stroke-width:1;' x1='1037' y1='401' x2='1042' y2='365' />
+<line style='stroke:black; stroke-width:1;' x1='1139' y1='401' x2='1042' y2='365' />
+<text style='fill: green; font-size: 16px;' x='1238.4375' y='358'> ,</text>
+<text style='fill: blue; font-size: 16px;' x='1312.6875' y='358'>string</text>
+<line style='stroke:black; stroke-width:1;' x1='1333' y1='401' x2='1333' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='1367.421875' y='358'>name-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='1426' y1='401' x2='1426' y2='365' />
+<text style='fill: blue; font-size: 16px;' x='1709.390625' y='358'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1729' y1='401' x2='1729' y2='365' />
+<text style='fill: green; font-size: 16px;' x='46.5078125' y='292'> {</text>
+<text style='fill: blue; font-size: 16px;' x='689.9482421875' y='292'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='426' y1='335' x2='734' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='852' y1='335' x2='734' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='1042' y1='335' x2='734' y2='299' />
+<text style='fill: blue; font-size: 16px;' x='1182.6875' y='292'>value-separator</text>
+<line style='stroke:black; stroke-width:1;' x1='1240' y1='335' x2='1240' y2='299' />
+<text style='fill: blue; font-size: 16px;' x='1499.66796875' y='292'>member</text>
+<line style='stroke:black; stroke-width:1;' x1='1333' y1='335' x2='1531' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='1426' y1='335' x2='1531' y2='299' />
+<line style='stroke:black; stroke-width:1;' x1='1729' y1='335' x2='1531' y2='299' />
+<text style='fill: green; font-size: 16px;' x='2003.09375' y='292'> }</text>
+<text style='fill: blue; font-size: 16px;' x='5.0' y='226'>begin-object</text>
+<line style='stroke:black; stroke-width:1;' x1='50' y1='269' x2='50' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='1088.40185546875' y='226'>member-list</text>
+<line style='stroke:black; stroke-width:1;' x1='734' y1='269' x2='1132' y2='233' />
+<line style='stroke:black; stroke-width:1;' x1='1240' y1='269' x2='1132' y2='233' />
+<line style='stroke:black; stroke-width:1;' x1='1531' y1='269' x2='1132' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='1967.625' y='226'>end-object</text>
+<line style='stroke:black; stroke-width:1;' x1='2006' y1='269' x2='2006' y2='233' />
+<text style='fill: blue; font-size: 16px;' x='1005.3125' y='160'>object</text>
+<line style='stroke:black; stroke-width:1;' x1='50' y1='203' x2='1028' y2='167' />
+<line style='stroke:black; stroke-width:1;' x1='1132' y1='203' x2='1028' y2='167' />
+<line style='stroke:black; stroke-width:1;' x1='2006' y1='203' x2='1028' y2='167' />
+<text style='fill: blue; font-size: 16px;' x='1008.578125' y='94'>value</text>
+<line style='stroke:black; stroke-width:1;' x1='1028' y1='137' x2='1028' y2='101' />
+<text style='fill: blue; font-size: 16px;' x='992.359375' y='28'>JSON-text</text>
+<line style='stroke:black; stroke-width:1;' x1='1028' y1='71' x2='1028' y2='35' />
+</svg>

data/examples/data_formats/JSON/sample03.json

@@ -0,0 +1,88 @@
+{"web-app": {
+  "servlet": [   
+    {
+      "servlet-name": "cofaxCDS",
+      "servlet-class": "org.cofax.cds.CDSServlet",
+      "init-param": {
+        "configGlossary:installationAt": "Philadelphia, PA",
+        "configGlossary:adminEmail": "ksm@pobox.com",
+        "configGlossary:poweredBy": "Cofax",
+        "configGlossary:poweredByIcon": "/images/cofax.gif",
+        "configGlossary:staticPath": "/content/static",
+        "templateProcessorClass": "org.cofax.WysiwygTemplate",
+        "templateLoaderClass": "org.cofax.FilesTemplateLoader",
+        "templatePath": "templates",
+        "templateOverridePath": "",
+        "defaultListTemplate": "listTemplate.htm",
+        "defaultFileTemplate": "articleTemplate.htm",
+        "useJSP": false,
+        "jspListTemplate": "listTemplate.jsp",
+        "jspFileTemplate": "articleTemplate.jsp",
+        "cachePackageTagsTrack": 200,
+        "cachePackageTagsStore": 200,
+        "cachePackageTagsRefresh": 60,
+        "cacheTemplatesTrack": 100,
+        "cacheTemplatesStore": 50,
+        "cacheTemplatesRefresh": 15,
+        "cachePagesTrack": 200,
+        "cachePagesStore": 100,
+        "cachePagesRefresh": 10,
+        "cachePagesDirtyRead": 10,
+        "searchEngineListTemplate": "forSearchEnginesList.htm",
+        "searchEngineFileTemplate": "forSearchEngines.htm",
+        "searchEngineRobotsDb": "WEB-INF/robots.db",
+        "useDataStore": true,
+        "dataStoreClass": "org.cofax.SqlDataStore",
+        "redirectionClass": "org.cofax.SqlRedirection",
+        "dataStoreName": "cofax",
+        "dataStoreDriver": "com.microsoft.jdbc.sqlserver.SQLServerDriver",
+        "dataStoreUrl": "jdbc:microsoft:sqlserver://LOCALHOST:1433;DatabaseName=goon",
+        "dataStoreUser": "sa",
+        "dataStorePassword": "dataStoreTestQuery",
+        "dataStoreTestQuery": "SET NOCOUNT ON;select test='test';",
+        "dataStoreLogFile": "/usr/local/tomcat/logs/datastore.log",
+        "dataStoreInitConns": 10,
+        "dataStoreMaxConns": 100,
+        "dataStoreConnUsageLimit": 100,
+        "dataStoreLogLevel": "debug",
+        "maxUrlLength": 500}},
+    {
+      "servlet-name": "cofaxEmail",
+      "servlet-class": "org.cofax.cds.EmailServlet",
+      "init-param": {
+      "mailHost": "mail1",
+      "mailHostOverride": "mail2"}},
+    {
+      "servlet-name": "cofaxAdmin",
+      "servlet-class": "org.cofax.cds.AdminServlet"},
+ 
+    {
+      "servlet-name": "fileServlet",
+      "servlet-class": "org.cofax.cds.FileServlet"},
+    {
+      "servlet-name": "cofaxTools",
+      "servlet-class": "org.cofax.cms.CofaxToolsServlet",
+      "init-param": {
+        "templatePath": "toolstemplates/",
+        "log": 1,
+        "logLocation": "/usr/local/tomcat/logs/CofaxTools.log",
+        "logMaxSize": "",
+        "dataLog": 1,
+        "dataLogLocation": "/usr/local/tomcat/logs/dataLog.log",
+        "dataLogMaxSize": "",
+        "removePageCache": "/content/admin/remove?cache=pages&id=",
+        "removeTemplateCache": "/content/admin/remove?cache=templates&id=",
+        "fileTransferFolder": "/usr/local/tomcat/webapps/content/fileTransferFolder",
+        "lookInContext": 1,
+        "adminGroupID": 4,
+        "betaServer": true}}],
+  "servlet-mapping": {
+    "cofaxCDS": "/",
+    "cofaxEmail": "/cofaxutil/aemail/*",
+    "cofaxAdmin": "/admin/*",
+    "fileServlet": "/static/*",
+    "cofaxTools": "/tools/*"},
+ 
+  "taglib": {
+    "taglib-uri": "cofax.tld",
+    "taglib-location": "/WEB-INF/tlds/cofax.tld"}}}

data/examples/general/calc_iter1/README.md

@@ -0,0 +1,26 @@
+## Demo calculator (iteration 1)
+
+### Source files
+This sample project consists of the following files:  
+- **calc_ast_builder.rb** containing the class `CalcASTBuilder` that creates
+  the abstract syntax trees (AST) representation for a given parse result.  
+
+- **calc_ast_nodes.rb** containing the node classes for implementing the AST.
+The nodes have a 'interpret' method that computes of the numeric value of the
+node.
+
+- **calc_demo.rb** a demo command-line program.  
+  It parses the expression in the command-line and outputs the numeric result
+  For more details, run the command-line: `calc_demo.rb`
+
+- **calc_grammar.rb** implementing the class `CalcGrammar`.  
+  The grammar is a list of (production) rules that specifies the syntax of math 
+  expression.   A little examination of the grammar will reveal interesting 
+  grammar features such as:  
+    * Recursive rules (e.g. terminal in left-side of rule also appears in right-hand side)
+
+- **calc_lexer.rb** implementing  the class `CalcLexer`.  
+  The purpose of the lexer is to break the math expression into a sequence of tokens.  
+
+- **calc_parser.rb** implementing a `CalcParser` class.  
+  The parser processes the tokens stream from the lexer and delivers its results  

data/examples/general/calc_iter2/README.md

@@ -0,0 +1,55 @@
+## Demo calculator (iteration 2)
+Compared to the first iteration, this calculator supports:
+- the exponentiation operator '**',
+- the unary '-' operator (sign change),
+- the PI and E constants,
+- and several functions from Ruby's Math module.
+
+
+It also prints out:
+- The Concrete Syntax Tree (CST), a complete but verbose parse tree representation
+- The Abstract Syntax Tree (AST), a customized parse tree representation that is more
+convenient for further processing (here calculation.).
+
+BUILT-IN OPERATORS AND FUNCTIONS
+---------------------------------
+Operators: `+`, `-`, `*`, `/`, `**`  
+Numeric constants: `PI` (= 3.14159...), `E` (= 2.71828...)
+Functions from Ruby's Math module:
+`sqrt` (square root), `cbrt` (cubic root),  
+`exp` (exponential), `ln` (natural or neperian logarithm), `log10` (logarithm in base 10)  
+Trigonometric functions: `sin`, `cos`, `tan`  
+Inverse trigonometric functions: `asin`, `acos`, `atan`  
+
+
+### Source files
+This sample project consists of the following files:  
+- **calc_ast_builder.rb** containing the class `CalcASTBuilder` that creates
+  the abstract syntax trees (AST) representation for a given parse result.  
+
+- **calc_ast_nodes.rb** containing the node classes for implementing the AST.
+The nodes have a 'interpret' method that computes of the numeric value of the
+node.
+
+- **calc_demo.rb** a demo command-line program.  
+  It parses the expression in the command-line and outputs the numeric result
+  For more details, run the command-line: `calc_demo.rb`
+
+- **calc_grammar.rb** implementing the class `CalcGrammar`.  
+  The grammar is a list of (production) rules that specifies the syntax of math
+  expression.   A little examination of the grammar will reveal interesting
+  grammar features such as:  
+    * Recursive rules (e.g. terminal in left-side of rule also appears in right-hand side)
+
+- **calc_lexer.rb** implementing  the class `CalcLexer`.  
+  The purpose of the lexer is to break the math expression into a sequence of tokens.  
+
+- **calc_parser.rb** implementing a `CalcParser` class.  
+  The parser processes the tokens stream from the lexer and delivers its results
+
+- **sample_result.txt** a sample text containing the output of calculator for the expression:
+2 * 3 + (1 + 3 ** 2). It illustrates the huge difference in size and nesting between the CST
+(Concrete Syntax Tree) and the AST (Abstract Syntax Tree) representations. The generation of CSTs
+comes out-of-the-box with **Rley**. Creating ASTs requires customization. Here, the class for the
+AST nodes of the AST are defined in the **calc_ast_nodes.rb** and the methods for assembling the
+AST from the grammar rules are implemented in the `CalcASTBuilder` class.

data/examples/general/general_examples.md

@@ -0,0 +1,37 @@
+## Directory contents
+This directory contains a number sample projects and short demos.  
+
+- [calc_iter1](#demo-calculators)  
+- [calc_iter2](#demo-calculators)  
+- [SPPF](#Shared-Packed-Parse-Forest)  
+- [SRL](#Simple-Regex-Language)  
+- [left.rb](#recursive-rules)  
+- [right.rb](#recursive-rules)
+
+
+### Demo calculators
+Two command-line tools that parse basic math expressions
+and calculate their numeric values.
+
+There are two variants of the calculator:
+- **Iteration 1**. A simple calculator program that handles expressions with the  
+  4 basic arithmetic operators: + - * and /
+- **Iteration 2**. A significantly more elaborated calculator that adds:  
+ support for the exponentiation operator and the unary minus (sign change),  
+ PI and E constants,  
+ trigonometric functions, inverse trigonometric functions,  
+ square root, exponential and natural logarithm functions.
+
+As a bonus, the iteration 2 calculator prints out:
+- The Concrete Syntax Tree (**CST**), a complete but verbose parse tree representation
+- The Abstract Syntax Tree (**AST**), a customized parse tree representation that is simpler
+for further processing (i.e. calculation, execution,...).
+
+Although these calculators are demo apps (read: they lack robust error handling and user friendly
+error reporting), great care was taken about their modularity.
+
+### Shared Packed Parse Forest
+This directory will contain code showing how to use and manipulate SPPFs.
+
+### Recursive rules
+The files `left.rb` and `right.rb` show how to define left- and right-recursive rules respectively. These examples were used to benchmark the parsing. Although `Rley` can handle right-recursive rules, one should avoid deeply-nested right-recursive rule calls. The reason is that in these situations the number of possible parse states increases rapidly and affects severely the parsing speed. There are optimization techniques addressing the issue e.g. [Leo's optimization](http://www.sciencedirect.com/science/article/pii/030439759190180A) that may eventually be implemented in Rley provided that they don't put limits on the NLP capabilities. Here is another link about [Leo's optimization](http://loup-vaillant.fr/tutorials/earley-parsing/right-recursion)

data/examples/tokenizer/README.md

@@ -0,0 +1,46 @@
+Integrating a scanner generated with `oedipus_lex` gem with a Rley parser.
+===
+
+This folder contains a demo tokenizer for the `Lox` programming language.  
+While tokenizers from other examples were handwritten, this one was generated with a tool.
+
+The resulting tokenizer consists of two classes:
+- The generated `LoxxyRawScanner` class (file: `loxxy_raw_scanner.rex.rb`).
+- The handwritten `LoxxyTokenizer` class. Its purpose is explained later (file: `loxxy_tokenizer.rb`).
+
+## How was the scanner class generated?
+
+The `LoxxyRawScanner` class was generated from the specification file `loxxy_raw_scanner.rex`.  
+This file has a format that can be read by the `oedipus lex`gem (the 'scanner generator').
+The scanner generator then generates a Ruby class that implements a scanner.
+
+The generation process is controled with a `Rakefile`.
+Assuming that the gem is already installed, launch the following command line in this folder:
+```ruby
+rake tokenizer
+```
+
+Rake script should display the following message:
+```ruby
+Generating loxxy_raw_scanner.rex.rb from loxxy_raw_scanner.rex
+```
+
+## How to install `oedipus_lex`?
+Use the standard installation step:
+```ruby
+gem install oedipus_lex
+```
+
+## Why the `oedipus_lex` scanner generator?
+This gem was created as a companion to the `Racc` parser (part of Ruby's standard library).  
+But the code it produces has no dependency towards a specific parser, 
+so that it can used for building scanners for Rley parsers.
+
+## What is the purpose of the`LoxxyTokenizer` class?
+If the scanner can be generated, why do we need to handcode another class?
+There were two reasons:
+- First, `rex` files use particular syntax a domain-specific language (DSL). So I tend to minimize its use.
+  Without the flexbility of Ruby, handling keywords directly in the `rex` can become cumbersone.
+- Second, the `LoxxyTokenizer` class acts as an Adapter between the parser-neutral generated scanner and the expectations of a Rley parser.
+  For instance, Rley expects the tokenizer to deliver a sequence of `Rley::Lexical::Token` instances.
+  In addition, that class performs some convertion methods that are better implemented directly in Ruby.