RubyGems - citrus - Versions diffs - 2.1.2 → 2.2.0 - Mend

citrus 2.1.2 → 2.2.0

Files changed (36) hide show

data/README +106 -49
data/benchmark/after.dat +192 -0
data/benchmark/before.dat +192 -0
data/citrus.gemspec +0 -1
data/doc/extras.markdown +16 -0
data/doc/syntax.markdown +76 -29
data/doc/testing.markdown +12 -20
data/examples/calc.citrus +12 -11
data/examples/calc.rb +12 -11
data/lib/citrus.rb +416 -253
data/lib/citrus/file.rb +66 -33
data/test/_files/super.citrus +1 -1
data/test/_files/super2.citrus +13 -0
data/test/alias_test.rb +18 -34
data/test/and_predicate_test.rb +15 -10
data/test/but_predicate_test.rb +22 -17
data/test/calc_file_test.rb +1 -1
data/test/choice_test.rb +12 -37
data/test/{rule_test.rb → extension_test.rb} +17 -16
data/test/file_test.rb +350 -244
data/test/grammar_test.rb +5 -11
data/test/helper.rb +1 -17
data/test/input_test.rb +172 -2
data/test/label_test.rb +0 -10
data/test/match_test.rb +91 -35
data/test/multibyte_test.rb +4 -4
data/test/not_predicate_test.rb +15 -10
data/test/parse_error_test.rb +1 -3
data/test/repeat_test.rb +59 -32
data/test/sequence_test.rb +19 -31
data/test/string_terminal_test.rb +55 -0
data/test/super_test.rb +31 -31
data/test/terminal_test.rb +12 -37
metadata +13 -23
data/lib/citrus/debug.rb +0 -69
data/test/debug_test.rb +0 -23

data/README CHANGED Viewed

@@ -113,46 +113,57 @@ already be familiar to Ruby programmers.
 Terminals may be represented by a string or a regular expression. Both follow
 the same rules as Ruby string and regular expression literals.
-    'abc'
-    "abc\n"
-    /\xFF/
+    'abc'         # match "abc"
+    "abc\n"       # match "abc\n"
+    /abc/i        # match "abc" in any case
+    /\xFF/        # match "\xFF"
 Character classes and the dot (match anything) symbol are supported as well for
 compatibility with other parsing expression implementations.
     [a-z0-9]      # match any lowercase letter or digit
     [\x00-\xFF]   # match any octet
-    .             # match anything, even new lines
+    .             # match any single character, including new lines
-See [Terminal](api/classes/Citrus/Terminal.html) for more information.
+Also, strings may use backticks instead of quotes to indicate that they should
+match in a case-insensitive manner.
+    `abc`         # match "abc" in any case
+See [Terminal](api/classes/Citrus/Terminal.html) and
+[StringTerminal](api/classes/Citrus/StringTerminal.html) for more information.
 ## Repetition
 Quantifiers may be used after any expression to specify a number of times it
-must match. The universal form of a quantifier is N*M where N is the minimum and
-M is the maximum number of times the expression may match.
+must match. The universal form of a quantifier is `N*M` where `N` is the minimum
+and `M` is the maximum number of times the expression may match.
-    'abc'1*2      # match "abc" a minimum of one, maximum
-                  # of two times
+    'abc'1*2      # match "abc" a minimum of one, maximum of two times
     'abc'1*       # match "abc" at least once
     'abc'*2       # match "abc" a maximum of twice
-The + and ? operators are supported as well for the common cases of 1* and *1
-respectively.
+Additionally, the minimum and maximum may be omitted entirely to specify that an
+expression may match zero or more times.
+    'abc'*        # match "abc" zero or more times
-    'abc'+        # match "abc" at least once
-    'abc'?        # match "abc" a maximum of once
+The `+` and `?` operators are supported as well for the common cases of `1*` and
+`*1` respectively.
+    'abc'+        # match "abc" one or more times
+    'abc'?        # match "abc" zero or one time
 See [Repeat](api/classes/Citrus/Repeat.html) for more information.
 ## Lookahead
-Both positive and negative lookahead are supported in Citrus. Use the & and !
-operators to indicate that an expression either should or should not match. In
-neither case is any input consumed.
+Both positive and negative lookahead are supported in Citrus. Use the `&` and
+`!` operators to indicate that an expression either should or should not match.
+In neither case is any input consumed.
     &'a' 'b'      # match a "b" preceded by an "a"
-    !'a' 'b'      # match a "b" that is not preceded by an "a"
+    'a' !'b'      # match an "a" that is not followed by a "b"
     !'a' .        # match any character except for "a"
 A special form of lookahead is also supported which will match any character
@@ -178,20 +189,17 @@ See [Sequence](api/classes/Citrus/Sequence.html) for more information.
 ## Choices
 Ordered choice is indicated by a vertical bar that separates two expressions.
-Note that any operator binds more tightly than the bar.
+When using choice, each expression is tried in order. When one matches, the
+rule returns the match immediately without trying the remaining rules.
     'a' | 'b'       # match "a" or "b"
     'a' 'b' | 'c'   # match "a" then "b" (in sequence), or "c"
-See [Choice](api/classes/Citrus/Choice.html) for more information.
-## Super
-When including a grammar inside another, all rules in the child that have the
-same name as a rule in the parent also have access to the "super" keyword to
-invoke the parent rule.
+It is important to note when using ordered choice that any operator binds more
+tightly than the vertical bar. A full chart of operators and their respective
+levels of precedence is below.
-See [Super](api/classes/Citrus/Super.html) for more information.
+See [Choice](api/classes/Citrus/Choice.html) for more information.
 ## Labels
@@ -199,12 +207,50 @@ Match objects may be referred to by a different name than the rule that
 originally generated them. Labels are created by placing the label and a colon
 immediately preceding any expression.
-    chars:/[a-z]+/  # the characters matched by the regular
-                    # expression may be referred to as "chars"
-                    # in a block method
+    chars:/[a-z]+/  # the characters matched by the regular expression
+                    # may be referred to as "chars" in an extension
+                    # method
 See [Label](api/classes/Citrus/Label.html) for more information.
+## Grouping
+As is common in many programming languages, parentheses may be used to override
+the normal binding order of operators.
+    'a' ('b' | 'c')   # match "a", then "b" or "c"
+## Extensions
+Extensions may be specified using either "module" or "block" syntax. When using
+module syntax, specify the name of a module that is used to extend match objects
+in between less than and greater than symbols.
+    [a-z0-9]5*9 <CouponCode>  # match a string that consists of any lower
+                              # cased letter or digit between 5 and 9
+                              # times and extend the match with the
+                              # CouponCode module
+Additionally, extensions may be specified inline using curly braces. Inside the
+curly braces you may embed method definitions that will be used to extend match
+objects.
+    # match any digit and return its integer value when calling the
+    # #value method on the match object
+    [0-9] {
+      def value
+        to_i
+      end
+    }
+## Super
+When including a grammar inside another, all rules in the child that have the
+same name as a rule in the parent also have access to the `super` keyword to
+invoke the parent rule.
+See [Super](api/classes/Citrus/Super.html) for more information.
 ## Precedence
 The following table contains a list of all Citrus symbols and operators and
@@ -214,6 +260,7 @@ Operator  | Name                      | Precedence
 --------- | ------------------------- | ----------
 ''        | String (single quoted)    | 6
 ""        | String (double quoted)    | 6
+``        | String (case insensitive) | 6
 []        | Character class           | 6
 .         | Dot (any character)       | 6
 //        | Regular expression        | 6
@@ -410,12 +457,11 @@ case that could be used to test that our grammar works properly.
       end
     end
-The key here is using the `root`
-[option](api/classes/Citrus/GrammarMethods.html#M000031) when performing the
-parse to specify the name of the rule at which the parse should start. In
-`test_number`, since `:number` was given the parse will start at that rule as if
-it were the root rule of the entire grammar. The ability to change the root rule
-on the fly like this enables easy unit testing of the entire grammar.
+The key here is using the `:root` option when performing the parse to specify
+the name of the rule at which the parse should start. In `test_number`, since
+`:number` was given the parse will start at that rule as if it were the root
+rule of the entire grammar. The ability to change the root rule on the fly like
+this enables easy unit testing of the entire grammar.
 Also note that because match objects are themselves strings, assertions may be
 made to test equality of match objects with string values.
@@ -424,9 +470,9 @@ made to test equality of match objects with string values.
 When a parse fails, a [ParseError](api/classes/Citrus/ParseError.html) object is
 generated which provides a wealth of information about exactly where the parse
-failed. Using this object, you could possibly provide some useful feedback to
-the user about why the input was bad. The following code demonstrates one way
-to do this.
+failed including the offset, line number, line text, and line offset. Using this
+object, you could possibly provide some useful feedback to the user about why
+the input was bad. The following code demonstrates one way to do this.
     def parse_some_stuff(stuff)
       match = StuffGrammar.parse(stuff)
@@ -435,17 +481,28 @@ to do this.
         [e.line_number, e.line_offset]
     end
-In addition to useful error objects, Citrus also includes a special file that
-should help grammar authors when debugging grammars. To get this extra
-functionality, simply `require 'citrus/debug'` instead of `require 'citrus'`
-when running your code.
-When debugging is enabled, you can visualize parse trees in the console as XML
-documents. This can help when determining which rules are generating which
-matches and how they are organized in the output. Also when debugging, each
-match object automatically records its offset in the original input, which can
-also be very helpful in keeping track of which offsets in the input generated
-which matches.
+In addition to useful error objects, Citrus also includes a means of visualizing
+match trees in the console via `Match#dump`. This can help when determining
+which rules are generating which matches and how they are organized in the
+match tree.
+# Extras
+Several files are included in the Citrus repository that make it easier to work
+with grammar files in various editors.
+## TextMate
+To install the Citrus [TextMate](http://macromates.com/) bundle, simply
+double-click on the `Citrus.tmbundle` file in the `extras` directory.
+## Vim
+To install the [Vim](http://www.vim.org/) scripts, copy the files in
+`extras/vim` to a directory in Vim's
+[runtimepath](http://vimdoc.sourceforge.net/htmldoc/options.html#\'runtimepath\').
 # Links

data/benchmark/after.dat ADDED Viewed

@@ -0,0 +1,192 @@
+12 0
+30 0
+35 0
+1749 19
+1962 9
+2383 20
+3728 29
+3919 30
+3952 30
+3995 30
+4063 30
+4325 40
+4527 39
+4570 39
+4607 50
+4654 40
+4679 39
+4774 40
+4968 40
+5059 50
+5383 39
+5915 49
+6109 50
+6122 49
+6218 50
+6332 50
+6681 59
+7440 60
+7530 60
+7605 70
+8155 60
+8402 69
+8420 70
+8617 80
+8635 70
+8841 79
+8843 79
+8852 69
+9151 80
+9271 80
+9521 80
+9525 80
+9566 80
+9584 80
+9642 70
+10138 89
+10181 80
+10225 80
+10338 80
+10449 89
+10629 90
+10763 89
+10817 89
+11059 90
+11062 90
+11215 89
+11698 99
+11891 99
+11945 100
+11956 100
+12018 100
+12053 100
+12178 99
+12283 100
+12326 109
+12430 99
+12438 99
+12572 99
+12638 99
+12687 99
+12703 109
+12896 109
+12922 109
+12996 99
+13137 110
+13211 129
+13462 109
+13477 109
+13576 109
+13577 120
+13584 110
+13605 109
+13631 109
+14216 120
+14237 120
+14260 130
+14367 119
+14371 120
+14741 120
+14893 120
+14910 120
+14917 129
+14977 130
+15049 119
+15191 130
+15382 129
+15618 129
+15623 130
+15629 129
+15856 129
+16496 130
+16512 159
+16956 129
+17074 140
+17237 139
+17371 150
+17568 149
+17945 140
+18147 149
+18343 150
+18417 160
+18823 159
+18970 149
+19285 170
+19333 160
+19500 160
+19548 150
+19634 160
+19673 160
+19689 179
+19909 169
+20054 160
+20107 169
+20248 169
+20580 169
+20744 169
+20806 189
+20954 169
+21034 179
+21187 179
+21303 189
+21450 169
+21626 179
+21931 179
+21950 169
+22359 189
+22626 190
+22638 180
+22772 189
+22885 190
+22897 190
+23114 190
+23242 189
+23428 189
+23452 209
+23495 199
+23499 189
+23558 190
+23744 190
+23881 219
+23945 189
+24361 210
+24501 210
+24642 199
+24672 200
+24694 210
+24706 210
+24931 210
+25065 210
+25140 199
+25402 210
+25702 209
+25743 209
+27139 230
+27316 240
+27333 229
+27414 220
+27771 230
+27798 229
+28583 240
+28906 239
+29025 240
+29209 240
+29272 239
+29273 250
+29359 240
+29577 240
+30886 259
+31170 250
+31593 259
+32460 269
+32486 259
+32630 269
+33010 269
+33137 289
+33142 280
+33739 280
+39880 319
+39940 339
+42952 350
+43227 350
+52855 439

data/benchmark/before.dat ADDED Viewed

@@ -0,0 +1,192 @@
+12 0
+30 0
+35 0
+1749 20
+1962 20
+2383 30
+3728 39
+3919 49
+3952 99
+3995 120
+4063 49
+4325 49
+4527 109
+4570 89
+4607 100
+4654 59
+4679 80
+4774 130
+4968 59
+5059 60
+5383 90
+5915 70
+6109 70
+6122 150
+6218 190
+6332 89
+6681 160
+7440 149
+7530 100
+7605 129
+8155 169
+8402 210
+8420 160
+8617 179
+8635 200
+8841 129
+8843 159
+8852 140
+9151 100
+9271 170
+9521 200
+9525 179
+9566 119
+9584 149
+9642 119
+10138 200
+10181 120
+10225 160
+10338 149
+10449 160
+10629 179
+10763 190
+10817 200
+11059 169
+11062 200
+11215 179
+11698 179
+11891 219
+11945 229
+11956 209
+12018 219
+12053 179
+12178 239
+12283 150
+12326 239
+12430 210
+12438 160
+12572 190
+12638 250
+12687 219
+12703 219
+12896 190
+12922 239
+12996 189
+13137 230
+13211 230
+13462 230
+13477 199
+13576 209
+13577 219
+13584 279
+13605 210
+13631 219
+14216 230
+14237 240
+14260 190
+14367 210
+14371 240
+14741 269
+14893 229
+14910 239
+14917 239
+14977 260
+15049 260
+15191 199
+15382 260
+15618 230
+15623 250
+15629 239
+15856 269
+16496 300
+16512 250
+16956 300
+17074 250
+17237 259
+17371 300
+17568 290
+17945 300
+18147 270
+18343 269
+18417 279
+18823 400
+18970 280
+19285 269
+19333 329
+19500 269
+19548 299
+19634 280
+19673 330
+19689 359
+19909 300
+20054 280
+20107 359
+20248 319
+20580 320
+20744 299
+20806 320
+20954 359
+21034 299
+21187 309
+21303 429
+21450 310
+21626 340
+21931 369
+21950 320
+22359 370
+22626 339
+22638 359
+22772 359
+22885 350
+22897 339
+23114 320
+23242 310
+23428 359
+23452 359
+23495 379
+23499 370
+23558 350
+23744 380
+23881 400
+23945 330
+24361 350
+24501 370
+24642 369
+24672 360
+24694 429
+24706 359
+24931 370
+25065 400
+25140 330
+25402 349
+25702 380
+25743 339
+27139 410
+27316 449
+27333 430
+27414 349
+27771 390
+27798 399
+28583 370
+28906 410
+29025 490
+29209 399
+29272 460
+29273 460
+29359 420
+29577 459
+30886 500
+31170 459
+31593 500
+32460 510
+32486 509
+32630 509
+33010 480
+33137 510
+33142 470
+33739 530
+39880 629
+39940 619
+42952 659
+43227 619
+52855 780