rattler 0.4.2 → 0.5.0
Sign up to get free protection for your applications and to get access to all the features.
- data/README.rdoc +9 -3
- data/features/command_line/dest_option.feature +1 -1
- data/features/command_line/lib_option.feature +2 -2
- data/features/command_line/output_option.feature +2 -2
- data/features/command_line/parser_generator.feature +3 -3
- data/features/grammar/any_character.feature +0 -4
- data/features/grammar/back_reference.feature +0 -4
- data/features/grammar/character_class.feature +0 -4
- data/features/grammar/comments.feature +0 -4
- data/features/grammar/e_symbol.feature +18 -0
- data/features/grammar/eof.feature +0 -4
- data/features/grammar/fail.feature +0 -4
- data/features/grammar/list_matching.feature +52 -3
- data/features/grammar/literal.feature +0 -4
- data/features/grammar/negative_lookahead.feature +0 -4
- data/features/grammar/negative_semantic_predicate.feature +99 -0
- data/features/grammar/node_action.feature +17 -4
- data/features/grammar/nonterminal.feature +3 -7
- data/features/grammar/one_or_more.feature +0 -4
- data/features/grammar/optional.feature +0 -4
- data/features/grammar/ordered_choice.feature +0 -4
- data/features/grammar/positive_lookahead.feature +0 -4
- data/features/grammar/positive_semantic_predicate.feature +99 -0
- data/features/grammar/posix_class.feature +3 -6
- data/features/grammar/repeat.feature +50 -0
- data/features/grammar/sequence.feature +0 -4
- data/features/grammar/side_effect.feature +81 -0
- data/features/grammar/skip_operator.feature +2 -5
- data/features/grammar/start_rule.feature +1 -5
- data/features/grammar/symantic_action.feature +9 -4
- data/features/grammar/token.feature +0 -4
- data/features/grammar/whitespace.feature +0 -4
- data/features/grammar/word_literal.feature +0 -4
- data/features/grammar/zero_or_more.feature +0 -4
- data/features/step_definitions/grammar_steps.rb +5 -1
- data/lib/rattler/back_end/optimizer.rb +1 -0
- data/lib/rattler/back_end/optimizer/reduce_repeat_match.rb +20 -8
- data/lib/rattler/back_end/optimizer/simplify_redundant_repeat.rb +10 -7
- data/lib/rattler/back_end/optimizer/specialize_repeat.rb +40 -0
- data/lib/rattler/back_end/parser_generator.rb +11 -6
- data/lib/rattler/back_end/parser_generator/delegating_generator.rb +20 -0
- data/lib/rattler/back_end/parser_generator/e_symbol_generator.rb +52 -0
- data/lib/rattler/back_end/parser_generator/eof_generator.rb +58 -0
- data/lib/rattler/back_end/parser_generator/expr_generator.rb +6 -2
- data/lib/rattler/back_end/parser_generator/gen_method_names.rb +15 -5
- data/lib/rattler/back_end/parser_generator/general_list_generator.rb +194 -0
- data/lib/rattler/back_end/parser_generator/general_repeat_generator.rb +177 -0
- data/lib/rattler/back_end/parser_generator/label_generator.rb +4 -1
- data/lib/rattler/back_end/parser_generator/list0_generating.rb +36 -0
- data/lib/rattler/back_end/parser_generator/list1_generating.rb +30 -0
- data/lib/rattler/back_end/parser_generator/list_generator.rb +16 -20
- data/lib/rattler/back_end/parser_generator/one_or_more_generating.rb +25 -0
- data/lib/rattler/back_end/parser_generator/{optional_generator.rb → optional_generating.rb} +66 -81
- data/lib/rattler/back_end/parser_generator/predicate_propogating.rb +4 -4
- data/lib/rattler/back_end/parser_generator/repeat_generator.rb +53 -0
- data/lib/rattler/back_end/parser_generator/skip_propogating.rb +2 -2
- data/lib/rattler/back_end/parser_generator/sub_generating.rb +16 -18
- data/lib/rattler/back_end/parser_generator/token_propogating.rb +1 -1
- data/lib/rattler/back_end/parser_generator/zero_or_more_generating.rb +31 -0
- data/lib/rattler/grammar/grammar_parser.rb +20 -0
- data/lib/rattler/grammar/metagrammar.rb +131 -13
- data/lib/rattler/grammar/rattler.rtlr +25 -12
- data/lib/rattler/parsers.rb +9 -5
- data/lib/rattler/parsers/assert_code.rb +31 -0
- data/lib/rattler/parsers/atomic.rb +19 -0
- data/lib/rattler/parsers/direct_action.rb +12 -4
- data/lib/rattler/parsers/disallow_code.rb +31 -0
- data/lib/rattler/parsers/dispatch_action.rb +3 -2
- data/lib/rattler/parsers/e_symbol.rb +47 -0
- data/lib/rattler/parsers/effect_code.rb +31 -0
- data/lib/rattler/parsers/eof.rb +6 -13
- data/lib/rattler/parsers/list_parser.rb +33 -14
- data/lib/rattler/parsers/match.rb +1 -6
- data/lib/rattler/parsers/parser_dsl.rb +78 -35
- data/lib/rattler/parsers/predicate.rb +1 -4
- data/lib/rattler/parsers/repeat.rb +76 -0
- data/lib/rattler/parsers/semantic_assert.rb +19 -0
- data/lib/rattler/parsers/semantic_disallow.rb +19 -0
- data/lib/rattler/parsers/side_effect.rb +19 -0
- data/lib/rattler/parsers/skip.rb +1 -9
- data/lib/rattler/parsers/token.rb +1 -6
- data/lib/rattler/util/graphviz/node_builder.rb +6 -3
- data/spec/rattler/back_end/assert_compiler_examples.rb +187 -0
- data/spec/rattler/back_end/direct_action_compiler_examples.rb +227 -0
- data/spec/rattler/back_end/disallow_compiler_examples.rb +187 -0
- data/spec/rattler/back_end/dispatch_action_compiler_examples.rb +225 -0
- data/spec/rattler/back_end/optimizer/reduce_repeat_match_spec.rb +45 -33
- data/spec/rattler/back_end/optimizer/simplify_redundant_repeat_spec.rb +32 -172
- data/spec/rattler/back_end/parser_generator/e_symbol_generator_spec.rb +149 -0
- data/spec/rattler/back_end/parser_generator/eof_generator_spec.rb +152 -0
- data/spec/rattler/back_end/parser_generator/label_generator_spec.rb +4 -4
- data/spec/rattler/back_end/parser_generator/list0_generator_examples.rb +322 -0
- data/spec/rattler/back_end/parser_generator/list1_generator_examples.rb +343 -0
- data/spec/rattler/back_end/parser_generator/list_generator_spec.rb +597 -140
- data/spec/rattler/back_end/parser_generator/one_or_more_generator_examples.rb +292 -0
- data/spec/rattler/back_end/parser_generator/optional_generator_examples.rb +233 -0
- data/spec/rattler/back_end/parser_generator/repeat_generator_spec.rb +281 -0
- data/spec/rattler/back_end/parser_generator/zero_or_more_generator_examples.rb +277 -0
- data/spec/rattler/back_end/semantic_assert_compiler_examples.rb +152 -0
- data/spec/rattler/back_end/semantic_disallow_compiler_examples.rb +152 -0
- data/spec/rattler/back_end/shared_compiler_examples.rb +106 -651
- data/spec/rattler/back_end/side_effect_compiler_examples.rb +227 -0
- data/spec/rattler/back_end/skip_compiler_examples.rb +161 -0
- data/spec/rattler/back_end/token_compiler_examples.rb +262 -0
- data/spec/rattler/grammar/grammar_parser_spec.rb +302 -23
- data/spec/rattler/parsers/apply_spec.rb +6 -0
- data/spec/rattler/parsers/assert_spec.rb +18 -7
- data/spec/rattler/parsers/back_reference_spec.rb +9 -0
- data/spec/rattler/parsers/choice_spec.rb +36 -21
- data/spec/rattler/parsers/direct_action_spec.rb +76 -36
- data/spec/rattler/parsers/disallow_spec.rb +18 -7
- data/spec/rattler/parsers/dispatch_action_spec.rb +128 -22
- data/spec/rattler/parsers/e_symbol_spec.rb +30 -0
- data/spec/rattler/parsers/eof_spec.rb +15 -6
- data/spec/rattler/parsers/label_spec.rb +15 -2
- data/spec/rattler/parsers/list_parser_spec.rb +187 -0
- data/spec/rattler/parsers/match_spec.rb +16 -7
- data/spec/rattler/parsers/parser_dsl_spec.rb +82 -23
- data/spec/rattler/parsers/repeat_spec.rb +233 -0
- data/spec/rattler/parsers/semantic_assert_spec.rb +83 -0
- data/spec/rattler/parsers/semantic_disallow_spec.rb +83 -0
- data/spec/rattler/parsers/sequence_spec.rb +34 -20
- data/spec/rattler/parsers/side_effect_spec.rb +214 -0
- data/spec/rattler/parsers/skip_spec.rb +17 -6
- data/spec/rattler_spec.rb +2 -0
- data/spec/support/compiler_spec_helper.rb +8 -0
- metadata +156 -447
- data/bin/rtlr.compiled.rbc +0 -201
- data/features/step_definitions/cli_steps.rbc +0 -149
- data/features/step_definitions/grammar_steps.rbc +0 -1211
- data/features/support/env.rbc +0 -389
- data/lib/rattler.rbc +0 -1231
- data/lib/rattler/back_end.rbc +0 -286
- data/lib/rattler/back_end/compiler.rbc +0 -1394
- data/lib/rattler/back_end/optimizer.rbc +0 -2000
- data/lib/rattler/back_end/optimizer/composite_reducing.rbc +0 -337
- data/lib/rattler/back_end/optimizer/flatten_choice.rbc +0 -443
- data/lib/rattler/back_end/optimizer/flatten_sequence.rbc +0 -827
- data/lib/rattler/back_end/optimizer/flattening.rbc +0 -570
- data/lib/rattler/back_end/optimizer/inline_regular_rules.rbc +0 -691
- data/lib/rattler/back_end/optimizer/join_match_capturing_sequence.rbc +0 -1299
- data/lib/rattler/back_end/optimizer/join_match_choice.rbc +0 -523
- data/lib/rattler/back_end/optimizer/join_match_matching_sequence.rbc +0 -619
- data/lib/rattler/back_end/optimizer/join_match_sequence.rbc +0 -174
- data/lib/rattler/back_end/optimizer/join_predicate_bare_match.rbc +0 -1505
- data/lib/rattler/back_end/optimizer/join_predicate_match.rbc +0 -174
- data/lib/rattler/back_end/optimizer/join_predicate_nested_match.rbc +0 -620
- data/lib/rattler/back_end/optimizer/join_predicate_or_bare_match.rbc +0 -1502
- data/lib/rattler/back_end/optimizer/join_predicate_or_match.rbc +0 -174
- data/lib/rattler/back_end/optimizer/join_predicate_or_nested_match.rbc +0 -616
- data/lib/rattler/back_end/optimizer/match_joining.rbc +0 -1454
- data/lib/rattler/back_end/optimizer/optimization.rbc +0 -1366
- data/lib/rattler/back_end/optimizer/optimization_context.rbc +0 -1386
- data/lib/rattler/back_end/optimizer/optimization_sequence.rbc +0 -520
- data/lib/rattler/back_end/optimizer/optimize_children.rbc +0 -793
- data/lib/rattler/back_end/optimizer/reduce_repeat_match.rbc +0 -788
- data/lib/rattler/back_end/optimizer/remove_meaningless_wrapper.rbc +0 -508
- data/lib/rattler/back_end/optimizer/simplify_redundant_repeat.rbc +0 -807
- data/lib/rattler/back_end/optimizer/simplify_token_match.rbc +0 -561
- data/lib/rattler/back_end/parser_generator.rbc +0 -1326
- data/lib/rattler/back_end/parser_generator/apply_generator.rbc +0 -2148
- data/lib/rattler/back_end/parser_generator/assert_generator.rbc +0 -1967
- data/lib/rattler/back_end/parser_generator/back_reference_generator.rbc +0 -1665
- data/lib/rattler/back_end/parser_generator/choice_generator.rbc +0 -2793
- data/lib/rattler/back_end/parser_generator/direct_action_generator.rbc +0 -1071
- data/lib/rattler/back_end/parser_generator/disallow_generator.rbc +0 -1967
- data/lib/rattler/back_end/parser_generator/dispatch_action_generator.rbc +0 -1071
- data/lib/rattler/back_end/parser_generator/expr_generator.rbc +0 -2295
- data/lib/rattler/back_end/parser_generator/fail_generator.rbc +0 -1216
- data/lib/rattler/back_end/parser_generator/gen_method_names.rbc +0 -296
- data/lib/rattler/back_end/parser_generator/group_match.rbc +0 -612
- data/lib/rattler/back_end/parser_generator/group_match_generator.rbc +0 -1647
- data/lib/rattler/back_end/parser_generator/label_generator.rbc +0 -1401
- data/lib/rattler/back_end/parser_generator/list1_generator.rb +0 -54
- data/lib/rattler/back_end/parser_generator/list1_generator.rbc +0 -1237
- data/lib/rattler/back_end/parser_generator/list_generating.rb +0 -71
- data/lib/rattler/back_end/parser_generator/list_generating.rbc +0 -1900
- data/lib/rattler/back_end/parser_generator/list_generator.rbc +0 -1068
- data/lib/rattler/back_end/parser_generator/match_generator.rbc +0 -1743
- data/lib/rattler/back_end/parser_generator/nested.rbc +0 -496
- data/lib/rattler/back_end/parser_generator/one_or_more_generator.rb +0 -56
- data/lib/rattler/back_end/parser_generator/one_or_more_generator.rbc +0 -1277
- data/lib/rattler/back_end/parser_generator/optional_generator.rbc +0 -2025
- data/lib/rattler/back_end/parser_generator/predicate_propogating.rbc +0 -648
- data/lib/rattler/back_end/parser_generator/repeat_generating.rb +0 -57
- data/lib/rattler/back_end/parser_generator/repeat_generating.rbc +0 -1549
- data/lib/rattler/back_end/parser_generator/rule_generator.rbc +0 -1239
- data/lib/rattler/back_end/parser_generator/rule_set_generator.rbc +0 -2641
- data/lib/rattler/back_end/parser_generator/sequence_generator.rbc +0 -4867
- data/lib/rattler/back_end/parser_generator/skip_generator.rbc +0 -1278
- data/lib/rattler/back_end/parser_generator/skip_propogating.rbc +0 -432
- data/lib/rattler/back_end/parser_generator/sub_generating.rbc +0 -2785
- data/lib/rattler/back_end/parser_generator/token_generator.rbc +0 -755
- data/lib/rattler/back_end/parser_generator/token_propogating.rbc +0 -324
- data/lib/rattler/back_end/parser_generator/top_level.rbc +0 -352
- data/lib/rattler/back_end/parser_generator/zero_or_more_generator.rb +0 -53
- data/lib/rattler/back_end/parser_generator/zero_or_more_generator.rbc +0 -1111
- data/lib/rattler/back_end/ruby_generator.rbc +0 -1841
- data/lib/rattler/grammar.rbc +0 -557
- data/lib/rattler/grammar/analysis.rbc +0 -1944
- data/lib/rattler/grammar/grammar.rbc +0 -1090
- data/lib/rattler/grammar/grammar_dsl.rbc +0 -1401
- data/lib/rattler/grammar/grammar_parser.rbc +0 -2096
- data/lib/rattler/grammar/metagrammar.rbc +0 -11014
- data/lib/rattler/parsers.rbc +0 -1006
- data/lib/rattler/parsers/action_code.rbc +0 -1577
- data/lib/rattler/parsers/apply.rbc +0 -562
- data/lib/rattler/parsers/assert.rbc +0 -378
- data/lib/rattler/parsers/back_reference.rbc +0 -871
- data/lib/rattler/parsers/choice.rbc +0 -607
- data/lib/rattler/parsers/combinator_parser.rbc +0 -612
- data/lib/rattler/parsers/combining.rbc +0 -570
- data/lib/rattler/parsers/direct_action.rbc +0 -1472
- data/lib/rattler/parsers/disallow.rbc +0 -379
- data/lib/rattler/parsers/dispatch_action.rbc +0 -2078
- data/lib/rattler/parsers/eof.rbc +0 -567
- data/lib/rattler/parsers/fail.rbc +0 -745
- data/lib/rattler/parsers/label.rbc +0 -749
- data/lib/rattler/parsers/list.rbc +0 -292
- data/lib/rattler/parsers/list0.rb +0 -26
- data/lib/rattler/parsers/list0.rbc +0 -292
- data/lib/rattler/parsers/list1.rb +0 -26
- data/lib/rattler/parsers/list1.rbc +0 -305
- data/lib/rattler/parsers/list_parser.rbc +0 -886
- data/lib/rattler/parsers/match.rbc +0 -621
- data/lib/rattler/parsers/node_code.rbc +0 -1064
- data/lib/rattler/parsers/one_or_more.rb +0 -47
- data/lib/rattler/parsers/one_or_more.rbc +0 -475
- data/lib/rattler/parsers/optional.rb +0 -42
- data/lib/rattler/parsers/optional.rbc +0 -450
- data/lib/rattler/parsers/parser.rbc +0 -994
- data/lib/rattler/parsers/parser_dsl.rbc +0 -4765
- data/lib/rattler/parsers/predicate.rbc +0 -490
- data/lib/rattler/parsers/rule.rbc +0 -777
- data/lib/rattler/parsers/rule_set.rbc +0 -1438
- data/lib/rattler/parsers/sequence.rbc +0 -1040
- data/lib/rattler/parsers/skip.rbc +0 -613
- data/lib/rattler/parsers/token.rbc +0 -457
- data/lib/rattler/parsers/zero_or_more.rb +0 -40
- data/lib/rattler/parsers/zero_or_more.rbc +0 -462
- data/lib/rattler/runner.rbc +0 -3813
- data/lib/rattler/runtime.rbc +0 -370
- data/lib/rattler/runtime/extended_packrat_parser.rbc +0 -2351
- data/lib/rattler/runtime/packrat_parser.rbc +0 -1390
- data/lib/rattler/runtime/parse_failure.rbc +0 -909
- data/lib/rattler/runtime/parse_node.rbc +0 -1007
- data/lib/rattler/runtime/parser.rbc +0 -2267
- data/lib/rattler/runtime/parser_helper.rbc +0 -337
- data/lib/rattler/runtime/recursive_descent_parser.rbc +0 -942
- data/lib/rattler/util.rbc +0 -286
- data/lib/rattler/util/graphviz.rbc +0 -327
- data/lib/rattler/util/graphviz/node_builder.rbc +0 -2207
- data/lib/rattler/util/line_counter.rbc +0 -988
- data/lib/rattler/util/node.rbc +0 -2393
- data/lib/rattler/util/parser_spec_helper.rbc +0 -2533
- data/spec/rattler/back_end/compiler_spec.rbc +0 -1187
- data/spec/rattler/back_end/optimizer/flatten_choice_spec.rbc +0 -2093
- data/spec/rattler/back_end/optimizer/flatten_sequence_spec.rbc +0 -4055
- data/spec/rattler/back_end/optimizer/inline_regular_rules_spec.rbc +0 -2345
- data/spec/rattler/back_end/optimizer/join_match_capturing_sequence_spec.rbc +0 -7006
- data/spec/rattler/back_end/optimizer/join_match_choice_spec.rbc +0 -3252
- data/spec/rattler/back_end/optimizer/join_match_matching_sequence_spec.rbc +0 -3681
- data/spec/rattler/back_end/optimizer/join_predicate_bare_match_spec.rbc +0 -5603
- data/spec/rattler/back_end/optimizer/join_predicate_nested_match_spec.rbc +0 -5232
- data/spec/rattler/back_end/optimizer/join_predicate_or_bare_match_spec.rbc +0 -4272
- data/spec/rattler/back_end/optimizer/join_predicate_or_nested_match_spec.rbc +0 -4342
- data/spec/rattler/back_end/optimizer/reduce_repeat_match_spec.rbc +0 -2960
- data/spec/rattler/back_end/optimizer/simplify_redundant_repeat_spec.rbc +0 -6929
- data/spec/rattler/back_end/optimizer/simplify_token_match_spec.rbc +0 -2327
- data/spec/rattler/back_end/optimizer_spec.rbc +0 -372
- data/spec/rattler/back_end/parser_generator/apply_generator_spec.rbc +0 -4710
- data/spec/rattler/back_end/parser_generator/assert_generator_spec.rbc +0 -4697
- data/spec/rattler/back_end/parser_generator/back_reference_generator_spec.rbc +0 -4855
- data/spec/rattler/back_end/parser_generator/choice_generator_spec.rbc +0 -5350
- data/spec/rattler/back_end/parser_generator/direct_action_generator_spec.rbc +0 -4737
- data/spec/rattler/back_end/parser_generator/disallow_generator_spec.rbc +0 -4697
- data/spec/rattler/back_end/parser_generator/dispatch_action_generator_spec.rbc +0 -4731
- data/spec/rattler/back_end/parser_generator/fail_generator_spec.rbc +0 -2115
- data/spec/rattler/back_end/parser_generator/group_match_generator_spec.rbc +0 -4195
- data/spec/rattler/back_end/parser_generator/group_match_spec.rbc +0 -1414
- data/spec/rattler/back_end/parser_generator/label_generator_spec.rbc +0 -4696
- data/spec/rattler/back_end/parser_generator/list1_generator_spec.rb +0 -309
- data/spec/rattler/back_end/parser_generator/list1_generator_spec.rbc +0 -5213
- data/spec/rattler/back_end/parser_generator/list_generator_spec.rbc +0 -5179
- data/spec/rattler/back_end/parser_generator/match_generator_spec.rbc +0 -4681
- data/spec/rattler/back_end/parser_generator/one_or_more_generator_spec.rb +0 -259
- data/spec/rattler/back_end/parser_generator/one_or_more_generator_spec.rbc +0 -4957
- data/spec/rattler/back_end/parser_generator/optional_generator_spec.rb +0 -190
- data/spec/rattler/back_end/parser_generator/optional_generator_spec.rbc +0 -4704
- data/spec/rattler/back_end/parser_generator/rule_generator_spec.rbc +0 -545
- data/spec/rattler/back_end/parser_generator/rule_set_generator_spec.rbc +0 -1110
- data/spec/rattler/back_end/parser_generator/sequence_generator_spec.rbc +0 -5453
- data/spec/rattler/back_end/parser_generator/skip_generator_spec.rbc +0 -4691
- data/spec/rattler/back_end/parser_generator/token_generator_spec.rbc +0 -4691
- data/spec/rattler/back_end/parser_generator/zero_or_more_generator_spec.rb +0 -244
- data/spec/rattler/back_end/parser_generator/zero_or_more_generator_spec.rbc +0 -4924
- data/spec/rattler/back_end/ruby_generator_spec.rbc +0 -2460
- data/spec/rattler/back_end/shared_compiler_examples.rbc +0 -41886
- data/spec/rattler/grammar/analysis_spec.rbc +0 -4365
- data/spec/rattler/grammar/grammar_parser_spec.rbc +0 -10344
- data/spec/rattler/grammar/grammar_spec.rbc +0 -1701
- data/spec/rattler/parsers/action_code_spec.rbc +0 -4674
- data/spec/rattler/parsers/apply_spec.rbc +0 -851
- data/spec/rattler/parsers/assert_spec.rbc +0 -752
- data/spec/rattler/parsers/back_reference_spec.rbc +0 -1002
- data/spec/rattler/parsers/choice_spec.rbc +0 -1696
- data/spec/rattler/parsers/combinator_parser_spec.rbc +0 -361
- data/spec/rattler/parsers/direct_action_spec.rbc +0 -5222
- data/spec/rattler/parsers/disallow_spec.rbc +0 -752
- data/spec/rattler/parsers/dispatch_action_spec.rbc +0 -3033
- data/spec/rattler/parsers/eof_spec.rbc +0 -728
- data/spec/rattler/parsers/fail_spec.rbc +0 -548
- data/spec/rattler/parsers/label_spec.rbc +0 -1091
- data/spec/rattler/parsers/list0_spec.rb +0 -82
- data/spec/rattler/parsers/list0_spec.rbc +0 -2308
- data/spec/rattler/parsers/list1_spec.rb +0 -82
- data/spec/rattler/parsers/list1_spec.rbc +0 -2287
- data/spec/rattler/parsers/list_spec.rbc +0 -2308
- data/spec/rattler/parsers/match_spec.rbc +0 -780
- data/spec/rattler/parsers/node_code_spec.rbc +0 -1754
- data/spec/rattler/parsers/one_or_more_spec.rb +0 -64
- data/spec/rattler/parsers/one_or_more_spec.rbc +0 -1698
- data/spec/rattler/parsers/optional_spec.rb +0 -64
- data/spec/rattler/parsers/optional_spec.rbc +0 -1717
- data/spec/rattler/parsers/parser_dsl_spec.rbc +0 -10150
- data/spec/rattler/parsers/rule_set_spec.rbc +0 -1060
- data/spec/rattler/parsers/sequence_spec.rbc +0 -2899
- data/spec/rattler/parsers/skip_spec.rbc +0 -753
- data/spec/rattler/parsers/token_spec.rbc +0 -1511
- data/spec/rattler/parsers/zero_or_more_spec.rb +0 -64
- data/spec/rattler/parsers/zero_or_more_spec.rbc +0 -1719
- data/spec/rattler/runtime/extended_packrat_parser_spec.rbc +0 -1341
- data/spec/rattler/runtime/packrat_parser_spec.rbc +0 -210
- data/spec/rattler/runtime/parse_failure_spec.rbc +0 -2244
- data/spec/rattler/runtime/parse_node_spec.rbc +0 -2008
- data/spec/rattler/runtime/parser_spec.rbc +0 -2757
- data/spec/rattler/runtime/recursive_descent_parser_spec.rbc +0 -210
- data/spec/rattler/runtime/shared_parser_examples.rbc +0 -2567
- data/spec/rattler/util/graphviz/node_builder_spec.rbc +0 -3439
- data/spec/rattler/util/line_counter_spec.rbc +0 -2272
- data/spec/rattler/util/node_spec.rbc +0 -15023
- data/spec/rattler_spec.rbc +0 -1591
- data/spec/spec_helper.rbc +0 -336
- data/spec/support/combinator_parser_spec_helper.rbc +0 -1284
- data/spec/support/compiler_spec_helper.rbc +0 -1941
- data/spec/support/parser_generator_spec_helper.rbc +0 -718
- data/spec/support/runtime_parser_spec_helper.rbc +0 -313
data/README.rdoc
CHANGED
@@ -4,6 +4,8 @@
|
|
4
4
|
|
5
5
|
=== Rattler - Ruby Tool for Language Recognition
|
6
6
|
|
7
|
+
Parsing for Ruby that's so easy it feels like cheating.
|
8
|
+
|
7
9
|
Rattler is a parser generator for Ruby based on parsing expression grammars.
|
8
10
|
Rattler's purpose is to make language design and recognition as simple and fun
|
9
11
|
as possible. To that end, the normal parsing expression grammar syntax is
|
@@ -14,7 +16,9 @@ A language syntax is specified in a grammar using the Rattler syntax. Parser
|
|
14
16
|
classes and modules can be generated statically using the "rtlr" command or
|
15
17
|
dynamically from strings.
|
16
18
|
|
17
|
-
|
19
|
+
The cucumber features are currently the best documentation.
|
20
|
+
|
21
|
+
{RDoc}[http://rubydoc.info/gems/rattler/0.5.0/frames]
|
18
22
|
|
19
23
|
== FEATURES:
|
20
24
|
|
@@ -30,9 +34,9 @@ dynamically from strings.
|
|
30
34
|
|
31
35
|
* Only strings can be parsed, so files have to be read completely before parsing
|
32
36
|
* There are many holes in the tests so there are undoubtedly many bugs
|
33
|
-
* Optimzations are not complete
|
34
37
|
* Support for composable grammars is not yet implemented
|
35
|
-
*
|
38
|
+
* Semantics of automatic whitespace skipping could be refined
|
39
|
+
* Incomplete documentation
|
36
40
|
|
37
41
|
== EXAMPLES:
|
38
42
|
|
@@ -50,6 +54,8 @@ dynamically from strings.
|
|
50
54
|
|
51
55
|
%whitespace (SPACE+ / comment)* {
|
52
56
|
|
57
|
+
json_text <- (object / array) EOF
|
58
|
+
|
53
59
|
object <- ~'{' members ~'}' { object _ }
|
54
60
|
|
55
61
|
members <- pair *, ','
|
@@ -12,7 +12,7 @@ Feature: --dest option
|
|
12
12
|
expr <- [01]*
|
13
13
|
"""
|
14
14
|
And a directory named "lib/my_examples"
|
15
|
-
When I run
|
15
|
+
When I run `rtlr --dest lib/my_examples binary.rtlr`
|
16
16
|
Then the output should contain "binary.rtlr -> lib/my_examples/binary_grammar.rb"
|
17
17
|
And the file "lib/my_examples/binary_grammar.rb" should contain:
|
18
18
|
"""
|
@@ -14,7 +14,7 @@ Feature: --lib option
|
|
14
14
|
expr <- [01]*
|
15
15
|
"""
|
16
16
|
And a directory named "lib"
|
17
|
-
When I run
|
17
|
+
When I run `rtlr --lib lib binary.rtlr`
|
18
18
|
Then the output should contain "binary.rtlr -> lib/binary_grammar.rb"
|
19
19
|
And the file "lib/binary_grammar.rb" should contain:
|
20
20
|
"""
|
@@ -29,7 +29,7 @@ Feature: --lib option
|
|
29
29
|
expr <- [01]*
|
30
30
|
"""
|
31
31
|
And a directory named "lib/examples"
|
32
|
-
When I run
|
32
|
+
When I run `rtlr --lib lib binary.rtlr`
|
33
33
|
Then the output should contain "binary.rtlr -> lib/examples/binary_grammar.rb"
|
34
34
|
And the file "lib/examples/binary_grammar.rb" should contain:
|
35
35
|
"""
|
@@ -11,7 +11,7 @@ Feature: --output option
|
|
11
11
|
expr <- [01]*
|
12
12
|
"""
|
13
13
|
And a directory named "lib"
|
14
|
-
When I run
|
14
|
+
When I run `rtlr --output my_grammar.rb binary.rtlr`
|
15
15
|
Then the output should contain "binary.rtlr -> my_grammar.rb"
|
16
16
|
And the file "my_grammar.rb" should contain:
|
17
17
|
"""
|
@@ -26,7 +26,7 @@ Feature: --output option
|
|
26
26
|
expr <- [01]*
|
27
27
|
"""
|
28
28
|
And a directory named "lib"
|
29
|
-
When I run
|
29
|
+
When I run `rtlr --output - binary.rtlr`
|
30
30
|
Then the output should contain:
|
31
31
|
"""
|
32
32
|
module BinaryGrammar
|
@@ -5,7 +5,7 @@ Feature: Parser Generator
|
|
5
5
|
|
6
6
|
@command-line
|
7
7
|
Scenario: Getting help
|
8
|
-
When I run
|
8
|
+
When I run `rtlr --help`
|
9
9
|
Then the output should contain:
|
10
10
|
"""
|
11
11
|
Usage: rtlr FILENAME [options]
|
@@ -27,7 +27,7 @@ Feature: Parser Generator
|
|
27
27
|
grammar BinaryGrammar
|
28
28
|
expr <- [01]*
|
29
29
|
"""
|
30
|
-
When I run
|
30
|
+
When I run `rtlr binary.rtlr --standalone`
|
31
31
|
Then the output should contain "binary.rtlr -> binary_grammar.rb"
|
32
32
|
And the file "binary_grammar.rb" should contain:
|
33
33
|
"""
|
@@ -59,7 +59,7 @@ Feature: Parser Generator
|
|
59
59
|
parser BinaryParser < Rattler::Runtime::PackratParser
|
60
60
|
expr <- [01]*
|
61
61
|
"""
|
62
|
-
When I run
|
62
|
+
When I run `rtlr binary.rtlr --standalone`
|
63
63
|
Then the output should contain "binary.rtlr -> binary_parser.rb"
|
64
64
|
And the file "binary_parser.rb" should contain:
|
65
65
|
"""
|
@@ -4,10 +4,6 @@ Feature: Back References
|
|
4
4
|
The reference refers to the previous parse result and it means to match the
|
5
5
|
same exact input again.
|
6
6
|
|
7
|
-
In order to reuse a previous parse result to match the same text again
|
8
|
-
As a language designer
|
9
|
-
I want to use back references in my grammar
|
10
|
-
|
11
7
|
Scenario Outline: Parsing
|
12
8
|
Given a grammar with:
|
13
9
|
"""
|
@@ -4,10 +4,6 @@ Feature: Character Classes
|
|
4
4
|
any single character in the set. They are identical to character classes in
|
5
5
|
Ruby's regular expressions.
|
6
6
|
|
7
|
-
In order to match specific sets of character
|
8
|
-
As a language designer
|
9
|
-
I want to use character classes in my grammar
|
10
|
-
|
11
7
|
Scenario Outline: Parsing
|
12
8
|
Given a grammar with:
|
13
9
|
"""
|
@@ -0,0 +1,18 @@
|
|
1
|
+
Feature: EOF Symbol
|
2
|
+
|
3
|
+
The symbol "E" always matches without consuming any input.
|
4
|
+
|
5
|
+
Background: A grammar with the EOF symbol
|
6
|
+
Given a grammar with:
|
7
|
+
"""
|
8
|
+
foo <- E
|
9
|
+
"""
|
10
|
+
|
11
|
+
Scenario: Before end-of-input
|
12
|
+
When I parse "foo"
|
13
|
+
Then the parse result should be true
|
14
|
+
|
15
|
+
Scenario: At end-of-input
|
16
|
+
When I parse "foo"
|
17
|
+
And the parse position is 3
|
18
|
+
Then the parse result should be true
|
@@ -3,10 +3,6 @@ Feature: EOF Symbol
|
|
3
3
|
The symbol "EOF" means to match the end of the source, i.e. match only if
|
4
4
|
there is no more input.
|
5
5
|
|
6
|
-
In order to match the end of the input
|
7
|
-
As a language designer
|
8
|
-
I want to use the EOF symbol in my grammar
|
9
|
-
|
10
6
|
Background: A grammar with the EOF symbol
|
11
7
|
Given a grammar with:
|
12
8
|
"""
|
@@ -6,10 +6,6 @@ Feature: Fail Expressions
|
|
6
6
|
as a string literal. Using "fail_rule" causes the entire rule to fail and
|
7
7
|
"fail_parse" causes the entire parse to fail.
|
8
8
|
|
9
|
-
In order to define my own failure messages
|
10
|
-
As a language designer
|
11
|
-
I want to use fail expressions in my grammar
|
12
|
-
|
13
9
|
Scenario: Fail-expression
|
14
10
|
Given a grammar with:
|
15
11
|
"""
|
@@ -4,9 +4,10 @@ Feature: List Matching
|
|
4
4
|
to match a list of terms with separators between them. "*," matches a list of
|
5
5
|
zero or more terms, "+," matches a list of one or more term.
|
6
6
|
|
7
|
-
|
8
|
-
|
9
|
-
|
7
|
+
Repetition counts or ranges can also be used for generalized list matching
|
8
|
+
expressions. A count by itself means to match the preceding expression exactly
|
9
|
+
that many times. A range can be written with ".." between the lower and upper
|
10
|
+
bounds. The upper bound is optional.
|
10
11
|
|
11
12
|
Scenario Outline: Zero or more terms
|
12
13
|
Given a grammar with:
|
@@ -40,6 +41,54 @@ Feature: List Matching
|
|
40
41
|
| "foo,bar," | ["foo", "bar"] | 7 |
|
41
42
|
| " " | FAIL | 0 |
|
42
43
|
|
44
|
+
Scenario Outline: Range
|
45
|
+
Given a grammar with:
|
46
|
+
"""
|
47
|
+
words <- @WORD+ 2..4, ','
|
48
|
+
"""
|
49
|
+
When I parse <input>
|
50
|
+
Then the parse result should be <result>
|
51
|
+
And the parse position should be <pos>
|
52
|
+
|
53
|
+
Examples:
|
54
|
+
| input | result | pos |
|
55
|
+
| "foo,bar" | ["foo", "bar"] | 7 |
|
56
|
+
| "foo,bar," | ["foo", "bar"] | 7 |
|
57
|
+
| "a,b,c,d,e" | ["a", "b", "c", "d"] | 7 |
|
58
|
+
| "foo" | FAIL | 0 |
|
59
|
+
|
60
|
+
Scenario Outline: Lower bound only
|
61
|
+
Given a grammar with:
|
62
|
+
"""
|
63
|
+
words <- @WORD+ 2.., ','
|
64
|
+
"""
|
65
|
+
When I parse <input>
|
66
|
+
Then the parse result should be <result>
|
67
|
+
And the parse position should be <pos>
|
68
|
+
|
69
|
+
Examples:
|
70
|
+
| input | result | pos |
|
71
|
+
| "foo,bar" | ["foo", "bar"] | 7 |
|
72
|
+
| "foo,bar," | ["foo", "bar"] | 7 |
|
73
|
+
| "a,b,c,d,e" | ["a","b","c","d","e"] | 9 |
|
74
|
+
| "foo" | FAIL | 0 |
|
75
|
+
|
76
|
+
Scenario Outline: Specific count
|
77
|
+
Given a grammar with:
|
78
|
+
"""
|
79
|
+
words <- @WORD+ 2, ','
|
80
|
+
"""
|
81
|
+
When I parse <input>
|
82
|
+
Then the parse result should be <result>
|
83
|
+
And the parse position should be <pos>
|
84
|
+
|
85
|
+
Examples:
|
86
|
+
| input | result | pos |
|
87
|
+
| "foo,bar" | ["foo", "bar"] | 7 |
|
88
|
+
| "foo,bar," | ["foo", "bar"] | 7 |
|
89
|
+
| "a,b,c,d,e" | ["a","b"] | 3 |
|
90
|
+
| "foo" | FAIL | 0 |
|
91
|
+
|
43
92
|
Scenario: Using whitespace
|
44
93
|
Given a grammar with:
|
45
94
|
"""
|
@@ -3,10 +3,6 @@ Feature: Literal Expressions
|
|
3
3
|
A string literal has the same syntax as a Ruby string literal and it means to
|
4
4
|
match if that exact text is next in the input.
|
5
5
|
|
6
|
-
In order to define symbols and keywords
|
7
|
-
As a language designer
|
8
|
-
I want to use string literals in my grammar
|
9
|
-
|
10
6
|
Scenario Outline: Normal String
|
11
7
|
Given a grammar with:
|
12
8
|
"""
|
@@ -4,10 +4,6 @@ Feature: Negative Lookahead Operator
|
|
4
4
|
would not match here and never consume any input. This is known as zero-width
|
5
5
|
negative lookahead.
|
6
6
|
|
7
|
-
In order to define zero-width negative lookahead expressions
|
8
|
-
As a language designer
|
9
|
-
I want to use the negative lookahead operator in my grammar
|
10
|
-
|
11
7
|
Scenario Outline: Parsing
|
12
8
|
Given a grammar with:
|
13
9
|
"""
|
@@ -0,0 +1,99 @@
|
|
1
|
+
Feature: Negative Symantic Predicates
|
2
|
+
|
3
|
+
A negative symantic predicate can be defined by placing a "!" in front of a
|
4
|
+
symantic action. If the action results in a false value the parse succeeds
|
5
|
+
with no parse result, otherwise the parse fails.
|
6
|
+
|
7
|
+
Scenario Outline: Single token
|
8
|
+
Given a grammar with:
|
9
|
+
"""
|
10
|
+
expr <- @DIGIT+ !{|s| s.to_i > 20 }
|
11
|
+
"""
|
12
|
+
When I parse <input>
|
13
|
+
Then the parse result should be <result>
|
14
|
+
|
15
|
+
Examples:
|
16
|
+
| input | result |
|
17
|
+
| "17" | "17" |
|
18
|
+
| "42" | FAIL |
|
19
|
+
|
20
|
+
Scenario Outline: Sequence
|
21
|
+
Given a grammar with:
|
22
|
+
"""
|
23
|
+
%whitespace SPACE*
|
24
|
+
expr <- @DIGIT+ @DIGIT+ !{|a,b| a.to_i < b.to_i }
|
25
|
+
"""
|
26
|
+
When I parse <input>
|
27
|
+
Then the parse result should be <result>
|
28
|
+
|
29
|
+
Examples:
|
30
|
+
| input | result |
|
31
|
+
| "16 3" | ["16", "3"] |
|
32
|
+
| "3 16" | FAIL |
|
33
|
+
|
34
|
+
Scenario Outline: Sequence with non-capturing expressions
|
35
|
+
Given a grammar with:
|
36
|
+
"""
|
37
|
+
expr <- ~"(" @DIGIT+ ~">" @DIGIT+ ~")" !{|a,b| a.to_i <= b.to_i }
|
38
|
+
"""
|
39
|
+
When I parse <input>
|
40
|
+
Then the parse result should be <result>
|
41
|
+
|
42
|
+
Examples:
|
43
|
+
| input | result |
|
44
|
+
| "(23>17)" | ["23", "17"] |
|
45
|
+
| "(17>23)" | FAIL |
|
46
|
+
|
47
|
+
Scenario Outline: Sequence with labeled expressions
|
48
|
+
Given a grammar with:
|
49
|
+
"""
|
50
|
+
expr <- "(" left:@DIGIT+ "<" right:@DIGIT+ ")" !{ left.to_i >= right.to_i }
|
51
|
+
"""
|
52
|
+
When I parse <input>
|
53
|
+
Then the parse result should be <result>
|
54
|
+
|
55
|
+
Examples:
|
56
|
+
| input | result |
|
57
|
+
| "(17<23)" | ["(", "17", "<", "23", ")"] |
|
58
|
+
| "(23<17)" | FAIL |
|
59
|
+
|
60
|
+
Scenario Outline: Single token using "_"
|
61
|
+
Given a grammar with:
|
62
|
+
"""
|
63
|
+
expr <- @DIGIT+ !{ _.to_i > 20 }
|
64
|
+
"""
|
65
|
+
When I parse <input>
|
66
|
+
Then the parse result should be <result>
|
67
|
+
|
68
|
+
Examples:
|
69
|
+
| input | result |
|
70
|
+
| "17" | "17" |
|
71
|
+
| "42" | FAIL |
|
72
|
+
|
73
|
+
Scenario Outline: Sequence using "_"
|
74
|
+
Given a grammar with:
|
75
|
+
"""
|
76
|
+
%whitespace SPACE*
|
77
|
+
expr <- @DIGIT+ @DIGIT+ !{ _[0].to_i < _[1].to_i }
|
78
|
+
"""
|
79
|
+
When I parse <input>
|
80
|
+
Then the parse result should be <result>
|
81
|
+
|
82
|
+
Examples:
|
83
|
+
| input | result |
|
84
|
+
| "16 3" | ["16", "3"] |
|
85
|
+
| "3 16" | FAIL |
|
86
|
+
|
87
|
+
Scenario Outline: Sequence using "_" as a parameter name
|
88
|
+
Given a grammar with:
|
89
|
+
"""
|
90
|
+
%whitespace SPACE*
|
91
|
+
expr <- @DIGIT+ @DIGIT+ !{|_| _.to_i < 20 }
|
92
|
+
"""
|
93
|
+
When I parse <input>
|
94
|
+
Then the parse result should be <result>
|
95
|
+
|
96
|
+
Examples:
|
97
|
+
| input | result |
|
98
|
+
| "42 7" | ["42", "7"] |
|
99
|
+
| "3 16" | FAIL |
|
@@ -7,10 +7,6 @@ Feature: Node Actions
|
|
7
7
|
Array and any attributes are included in a Hash. The action can optionally
|
8
8
|
include a name.
|
9
9
|
|
10
|
-
In order to create node objects as parse results
|
11
|
-
As a language designer
|
12
|
-
I want to use node actions in my grammar
|
13
|
-
|
14
10
|
Scenario: Explicit node class
|
15
11
|
Given a grammar with:
|
16
12
|
"""
|
@@ -42,3 +38,20 @@ Feature: Node Actions
|
|
42
38
|
"""
|
43
39
|
When I parse "42+23"
|
44
40
|
Then the parse result should be Expr[["42", "23"], {:name => "sum"}]
|
41
|
+
|
42
|
+
Scenario: Just a node name
|
43
|
+
Given a grammar with:
|
44
|
+
"""
|
45
|
+
integer <- @DIGIT+ <"num">
|
46
|
+
"""
|
47
|
+
When I parse "42"
|
48
|
+
Then the parse result should be Rattler::Runtime::ParseNode["42", {:name => "num"}]
|
49
|
+
|
50
|
+
Scenario: Lone action
|
51
|
+
Given a grammar with:
|
52
|
+
"""
|
53
|
+
default <- <>
|
54
|
+
"""
|
55
|
+
When I parse "anything"
|
56
|
+
Then the parse result should be Rattler::Runtime::ParseNode[]
|
57
|
+
And the parse position should be 0
|
@@ -1,12 +1,7 @@
|
|
1
1
|
Feature: Nonterminals
|
2
2
|
|
3
3
|
A nonterminal is an identifier that refers to a parse rule and means to parse
|
4
|
-
according to the rule.
|
5
|
-
used in.
|
6
|
-
|
7
|
-
In order to define context-free languages with recursive definitions
|
8
|
-
As a language designer
|
9
|
-
I want to use nonterminal expressions in my grammar
|
4
|
+
according to the rule. Nonterminals can be used to define recursive rules.
|
10
5
|
|
11
6
|
Scenario: Simple example
|
12
7
|
Given a grammar with:
|
@@ -25,4 +20,5 @@ Feature: Nonterminals
|
|
25
20
|
/ "a"
|
26
21
|
"""
|
27
22
|
When I parse "aaab"
|
28
|
-
Then the parse result should be [["a", ["a", "a"]], "b"]
|
23
|
+
Then the parse result should be [["a", ["a", "a"]], "b"]
|
24
|
+
|