treetop 1.1.2 → 1.1.4

data/Rakefile

@@ -1,21 +1,21 @@
 dir = File.dirname(__FILE__)
-
 require 'rubygems'
 require 'rake'
+$LOAD_PATH.unshift(File.join(dir, 'vendor', 'rspec', 'lib'))
 require 'spec/rake/spectask'
+
 Gem::manage_gems
 require 'rake/gempackagetask'
-require 'rake/testtask'
 
-task :default => :test
+task :default => :spec
 
-Rake::TestTask.new do |t|
-  t.pattern = 'test/**/*test.rb'
+Spec::Rake::SpecTask.new do |t|
+  t.pattern = 'spec/**/*spec.rb'
 end
 
 gemspec = Gem::Specification.new do |s|
   s.name = "treetop"
-  s.version = "1.1.2"
+  s.version = "1.1.4"
   s.author = "Nathan Sobo"
   s.email = "nathansobo@gmail.com"
   s.homepage = "http://functionalform.blogspot.com"

data/doc/index.markdown

@@ -1,22 +1,27 @@
-#Overview
-Treetop blends cutting-edge parser research with the elegance of Ruby deliver the power of grammar-based syntactic analysis without its traditional conceptual and technical overhead. It is based on parsing expression grammars, and compiles clean, intuitive, and _composable_ language descriptions into _packrat parsers_ written in pure, readable Ruby.
+<p class="intro_text">
+	
+Treetop is a language for describing languages. Combining the elegance of Ruby with cutting-edge <em>parsing expression grammars</em>, it helps you analyze syntax with revolutionarily ease.	
 
-##Intuitive Grammar Specifications
-Treetop's packrat parsers use _memoization_ to make the time-complexity of backtracking a non-issue. This cuts the gordian knot of grammar design. There's no need to look ahead and no need to lex. Worry about the structure of the language, not the idiosyncrasies of the parser.
+</p>
 
-##Syntax-Oriented Programming
+
+#Intuitive Grammar Specifications
+Treetop's packrat parsers use _memoization_ to make the backtracking possible in linear time. This cuts the gordian knot of grammar design. There's no need to look ahead and no need to lex. Worry about the structure of the language, not the idiosyncrasies of the parser.
+
+#Syntax-Oriented Programming
 Rather than implementing semantic actions that construct parse trees, define methods on the trees that Treetop automatically constructs–and write this code directly inside the grammar.
 
-##Reusable, Composable Language Descriptions
+#Reusable, Composable Language Descriptions
 Break grammars into modules and compose them via Ruby's mixin semantics. Or combine grammars written by others in novel ways. Or extend existing grammars with your own syntactic constructs by overriding rules with access to a `super` keyword. Compositionally means your investment of time into grammar writing is secure–you can always extend and reuse your code.
 
-##Acknowledgements
+#Acknowledgements
 First, thank you to my employer Rob Mee of Pivotal Labs for funding a substantial portion of Treetop's development. He gets it.
 
 I'd also like to thank:
 
 * Damon McCormick for several hours of pair programming.
-* Nick Kallen for constant, well-considered feedback, and a few hours of programming to boot. 
+* Nick Kallen for lots of well-considered feedback and a few afternoons of programming.
+* Brian Takita for a night of pair programming.
 * Eliot Miranda for urging me rewrite as a compiler right away rather than putting it off.
 * Ryan Davis and Eric Hodel for hurting my code.
 * Dav Yaginuma for kicking me into action on my idea.

data/doc/screen.css

@@ -0,0 +1,52 @@
+body {
+	margin: 0;
+	padding: 0;
+	background: #666666;
+	font-family: "Lucida Grande", Geneva, Arial, Verdana, sans-serif;
+	color: #333333;
+}
+
+div {
+	margin: 0;
+	background-position: center;
+	background-repeat: none;	
+}
+
+h1 {
+	font-size: 120%;
+	margin-top: 1.5em;
+	margin-bottom: .5em;
+}
+
+div#top {
+	background-image: url("images/top_background.png");
+	height: 200px;
+	width: 100%;
+}
+
+div#middle {
+	padding-top: 10px;
+	background-image: url("images/middle_background.png");
+	background-repeat: repeat-y;
+}
+
+div#bottom {
+	background-image: url("images/bottom_background.png");
+	height: 42px;
+	margin-bottom: 30px;
+}
+
+div#content {
+	width: 481px;
+	margin: 0 auto 0 auto;
+	padding: 0 60px 25px 60px;
+}
+
+p {
+	line-height: 150%;
+}
+
+p.intro_text {
+	color: #C45900;
+	font-size: 115%;
+}

data/doc/site.html

@@ -0,0 +1,34 @@
+<html><head><link type="text/css" href="./screen.css" rel="stylesheet"></link></head><body><div id="top"></div><div id="middle"><div id="content"><p class="intro_text">
+
+Treetop is a language for describing languages. Combining the elegance of Ruby with cutting-edge <em>parsing expression grammars</em>, it helps you analyze syntax with revolutionarily ease.   
+
+</p>
+
+<h1>Intuitive Grammar Specifications</h1>
+
+<p>Treetop's packrat parsers use <em>memoization</em> to make the backtracking possible in linear time. This cuts the gordian knot of grammar design. There's no need to look ahead and no need to lex. Worry about the structure of the language, not the idiosyncrasies of the parser.</p>
+
+<h1>Syntax-Oriented Programming</h1>
+
+<p>Rather than implementing semantic actions that construct parse trees, define methods on the trees that Treetop automatically constructs–and write this code directly inside the grammar.</p>
+
+<h1>Reusable, Composable Language Descriptions</h1>
+
+<p>Break grammars into modules and compose them via Ruby's mixin semantics. Or combine grammars written by others in novel ways. Or extend existing grammars with your own syntactic constructs by overriding rules with access to a <code>super</code> keyword. Compositionally means your investment of time into grammar writing is secure–you can always extend and reuse your code.</p>
+
+<h1>Acknowledgements</h1>
+
+<p>First, thank you to my employer Rob Mee of Pivotal Labs for funding a substantial portion of Treetop's development. He gets it.</p>
+
+<p>I'd also like to thank:</p>
+
+<ul>
+<li>Damon McCormick for several hours of pair programming.</li>
+<li>Nick Kallen for lots of well-considered feedback and a few afternoons of programming.</li>
+<li>Brian Takita for a night of pair programming.</li>
+<li>Eliot Miranda for urging me rewrite as a compiler right away rather than putting it off.</li>
+<li>Ryan Davis and Eric Hodel for hurting my code.</li>
+<li>Dav Yaginuma for kicking me into action on my idea.</li>
+<li>Bryan Ford for his seminal work on Packrat Parsers.</li>
+<li>The editors of Lambda the Ultimate, where I discovered parsing expression grammars.</li>
+</ul></div></div><div id="bottom"></div></body></html>

data/doc/site.rb

@@ -0,0 +1,41 @@
+require 'rubygems'
+require 'erector'
+
+class Layout < Erector::Widget
+  def render
+    html do
+      head do
+        link :rel => "stylesheet",
+             :type => "text/css",
+             :href => "./screen.css"
+      end
+      
+      body do
+        div :id => 'top' do
+        end
+        div :id => 'middle' do
+          div :id => 'content' do
+            content
+          end
+        end
+        div :id => 'bottom' do
+          
+        end
+      end
+    end    
+  end
+  
+  def bluecloth(path)
+    File.open(path) do |file|
+      text BlueCloth.new(file.read).to_html
+    end
+  end
+end
+
+class Index < Layout
+  def content
+    bluecloth "index.markdown"
+  end
+end
+
+puts Index.new.to_s

data/examples/lambda_calculus/lambda_calculus.rb

@@ -1,5 +1,5 @@
 module LambdaCalculus
-  include ::Treetop::Runtime
+  include Treetop::Runtime
   
   def root
     result = _nt_program
@@ -746,6 +746,6 @@ module LambdaCalculus
   
 end
 
-class LambdaCalculusParser < ::Treetop::Runtime::CompiledParser
+class LambdaCalculusParser < Treetop::Runtime::CompiledParser
   include LambdaCalculus
 end

data/lib/treetop/bootstrap_gen_1_metagrammar.rb

@@ -0,0 +1,37 @@
+require 'rubygems'
+dir = File.dirname(__FILE__)
+
+# Loading trusted version of Treetop to compile the compiler
+trusted_treetop_path = Gem.source_index.find_name('treetop').last.full_gem_path
+require File.join(trusted_treetop_path, 'lib', 'treetop')
+
+# Relocating trusted version of Treetop to Trusted::Treetop
+Trusted = Module.new
+Trusted::Treetop = Treetop
+Object.send(:remove_const, :Treetop)
+Object.send(:remove_const, :TREETOP_ROOT)
+
+# Requiring version of Treetop that is under test
+require File.expand_path(File.join(dir, '..', 'treetop'))
+# Remove stale Metagrammar defined by the generated metagrammar.rb in system under test
+Treetop::Compiler.send(:remove_const, :Metagrammar)
+Treetop::Compiler.send(:remove_const, :MetagrammarParser)
+
+# Compile and evaluate freshly generated metagrammar source
+METAGRAMMAR_PATH = File.join(TREETOP_ROOT, 'compiler', 'metagrammar.treetop')
+compiled_metagrammar_source = Trusted::Treetop::Compiler::GrammarCompiler.new.ruby_source(METAGRAMMAR_PATH)
+Object.class_eval(compiled_metagrammar_source)
+
+# The compiler under test was compiled with the trusted grammar and therefore depends on its runtime
+# But the runtime in the global namespace is the new runtime. We therefore inject the tursted runtime
+# into the compiler so its parser functions correctly
+Treetop::Compiler::Metagrammar.module_eval do
+  include Trusted::Treetop::Runtime
+end
+
+Treetop::Compiler.send(:remove_const, :MetagrammarParser)
+class Treetop::Compiler::MetagrammarParser < Trusted::Treetop::Runtime::CompiledParser
+  include Treetop::Compiler::Metagrammar
+end
+
+$bootstrapped_gen_1_metagrammar = true

data/lib/treetop/compiler/metagrammar.rb

@@ -1,15 +1,10 @@
 module Treetop
   module Compiler
     module Metagrammar
-      include ::Treetop::Runtime
+      include Treetop::Runtime
       
       def root
-        result = _nt_treetop_file
-        if index == input.size
-          return result
-        else
-          return ParseFailure.new(input, index, result.nested_failures)
-        end
+        @root || :treetop_file
       end
       
       module TreetopFile0
@@ -505,7 +500,23 @@ module Treetop
             if r3.success?
               s4, nr4, i4 = [], [], index
               loop do
-                r5 = _nt_alphanumeric_char
+                i5, nr5 = index, []
+                r6 = _nt_alphanumeric_char
+                nr5 << r6
+                if r6.success?
+                  r5 = r6
+                  r6.update_nested_results(nr5)
+                else
+                  r7 = parse_terminal('::', SyntaxNode)
+                  nr5 << r7
+                  if r7.success?
+                    r5 = r7
+                    r7.update_nested_results(nr5)
+                  else
+                    self.index = i5
+                    r5 = ParseFailure.new(input, i5, nr5)
+                  end
+                end
                 nr4 << r5
                 if r5.success?
                   s4 << r5
@@ -2438,7 +2449,7 @@ module Treetop
       
     end
     
-    class MetagrammarParser < ::Treetop::Runtime::CompiledParser
+    class MetagrammarParser < Treetop::Runtime::CompiledParser
       include Metagrammar
     end
 

data/lib/treetop/compiler/metagrammar.treetop

@@ -47,7 +47,7 @@ module Treetop
       end
       
       rule include_declaration
-        'include' space [A-Z] alphanumeric_char* {
+        'include' space [A-Z] (alphanumeric_char / '::')* {
           def compile(builder)
             builder << text_value
           end

data/lib/treetop/compiler/node_classes/declaration_sequence.rb

@@ -5,13 +5,7 @@ module Treetop
       def compile(builder)
         unless rules.empty?
           builder.method_declaration("root") do
-            builder.assign 'result', rules.first.method_name
-            builder.if__ 'index == input.size' do
-              builder << 'return result'
-            end
-            builder.else_ do
-              builder << 'return ParseFailure.new(input, index, result.nested_failures)'
-            end
+            builder << "@root || :#{rules.first.name}"
           end
           builder.newline
         end

data/lib/treetop/compiler/node_classes/grammar.rb

@@ -6,12 +6,12 @@ module Treetop
         
         builder.module_declaration "#{grammar_name.text_value}" do
           builder.in(indent_level) # account for initial indentation of grammar declaration
-          builder << "include ::Treetop::Runtime"
+          builder << "include Treetop::Runtime"
           builder.newline
           declaration_sequence.compile(builder)
         end
         builder.newline
-        builder.class_declaration "#{parser_name} < ::Treetop::Runtime::CompiledParser" do
+        builder.class_declaration "#{parser_name} < Treetop::Runtime::CompiledParser" do
           builder << "include #{grammar_name.text_value}"
         end
       end

data/lib/treetop/runtime/compiled_parser.rb

@@ -3,11 +3,24 @@ module Treetop
     class CompiledParser
       include Treetop::Runtime
       
+      attr_accessor :consume_all_input
+      attr_writer :root
+      alias :consume_all_input? :consume_all_input
       attr_reader :input, :index
 
-      def parse(input)
+      def initialize
+        self.consume_all_input = true
+      end
+
+      def parse(input, options = {})
         prepare_to_parse(input)
-        return root
+        @index = options[:index] if options[:index]
+        result = send("_nt_#{root}")
+        if consume_all_input? && index != input.size
+          return ParseFailure.new(input, index, result.nested_failures)
+        else
+          return result
+        end
       end
       
       protected

metadata

@@ -1,10 +1,10 @@
 --- !ruby/object:Gem::Specification 
-rubygems_version: 0.9.2
+rubygems_version: 0.9.4
 specification_version: 1
 name: treetop
 version: !ruby/object:Gem::Version 
-  version: 1.1.2
-date: 2007-11-05 00:00:00 -08:00
+  version: 1.1.4
+date: 2007-12-02 00:00:00 -08:00
 summary: A Ruby-based text parsing and interpretation DSL
 require_paths: 
 - lib
@@ -31,79 +31,9 @@ authors:
 files: 
 - README
 - Rakefile
-- test/compilation_target
-- test/compiler
-- test/composition
-- test/parser
-- test/ruby_extensions
-- test/screw
-- test/test_helper.rb
-- test/compilation_target/target.rb
-- test/compilation_target/target.treetop
-- test/compilation_target/target_test.rb
-- test/compiler/and_predicate_test.rb
-- test/compiler/anything_symbol_test.rb
-- test/compiler/character_class_test.rb
-- test/compiler/choice_test.rb
-- test/compiler/circular_compilation_test.rb
-- test/compiler/failure_propagation_functional_test.rb
-- test/compiler/grammar_compiler_test.rb
-- test/compiler/grammar_test.rb
-- test/compiler/nonterminal_symbol_test.rb
-- test/compiler/not_predicate_test.rb
-- test/compiler/one_or_more_test.rb
-- test/compiler/optional_test.rb
-- test/compiler/parenthesized_expression_test.rb
-- test/compiler/parsing_rule_test.rb
-- test/compiler/sequence_test.rb
-- test/compiler/terminal_symbol_test.rb
-- test/compiler/test_grammar.treetop
-- test/compiler/zero_or_more_test.rb
-- test/composition/a.treetop
-- test/composition/b.treetop
-- test/composition/c.treetop
-- test/composition/d.treetop
-- test/composition/grammar_composition_test.rb
-- test/parser/syntax_node_test.rb
-- test/parser/terminal_parse_failure_test.rb
-- test/ruby_extensions/string_test.rb
-- test/screw/Rakefile
-- test/screw/unit
-- test/screw/unit.rb
-- test/screw/unit/assertion_failed_error.rb
-- test/screw/unit/assertions.rb
-- test/screw/unit/auto_runner.rb
-- test/screw/unit/collector
-- test/screw/unit/collector.rb
-- test/screw/unit/error.rb
-- test/screw/unit/failure.rb
-- test/screw/unit/sugar.rb
-- test/screw/unit/test_case.rb
-- test/screw/unit/test_result.rb
-- test/screw/unit/test_suite.rb
-- test/screw/unit/ui
-- test/screw/unit/ui.rb
-- test/screw/unit/util
-- test/screw/unit/util.rb
-- test/screw/unit/collector/dir.rb
-- test/screw/unit/collector/objectspace.rb
-- test/screw/unit/ui/console
-- test/screw/unit/ui/fox
-- test/screw/unit/ui/gtk
-- test/screw/unit/ui/gtk2
-- test/screw/unit/ui/test_runner_mediator.rb
-- test/screw/unit/ui/test_runner_utilities.rb
-- test/screw/unit/ui/tk
-- test/screw/unit/ui/console/test_runner.rb
-- test/screw/unit/ui/fox/test_runner.rb
-- test/screw/unit/ui/gtk/test_runner.rb
-- test/screw/unit/ui/gtk2/testrunner.rb
-- test/screw/unit/ui/tk/test_runner.rb
-- test/screw/unit/util/backtrace_filter.rb
-- test/screw/unit/util/observable.rb
-- test/screw/unit/util/proc_wrapper.rb
 - lib/treetop
 - lib/treetop.rb
+- lib/treetop/bootstrap_gen_1_metagrammar.rb
 - lib/treetop/compiler
 - lib/treetop/compiler.rb
 - lib/treetop/ruby_extensions
@@ -113,6 +43,7 @@ files:
 - lib/treetop/compiler/grammar_compiler.rb
 - lib/treetop/compiler/lexical_address_space.rb
 - lib/treetop/compiler/load_grammar.rb
+- lib/treetop/compiler/metagrammar.
 - lib/treetop/compiler/metagrammar.rb
 - lib/treetop/compiler/metagrammar.treetop
 - lib/treetop/compiler/node_classes
@@ -150,10 +81,17 @@ files:
 - doc/images
 - doc/index.markdown
 - doc/pitfalls_and_advanced_techniques.markdown
+- doc/screen.css
 - doc/semantic_interpretation.markdown
+- doc/site.html
+- doc/site.rb
 - doc/syntactic_recognition.markdown
 - doc/using_in_ruby.markdown
+- doc/images/bottom_background.png
+- doc/images/middle_backgound.png
+- doc/images/middle_background.png
 - doc/images/paren_language_output.png
+- doc/images/top_background.png
 - examples/lambda_calculus
 - examples/TALK
 - examples/lambda_calculus/arithmetic.treetop