typo 3.99.0

data/vendor/syntax/doc/manual/parts/0000.txt

@@ -0,0 +1,5 @@
+Syntax is, first and foremost, a lexical analysis framework. It supports pluggable syntax modules, and comes with modules for Ruby, XML, and YAML.
+
+What does it mean, a "lexical analysis framework"? It means you can use Syntax to take a body of text representing instructions in some syntax (like Ruby), and break that text into _tokens_.
+
+What is that good for? Well, a few things. You could concievably use it to count the number of classes in a system, or the number of lines of code per method. But Syntax was written specifically with _syntax highlighting_ in mind.

data/vendor/syntax/doc/manual/parts/0001.txt

@@ -0,0 +1,16 @@
+Because Syntax was written primarily to support syntax highlighting, that is going to be the task most clients of the library will use it for. So, to get you up and running quickly, converting, for example, Ruby code to HTML, consider the following:
+
+{{{lang=ruby,number=true,caption=Convert a Ruby script to HTML
+require 'syntax/convertors/html'
+
+convertor = Syntax::Convertors::HTML.for_syntax "ruby"
+html = convertor.convert( File.read( "program.rb" ) )
+
+puts html
+}}}
+
+The above script will convert a Ruby file called @program.rb@ to HTML, writing the HTML to stdout. HTML @span@ tags will be used in conjunction with "class" attributes to describe the various tokens--this allows you to use CSS to colorize the result.
+
+Incidentally, this entire document uses the Syntax library to colorize the Ruby code snippets!
+
+_Note: currently, only HTML conversion is supported, but it should be possible to create a convertor that takes the tokenized output and emits other formats, like LaTeX, PDF, or Rich Text._

data/vendor/syntax/doc/manual/parts/0002.txt

@@ -0,0 +1,24 @@
+Lexical analysis is (at least in part) the process of converting a body of text into _tokens_. It is also the process of identifying the _class_ of each token. The Syntax library refers to these classes as _groups_.
+
+Each syntax module may define its own groups. The Ruby module, for instance, defines 18 different groups:
+
+# normal: whitespace and the like. Basically, any text not grouped in any of the other groups.
+# comment: the delimiters and contents of a comment
+# keyword: any recognized keyword of the Ruby language
+# method: the name of a method when it is being declared
+# class: the name of a class when it is being declared
+# module: the name of a module when it is being declared
+# punct: any punctuation character
+# symbol: a Ruby symbol token
+# string: the contents (but not delimiters) of a string
+# char: a character literal (@?g@)
+# ident: an identifier, not otherwise recognized as a keyword
+# constant: a constant (beginning with an uppercase letter)
+# regex: the contents (but not delimiters) of a regular expression
+# number: a numeric literal
+# attribute: an instance variable
+# global: a global variable
+# expr: a nested (interpolated) expression within a string or regex
+# escape: an escape squence within a string or regex
+
+The only group common to all modules is @normal@. (When converting text to HTML, the name of the class used in a span will be the name of the corresponding group--this makes it straightforward to determine what CSS classes need to be defined.)

data/vendor/syntax/doc/manual/parts/0003.txt

@@ -0,0 +1,6 @@
+In addition to groups, each token has an associated _instruction_. For most tokens, this instruction is the symbol @:none@, meaning "do nothing special". However, there are two other instructions defined by the framework:
+
+* @:region_open@: begin a "region". This region is a sequence of tokens that are all nested inside the group of the current token. This is useful for strings and regular expressions, which may contain other kinds of tokens (like @expr@ and @escape@, in Ruby's case).
+* @:region_close@: close the current region.
+
+The HTML convertors uses these instructions to know whether to emit just an opened span tag, or a closed one, or whether to emit both. Other convertors may use these instructions in similar ways.

data/vendor/syntax/doc/manual/parts/0004.txt

@@ -0,0 +1,32 @@
+Lexical analysis is performed by obtaining a tokenizer of the appropriate class and calling @tokenize@ on it, passing the text to be tokenized. Each token is yielded to the associated block as it is discovered.
+
+{{{lang=ruby,number=true,caption=Tokenizing a Ruby script
+require 'syntax'
+
+tokenizer = Syntax.load "ruby"
+tokenizer.tokenize( File.read( "program.rb" ) ) do |token|
+  puts token
+  puts "  group: #{token.group}"
+  puts "  instruction: #{token.instruction}"
+end
+}}}
+
+If you need finer control over the process, you can use the lower-level API:
+
+{{{lang=ruby,number=true,caption=Tokenizing a Ruby script via step
+require 'syntax'
+
+tokenizer = Syntax.load "ruby"
+tokenizer.start( File.read( "program.rb" ) ) do |token|
+  puts token
+  puts "  group: #{token.group}"
+  puts "  instruction: #{token.instruction}"
+end
+
+tokenizer.step
+tokenizer.step
+...
+tokenizer.finish
+}}}
+
+In this case, each time @#step@ is invoked, it results in tokens being consumed and yielded to the block. However, a single step may result in multiple tokens being detected and yielded--there is no way to guarantee a single token at a time, unless the corresponding syntax module was written to work that way. For efficiency, the existing modules will yield multiple tokens when processing (for instance) strings, regular expressions, and heredocs.

data/vendor/syntax/doc/manual/parts/0005.txt

@@ -0,0 +1,18 @@
+Colorizing code has already been touched on briefly in the Introduction chapter (under "Quick Start"). To review, the process is a mere three steps:
+
+# Require the class for the type of the output you want (currently, only HTML is supported).
+# Obtain an instance of the convertor, for the syntax you wish to convert.
+# Call @#convert@ on that convertor, passing in the text you want to convert. The return value is the HTML representation of the colorized text.
+
+For example:
+
+{{{lang=ruby,number=true,caption=Colorizing a Ruby script
+# Step 1: require the HTML convertor
+require 'syntax/convertor/html'
+
+# Step 2: get an instance of the HTML convertor for the Ruby syntax
+convertor = Syntax::Convertor::HTML.for_syntax "ruby"
+
+# Step 3: convert the text to HTML
+puts convertor.convert( File.read( "program.rb" ) )
+}}}

data/vendor/syntax/doc/manual/parts/0006.txt

@@ -0,0 +1,62 @@
+To write your own custom highlighter module, you just need to:
+
+# inherit from @Syntax::Convertors::Abstract@
+# implement the @convert@ method
+
+You can use the @syntax/convertors/html.rb@ file as an example:
+
+{{{lang=ruby,number=true,caption=syntax/convertors/html.rb
+require 'syntax/convertors/abstract'
+
+module Syntax
+  module Convertors
+
+ # A simple class for converting a text into HTML.
+    class HTML < Abstract
+
+      # Converts the given text to HTML, using spans to represent token groups
+      # of any type but <tt>:normal</tt> (which is always unhighlighted). If
+      # +pre+ is +true+, the html is automatically wrapped in pre tags.
+      def convert( text, pre=true )
+        html = ""
+        html << "<pre>" if pre
+        regions = []
+        @tokenizer.tokenize( text ) do |tok|
+          value = html_escape(tok)
+   case tok.instruction
+            when :region_close then
+              regions.pop
+              html << "</span>"
+            when :region_open then
+              regions.push tok.group
+              html << "<span class=\"#{tok.group}\">#{value}"
+            else
+         if tok.group == ( regions.last || :normal )
+                html << value
+ else
+                html << "<span class=\"#{tok.group}\">#{value}</span>"
+              end
+          end
+        end
+        html << "</span>" while regions.pop
+        html << "</pre>" if pre
+        html
+      end
+
+      private
+
+        # Replaces some characters with their corresponding HTML entities.
+        def html_escape( string )
+          string.gsub( /&/, "&amp;" ).
+                 gsub( /</, "&lt;" ).
+                 gsub( />/, "&gt;" ).
+                 gsub( /"/, "&quot;" )
+        end
+
+    end
+
+  end
+end
+}}}
+
+Within the @#convert@ method, you will automatically have access to the @tokenizer@ instance variable--instantiated for you by the framework. The rest is up to you.

data/vendor/syntax/doc/manual/parts/0007.txt

@@ -0,0 +1,3 @@
+Because of Syntax's modular design, it is pretty straightforward to create your own syntax modules. The hardest part is doing the actual tokenizing of your chosen syntax.
+
+You can use the existing syntax modules to guide your own implementation if you wish, but note that each module will have to solve unique problems, because of the uniqueness of each different syntax.

data/vendor/syntax/doc/manual/parts/0008.txt

@@ -0,0 +1,5 @@
+Your new syntax implementation should extend @Syntax::Tokenizer@--this sets up a rich domain-specific language for scanning and tokenizing.
+
+Then, all you need to implement is the @#step@ method, which should take no parameters. Each invocation of @#step@ should extract at least one token, but may extract as many as you need it to. (Fewer is generally better, though.)
+
+Additionally, you may also implement @#setup@, to perform any initialization that should occur when tokenizing begins. Similarly, @#teardown@ may be implemented to do any cleanup that is needed.

data/vendor/syntax/doc/manual/parts/0009.txt

@@ -0,0 +1,29 @@
+Within a tokenizer, you have access to a rich set of methods for scanning the text. These methods correspond to the methods of the StringScanner class (i.e., @scan@, @scan_until@, @bol?@, etc.).
+
+Additionally, subgroups of recent regexps (used in @scan@, etc.) can be obtained via @subgroup@, which takes as a parameter the group you want to query.
+
+Tokenizing proceeds as follows:
+
+# Identify a token (using @#peek@, @#scan@, etc.).
+# Start a new token group (using @#start_group@, passing the symbol for the group and optionally any text you want to seed the group with).
+# Append text to the current group either with additional calls to @#start_group@ using the same group, or with @#append@ (which just takes the text to append to the current group)
+
+Instead of @#start_group@, you can also use @#start_region@, which begins a new region for the given group, and @#end_region@, which closes the region.
+
+Here is an example of a very, very simple tokenizer, that simple extracts words and numbers from the text:
+
+{{{lang=ruby,number=true,caption=Simple tokenizer
+require 'syntax'
+
+class SimpleTokenizer < Syntax::Tokenizer
+  def step
+    if digits = scan(/\d+/)
+      start_group :digits, digits
+    elsif words = scan(/\w+/)
+      start_group :words, words
+    else
+      start_group :normal, scan(/./)
+    end
+  end
+end
+}}}

data/vendor/syntax/doc/manual/parts/0010.txt

@@ -0,0 +1,21 @@
+Once you've written your new syntax module, you need to register it with the Syntax library so that it can be found and used by the framework. To do this, just add it to the @Syntax::SYNTAX@ hash:
+
+{{{lang=ruby,number=true,caption=Registering a new syntax
+require 'syntax'
+
+class SimpleTokenizer < Syntax::Tokenizer
+  ...
+end
+
+Syntax::SYNTAX['simple'] = SimpleTokenizer
+}}}
+
+That's it! Once you've done that, you can now use your syntax just by requiring the file that defines it, and then using the standard Syntax framework methods:
+
+{{{lang=ruby,number=true,caption=Using your new syntax
+require 'simple-tokenizer'
+require 'syntax/convertor/html'
+
+convertor = Syntax::Convertors::HTML.for_syntax "simple"
+puts convertor.convert( "hello 15 worlds!" )
+}}}

data/vendor/syntax/doc/manual/stylesheets/manual.css

@@ -0,0 +1,270 @@
+body {
+  background: #FFF;
+  font-family: sans-serif;
+  color: #000;
+  margin: 0px;
+}
+
+a {
+  color: #00F;
+  text-decoration: none;
+}
+
+a:hover {
+  text-decoration: underline;
+}
+
+.product-manual {
+  text-align: center;
+  font-size: x-large;
+  font-variant: small-caps;
+  font-weight: bold;
+  color: #005;
+}
+
+.copyright {
+  text-align: center;
+  font-size: small;
+  font-style: italic;
+  color: #005;
+  margin-top: 1in;
+}
+
+#banner {
+  background: #005;
+  color: #FFF;
+  border-bottom: 1px solid #000;
+  padding-top: 0.5em;
+  padding-bottom: 0.5em;
+  padding-left: 1em;
+  margin-bottom: 1em;
+}
+
+#banner .title {
+  font-size: x-large;
+}
+
+#banner .title:first-letter {
+  font-size: 250%;
+  font-weight: normal;
+  float: left;
+  margin-top: -7px;
+  margin-bottom: -7px;
+}
+
+#banner .product {
+  font-weight: bold;
+  color: #FF7;
+  letter-spacing: 0.5em;
+}
+
+#banner .tagline {
+  font-style: italic;
+  font-size: large;
+  letter-spacing: 0.1em;
+  color: #FFF;
+}
+
+#banner .info {
+  color: white;
+  font-size: small;
+  padding-right: 1em;
+}
+
+#content {
+  margin-left: 0cm;
+  margin-right: 1cm;
+}
+
+#navigation {
+  font-size: x-small;
+  border-right: 2px groove black;
+  border-top: 2px groove black;
+  margin-right: 1em;
+  width: 200px;
+  height: 100%;
+  background: #FFD;
+}
+
+#navigation ul, #navigation ol {
+  margin-left: 1.2em;
+  padding-left: 1.2em;
+}
+
+#navigation .license {
+  font-size: x-small;
+  text-align: center;
+  border-top: 1px dashed #005;
+  margin-top: 2em;
+  padding: 1em;
+}
+
+.section {
+  margin-bottom: 1em;
+}
+
+.section p {
+  text-align: justify;
+}
+
+#content h1 {
+  background: #005;
+  color: #FFF;
+  font-size: x-large;
+  font-weight: bold;
+  font-variant: small-caps;
+  padding: 0.5em;
+  border: 1px solid #000;
+  margin-top: 0px;
+  margin-bottom: 1em;
+}
+
+#content h2 {
+  background: #005;
+  color: #FFF;
+  font-size: large;
+  font-weight: bold;
+  font-variant: small-caps;
+  padding: 0.25em;
+  padding-left: 0.5em;
+  border: 1px solid #000;
+  margin-bottom: 1em;
+}
+
+#content h3 {
+  background: #FFD;
+  color: #000;
+  font-size: normal;
+  font-weight: bold;
+  font-variant: small-caps;
+  padding: 0.25em;
+  padding-left: 0.5em;
+  border: 1px dotted #000;
+  margin-bottom: 1em;
+}
+
+#content h4 {
+  background: #FFE;
+  color: #000;
+  font-size: normal;
+  font-weight: bold;
+  font-variant: small-caps;
+  padding: 0.25em;
+  padding-left: 0.5em;
+  border: 1px dotted #777;
+  margin-bottom: 1em;
+}
+
+#navigation h1 {
+  margin: 0px;
+  padding: 1em;
+  color: #005;
+  background: transparent;
+  font-weight: bold;
+  font-size: 150%;
+  font-variant: small-caps;
+  text-align: center;
+}
+
+#navigation h2 {
+  margin: 0px;
+  margin-bottom: 1em;
+  padding: 0.5em;
+  border-top: 1px dashed #005;
+  border-bottom: 1px dashed #005;
+  color: #005;
+  background: transparent;
+  font-weight: bold;
+  font-size: 125%;
+  font-variant: small-caps;
+  text-align: center;
+}
+
+table.list {
+  margin: 2em;
+  border: 1px solid black;
+  background: #FFD;
+  padding: 0px;
+  border-spacing: 0px;
+}
+
+table.list th {
+  border-bottom: 1px solid #005;
+  padding-bottom: 5px;
+  background: #008;
+  color: white;
+  padding: 0.5em;
+  text-align: left;
+}
+
+table.list td {
+  padding: 0.2em;
+  text-align: left;
+  vertical-align: top;
+  border-bottom: 1px solid;
+}
+
+.prevnext {
+  padding: 0.5em 1em 0.5em 1em;
+  background: #557;
+  color: #FFF;
+  font-size: small;
+  font-weight: bold;
+  border: 1px solid #000;
+}
+
+.prevnext a {
+  color: #FF0;
+}
+
+.top .prevnext {
+  margin: 0 0 1em 0;
+  text-align: left;
+}
+
+.bottom .prevnext {
+  margin: 1em 0 0 0;
+  text-align: right;
+}
+
+.figure {
+  border: 1px solid black;
+  line-height: normal;
+  background: #FFD;
+  margin: 2em;
+}
+
+.figure .caption {
+  background: #008;
+  color: white;
+  font-weight: bold;
+  font-size: small;
+  padding: 4px 24px 4px 8px;
+  margin-left: -4px;
+  border: 1px dotted #77F;
+}
+
+.figure table {
+  padding-top: 0.8em;
+  padding-bottom: 0.5em;
+}
+
+.figure .body {
+  padding-left: 1em;
+}
+
+.figure pre {
+  padding: 0px;
+  background: transparent;
+  border: none;
+  font-size: small;
+  font-family: monospace;
+}
+
+.figure .lineno {
+  text-align: right;
+  color: #B00;
+  font-family: monospace;
+  font-size: small;
+  padding-right: 1em;
+}