asciidoctor 0.0.7 → 0.0.9

data/Gemfile

@@ -0,0 +1,2 @@
+source :rubygems
+gemspec

data/README.asciidoc

@@ -1,7 +1,11 @@
 [float]
 Asciidoctor
 ===========
+:asciidoctor: http://github.com/erebor/asciidoctor
+:asciidoc: http://www.methods.co.nz/asciidoc/index.html
 :gitscm-next: https://github.com/github/gitscm-next
+:asciidoctorseed: https://github.com/github/gitscm-next/commits/master/lib/asciidoc.rb
+:templates: https://github.com/erebor/asciidoctor/blob/master/lib/asciidoctor/render_templates.rb
 :tilt: https://github.com/rtomayko/tilt
 :freesoftware: http://www.fsf.org/licensing/essays/free-sw.html
 :issues: https://github.com/erebor/asciidoctor/issues
@@ -11,11 +15,11 @@ Asciidoctor
 :pr: http://help.github.com/send-pull-requests/
 :license: https://github.com/erebor/asciidoctor/blob/master/LICENSE
 
-http://github.com/erebor/asciidoctor[Asciidoctor] is a pure-Ruby
-processor for turning
-http://www.methods.co.nz/asciidoc/index.html[AsciiDoc] documents or
-strings into HTML (and, eventually other formats as well...'stay
-tuned!').
+{asciidoctor}[Asciidoctor] is a pure-Ruby processor for turning
+{asciidoc}[AsciiDoc] documents or strings into HTML (and, eventually
+other formats as well...'stay tuned!').
+
+image::https://travis-ci.org/erebor/asciidoctor.png?branch=master["Build Status", link="https://travis-ci.org/erebor/asciidoctor"]
 
 Asciidoctor uses simple built-in ERB templates to style the output in
 a way that roughly matches the default HTML output of the native
@@ -28,14 +32,15 @@ know of any reason it shouldn't work with more exotic Ruby versions,
 and would welcome help in testing that out.
 
 The initial code from which Asciidoctor started was from the
-{gitscm-next}[Git SCM site repo].
+{gitscm-next}[Git SCM site repo]. Refer to commit history of
+{asciidoctorseed}[asciidoc.rb] to view the initial contributions.
 
 == Installation
 
-NOTE: This gem is very immature, and as yet only supports a small
-subset of AsciiDoc features. Thus, you should only use it if you have
-a high tolerance for bugs, failures, and generally bad and intemperate
-behavior.
+NOTE: This gem is very immature, and as yet only supports a small subset
+of AsciiDoc features (but it's getting better!). Thus, you should only
+use it if you have a high tolerance for bugs, failures, and generally
+bad and intemperate behavior.
 
 To install the gem:
 
@@ -61,27 +66,29 @@ To render an AsciiDoc-formatted string:
  doc = Asciidoctor::Document.new("*This* is it.")
  puts doc.render
 
-Asciidoctor allows you to override the default template used to render
-almost any individual AsciiDoc element. If you provide a directory of
-{tilt}[Tilt]-compatible templates, named in such a way that Asciidoctor
-can figure out which template goes with which element, Asciidoctor will
-use the templates in this directory instead of its built-in templates
-for any elements for which it finds a matching template. It will
-fallback to its default templates for everything else.
+Asciidoctor allows you to override the {templates}[built-in templates]
+used to render almost any individual AsciiDoc element. If you provide a
+directory of {tilt}[Tilt]-compatible templates, named in such a way that
+Asciidoctor can figure out which template goes with which element,
+Asciidoctor will use the templates in this directory instead of its
+built-in templates for any elements for which it finds a matching
+template. It will fallback to its default templates for everything else.
 
  doc = Asciidoctor::Document.new("*This* is it.", :template_dir => 'templates')
  puts doc.render
 
 The Document and Section templates should begin with `document.` and
-`section.`, respectively. The file extension will depend on which
-Tilt-compatible format you've chosen. For ERB, the template names would be
-`document.html.erb` and `section.html.erb`, for instance. For Haml, they
-would be `document.html.haml` and `section.html.haml`.
-
-Templates for specific elements, like a Paragraph or Anchor, would begin
-with `section_<element>.`. For instance, to override the default Paragraph
-template with an ERB template, put a file called `section_paragraph.html.erb`
-in the template directory you pass in to `Document.new`.
+`section.`, respectively. The file extension is used by Tilt to
+determine which view framework it will use to use to render the
+template. For instance, if you want to write the template in ERB, you'd
+name these two templates `document.html.erb` and `section.html.erb`. To
+use Haml, you'd name them `document.html.haml` and `section.html.haml`.
+
+Templates for block elements, like a Paragraph or Sidebar, would begin
+with `block_<style>.`. For instance, to override the default Paragraph
+template with an ERB template, put a file named
+`block_paragraph.html.erb` in the template directory you pass to the
+`Document` constructor using the `template_dir` option.
 
 For more usage examples, see the test suite.
 
@@ -154,3 +161,5 @@ support for that Ruby version may be dropped.
 
 Copyright (c) 2012 Ryan Waldron.
 See {license}[LICENSE] for details.
+
+// vim: tw=72

data/Rakefile

@@ -60,12 +60,15 @@ task :coverage do
   sh "open coverage/index.html"
 end
 
-require 'rdoc/task'
-Rake::RDocTask.new do |rdoc|
-  rdoc.rdoc_dir = 'rdoc'
-  rdoc.title = "#{name} #{version}"
-  rdoc.rdoc_files.include('README*')
-  rdoc.rdoc_files.include('lib/**/*.rb')
+begin
+  require 'rdoc/task'
+  Rake::RDocTask.new do |rdoc|
+    rdoc.rdoc_dir = 'rdoc'
+    rdoc.title = "#{name} #{version}"
+    rdoc.rdoc_files.include('README*')
+    rdoc.rdoc_files.include('lib/**/*.rb')
+  end
+rescue LoadError
 end
 
 desc "Open an irb session preloaded with this library"

data/asciidoctor.gemspec

@@ -13,8 +13,8 @@ Gem::Specification.new do |s|
   ## If your rubyforge_project name is different, then edit it and comment out
   ## the sub! line in the Rakefile
   s.name              = 'asciidoctor'
-  s.version           = '0.0.7'
-  s.date              = '2012-12-17'
+  s.version           = '0.0.9'
+  s.date              = '2013-01-16'
   s.rubyforge_project = 'asciidoctor'
 
   ## Make sure your summary is short. The description may be as long
@@ -25,7 +25,7 @@ Gem::Specification.new do |s|
   ## List the primary authors. If there are a bunch of authors, it's probably
   ## better to set the email to an email list or something. If you don't have
   ## a custom homepage, consider using your GitHub URL or the like.
-  s.authors  = ["Ryan Waldron", "Jeremy McAnally"]
+  s.authors  = ["Ryan Waldron", "Dan Allen", "Jeremy McAnally"]
   s.email    = 'rew@erebor.com'
   s.homepage = 'http://github.com/erebor/asciidoctor'
 
@@ -43,46 +43,62 @@ Gem::Specification.new do |s|
 
   ## List your runtime dependencies here. Runtime dependencies are those
   ## that are needed for an end user to actually USE your code.
-  s.add_dependency('json')
-  s.add_dependency('tilt')
 
   ## List your development dependencies here. Development dependencies are
   ## those that are only needed during development
+  s.add_development_dependency('coderay')
+  s.add_development_dependency('erubis')
+  s.add_development_dependency('htmlentities')
   s.add_development_dependency('mocha')
   s.add_development_dependency('nokogiri')
-  s.add_development_dependency('htmlentities')
   s.add_development_dependency('pending')
+  s.add_development_dependency('rake')
+  s.add_development_dependency('tilt')
 
   ## Leave this section as-is. It will be automatically generated from the
   ## contents of your Git repository via the gemspec task. DO NOT REMOVE
   ## THE MANIFEST COMMENTS, they are used as delimiters by the task.
   # = MANIFEST =
   s.files = %w[
+    Gemfile
     LICENSE
     README.asciidoc
     Rakefile
     asciidoctor.gemspec
     bin/asciidoctor
     lib/asciidoctor.rb
+    lib/asciidoctor/abstract_block.rb
+    lib/asciidoctor/abstract_node.rb
+    lib/asciidoctor/attribute_list.rb
+    lib/asciidoctor/backends/base_template.rb
+    lib/asciidoctor/backends/docbook45.rb
+    lib/asciidoctor/backends/html5.rb
     lib/asciidoctor/block.rb
+    lib/asciidoctor/callouts.rb
     lib/asciidoctor/debug.rb
     lib/asciidoctor/document.rb
     lib/asciidoctor/errors.rb
+    lib/asciidoctor/inline.rb
     lib/asciidoctor/lexer.rb
     lib/asciidoctor/list_item.rb
     lib/asciidoctor/reader.rb
-    lib/asciidoctor/render_templates.rb
     lib/asciidoctor/renderer.rb
     lib/asciidoctor/section.rb
-    lib/asciidoctor/string.rb
+    lib/asciidoctor/substituters.rb
+    lib/asciidoctor/table.rb
     lib/asciidoctor/version.rb
     noof.rb
     test/attributes_test.rb
+    test/blocks_test.rb
     test/document_test.rb
     test/fixtures/asciidoc.txt
     test/fixtures/asciidoc_index.txt
     test/fixtures/ascshort.txt
+    test/fixtures/dot.gif
+    test/fixtures/encoding.asciidoc
+    test/fixtures/include-file.asciidoc
     test/fixtures/list_elements.asciidoc
+    test/fixtures/tip.gif
     test/headers_test.rb
     test/lexer_test.rb
     test/links_test.rb
@@ -90,6 +106,9 @@ Gem::Specification.new do |s|
     test/paragraphs_test.rb
     test/preamble_test.rb
     test/reader_test.rb
+    test/renderer_test.rb
+    test/substitutions_test.rb
+    test/tables_test.rb
     test/test_helper.rb
     test/text_test.rb
   ]

data/bin/asciidoctor

@@ -1,4 +1,4 @@
-#!/bin/env ruby
+#!/usr/bin/env ruby
 
 $:.unshift File.dirname(__FILE__) + '/../lib'
 

data/lib/asciidoctor.rb

@@ -1,13 +1,11 @@
 require 'rubygems'
-require 'cgi'
-require 'erb'
-require 'tilt'
+require 'strscan'
 
 $:.unshift(File.dirname(__FILE__))
-$:.unshift(File.join(File.dirname(__FILE__), '..', 'vendor'))
+#$:.unshift(File.join(File.dirname(__FILE__), '..', 'vendor'))
 
 # Public: Methods for parsing Asciidoc input files and rendering documents
-# using erb templates.
+# using eRuby templates.
 #
 # Asciidoc documents comprise a header followed by zero or more sections.
 # Sections are composed of blocks of content.  For example:
@@ -47,27 +45,104 @@ $:.unshift(File.join(File.dirname(__FILE__), '..', 'vendor'))
 #     file.puts html
 #   end
 module Asciidoctor
+
+  module SafeMode
+
+    # A safe mode level that disables any of the security features enforced
+    # by Asciidoctor (Ruby is still subject to its own restrictions).
+    UNSAFE = 0;
+
+    # A safe mode level that closely parallels safe mode in AsciiDoc. This value
+    # prevents access to files which reside outside of the parent directory of
+    # the source file and disables any macro other than the include::[] macro.
+    SAFE = 1;
+
+    # A safe mode level that disallows the document from attempting to read files
+    # from the file system and including the contents of them into the document.
+    # This value disallows use of the include::[] macro and the embedding of
+    # binary content (data uri), stylesheets and JavaScripts referenced by the
+    # document. (Asciidoctor and trusted extensions may still be allowed to embed
+    # trusted content into the document). Since Asciidoctor is aiming for wide
+    # adoption, this value is the default and is recommended for server-side
+    # deployments.
+    SECURE = 10;
+
+    # A planned safe mode level that disallows the use of passthrough macros and
+    # prevents the document from setting any known attributes, in addition to all
+    # the security features of SafeMode::SECURE
+    #
+    # Please note that this level is not currently implemented (and therefore not
+    # enforced)!
+    #PARANOID = 100;
+
+  end
+
   # The default document type
   # Can influence markup generated by render templates
   DEFAULT_DOCTYPE = 'article'
 
-  LIST_CONTEXTS = [:ulist, :olist, :dlist]
+  # Backend determines the format of the rendered output, default to html5
+  DEFAULT_BACKEND = 'html5'
+
+  # Default page widths for calculating absolute widths
+  DEFAULT_PAGE_WIDTHS = {
+    'docbook' => 425
+  }
+
+  LIST_CONTEXTS = [:ulist, :olist, :dlist, :colist]
+
+  NESTABLE_LIST_CONTEXTS = [:ulist, :olist, :dlist]
+
+  ORDERED_LIST_STYLES = [:arabic, :loweralpha, :lowerroman, :upperalpha, :upperroman]
+
+  ORDERED_LIST_MARKER_PATTERNS = {
+    :arabic => /\d+[\.>]/,
+    :loweralpha => /[a-z]\./,
+    :upperalpha => /[A-Z]\./,
+    :lowerroman => /[ivx]+\)/,
+    :upperroman => /[IVX]+\)/
+  }
 
   LIST_CONTINUATION = '+'
 
+  BLANK_LINES_PATTERN = /^\s*\n/
+
+  LINE_FEED_ENTITY = '
' # or 

+
   REGEXP = {
-    # [[Foo]]  (also allows, say, [[[]] or [[[Foo[f]], but I don't think it is supposed to (TODO))
-    :anchor           => /^\[(\[.+\])\]\s*$/,
+    # [[Foo]]
+    :anchor           => /^\[\[([^\[\]]+)\]\]\s*$/,
 
     # Foowhatevs [[Bar]]
-    :anchor_embedded  => /^(.*)\[\[([^\]]+)\]\]\s*$/,
+    :anchor_embedded  => /^(.*?)\s*\[\[([^\[\]]+)\]\]\s*$/,
+
+    # [[ref]] (anywhere inline)
+    :anchor_macro     => /\\?\[\[([\w":].*?)\]\]/,
+
+    # matches any block delimiter:
+    #   open, listing, example, literal, comment, quote, sidebar, passthrough, table
+    # NOTE position the most common blocks towards the front of the pattern
+    :any_blk          => %r{^(?:\-\-|(?:\-|=|\.|/|_|\*|\+){4,}|[\|!]={3,})\s*$},
+
+    # optimization when scanning lines for blocks
+    # NOTE accessing the first element before calling ord is first Ruby 1.8.7 compat
+    :any_blk_ord      => %w(- = . / _ * + | !).map {|c| c[0].ord },
+
+    # :foo: bar
+    :attr_assign      => /^:([^:!]+):\s*(.*)\s*$/,
+
+    # {name?value}
+    :attr_conditional => /^\s*\{([^\?]+)\?\s*([^\}]+)\s*\}/,
 
     # +   Attribute values treat lines ending with ' +' as a continuation,
     #     not a line-break as elsewhere in the document, where this is
     #     a forced line break. This should be the same regexp as :line_break,
     #     below, but it gets its own entry because readability ftw, even
     #     though repeating regexps ftl.
-    :attr_continue    => /^(.*)[[:blank:]]\+[[:blank:]]*$/,
+    :attr_continue    => /^[[:blank:]]*(.*)[[:blank:]]\+[[:blank:]]*$/,
+
+    # :foo!:
+    :attr_delete      => /^:([^:]+)!:\s*$/,
 
     # An attribute list above a block element
     #
@@ -75,33 +150,60 @@ module Asciidoctor
     # [quote, Adam Smith, Wealth of Nations]
     # Or can have name/value pairs
     # [NOTE, caption="Good to know"]
-    :attr_list_blk    => /^\[(\w.*)\]$/,
+    # Can be defined by an attribute
+    # [{lead}]
+    :blk_attr_list    => /^\[([\w\{].*)\]$/,
+
+    # attribute list or anchor (indicates a paragraph break)
+    :attr_line        => /^\[([\w\{].*|\[[^\[\]]+\])\]$/,
+
+    # attribute reference
+    # {foo}
+    :attr_ref         => /(\\?)\{(\w|\w[\w\-]*\w)(\\?)\}/,
+
+    # The author info line the appears immediately following the document title
+    # John Doe <john@anonymous.com>
+    :author_info      => /^\s*([\w\-]+)(?: +([\w\-]+))?(?: +([\w\-]+))?(?: +<([^>]+)>)?\s*$/,
 
     # [[[Foo]]]  (does not suffer quite the same malady as :anchor, but almost. Allows [ but not ] in internal capture
-    :biblio           => /\[\[\[([^\]]+)\]\]\]/,
+    :biblio           => /\[\[\[([^\[\]]+)\]\]\]/,
+
+    # callout reference inside literal text
+    # <1>
+    # special characters will already be replaced, hence their use in the regex
+    :callout_render   => /\\?&lt;(\d+)&gt;/,
+    # ...but not while scanning
+    :callout_scan     => /\\?<(\d+)>/,
 
     # <1> Foo
-    :colist           => /^(\<\d+\>)\s*(.*)/,
+    :colist           => /^<?(\d+)> (.*)/,
 
     # ////
     # comment block
     # ////
-    :comment_blk      => /^\/{4,}\s*$/,
+    :comment_blk      => %r{^/{4,}\s*$},
 
     # // (and then whatever)
-    :comment          => /^\/\/([^\/]|$)/,
+    :comment          => %r{^//([^/].*|)$},
+
+    # 29
+    :digits           => /^\d+$/,
 
-    # foo::  ||  foo;;
-    # Should be followed by a definition line, e.g.,
+    # foo::  ||  foo::: || foo:::: || foo;;
+    # Should be followed by a definition, on the same line...
+    # foo:: That which precedes 'bar' (see also, <<bar>>)
+    # ...or on a separate line
     # foo::
-    #    That which precedes 'bar' (see also, bar)
-    :dlist            => /^\s*(?:\[\[([^\]]*)\]\])?(\w.*?)(:{2,4}|;;)(\s+(.*))?$/,
+    #   That which precedes 'bar' (see also, <<bar>>)
+    # The term may be an attribute reference
+    # {term_foo}:: {def_foo}
+    :dlist            => /^\s*(.*?)(:{2,4}|;;)(?:[[:blank:]]+(.*))?$/,
     :dlist_siblings   => {
                            # (?:.*?[^:])? - a non-capturing group which grabs longest sequence of characters that doesn't end w/ colon
-                           '::' => /^\s*(?:\[\[([^\]]*)\]\])?(\w(?:.*[^:])?)(::)(\s+(.*))?$/,
-                           ':::' => /^\s*(?:\[\[([^\]]*)\]\])?(\w(?:.*[^:])?)(:::)(\s+(.*))?$/,
-                           '::::' => /^\s*(?:\[\[([^\]]*)\]\])?(\w(?:.*[^:])?)(::::)(\s+(.*))?$/,
-                           ';;' => /^\s*(?:\[\[([^\]]*)\]\])?(\w.*)(;;)(\s+(.*))?$/
+                           '::' => /^\s*((?:.*[^:])?)(::)(?:[[:blank:]]+(.*))?$/,
+                           ':::' => /^\s*((?:.*[^:])?)(:::)(?:[[:blank:]]+(.*))?$/,
+                           '::::' => /^\s*((?:.*[^:])?)(::::)(?:[[:blank:]]+(.*))?$/,
+                           ';;' => /^\s*(.*)(;;)(?:[[:blank:]]+(.*))?$/
                          },
     # ====
     :example          => /^={4,}\s*$/,
@@ -109,23 +211,11 @@ module Asciidoctor
     # image::filename.png[Caption]
     :image_blk        => /^image::(\S+?)\[(.*?)\]$/,
 
-    # == Foo
-    # ^ yields a level 2 title
-    #
-    # == Foo ==
-    # ^ also yields a level 2 title
-    #
-    # both equivalent to this two-line version:
-    # Foo
-    # ~~~
-    #
-    # match[1] is the delimiter, whose length determines the level
-    # match[2] is the title itself
-    # match[3] is an optional repeat of the delimiter, which is dropped
-    :level_title      => /^(={1,5})\s+(\S.*?)\s*(\s\1)?$/,
+    # image:filename.png[Alt]
+    :image_macro      => /\\?image:([^\[]+)(?:\[([^\]]*)\])/,
 
-    # ======  || ------ || ~~~~~~ || ^^^^^^ || ++++++
-    :line             => /^([=\-~^\+])+\s*$/,
+    # whitespace at the beginning of the line
+    :leading_blanks   => /^([[:blank:]]*)/,
 
     # +   From the Asciidoc User Guide: "A plus character preceded by at
     #     least one space character at the end of a non-blank line forces
@@ -134,44 +224,138 @@ module Asciidoctor
     # +      (would not match because there's no space before +)
     #  +     (would match and capture '')
     # Foo +  (would and capture 'Foo')
-    :line_break       => /([[:blank:]])\+[[:blank:]]*$/,
+    :line_break       => /^(.*)[[:blank:]]\+[[:blank:]]*$/,
+
+    # inline link and some inline link macro
+    # FIXME revisit!
+    :link_inline      => %r{(^|link:|\s|>|&lt;|[\(\)\]])(\\?https?://[^\[ ]*[^\. \)\[])(?:\[((?:\\\]|[^\]])*?)\])?},
+
+    # inline link macro
+    # link:path[label]
+    :link_macro       => /\\?link:([^\[ ]+)(?:\[((?:\\\]|[^\]])*?)\])/,
 
     # ----
     :listing          => /^\-{4,}\s*$/,
 
     # ....
-    :lit_blk          => /^\.{4,}\s*$/,
+    :literal          => /^\.{4,}\s*$/,
 
     # <TAB>Foo  or one-or-more-spaces-or-tabs then whatever
-    :lit_par          => /^([ \t]+.*)$/,
-
-    # "Wooble"  ||  Wooble
-    :name             => /^(["A-Za-z].*)\s*$/,  # I believe this fails to require " chars to be paired (TODO)
+    :lit_par          => /^([[:blank:]]+.*)$/,
 
     # --
-    :oblock           => /^\-\-\s*$/,
+    :open_blk         => /^\-\-\s*$/,
+
+    # . Foo (up to 5 consecutive dots)
+    # 1. Foo (arabic, default)
+    # a. Foo (loweralpha)
+    # A. Foo (upperalpha)
+    # i. Foo (lowerroman)
+    # I. Foo (upperroman)
+    :olist            => /^\s*(\d+\.|[a-z]\.|[ivx]+\)|\.{1,5}) +(.*)$/i,
 
-    # 1.Foo  ||  1. Foo  ||  . Foo
-    :olist            => /^\s*(\d+\.|\. )(.*)$/,
+    # ++++
+    :pass             => /^\+{4,}\s*$/,
+
+    # inline passthrough macros
+    # +++text+++
+    # $$text$$
+    # pass:quotes[text]
+    :pass_macro       => /\\?(?:(\+{3}|\${2})(.*?)\1|pass:([a-z,]*)\[((?:\\\]|[^\]])*?)\])/m,
+
+    # passthrough macro allowed in value of attribute assignment
+    # pass:[text]
+    :pass_macro_basic => /^pass:([a-z,]*)\[(.*)\]$/,
+
+    # inline literal passthrough macro
+    # `text`
+    :pass_lit         => /(^|[^`\w])(\\?`([^`\s]|[^`\s].*?\S)`)(?![`\w])/m,
+
+    # placeholder for extracted passthrough text
+    :pass_placeholder => /\x0(\d+)\x0/,
 
     # ____
     :quote            => /^_{4,}\s*$/,
 
+    # The document revision info line the appears immediately following the
+    # document title author info line, if present
+    # v1.0, 2013-01-01: Ring in the new year release
+    :revision_info    => /^\s*(?:\D*(.*?),)?(?:\s*(.*?))(?:\s*:\s*(.*)\s*)?$/,
+
     # '''
     :ruler            => /^'{3,}\s*$/,
 
     # ****
     :sidebar_blk      => /^\*{4,}\s*$/,
 
-    #     and blah blah blah
-    # ^^^^  <--- whitespace
-    :starts_with_whitespace => /\s+(.+)\s+\+\s*$/,
+    # \' within a word
+    :single_quote_esc => /(\w)\\'(\w)/,
+    # an alternative if our backend generated single-quoted html/xml attributes
+    #:single_quote_esc => /(\w|=)\\'(\w)/,
+
+    # |===
+    # |table
+    # |===
+    :table            => /^\|={3,}\s*$/,
+
+    # !===
+    # !table
+    # !===
+    :table_nested     => /^!={3,}\s*$/,
+
+    # 1*h,2*,^3e
+    :table_colspec    => /^(?:(\d+)\*)?([<^>](?:\.[<^>]?)?|(?:[<^>]?\.)?[<^>])?(\d+)?([a-z])?$/,
+
+    # 2.3+<.>m
+    # TODO might want to use step-wise scan rather than this mega-regexp
+    :table_cellspec => {
+      :start => /^[[:blank:]]*(?:(\d+(?:\.\d*)?|(?:\d*\.)?\d+)([*+]))?([<^>](?:\.[<^>]?)?|(?:[<^>]?\.)?[<^>])?([a-z])?\|/,
+      :end => /[[:blank:]]+(?:(\d+(?:\.\d*)?|(?:\d*\.)?\d+)([*+]))?([<^>](?:\.[<^>]?)?|(?:[<^>]?\.)?[<^>])?([a-z])?$/
+    },
 
     # .Foo   but not  . Foo or ..Foo
-    :title            => /^\.([^\s\.].*)\s*$/,
+    :blk_title        => /^\.([^\s\.].*)\s*$/,
 
-    # * Foo  ||  - Foo
-    :ulist            => /^ \s* (- | \*{1,5}) \s+ (.*) $/x
+    # == Foo
+    # ^ yields a level 2 title
+    #
+    # == Foo ==
+    # ^ also yields a level 2 title
+    #
+    # both equivalent to this two-line version:
+    # Foo
+    # ~~~
+    #
+    # match[1] is the delimiter, whose length determines the level
+    # match[2] is the title itself
+    # match[3] is an optional repeat of the delimiter, which is dropped
+    :section_title     => /^(={1,5})\s+(\S.*?)\s*(?:\[\[([^\[]+)\]\]\s*)?(\s\1)?$/,
+
+    # does not begin with a dot and has at least one alphanumeric character
+    :section_name      => /^((?=.*\w+.*)[^\.].*?)\s*$/,
+
+    # ======  || ------ || ~~~~~~ || ^^^^^^ || ++++++
+    :section_underline => /^([=\-~^\+])+\s*$/,
+
+    # * Foo (up to 5 consecutive asterisks)
+    # - Foo
+    :ulist            => /^ \s* (- | \*{1,5}) \s+ (.*) $/x,
+
+    # inline xref macro
+    # <<id,reftext>> (special characters have already been escaped, hence the entity references)
+    # xref:id[reftext]
+    :xref_macro       => /\\?(?:&lt;&lt;([\w":].*?)&gt;&gt;|xref:([\w":].*?)\[(.*?)\])/m,
+
+    # ifdef::basebackend-html[]
+    # ifndef::theme[]
+    :ifdef_macro      => /^(ifdef|ifndef)::([^\[]+)\[\]/,
+
+    # endif::theme[]
+    # endif::basebackend-html[]
+    :endif_macro      => /^endif::/,
+
+    # include::chapter1.ad[]
+    :include_macro    => /^\\?include::([^\[]+)\[\]\s*\n?\z/
   }
 
   ADMONITION_STYLES = ['NOTE', 'TIP', 'IMPORTANT', 'WARNING', 'CAUTION']
@@ -184,18 +368,20 @@ module Asciidoctor
     'caret'      => '^',
     'asterisk'   => '*',
     'tilde'      => '~',
-    'litdd'      => '--',
-    'plus'       => '+',
+    'plus'       => '&#43;',
     'apostrophe' => '\'',
     'backslash'  => '\\',
     'backtick'   => '`',
     'empty'      => '',
     'sp'         => ' ',
+    'space'      => ' ',
     'two-colons' => '::',
     'two-semicolons' => ';;',
     'nbsp'       => '&#160;',
     'deg'        => '&#176;',
     'zwsp'       => '&#8203;',
+    'quot'       => '&#34;',
+    'apos'       => '&#39;',
     'lsquo'      => '&#8216;',
     'rsquo'      => '&#8217;',
     'ldquo'      => '&#8220;',
@@ -203,7 +389,7 @@ module Asciidoctor
     'wj'         => '&#8288;',
     'amp'        => '&',
     'lt'         => '<',
-    'gt'         => '>',
+    'gt'         => '>'
     }
   )
 
@@ -213,20 +399,122 @@ module Asciidoctor
     '&' => '&amp;'
   }
 
-  HTML_ELEMENTS = {
-    'br-asciidoctor' => '<br/>'
-  }
-
+  SPECIAL_CHARS_PATTERN = /[#{SPECIAL_CHARS.keys.join}]/
+
+  # unconstrained quotes:: can appear anywhere
+  # constrained quotes:: must be bordered by non-word characters
+  # NOTE these substituions are processed in the order they appear here and
+  # the order in which they are replaced is important
+  QUOTE_SUBS = [
+
+    # **strong**
+    [:strong, :unconstrained, /\\?(?:\[([^\]]+?)\])?\*\*(.+?)\*\*/m],
+
+    # *strong*
+    [:strong, :constrained, /(^|[^\w;:}])(?:\[([^\]]+?)\])?\*(\S|\S.*?\S)\*(?=\W|$)/m],
+
+    # ``double-quoted''
+    [:double, :constrained, /(^|[^\w;:}])(?:\[([^\]]+?)\])?``(\S|\S.*?\S)''(?=\W|$)/m],
+
+    # 'emphasis'
+    [:emphasis, :constrained, /(^|[^\w;:}])(?:\[([^\]]+?)\])?'(\S|\S.*?\S)'(?=\W|$)/m],
+
+    # `single-quoted'
+    [:single, :constrained, /(^|[^\w;:}])(?:\[([^\]]+?)\])?`(\S|\S.*?\S)'(?=\W|$)/m],
+
+    # ++monospaced++
+    [:monospaced, :unconstrained, /\\?(?:\[([^\]]+?)\])?\+\+(.+?)\+\+/m],
+
+    # +monospaced+
+    [:monospaced, :constrained, /(^|[^\w;:}])(?:\[([^\]]+?)\])?\+(\S|\S.*?\S)\+(?=\W|$)/m],
+
+    # __emphasis__
+    [:emphasis, :unconstrained, /\\?(?:\[([^\]]+?)\])?\_\_(.+?)\_\_/m],
+
+    # _emphasis_
+    [:emphasis, :constrained, /(^|[^\w;:}])(?:\[([^\]]+?)\])?_(\S|\S.*?\S)_(?=\W|$)/m],
+
+    # ##unquoted##
+    [:none, :unconstrained, /\\?(?:\[([^\]]+?)\])?##(.+?)##/m],
+
+    # #unquoted#
+    [:none, :constrained, /(^|[^\w;:}])(?:\[([^\]]+?)\])?#(\S|\S.*?\S)#(?=\W|$)/m],
+
+    # ^superscript^
+    [:superscript, :unconstrained, /\\?(?:\[([^\]]+?)\])?\^(.+?)\^/m],
+
+    # ~subscript~
+    [:subscript, :unconstrained, /\\?(?:\[([^\]]+?)\])?\~(.+?)\~/m]
+  ]
+
+  # NOTE in Ruby 1.8.7, [^\\] does not match start of line,
+  # so we need to match it explicitly
+  # order is significant
+  REPLACEMENTS = [
+    # (C)
+    [/(^|[^\\])\(C\)/, '\1&#169;'], 
+    # (R)
+    [/(^|[^\\])\(R\)/, '\1&#174;'],
+    # (TM)
+    [/(^|[^\\])\(TM\)/, '\1&#8482;'],
+    # foo -- bar
+    [/ -- /, '&#8201;&#8212;&#8201;'],
+    # foo--bar
+    [/(\w)--(?=\w)/, '\1&#8212;'],
+    # ellipsis
+    [/(^|[^\\])\.\.\./, '\1&#8230;'],
+    # single quotes
+    [/(\w)'(\w)/, '\1&#8217;\2'],
+    # escaped single quotes
+    [/(\w)\\'(\w)/, '\1\'\2'],
+    # and so on...
+    
+    # restore entities; TODO needs cleanup
+    [/&amp;(#[a-z0-9]+;)/i, '&\1']
+  ]
+
+  # Internal: Prior to invoking Kernel#require, issues a warning urging a
+  # manual require if running in a threaded environment.
+  #
+  # name  - the String name of the library to require.
+  #
+  # returns nothing
+  def self.require_library(name)
+    if Thread.list.size > 1
+      main_script = "#{name}.rb"
+      main_script_path_segment = "/#{name}.rb"
+      if !$LOADED_FEATURES.detect {|p| p == main_script || p.end_with?(main_script_path_segment) }.nil?
+        return
+      else
+        warn "WARN: asciidoctor is autoloading '#{name}' in threaded environment. " +
+           "The use of an explicit require '#{name}' statement is recommended."
+      end
+    end
+    require name
+    nil
+  end
+
+  # modules
+  require 'asciidoctor/substituters'
+
+  # abstract classes
+  require 'asciidoctor/abstract_node'
+  require 'asciidoctor/abstract_block'
+
+  # concrete classes
+  require 'asciidoctor/attribute_list'
+  require 'asciidoctor/backends/base_template'
   require 'asciidoctor/block'
+  require 'asciidoctor/callouts'
   require 'asciidoctor/debug'
   require 'asciidoctor/document'
   require 'asciidoctor/errors'
+  require 'asciidoctor/inline'
   require 'asciidoctor/lexer'
   require 'asciidoctor/list_item'
   require 'asciidoctor/reader'
-  require 'asciidoctor/render_templates'
   require 'asciidoctor/renderer'
   require 'asciidoctor/section'
-  require 'asciidoctor/string'
+  require 'asciidoctor/table'
   require 'asciidoctor/version'
 end