asciidoctor 0.0.9 → 0.1.0

data/README.asciidoc

@@ -1,46 +1,53 @@
-[float]
 Asciidoctor
 ===========
-:asciidoctor: http://github.com/erebor/asciidoctor
-:asciidoc: http://www.methods.co.nz/asciidoc/index.html
+:asciidoctor: http://asciidoctor.org
+:asciidoctor-source: http://github.com/asciidoctor/asciidoctor
+:asciidoc: http://asciidoc.org
 :gitscm-next: https://github.com/github/gitscm-next
-:asciidoctorseed: https://github.com/github/gitscm-next/commits/master/lib/asciidoc.rb
-:templates: https://github.com/erebor/asciidoctor/blob/master/lib/asciidoctor/render_templates.rb
+:asciidoctor-seed: https://github.com/github/gitscm-next/commits/master/lib/asciidoc.rb
+:templates: https://github.com/asciidoctor/asciidoctor/blob/master/lib/asciidoctor/backends
 :tilt: https://github.com/rtomayko/tilt
 :freesoftware: http://www.fsf.org/licensing/essays/free-sw.html
-:issues: https://github.com/erebor/asciidoctor/issues
+:issues: https://github.com/asciidoctor/asciidoctor/issues
 :gist: https://gist.github.com
 :fork: http://help.github.com/fork-a-repo/
 :branch: http://learn.github.com/p/branching.html
 :pr: http://help.github.com/send-pull-requests/
-:license: https://github.com/erebor/asciidoctor/blob/master/LICENSE
+:license: https://github.com/asciidoctor/asciidoctor/blob/master/LICENSE
+:idprefix:
 
-{asciidoctor}[Asciidoctor] is a pure-Ruby processor for turning
-{asciidoc}[AsciiDoc] documents or strings into HTML (and, eventually
-other formats as well...'stay tuned!').
+{asciidoctor}[Asciidoctor] is a pure Ruby processor for converting
+{asciidoc}[AsciiDoc] source files and strings into HTML 5, DocBook 4.5
+and other formats. It's
+http://rubygems.org/gems/asciidoctor[published as a RubyGem] and is
+available under the MIT open source license.
 
-image::https://travis-ci.org/erebor/asciidoctor.png?branch=master["Build Status", link="https://travis-ci.org/erebor/asciidoctor"]
+image::https://travis-ci.org/asciidoctor/asciidoctor.png?branch=master["Build Status", link="https://travis-ci.org/asciidoctor/asciidoctor"]
 
-Asciidoctor uses simple built-in ERB templates to style the output in
-a way that roughly matches the default HTML output of the native
-Python processor. You can override this behavior by providing
-{tilt}[Tilt]-compatible templates. See the xref:usage[Usage section]
-for more details.
+Asciidoctor uses a set of built-in ERB templates to render the document
+to HTML 5 or DocBook 4.5. We've matched the rendered output as close as
+possible to the default output of the native Python processor. You can
+override this behavior by providing {tilt}[Tilt]-compatible templates.
+See the xref:usage[Usage section] for more details.
 
-Asciidoctor currently works with Ruby 1.8.7 and 1.9.3, though I don't
-know of any reason it shouldn't work with more exotic Ruby versions,
-and would welcome help in testing that out.
+Asciidoctor currently works (read as 'tested') with Ruby 1.8.7, Ruby
+1.9.3 and JRuby 1.7.2 (on Linux, Mac and Windows). We expect it will
+work with other versions of Ruby as well and would welcome help in
+testing it out.
 
-The initial code from which Asciidoctor started was from the
+The initial code from which Asciidoctor started emerged from the
 {gitscm-next}[Git SCM site repo]. Refer to commit history of
-{asciidoctorseed}[asciidoc.rb] to view the initial contributions.
+{asciidoctor-seed}[asciidoc.rb] to view the initial contributions and
+individual contributors.
+
+The source code can now be found in the {asciidoctor-source}[Asciidoctor
+source repository] on GitHub.
 
 == Installation
 
-NOTE: This gem is very immature, and as yet only supports a small subset
-of AsciiDoc features (but it's getting better!). Thus, you should only
-use it if you have a high tolerance for bugs, failures, and generally
-bad and intemperate behavior.
+NOTE: We are working hard to make Asciidoctor a drop-in replacement for
+AsciiDoc. We're very close, with nearly 600 tests that ensure
+compatibility. The march is on towards full compliance and beyond.
 
 To install the gem:
 
@@ -52,20 +59,91 @@ Or if you prefer bundler:
 
 == Usage
 
-To render a file containing AsciiDoc markup to HTML:
+Asciidoctor has both a command line interface (CLI) and an API. The
+CLI is a drop-in replacement for the `asciidoc.py` command from the
+python implementation. The API is intended for integration with other
+software projects and is suitable for server-side applications, such
+as Rails, Sinatra and GitHub.
+
+=== Command line interface (CLI)
+
+After installing the `asciidoctor` gem, the `asciidoctor` commandline
+interface should be available on your PATH after installing the gem.
+To invoke it, simply execute:
+
+ asciidoctor <asciidoc_file>
+
+This will use the built-in defaults for options and create a new file
+in the same directory as the input file, with the same base name, but
+with the .html extention.
+
+There are many other options available and full help is provided via:
+
+ asciidoctor --help
+
+, or in the http://asciidoctor.org/man/asciidoctor[man page].
+
+There is also an `asciidoctor-safe` command, which turns on safe mode
+by default, preventing access to files outside the parent directory of
+the source file. This mode is very similar to the safe mode of `asciidoc.py`.
+
+=== Ruby API
+
+To use Asciidoctor in your application, you first need to require the
+gem:
+
+ require 'asciidoctor'
+
+With that in place, you can start processing AsciiDoc documents.
+
+.Loading a document
+To parse a file into an `Asciidoctor::Document` object:
 
- lines = File.readlines('your_file.asciidoc')
- doc = Asciidoctor::Document.new(lines)
- html = doc.render
- File.open('your_file.html', 'w+') do |file|
-   file.puts html
- end
+ doc = Asciidoctor.load_file('your_file.asciidoc')
 
+You can get information about the document:
+
+ puts doc.doctitle
+ puts doc.attributes
+
+More than likely, you want to just render the document.
+
+.Rendering files
+To render a file containing AsciiDoc markup to HTML 5:
+
+ Asciidoctor.render_file('your_file.asciidoc', :in_place => true)
+
+The command will output to the file `your_file.html` in the same
+directory. You can render the file to DocBook 4.5 by setting the
+`backend` attribute to 'docbook':
+
+ Asciidoctor.render_file('your_file.asciidoc', :in_place => true,
+   :attributes => {'backend' => 'docbook'})
+
+The command will output to the file `your_file.xml` in the same
+directory. (If you're on Linux, you can view the file using yelp).
+
+.Rendering strings
 To render an AsciiDoc-formatted string:
 
- doc = Asciidoctor::Document.new("*This* is it.")
- puts doc.render
+ puts Asciidoctor.render('*This* is it.')
+
+When rendering a string, the header and footer are excluded by default
+to make Asciidoctor consistent with other lightweight markup engines
+like Markdown. If you want the header and footer, just declare it as
+an option:
+
+ puts Asciidoctor.render('*This* is it.', :header_footer => true)
+
+Now you'll get a full HTML 5 file. As before, you can also produce
+DocBook 4.5:
 
+ puts Asciidoctor.render('*This* is it.', :header_footer => true,
+   :attributes => {'backend' => 'docbook'})
+
+If you don't like the output you see, you can change it. Any of it!
+
+.Custom templates
 Asciidoctor allows you to override the {templates}[built-in templates]
 used to render almost any individual AsciiDoc element. If you provide a
 directory of {tilt}[Tilt]-compatible templates, named in such a way that
@@ -74,8 +152,8 @@ Asciidoctor will use the templates in this directory instead of its
 built-in templates for any elements for which it finds a matching
 template. It will fallback to its default templates for everything else.
 
- doc = Asciidoctor::Document.new("*This* is it.", :template_dir => 'templates')
- puts doc.render
+ puts Asciidoctor.render('*This* is it.', :header_footer => true,
+   :template_dir => 'templates')
 
 The Document and Section templates should begin with `document.` and
 `section.`, respectively. The file extension is used by Tilt to
@@ -90,7 +168,47 @@ template with an ERB template, put a file named
 `block_paragraph.html.erb` in the template directory you pass to the
 `Document` constructor using the `template_dir` option.
 
-For more usage examples, see the test suite.
+For more usage examples, see the (massive) test suite.
+
+== Differences from AsciiDoc
+
+While Asciidoctor aims to be compliant with the AsciiDoc syntax, there are some differences which are important to keep in mind. In some cases, it's to enforce a rule we believe is too lax or ambiguous in AsciiDoc. In other cases, it's a tradeoff for speed, smarter processing or a feature we just haven't yet implemented. (You'll also notice that Asciidoctor is about 20x faster than AsciiDoc).
+
+Here are the known cases where Asciidoctor differs from AsciiDoc:
+
+* In Asciidoctor, safe mode is on by default when using the API (safe mode level SECURE),
+* Asciidoctor safe mode is even more safe than AsciiDoc's safe mode
+* Asciidoctor enforces symmetric block delimiters (the length of start and end delimiters for a block must match)
+* Section title underlines must be within +/- 1 of the length of the title (AsciiDoc is +/- 3)
+* Asciidoctor's default HTML backend matches AsciiDoc's HTML 5 backend (whereas XHTML 1.1 is the default HTML backend in AsciiDoc)
+* Asciidoctor handles inline anchors more cleanly
+** AsciiDoc adds an `<a>` tag in the line and that markup gets caught in the generated id
+** Asciidoctor promotes the id of the anchor as the section id
+* Asciidoctor strips XML entities from the section title before generating the id (makes for cleaner section ids)
+* Asciidoctor use `<tt>` instead of `<span class="monospace">` around inline literal text in the HTML backend
+* Asciidoctor is much more lenient about attribute list parsing (double quotes are rarely needed)
+* Asciidoctor creates xref labels using the text from the linked section title when rendering HTML to match how DocBook works
+* Asciidoctor allows commas to be used in xref labels, whereas AsciiDoc cuts off the label at the location of the first comma
+* Asciidoctor removes indentation for non-literal paragraphs in a list item
+** In general, Asciidoctor handles whitespace much more intelligently
+* In Asciidoctor, a ruler can have attributes
+* Asciidoctor skips over line comments in tables, whereas AsciiDoc does not
+* Asciidoctor uses its own API rather than a command line invocation to handle table cells that have AsciiDoc content
+* Asciidoctor supports resolving variables from parent document in table cells with AsciiDoc content
+* AsciiDoc doesn't carry over the doctype attribute passed from the commandline when rendering AsciiDoc content cells, whereas Asciidoctor does
+* Asciidoctor strips the file extension from the target image when generating alt text if no alt text is provided
+* Asciidoctor reifies the toc in the header of the document instead of relying on JavaScript to create it
+* Asciidoctor is nice about using a section title syntax inside a delimited block by simply ignoring it (AsciiDoc issues warnings)
+* Asciidoctor honors the alternate style name "discrete" for a floating title (i.e., [discrete])
+* Asciidoctor supports syntax highlighting of listing or literal blocks that have the "source" style out of the box
+** Asciidoctor honors the source-highlighter values `coderay` and `highlightjs`, using CodeRay or highlight.js, respectively
+** Asciidoctor does not currently support Pygments for source highlighting
+* Asciidoctor sets these additional intrinsic attributes
+`asciidoctor`:: indicates Asciidoctor is being used; useful for conditional processing
+`asciidoctor-version`:: indicates which version of Asciidoctor is in use
+* Asciidoctor does not support deprecated tables (you don't want them anyway)
+
+If there's a difference you don't see in this list, check the {issues}[issue tracker] to see if it's an outstanding feature, or file an issue to report the difference.
 
 == Contributing
 
@@ -104,8 +222,11 @@ Here are some ways *you* can contribute:
 * by suggesting new features
 * by writing or editing documentation
 * by writing specifications
-* by writing code ('No patch is too small.' Fix typos, add comments,
-  clean up inconsistent whitespace, etc.)
+* by writing code -- 'No patch is too small.'
+** fix typos
+** add comments
+** clean up inconsistent whitespace
+** write tests!
 * by refactoring code
 * by fixing {issues}[issues]
 * by reviewing patches
@@ -138,13 +259,14 @@ An ideal bug report would include a pull request with failing specs.
 . Add, commit, and push your changes.
 . {pr}[Submit a pull request].
 
-
 == Supported Ruby Versions
 
 This library aims to support the following Ruby implementations:
 
 * Ruby 1.8.7
 * Ruby 1.9.3
+* JRuby 1.7.2
+* Rubinius 1.2.4
 
 If something doesn't work on one of these interpreters, it should be
 considered a bug.
@@ -159,7 +281,7 @@ support for that Ruby version may be dropped.
 
 == Copyright
 
-Copyright (c) 2012 Ryan Waldron.
+Copyright (C) 2012 Ryan Waldron.
 See {license}[LICENSE] for details.
 
 // vim: tw=72

data/Rakefile

@@ -49,6 +49,7 @@ require 'rake/testtask'
 Rake::TestTask.new(:test) do |test|
   test.libs << 'lib' << 'test'
   test.pattern = 'test/**/*_test.rb'
+  test.warning = true
   test.verbose = true
 end
 
@@ -62,9 +63,10 @@ end
 
 begin
   require 'rdoc/task'
-  Rake::RDocTask.new do |rdoc|
+  RDoc::Task.new do |rdoc|
     rdoc.rdoc_dir = 'rdoc'
     rdoc.title = "#{name} #{version}"
+    rdoc.markup = 'tomdoc' if rdoc.respond_to?(:markup)
     rdoc.rdoc_files.include('README*')
     rdoc.rdoc_files.include('lib/**/*.rb')
   end

data/asciidoctor.gemspec

@@ -13,8 +13,8 @@ Gem::Specification.new do |s|
   ## If your rubyforge_project name is different, then edit it and comment out
   ## the sub! line in the Rakefile
   s.name              = 'asciidoctor'
-  s.version           = '0.0.9'
-  s.date              = '2013-01-16'
+  s.version           = '0.1.0'
+  s.date              = '2013-02-04'
   s.rubyforge_project = 'asciidoctor'
 
   ## Make sure your summary is short. The description may be as long
@@ -27,7 +27,7 @@ Gem::Specification.new do |s|
   ## a custom homepage, consider using your GitHub URL or the like.
   s.authors  = ["Ryan Waldron", "Dan Allen", "Jeremy McAnally"]
   s.email    = 'rew@erebor.com'
-  s.homepage = 'http://github.com/erebor/asciidoctor'
+  s.homepage = 'http://github.com/asciidoctor'
 
   ## This gets added to the $LOAD_PATH so that 'lib/NAME.rb' can be required as
   ## require 'NAME.rb' or'/lib/NAME/file.rb' can be as require 'NAME/file.rb'
@@ -53,6 +53,7 @@ Gem::Specification.new do |s|
   s.add_development_dependency('nokogiri')
   s.add_development_dependency('pending')
   s.add_development_dependency('rake')
+  s.add_development_dependency('rdoc', '~> 3.12')
   s.add_development_dependency('tilt')
 
   ## Leave this section as-is. It will be automatically generated from the
@@ -66,6 +67,7 @@ Gem::Specification.new do |s|
     Rakefile
     asciidoctor.gemspec
     bin/asciidoctor
+    bin/asciidoctor-safe
     lib/asciidoctor.rb
     lib/asciidoctor/abstract_block.rb
     lib/asciidoctor/abstract_node.rb
@@ -75,6 +77,8 @@ Gem::Specification.new do |s|
     lib/asciidoctor/backends/html5.rb
     lib/asciidoctor/block.rb
     lib/asciidoctor/callouts.rb
+    lib/asciidoctor/cli/invoker.rb
+    lib/asciidoctor/cli/options.rb
     lib/asciidoctor/debug.rb
     lib/asciidoctor/document.rb
     lib/asciidoctor/errors.rb
@@ -87,7 +91,8 @@ Gem::Specification.new do |s|
     lib/asciidoctor/substituters.rb
     lib/asciidoctor/table.rb
     lib/asciidoctor/version.rb
-    noof.rb
+    man/asciidoctor.1
+    man/asciidoctor.ad
     test/attributes_test.rb
     test/blocks_test.rb
     test/document_test.rb
@@ -98,15 +103,18 @@ Gem::Specification.new do |s|
     test/fixtures/encoding.asciidoc
     test/fixtures/include-file.asciidoc
     test/fixtures/list_elements.asciidoc
+    test/fixtures/sample.asciidoc
     test/fixtures/tip.gif
-    test/headers_test.rb
+    test/invoker_test.rb
     test/lexer_test.rb
     test/links_test.rb
     test/lists_test.rb
+    test/options_test.rb
     test/paragraphs_test.rb
     test/preamble_test.rb
     test/reader_test.rb
     test/renderer_test.rb
+    test/sections_test.rb
     test/substitutions_test.rb
     test/tables_test.rb
     test/test_helper.rb

data/bin/asciidoctor

@@ -1,8 +1,11 @@
 #!/usr/bin/env ruby
 
-$:.unshift File.dirname(__FILE__) + '/../lib'
+$:.unshift File.join(File.dirname(__FILE__), '..', 'lib')
 
 require 'asciidoctor'
-require 'asciidoctor/cli'
+require 'asciidoctor/cli/options'
+require 'asciidoctor/cli/invoker'
 
-Asciidoctor::Cli.new.run
+invoker = Asciidoctor::Cli::Invoker.new(ARGV)
+invoker.invoke!
+exit invoker.code

data/bin/asciidoctor-safe

@@ -0,0 +1,13 @@
+#!/usr/bin/env ruby
+
+$:.unshift File.join(File.dirname(__FILE__), '..', 'lib')
+
+require 'asciidoctor'
+require 'asciidoctor/cli/options'
+require 'asciidoctor/cli/invoker'
+
+options = Asciidoctor::Cli::Options.parse! ARGV
+options[:safe] = Asciidoctor::SafeMode::SAFE
+invoker = Asciidoctor::Cli::Invoker.new(options)
+invoker.invoke!
+exit invoker.code

data/lib/asciidoctor.rb

@@ -57,19 +57,29 @@ module Asciidoctor
     # the source file and disables any macro other than the include::[] macro.
     SAFE = 1;
 
-    # A safe mode level that disallows the document from attempting to read files
-    # from the file system and including the contents of them into the document.
-    # This value disallows use of the include::[] macro and the embedding of
-    # binary content (data uri), stylesheets and JavaScripts referenced by the
-    # document. (Asciidoctor and trusted extensions may still be allowed to embed
-    # trusted content into the document). Since Asciidoctor is aiming for wide
-    # adoption, this value is the default and is recommended for server-side
-    # deployments.
-    SECURE = 10;
+    # A safe mode level that disallows the document from setting attributes
+    # that would affect the rendering of the document, in addition to all the
+    # security features of SafeMode::SAFE. For instance, this level disallows
+    # changing the backend or the source-highlighter using an attribute defined
+    # in the source document. This is the most fundamental level of security
+    # for server-side deployments (hence the name).
+    SERVER = 10;
+
+    # A safe mode level that disallows the document from attempting to read
+    # files from the file system and including the contents of them into the
+    # document, in additional to all the security features of SafeMode::SERVER.
+    # For instance, this level disallows use of the include::[] macro and the
+    # embedding of binary content (data uri), stylesheets and JavaScripts
+    # referenced by the document.(Asciidoctor and trusted extensions may still
+    # be allowed to embed trusted content into the document).
+    #
+    # Since Asciidoctor is aiming for wide adoption, this level is the default
+    # and is recommended for server-side deployments.
+    SECURE = 20;
 
     # A planned safe mode level that disallows the use of passthrough macros and
     # prevents the document from setting any known attributes, in addition to all
-    # the security features of SafeMode::SECURE
+    # the security features of SafeMode::SECURE.
     #
     # Please note that this level is not currently implemented (and therefore not
     # enforced)!
@@ -81,14 +91,28 @@ module Asciidoctor
   # Can influence markup generated by render templates
   DEFAULT_DOCTYPE = 'article'
 
-  # Backend determines the format of the rendered output, default to html5
+  # The backend determines the format of the rendered output, default to html5
   DEFAULT_BACKEND = 'html5'
 
+  # Pointers to the preferred version for a given backend.
+  BACKEND_ALIASES = {
+    'html' => 'html5',
+    'docbook' => 'docbook45'
+  }
+
   # Default page widths for calculating absolute widths
   DEFAULT_PAGE_WIDTHS = {
     'docbook' => 425
   }
 
+  # Default extensions for the respective base backends
+  DEFAULT_EXTENSIONS = {
+    'html' => '.html',
+    'docbook' => '.xml',
+    'asciidoc' => '.ad',
+    'markdown' => '.md'
+  }
+
   LIST_CONTEXTS = [:ulist, :olist, :dlist, :colist]
 
   NESTABLE_LIST_CONTEXTS = [:ulist, :olist, :dlist]
@@ -109,6 +133,13 @@ module Asciidoctor
 
   LINE_FEED_ENTITY = '
' # or 

 
+  # The following pattern, which appears frequently, captures the contents between square brackets,
+  # ignoring escaped closing brackets (closing brackets prefixed with a backslash '\' character)
+  #
+  # Pattern:
+  # (?:\[((?:\\\]|[^\]])*?)\])
+  # Matches:
+  # [enclosed text here] or [enclosed [text\] here]
   REGEXP = {
     # [[Foo]]
     :anchor           => /^\[\[([^\[\]]+)\]\]\s*$/,
@@ -159,14 +190,15 @@ module Asciidoctor
 
     # attribute reference
     # {foo}
-    :attr_ref         => /(\\?)\{(\w|\w[\w\-]*\w)(\\?)\}/,
+    # {counter:pcount:1}
+    :attr_ref         => /(\\?)\{(\w|\w[\w\-:]*\w)(\\?)\}/,
 
     # The author info line the appears immediately following the document title
     # John Doe <john@anonymous.com>
-    :author_info      => /^\s*([\w\-]+)(?: +([\w\-]+))?(?: +([\w\-]+))?(?: +<([^>]+)>)?\s*$/,
+    :author_info      => /^\s*(\w[\w\-'\.]*)(?: +(\w[\w\-'\.]*))?(?: +(\w[\w\-'\.]*))?(?: +<([^>]+)>)?\s*$/,
 
-    # [[[Foo]]]  (does not suffer quite the same malady as :anchor, but almost. Allows [ but not ] in internal capture
-    :biblio           => /\[\[\[([^\[\]]+)\]\]\]/,
+    # [[[Foo]]] (anywhere inline)
+    :biblio_macro     => /\\?\[\[\[([\w:][\w:\.-]*?)\]\]\]/,
 
     # callout reference inside literal text
     # <1>
@@ -186,6 +218,11 @@ module Asciidoctor
     # // (and then whatever)
     :comment          => %r{^//([^/].*|)$},
 
+    # one,two
+    # one, two
+    # one , two
+    :csv_delimiter    => /[[:space:]]*,[[:space:]]*/,
+
     # 29
     :digits           => /^\d+$/,
 
@@ -208,12 +245,25 @@ module Asciidoctor
     # ====
     :example          => /^={4,}\s*$/,
 
+    # footnote:[text]
+    # footnoteref:[id,text]
+    # footnoteref:[id]
+    :footnote_macro   => /\\?(footnote|footnoteref):\[((?:\\\]|[^\]])*?)\]/m,
+
     # image::filename.png[Caption]
-    :image_blk        => /^image::(\S+?)\[(.*?)\]$/,
+    :image_blk        => /^image::(\S+?)\[(.*?)\]\s*$/,
 
-    # image:filename.png[Alt]
+    # image:filename.png[Alt Text]
     :image_macro      => /\\?image:([^\[]+)(?:\[([^\]]*)\])/,
 
+    # indexterm:[Tigers,Big cats]
+    # (((Tigers,Big cats)))
+    :indexterm_macro  => /\\?(?:indexterm:(?:\[((?:\\\]|[^\]])*?)\])|\(\(\((.*?)\)\)\)(?!\)))/m,
+
+    # indexterm2:[Tigers]
+    # ((Tigers))
+    :indexterm2_macro  => /\\?(?:indexterm2:(?:\[((?:\\\]|[^\]])*?)\])|\(\((.*?)\)\)(?!\)))/m,
+
     # whitespace at the beginning of the line
     :leading_blanks   => /^([[:blank:]]*)/,
 
@@ -228,11 +278,11 @@ module Asciidoctor
 
     # inline link and some inline link macro
     # FIXME revisit!
-    :link_inline      => %r{(^|link:|\s|>|&lt;|[\(\)\]])(\\?https?://[^\[ ]*[^\. \)\[])(?:\[((?:\\\]|[^\]])*?)\])?},
+    :link_inline      => %r{(^|link:|\s|>|&lt;|[\(\)\[\]])(\\?https?://[^\s\[<]*[^\s\.\)\[<])(?:\[((?:\\\]|[^\]])*?)\])?},
 
     # inline link macro
     # link:path[label]
-    :link_macro       => /\\?link:([^\[ ]+)(?:\[((?:\\\]|[^\]])*?)\])/,
+    :link_macro       => /\\?link:([^\s\[]+)(?:\[((?:\\\]|[^\]])*?)\])/,
 
     # ----
     :listing          => /^\-{4,}\s*$/,
@@ -328,8 +378,8 @@ module Asciidoctor
     #
     # match[1] is the delimiter, whose length determines the level
     # match[2] is the title itself
-    # match[3] is an optional repeat of the delimiter, which is dropped
-    :section_title     => /^(={1,5})\s+(\S.*?)\s*(?:\[\[([^\[]+)\]\]\s*)?(\s\1)?$/,
+    # match[3] is an inline anchor, which becomes the section id
+    :section_title     => /^(={1,5})\s+(\S.*?)\s*(?:\[\[([^\[]+)\]\]\s*)?(?:\s\1)?$/,
 
     # does not begin with a dot and has at least one alphanumeric character
     :section_name      => /^((?=.*\w+.*)[^\.].*?)\s*$/,
@@ -458,7 +508,7 @@ module Asciidoctor
     # (TM)
     [/(^|[^\\])\(TM\)/, '\1&#8482;'],
     # foo -- bar
-    [/ -- /, '&#8201;&#8212;&#8201;'],
+    [/(^|\n| )-- /, '&#8201;&#8212;&#8201;'],
     # foo--bar
     [/(\w)--(?=\w)/, '\1&#8212;'],
     # ellipsis
@@ -467,31 +517,192 @@ module Asciidoctor
     [/(\w)'(\w)/, '\1&#8217;\2'],
     # escaped single quotes
     [/(\w)\\'(\w)/, '\1\'\2'],
+    # right arrow ->
+    [/(^|[^\\])-&gt;/, '\1&#8594;'],
+    # right double arrow =>
+    [/(^|[^\\])=&gt;/, '\1&#8658;'],
+    # left arrow <-
+    [/(^|[^\\])&lt;-/, '\1&#8592;'],
+    # right left arrow <=
+    [/(^|[^\\])&lt;=/, '\1&#8656;'],
     # and so on...
     
     # restore entities; TODO needs cleanup
-    [/&amp;(#[a-z0-9]+;)/i, '&\1']
+    [/(^|[^\\])&amp;(#[a-z0-9]+;)/i, '\1&\2']
   ]
 
+  # Public: Parse the AsciiDoc source input into an Asciidoctor::Document
+  #
+  # Accepts input as an IO (or StringIO), String or String Array object. If the
+  # input is a File, information about the file is stored in attributes on the
+  # Document object.
+  #
+  # input   - the AsciiDoc source as a IO, String or Array.
+  # options - a Hash of options to control processing (default: {})
+  #           see Asciidoctor::Document#initialize for details
+  # block   - a callback block for handling include::[] directives
+  #
+  # returns the Asciidoctor::Document
+  def self.load(input, options = {}, &block)
+    lines = nil
+    if input.is_a?(File)
+      options[:attributes] ||= {}
+      attrs = options[:attributes]
+      lines = input.readlines
+      input_mtime = input.mtime
+      input_path = File.expand_path(input.path)
+      # hold off on setting infile and indir until we get a better sense of their purpose
+      attrs['docfile'] = input_path
+      attrs['docdir'] = File.dirname(input_path)
+      attrs['docname'] = File.basename(input_path, File.extname(input_path))
+      attrs['docdate'] = input_mtime.strftime('%Y-%m-%d')
+      attrs['doctime'] = input_mtime.strftime('%H:%M:%S %Z')
+      attrs['docdatetime'] = [attrs['docdate'], attrs['doctime']] * ' '
+    elsif input.respond_to?(:readlines)
+      input.rewind rescue nil
+      lines = input.readlines
+    elsif input.is_a?(String)
+      lines = input.lines.entries
+    elsif input.is_a?(Array)
+      lines = input.dup
+    else
+      raise "Unsupported input type: #{input.class}"
+    end
+
+    Document.new(lines, options, &block) 
+  end
+
+  # Public: Parse the contents of the AsciiDoc source file into an Asciidoctor::Document
+  #
+  # Accepts input as an IO, String or String Array object. If the
+  # input is a File, information about the file is stored in
+  # attributes on the Document.
+  #
+  # input   - the String AsciiDoc source filename
+  # options - a Hash of options to control processing (default: {})
+  #           see Asciidoctor::Document#initialize for details
+  # block   - a callback block for handling include::[] directives
+  #
+  # returns the Asciidoctor::Document
+  def self.load_file(filename, options = {}, &block)
+    Asciidoctor.load(File.new(filename), options, &block)
+  end
+
+  # Public: Parse the AsciiDoc source input into an Asciidoctor::Document and render it
+  # to the specified backend format
+  #
+  # Accepts input as an IO, String or String Array object. If the
+  # input is a File, information about the file is stored in
+  # attributes on the Document.
+  #
+  # If the :in_place option is true, and the input is a File, the output is
+  # written to a file adjacent to the input file, having an extension that
+  # corresponds to the backend format. Otherwise, if the :to_file option is
+  # specified, the file is written to that file. If :to_file is not an absolute
+  # path, it is resolved relative to :to_dir, if given, otherwise the
+  # Document#base_dir. If the target directory does not exist, it will not be
+  # created unless the :mkdirs option is set to true. If the file cannot be
+  # written because the target directory does not exist, or because it falls
+  # outside of the Document#base_dir in safe mode, an IOError is raised.
+  #
+  # If the output is going to be written to a file, the header and footer are
+  # rendered unless specified otherwise (writing to a file implies creating a
+  # standalone document). Otherwise, the header and footer are not rendered by
+  # default and the rendered output is returned.
+  #
+  # input   - the String AsciiDoc source filename
+  # options - a Hash of options to control processing (default: {})
+  #           see Asciidoctor::Document#initialize for details
+  # block   - a callback block for handling include::[] directives
+  #
+  # returns nothing if the rendered output String is written to a file,
+  # otherwise the rendered output String is returned
+  def self.render(input, options = {}, &block)
+    in_place = options.delete(:in_place) || false
+    to_file = options.delete(:to_file)
+    to_dir = options.delete(:to_dir)
+    mkdirs = options.delete(:mkdirs) || false
+
+    write_in_place = in_place && input.is_a?(File)
+    write_to_target = to_file || to_dir
+
+    raise ArgumentError, ':in_place with input file must not accompany :to_dir or :to_file' if write_in_place && write_to_target
+
+    if !options.has_key?(:header_footer) && (write_in_place || write_to_target)
+      options[:header_footer] = true
+    end
+
+    doc = Asciidoctor.load(input, options, &block)
+
+    if write_in_place
+      to_file = File.join(File.dirname(input.path), "#{doc.attributes['docname']}#{doc.attributes['outfilesuffix']}")
+    elsif write_to_target
+      if to_dir
+        to_dir = doc.normalize_asset_path(to_dir, 'to_dir', false)
+        if to_file
+          # normalize again, to_file could have dirty bits
+          to_file = doc.normalize_asset_path(File.expand_path(to_file, to_dir), 'to_file', false)
+          # reestablish to_dir as the final target directory (in the case to_file had directory segments)
+          to_dir = File.dirname(to_file)
+        else
+          to_file = File.join(to_dir, "#{doc.attributes['docname']}#{doc.attributes['outfilesuffix']}")
+        end
+      elsif to_file
+        to_file = doc.normalize_asset_path(to_file, 'to_file', false)
+        to_dir = File.dirname(to_file)
+      end
+
+      if !File.directory? to_dir
+        if mkdirs
+          require_library 'fileutils'
+          FileUtils.mkdir_p to_dir
+        else
+          raise IOError, "target directory does not exist: #{to_dir}"
+        end
+      end
+    end
+
+    if to_file
+      File.open(to_file, 'w') {|file| file.write doc.render }
+      nil
+    else
+      doc.render
+    end
+  end
+
+  # Public: Parse the contents of the AsciiDoc source file into an Asciidoctor::Document
+  # and render it to the specified backend format
+  #
+  # input   - the String AsciiDoc source filename
+  # options - a Hash of options to control processing (default: {})
+  #           see Asciidoctor::Document#initialize for details
+  # block   - a callback block for handling include::[] directives
+  #
+  # returns nothing if the rendered output String is written to a file,
+  # otherwise the rendered output String is returned
+  def self.render_file(filename, options = {}, &block)
+    Asciidoctor.render(File.new(filename), options, &block)
+  end
+
   # Internal: Prior to invoking Kernel#require, issues a warning urging a
   # manual require if running in a threaded environment.
   #
   # name  - the String name of the library to require.
   #
-  # returns nothing
+  # returns false if the library is detected on the load path or the return
+  # value of delegating to Kernel#require
   def self.require_library(name)
     if Thread.list.size > 1
       main_script = "#{name}.rb"
       main_script_path_segment = "/#{name}.rb"
       if !$LOADED_FEATURES.detect {|p| p == main_script || p.end_with?(main_script_path_segment) }.nil?
-        return
+        return false
       else
         warn "WARN: asciidoctor is autoloading '#{name}' in threaded environment. " +
            "The use of an explicit require '#{name}' statement is recommended."
       end
     end
     require name
-    nil
   end
 
   # modules