asciidoctor-rfc 0.8.0 → 0.8.2

data/.rubocop.yml

@@ -1,32 +1,15 @@
-inherit_from: .oss-guides.rubocop.yml
-
+# This project follows the Ribose OSS style guide.
+# https://github.com/riboseinc/oss-guides
+# All project-specific additions and overrides should be specified in this file.
+
+inherit_from:
+  # Thoughtbot's style guide from: https://github.com/thoughtbot/guides
+  - ".rubocop.tb.yml"
+  # Overrides from Ribose
+  - ".rubocop.ribose.yml"
 AllCops:
-  TargetRubyVersion: 2.3
-
-Metrics/LineLength:
-  Exclude:
-  - "spec/**/*"
-
+  DisplayCopNames: false
+  StyleGuideCopsOnly: false
+  TargetRubyVersion: 2.4
 Rails:
-  Enabled: false
-
-Metrics/AbcSize:
-  Enabled: false
-
-Metrics/LineLength:
-  Enabled: false
-
-Metrics/ModuleLength:
-  Enabled: false
-
-Metrics/MethodLength:
-  Enabled: false
-
-Metrics/PerceivedComplexity:
-  Enabled: false
-
-Metrics/CyclomaticComplexity:
-  Enabled: false
-
-Naming/VariableName:
-  Enabled: false
+  Enabled: true

data/.travis.yml

@@ -1,6 +1,11 @@
-sudo: false
+sudo: true
 language: ruby
 rvm:
-  - 2.3
+  - 2.4.0
+before_install: 
+  - gem install bundler -v 1.16.0
+  - bundle install
+install:
+  - sudo pip install xml2rfc
 script:
-  - bundle exec rspec
+  - bundle exec rspec --color --format documentation --require spec_helper 

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at ronald.tse@ribose.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -1,4 +1,11 @@
 source "https://rubygems.org"
 
-# Specify your gem's dependencies in ribose.gemspec
+group :development, :test do
+  gem "asciidoctor-bibliography",
+      git: "https://github.com/riboseinc/asciidoctor-bibliography.git"
+  gem "rspec"
+  gem "rspec-rails", "~> 3.0"
+end
+
+# Specify your gem"s dependencies in ribose.gemspec
 gemspec

data/LICENSE

@@ -1,21 +1,25 @@
-MIT License
+BSD 2-Clause License
 
-Copyright (c) 2017 Ribose
+Copyright (c) 2018, Ribose
+All rights reserved.
 
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
 
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.adoc

@@ -1,4 +1,4 @@
-= Writing Internet-Drafts and RFCs in AsciiDoc
+= AsciiRFC: Writing Internet-Drafts and RFCs in AsciiDoc
 :source-highlighter: coderay
 :icons: font
 
@@ -23,7 +23,15 @@ RFC XML output in both v3 and v2 formats:
 * v3 RFC XML (https://tools.ietf.org/html/rfc7991[RFC 7991])
 * v2 RFC XML (https://tools.ietf.org/html/rfc7749[RFC 7749])
 
-Its syntax is designed to be "`native-asciidoctor`":
+Its syntax is designed to be "`native-asciidoctor`" as much as possible, with
+some templated use of attributes to convey added information for RFC XML
+output.
+
+We refer to the version of Asciidoc markup processed by this gem as AsciiRFC.
+
+This README documents the AsciiRFC markup in full detail. For further
+information about AsciiRFC, please refer to the 
+https://datatracker.ietf.org/doc/draft-ribose-asciirfc/[AsciiRFC IETF Internet-Draft].
 
 
 == Installation
@@ -50,15 +58,30 @@ $ gem install asciidoctor-rfc
 ----
 
 
-== Quick Start
+== Quick Start / Template
+
+Clone the
+https://github.com/riboseinc/rfc-in-asciidoc-template[`rfc-in-asciidoc-template`]
+repository as a template, and populate it for your Asciidoc RFCs and
+Internet-Drafts:
+
+[source,console]
+----
+$ git clone https://github.com/riboseinc/rfc-in-asciidoc-template
+----
+
+See the https://github.com/riboseinc/rfc-in-asciidoc-template[README] of that document for
+more information.
+
+== Quick Example
 
 Illustrating with v2 RFC XML, and Internet Draft rather than RFC
 
 [source,asciidoc]
 ----
-=  A Standard for the Transmission of IP Datagrams on Avian Carriers
+= A Standard for the Transmission of IP Datagrams on Avian Carriers
 David Waitzman <dwaitzman@BBN.COM>
-:doctype: internet-draft 
+:doctype: internet-draft
 :abbrev: IP Datagrams on Avian Carriers
 :obsoletes: 10, 120
 :updates: 2010, 2120
@@ -87,7 +110,7 @@ IEEE802.3.  The carriers have an intrinsic collision avoidance
 system, which increases availability.  Unlike some network
 technologies, such as packet radio, communication is not limited to
 line-of-sight distance.  Connection oriented service is available in
-some cities, usually based upon a central hub topology. 
+some cities, usually based upon a central hub topology.
 
 NOTE: Yes, this is an April Fool's RFC.
 
@@ -141,26 +164,28 @@ typical MTU is 256 milligrams.  Some datagram padding may be needed.<<RFC7253>>
 ++++
 ----
 
+
 == Usage
 
 Converting your AsciiDoc to RFC XML is a simple as running the appropriate
-`./bin/asciidoctor-rfc` script using Ruby, and passing our
+`./bin/asciidoctor-rfc` script using Ruby, and passing your
 AsciiDoc document file as the first argument.
 
+Usually you would use the `asciidoctor` executable, specifying this gem as a
+library:
+
 [source,console]
 ----
-$ ruby ./bin/asciidoctor-rfc3 draft-example-00.adoc  # RFC XML v3 output
-$ ruby ./bin/asciidoctor-rfc2 draft-example-00.adoc  # RFC XML v2 output
-## WIP: bundle exec asciidoctor -b asciidoctor-rfc3 draft-example-00.adoc # RFC XML v3 output
-## WIP: bundle exec asciidoctor -b asciidoctor-rfc2 draft-example-00.adoc # RFC XML v2 output
+$ asciidoctor -b rfc3 -r 'asciidoctor-rfc' a.adoc  # RFC XML v3 output
+$ asciidoctor -b rfc2 -r 'asciidoctor-rfc' a.adoc  # RFC XML v2 output
 ----
 
-You can also use the asciidoctor executable, specifying this gem as a library:
+or through the included bin-stub:
 
 [source,console]
 ----
-$ asciidoctor -b rfc3 -r 'asciidoctor/rfc' a.adoc  # RFC XML v3 output
-$ asciidoctor -b rfc2 -r 'asciidoctor/rfc' a.adoc  # RFC XML v2 output
+$ ./bin/asciidoctor-rfc3 draft-example-00.adoc  # RFC XML v3 output
+$ ./bin/asciidoctor-rfc2 draft-example-00.adoc  # RFC XML v2 output
 ----
 
 When the script completes, you should see the RFC XML file `draft-example-00.xml` in the
@@ -185,7 +210,7 @@ not permit anchors for the `note` or `abstract` elements, while RFC XML v2
 uses anchors for much fewer block elements (e.g. `artwork`, `list`.)
 * Asciidoctor has a more restrctive block model: it regards elements such as
 admonitions, lists, and tables as blocks, and does not allow them to be nested
-within paragraphs. 
+within paragraphs.
 
 As a result, it may be necessary to postedit the RFC XML output, if the block
 model produced does not reflect the intended structure accurately.
@@ -193,10 +218,14 @@ model produced does not reflect the intended structure accurately.
 
 NOTE: The gem validates all RFC XML generated through the RELAXNG schema definitions
 of RFC XML. While the gem attempts to generate valid RFC XML, some Asciidoctor
-text may not align with the RFC XML document model, and any highlighted syntax 
-errors will also need to be rectified in postediting, before processing the 
+text may not align with the RFC XML document model, and any highlighted syntax
+errors will also need to be rectified in postediting, before processing the
 generated XML further.
 
+NOTE: A summary guide to authoring RFC XML in Asciidoctor is also available at
+https://tools.ietf.org/html/draft-ribose-asciirfc-02 (with source at
+https://github.com/riboseinc/rfc-asciidoc-rfc ).
+
 
 === Document Type: `rfc` or `internet-draft` (Mandatory)
 
@@ -215,7 +244,7 @@ Set the `doctype` attribute to set status of this document:
 * `rfc@number` will be set to the `:name:` attribute (in v2)
 
 
-=== Gem Options
+=== Global Options
 
 [cols="2", options="header"]
 |===
@@ -229,7 +258,7 @@ Override default assumption that boldface uppercase BCP14 word is to be rendered
 
 |`:smart-quotes:`
 |Optional. Default value `true`. Allowed values: `true`, `false`.
-Permit smart quotes, when they are specified explicitly in Asciidoc (as `"`...`"`, `'`...`'`.) 
+Permit smart quotes, when they are specified explicitly in Asciidoc (as `"`...`"`, `'`...`'`.)
 When disabled, smart quotes are rendered as straight quotes, and Asciidoc's default conversion
 of straight apostrophes to smart is undone.
 
@@ -239,10 +268,58 @@ Only applies to v2. By default, `<vspace blankLines="1">` is inserted after
 the definition in a v2 definition list, to satisfy the requirement from `idnits` validation
 that definition terms be separated by a carriage return from the definition. That is, by
 default inline definition lists are rendered as paragraphing definition lists. If the option is
-on, the additional `vspace` element is not added, and inline definition lists are left as is..
+on, the additional `vspace` element is not added, and inline definition lists are left as is.
+
+|`:flush-caches:`
+|Optional. Default value `false`. Allowed values: `true`, `false`.
+Delete and reload the caches of references to be included externally, and of workgroups,
+during processing of this document.
+The caches are stored in `~/.asciidoc-rfc-biblio-cache.json` and `~/.asciidoc-rfc-workgroup-cache.json`.
+
+|`:biblio-dir:`
+|Optional. Name of directory. If present, gives the name of a directory from which RFC XML references are to be read into the document, rather than assuming the references are already present in the document.
+
+|`:normative:`
+|Optional. Comma-delimited list of reference anchors. Used in conjunction with `:biblio-dir:`, which uses a single directory for all references: this attribute
+lists those references which are to be considered normative, and listed under the _Normative References_ heading.
 
 |===
 
+Any of these global options can also be included in the command line call, for either
+the asciidoctor executable or the local bin-stubs, through the command line option `-a`;
+e.g.
+
+[source,console]
+----
+$ asciidoctor -a flush-biblio=true -b rfc3 -r 'asciidoctor-rfc' a.adoc  
+$ ./bin/asciidoctor-rfc2 -a smart-quotes=false draft-example-00.adoc  
+----
+
+[[caches]]
+==== Caches
+
+The cache of externally addressable bibliographic information is built from screenscraping the contents of:
+
+* https://xml2rfc.tools.ietf.org/public/rfc/bibxml/
+* https://xml2rfc.tools.ietf.org/public/rfc/bibxml2/ 
+* https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/
+* https://xml2rfc.tools.ietf.org/public/rfc/bibxml4/ 
+* https://xml2rfc.tools.ietf.org/public/rfc/bibxml5/
+
+The cache of current IETF and IRTF workgroups is built from screenscraping the contents of:
+
+* https://tools.ietf.org/wg/
+* https://irtf.org/groups
+
+The caches are not rebuilt daily, as the bibliographic cache is with xml2rfc. 
+If you want to refresh the caches, 
+
+* delete
+your `~/.asciidoc-rfc-biblio-cache.json` and `~/.asciidoc-rfc-workgroup-cache.json` files; 
+* insert the document attribute `:flush-caches: true` into
+the header of the document being processed; or 
+* run the asciidoctor executable with option `-a flush-caches=true`
+(which has the same effect).
 
 === Basic Document Attributes
 
@@ -312,7 +389,7 @@ https://tools.ietf.org/html/rfc7991#section-2.4[here].
 | `front/area`
 
 | `:workgroup:`
-| Optional. Comma delimited text on which IETF workgroup or research group this
+| Optional. Comma delimited text on which IETF or IRTF workgroup or research group this
 document originates from. See https://tools.ietf.org/html/rfc7991#section-2.65[here].
 | `front/workgroup`
 
@@ -326,11 +403,11 @@ metadata.
 === Processing Instructions
 The `xml2rfc` tool accepts processing instructions of the form `<?rfc keyword='value'?>`:
 see https://xml2rfc.tools.ietf.org/authoring/README.html#processing.instructions .
-(Of these, `sort-refs`, `sym-refs` and `toc-include`  are also present in the 
+(Of these, `sort-refs`, `sym-refs` and `toc-include`  are also present in the
 v3 RFC XML specifcation, as attributes of the
 root `rfc` element: <<v3documentattributes,v3-specific document attributes>>.)
-Those processing instructions which apply to the entire document can also be 
-specified for this gem as document options. 
+Those processing instructions which apply to the entire document can also be
+specified for this gem as document options.
 
 |===
 |keyword |meaning
@@ -365,7 +442,7 @@ specified for this gem as document options.
 |text-list-symbols |	modify the list of symbols used (when generated text) for list type="symbols". For example, specifying "abcde" will cause "a" to be used for 1st level, "b" for the 2nd level, etc, cycling back to the first character "a" at the 6th level. Specifying "o*" will cause the characters "o" and "*" to be alternated for each successive level.
 |toc-include |	(`toc`) generate a table-of-contents
 |tocappendix |	control whether the word "Appendix" appears in the table-of-content
-|tocdepth |	if toc is "yes", then this determines the depth of the table-of-contents
+|toc-depth |	if toc is "yes", then this determines the depth of the table-of-contents
 |tocindent |	if toc is "yes", then setting this to "yes" will indent subsections in the table-of-contents
 |tocnarrow |	affects horizontal spacing in the table-of-content
 |tocompact |	if toc is "yes", then setting this to "no" will make it a little less compact
@@ -375,6 +452,9 @@ specified for this gem as document options.
 
 Exceptionally, `compact`, `toc-include`, `sym-refs`, `sort-refs` and `strict` are is set by default to `yes`, `subcompact` to `no`, and `toc-depth` to 4.
 
+The additional document option `rfc2629xslt` (default value: true) injects into the document header the processing
+instruction `<?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?>`, which impacts on the output of xml2rfc.
+
 === Document Name / Number (Mandatory)
 
 The `name` attribute sets the document's name. This should be a number if
@@ -397,8 +477,8 @@ When `doctype` is set to:
 
 Set the `status` attribute to set the current status of this document.
 
-The following values are allowed: `standard`, `informational`, `experimental`, `bcp`, `fyi`, 
-`full-standard` (v3 only), `historic` (v2 only).  
+The following values are allowed: `standard`, `informational`, `experimental`, `bcp`, `fyi`,
+`full-standard` (v3 only), `historic` (v2 only).
 
 * In v3, this sets the first `front/seriesInfo` element with `@status` as one
   of: `standard`, `informational`, `experimental`, `bcp`, `fyi`, `full-standard`.
@@ -410,7 +490,7 @@ The following values are allowed: `standard`, `informational`, `experimental`, `
 Set the `intended-series` attribute to set the intended series of this
 document.
 
-The following values are allowed: `standard`, `informational`, `experimental`, `bcp`, `fyi`, `full-standard`. 
+The following values are allowed: `standard`, `informational`, `experimental`, `bcp`, `fyi`, `full-standard`.
 
 When `doctype` is set to:
 
@@ -508,8 +588,8 @@ formatter should use symbolic references (such as "`[RFC2119]`") or not
 formatter should contain a table of contents. Supported in v2 as a processing instruction.
 | `rfc@tocInclude`
 
-| `:link{_i_}: _URL_` or `:link{_i_}: _URL_ _REL_`
-a| Optional. A link to an external document related to this document.
+| `:link: _URL_, _URL_ _REL_`
+a| Optional. Comma-delimited links to an external document related to this document.
 
 There are 4 types of values:
 
@@ -536,7 +616,7 @@ You will need to provide the following information.
 
 ==== Multiple Author Names
 
-Just like a normal AsciiDoc, you can provide author information in the author 
+Just like a normal AsciiDoc, you can provide author information in the author
 header (sample of 3 authors):
 
 [source,asciidoc]
@@ -800,6 +880,10 @@ The following represent the v3 relref element
 In v2, the relref style crossreferences are rendered as equivalent `xref` crossreferences,
 inserting section numbers as appropriate.
 
+Note that fragments (e.g. `crossreference#fragment`) are not supported on the `xref@target` attribute,
+in either v2 or v3: the RFC XML specification requires that the `xref@target` attribute equals
+the value of an anchor attribute elsewhere in the document.
+
 
 === Indexing
 
@@ -844,6 +928,8 @@ NOTE: The delimiters must occur within the one line; the following is invalid in
 PROBABLY*
 --
 
+Any formatting XML spans within `spanx` elements are stripped in postprocessing.
+
 === Paragraphs
 
 [source,asciidoc]
@@ -871,6 +957,84 @@ Quotation <3>
 
 === Comments
 
+==== Asciidoctor comments
+
+Asciidoctor implements both inline comments (prefixed with `\\`)
+and block comments (prefixed with `\\\\`). Both are ignored by
+the Asciidoctor processor, and are not rendered in any output,
+including RFC XML.
+
+Asciidoctor also permits paragraphs and open blocks (which can contain
+multiple paragraphs) to be treated as Asciidoctor comments, if
+they have the style attribute `[comment]`:
+
+[source,asciidoc]
+----
+
+// This is an inline Asciidoctor comment, which will not be output to XML.
+
+[comment]
+This is a single paragraph Asciidoctor comment,
+which will not be output to XML.
+
+////
+This is a block Asciidoctor comment,
+
+which will not be output to XML.
+////
+
+[comment]
+--
+This is a 
+
+multiple paragraph
+
+Asciidoctor comment, which will not be output to XML.
+--
+----
+
+==== XML comments
+
+XML inline comments may be introduced into XML through the `[comment]`
+formatting macro: any such comments may not span more than one line.
+
+[source,asciidoc]
+--
+Text [comment]#This is a comment# Text
+--
+
+The foregoing will be rendered in RFC XML as:
+
+[source,xml]
+--
+<t>Text <!-- This is a comment --> Text</t>
+--
+
+XML block comments are introduced through the role attribute
+`[.comment]`, which can be prefied to a paragraph or an open
+block (which can contain multiple paragraphs):
+
+[source,asciidoc]
+----
+[.comment]
+This is a single paragraph XML comment.
+
+[.comment]
+--
+This is a 
+
+multiple paragraph
+
+XML comment.
+--
+----
+
+==== Text Comments 
+
+RFC XML provides for editorial comments which may optionally appear
+in the published text (subject to either the v3 `cref@display`
+attribute, or the `comments` processing instruction).
+
 In v2 RFC XML, comment text is stripped of all formatting.
 
 [source,asciidoc]
@@ -893,7 +1057,12 @@ Any admonition inside the body of the text is a comment.
 
 === Source Code Listings
 
+In RFC XML, `sourcecode` (v3) and `artwork` (v2) elements only occur within a 
+`figure` wrapper; this gem supplies that wrapper if it is not provided
+explicitly.
+
 [source,asciidoc]
+.Without Figure Wrapper
 --
 [[id]] <1>
 .Source code listing title <2>
@@ -904,15 +1073,83 @@ begin() {
 }
 ----
 --
-<1> v3: `figure/sourcecode@anchor`; v2: `figure/artwork@anchor`
+<1> v3: `figure/sourcecode@anchor`; v2: `figure@anchor` (moved into supplied wrapper: anchors not supported on `artwork`)
 <2> v3: `figure/sourcecode@name`; v2: `figure/artwork@name`
 <3> v3: `figure/sourcecode@type`; `figure/sourcecode@src` (`align` and `alt` not supported). If `src` is present, the listing is not expected to have any content: content is taken from the hyperlink in the attribute. v2: `figure/artwork@type`, `figure/artwork@src`, `figure/artwork@align`, `figure/artwork@alt`.
 <4> v3: `figure/sourcecode`; v2: `figure/artwork`
 
+[source,asciidoc]
+.With Figure Wrapper
+--
+[[id]] <1>
+[align,alt,suppress-title] <2>
+.Figure 1 <3>
+====
+Preamble text <4>
+
+[[id1]] <5>
+.Source code listing title <6>
+[source,type,src=uri,align,alt] <7>
+----
+begin() {
+  source code listing <8>
+}
+----
+
+Postamble text <9>
+====
+--
+<1> `figure@anchor`
+<2> v2 only: `figure/artwork@align`, `figure/artwork@alt`, `figure@suppress-title` (attributes only available in v2)
+<3> `figure/name`
+<4> v2 only: `figure/preamble` (only available in v2)
+<5> v3: `figure/sourcecode@anchor`; v2: Not supported: use `figure@anchor`
+<6> v3: `figure/sourcecode@name`; v2: `figure/artwork@name`
+<7> v3: `figure/sourcecode@type`; `figure/sourcecode@src` (`align` and `alt` not supported). If `src` is present, the listing is not expected to have any content: content is taken from the hyperlink in the attribute. v2: `figure/artwork@type`, `figure/artwork@src`, `figure/artwork@align`, `figure/artwork@alt`.
+<8> v3: `figure/sourcecode`; v2: `figure/artwork`
+<9> v2 only: `figure/postamble` (only available in v2)
+
 
 === ASCII Art and Images
 
+In RFC XML, `artwork` elements only occur within a 
+`figure` wrapper; this gem supplies that wrapper if it is not provided
+explicitly.
+
+
+[source,asciidoc]
+.Ascii-Art Without Figure Wrapper
+--
+[[id]] <1>
+.Figure2.jpg <2>
+[align=left|center|right,alt=Ascii Art,type=text/plain] <3>
+....
+------------------------
+|        Ascii Art     |
+------------------------ <4>
+....
+--
+<1> v3 only: `figure/artwork@anchor`; v2: `figure@anchor` (moved into supplied wrapper: anchors not supported on `artwork`)
+<2> `figure/artwork@name`
+<3> `figure/artwork@align`, `figure/artwork@alt`; `figure@type` (attribute only available in v2)
+<4> `figure/artwork`
+
+[source,asciidoc]
+.Image Without Figure Wrapper
+--
+[[id]] <1>
+.Figure2.jpg <2>
+[align=left|center|right,alt=alt_text,type=img/jpeg] <3>
+image::filename.jpg[alt_text,700,200] <4>
+--
+<1> v3 only: `figure/artwork@anchor`; v2: `figure@anchor` (moved into supplied wrapper: anchors not supported on `artwork`)
+<2> `figure/artwork@name`
+<3> `figure/artwork@align`, `figure/artwork@alt`; `figure/artwork@type` (only available in v2, intended to be a MIME type; v3: populated as either `svg` or `binary-art` depending on file suffix)
+<4> `figure/artwork@src`, `figure/artwork@alt`, `figure/artwork@width` (deprecated in v3), `figure/artwork@height` (deprecated in v3)
+
+
 [source,asciidoc]
+.With Figure Wrapper
 --
 [[id]] <1>
 [align,alt,suppress-title] <2>
@@ -922,14 +1159,16 @@ Preamble text <4>
 
 [[id]] <5>
 .Figure2.jpg <8>
-[align=left|center|right,alt=alt_text,type] <6>
+[align=left|center|right,alt=alt_text,type=text/plain] <6>
 ....
-Figures are only permitted to contain listings (sourcecode), images (artwork),
+Figures are 
+      only permitted to contain listings (sourcecode),  
+           images (artwork),
 or literal (artwork) <7>
 ....
 [[id]] <5>
 .Figure2.jpg <8>
-[align=left|center|right,alt=alt_text,type] <9>
+[align=left|center|right,alt=alt_text,type=img/jpeg] <9>
 image::filename.jpg[alt_text,700,200] <10>
 
 Postamble text <11>
@@ -939,7 +1178,7 @@ Postamble text <11>
 <2> v2 only: `figure/artwork@align`, `figure/artwork@alt`, `figure@suppress-title` (attributes only available in v2)
 <3> `figure/name`
 <4> v2 only: `figure/preamble` (only available in v2)
-<5> `figure/artwork@anchor`
+<5> v3: `figure/artwork@anchor`; v2: Not supported: use `figure@anchor`
 <6> `figure/artwork@align`, `figure/artwork@alt`; `figure@type` (attribute only available in v2)
 <7> `figure/artwork`
 <8> `figure/artwork@name`
@@ -949,6 +1188,8 @@ Postamble text <11>
 
 === Mathematical examples
 
+In order for mathematical formatting to be recognised in Asciidoc, the document attribute `:stem:` needs to be set.
+
 [source,asciidoc]
 --
 :stem:
@@ -960,8 +1201,8 @@ sqrt(4) = 2
 --
 
 Mathematical examples are treated identically to literals, and are rendered as `artwork` in both v2 and v3;
-however their default alignment is set as `center`. As with inline stem expressions, they are treated identically 
-to monospace expressions; they are not currently rendered as MathML or any other notation.
+however their default alignment is set as `center`. As with inline stem expressions, they are treated identically
+to monospace expressions in the RFC XML output; they are not currently rendered as MathML or any other notation.
 
 === Unordered and Ordered Lists
 
@@ -974,7 +1215,7 @@ to monospace expressions; they are not currently rendered as MathML or any other
 ** Nested list <4>
 
 [[id]] <5>
-[spacing=compact,start=n,group=n,counter=token,hang-indent=n,format=List #%d,arabic|loweralpha|upperralpha|lowerroman|upperroman] <6>
+[spacing=compact,empty=true,start=n,group=n,counter=token,hang-indent=n,format=List #%d,arabic|loweralpha|upperralpha|lowerroman|upperroman] <6>
 . A <7>
 . B <7>
 --
@@ -983,7 +1224,7 @@ to monospace expressions; they are not currently rendered as MathML or any other
 <3> v2: `list[@style="symbols"]/t`; v3: ul/li
 <4> v2: `list[@style="symbols"]/t/list[@style="symbols"]/t`; v3: `ul/li/ul/li`
 <5> v3: `ol@anchor`; attribute only available in v3
-<6> v2: `list/counter`,  `list@hangIndent`, `list@style = format List #%d`, `list@style` (for arabic|loweralpha|upperralpha|lowerroman|upperroman) (`start`, `empty` and `group` not available) v3: `ol@empty`, `ol@start`, `ol@group`, `ol@type = "#%d", `ol@type` (for arabic|loweralpha|upperralpha|lowerroman|upperroman) (`counter`, `hangIndent` not available)
+<6> v2: `list/counter`,  `list@hangIndent`, `list@style = format List #%d`, `list@style` (for arabic|loweralpha|upperralpha|lowerroman|upperroman) (`spacing`, `start`, `empty` and `group` not available) v3: `ol@spacing`, `ol@empty`, `ol@start`, `ol@group`, `ol@type = "#%d", `ol@type` (for arabic|loweralpha|upperralpha|lowerroman|upperroman) (`counter`, `hangIndent` not available)
 <7> v2: `list/t`; v3: `ol/li`
 
 NOTE: Asciidoctor does not permit anchors on list items: the anchors in the following are ignored.
@@ -1028,7 +1269,7 @@ paragraphs within v2 list items are replaced with `vspace` tages.
 === Tables
 
 The converter respects the AsciiDoc (horizontal) align attributes of cells (v2,
-v3), column widths (v2), and `colspan`, `rowspan` attributes (v3). 
+v3), column widths (v2), and `colspan`, `rowspan` attributes (v3).
 
 (Exceptionally,
 column widths specified for v2 as `"1,1,1,1,1,1...."` will be ignored, since Asciidoctor
@@ -1037,7 +1278,7 @@ internally treats them identically to unspecified column widths on a table.)
 [source,asciidoc]
 --
 [[id]] <1>
-[suppress-title=true|false,align=left|center|right,grid=all|cols|none] <2>
+[suppress-title=true|false,align=left|center|right,grid=all|cols|none|rows] <2>
 .Table Title <3>
 |===
 |[[id]] head | head <4>
@@ -1049,7 +1290,7 @@ h|header cell | body cell <5>
 |===
 --
 <1> v3: `table@anchor`; v2: `texttable@anchor`
-<2> v2: `texttable@suppress-title`, `texttable@align`, `texttable@style` (attributes only available in v2). Mapping of Asciidoc grid attribute to RFC XML style attribute is: `all` > `all`, `cols` > `full`, `none` > `none`. The RFC XML `headers` attribute is not supported. The Asciidoc attribute `rows` is not supported.
+<2> v2: `texttable@suppress-title`, `texttable@align`, `texttable@style` (attributes only available in v2). Mapping of Asciidoc grid attribute to RFC XML style attribute is: `all` > `all`, `cols` > `full`, `none` > `none`, `rows` > `headers` (although the two are not strictly equivalent).
 <3> v3: `table/name`; v2: `texttable@title`
 <4> v3: `table/thead/tr/td`; v2: `texttable/ttcol@id` (attribute only available in v2), `texttable/ttcol`
 <5> v3: `table/tbody/tr/th`, `table/tbody/tr/td`; v2: `texttable/c`, `texttable/c`
@@ -1072,7 +1313,7 @@ Sidebar <2>
 <2> `<aside>Sidebar</aside>`
 
 
-=== References
+=== References: Embedded in Document
 
 References are expected to be provided in raw RFC XML v2 format. For v3, a list of crossreferences may
 precede the block of references, with alternative text. This will not be rendered, but it will be used
@@ -1084,6 +1325,9 @@ RFC requires two separate bibliographies, one for normative and one for informat
 either can be omitted. All bibliography sections in the must be styled with the prefix `[bibliography]`,
 and must appear in sequence, before any appendices.
 
+By default, the references cited must be included as raw RFC XML, and separated
+into the normative and informative sections.
+
 [source,asciidoc]
 --
 [[id]] <1>
@@ -1101,10 +1345,63 @@ and must appear in sequence, before any appendices.
 (raw XML) <2>
 ++++
 --
-<1> `back/references@anchor`
+<1> `back/references@anchor` (only in v3)
 <2> `back/displayreference@target`, `back/displayreference@to` (only in v3)
 <3> `back/references/reference`
 
+In postprocessing, bibliographic entries available from http://xml.resource.org/public/rfc are replaced
+with external references to that entry, using XML entities in RFC XML v2, and XML includes
+in RFC XML v3. Do not insert your own entities or XML includes into the references; the gem will have difficulty
+processing them.
+
+[[external-directory-refs]]
+=== References: External Directory
+
+As an alternative, the document attribute `:biblio-dir:` can nominate a directory in which separate XML files can be placed, one for each reference to be included. (RFC XML v3 referencegroup elements will also be recognised as files.) The gem will read in from that directory only the files that have actually been cited, and insert them into the appropriate bibliography, without the references needing to be given under the bibliographies as above. (In fact, any XML already provided will be deleted.) By default, references will be considered informative; the document attribute `:normative:` can be used to specify a comma-delimited list of normative references.
+
+The gem will issue a warning if any cited reference is not included in the directory. However, external references do not have to be included in the directory: they will be recognised by comparing their anchors against the external bibliography cache, and referenced as entities or includes. However, particular drafts of Internet-Draft documents do still need to be included as separate documents (see <<external-ref-lookup>>.)
+
+For example:
+
+[source,asciidoc]
+--
+= The Holy Hand Grenade of Antioch
+Arthur Pendragon
+:doctype: internet-draft
+:workgroup: silly
+:biblio-dir: refs <1>
+:normative: RFC2119, AsciiDoc <1>
+
+[[xyz]]
+== Hello
+Hello
+
+* a <<RFC2119,2.3 of: See internet draft subsection>> <3>
+* b <<I-D.abarth-cake>>
+* b2 <<I-D.abarth-cake,what>>
+* b1 <<I-D.abarth-cake,2.3 of: See internet draft subsection>> <3>
+* c <<xyz,format=counter: xyzzy>> <4>
+* d <<biblio>> <4>
+* e <<AsciiDoc,AsciiDoctor>>
+* f <<mathrefs>>
+
+
+[[biblio]]
+=== Biblio
+See biblio
+
+[bibliography]
+== Normative References <5>
+
+[bibliography]
+== Informative References <6>
+--
+<1> The RFC XML references are included in the directory `./refs`, with one file per reference. For example, we would expect it to contain a file corresponding to the reference `mathrefs`. A file corresponding to `RFC2119` is optional, and in fact will be ignored, since the anchor is recognised as an external reference. A file corresponding to `I-D.abarth-cake` will not be ignored, if that file contains a `seriesInfo` element nominating a specific draft version.
+<2> The references in the `./refs` directory are by default considered informative; this attribute indicates that `RFC2119` and `AsciiDoc` are to be considered normative.
+<3> References are recognised in `relref` as well as `xref` elements.
+<4> The gem differentiates between bibliographic references and crossreferences to other anchors within the document.
+<5> The bibliographic headers need to be provided as above, and its titles are expected to be "Normative References" and "Informative References"; the gem will look for those titles specifically in order to insert the references it identifies from the file. However, no XML content is expected to be provided under each heading, and any XML content that is provided will be ignored.
+
 
 === Appendices
 
@@ -1129,16 +1426,12 @@ asciidoctor commands:
 
 |===
 | RFC XML element                  | RFC XML v3 | RFC XML v2
+
 | `front/boilerplate`              | Not added  | N/A
 | `iref@primary`                   | N          | N
-| `reference/*` (and all children) | N/A        | N
-| `reference/front`                | N          | N
-| `reference/annotation`           | N          | N
-| `reference@quote-title`          | N          | N/A
-| `reference@target`               | N          | N
+| `reference` (and all children) 2+| Supported only as pass-through or through asciidoc-bibliography gem
 | `table/preamble`                 | Deprecated | N
 | `table/postamble`                | Deprecated | N
-| `table@style = headers`          | N/A        | N
 | `artwork@width`                  | Only on images | Only on images
 | `artwork@height`                 | Only on images | Only on images
 |===
@@ -1149,6 +1442,7 @@ These asciidoctor features are not supported in output of these formats:
 
 |===
 | asciidoctor feature | RFC XML v2 | RFC XML v3
+
 | http://asciidoctor.org/docs/user-manual/#quote[Quote] | Rendered as normal paragraph | Supported
 | http://asciidoctor.org/docs/user-manual/#quote[Quote]: Non-URL Citations 2+| Ignored
 | http://asciidoctor.org/docs/user-manual/#verse[Verse] | Rendered as normal paragraph | Rendered as Quote
@@ -1169,7 +1463,7 @@ These asciidoctor features are not supported in output of these formats:
 | Macro: http://asciidoctor.org/docs/user-manual/#ui-buttons[UI buttons] 2+| Ignored
 | Table: http://asciidoctor.org/docs/user-manual/#footer-row[Distinction between table body and table footer] | No | Supported
 | Table: http://asciidoctor.org/docs/user-manual/#header-row[Multiple table header rows] | No | Supported
-| Table: http://asciidoctor.org/docs/user-manual/#table-borders[Table borders: grid = rows] | No | Ignored
+| Table: http://asciidoctor.org/docs/user-manual/#table-borders[Table borders: grid = rows] | Rendered as `headers` | Ignored
 | Table: http://asciidoctor.org/docs/user-manual/#table-borders[Table borders: grid = cols\|none] | Supported | Ignored
 | Table: http://asciidoctor.org/docs/user-manual/#table-borders[Table borders: frame] 2+| Ignored
 | Table: http://asciidoctor.org/docs/user-manual/#cell[Table cells: rowspan] | Ignored | Supported
@@ -1178,6 +1472,131 @@ These asciidoctor features are not supported in output of these formats:
 | Table: http://asciidoctor.org/docs/user-manual/#cols-format[Table columns: width] | Supported | Ignored
 |===
 
+[[external-ref-lookup]]
+== Lookup of external references
+
+In order to speed the lookup of references on the http://xml.resource.org/public/rfc website for external
+references, a cache of references is built the first time the gem is run, in the user's home directory:
+`~/.asciidoc-rfc-biblio-cache.json`. This maps all the canonical anchors for external references as defined
+by the IETF, such as `RFC2119` or `CCITT.E163.1988`, to the URLs that their RFC reference is stored on.
+References are detected for replacement in the Asciidoc document by matching the `references@anchor` 
+attribute to one of those canonical anchors; the content of the local RFC XML reference is not checked.
+
+(For rebuilding the cache, see <<caches>>.)
+
+Rather than hand crafting RFC XML references for RFC documents, or other references hosted at `http://xml.resource.org/public/rfc/`,
+you need only create a dummy `<reference>` element containing the IETF-defined anchor for that citation. In postprocessing, any
+references that are hosted at xml.resource.org will be identified by their anchor, and replaced by entities or XML includes,
+for RFC XML v2 and v3 respectively. (The complete list of authoritative
+RFC XML bibliographies is available from https://xml2rfc.tools.ietf.org , online and in zipped form.)
+
+Because the content of any externally defined `<reference>` element is overwritten with an entity or include,
+you do not need to provide a full reference. You can supply a minimal reference like `<reference anchor="RFC2119"/>`,
+but note that such a reference is invalid in the RFC XML schema, and the gem will report a missing element during processing.
+(The document will still be processed successfully.) To prevent any validation error reports, the minimal syntactically valid reference is:
+
+[source,xml]
+--
+<reference anchor="RFC2119">
+  <front>
+    <title/>
+    <author/>
+    <date/>
+  </front>
+</reference>
+--
+
+If you wish to cite a specific version of an Internet-Draft, you will need to include the `seriesInfo` element
+in the reference that identifies the specific version; the anchor is the same for all internet drafts. For example:
+
+[source,xml]
+--
+<reference anchor="I-D.abarth-cake">
+  <front>
+    <title/>
+    <author/>
+    <date/>
+  </front>
+  <seriesInfo name="Internet-Draft" value="draft-abarth-cake-00"/>
+</reference>
+--
+
+As discussed under <<external-directory-refs>>, any citations of specific versions of an Internet-Draft need to be included as explicit files in an external directory of RFC XML references, since the seriesInfo draft information cannot be recovered by the anchor. However, any other external references do not require a corresponding directory file (although normative references do still need to be named in the `:normative:` document attribute.)
+
+== asciidoc-bibliography integration
+
+[TODO]
+
+////
+The https://github.com/riboseinc/asciidoctor-bibliography[asciidoc-bibliography] gem allows citations to be imported
+into asciidoc from BibTex or RFC XML files. The gem natively requires the user to provide the correct, presorted
+RFC XML references within asciidoc separately for normative and for informative references. Using the asciidoc-bibliography
+gem, the user can instead have a single file of RFC XML references, and indicate in place whether each reference
+is intended to be normative or informative, as is the case for MMark; the gem will then extract the correct RFC XML
+references from the references file, and insert them in order into the normative and informative references respectively.
+
+Integration with the asciidoc-bibliography gem proceeds as follows:
+
+. Create an RFC XML references file, consisting of a `<references>` element with individual `<reference>` elements inserted,
+as would be done for the informative and normative references normally. The references file will contain all possible
+references to be used
+in the file; the bibliography gem will select which references have actually been cited in the document.
+.. The caveats for <<external-ref-lookup,externally available references>> also apply to the references file.
+Unlike the case for RFC XML documents created manually, the references file does not recognise XML entities
+and will not attempt to download them during processing; postprocessing means you do not need to create your own
+XML entities anyway.
+.. The RFC XML in the references file will need to be appropriate to the version of RFC XML used in the main document, as
+usual. Note that RFC XML v2 references are forward compatible with v3; v3 contains a couple of additional elements.
+. Add the following header attributes to the main document:
++
+--
+`:bibliography-database:` :: The name of the references file; this is expected to be in the same directory as
+the main document.
+`:bibliography-passthrough:` :: `citations`. This instructs the asciidoctor-bibliography gem to wrap the citations
+in an Asciidoctor passthrough block. If `true` is used, the references introduced through `cite` macros will also
+be treated as inline Asciidoctor passthrough text.
+`:bibliography-prepend-empty:` :: `true|false`. No current effect on file
+`:bibliography-hyperlinks:` :: `true|false` (default `true`): inserts bibliographic link around reference. Use `false`.
+`:bibliography-style:` :: The style of reference used in the references file; the recognised styles are `rfc-v2` and
+`rfc-v3`, for RFC XML v2 and RFC XML v3 respectively.
+--
+
+. References to a normative reference are inserted with the macro `cite:norm[id]` instead of `<<id>>`, where `id` is
+the anchor of the reference.
+. References to an informative reference are inserted with the macro `cite:info[id]` instead of `<<id>>`, where `id` is
+the anchor of the reference.
+.. Text may be prefixed or suffixed to the crossreference with the `prefix` and `suffix` attributes.
+.. Arbitrary text associated with a crossreference is inserted with a `text` attribute (with the citation itself in curly brackets), and fragment references are inserted with a `locator` attribute.
+.. Formatted crossreferences and `relref` crossreferences are entered by inserting the expected raw XML in the `text` attribute. Do not use the `{cite}` interpolation of the citation. Using `cite:norm`, the following equivalence obtain between normal Asciidoc treatment of crossreferences and asciidoc-bibliography macros:
+*** `<<id,words>>` = `cite:norm[id, text="<xref target='id'>words</xref>"]`
+*** `<<id,format=counter: words>>` (processed as a formatted crossreference) = `cite:norm[id, text="<xref format='counter' target='id'>words</xref>"]`
+*** `<<id,2.4 comma: words>>` (processed as relref) = `cite:norm[id, text="<relref displayFormat='comma' section='2.4' target='id'}/>"]`
+*** `<<id#section2_4,2.4 comma: words>>` (processed as relref with a cross-document internal reference) = `cite:norm[id, text="<relref relative='section2_4' displayFormat='comma' section='2.4' target='id'/>"]`
+. Normative and Informative References are inserted in the document through a macro, which occurs where the RFC XML
+references would be inserted
++
+[source,asciidoc]
+--
+[bibliography]
+== Normative References
+
+++++
+bibliography::norm[]
+++++
+
+[bibliography]
+== Informative References
+
+++++
+bibliography::info[]
+++++
+--
+. In order to preprocess the macros, the asciidoctor-bibliography gem is passed as an `-r` argument to the
+asciidoctor-rfc invocation:
+** `asciidoctor -b rfc2 -r 'asciidoctor-bibliography' -r 'asciidoctor-rfc'  spec/examples/refs-v2.adoc`
+** `asciidoctor -b rfc3 -r 'asciidoctor-bibliography' -r 'asciidoctor-rfc'  spec/examples/refs-v3.adoc`
+. `referencegroup` citations are not currently supported by the asciidoctor-bibliography gem.
+////
 
 == Examples