commonmarker 0.14.3 → 0.14.4

data/ext/commonmarker/cmark/test/extensions.txt

@@ -44,7 +44,7 @@ Hello!
 | _abc_ | セン |
 | ----- | ---- |
 | 1. Block elements inside cells don't work. | |
-| But **_inline elements do_**. | x |
+| But _**inline elements do**_. | x |
 
 Hi!
 .
@@ -62,7 +62,7 @@ Hi!
 <td></td>
 </tr>
 <tr>
-<td>But <strong><em>inline elements do</em></strong>.</td>
+<td>But <em><strong>inline elements do</strong></em>.</td>
 <td>x</td>
 </tr></tbody></table>
 <p>Hi!</p>
@@ -145,7 +145,7 @@ Hello!
 | _abc_ | セン |
 | ----- | ---- |
 | this row has a space at the end | | 
-| But **_inline elements do_**. | x |
+| But _**inline elements do**_. | x |
 
 Hi!
 .
@@ -163,7 +163,7 @@ Hi!
 <td></td>
 </tr>
 <tr>
-<td>But <strong><em>inline elements do</em></strong>.</td>
+<td>But <em><strong>inline elements do</strong></em>.</td>
 <td>x</td>
 </tr></tbody></table>
 <p>Hi!</p>
@@ -255,7 +255,7 @@ Tables with embedded pipes could be tricky.
 | --- | --- |
 | Escaped pipes are \|okay\|. | Like \| this. |
 | Within `|code| is okay` too. |
-| **_`c|`_** \| complex
+| _**`c|`**_ \| complex
 | don't **\_reparse\_**
 .
 <table>
@@ -275,7 +275,7 @@ Tables with embedded pipes could be tricky.
 <td></td>
 </tr>
 <tr>
-<td><strong><em><code>c|</code></em></strong> | complex</td>
+<td><em><strong><code>c|</code></strong></em> | complex</td>
 <td></td>
 </tr>
 <tr>

data/ext/commonmarker/cmark/test/roundtrip_tests.py

@@ -14,6 +14,8 @@ if __name__ == "__main__":
             default=None, help='limit to sections matching regex pattern')
     parser.add_argument('--library-dir', dest='library_dir', nargs='?',
             default=None, help='directory containing dynamic library')
+    parser.add_argument('--extensions', dest='extensions', nargs='?',
+            default=None, help='space separated list of extensions to enable')
     parser.add_argument('--no-normalize', dest='normalize',
             action='store_const', const=False, default=True,
             help='do not normalize HTML')
@@ -23,11 +25,11 @@ if __name__ == "__main__":
 
 spec = sys.argv[1]
 
-def converter(md):
-  cmark = CMark(prog=args.program, library_dir=args.library_dir)
-  [ec, result, err] = cmark.to_commonmark(md)
+def converter(md, exts):
+  cmark = CMark(prog=args.program, library_dir=args.library_dir, extensions=args.extensions)
+  [ec, result, err] = cmark.to_commonmark(md, exts)
   if ec == 0:
-    [ec, html, err] = cmark.to_html(result)
+    [ec, html, err] = cmark.to_html(result, exts)
     if ec == 0:
         # In the commonmark writer we insert dummy HTML
         # comments between lists, and between lists and code

data/ext/commonmarker/cmark/test/spec.txt

@@ -3161,7 +3161,187 @@ aaa
 <h1>aaa</h1>
 ````````````````````````````````
 
+<div class="extension">
 
+## Tables (extension)
+
+If the `table` extension is enabled, an additional leaf block type is
+available
+
+A [table](@) is an arrangement of data with rows and columns, consisting of a
+single header row, a [delimiter row] separating the header from the data, and
+zero or more data rows.
+
+Each row consists of cells containing arbitrary text, in which [inlines] are
+parsed, separated by pipes (`|`).  A leading and trailing pipe is also
+recommended for clarity of reading, and if there's otherwise parsing ambiguity.
+Spaces between pipes and cell content are trimmed.  Block-level elements cannot
+be inserted in a table.
+
+The [delimiter row](@) consists of cells whose only content are hyphens (`-`),
+and optionally, a leading or trailing colon (`:`), or both, to indicate left,
+right, or center alignment respectively.
+
+```````````````````````````````` example table
+| foo | bar |
+| --- | --- |
+| baz | bim |
+.
+<table>
+<thead>
+<tr>
+<th>foo</th>
+<th>bar</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>baz</td>
+<td>bim</td>
+</tr></tbody></table>
+````````````````````````````````
+
+Cells in one column don't need to match length, though it's easier to read if
+they are. Likewise, use of leading and trailing pipes may be inconsistent:
+
+```````````````````````````````` example table
+| abc | defghi |
+:-: | -----------:
+bar | baz
+.
+<table>
+<thead>
+<tr>
+<th align="center">abc</th>
+<th align="right">defghi</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td align="center">bar</td>
+<td align="right">baz</td>
+</tr></tbody></table>
+````````````````````````````````
+
+Include a pipe in a cell's content by escaping it. Pipes inside other inline
+spans (such as emphasis, code, etc.) will not break a cell:
+
+```````````````````````````````` example table
+| f\|oo  |
+| ------ |
+| b `|` az |
+| b **|** im |
+.
+<table>
+<thead>
+<tr>
+<th>f|oo</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>b <code>|</code> az</td>
+</tr>
+<tr>
+<td>b <strong>|</strong> im</td>
+</tr></tbody></table>
+````````````````````````````````
+
+The table is broken at the first empty line, or beginning of another
+block-level structure:
+
+```````````````````````````````` example table
+| abc | def |
+| --- | --- |
+| bar | baz |
+> bar
+.
+<table>
+<thead>
+<tr>
+<th>abc</th>
+<th>def</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>bar</td>
+<td>baz</td>
+</tr></tbody></table>
+<blockquote>
+<p>bar</p>
+</blockquote>
+````````````````````````````````
+
+```````````````````````````````` example table
+| abc | def |
+| --- | --- |
+| bar | baz |
+bar
+
+bar
+.
+<table>
+<thead>
+<tr>
+<th>abc</th>
+<th>def</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>bar</td>
+<td>baz</td>
+</tr>
+<tr>
+<td>bar</td>
+<td></td>
+</tr></tbody></table>
+<p>bar</p>
+````````````````````````````````
+
+The header row must match the [delimiter row] in the number of cells.  If not,
+a table will not be recognized:
+
+```````````````````````````````` example table
+| abc | def |
+| --- |
+| bar |
+.
+<p>| abc | def |
+| --- |
+| bar |</p>
+````````````````````````````````
+
+The remainder of the table's rows may vary in the number of cells.  If there
+are a number of cells than the header, empty cells are inserted.  If there are
+greater, the excess is ignored:
+
+```````````````````````````````` example table
+| abc | def |
+| --- | --- |
+| bar |
+| bar | baz | boo |
+.
+<table>
+<thead>
+<tr>
+<th>abc</th>
+<th>def</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>bar</td>
+<td></td>
+</tr>
+<tr>
+<td>bar</td>
+<td>baz</td>
+</tr></tbody></table>
+````````````````````````````````
+
+</div>
 
 # Container blocks
 
@@ -5976,8 +6156,8 @@ the following principles resolve ambiguity:
     an interpretation `<strong>...</strong>` is always preferred to
     `<em><em>...</em></em>`.
 
-14. An interpretation `<strong><em>...</em></strong>` is always
-    preferred to `<em><strong>..</strong></em>`.
+14. An interpretation `<em><strong>...</strong></em>` is always
+    preferred to `<strong><em>..</em></strong>`.
 
 15. When two potential emphasis or strong emphasis spans overlap,
     so that the second begins before the first ends and ends after
@@ -7000,14 +7180,14 @@ Rule 14:
 ```````````````````````````````` example
 ***foo***
 .
-<p><strong><em>foo</em></strong></p>
+<p><em><strong>foo</strong></em></p>
 ````````````````````````````````
 
 
 ```````````````````````````````` example
 _____foo_____
 .
-<p><strong><strong><em>foo</em></strong></strong></p>
+<p><em><strong><strong>foo</strong></strong></em></p>
 ````````````````````````````````
 
 
@@ -7108,6 +7288,43 @@ __a<http://foo.bar/?q=__>
 ````````````````````````````````
 
 
+<div class="extension">
+
+## Strikethrough (extension)
+
+If the `strikethrough` extension is enabled, an additional emphasis type is
+available.
+
+Strikethrough text is any text wrapped in tildes (`~`).
+
+```````````````````````````````` example strikethrough
+~Hi~ Hello, world!
+.
+<p><del>Hi</del> Hello, world!</p>
+````````````````````````````````
+
+Any number of tildes may be used on either side of the text; they do not need
+to match, and they cannot be nested.
+
+```````````````````````````````` example strikethrough
+This ~text~~~~ is ~~~~curious~.
+.
+<p>This <del>text</del> is <del>curious</del>.</p>
+````````````````````````````````
+
+As with regular emphasis delimiters, a new paragraph will cause the cessation
+of parsing a strikethrough:
+
+```````````````````````````````` example strikethrough
+This ~~has a
+
+new paragraph~~.
+.
+<p>This ~~has a</p>
+<p>new paragraph~~.</p>
+````````````````````````````````
+
+</div>
 
 ## Links
 
@@ -8529,6 +8746,166 @@ foo@bar.example.com
 <p>foo@bar.example.com</p>
 ````````````````````````````````
 
+<div class="extension">
+
+## Autolinks (extension)
+
+If the `autolink` extension is enabled, autolinks will be recognised in a
+greater number of conditions.
+
+[Autolink]s can also be constructed without requiring the use of `<` and to `>`
+to delimit them, although they will be recognized under a smaller set of
+circumstances.  All such recognized autolinks can only come after whitespace,
+or any of the delimiting characters `*`, `_`, `~`, `(`, and `[`.
+
+An [extended www autolink](@) will be recognized when a [valid domain] is
+found. A [valid domain](@) consists of the text `www.`, followed by
+alphanumeric characters, underscores (`_`), hyphens (`-`) and periods (`.`).
+There must be at least one period, and no underscores may be present in the
+last two segments of the domain.
+
+The scheme `http` will be inserted automatically:
+
+```````````````````````````````` example autolink
+www.commonmark.org
+.
+<p><a href="http://www.commonmark.org">www.commonmark.org</a></p>
+````````````````````````````````
+
+After a [valid domain], zero or more non-space non-`<` characters may follow:
+
+```````````````````````````````` example autolink
+Visit www.commonmark.org/help for more information.
+.
+<p>Visit <a href="http://www.commonmark.org/help">www.commonmark.org/help</a> for more information.</p>
+````````````````````````````````
+
+We then apply [extended autolink path validation](@) as follows:
+
+Trailing punctuation (specifically, `?`, `!`, `.`, `,`, `:`, `*`, `_`, and `~`)
+will not be considered part of the autolink, though they may be included in the
+interior of the link:
+
+```````````````````````````````` example autolink
+Visit www.commonmark.org.
+
+Visit www.commonmark.org/a.b.
+.
+<p>Visit <a href="http://www.commonmark.org">www.commonmark.org</a>.</p>
+<p>Visit <a href="http://www.commonmark.org/a.b">www.commonmark.org/a.b</a>.</p>
+````````````````````````````````
+
+When an autolink ends in `)`, we scan the entire autolink for the total number
+of parentheses.  If there is a greater number of closing parentheses than
+opening ones, we don't consider the last character part of the autolink, in
+order to facilitate including an autolink inside a parenthesis:
+
+```````````````````````````````` example autolink
+www.google.com/search?q=Markup+(business)
+
+(www.google.com/search?q=Markup+(business))
+.
+<p><a href="http://www.google.com/search?q=Markup+(business)">www.google.com/search?q=Markup+(business)</a></p>
+<p>(<a href="http://www.google.com/search?q=Markup+(business)">www.google.com/search?q=Markup+(business)</a>)</p>
+````````````````````````````````
+
+This check is only done when the link ends in a closing parentheses `)`, so if
+the only parentheses are in the interior of the autolink, no special rules are
+applied:
+
+```````````````````````````````` example autolink
+www.google.com/search?q=(business))+ok
+.
+<p><a href="http://www.google.com/search?q=(business))+ok">www.google.com/search?q=(business))+ok</a></p>
+````````````````````````````````
+
+If an autolink ends in a semicolon (`;`), we check to see if it appears to
+resemble an [entity reference][entity references]; if the preceding text is `&`
+followed by one or more alphanumeric characters.  If so, it is excluded from
+the autolink:
+
+```````````````````````````````` example autolink
+www.google.com/search?q=commonmark&hl=en
+
+www.google.com/search?q=commonmark&hl;
+.
+<p><a href="http://www.google.com/search?q=commonmark&amp;hl=en">www.google.com/search?q=commonmark&amp;hl=en</a></p>
+<p><a href="http://www.google.com/search?q=commonmark">www.google.com/search?q=commonmark</a>&amp;hl;</p>
+````````````````````````````````
+
+`<` immediately ends an autolink.
+
+```````````````````````````````` example autolink
+www.commonmark.org/he<lp
+.
+<p><a href="http://www.commonmark.org/he">www.commonmark.org/he</a>&lt;lp</p>
+````````````````````````````````
+
+An [extended url autolink](@) will be recognised when one of the schemes
+`http://`, `https://`, or `ftp://`, followed by a [valid domain], then zero or
+more non-space non-`<` characters according to
+[extended autolink path validation]:
+
+```````````````````````````````` example autolink
+http://commonmark.org
+
+(Visit https://encrypted.google.com/search?q=Markup+(business))
+
+Anonymous FTP is available at ftp://foo.bar.baz.
+.
+<p><a href="http://commonmark.org">http://commonmark.org</a></p>
+<p>(Visit <a href="https://encrypted.google.com/search?q=Markup+(business)">https://encrypted.google.com/search?q=Markup+(business)</a>)</p>
+<p>Anonymous FTP is available at <a href="ftp://foo.bar.baz">ftp://foo.bar.baz</a>.</p>
+````````````````````````````````
+
+
+An [extended email autolink](@) will be recognised when an email address is
+recognised within any text node.  Email addresses are recognised according to
+the following rules:
+
+* One ore more characters which are alphanumeric, or `.`, `-`, `_`, or `+`.
+* An `@` symbol.
+* One or more characters which are alphanumeric, or `.`, `-`, or `_`. At least
+  one of the characters here must be a period (`.`).  The last character must
+  not be one of `-` or `_`.  If the last character is a period (`.`), it will
+  be excluded from the autolink.
+
+The scheme `mailto:` will automatically be added to the generated link:
+
+```````````````````````````````` example autolink
+foo@bar.baz
+.
+<p><a href="mailto:foo@bar.baz">foo@bar.baz</a></p>
+````````````````````````````````
+
+`+` can occur before the `@`, but not after.
+
+```````````````````````````````` example autolink
+hello@mail+xyz.example isn't valid, but hello+xyz@mail.example is.
+.
+<p>hello@mail+xyz.example isn't valid, but <a href="mailto:hello+xyz@mail.example">hello+xyz@mail.example</a> is.</p>
+````````````````````````````````
+
+`.`, `-`, and `_` can occur on both sides of the `@`, but only `.` may occur at
+the end of the email address, in which case it will not be considered part of
+the address:
+
+```````````````````````````````` example autolink
+a.b-c_d@a.b
+
+a.b-c_d@a.b.
+
+a.b-c_d@a.b-
+
+a.b-c_d@a.b_
+.
+<p><a href="mailto:a.b-c_d@a.b">a.b-c_d@a.b</a></p>
+<p><a href="mailto:a.b-c_d@a.b">a.b-c_d@a.b</a>.</p>
+<p>a.b-c_d@a.b-</p>
+<p>a.b-c_d@a.b_</p>
+````````````````````````````````
+
+</div>
 
 ## Raw HTML
 
@@ -8800,6 +9177,45 @@ foo <a href="\*">
 ````````````````````````````````
 
 
+<div class="extension">
+
+## Raw HTML (extension)
+
+If the `tagfilter` extension is enabled, the following HTML tags will be
+filtered when rendering HTML output:
+
+* `<title>`
+* `<textarea>`
+* `<style>`
+* `<xmp>`
+* `<iframe>`
+* `<noembed>`
+* `<noframes>`
+* `<script>`
+* `<plaintext>`
+
+Filtering is done by replacing the leading `<` with the entity `&lt;`.  These
+tags are chosen in particular as they change how HTML is interpreted in a way
+unique to them (i.e. nested HTML is interpreted differently), and this is
+usually undesireable in the context of other rendered Markdown content.
+
+All other HTML tags are left untouched.
+
+```````````````````````````````` example tagfilter
+<strong> <title> <style> <em>
+
+<blockquote>
+  <xmp> is disallowed.
+</blockquote>
+.
+<p><strong> &lt;title> &lt;style> <em></p>
+<blockquote>
+  &lt;xmp> is disallowed.
+</blockquote>
+````````````````````````````````
+
+</div>
+
 ## Hard line breaks
 
 A line break (not in a code span or HTML tag) that is preceded