markly 0.18.0 → 0.19.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a6df3d157ca441e13f840de336bf04408669aae72ff791d35be091ee008da87c
-  data.tar.gz: 682356cc37b69d2a486624ffd0e0c02f75a717b07f32c0a41d838e2032730223
+  metadata.gz: e8acce4b2ae6c9e79d70bb21548ece0ec72bf0ef3e7cd77aaae270cc408daa2c
+  data.tar.gz: d48929fd7d802c68ad1ef18bb186f3f3b349d147be7b9c64dddb853f361eedac
 SHA512:
-  metadata.gz: 70315346cac721d22cbc67c0efaee8bd6a035198fec894388a827136c3710d60844c24e29a26a4374a899af01212618412d929acfd840db03ce887f79e2b80ef
-  data.tar.gz: 895682bb5cd75d95306c2e60b9bb446b4459ade9800d8a29e5ecd13b4fb9f8e1caa33518ac1d3f6779143af9f213d50a0c17460ded2fc94eea57872e7aba9c26
+  metadata.gz: e7f9545009613d2e140762a337babf3ddcac13520b585d8fa07ec4ab6c90a7248a4be8df2998afbdb9a4428c315bc969426f05d521219aba671dca05371f1e1c
+  data.tar.gz: 460d177e3361f7416b7d20ab48e900f9635c83c48c132bf489a025a50bfd532c678e0b9e7467d1e7441ef362030e0c96bd5f13a8e1c0d39f9746cd096934c4e8

data/context/extensions.md

@@ -0,0 +1,232 @@
+# Extensions
+
+This guide explains how to enable and use Markly's Markdown extensions.
+
+## Choosing a Markdown Dialect
+
+Markly parses standard CommonMark by default. Extensions change which syntax is
+recognized, so applications should enable them explicitly and consistently.
+This is especially important when several components parse or render the same
+document.
+
+Pass extension names as symbols using the `extensions:` keyword:
+
+``` ruby
+EXTENSIONS = %i[table tasklist strikethrough autolink].freeze
+
+html = Markly.render_html(markdown, extensions: EXTENSIONS)
+```
+
+When parsing and rendering separately, use the same configuration at both
+boundaries:
+
+``` ruby
+document = Markly.parse(markdown, extensions: EXTENSIONS)
+html = document.to_html(extensions: EXTENSIONS)
+```
+
+Extension names must be symbols. A string such as `"table"` raises `TypeError`,
+and an unknown symbol raises `ArgumentError`.
+
+## GitHub Flavored Markdown Extensions
+
+Markly includes the five syntax extensions defined by GitHub Flavored Markdown.
+None are enabled by default.
+
+### Tables
+
+The `:table` extension recognizes a paragraph followed by a delimiter row as a
+table. Colons in the delimiter row specify column alignment:
+
+``` ruby
+markdown = <<~MARKDOWN
+	| Package | Status |
+	| :--- | ---: |
+	| Markly | Ready |
+MARKDOWN
+
+document = Markly.parse(markdown, extensions: [:table])
+table = document.first_child
+
+table.type
+# => :table
+
+table.table_alignments
+# => [:left, :right]
+```
+
+The table AST contains `:table`, `:table_header`, `:table_row`, and
+`:table_cell` nodes. By default, HTML rendering uses `align` attributes for
+aligned cells. Pass `Markly::TABLE_PREFER_STYLE_ATTRIBUTES` when rendering to
+use `style="text-align: ..."` instead:
+
+``` ruby
+document.to_html(
+	flags: Markly::TABLE_PREFER_STYLE_ATTRIBUTES,
+	extensions: [:table],
+)
+```
+
+### Task Lists
+
+The `:tasklist` extension recognizes checked and unchecked list items:
+
+``` markdown
+- [x] Parse Markdown
+- [ ] Render HTML
+```
+
+Task-list state is available on the list-item node and can be changed before
+rendering:
+
+``` ruby
+document = Markly.parse("- [x] Parse Markdown", extensions: [:tasklist])
+item = document.first_child.first_child
+
+item.tasklist_item_checked?
+# => true
+
+item.tasklist_item_checked = false
+```
+
+### Strikethrough
+
+The `:strikethrough` extension renders strikethrough text using `<del>`:
+
+``` ruby
+Markly.render_html("~~obsolete~~", extensions: [:strikethrough])
+# => "<p><del>obsolete</del></p>\n"
+```
+
+Pass `Markly::STRIKETHROUGH_DOUBLE_TILDE` while parsing to accept only spans
+surrounded by exactly two tildes. This is useful when compatibility requires
+single or longer runs of tildes to remain literal:
+
+``` ruby
+Markly.render_html(
+	"~one~ ~~two~~ ~~~three~~~",
+	flags: Markly::STRIKETHROUGH_DOUBLE_TILDE,
+	extensions: [:strikethrough],
+)
+# => "<p>~one~ <del>two</del> ~~~three~~~</p>\n"
+```
+
+### Autolinks
+
+The `:autolink` extension turns plain URLs and email addresses into links
+without requiring angle brackets or Markdown link syntax:
+
+``` ruby
+Markly.render_html(
+	"Visit https://socketry.io or email hello@example.com.",
+	extensions: [:autolink],
+)
+```
+
+Use this extension when rendering prose where authors expect GitHub-style link
+detection. Leave it disabled when plain URL-like text must remain unchanged.
+
+### Tag Filtering
+
+The `:tagfilter` extension escapes the raw HTML tags prohibited by the GitHub
+Flavored Markdown specification. It is typically combined with
+`Markly::UNSAFE`, which otherwise permits raw HTML:
+
+``` ruby
+Markly.render_html(
+	"<script>alert('no')</script><strong>yes</strong>",
+	flags: Markly::UNSAFE,
+	extensions: [:tagfilter],
+)
+```
+
+Tag filtering is not a general-purpose HTML sanitizer. It filters the specific
+tag names required by the GFM specification, while other raw HTML remains
+available when `Markly::UNSAFE` is enabled. Sanitize untrusted HTML separately
+when the application requires a stricter policy.
+
+## Markly Syntax Features
+
+Markly also provides syntax features that are not named GFM extensions. Enable
+these with parser flags rather than adding names to `extensions:`.
+
+### Front Matter
+
+`Markly::FRONT_MATTER` recognizes a `---` delimited block only at the beginning
+of a document:
+
+``` ruby
+markdown = <<~MARKDOWN
+	--- yaml
+	title: Extensions
+	---
+	# Document
+MARKDOWN
+
+document = Markly.parse(markdown, flags: Markly::FRONT_MATTER)
+front_matter = document.first_child
+
+front_matter.type
+# => :front_matter
+
+front_matter.string_content
+# => "title: Extensions\n"
+
+front_matter.code_info
+# => "yaml"
+```
+
+Front matter is omitted from HTML and plain-text output. Markly exposes its raw
+contents but does not interpret YAML, TOML, or any other format. Treat the
+contents as untrusted input and parse them according to the application's own
+policy.
+
+The closing delimiter must be an exact `---` line. If it is missing, the rest
+of the document belongs to the front-matter node.
+
+### Inline Code Information
+
+`Markly::INLINE_CODE_INFO` recognizes a language prefix immediately before an
+inline code span:
+
+``` ruby
+document = Markly.parse(
+	"ruby:`Object.new`",
+	flags: Markly::INLINE_CODE_INFO,
+)
+code = document.first_child.first_child
+
+code.code_info
+# => "ruby"
+
+code.code_language
+# => "ruby"
+
+document.to_html
+# => "<p><code class=\"language-ruby\">Object.new</code></p>\n"
+```
+
+Without the flag, the prefix remains ordinary text. Inline code information is
+a single language token; richer code-block information belongs on a fenced code
+block instead.
+
+### Code Block Metadata
+
+`Node#code_info` is the general information-string accessor for fenced code
+blocks and front matter. `Node#code_language` returns the first token of that
+information string.
+
+For a fenced code block, `Node#fence` returns a {ruby Markly::Node::Fence}
+containing the fence character, length, and indentation:
+
+``` ruby
+block = Markly.parse("  ~~~~ ruby\n  Object.new\n  ~~~~").first_child
+
+block.code_info
+# => "ruby"
+
+block.fence
+# => #<struct Markly::Node::Fence character="~", length=4, indent=2>
+```
+
+Indented code blocks and other node types return `nil` from `Node#fence`.

data/context/getting-started.md

@@ -78,57 +78,21 @@ To have multiple options applied, `|` (or) the flags together:
 Markly.render_html("\"'Shelob' is my name.\"", flags: Markly::HARD_BREAKS|Markly::SOURCE_POSITION)
 ```
 
-Inline code language prefixes are opt-in. The language is available through
-`Node#code_info` and is rendered as a `language-...` class:
-
-``` ruby
-document = Markly.parse("ruby:`Object.new`", flags: Markly::INLINE_CODE_INFO)
-code = document.first_child.first_child
-
-code.code_info
-# => "ruby"
-
-code.code_language
-# => "ruby"
-
-document.to_html
-# => <p><code class="language-ruby">Object.new</code></p>
-```
-
-`Node#code_info` is also the general info-string accessor for fenced code
-blocks and front matter. `Node#code_language` returns the first token of that
-info string, while `Node#fence_info` remains available for compatibility on
-those block nodes.
-
-For a fenced code block, `Node#fence` returns a `Node::Fence` structure with
-the fence character, length, and indentation:
-
-``` ruby
-block = Markly.parse("  ~~~~ ruby\n  Object.new\n  ~~~~").first_child
-
-block.fence
-# => #<struct Markly::Node::Fence character="~", length=4, indent=2>
-```
-
-Indented code blocks and other node types return `nil`.
-
 ## Extensions
 
-Both `render_html` and `parse` take an optional `extensions:` argument defining the extensions you want enabled as your CommonMark document is being processed:
+Markly parses standard CommonMark by default. GitHub Flavored Markdown syntax
+and Markly-specific syntax are opt-in so applications can choose their accepted
+Markdown dialect explicitly:
 
 ``` ruby
-Markly.render_html("<script>hi</script>", flags: Markly::UNSAFE, extensions: [:tagfilter])
+Markly.render_html(
+	"| Name | Status |\n| --- | --- |\n| Markly | Ready |",
+	extensions: [:table],
+)
 ```
 
-The documentation for these extensions are [defined in this spec](https://github.github.com/gfm/), and the rationale is provided [in this blog post](https://githubengineering.com/a-formal-spec-for-github-markdown/).
-
-The available extensions are:
-
-  - `:table` - This provides support for tables.
-  - `:tasklist` - This provides support for task list items.
-  - `:strikethrough` - This provides support for strikethroughs.
-  - `:autolink` - This provides support for automatically converting URLs to anchor tags.
-  - `:tagfilter` - This escapes [several "unsafe" HTML tags](https://github.github.com/gfm/#disallowed-raw-html-extension-), causing them to not have any effect.
+See [Extensions](../extensions/index) for the supported extensions, related
+flags, and generated AST.
 
 ## Developing Locally
 

data/context/index.yaml

@@ -10,6 +10,9 @@ files:
 - path: getting-started.md
   title: Getting Started
   description: This guide explains now to install and use Markly.
+- path: extensions.md
+  title: Extensions
+  description: This guide explains how to enable and use Markly's Markdown extensions.
 - path: abstract-syntax-tree.md
   title: Abstract Syntax Tree
   description: This guide explains how to use Markly's abstract syntax tree (AST)

data/ext/markly/blocks.c

@@ -91,8 +91,8 @@ static CMARK_INLINE bool S_ends_on_current_line(cmark_parser *parser, cmark_node
          // similar to fenced code blocks.
          // Types 6-7 end at a blank line, so their last content line is
          // the previous line and they should NOT match here.
-         (S_type(b) == CMARK_NODE_HTML_BLOCK && b->as.html_block_type >= 1 &&
-          b->as.html_block_type <= 5) ||
+         (S_type(b) == CMARK_NODE_HTML_BLOCK && b->as.html_block.type >= 1 &&
+          b->as.html_block.type <= 5) ||
          // Single-line blocks: finalized on same line they started
          b->start_line == parser->line_number;
 }
@@ -1037,7 +1037,7 @@ static bool parse_code_block_prefix(cmark_parser *parser, cmark_chunk *input,
 static bool parse_html_block_prefix(cmark_parser *parser,
                                     cmark_node *container) {
   bool res = false;
-  int html_block_type = container->as.html_block_type;
+  int html_block_type = container->as.html_block.type;
 
   assert(html_block_type >= 1 && html_block_type <= 7);
   switch (html_block_type) {
@@ -1051,7 +1051,32 @@ static bool parse_html_block_prefix(cmark_parser *parser,
     break;
   case 6:
   case 7:
-    res = !parser->blank;
+    if (!(parser->options & CMARK_OPT_HTML_BLOCK_BLANK_LINES)) {
+      res = !parser->blank;
+    } else if (parser->blank) {
+      // Tentatively retain blank lines. The next nonblank line determines
+      // whether the HTML block continues:
+      res = true;
+    } else if (S_last_line_blank(container)) {
+      // Establish the content indentation lazily so a blank line may follow
+      // the opening tag. A non-indented line still terminates the block:
+      if (container->as.html_block.indent == 0 && parser->indent > 0) {
+        container->as.html_block.indent = parser->indent;
+      }
+
+      res = container->as.html_block.indent > 0 &&
+            parser->indent >= container->as.html_block.indent;
+    } else {
+      // Record the shallowest positive content indentation before a blank
+      // line. Deeper nested HTML may then continue without changing it:
+      if (parser->indent > 0 &&
+          (container->as.html_block.indent == 0 ||
+           parser->indent < container->as.html_block.indent)) {
+        container->as.html_block.indent = parser->indent;
+      }
+
+      res = true;
+    }
     break;
   }
 
@@ -1224,7 +1249,7 @@ static void open_new_blocks(cmark_parser *parser, cmark_node **container,
                                    input, parser->first_nonspace))))) {
       *container = add_child(parser, *container, CMARK_NODE_HTML_BLOCK,
                              parser->first_nonspace + 1);
-      (*container)->as.html_block_type = matched;
+      (*container)->as.html_block.type = matched;
       // note, we don't adjust parser->offset because the tag is part of the
       // text
     } else if (!indented && cont_type == CMARK_NODE_PARAGRAPH &&
@@ -1421,7 +1446,7 @@ static void add_text_to_container(cmark_parser *parser, cmark_node *container,
       add_line(container, input, parser);
 
       int matches_end_condition;
-      switch (container->as.html_block_type) {
+      switch (container->as.html_block.type) {
       case 1:
         // </script>, </style>, </pre>
         matches_end_condition =

data/ext/markly/cmark-gfm.h

@@ -798,6 +798,12 @@ char *cmark_render_latex_with_mem(cmark_node *root, int options, int width, cmar
  */
 #define CMARK_OPT_INLINE_CODE_INFO (1 << 19)
 
+/** Allow indented content in type 6 and 7 HTML blocks to continue across
+ * blank lines. The indentation established by the HTML content must be
+ * preserved after each blank line.
+ */
+#define CMARK_OPT_HTML_BLOCK_BLANK_LINES (1 << 20)
+
 /**
  * ## Version information
  */

data/ext/markly/node.h

@@ -33,6 +33,11 @@ typedef struct {
   int8_t fenced;
 } cmark_code;
 
+typedef struct {
+  int type;
+  int indent;
+} cmark_html_block;
+
 typedef struct {
   int level;
   bool setext;
@@ -104,7 +109,7 @@ struct cmark_node {
     cmark_heading heading;
     cmark_link link;
     cmark_custom custom;
-    int html_block_type;
+    cmark_html_block html_block;
     int cell_index; // For keeping track of TABLE_CELL table alignments
     void *opaque;
   } as;

data/lib/markly/flags.rb

@@ -25,10 +25,13 @@ module Markly
 	FRONT_MATTER = 1 << 18
 	# Parse language prefixes on inline code spans, e.g. ruby:`Object.new`.
 	INLINE_CODE_INFO = 1 << 19
+	# Allow consistently indented HTML content to continue across blank lines.
+	HTML_BLOCK_BLANK_LINES = 1 << 20
 	
 	PARSE_FLAGS = {
 		front_matter: FRONT_MATTER,
 		inline_code_info: INLINE_CODE_INFO,
+		html_block_blank_lines: HTML_BLOCK_BLANK_LINES,
 		validate_utf8: VALIDATE_UTF8,
 		smart_quotes: SMART,
 		liberal_html_tags: LIBERAL_HTML_TAG,

data/lib/markly/version.rb

@@ -9,5 +9,5 @@
 # @namespace
 module Markly
 	# @constant [String] The version of the Markly gem.
-	VERSION = "0.18.0"
+	VERSION = "0.19.0"
 end

data/readme.md

@@ -16,6 +16,8 @@ Please see the [project documentation](https://socketry.github.io/markly/) for m
 
   - [Getting Started](https://socketry.github.io/markly/guides/getting-started/index) - This guide explains now to install and use Markly.
 
+  - [Extensions](https://socketry.github.io/markly/guides/extensions/index) - This guide explains how to enable and use Markly's Markdown extensions.
+
   - [Abstract Syntax Tree](https://socketry.github.io/markly/guides/abstract-syntax-tree/index) - This guide explains how to use Markly's abstract syntax tree (AST) to parse and manipulate Markdown documents.
 
   - [Headings](https://socketry.github.io/markly/guides/headings/index) - This guide explains how to work with headings in Markly, including extracting them for navigation and handling duplicate heading text.
@@ -24,6 +26,10 @@ Please see the [project documentation](https://socketry.github.io/markly/) for m
 
 Please see the [project releases](https://socketry.github.io/markly/releases/index) for all releases.
 
+### v0.19.0
+
+  - Add `Markly::HTML_BLOCK_BLANK_LINES` for keeping consistently indented HTML content together across blank lines.
+
 ### v0.18.0
 
   - Preserve complete node and extension metadata when duplicating node trees.

data/releases.md

@@ -1,5 +1,9 @@
 # Releases
 
+## v0.19.0
+
+  - Add `Markly::HTML_BLOCK_BLANK_LINES` for keeping consistently indented HTML content together across blank lines.
+
 ## v0.18.0
 
   - Preserve complete node and extension metadata when duplicating node trees.

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: markly
 version: !ruby/object:Gem::Version
-  version: 0.18.0
+  version: 0.19.0
 platform: ruby
 authors:
 - Garen Torikian
@@ -65,6 +65,7 @@ extensions:
 extra_rdoc_files: []
 files:
 - context/abstract-syntax-tree.md
+- context/extensions.md
 - context/getting-started.md
 - context/headings.md
 - context/index.yaml