red_quilt 0.7.2 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 689739c1f6cf971cbeaacdbb65b50fcf7600cecef38bc25aa607ab5e87a559e5
-  data.tar.gz: 2d57a5a5993bfb8352c18ec8c6da5dc06169d59198e72870399c85d4cc1e1769
+  metadata.gz: 211bfba5adede6552fb64c3dfadf6c895325af2c58a577eadb86c8d426a3dd0b
+  data.tar.gz: 2e542ae344e626dc5a1bf0198b80761dfac3317b48f11011569235e5418048c9
 SHA512:
-  metadata.gz: b3f3b90749d7307db9121d9f1af968a72e94cad5aa06a3f9cf54505ff047e482246f5aa2c0fa14f1b74d9b3d03b516629b3f1c4e57e5ecb6cf5d171d4bc2b6f6
-  data.tar.gz: f8a8929d8075e0c9e3ec32145daac57ca4565ff299bed3f119faee01f5253727d44acea4dacac579ee676715138e4f03f16e0555544b4b25ee1e30c779db1de5
+  metadata.gz: 8526155c616d353741dd68ad7740a0ad3d18e08139c239a6d2ff2d48198721e4b1a84a8da3eca98dae033d1da73a59817a766fe0b3bf89391897559a0fc37047
+  data.tar.gz: 272a4918326acc93dc8699308a8597c9e621f98518f0307ef7a8f2eed9ba03dc3153f2fbea641e6b02eb76af0adf612a7d8e8fca7b08360c130d1e34b7b5266c

data/CHANGELOG.md

@@ -7,8 +7,107 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.9.0] - 2026-07-20
+
+### Changed
+
+- **Breaking: source positions now follow the unist Point convention**, which
+  cmark sourcepos and mdast both use. Three related changes:
+  - `source_location` columns are 1-based (they were 0-based; lines were
+    already 1-based). Both are counted in characters, and `end` remains
+    exclusive.
+  - Block spans cover the block as authored rather than its content alone.
+    `# H1` now starts at the `#`, `> quote` at the `>`, a list item at its
+    bullet, a fenced code block includes both fences, and a setext heading
+    includes its `===` / `---` underline.
+  - Leading indent is excluded from block spans, so `  text` starts at
+    column 3 rather than column 1. This applies to ATX and setext headings,
+    paragraphs, blockquotes, fenced code blocks, lists, list items, and
+    thematic breaks.
+  - The fenced code block change also fixes a wrong *line* number: the span
+    used to start at the first content line, reporting the block one line
+    below its opening fence.
+
+  Every expected value was verified against commonmark.js and
+  mdast-util-from-markdown; RedQuilt now agrees with mdast on all of them
+  except an unclosed fence, where the two references disagree and RedQuilt
+  follows cmark. Callers that added 1 to columns, or that relied on a
+  heading span excluding `# `, need updating.
+
+  `NodeRef#text` is unaffected: a heading's text remains its content, without
+  the marker.
+
+- **Breaking: `NodeRef#info` returns nil for nodes that are not code
+  blocks**, where 0.8.0 returned `""`. A code block written without an info
+  string still returns `""`, so the two cases are now distinguishable. This
+  matches the new attribute accessors, which all use nil for "this node's
+  type does not carry the attribute".
+- Lowered the minimum supported Ruby from 3.3 to 3.1. The only 3.1
+  incompatibility was `String#byteindex` (added in Ruby 3.2), used in two
+  inline hot paths; both operate on a binary (`String#b`) view of the source,
+  where `String#index` returns the same byte offsets. CI now runs the test
+  suite on 3.1 through 4.0.
+- `Gemfile.lock` is no longer committed. A lockfile resolved on one Ruby pins
+  development gem versions that do not exist for the others (e.g. `rdoc 8.0.0`
+  and `rbs 4.0.3` have no Ruby 3.1 release), so a single committed lockfile
+  cannot serve the supported range. Each Ruby now resolves its own.
+
+### Fixed
+
+- `source_location` reported wrong line numbers for any source containing a
+  multibyte character. The line-start table is indexed by byte offset, but it
+  was built with `String#index`, which counts characters, so every line after
+  the first multibyte character drifted. Existing multibyte coverage did not
+  catch this because it used single-line sources.
+- `Renderer::MDAST` emitted positions that violated the unist spec: `column`
+  was 0-based where unist requires 1-based, and `offset` was a byte index
+  where unist requires a 0-based character index. The latter made every
+  offset after a multibyte character wrong.
+
+### Added
+
+- Node attribute accessors on `NodeRef`: `heading_level`, `list_ordered?`,
+  `list_start`, `list_tight?`, `list_delimiter`, `link_destination`,
+  `link_title`, `footnote_label`, `footnote_number`, and `header?`. Each
+  returns nil when the node's type does not carry the attribute, so callers
+  can walk a document branching on `#type` and read that type's fields
+  directly. Previously the only public route was `#to_h`, which builds a Hash
+  for the whole subtree — 5391 objects for a 257-node list where these
+  accessors allocate none.
+
+  These wrappers are the safe way to read attributes. The `Arena` accessors
+  they delegate to are the raw layer and skip the type check, and several
+  attributes share a storage column, so reading one off a mismatched node
+  there returns another field's value rather than nil (`Arena#link_destination`
+  on a paragraph returns the paragraph's text).
+
+## [0.8.0] - 2026-06-24
+
+### Added
+
+- `NodeRef#info`: returns the fence info string of a code block (e.g. `ruby`
+  in ` ```ruby `, or `vtt audio="x.mp3"`); `""` for code blocks without one
+  and for every other node type. The raw code content remains available via
+  `NodeRef#text`.
+- `Renderer::HTML#render_fragment(nodes)`: renders an Array of `NodeRef` in
+  order and returns the HTML fragment without affecting the main render
+  output. Renderer state shared across nodes (e.g. the heading-id slugger) is
+  preserved between calls. Lets callers that partition a document render the
+  pieces separately without reaching into renderer internals.
+- `Arena` semantic payload accessors (`heading_level`, `list_ordered?`,
+  `code_block_info`, `link_destination`, `footnote_number`, …) for callers
+  that walk `Document#arena`, replacing direct use of the raw `int1`/`str2`
+  columns.
+
+## [0.7.2] - 2026-06-23
+
 ### Added
 
+- Opt-in YAML frontmatter support via the `frontmatter:` option on `parse` /
+  `render_html` and the `--frontmatter` CLI flag (off by default). A leading
+  `---` … `---` block is removed from the rendered body and exposed as
+  `Document#frontmatter`; in standalone output its `title` / `lang` keys fill
+  in `<title>` / `<html lang>`.
 - Opt-in Mermaid diagram support via the `mermaid:` option on `render_html` /
   `Document#to_html` and the `--mermaid` CLI flag (off by default). Fenced
   ` ```mermaid ` code blocks render as `<pre class="mermaid">` containers; in

data/README.md

@@ -48,6 +48,7 @@ RedQuilt.render_html("Hi <em>tag</em>", allow_html: true)
 | `extended_autolinks:` | `false` | GFM: linkify bare `http(s)://` / `www.` / email addresses |
 | `footnotes:` | `false` | GFM footnotes (see below) |
 | `lint:` | `false` | Collect lint diagnostics (empty links, missing image alt, heading-level skips) |
+| `frontmatter:` | `false` | Parse leading YAML frontmatter (`---`) as metadata (see below) |
 
 ### Footnotes (opt-in)
 
@@ -100,6 +101,41 @@ RedQuilt.parse("```mermaid\ngraph LR\n  A --> B\n```")
 In standalone output each diagram is made interactive with
 [svg-pan-zoom](https://github.com/bumbu/svg-pan-zoom) (loaded from a CDN).
 
+### YAML frontmatter (opt-in)
+
+`parse` / `render_html` accept `frontmatter:` to extract a leading YAML
+frontmatter block (the `---` … `---` fences used by Jekyll, Hugo).
+The block is parsed with `Psych.safe_load` and removed from the rendered body;
+the parsed Hash is exposed as `Document#frontmatter`.
+
+```ruby
+doc = RedQuilt.parse(<<~MD, frontmatter: true)
+  ---
+  title: My Page
+  lang: ja
+  ---
+
+  # Body
+MD
+doc.frontmatter            # => {"title" => "My Page", "lang" => "ja"}
+doc.to_html               # => "<h1>Body</h1>\n"  (frontmatter stripped)
+```
+
+In standalone output the frontmatter's `title` / `lang` fill in `<title>` /
+`<html lang>` when no explicit argument is given (explicit argument >
+frontmatter > default):
+
+```ruby
+doc.to_html(standalone: true)
+# <html lang="ja"> … <title>My Page</title> …
+```
+
+The feature is opt-in, so a bare `---` is never mistaken for frontmatter
+unless `frontmatter: true` is passed. Frontmatter lines are blanked rather
+than deleted, so body source spans and diagnostic line numbers stay relative
+to the start of the file. Invalid YAML records a `:frontmatter` warning
+diagnostic and leaves `Document#frontmatter` as `nil` without raising.
+
 ### Tilt integration
 
 RedQuilt ships a [Tilt](https://github.com/jeremyevans/tilt) adapter.
@@ -186,6 +222,8 @@ redquilt --mermaid --open input.md
                          Dir.tmpdir when -o is not given)
 --mermaid                Render `mermaid` code blocks as diagrams (loads
                          mermaid.js from a CDN in standalone output)
+--frontmatter            Parse leading YAML frontmatter (---) as metadata;
+                         fills <title>/lang in standalone output
 --diagnostics            Print diagnostics to stderr
 --diagnostics-only       Print diagnostics only (suppress output)
 -h, --help               Show help

data/docs/api.md

@@ -18,6 +18,7 @@ doc.source_map        # Line/column lookup (lazy memoized)
 doc.diagnostics       # Array of RedQuilt::Diagnostic collected while parsing
 doc.allow_html?       # Check HTML pass-through setting
 doc.disallow_raw_html? # Check GFM disallowed-raw-HTML filtering setting
+doc.frontmatter       # Parsed YAML frontmatter Hash, or nil (see below)
 
 # Standalone document with an embedded theme:
 doc.to_html(standalone: true, theme: :default, title: "My Doc", lang: "en")
@@ -27,6 +28,15 @@ doc.to_html(standalone: true, theme: :default, title: "My Doc", lang: "en")
 # Render `mermaid` code blocks as <pre class="mermaid"> diagrams; in
 # standalone mode the mermaid.js runtime is loaded from a CDN too.
 doc.to_html(standalone: true, mermaid: true)
+
+# Parse a leading YAML frontmatter block (--- ... ---). Off by default; when
+# enabled the block is removed from the rendered body and exposed as a Hash.
+doc = RedQuilt.parse("---\ntitle: Hi\nlang: ja\n---\n\n# Body", frontmatter: true)
+doc.frontmatter       # => {"title" => "Hi", "lang" => "ja"} (nil when absent/disabled)
+# In standalone output frontmatter title/lang fill <title>/<html lang> unless
+# an explicit argument overrides them. Invalid YAML adds a :frontmatter
+# warning diagnostic and leaves doc.frontmatter as nil.
+doc.to_html(standalone: true)
 ```
 
 ## NodeRef (AST node wrapper)
@@ -41,12 +51,34 @@ node.walk             # Enumerator[NodeRef] or { |node| ... } block
 node.find_all(:link)  # Array[NodeRef] with matching type
 node.text             # String (concatenated child text)
 
+# Node attributes, by type. Each returns nil when the node's type does not
+# carry the attribute, so you can branch on #type and read that type's own
+# fields without building a Hash per node.
+node.info             # CODE_BLOCK: fence info, e.g. "ruby". "" when the
+                      # block was written without one.
+node.heading_level    # HEADING: 1..6
+node.list_ordered?    # LIST: true for "1.", false for "-"
+node.list_start       # LIST: start number of an ordered list
+node.list_tight?      # LIST: tight vs loose
+node.list_delimiter   # LIST: delimiter as authored, e.g. "-" or "."
+node.link_destination # LINK / IMAGE: destination URL
+node.link_title       # LINK / IMAGE: title, nil when absent
+node.footnote_label   # FOOTNOTE_DEFINITION / FOOTNOTE_REFERENCE: label
+node.footnote_number  # FOOTNOTE_DEFINITION / FOOTNOTE_REFERENCE: 1-based
+node.header?          # TABLE_ROW / TABLE_CELL: part of the header row?
+
 # Position information (byte offset)
 node.source_span      # SourceSpan with start_byte, end_byte
 
 # Position information (line/column)
 node.source_location  # { start_line, start_column, end_line, end_column }
-                      # line: 1-indexed, column: 0-indexed (character-based)
+                      # line and column are both 1-indexed and counted in
+                      # characters, following the unist Point convention that
+                      # cmark sourcepos and mdast use. `end` is exclusive: it
+                      # is the position just past the node's last character.
+                      # Block spans cover the block as authored, including
+                      # markers ("# H1" starts at the "#", "```" fences are
+                      # part of the code block), but excluding leading indent.
 
 # AST export
 node.to_h             # Export subtree as Hash[Symbol, untyped]

data/lib/red_quilt/arena.rb

@@ -231,6 +231,116 @@ module RedQuilt
       @str2[id]
     end
 
+    # --- Semantic payload accessors -------------------------------------
+    #
+    # int1..int3 / str1 / str2 are anonymous columns; their meaning is
+    # per-NodeType (see the class comment). These readers are the single
+    # source of truth for those conventions, so callers (renderers, the
+    # NodeRef wrapper, AST/MDAST export) never need to know which raw
+    # column a field lives in. Writers use add_node's keyword args,
+    # update_*, or a small set of typed writers (e.g.
+    # #resolve_footnote_definition) when the intent is worth naming.
+    #
+    # The reader is responsible for calling these only on the matching
+    # NodeType; on a mismatching node they return whatever the raw column
+    # holds (typically the 0 / nil default).
+
+    # HEADING: nesting level (1..6).
+    def heading_level(id)
+      @int1[id]
+    end
+
+    # HEADING: byte range of the heading text alone, excluding the `#` marker
+    # and any closing hashes. The node's source_span covers the heading as
+    # authored (for position reporting), so inline parsing needs this narrower
+    # range to avoid lexing the marker as content. Setext headings carry no
+    # marker prefix and leave this zeroed, in which case the span is the
+    # content range.
+    def heading_content_start(id)
+      @int2[id]
+    end
+
+    def heading_content_len(id)
+      @int3[id]
+    end
+
+    # LIST: ordered (`1.`) vs bullet (`-`).
+    def list_ordered?(id)
+      @int1[id] == 1
+    end
+
+    # LIST: the start number of an ordered list (1 unless overridden).
+    def list_start(id)
+      @int2[id]
+    end
+
+    # LIST: tight (no blank lines between items) vs loose.
+    def list_tight?(id)
+      @int3[id] == 1
+    end
+
+    # LIST: the item delimiter as authored (e.g. "-", "1.").
+    def list_delimiter(id)
+      @str1[id]
+    end
+
+    # TABLE_ROW: header row (rendered in <thead>) vs body row.
+    def table_row_header?(id)
+      @int1[id] == 1
+    end
+
+    # TABLE_CELL: header cell (<th>) vs data cell (<td>).
+    def table_cell_header?(id)
+      @int1[id] == 1
+    end
+
+    # CODE_BLOCK: the fence info string (e.g. 'ruby', 'vtt audio="x"').
+    def code_block_info(id)
+      @str2[id]
+    end
+
+    # LINK / IMAGE: destination URL.
+    def link_destination(id)
+      @str1[id]
+    end
+
+    # LINK / IMAGE: optional title attribute (nil/empty when absent).
+    def link_title(id)
+      @str2[id]
+    end
+
+    # FOOTNOTE_REFERENCE: the assigned footnote number.
+    def footnote_number(id)
+      @int1[id]
+    end
+
+    # FOOTNOTE_REFERENCE: which occurrence of a repeated reference this is
+    # (1-based), used to give each backref a unique anchor.
+    def footnote_occurrence(id)
+      @int2[id]
+    end
+
+    # FOOTNOTE_REFERENCE / FOOTNOTE_DEFINITION: the author-written label.
+    def footnote_label(id)
+      @str1[id]
+    end
+
+    # FOOTNOTE_DEFINITION: total number of references to this footnote,
+    # materialized by FootnotePass so renderers can read it off the node
+    # instead of consulting the registry. (FOOTNOTE_REFERENCE reuses int2
+    # for its own occurrence index; see #footnote_occurrence.)
+    def footnote_total_references(id)
+      @int2[id]
+    end
+
+    # FOOTNOTE_DEFINITION: records the resolved number and total reference
+    # count onto the node (read back via #footnote_number /
+    # #footnote_total_references).
+    def resolve_footnote_definition(id, number, total_references)
+      @int1[id] = number
+      @int2[id] = total_references
+    end
+
     # Returns a SourceSpan for the node, or nil when the node has no
     # span (source_start < 0, meaning the content is held in str1).
     def source_span(id)
@@ -251,6 +361,13 @@ module RedQuilt
       start_byte = @source_start[id]
       return nil if start_byte.nil? || start_byte.negative?
 
+      # An ATX heading's span deliberately covers its `#` marker so that
+      # positions point at the heading as authored. Text is the content
+      # alone, so slice the narrower content range instead.
+      if @type[id] == NodeType::HEADING && @int2[id].positive?
+        return @source.byteslice(@int2[id], @int3[id])
+      end
+
       @source.byteslice(start_byte, @source_len[id])
     end