red_quilt 0.8.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 56e09a19cfb78fad2a7e9a377f3ec6b9033d2160d4c126719e8220245b340709
-  data.tar.gz: 35bcb8de686345a72b9d2edf0261c7b201fb138047b571e3cc14b4f62be2671a
+  metadata.gz: 211bfba5adede6552fb64c3dfadf6c895325af2c58a577eadb86c8d426a3dd0b
+  data.tar.gz: 2e542ae344e626dc5a1bf0198b80761dfac3317b48f11011569235e5418048c9
 SHA512:
-  metadata.gz: a5ad64e49f61ddcca8c3453b28ab06e7c04d76da574d82e6185b9a175189f313e043dcb586ff4293ad801b6bbb521ecca902f4dd9cac630186db97a2d8de5b91
-  data.tar.gz: fc3764a74f06f1d902afd5eeba1e572ddb97f3fa1c9306f4de937e1929937d22830f856c77b03d2e1fc658be3310d7f7535cb994a7945cf7f5f18ea52e0416c4
+  metadata.gz: 8526155c616d353741dd68ad7740a0ad3d18e08139c239a6d2ff2d48198721e4b1a84a8da3eca98dae033d1da73a59817a766fe0b3bf89391897559a0fc37047
+  data.tar.gz: 272a4918326acc93dc8699308a8597c9e621f98518f0307ef7a8f2eed9ba03dc3153f2fbea641e6b02eb76af0adf612a7d8e8fca7b08360c130d1e34b7b5266c

data/CHANGELOG.md

@@ -7,6 +7,82 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.9.0] - 2026-07-20
+
+### Changed
+
+- **Breaking: source positions now follow the unist Point convention**, which
+  cmark sourcepos and mdast both use. Three related changes:
+  - `source_location` columns are 1-based (they were 0-based; lines were
+    already 1-based). Both are counted in characters, and `end` remains
+    exclusive.
+  - Block spans cover the block as authored rather than its content alone.
+    `# H1` now starts at the `#`, `> quote` at the `>`, a list item at its
+    bullet, a fenced code block includes both fences, and a setext heading
+    includes its `===` / `---` underline.
+  - Leading indent is excluded from block spans, so `  text` starts at
+    column 3 rather than column 1. This applies to ATX and setext headings,
+    paragraphs, blockquotes, fenced code blocks, lists, list items, and
+    thematic breaks.
+  - The fenced code block change also fixes a wrong *line* number: the span
+    used to start at the first content line, reporting the block one line
+    below its opening fence.
+
+  Every expected value was verified against commonmark.js and
+  mdast-util-from-markdown; RedQuilt now agrees with mdast on all of them
+  except an unclosed fence, where the two references disagree and RedQuilt
+  follows cmark. Callers that added 1 to columns, or that relied on a
+  heading span excluding `# `, need updating.
+
+  `NodeRef#text` is unaffected: a heading's text remains its content, without
+  the marker.
+
+- **Breaking: `NodeRef#info` returns nil for nodes that are not code
+  blocks**, where 0.8.0 returned `""`. A code block written without an info
+  string still returns `""`, so the two cases are now distinguishable. This
+  matches the new attribute accessors, which all use nil for "this node's
+  type does not carry the attribute".
+- Lowered the minimum supported Ruby from 3.3 to 3.1. The only 3.1
+  incompatibility was `String#byteindex` (added in Ruby 3.2), used in two
+  inline hot paths; both operate on a binary (`String#b`) view of the source,
+  where `String#index` returns the same byte offsets. CI now runs the test
+  suite on 3.1 through 4.0.
+- `Gemfile.lock` is no longer committed. A lockfile resolved on one Ruby pins
+  development gem versions that do not exist for the others (e.g. `rdoc 8.0.0`
+  and `rbs 4.0.3` have no Ruby 3.1 release), so a single committed lockfile
+  cannot serve the supported range. Each Ruby now resolves its own.
+
+### Fixed
+
+- `source_location` reported wrong line numbers for any source containing a
+  multibyte character. The line-start table is indexed by byte offset, but it
+  was built with `String#index`, which counts characters, so every line after
+  the first multibyte character drifted. Existing multibyte coverage did not
+  catch this because it used single-line sources.
+- `Renderer::MDAST` emitted positions that violated the unist spec: `column`
+  was 0-based where unist requires 1-based, and `offset` was a byte index
+  where unist requires a 0-based character index. The latter made every
+  offset after a multibyte character wrong.
+
+### Added
+
+- Node attribute accessors on `NodeRef`: `heading_level`, `list_ordered?`,
+  `list_start`, `list_tight?`, `list_delimiter`, `link_destination`,
+  `link_title`, `footnote_label`, `footnote_number`, and `header?`. Each
+  returns nil when the node's type does not carry the attribute, so callers
+  can walk a document branching on `#type` and read that type's fields
+  directly. Previously the only public route was `#to_h`, which builds a Hash
+  for the whole subtree — 5391 objects for a 257-node list where these
+  accessors allocate none.
+
+  These wrappers are the safe way to read attributes. The `Arena` accessors
+  they delegate to are the raw layer and skip the type check, and several
+  attributes share a storage column, so reading one off a mismatched node
+  there returns another field's value rather than nil (`Arena#link_destination`
+  on a paragraph returns the paragraph's text).
+
+## [0.8.0] - 2026-06-24
+
 ### Added
 
 - `NodeRef#info`: returns the fence info string of a code block (e.g. `ruby`

data/docs/api.md

@@ -50,14 +50,35 @@ node.children         # Array[NodeRef]
 node.walk             # Enumerator[NodeRef] or { |node| ... } block
 node.find_all(:link)  # Array[NodeRef] with matching type
 node.text             # String (concatenated child text)
-node.info             # String fence info of a code block (e.g. "ruby")
+
+# Node attributes, by type. Each returns nil when the node's type does not
+# carry the attribute, so you can branch on #type and read that type's own
+# fields without building a Hash per node.
+node.info             # CODE_BLOCK: fence info, e.g. "ruby". "" when the
+                      # block was written without one.
+node.heading_level    # HEADING: 1..6
+node.list_ordered?    # LIST: true for "1.", false for "-"
+node.list_start       # LIST: start number of an ordered list
+node.list_tight?      # LIST: tight vs loose
+node.list_delimiter   # LIST: delimiter as authored, e.g. "-" or "."
+node.link_destination # LINK / IMAGE: destination URL
+node.link_title       # LINK / IMAGE: title, nil when absent
+node.footnote_label   # FOOTNOTE_DEFINITION / FOOTNOTE_REFERENCE: label
+node.footnote_number  # FOOTNOTE_DEFINITION / FOOTNOTE_REFERENCE: 1-based
+node.header?          # TABLE_ROW / TABLE_CELL: part of the header row?
 
 # Position information (byte offset)
 node.source_span      # SourceSpan with start_byte, end_byte
 
 # Position information (line/column)
 node.source_location  # { start_line, start_column, end_line, end_column }
-                      # line: 1-indexed, column: 0-indexed (character-based)
+                      # line and column are both 1-indexed and counted in
+                      # characters, following the unist Point convention that
+                      # cmark sourcepos and mdast use. `end` is exclusive: it
+                      # is the position just past the node's last character.
+                      # Block spans cover the block as authored, including
+                      # markers ("# H1" starts at the "#", "```" fences are
+                      # part of the code block), but excluding leading indent.
 
 # AST export
 node.to_h             # Export subtree as Hash[Symbol, untyped]

data/lib/red_quilt/arena.rb

@@ -250,6 +250,20 @@ module RedQuilt
       @int1[id]
     end
 
+    # HEADING: byte range of the heading text alone, excluding the `#` marker
+    # and any closing hashes. The node's source_span covers the heading as
+    # authored (for position reporting), so inline parsing needs this narrower
+    # range to avoid lexing the marker as content. Setext headings carry no
+    # marker prefix and leave this zeroed, in which case the span is the
+    # content range.
+    def heading_content_start(id)
+      @int2[id]
+    end
+
+    def heading_content_len(id)
+      @int3[id]
+    end
+
     # LIST: ordered (`1.`) vs bullet (`-`).
     def list_ordered?(id)
       @int1[id] == 1
@@ -347,6 +361,13 @@ module RedQuilt
       start_byte = @source_start[id]
       return nil if start_byte.nil? || start_byte.negative?
 
+      # An ATX heading's span deliberately covers its `#` marker so that
+      # positions point at the heading as authored. Text is the content
+      # alone, so slice the narrower content range instead.
+      if @type[id] == NodeType::HEADING && @int2[id].positive?
+        return @source.byteslice(@int2[id], @int3[id])
+      end
+
       @source.byteslice(start_byte, @source_len[id])
     end
 

data/lib/red_quilt/block_parser.rb

@@ -69,7 +69,11 @@ module RedQuilt
           append_heading(parent_id, line, heading, transformed)
           index += 1
         elsif thematic_break?(content)
-          @arena.append_child(parent_id, @arena.add_node(NodeType::THEMATIC_BREAK, source_start: line.start_byte, source_len: line.span_len))
+          break_start = line.start_byte + Indentation.leading_ws_bytes(content)
+          @arena.append_child(parent_id,
+                              @arena.add_node(NodeType::THEMATIC_BREAK,
+                                              source_start: break_start,
+                                              source_len: line.end_byte - break_start))
           index += 1
         elsif @footnotes && (footnote = FootnoteDefinition.match(content))
           index = @footnote_parser.parse(lines, index, footnote, @footnotes, @root_id)
@@ -188,11 +192,17 @@ module RedQuilt
 
     def append_heading(parent_id, line, heading, transformed)
       content = heading[:content].to_s.rstrip
-      source_start = line.start_byte + heading[:content_start]
+      # The span covers the heading as written — `#` marker, content, and any
+      # closing hashes — matching cmark sourcepos and mdast. Leading indent is
+      # excluded: `   # H1` starts at the `#`, not at the line start.
+      source_start = line.start_byte + heading[:marker_start].to_i
+      content_start = line.start_byte + heading[:content_start]
       node_id = @arena.add_node(NodeType::HEADING,
                                 source_start: source_start,
-                                source_len: content.bytesize,
+                                source_len: line.end_byte - source_start,
                                 int1: heading[:level],
+                                int2: content_start,
+                                int3: content.bytesize,
                                 str1: transformed ? content : nil)
       @arena.append_child(parent_id, node_id)
     end
@@ -201,6 +211,7 @@ module RedQuilt
       paragraph_lines = []
       start_index = index
       setext_level = nil
+      setext_underline = nil
       while index < lines.length
         line = lines[index]
         break if line.blank
@@ -212,6 +223,7 @@ module RedQuilt
         # thematic break.
         if paragraph_lines.any? && !line.lazy_continuation && (level = setext_underline_level(line.content))
           setext_level = level
+          setext_underline = line
           index += 1
           break
         end
@@ -246,13 +258,21 @@ module RedQuilt
       stripped[-1] = stripped[-1].sub(/[ \t]+\z/, "") if stripped.any?
       indent_was_stripped = stripped.zip(paragraph_lines).any? { |s, l| s.length != l.content.length }
       text = stripped.join("\n")
-      start_byte = paragraph_lines.first.start_byte
+      # Spans start at the first non-whitespace byte: leading indent is not
+      # part of the block as authored. Both cmark and mdast report `  text`
+      # as starting at column 3.
+      first_line = paragraph_lines.first
+      start_byte = first_line.start_byte + Indentation.leading_ws_bytes(first_line.content)
       end_byte = paragraph_lines.last.end_byte
 
       if setext_level
+        # The span runs through the `===` / `---` underline, which is part of
+        # the heading as authored — cmark and mdast both report it that way.
+        # The text is carried by str1, so widening the span does not leak the
+        # underline into the heading's content.
         heading_id = @arena.add_node(NodeType::HEADING,
                                      source_start: start_byte,
-                                     source_len: end_byte - start_byte,
+                                     source_len: setext_underline.end_byte - start_byte,
                                      int1: setext_level,
                                      str1: text.strip)
         @arena.append_child(parent_id, heading_id)
@@ -340,7 +360,10 @@ module RedQuilt
 
       content = match[2].to_s
       content_index = content.empty? ? text.length : (text.index(content) || text.bytesize)
-      { level: match[1].length, content: content, content_start: content_index }
+      # Offset of the `#` run itself. Only spaces may precede it (at most 3),
+      # so the character index doubles as a byte offset.
+      { level: match[1].length, content: content, content_start: content_index,
+        marker_start: match.begin(1) }
     end
 
     def store_reference(reference, source_span)

data/lib/red_quilt/blockquote.rb

@@ -17,6 +17,14 @@ module RedQuilt
       text.match?(BLOCKQUOTE_PREFIX_RE)
     end
 
+    # Byte offset of the `>` within the line, or nil when there is no prefix.
+    # Only spaces may precede it (at most 3), so the character index returned
+    # by the match doubles as a byte offset.
+    def marker_offset(text)
+      match = BLOCKQUOTE_PREFIX_RE.match(text)
+      match && (match.end(0) - 1)
+    end
+
     # Strip the leading `>` (and at most one column of whitespace after
     # it) from a blockquote line. Returns a new Line whose
     # content is the inner text. If the line has no `>` prefix, the
@@ -75,6 +83,11 @@ module RedQuilt
       end
 
       def parse(parent_id, lines, index)
+        # Captured before the loop: block_lines holds prefix-stripped lines,
+        # whose start_byte sits after the `>`. The span must start at the
+        # marker itself (excluding indent), as cmark and mdast both report.
+        first_line = lines[index]
+        source_start = first_line.start_byte + (Blockquote.marker_offset(first_line.content) || 0)
         block_lines = []
         paragraph_open = false
 
@@ -113,8 +126,8 @@ module RedQuilt
         end
 
         block_id = @arena.add_node(NodeType::BLOCKQUOTE,
-                                   source_start: block_lines.first.start_byte,
-                                   source_len: block_lines.last.end_byte - block_lines.first.start_byte)
+                                   source_start: source_start,
+                                   source_len: block_lines.last.end_byte - source_start)
         @arena.append_child(parent_id, block_id)
         @block_parser.parse_lines(block_id, block_lines, transformed: true)
         index

data/lib/red_quilt/code_block.rb

@@ -63,8 +63,14 @@ module RedQuilt
         indent_n = fence[:indent] || 0
         code = content_lines.map { |l| Indentation.strip_leading_spaces(l.content, indent_n) }.join("\n")
         code << "\n" unless content_lines.empty?
-        source_start = content_lines.empty? ? start_line.start_byte : content_lines.first.start_byte
-        source_end = content_lines.empty? ? start_line.end_byte : content_lines.last.end_byte
+        # The span covers the fences too, as cmark and mdast both report — a
+        # span of the content alone points a line below the block. Leading
+        # indent is excluded, so `  ```rb` starts at the backticks. `index`
+        # now sits past the closing fence, or past the last line when the
+        # fence was never closed, so lines[index - 1] is the block's last line
+        # either way.
+        source_start = start_line.start_byte + indent_n
+        source_end = lines[index - 1].end_byte
         code_id = @arena.add_node(NodeType::CODE_BLOCK,
                                   source_start: source_start,
                                   source_len: source_end - source_start,

data/lib/red_quilt/inline/builder.rb

@@ -27,10 +27,10 @@ module RedQuilt
       def initialize(arena, source, references, track_source: true, diagnostics: nil, footnotes: nil)
         @arena = arena
         @source = source
-        # Binary view of the source for String#byteindex hot paths:
-        # byteindex on a UTF-8 string raises when the byte offset falls
-        # inside a multibyte sequence; the binary view treats every byte
-        # as its own character.
+        # Binary view of the source for String#index hot paths: on a
+        # UTF-8 string, index counts characters, so the offsets would not
+        # line up with our byte positions. The binary view treats every
+        # byte as its own character, making index a byte-offset search.
         @source_b = source.b
         @references = references
         @track_source = track_source
@@ -229,14 +229,14 @@ module RedQuilt
       # source bytes directly. CommonMark: backslash escapes do not
       # apply inside a code span, so once we're past the opener every
       # backtick run is a real candidate (token-level ESCAPED_CHAR is
-      # ignored). byteindex jumps over non-backtick byte stretches at
-      # C speed.
+      # ignored). index on the binary view jumps over non-backtick byte
+      # stretches at C speed.
       def resolve_code_span(opener_id)
         run_len = @tokens.int1(opener_id)
         pos = @tokens.end_byte(opener_id)
         bytesize = @source_b.bytesize
         while pos < bytesize
-          run_start = @source_b.byteindex(BACKTICK_BYTE, pos)
+          run_start = @source_b.index(BACKTICK_BYTE, pos)
           break if run_start.nil?
 
           pos = run_start + 1

data/lib/red_quilt/inline/lexer.rb

@@ -22,8 +22,8 @@ module RedQuilt
         [0x2A, 0x5F, 0x60, 0x5B, 0x5D, 0x21, 0x3C, 0x26, 0x5C, 0x0A, 0x7E].each { |b| a[b] = true }
         a.freeze
       end
-      # Same set as SPECIAL_BYTES, for String#byteindex to jump over long
-      # plain-text runs at C speed.
+      # Same set as SPECIAL_BYTES, for String#index on the binary view to
+      # jump over long plain-text runs at C speed.
       SPECIAL_BYTE_RE = /[*_`\[\]!<&\\\n~]/
 
       # Anchored regexes for StringScanner#scan (still used by
@@ -116,12 +116,12 @@ module RedQuilt
             pos = @ss.pos
           else
             # Inlined scan_text. Always make progress: consume the
-            # current byte, then byteindex against the binary view to
+            # current byte, then index against the binary view to
             # leap to the next special byte at C speed.
             start = pos
             pos += 1
             if pos < end_pos
-              next_special = @source_b.byteindex(SPECIAL_BYTE_RE, pos)
+              next_special = @source_b.index(SPECIAL_BYTE_RE, pos)
               pos = next_special.nil? || next_special >= end_pos ? end_pos : next_special
             end
             tokens.emit(TokenKind::TEXT, start_byte: start, end_byte: pos)

data/lib/red_quilt/inline_pass.rb

@@ -35,8 +35,7 @@ module RedQuilt
                               diagnostics: @document.diagnostics,
                               footnotes: @document.footnotes).build(node_id, @tokens)
         else
-          start_byte = @arena.source_start(node_id)
-          end_byte = @arena.source_end(node_id)
+          start_byte, end_byte = inline_range(node_id)
           @lexer.lex_into(@tokens, start_byte, end_byte)
           @builder.build(node_id, @tokens)
         end
@@ -49,5 +48,19 @@ module RedQuilt
         child_id = @arena.raw_next_sibling_id(child_id)
       end
     end
+
+    # The byte range to lex as inline content. For most nodes that is the
+    # source span, but an ATX heading's span covers the `#` marker and any
+    # closing hashes so that positions point at the heading as authored;
+    # lexing that range would turn the marker into text. Setext headings
+    # leave the content range zeroed and fall back to the span.
+    def inline_range(node_id)
+      if @arena.type(node_id) == NodeType::HEADING && @arena.heading_content_start(node_id).positive?
+        start_byte = @arena.heading_content_start(node_id)
+        return [start_byte, start_byte + @arena.heading_content_len(node_id)]
+      end
+
+      [@arena.source_start(node_id), @arena.source_end(node_id)]
+    end
   end
 end

data/lib/red_quilt/list.rb

@@ -146,15 +146,19 @@ module RedQuilt
 
       def parse(parent_id, lines, index)
         first_match = List.match(lines[index].content)
+        # Leading indent is excluded, so `  - item` starts at the bullet.
+        list_start_byte = lines[index].start_byte + Indentation.leading_ws_bytes(lines[index].content)
         list_id = @arena.add_node(NodeType::LIST,
-                                  source_start: lines[index].start_byte,
+                                  source_start: list_start_byte,
                                   source_len: 0,
                                   int1: first_match[:ordered] ? 1 : 0,
                                   int2: first_match[:start_number],
                                   int3: 1,
                                   str1: first_match[:marker])
         @arena.append_child(parent_id, list_id)
-        start_byte = lines[index].start_byte
+        # update_span below rewrites the span once the list's extent is
+        # known, so it has to carry the indent-stripped start too.
+        start_byte = list_start_byte
         end_byte = lines[index].end_byte
         loose = false
 
@@ -167,11 +171,16 @@ module RedQuilt
           break unless match
           break unless List.same_group?(first_match, match)
 
+          # collect_item strips the bullet, so item_lines start at the item's
+          # content. The span starts at the marker instead — an item is
+          # authored as "- text", not "text".
+          marker_line = lines[index]
+          item_start = marker_line.start_byte + Indentation.leading_ws_bytes(marker_line.content)
           item_lines, index = collect_item(lines, index, match)
           end_byte = item_lines.last.end_byte
           item_id = @arena.add_node(NodeType::LIST_ITEM,
-                                    source_start: item_lines.first.start_byte,
-                                    source_len: item_lines.last.end_byte - item_lines.first.start_byte)
+                                    source_start: item_start,
+                                    source_len: item_lines.last.end_byte - item_start)
           @arena.append_child(list_id, item_id)
           # CommonMark: an item is loose when "two block-level elements
           # with a blank line between them" appear at its top level.

data/lib/red_quilt/node_ref.rb

@@ -46,13 +46,83 @@ module RedQuilt
       text
     end
 
-    # Returns the fence info string of a CODE_BLOCK node.
+    # Node attributes, by NodeType.
+    #
+    # Each returns nil when the node does not carry the attribute, so callers
+    # can walk the tree branching on #type and read the type's own fields
+    # without a Hash allocation per node. The Arena's accessors are the raw
+    # layer and deliberately skip that check: several attributes share a
+    # storage column, so reading one off a mismatched node yields another
+    # field's value rather than nil (e.g. Arena#link_destination on a
+    # paragraph returns the paragraph's text). These wrappers are the safe
+    # way to read attributes.
+
+    # CODE_BLOCK: the fence info string, e.g. "ruby" or 'vtt audio="x.mp3"'.
+    # A code block written without one has an empty info string, so "" means
+    # "no info given" while nil means "not a code block".
     def info
-      return "" unless @arena.type(@node_id) == NodeType::CODE_BLOCK
+      return nil unless type?(NodeType::CODE_BLOCK)
 
       @arena.code_block_info(@node_id).to_s
     end
 
+    # HEADING: nesting level (1..6).
+    def heading_level
+      @arena.heading_level(@node_id) if type?(NodeType::HEADING)
+    end
+
+    # LIST: ordered (`1.`) vs bullet (`-`).
+    def list_ordered?
+      @arena.list_ordered?(@node_id) if type?(NodeType::LIST)
+    end
+
+    # LIST: start number of an ordered list.
+    def list_start
+      @arena.list_start(@node_id) if type?(NodeType::LIST)
+    end
+
+    # LIST: tight (no blank lines between items) vs loose.
+    def list_tight?
+      @arena.list_tight?(@node_id) if type?(NodeType::LIST)
+    end
+
+    # LIST: the item delimiter as authored, e.g. "-" or "1.".
+    def list_delimiter
+      @arena.list_delimiter(@node_id) if type?(NodeType::LIST)
+    end
+
+    # LINK / IMAGE: the destination URL.
+    def link_destination
+      @arena.link_destination(@node_id) if link_like?
+    end
+
+    # LINK / IMAGE: the optional title, nil when absent.
+    def link_title
+      @arena.link_title(@node_id) if link_like?
+    end
+
+    # FOOTNOTE_DEFINITION / FOOTNOTE_REFERENCE: the label as authored.
+    def footnote_label
+      @arena.footnote_label(@node_id) if footnote_like?
+    end
+
+    # FOOTNOTE_DEFINITION / FOOTNOTE_REFERENCE: the resolved 1-based number.
+    def footnote_number
+      @arena.footnote_number(@node_id) if footnote_like?
+    end
+
+    # TABLE_ROW / TABLE_CELL: whether this belongs to the header row.
+    #
+    # nil rather than false outside a table, matching the other attribute
+    # accessors: a paragraph is not a non-header row, it has no header-ness
+    # at all. Both answers are falsy, so `if node.header?` reads the same.
+    def header?
+      return @arena.table_row_header?(@node_id) if type?(NodeType::TABLE_ROW)
+      return @arena.table_cell_header?(@node_id) if type?(NodeType::TABLE_CELL)
+
+      nil # rubocop:disable Style/ReturnNilInPredicateMethodDefinition
+    end
+
     def source_span
       @arena.source_span(@node_id)
     end
@@ -90,6 +160,22 @@ module RedQuilt
 
     private
 
+    def type?(node_type)
+      @arena.type(@node_id) == node_type
+    end
+
+    # Kept allocation-free (no splat) because these accessors are meant for
+    # per-node use while walking a whole document.
+    def link_like?
+      node_type = @arena.type(@node_id)
+      node_type == NodeType::LINK || node_type == NodeType::IMAGE
+    end
+
+    def footnote_like?
+      node_type = @arena.type(@node_id)
+      node_type == NodeType::FOOTNOTE_DEFINITION || node_type == NodeType::FOOTNOTE_REFERENCE
+    end
+
     def ast_attributes
       case @arena.type(@node_id)
       when NodeType::HEADING

data/lib/red_quilt/renderer/mdast.rb

@@ -127,19 +127,23 @@ module RedQuilt
         result
       end
 
+      # unist Point: line and column are 1-based, offset is a 0-based
+      # *character* index. SourceSpan carries bytes, so offset is converted
+      # rather than passed through — the two diverge on multibyte sources.
       def position(span)
-        start_loc = @document.source_map.line_column(span.start_byte)
-        end_loc = @document.source_map.line_column(span.end_byte)
+        source_map = @document.source_map
+        start_loc = source_map.line_column(span.start_byte)
+        end_loc = source_map.line_column(span.end_byte)
         {
           "start" => {
             "line" => start_loc[:line],
             "column" => start_loc[:column],
-            "offset" => span.start_byte,
+            "offset" => source_map.char_offset(span.start_byte),
           },
           "end" => {
             "line" => end_loc[:line],
             "column" => end_loc[:column],
-            "offset" => span.end_byte,
+            "offset" => source_map.char_offset(span.end_byte),
           },
         }
       end

data/lib/red_quilt/source_map.rb

@@ -1,29 +1,69 @@
 # frozen_string_literal: true
 
 module RedQuilt
+  # Translates byte offsets from SourceSpan into human-facing positions.
+  #
+  # Positions follow the unist Point convention, which cmark sourcepos and
+  # mdast both use: `line` and `column` are 1-based, while character offsets
+  # are 0-based. Columns and offsets are counted in characters, not bytes, so
+  # a multibyte source reports the position a reader would count.
   class SourceMap
     def initialize(source)
       @source = source
       @line_starts = build_line_starts(source)
+      @line_char_starts = build_line_char_starts(source, @line_starts)
     end
 
+    # Returns { line:, column: }, both 1-based.
     def line_column(byte_offset)
-      line = (@line_starts.bsearch_index { |s| s > byte_offset } || @line_starts.length) - 1
-      line_start = @line_starts[line]
-      column = @source.byteslice(line_start, byte_offset - line_start).to_s.length
-      { line: line + 1, column: column }
+      line = line_index(byte_offset)
+      { line: line + 1, column: chars_from_line_start(line, byte_offset) + 1 }
+    end
+
+    # Returns the 0-based character offset, as unist Point#offset requires.
+    # Byte offsets are what SourceSpan carries, and the two differ as soon as
+    # the source contains a multibyte character.
+    def char_offset(byte_offset)
+      line = line_index(byte_offset)
+      @line_char_starts[line] + chars_from_line_start(line, byte_offset)
     end
 
     private
 
+    def line_index(byte_offset)
+      (@line_starts.bsearch_index { |s| s > byte_offset } || @line_starts.length) - 1
+    end
+
+    def chars_from_line_start(line, byte_offset)
+      line_start = @line_starts[line]
+      @source.byteslice(line_start, byte_offset - line_start).to_s.length
+    end
+
+    # Line starts are byte offsets, matching SourceSpan#start_byte / #end_byte
+    # and the argument of #line_column. Scanning the binary view is what makes
+    # them bytes: index on a UTF-8 string counts characters, so a multibyte
+    # source would yield offsets that line_column then misreads as bytes.
     def build_line_starts(source)
       starts = [0]
+      source_b = source.b
       pos = 0
-      while (idx = source.index("\n", pos))
+      while (idx = source_b.index("\n", pos))
         starts << (idx + 1)
         pos = idx + 1
       end
       starts
     end
+
+    # Character offset of each line start, so char_offset stays O(1) per call
+    # instead of recounting the source prefix every time.
+    def build_line_char_starts(source, line_starts)
+      chars = 0
+      prev_byte = 0
+      line_starts.map do |byte_start|
+        chars += source.byteslice(prev_byte, byte_start - prev_byte).to_s.length
+        prev_byte = byte_start
+        chars
+      end
+    end
   end
 end

data/lib/red_quilt/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RedQuilt
-  VERSION = "0.8.0"
+  VERSION = "0.9.0"
 end

data/sig/red_quilt.rbs

@@ -61,11 +61,23 @@ module RedQuilt
     def walk: () { (NodeRef) -> void } -> void
             | () -> Enumerator[NodeRef, void]
     def text: () -> String?
-    def info: () -> String
     def source_span: () -> SourceSpan?
     def find_all: (Symbol type) -> Array[NodeRef]
     def source_location: () -> { start_line: Integer, start_column: Integer, end_line: Integer, end_column: Integer }?
     def to_h: () -> Hash[Symbol, untyped]
+
+    # Node attributes: nil when the node's type does not carry the attribute.
+    def info: () -> String?
+    def heading_level: () -> Integer?
+    def list_ordered?: () -> bool?
+    def list_start: () -> Integer?
+    def list_tight?: () -> bool?
+    def list_delimiter: () -> String?
+    def link_destination: () -> String?
+    def link_title: () -> String?
+    def footnote_label: () -> String?
+    def footnote_number: () -> Integer?
+    def header?: () -> bool?
   end
 
   # Low-level parallel-array storage for the AST. See docs/arena-usage.ja.md.
@@ -118,6 +130,8 @@ module RedQuilt
 
     # Semantic payload accessors (per-NodeType column conventions)
     def heading_level: (Integer id) -> Integer
+    def heading_content_start: (Integer id) -> Integer
+    def heading_content_len: (Integer id) -> Integer
     def list_ordered?: (Integer id) -> bool
     def list_start: (Integer id) -> Integer
     def list_tight?: (Integer id) -> bool
@@ -150,9 +164,12 @@ module RedQuilt
     def to_h: () -> { start_byte: Integer, end_byte: Integer }
   end
 
+  # Positions follow the unist Point convention: line and column are 1-based
+  # and counted in characters, while char_offset is 0-based.
   class SourceMap
     def initialize: (String source) -> void
     def line_column: (Integer byte_offset) -> { line: Integer, column: Integer }
+    def char_offset: (Integer byte_offset) -> Integer
   end
 
   class Diagnostic

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: red_quilt
 version: !ruby/object:Gem::Version
-  version: 0.8.0
+  version: 0.9.0
 platform: ruby
 authors:
 - takahashim
@@ -94,14 +94,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 3.3.0
+      version: 3.1.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.10
+rubygems_version: 4.0.16
 specification_version: 4
 summary: CommonMark-based Markdown processor written in pure Ruby
 test_files: []