beid 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 203b09a7d1880b4113345b87d2571e0c84aefb40371e6a4b9f97fdf32b417c7e
+  data.tar.gz: 72145389c81cb8e86045bdef7bc3373f5a371ea57717676b3990a580cb9a7e7a
+SHA512:
+  metadata.gz: 68d48286462af7a869d4007540357c2a42957df7c2dee6a98e9d827309de9d3c426ba26d29de9262210b868f0a08ac52b39cfb3a98eb01568e0494a96c2d5f8d
+  data.tar.gz: d45e9fd574d845ddc0d69567f044addd222882d9bc2644ca18849c8ee93ff45ec14d7f0455acfbe49c8994eea1816edd227f0c516b4bfe7b378b2da2668284e1

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## 0.1.0 - 2026-09-23
+
+- Initial release.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Yudai Takada
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,52 @@
+# Beid
+
+Beid (ο¹ Eridani; Arabic *bayḍ*, “egg”) parses Markdown into a source-positioned tree and edits the original bytes in place. It is intended for applications that need to update part of a Markdown document without normalizing untouched formatting.
+
+Beid uses Ruby's standard library at runtime and supports Ruby 3.1 and later.
+
+## Installation
+
+```ruby
+gem "beid"
+```
+
+## Usage
+
+```ruby
+require "beid"
+
+source = "# Title\n\nKeep this *formatting*.\n"
+document = Beid::Document.parse(source)
+paragraph = document.root.children.find { |node| node.type == :paragraph }
+
+updated = Beid::Editing.replace_text(document, paragraph, "A new sentence.")
+puts updated
+# # Title
+#
+# A new sentence.
+```
+
+`Document#source` and `#to_s` return the exact source string. Node ranges and all offsets are half-open byte ranges in that original UTF-8 string. `position_at` returns zero-based line and Unicode-codepoint column; `utf16_position_at` returns a zero-based LSP position. `nodes_at` returns the document-to-deepest-node path at a byte offset.
+
+`Beid::Editing` provides `replace`, `replace_text`, `insert_before`, `insert_after`, `remove`, `set_attribute`, `set_directive`, and `move`. Each operation returns a new parsed `Document`; the original remains unchanged. Edits splice only the requested byte range. Replacements are reparsed, and nodes from another document are rejected.
+
+```ruby
+heading = document.root.children.first
+updated = Beid::Editing.set_attribute(document, heading, :level, 2)
+```
+
+`Document.parse(text, gfm: true, front_matter: true)` retains YAML front matter as raw text, recognizes HTML-comment directives (`<!-- layout: two-column -->`), fenced code blocks, headings, paragraphs, nested lists, block quotes, tables, task-list items, strikethrough, and footnote references/definitions. `::: name` fenced divs are retained as directive nodes. Inline nodes include emphasis, strong emphasis, inline and reference links, autolinks, images, code spans, and text. Reference definitions are retained as `:link_definition` nodes and resolved link nodes carry their destination and source ranges.
+
+Beid does not evaluate YAML or render HTML. It is not a complete CommonMark/GFM implementation: unsupported or ambiguous syntax may be represented as plain text or raw HTML. The official CommonMark 0.31.2 fixture is bundled under `spec/fixtures/commonmark`. CI verifies exact source round-tripping and node byte ranges, and compares a test-only HTML rendering of Beid's tree with the fixture's expected output after HTML5 DOM normalization. The current semantic HTML DOM agreement is 620/652 (95.1%), meeting the design's 95% CommonMark semantic-conformance gate; the remaining mismatches are listed by section in CI output. Nokogiri is a development-only dependency for this oracle and is not a runtime gem dependency. Regardless of parser coverage, `Document#to_s` is an exact round trip because serialization returns the original source rather than regenerating Markdown from the tree. Treat the tree as a best-effort editing view and keep application-level edits within recognized node ranges.
+
+## Development
+
+```sh
+bundle install
+bundle exec rake
+bundle exec rbs -I sig validate
+```
+
+## License
+
+MIT. See [LICENSE.txt](LICENSE.txt).

data/docs/adr/001-byte-range-edits.md

@@ -0,0 +1,16 @@
+# ADR 001: Byte-range edits instead of AST reserialization
+
+- Status: Accepted
+- Date: 2026-09-23
+
+## Context
+
+Beid is meant to support editing Markdown documents without changing untouched formatting. Rebuilding source from a parsed tree would normalize markers, spacing, line endings, and syntax Beid does not understand.
+
+## Decision
+
+Each node keeps a half-open byte range into an immutable source snapshot. Editing operations splice only that range and reparse the result; `Document#to_s` returns the stored source verbatim.
+
+## Consequences
+
+Unedited bytes remain unchanged, including syntax outside Beid's supported subset. Consumers must use a node from the same document snapshot and reparse after edits; ranges from an earlier document are not reusable.

data/docs/adr/002-html-comment-directives.md

@@ -0,0 +1,16 @@
+# ADR 002: HTML comments as directives
+
+- Status: Accepted
+- Date: 2026-09-23
+
+## Context
+
+Applications built on Markdown need lightweight metadata without introducing a custom block syntax that ordinary Markdown tools cannot preserve.
+
+## Decision
+
+Beid recognizes a standalone HTML comment containing one `key: value` pair, such as `<!-- layout: two-column -->`, as a directive. It also retains `::: name` fenced div blocks as source-positioned directive nodes, without interpreting their contents.
+
+## Consequences
+
+Directive metadata remains valid Markdown and round-trips unchanged. YAML and directive values are not evaluated; applications define their meaning. Beid's directive helper deliberately handles one key-value pair per HTML comment.

data/lib/beid/directive.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module Beid
+  module Directive
+    PATTERN = /\A<!--[ \t]*([A-Za-z][A-Za-z0-9_-]*)[ \t]*:[ \t]*(.*?)[ \t]*-->\z/m
+
+    def self.parse(html_comment)
+      match = PATTERN.match(html_comment)
+      return nil unless match
+
+      { match[1] => match[2] }
+    end
+
+    def self.render(values)
+      raise TypeError, "directive must be a Hash" unless values.is_a?(Hash)
+      raise ArgumentError, "one key-value pair per comment is supported" unless values.length == 1
+
+      fields = values.map do |key, value|
+        key = key.to_s
+        value = value.to_s
+        raise ArgumentError, "invalid directive key" unless key.match?(/\A[A-Za-z][A-Za-z0-9_-]*\z/)
+        raise ArgumentError, "directive value cannot contain a comment terminator or newline" if value.include?("-->") || value.match?(/[\r\n]/)
+
+        "#{key}: #{value}"
+      end
+      "<!-- #{fields.join("; ")} -->"
+    end
+  end
+end

data/lib/beid/document.rb

@@ -0,0 +1,112 @@
+# frozen_string_literal: true
+
+module Beid
+  class Document
+    attr_reader :source, :root, :front_matter, :options, :diagnostics
+
+    def self.parse(text, gfm: true, front_matter: true)
+      raise TypeError, "source must be a String" unless text.is_a?(String)
+
+      source = text.dup.freeze
+      options = { gfm: gfm, front_matter: front_matter }.freeze
+      root, raw_front_matter, diagnostics = Parser.new(source, **options).parse
+      new(source, root, raw_front_matter, options, diagnostics)
+    end
+
+    def initialize(source, root, front_matter, options, diagnostics)
+      @source, @root, @front_matter = source, root, front_matter&.freeze
+      @options, @diagnostics = options, diagnostics.freeze
+      @nodes = {}
+      walk(@root) { |node| @nodes[node.object_id] = node }
+    end
+
+    def nodes_at(offset)
+      check_offset(offset)
+      return [@root] if @source.empty? && offset.zero?
+      return [] unless @root.range.cover?(offset)
+
+      result = [@root]
+      current = @root
+      loop do
+        child = current.children.find { |entry| entry.range.cover?(offset) }
+        break unless child
+
+        result << child
+        current = child
+      end
+      result
+    end
+
+    def node_at(offset)
+      nodes_at(offset).last
+    rescue RangeError
+      nil
+    end
+
+    def range_of(node)
+      raise TypeError, "node must be a Beid::Node" unless node.is_a?(Node)
+      raise ArgumentError, "node does not belong to this document" unless @nodes[node.object_id].equal?(node)
+
+      node.range
+    end
+
+    # Returns zero-based [line, Unicode-codepoint column] for a UTF-8 byte offset.
+    def position_at(offset)
+      check_offset(offset)
+      raise EncodingError, "positions require valid UTF-8" unless @source.valid_encoding?
+
+      line_index = line_index_at(offset)
+      start = line_starts[line_index]
+      prefix = @source.byteslice(start...offset).to_s.sub(/(?:\r\n|\r|\n)\z/, "")
+      [line_index, prefix.length]
+    end
+
+    # Returns zero-based LSP [line, UTF-16 code-unit column].
+    def utf16_position_at(offset)
+      line, = position_at(offset)
+      start = line_starts[line]
+      prefix = @source.byteslice(start...offset).to_s.sub(/(?:\r\n|\r|\n)\z/, "")
+      [line, prefix.encode("UTF-16LE").bytesize / 2]
+    end
+
+    def valid?
+      @source.valid_encoding? && @diagnostics.empty?
+    end
+
+    def to_s
+      @source
+    end
+
+    def include_node?(node)
+      node.is_a?(Node) && @nodes[node.object_id].equal?(node)
+    end
+
+    private
+
+    def walk(node, &block)
+      yield node
+      node.children.each { |child| walk(child, &block) }
+    end
+
+    def check_offset(offset)
+      unless offset.is_a?(Integer) && (0..@source.bytesize).cover?(offset)
+        raise RangeError, "Byte offset is outside the document"
+      end
+      if @source.valid_encoding? && offset < @source.bytesize && (@source.getbyte(offset) & 0xc0) == 0x80
+        raise RangeError, "Byte offset splits a UTF-8 character"
+      end
+    end
+
+    def line_index_at(offset)
+      (line_starts.bsearch_index { |start| start > offset } || line_starts.length) - 1
+    end
+
+    def line_starts
+      @line_starts ||= begin
+        starts = [0]
+        @source.b.to_enum(:scan, /\r\n|\r|\n/n).each { starts << Regexp.last_match.end(0) }
+        starts
+      end
+    end
+  end
+end

data/lib/beid/editing.rb

@@ -0,0 +1,173 @@
+# frozen_string_literal: true
+
+module Beid
+  module Editing
+    def self.replace(document, node, markdown)
+      document.range_of(node)
+      raise TypeError, "markdown must be a String" unless markdown.is_a?(String)
+      terminator = document.source.byteslice(node.range.end - 2...node.range.end).to_s
+      terminator = document.source.byteslice(node.range.end - 1...node.range.end).to_s unless ["\r\n", "\n", "\r"].include?(terminator)
+      markdown += terminator if !markdown.empty? && ["\r\n", "\n", "\r"].include?(terminator) && !markdown.end_with?("\r", "\n")
+      edit_range(document, node, node.range, markdown)
+    end
+
+    def self.replace_text(document, node, text)
+      raise TypeError, "text must be a String" unless text.is_a?(String)
+      if node.type == :code_span
+        marker_size = [node.marker.length, text.scan(/`+/).map(&:length).max.to_i + 1].max
+        marker = "`" * marker_size
+        return edit(document, node, "#{marker}#{text}#{marker}")
+      end
+      range = case node.type
+      when :text
+        document.range_of(node)
+      when :heading, :paragraph, :footnote_definition
+        node.attributes.fetch(:content_range)
+      when :link, :image
+        node.attributes.fetch(:label_range)
+      when :emphasis, :strong, :strikethrough
+        node.range.begin + node.marker.bytesize...(node.range.end - node.marker.bytesize)
+      else
+        raise ArgumentError, "#{node.type} does not have replaceable text"
+      end
+      edit_range(document, node, range, escape_text(text))
+    end
+
+    def self.insert_after(document, node, markdown)
+      insert(document, node, markdown, after: true)
+    end
+
+    def self.insert_before(document, node, markdown)
+      insert(document, node, markdown, after: false)
+    end
+
+    def self.remove(document, node)
+      edit(document, node, "")
+    end
+
+    def self.set_attribute(document, node, key, value)
+      document.range_of(node)
+      case [node.type, key.to_sym]
+      when [:heading, :level]
+        level = Integer(value)
+        raise ArgumentError, "heading level must be between 1 and 6" unless (1..6).cover?(level)
+
+        if node.attributes[:style] == :atx
+          range = node.attributes.fetch(:marker_range)
+          edit_range(document, node, range, "#" * level)
+        else
+          edit_range(document, node, node.attributes.fetch(:marker_range), (level == 1 ? "=" : "-") * [node.marker.length, 3].max)
+        end
+      when [:link, :destination], [:image, :destination]
+        edit_range(document, node, node.attributes.fetch(:destination_range), escape_destination(value.to_s))
+      else
+        raise ArgumentError, "unsupported attribute #{key.inspect} for #{node.type}"
+      end
+    end
+
+    def self.set_directive(document, node, key, value)
+      document.range_of(node)
+      raise ArgumentError, "node is not an HTML directive" unless node.type == :directive && node.attributes[:kind] == :html_comment
+
+      values = Directive.parse(document.source.byteslice(node.range)) || {}
+      values[key.to_s] = value.to_s
+      edit_range(document, node, node.attributes.fetch(:comment_range), Directive.render(values))
+    end
+
+    def self.move(document, node, before: nil, after: nil)
+      document.range_of(node)
+      raise ArgumentError, "specify exactly one of before or after" unless (!!before ^ !!after)
+      target = before || after
+      target_range = document.range_of(target)
+      source_range = node.range
+      raise ArgumentError, "cannot move a node relative to itself" if node.equal?(target)
+      raise ArgumentError, "move ranges overlap" if source_range.begin < target_range.end && target_range.begin < source_range.end
+
+      source = document.source.b
+      moved = source.byteslice(source_range)
+      remainder = source.byteslice(0...source_range.begin) + source.byteslice(source_range.end..-1).to_s
+      destination = before ? target_range.begin : target_range.end
+      destination -= source_range.size if source_range.end <= destination
+      result = remainder.byteslice(0...destination) + moved + remainder.byteslice(destination..-1).to_s
+      parse_result(document, result)
+    end
+
+    def self.insert(document, node, markdown, after:)
+      document.range_of(node)
+      raise TypeError, "markdown must be a String" unless markdown.is_a?(String)
+      markdown = adapt_list_marker(markdown, node.marker) if node.type == :list_item
+      source = document.source.b
+      range = node.range
+      newline = newline_for(document.source)
+      if after
+        offset = range.end
+        if node.type == :list_item
+          item = source.byteslice(range)
+          prefix = item.end_with?("\r", "\n") ? "" : newline
+          insertion = prefix + markdown + (offset < source.bytesize ? newline : "")
+        else
+          insertion = newline + markdown
+          insertion += newline if offset < source.bytesize
+        end
+      else
+        offset = range.begin
+        insertion = if node.type == :list_item
+          markdown + newline
+        else
+          markdown + newline * 2
+        end
+      end
+      result = source.byteslice(0...offset) + insertion.b + source.byteslice(offset..-1).to_s
+      parse_result(document, result)
+    end
+
+    def self.edit(document, node, markdown)
+      document.range_of(node)
+      raise TypeError, "markdown must be a String" unless markdown.is_a?(String)
+      edit_range(document, node, node.range, markdown)
+    end
+
+    def self.edit_range(document, node, range, replacement)
+      document.range_of(node)
+      unless range.is_a?(Range) && range.begin >= node.range.begin && range.end <= node.range.end
+        raise ArgumentError, "edit range must be within the node"
+      end
+
+      source = document.source.b
+      result = source.byteslice(0...range.begin) + replacement.b + source.byteslice(range.end..-1).to_s
+      parse_result(document, result)
+    end
+
+    def self.parse_result(document, source)
+      result = Document.parse(source.force_encoding(document.source.encoding), **document.options)
+      raise Error, "edit produced invalid Markdown" unless result.valid?
+
+      result
+    end
+    private_class_method :parse_result
+
+    def self.escape_text(text)
+      text.gsub(/([\\`*_{}\[\]<>!])/) { |character| "\\#{character}" }
+    end
+    private_class_method :escape_text
+
+    def self.escape_destination(value)
+      raise ArgumentError, "destination cannot contain newlines or angle brackets" if value.match?(/[\r\n<>]/)
+
+      value.gsub(/([\\() ])/) { |character| "\\#{character}" }
+    end
+    private_class_method :escape_destination
+
+    def self.adapt_list_marker(markdown, marker)
+      return markdown unless marker && (match = /\A( {0,3})(?:[-+*]|\d{1,9}[.)])(?=[ \t])/.match(markdown))
+
+      markdown.sub(/\A( {0,3})(?:[-+*]|\d{1,9}[.)])(?=[ \t])/, "\\1#{marker}")
+    end
+    private_class_method :adapt_list_marker
+
+    def self.newline_for(source)
+      source[/\r\n|\n|\r/] || "\n"
+    end
+    private_class_method :newline_for
+  end
+end