commonmarker-merge 1.0.0

data/RUBOCOP.md

@@ -0,0 +1,71 @@
+# RuboCop Usage Guide
+
+## Overview
+
+A tale of two RuboCop plugin gems.
+
+### RuboCop Gradual
+
+This project uses `rubocop_gradual` instead of vanilla RuboCop for code style checking. The `rubocop_gradual` tool allows for gradual adoption of RuboCop rules by tracking violations in a lock file.
+
+### RuboCop LTS
+
+This project uses `rubocop-lts` to ensure, on a best-effort basis, compatibility with Ruby >= 1.9.2.
+RuboCop rules are meticulously configured by the `rubocop-lts` family of gems to ensure that a project is compatible with a specific version of Ruby. See: https://rubocop-lts.gitlab.io for more.
+
+## Checking RuboCop Violations
+
+To check for RuboCop violations in this project, always use:
+
+```bash
+bundle exec rake rubocop_gradual:check
+```
+
+**Do not use** the standard RuboCop commands like:
+- `bundle exec rubocop`
+- `rubocop`
+
+## Understanding the Lock File
+
+The `.rubocop_gradual.lock` file tracks all current RuboCop violations in the project. This allows the team to:
+
+1. Prevent new violations while gradually fixing existing ones
+2. Track progress on code style improvements
+3. Ensure CI builds don't fail due to pre-existing violations
+
+## Common Commands
+
+- **Check violations**
+    - `bundle exec rake rubocop_gradual`
+    - `bundle exec rake rubocop_gradual:check`
+- **(Safe) Autocorrect violations, and update lockfile if no new violations**
+  - `bundle exec rake rubocop_gradual:autocorrect`
+- **Force update the lock file (w/o autocorrect) to match violations present in code**
+  - `bundle exec rake rubocop_gradual:force_update`
+
+## Workflow
+
+1. Before submitting a PR, run `bundle exec rake rubocop_gradual:autocorrect`
+   a. or just the default `bundle exec rake`, as autocorrection is a pre-requisite of the default task.
+2. If there are new violations, either:
+   - Fix them in your code
+   - Run `bundle exec rake rubocop_gradual:force_update` to update the lock file (only for violations you can't fix immediately)
+3. Commit the updated `.rubocop_gradual.lock` file along with your changes
+
+## Never add inline RuboCop disables
+
+Do not add inline `rubocop:disable` / `rubocop:enable` comments anywhere in the codebase (including specs, except when following the few existing `rubocop:disable` patterns for a rule already being disabled elsewhere in the code). We handle exceptions in two supported ways:
+
+- Permanent/structural exceptions: prefer adjusting the RuboCop configuration (e.g., in `.rubocop.yml`) to exclude a rule for a path or file pattern when it makes sense project-wide.
+- Temporary exceptions while improving code: record the current violations in `.rubocop_gradual.lock` via the gradual workflow:
+  - `bundle exec rake rubocop_gradual:autocorrect` (preferred; will autocorrect what it can and update the lock only if no new violations were introduced)
+  - If needed, `bundle exec rake rubocop_gradual:force_update` (as a last resort when you cannot fix the newly reported violations immediately)
+
+In general, treat the rules as guidance to follow; fix violations rather than ignore them. For example, RSpec conventions in this project expect `described_class` to be used in specs that target a specific class under test.
+
+## Benefits of rubocop_gradual
+
+- Allows incremental adoption of code style rules
+- Prevents CI failures due to pre-existing violations
+- Provides a clear record of code style debt
+- Enables focused efforts on improving code quality over time

data/SECURITY.md

@@ -0,0 +1,21 @@
+# Security Policy
+
+## Supported Versions
+
+| Version  | Supported |
+|----------|-----------|
+| 1.latest | ✅         |
+
+## Security contact information
+
+To report a security vulnerability, please use the
+[Tidelift security contact](https://tidelift.com/security).
+Tidelift will coordinate the fix and disclosure.
+
+## Additional Support
+
+If you are interested in support for versions older than the latest release,
+please consider sponsoring the project / maintainer @ https://liberapay.com/pboling/donate,
+or find other sponsorship links in the [README].
+
+[README]: README.md

data/lib/commonmarker/merge/backend.rb

@@ -0,0 +1,336 @@
+# frozen_string_literal: true
+
+module Commonmarker
+  module Merge
+    # Commonmarker backend using the Commonmarker gem (comrak Rust parser)
+    #
+    # This backend wraps Commonmarker, a Ruby gem that provides bindings to
+    # comrak, a fast CommonMark-compliant Markdown parser written in Rust.
+    #
+    # @note This backend only parses Markdown source code
+    # @see https://github.com/gjtorikian/commonmarker Commonmarker gem
+    #
+    # @example Basic usage
+    #   parser = TreeHaver::Parser.new
+    #   parser.language = Commonmarker::Merge::Backend::Language.markdown
+    #   tree = parser.parse(markdown_source)
+    #   root = tree.root_node
+    #   puts root.type  # => "document"
+    module Backend
+      @load_attempted = false
+      @loaded = false
+
+      # Check if the Commonmarker backend is available
+      #
+      # @return [Boolean] true if commonmarker gem is available
+      class << self
+        def available?
+          return @loaded if @load_attempted # rubocop:disable ThreadSafety/ClassInstanceVariable
+          @load_attempted = true # rubocop:disable ThreadSafety/ClassInstanceVariable
+          begin
+            require "commonmarker"
+            @loaded = true # rubocop:disable ThreadSafety/ClassInstanceVariable
+          rescue LoadError
+            @loaded = false # rubocop:disable ThreadSafety/ClassInstanceVariable
+          rescue StandardError
+            @loaded = false # rubocop:disable ThreadSafety/ClassInstanceVariable
+          end
+          @loaded # rubocop:disable ThreadSafety/ClassInstanceVariable
+        end
+
+        # Reset the load state (primarily for testing)
+        #
+        # @return [void]
+        # @api private
+        def reset!
+          @load_attempted = false # rubocop:disable ThreadSafety/ClassInstanceVariable
+          @loaded = false # rubocop:disable ThreadSafety/ClassInstanceVariable
+        end
+
+        # Get capabilities supported by this backend
+        #
+        # @return [Hash{Symbol => Object}] capability map
+        def capabilities
+          return {} unless available?
+          {
+            backend: :commonmarker,
+            query: false,
+            bytes_field: false,       # Commonmarker uses line/column
+            incremental: false,
+            pure_ruby: false,         # Uses Rust via FFI
+            markdown_only: true,
+            error_tolerant: true,     # Markdown is forgiving
+          }
+        end
+      end
+
+      # Commonmarker language wrapper
+      #
+      # Commonmarker only parses Markdown. This class exists for API compatibility.
+      #
+      # @example
+      #   language = Commonmarker::Merge::Backend::Language.markdown
+      #   parser.language = language
+      class Language < TreeHaver::Base::Language
+        # Create a new Commonmarker language instance
+        #
+        # @param name [Symbol] Language name (should be :markdown)
+        # @param options [Hash] Commonmarker parse options
+        def initialize(name = :markdown, options: {})
+          super(name, backend: :commonmarker, options: options)
+        end
+
+        class << self
+          # Create a Markdown language instance
+          #
+          # @param options [Hash] Commonmarker parse options
+          # @return [Language] Markdown language
+          def markdown(options: {})
+            new(:markdown, options: options)
+          end
+
+          # Load language from library path (API compatibility)
+          #
+          # @param _path [String] Ignored - Commonmarker doesn't load external grammars
+          # @param symbol [String, nil] Ignored
+          # @param name [String, nil] Language name hint (defaults to :markdown)
+          # @return [Language] Markdown language
+          # @raise [TreeHaver::NotAvailable] if requested language is not Markdown
+          def from_library(_path = nil, symbol: nil, name: nil)
+            lang_name = name || symbol&.to_s&.sub(/^tree_sitter_/, "")&.to_sym || :markdown
+
+            unless lang_name == :markdown
+              raise TreeHaver::NotAvailable,
+                "Commonmarker backend only supports Markdown, not #{lang_name}."
+            end
+
+            markdown
+          end
+        end
+      end
+
+      # Commonmarker parser wrapper
+      class Parser < TreeHaver::Base::Parser
+        # Parse Markdown source code
+        #
+        # @param source [String] Markdown source to parse
+        # @return [Tree] Parsed tree
+        def parse(source)
+          raise "Language not set" unless language
+          Backend.available? or raise "Commonmarker not available"
+
+          opts = language.options || {}
+          doc = ::Commonmarker.parse(source, options: opts)
+          Tree.new(doc, source)
+        end
+      end
+
+      # Commonmarker tree wrapper
+      class Tree < TreeHaver::Base::Tree
+        def initialize(document, source)
+          super(document, source: source)
+        end
+
+        def root_node
+          Node.new(inner_tree, source: source, lines: lines)
+        end
+      end
+
+      # Commonmarker node wrapper
+      #
+      # Wraps Commonmarker::Node to provide TreeHaver::Node-compatible interface.
+      class Node < TreeHaver::Base::Node
+        # Get the node type as a string
+        #
+        # @return [String] Node type
+        def type
+          inner_node.type.to_s
+        end
+
+        # Alias for TreeHaver compatibility
+        alias_method :kind, :type
+
+        # Get the text content of this node
+        #
+        # @return [String] Node text
+        def text
+          if inner_node.respond_to?(:string_content)
+            begin
+              content = inner_node.string_content.to_s
+              return content unless content.empty?
+            rescue TypeError
+              # Container node - fall through
+            end
+          end
+          children.map(&:text).join
+        end
+
+        # Get child nodes
+        #
+        # @return [Array<Node>] Child nodes
+        def children
+          return [] unless inner_node.respond_to?(:each)
+
+          result = []
+          inner_node.each { |child| result << Node.new(child, source: source, lines: lines) }
+          result
+        end
+
+        # Get start byte offset
+        def start_byte
+          sp = start_point
+          calculate_byte_offset(sp[:row], sp[:column])
+        end
+
+        # Get end byte offset
+        def end_byte
+          ep = end_point
+          calculate_byte_offset(ep[:row], ep[:column])
+        end
+
+        # Get start point (0-based row/column)
+        # @return [Point] Start position
+        def start_point
+          if inner_node.respond_to?(:source_position)
+            begin
+              pos = inner_node.source_position
+              if pos && pos[:start_line]
+                return Point.new(pos[:start_line] - 1, (pos[:start_column] || 1) - 1)
+              end
+            rescue
+              nil
+            end
+          end
+
+          # Fallback: check sourcepos (old API)
+          begin
+            pos = inner_node.sourcepos
+            return Point.new(pos[0] - 1, pos[1] - 1) if pos
+          rescue
+            nil
+          end
+
+          Point.new(0, 0)
+        end
+
+        # Get end point (0-based row/column)
+        # @return [Point] End position
+        def end_point
+          if inner_node.respond_to?(:source_position)
+            begin
+              pos = inner_node.source_position
+              if pos && pos[:end_line]
+                return Point.new(pos[:end_line] - 1, (pos[:end_column] || 1) - 1)
+              end
+            rescue
+              nil
+            end
+          end
+
+          begin
+            pos = inner_node.sourcepos
+            return Point.new(pos[2] - 1, pos[3] - 1) if pos
+          rescue
+            nil
+          end
+
+          Point.new(0, 0)
+        end
+
+        # Commonmarker-specific methods
+
+        # Get heading level (1-6)
+        # @return [Integer, nil]
+        def header_level
+          return unless type == "heading"
+          begin
+            inner_node.header_level
+          rescue
+            nil
+          end
+        end
+
+        # Get fence info for code blocks
+        # @return [String, nil]
+        def fence_info
+          return unless type == "code_block"
+          begin
+            inner_node.fence_info
+          rescue
+            nil
+          end
+        end
+
+        # Get URL for links/images
+        # @return [String, nil]
+        def url
+          inner_node.url
+        rescue
+          nil
+        end
+
+        # Get title for links/images
+        # @return [String, nil]
+        def title
+          inner_node.title
+        rescue
+          nil
+        end
+
+        # Get the next sibling
+        # @return [Node, nil]
+        def next_sibling
+          sibling = begin
+            inner_node.next_sibling
+          rescue
+            nil
+          end
+          sibling ? Node.new(sibling, source: source, lines: lines) : nil
+        end
+
+        # Get the previous sibling
+        # @return [Node, nil]
+        def prev_sibling
+          sibling = begin
+            inner_node.previous_sibling
+          rescue
+            nil
+          end
+          sibling ? Node.new(sibling, source: source, lines: lines) : nil
+        end
+
+        # Get the parent node
+        # @return [Node, nil]
+        def parent
+          p = begin
+            inner_node.parent
+          rescue
+            nil
+          end
+          p ? Node.new(p, source: source, lines: lines) : nil
+        end
+      end
+
+      # Alias Point to the base class for compatibility
+      Point = TreeHaver::Base::Point
+
+      # Register this backend with TreeHaver
+      # Register for generic :markdown language
+      ::TreeHaver.register_language(
+        :markdown,
+        backend_type: :commonmarker,
+        backend_module: self,
+        gem_name: "commonmarker",
+      )
+
+      # Register the full tag for RSpec dependency tags with require path
+      # This enables tree_haver to lazily load this gem when checking availability
+      ::TreeHaver::BackendRegistry.register_tag(
+        :commonmarker_backend,
+        category: :backend,
+        backend_name: :commonmarker,
+        require_path: "commonmarker/merge",
+      ) { available? }
+    end
+  end
+end

data/lib/commonmarker/merge/debug_logger.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module Commonmarker
+  module Merge
+    # Debug logging utility for Commonmarker::Merge operations.
+    #
+    # Extends Ast::Merge::DebugLogger to provide consistent logging
+    # across all merge gems. Logs are controlled via environment variables.
+    #
+    # @example Enable debug logging
+    #   ENV["COMMONMARKER_MERGE_DEBUG"] = "1"
+    #   DebugLogger.debug("Parsing markdown", { file: "README.md" })
+    #
+    # @example Time an operation
+    #   result = DebugLogger.time("parse") { Commonmarker.parse(source) }
+    #
+    # @see Ast::Merge::DebugLogger Base module
+    module DebugLogger
+      extend Ast::Merge::DebugLogger
+
+      # Configure for commonmarker-merge
+      self.env_var_name = "COMMONMARKER_MERGE_DEBUG"
+      self.log_prefix = "[commonmarker-merge]"
+    end
+  end
+end

data/lib/commonmarker/merge/file_analysis.rb

@@ -0,0 +1,51 @@
+# frozen_string_literal: true
+
+module Commonmarker
+  module Merge
+    # File analysis for Markdown files using CommonMarker.
+    #
+    # This is a thin wrapper around Markdown::Merge::FileAnalysis that:
+    # - Forces the :commonmarker backend
+    # - Sets the default freeze token to "commonmarker-merge"
+    # - Exposes commonmarker-specific options
+    #
+    # @example Basic usage
+    #   analysis = FileAnalysis.new(markdown_source)
+    #   analysis.statements.each do |node|
+    #     puts "#{node.merge_type}: #{node.type}"
+    #   end
+    #
+    # @example With custom freeze token
+    #   analysis = FileAnalysis.new(source, freeze_token: "my-merge")
+    #
+    # @see Markdown::Merge::FileAnalysis Underlying implementation
+    class FileAnalysis < Markdown::Merge::FileAnalysis
+      # Default freeze token for commonmarker-merge
+      # @return [String]
+      DEFAULT_FREEZE_TOKEN = "commonmarker-merge"
+
+      # Initialize file analysis with CommonMarker backend.
+      #
+      # @param source [String] Markdown source code to analyze
+      # @param freeze_token [String] Token for freeze block markers (default: "commonmarker-merge")
+      # @param signature_generator [Proc, nil] Custom signature generator
+      # @param options [Hash] CommonMarker parse options
+      def initialize(source, freeze_token: DEFAULT_FREEZE_TOKEN, signature_generator: nil, options: {})
+        super(
+          source,
+          backend: :commonmarker,
+          freeze_token: freeze_token,
+          signature_generator: signature_generator,
+          options: options,
+        )
+      end
+
+      # Returns the FreezeNode class to use.
+      #
+      # @return [Class] Commonmarker::Merge::FreezeNode
+      def freeze_node_class
+        FreezeNode
+      end
+    end
+  end
+end

data/lib/commonmarker/merge/freeze_node.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+module Commonmarker
+  module Merge
+    # Represents a frozen block of Markdown content that should be preserved during merges.
+    #
+    # Inherits from Markdown::Merge::FreezeNode which provides the generic
+    # freeze block handling.
+    #
+    # Freeze blocks are marked with HTML comments:
+    #   <!-- commonmarker-merge:freeze -->
+    #   ... frozen content ...
+    #   <!-- commonmarker-merge:unfreeze -->
+    #
+    # @example Basic freeze block
+    #   <!-- commonmarker-merge:freeze -->
+    #   ## Custom Section
+    #   This content will not be modified by merge operations.
+    #   <!-- commonmarker-merge:unfreeze -->
+    #
+    # @example Freeze block with reason
+    #   <!-- commonmarker-merge:freeze Manual TOC -->
+    #   ## Table of Contents
+    #   - [Introduction](#introduction)
+    #   - [Usage](#usage)
+    #   <!-- commonmarker-merge:unfreeze -->
+    #
+    # @see Markdown::Merge::FreezeNode
+    class FreezeNode < Markdown::Merge::FreezeNode
+    end
+  end
+end

data/lib/commonmarker/merge/smart_merger.rb

@@ -0,0 +1,139 @@
+# frozen_string_literal: true
+
+module Commonmarker
+  module Merge
+    # Orchestrates the smart merge process for Markdown files using CommonMarker.
+    #
+    # This is a thin wrapper around Markdown::Merge::SmartMerger that:
+    # - Forces the :commonmarker backend
+    # - Sets commonmarker-specific defaults (freeze token, inner_merge_code_blocks)
+    # - Exposes commonmarker-specific options (options hash)
+    #
+    # @example Basic merge (destination customizations preserved)
+    #   merger = SmartMerger.new(template_content, dest_content)
+    #   result = merger.merge
+    #   if result.success?
+    #     File.write("output.md", result.content)
+    #   end
+    #
+    # @example Template updates win
+    #   merger = SmartMerger.new(
+    #     template_content,
+    #     dest_content,
+    #     preference: :template,
+    #     add_template_only_nodes: true
+    #   )
+    #   result = merger.merge
+    #
+    # @example Custom signature matching
+    #   sig_gen = ->(node) {
+    #     canonical_type = Ast::Merge::NodeTyping.merge_type_for(node) || node.type
+    #     if canonical_type == :heading
+    #       [:heading, node.header_level]  # Match by level only, not content
+    #     else
+    #       node  # Fall through to default
+    #     end
+    #   }
+    #   merger = SmartMerger.new(
+    #     template_content,
+    #     dest_content,
+    #     signature_generator: sig_gen
+    #   )
+    #
+    # @see Markdown::Merge::SmartMerger Underlying implementation
+    class SmartMerger < Markdown::Merge::SmartMerger
+      # Creates a new SmartMerger for intelligent Markdown file merging.
+      #
+      # @param template_content [String] Template Markdown source code
+      # @param dest_content [String] Destination Markdown source code
+      #
+      # @param signature_generator [Proc, nil] Optional proc to generate custom node signatures.
+      #   The proc receives a node (wrapped with canonical merge_type) and should return one of:
+      #   - An array representing the node's signature
+      #   - `nil` to indicate the node should have no signature
+      #   - The original node to fall through to default signature computation
+      #
+      # @param preference [Symbol] Controls which version to use when nodes
+      #   have matching signatures but different content:
+      #   - `:destination` (default) - Use destination version (preserves customizations)
+      #   - `:template` - Use template version (applies updates)
+      #
+      # @param add_template_only_nodes [Boolean] Controls whether to add nodes that only
+      #   exist in template:
+      #   - `false` (default) - Skip template-only nodes
+      #   - `true` - Add template-only nodes to result
+      #
+      # @param freeze_token [String] Token to use for freeze block markers.
+      #   Default: "commonmarker-merge"
+      #   Looks for: <!-- commonmarker-merge:freeze --> / <!-- commonmarker-merge:unfreeze -->
+      #
+      # @param options [Hash] CommonMarker parse options
+      #
+      # @param match_refiner [#call, nil] Optional match refiner for fuzzy matching of
+      #   unmatched nodes. Default: nil (fuzzy matching disabled).
+      #   Set to TableMatchRefiner.new to enable fuzzy table matching.
+      #
+      # @param node_typing [Hash{Symbol,String => #call}, nil] Node typing configuration
+      #   for per-node-type merge preferences.
+      # @param extra_options [Hash] Additional options for forward compatibility
+      #
+      # @raise [TemplateParseError] If template has syntax errors
+      # @raise [DestinationParseError] If destination has syntax errors
+      def initialize(
+        template_content,
+        dest_content,
+        signature_generator: nil,
+        preference: :destination,
+        add_template_only_nodes: false,
+        freeze_token: DEFAULT_FREEZE_TOKEN,
+        options: {},
+        match_refiner: nil,
+        node_typing: nil,
+        **extra_options
+      )
+        super(
+          template_content,
+          dest_content,
+          backend: :commonmarker,
+          signature_generator: signature_generator,
+          preference: preference,
+          add_template_only_nodes: add_template_only_nodes,
+          inner_merge_code_blocks: DEFAULT_INNER_MERGE_CODE_BLOCKS,
+          freeze_token: freeze_token,
+          match_refiner: match_refiner,
+          node_typing: node_typing,
+          options: options,
+          **extra_options
+        )
+      end
+
+      # Returns the TemplateParseError class to use.
+      #
+      # @return [Class] Commonmarker::Merge::TemplateParseError
+      def template_parse_error_class
+        TemplateParseError
+      end
+
+      # Returns the DestinationParseError class to use.
+      #
+      # @return [Class] Commonmarker::Merge::DestinationParseError
+      def destination_parse_error_class
+        DestinationParseError
+      end
+
+      # Create a FileAnalysis instance for parsing.
+      #
+      # @param content [String] Markdown content to analyze
+      # @param options [Hash] Analysis options
+      # @return [Commonmarker::Merge::FileAnalysis] File analysis instance
+      def create_file_analysis(content, **opts)
+        FileAnalysis.new(
+          content,
+          freeze_token: opts[:freeze_token],
+          signature_generator: opts[:signature_generator],
+          options: opts[:options] || {},
+        )
+      end
+    end
+  end
+end

data/lib/commonmarker/merge/version.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+module Commonmarker
+  module Merge
+    # Version information for Commonmarker::Merge
+    module Version
+      # Current version of the commonmarker-merge gem
+      VERSION = "1.0.0"
+    end
+    VERSION = Version::VERSION # traditional location
+  end
+end