canon 0.1.7 → 0.1.9

data/docs/understanding/comparison-pipeline.adoc

@@ -83,6 +83,128 @@ graph TD
 
 **Documentation**: See link:../features/diff-formatting/[Diff Formatting] and link:../features/diff-formatting/algorithm-specific-output.adoc[Algorithm-Specific Output]
 
+== DiffNode Data Flow
+
+=== How differences flow through the layers
+
+DiffNode objects are created in Layer 2, enriched with metadata, and flow through to Layer 4 for rendering:
+
+[mermaid]
+----
+graph LR
+    A[Layer 2: Algorithm] --> B[Create DiffNode]
+    B --> C[Enrich Metadata]
+    C --> D[Layer 3: Classification]
+    D --> E[Layer 4: Rendering]
+    E --> F[Formatted Output]
+
+    C --> C1[PathBuilder]
+    C --> C2[NodeSerializer]
+    C1 --> G[path]
+    C2 --> H[serialized_before/after]
+    C2 --> I[attributes_before/after]
+
+    style A fill:#fff4e1
+    style C fill:#e1f5ff
+    style E fill:#e1ffe1
+----
+
+=== Layer 2: DiffNode creation
+
+Each algorithm creates DiffNode objects when it finds differences:
+
+[source,ruby]
+----
+# DOM algorithm: Creates DiffNode during element-by-element comparison
+diff_node = Canon::Diff::DiffNode.new(
+  node1: element1,
+  node2: element2,
+  dimension: :text_content,
+  reason: "Text content differs"
+)
+
+# Semantic algorithm: Creates DiffNode from tree operations
+diff_node = Canon::Diff::DiffNode.new(
+  node1: nil,
+  node2: inserted_node,
+  dimension: :element_structure,
+  reason: "Element inserted"
+)
+----
+
+=== Metadata enrichment
+
+After creation, DiffNodes are enriched with metadata for Layer 4 rendering:
+
+[source,ruby]
+----
+# Enriched with:
+{
+  path: "/#document/div[0]/p[1]/span[2]",      # Canonical location
+  serialized_before: "<span>Old text</span>",  # Captured state
+  serialized_after: "<span>New text</span>",   # Captured state
+  attributes_before: {"id" => "old"},          # Normalized attrs
+  attributes_after: {"id" => "new"}            # Normalized attrs
+}
+----
+
+**Enrichment utilities**:
+
+* **PathBuilder**: Generates canonical paths with ordinal indices
+* **NodeSerializer**: Library-agnostic serialization of node content
+* **Attribute extraction**: Normalized attribute hashes
+
+See link:../internals/diffnode-enrichment.adoc[DiffNode Enrichment] for implementation details.
+
+=== Layer 3: Classification
+
+DiffNodes are classified to determine their impact:
+
+[source,ruby]
+----
+diff_node.normative = true   # Affects semantic equivalence
+diff_node.formatting = true  # Purely cosmetic difference
+----
+
+This classification affects whether differences cause `equivalent?` to return false.
+
+=== Layer 4: Rendering
+
+Layer 4 formatters use enriched metadata to display differences:
+
+[source,text]
+----
+🔍 DIFFERENCE #1/3 [NORMATIVE]
+════════════════════════════════════════════════════════════════════════
+Dimension: text_content
+Location:  /#document/div[0]/p[1]/span[2]
+
+⊖ Expected (File 1):
+   <span>Old text</span>
+
+⊕ Actual (File 2):
+   <span>New text</span>
+
+✨ Changes:
+   Text content changed from "Old text" to "New text"
+----
+
+* The `Location` field uses the enriched `path`
+* The before/after content uses `serialized_before/after`
+* Attribute differences use `attributes_before/after`
+
+This ensures accurate display regardless of which parsing library was used.
+
+=== Benefits of enriched DiffNodes
+
+**Library flexibility**: Layer 4 works with any parsing library through enriched metadata
+
+**Performance**: Metadata captured once at diff creation, not recomputed during rendering
+
+**Accuracy**: Shows actual node state when difference was found, not current state
+
+**Debuggability**: Ordinal indices in paths make it easy to locate specific elements
+
 == Complete Example
 
 Here's a full 4-layer configuration showing all layers working together:

data/lib/canon/cache.rb

@@ -0,0 +1,129 @@
+# frozen_string_literal: true
+
+require "digest"
+
+module Canon
+  # Cache for expensive operations during document comparison
+  #
+  # Provides thread-safe caching with size limits to prevent memory bloat.
+  # Uses LRU (Least Recently Used) eviction when cache is full.
+  #
+  # @example Cache a parsed document
+  #   key = Cache.key_for_document(xml_string, :xml, :none)
+  #   parsed = Cache.fetch(:document_parse, key) { parse_xml(xml_string) }
+  #
+  # @example Clear all caches (e.g., between test cases)
+  #   Cache.clear_all
+  module Cache
+    class << self
+      # Maximum number of entries per cache category
+      MAX_CACHE_SIZE = 100
+
+      # Fetch a value from cache, or compute and cache it
+      #
+      # @param category [Symbol] Cache category (:document_parse, :format_detect, etc.)
+      # @param key [String] Cache key
+      # @yield Block to compute value if not cached
+      # @return [Object] Cached or computed value
+      def fetch(category, key)
+        cache = cache_for(category)
+
+        # Check if key exists
+        if cache.key?(key)
+          # Update access time for LRU
+          cache[key][:accessed] = Time.now
+          return cache[key][:value]
+        end
+
+        # Compute and cache the value
+        value = yield
+
+        # Evict oldest entry if cache is full
+        if cache.size >= MAX_CACHE_SIZE
+          oldest_key = cache.min_by { |_, v| v[:accessed] }&.first
+          cache.delete(oldest_key) if oldest_key
+        end
+
+        cache[key] = { value: value, accessed: Time.now }
+        value
+      end
+
+      # Clear all caches
+      #
+      # Useful for tests or when memory needs to be freed
+      def clear_all
+        @caches&.each_value(&:clear)
+        @caches = nil
+      end
+
+      # Clear a specific cache category
+      #
+      # @param category [Symbol] Cache category to clear
+      def clear_category(category)
+        return unless @caches&.key?(category)
+
+        @caches[category]&.clear
+      end
+
+      # Get cache statistics
+      #
+      # @return [Hash] Statistics about cache usage
+      def stats
+        @caches&.transform_values(&:size) || {}
+      end
+
+      # Generate cache key for document parsing
+      #
+      # @param content [String] Document content
+      # @param format [Symbol] Document format
+      # @param preprocessing [Symbol] Preprocessing option
+      # @return [String] Cache key
+      def key_for_document(content, format, preprocessing)
+        digest = Digest::SHA256.hexdigest(content)
+        "doc:#{format}:#{preprocessing}:#{digest[0..16]}"
+      end
+
+      # Generate cache key for format detection
+      #
+      # @param content [String] Document content
+      # @return [String] Cache key
+      def key_for_format_detection(content)
+        # Use first 100 chars for quick key, plus length
+        preview = content[0..100]
+        digest = Digest::SHA256.hexdigest(preview + content.length.to_s)
+        "fmt:#{digest[0..16]}"
+      end
+
+      # Generate cache key for XML canonicalization
+      #
+      # @param content [String] XML content
+      # @param with_comments [Boolean] Whether to include comments
+      # @return [String] Cache key
+      def key_for_c14n(content, with_comments)
+        digest = Digest::SHA256.hexdigest(content)
+        "c14n:#{with_comments}:#{digest[0..16]}"
+      end
+
+      # Generate cache key for preprocessing
+      #
+      # @param content [String] Original content
+      # @param preprocessing [Symbol] Preprocessing type
+      # @return [String] Cache key
+      def key_for_preprocessing(content, preprocessing)
+        digest = Digest::SHA256.hexdigest(content)
+        "pre:#{preprocessing}:#{digest[0..16]}"
+      end
+
+      private
+
+      # Get or create cache for a category
+      #
+      # @param category [Symbol] Cache category
+      # @return [Hash] Cache hash for category
+      def cache_for(category)
+        @caches ||= {}
+        @caches[category] ||= {}
+      end
+    end
+  end
+end

data/lib/canon/comparison/dimensions/attribute_order_dimension.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+require_relative "base_dimension"
+
+module Canon
+  module Comparison
+    module Dimensions
+      # Attribute order dimension
+      #
+      # Handles comparison of attribute ordering.
+      # Supports :strict and :ignore behaviors.
+      #
+      # Behaviors:
+      # - :strict - Attributes must appear in the same order
+      # - :ignore - Attribute order doesn't matter
+      class AttributeOrderDimension < BaseDimension
+        # Extract attribute order from a node
+        #
+        # @param node [Moxml::Node, Nokogiri::XML::Node] Node to extract from
+        # @return [Array<Symbol>] Array of attribute names in order
+        def extract_data(node)
+          return [] unless node
+
+          # Handle Moxml nodes
+          if node.is_a?(Moxml::Node)
+            extract_from_moxml(node)
+          # Handle Nokogiri nodes
+          elsif node.is_a?(Nokogiri::XML::Node)
+            extract_from_nokogiri(node)
+          else
+            []
+          end
+        end
+
+        # Strict attribute order comparison
+        #
+        # @param order1 [Array<Symbol>] First attribute order
+        # @param order2 [Array<Symbol>] Second attribute order
+        # @return [Boolean] true if attribute order is exactly the same
+        def compare_strict(order1, order2)
+          order1 == order2
+        end
+
+        private
+
+        # Extract attribute order from Moxml node
+        #
+        # @param node [Moxml::Node] Moxml node
+        # @return [Array<Symbol>] Array of attribute names in order
+        def extract_from_moxml(node)
+          return [] unless node.node_type == :element
+
+          node.attributes.map { |attr| attr.name.to_sym }
+        end
+
+        # Extract attribute order from Nokogiri node
+        #
+        # @param node [Nokogiri::XML::Node] Nokogiri node
+        # @return [Array<Symbol>] Array of attribute names in order
+        def extract_from_nokogiri(node)
+          return [] unless node.node_type == Nokogiri::XML::Node::ELEMENT_NODE
+
+          node.attribute_nodes.map { |attr| attr.name.to_sym }
+        end
+      end
+    end
+  end
+end

data/lib/canon/comparison/dimensions/attribute_presence_dimension.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+require_relative "base_dimension"
+
+module Canon
+  module Comparison
+    module Dimensions
+      # Attribute presence dimension
+      #
+      # Handles comparison of attribute presence (which attributes exist).
+      # Supports :strict and :ignore behaviors.
+      #
+      # Behaviors:
+      # - :strict - Attribute names must match exactly
+      # - :ignore - Skip attribute presence comparison
+      class AttributePresenceDimension < BaseDimension
+        # Extract attribute names from a node
+        #
+        # @param node [Moxml::Node, Nokogiri::XML::Node] Node to extract from
+        # @return [Array<Symbol>] Array of attribute names
+        def extract_data(node)
+          return [] unless node
+
+          # Handle Moxml nodes
+          if node.is_a?(Moxml::Node)
+            extract_from_moxml(node)
+          # Handle Nokogiri nodes
+          elsif node.is_a?(Nokogiri::XML::Node)
+            extract_from_nokogiri(node)
+          else
+            []
+          end
+        end
+
+        # Strict attribute presence comparison
+        #
+        # @param names1 [Array<Symbol>] First attribute names
+        # @param names2 [Array<Symbol>] Second attribute names
+        # @return [Boolean] true if attribute names are exactly equal
+        def compare_strict(names1, names2)
+          names1.sort == names2.sort
+        end
+
+        private
+
+        # Extract attribute names from Moxml node
+        #
+        # @param node [Moxml::Node] Moxml node
+        # @return [Array<Symbol>] Array of attribute names
+        def extract_from_moxml(node)
+          return [] unless node.node_type == :element
+
+          node.attributes.map { |attr| attr.name.to_sym }
+        end
+
+        # Extract attribute names from Nokogiri node
+        #
+        # @param node [Nokogiri::XML::Node] Nokogiri node
+        # @return [Array<Symbol>] Array of attribute names
+        def extract_from_nokogiri(node)
+          return [] unless node.node_type == Nokogiri::XML::Node::ELEMENT_NODE
+
+          node.attribute_nodes.map { |attr| attr.name.to_sym }
+        end
+      end
+    end
+  end
+end

data/lib/canon/comparison/dimensions/attribute_values_dimension.rb

@@ -0,0 +1,171 @@
+# frozen_string_literal: true
+
+require_relative "base_dimension"
+require_relative "../match_options"
+
+module Canon
+  module Comparison
+    module Dimensions
+      # Attribute values dimension
+      #
+      # Handles comparison of attribute values.
+      # Supports :strict, :strip, :compact, :normalize, and :ignore behaviors.
+      #
+      # Behaviors:
+      # - :strict - Exact attribute value comparison
+      # - :strip - Compare with leading/trailing whitespace removed
+      # - :compact - Compare with internal whitespace collapsed
+      # - :normalize - Compare with whitespace stripped and collapsed
+      # - :ignore - Skip attribute value comparison
+      class AttributeValuesDimension < BaseDimension
+        # Extract attribute values from a node
+        #
+        # Returns a hash of attribute name to value.
+        #
+        # @param node [Moxml::Node, Nokogiri::XML::Node] Node to extract from
+        # @return [Hash] Attribute name to value mapping
+        def extract_data(node)
+          return {} unless node
+
+          # Handle Moxml nodes
+          if node.is_a?(Moxml::Node)
+            extract_from_moxml(node)
+          # Handle Nokogiri nodes
+          elsif node.is_a?(Nokogiri::XML::Node)
+            extract_from_nokogiri(node)
+          else
+            {}
+          end
+        end
+
+        # Strict attribute value comparison
+        #
+        # @param attrs1 [Hash] First attributes hash
+        # @param attrs2 [Hash] Second attributes hash
+        # @return [Boolean] true if all attribute values are exactly equal
+        def compare_strict(attrs1, attrs2)
+          # Get all unique attribute names
+          all_keys = (attrs1.keys | attrs2.keys)
+
+          all_keys.all? do |key|
+            attrs1[key].to_s == attrs2[key].to_s
+          end
+        end
+
+        # Strip comparison
+        #
+        # Compare with leading/trailing whitespace removed.
+        #
+        # @param attrs1 [Hash] First attributes hash
+        # @param attrs2 [Hash] Second attributes hash
+        # @return [Boolean] true if stripped values are equal
+        def compare_strip(attrs1, attrs2)
+          all_keys = (attrs1.keys | attrs2.keys)
+
+          all_keys.all? do |key|
+            attrs1[key].to_s.strip == attrs2[key].to_s.strip
+          end
+        end
+
+        # Compact comparison
+        #
+        # Compare with internal whitespace collapsed.
+        #
+        # @param attrs1 [Hash] First attributes hash
+        # @param attrs2 [Hash] Second attributes hash
+        # @return [Boolean] true if compacted values are equal
+        def compare_compact(attrs1, attrs2)
+          all_keys = (attrs1.keys | attrs2.keys)
+
+          all_keys.all? do |key|
+            compact_whitespace(attrs1[key].to_s) == compact_whitespace(attrs2[key].to_s)
+          end
+        end
+
+        # Normalized comparison
+        #
+        # Compare with whitespace stripped and collapsed.
+        #
+        # @param attrs1 [Hash] First attributes hash
+        # @param attrs2 [Hash] Second attributes hash
+        # @return [Boolean] true if normalized values are equal
+        def compare_normalize(attrs1, attrs2)
+          all_keys = (attrs1.keys | attrs2.keys)
+
+          all_keys.all? do |key|
+            normalize_text(attrs1[key].to_s) == normalize_text(attrs2[key].to_s)
+          end
+        end
+
+        # Compare with custom behavior
+        #
+        # Supports the extended behaviors for attribute values.
+        #
+        # @param data1 [Object] First data
+        # @param data2 [Object] Second data
+        # @param behavior [Symbol] Comparison behavior
+        # @return [Boolean] true if data matches according to behavior
+        def compare(data1, data2, behavior)
+          case behavior
+          when :strip
+            compare_strip(data1, data2)
+          when :compact
+            compare_compact(data1, data2)
+          else
+            super
+          end
+        end
+
+        private
+
+        # Extract attributes from Moxml node
+        #
+        # @param node [Moxml::Node] Moxml node
+        # @return [Hash] Attribute name to value mapping
+        def extract_from_moxml(node)
+          return {} unless node.node_type == :element
+
+          attrs = {}
+          node.attributes.each do |attr|
+            attrs[attr.name] = attr.value
+          end
+          attrs
+        end
+
+        # Extract attributes from Nokogiri node
+        #
+        # @param node [Nokogiri::XML::Node] Nokogiri node
+        # @return [Hash] Attribute name to value mapping
+        def extract_from_nokogiri(node)
+          return {} unless node.node_type == Nokogiri::XML::Node::ELEMENT_NODE
+
+          attrs = {}
+          node.attribute_nodes.each do |attr|
+            attrs[attr.name] = attr.value
+          end
+          attrs
+        end
+
+        # Compact whitespace
+        #
+        # Collapses internal whitespace without trimming.
+        #
+        # @param text [String] Text to compact
+        # @return [String] Compacted text
+        def compact_whitespace(text)
+          text.gsub(/[\p{Space}\u00a0]+/, " ")
+        end
+
+        # Normalize text
+        #
+        # Collapses and trims whitespace.
+        #
+        # @param text [String] Text to normalize
+        # @return [String] Normalized text
+        def normalize_text(text)
+          MatchOptions.normalize_text(text)
+        end
+      end
+    end
+  end
+end

data/lib/canon/comparison/dimensions/base_dimension.rb

@@ -0,0 +1,107 @@
+# frozen_string_literal: true
+
+module Canon
+  module Comparison
+    module Dimensions
+      # Base class for comparison dimensions
+      #
+      # A dimension represents "WHAT to compare" - a specific aspect of a document
+      # that can be compared (e.g., text content, attributes, comments).
+      #
+      # Each dimension knows how to:
+      # - Extract relevant data from a node
+      # - Compare data according to a behavior (:strict, :normalize, :ignore)
+      #
+      # Subclasses must implement:
+      # - extract_data(node) - Extract relevant data from a node
+      # - compare_strict(data1, data2) - Strict comparison
+      # - compare_normalize(data1, data2) - Normalized comparison (optional)
+      #
+      # @abstract Subclass and implement abstract methods
+      class BaseDimension
+        # Behavior constants
+        STRICT = :strict
+        NORMALIZE = :normalize
+        IGNORE = :ignore
+
+        # Get the dimension name
+        #
+        # @return [Symbol] Dimension name
+        def dimension_name
+          @dimension_name ||= self.class.name.split("::").last.gsub(
+            /Dimension$/, ""
+          ).downcase.to_sym
+        end
+
+        # Compare extracted data according to behavior
+        #
+        # @param data1 [Object] First data
+        # @param data2 [Object] Second data
+        # @param behavior [Symbol] Comparison behavior (:strict, :normalize, :ignore)
+        # @return [Boolean] true if data matches according to behavior
+        def compare(data1, data2, behavior)
+          case behavior
+          when STRICT
+            compare_strict(data1, data2)
+          when NORMALIZE
+            compare_normalize(data1, data2)
+          when IGNORE
+            true
+          else
+            raise Error, "Unknown behavior: #{behavior}"
+          end
+        end
+
+        # Check if two nodes are equivalent for this dimension
+        #
+        # @param node1 [Object] First node
+        # @param node2 [Object] Second node
+        # @param behavior [Symbol] Comparison behavior
+        # @return [Boolean] true if nodes match for this dimension
+        def equivalent?(node1, node2, behavior)
+          data1 = extract_data(node1)
+          data2 = extract_data(node2)
+          compare(data1, data2, behavior)
+        end
+
+        # Extract data from a node
+        #
+        # @param node [Object] Node to extract data from
+        # @return [Object] Extracted data
+        # @abstract Subclass must implement
+        def extract_data(node)
+          raise NotImplementedError, "#{self.class} must implement extract_data"
+        end
+
+        # Strict comparison
+        #
+        # @param data1 [Object] First data
+        # @param data2 [Object] Second data
+        # @return [Boolean] true if data matches strictly
+        # @abstract Subclass must implement
+        def compare_strict(data1, data2)
+          raise NotImplementedError,
+                "#{self.class} must implement compare_strict"
+        end
+
+        # Normalized comparison
+        #
+        # @param data1 [Object] First data
+        # @param data2 [Object] Second data
+        # @return [Boolean] true if data matches after normalization
+        def compare_normalize(data1, data2)
+          # Default implementation: delegate to strict comparison
+          compare_strict(data1, data2)
+        end
+
+        # Check if this dimension supports normalization
+        #
+        # @return [Boolean] true if normalization is supported
+        def supports_normalization?
+          # Check if compare_normalize is overridden (not the default implementation)
+          method(:compare_normalize).owner != BaseDimension
+        end
+      end
+    end
+  end
+end