canon 0.1.8 → 0.1.9

data/docs/features/match-options/index.adoc

@@ -45,10 +45,37 @@ Match dimensions are orthogonal aspects that can be configured independently.
 
 `:strict`:: Text must match exactly, character-for-character including all whitespace
 
-`:normalize`:: Whitespace is normalized (collapsed/trimmed) before comparison
+`:normalize`:: Whitespace is normalized (collapsed/trimmed) before comparison.
+Formatting-only differences (e.g., extra spaces around text) are classified as
+*informative* rather than normative. This means documents with only whitespace
+differences in text content are considered equivalent.
 
 `:ignore`:: Text content is completely ignored in comparison
 
+.Using text_content: :normalize
+[example]
+====
+[source,ruby]
+----
+# These are equivalent with :normalize
+# Whitespace differences are formatting-only (informative)
+Canon.equivalent?(
+  '<p>  text  </p>',
+  '<p>text</p>',
+  match: { text_content: :normalize }
+)
+# => true
+
+# These differ in :strict mode
+Canon.equivalent?(
+  '<p>  text  </p>',
+  '<p>text</p>',
+  match: { text_content: :strict }
+)
+# => false
+----
+====
+
 === structural_whitespace
 
 **Applies to**: All formats
@@ -63,6 +90,200 @@ Match dimensions are orthogonal aspects that can be configured independently.
 
 `:ignore`:: Structural whitespace is completely ignored
 
+
+=== Whitespace sensitivity at element level
+
+==== General
+
+In XML, whitespace sensitivity can vary by schema and element:
+
+* Elements that apply `xml:space="preserve"` are whitespace-sensitive.
+
+* Other elements may be defined as sensitive by schema (e.g.
+`xs:space="preserve"` in XML Schema) or unannounced conventions, such as
+for mixed content.
+
+In HTML, elements like `<pre>` and `<code>` preserve whitespace, while others
+like `<div>` and `<p>` do not.
+
+In the unannounced cases, the developer must indicate which elements are
+whitespace-sensitive.
+
+In Canon, you can control whitespace sensitivity at the element level using
+`structural_whitespace: :strict` or `text_content: :normalize`.
+
+Element-level sensitivity controls both:
+
+* `structural_whitespace`: Whether whitespace between elements in the element is
+preserved
+
+* `text_content`: Whether whitespace within text nodes of the element is
+normalized
+
+Options for controlling element-level sensitivity include:
+
+* **xml:space attribute** - XML standard for declaring whitespace sensitivity in documents
+* **whitelist/blacklist options** - User-specified element lists
+* **Format defaults** - HTML has built-in sensitive elements
+* **respect_xml_space option** - Control whether xml:space is honored
+
+For elements marked as sensitive, whitespace differences are always normative.
+
+For non-sensitive elements using `text_content: :normalize`, whitespace
+differences are classified as formatting-only (informative).
+
+
+==== xml:space attribute support
+
+The `xml:space` attribute is the XML standard way to declare whitespace
+sensitivity in XML instance documents:
+
+[source,xml]
+----
+<!-- Preserve whitespace in this element -->
+<code xml:space="preserve">
+  Indentation and newlines matter here
+</code>
+
+<!-- Use default behavior -->
+<text xml:space="default">
+  Whitespace handling follows configured behavior
+</text>
+----
+
+==== Whitelist and blacklist options
+
+You can explicitly specify which elements are whitespace-sensitive:
+
+[source,ruby]
+----
+# Specify elements that preserve whitespace
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: {
+    structural_whitespace: :strict,
+    whitespace_sensitive_elements: [:pre, :code, :sample],
+    whitespace_insensitive_elements: [:p, :div]  # Override defaults/whitelist
+  }
+)
+----
+
+==== respect_xml_space option
+
+Control whether xml:space attributes in the document are honored:
+
+[source,ruby]
+----
+# Honor xml:space (default)
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: {
+    structural_whitespace: :strict,
+    respect_xml_space: true  # Use xml:space attributes in document
+  }
+)
+
+# Ignore xml:space, use only user configuration
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: {
+    structural_whitespace: :strict,
+    respect_xml_space: false  # Override document declarations
+  }
+)
+----
+
+==== Priority order
+
+When determining if an element is whitespace-sensitive, Canon uses this priority:
+
+[source]
+----
+1. respect_xml_space: false → User config only (ignore xml:space)
+   ↓
+2. User whitelist → Use whitelist (user explicitly declared)
+   ↓
+3. Format defaults → HTML: [:pre, :textarea, :script, :style], XML: []
+   ↓
+4. User blacklist → Remove from defaults/whitelist
+   ↓
+5. xml:space="preserve" → Element is sensitive
+   ↓
+6. xml:space="default" → Use steps 1-4
+----
+
+==== Format-specific defaults
+
+**HTML**:: `[:pre, :textarea, :script, :style]` - These elements preserve whitespace by HTML specification
+**XML**:: `[]` - No default whitespace-sensitive elements, purely user-controlled
+
+==== Examples
+
+.Using xml:space attribute
+[source,ruby]
+----
+xml1 = '<root><code xml:space="preserve">  indented  </code></root>'
+xml2 = '<root><code xml:space="preserve">indented</code></root>'
+
+# These are NOT equivalent (whitespace matters in xml:space="preserve")
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: { structural_whitespace: :strict }
+)
+# => false
+----
+
+.Using whitelist
+[source,ruby]
+----
+# Make <p> elements whitespace-sensitive
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: {
+    structural_whitespace: :strict,
+    whitespace_sensitive_elements: [:p, :pre]
+  }
+)
+----
+
+.Overriding HTML defaults
+[source,ruby]
+----
+# Make <script> NOT whitespace-sensitive (override HTML default)
+Canon::Comparison.equivalent?(html1, html2,
+  format: :html,
+  match: {
+    structural_whitespace: :strict,
+    whitespace_insensitive_elements: [:script]
+  }
+)
+----
+
+.Using text_content: :normalize with whitespace_insensitive_elements
+[source,ruby]
+----
+# HTML defaults: [:pre, :code, :textarea, :script, :style]
+# Excluding :code means it's no longer whitespace-sensitive
+html1 = '<root><pre>  indented  </pre><code>  code  </code></root>'
+html2 = '<root><pre>  indented  </pre><code>code</code></root>'
+
+# With :code blacklisted, whitespace in <code> is normalized (formatting-only)
+# HTML uses text_content: :normalize by default
+Canon::Comparison.equivalent?(html1, html2,
+  format: :html,
+  match: {
+    whitespace_insensitive_elements: [:code],
+  }
+)
+# => true (whitespace differences in <code> are formatting-only)
+
+# Without blacklisting, <code> is sensitive (whitespace matters)
+Canon::Comparison.equivalent?(html1, html2,
+  format: :html,
+  match: {
+    structural_whitespace: :strict,
+  }
+)
+# => false (whitespace in <code> is normative)
+----
+
+
+
 === attribute_whitespace
 
 **Applies to**: XML, HTML only
@@ -414,6 +635,23 @@ expect(actual).to be_xml_equivalent_to(expected,
     element_position: :ignore,
     element_hierarchy: :ignore
   )
+
+# Element-level whitespace sensitivity
+expect(actual).to be_xml_equivalent_to(expected,
+  match: { structural_whitespace: :strict }
+)
+  .with_options(
+    whitespace_sensitive_elements: [:pre, :code, :sample],
+    respect_xml_space: true
+  )
+
+# Override HTML default whitespace-sensitive elements
+expect(html).to be_html_equivalent_to(expected,
+  match: { structural_whitespace: :strict }
+)
+  .with_options(
+    whitespace_insensitive_elements: [:script, :style]
+  )
 ====
 
 == Comments dimension

data/lib/canon/comparison/format_detector.rb

@@ -72,7 +72,8 @@ module Canon
           return :json if trimmed.start_with?("{", "[")
 
           # HTML indicators
-          return :html if trimmed.start_with?("<!DOCTYPE html", "<html", "<HTML")
+          return :html if trimmed.start_with?("<!DOCTYPE html", "<html",
+                                              "<HTML")
 
           # XML indicators - must start with < and end with >
           return :xml if trimmed.start_with?("<") && trimmed.end_with?(">")

data/lib/canon/comparison/html_comparator.rb

@@ -13,6 +13,8 @@ require_relative "../diff/diff_classifier"
 require_relative "strategies/match_strategy_factory"
 require_relative "../html/data_model"
 require_relative "xml_node_comparison"
+# Whitespace sensitivity module (single source of truth for sensitive elements)
+require_relative "whitespace_sensitivity"
 
 module Canon
   module Comparison
@@ -542,16 +544,22 @@ compare_profile = nil)
           return if match_opts[:text_content] == :strict
 
           # Elements where whitespace is significant - don't normalize
-          # Use profile if available, otherwise use default list
+          # SINGLE SOURCE OF TRUTH: WhitespaceSensitivity.format_default_sensitive_elements
+          # This ensures consistency between preprocessing and comparison logic
+          # SINGLE SOURCE OF TRUTH: WhitespaceSensitivity.format_default_sensitive_elements
+          # This ensures consistency between preprocessing and comparison logic
           preserve_whitespace = if compare_profile.is_a?(HtmlCompareProfile)
                                   # Profile handles HTML-specific whitespace rules
-                                  %w[pre code textarea script
-                                     style].select do |elem|
-                                    compare_profile.preserve_whitespace?(elem)
-                                  end
+                                  # Get default list and filter by profile
+                                  WhitespaceSensitivity
+                                    .format_default_sensitive_elements(match_opts)
+                                    .select do |elem|
+                                      compare_profile.preserve_whitespace?(elem.to_s)
+                                    end
+                                    .map(&:to_s)
                                 else
-                                  # Fallback to default list
-                                  %w[pre code textarea script style]
+                                  # Use default list from WhitespaceSensitivity (single source of truth)
+                                  WhitespaceSensitivity.format_default_sensitive_elements(match_opts).map(&:to_s)
                                 end
 
           # Walk all text nodes
@@ -607,9 +615,12 @@ compare_profile = nil)
         #
         # CRITICAL: Do NOT remove whitespace-only text nodes from whitespace-sensitive
         # elements like <pre>, <code>, <textarea>, <script>, <style>
+        #
+        # SINGLE SOURCE OF TRUTH: Uses WhitespaceSensitivity.format_default_sensitive_elements
         def remove_whitespace_only_text_nodes(doc)
           # Elements where whitespace is significant - don't remove whitespace-only nodes
-          preserve_whitespace = %w[pre code textarea script style]
+          # SINGLE SOURCE OF TRUTH: WhitespaceSensitivity.format_default_sensitive_elements
+          preserve_whitespace = WhitespaceSensitivity.format_default_sensitive_elements(format: :html).map(&:to_s)
 
           doc.xpath(".//text()").each do |text_node|
             # CRITICAL: Skip if this text node is inside a whitespace-preserving element

data/lib/canon/comparison/html_compare_profile.rb

@@ -1,6 +1,8 @@
 # frozen_string_literal: true
 
 require_relative "compare_profile"
+# Whitespace sensitivity module (single source of truth for sensitive elements)
+require_relative "whitespace_sensitivity"
 
 module Canon
   module Comparison
@@ -82,9 +84,13 @@ module Canon
       private
 
       # Elements where whitespace is semantically significant in HTML
-      # @return [Array<String>] List of element names
+      #
+      # SINGLE SOURCE OF TRUTH: Delegates to WhitespaceSensitivity.format_default_sensitive_elements
+      # This ensures consistency across the codebase.
+      #
+      # @return [Array<String>] List of element names (as strings)
       def whitespace_sensitive_elements
-        %w[pre code textarea script style]
+        WhitespaceSensitivity.format_default_sensitive_elements(format: @html_version).map(&:to_s)
       end
 
       # Check if a dimension is explicitly set to :strict

data/lib/canon/comparison/match_options/base_resolver.rb

@@ -27,6 +27,9 @@ module Canon
             # Start with format-specific defaults
             options = format_defaults(format).dup
 
+            # Store format for later use (e.g., WhitespaceSensitivity needs it)
+            options[:format] = format
+
             # Apply global profile if specified
             if global_profile
               profile_opts = get_profile_options(global_profile)
@@ -111,12 +114,16 @@ module Canon
           def validate_match_options!(match_options)
             # Special options that don't need validation as dimensions
             special_options = %i[
+              format
               preprocessing
               semantic_diff
               similarity_threshold
               hash_matching
               similarity_matching
               propagation
+              whitespace_sensitive_elements
+              whitespace_insensitive_elements
+              respect_xml_space
             ]
 
             match_options.each do |dimension, behavior|

data/lib/canon/comparison/whitespace_sensitivity.rb

@@ -0,0 +1,208 @@
+# frozen_string_literal: true
+
+module Canon
+  module Comparison
+    # Whitespace sensitivity utilities for element-level control
+    #
+    # This module provides logic to determine whether whitespace should be
+    # preserved during comparison based on:
+    # - Format-specific defaults (HTML has built-in sensitive elements)
+    # - User-configured whitelist (elements that care about whitespace)
+    # - User-configured blacklist (elements that don't care about whitespace)
+    # - xml:space attribute in the document itself
+    # - respect_xml_space flag (whether to honor or override xml:space)
+    #
+    # == Priority Order
+    #
+    # 1. respect_xml_space: false → User config only (ignore xml:space)
+    # 2. User whitelist → Use whitelist (user explicitly declared)
+    # 3. Format defaults → HTML: [:pre, :textarea, :script, :style], XML: []
+    # 4. User blacklist → Remove from defaults/whitelist
+    # 5. xml:space="preserve" → Element is sensitive
+    # 6. xml:space="default" → Use steps 1-4
+    #
+    # == Usage
+    #
+    #   WhitespaceSensitivity.element_sensitive?(node, opts)
+    #   => true if whitespace should be preserved for this element
+    module WhitespaceSensitivity
+      class << self
+        # Check if an element is whitespace-sensitive based on configuration
+        #
+        # @param node [Object] The element node to check
+        # @param opts [Hash] Comparison options containing match_opts
+        # @return [Boolean] true if whitespace should be preserved for this element
+        def element_sensitive?(node, opts)
+          match_opts = opts[:match_opts]
+          return false unless match_opts
+          return false unless text_node_parent?(node)
+
+          parent = node.parent
+
+          # 1. Check if we should ignore xml:space (user override)
+          if !respect_xml_space?(match_opts)
+            return user_config_sensitive?(parent, match_opts)
+          end
+
+          # 2. Check xml:space="preserve" (document declaration)
+          return true if xml_space_preserve?(parent)
+
+          # 3. Check xml:space="default" (use configured behavior)
+          return false if xml_space_default?(parent)
+
+          # 4. Use user configuration + format defaults
+          configured_sensitive?(parent, match_opts)
+        end
+
+        # Check if whitespace-only text node should be filtered
+        #
+        # @param node [Object] The text node to check
+        # @param opts [Hash] Comparison options
+        # @return [Boolean] true if node should be preserved (not filtered)
+        def preserve_whitespace_node?(node, opts)
+          return false unless node.respond_to?(:parent)
+          return false unless node.parent
+
+          element_sensitive?(node, opts)
+        end
+
+        # Get format-specific default sensitive elements
+        #
+        # This is the SINGLE SOURCE OF TRUTH for default whitespace-sensitive
+        # elements. All other code should use this method to get the list.
+        #
+        # @param match_opts [Hash] Resolved match options
+        # @return [Array<Symbol>] Default sensitive element names
+        def format_default_sensitive_elements(match_opts)
+          format = match_opts[:format] || :xml
+
+          case format
+          when :html, :html4, :html5
+            # HTML specification: these elements preserve whitespace
+            %i[pre code textarea script style].freeze
+          when :xml
+            # XML has no default sensitive elements - purely user-controlled
+            [].freeze
+          else
+            [].freeze
+          end
+        end
+
+        # Check if an element is in the default sensitive list for its format
+        #
+        # Convenience method for checking element sensitivity without building
+        # the full list first.
+        #
+        # @param element_name [String, Symbol] The element name to check
+        # @param match_opts [Hash] Resolved match options
+        # @return [Boolean] true if element is in default sensitive list
+        def default_sensitive_element?(element_name, match_opts)
+          format_default_sensitive_elements(match_opts)
+            .include?(element_name.to_sym)
+        end
+
+        private
+
+        # Check if we should respect xml:space attribute
+        #
+        # @param match_opts [Hash] Resolved match options
+        # @return [Boolean] true if xml:space should be respected
+        def respect_xml_space?(match_opts)
+          if match_opts.key?(:respect_xml_space)
+            match_opts[:respect_xml_space]
+          else
+            true
+          end
+        end
+
+        # Check if xml:space="preserve" is set
+        #
+        # @param element [Object] The element to check
+        # @return [Boolean] true if xml:space="preserve"
+        def xml_space_preserve?(element)
+          if element.is_a?(Canon::Xml::Nodes::ElementNode)
+            # Check attribute_nodes for xml:space attribute
+            # xml:space is stored with name="space" and namespace_uri="http://www.w3.org/XML/1998/namespace"
+            element.attribute_nodes.any? do |attr|
+              attr.name == "space" &&
+                attr.namespace_uri == "http://www.w3.org/XML/1998/namespace" &&
+                attr.value == "preserve"
+            end
+          elsif element.respond_to?(:[])
+            element["xml:space"] == "preserve"
+          else
+            false
+          end
+        end
+
+        # Check if xml:space="default" is set
+        #
+        # @param element [Object] The element to check
+        # @return [Boolean] true if xml:space="default"
+        def xml_space_default?(element)
+          if element.is_a?(Canon::Xml::Nodes::ElementNode)
+            # Check attribute_nodes for xml:space attribute
+            # xml:space is stored with name="space" and namespace_uri="http://www.w3.org/XML/1998/namespace"
+            element.attribute_nodes.any? do |attr|
+              attr.name == "space" &&
+                attr.namespace_uri == "http://www.w3.org/XML/1998/namespace" &&
+                attr.value == "default"
+            end
+          elsif element.respond_to?(:[])
+            element["xml:space"] == "default"
+          else
+            false
+          end
+        end
+
+        # Check sensitivity based on user configuration
+        #
+        # @param element [Object] The element to check
+        # @param match_opts [Hash] Resolved match options
+        # @return [Boolean] true if element is in whitelist
+        def user_config_sensitive?(element, match_opts)
+          return false unless match_opts[:whitespace_sensitive_elements]
+
+          match_opts[:whitespace_sensitive_elements].include?(element.name.to_sym)
+        end
+
+        # Check sensitivity based on user config + format defaults
+        #
+        # @param element [Object] The element to check
+        # @param match_opts [Hash] Resolved match options
+        # @return [Boolean] true if element should be sensitive
+        def configured_sensitive?(element, match_opts)
+          # Start with format defaults
+          sensitive = format_default_sensitive_elements(match_opts).to_set
+
+          # Apply whitelist (adds to defaults)
+          if match_opts[:whitespace_sensitive_elements]
+            sensitive |= match_opts[:whitespace_sensitive_elements]
+          end
+
+          # Apply blacklist (removes from everything)
+          if match_opts[:whitespace_insensitive_elements]
+            sensitive -= match_opts[:whitespace_insensitive_elements]
+          end
+
+          sensitive.include?(element.name.to_sym)
+        end
+
+        # Check if node has a parent that's an element (not document root)
+        #
+        # @param node [Object] The node to check
+        # @return [Boolean] true if node has an element parent
+        def text_node_parent?(node)
+          return false unless node.respond_to?(:parent)
+          return false unless node.parent
+
+          parent = node.parent
+          return true if parent.respond_to?(:element?) && parent.element?
+
+          # Nokogiri compatibility
+          parent.respond_to?(:node_type) && parent.node_type == :element
+        end
+      end
+    end
+  end
+end

data/lib/canon/comparison/xml_comparator/child_comparison.rb

@@ -26,7 +26,8 @@ module Canon
           # @param diff_children [Boolean] Whether to diff children
           # @param differences [Array] Array to collect differences
           # @return [Integer] Comparison result code
-          def compare(node1, node2, comparator, opts, child_opts, diff_children, differences)
+          def compare(node1, node2, comparator, opts, child_opts,
+diff_children, differences)
             children1 = comparator.send(:filter_children, node1.children, opts)
             children2 = comparator.send(:filter_children, node2.children, opts)
 
@@ -51,7 +52,9 @@ module Canon
           # method that returns symbols, and only works with element nodes.
           def can_use_element_matcher?(children1, children2)
             !children1.empty? && !children2.empty? &&
-              children1.all? { |c| c.is_a?(Canon::Xml::Node) && c.node_type == :element } &&
+              children1.all? do |c|
+                c.is_a?(Canon::Xml::Node) && c.node_type == :element
+              end &&
               children2.all? { |c| c.is_a?(Canon::Xml::Node) && c.node_type == :element }
           end
 
@@ -140,7 +143,8 @@ module Canon
                                         opts, child_opts, diff_children, differences)
             # Length check
             unless children1.length == children2.length
-              dimension = determine_dimension_for_mismatch(children1, children2, comparator)
+              dimension = determine_dimension_for_mismatch(children1,
+                                                           children2, comparator)
               comparator.send(:add_difference, parent_node, parent_node,
                               Comparison::MISSING_NODE, Comparison::MISSING_NODE,
                               dimension, opts, differences)
@@ -167,15 +171,19 @@ module Canon
             (0...max_len).each do |i|
               if i >= children1.length
                 # Extra child in children2
-                dimension = comparator.send(:determine_node_dimension, children2[i])
+                dimension = comparator.send(:determine_node_dimension,
+                                            children2[i])
                 break
               elsif i >= children2.length
                 # Extra child in children1
-                dimension = comparator.send(:determine_node_dimension, children1[i])
+                dimension = comparator.send(:determine_node_dimension,
+                                            children1[i])
                 break
-              elsif !comparator.send(:same_node_type?, children1[i], children2[i])
+              elsif !comparator.send(:same_node_type?, children1[i],
+                                     children2[i])
                 # Different node types at same position
-                dimension = comparator.send(:determine_node_dimension, children1[i])
+                dimension = comparator.send(:determine_node_dimension,
+                                            children1[i])
                 break
               end
             end

data/lib/canon/comparison/xml_comparator/node_parser.rb

@@ -13,21 +13,24 @@ module Canon
         #
         # @param node [String, Object] Node to parse
         # @param preprocessing [Symbol] Preprocessing mode (:none, :normalize, :c14n, :format)
+        # @param preserve_whitespace [Boolean] Whether to preserve whitespace-only text nodes
         # @return [Canon::Xml::Node] Parsed node
-        def self.parse(node, preprocessing = :none)
+        def self.parse(node, preprocessing = :none, preserve_whitespace: false)
           # If already a Canon::Xml::Node, return as-is
           return node if node.is_a?(Canon::Xml::Node)
 
           # If it's a Nokogiri or Moxml node, convert to DataModel
           unless node.is_a?(String)
-            return convert_from_node(node)
+            return convert_from_node(node,
+                                     preserve_whitespace: preserve_whitespace)
           end
 
           # Apply preprocessing to XML string before parsing
           xml_string = apply_preprocessing(node, preprocessing)
 
           # Use Canon::Xml::DataModel for parsing to get Canon::Xml::Node instances
-          Canon::Xml::DataModel.from_xml(xml_string)
+          Canon::Xml::DataModel.from_xml(xml_string,
+                                         preserve_whitespace: preserve_whitespace)
         end
 
         # Apply preprocessing transformation to XML string
@@ -55,8 +58,9 @@ module Canon
         # Convert from Nokogiri/Moxml node to Canon::Xml::Node
         #
         # @param node [Object] Nokogiri or Moxml node
+        # @param preserve_whitespace [Boolean] Whether to preserve whitespace-only text nodes
         # @return [Canon::Xml::Node] Converted node
-        def self.convert_from_node(node)
+        def self.convert_from_node(node, preserve_whitespace: false)
           # Convert to XML string then parse through DataModel
           xml_str = if node.respond_to?(:to_xml)
                       node.to_xml
@@ -66,7 +70,8 @@ module Canon
                       raise Canon::Error,
                             "Unable to convert node to string: #{node.class}"
                     end
-          Canon::Xml::DataModel.from_xml(xml_str)
+          Canon::Xml::DataModel.from_xml(xml_str,
+                                         preserve_whitespace: preserve_whitespace)
         end
       end
     end