canon 0.1.7 → 0.1.9

data/docs/advanced/diff-classification.adoc

@@ -229,6 +229,69 @@ result = Canon::Comparison.equivalent?(
 ----
 ====
 
+==== Text Content
+
+* **`:strict` behavior** → Normative
+  - Text must match exactly, including all whitespace
+  - Any text difference causes non-equivalence
+
+* **`:normalize` behavior** → Normative (after normalization) or Informative (if formatting-only)
+  - Whitespace is normalized (collapsed/trimmed) before comparison
+  - If normalized texts match but originals differ, classified as formatting-only (informative)
+  - This ensures that whitespace-only differences don't affect equivalence
+  - Element-level sensitivity is respected (e.g., `<pre>`, `<code>` preserve whitespace)
+
+* **`:ignore` behavior** → Informative
+  - Text content differences tracked but don't affect equivalence
+
+.Example: Text content with normalize behavior
+====
+[source,ruby]
+----
+# Formatting-only difference - normalized texts match
+xml1 = '<p>Hello  world</p>'
+xml2 = '<p>Hello world</p>'
+
+result = Canon::Comparison.equivalent?(
+  xml1, xml2,
+  match: { text_content: :normalize }
+)
+# => true (extra space is formatting-only, classified as informative)
+
+# Shows as informative in verbose output
+result.differences.first.normative?
+# => false
+result.differences.first.formatting?
+# => true
+----
+
+.Using text_content: :normalize with element-level sensitivity
+====
+[source,ruby]
+----
+# HTML defaults: <code> is whitespace-sensitive
+html1 = '<code>  indented  </code><p>  text  </p>'
+html2 = '<code>indented</code><p>text</p>'
+
+# With <code> blacklisted from sensitive elements
+Canon::Comparison.equivalent?(html1, html2,
+  format: :html,
+  match: {
+    whitespace_insensitive_elements: [:code],
+  }
+)
+# => true
+# - <code> whitespace: formatting-only (informative)
+# - <p> whitespace: formatting-only (informative)
+
+# Without blacklisting (default HTML behavior)
+Canon::Comparison.equivalent?(html1, html2, format: :html)
+# => false
+# - <code> whitespace: normative (sensitive element)
+# - <p> whitespace: formatting-only (informative)
+----
+====
+
 === FormattingDetector Integration
 
 For dimensions that support it (`:text_content`, `:structural_whitespace`),
@@ -262,12 +325,23 @@ The [`CompareProfile`](../../lib/canon/comparison/compare_profile.rb) class prov
 * `affects_equivalence?(dimension)` - Does this dimension affect equivalence?
 * `supports_formatting_detection?(dimension)` - Can this dimension have formatting-only diffs?
 
-The [`DiffClassifier`](../../lib/canon/diff/diff_classifier.rb) uses CompareProfile to classify:
+The [`DiffClassifier`](../../lib/canon/diff/diff_classifier.rb) uses CompareProfile to classify differences, with special handling for `text_content: :normalize`:
 
 [source,ruby]
 ----
 def classify(diff_node)
-  # Check normative status based on policy
+  # SPECIAL CASE: text_content with :normalize behavior
+  # Formatting-only differences (whitespace-only) are marked as non-normative
+  if diff_node.dimension == :text_content &&
+      profile.send(:behavior_for, :text_content) == :normalize &&
+      !inside_whitespace_sensitive_element?(diff_node) &&
+      formatting_only_diff?(diff_node)
+    diff_node.formatting = true
+    diff_node.normative = false
+    return diff_node
+  end
+
+  # Standard classification flow
   is_normative = profile.normative_dimension?(diff_node.dimension)
 
   # Only check formatting for non-normative dimensions
@@ -284,6 +358,12 @@ def classify(diff_node)
 end
 ----
 
+The key distinction for `text_content: :normalize`:
+
+* **Formatting-only detection**: Uses `normalized_equivalent?` method to compare normalized texts
+* **Element sensitivity**: Respects element-level whitespace sensitivity (`<pre>`, `<code>`, etc.)
+* **Result**: Whitespace-only differences are classified as *informative* (non-normative) when using `:normalize`
+
 == Visual Indicators
 
 === Normative Diffs

data/docs/advanced/extending-canon.adoc

@@ -0,0 +1,193 @@
+---
+title: Extending Canon
+parent: Advanced
+nav_order: 8
+---
+= Extending Canon
+
+== Purpose
+
+This document explains how to extend Canon with custom functionality, including creating custom comparators, formatters, and adapters for different document formats.
+
+== Overview
+
+Canon is designed to be extensible at multiple layers:
+
+* **Layer 1**: Custom preprocessing/normalization
+* **Layer 2**: Custom comparison algorithms
+* **Layer 3**: Custom match options and dimensions
+* **Layer 4**: Custom diff formatters and renderers
+
+== Adapter Pattern
+
+Canon uses an adapter pattern to work with different parsing libraries (Nokogiri, Moxml, etc.).
+
+=== Adapter Structure
+
+[source,ruby]
+----
+module Canon
+  module Adapters
+    class NokogiriAdapter
+      def parse(input)
+        # Parse with Nokogiri
+        Nokogiri::XML(input)
+      end
+
+      def serialize(node)
+        # Serialize with Nokogiri
+        node.to_xml
+      end
+    end
+  end
+end
+----
+
+=== Creating a Custom Adapter
+
+To add support for a new parsing library:
+
+1. Create an adapter class that implements `parse` and `serialize` methods
+2. Register the adapter with the format system
+3. Add tests for the new adapter
+
+== Custom Comparators
+
+=== Creating a Custom Comparison Algorithm
+
+[source,ruby]
+----
+module Canon
+  module Comparison
+    class CustomComparator < BaseComparator
+      def compare(node1, node2, opts)
+        # Your comparison logic here
+        differences = []
+
+        # Your algorithm implementation
+
+        differences
+      end
+    end
+  end
+end
+----
+
+=== Registering Your Algorithm
+
+[source,ruby]
+----
+Canon::Comparison.register_algorithm(:custom, CustomComparator)
+----
+
+Then use it:
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2, diff_algorithm: :custom)
+----
+
+== Custom Formatters
+
+=== Creating a Custom Diff Formatter
+
+[source,ruby]
+----
+module Canon
+  class DiffFormatter
+    class CustomFormatter
+      def format(differences, opts)
+        # Your formatting logic here
+        formatted_output = ""
+
+        differences.each do |diff|
+          formatted_output += format_difference(diff, opts)
+        end
+
+        formatted_output
+      end
+    end
+  end
+end
+----
+
+=== Using Your Formatter
+
+[source,ruby]
+----
+result = Canon::Comparison.equivalent?(doc1, doc2, verbose: true)
+
+formatter = Canon::DiffFormatter::CustomFormatter.new
+output = formatter.format(result.differences, use_color: true)
+puts output
+----
+
+== Custom Match Options
+
+=== Defining Custom Dimensions
+
+[source,ruby]
+----
+module Canon
+  module Comparison
+    class CustomDimension
+      def self.key
+        :custom_dimension
+      end
+
+      def self.compare(node1, node2, behavior, opts)
+        # Your comparison logic for this dimension
+        case behavior
+        when :strict
+          node1 == node2
+        when :normalize
+          normalize(node1) == normalize(node2)
+        when :ignore
+          true
+        end
+      end
+    end
+  end
+end
+----
+
+Register your dimension:
+
+[source,ruby]
+----
+Canon::Comparison.register_dimension(CustomDimension)
+----
+
+== Best Practices
+
+=== Testing Your Extensions
+
+1. Write comprehensive tests for your extensions
+2. Use the existing test helpers and fixtures
+3. Test edge cases and error conditions
+
+=== Performance Considerations
+
+1. Cache expensive computations
+2. Use lazy evaluation where appropriate
+3. Avoid unnecessary node cloning
+
+=== Error Handling
+
+1. Provide clear error messages
+2. Use Canon's error classes consistently
+3. Document error conditions
+
+== Examples
+
+See the source code for examples of:
+
+* link:xml-comparator[DOM Comparator implementation]
+* link:semantic-tree-diff-internals[Semantic Diff implementation]
+* link:diff-formatting/[Diff Formatter implementations]
+
+== See Also
+
+* link:../understanding/architecture.adoc[Architecture] - 4-layer architecture overview
+* link:../features/diff-formatting/[Diff Formatting] - Layer 4 rendering options
+* link:diff-pipeline[Comparison Pipeline] - Technical pipeline details

data/docs/features/match-options/index.adoc

@@ -45,10 +45,37 @@ Match dimensions are orthogonal aspects that can be configured independently.
 
 `:strict`:: Text must match exactly, character-for-character including all whitespace
 
-`:normalize`:: Whitespace is normalized (collapsed/trimmed) before comparison
+`:normalize`:: Whitespace is normalized (collapsed/trimmed) before comparison.
+Formatting-only differences (e.g., extra spaces around text) are classified as
+*informative* rather than normative. This means documents with only whitespace
+differences in text content are considered equivalent.
 
 `:ignore`:: Text content is completely ignored in comparison
 
+.Using text_content: :normalize
+[example]
+====
+[source,ruby]
+----
+# These are equivalent with :normalize
+# Whitespace differences are formatting-only (informative)
+Canon.equivalent?(
+  '<p>  text  </p>',
+  '<p>text</p>',
+  match: { text_content: :normalize }
+)
+# => true
+
+# These differ in :strict mode
+Canon.equivalent?(
+  '<p>  text  </p>',
+  '<p>text</p>',
+  match: { text_content: :strict }
+)
+# => false
+----
+====
+
 === structural_whitespace
 
 **Applies to**: All formats
@@ -63,6 +90,200 @@ Match dimensions are orthogonal aspects that can be configured independently.
 
 `:ignore`:: Structural whitespace is completely ignored
 
+
+=== Whitespace sensitivity at element level
+
+==== General
+
+In XML, whitespace sensitivity can vary by schema and element:
+
+* Elements that apply `xml:space="preserve"` are whitespace-sensitive.
+
+* Other elements may be defined as sensitive by schema (e.g.
+`xs:space="preserve"` in XML Schema) or unannounced conventions, such as
+for mixed content.
+
+In HTML, elements like `<pre>` and `<code>` preserve whitespace, while others
+like `<div>` and `<p>` do not.
+
+In the unannounced cases, the developer must indicate which elements are
+whitespace-sensitive.
+
+In Canon, you can control whitespace sensitivity at the element level using
+`structural_whitespace: :strict` or `text_content: :normalize`.
+
+Element-level sensitivity controls both:
+
+* `structural_whitespace`: Whether whitespace between elements in the element is
+preserved
+
+* `text_content`: Whether whitespace within text nodes of the element is
+normalized
+
+Options for controlling element-level sensitivity include:
+
+* **xml:space attribute** - XML standard for declaring whitespace sensitivity in documents
+* **whitelist/blacklist options** - User-specified element lists
+* **Format defaults** - HTML has built-in sensitive elements
+* **respect_xml_space option** - Control whether xml:space is honored
+
+For elements marked as sensitive, whitespace differences are always normative.
+
+For non-sensitive elements using `text_content: :normalize`, whitespace
+differences are classified as formatting-only (informative).
+
+
+==== xml:space attribute support
+
+The `xml:space` attribute is the XML standard way to declare whitespace
+sensitivity in XML instance documents:
+
+[source,xml]
+----
+<!-- Preserve whitespace in this element -->
+<code xml:space="preserve">
+  Indentation and newlines matter here
+</code>
+
+<!-- Use default behavior -->
+<text xml:space="default">
+  Whitespace handling follows configured behavior
+</text>
+----
+
+==== Whitelist and blacklist options
+
+You can explicitly specify which elements are whitespace-sensitive:
+
+[source,ruby]
+----
+# Specify elements that preserve whitespace
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: {
+    structural_whitespace: :strict,
+    whitespace_sensitive_elements: [:pre, :code, :sample],
+    whitespace_insensitive_elements: [:p, :div]  # Override defaults/whitelist
+  }
+)
+----
+
+==== respect_xml_space option
+
+Control whether xml:space attributes in the document are honored:
+
+[source,ruby]
+----
+# Honor xml:space (default)
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: {
+    structural_whitespace: :strict,
+    respect_xml_space: true  # Use xml:space attributes in document
+  }
+)
+
+# Ignore xml:space, use only user configuration
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: {
+    structural_whitespace: :strict,
+    respect_xml_space: false  # Override document declarations
+  }
+)
+----
+
+==== Priority order
+
+When determining if an element is whitespace-sensitive, Canon uses this priority:
+
+[source]
+----
+1. respect_xml_space: false → User config only (ignore xml:space)
+   ↓
+2. User whitelist → Use whitelist (user explicitly declared)
+   ↓
+3. Format defaults → HTML: [:pre, :textarea, :script, :style], XML: []
+   ↓
+4. User blacklist → Remove from defaults/whitelist
+   ↓
+5. xml:space="preserve" → Element is sensitive
+   ↓
+6. xml:space="default" → Use steps 1-4
+----
+
+==== Format-specific defaults
+
+**HTML**:: `[:pre, :textarea, :script, :style]` - These elements preserve whitespace by HTML specification
+**XML**:: `[]` - No default whitespace-sensitive elements, purely user-controlled
+
+==== Examples
+
+.Using xml:space attribute
+[source,ruby]
+----
+xml1 = '<root><code xml:space="preserve">  indented  </code></root>'
+xml2 = '<root><code xml:space="preserve">indented</code></root>'
+
+# These are NOT equivalent (whitespace matters in xml:space="preserve")
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: { structural_whitespace: :strict }
+)
+# => false
+----
+
+.Using whitelist
+[source,ruby]
+----
+# Make <p> elements whitespace-sensitive
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: {
+    structural_whitespace: :strict,
+    whitespace_sensitive_elements: [:p, :pre]
+  }
+)
+----
+
+.Overriding HTML defaults
+[source,ruby]
+----
+# Make <script> NOT whitespace-sensitive (override HTML default)
+Canon::Comparison.equivalent?(html1, html2,
+  format: :html,
+  match: {
+    structural_whitespace: :strict,
+    whitespace_insensitive_elements: [:script]
+  }
+)
+----
+
+.Using text_content: :normalize with whitespace_insensitive_elements
+[source,ruby]
+----
+# HTML defaults: [:pre, :code, :textarea, :script, :style]
+# Excluding :code means it's no longer whitespace-sensitive
+html1 = '<root><pre>  indented  </pre><code>  code  </code></root>'
+html2 = '<root><pre>  indented  </pre><code>code</code></root>'
+
+# With :code blacklisted, whitespace in <code> is normalized (formatting-only)
+# HTML uses text_content: :normalize by default
+Canon::Comparison.equivalent?(html1, html2,
+  format: :html,
+  match: {
+    whitespace_insensitive_elements: [:code],
+  }
+)
+# => true (whitespace differences in <code> are formatting-only)
+
+# Without blacklisting, <code> is sensitive (whitespace matters)
+Canon::Comparison.equivalent?(html1, html2,
+  format: :html,
+  match: {
+    structural_whitespace: :strict,
+  }
+)
+# => false (whitespace in <code> is normative)
+----
+
+
+
 === attribute_whitespace
 
 **Applies to**: XML, HTML only
@@ -414,6 +635,23 @@ expect(actual).to be_xml_equivalent_to(expected,
     element_position: :ignore,
     element_hierarchy: :ignore
   )
+
+# Element-level whitespace sensitivity
+expect(actual).to be_xml_equivalent_to(expected,
+  match: { structural_whitespace: :strict }
+)
+  .with_options(
+    whitespace_sensitive_elements: [:pre, :code, :sample],
+    respect_xml_space: true
+  )
+
+# Override HTML default whitespace-sensitive elements
+expect(html).to be_html_equivalent_to(expected,
+  match: { structural_whitespace: :strict }
+)
+  .with_options(
+    whitespace_insensitive_elements: [:script, :style]
+  )
 ====
 
 == Comments dimension