canon 0.1.6 → 0.1.7

data/docs/features/match-options/algorithm-specific-behavior.adoc

@@ -0,0 +1,365 @@
+---
+title: Algorithm-Specific Behavior
+parent: Match Options
+grand_parent: Features
+nav_order: 4
+---
+= Algorithm-Specific Behavior
+
+== Purpose
+
+Match options control what to compare and how strictly, but **different algorithms interpret these options differently**. Understanding these differences is crucial for choosing the right configuration.
+
+This page explains how the DOM and Semantic algorithms each handle match dimensions and provides guidance for migrating between algorithms.
+
+== Key Concept
+
+The same match option settings can produce different comparison behavior depending on which algorithm you choose:
+
+* **DOM algorithm** uses options for element-by-element positional comparison
+* **Semantic algorithm** uses options during signature calculation and similarity matching
+
+== Algorithm Comparison
+
+=== DOM Algorithm Match Behavior
+
+The DOM algorithm applies match options during **positional element comparison**:
+
+**Characteristics**:
+* Elements matched by position in document tree
+* Match options control comparison strictness at each position
+* No understanding of semantic relationships
+* Order matters significantly
+
+**How Options Are Used**:
+* `text_content` - Controls how text at each position is compared
+* `structural_whitespace` - Controls whitespace comparison in structure
+* `attribute_order` - Controls whether attribute order must match
+* `attribute_values` - Controls how attribute values are compared
+
+**Best For**:
+* Documents with similar structure
+* Traditional diff workflows
+* Fast comparisons
+* Stable, predictable results
+
+=== Semantic Algorithm Match Behavior
+
+The Semantic algorithm applies match options during **signature calculation and similarity matching**:
+
+**Characteristics**:
+* Elements matched by semantic signatures
+* Match options influence signature generation
+* Understands moves, merges, splits
+* Order less critical (uses similarity scoring)
+
+**How Options Are Used**:
+* `text_content` - Included in element signature
+* `structural_whitespace` - Affects structural signatures
+* `attribute_order` - Ignored (attributes are unordered in signatures)
+* `attribute_values` - Included in element signature
+
+**Best For**:
+* Restructured documents
+* Detecting semantic changes
+* Operation-level analysis
+* Content evolution tracking
+
+== Match Dimension Handling by Algorithm
+
+This table shows how each algorithm interprets each match dimension:
+
+[cols="2,3,3"]
+|===
+|Match Dimension |DOM Algorithm |Semantic Algorithm
+
+|**text_content**
+|Compares text at each position. `strict` requires exact match, `normalize` normalizes whitespace, `ignore` skips text comparison
+|Influences element signature. `strict` includes exact text, `normalize` includes normalized text, `ignore` excludes text from signature
+
+|**structural_whitespace**
+|Compares whitespace-only text nodes at each position
+|Affects structural signature calculation. Normalized whitespace creates different signatures
+
+|**attribute_whitespace**
+|Compares whitespace in attribute values at each position
+|Affects attribute value signatures. Normalized values create different signatures
+
+|**attribute_order**
+|`strict` requires same attribute order, `ignore` allows any order at each position
+|Always ignored - attributes are unordered in semantic signatures
+
+|**attribute_values**
+|Compares attribute values at each position
+|Attribute values included in element signature
+
+|**key_order** (JSON/YAML)
+|`strict` requires same key order, `ignore` allows any order
+|Always ignored - keys are unordered in semantic signatures
+
+|**comments**
+|Compares comments at each position. `strict` requires exact match, `normalize` normalizes, `ignore` skips
+|Comments can be included in signatures or ignored. Less impact than in DOM
+
+|**namespace_uri** (XML)
+|Always compared strictly - elements must have same namespace URI to match at each position
+|Always included in element signature - elements must have same namespace URI for signatures to match
+|===
+
+== Example: Same Options, Different Results
+
+Here's an example showing how the same match options produce different results with each algorithm:
+
+=== Document Pair
+
+[source,xml]
+----
+<!-- Document 1 -->
+<book>
+  <title>Canon Guide</title>
+  <author>John Doe</author>
+</book>
+
+<!-- Document 2 -->
+<book>
+  <author>John Doe</author>
+  <title>Canon Guide</title>
+</book>
+----
+
+=== DOM Algorithm Result
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :dom,
+  match: {
+    text_content: :normalize,
+    attribute_order: :ignore
+  },
+  verbose: true
+)
+# Result: NOT EQUIVALENT
+# Reason: Elements at positions don't match (title vs author)
+# Even though content is identical, position matters
+----
+
+=== Semantic Algorithm Result
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  match: {
+    text_content: :normalize,
+    attribute_order: :ignore
+  },
+  verbose: true
+)
+# Result: EQUIVALENT (with MOVE operation)
+# Reason: Elements have same signatures, just reordered
+# Semantic algorithm detects this as a MOVE operation
+----
+
+== Match Profile Behavior Differences
+
+Match profiles also behave differently with each algorithm:
+
+=== `strict` Profile
+
+**DOM Algorithm**:
+* Exact positional matching
+* All elements must be in same positions
+* Whitespace must match exactly
+* Fast comparison
+
+**Semantic Algorithm**:
+* Exact signature matching
+* Elements can be reordered
+* Signatures must match exactly
+* Slower but detects moves
+
+=== `spec_friendly` Profile
+
+**DOM Algorithm**:
+* Ignores formatting at each position
+* Position still matters
+* Good for test assertions with similar structure
+
+**Semantic Algorithm**:
+* Ignores formatting in signatures
+* Position doesn't matter
+* Good for test assertions with any structure
+
+=== `content_only` Profile
+
+**DOM Algorithm**:
+* Compares only text content at positions
+* Still position-dependent
+* Ignores all structural differences at each position
+
+**Semantic Algorithm**:
+* Generates signatures from content only
+* Position-independent
+* True content-only comparison
+
+== Migration Guide
+
+=== Switching from DOM to Semantic
+
+When migrating from DOM to Semantic algorithm:
+
+**Expected Changes**:
+1. **Reordered elements** will be detected as MOVEs instead of DELETE+INSERT
+2. **attribute_order** setting becomes irrelevant (always ignored)
+3. **Performance** will be slower but more intelligent
+4. **Output format** changes to operation-based
+
+**Configuration Adjustments**:
+
+[source,ruby]
+----
+# Before (DOM)
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :dom,
+  match: {
+    attribute_order: :strict  # This mattered
+  },
+  diff_mode: :by_line
+)
+
+# After (Semantic)
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  match: {
+    attribute_order: :ignore  # Changed (but actually doesn't matter)
+  },
+  diff_mode: :by_object  # Better for semantic output
+)
+----
+
+**What to Watch For**:
+* Tests expecting positional differences may now pass (moves detected)
+* Diff output format changes significantly
+* Performance may be slower on large documents
+
+=== Switching from Semantic to DOM
+
+When migrating from Semantic to DOM algorithm:
+
+**Expected Changes**:
+1. **MOVE operations** will become DELETE+INSERT pairs
+2. **Reordered content** will show as differences
+3. **Performance** will be faster
+4. **Output format** changes to line-based
+
+**Configuration Adjustments**:
+
+[source,ruby]
+----
+# Before (Semantic)
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  diff_mode: :by_object
+)
+
+# After (DOM)
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :dom,
+  match: {
+    attribute_order: :ignore  # May want to add this
+  },
+  diff_mode: :by_line  # Better for DOM output
+)
+----
+
+**What to Watch For**:
+* Tests may now fail on reordered content
+* Need to add `attribute_order: :ignore` if attribute order shouldn't matter
+* Diff output is less semantic, more positional
+
+== Choosing the Right Algorithm
+
+=== Use DOM Algorithm When
+
+* Documents have similar structure
+* Position matters
+* Fast performance is critical
+* Traditional diff output is sufficient
+* Stability is important (production use)
+
+=== Use Semantic Algorithm When
+
+* Documents may be restructured
+* Need to detect moves/reorders
+* Operation-level analysis is valuable
+* Content evolution tracking is needed
+* Willing to accept experimental status
+
+== Common Patterns
+
+=== Pattern 1: Test-Friendly DOM Comparison
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(expected, actual,
+  diff_algorithm: :dom,
+  match_profile: :spec_friendly,
+  verbose: true
+)
+# Ignores formatting but requires same structure
+----
+
+=== Pattern 2: Content-Only Semantic Comparison
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  match_profile: :content_only,
+  verbose: true,
+  diff_mode: :by_object
+)
+# True content comparison, structure-independent
+----
+
+=== Pattern 3: Hybrid Approach
+
+[source,ruby]
+----
+# Try DOM first (fast)
+if Canon::Comparison.equivalent?(doc1, doc2, diff_algorithm: :dom)
+  puts "Documents identical"
+else
+  # Use semantic for detailed analysis
+  result = Canon::Comparison.equivalent?(doc1, doc2,
+    diff_algorithm: :semantic,
+    verbose: true,
+    diff_mode: :by_object
+  )
+  puts result.operations
+end
+----
+
+== Performance Implications
+
+=== DOM Algorithm Performance
+
+* **Speed**: Fast (linear with document size)
+* **Memory**: Low (processes line-by-line)
+* **Best for**: Documents < 100KB
+
+=== Semantic Algorithm Performance
+
+* **Speed**: Slower (quadratic worst case)
+* **Memory**: Higher (builds tree structures)
+* **Best for**: Documents < 10KB or where intelligence is worth the cost
+
+== See Also
+
+* link:index.adoc[Match Options Overview]
+* link:../../understanding/algorithms/[Algorithms] - Detailed algorithm documentation
+* link:dimensions.adoc[Match Dimensions] - All available dimensions
+* link:profiles.adoc[Match Profiles] - Preset configurations
+* link:../diff-formatting/algorithm-specific-output.adoc[Algorithm-Specific Output] - How output differs
+* link:../../guides/choosing-configuration.adoc[Choosing Configuration] - Decision guide

data/docs/features/match-options/html-policies.adoc

@@ -0,0 +1,312 @@
+---
+layout: default
+title: HTML-Specific Policies
+parent: Match Options
+grand_parent: Features
+nav_order: 4
+---
+
+:toc:
+:toclevels: 3
+
+== HTML-Specific Comparison Policies
+
+=== Overview
+
+HTML comparison has specific policies that differ from XML due to HTML's unique
+characteristics and rendering behavior. Canon uses `HtmlCompareProfile` to
+implement these format-specific policies.
+
+=== Default Policies
+
+HTML uses the `:rendered` profile by default:
+
+[source,ruby]
+----
+{
+  preprocessing: :rendered,
+  text_content: :normalize,
+  structural_whitespace: :normalize,
+  comments: :ignore,
+  attribute_order: :ignore
+}
+----
+
+This reflects how browsers render HTML - whitespace is normalized and comments
+are presentational.
+
+=== HTML Version Detection
+
+Canon automatically detects HTML version:
+
+* **HTML4**: Case-insensitive element/attribute names
+* **HTML5**: Case-sensitive (preserves case)
+
+Detection is based on DOCTYPE or parsing mode.
+
+=== Whitespace Preservation
+
+Certain HTML elements require strict whitespace preservation regardless of the
+`text_content` policy:
+
+[cols="1,3"]
+|===
+|Element |Purpose
+
+|`<pre>`
+|Preformatted text blocks
+
+|`<code>`
+|Code snippets
+
+|`<textarea>`
+|Form input fields
+
+|`<script>`
+|JavaScript code
+
+|`<style>`
+|CSS stylesheets
+|===
+
+Inside these elements, ALL whitespace is preserved even when `text_content:
+:normalize` is set.
+
+.Example: Whitespace preservation in <pre>
+====
+[source,ruby]
+----
+html1 = '<pre>Line 1\n  Line 2</pre>'
+html2 = '<pre>Line 1\nLine 2</pre>'
+
+# Whitespace is preserved - not equivalent
+Canon::Comparison.equivalent?(html1, html2, preprocessing: :rendered)
+# => false
+----
+
+The indentation difference matters in `<pre>` elements.
+====
+
+.Example: Whitespace normalization in <div>
+====
+[source,ruby]
+----
+html1 = '<div>Text    with    spaces</div>'
+html2 = '<div>Text with spaces</div>'
+
+# Whitespace is normalized - equivalent
+Canon::Comparison.equivalent?(html1, html2, preprocessing: :rendered)
+# => true
+----
+
+Multiple spaces are normalized to single spaces in regular elements.
+====
+
+=== Comment Handling
+
+HTML comments are presentational by default (like CSS styles):
+
+[source,ruby]
+----
+# Default: comments ignored (informative)
+html1 = '<div><!-- comment --><p>Text</p></div>'
+html2 = '<div><p>Text</p></div>'
+
+Canon::Comparison.equivalent?(html1, html2)
+# => true (comments don't affect equivalence)
+
+# Strict mode: comments compared (normative)
+Canon::Comparison.equivalent?(html1, html2, match: { comments: :strict })
+# => false (comments affect equivalence)
+----
+
+=== Why comments are ignored by default
+
+In HTML, comments serve similar purposes to CSS:
+* Developer notes
+* Conditional comments (IE hacks)
+* Disabled code blocks
+* Build tool markers
+
+They don't affect rendering or semantic meaning, so they're
+informative by default.
+
+=== Case Sensitivity
+
+HTML4 and HTML5 have different case sensitivity rules:
+
+.HTML4 (case-insensitive)
+====
+[source,ruby]
+----
+html1 = '<DIV CLASS="test">Content</DIV>'
+html2 = '<div class="test">Content</div>'
+
+Canon::Comparison.equivalent?(html1, html2, format: :html4)
+# => true (case doesn't matter in HTML4)
+----
+====
+
+.HTML5 (case-sensitive)
+====
+[source,ruby]
+----
+html1 = '<DIV CLASS="test">Content</DIV>'
+html2 = '<div class="test">Content</div>'
+
+Canon::Comparison.equivalent?(html1, html2, format: :html5)
+# => false (case matters in HTML5, though uncommon)
+----
+====
+
+=== Usage Examples
+
+=== Default HTML comparison
+
+[source,ruby]
+----
+require 'canon/comparison'
+
+html1 = '<div>  <p>  Text  </p>  </div>'
+html2 = '<div><p>Text</p></div>'
+
+# Uses HtmlCompareProfile automatically
+result = Canon::Comparison.equivalent?(html1, html2)
+# => true (whitespace normalized, comments ignored)
+----
+
+=== Strict HTML comparison
+
+[source,ruby]
+----
+# All differences matter
+result = Canon::Comparison.equivalent?(html1, html2,
+  match: {
+    text_content: :strict,
+    structural_whitespace: :strict,
+    comments: :strict,
+    attribute_order: :strict
+  }
+)
+# => false (whitespace differences are normative)
+----
+
+=== Mixed policies
+
+[source,ruby]
+----
+# Normalize whitespace but compare comments strictly
+result = Canon::Comparison.equivalent?(html1, html2,
+  match: {
+    text_content: :normalize,
+    structural_whitespace: :normalize,
+    comments: :strict
+  }
+)
+----
+
+=== Preprocessing Options
+
+HTML supports several preprocessing modes:
+
+=== `:rendered` (default)
+
+Simulates browser rendering:
+- Normalizes whitespace
+- Preserves whitespace in special elements
+- Ignores comments
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(html1, html2, preprocessing: :rendered)
+----
+
+=== `:format`
+
+Pretty-prints before comparison:
+- Consistent indentation
+- One element per line
+- Good for visual diffs
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(html1, html2, preprocessing: :format)
+----
+
+=== `:none`
+
+No preprocessing:
+- Raw comparison
+- Useful for exact matching
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(html1, html2, preprocessing: :none)
+----
+
+=== Advanced Examples
+
+=== Compare HTML with mixed content
+
+[source,ruby]
+----
+html1 = '<p>This is <em>important</em> text.</p>'
+html2 = '<p>This is  <em>important</em>  text.</p>'
+
+result = Canon::Comparison.equivalent?(
+  html1, html2,
+  verbose: true,
+  match: { text_content: :normalize, structural_whitespace: :normalize }
+)
+
+result.equivalent?  # => true
+result.differences  # => [#<DiffNode formatting: true, normative: false>]
+----
+
+=== Compare with element-specific preservation
+
+[source,ruby]
+----
+html1 = '<div><pre>  Code  </pre></div>'
+html2 = '<div><pre>Code</pre></div>'
+
+# Whitespace preserved in <pre>, normalized in <div>
+result = Canon::Comparison.equivalent?(html1, html2)
+# => false (whitespace matters in <pre>)
+----
+
+=== Detect normative vs informative differences
+
+[source,ruby]
+----
+html1 = '<div class="a" id="1"><!-- v1 --><p>Text</p></div>'
+html2 = '<div id="1" class="b"><!-- v2 --><p>Text</p></div>'
+
+result = Canon::Comparison.equivalent?(
+  html1, html2,
+  verbose: true,
+  match: { attribute_order: :ignore, comments: :ignore }
+)
+
+# Attribute order: informative (ignored)
+# Comments: informative (ignored)
+# Attribute value (class): normative (different)
+
+result.equivalent?  # => false
+result.differences.select(&:normative?)  # => [class attribute diff]
+result.differences.reject(&:normative?) # => [order diff, comment diff]
+----
+
+=== Implementation
+
+See the following files for implementation details:
+
+* [`lib/canon/comparison/html_compare_profile.rb`](../../lib/canon/comparison/html_compare_profile.rb) - HTML-specific profile
+* [`lib/canon/comparison/compare_profile.rb`](../../lib/canon/comparison/compare_profile.rb) - Base profile
+* [`spec/canon/comparison/html_compare_profile_spec.rb`](../../spec/canon/comparison/html_compare_profile_spec.rb) - Comprehensive examples
+
+=== See Also
+
+* link:index.html[Match Options] - Overview of match system
+* link:algorithm-specific-behavior.html[Algorithm-Specific Behavior] - How algorithms handle options
+* link:../../advanced/diff-classification.html[Diff Classification] - Normative vs informative