canon 0.1.5 → 0.1.7

data/docs/understanding/comparison-pipeline.adoc

@@ -0,0 +1,317 @@
+---
+title: Comparison Pipeline
+parent: Understanding
+nav_order: 2
+---
+= Comparison Pipeline
+
+== Purpose
+
+Canon's comparison system uses a 4-layer architecture where each layer has a distinct responsibility. Understanding this pipeline is essential for configuring Canon effectively.
+
+== The 4-Layer Flow
+
+Canon processes document comparisons through four sequential layers:
+
+[mermaid]
+----
+graph TD
+    A[Input Documents] --> B[Layer 1: Preprocessing]
+    B --> C[Layer 2: Algorithm Selection]
+    C --> D[Layer 3: Match Options]
+    D --> E[Layer 4: Diff Formatting]
+    E --> F[Output]
+
+    style B fill:#e1f5ff
+    style C fill:#fff4e1
+    style D fill:#ffe1f5
+    style E fill:#e1ffe1
+----
+
+== Layer Overview
+
+=== Layer 1: Preprocessing
+
+**Purpose**: Normalize documents before comparison
+
+**Options**:
+* `none` - No preprocessing (default)
+* `c14n` - Canonicalize using format-specific rules
+* `normalize` - Normalize whitespace
+* `format` - Pretty-print with consistent formatting
+
+**When this runs**: Before any comparison takes place
+
+**Documentation**: See link:../features/preprocessing/[Preprocessing]
+
+=== Layer 2: Algorithm Selection
+
+**Purpose**: Choose the comparison strategy
+
+**Options**:
+* `dom` - DOM-based positional comparison (default, stable)
+* `semantic` - Tree-based semantic diff (experimental, intelligent)
+
+**Impact**: Determines how Layers 3 and 4 behave
+
+**Documentation**: See link:algorithms/[Algorithms]
+
+=== Layer 3: Match Options
+
+**Purpose**: Configure what to compare and how strictly
+
+**Key Concept**: This layer is **algorithm-specific** - each algorithm interprets match options differently.
+
+**Components**:
+* Match dimensions (granular control)
+* Match profiles (preset combinations)
+* Algorithm-specific behaviors
+
+**Documentation**: See link:../features/match-options/[Match Options] and link:../features/match-options/algorithm-specific-behavior.adoc[Algorithm-Specific Behavior]
+
+=== Layer 4: Diff Formatting
+
+**Purpose**: Control how differences are displayed
+
+**Key Concept**: This layer is **algorithm-specific** - each algorithm generates different output types.
+
+**Components**:
+* Diff mode (`by_line`, `by_object`)
+* Colors and symbols
+* Context and grouping
+* Character visualization
+
+**Documentation**: See link:../features/diff-formatting/[Diff Formatting] and link:../features/diff-formatting/algorithm-specific-output.adoc[Algorithm-Specific Output]
+
+== Complete Example
+
+Here's a full 4-layer configuration showing all layers working together:
+
+=== Ruby API
+
+[source,ruby]
+----
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  # Layer 1: Preprocessing
+  preprocessing: :normalize,
+
+  # Layer 2: Algorithm
+  diff_algorithm: :semantic,
+
+  # Layer 3: Match Options
+  match_profile: :spec_friendly,
+
+  # Layer 4: Diff Formatting
+  verbose: true,
+  diff_mode: :by_object,
+  use_color: true,
+  context_lines: 3
+)
+----
+
+=== CLI
+
+[source,bash]
+----
+canon diff file1.xml file2.xml \
+  --preprocessing normalize \          # Layer 1
+  --diff-algorithm semantic \          # Layer 2
+  --match-profile spec_friendly \      # Layer 3
+  --diff-mode by_object \              # Layer 4
+  --verbose \                          # Enable Layer 4 output
+  --context-lines 5                    # Layer 4 option
+----
+
+== Layer-by-Layer Build-Up
+
+Let's see how each layer adds to the configuration:
+
+=== Just Layer 1
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :normalize
+)
+# Documents are normalized, then compared using default DOM algorithm
+# with strict matching and by-line diff output
+----
+
+=== Layers 1 + 2
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :normalize,
+  diff_algorithm: :semantic
+)
+# Documents are normalized, then compared using semantic algorithm
+# with strict matching and by-object diff output (semantic's natural mode)
+----
+
+=== Layers 1 + 2 + 3
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :normalize,
+  diff_algorithm: :semantic,
+  match_profile: :spec_friendly
+)
+# Documents are normalized, compared using semantic algorithm
+# with spec_friendly matching (ignores formatting differences)
+# Default diff output is by-object
+----
+
+=== All 4 Layers
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :normalize,
+  diff_algorithm: :semantic,
+  match_profile: :spec_friendly,
+  verbose: true,
+  diff_mode: :by_line,
+  use_color: true
+)
+# Complete configuration with all layers specified
+# Documents normalized, semantic comparison, spec-friendly matching,
+# traditional line-based diff output with colors
+----
+
+== Layer Interaction Matrix
+
+This table shows common configuration patterns:
+
+[cols="1,1,1,1,2"]
+|===
+|Layer 1 |Layer 2 |Layer 3 |Layer 4 |Result
+
+|none
+|dom
+|strict
+|by_line
+|Traditional exact comparison
+
+|normalize
+|dom
+|spec_friendly
+|by_line
+|Test-friendly comparison
+
+|c14n
+|dom
+|content_only
+|by_object
+|Canonical structure view
+
+|none
+|semantic
+|strict
+|by_object
+|Semantic operations view
+
+|normalize
+|semantic
+|rendered
+|by_line
+|Rendered diff with operations
+|===
+
+== Key Principles
+
+=== Layer Independence
+
+Each layer has a distinct purpose:
+* **Layer 1**: Document transformation
+* **Layer 2**: Comparison strategy
+* **Layer 3**: Match criteria
+* **Layer 4**: Output presentation
+
+=== Algorithm Specificity
+
+Layers 3 and 4 are interpreted differently by each algorithm:
+* The same match options may behave differently with DOM vs Semantic
+* Diff modes have different natural fits for each algorithm
+* Understanding this is crucial for effective configuration
+
+=== Default Behavior
+
+If you don't specify a layer:
+* **Layer 1**: `none` (no preprocessing)
+* **Layer 2**: `dom` (stable algorithm)
+* **Layer 3**: `strict` (exact matching)
+* **Layer 4**: `by_line` for DOM, `by_object` for Semantic
+
+== Common Patterns
+
+=== Testing XML Generation
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(expected, actual,
+  preprocessing: :normalize,       # Ignore formatting
+  match_profile: :spec_friendly,   # Focus on content
+  verbose: true                    # Show differences
+)
+----
+
+=== Debugging Test Failures
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(expected, actual,
+  diff_algorithm: :semantic,       # Detect moves/changes
+  verbose: true,
+  diff_mode: :by_object,           # See operations
+  use_color: true                  # Easier to read
+)
+----
+
+=== Content-Only Comparison
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :format,          # Normalize structure
+  match_profile: :content_only,    # Ignore all formatting
+  verbose: true
+)
+----
+
+== Anti-Patterns to Avoid
+
+=== Over-Configuration
+
+[source,ruby]
+----
+# DON'T: Too many conflicting options
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :c14n,
+  diff_algorithm: :dom,
+  match: {
+    text_content: :ignore,         # Conflicts with preprocessing
+    structural_whitespace: :strict  # Conflicts with preprocessing
+  }
+)
+----
+
+=== Wrong Algorithm/Mode Combination
+
+[source,ruby]
+----
+# SUBOPTIMAL: Semantic algorithm with by-line mode loses operation info
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  diff_mode: :by_line  # Better to use by_object with semantic
+)
+----
+
+== See Also
+
+* link:architecture.adoc[Architecture] - Overall system design
+* link:algorithms/[Algorithms] - Detailed algorithm documentation
+* link:../features/match-options/algorithm-specific-behavior.adoc[Algorithm-Specific Behavior] - How algorithms interpret match options
+* link:../features/diff-formatting/algorithm-specific-output.adoc[Algorithm-Specific Output] - Different output formats
+* link:../guides/choosing-configuration.adoc[Choosing Configuration] - Decision guide for all layers

data/docs/understanding/formats/html.adoc

@@ -0,0 +1,380 @@
+---
+title: HTML Format
+parent: Format Support
+grand_parent: Understanding
+nav_order: 2
+---
+= HTML format
+:toc:
+:toclevels: 3
+
+== Purpose
+
+This page describes Canon's HTML format support, including automatic HTML4/HTML5/XHTML detection, browser rendering simulation, and HTML-specific features.
+
+== Canonicalization
+
+Canon supports HTML 4, HTML5, and XHTML with automatic format detection.
+
+**Key features:**
+
+* Automatic HTML vs XHTML detection
+* HTML5 parser for modern HTML
+* XML parser for XHTML
+* Consistent attribute ordering
+* Whitespace normalization
+* Comment handling in `<style>` and `<script>` tags
+
+.HTML canonicalization example
+[example]
+====
+[source,ruby]
+----
+html = <<~HTML
+  <!DOCTYPE html>
+  <html>
+    <body>
+      <div   class="foo"   id="bar">
+        Content
+      </div>
+    </body>
+  </html>
+HTML
+
+Canon.format(html, :html)
+# => Normalized structure with consistent formatting
+----
+====
+
+== Format defaults
+
+[cols="1,1"]
+|===
+|Dimension |Default Behavior
+
+|`text_content`
+|`:normalize`
+
+|`structural_whitespace`
+|`:normalize`
+
+|`attribute_whitespace`
+|`:normalize`
+
+|`attribute_order`
+|`:ignore`
+
+|`attribute_values`
+|`:strict`
+
+|`comments`
+|`:ignore`
+|===
+
+Default diff mode: `:by_line` (line-based diff)
+
+== Match profiles for HTML
+
+Canon provides predefined profiles optimized for HTML documents. Each profile configures preprocessing, match options, diff algorithm, and formatting.
+
+=== strict profile
+
+**Purpose**: Character-perfect HTML matching
+
+**Configuration**:
+
+[source,ruby]
+----
+{
+  preprocessing: :none,
+  diff_algorithm: :dom,
+  diff_mode: :by_line,       # Line-based diff output (HTML default)
+  match: {
+    text_content: :strict,
+    structural_whitespace: :strict,
+    attribute_whitespace: :strict,
+    attribute_order: :strict,
+    attribute_values: :strict,
+    comments: :strict
+  }
+}
+----
+
+**Use when**: Testing exact HTML formatter output, verifying HTML formatting compliance.
+
+=== rendered profile
+
+**Purpose**: Browser-rendered equivalence (most common for HTML)
+
+**Configuration**:
+
+[source,ruby]
+----
+{
+  preprocessing: :none,
+  diff_algorithm: :dom,
+  diff_mode: :by_line,
+  match: {
+    text_content: :normalize,
+    structural_whitespace: :normalize,
+    attribute_whitespace: :normalize,
+    attribute_order: :ignore,        # HTML attributes are unordered
+    attribute_values: :strict,
+    comments: :ignore
+  }
+}
+----
+
+**Use when**: Comparing HTML as browsers render it, testing web page output, ignoring formatting that doesn't affect display. This is the recommended profile for most HTML comparisons.
+
+=== spec_friendly profile
+
+**Purpose**: Test-friendly comparison for RSpec
+
+**Configuration**:
+
+[source,ruby]
+----
+{
+  preprocessing: :normalize,
+  diff_algorithm: :dom,
+  diff_mode: :by_object,     # Tree-based for better test output
+  match: {
+    text_content: :normalize,
+    structural_whitespace: :ignore,
+    attribute_whitespace: :normalize,
+    attribute_order: :ignore,
+    attribute_values: :strict,
+    comments: :ignore
+  }
+}
+----
+
+**Use when**: Writing RSpec tests for HTML generation, testing semantic HTML correctness.
+
+=== content_only profile
+
+**Purpose**: Maximum tolerance - only structure matters
+
+**Configuration**:
+
+[source,ruby]
+----
+{
+  preprocessing: :normalize,
+  diff_algorithm: :dom,
+  diff_mode: :by_object,
+  match: {
+    text_content: :normalize,
+    structural_whitespace: :ignore,
+    attribute_whitespace: :ignore,
+    attribute_order: :ignore,
+    attribute_values: :ignore,
+    comments: :ignore
+  }
+}
+----
+
+**Use when**: Only HTML structure needs to match, maximum flexibility for all formatting and attribute differences.
+
+== HTML-specific features
+
+=== Format detection
+
+Automatically detects HTML5, HTML4, or XHTML based on DOCTYPE and structure.
+
+.Format detection examples
+[example]
+====
+[source,html]
+----
+<!-- HTML5 detected -->
+<!DOCTYPE html>
+<html>...</html>
+
+<!-- HTML4 detected -->
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
+<html>...</html>
+
+<!-- XHTML detected -->
+<?xml version="1.0"?>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN">
+<html xmlns="http://www.w3.org/1999/xhtml">...</html>
+----
+====
+
+=== Whitespace handling
+
+HTML whitespace is collapsed per CSS rendering rules. Empty text nodes between elements are removed.
+
+.Whitespace handling example
+[example]
+====
+[source,html]
+----
+<!-- Before -->
+<div>
+  <p>Hello     world</p>
+  <p>Second   paragraph</p>
+</div>
+
+<!-- After normalization (with normalize) -->
+<div>
+  <p>Hello world</p>
+  <p>Second paragraph</p>
+</div>
+----
+
+Multiple spaces within text content are collapsed to single spaces when `text_content: :normalize` is used.
+====
+
+=== Attribute order
+
+HTML attributes are inherently unordered per the HTML specification, so default is `:ignore`.
+
+.Attribute order example
+[example]
+====
+[source,html]
+----
+<!-- These are always equivalent for HTML -->
+<input type="text" id="name" class="form-control">
+<input class="form-control" id="name" type="text">
+----
+
+The HTML specification states that attribute order has no meaning, so Canon ignores attribute order by default for HTML.
+====
+
+=== Special tags
+
+Comments in `<style>` and `<script>` tags are normalized specially to handle CSS/JavaScript syntax.
+
+.Special tag handling
+[example]
+====
+[source,html]
+----
+<style>
+  /* CSS comments preserved */
+  body { margin: 0; }
+</style>
+
+<script>
+  // JavaScript comments preserved
+  console.log("test");
+</script>
+----
+
+Canon recognizes that `<style>` and `<script>` tags contain non-HTML content and handles them appropriately.
+====
+
+=== Class attribute normalization
+
+The HTML `class` attribute contains space-separated class names, making normalization particularly useful.
+
+.Class attribute example
+[example]
+====
+[source,html]
+----
+<!-- These are equivalent with attribute_whitespace: :normalize -->
+<div class="btn  primary   active">Click</div>
+<div class="btn primary active">Click</div>
+----
+
+Multiple spaces between class names are normalized to single spaces.
+====
+
+== Usage examples
+
+=== Basic HTML comparison
+
+[source,ruby]
+----
+html1 = File.read("page1.html")
+html2 = File.read("page2.html")
+
+Canon::Comparison.equivalent?(html1, html2,
+  match_profile: :rendered
+)
+----
+
+=== Test-friendly HTML comparison
+
+[source,ruby]
+----
+expect(actual_html).to be_html_equivalent_to(expected_html)
+  .with_profile(:rendered)
+----
+
+=== Using HTML comparator directly
+
+[source,ruby]
+----
+Canon::Comparison::HtmlComparator.equivalent?(html1, html2,
+  match_profile: :rendered
+)
+----
+
+=== CLI usage
+
+[source,bash]
+----
+# Basic comparison with rendered profile
+canon diff page1.html page2.html \
+  --match-profile rendered \
+  --verbose
+
+# Strict HTML comparison
+canon diff file1.html file2.html \
+  --match-profile strict \
+  --verbose
+----
+
+== HTML vs XHTML
+
+Canon handles HTML and XHTML differently:
+
+=== HTML (HTML4/HTML5)
+
+* Uses HTML parser (more lenient)
+* Attribute order ignored by default
+* Whitespace normalized by default
+* Comments ignored by default
+
+=== XHTML
+
+* Uses XML parser (stricter)
+* Follows XML rules
+* Can use XML-specific features
+* Namespace-aware
+
+.XHTML example
+[example]
+====
+[source,xhtml]
+----
+<?xml version="1.0"?>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
+  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+  <head>
+    <title>XHTML Document</title>
+  </head>
+  <body>
+    <p>Content</p>
+  </body>
+</html>
+----
+
+XHTML is treated as XML and follows stricter rules.
+====
+
+== See also
+
+* link:../comparison-pipeline.adoc[Comparison Pipeline] - Understanding the 4 layers
+* link:../../features/match-options/[Match Options] - All matching options
+* link:../../guides/choosing-configuration.adoc[Choosing Configuration] - Decision guide
+* link:index.adoc[Format Support] - Overview of all formats
+* link:xml.adoc[XML Format] - XML-specific features
+* link:json.adoc[JSON Format] - JSON-specific features