canon 0.1.6 → 0.1.7

data/docs/advanced/semantic-diff-report.adoc

@@ -0,0 +1,390 @@
+---
+layout: default
+title: Semantic Diff Report
+parent: Advanced
+nav_order: 2
+---
+
+:toc:
+:toclevels: 3
+
+== Purpose
+
+The Semantic Diff Report provides dimension-specific, actionable details for each difference found during comparison. It focuses on **what changed** and **why it matters**, rather than just showing every changed line.
+
+The report is automatically shown in verbose mode when differences exist, appearing before the detailed diff output.
+
+== Key Features
+
+* XPath locations for XML/HTML elements
+* JSON path locations for JSON/YAML data
+* Dimension-specific formatting optimized for each type of difference
+* Colorized output for easy visual scanning
+* Whitespace preservation detection for `<pre>`, `<code>`, etc.
+* Actionable change summaries (e.g., "Added: +xmlns:v, +xmlns:o")
+
+== Report Format
+
+Each difference is displayed as a vertical section:
+
+[source]
+----
+🔍 DIFFERENCE #1/3 [NORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  attribute_presence
+Location:   /iso-standard/preface/foreword
+
+⊖ Expected (File 1):
+   <foreword> with 3 attributes: displayorder, id, semx-id
+
+⊕ Actual (File 2):
+   <foreword> with 2 attributes: displayorder, id
+
+✨ Changes:
+   Removed: -semx-id
+----
+
+Where:
+
+`STATUS`:: Either `[NORMATIVE]` (green) or `[INFORMATIVE]` (yellow)
+`Dimension`:: The match dimension that detected this difference (magenta)
+`Location`:: XPath for XML/HTML, JSON path for JSON/YAML (blue)
+`Expected`:: What was in File 1 (red heading)
+`Actual`:: What was in File 2 (green heading)
+`Changes`:: Actionable summary (yellow)
+
+== Color Scheme
+
+* **Dimension name**: Magenta
+* **XPath/JSON path**: Blue
+* **Expected heading**: Red (bold)
+* **Actual heading**: Green (bold)
+* **Changes heading**: Yellow (bold)
+* **Status [NORMATIVE]**: Green (bold)
+* **Status [INFORMATIVE]**: Yellow (bold)
+* **Added items**: Green (with `+` prefix)
+* **Removed items**: Red (with `-` prefix)
+* **Element names**: Magenta
+* **Attribute names**: Cyan
+
+== XML/HTML Dimensions
+
+=== Attribute Presence
+
+Shows when attributes are missing or extra.
+
+.Example: Missing attribute
+[example]
+====
+[source]
+----
+🔍 DIFFERENCE #1/1 [NORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  attribute_presence
+Location:   /html/body/div/div
+
+⊖ Expected (File 1):
+   <div> with 2 attributes: class, id
+
+⊕ Actual (File 2):
+   <div> with 1 attribute: id
+
+✨ Changes:
+   Removed: -class
+----
+====
+
+=== Attribute Values
+
+Shows when an attribute value differs.
+
+.Example: Whitespace difference
+[example]
+====
+[source]
+----
+🔍 DIFFERENCE #1/1 [NORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  attribute_values
+Location:   /html/body/div[@id="main"]
+
+⊖ Expected (File 1):
+   <div> class="  container  fluid  "
+
+⊕ Actual (File 2):
+   <div> class="container fluid"
+
+✨ Changes:
+   Whitespace normalization difference
+----
+====
+
+=== Text Content
+
+Shows when element text differs.
+
+.Example: Text content change
+[example]
+====
+[source]
+----
+🔍 DIFFERENCE #1/1 [NORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  text_content
+Location:   /html/body/div/table/tbody/tr/td/pre/text
+
+⊖ Expected (File 1):
+   <text> "
+                      puts \"Hello, world.\"
+                      "
+
+⊕ Actual (File 2):
+   <text> "puts \"Hello, world.\" "
+
+✨ Changes:
+   ⚠️  Whitespace preserved (inside <pre>, <code>, etc. - whitespace is significant)
+----
+====
+
+The warning appears for text inside whitespace-preserving elements where Canon automatically switches to strict mode.
+
+=== Structural Whitespace
+
+Shows whitespace-only differences (usually informative).
+
+.Example: Whitespace-only difference
+[example]
+====
+[source]
+----
+🔍 DIFFERENCE #1/1 [INFORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  structural_whitespace
+Location:   /root/section/p
+
+⊖ Expected (File 1):
+   <p> "hello␣␣world"
+
+⊕ Actual (File 2):
+   <p> "hello␣world"
+
+✨ Changes:
+   Whitespace-only difference (informative)
+----
+====
+
+=== Comments
+
+Shows when comment content differs.
+
+.Example: Comment difference
+[example]
+====
+[source]
+----
+🔍 DIFFERENCE #1/1 [INFORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  comments
+Location:   /html/head
+
+⊖ Expected (File 1):
+   <!-- Original comment text -->
+
+⊕ Actual (File 2):
+   <!-- Modified comment text -->
+
+✨ Changes:
+   Comment content differs
+----
+====
+
+== JSON/YAML Dimensions
+
+JSON and YAML use path-based reporting:
+
+=== Hash Key Differences
+
+.Example: Missing key
+[example]
+====
+[source]
+----
+🔍 DIFFERENCE #1/1 [NORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  hash_key
+Location:   user.email
+
+⊖ Expected (File 1):
+   user.email = "alice@example.com"
+
+⊕ Actual (File 2):
+   user.email = nil
+
+✨ Changes:
+   Key missing
+----
+====
+
+=== Primitive Value Differences
+
+.Example: Value changed
+[example]
+====
+[source]
+----
+🔍 DIFFERENCE #1/1 [NORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  primitive_value
+Location:   users[0].age
+
+⊖ Expected (File 1):
+   users[0].age = 25
+
+⊕ Actual (File 2):
+   users[0].age = 30
+
+✨ Changes:
+   Value changed
+----
+====
+
+=== Array Differences
+
+.Example: Array length differs
+[example]
+====
+[source]
+----
+🔍 DIFFERENCE #1/1 [NORMATIVE]
+──────────────────────────────────────────────────────────────────────
+Dimension:  array_length
+Location:   items
+
+⊖ Expected (File 1):
+   items = [...] (5 items)
+
+⊕ Actual (File 2):
+   items = [...] (3 items)
+
+✨ Changes:
+   Array length differs
+----
+====
+
+== Special Features
+
+=== XPath Location Extraction
+
+For XML/HTML differences, the report extracts XPath with:
+
+* Full path from root: `/html/body/div/section/p`
+* Position predicates when multiple siblings: `/p[2]`, `/div[3]`
+* Safe traversal with depth limits to prevent infinite loops
+* Graceful error handling for circular references
+
+=== Whitespace Preservation Detection
+
+The report detects when text is inside whitespace-preserving HTML elements and shows a special warning:
+
+[source]
+----
+✨ Changes: ⚠️  Whitespace preserved (inside <pre>, <code>, etc. - whitespace is significant)
+----
+
+Elements detected:
+
+* `<pre>` - Preformatted text
+* `<code>` - Code blocks
+* `<textarea>` - Text input areas
+* `<script>` - JavaScript code
+* `<style>` - CSS style sheets
+
+This helps developers understand why whitespace differences cause test failures.
+
+=== Comprehensive Error Handling
+
+The formatter includes multiple layers of error handling:
+
+* Top-level rescue in formatting functions
+* Safe XPath extraction with depth limits
+* Safe parent traversal with document node checks
+* Graceful fallbacks when node types are unexpected
+
+This ensures the report never crashes, even with unusual DOM structures.
+
+== Integration
+
+The Semantic Diff Report is integrated at the `format_comparison_result()` level:
+
+[source,ruby]
+----
+def format_comparison_result(comparison_result, expected, actual)
+  output = []
+
+  # 1. Algorithm name
+  output << "Algorithm: SEMANTIC TREE DIFF"
+
+  # 2. CANON VERBOSE tables (optional)
+  output << verbose_tables if ENV['CANON_VERBOSE'] == '1'
+
+  # 3. Semantic Diff Report (always if diffs exist)
+  if comparison_result.differences.any?
+    output << DiffDetailFormatter.format_report(differences)
+  end
+
+  # 4. Main diff output (always)
+  output << format(...)
+
+  output.compact.join("\n")
+end
+----
+
+This ensures the Semantic Diff Report is part of the main output, not debug information.
+
+== When It Appears
+
+The Semantic Diff Report appears automatically when:
+
+1. Comparison finds differences
+2. Verbose mode is enabled (`verbose: true` or `--verbose`)
+3. Files are not equivalent
+
+It does **not** appear when:
+
+* Files are equivalent
+* Verbose mode is disabled
+* No differences exist
+
+== Practical Uses
+
+=== Debugging Test Failures
+
+When a test fails, the report immediately shows:
+
+* Exactly which element changed
+* What dimension detected the change
+* Whether it's normative or informative
+* Actionable summary of what to fix
+
+=== Understanding Match Behavior
+
+The report helps verify match options are working:
+
+* Informative diffs show what's being ignored
+* Normative diffs show what causes failures
+* Dimension names clarify which rules apply
+
+=== Code Review
+
+During code review, the report provides:
+
+* High-level summary of changes
+* Location-based organization
+* Clear separation of semantic vs formatting changes
+
+== See Also
+
+* link:diff-classification.html[Diff Classification] - Normative vs informative
+* link:../features/diff-formatting/colors-and-symbols.html[Colors and Symbols] - Visual indicators
+* link:verbose-mode-architecture.html[Verbose Mode] - Debug output details
+* link:../understanding/comparison-pipeline.html[Comparison Pipeline] - How comparison works

data/docs/{VERBOSE.adoc → advanced/verbose-mode-architecture.adoc}

@@ -1,25 +1,25 @@
 ---
-layout: default
-title: Verbose Mode
-nav_order: 40
-parent: Advanced Topics
+title: Verbose Mode Architecture
+parent: Advanced
+nav_order: 3
 ---
 = Canon verbose mode guide
 :toc:
 :toclevels: 3
 
-== General
+== Purpose
 
-Canon provides a two-tier verbose output architecture for debugging
-comparison failures:
+Canon provides a two-tier verbose output architecture for debugging comparison failures. This document explains the output structure, when to use each tier, and implementation details.
 
-* **Semantic Diff Report**: Always shown in verbose mode - provides
-  actionable, dimension-specific details for each difference
-* **CANON VERBOSE tables**: Extra detailed option tables shown only when
-  `CANON_VERBOSE=1` environment variable is set
+== Overview
 
-This progressive disclosure ensures developers get useful information by
-default, with additional debugging details available when needed.
+Canon's verbose mode enhances diff output with configuration tables and summaries:
+
+* **Diffs**: Always shown when files differ (default behavior)
+* **Configuration Tables**: Shown with `--verbose` flag or `CANON_VERBOSE=1`
+* **Semantic Diff Report**: Always shown when differences exist in verbose mode
+
+The `--verbose` flag adds diagnostic information, not gates diff display.
 
 == Architecture
 
@@ -28,74 +28,71 @@ The output architecture follows a clear three-tier structure:
 [source]
 ----
 ╔═════════════════════════════════════════════════════════════════════╗
-║              CANON VERBOSE MODE OUTPUT ARCHITECTURE                 ║
+║              CANON DIFF OUTPUT ARCHITECTURE                         ║
 ╚═════════════════════════════════════════════════════════════════════╝
 
-When verbose: true is used:
+When files differ:
 
 ┌─────────────────────────────────────────────────────────────────────┐
-│ TIER 1: CANON VERBOSE Tables (ONLY if CANON_VERBOSE=1)             │
+│ TIER 1: Configuration Tables (with --verbose flag)                  │
 │                                                                      │
 │  ┌────────────────────────────────────────────────────────────────┐ │
-│  │ Match Options Table                                            │ │
+│  │ Match Options Table (if CANON_VERBOSE=1)                       │ │
 │  │  • Shows preprocessing behavior                                │ │
 │  │  • Shows dimension behaviors (strict/normalize/ignore)         │ │
 │  │  • Explains what each setting means                            │ │
 │  └────────────────────────────────────────────────────────────────┘ │
 │  ┌────────────────────────────────────────────────────────────────┐ │
-│  │ Formatter Options Table                                        │ │
+│  │ Formatter Options Table (if CANON_VERBOSE=1)                   │ │
 │  │  • Shows mode (by_line vs by_object)                           │ │
 │  │  • Shows context_lines, diff_grouping_lines                    │ │
 │  │  • Shows show_diffs filter setting                             │ │
 │  └────────────────────────────────────────────────────────────────┘ │
 │  ┌────────────────────────────────────────────────────────────────┐ │
-│  │ Comparison Result Summary                                      │ │
+│  │ Comparison Result Summary (if CANON_VERBOSE=1)                 │ │
 │  │  • Equivalent? (YES/NO)                                        │ │
-│  │  • Normative/Informative/Total diff counts                           │ │
+│  │  • Normative/Informative/Total diff counts                     │ │
 │  └────────────────────────────────────────────────────────────────┘ │
 └─────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
 ┌─────────────────────────────────────────────────────────────────────┐
-│ TIER 2: Semantic Diff Report (ALWAYS if diffs exist)               │
+│ TIER 2: Semantic Diff Report (if --verbose and diffs exist)        │
 │                                                                      │
 │  For each difference:                                               │
 │   • XPath location (e.g., /html/body/div/table/pre/text)           │
 │   • Dimension classification (attribute_presence, text_content)     │
 │   • Specific changes (Added: +xmlns:v, +xmlns:o)                    │
-│   • Normative/Informative status                                          │
+│   • Normative/Informative status                                    │
 │   • Dimension-specific formatting                                   │
 └─────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
 ┌─────────────────────────────────────────────────────────────────────┐
-│ TIER 3: Detailed Diff (ALWAYS)                                     │
+│ TIER 3: Detailed Diff (ALWAYS when files differ)                   │
 │                                                                      │
 │  Either:                                                            │
-│   • Line-by-line diff (for HTML, or with --by-line flag)           │
-│   • Object tree diff (for XML/JSON/YAML by default)                │
+│   • Line-by-line diff (for HTML, or with --diff-mode by_line)      │
+│   • Object tree diff (for XML/JSON/YAML, or with --diff-mode       │
+│     by_object)                                                      │
 └─────────────────────────────────────────────────────────────────────┘
 ----
 
 === Output flow
 
-The `DiffFormatter.format_comparison_result()` method orchestrates the
-output:
+The `DiffCommand` and `DiffFormatter.format_comparison_result()` orchestrate output:
 
-. Check if `CANON_VERBOSE=1` → Render option tables
-. Check if differences exist → Render Semantic Diff Report
-. Always render detailed diff (by-line or by-object)
+. Always show diff when files differ (Tier 3)
+. If `--verbose`: Add configuration tables (Tier 1 if CANON_VERBOSE=1)
+. If `--verbose` and diffs exist: Add Semantic Diff Report (Tier 2)
 
 == Semantic diff report
 
-=== General
+=== Purpose
 
-The Semantic Diff Report is the core verbose output, always shown when
-differences exist. It provides dimension-specific, actionable details for
-each difference.
+The Semantic Diff Report is the core verbose output, always shown when differences exist. It provides dimension-specific, actionable details for each difference.
 
-Unlike the detailed diff (which shows every changed line), the Semantic
-Diff Report shows a high-level summary of WHAT changed and WHY it matters.
+Unlike the detailed diff (which shows every changed line), the Semantic Diff Report shows a high-level summary of WHAT changed and WHY it matters.
 
 === Output format
 
@@ -135,8 +132,7 @@ Each difference displays:
 * **Location**: XPath for XML/HTML, path for JSON/YAML (colorized in blue)
 * **Expected section**: What was in File 1 (red heading, bold)
 * **Actual section**: What was in File 2 (green heading, bold)
-* **Changes summary**: Actionable description of the difference (yellow,
-  bold)
+* **Changes summary**: Actionable description of the difference (yellow, bold)
 
 === Dimension-specific formats
 
@@ -162,8 +158,7 @@ Shows:
 
 * Element name (`<p>`)
 * How many attributes each has
-* Which attributes were added (green with `+` prefix) or removed (red with
-  `-` prefix)
+* Which attributes were added (green with `+` prefix) or removed (red with `-` prefix)
 
 ==== Attribute value differences
 
@@ -187,8 +182,7 @@ Shows:
 
 * Which specific attribute differs (highlighted in cyan)
 * Exact values on both sides
-* Analysis: "Whitespace difference only", "Whitespace normalization
-  difference", or "Value changed"
+* Analysis: "Whitespace difference only", "Whitespace normalization difference", or "Value changed"
 
 ==== Text content differences
 
@@ -214,8 +208,7 @@ Location:   /html/body/div/table/tbody/tr/td/pre/text
 Shows:
 
 * Text preview (truncated at 100 characters)
-* Special warning if inside `<pre>`, `<code>`, `<textarea>`, `<script>`,
-  or `<style>` elements (where whitespace is significant)
+* Special warning if inside `<pre>`, `<code>`, `<textarea>`, `<script>`, or `<style>` elements (where whitespace is significant)
 
 ==== Structural whitespace differences
 
@@ -260,10 +253,9 @@ Location:   user.email
 
 == CANON VERBOSE mode
 
-=== General
+=== Purpose
 
-CANON VERBOSE mode adds detailed option tables BEFORE the Semantic Diff
-Report. These tables help understand:
+CANON VERBOSE mode adds detailed option tables BEFORE the Semantic Diff Report. These tables help understand:
 
 * What match options are in effect
 * How the diff formatter is configured
@@ -290,10 +282,10 @@ Shows preprocessing and dimension behaviors:
 │ Dimension          │ Behavior  │ Meaning                            │
 ├────────────────────┼───────────┼────────────────────────────────────┤
 │ preprocessing      │ rendered  │ As browser-rendered (compacted wh… │
-│ text_content       │ normalize │ Normalized then compared (normative…  │
+│ text_content       │ normalize │ Normalized then compared (normative)  │
 │ structural_whit…   │ ignore    │ Differences IGNORED (informative)     │
 │ attribute_presence │ strict    │ Must match exactly (normative)        │
-│ attribute_values   │ normalize │ Normalized then compared (normative…  │
+│ attribute_values   │ normalize │ Normalized then compared (normative)  │
 │ comments           │ ignore    │ Differences IGNORED (informative)     │
 ╰────────────────────┴───────────┴────────────────────────────────────╯
 ----
@@ -309,10 +301,9 @@ Preprocessing behaviors:
 
 Dimension behaviors:
 
-* `:ignore` - Differences IGNORED (innormative, won't fail test)
+* `:ignore` - Differences IGNORED (informative, won't fail test)
 * `:strict` - Must match exactly (normative, will fail test)
-* `:normalize` - Normalized then compared (normative if different after
-  normalization)
+* `:normalize` - Normalized then compared (normative if different after normalization)
 * `:strip` - Strip leading/trailing whitespace only
 * `:compact` - Collapse whitespace runs to single space
 
@@ -328,7 +319,7 @@ Shows diff formatting settings:
 │                         Formatter Options                          │
 ├─────────────────────┬─────────┬────────────────────────────────────┤
 │ Option              │ Value   │ Impact                             │
-├─────────────────────┼─────────┼─────────────────────────────────────┤
+├─────────────────────┼─────────┼────────────────────────────────────┤
 │ mode                │ by_line │ Line-by-line diff                  │
 │ context_lines       │ 3       │ 3 lines of context around diffs    │
 │ show_diffs          │ all     │ Show all diffs (normative + informative) │
@@ -480,3 +471,10 @@ end
 ----
 
 This ensures the correct output order and separation of concerns.
+
+== See also
+
+* link:../features/diff-formatting/[Diff Formatting] - Diff output customization
+* link:../understanding/comparison-pipeline.adoc[Comparison Pipeline] - 4-layer architecture
+* link:../interfaces/rspec/[RSpec Matchers] - Using verbose mode in tests
+* link:../interfaces/cli/[CLI] - Command-line verbose mode