canon 0.1.3 → 0.1.5

data/docs/MODES.adoc

@@ -0,0 +1,432 @@
+---
+layout: default
+title: Diff Modes
+nav_order: 21
+parent: Understanding Canon
+---
+= Diff modes
+:toc:
+:toclevels: 3
+
+== Scope
+
+This document explains Canon's two diff modes: by-line and by-object. Each
+mode is optimized for different use cases and document formats.
+
+For usage examples, see link:RUBY_API[Ruby API], link:CLI[CLI], or
+link:RSPEC[RSpec documentation].
+
+== General
+
+Canon provides two distinct diff modes that present differences in different
+ways:
+
+* **by-line**: Traditional line-by-line diff after canonicalization
+* **by-object**: Tree-based semantic diff showing structural changes
+
+The choice of mode affects how differences are calculated, displayed, and
+what information is emphasized.
+
+== by-line mode
+
+=== Purpose
+
+Show differences as traditional line-by-line changes, similar to `diff` or
+`git diff`. Best for reviewing textual changes and understanding exact line
+modifications.
+
+=== How it works
+
+. Canonicalize both documents
+. Perform line-by-line comparison using Longest Common Subsequence (LCS)
+. For XML: Apply DOM-guided semantic matching to pair corresponding elements
+. Display with line numbers, diff markers, and context
+
+=== When to use
+
+**Best for:**
+
+* HTML documents where line-level changes matter
+* Reviewing markup structure changes
+* Understanding exact textual differences
+* When you need full document context
+* Debugging formatting issues
+
+**Default for:**
+
+* HTML format (always uses by-line mode)
+* XML format when `--by-line` flag is specified
+
+=== Output format
+
+[source]
+----
+   4|  -  | <foreword id="fwd">
+   4|  +  | <foreword displayorder="2" id="fwd">
+   5|     | <p>First paragraph</p>
+  ...
+  10|  +  | <p>New content</p>
+  11|     | </clause>
+----
+
+Where:
+
+* Left column: Line number in first document
+* Middle column: Line number in second document
+* Marker: `-` (removed), `+` (added), ` ` (unchanged)
+* Right column: Line content
+
+=== Features
+
+**DOM-guided semantic matching** (XML only)::
+Elements are matched by identity attributes (`id`, `ref`, `name`, `key`) and
+element paths, ensuring corresponding elements are compared even when at
+different line positions.
+
+**Token-level highlighting**::
+Within changed lines, individual tokens (element names, attribute names,
+values) are highlighted to show exactly what changed.
+
+**Context lines**::
+Shows unchanged lines around changes for context (configurable via
+`context_lines`).
+
+**Whitespace visualization**::
+Makes invisible characters visible when `use_color` is enabled.
+
+.by-line mode example
+[example]
+====
+[source,ruby]
+----
+# Ruby API - XML with by-line mode
+result = Canon::Comparison.equivalent?(xml1, xml2,
+  verbose: true,
+  diff: { mode: :by_line }
+)
+
+# CLI
+$ canon diff document1.xml document2.xml --by-line --verbose
+----
+
+Output shows line-by-line changes with full context:
+
+[source]
+----
+   1|     | <?xml version="1.0" encoding="UTF-8"?>
+   2|     | <document>
+   3|     |   <preface>
+   4|  -  |     <foreword id="fwd">
+   4|  +  |     <foreword displayorder="2" id="fwd">
+   5|     |       <p>First paragraph</p>
+   6|     |     </foreword>
+----
+====
+
+== by-object mode
+
+=== Purpose
+
+Show differences as structured tree changes, focusing on what changed in the
+document's semantic structure rather than line-by-line text.
+
+=== How it works
+
+. Parse both documents into object trees (DOM for XML/HTML, objects for
+JSON/YAML)
+. Perform semantic tree comparison
+. Identify additions, deletions, and modifications
+. Display as visual tree showing only changes
+
+=== When to use
+
+**Best for:**
+
+* JSON and YAML documents (configuration files, API responses)
+* XML when you care about structural changes
+* Understanding what values changed
+* Reviewing configuration differences
+* Comparing API responses
+
+**Default for:**
+
+* JSON format
+* YAML format
+* XML format (unless `--by-line` is specified)
+
+=== Output format
+
+[source]
+----
+Visual Diff:
+├── settings.debug:
+│   ├── - true
+│   └── + false
+└── version:
+    ├── - "1.0.0"
+    └── + "2.0.0"
+----
+
+The tree shows:
+
+* `├──` Tree structure using box-drawing characters
+* `│` Path to the changed value
+* `-` Removed/old value (red)
+* `+` Added/new value (green)
+
+=== Features
+
+**Structural focus**::
+Shows only what changed in the object hierarchy, ignoring formatting.
+
+**Path display**::
+Full path to each change (e.g., `settings.debug`, `user.address.city`).
+
+**Value highlighting**::
+Clear before/after values with color coding.
+
+**Compact output**::
+Omits unchanged parts of the structure.
+
+.by-object mode example
+[example]
+====
+[source,ruby]
+----
+# Ruby API - JSON with by-object mode (default)
+result = Canon::Comparison.equivalent?(json1, json2,
+  verbose: true
+  # mode: :by_object is default for JSON
+)
+
+# CLI
+$ canon diff config1.json config2.json --verbose
+----
+
+Output shows semantic changes in tree form:
+
+[source]
+----
+Visual Diff:
+├── database.host:
+│   ├── - "localhost"
+│   └── + "production.db.example.com"
+├── database.port:
+│   ├── - 5432
+│   └── + 5433
+└── logging.level:
+    ├── - "info"
+    └── + "debug"
+----
+====
+
+== Mode comparison
+
+[cols="1,1,1"]
+|===
+|Aspect |by-line |by-object
+
+|**View**
+|Line-by-line text
+|Tree structure
+
+|**Focus**
+|Textual changes
+|Semantic changes
+
+|**Shows**
+|All lines with context
+|Only changed values
+
+|**Best for**
+|HTML, markup review
+|JSON, YAML, config files
+
+|**Default formats**
+|HTML
+|XML, JSON, YAML
+
+|**Whitespace**
+|Visualized in diff lines
+|Not shown (normalized)
+
+|**Context**
+|Surrounding lines
+|Object hierarchy path
+
+|**Output size**
+|Larger (full context)
+|Smaller (changes only)
+|===
+
+== Choosing the right mode
+
+=== Use by-line mode when
+
+* Reviewing HTML markup changes
+* Debugging formatting or whitespace issues
+* Need to see exact line positions
+* Want full document context
+* Working with primarily text-based formats
+* Need to match with traditional diff tools
+
+=== Use by-object mode when
+
+* Comparing configuration files (JSON/YAML)
+* Reviewing API response differences
+* Focus on what values changed
+* Don't care about formatting
+* Want compact diff output
+* Working with nested data structures
+
+== Format-specific defaults
+
+[cols="1,1,2"]
+|===
+|Format |Default Mode |Rationale
+
+|**HTML**
+|by-line
+|Markup structure matters; line-level changes important
+
+|**XML**
+|by-object
+|Semantic structure focus; DOM-based comparison more meaningful
+
+|**JSON**
+|by-object
+|Object graph comparison; structure over formatting
+
+|**YAML**
+|by-object
+|Configuration focus; value changes matter most
+|===
+
+== Configuration
+
+=== Ruby API
+
+[source,ruby]
+----
+# Specify mode explicitly
+Canon::Comparison.equivalent?(doc1, doc2,
+  verbose: true,
+  diff: { mode: :by_line }
+)
+
+# Global configuration for RSpec
+Canon::RSpecMatchers.configure do |config|
+  config.xml.diff.mode = :by_line
+  config.json.diff.mode = :by_object
+end
+----
+
+=== CLI
+
+[source,bash]
+----
+# Force by-line mode for XML
+$ canon diff file1.xml file2.xml --by-line --verbose
+
+# by-object is default for JSON
+$ canon diff config1.json config2.json --verbose
+----
+
+=== RSpec
+
+[source,ruby]
+----
+# Global configuration
+Canon::RSpecMatchers.configure do |config|
+  config.xml.diff.mode = :by_line
+end
+
+# Override per-test
+expect(actual).to be_xml_equivalent_to(expected, verbose: true)
+  # Uses global config mode
+----
+
+== Advanced features
+
+=== by-line mode advanced features
+
+**DOM-guided semantic matching** (XML only)::
+Matches elements across documents using identity attributes, ensuring
+corresponding elements are compared even when at different positions.
+
+**Token highlighting**::
+Within changed lines, highlights specific tokens that differ (element names,
+attributes, values).
+
+**Grouped contexts**::
+Groups nearby changes into context blocks with `diff_grouping_lines`.
+
+.XML DOM-guided matching example
+[example]
+====
+When elements are reordered but have unique IDs, by-line mode matches them
+semantically:
+
+[source,xml]
+----
+<!-- File 1 -->
+<items>
+  <item id="1" name="First"/>
+  <item id="2" name="Second"/>
+</items>
+
+<!-- File 2 -->
+<items>
+  <item id="2" name="Second"/>
+  <item id="1" name="First Edited"/>
+</items>
+----
+
+Diff shows only the actual change:
+
+[source]
+----
+Visual Diff:
+└── items.item[id="1"].name:
+    ├── - "First"
+    └── + "First Edited"
+----
+
+Elements matched by `id` attribute, position change ignored.
+====
+
+=== by-object mode advanced features
+
+**Deep nesting support**::
+Handles arbitrarily nested structures with clear path display.
+
+**Type-aware comparison**::
+Distinguishes between different data types (string "1" vs number 1).
+
+**Array handling**::
+Shows array element changes with index notation.
+
+.Nested structure example
+[example]
+====
+[source]
+----
+Visual Diff:
+├── users[0].profile.settings.notifications.email:
+│   ├── - true
+│   └── + false
+└── users[1].name:
+    ├── - "John Doe"
+    └── + "Jane Doe"
+----
+
+Clear path to each change in deeply nested structure.
+====
+
+== See also
+
+* link:RUBY_API[Ruby API documentation]
+* link:CLI[Command-line interface]
+* link:RSPEC[RSpec matchers]
+* link:DIFF_FORMATTING[Diff formatting options]
+* link:MATCH_ARCHITECTURE[Match architecture]

data/docs/NORMATIVE_INFORMATIVE_DIFFS.adoc

@@ -0,0 +1,219 @@
+---
+layout: default
+title: Normative vs Informative Diffs
+nav_order: 42
+parent: Advanced Topics
+---
+= Canon Normative vs Informative Diffs Architecture
+:toc:
+:toclevels: 3
+
+== Overview
+
+Canon distinguishes between two types of differences when comparing documents:
+
+* **Normative diffs**: Semantic differences that affect the match result based on your match options
+* **Informative diffs**: Textual differences that don't affect the match (semantically equivalent per match options)
+
+This allows you to focus on differences that matter while optionally viewing formatting-only changes.
+
+== Architecture
+
+----
+╔═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
+║                                              CANON DIFF ARCHITECTURE                                                                             ║
+║                                         Normative vs Informative Differences                                                                           ║
+╚═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
+
+┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
+│ COMPARISON LAYER: Semantic Analysis                                                                                                              │
+│                                                                                                                                                  │
+│  Input: Two documents (HTML, XML, JSON, YAML)                                                                                                   │
+│  Output: Semantic differences between document trees                                                                                            │
+│                                                                                                                                                  │
+│  ┌────────────────────┐    ┌────────────────────┐    ┌────────────────────┐                  ┌──────────────────────────────────────┐         │
+│  │  Tree Comparator   │    │  Tree Comparator   │    │  Tree Comparator   │                  │      Match Options                   │         │
+│  │  (HTML/XML)        │    │  (JSON)            │    │  (YAML)            │                  │──────────────────────────────────────│         │
+│  └─────────┬──────────┘    └─────────┬──────────┘    └─────────┬──────────┘                  │ Define what matters:                 │         │
+│            └───────────────────────────┴───────────────────────┘                             │  • text_content                      │         │
+│                                        │                                                      │  • structural_whitespace             │         │
+│                                        ▼                                                      │  • attribute_whitespace              │         │
+│                         Creates DiffNodes (semantic differences)                             │  • comments                          │         │
+│                                        │                                                      │  • key_order                         │         │
+│                                        │                                                      │                                      │         │
+│  ┌─────────────────────────────────────┴───────────────────────────────────┐                │ Each dimension has behavior:         │         │
+│  │  DiffNode                                                                │                │  • :strict    → must match exactly   │         │
+│  │  Represents one semantic difference                                     │                │  • :normalize → match after cleanup  │         │
+│  │──────────────────────────────────────────────────────────────────────────│                │  • :ignore    → don't care           │         │
+│  │  • References nodes in both trees                                       │◄───Classified──┤                                      │         │
+│  │  • Has a dimension (what differs)                                       │    based on    │ Classification:                      │         │
+│  │  • Has a reason code                                                    │                │  If dimension is :ignore             │         │
+│  │  • Marked as normative or informative                                         │                │    → INFORMATIVE diff                   │         │
+│  └──────────────────────────────────────────────────────────────────────────┘                │  If dimension is :strict or :normalize│        │
+│                                                                                               │    → NORMATIVE diff                     │         │
+│  Normative diff = Difference that affects the match result                                      └──────────────────────────────────────┘         │
+│  Informative diff = Difference that doesn't affect the match (semantically equivalent)                                                            │
+└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
+
+┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
+│ RENDERING LAYER: Text Display                                                                                                                   │
+│                                                                                                                                                  │
+│  Input: DiffNodes from comparison, original text documents                                                                                      │
+│  Output: Formatted diff with line numbers and markers                                                                                           │
+│                                                                                                                                                  │
+│  ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐           │
+│  │ STEP 1: Map Semantic Diffs to Text Lines                                                                                        │           │
+│  │                                                                                                                                  │           │
+│  │  DiffNodeMapper bridges semantic differences to text representation:                                                            │           │
+│  │   • Uses line-by-line text diff (Diff::LCS)                                                                                     │           │
+│  │   • Maps each changed line to its corresponding DiffNode                                                                        │           │
+│  │   • Creates DiffLine objects that link text ↔ semantics                                                                        │           │
+│  │                                                                                                                                  │           │
+│  │  DiffLine                                                                                                                        │           │
+│  │  ────────                                                                                                                        │           │
+│  │  • Line numbers in both documents                                                                                               │           │
+│  │  • Content of the line                                                                                                          │           │
+│  │  • Reference to DiffNode (may be nil for purely textual changes)                                                                │           │
+│  │  • Inherits normative/informative status from DiffNode                                                                                │           │
+│  └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘           │
+│                                              │                                                                                                  │
+│                                              ▼                                                                                                  │
+│  ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐           │
+│  │ STEP 2: Group Lines into Blocks                                                                                                 │           │
+│  │                                                                                                                                  │           │
+│  │  DiffBlock                                                                                                                       │           │
+│  │  ──────────                                                                                                                      │           │
+│  │  • Contiguous run of changed lines                                                                                              │           │
+│  │  • Contains one or more DiffLines                                                                                                │           │
+│  │  • Has reference to DiffNode if block maps to single semantic change                                                            │           │
+│  │  • Normative if any contained line is normative                                                                                       │           │
+│  └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘           │
+│                                              │                                                                                                  │
+│                                              ▼                                                                                                  │
+│  ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐           │
+│  │ STEP 3: Filter Based on Rendering Options                                                                                       │           │
+│  │                                                                                                                                  │           │
+│  │  show_diffs option controls which blocks are displayed:                                                                         │           │
+│  │   • :normative    → Show only blocks with normative differences                                                                       │           │
+│  │   • :informative  → Show only blocks with informative differences                                                                     │           │
+│  │   • :all       → Show all differences                                                                                           │           │
+│  └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘           │
+│                                              │                                                                                                  │
+│                                              ▼                                                                                                  │
+│  ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐           │
+│  │ STEP 4: Add Context and Group for Display                                                                                       │           │
+│  │                                                                                                                                  │           │
+│  │  DiffContext                                                                                                                     │           │
+│  │  ───────────                                                                                                                     │           │
+│  │  • One or more DiffBlocks grouped by proximity                                                                                  │           │
+│  │  • Includes surrounding unchanged lines for context                                                                             │           │
+│  │  • Controlled by context_lines and diff_grouping_lines options                                                                  │           │
+│  └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘           │
+│                                              │                                                                                                  │
+│                                              ▼                                                                                                  │
+│  ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐           │
+│  │ STEP 5: Format for Display                                                                                                      │           │
+│  │                                                                                                                                  │           │
+│  │  Normative differences:                        Informative differences:                                                               │           │
+│  │    18|  -  <div class="TOC" id="_">           18|  ~  <div class="TOC" id="_">                                                  │           │
+│  │    18|  +  <div id="_" class="TOC">           18|  ~  <div id="_" class="TOC">                                                  │           │
+│  │         ▲                                           ▲                                                                            │           │
+│  │     Red/Green markers                           Cyan marker                                                                     │           │
+│  │     (semantic difference)                       (textual difference only)                                                       │           │
+│  └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘           │
+└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
+----
+
+== Example: Attribute Order
+
+Consider comparing these two HTML elements:
+
+[source,html]
+----
+<!-- Document 1 -->
+<div class="TOC" id="_">
+
+<!-- Document 2 -->
+<div id="_" class="TOC">
+----
+
+=== Comparison Layer
+
+The comparator detects that attribute order differs and creates a `DiffNode`:
+
+* `dimension`: `:attribute_whitespace`
+* `reason`: `ATTRIBUTE_ORDER_DIFFERS`
+
+=== Classification
+
+The `DiffNode` is classified based on your match options:
+
+* If `attribute_whitespace: :strict` → **NORMATIVE** (order matters to you)
+** Display: `18| - <div class="TOC" id="_">` (red)
+** Display: `18| + <div id="_" class="TOC">` (green)
+
+* If `attribute_whitespace: :normalize` → **INFORMATIVE** (order doesn't matter)
+** Display: `18| ~ <div class="TOC" id="_">` (cyan)
+** Display: `18| ~ <div id="_" class="TOC">` (cyan)
+
+=== Rendering Options
+
+Control which diffs are displayed with `show_diffs`:
+
+* `:normative` → Skip this block (it's informative when normalized)
+* `:informative` → Show with `~` marker in cyan
+* `:all` → Show all diffs
+
+== Usage
+
+=== RSpec Matchers
+
+[source,ruby]
+----
+# Show only normative diffs (default)
+expect(actual).to be_html4_equivalent_to(expected, show_diffs: :normative)
+
+# Show only informative diffs
+expect(actual).to be_html4_equivalent_to(expected, show_diffs: :informative)
+
+# Show all diffs
+expect(actual).to be_html4_equivalent_to(expected, show_diffs: :all)
+----
+
+=== CLI
+
+[source,bash]
+----
+# Show only normative diffs (default)
+canon diff file1.html file2.html
+
+# Show informative diffs
+canon diff file1.html file2.html --show-diffs=informative
+
+# Show all diffs
+canon diff file1.html file2.html --show-diffs=all
+----
+
+== Implementation Components
+
+=== New Classes
+
+* `Canon::Diff::DiffNode` - Represents a semantic difference
+* `Canon::Diff::DiffLine` - Represents a changed text line
+* `Canon::Diff::DiffNodeMapper` - Bridges semantic and textual differences
+* `Canon::Diff::DiffClassifier` - Classifies diffs as normative/informative
+
+=== Enhanced Classes
+
+* `Canon::Diff::DiffBlock` - Add `diff_lines`, `diff_node`, `normative?`
+* `Canon::Diff::DiffContext` - Add `has_normative_diffs?`, `has_informative_diffs?`
+* `Canon::DiffFormatter` - Add `show_diffs` option
+* Line formatters - Support `~` marker and cyan color for informative diffs
+
+=== Comparators
+
+Update `XmlComparator`, `JsonComparator`, `YamlComparator` to:
+
+* Track dimension for each difference
+* Create `DiffNode` objects instead of simple hashes
+* Store differences with semantic context