canon 0.1.6 → 0.1.7

data/docs/features/diff-formatting/algorithm-specific-output.adoc

@@ -0,0 +1,533 @@
+---
+title: Algorithm-Specific Output
+parent: Diff Formatting
+grand_parent: Features
+nav_order: 2
+---
+= Algorithm-Specific Output
+
+== Purpose
+
+Different comparison algorithms produce fundamentally different types of output. Understanding these differences is essential for choosing the right diff mode and interpreting results correctly.
+
+This page explains the output formats from DOM and Semantic algorithms and provides guidance on choosing the appropriate diff mode for each.
+
+== Key Concept
+
+**Each algorithm has a natural output format**:
+
+* **DOM algorithm** generates line-based differences (natural fit: `by_line`)
+* **Semantic algorithm** generates operation-based differences (natural fit: `by_object`)
+
+Both algorithms support both diff modes, but the natural fit provides the most useful output.
+
+== DOM Algorithm Output
+
+=== Natural Output Format
+
+The DOM algorithm produces **line-based differences** showing positional changes:
+
+**Characteristics**:
+* Each line is compared at its position
+* Shows additions, deletions, and modifications
+* Traditional diff format (similar to `git diff`)
+* No move detection
+
+**Output Structure**:
+[source]
+----
+- <line removed at position>
++ <line added at position>
+  <unchanged line>
+  <unchanged line>
+- <another removed line>
++ <another added line>
+----
+
+=== DOM with `by_line` Mode (Natural Fit)
+
+This is the recommended mode for DOM algorithm.
+
+[source,ruby]
+----
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :dom,
+  diff_mode: :by_line,
+  verbose: true
+)
+
+puts result.diff
+----
+
+**Example Output**:
+[source,diff]
+----
+  <book>
+-   <title>Old Title</title>
++   <title>New Title</title>
+    <author>John Doe</author>
+-   <year>2020</year>
++   <year>2024</year>
+  </book>
+----
+
+**Best For**:
+* Code review workflows
+* Understanding line-by-line changes
+* Traditional diff tools integration
+* Quick visual scanning
+
+=== DOM with `by_object` Mode
+
+DOM algorithm can also produce tree-based output, though it's less natural.
+
+[source,ruby]
+----
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :dom,
+  diff_mode: :by_object,
+  verbose: true
+)
+----
+
+**Example Output**:
+[source]
+----
+book
+  title
+    - Old Title
+    + New Title
+  author
+    = John Doe
+  year
+    - 2020
+    + 2024
+----
+
+**Characteristics**:
+* Shows tree structure
+* Still position-based (no moves)
+* Can be useful for structured view
+* No semantic operations
+
+**Use When**:
+* You want tree structure visualization
+* Working with deeply nested documents
+* Need hierarchical context
+
+== Semantic Algorithm Output
+
+=== Natural Output Format
+
+The Semantic algorithm produces **operation-based differences** showing semantic changes:
+
+**Characteristics**:
+* Detects INSERT, DELETE, UPDATE, MOVE operations
+* Understands structural changes
+* Shows element paths
+* Provides operation statistics
+
+**Output Structure**:
+[source]
+----
+INSERT: <path> <content>
+DELETE: <path> <content>
+UPDATE: <path> <old> -> <new>
+MOVE:   <from-path> -> <to-path>
+----
+
+=== Semantic with `by_object` Mode (Natural Fit)
+
+This is the recommended mode for Semantic algorithm.
+
+[source,ruby]
+----
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  diff_mode: :by_object,
+  verbose: true
+)
+
+puts result.operations
+puts result.statistics
+----
+
+**Example Output**:
+[source]
+----
+UPDATE: book/title: "Old Title" -> "New Title"
+UPDATE: book/year: "2020" -> "2024"
+MOVE:   book/chapter[2] -> book/chapter[1]
+
+Statistics:
+  INSERT: 0
+  DELETE: 0
+  UPDATE: 2
+  MOVE:   1
+----
+
+**Best For**:
+* Understanding semantic changes
+* Tracking content evolution
+* Detecting restructuring
+* Operation-level analysis
+
+=== Semantic with `by_line` Mode
+
+Semantic algorithm can also produce line-based output for traditional workflows.
+
+[source,ruby]
+----
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  diff_mode: :by_line,
+  verbose: true
+)
+----
+
+**Example Output**:
+[source,diff]
+----
+  <book>
+~   <title>New Title</title>  [UPDATE]
+    <author>John Doe</author>
+~   <year>2024</year>         [UPDATE]
+→   <chapter id="2">          [MOVE from position 2]
+    <chapter id="1">
+  </book>
+----
+
+**Characteristics**:
+* Traditional line-based format
+* Annotated with operation types
+* Shows move indicators
+* Combines both worlds
+
+**Use When**:
+* Need traditional diff format
+* Want operation annotations
+* Integrating with line-based tools
+* Users familiar with git diff
+
+== Output Comparison Table
+
+[cols="2,3,3"]
+|===
+|Feature |DOM Output |Semantic Output
+
+|**Primary Format**
+|Line-based positional differences
+|Operation-based semantic changes
+
+|**Move Detection**
+|No (shows as DELETE + INSERT)
+|Yes (shows as MOVE operation)
+
+|**Operation Types**
+|Add, Remove, Modify
+|INSERT, DELETE, UPDATE, MOVE
+
+|**Best Diff Mode**
+|`by_line`
+|`by_object`
+
+|**Statistics**
+|Line counts
+|Operation counts
+
+|**Path Information**
+|Line numbers
+|Element paths
+
+|**Performance**
+|Fast
+|Slower
+
+|**Best For**
+|Traditional workflows
+|Semantic analysis
+|===
+
+== Choosing the Right Combination
+
+=== Recommended Configurations
+
+[cols="2,2,2,3"]
+|===
+|Algorithm |Diff Mode |Use Case |Output Type
+
+|DOM
+|by_line
+|Code review, quick diffs
+|Traditional line-based diff
+
+|DOM
+|by_object
+|Structured document view
+|Tree view without operations
+
+|Semantic
+|by_object
+|Semantic analysis
+|Operation-based tree diff
+
+|Semantic
+|by_line
+|Traditional format with operations
+|Annotated line-based diff
+|===
+
+== Detailed Examples
+
+=== Example 1: Simple Text Change
+
+**Input Documents**:
+[source,xml]
+----
+<!-- doc1.xml -->
+<message>Hello World</message>
+
+<!-- doc2.xml -->
+<message>Hello Universe</message>
+----
+
+**DOM by_line Output**:
+[source,diff]
+----
+- <message>Hello World</message>
++ <message>Hello Universe</message>
+----
+
+**Semantic by_object Output**:
+[source]
+----
+UPDATE: message: "Hello World" -> "Hello Universe"
+----
+
+=== Example 2: Element Reordering
+
+**Input Documents**:
+[source,xml]
+----
+<!-- doc1.xml -->
+<book>
+  <title>Canon</title>
+  <author>John</author>
+</book>
+
+<!-- doc2.xml -->
+<book>
+  <author>John</author>
+  <title>Canon</title>
+</book>
+----
+
+**DOM by_line Output**:
+[source,diff]
+----
+  <book>
+-   <title>Canon</title>
+    <author>John</author>
++   <title>Canon</title>
+  </book>
+----
+
+**Semantic by_object Output**:
+[source]
+----
+MOVE: book/title -> book/title (from position 1 to position 2)
+
+Statistics:
+  INSERT: 0
+  DELETE: 0
+  UPDATE: 0
+  MOVE:   1
+----
+
+=== Example 3: Complex Restructuring
+
+**Input Documents**:
+[source,xml]
+----
+<!-- doc1.xml -->
+<doc>
+  <section id="1">
+    <para>Text A</para>
+    <para>Text B</para>
+  </section>
+  <section id="2">
+    <para>Text C</para>
+  </section>
+</doc>
+
+<!-- doc2.xml -->
+<doc>
+  <section id="2">
+    <para>Text C</para>
+    <para>Text B</para>
+  </section>
+  <section id="1">
+    <para>Text A</para>
+  </section>
+</doc>
+----
+
+**DOM by_line Output** (shows many line changes):
+[source,diff]
+----
+  <doc>
+-   <section id="1">
+-     <para>Text A</para>
+-     <para>Text B</para>
+-   </section>
+    <section id="2">
+      <para>Text C</para>
++     <para>Text B</para>
+    </section>
++   <section id="1">
++     <para>Text A</para>
++   </section>
+  </doc>
+----
+
+**Semantic by_object Output** (shows semantic operations):
+[source]
+----
+MOVE: doc/section[@id='1'] -> doc/section[@id='1'] (position 1 to 2)
+MOVE: doc/section[@id='1']/para[2] -> doc/section[@id='2']/para[2]
+
+Statistics:
+  INSERT: 0
+  DELETE: 0
+  UPDATE: 0
+  MOVE:   2
+----
+
+== Accessing Algorithm-Specific Output
+
+=== Ruby API
+
+[source,ruby]
+----
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  diff_mode: :by_object,
+  verbose: true
+)
+
+# Check which algorithm was used
+case result.diff_algorithm
+when :dom
+  puts result.diff_lines    # Array of diff lines
+  puts result.line_count    # Statistics
+when :semantic
+  puts result.operations    # Array of operations
+  puts result.statistics    # Operation counts
+  puts result.tree_diff     # Tree representation
+end
+----
+
+=== Operation Objects (Semantic Only)
+
+[source,ruby]
+----
+result.operations.each do |op|
+  puts "Type: #{op.type}"           # :insert, :delete, :update, :move
+  puts "Path: #{op.path}"           # Element path
+  puts "Old: #{op.old_value}"       # For UPDATE
+  puts "New: #{op.new_value}"       # For UPDATE/INSERT
+  puts "From: #{op.from_path}"      # For MOVE
+  puts "To: #{op.to_path}"          # For MOVE
+end
+----
+
+== Performance Implications
+
+=== DOM Output Generation
+
+* **Speed**: Fast (linear with document size)
+* **Memory**: Low (line-by-line processing)
+* **Scaling**: Handles large documents well
+
+=== Semantic Output Generation
+
+* **Speed**: Slower (quadratic worst case)
+* **Memory**: Higher (tree structures in memory)
+* **Scaling**: Best for smaller documents (< 10KB)
+
+== Common Patterns
+
+=== Pattern 1: Quick Diff Review
+
+[source,ruby]
+----
+# Fast line-based diff
+Canon::Comparison.equivalent?(expected, actual,
+  diff_algorithm: :dom,
+  diff_mode: :by_line,
+  verbose: true,
+  use_color: true
+)
+----
+
+=== Pattern 2: Semantic Analysis
+
+[source,ruby]
+----
+# Detailed operation analysis
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  diff_mode: :by_object,
+  verbose: true
+)
+
+# Analyze operations
+puts "Total changes: #{result.statistics.total}"
+puts "Moves detected: #{result.statistics.moves}"
+----
+
+=== Pattern 3: Hybrid View
+
+[source,ruby]
+----
+# Use semantic algorithm but traditional output
+Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  diff_mode: :by_line,  # Traditional format
+  verbose: true,
+  use_color: true
+)
+# Get operation annotations in traditional diff
+----
+
+== Migration Guide
+
+=== Migrating Output Format
+
+When switching algorithms, update your output expectations:
+
+[source,ruby]
+----
+# Before: DOM algorithm
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :dom,
+  diff_mode: :by_line
+)
+# Expect: result.diff_lines
+
+# After: Semantic algorithm
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  diff_algorithm: :semantic,
+  diff_mode: :by_object
+)
+# Expect: result.operations, result.statistics
+----
+
+== See Also
+
+* link:index.adoc[Diff Formatting Overview]
+* link:diff-modes.adoc[Diff Modes] - by_line vs by_object details
+* link:../match-options/algorithm-specific-behavior.adoc[Algorithm-Specific Behavior] - How algorithms work
+* link:../../understanding/algorithms/[Algorithms] - Detailed algorithm documentation
+* link:colors-and-symbols.adoc[Colors and Symbols] - Visual formatting options
+* link:../../guides/choosing-configuration.adoc[Choosing Configuration] - Decision guide

data/docs/{CHARACTER_VISUALIZATION.adoc → features/diff-formatting/character-visualization.adoc}

@@ -1,42 +1,28 @@
 ---
-layout: default
 title: Character Visualization
-nav_order: 34
-parent: Customizing Behavior
+parent: Diff Formatting
+grand_parent: Features
+nav_order: 4
 ---
-= Canon character visualization
+= Character visualization
 :toc:
 :toclevels: 3
 
-== Scope
+== Purpose
 
-This document describes Canon's whitespace and special character visualization
-system, which makes invisible characters visible in diff output.
+Canon's character visualization system makes invisible characters (spaces, tabs, zero-width characters) visible in diff output, helping you quickly identify whitespace differences that cause test failures.
 
-For diff formatting options, see link:DIFF_FORMATTING[Diff formatting].
-
-== General
-
-When comparing documents, invisible characters like spaces, tabs, and
-zero-width characters can cause mysterious test failures. Canon's character
-visualization makes these characters visible in diff output, helping you
-quickly identify the exact difference.
-
-Visualization is **CJK-safe**, using Unicode symbols that don't conflict with
-Chinese, Japanese, or Korean text.
+Visualization is **CJK-safe**, using Unicode symbols that don't conflict with Chinese, Japanese, or Korean text.
 
 == When visualization is applied
 
-Character visualization is applied **only to diff lines** (additions,
-deletions, and changes), not to context lines (unchanged lines). This ensures:
+Character visualization is applied **only to diff lines** (additions, deletions, and changes), not to context lines (unchanged lines). This ensures:
 
 * Context lines display content in original form
 * Only actual changes show visualization
 * Differences are easier to spot
 
-Within changed lines showing token-level diffs, unchanged tokens are displayed
-in the terminal's default color (not red/green) to distinguish them from
-actual changes.
+Within changed lines showing token-level diffs, unchanged tokens are displayed in the terminal's default color (not red/green) to distinguish them from actual changes.
 
 == Default character map
 
@@ -261,8 +247,7 @@ Canon provides a comprehensive CJK-safe character mapping.
 
 == CJK safety
 
-The visualization characters are specifically chosen to avoid conflicts with
-CJK text:
+The visualization characters are specifically chosen to avoid conflicts with CJK text:
 
 **Avoided characters**:
 
@@ -290,8 +275,7 @@ CJK text:
     |  10+| <tag>░Value</tag>           # Space added (green light shade)
 ----
 
-The `░` symbol clearly shows a regular space was added between `<tag>` and
-`Value`.
+The `░` symbol clearly shows a regular space was added between `<tag>` and `Value`.
 ====
 
 === Tab vs spaces
@@ -305,8 +289,7 @@ The `░` symbol clearly shows a regular space was added between `<tag>` and
     |  15+| <tag>░░Value</tag>          # Two spaces (green light shades)
 ----
 
-The difference between a tab (`⇥`) and two spaces (`░░`) is immediately
-visible.
+The difference between a tab (`⇥`) and two spaces (`░░`) is immediately visible.
 ====
 
 === Non-breaking space
@@ -330,8 +313,7 @@ With visualization:
     |   4+| <foreword␣id="fwd">          # Non-breaking space (U+00A0)
 ----
 
-The different symbols (`░` vs `␣`) clearly show that one uses a regular space
-while the other uses a non-breaking space, likely from copying from a web page.
+The different symbols (`░` vs `␣`) clearly show that one uses a regular space while the other uses a non-breaking space, likely from copying from a web page.
 ====
 
 === Zero-width space
@@ -358,26 +340,11 @@ The diff shows:
 The rightwards arrow (`→`) reveals the presence of a zero-width space.
 ====
 
-=== Mixed invisible characters
-
-.Multiple whitespace types
-[example]
-====
-[source]
-----
-  30|     -| <p>Text▬more</p>           # Em space (red black rectangle)
-    |  30+| <p>Text░more</p>            # Regular space (green light shade)
-----
-
-Different space types shown with different symbols.
-====
-
 == Real-world scenarios
 
 === Web copy-paste
 
-**Problem**: Text copied from web pages often contains non-breaking spaces
-(U+00A0) instead of regular spaces.
+**Problem**: Text copied from web pages often contains non-breaking spaces (U+00A0) instead of regular spaces.
 
 .Detection example
 [example]
@@ -393,8 +360,7 @@ The `␣` symbol immediately identifies the non-breaking space.
 
 === Smart quotes
 
-**Problem**: Text editors may automatically convert straight quotes to curly
-quotes.
+**Problem**: Text editors may automatically convert straight quotes to curly quotes.
 
 .Detection example
 [example]
@@ -446,9 +412,6 @@ formatter = Canon::DiffFormatter.new(
   use_color: true,
   visualization_map: custom_map
 )
-
-# The custom map merges with defaults, so unspecified
-# characters still use the default visualization
 ----
 
 === When to customize
@@ -468,8 +431,7 @@ formatter = Canon::DiffFormatter.new(
 
 == Configuration
 
-Character visualization is automatically enabled when `use_color: true` and
-applies across all Canon interfaces.
+Character visualization is automatically enabled when `use_color: true` and applies across all Canon interfaces.
 
 === Enabling/disabling
 
@@ -511,10 +473,10 @@ Canon::Comparison.equivalent?(doc1, doc2,
 [source,bash]
 ----
 # Enable (default)
-$ canon diff file1.xml file2.xml --verbose
+canon diff file1.xml file2.xml --verbose
 
 # Disable
-$ canon diff file1.xml file2.xml --no-color --verbose
+canon diff file1.xml file2.xml --no-color --verbose
 ----
 ====
 
@@ -556,12 +518,11 @@ end
 
 **Problem**: Visualization conflicts with CJK text.
 
-**Solution**: Canon's defaults are CJK-safe. If using custom map, avoid the
-characters listed in "CJK safety" section.
+**Solution**: Canon's defaults are CJK-safe. If using custom map, avoid the characters listed in "CJK safety" section.
 
 == See also
 
-* link:DIFF_FORMATTING[Diff formatting]
-* link:MODES[Diff modes]
-* link:MATCH_ARCHITECTURE[Match architecture]
-* link:RUBY_API[Ruby API documentation]
+* link:index.adoc[Diff Formatting] - Overview of formatting options
+* link:colors-and-symbols.adoc[Colors and Symbols] - Color scheme details
+* link:../../interfaces/cli/index.adoc[CLI Interface] - Command-line usage
+* link:../../interfaces/ruby-api/index.adoc[Ruby API] - Programmatic usage