canon 0.1.6 → 0.1.7

data/README.adoc

@@ -49,11 +49,17 @@ require 'canon'
 
 # Canonical form (compact)
 Canon.format('<root><b>2</b><a>1</a></root>', :xml)
-# => "<root><a>1</a><b>2</b></root>"
+# => Pretty-printed XML (default behavior)
 
-# Pretty-print (human-readable)
+# Compact canonical form
+require 'canon/xml/c14n'
+Canon::Xml::C14n.canonicalize('<root><b>2</b><a>1</a></root>', with_comments: false)
+# => "<root><b>2</b><a>1</a></root>"
+
+# Pretty-print (human-readable with custom indent)
 require 'canon/pretty_printer/xml'
-Canon::Xml::PrettyPrinter.new(indent: 2).format(xml_input)
+xml_input = '<root><b>2</b><a>1</a></root>'
+Canon::PrettyPrinter::Xml.new(indent: 2).format(xml_input)
 ----
 
 === Compare documents
@@ -67,6 +73,13 @@ xml2 = '<root>  <b>2</b>  <a>1</a>  </root>'
 
 Canon::Comparison.equivalent?(xml1, xml2)
 # => true (semantically equivalent despite formatting differences)
+
+# Use semantic tree diff for operation-level analysis
+result = Canon::Comparison.equivalent?(xml1, xml2,
+  verbose: true,
+  diff_algorithm: :semantic
+)
+result.operations  # => [INSERT, DELETE, UPDATE, MOVE operations]
 ----
 
 === Use in tests
@@ -117,6 +130,10 @@ $ canon help
   options
 * **link:docs/MATCH_OPTIONS[Match options]** - Match dimensions and
   profiles
+* **link:docs/TREE_DIFF[Semantic tree diff]** - Operation-level tree
+  comparison
+* **link:docs/SEMANTIC_TREE_DIFF[Semantic tree diff algorithm]** - Comprehensive guide to semantic diff
+* **link:docs/ENV_CONFIG[Environment configuration]** - Configure via ENV variables including size limits
 * **link:docs/DIFF_FORMATTING[Diff formatting]** - Customizing diff output
 * **link:docs/CHARACTER_VISUALIZATION[Character visualization]** -
   Whitespace and special characters
@@ -131,6 +148,7 @@ $ canon help
   classification
 * **link:docs/DIFF_ARCHITECTURE[Diff architecture]** - Technical pipeline
   details
+* **link:docs/COMPARE_PROFILE[CompareProfile architecture]** - Format-specific policies
 
 == Features
 
@@ -152,12 +170,182 @@ Compare documents based on meaning, not formatting:
 
 * Whitespace normalization options
 * Attribute/key order handling
-* Comment handling
+* Comment handling with display control
 * Multiple match dimensions with behaviors
 * Predefined match profiles (strict, rendered, spec_friendly, content_only)
 
 See link:docs/MATCH_OPTIONS[Match options] for details.
 
+==== Comment display control
+
+Control which differences are displayed in diff output:
+
+[source,ruby]
+----
+# Show all differences (default)
+result = Canon::Comparison.equivalent?(xml1, xml2,
+  verbose: true,
+  match: { comments: :ignore },
+  show_diffs: :all
+)
+
+# Show only normative differences (affect equivalence)
+result = Canon::Comparison.equivalent?(xml1, xml2,
+  verbose: true,
+  match: { comments: :ignore },
+  show_diffs: :normative
+)
+
+# Show only informative differences
+result = Canon::Comparison.equivalent?(xml1, xml2,
+  verbose: true,
+  match: { comments: :ignore },
+  show_diffs: :informative
+)
+----
+
+**CLI usage:**
+[source,bash]
+----
+# Show all differences
+$ canon diff file1.xml file2.xml --show-diffs all
+
+# Show only normative differences
+$ canon diff file1.xml file2.xml --show-diffs normative
+
+# Show only informative differences
+$ canon diff file1.xml file2.xml --show-diffs informative
+----
+
+**RSpec usage:**
+[source,ruby]
+----
+expect(actual).to be_xml_equivalent_to(expected)
+  .show_diffs(:normative)
+----
+
+=== Original input string display
+
+When debugging test failures, it's often helpful to see the exact strings that
+were passed to the comparison before any preprocessing or normalization. The
+`verbose_diff` option displays the original input strings in an RSpec-style
+format with line numbers.
+
+[source,ruby]
+----
+# Enable original string display in configuration
+Canon::Config.configure do |config|
+  config.xml.diff.verbose_diff = true
+end
+
+# Or programmatically for a specific comparison
+result = Canon::Comparison.equivalent?(xml1, xml2,
+  verbose: true,
+  verbose_diff: true
+)
+----
+
+**Output format:**
+----
+==================================================================
+  ORIGINAL INPUT STRINGS
+==================================================================
+
+Expected (as string):
+     1 | <root>
+     2 |   <element>value1</element>
+     3 | </root>
+
+Actual (as string):
+     1 | <root>
+     2 |   <element>value2</element>
+     3 | </root>
+
+==================================================================
+----
+
+**When to use this feature:**
+
+* Debugging why two documents are considered different
+* Understanding preprocessing effects (c14n, normalization, etc.)
+* Verifying the exact input received by the comparison
+* Comparing raw vs processed content
+
+**Environment variable:**
+[source,bash]
+----
+export CANON_XML_DIFF_VERBOSE_DIFF=true
+export CANON_HTML_DIFF_VERBOSE_DIFF=true
+export CANON_JSON_DIFF_VERBOSE_DIFF=true
+export CANON_YAML_DIFF_VERBOSE_DIFF=true
+----
+
+=== Algorithm choice
+
+Canon provides two diff algorithms:
+
+* **DOM diff** (default): Stable, position-based comparison for traditional line-by-line output
+* **Semantic tree diff** (experimental): Advanced operation detection (INSERT, DELETE, UPDATE, MOVE, MERGE, SPLIT, UPGRADE, DOWNGRADE)
+
+[source,ruby]
+----
+# Use DOM diff (default, stable)
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  verbose: true,
+  diff_algorithm: :dom
+)
+
+# Use semantic tree diff (experimental, more intelligent)
+result = Canon::Comparison.equivalent?(doc1, doc2,
+  verbose: true,
+  diff_algorithm: :semantic
+)
+----
+
+**When to use semantic tree diff:**
+
+* Need to detect high-level operations (moves, merges, splits)
+* Documents have significant rearrangement
+* Want statistical analysis of changes
+* Need operation-level transformation analysis
+
+**When to use DOM diff:**
+
+* Need stable, well-tested comparison
+* Want traditional line-by-line output
+* Documents are similar in structure
+* Maximum performance for large files
+
+See link:docs/SEMANTIC_TREE_DIFF[Semantic tree diff algorithm] for comprehensive guide.
+
+=== Size limits for large files
+
+Canon provides configurable size limits to prevent hangs on pathologically large files:
+
+* **File size limit**: Default 5MB (configurable)
+* **Node count limit**: Default 10,000 nodes (configurable)
+* **Diff output limit**: Default 10,000 lines (configurable)
+
+[source,bash]
+----
+# Configure via environment variables
+export CANON_MAX_FILE_SIZE=10485760      # 10MB
+export CANON_MAX_NODE_COUNT=50000        # 50,000 nodes
+export CANON_MAX_DIFF_LINES=20000        # 20,000 lines
+
+bundle exec rspec
+----
+
+[source,ruby]
+----
+# Or programmatically
+Canon::Config.instance.xml.diff.max_file_size = 10_485_760
+Canon::Config.instance.xml.diff.max_node_count = 50_000
+Canon::Config.instance.xml.diff.max_diff_lines = 20_000
+----
+
+See link:docs/ENV_CONFIG#size-limits[ENV_CONFIG] for details on size limit configuration.
+
 === Smart diff output
 
 **By-line mode**: Traditional line-by-line diff with:
@@ -177,8 +365,12 @@ See link:docs/MODES[Diff modes] for details.
 
 === Enhanced diff features
 
-* **Color-coded output**: Red (normative deletions), green (normative additions), yellow
-  (normative structure), cyan (informative diffs)
+* **Three-tier diff classification**: Formatting-only (`[` dark gray/`]` light gray), informative (`<` blue/`>` cyan), and normative (`-` red/`+` green) differences with directional colors
+* **Directional color coding**: Removals and additions use different colors within each tier (red/green for normative, blue/cyan for informative, dark gray/light gray for formatting)
+* **Namespace declaration tracking**: Separate dimension for tracking `xmlns` and `xmlns:*` attribute changes, reported independently from regular data attributes
+* **Namespace rendering**: Explicit namespace display in XML diffs using `ns:[uri]` or `ns:[{blank}]` format
+* **Informative diff visualization**: Visually distinct blue/cyan markers for differences that don't affect equivalence
+* **Formatting diff detection**: Automatically detects and highlights purely cosmetic whitespace/line break differences
 * **Whitespace visualization**: Make invisible characters visible with CJK-safe
   Unicode symbols
 * **Non-ASCII detection**: Warnings for unexpected Unicode characters
@@ -233,7 +425,7 @@ See link:docs/CLI[CLI documentation].
 [source,ruby]
 ----
 # Configure globally
-Canon::RSpecMatchers.configure do |config|
+Canon::Config.configure do |config|
   config.xml.match.profile = :spec_friendly
   config.xml.diff.use_color = true
 end
@@ -267,6 +459,207 @@ delegation to mode-specific formatters (by-line, by-object).
 
 See link:docs/MATCH_ARCHITECTURE[Match architecture] for details.
 
+=== CompareProfile architecture
+
+Canon uses the **CompareProfile** class to encapsulate policy decisions about how differences in various dimensions should be handled during comparison. This provides clean separation of concerns between policy decisions, comparison logic, and difference classification.
+
+==== Separation of concerns
+
+The comparison system is divided into four distinct components:
+
+**CompareProfile**:: Policy decisions (what to track, what affects equivalence)
+**XmlComparator/HtmlComparator**:: Comparison logic (detect differences)
+**DiffNode**:: Data representation (represents a difference)
+**DiffClassifier**:: Classification logic (normative vs informative vs formatting)
+
+Each component has ONE responsibility with no overlapping concerns:
+
+* CompareProfile does NOT classify differences
+* XmlComparator does NOT make policy decisions
+* DiffClassifier does NOT compare documents
+
+==== Policy methods
+
+CompareProfile provides four key policy methods:
+
+`track_dimension?(dimension)`:: Should DiffNodes be created for this dimension? Returns `true` in verbose mode to track all differences for reporting.
+
+`affects_equivalence?(dimension)`:: Should differences affect equivalence? Determines the return value of the comparison.
+Returns `false` for dimensions with `:ignore` behavior.
+
+`normative_dimension?(dimension)`:: Is this dimension normative (affects equivalence) or informative (display only)?
+Used by DiffClassifier to set the normative flag on DiffNodes.
+
+`supports_formatting_detection?(dimension)`:: Can FormattingDetector apply to this dimension?
+Returns `true` only for text/content dimensions (`:text_content`, `:structural_whitespace`, `:comments`).
+
+=== CompareProfile architecture
+
+Canon uses a `CompareProfile` system to define format-specific comparison policies.
+This allows different formats (HTML, XML, JSON, YAML) to have their own default
+behaviors while maintaining a consistent architecture.
+
+==== How CompareProfile works
+
+The `CompareProfile` class provides the foundation for policy-based comparison:
+
+**Normative policy**: Determines what differences matter for equivalence. Each
+dimension (`:text_content`, `:structural_whitespace`, `:comments`, etc.) has a
+behavior (`:strict`, `:normalize`, `:ignore`) that determines whether differences
+in that dimension affect equivalence.
+
+**Dimension-based classification**: Each difference has a dimension and the
+profile determines if that dimension is:
+
+* **Normative**: Affects equivalence (documents not equivalent if different)
+* **Informative**: Tracked but doesn't affect equivalence
+* **Formatting-only**: Pure whitespace differences when normalized content matches
+
+**Classification hierarchy**:
+
+1. **Normative** (highest priority): Differences that make documents non-equivalent
+2. ** Informative** (medium priority): Differences that are tracked but don't affect equivalence
+3. **Formatting-only** (lowest priority): Pure whitespace/formatting differences
+
+==== Dimension behaviors
+
+Each dimension can have one of three behaviors:
+
+* **`:strict`**: Differences in this dimension are normative (affect equivalence)
+* **`:normalize`**: Differences are normalized; only semantic changes are normative
+* **`:ignore`**: Differences are informative only (don't affect equivalence)
+
+.Example: Whitespace handling
+[example]
+====
+----
+# Default (strict mode): whitespace differences are normative
+xml1 = '<root><p>Hello world</p></root>'
+xml2 = '<root><p>Hello\nworld</p></root>'
+Canon::Comparison.equivalent?(xml1, xml2)  # => false
+
+# Normalize mode: whitespace-only differences are formatting-only
+Canon::Comparison.equivalent?(xml1, xml2,
+  match: { text_content: :normalize, structural_whitespace: :normalize }
+)  # => true
+----
+====
+
+In normalize mode, the line break is detected as formatting-only because the
+normalized content ("Hello world") is the same.
+
+==== Format-specific profiles
+
+Different formats can extend `CompareProfile` with format-specific policies:
+
+* **XML** (base): Strict policies for all dimensions
+* **HTML** (HtmlCompareProfile): Comments ignored by default, whitespace preserved in certain elements
+* **JSON/YAML** (future): Key order policies, type handling
+
+See `lib/canon/comparison/compare_profile.rb` for the base implementation and
+`lib/canon/comparison/html_compare_profile.rb` for HTML-specific policies.
+
+==== Format-specific policies for HTML
+
+Canon provides a format-specific CompareProfile implementation called
+HtmlCompareProfile that encapsulates policies specific to HTML comparison.
+This profile is automatically used by HtmlComparator based on detected
+HTML version.
+
+**Comments**: Default behavior is `:ignore` (presentational content in HTML),
+unless explicitly set to `:strict`. When comments are set to `:strict`,
+they will affect equivalence.
+
+**Whitespace preservation**: HtmlCompareProfile automatically preserves
+whitespace in elements where it's semantically significant (e.g., `<pre>`,
+`<code>`, `<textarea>`, `<script>`, `<style>`). In other elements, whitespace
+is normalized.
+
+**Case sensitivity**: HTML5 is case-sensitive for element names, while
+HTML4 is case-insensitive. HtmlCompareProfile uses HTML5 case-sensitivity
+by default.
+
+
+==== Usage example
+
+When using `match: { comments: :ignore }`:
+
+* `track_dimension?(:comments)` returns `true` (track in verbose mode)
+* `affects_equivalence?(:comments)` returns `false` (doesn't affect equivalence)
+* `normative_dimension?(:comments)` returns `false` (informative only)
+
+This ensures that comment differences are tracked and displayed in verbose mode
+but don't make documents non-equivalent.
+
+.Example: Comment differences with :ignore behavior
+====
+[source,ruby]
+----
+xml1 = '<root><!-- comment 1 --><data>value</data></root>'
+xml2 = '<root><!-- comment 2 --><data>value</data></root>'
+
+result = Canon::Comparison.equivalent?(xml1, xml2,
+  verbose: true,
+  match: { comments: :ignore }
+)
+
+result.differences           # => [#<DiffNode @dimension=:comments>]
+result.differences[0].normative?  # => false (informative)
+result.equivalent?           # => true (doesn't affect equivalence)
+----
+
+The comment difference is tracked and displayed, but the documents are still
+considered equivalent because comments are set to `:ignore`.
+====
+
+.Example: HTML comment handling
+====
+[source,ruby]
+----
+html1 = '<div><!-- comment --><p>Text</p></div>'
+html2 = '<div><p>Text</p></div>'
+
+# HTML defaults: comments are ignored (presentational)
+result = Canon::Comparison.equivalent?(html1, html2)
+# => true (comments don't affect HTML equivalence by default)
+
+# Explicit strict matching
+result = Canon::Comparison.equivalent?(html1, html2,
+  match: { comments: :strict }
+)
+# => false (comments now affect equivalence)
+----
+
+Comments in HTML are considered presentational content (like CSS styles) and
+don't affect the semantic meaning unless explicitly configured to `:strict`.
+====
+
+.Example: HTML whitespace preservation
+====
+[source,ruby]
+----
+html1 = '<pre>Line 1\n  Line 2</pre>'
+html2 = '<pre>Line 1\nLine 2</pre>'
+
+# Whitespace is preserved in <pre> elements
+result = Canon::Comparison.equivalent?(html1, html2)
+# => false (whitespace differs in pre element)
+
+# But normalized in other elements
+html3 = '<div>Text    with    spaces</div>'
+html4 = '<div>Text with spaces</div>'
+result = Canon::Comparison.equivalent?(html3, html4)
+# => true (whitespace normalized in regular elements)
+----
+
+HtmlCompareProfile automatically preserves whitespace in elements where it's
+semantically significant (`<pre>`, `<code>`, `<textarea>`, `<script>`,
+`<style>`), while normalizing it in other elements.
+====
+
+**Future format profiles**: The architecture supports additional format-specific
+profiles for JSON, YAML, and other formats as needed.
+
 == Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then run

data/docs/Gemfile

@@ -0,0 +1,9 @@
+source "https://rubygems.org"
+
+gem "jekyll", "~> 4.3"
+gem "jekyll-asciidoc"
+gem "just-the-docs"
+
+group :jekyll_plugins do
+  gem "jekyll-seo-tag"
+end