canon 0.1.3 → 0.1.5

data/docs/PREPROCESSING.adoc

@@ -0,0 +1,491 @@
+---
+layout: default
+title: Preprocessing
+nav_order: 31
+parent: Customizing Behavior
+---
+= Canon preprocessing options
+:toc:
+:toclevels: 3
+
+== Scope
+
+This document describes Canon's preprocessing options that transform documents
+before comparison. Preprocessing is Phase 1 of Canon's three-phase comparison
+architecture.
+
+For the complete architecture, see link:MATCH_ARCHITECTURE[Match
+architecture].
+
+== General
+
+Preprocessing transforms documents into a normalized form before semantic
+matching. This eliminates format-specific variations that should not affect
+semantic equivalence.
+
+Preprocessing is optional and configured per-comparison. The default is
+`none` (no preprocessing).
+
+== Architecture context
+
+Preprocessing is Phase 1 in Canon's comparison flow:
+
+[source]
+----
+Phase 1: PREPROCESSING → Phase 2: MATCHING → Phase 3: RENDERING
+----
+
+Documents are preprocessed identically before comparison, ensuring consistent
+input to the semantic matching phase.
+
+== Preprocessing options
+
+=== none (default)
+
+**Purpose**: No preprocessing - compare documents exactly as provided.
+
+**When to use**:
+
+* Documents are already in comparable form
+* You want to detect any differences, including formatting
+* Testing exact output from generators
+* Maximum strictness required
+
+**Behavior**:
+
+* Documents passed directly to comparison
+* No normalization applied
+* All whitespace preserved
+* All formatting preserved
+
+.none example
+[example]
+====
+[source,ruby]
+----
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :none  # or omit (default)
+)
+----
+
+Documents compared exactly as-is.
+====
+
+=== c14n
+
+**Purpose**: Apply canonical form according to format-specific rules.
+
+**When to use**:
+
+* Eliminate all formatting differences
+* Focus purely on semantic content
+* Compare documents from different sources
+* Maximum normalization needed
+
+**Behavior per format**:
+
+**XML**::
+* W3C Canonical XML Version 1.1
+* Namespace prefixes sorted
+* Attributes sorted by namespace URI then local name
+* Whitespace normalized
+* Comments removed (unless `--with-comments`)
+
+**HTML**::
+* Normalized HTML structure
+* Attributes sorted
+* Whitespace collapsed per CSS rules
+* Empty text nodes removed
+
+**JSON**::
+* Keys sorted alphabetically at all levels
+* Whitespace normalized
+* Consistent formatting
+
+**YAML**::
+* Keys sorted alphabetically
+* Standard YAML 1.2 format
+* Consistent indentation
+
+.c14n examples
+[example]
+====
+[source,ruby]
+----
+# Ruby API
+Canon::Comparison.equivalent?(xml1, xml2,
+  preprocessing: :c14n
+)
+
+# CLI
+$ canon diff file1.xml file2.xml --preprocessing c14n --verbose
+----
+
+**XML before c14n**:
+
+[source,xml]
+----
+<root xmlns:b="http://b.com" xmlns:a="http://a.com">
+  <item b:attr="2" a:attr="1">
+    Text
+  </item>
+</root>
+----
+
+**After c14n**:
+
+[source,xml]
+----
+<root xmlns:a="http://a.com" xmlns:b="http://b.com"><item a:attr="1" b:attr="2">Text</item></root>
+----
+
+Namespaces and attributes sorted, whitespace removed.
+====
+
+=== normalize
+
+**Purpose**: Normalize whitespace while preserving structure.
+
+**When to use**:
+
+* Ignore whitespace differences but preserve element/key order
+* Compare documents with different pretty-printing
+* Focus on content with flexible whitespace handling
+* Middle ground between `none` and `c14n`
+
+**Behavior**:
+
+* Collapse multiple whitespace to single space
+* Trim leading/trailing whitespace
+* Normalize line endings (LF)
+* Preserve element/attribute/key order
+* Preserve structural whitespace
+
+.normalize examples
+[example]
+====
+[source,ruby]
+----
+# Ruby API
+Canon::Comparison.equivalent?(xml1, xml2,
+  preprocessing: :normalize
+)
+
+# CLI
+$ canon diff file1.xml file2.xml --preprocessing normalize --verbose
+----
+
+**Before normalize**:
+
+[source,xml]
+----
+<root>
+  <item>  Text   with    spaces  </item>
+</root>
+----
+
+**After normalize**:
+
+[source,xml]
+----
+<root>
+<item>Text with spaces</item>
+</root>
+----
+
+Whitespace collapsed and trimmed, structure preserved.
+====
+
+=== format
+
+**Purpose**: Pretty-print documents with standard formatting.
+
+**When to use**:
+
+* Compare documents where both should be well-formatted
+* Ensure consistent indentation before comparison
+* Generate readable diff output
+* Standardize formatting across sources
+
+**Behavior**:
+
+* 2-space indentation (default)
+* One element/property per line
+* Consistent structure
+* Attributes/keys in canonical order
+
+.format examples
+[example]
+====
+[source,ruby]
+----
+# Ruby API
+Canon::Comparison.equivalent?(xml1, xml2,
+  preprocessing: :format
+)
+
+# CLI
+$ canon diff file1.xml file2.xml --preprocessing format --verbose
+----
+
+**Before format**:
+
+[source,xml]
+----
+<root><a>1</a><b>2</b></root>
+----
+
+**After format**:
+
+[source,xml]
+----
+<root>
+  <a>1</a>
+  <b>2</b>
+</root>
+----
+
+Consistent 2-space indentation applied.
+====
+
+== Comparison table
+
+[cols="1,1,1,1,1"]
+|===
+|Aspect |none |c14n |normalize |format
+
+|**Whitespace**
+|Preserved
+|Removed
+|Collapsed
+|Standardized
+
+|**Element order**
+|Preserved
+|May change (XML)
+|Preserved
+|Preserved
+
+|**Attribute/key order**
+|Preserved
+|Sorted
+|Preserved
+|Sorted
+
+|**Formatting**
+|Preserved
+|Compact
+|Minimal
+|Pretty-printed
+
+|**Use case**
+|Exact match
+|Maximum normalization
+|Flexible whitespace
+|Consistent formatting
+|===
+
+== Choosing preprocessing options
+
+=== Decision guide
+
+**Use `none` when**:
+
+* Testing exact generator output
+* Formatting matters
+* Maximum strictness needed
+* Documents should be identical
+
+**Use `c14n` when**:
+
+* Comparing from different sources
+* Formatting doesn't matter at all
+* Maximum normalization needed
+* Focus purely on semantic content
+
+**Use `normalize` when**:
+
+* Whitespace differences should be ignored
+* Want to preserve element/key order
+* Middle ground between strict and normalized
+* Comparing hand-written vs generated content
+
+**Use `format` when**:
+
+* Both documents should be well-formatted
+* Want readable diff output
+* Standardizing before comparison
+* Pretty-printing for review
+
+=== Common scenarios
+
+.Configuration file comparison
+[example]
+====
+**Scenario**: Compare JSON config files from different environments.
+
+**Recommendation**: `c14n`
+
+**Reason**: Key order and whitespace don't matter; focus on values.
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(prod_config, dev_config,
+  preprocessing: :c14n
+)
+----
+====
+
+.HTML output testing
+[example]
+====
+**Scenario**: Test HTML generator output against expected fixture.
+
+**Recommendation**: `normalize`
+
+**Reason**: Ignore whitespace differences but preserve structure.
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(generated_html, expected_html,
+  preprocessing: :normalize
+)
+----
+====
+
+.XML document comparison
+[example]
+====
+**Scenario**: Compare XML from manual edits vs programmatic generation.
+
+**Recommendation**: `format`
+
+**Reason**: Standardize formatting for readable diff.
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(manual_xml, generated_xml,
+  preprocessing: :format,
+  verbose: true
+)
+----
+====
+
+.Exact output verification
+[example]
+====
+**Scenario**: Verify serializer produces exactly correct output.
+
+**Recommendation**: `none`
+
+**Reason**: Formatting and whitespace matter.
+
+[source,ruby]
+----
+Canon::Comparison.equivalent?(serialized, expected,
+  preprocessing: :none  # or omit
+)
+----
+====
+
+== Combining with match options
+
+Preprocessing works in combination with match options (Phase 2):
+
+.Preprocessing + match options
+[example]
+====
+[source,ruby]
+----
+# Normalize whitespace in preprocessing,
+# then ignore comments in matching
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :normalize,
+  match: {
+    comments: :ignore
+  }
+)
+
+# Format for consistency,
+# then strict matching on content
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :format,
+  match: {
+    text_content: :strict,
+    structural_whitespace: :strict
+  }
+)
+----
+
+Preprocessing transforms, then matching compares.
+====
+
+== Usage across interfaces
+
+=== Ruby API
+
+[source,ruby]
+----
+# Basic usage
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :c14n
+)
+
+# With match options
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :normalize,
+  match: { comments: :ignore }
+)
+
+# With diff options
+Canon::Comparison.equivalent?(doc1, doc2,
+  preprocessing: :format,
+  verbose: true,
+  diff: { mode: :by_line }
+)
+----
+
+=== CLI
+
+[source,bash]
+----
+# Basic usage
+$ canon diff file1.xml file2.xml --preprocessing c14n --verbose
+
+# With match options
+$ canon diff file1.xml file2.xml \
+  --preprocessing normalize \
+  --comments ignore \
+  --verbose
+
+# Format before comparison
+$ canon diff file1.json file2.json \
+  --preprocessing format \
+  --verbose
+----
+
+=== RSpec
+
+Preprocessing is not typically configured globally for RSpec since it's
+usually task-specific. Use match options for global configuration instead.
+
+[source,ruby]
+----
+# Per-test preprocessing
+it 'compares with c14n preprocessing' do
+  # Note: RSpec matchers don't directly support preprocessing parameter
+  # Use Canon::Comparison.equivalent? directly instead
+  result = Canon::Comparison.equivalent?(actual, expected,
+    preprocessing: :c14n
+  )
+  expect(result).to be true
+end
+----
+
+== See also
+
+* link:MATCH_ARCHITECTURE[Match architecture]
+* link:MATCH_OPTIONS[Match options]
+* link:FORMATS[Format support]
+* link:RUBY_API[Ruby API documentation]
+* link:CLI[Command-line interface]