canon 0.3.100 → 0.3.101

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 47ebab684f05c7445139ea4b0ed5133ab734c1d16918f3dadc055b3ddbc5692f
-  data.tar.gz: d80c1ae4a66f43ffb92c3eb8a14ebca8e2bcd2eb373130797a71e7f79a748d9a
+  metadata.gz: dc66e03ef02db4a7cbb21e496076e2794027a7a52996eb797b90b697c77013ee
+  data.tar.gz: 6ab70c76adfd5fe466b0fa8a54f542097344f4726c34192b1f85f59a542b5752
 SHA512:
-  metadata.gz: 3555261e4b8280e6967bd63900dec75bb19d2225ff2da16748136ee7ddcb076fcee4f04b6502801afd92d344ef67b578c692868e63c3c181dff8008efc472986
-  data.tar.gz: 590b8d12f49526bd3965d78f2000b4d7e64452e0b026547285d0762f6fb0565311ffb00c595aaac85813c90658656842aca1a8970e25f88f20e4f6d09e6ddeb8
+  metadata.gz: efc3813ef9fa91be57ddd8fa537506cb938361e5c582f4bdc06ac272c7ffd299b92172c9bf41cf194f1a5d0c50237b82c8a01de801f652545dc51693ed96f08b
+  data.tar.gz: a867f50610cb84caa68f469561f9bd5cc15ec49cb4732543fd76efbca201787520245f013edca9454349e7a164be73f8f884bc503458f007f7ee7206bd5fedc9

data/CLAUDE.md

@@ -136,7 +136,7 @@ Canon is engine-agnostic across three seams (MECE — one concern per module):
 
 - `Canon::XmlBackend` — XML engine selection. The default follows moxml's resolved adapter: **leptris whenever it is installed** (parse ~1.6x, serialize ~3.8x vs Nokogiri), raw Nokogiri otherwise (wrapping Nokogiri in moxml buys nothing — the wrapper adds 2-3x overhead). `:moxml` under Opal. `CANON_XML_BACKEND=nokogiri|moxml` forces either engine.
 - `Canon::XmlParsing` — the only place that talks to engines for XML parse/serialize (moxml parses pass `readonly: true` — canon never mutates engine documents); node type queries answer for ANY recognized node (Nokogiri or moxml) by type, never by active backend — user-supplied Nokogiri nodes keep working under the moxml engine.
-- `Canon::Html::NokogiriSupport` — HTML engine seam. **HTML comparison parses through the engine** since 0.3.86: `HtmlParser.parse` routes through moxml `Context#parse_html_fragment` (WHATWG fragment parsing, moxml#353 riding leptris) and builds a Canon tree via the materialize-records fast path — whitespace text nodes survive the parse unconditionally (HTML whitespace significance is the comparison profile's decision, #112/#137), and `parse_node` walks Canon trees directly. The Nokogiri HTML4/HTML5 fragment path remains for runtimes without an engine HTML adapter and for user-supplied Nokogiri nodes. Nokogiri still serves the other HTML lanes: the pretty-printer's HTML5/fixture-ready display modes, the canonical HTML formatters, and the semantic lane's `DataModel.from_html`. The fragment-records lane once delivered nil URIs for prefixed attributes (moxml#367, fixed in moxml 0.5.130 — the bulk read now falls back to the reserved-prefix table); the TreeBuilder keeps the one legal `xml`-prefix binding as standing defense.
+- `Canon::Html::NokogiriSupport` — HTML engine seam. **HTML is engine-served end-to-end** since 0.3.86-0.3.101: `HtmlParser.parse` routes through moxml `Context#parse_html_fragment` (WHATWG fragment parsing, moxml#353 riding leptris) and builds a Canon tree via the materialize-records fast path — whitespace text nodes survive the parse unconditionally (HTML whitespace significance is the comparison profile's decision, #112/#137), and `parse_node` walks Canon trees directly. The semantic lane's `DataModel.from_html`, the `:format` string normalizer, the canonical HTML formatters, `format_html`'s default/XHTML display modes, and `validate_xhtml!` are all engine-served (0.3.99/0.3.100/0.3.101) — **HTML comparison and formatting run nokogiri-less** (pinned by the optionality contract spec). The Nokogiri HTML fragment path remains only for runtimes without an engine HTML adapter and for user-supplied Nokogiri nodes. Nokogiri's residual roles: the `fixture_ready` display mode (libxml FORMAT|AS_XHTML — moxml#390 filed for an engine XHTML mode), the HTML5 validation lint layer (gumbo error-level classification — leptris#1771 filed), and the fragment-records lane once delivered nil URIs for prefixed attributes (moxml#367, fixed in moxml 0.5.130 — the bulk read now falls back to the reserved-prefix table); the TreeBuilder keeps the one legal `xml`-prefix binding as standing defense.
 - `Canon::Xml::Sax` — SAX driver selection follows the resolved adapter: `MoxmlDriver` (leptris SAX — the comparison hot path, CI-gated in the split flip) under the leptris default, `NokogiriDriver` when the resolved adapter is nokogiri, `MoxmlDriver` under Opal. The builder (`SaxBuilder`) is engine-neutral.
 
 Engine parity is complete through libleptris 1.9.8 / leptris-ruby 1.9.33 / moxml 0.5.12: attribute-value normalization, prolog/epilog PIs and comments, DTD ATTLIST defaults off by default (opt-in via `ParseOptions::DTDATTR`), PI-data whitespace matching libxml2, materialize with namespace declarations and a root-subtree-only record contract (moxml#140), readonly parse, deterministic `Document#free` (wired into `from_moxml_xml` — moxml#134). Canon's moxml conversion is record-based (`build_from_moxml` consumes `materialize` records — no per-node wrapper allocation; ~0.55x the Nokogiri conversion). `spec/canon/xml/engine_parity_spec.rb` is fully green. Pretty-printing runs the leptris serializer through `PrettyPrinter::Xml#moxml_format` (byte parity via moxml#153/#155/#156; the #167 escaping race fixed in 0.5.28), with Nokogiri serving the fallback engine and Opal — pretty-printed bytes are canon's product, gated by the fixture-integrity suite and the CI performance gate.

data/lib/canon/formatters/html4_formatter.rb

@@ -2,13 +2,14 @@
 
 module Canon
   module Formatters
-    # HTML4 formatter using Nokogiri::HTML parser
+    # HTML4 formatter — parsed through the engine WHATWG lane (the
+    # #118 html4/html5 unification: one parse serves both flavors).
     class Html4Formatter < HtmlFormatterBase
       # Parse HTML4 document
       # @param html [String] HTML document to parse
-      # @return [Nokogiri::HTML::Document] Parsed HTML4 document
+      # @return [Moxml::Document] Parsed document
       def self.parse(html)
-        Nokogiri::HTML(html)
+        Canon::Html::DataModel.engine_html_parse(html)
       end
     end
   end

data/lib/canon/formatters/html5_formatter.rb

@@ -2,13 +2,13 @@
 
 module Canon
   module Formatters
-    # HTML5 formatter using Nokogiri::HTML5 parser
+    # HTML5 formatter — parsed through the engine WHATWG lane.
     class Html5Formatter < HtmlFormatterBase
       # Parse HTML5 document
       # @param html [String] HTML document to parse
-      # @return [Nokogiri::HTML5::Document] Parsed HTML5 document
+      # @return [Moxml::Document] Parsed document
       def self.parse(html)
-        Nokogiri::HTML5(html)
+        Canon::Html::DataModel.engine_html_parse(html)
       end
     end
   end

data/lib/canon/formatters/html_formatter.rb

@@ -1,23 +1,34 @@
 # frozen_string_literal: true
 
-Canon::NokogiriLoader.require!("HTML formatting")
-
 module Canon
   module Formatters
-    # HTML formatter for HTML 4/5 and XHTML
+    # HTML formatter for HTML 4/5 and XHTML — engine-parsed.
     class HtmlFormatter < HtmlFormatterBase
-      # Parse HTML into a Nokogiri document
+      # Parse HTML into an engine document
       # @param html [String] HTML document to parse
-      # @return [Nokogiri::HTML::Document, Nokogiri::XML::Document]
-      #   Parsed HTML or XML document
+      # @return [Moxml::Document] Parsed document
       def self.parse(html)
         # Validate before parsing
         Canon::Validators::HtmlValidator.validate!(html)
 
         if xhtml?(html)
-          Nokogiri::XML(html)
+          # XHTML is XML — the recover-and-record parse matches the old
+          # non-strict Nokogiri::XML behaviour.
+          Canon::XmlParsing.moxml_context.parse(html)
+        else
+          Canon::Html::DataModel.engine_html_parse(html)
+        end
+      end
+
+      # XHTML input must serialize XML-shaped (self-closed voids) so
+      # the canonical output stays well-formed and re-validates.
+      def self.format(html)
+        if xhtml?(html)
+          doc = parse(html)
+          canonicalize(doc, serializer: :xml)
         else
-          Nokogiri::HTML5(html)
+          doc = Canon::Html::DataModel.engine_html_parse(html)
+          canonicalize(doc, serializer: :html)
         end
       end
 

data/lib/canon/formatters/html_formatter_base.rb

@@ -1,6 +1,5 @@
 # frozen_string_literal: true
 
-Canon::NokogiriLoader.require!("HTML formatting")
 require "set"
 
 module Canon
@@ -63,7 +62,7 @@ module Canon
       # Compiled once — building the alternation and compiling the
       # regex per format call cost more than the gsub it drives.
       BLOCK_SPACING_PATTERN =
-        Regexp.new("(</(?:#{BLOCK_ELEMENTS.join('|')})>)(<(?:#{BLOCK_ELEMENTS.join('|')})[\s>])").freeze
+        Regexp.new("(</(?:#{BLOCK_ELEMENTS.join('|')})>)(?:\n*)(<(?:#{BLOCK_ELEMENTS.join('|')})[\s>])").freeze
       # Format HTML using canonical form
       # @param html [String] HTML document to canonicalize
       # @return [String] Canonical form of HTML
@@ -82,19 +81,25 @@ module Canon
       end
 
       # Canonicalize HTML document
-      # @param doc [Nokogiri::HTML::Document] Parsed HTML document
+      # @param doc [Moxml::Document] Parsed document
+      # @param serializer [:html, :xml] XHTML input must serialize
+      #   XML-shaped (self-closed voids) so the output stays
+      #   well-formed; HTML input takes the HTML shape.
       # @return [String] Canonical HTML string
-      def self.canonicalize(doc)
+      def self.canonicalize(doc, serializer: :html)
         # Sort attributes for consistency
         sort_attributes(doc)
 
         # Normalize whitespace between elements
         normalize_whitespace(doc)
 
-        # Serialize with consistent formatting
-        html = doc.to_html(
-          save_with: Nokogiri::XML::Node::SaveOptions::NO_DECLARATION,
-        ).strip
+        # Serialize with consistent formatting — the engine emits no
+        # declaration; empty elements self-close in the XML shape.
+        html = (if serializer == :xml
+                  doc.to_xml(expand_empty: false)
+                else
+                  doc.to_html
+                end).strip
 
         # Post-process: ensure spaces between block element tags
         # This is needed because Nokogiri's serialization may remove
@@ -103,74 +108,105 @@ module Canon
       end
 
       # Sort element attributes alphabetically throughout document
-      # @param doc [Nokogiri::HTML::Document] Document to process
+      # @param doc [Moxml::Document] Parsed document
       def self.sort_attributes(doc)
-        doc.traverse do |node|
-          next unless node.element?
-          next if node.attributes.empty?
-
-          names = node.attributes.keys
-          # Already-sorted un-namespaced is the common case — removing
-          # and re-adding every attribute is expensive, so check first.
-          # Namespaced attributes must take the slow path: the
-          # remove/re-add below flattens their prefix, and skipping
-          # would change the canonical output.
-          next if names.each_cons(2).all? { |a, b| (a <=> b) <= 0 } &&
-            node.attributes.each_value.all? { |a| a.namespace.nil? }
-
-          sorted_attrs = node.attributes.sort_by { |name, _| name }
-          node.attributes.each_key { |name| node.remove_attribute(name) }
-          sorted_attrs.each { |name, attr| node[name] = attr.value }
+        walk_elements(doc.root) do |element|
+          attrs = element.attributes
+          next if attrs.empty?
+
+          names = attrs.map(&:name)
+          # Already-sorted is the common case — removing and re-adding
+          # every attribute is expensive, so check first.
+          next if names.each_cons(2).all? { |a, b| (a <=> b) <= 0 }
+
+          sorted_attrs = attrs.sort_by(&:name)
+          names.each { |name| element.remove_attribute(name) }
+          sorted_attrs.each { |attr| element[attr.name] = attr.value }
         end
       end
 
+      # Depth-first element walk (pre-order).
+      def self.walk_elements(node, &block)
+        return unless node.is_a?(Moxml::Element)
+
+        yield(node)
+        node.children.to_a.each { |child| walk_elements(child, &block) }
+      end
+
       # Normalize whitespace by removing whitespace-only text nodes
       # between elements and collapsing whitespace within text content
-      # @param doc [Nokogiri::HTML::Document] Document to process
+      # @param doc [Moxml::Document] Parsed document
       def self.normalize_whitespace(doc)
-        # Normalize whitespace in text nodes
-        doc.traverse do |node|
-          next unless node.text?
-
+        # Walk parented text nodes only — the engine's content= on a
+        # parentless node raises an opaque Invalid argument (upstream
+        # note), and parentless text has no canonical meaning anyway.
+        walk_text_nodes(doc.root) do |text, parent|
           # CRITICAL: Skip normalization for whitespace-sensitive elements
           # In elements like <pre>, <code>, etc., whitespace is semantically
           # significant and must be preserved exactly as-is
-          if whitespace_sensitive_element?(node.parent)
-            next
-          end
-
-          # Handle whitespace-only text nodes
-          if node.text.match?(Canon::Xml::WhitespacePolicy::STRIP_ONLY) && node.parent&.element?
-            # Check if this text node is between block-level elements
-            prev_sibling = node.previous_sibling
-            next_sibling = node.next_sibling
-
-            # If between block elements, preserve one space
-            if block_element?(prev_sibling) || block_element?(next_sibling) ||
-                block_element?(node.parent)
-              node.content = " "
+          next if whitespace_sensitive_element?(parent)
+
+          if text.content.match?(Canon::Xml::WhitespacePolicy::STRIP_ONLY)
+            # Whitespace-only text: keep one space between block
+            # elements (information-preserving canonical form),
+            # otherwise remove. The walk yields parented nodes only —
+            # the engine content= raises on parentless text upstream.
+            if block_element?(prev_element_sibling(parent, text)) ||
+                block_element?(next_element_sibling(parent, text)) ||
+                block_element?(parent)
+              text.content = " "
             else
-              # Otherwise remove it
-              node.remove
+              text.remove
             end
           else
             # Collapse multiple whitespace characters into single spaces
             # but preserve leading/trailing single spaces for inline content
-            text = node.text
-            normalized = text.gsub(/\s+/, " ")
+            content = text.content
+            normalized = content.gsub(/\s+/, " ")
             # Only strip if the entire parent chain suggests it's appropriate
             # (e.g., at document boundaries)
-            if node.parent&.name == "body" &&
-                (node.previous_sibling.nil? || node.next_sibling.nil?)
+            if parent.name == "body" &&
+                (first_real_child?(parent, text) || last_real_child?(parent, text))
               normalized = normalized.strip
             end
-            # node.content= re-parses the string — skip it when nothing
+            # content= re-parses the string — skip it when nothing
             # changed (text with no collapsible whitespace).
-            node.content = normalized unless normalized == text
+            text.content = normalized unless normalized == content
           end
         end
       end
 
+      def self.walk_text_nodes(node, parent = nil, &block)
+        case node
+        when Moxml::Text
+          yield(node, parent) if parent
+        when Moxml::Element
+          node.children.to_a.each { |child| walk_text_nodes(child, node, &block) }
+        end
+      end
+
+      def self.prev_element_sibling(parent, text)
+        siblings = parent.children.to_a
+        idx = siblings.index(text)
+        siblings[0...idx].reverse_each.find { |s| s.is_a?(Moxml::Element) }
+      end
+
+      def self.next_element_sibling(parent, text)
+        siblings = parent.children.to_a
+        idx = siblings.index(text)
+        siblings[(idx + 1)..].find { |s| s.is_a?(Moxml::Element) }
+      end
+
+      def self.first_real_child?(parent, text)
+        siblings = parent.children.to_a
+        siblings.find { |s| !(s.is_a?(Moxml::Text) && s.content.strip.empty?) }.equal?(text)
+      end
+
+      def self.last_real_child?(parent, text)
+        siblings = parent.children.to_a
+        siblings.reverse_each.find { |s| !(s.is_a?(Moxml::Text) && s.content.strip.empty?) }.equal?(text)
+      end
+
       # Ensure spacing between block element tags in serialized HTML
       # @param html [String] Serialized HTML string
       # @return [String] HTML with proper spacing between block elements
@@ -184,7 +220,7 @@ module Canon
       # @param node [Nokogiri::XML::Node, nil] Node to check
       # @return [Boolean] true if node is a block element
       def self.block_element?(node)
-        return false unless node&.element?
+        return false unless node.is_a?(Moxml::Element)
 
         name = node.name
         BLOCK_ELEMENT_SET.include?(name) ||
@@ -192,25 +228,18 @@ module Canon
       end
 
       # Check if a node is a whitespace-sensitive element
-      # @param node [Nokogiri::XML::Node, nil] Node to check
+      # @param node [Moxml::Element, nil] Node to check
       # @return [Boolean] true if node is whitespace-sensitive
       def self.whitespace_sensitive_element?(node)
-        return false unless node&.element?
-
-        # Check if this element or any ancestor is whitespace-sensitive
         current = node
-        while current
-          if current.element?
-            name = current.name
-            # Nokogiri lowercases HTML names — the downcase fallback
-            # only allocates for unusual mixed-case input.
-            if WHITESPACE_SENSITIVE_ELEMENTS.include?(name) ||
-                WHITESPACE_SENSITIVE_ELEMENTS.include?(name.downcase)
-              return true
-            end
+        while current.is_a?(Moxml::Element)
+          name = current.name
+          # The downcase fallback only allocates for unusual mixed-case
+          # input.
+          if WHITESPACE_SENSITIVE_ELEMENTS.include?(name) ||
+              WHITESPACE_SENSITIVE_ELEMENTS.include?(name.downcase)
+            return true
           end
-          # Stop at document root - documents don't have parents
-          break if current.is_a?(Nokogiri::XML::Document) || current.is_a?(Nokogiri::HTML5::Document)
 
           current = current.parent
         end

data/lib/canon/pretty_printer/html.rb

@@ -98,57 +98,31 @@ module Canon
       # indentation; +AS_XHTML+ produces well-shaped output (void
       # elements self-closed, non-void paired); +NO_DECLARATION+
       # suppresses the +<?xml ...?>+ prefix.
+      # Engine parse + significance-aware strip + the #354 indent
+      # knobs + a canon-side void self-closing pass for the AS_XHTML
+      # contract (an engine XHTML mode is asked upstream as moxml#390;
+      # until then the shape transform is ours).
       def format_fixture_ready(html_string)
-        doc = Nokogiri::HTML5(html_string)
-        strip_structural_whitespace!(doc)
-        io = StringIO.new
-        if @indent_type == "tab"
-          doc.write_to(io, save_with: fixture_ready_save_options,
-                           indent: 1, indent_text: "\t")
-        else
-          doc.write_to(io, save_with: fixture_ready_save_options,
-                           indent: @indent)
-        end
-        io.string
+        doc = Canon::Html::DataModel.engine_html_parse(html_string)
+        Canon::Html::DataModel.strip_insignificant_whitespace!(doc.root)
+        html = if @indent_type == "tab"
+                 doc.to_html(indent: 1, indent_text: "\t")
+               else
+                 doc.to_html(indent: @indent)
+               end
+        self_closing_voids(html)
       end
 
-      # libxml's +FORMAT+ save flag does not insert indentation around
-      # the children of any element it sees as mixed content (any
-      # non-whitespace-only text node child).  +Nokogiri::HTML5+ does
-      # not accept the +noblanks+ option that the XML parser uses to
-      # strip these inter-sibling text nodes pre-serialisation, so we
-      # do it manually here: drop whitespace-only text nodes whose
-      # parent is structural (no real text content) and not a
-      # whitespace-preserving element.  Mixed-content runs like
-      # +<p>foo <em>bar</em> baz</p>+ are left alone.
-      def strip_structural_whitespace!(doc)
-        to_remove = []
-        doc.traverse do |node|
-          next unless node.text?
-          next unless node.content.strip.empty?
-
-          parent = node.parent
-          next if parent.nil?
-          next if WHITESPACE_PRESERVING_ELEMENTS.include?(parent.name)
-          next if parent_has_real_text?(parent)
-
-          to_remove << node
-        end
-        to_remove.each(&:remove)
-      end
+      # AS_XHTML: void elements self-closed (<br/>, <img ... />).
+      def self_closing_voids(html)
+        html.gsub(%r{<([A-Za-z][A-Za-z0-9:-]*)((?:\s+[^<>"]*(?:"[^"]*"[^<>"]*)*)?)>}) do
+          name = ::Regexp.last_match(1)
+          next ::Regexp.last_match(0) unless HtmlVoidElements.void?(name)
 
-      def parent_has_real_text?(parent)
-        parent.children.any? do |c|
-          c.text? && !c.content.strip.empty?
+          "<#{name}#{::Regexp.last_match(2)}/>"
         end
       end
 
-      def fixture_ready_save_options
-        Nokogiri::XML::Node::SaveOptions::FORMAT |
-          Nokogiri::XML::Node::SaveOptions::AS_XHTML |
-          Nokogiri::XML::Node::SaveOptions::NO_DECLARATION
-      end
-
       # Rewrite +<tag …/>+ into +<tag …></tag>+ for every element name
       # that is not an HTML5 void element. +<a/>+ is illegal HTML;
       # void tags like +<br/>+ and +<img …/>+ pass through unchanged.

data/lib/canon/tree_diff/adapters/xml_adapter.rb

@@ -45,6 +45,8 @@ module Canon
             return to_tree_from_canon_text(node)
           when Canon::Xml::Nodes::CommentNode
             return to_tree_from_canon_comment(node)
+          when Canon::Xml::Nodes::ProcessingInstructionNode
+            return to_tree_from_canon_processing_instruction(node)
           end
 
           # Fallback to raw engine documents (legacy support). Nokogiri
@@ -61,6 +63,8 @@ module Canon
             to_tree(root)
           when Nokogiri::XML::Element
             convert_element(node)
+          when Nokogiri::XML::ProcessingInstruction
+            to_tree_from_canon_processing_instruction(node)
           else
             raise ArgumentError, "Unsupported node type: #{node.class}"
           end
@@ -271,6 +275,29 @@ module Canon
           )
         end
 
+        # Convert Canon::Xml::Nodes::ProcessingInstructionNode (or a raw
+        # engine processing instruction) to TreeNode. The value carries
+        # both target and data so differing targets diff.
+        #
+        # @param pi_node [Canon::Xml::Nodes::ProcessingInstructionNode,
+        #   Nokogiri::XML::ProcessingInstruction] Processing instruction
+        # @return [Core::TreeNode] Tree node
+        def to_tree_from_canon_processing_instruction(pi_node)
+          target = pi_node.respond_to?(:target) ? pi_node.target : pi_node.name
+          data = if pi_node.respond_to?(:data)
+                   pi_node.data
+                 else
+                   pi_node.content.to_s
+                 end
+          Core::TreeNode.new(
+            label: "processing-instruction",
+            value: "#{target} #{data}".strip,
+            attributes: {},
+            children: [],
+            source_node: pi_node,
+          )
+        end
+
         # Extract attributes from Canon::Xml::Nodes::ElementNode
         #
         # @param element_node [Canon::Xml::Nodes::ElementNode] Element node

data/lib/canon/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Canon
-  VERSION = "0.3.100"
+  VERSION = "0.3.101"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: canon
 version: !ruby/object:Gem::Version
-  version: 0.3.100
+  version: 0.3.101
 platform: ruby
 authors:
 - Ribose Inc.