canon 0.3.115 → 0.3.117

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f0837e14a667e91da2ce1818ecab8ca194f7ab1353ac8db3703ba6bee4ea6857
-  data.tar.gz: 4a9345754f64e4cae207ee7285ab0bc88b8aa5747090897aa479f9b30cf07f45
+  metadata.gz: 223b5ab0201bbebfe8bfee1de2f23229260f574bd6d42fed426256b199f03d6c
+  data.tar.gz: 803eaba7a9fd850a38669c9ea2bbd567503fbc936fd39147a473d2526b3539ce
 SHA512:
-  metadata.gz: 6524509bb0f1c3aa27b419c45959fee9eb84c38c44859373f0d926cb729bb0796cf59cc4fe12567cd2480090bbc6dad640ca759e166cbd6e629c38e254d20832
-  data.tar.gz: 01fb13b4b41701f57127324a0f667c721a7a55121a08e64eae8ac291595a0596544916daf274e62da30de07a4f2b1153f2f73afa76d63c1f5fca31751763a1a7
+  metadata.gz: 24ae7e292b598242df4c98bad7d98d98815d7d28ddebd6d1ed79df4451e40017cd3840f2a3556bf4b78681ee7b6653ee91a67623d806d2aec1d8e170093a19d8
+  data.tar.gz: 35a5642e0e6b5b0268935d6888470614058888a3e4cc33d587ea079737a3817b9543aa63fbae986da95240a2c4c72174419eae1fba9336e871859ae6aae6ee3d

data/docs/_references/canonical-digests.adoc

@@ -0,0 +1,50 @@
+= The canonical digests (W5.3)
+
+This document specifies canon's canonical digest API as shipped by
+`Canon::Digest`. Reviewed against the implementation by
+`spec/canon/docs/canonical_digests_spec.rb` (the W6.3 review-gate
+pattern), with the cross-release byte pins carried by
+`spec/canon/digest_stability_spec.rb`.
+
+== 1. The law
+
+The digest of a document is the OpenSSL hash of its CANONICAL
+FORM — the form under which the format's comparison pipeline deems
+documents equivalent. Two documents that compare equivalent produce
+identical digests; two that compare distinct produce distinct
+digests. The digest is an API over the comparison semantics, not a
+new canonicalization.
+
+== 2. Per-format canonical forms
+
+* *XML* rides Canonical XML 1.1
+  (`Canon::Xml::C14n.canonicalize`, reached as
+  `Formatters::XmlFormatter.format(pretty: false)`). The
+  pretty-printed XML form preserves attribute order and is NOT
+  canonical; the digest lane never touches it.
+* *JSON and YAML* ride the key-sorted formatter forms (the JCS
+  profile for JSON's canonical lane; the `[class, to_s]` TOTAL
+  key order for YAML).
+* *HTML* rides the whitespace-law form with sorted attributes
+  (the W1.3 specification).
+
+Equivalence consequences the digest inherits: attribute order is
+insignificant in XML and HTML; mapping key order is insignificant
+in JSON and YAML; element/sequence order is significant everywhere.
+
+== 3. Entry points
+
+`Canon::Digest.digest(content, format, algorithm: "sha256")`
+accepts any OpenSSL digest name; `canonical_form(content, format)`
+exposes the hashed bytes. Per-format shorthands
+(`Canon.digest_xml`, `digest_yaml`, `digest_json`, ...) mirror the
+format lane's shorthands.
+
+== 4. Stability contract
+
+The digests are stable across releases for as long as the
+underlying canonical form is. The stability suite pins the digest
+hex of a representative document per format; a canonical-form
+change moves a digest and MUST fail that suite, forcing a
+conscious pin update (and a release note) rather than silent drift
+for downstream dedup stores.

data/docs/_references/jcs-rfc8785.adoc

@@ -0,0 +1,52 @@
+= The JCS canonical profile (W6.1, RFC 8785)
+
+This document specifies canon's JSON JCS canonical form as shipped
+by `Canon::Json::JCS`. Reviewed against the implementation by
+`spec/canon/docs/jcs_spec_review.rb` (the W6.3 review-gate
+pattern).
+
+== 1. Scope
+
+RFC 8785 — the I-JSON Canonicalization Scheme. Input must be I-JSON
+(RFC 7493): valid UTF-8 without lone surrogates, no duplicate
+object keys, numbers expressible as IEEE 754 doubles. Violations
+raise `Canon::Json::JCS::Error`.
+
+== 2. The canonical laws
+
+* *Strings* (`escape_string`): RFC 8785 §3.2.2.2 — the
+  JSON.stringify escaping set: short forms for `\b \t \n \f \r`
+  (the SHORT_ESCAPES table), two-hex lowercase `\u00xx` for the
+  remaining C0 controls, `\"` and `\\`; every other code point
+  emits verbatim as UTF-8 (DEL U+007F is NOT escaped).
+
+* *Numbers* (`es6_number`): ECMAScript `Number::toString`
+  serialization — the shortest round-tripping decimal, fixed
+  notation inside [1e-6, 1e21), exponential outside; integers
+  via `ijson_integer` (the I-JSON 2^53 range).
+
+* *Object keys* (`write_object`): sorted by their
+  UTF-16 code units (`utf16_code_units` — the RFC's sort is over
+  UTF-16 code units, not code points, so surrogate pairs and BMP
+  characters interleave per the RFC's examples; e.g. the key
+  U+1D160 (a surrogate pair, code units 0xD834 0xDD60) sorts
+  BEFORE U+20AC (a single BMP code unit 0x20AC), which sorts
+  BEFORE "a" (0x0061)).
+
+* *Containers*: arrays comma-separated in order; objects
+  comma-separated in sorted key order; no whitespace anywhere.
+
+== 3. Entry points
+
+`canonicalize(json)` accepts a JSON string (parsed through
+`JSON.parse`) or an already-parsed Ruby graph; `serialize(value)`
+skips the parse. The canonical form is byte-stable across
+platforms and suitable for hashing and signing.
+
+== 4. Fixed point and parity
+
+JCS output re-parses to the same graph, and the form is a fixed
+point. The phase 02 property suite (JSON/YAML equivalence legs)
+and the diff-display corpus pin the behavior; the JCS spec
+vectors (RFC 8785 §3.2.3 examples) pin the number and sorting
+laws.

data/lib/canon/comparison/schema_gate.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module Canon
+  module Comparison
+    # W5.1 validate-then-compare: runs both documents through the
+    # engine's XSD assessment before the diff. Returns a hash of
+    # side-index (0 or 1) to Canon::ValidationError — empty when both
+    # documents pass and the comparison may proceed.
+    module SchemaGate
+      module_function
+
+      # @param obj1 [String] first XML document
+      # @param obj2 [String] second XML document
+      # @param schema [String, Leptris::XML::XSD::Schema] schema text
+      #   (compiled once here) or a compiled schema
+      # @return [Hash{Integer=>Canon::ValidationError}]
+      def gate(obj1, obj2, schema)
+        schema = Canon::Validators::XsdValidator.compile(schema) if schema.is_a?(String)
+        failures = {}
+        [obj1, obj2].each_with_index do |obj, side|
+          Canon::Validators::XsdValidator.validate!(obj, schema: schema)
+        rescue Canon::ValidationError => e
+          failures[side] = e
+        end
+        failures
+      end
+    end
+  end
+end

data/lib/canon/comparison.rb

@@ -95,6 +95,7 @@ module Canon
     autoload :DiffNodeBuilder, "canon/comparison/diff_node_builder"
     autoload :Dimensions, "canon/comparison/dimensions"
     autoload :FormatDetector, "canon/comparison/format_detector"
+    autoload :SchemaGate, "canon/comparison/schema_gate"
     autoload :HtmlComparator, "canon/comparison/html_comparator"
     autoload :HtmlCompareProfile, "canon/comparison/html_compare_profile"
     autoload :HtmlParser, "canon/comparison/html_parser"
@@ -246,6 +247,33 @@ module Canon
                                                        :semantic_tree))
         end
 
+        # W5.1 schema gate: when a schema rides the options, both XML
+        # documents must pass the engine's XSD assessment BEFORE the
+        # diff runs — a validation failure is normative regardless of
+        # what the tree comparison would say.
+        if (xsd = opts[:xsd])
+          gate = SchemaGate.gate(obj1, obj2, xsd)
+          unless gate.empty?
+            diffs = gate.map do |side, error|
+              node = Diff::DiffNode.new(node1: side.zero? ? obj1 : obj2,
+                                        node2: side.zero? ? obj2 : obj1,
+                                        dimension: :schema_validation,
+                                        reason: error.message)
+              node.normative = true # a schema failure is normative by definition
+              node
+            end
+            return false unless opts[:verbose]
+
+            return ComparisonResult.new(
+              differences: diffs,
+              preprocessed_strings: [obj1.to_s, obj2.to_s],
+              format: opts[:format] || :xml,
+              algorithm: :dom,
+              original_strings: [obj1.to_s, obj2.to_s],
+            )
+          end
+        end
+
         if %i[semantic semantic_tree].include?(opts[:diff_algorithm])
           return semantic_diff(obj1, obj2, opts)
         end

data/lib/canon/digest.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+require "openssl"
+
+module Canon
+  # W5.3 — canonical digests: the hash of a document's canonical form
+  # as a first-class API for deduplication and change detection.
+  #
+  # The digest is taken over the format's CANONICAL form — the form
+  # under which the comparison pipeline deems documents equivalent:
+  # XML rides Canonical XML 1.1 (attribute order sorted,
+  # namespace-normalized); JSON/YAML ride the key-sorted forms; HTML
+  # rides the whitespace-law form with sorted attributes. Two
+  # documents that compare equivalent under the canonical form
+  # produce identical digests. The digests are stable across releases
+  # for as long as the canonical form is (the digest-stability suite
+  # pins the bytes per release pair).
+  module Digest
+    DEFAULT_ALGORITHM = "sha256"
+
+    class << self
+      # @param content [String] the document
+      # @param format [Symbol] the format (see SUPPORTED_FORMATS)
+      # @param algorithm [String, Symbol] any OpenSSL digest name
+      # @return [String] the hex digest of the canonical form
+      def digest(content, format, algorithm: DEFAULT_ALGORITHM)
+        OpenSSL::Digest.new(algorithm.to_s)
+          .hexdigest(canonical_form(content, format))
+      end
+
+      # The canonical form the digest rides: XML's is C14N (the
+      # pretty-printed form preserves attribute order and is NOT
+      # canonical); every other format's formatter output IS the
+      # canonical form.
+      def canonical_form(content, format)
+        if format.to_sym == :xml
+          Formatters::XmlFormatter.format(content, pretty: false)
+        else
+          Canon.format(content, format)
+        end
+      end
+    end
+  end
+end

data/lib/canon/validators/xsd_validator.rb

@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+require "leptris" # the XSD lane rides the engine's tier-1 validator
+# (libleptris >= 1.9.321, leptris#1075)
+
+module Canon
+  module Validators
+    # XSD validation (W5.1) through the engine's tier-1 validator:
+    # schema compilation, whole-instance assessment, enumerated
+    # failures. The schema is a compiled `Leptris::XML::XSD::Schema`
+    # — callers validating many documents against one schema compile
+    # once and reuse (compilation is the expensive step).
+    #
+    #     schema = Canon::Validators::XsdValidator.compile(schema_xml)
+    #     Canon::Validators::XsdValidator.validate!(xml, schema: schema)
+    class XsdValidator < BaseValidator
+      # Compile a schema from XSD text. Raises Canon::ValidationError
+      # when the text is not well-formed XML, and when the engine
+      # reports schema-level problems.
+      #
+      # @param schema_xml [String] the XSD document
+      # @return [Leptris::XML::XSD::Schema] the compiled schema
+      def self.compile(schema_xml)
+        schema = Leptris::XML::XSD.compile(schema_xml)
+        if (detail = schema.error)
+          raise Canon::ValidationError.new(
+            "schema invalid: #{detail}",
+            format: :xml,
+          )
+        end
+        schema
+      end
+
+      # Validate an XML instance against a compiled schema
+      #
+      # @param input [String] the XML instance
+      # @param schema [Leptris::XML::XSD::Schema] a compiled schema
+      # @raise [Canon::ValidationError] if the instance fails
+      #   assessment; the message carries the first failure, details
+      #   carry the full enumerated list
+      # @return [void]
+      def self.validate!(input, schema:)
+        return if input.nil? || input.strip.empty?
+
+        doc = Leptris::XML.parse(input, readonly: true)
+        errors = schema.validate_errors(doc)
+        return if errors.empty?
+
+        raise Canon::ValidationError.new(
+          errors.first.to_s,
+          format: :xml,
+          details: errors.join("\n"),
+        )
+      end
+    end
+  end
+end

data/lib/canon/validators.rb

@@ -7,6 +7,7 @@ module Canon
     autoload :HtmlValidator, "canon/validators/html_validator"
     autoload :JsonValidator, "canon/validators/json_validator"
     autoload :XmlValidator, "canon/validators/xml_validator"
+    autoload :XsdValidator, "canon/validators/xsd_validator"
     autoload :YamlValidator, "canon/validators/yaml_validator"
   end
 end

data/lib/canon/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Canon
-  VERSION = "0.3.115"
+  VERSION = "0.3.117"
 end

data/lib/canon/xml/exclusive_c14n.rb

@@ -55,12 +55,30 @@ module Canon
       # — every node in the subtrees rooted at pfx:name. Returns
       # [prefix, local_name], or nil for anything else.
       def self.c14n_selection_idiom(xpath)
-        # the suite's .xpath files are XML documents wrapping the
-        # expression (with prefix declarations and inline comments)
-        text = xpath.sub(/\A<\?xml[^?]*\?>/, "")
-          .gsub(/<!--[\s\S]*?-->/, "")
-          .gsub(/<[^>]*>/, " ")
-          .squeeze(" ")
+        # The suite's .xpath files are XML documents wrapping the
+        # expression (with prefix declarations and inline comments).
+        # Extraction is index-based string surgery — the CodeQL
+        # polynomial-redos query flags the gsub-chain form.
+        close_tag = xpath.rindex("</XPath>")
+        text = close_tag ? xpath[0...close_tag] : xpath.dup
+        # drop the XML declaration and the <XPath> open tag: each
+        # ends at the first '>' at/after its '<'
+        if (decl = text.index("<"))
+          gt = text.index(">", decl)
+          if (xopen = text.index("<XPath", decl)) && (xopen < gt)
+            gt = text.index(">", xopen)
+          end
+          text = text[(gt + 1)..] if gt
+        end
+        # strip <!-- --> comments by index (no regex backtracking)
+        loop do
+          open_c = text.index("<!--")
+          break unless open_c
+
+          close_c = text.index("-->", open_c + 4)
+          text = close_c ? text[0...open_c] + text[(close_c + 3)..] : text[0...open_c]
+        end
+        text = text.squeeze(" ")
         m = text.match(%r{\(//\.\s*\|\s*//@\*\s*\|\s*//namespace::\*\)\s*\[\s*ancestor-or-self::([A-Za-z_][\w.-]*)(?::([\w.-]+))?\s*\]\s*\z})
         return nil unless m
 

data/lib/canon.rb

@@ -32,6 +32,7 @@ module Canon
   autoload :Cli, "canon/cli"
   autoload :ColorDetector, "canon/color_detector"
   autoload :DiffFormatter, "canon/diff_formatter"
+  autoload :Digest, "canon/digest"
   autoload :Json, "canon/json"
 
   SUPPORTED_FORMATS = %i[xml yaml json html html4 html5 string].freeze
@@ -84,5 +85,15 @@ module Canon
     define_singleton_method("format_#{format}") do |content|
       format(content, format)
     end
+
+    define_singleton_method("digest_#{format}") do |content, algorithm: Canon::Digest::DEFAULT_ALGORITHM|
+      Canon::Digest.digest(content, format, algorithm: algorithm)
+    end
+  end
+
+  # The canonical digest of a document (W5.3): the hash of its
+  # canonical form. See Canon::Digest for the stability contract.
+  def self.digest(content, format = :xml, algorithm: Canon::Digest::DEFAULT_ALGORITHM)
+    Canon::Digest.digest(content, format, algorithm: algorithm)
   end
 end

data/lib/tasks/performance_comparator.rb

@@ -30,16 +30,33 @@ class PerformanceComparator
   DIM    = "\e[2m"
   CLEAR  = "\e[0m"
 
+  BEST_OF = Integer(ENV.fetch("CANON_PERF_BEST_OF", "3"))
+
   def run
     clone_base
-    current = run_report(REPO_ROOT, "current")
-    base = run_report(base_clone_dir, "base (#{DEFAULT_BASE})")
+    current = best_of(REPO_ROOT, "current")
+    base = best_of(base_clone_dir, "base (#{DEFAULT_BASE})")
     print_report(current, base)
     exit(1) if regressions?(current, base)
   ensure
     cleanup
   end
 
+  # The html_parsing lanes drift up to ~27% BETWEEN SAME-CONTENT runs
+  # on shared runners (main failed against itself: run 40047205814) —
+  # single-run comparisons measure the runner, not the code. Each side
+  # runs BEST_OF times; per benchmark, the run with the highest upper
+  # bound stands (the capability estimate a fair gate compares).
+  def best_of(working_dir, label)
+    runs = Array.new(BEST_OF) { |i| run_report(working_dir, "#{label} ##{i + 1}") }
+    runs.first.fetch("benchmarks").keys.each_with_object({}) do |bench, merged|
+      best = runs.max_by { |r| r.fetch("benchmarks").fetch(bench).fetch("upper").to_f }
+      merged[bench] = best.fetch("benchmarks").fetch(bench)
+    end.then do |benchmarks|
+      runs.first.merge("benchmarks" => benchmarks)
+    end
+  end
+
   private
 
   def clone_base

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: canon
 version: !ruby/object:Gem::Version
-  version: 0.3.115
+  version: 0.3.117
 platform: ruby
 authors:
 - Ribose Inc.
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-10-23 00:00:00.000000000 Z
+date: 2026-10-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: diff-lcs
@@ -173,8 +173,10 @@ files:
 - docs/Gemfile
 - docs/INDEX.adoc
 - docs/_config.yml
+- docs/_references/canonical-digests.adoc
 - docs/_references/comparison-theory.adoc
 - docs/_references/html-canonical-form.adoc
+- docs/_references/jcs-rfc8785.adoc
 - docs/_references/yaml-canonical-form.adoc
 - docs/advanced/diff-classification.adoc
 - docs/advanced/diff-pipeline.adoc
@@ -267,6 +269,7 @@ files:
 - lib/canon/comparison/pipeline.rb
 - lib/canon/comparison/profile_definition.rb
 - lib/canon/comparison/ruby_object_comparator.rb
+- lib/canon/comparison/schema_gate.rb
 - lib/canon/comparison/strategies.rb
 - lib/canon/comparison/strategies/base_match_strategy.rb
 - lib/canon/comparison/strategies/match_strategy_factory.rb
@@ -338,6 +341,7 @@ files:
 - lib/canon/diff_formatter/legend.rb
 - lib/canon/diff_formatter/pretty_diff_formatter.rb
 - lib/canon/diff_formatter/theme.rb
+- lib/canon/digest.rb
 - lib/canon/errors.rb
 - lib/canon/formatters.rb
 - lib/canon/formatters/html4_formatter.rb
@@ -404,6 +408,7 @@ files:
 - lib/canon/validators/html_validator.rb
 - lib/canon/validators/json_validator.rb
 - lib/canon/validators/xml_validator.rb
+- lib/canon/validators/xsd_validator.rb
 - lib/canon/validators/yaml_validator.rb
 - lib/canon/version.rb
 - lib/canon/xml.rb