canon 0.3.113 → 0.3.114
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/docs/_references/html-canonical-form.adoc +67 -0
- data/docs/_references/yaml-canonical-form.adoc +45 -0
- data/lib/canon/version.rb +1 -1
- data/lib/canon/xml/attribute_handler.rb +6 -2
- data/lib/canon/xml/c14n.rb +3 -42
- data/lib/canon/xml/exclusive_c14n.rb +107 -0
- data/lib/canon/xml/exclusive_namespace_handler.rb +87 -0
- data/lib/canon/xml/node_set_marker.rb +37 -0
- data/lib/canon/xml/processor.rb +6 -3
- data/lib/canon/xml/xpath_engine.rb +2 -0
- data/lib/canon/xml.rb +3 -0
- data/scripts/exc_c14n_differential.rb +61 -0
- metadata +7 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e1fba0743a7127443ef7837004ae95b97be4760617237be10a63de01147e6812
|
|
4
|
+
data.tar.gz: 7853cf8424906ab05947560615db5d3c3ff5620564a86ee38b8fe19f306e8dac
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4b222586fe5831e76e443fa61bb4ee0601c31e5c048f3a9e39a37649437c738f03b9be5f89e7b97a823f311245a1ce1275cf215649a07758307715c86242aecb
|
|
7
|
+
data.tar.gz: 32d38e3eaaa8703697f2e14ff7610f44b76a69ba8d7ff18a4951d57f0d36ba83d9e02525526643103c2d62f37bd37cc0f7174d707a1fb96d87de262430251bad
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
= The HTML canonical form (W1.3)
|
|
2
|
+
|
|
3
|
+
This document specifies canon's HTML canonical form as shipped by
|
|
4
|
+
`Canon::Formatters::HtmlFormatter` (and the html4/html5 variants).
|
|
5
|
+
It is reviewed against the implementation by
|
|
6
|
+
`spec/canon/docs/canonical_form_spec.rb` (the W6.3 review-gate
|
|
7
|
+
pattern: law tokens asserted against the formatter's constants and
|
|
8
|
+
methods).
|
|
9
|
+
|
|
10
|
+
== 1. Pipeline
|
|
11
|
+
|
|
12
|
+
format(input):
|
|
13
|
+
|
|
14
|
+
1. *Parse* through the engine HTML adapter (WHATWG fragment
|
|
15
|
+
parsing; the fixture materialization keeps whitespace text nodes —
|
|
16
|
+
whitespace significance is the comparison profile's decision, not
|
|
17
|
+
the parser's).
|
|
18
|
+
2. *Canonicalize the tree* (attribute order, whitespace law below).
|
|
19
|
+
3. *Serialize* through the engine's HTML serializer.
|
|
20
|
+
4. *Post-serialize spacing* (the two regex laws below).
|
|
21
|
+
|
|
22
|
+
== 2. Whitespace law
|
|
23
|
+
|
|
24
|
+
*Whitespace-sensitive elements* — `pre`, `code`, `textarea`,
|
|
25
|
+
`script`, `style` — are never normalized: their text content rides
|
|
26
|
+
verbatim.
|
|
27
|
+
|
|
28
|
+
Every other text node normalizes:
|
|
29
|
+
|
|
30
|
+
* *Whitespace-only text* between/inside block elements collapses to
|
|
31
|
+
one space (the information-preserving canonical form);
|
|
32
|
+
whitespace-only text elsewhere is removed.
|
|
33
|
+
* *Text with content*: whitespace runs collapse to single spaces;
|
|
34
|
+
leading/trailing strips only at `body` boundaries (first/last
|
|
35
|
+
real child).
|
|
36
|
+
|
|
37
|
+
*Block elements* (the formatter's BLOCK_ELEMENTS registry):
|
|
38
|
+
address, article, aside, blockquote, dd, details, dialog, div, dl,
|
|
39
|
+
dt, fieldset, figcaption, figure, footer, form, h1, h2, h3, h4, h5,
|
|
40
|
+
h6, header, hgroup, hr, li, main, nav, ol, p, pre, section, table,
|
|
41
|
+
tbody, td, tfoot, th, thead, tr, ul.
|
|
42
|
+
|
|
43
|
+
== 3. Attribute order
|
|
44
|
+
|
|
45
|
+
Attributes sort by name (the shared `Canon::Xml::AttributeSorter`
|
|
46
|
+
walk — the canonical order matches the XML lane's sorter; the
|
|
47
|
+
formatter's `sort_attributes` delegates to it).
|
|
48
|
+
|
|
49
|
+
== 4. Post-serialize spacing
|
|
50
|
+
|
|
51
|
+
Two serialization laws, applied in order (the formatter's compiled
|
|
52
|
+
constants carry the same names):
|
|
53
|
+
|
|
54
|
+
* *Block spacing* (`BLOCK_SPACING_PATTERN`): a block close tag
|
|
55
|
+
followed by newlines and a block open tag re-spells exactly one
|
|
56
|
+
newline between them.
|
|
57
|
+
* *After-block fold* (`AFTER_BLOCK_NEWLINE_PATTERN`): a block close
|
|
58
|
+
tag followed by a newline run and any open tag folds the run to a
|
|
59
|
+
single space when the follower is not a close tag (the
|
|
60
|
+
engine-inserted newlines after block closes fold to the canonical
|
|
61
|
+
single space).
|
|
62
|
+
|
|
63
|
+
== 5. Idempotence
|
|
64
|
+
|
|
65
|
+
The form is a fixed point: format(format(x)) == format(x). The
|
|
66
|
+
property suite (canonical-form invariants) and the corpus gates pin
|
|
67
|
+
this per release.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
= The YAML canonical form (W1.4)
|
|
2
|
+
|
|
3
|
+
This document specifies canon's YAML canonical form as shipped by
|
|
4
|
+
`Canon::Formatters::YamlFormatter`. Reviewed against the
|
|
5
|
+
implementation by `spec/canon/docs/canonical_form_spec.rb` (the
|
|
6
|
+
W6.3 review-gate pattern).
|
|
7
|
+
|
|
8
|
+
== 1. Pipeline
|
|
9
|
+
|
|
10
|
+
format(input):
|
|
11
|
+
|
|
12
|
+
1. *Validate* through the YAML validator (engine-resolved: yeptris
|
|
13
|
+
when loadable, Psych otherwise; a validator rejection is a
|
|
14
|
+
Canon::ValidationError with position information).
|
|
15
|
+
2. *Parse* to the Ruby object graph (`safe_load`, aliases enabled).
|
|
16
|
+
3. *Sort* every mapping's keys (the law below).
|
|
17
|
+
4. *Dump* through the engine's emitter (the engine-dump byte-parity
|
|
18
|
+
is pinned by the YAML engine-parity spec).
|
|
19
|
+
|
|
20
|
+
== 2. The key-order law
|
|
21
|
+
|
|
22
|
+
Every mapping's keys sort under the TOTAL order over
|
|
23
|
+
`[key.class.name, key.to_s]`:
|
|
24
|
+
|
|
25
|
+
* Homogeneous String keys sort exactly as a plain string sort.
|
|
26
|
+
* Heterogeneous keys (strings, symbols, integers, nil, complex
|
|
27
|
+
collections — YAML mappings allow any key type) order by class
|
|
28
|
+
name first, then string form: the order is deterministic and
|
|
29
|
+
total, and the sort cannot raise on mixed types (the pre-0.3.113
|
|
30
|
+
`keys.sort` crashed on the yaml-test-suite's mixed-key shapes).
|
|
31
|
+
|
|
32
|
+
The sort is applied recursively: nested mappings, and mappings
|
|
33
|
+
inside arrays, sort by the same law.
|
|
34
|
+
|
|
35
|
+
== 3. Equivalence contract
|
|
36
|
+
|
|
37
|
+
The canonical form respects key order: two documents differing only
|
|
38
|
+
in mapping key order produce identical canonical bytes (pinned by
|
|
39
|
+
the property suite's shuffled-YAML leg).
|
|
40
|
+
|
|
41
|
+
== 4. Fixed point
|
|
42
|
+
|
|
43
|
+
format(format(x)) == format(x) for every accepted document (the
|
|
44
|
+
property suite pins this across the generated corpus and the
|
|
45
|
+
yaml-test-suite gate across the vendored suite).
|
data/lib/canon/version.rb
CHANGED
|
@@ -5,8 +5,12 @@ module Canon
|
|
|
5
5
|
# Attribute handler for C14N 1.1
|
|
6
6
|
# Handles attribute processing per spec
|
|
7
7
|
class AttributeHandler
|
|
8
|
-
def initialize(encoder)
|
|
8
|
+
def initialize(encoder, inherit_xml_attributes: true)
|
|
9
9
|
@encoder = encoder
|
|
10
|
+
# C14N 1.1 inherits xml:lang/xml:base from omitted ancestors in
|
|
11
|
+
# subsets; Exclusive C14N drops all xml:* inheritance (the
|
|
12
|
+
# spec's own point) and disables it here.
|
|
13
|
+
@inherit_xml_attributes = inherit_xml_attributes
|
|
10
14
|
end
|
|
11
15
|
|
|
12
16
|
# Process attribute axis of an element
|
|
@@ -35,7 +39,7 @@ module Canon
|
|
|
35
39
|
attributes = element.sorted_attribute_nodes.select(&:in_node_set?)
|
|
36
40
|
|
|
37
41
|
# Add inherited simple inheritable attributes if needed
|
|
38
|
-
if omitted_ancestors.any?
|
|
42
|
+
if @inherit_xml_attributes && omitted_ancestors.any?
|
|
39
43
|
inherited = collect_inherited_attributes(element, omitted_ancestors)
|
|
40
44
|
attributes = merge_attributes(attributes, inherited)
|
|
41
45
|
end
|
data/lib/canon/xml/c14n.rb
CHANGED
|
@@ -108,54 +108,15 @@ module Canon
|
|
|
108
108
|
def self.canonicalize_subset(xml, xpath, with_comments: false)
|
|
109
109
|
root_node = DataModel.from_xml(xml)
|
|
110
110
|
|
|
111
|
-
#
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
# Evaluate XPath and mark matched nodes
|
|
111
|
+
# Evaluate XPath and mark the node-set (the C14N marking law:
|
|
112
|
+
# matched nodes and their descendants; the root always rides)
|
|
115
113
|
matched = XPathEngine.evaluate(root_node, xpath)
|
|
116
|
-
|
|
117
|
-
# If XPath matches root or is empty, fall back to full canonicalization
|
|
118
|
-
if matched.empty?
|
|
119
|
-
mark_all_nodes(root_node, true)
|
|
120
|
-
else
|
|
121
|
-
# Mark matched nodes and their ancestors/descendants
|
|
122
|
-
mark_subset(root_node, matched)
|
|
123
|
-
end
|
|
114
|
+
NodeSetMarker.mark(root_node, matched)
|
|
124
115
|
|
|
125
116
|
# Process to canonical form
|
|
126
117
|
processor = Processor.new(with_comments: with_comments)
|
|
127
118
|
processor.process(root_node)
|
|
128
119
|
end
|
|
129
|
-
|
|
130
|
-
class << self
|
|
131
|
-
private
|
|
132
|
-
|
|
133
|
-
# Recursively set in_node_set on all nodes
|
|
134
|
-
def mark_all_nodes(node, value)
|
|
135
|
-
node.in_node_set = value
|
|
136
|
-
node.children.each { |child| mark_all_nodes(child, value) }
|
|
137
|
-
end
|
|
138
|
-
|
|
139
|
-
# Mark matched nodes and all required supporting nodes.
|
|
140
|
-
#
|
|
141
|
-
# Per W3C C14N 1.1, only nodes in the node-set are rendered.
|
|
142
|
-
# Ancestors not in the node-set become "omitted ancestors" —
|
|
143
|
-
# the Processor handles namespace/attribute inheritance from them.
|
|
144
|
-
def mark_subset(root_node, matched)
|
|
145
|
-
# Mark matched nodes and their descendants
|
|
146
|
-
matched.each do |node|
|
|
147
|
-
mark_node_and_descendants(node)
|
|
148
|
-
end
|
|
149
|
-
|
|
150
|
-
# Root node is always in the set so processing starts
|
|
151
|
-
root_node.in_node_set = true
|
|
152
|
-
end
|
|
153
|
-
|
|
154
|
-
def mark_node_and_descendants(node)
|
|
155
|
-
node.in_node_set = true
|
|
156
|
-
node.children.each { |child| mark_node_and_descendants(child) }
|
|
157
|
-
end
|
|
158
|
-
end
|
|
159
120
|
end
|
|
160
121
|
end
|
|
161
122
|
end
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Canon
|
|
4
|
+
module Xml
|
|
5
|
+
# Exclusive XML Canonicalization (W3C REC xml-exc-c14n-20080702),
|
|
6
|
+
# the Ruby reference processor. Differs from C14N 1.1 only in the
|
|
7
|
+
# namespace axis: namespaces render when VISIBLY UTILIZED (the
|
|
8
|
+
# element's own prefix or an attribute's prefix) or named in the
|
|
9
|
+
# InclusiveNamespaces PrefixList — see ExclusiveNamespaceHandler.
|
|
10
|
+
#
|
|
11
|
+
# ExclusiveC14n.canonicalize(xml)
|
|
12
|
+
# ExclusiveC14n.canonicalize(xml, inclusive_ns_prefixes:
|
|
13
|
+
# ["#default", "ds"])
|
|
14
|
+
# ExclusiveC14n.canonicalize(xml, xpath: "(//. | //@*)
|
|
15
|
+
# [ancestor-or-self::x]")
|
|
16
|
+
#
|
|
17
|
+
# The leptris native exclusive lane rides this face once the
|
|
18
|
+
# engine ships it (the canon acceptance protocol: byte-identity
|
|
19
|
+
# through this reference first).
|
|
20
|
+
class ExclusiveC14n
|
|
21
|
+
# Canonicalize a document (or an XPath-selected subset)
|
|
22
|
+
# @param xml [String] XML document as string
|
|
23
|
+
# @param with_comments [Boolean] Include comments in canonical form
|
|
24
|
+
# @param inclusive_ns_prefixes [Array<String>] InclusiveNamespaces
|
|
25
|
+
# PrefixList tokens (`#default`, bare prefixes)
|
|
26
|
+
# @param xpath [String, nil] subset selection (C14N node-set
|
|
27
|
+
# semantics); nil canonicalizes the whole document
|
|
28
|
+
# @return [String] Canonical form in UTF-8
|
|
29
|
+
def self.canonicalize(xml, with_comments: false,
|
|
30
|
+
inclusive_ns_prefixes: [], xpath: nil)
|
|
31
|
+
root_node = DataModel.from_xml(xml)
|
|
32
|
+
|
|
33
|
+
if xpath
|
|
34
|
+
if (idiom = c14n_selection_idiom(xpath))
|
|
35
|
+
matched = select_ancestor_or_self(root_node, *idiom)
|
|
36
|
+
NodeSetMarker.mark(root_node, matched, keep_root: false)
|
|
37
|
+
else
|
|
38
|
+
matched = XPathEngine.evaluate(root_node, xpath)
|
|
39
|
+
NodeSetMarker.mark(root_node, matched)
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
processor = Processor.new(
|
|
44
|
+
with_comments: with_comments,
|
|
45
|
+
namespace_handler: ExclusiveNamespaceHandler.new(
|
|
46
|
+
CharacterEncoder.new, inclusive_ns_prefixes
|
|
47
|
+
),
|
|
48
|
+
inherit_xml_attributes: false,
|
|
49
|
+
)
|
|
50
|
+
processor.process(root_node)
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
# The C14N selection idiom the suite drives:
|
|
54
|
+
# (//. | //@* | //namespace::*)[ancestor-or-self::pfx:name]
|
|
55
|
+
# — every node in the subtrees rooted at pfx:name. Returns
|
|
56
|
+
# [prefix, local_name], or nil for anything else.
|
|
57
|
+
def self.c14n_selection_idiom(xpath)
|
|
58
|
+
# the suite's .xpath files are XML documents wrapping the
|
|
59
|
+
# expression (with prefix declarations and inline comments)
|
|
60
|
+
text = xpath.sub(/\A<\?xml[^?]*\?>/, "")
|
|
61
|
+
.gsub(/<!--[\s\S]*?-->/, "")
|
|
62
|
+
.gsub(/<[^>]*>/, " ")
|
|
63
|
+
.squeeze(" ")
|
|
64
|
+
m = text.match(%r{\(//\.\s*\|\s*//@\*\s*\|\s*//namespace::\*\)\s*\[\s*ancestor-or-self::([A-Za-z_][\w.-]*)(?::([\w.-]+))?\s*\]\s*\z})
|
|
65
|
+
return nil unless m
|
|
66
|
+
|
|
67
|
+
m[2] ? [m[1], m[2]] : [nil, m[1]]
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
# The subtree roots whose ancestor-or-self chain touches
|
|
71
|
+
# prefix:name (resolved through the document's bindings), with
|
|
72
|
+
# deeper matches subsumed by their topmost ancestor.
|
|
73
|
+
def self.select_ancestor_or_self(root_node, prefix, local_name)
|
|
74
|
+
target_uri = nil
|
|
75
|
+
walk = lambda do |el|
|
|
76
|
+
if el.respond_to?(:namespace_nodes)
|
|
77
|
+
el.namespace_nodes.each do |ns|
|
|
78
|
+
next if ns.default_namespace?
|
|
79
|
+
|
|
80
|
+
if ns.prefix == prefix || (prefix.nil? && ns.prefix.nil?)
|
|
81
|
+
target_uri = ns.uri
|
|
82
|
+
break
|
|
83
|
+
end
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
break if target_uri
|
|
87
|
+
|
|
88
|
+
el.children.each { |c| walk.call(c) if c.node_type == :element }
|
|
89
|
+
end
|
|
90
|
+
root_node.children.each { |c| walk.call(c) if c.node_type == :element }
|
|
91
|
+
|
|
92
|
+
tops = []
|
|
93
|
+
collect = lambda do |el|
|
|
94
|
+
hit = target_uri.nil? ? false : (el.namespace_uri == target_uri)
|
|
95
|
+
hit ||= prefix.nil? && el.name == local_name
|
|
96
|
+
if hit
|
|
97
|
+
tops << el
|
|
98
|
+
else
|
|
99
|
+
el.children.each { |c| collect.call(c) if c.node_type == :element }
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
root_node.children.each { |c| collect.call(c) if c.node_type == :element }
|
|
103
|
+
tops
|
|
104
|
+
end
|
|
105
|
+
end
|
|
106
|
+
end
|
|
107
|
+
end
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Canon
|
|
4
|
+
module Xml
|
|
5
|
+
# Exclusive XML Canonicalization namespace law (W3C REC
|
|
6
|
+
# xml-exc-c14n-20080702). Unlike C14N 1.1's render-all-in-scope
|
|
7
|
+
# namespace axis, the exclusive law renders only namespaces that
|
|
8
|
+
# are:
|
|
9
|
+
#
|
|
10
|
+
# 1. VISIBLY UTILIZED — the element's own prefix, or the prefix
|
|
11
|
+
# of an in-node-set attribute of the element; or
|
|
12
|
+
# 2. IN THE InclusiveNamespaces PrefixList — the transform
|
|
13
|
+
# parameter that re-imports ancestor context.
|
|
14
|
+
#
|
|
15
|
+
# and then only when no rendered ancestor already carries the
|
|
16
|
+
# same prefix=uri binding (the ancestor-dedup law C14N shares).
|
|
17
|
+
# `#default` in the prefix list imports the default namespace.
|
|
18
|
+
class ExclusiveNamespaceHandler < NamespaceHandler
|
|
19
|
+
# @param encoder [CharacterEncoder]
|
|
20
|
+
# @param inclusive_ns_prefixes [Array<String>] the
|
|
21
|
+
# InclusiveNamespaces PrefixList tokens (`#default`,
|
|
22
|
+
# bare prefixes; a trailing ':' on a token is spelling)
|
|
23
|
+
def initialize(encoder, inclusive_ns_prefixes = [])
|
|
24
|
+
super(encoder)
|
|
25
|
+
@inclusive = inclusive_ns_prefixes.map { |p| p.to_s.chomp(":") }
|
|
26
|
+
@inclusive_default = @inclusive.delete("#default")
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
private
|
|
30
|
+
|
|
31
|
+
# The exclusive rendering set: visibly utilized ∪ inclusive
|
|
32
|
+
# list, intersected with the element's in-scope namespace nodes.
|
|
33
|
+
def renderable_namespaces(element)
|
|
34
|
+
utilized = visibly_utilized_prefixes(element)
|
|
35
|
+
element.namespace_nodes.sort_by(&:local_name).select do |ns|
|
|
36
|
+
next false unless ns.in_node_set?
|
|
37
|
+
|
|
38
|
+
if ns.default_namespace?
|
|
39
|
+
@inclusive_default
|
|
40
|
+
else
|
|
41
|
+
utilized.include?(ns.prefix) || @inclusive.include?(ns.prefix)
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# The element's own prefix plus the prefixes of its in-node-set
|
|
47
|
+
# attributes (xml:* excluded — the XML namespace never renders).
|
|
48
|
+
def visibly_utilized_prefixes(element)
|
|
49
|
+
prefixes = []
|
|
50
|
+
prefixes << element.prefix if element.prefix && !element.prefix.empty?
|
|
51
|
+
element.attribute_nodes.each do |attr|
|
|
52
|
+
next unless attr.in_node_set?
|
|
53
|
+
next if attr.xml_base?
|
|
54
|
+
next if attr.name == "xml:base"
|
|
55
|
+
|
|
56
|
+
p = attr.prefix.to_s
|
|
57
|
+
prefixes << p unless p.empty? || p == "xmlns"
|
|
58
|
+
end
|
|
59
|
+
prefixes.uniq
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
# The exclusive law replaces C14N's render-all selection.
|
|
63
|
+
# (the `ns` param name is the file's shared vocabulary)
|
|
64
|
+
def should_skip_namespace?(ns, elem, parent_element) # rubocop:disable Naming/MethodParameterName
|
|
65
|
+
return true if ns.xml_namespace?
|
|
66
|
+
|
|
67
|
+
namespace_declared_by_ancestor?(ns, elem, parent_element)
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
# The empty-default emission follows the inclusive list: only
|
|
71
|
+
# when #default is imported and the in-scope default is
|
|
72
|
+
# non-empty does xmlns="" re-appear.
|
|
73
|
+
def should_emit_empty_default_namespace?(_element, namespaces,
|
|
74
|
+
parent_element)
|
|
75
|
+
return false unless @inclusive_default
|
|
76
|
+
return false if namespaces.first&.default_namespace?
|
|
77
|
+
return false unless parent_element
|
|
78
|
+
|
|
79
|
+
parent_default_ns = parent_element.namespace_nodes.find do |ns|
|
|
80
|
+
ns.default_namespace? && ns.in_node_set?
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
parent_default_ns && !parent_default_ns.uri.empty?
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
end
|
|
87
|
+
end
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Canon
|
|
4
|
+
module Xml
|
|
5
|
+
# Marks the XPath node-set on a data-model tree for both C14N
|
|
6
|
+
# lanes (C14n and ExclusiveC14n): all nodes cleared, the matched
|
|
7
|
+
# nodes and their descendants marked, the root always kept so
|
|
8
|
+
# processing starts.
|
|
9
|
+
module NodeSetMarker
|
|
10
|
+
module_function
|
|
11
|
+
|
|
12
|
+
# keep_root: the C14N subset lane pins the root into the set so
|
|
13
|
+
# processing starts there (its selectors include it); the
|
|
14
|
+
# exclusive corpus's selectors exclude the document element and
|
|
15
|
+
# the output renders the included subtrees as fragments.
|
|
16
|
+
def mark(root_node, matched, keep_root: true)
|
|
17
|
+
mark_all(root_node, false)
|
|
18
|
+
if matched.nil? || matched.empty?
|
|
19
|
+
mark_all(root_node, true)
|
|
20
|
+
else
|
|
21
|
+
matched.each { |node| mark_node_and_descendants(node) }
|
|
22
|
+
root_node.in_node_set = true if keep_root
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def mark_all(node, value)
|
|
27
|
+
node.in_node_set = value
|
|
28
|
+
node.children.each { |child| mark_all(child, value) }
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def mark_node_and_descendants(node)
|
|
32
|
+
node.in_node_set = true
|
|
33
|
+
node.children.each { |child| mark_node_and_descendants(child) }
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
data/lib/canon/xml/processor.rb
CHANGED
|
@@ -5,11 +5,14 @@ module Canon
|
|
|
5
5
|
# C14N 1.1 processor
|
|
6
6
|
# Processes XPath data model and generates canonical form
|
|
7
7
|
class Processor
|
|
8
|
-
def initialize(with_comments: false
|
|
8
|
+
def initialize(with_comments: false, namespace_handler: nil,
|
|
9
|
+
inherit_xml_attributes: true)
|
|
9
10
|
@with_comments = with_comments
|
|
10
11
|
@encoder = CharacterEncoder.new
|
|
11
|
-
@namespace_handler = NamespaceHandler.new(@encoder)
|
|
12
|
-
@attribute_handler = AttributeHandler.new(
|
|
12
|
+
@namespace_handler = namespace_handler || NamespaceHandler.new(@encoder)
|
|
13
|
+
@attribute_handler = AttributeHandler.new(
|
|
14
|
+
@encoder, inherit_xml_attributes: inherit_xml_attributes
|
|
15
|
+
)
|
|
13
16
|
@xml_base_handler = XmlBaseHandler.new
|
|
14
17
|
end
|
|
15
18
|
|
data/lib/canon/xml.rb
CHANGED
|
@@ -21,10 +21,13 @@ module Canon
|
|
|
21
21
|
autoload :CharacterEncoder, "canon/xml/character_encoder"
|
|
22
22
|
autoload :DataModel, "canon/xml/data_model"
|
|
23
23
|
autoload :ElementMatcher, "canon/xml/element_matcher"
|
|
24
|
+
autoload :ExclusiveC14n, "canon/xml/exclusive_c14n"
|
|
25
|
+
autoload :ExclusiveNamespaceHandler, "canon/xml/exclusive_namespace_handler"
|
|
24
26
|
autoload :LineRangeMapper, "canon/xml/line_range_mapper"
|
|
25
27
|
autoload :NamespaceHandler, "canon/xml/namespace_handler"
|
|
26
28
|
autoload :NamespaceHelper, "canon/xml/namespace_helper"
|
|
27
29
|
autoload :Node, "canon/xml/node"
|
|
30
|
+
autoload :NodeSetMarker, "canon/xml/node_set_marker"
|
|
28
31
|
autoload :Nodes, "canon/xml/nodes"
|
|
29
32
|
autoload :Processor, "canon/xml/processor"
|
|
30
33
|
autoload :Sax, "canon/xml/sax"
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
require_relative "../lib/canon"
|
|
2
|
+
require "nokogiri"
|
|
3
|
+
|
|
4
|
+
# Diagnostic: the ExclusiveC14n differential against nokogiri/libxml2
|
|
5
|
+
# over the vendored merlin-c14n-two corpus. NOT a battery gate — see
|
|
6
|
+
# TODO.sota/05 for the residual map (the extended-predicate subset
|
|
7
|
+
# semantics and the default-ns law disposition).
|
|
8
|
+
CORPUS = File.join(__dir__, "../spec/fixtures/corpora/exc-c14n")
|
|
9
|
+
results = Hash.new { |h, k| h[k] = [] }
|
|
10
|
+
|
|
11
|
+
Dir.glob("#{CORPUS}/*.xml").each do |input|
|
|
12
|
+
base = File.basename(input, ".xml")
|
|
13
|
+
xpath_file = "#{CORPUS}/#{base}.xpath"
|
|
14
|
+
ns_file = "#{CORPUS}/#{base}.ns"
|
|
15
|
+
ns_list = File.exist?(ns_file) ? File.read(ns_file).split : []
|
|
16
|
+
|
|
17
|
+
doc = Nokogiri::XML(File.read(input))
|
|
18
|
+
# compact product: strip whitespace-only text nodes (canon's C14N
|
|
19
|
+
# lane parses noblanks; the compact bytes are the product)
|
|
20
|
+
doc.xpath("//text()").select { |t| t.content.strip.empty? }.each(&:remove)
|
|
21
|
+
|
|
22
|
+
if File.exist?(xpath_file)
|
|
23
|
+
# subset case: canon evaluates its own XPath marking
|
|
24
|
+
mine = Canon::Xml::ExclusiveC14n.canonicalize(
|
|
25
|
+
File.read(input), xpath: File.read(xpath_file).strip,
|
|
26
|
+
inclusive_ns_prefixes: ns_list
|
|
27
|
+
)
|
|
28
|
+
# oracle: subtree selection — evaluate the same selector, take
|
|
29
|
+
# the topmost matched element(s), canonicalize exclusive with
|
|
30
|
+
# the same prefix list
|
|
31
|
+
xp = Nokogiri::XML(File.read(xpath_file))
|
|
32
|
+
prefix_map = xp.root.namespaces.transform_values { |v| v.to_s.sub(%r{^xmlns(?::)?}, "") }
|
|
33
|
+
expr = xp.root.text.strip
|
|
34
|
+
matched = doc.xpath(expr, prefix_map).grep(Nokogiri::XML::Element)
|
|
35
|
+
tops = matched.reject { |n| n.ancestors.any? { |a| a.element? && matched.include?(a) } }
|
|
36
|
+
ref = tops.map do |t|
|
|
37
|
+
t.canonicalize(Nokogiri::XML::XML_C14N_EXCLUSIVE_1_0, ns_list, false)
|
|
38
|
+
end.join
|
|
39
|
+
else
|
|
40
|
+
mine = Canon::Xml::ExclusiveC14n.canonicalize(
|
|
41
|
+
File.read(input), inclusive_ns_prefixes: ns_list
|
|
42
|
+
)
|
|
43
|
+
ref = doc.root.canonicalize(Nokogiri::XML::XML_C14N_EXCLUSIVE_1_0, ns_list, false)
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
if mine == ref
|
|
47
|
+
results[:match] << base
|
|
48
|
+
elsif base == "merlin-c14n-two-10"
|
|
49
|
+
File.write("/tmp/exc_ref.txt", ref)
|
|
50
|
+
File.write("/tmp/exc_mine.txt", mine)
|
|
51
|
+
elsif mine.include?("xmlns=") == ref.include?("xmlns=") &&
|
|
52
|
+
mine.gsub(/ xmlns(:\w+)?="[^"]*"/, "") == ref.gsub(/ xmlns(:\w+)?="[^"]*"/, "")
|
|
53
|
+
results[:default_ns_only] << base
|
|
54
|
+
else
|
|
55
|
+
results[:differ] << base
|
|
56
|
+
end
|
|
57
|
+
rescue StandardError => e
|
|
58
|
+
results[:error] << "#{base} (#{e.class.name.split('::').last}: #{e.message[0, 60]})"
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
results.each { |k, v| puts "#{k}: #{v.count} #{v.first(6).inspect}" }
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: canon
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.3.
|
|
4
|
+
version: 0.3.114
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Ribose Inc.
|
|
@@ -174,6 +174,8 @@ files:
|
|
|
174
174
|
- docs/INDEX.adoc
|
|
175
175
|
- docs/_config.yml
|
|
176
176
|
- docs/_references/comparison-theory.adoc
|
|
177
|
+
- docs/_references/html-canonical-form.adoc
|
|
178
|
+
- docs/_references/yaml-canonical-form.adoc
|
|
177
179
|
- docs/advanced/diff-classification.adoc
|
|
178
180
|
- docs/advanced/diff-pipeline.adoc
|
|
179
181
|
- docs/advanced/extending-canon.adoc
|
|
@@ -412,10 +414,13 @@ files:
|
|
|
412
414
|
- lib/canon/xml/data_model.rb
|
|
413
415
|
- lib/canon/xml/digest_gate.rb
|
|
414
416
|
- lib/canon/xml/element_matcher.rb
|
|
417
|
+
- lib/canon/xml/exclusive_c14n.rb
|
|
418
|
+
- lib/canon/xml/exclusive_namespace_handler.rb
|
|
415
419
|
- lib/canon/xml/line_range_mapper.rb
|
|
416
420
|
- lib/canon/xml/namespace_handler.rb
|
|
417
421
|
- lib/canon/xml/namespace_helper.rb
|
|
418
422
|
- lib/canon/xml/node.rb
|
|
423
|
+
- lib/canon/xml/node_set_marker.rb
|
|
419
424
|
- lib/canon/xml/nodes.rb
|
|
420
425
|
- lib/canon/xml/nodes/attribute_node.rb
|
|
421
426
|
- lib/canon/xml/nodes/comment_node.rb
|
|
@@ -445,6 +450,7 @@ files:
|
|
|
445
450
|
- lib/tasks/performance_helpers.rb
|
|
446
451
|
- lib/tasks/performance_report.rb
|
|
447
452
|
- lib/xml-c14n.rb
|
|
453
|
+
- scripts/exc_c14n_differential.rb
|
|
448
454
|
- sig/xml/c14n.rbs
|
|
449
455
|
homepage: https://github.com/lutaml/canon
|
|
450
456
|
licenses:
|