moxml 0.1.25 → 0.2.0

data/docs/signature/security.md

@@ -0,0 +1,143 @@
+# Security considerations
+
+XML signature is a complex spec with a long history of attacks. This
+document explains what `Moxml::Signature` does to mitigate them and
+what the application is responsible for.
+
+## Built-in mitigations
+
+### Best Practice 1 — authenticate before transforms (DONE)
+
+The Verifier validates `SignatureValue` before running any reference
+transforms. A hostile signature with expensive XSLT or XPath transforms
+cannot consume server resources unless its `SignatureValue` verifies
+against a key the application trusts.
+
+### Best Practice 26 — HMAC truncation floor (DONE)
+
+`HMACOutputLength` values below `max(hash_bits/2, 80)` are rejected
+during algorithm instantiation (spec §4.4.2). Signatures with
+sub-minimum truncation are deemed invalid.
+
+### Best Practice 11 — opaque certificate handling (DONE)
+
+`X509Certificate` payloads are decoded as raw DER and passed to
+`OpenSSL::X509::Certificate.new`. We never re-encode certificates, so
+the signature on the certificate itself is preserved.
+
+## What the application must do
+
+The library cannot make trust decisions; the application must.
+
+### Best Practice 2 — establish trust in the key
+
+Just because `SignatureValue` verifies against a public key in
+`KeyInfo` does NOT mean the signature should be trusted. The key must
+come from a trusted source:
+
+```ruby
+# Application-side: pin the expected key, ignore KeyInfo entirely
+result = Moxml::Signature.verify(
+  context: ctx,
+  document: doc,
+  key: trusted_public_key,   # do NOT rely on KeyInfo extraction
+)
+
+# Or, with certificate validation:
+cert = OpenSSL::X509::Certificate.new(cert_der)
+store = OpenSSL::X509::Store.new
+store.add_trust_file("cacert.pem")
+unless store.verify(cert)
+  raise "certificate chain invalid"
+end
+
+result = Moxml::Signature.verify(
+  context: ctx,
+  document: doc,
+  key: cert.public_key,
+)
+```
+
+### Best Practice 12 — see what was signed
+
+Use `SingleVerificationResult#references` to inspect what the signature
+actually covers:
+
+```ruby
+result.results.first.references.each do |ref|
+  puts "#{ref.uri}: #{ref.valid?}"
+  # Confirm ref.uri matches what the application expects to be signed.
+  # Wrapping attacks work by getting the verifier to confirm a different
+  # node than the application acts on.
+end
+```
+
+### Best Practice 14 — check name AND position
+
+When checking a reference URI, don't just verify the element name. A
+wrapping attack moves the signed element into an `<Object>` and points
+the reference at it; the application then acts on a different (unsigned)
+element with the same name.
+
+### Best Practice 8 — control external references
+
+External URI dereferencing is **not enabled by default**. If you need
+it, wrap the resolver in a policy that:
+
+- Allows only same-document URIs (`#id`, `""`)
+- Allows only specific schemes (`https://`, never `file://`)
+- Caps size and timeout
+- Disallows query parameters that mutate server state
+
+## What this library does NOT do
+
+### Not a chain validator
+
+`Moxml::Signature` does not validate X.509 certificate chains, check
+revocation, or evaluate certificate policies. The application is
+responsible for these (see Best Practice 2 above).
+
+### Not a timestamp authority
+
+Long-lived signatures need RFC 3161 timestamps from a TSA. This library
+does not implement timestamp verification.
+
+### Not a wrapping-attack detector
+
+Wrapping attacks succeed when the application acts on a different node
+than the one the signature actually covers. The library returns the
+list of references; the application must verify they cover the right
+nodes.
+
+## SHA-1 warning
+
+The W3C spec marks SHA-1 as REQUIRED for backwards compatibility but
+DISCOURAGED for new signatures. Cryptanalytic advances (SHAttered,
+2017) demonstrated practical collisions. Use SHA-256 or stronger for
+new signatures.
+
+## HMAC security
+
+HMAC signatures require a shared secret. Any verifier with the secret
+can forge signatures. Use distinct keys for signing vs. encryption
+(Best Practice 27).
+
+## Limited XPath support
+
+The library implements the Enveloped Signature transform directly
+(no XPath). XPath Filter and XSLT transforms are not implemented;
+Best Practices 3, 5, 6, 22 say avoid them.
+
+## Performance and DoS
+
+Canonicalization is O(N) for tree size. XSLT and complex XPath are
+not supported, eliminating the most common DoS vectors (Best Practices
+3, 5, 6). Wrap calls in a timeout if processing untrusted input:
+
+```ruby
+require "timeout"
+
+Timeout.timeout(5) do
+  Moxml::Signature.verify(context: ctx, document: untrusted_doc, key:)
+end
+```

data/examples/signature/auto_key_extraction.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+# Example: auto-extract the verification key from a KeyInfo that
+# contains an embedded X509Certificate. The Verifier derives the
+# OpenSSL key without application help.
+#
+# Run with: bundle exec ruby examples/signature/auto_key_extraction.rb
+
+require "moxml"
+require "moxml/signature"
+
+ctx = Moxml.new(:nokogiri)
+
+fixture = File.expand_path(
+  "../../spec/fixtures/xmldsig/sign3-result.xml",
+  __dir__,
+)
+xml = File.read(fixture)
+doc = ctx.parse(xml)
+
+# No explicit key — Verifier auto-extracts from X509Certificate.
+result = Moxml::Signature.verify(context: ctx, document: doc)
+puts "Auto-extracted from X509Certificate: #{result.valid?}"
+puts "Signature count: #{result.signature_count}"

data/examples/signature/enveloped_rsa.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+# Example: enveloped RSA-SHA256 signature with verification.
+#
+# Run with: bundle exec ruby examples/signature/enveloped_rsa.rb
+
+require "moxml"
+require "moxml/signature"
+require "openssl"
+
+ctx = Moxml.new(:nokogiri)
+key = OpenSSL::PKey::RSA.generate(2048)
+
+doc = ctx.parse("<doc><greeting>Hello, World!</greeting></doc>")
+puts "=== Before signing ==="
+puts doc.to_xml
+
+signature = Moxml::Signature.sign(
+  context: ctx,
+  document: doc,
+  key: key,
+  signature_method: "http://www.w3.org/2001/04/xmldsig-more#rsa-sha256",
+  canonicalization_method: "http://www.w3.org/2001/10/xml-exc-c14n#",
+  digest_method: "http://www.w3.org/2001/04/xmlenc#sha256",
+  reference_uri: "",
+  transforms: ["http://www.w3.org/2000/09/xmldsig#enveloped-signature"],
+)
+
+serialized = Moxml::Signature::Serializer.new(context: ctx).serialize(signature)
+doc.root.add_child(serialized.root)
+
+puts ""
+puts "=== After signing ==="
+puts doc.to_xml
+
+result = Moxml::Signature.verify(context: ctx, document: doc, key: key)
+puts ""
+puts "=== Verification ==="
+puts "Valid: #{result.valid?}"
+
+# Tamper test
+doc2 = ctx.parse(doc.to_xml(indent: 0))
+doc2.at_xpath("//greeting").text = "Goodbye!"
+tampered = Moxml::Signature.verify(context: ctx, document: doc2, key: key)
+puts "Tampered: #{tampered.valid?} (expected false)"

data/examples/signature/hmac.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+# Example: HMAC-SHA256 signature with a shared secret.
+#
+# Run with: bundle exec ruby examples/signature/hmac.rb
+
+require "moxml"
+require "moxml/signature"
+
+ctx = Moxml.new(:nokogiri)
+secret = "super-secret-shared-key"
+
+doc = ctx.parse("<message><body>hello</body></message>")
+
+signature = Moxml::Signature.sign(
+  context: ctx,
+  document: doc,
+  key: secret,
+  signature_method: "http://www.w3.org/2001/04/xmldsig-more#hmac-sha256",
+  canonicalization_method: "http://www.w3.org/2001/10/xml-exc-c14n#",
+  digest_method: "http://www.w3.org/2001/04/xmlenc#sha256",
+  reference_uri: "",
+  transforms: ["http://www.w3.org/2000/09/xmldsig#enveloped-signature"],
+)
+
+serialized = Moxml::Signature::Serializer.new(context: ctx).serialize(signature)
+doc.root.add_child(serialized.root)
+
+result = Moxml::Signature.verify(context: ctx, document: doc, key: secret)
+puts "Valid: #{result.valid?}"
+
+wrong = Moxml::Signature.verify(context: ctx, document: doc, key: "wrong")
+puts "Wrong secret: #{wrong.valid?} (expected false)"

data/lib/compat/opal/moxml_boot.rb

@@ -24,6 +24,9 @@ require "moxml/namespace"
 require "moxml/doctype"
 require "moxml/entity_reference"
 require "moxml/entity_registry"
+require "moxml/entity"
+require "moxml/entity/reference"
+require "moxml/xml_emitter"
 require "moxml/adapter"
 require "moxml/adapter/base"
 require "moxml/adapter/customized_rexml"
@@ -36,6 +39,8 @@ require "moxml/sax/block_handler"
 require "moxml/sax/namespace_splitter"
 require "moxml/adapter/rexml"
 require "moxml/adapter/customized_oga"
+require "moxml/adapter/customized_oga/entity_decoder"
+require "moxml/adapter/customized_oga/raw_value_override"
 require "moxml/adapter/customized_oga/xml_declaration"
 require "moxml/adapter/customized_oga/xml_generator"
 require "moxml/adapter/oga"

data/lib/moxml/adapter/base.rb

@@ -5,62 +5,29 @@ module Moxml
     class Base
       # include XmlUtils
 
-      # Entity marker for adapters that resolve entities during parsing.
-      # U+FFFC (Object Replacement Character) + U+FEFF (BOM) is a two-character
-      # sentinel chosen because this exact sequence followed by a valid entity
-      # name pattern is vanishingly unlikely in real XML content.
-      # Non-standard entities like © are converted to this marker before
-      # parsing, then restored during serialization.
-      # Standard XML entities (& < > " ') are NOT converted.
-      ENTITY_MARKER = "\u{FFFC}\u{FEFF}"
-      ENTITY_NAME_PATTERN = "[a-zA-Z_][\\w.:-]*"
-      ENTITY_NAME_RE = /&(#{ENTITY_NAME_PATTERN});/
-      ENTITY_MARKER_RE = /\u{FFFC}\u{FEFF}(#{ENTITY_NAME_PATTERN});/
-      SERIALIZED_ENTITY_MARKER_RE = /(#{ENTITY_NAME_PATTERN});/
-      STANDARD_ENTITIES = %w[amp lt gt quot apos].freeze
+      # Entity round-trip pipeline lives in Moxml::Entity; these
+      # constants and methods stay on the adapter protocol so
+      # adapters and wrappers keep their calling convention.
+      ENTITY_MARKER = Entity::MARKER
+      ENTITY_NAME_PATTERN = Entity::NAME_PATTERN
+      ENTITY_NAME_RE = Entity::NAME_RE
+      ENTITY_MARKER_RE = Entity::MARKER_RE
+      SERIALIZED_ENTITY_MARKER_RE = Entity::SERIALIZED_MARKER_RE
+      STANDARD_ENTITIES = Entity::STANDARD_ENTITIES
 
       class << self
         include XmlUtils
 
-        # Replace non-standard entity references with markers before parsing.
-        # Always returns a UTF-8 encoded string.
         def preprocess_entities(xml)
-          return "" if xml.nil?
-
-          str = if xml.encoding == Encoding::BINARY
-                  # Binary strings are assumed to be UTF-8. If the bytes are
-                  # not valid UTF-8, fall back to encoding as UTF-8 with
-                  # replacement to avoid raising on gsub.
-                  dup = xml.dup.force_encoding("UTF-8")
-                  if dup.valid_encoding?
-                    dup
-                  else
-                    xml.dup.encode("UTF-8",
-                                   "ASCII-8BIT", invalid: :replace, undef: :replace)
-                  end
-                elsif xml.encoding == Encoding::UTF_8
-                  xml
-                else
-                  xml.encode("UTF-8")
-                end
-          # Fast path: no `&` means no entity references to mark — skip
-          # the regex scan and string allocation entirely. The vast
-          # majority of XML payloads contain no entity references.
-          return str unless str.include?("&")
-
-          str.gsub(ENTITY_NAME_RE) do |match|
-            STANDARD_ENTITIES.include?(::Regexp.last_match(1)) ? match : "#{ENTITY_MARKER}#{::Regexp.last_match(1)};"
-          end
+          Entity.preprocess_entities(xml)
         end
 
-        # Restore entity markers back to named entity references.
-        def restore_entities(text)
-          return text unless text.is_a?(String)
+        def decode_entities(text)
+          Entity.decode_entities(text)
+        end
 
-          # Force UTF-8 encoding since markers are UTF-8 characters
-          str = text.encoding == Encoding::UTF_8 ? text : text.dup.force_encoding("UTF-8")
-          result = str.gsub(ENTITY_MARKER_RE, '&\1;')
-          result.gsub(SERIALIZED_ENTITY_MARKER_RE, '&\1;')
+        def restore_entities(text)
+          Entity.restore_entities(text)
         end
 
         def set_root(_doc, _element)
@@ -71,7 +38,7 @@ module Moxml
           )
         end
 
-        def parse(_xml, _options = {})
+        def parse(_xml, _options = {}, _context = nil)
           raise Moxml::NotImplementedError.new(
             "parse not implemented",
             feature: "parse",
@@ -172,12 +139,35 @@ namespace_validation_mode: :strict)
           create_native_entity_reference(name)
         end
 
+        # Mutation return contract: protocol methods that may change
+        # which native a wrapper tracks (set_attribute_name,
+        # set_namespace, set_attribute_value) always return the native
+        # the wrapper must keep tracking — the same object when
+        # mutated in place, a fresh object when the adapter recreates
+        # the node.
         def set_attribute_name(attribute, name)
           attribute.name = name
+          attribute
+        end
+
+        def set_namespace(_node, _namespace)
+          raise Moxml::NotImplementedError.new(
+            "set_namespace not implemented",
+            feature: "set_namespace",
+            adapter: name,
+          )
         end
 
         def set_attribute_value(attribute, value)
           attribute.value = value
+          attribute
+        end
+
+        # Remove a specific native attribute node from its owning
+        # element. Semantics (which attribute a name addresses) live
+        # in Moxml::AttributeResolver; this is the raw primitive.
+        def remove_attribute_native(attr)
+          attr.remove
         end
 
         def entity_reference_name(node)
@@ -212,6 +202,20 @@ namespace_validation_mode: :strict)
         # Called when a Declaration node is removed from a document.
         def remove_declaration(_native_doc); end
 
+        # Source line of a native node (1-based), or nil when the
+        # underlying backend does not track source positions.
+        # Adapters that track lines (Nokogiri, LibXML) override this.
+        def line_number(_node)
+          nil
+        end
+
+        # Local name of a native attribute node. Adapters whose natives
+        # carry qualified names override this to expose the local part;
+        # the wrapper composes the prefix.
+        def attribute_name(attr)
+          attr.name
+        end
+
         # Return the actual native node after an add_child operation.
         # Override for adapters where node identity may change (e.g., LibXML doc.root=).
         def actual_native(child_native, _parent_native)

data/lib/moxml/adapter/customized_leptris/declaration.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+module Moxml
+  module Adapter
+    module CustomizedLeptris
+      # Wrapper for an XML declaration. libleptris stores declaration
+      # data on the document struct (read-only, serialized via options)
+      # and has no first-class declaration node, so Moxml exposes this
+      # mutable value object instead.
+      class Declaration
+        attr_accessor :version, :encoding, :standalone, :parent_doc
+
+        def initialize(version = "1.0", encoding = "UTF-8", standalone = nil)
+          @version = version
+          @encoding = encoding
+          @standalone = standalone
+        end
+
+        def to_xml
+          XmlEmitter.declaration_xml(version, encoding, standalone)
+        end
+
+        def ==(other)
+          other.is_a?(self.class) &&
+            version == other.version &&
+            encoding == other.encoding &&
+            standalone == other.standalone
+        end
+      end
+    end
+  end
+end

data/lib/moxml/adapter/customized_leptris/doctype.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module Moxml
+  module Adapter
+    module CustomizedLeptris
+      # Wrapper for a programmatic DOCTYPE. libleptris only parses
+      # DOCTYPEs from source; it has no API to create one, so Moxml
+      # stores this value object in the document's attachments.
+      class Doctype
+        attr_accessor :name, :external_id, :system_id, :parent_doc
+
+        def initialize(name, external_id = nil, system_id = nil)
+          @name = name
+          @external_id = external_id
+          @system_id = system_id
+        end
+
+        def to_xml
+          XmlEmitter.doctype_xml(name, external_id, system_id)
+        end
+
+        def ==(other)
+          other.is_a?(self.class) &&
+            name == other.name &&
+            external_id == other.external_id &&
+            system_id == other.system_id
+        end
+      end
+    end
+  end
+end

data/lib/moxml/adapter/customized_leptris/entity_reference.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+module Moxml
+  module Adapter
+    module CustomizedLeptris
+      # libleptris expands the five built-in entities at parse time
+      # and has no custom ones; the adapter carries entity references
+      # as Moxml::Entity::Reference values over marker-bearing text.
+      EntityReference = ::Moxml::Entity::Reference
+    end
+  end
+end

data/lib/moxml/adapter/customized_leptris/text_segment.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module Moxml
+  module Adapter
+    module CustomizedLeptris
+      # A text run split out of a native text node that carried entity
+      # markers. Value object: the moxml contract exposes text nodes as
+      # separate children around entity references, while libleptris
+      # stores them as one marker-bearing text node.
+      class TextSegment
+        attr_accessor :parent
+
+        attr_reader :content
+
+        def initialize(content, parent = nil)
+          @content = content
+          @parent = parent
+        end
+
+        def ==(other)
+          other.is_a?(self.class) && content == other.content
+        end
+      end
+    end
+  end
+end

data/lib/moxml/adapter/customized_leptris.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Moxml
+  module Adapter
+    # Pure-Ruby node kinds that libleptris has no native representation
+    # for: XML declarations, programmatic DOCTYPEs, and entity references.
+    # The adapter stores them via NativeAttachment and serializes them
+    # from #to_xml.
+    module CustomizedLeptris
+      autoload :Declaration, "moxml/adapter/customized_leptris/declaration"
+      autoload :Doctype, "moxml/adapter/customized_leptris/doctype"
+      autoload :EntityReference, "moxml/adapter/customized_leptris/entity_reference"
+      autoload :TextSegment, "moxml/adapter/customized_leptris/text_segment"
+    end
+  end
+end

data/lib/moxml/adapter/customized_libxml/cdata.rb

@@ -5,19 +5,10 @@ module Moxml
     module CustomizedLibxml
       # Wrapper for LibXML CDATA section nodes
       class Cdata < Node
-        # Serialize as XML CDATA section
-        # LibXML auto-escapes content, we need to un-escape it
+        # libxml stores CDATA payload verbatim. Only the `]]>` end-marker
+        # needs splitting before re-wrapping.
         def to_xml
-          content = @native.content
-            .gsub("&quot;", '"')
-            .gsub("&apos;", "'")
-            .gsub("&lt;", "<")
-            .gsub("&gt;", ">")
-            .gsub("&amp;", "&")
-
-          # Handle CDATA end marker escaping (]]> becomes ]]]]><![CDATA[>)
-          # Replace all ]]> markers in the content before wrapping
-          escaped_content = content.gsub("]]>", "]]]]><![CDATA[>")
+          escaped_content = @native.content.gsub("]]>", "]]]]><![CDATA[>")
           "<![CDATA[#{escaped_content}]]>"
         end
       end

data/lib/moxml/adapter/customized_libxml/comment.rb

@@ -5,16 +5,9 @@ module Moxml
     module CustomizedLibxml
       # Wrapper for LibXML comment nodes
       class Comment < Node
-        # Serialize as XML comment
-        # LibXML auto-escapes content, we need to un-escape it
+        # libxml stores comment payload verbatim.
         def to_xml
-          content = @native.content
-            .gsub("&quot;", '"')
-            .gsub("&apos;", "'")
-            .gsub("&lt;", "<")
-            .gsub("&gt;", ">")
-            .gsub("&amp;", "&")
-          "<!--#{content}-->"
+          "<!--#{@native.content}-->"
         end
       end
     end

data/lib/moxml/adapter/customized_libxml/declaration.rb

@@ -47,16 +47,7 @@ module Moxml
 
         # Generate XML declaration string
         def to_xml
-          output = "<?xml version=\"#{@version}\""
-          if @encoding && !@encoding.empty?
-            output << " encoding=\"#{@encoding}\""
-          end
-          # Include standalone attribute if explicitly set (true or false)
-          unless @standalone_value.nil?
-            output << " standalone=\"#{standalone}\""
-          end
-          output << "?>"
-          output
+          XmlEmitter.declaration_xml(@version, @encoding, standalone)
         end
 
         private

data/lib/moxml/adapter/customized_libxml/node.rb

@@ -18,6 +18,14 @@ module Moxml
           @native = native_node
         end
 
+        # Swap the wrapped native node. Used by the Libxml adapter when
+        # libxml-ruby's content= setter would silently re-escape stored
+        # text; replacing the node with a fresh raw-storage instance is
+        # the only way to preserve verbatim content.
+        def replace_native!(fresh)
+          @native = fresh
+        end
+
         # Compare wrappers based on their native nodes
         def ==(other)
           return false unless other

data/lib/moxml/adapter/customized_libxml/processing_instruction.rb

@@ -5,20 +5,12 @@ module Moxml
     module CustomizedLibxml
       # Wrapper for LibXML processing instruction nodes
       class ProcessingInstruction < Node
-        # Serialize as XML processing instruction
-        # LibXML auto-escapes content, we need to un-escape it
+        # XML 1.0 §2.6: PI content is verbatim — no entity resolution, no escaping.
         def to_xml
           target = @native.name
           content = @native.content
-
-          # Un-escape LibXML's automatic escaping
           if content && !content.empty?
-            unescaped = content.gsub("&quot;", '"')
-              .gsub("&apos;", "'")
-              .gsub("&lt;", "<")
-              .gsub("&gt;", ">")
-              .gsub("&amp;", "&")
-            "<?#{target} #{unescaped}?>"
+            "<?#{target} #{content}?>"
           else
             "<?#{target}?>"
           end

data/lib/moxml/adapter/customized_libxml/text.rb

@@ -13,16 +13,9 @@ module Moxml
           @native.content
         end
 
-        # Serialize as XML with proper escaping
-        # LibXML's .content already contains escaped text, but it over-escapes
-        # quotes which don't need escaping in text nodes (only in attributes)
+        # @native.to_s escapes & < > but leaves quotes alone, which text nodes need.
         def to_xml
-          content = @native.content
-          # Skip the gsub allocation entirely when there's nothing to undo —
-          # the common case for parsed text without literal quotes.
-          return content unless content.include?("&quot;")
-
-          content.gsub("&quot;", '"')
+          @native.to_s
         end
       end
     end