moxml 0.1.10 → 0.1.11

data/README.adoc

@@ -592,8 +592,82 @@ book.add_child(doc.create_comment('Book details'))
 title = doc.create_element('title')
 title.text = 'Ruby Programming'
 book.add_child(title)
+
+# Add entity reference (for declared entities)
+book.add_child(doc.create_entity_reference('mdash'))
 ----
 
+=== Entity References
+
+Moxml supports `EntityReference` nodes for preserving entity syntax in XML documents. This enables round-trip preservation of entity references like ` `, `©`, and custom entities defined in the DOCTYPE.
+
+[source,ruby]
+----
+# Create entity reference programmatically
+ref = doc.create_entity_reference('nbsp')
+element.add_child(ref)
+
+# Or using the builder pattern
+doc = Moxml::Builder.new(Moxml.new).build do
+  element 'text' do
+    entity_reference 'ndash'
+    entity_reference 'copy'
+  end
+end
+----
+
+**Parsing and Round-Trip:**
+
+When parsing XML with declared entities, Moxml preserves entity references:
+
+[source,ruby]
+----
+# Parse document with custom entity
+xml = <<-XML
+<!DOCTYPE root [<!ENTITY nbsp " "> ]>
+<root>hello&nbsp;world</root>
+XML
+
+doc = Moxml.new(:nokogiri).parse(xml)
+doc.to_xml  # => preserves &nbsp; entity reference
+----
+
+**Adapter Notes:**
+
+- *Nokogiri*: Preserves custom declared entities as `EntityReference` nodes
+- *Ox, Oga*: These adapters resolve entities during parsing and do not expose entity reference nodes. Use Nokogiri or LibXML for entity preservation.
+
+**Entity Loading Configuration:**
+
+Moxml provides configurable entity loading with four modes to balance between functionality, performance, and security:
+
+[source,ruby]
+----
+# Default: Load all W3C entities (HTML + MathML + ISO entity sets)
+# Raises error if entity data is unavailable
+context = Moxml.new
+
+# Optional: Load entities if available, silently skip if not
+context = Moxml.new do |config|
+  config.entity_load_mode = :optional
+end
+
+# Disabled: No entity loading (fastest, for controlled XML sources)
+context = Moxml.new do |config|
+  config.entity_load_mode = :disabled
+end
+
+# Custom: Load entities from your own source
+context = Moxml.new do |config|
+  config.entity_load_mode = :custom
+  config.entity_provider = -> { MyEntitySource.all_entities }
+end
+----
+
+The entity data comes from the W3C XML Core WG Character Entities specification (HTMLMathML set), bundled locally in `data/w3c_entities.json` for offline capability. Set the `MOXML_ENTITY_DEFINITIONS_PATH` environment variable to use a custom entity data source.
+
+For backward compatibility, `config.load_external_entities = false` maps to `:disabled` mode, and `config.load_external_entities = true` maps to `:required` mode.
+
 === Fluent interface API
 
 Moxml provides a fluent, chainable API for improved developer experience:
@@ -741,6 +815,28 @@ context = Moxml.new do |config|
 end
 ----
 
+=== Namespace URI validation
+
+Moxml validates namespace URIs against
+https://www.rfc-editor.org/rfc/rfc3986[RFC 3986] by default, as required by the
+https://www.w3.org/TR/xml-names/[W3C Namespaces in XML] specification.
+
+For documents that use non-standard namespace identifiers, a lenient mode is
+available:
+
+[source,ruby]
+----
+# Strict mode (default) — rejects invalid URIs per RFC 3986
+context = Moxml.new do |config|
+  config.namespace_uri_mode = :strict
+end
+
+# Lenient mode — accepts any string as a namespace URI
+context = Moxml.new do |config|
+  config.namespace_uri_mode = :lenient
+end
+----
+
 For all configuration options, adapter selection, serialization options, and
 environment-based configuration, see
 link:docs/_pages/configuration.adoc[Configuration Guide].
@@ -846,6 +942,146 @@ limitations. Use these adapters when you need full XPath and namespace support.
 
 
 
+== Round-trip XML Testing
+
+Moxml includes comprehensive round-trip testing to verify that XML documents remain
+semantically equivalent when parsed and serialized across different adapters.
+
+=== Purpose
+
+Round-trip testing ensures:
+
+* **Cross-adapter compatibility** - XML parsed with one adapter (e.g., Nokogiri) can be
+  serialized and re-parsed with another adapter (e.g., Oga) while preserving content
+* **Structural fidelity** - Element names, attributes, and document structure are maintained
+* **Content preservation** - Text content and entity references survive multiple parse/serialize cycles
+* **Double round-trip verification** - Source → Target → Source sequences produce semantically
+  equivalent output
+
+=== Test Fixtures
+
+Round-trip tests use real-world XML documents organized into collections:
+
+**rfcxml** - IETF RFC documents in XML format. These provide complex, standards-compliant
+XML with mixed content, namespaces, and attributes. The collection includes:
+
+* Large documents (500KB-2.4MB) for stress testing
+* Rich metadata and cross-references
+* Various XML schema patterns
+
+**metanorma** - Metanorma document processing XML. These test:
+
+* Document structure preservation
+* Nested elements and complex hierarchies
+* Standard XML vocabularies
+
+**niso-jats** - NISO Journal Article Tag Suite XML. These provide:
+
+* Scholarly publishing XML schemas
+* Rich bibliographic metadata
+* Mixed content models
+
+=== Running Round-trip Tests
+
+[source,bash]
+----
+# Run all round-trip tests
+bundle exec rake spec:consistency
+
+# Exclude REXML for larger fixtures (faster, REXML is pure Ruby)
+MOXML_ROUNDTRIP_REXML_MAX_SIZE=0 bundle exec rake spec:consistency
+
+# Adjust the per-example timeout (default: 120 seconds)
+MOXML_ROUNDTRIP_TIMEOUT=300 bundle exec rake spec:consistency
+----
+
+REXML is a pure Ruby XML parser and becomes very slow on large documents (500KB+).
+By default, REXML adapter pairs are skipped for fixtures exceeding 500KB. All other
+adapters (Nokogiri, Oga, Ox) are tested against every fixture.
+
+=== Test Mechanics
+
+For each fixture, tests run across all adapter pairs (4 adapters = 12 combinations):
+
+1. Parse with source adapter
+2. Serialize to XML string
+3. Parse serialized output with target adapter
+4. Compare semantic equivalence (element names, attributes, text content)
+
+A "double round-trip" test additionally verifies: Source → Target → Source → Target
+produces consistent results.
+
+NOTE: REXML is excluded from adapter pairs for fixtures larger than 500KB (configurable
+via `MOXML_ROUNDTRIP_REXML_MAX_SIZE`). This is because REXML is pure Ruby and cannot
+parse large XML documents in a practical timeframe. A per-example timeout
+(`MOXML_ROUNDTRIP_TIMEOUT`, default 120s) prevents tests from hanging indefinitely.
+
+=== Ox Adapter Element Ordering Caveat
+
+The Ox adapter produces elements in a different order than other adapters for certain
+fixtures with complex nested structures (e.g., `element_citation.xml`,
+`collection1nested.xml`, `pnas_sample.xml`). This causes the `elements_with_attributes`
+comparison to fail with "Array length mismatch" even though the semantic equivalence
+check (double round-trip) passes.
+
+Round-trip tests automatically skip the `elements_with_attributes` comparison for these
+known Ox ordering issues. The `ruby-versions` CI job tests only Nokogiri and Oga adapters;
+the `nokogiri-ox` and `nokogiri-rexml` CI jobs test Ox and REXML respectively but are
+marked as **experimental** since these adapters lack full XML feature support:
+
+* **Ox**: Lacks proper namespace support, XPath with predicates, and uses a custom
+  `locate()` method instead of standard XPath
+* **REXML**: Pure Ruby, exponential time complexity with document size, impractical for
+  documents over ~500KB
+
+For production use, prefer Nokogiri or Oga which provide complete XML conformance.
+
+To run tests with a specific adapter set locally:
+
+[source,bash]
+----
+# Nokogiri + Oga only (fast, full test suite)
+MOXML_ROUNDTRIP_ADAPTERS=nokogiri,oga bundle exec rspec spec/consistency/ --tag round_trip
+
+# Nokogiri × Ox only (experimental)
+MOXML_ROUNDTRIP_ADAPTERS=nokogiri,ox MOXML_ROUNDTRIP_TIMEOUT=300 bundle exec rspec spec/consistency/ --tag round_trip
+
+# Nokogiri × REXML only (experimental, small fixtures due to exponential complexity)
+MOXML_ROUNDTRIP_ADAPTERS=nokogiri,rexml MOXML_ROUNDTRIP_TIMEOUT=300 MOXML_ROUNDTRIP_REXML_MAX_SIZE=50000 bundle exec rspec spec/consistency/ --tag round_trip
+----
+
+=== Why Semantic Equivalence?
+
+While a pure round-trip test with raw XML comparison would be ideal, different XML adapters
+have fundamentally different philosophies for handling:
+
+* **Element ordering** - Some preserve document order, others sort alphabetically
+* **Whitespace handling** - Some normalize spaces, others preserve exactly
+* **Attribute representation** - Different data structures for the same attributes
+* **Text extraction** - Varying approaches to concatenating text content
+
+Instead of raw comparison, Moxml implements semantic equivalence testing that focuses on
+meaningful XML structure and content:
+
+[source,ruby]
+----
+# Element name must match
+expect(target_element.name).to eq(source_element.name)
+
+# Attributes must be semantically equivalent
+expect(target_attributes).to eq(source_attributes)
+
+# Text content must be preserved (whitespace-normalized)
+expect(normalized_text(target)).to eq(normalized_text(source))
+
+# Document structure (element count) must match
+expect(doc.xpath("//*").size).to eq(original.xpath("//*").size)
+----
+
+This approach tolerates adapter-specific serialization differences while ensuring
+the actual XML content remains intact.
+
+
 == Development and testing
 
 For complete information on development setup, testing strategies, benchmarking,

data/Rakefile

@@ -30,6 +30,17 @@ namespace :spec do
     t.pattern = "spec/consistency/**/*_spec.rb"
   end
 
+  namespace :consistency do
+    desc "Run round-trip tests for a specific fixture category (CATEGORIES=metanorma,rfcxml,niso-jats)"
+    task :by_category do
+      categories = ENV.fetch("CATEGORIES", "").split(",").map(&:strip)
+      abort "Usage: CATEGORIES=metanorma,rfcxml rake spec:consistency:by_category" if categories.empty?
+
+      include_filters = categories.map { |c| "--tag fixture_category:#{c}" }.join(" ")
+      sh "bundle exec rspec spec/consistency/ --tag round_trip #{include_filters}"
+    end
+  end
+
   desc "Run example tests"
   RSpec::Core::RakeTask.new(:examples) do |t|
     t.pattern = "spec/examples/**/*_spec.rb"