nori 2.7.1 → 2.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e562b494afc72df387f136e51af0c5e0445b528b5515eedad7f60b3d49d207a5
-  data.tar.gz: 5eabad83b50392e39973058c70d6307ddaaeea47934c292fbd472ee0f93680f9
+  metadata.gz: d2ffd841cec28588977a4cf413ade2a33ceba13466046a1299e27016231db0ef
+  data.tar.gz: 51edbdf4ab6413ca7583bb02e05a88ad13fac97741797cf5dde6135edd99d5f8
 SHA512:
-  metadata.gz: 6366a2ac0d9dd99e4fd7b0f32d95dfb21f76885ab18fc7f3f7767c5e257b61e3817d2bf13f9a113feb2482fdb6f5a912956ef5d17a3f868a9139e9b55683402f
-  data.tar.gz: cf295850b06961ed67644a8c06c61152cecf5c8512f8a74a03115e47fe8f9a567300d9af8d91504013777400882869cb7016c80563015e6423e86718c7b828bd
+  metadata.gz: 1700483ec9b91f559ab0fa138c5760fe237f7bf92f5b927bbde60dc437175af5e8c8cc12998195bd875ea9272f62d25727e3fe85a5954918cc0ea31aa5a413a3
+  data.tar.gz: 8713852dfe40adfe96ae00debff569d58f6d8c394fd48fe8df7ad34a767e886605df4163d15cd8573922926ad7abb9f3af1135ddd7180301172dfef5b6b718c6

data/CHANGELOG.md

@@ -1,19 +1,48 @@
-# 2.7.1 (2024-07-28)
+# Nori changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [2.9.0] - 2026-07-05
+
+### Changed
+
+* The `:standards` profile no longer applies schema-less typing. Under `Nori.new(standards: true)`, `:advanced_typecasting` now defaults to `false` (an explicit `advanced_typecasting: true` still wins), and the bare un-namespaced `type=` and `nil=` attributes become ordinary attributes instead of casting instructions. No more `type=` conversions (`integer`, `date`, `decimal`, `array`, `file`, ...), and only a prefixed `xsi:nil="true"` marks an element nil. These conventions come from Rails' `Hash.from_xml` (inherited via crack and merb), not from any XML spec. Without a schema, character data is just text. Knowing that `<id>123</id>` holds an integer is the business of a schema-aware layer, not a guess from string shape. Parsing without the profile is completely unchanged.
+
+### Added
+
+* Add the `:serializable` profile option. `Nori.new(serializable: true)` returns plain, directly-serializable data with no custom value classes. A text node that also has attributes becomes a Hash in the XML JSON convention (`{"#text" => content}` merged with our existing `@`-prefixed attributes instead of a `Nori::StringWithAttributes`, and a text node without attributes becomes a plain `String`. A `type="file"` node is no longer decoded into a `Nori::StringIOFile`, it folds into text and attributes like any other node, leaving the base64 content untouched for the caller to decode. That `type=` decoding is a Rails/ActiveSupport `Hash.from_xml` convention Nori inherited via crack and merb, not part of any XML spec, which is why the plain-data profile drops it. With this, values survive `to_json`, `to_yaml`, and `Marshal`. Opt-in on the 2.x line, probably the default in Nori 3.0. Reported by @ArnoldMEDLINQ ([#107](https://github.com/savonrb/nori/issues/107)), thanks to @dub357 for pinpointing the cause and finding [#106](https://github.com/savonrb/nori/pull/106), and thanks to @ekzobrain whose hash-shape approach this builds on.
+
+## [2.8.0] - 2026-07-04
+
+### Added
+
+* Add the `:standards` profile option. `Nori.new(standards: true)` turns on Nori's spec-correct parsing as a group rather than one flag at a time. Its first members are xml:space honoring (whitespace-only text under `xml:space="preserve"` is kept instead of stripped, with inheritance and `xml:space="default"` reset per [XML 1.0 §2.10](https://www.w3.org/TR/xml/#sec-white-space)) and the XML string-value model for empty elements (implies `:consistent_empty_tags` with an empty-string `:empty_tag_value`, both still overridable). Opt-in on the 2.x line, default in 3.0. Thanks to @md5 for raising the `xml:space="preserve"` case on [#97](https://github.com/savonrb/nori/issues/97).
+
+* [#97](https://github.com/savonrb/nori/issues/97) Add the `:consistent_empty_tags` option ([#109](https://github.com/savonrb/nori/pull/109)). When enabled, every empty tag becomes the `:empty_tag_value`, whether it has attributes or not. A string value keeps the attributes accessible via `#attributes`, and an explicit `xsi:nil="true"` always becomes `nil`. Reported by @lukasbischof, who also proposed a fix in [#98](https://github.com/savonrb/nori/pull/98). Thanks to @md5 for surfacing another instance of it and pinpointing the cause.
+
+### Fixed
+
+* Whitespace-only CDATA content is no longer stripped. A `<![CDATA[   ]]>` section is the author's explicit literal-data marker, so its content is now preserved as text instead of being dropped, on both the Nokogiri and REXML parsers. Previously such a tag parsed to `nil` (or, with attributes, dropped the content and kept only the attributes). Non-whitespace CDATA is unchanged, and ordinary whitespace-only text is still stripped as before. Surfaced while auditing whitespace handling for [#97](https://github.com/savonrb/nori/issues/97), the same empty-tag/whitespace spec-compliance work as the `:consistent_empty_tags` option above.
+
+## 2.7.1 (2024-07-28)
 
 * Stop monkey-patching String with #snakecase by @mchu in https://github.com/savonrb/nori/pull/102
 
-# 2.7.0 (2024-02-13)
+## 2.7.0 (2024-02-13)
 
 * Added support for ruby 3.1, 3.2, 3.3. Dropped support for ruby 2.7 and below.
 * Feature: `Nori::Parser` has a new option, `:scrub_xml`, which defaults to true, for scrubbing invalid characters ([#72](https://github.com/savonrb/nori/pull/72)). This should allow documents containing invalid characters to still be parsed.
 * Fix: REXML parser changes `&lt;` inside CDATA to `<` ([#94](https://github.com/savonrb/nori/pull/94))
 * Change: `Object#blank?` is no longer patched in.
 
-# 2.6.0 (2015-05-06)
+## 2.6.0 (2015-05-06)
 
 * Feature: [#69](https://github.com/savonrb/nori/pull/69) Add option to convert empty tags to a value other than nil.
 
-# 2.5.0 (2015-03-31)
+## 2.5.0 (2015-03-31)
 
 * Formally drop support for ruby 1.8.7. Installing Nori from rubygems for that version should no longer attempt to install versions that will not work.
 * BREAKING CHANGE: Newlines are now preserved when present in the value of inner text nodes. See the example below:
@@ -31,7 +60,7 @@ Nori.new.parse("<outer>\n&lt;embedded&gt;\n&lt;one&gt;&lt;/one&gt;\n&lt;two&gt;&
 => {"outer"=>"<embedded>\n<one></one>\n<two></two>\n<embedded>\n"}
 ```
 
-# 2.4.0 (2014-04-19)
+## 2.4.0 (2014-04-19)
 
 * Change: Dropped support for ruby 1.8, rubinius and ree
 
@@ -39,42 +68,42 @@ Nori.new.parse("<outer>\n&lt;embedded&gt;\n&lt;one&gt;&lt;/one&gt;\n&lt;two&gt;&
 
 * Feature: Added `:convert_dashes_to_underscore` option
 
-# 2.3.0 (2013-07-26)
+## 2.3.0 (2013-07-26)
 
 * Change: `Nori#find` now ignores namespace prefixes in Hash keys it is searching through.
 
 * Fix: Limited Nokogiri to < 1.6, because v1.6 dropped support for Ruby 1.8.
 
-# 2.2.0 (2013-04-25)
+## 2.2.0 (2013-04-25)
 
 * Feature: [#42](https://github.com/savonrb/nori/pull/42) adds the `:delete_namespace_attributes`
   option to remove namespace attributes like `xmlns:*` or `xsi:*`.
 
-# 2.1.0 (2013-04-21)
+## 2.1.0 (2013-04-21)
 
 * Feature: Added `Nori.hash_key` and `Nori#find` to work with Hash keys generated by Nori.
   Original issue: [savonrb/savon#393](https://github.com/savonrb/savon/pull/393)
 
-# 2.0.4 (2013-02-26)
+## 2.0.4 (2013-02-26)
 
 * Fix: [#37](https://github.com/savonrb/nori/issues/37) special characters
   problem on Ruby 1.9.3-p392.
 
-# 2.0.3 (2013-01-10)
+## 2.0.3 (2013-01-10)
 
 * Fix for remote code execution bug. For more in-depth information, read about the
   recent [Rails hotfix](https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/61bkgvnSGTQ).
   Please make sure to upgrade now!
 
-# 2.0.2 (YANKED)
+## 2.0.2 (YANKED)
 
 * Yanked because of a problem with XML that starts with an instruction tag.
 
-# 2.0.1 (YANKED)
+## 2.0.1 (YANKED)
 
 * Yanked because of a problem with XML that starts with an instruction tag.
 
-# 2.0.0 (2012-12-12)
+## 2.0.0 (2012-12-12)
 
 Please make sure to read the updated README for how to use the new version.
 
@@ -235,3 +264,6 @@ Please make sure to read the updated README for how to use the new version.
 
 ## 0.1.0 2009-03-28
 * Initial release.
+
+[2.9.0]: https://github.com/savonrb/nori/compare/v2.8.0...v2.9.0
+[2.8.0]: https://github.com/savonrb/nori/compare/v2.7.1...v2.8.0

data/README.md

@@ -2,8 +2,8 @@ Nori
 ====
 
 [![CI](https://github.com/savonrb/nori/actions/workflows/test.yml/badge.svg)](https://github.com/savonrb/nori/actions/workflows/test.yml)
-[![Gem Version](https://badge.fury.io/rb/nori.svg)](http://badge.fury.io/rb/nori)
-[![Code Climate](https://codeclimate.com/github/savonrb/nori.svg)](https://codeclimate.com/github/savonrb/nori)
+[![Gem Version](https://img.shields.io/gem/v/nori.svg)](https://rubygems.org/gems/nori)
+[![Coverage Status](https://coveralls.io/repos/github/savonrb/nori/badge.svg?branch=main)](https://coveralls.io/github/savonrb/nori?branch=main)
 
 Really simple XML parsing ripped from Crack, which ripped it from Merb.  
 
@@ -50,6 +50,11 @@ result["foo"].attributes
 # => {"bar"=>"baz"}
 ```
 
+Because `StringWithAttributes` hides the attributes on an accessor, they are
+lost on a plain `to_json` and callers have to type-check each value. The
+[`serializable`](#serializable) profile returns a plain Hash instead and will
+probably be the default in 3.0.
+
 ## advanced_typecasting
 
 Nori can automatically convert string values to `TrueClass`, `FalseClass`, `Time`, `Date`, and `DateTime`:
@@ -107,3 +112,132 @@ parser = Nori.new(:convert_dashes_to_underscores => false)
 parser.parse('<any-tag>foo bar</any-tag>')
 # => {"any-tag"=>"foo bar"}
 ```
+
+## empty_tag_value
+
+By default, an empty tag becomes `nil`. The `empty_tag_value` option replaces that value.
+
+```ruby
+Nori.new.parse('<foo/>')
+# => {"foo"=>nil}
+
+Nori.new(:empty_tag_value => "").parse('<foo/>')
+# => {"foo"=>""}
+```
+
+The option does not apply to empty tags with attributes.
+They become a hash of their prefixed attributes instead.
+
+```ruby
+Nori.new(:empty_tag_value => "").parse('<foo bar="baz"/>')
+# => {"foo"=>{"@bar"=>"baz"}}
+```
+
+## consistent_empty_tags
+
+With `consistent_empty_tags`, every empty tag (with or without attributes) becomes the `empty_tag_value`.
+
+```ruby
+Nori.new(:consistent_empty_tags => true).parse('<foo bar="baz"/>')
+# => {"foo"=>nil}
+```
+
+A string `empty_tag_value` keeps the attributes accessible on the value,
+and an explicit `xsi:nil="true"` always becomes `nil`.
+
+```ruby
+parser = Nori.new(:consistent_empty_tags => true, :empty_tag_value => "")
+
+result = parser.parse('<foo bar="baz"/>')
+# => {"foo"=>""}
+
+result["foo"].attributes
+# => {"bar"=>"baz"}
+
+parser.parse('<foo xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>')
+# => {"foo"=>nil}
+```
+
+## standards
+
+`standards` is a profile that turns on Nori's spec-correct parsing as a group,
+rather than one flag at a time. It is opt-in on the 2.x line and becomes the
+default in Nori 3.0.
+
+```ruby
+Nori.new(:standards => true)
+```
+
+Members of the profile:
+
+- **xml:space honoring** ([XML 1.0 §2.10](https://www.w3.org/TR/xml/#sec-white-space)).
+  Whitespace-only text under `xml:space="preserve"` is kept instead of stripped.
+  The nearest ancestor that sets `xml:space` wins, and `xml:space="default"`
+  resets to the stripping behavior.
+
+  ```ruby
+  Nori.new(:standards => true).parse('<name xml:space="preserve">   </name>')
+  # => {"name"=>"   "}
+  ```
+
+- **the XML string-value model for empty elements.** The profile implies
+  `consistent_empty_tags: true` with `empty_tag_value: ""`. Both remain
+  overridable, so passing either option explicitly wins over the profile.
+
+  ```ruby
+  Nori.new(:standards => true).parse('<foo bar="baz"/>')
+  # => {"foo"=>""}
+  ```
+
+- **no schema-less typing.** Without a schema, character data is just text, so
+  the profile implies `advanced_typecasting: false` (overridable) and ignores
+  the bare `type=` and `nil=` attribute conventions that Nori inherited from
+  Rails' `Hash.from_xml`. A `type="integer"` no longer casts, a `type="array"`
+  no longer folds children into an Array, and both stay visible as ordinary
+  attributes. Only the prefixed `xsi:nil="true"` still marks an element nil,
+  because that convention comes from XML Schema Instance itself.
+
+  ```ruby
+  Nori.new(:standards => true).parse('<id type="integer">123</id>')
+  # => {"id"=>"123"}    (with "type" accessible via #attributes)
+  ```
+
+## serializable
+
+`serializable` is a profile that makes Nori return plain, directly-serializable
+data with no custom value classes. It is opt-in on the 2.x line and will
+probably become the default in 3.0.
+
+```ruby
+Nori.new(:serializable => true)
+```
+
+Members of the profile:
+
+- **text nodes with attributes become a Hash.** A tag with both text content
+  and attributes maps to the XML JSON convention (`{"#text" => content}` merged
+  with the `@`-prefixed attributes) instead of a `Nori::StringWithAttributes`.
+  This is the same `@`-keyed shape element nodes already use, so the attributes
+  survive `to_json`, `to_yaml`, and `Marshal`. A text node without attributes
+  stays a plain `String`.
+
+  ```ruby
+  Nori.new(:serializable => true).parse('<foo bar="baz">Content</foo>')
+  # => {"foo"=>{"#text"=>"Content", "@bar"=>"baz"}}
+
+  Nori.new(:serializable => true).parse('<foo>Content</foo>')
+  # => {"foo"=>"Content"}
+  ```
+
+- **`type="file"` nodes are not decoded.** Without the profile, a node carrying
+  `type="file"` is base64-decoded into a `Nori::StringIOFile` with its filename
+  and content type. The profile skips that. The node folds into text and
+  attributes like any other, leaving the base64 content as a plain `String` for
+  the caller to decode. This `type=` decoding is a Rails/ActiveSupport
+  `Hash.from_xml` convention Nori inherited through crack and merb, not part of
+  any XML specification, so the plain-data profile leaves it behind.
+
+  ```ruby
+  Nori.new(:serializable => true).parse('<doc type="file" name="x.pdf">aGVsbG8=</doc>')
+  # => {"doc"=>{"#text"=>"aGVsbG8=", "@type"=>"file", "@name"=>"x.pdf"}}
+  ```

data/lib/nori/parser/nokogiri.rb

@@ -9,23 +9,44 @@ class Nori
     module Nokogiri
 
       class Document < ::Nokogiri::XML::SAX::Document
+        # Marks a text node that came from a CDATA section so the whitespace
+        # stripping in {#end_element} and {#characters} leaves it alone.
+        # A CDATA section is the author's explicit literal-data marker
+        # (XML 1.0 §2.7), so its content is real even when it is only
+        # whitespace and is never trimmed. The marker never reaches the
+        # output because {XMLUtilityNode#inner_html} joins children into
+        # a plain string.
+        class CDataText < ::String; end
+
         attr_accessor :options
 
         def stack
           @stack ||= []
         end
 
+        # Tracks the xml:space value in effect for each open element so
+        # {#preserve_whitespace?} can honor XML 1.0 §2.10 inheritance.
+        def space_stack
+          @space_stack ||= []
+        end
+
         def start_element(name, attrs = [])
-          stack.push Nori::XMLUtilityNode.new(options, name, Hash[*attrs.flatten])
+          attributes = Hash[*attrs.flatten]
+          space_stack.push xml_space(attributes)
+          stack.push Nori::XMLUtilityNode.new(options, name, attributes)
         end
 
         # To keep backward behaviour compatibility
-        # delete last child if it is a space-only text node
+        # delete last child if it is a space-only text node.
+        # CDATA content is literal and is never stripped, and under
+        # xml:space="preserve" whitespace-only text is kept too.
         def end_element(name)
+          preserve = preserve_whitespace?
+          space_stack.pop
           if stack.size > 1
             last = stack.pop
             maybe_string = last.children.last
-            if maybe_string.is_a?(String) and maybe_string.strip.empty?
+            if !preserve and maybe_string.is_a?(String) and !maybe_string.is_a?(CDataText) and maybe_string.strip.empty?
               last.children.pop
             end
             stack.last.add_node last
@@ -34,15 +55,41 @@ class Nori
 
         # If this node is a successive character then add it as is.
         # First child being a space-only text node will not be added
-        # because there is no previous characters.
+        # because there is no previous characters, unless xml:space="preserve"
+        # is in effect.
         def characters(string)
           last = stack.last
-          if last and last.children.last.is_a?(String) or string.strip.size > 0
+          if last and (preserve_whitespace? or last.children.last.is_a?(String) or string.strip.size > 0)
             last.add_node(string)
           end
         end
 
-        alias cdata_block characters
+        # Adds the CDATA section content verbatim.
+        def cdata_block(string)
+          last = stack.last
+          last.add_node(CDataText.new(string)) if last
+        end
+
+        private
+
+        # The xml:space value for an element. Its own declaration if it has one,
+        # otherwise the value inherited from the nearest ancestor.
+        # Defaults to "default". See XML 1.0 §2.10.
+        #
+        # @param attributes [Hash{String => String}] the element's attributes
+        # @return [String] "preserve", "default", or an inherited value
+        def xml_space(attributes)
+          attributes["xml:space"] || space_stack.last || "default"
+        end
+
+        # Whether whitespace-only text must be kept for the current element.
+        # Only the +:standards+ profile honors xml:space. Otherwise
+        # insignificant whitespace is stripped as before.
+        #
+        # @return [Boolean]
+        def preserve_whitespace?
+          options[:standards] && space_stack.last == "preserve"
+        end
 
       end
 

data/lib/nori/parser/rexml.rb

@@ -12,6 +12,9 @@ class Nori
 
       def self.parse(xml, options)
         stack = []
+        # Tracks the xml:space value in effect for each open element so
+        # whitespace-only text can be kept under xml:space="preserve".
+        space_stack = []
         parser = ::REXML::Parsers::BaseParser.new(xml)
 
         while true
@@ -23,21 +26,47 @@ class Nori
           when :end_doctype, :start_doctype
             # do nothing
           when :start_element
+            space_stack.push xml_space(event[2], space_stack)
             stack.push Nori::XMLUtilityNode.new(options, event[1], event[2])
           when :end_element
+            space_stack.pop
             if stack.size > 1
               temp = stack.pop
               stack.last.add_node(temp)
             end
           when :text
-            stack.last.add_node(event[1]) unless event[1].strip.length == 0 || stack.empty?
+            # Whitespace-only text is insignificant and stripped, unless
+            # xml:space="preserve" is in effect under the :standards profile.
+            keep = preserve_whitespace?(options, space_stack) || event[1].strip.length > 0
+            stack.last.add_node(event[1]) if keep && !stack.empty?
           when :cdata
-            stack.last.add_node(raw_data[1]) unless raw_data[1].strip.length == 0 || stack.empty?
+            # CDATA is the author's explicit literal-data marker (XML 1.0 §2.7),
+            # so whitespace-only CDATA is kept rather than stripped like text.
+            stack.last.add_node(raw_data[1]) unless stack.empty?
           end
         end
         stack.length > 0 ? stack.pop.to_hash : {}
       end
 
+      # The xml:space value for an element: its own declaration if it has
+      # one, otherwise the value inherited from the nearest ancestor.
+      # Defaults to "default". See XML 1.0 §2.10.
+      #
+      # @param attributes [Hash{String => String}] the element's attributes
+      # @param space_stack [Array<String>] the ancestor xml:space values
+      # @return [String] "preserve", "default", or an inherited value
+      def self.xml_space(attributes, space_stack)
+        (attributes && attributes["xml:space"]) || space_stack.last || "default"
+      end
+
+      # Whether whitespace-only text must be kept for the current element.
+      # Only the +:standards+ profile honors xml:space.
+      #
+      # @return [Boolean]
+      def self.preserve_whitespace?(options, space_stack)
+        options[:standards] && space_stack.last == "preserve"
+      end
+
       def self.unnormalize(event)
         event.map do |el|
           if el.is_a?(String)

data/lib/nori/string_io_file.rb

@@ -1,3 +1,5 @@
+require "stringio"
+
 class Nori
   class StringIOFile < StringIO
 

data/lib/nori/version.rb

@@ -1,3 +1,3 @@
 class Nori
-  VERSION = '2.7.1'
+  VERSION = '2.9.0'
 end