lutaml-model 0.7.1 → 0.7.2

data/RELEASE_NOTES.adoc

@@ -0,0 +1,346 @@
+= Release Notes: v0.6.7 to v0.7.1
+
+== Overview
+
+This release introduces significant enhancements to the lutaml-model library, focusing on improved handling of missing values, comprehensive polymorphic model support, and important bug fixes. This document outlines the key changes and provides guidance for upgrading your codebase.
+
+== Key changes
+
+=== Missing values handling family
+
+A comprehensive set of features for handling the "missing values" family (empty,
+non-existent, and undefined values) across different serialization formats has
+been introduced.
+
+WARNING: The default behavior for handling uninitialized collection attributes
+is changed from before.
+
+In version 0.6.7, an uninitialized collection attribute in Lutaml::Model is
+represented as an empty array (`[]`).
+
+In version 0.7.1, an uninitialized collection attribute in Lutaml::Model is
+represented as `nil`.
+
+This change is made to ensure consistency across different serialization formats
+and to provide a predictable and consistent behavior based on explicit
+declarations when dealing with missing values.
+
+Previously, Lutaml::Model had a limited ability to handle the missing value types,
+for example, the appearance and round-trip behavior of the blank XML element
+is uncertain. Lutaml::Model also did not support XML `nil` and YAML `null` values
+in a consistent manner, nor the inconsistency where TOML does not support
+the notion of `nil` or `null` values.
+
+This change may break existing code that relies on the previous behavior. If you
+were using uninitialized collection attributes in your code, you may need to
+update your code to handle the new behavior.
+
+There is now a series of features to support the missing values family, including:
+
+* Attribute-level `initialize_empty` option, which controls the default
+initialization behavior of collection attributes. Previously, collection
+attributes defaulted to an empty array (`[]`) when uninitialized. The default
+is now the value `nil`.
++
+Revert to legacy behavior: You can return to the previous behavior by setting
+`initialize_empty: true` in the attribute definition.
+
+* Mapping-rule-level `render_nil` and `render_empty` options have been revamped
+into accepting additional options that are used to control how nil and empty
+values are parsed from and rendered in serialization formats.
++
+The behavior of `render_nil: true` and `render_empty: true` is now equivalent to
+`render_nil: :as_empty` and `render_empty: :as_empty`, respectively.
+It is strongly recommended that the legacy behavior be replaced with the new
+options.
++
+The reason being that legacy behavior breaks round-trips between conversions of
+the same format. In the case of `render_nil: true`, a model value of `nil` is
+serialized into an XML blank element, but when the XML blank element is
+deserialized, it is deserialized into an empty string (`""`, as per the W3C XML
+standard), not `nil`.
+
+* Mapping-rule-level `value_map` option, which provides fine-grained control
+over how missing values are handled during serialization and deserialization.
+
+* Support for a new `Uninitialized` value at the `Lutaml::Model` level, which is
+used to represent uninitialized attributes.
++
+This means that if you have a model with an attribute that is not set, it will
+be represented as `Uninitialized` instead of `nil`. This is useful for
+distinguishing between an attribute that is not set and an attribute that is
+set to `nil`, and as a result will preserve the round-trip behavior of
+serialization formats.
+
+* Support for `nil` values in JSON and YAML, and XML serialization formats.
++
+The `nil` value is now supported in JSON, YAML and XML serialization formats.
+This means that if you have a model with an attribute that is set to `nil`, it
+will be serialized as `null` in JSON/YAML and as an empty element in XML. This
+is useful for representing missing values in a consistent manner across
+different serialization formats.
+
+
+
+==== `initialize_empty` option
+
+The `initialize_empty` option controls the default initialization behavior of
+collection attributes:
+
+[source,ruby]
+----
+# Default to `nil`
+class SomeModel < Lutaml::Model::Serializable
+  attribute :coll, :string, collection: true
+
+  xml do
+    root "some-model"
+    map_element 'collection', to: :coll
+  end
+end
+puts SomeModel.new.coll  # => nil
+
+# Default to empty array
+class SomeModel < Lutaml::Model::Serializable
+  attribute :coll, :string, collection: true, initialize_empty: true
+
+  xml do
+    map_element 'collection', to: :coll
+  end
+end
+puts SomeModel.new.coll  # => []
+----
+
+==== `value_map` option
+
+The `value_map` option provides fine-grained control over how missing values are handled during serialization and deserialization. This is especially useful when different serialization formats represent missing values differently.
+
+[source,ruby]
+----
+class ExampleClass < Lutaml::Model::Serializable
+  attribute :status, :string
+
+  xml do
+    map_element 'status', to: :status, value_map: {
+      from: { empty: :nil, omitted: :omitted, nil: :nil },
+      to: { empty: :nil, omitted: :omitted, nil: :nil }
+    }
+  end
+
+  json do
+    map 'status', to: :status, value_map: {
+      from: { empty: :nil, omitted: :omitted, nil: :nil },
+      to: { empty: :nil, omitted: :omitted, nil: :nil }
+    }
+  end
+
+  toml do
+    map 'status', to: :status, value_map: {
+      from: { empty: :nil, omitted: :omitted },
+      to: { empty: :nil, omitted: :omitted, nil: :omitted }
+    }
+  end
+end
+----
+
+For collection attributes, the value_map behaves differently depending on the `initialize_empty` setting.
+
+==== `render_nil` and `render_empty` modes
+
+These options provide shorthand methods to control how nil and empty values are rendered in serialization formats:
+
+* `render_nil: true | :as_empty | :as_blank | :nil | :omit` - Controls how nil values are rendered
+* `render_empty: :as_empty | :as_blank | :nil | :omit` - Controls how empty collections are rendered
+
+[source,ruby]
+----
+class SomeModel < Lutaml::Model::Serializable
+  attribute :coll, :string, collection: true
+
+  xml do
+    root "some-model"
+    map_element 'collection', to: :coll, render_nil: :omit
+  end
+
+  json do
+    map 'collection', to: :coll, render_empty: :as_nil
+  end
+end
+----
+
+=== Polymorphic model support
+
+From version 0.7.1, Lutaml::Model now supports polymorphic models for attribute
+types.
+
+Comprehensive support for polymorphic models has been introduced, allowing for
+flexible modeling of inheritance relationships and proper
+serialization/deserialization.
+
+This means that you can define attributes that can accept multiple types of
+objects, and the library will handle serialization and deserialization for these
+types seamlessly.
+
+Specifically, the following features have been added:
+
+* Polymorphic attribute definition
+* Polymorphic class differentiation in model and serializations
+
+==== Polymorphic attribute definition
+
+Polymorphic attributes can be defined using the `polymorphic` option.
+
+It is possible to define polymorphic attribute classes in the superclass
+and subclasses.
+
+The `polymorphic` option can be set to a collection of classes, and the
+`polymorphic_class` option can be set to `true` to indicate acceptance of any
+subclass of the polymorphic class. Alternatively, you can specify a collection
+of classes to restrict the accepted types.
+
+[source,ruby]
+----
+class ReferenceSet < Lutaml::Model::Serializable
+  attribute :references, Reference, collection: true, polymorphic: [
+    DocumentReference,
+    AnchorReference,
+  ]
+end
+----
+
+When you are not requiring a specific set of subclasses, you can use the
+`polymorphic: true` option to indicate that any subclass of the specified class is
+acceptable.
+
+[source,ruby]
+----
+class ReferenceSet < Lutaml::Model::Serializable
+  attribute :references, Reference, collection: true, polymorphic: true
+end
+----
+
+
+==== Polymorphic class differentiator
+
+When serializing polymorphic models, a differentiator attribute is required to
+also be serialized to identify the specific subclass of the polymorphic class.
+
+This differentiator attribute is typically a string that indicates the type of
+the object being serialized. The differentiator attribute can be defined in the
+superclass or subclasses of the polymorphic class.
+
+Typically, the differentiator attribute is an XML element or attribute (e.g. `type="document-ref"`), or in JSON a `@`-prefixed key (e.g. `@type`).
+
+A polymorphic differentiator attribute can be set in either the superclass or subclasses:
+
+[source,ruby]
+----
+# In superclass
+class Reference < Lutaml::Model::Serializable
+  attribute :_class, :string, polymorphic_class: true
+  # ...
+end
+
+# Or in subclasses
+class DocumentReference < Reference
+  attribute :_class, :string, polymorphic_class: true
+  # ...
+end
+
+# Or in subclasses
+class AnchorReference < Reference
+  attribute :_class, :string, polymorphic_class: true
+  # ...
+end
+----
+
+Given the differentiator attribute being `_class`, we still need to define the
+mapping for the differentiator attribute in the superclass or subclasses.
+
+Polymorphic mapping in serialization is supported through the `polymorphic_map` option:
+
+[source,ruby]
+----
+class Reference < Lutaml::Model::Serializable
+  attribute :_class, :string, polymorphic_class: true
+
+  xml do
+    map_attribute "reference-type", to: :_class, polymorphic_map: {
+      "document-ref" => "DocumentReference",
+      "anchor-ref" => "AnchorReference"
+    }
+  end
+
+  key_value do
+    map "_class", to: :_class, polymorphic_map: {
+      "Document" => "DocumentReference",
+      "Anchor" => "AnchorReference"
+    }
+  end
+end
+----
+
+The `polymorphic_map` option is used to indicate that when serializing a
+`DocumentReference` object, the `_class` attribute will be serialized as
+`document-ref`, and when serializing an `AnchorReference` object, the `_class`
+attribute will be serialized as `anchor-ref`.
+
+This is a mapping-level option so that it can be used in serialization formats
+independently.
+
+This will produce a differentiator attribute in the serialized output as such.
+
+[source,yaml]
+----
+---
+references:
+- _class: Document
+  # other attributes...
+- _class: Anchor
+  # other attributes...
+----
+
+
+=== Importable model improvements
+
+Importable model functionality has been improved, with better support for reusable models:
+
+* `import_model` - Imports both attributes and mappings
+* `import_model_attributes` - Imports only attributes
+* `import_model_mappings` - Imports only mappings
+
+Bug fixes for the import_model functionality ensure more reliable model reuse.
+
+=== Circular reference handling
+
+Improved handling of circular references in the `ComparableModel` module prevents stack overflow errors when comparing models with self-referential structures.
+
+== Upgrade guide
+
+=== Missing values handling
+
+If you were previously using `render_nil: true`, you can continue using it, but
+you may want to explore the more flexible `value_map` option for fine-grained
+control over different serialization formats.
+
+For collection attributes, consider whether you want collections to initialize
+as `nil` or as an empty array by setting the `initialize_empty` option
+accordingly.
+
+=== Polymorphic models
+
+If you were previously using class detection for polymorphic models without
+explicit differentiators, you should now define a polymorphic differentiator
+attribute and use the `polymorphic_class: true` option.
+
+== Contributors
+
+Thank you to all contributors who made this release possible, especially:
+
+* HassanAkbar
+* Ronald Tse
+* suleman-uzair
+
+== Compatibility
+
+This release maintains compatibility with Ruby 2.7 and above.

data/docs/custom_adapters.adoc

@@ -0,0 +1,144 @@
+= Custom Adapters in Lutaml::Model
+
+Lutaml::Model provides a flexible system for creating custom adapters to handle different data formats. This guide explains how to create and use custom adapters in your application.
+
+== Overview
+
+Custom adapters allow you to extend Lutaml::Model to support additional data formats beyond the built-in ones (TOML, YAML, JSON, XML). Each adapter consists of three main components:
+
+. An adapter class that handles parsing and serialization
+. A mapping class that defines how data maps to your model
+. A transform class that handles data transformation
+
+== Creating a Custom Adapter
+
+=== 1. Adapter Class
+
+The adapter class is responsible for parsing input data and converting it back to the target format. It must implement:
+
+* `self.parse(data, options = {})` - Class method to parse input data
+* `to_<format_name>` - Instance method to convert data back to the target format
+
+.Example of a custom adapter for parsing string pairs
+[source,ruby]
+----
+class PairsAdapter < Lutaml::Model::KeyValueDocument
+  attr_reader :parsed_data
+
+  def initialize(parsed_data)
+    @parsed_data = parsed_data
+  end
+
+  def self.parse(data, options = {})
+    # Example input: "name:John|age:30"
+    parsed = data.split("|").map { |pair| pair.split(":") }.to_h
+    new(parsed)
+  end
+
+  def to_pairs_format
+    # Example output: "name:John|age:30"
+    parsed_data.map { |k, v| "#{k}:#{v}" }.join("|")
+  end
+end
+----
+
+=== 2. Mapping Class
+
+The mapping class defines how data fields map to your model attributes. It should inherit from `Lutaml::Model::Mapping`.
+
+[source,ruby]
+----
+class PairsMappingRule < Lutaml::Model::MappingRule
+end
+
+class PairsMapping < Lutaml::Model::Mapping
+  def map_field(name, to:)
+    @mappings << PairsMappingRule.new(name, to: to)
+  end
+
+  def mappings
+    @mappings
+  end
+end
+----
+
+=== 3. Transform Class
+
+The transform class handles the conversion between your data format and the model. It should inherit from `Lutaml::Model::Transform`.
+
+[source,ruby]
+----
+class PairsTransform < Lutaml::Model::Transform
+  def self.data_to_model(context, data, format, options = {})
+    model = context.model.new
+    model.name = data["name"]
+    model.age = data["age"]
+    model
+  end
+
+  def self.model_to_data(context, model, format, options = {})
+    { "name" => model.name, "age" => model.age }
+  end
+end
+----
+
+== Registering Your Adapter
+
+Register your custom adapter with the FormatRegistry:
+
+[source,ruby]
+----
+Lutaml::Model::FormatRegistry.register(
+  :pairs,
+  mapping_class: PairsMapping,
+  adapter_class: PairsAdapter,
+  transformer: PairsTransform
+)
+----
+
+== Using Custom Adapters
+
+Once registered, you can use your custom adapter with your model classes:
+
+[source,ruby]
+----
+class Person < Lutaml::Model::Serializable
+  attribute :name, :string
+  attribute :age, :string
+
+  pairs do
+    map_field "name", to: :name
+    map_field "age", to: :age
+  end
+end
+----
+
+=== Step-by-step Example
+
+[source,ruby]
+----
+input = "name:John|age:30"
+
+person = Person.from_pairs(input)
+# => #<Person @name="John", @age="30">
+
+output = person.to_pairs_format
+# => "name:John|age:30"
+----
+
+== Best Practices
+
+. *Separation of Concerns*: Keep parsing, mapping, and transformation logic in their respective classes.
+. *Error Handling*: Add meaningful error handling in your adapter's `.parse` method.
+. *Documentation*: Clearly document the format and adapter usage.
+. *Testing*: Write tests for your adapter's logic and behavior.
+. *Leverage Format Features*: Support format-specific options while staying consistent with Lutaml::Model’s interfaces.
+
+== Example Implementations
+
+For complete examples of custom adapter implementations, see:
+
+* `spec/lutaml/model/custom_bibtex_adapter_spec.rb`
+* `spec/lutaml/model/custom_vobject_adapter_spec.rb`
+
+These demonstrate how to build complete, custom adapters that integrate cleanly with `Lutaml::Model`.

data/lib/lutaml/model/attribute.rb

@@ -186,7 +186,7 @@ module Lutaml
 
       def validate_polymorphic(value)
         return value.all? { |v| validate_polymorphic!(v) } if value.is_a?(Array)
-        return true unless polymorphic_enabled?
+        return true unless options[:polymorphic]
 
         valid_polymorphic_type?(value)
       end
@@ -194,7 +194,7 @@ module Lutaml
       def validate_polymorphic!(value)
         return true if validate_polymorphic(value)
 
-        raise Lutaml::Model::PolymorphicError.new(value, options)
+        raise Lutaml::Model::PolymorphicError.new(value, options, type)
       end
 
       def validate_collection_range
@@ -290,6 +290,10 @@ module Lutaml
         end
       end
 
+      def serializable?
+        type <= Serialize
+      end
+
       def deep_dup
         self.class.new(name, type, Utils.deep_dup(options))
       end
@@ -304,16 +308,16 @@ module Lutaml
         Object.const_get(klass_name)
       end
 
-      def polymorphic_map_defined?(options, value)
+      def polymorphic_map_defined?(polymorphic_options, value)
         !value.nil? &&
-          options[:polymorphic] &&
-          !options[:polymorphic].empty? &&
-          value[options[:polymorphic][:attribute]]
+          polymorphic_options[:polymorphic] &&
+          !polymorphic_options[:polymorphic].empty? &&
+          value[polymorphic_options[:polymorphic][:attribute]]
       end
 
       def castable?(value, format)
         value.is_a?(Hash) ||
-          (format == :xml && value.is_a?(Lutaml::Model::XmlAdapter::XmlElement))
+          (format == :xml && value.is_a?(Lutaml::Model::Xml::XmlElement))
       end
 
       def castable_serialized_type?(value)
@@ -397,12 +401,14 @@ module Lutaml
               "Invalid type: #{type}, must be a Symbol, String or a Class"
       end
 
-      def polymorphic_enabled?
-        options[:polymorphic]&.any?
+      def valid_polymorphic_type?(value)
+        return value.is_a?(type) unless has_polymorphic_list?
+
+        options[:polymorphic].include?(value.class) && value.is_a?(type)
       end
 
-      def valid_polymorphic_type?(value)
-        value.is_a?(type) && options[:polymorphic].include?(value.class)
+      def has_polymorphic_list?
+        options[:polymorphic]&.is_a?(Array)
       end
     end
   end

data/lib/lutaml/model/config.rb

@@ -3,12 +3,7 @@ module Lutaml
     module Config
       extend self
 
-      # Default values are set for these so the readers are defined below
-      attr_writer :json_adapter, :yaml_adapter
-
-      attr_accessor :xml_adapter, :toml_adapter
-
-      AVAILABLE_FORMATS = %i[xml json yaml toml].freeze
+      AVAILABLE_FORMATS = %i[xml json yaml toml hash].freeze
       KEY_VALUE_FORMATS = AVAILABLE_FORMATS - %i[xml]
 
       def configure
@@ -42,55 +37,66 @@ module Lutaml
       #     one of [:tomlib, :toml_rb]
       #   @example
       #     Lutaml::Model::Config.toml_adapter = :tomlib
+      #
+      # AVAILABLE_FORMATS.each do |adapter_name|
+      #   define_method(:"#{adapter_name}_adapter_type=") do |type_name|
+      #     Lutaml::Model::FormatRegistry.send(:"#{adapter_name}_adapter_type=", type_name)
+      #   end
+      # end
+
       AVAILABLE_FORMATS.each do |adapter_name|
         define_method(:"#{adapter_name}_adapter_type=") do |type_name|
-          adapter = "#{adapter_name}_adapter"
-          type = "#{type_name}_adapter"
+          adapter = adapter_name.to_s
+          type = normalize_type_name(type_name, adapter_name)
+          load_adapter_file(adapter, type)
+          load_moxml_adapter(type_name, adapter_name)
+          set_adapter_for(adapter_name, class_for(adapter, type))
+        end
+      end
 
-          begin
-            adapter_file = File.join(adapter, type)
-            require_relative adapter_file
-          rescue LoadError
-            raise(
-              Lutaml::Model::UnknownAdapterTypeError.new(
-                adapter_name,
-                type_name,
-              ),
-              cause: nil,
-            )
-          end
-          Moxml::Adapter.load(type_name) unless KEY_VALUE_FORMATS.include?(adapter_name)
+      def adapter_for(format)
+        public_send(:"#{format}_adapter")
+      end
 
-          instance_variable_set(
-            :"@#{adapter}",
-            Lutaml::Model.const_get(to_class_name(adapter))
-                         .const_get(to_class_name(type)),
-          )
-        end
+      def set_adapter_for(format, adapter)
+        public_send(:"#{format}_adapter=", adapter)
       end
 
-      # Return JSON adapter. By default StandardJsonAdapter is used
-      #
-      # @example
-      #   Lutaml::Model::Config.json_adapter
-      #   # => Lutaml::Model::YamlAdapter::StandardJsonAdapter
-      def json_adapter
-        @json_adapter || Lutaml::Model::JsonAdapter::StandardJsonAdapter
+      def mappings_class_for(format)
+        Lutaml::Model::FormatRegistry.mappings_class_for(format)
       end
 
-      # Return YAML adapter. By default StandardYamlAdapter is used
-      #
-      # @example
-      #   Lutaml::Model::Config.yaml_adapter
-      #   # => Lutaml::Model::YamlAdapter::StandardYamlAdapter
-      def yaml_adapter
-        @yaml_adapter || Lutaml::Model::YamlAdapter::StandardYamlAdapter
+      def transformer_for(format)
+        Lutaml::Model::FormatRegistry.transformer_for(format)
+      end
+
+      def class_for(adapter, type)
+        Lutaml::Model.const_get(to_class_name(adapter))
+          .const_get(to_class_name(type))
       end
 
-      # @api private
       def to_class_name(str)
         str.to_s.split("_").map(&:capitalize).join
       end
+
+      private
+
+      def normalize_type_name(type_name, adapter_name)
+        "#{type_name.to_s.gsub("_#{adapter_name}", '')}_adapter"
+      end
+
+      def load_adapter_file(adapter, type)
+        adapter_file = File.join(adapter, type)
+        require_relative adapter_file
+      rescue LoadError
+        raise UnknownAdapterTypeError.new(adapter, type), cause: nil
+      end
+
+      def load_moxml_adapter(type_name, adapter_name)
+        return if KEY_VALUE_FORMATS.include?(adapter_name)
+
+        Moxml::Adapter.load(type_name)
+      end
     end
   end
 end

data/lib/lutaml/model/error/polymorphic_error.rb

@@ -1,8 +1,13 @@
 module Lutaml
   module Model
     class PolymorphicError < Error
-      def initialize(value, options)
-        super("#{value.class} not in #{options[:polymorphic]}")
+      def initialize(value, options, type)
+        error = if options[:polymorphic].is_a?(Array)
+                  "#{value.class} not in #{options[:polymorphic]}"
+                else
+                  "#{value.class} is not valid sub class of #{type}"
+                end
+        super(error)
       end
     end
   end