lutaml-model 0.8.56 → 0.8.106

data/docs/_guides/native-engines.adoc

@@ -67,11 +67,22 @@ When the XML adapter resolves to leptris, XML (de)serialization
 compiles the model's mapping into a leptris descriptor plan and runs
 the whole document through one native pass — measured 6.7x parse and
 4x serialize with -76% / -80% allocations on the plan-path corpus.
-Models the plan cannot express exactly (same-name discriminator
-partitions, namespace-qualified models with deferred-rule shapes) and
-shapes with custom semantics (custom methods, polymorphism, unions,
-ordered children) hydrate through the interpretive pipeline or a
-per-subtree fragment of it, unchanged.
+Namespace-qualified models ride the path too: serialization spells
+prefixed element and attribute names through the same namespace
+planning the interpretive writer uses, and every `(prefix, uri)`
+pair is declared once on the output root. Models the plan cannot
+express exactly (same-name discriminator partitions, deferred-rule
+shapes) and shapes with custom semantics (custom methods,
+polymorphism, unions, ordered children) hydrate through the
+interpretive pipeline or a per-subtree fragment of it, unchanged.
+
+Version floor note: leptris 1.9.320+ retains child-row namespace
+URIs engine-side and matches plain attribute rows against
+namespace-qualified wire spellings. On older engines, models whose
+plans carry exact-URI child rows or plain attribute rows under a
+namespaced chain parse interpretively for correctness — a per-model
+fallback, not an error. Bundles resolving leptris 1.9.320+ get the
+full fast path.
 
 The path is on by default and is a no-op without leptris (standard
 adapters keep the classic pipeline). Opt out for the whole process:
@@ -106,6 +117,13 @@ Lutaml::Model::Config.toml_adapter_type = :teptris
   raising `Psych::DisallowedClass`.
 * **JSON**: `yeptris` parsing targets exact `JSON.parse` semantics;
   generation stays on the `json` gem.
+* **Error classes**: parse failures keep the standard adapters' contract
+  on every engine — the yeptris adapters re-raise the engine's syntax
+  errors as `Psych::SyntaxError` (YAML, with the engine's line, column,
+  and problem text) and `JSON::ParserError` (JSON), and the model layer
+  wraps them as `InvalidFormatError` just as it does for Psych and the
+  `json` gem. Code that rescues those classes does not change when a
+  native engine is added to the bundle.
 * **TOML**: the teptris adapter mirrors tomlib value semantics — offset
   and local datetimes become `Time`, dates become `Date`, local times
   stay `String`. Parse failures raise `Teptris::ParseError` with line

data/docs/_guides/schema-generation.adoc

@@ -901,3 +901,28 @@ The generated schema will include validation constraints:
   </define>
 </grammar>
 ----
+
+== Documentation metadata (desc)
+
+Models and attributes carry documentation strings through `desc` —
+set on the model class or on any `attribute` declaration, read back
+by documentation and schema tooling. Serialization is never affected
+by it.
+
+[source,ruby]
+----
+class Title < Lutaml::Model::Serializable
+  desc "This is a title."
+
+  attribute :type, :string, desc: "This attribute is used for..."
+end
+
+Title.desc # => "This is a title."
+Title.attributes[:type].desc # => "This attribute is used for..."
+----
+
+Subclasses inherit the nearest ancestor's class description; an
+explicit `desc` overrides it. A consumer override-free `desc` value
+does not change any generated schema today — wiring `desc` into the
+generated `description` fields is deliberate product scope, not a
+free rider on this option.

data/docs/_guides/type-registers-contexts.adoc

@@ -0,0 +1,126 @@
+---
+title: Type Registers and Contexts
+nav_order: 20
+---
+
+= Type registers and contexts
+
+Lutaml::Model resolves every attribute type — Symbol or class —
+through a per-application *register*: a named table of types scoped
+so that different class trees can share mapping definitions without
+colliding. Registers are the mechanism behind multi-version model
+suites and model substitution.
+
+== Registers
+
+A register maps type ids to classes. Classes declare the register
+they belong to:
+
+[source,ruby]
+----
+module MyModels
+  class Mi < Lutaml::Model::Serializable
+    def self.lutaml_default_register
+      :my_models
+    end
+  end
+end
+----
+
+Symbol attribute types resolve in the declaring class's register,
+so the same symbol can mean different classes per tree — the
+multi-version pattern below builds on this.
+
+== Contexts: derived registers with substitutions
+
+A *context* is a derived type-resolution scope: it carries a
+registry, fallback registers, and — the part that powers model
+substitution — a list of type substitutions. Contexts are created
+through the global registry:
+
+[source,ruby]
+----
+Lutaml::Model::GlobalContext.create_context(
+  id: :custom_models,
+  fallback_to: [:my_models],
+  substitutions: [
+    { from_type: MyModels::Mover, to_type: Overset },
+  ],
+)
+
+instance = MyModels::Math.from_xml(xml, context: :custom_models)
+instance.mover_value.first # => #<Overset>
+----
+
+Every attribute whose type resolves to `MyModels::Mover` — at any
+depth of the tree — hydrates `Overset` in that context. Substitution
+is type-based, so "swap this class everywhere it is referenced" and
+"swap it only inside this one path" are the same operation scoped by
+context: create two contexts with different substitution lists.
+
+Contexts are cheap and disposable:
+
+[source,ruby]
+----
+Lutaml::Model::GlobalContext.unregister_context(:custom_models)
+----
+
+=== The custom-models convenience
+
+A whole-table convenience reads exactly like a class-swap hash and
+derives the context for you:
+
+[source,ruby]
+----
+models = { MyModels::Mover => Overset, MyModels::Mi => BoldMi }
+Lutaml::Model::GlobalContext.create_context(
+  id: :custom, fallback_to: [:my_models],
+  substitutions: models.map { |from, to| { from_type: from, to_type: to } },
+)
+MyModels::Math.from_xml(xml, context: :custom)
+----
+
+Substitute classes should inherit from the class they replace (or
+provide its full attribute surface) so mappings and serialization
+keep working.
+
+=== Substituting a symbol entry instead of a class
+
+Registers accept direct class registrations under a type id — useful
+when a tree declares `attribute :row, :row` and you want the symbol
+to resolve to your own class within a derived register:
+
+[source,ruby]
+----
+derived = Lutaml::Model::Register.new(:derived, fallback: [:my_models])
+derived.register_model(NewRow, id: :row)
+MyModels::Doc.from_xml(xml, register: derived)
+----
+
+(`register:` is the pre-context spelling; prefer `context:` with
+substitutions in new code.)
+
+== Multi-version class trees
+
+The flagship pattern combines registers with per-version classes
+and shared mapping modules:
+
+* a `Base::Mrow` module holds the mapping DSL once;
+* each version declares `class Mrow` mixing the module in and
+  pointing `lutaml_default_register` at its version's register;
+* attributes use symbol types, so `attribute :mrow, :mrow` means
+  the right class per tree;
+* a substitution context swaps individual classes for one call
+  without re-opening or redefining anything.
+
+Nothing re-opens generated classes, and each version's tree stays
+independent — no hidden delegation.
+
+== Rules of thumb
+
+* Prefer contexts over global configuration: substitutions scoped
+  to a context cannot leak into other resolutions.
+* Substitute classes should inherit the original; substitution then
+  preserves every mapping declared on the base.
+* Clean up exploratory contexts with `unregister_context` in test
+  teardowns — contexts are global state.

data/lib/compat/opal/lutaml_model_boot.rb

@@ -147,6 +147,7 @@ require "lutaml/xml/format"
 require "lutaml/xml/format_chooser"
 require "lutaml/xml/hoisting_algorithm"
 require "lutaml/xml/html_entities"
+require "lutaml/xml/leptris"
 require "lutaml/xml/listener"
 require "lutaml/xml/mapping"
 require "lutaml/xml/mapping_rule"
@@ -170,11 +171,6 @@ require "lutaml/xml/oga"
 require "lutaml/xml/parse_session"
 require "lutaml/xml/parsed_namespace_declaration"
 require "lutaml/xml/parsed_namespace_set"
-require "lutaml/xml/plan_compiler"
-require "lutaml/xml/plan_hydrator"
-require "lutaml/xml/plan_order"
-require "lutaml/xml/plan_serializer"
-require "lutaml/xml/plan_walk"
 require "lutaml/xml/polymorphic_value_handler"
 require "lutaml/xml/qualified_inheritance_strategy"
 require "lutaml/xml/rexml"
@@ -386,6 +382,12 @@ require "lutaml/xml/error/namespace_mismatch_error"
 require "lutaml/xml/error/schema_validation_error"
 require "lutaml/xml/error/xml_configuration_error"
 require "lutaml/xml/error/xml_error"
+require "lutaml/xml/leptris/native_emission"
+require "lutaml/xml/leptris/plan_compiler"
+require "lutaml/xml/leptris/plan_hydrator"
+require "lutaml/xml/leptris/plan_order"
+require "lutaml/xml/leptris/plan_serializer"
+require "lutaml/xml/leptris/plan_walk"
 require "lutaml/xml/oga/element"
 require "lutaml/xml/ox/element"
 require "lutaml/xml/rexml/element"

data/lib/lutaml/json/adapter/yeptris_adapter.rb

@@ -12,9 +12,32 @@ module Lutaml
       # load-only), so everything else inherits from the standard adapter.
       class YeptrisAdapter < StandardAdapter
         def self.parse(json, _options = {})
-          require "yeptris"
+          require_engine
           ::Yeptris::JSON.load(json)
+        rescue ::Yeptris::JSON::ParseError => e
+          raise parser_error(e)
+        end
+
+        # The engine require sits outside parse's rescue so a broken
+        # install still surfaces its LoadError (the detection fallback's
+        # contract) instead of a NameError from the rescue clause.
+        def self.require_engine
+          require "yeptris"
+        end
+        private_class_method :require_engine
+
+        # The parse-error contract is the standard adapter's: invalid
+        # input raises JSON::ParserError regardless of the engine, so
+        # consumers' rescue ladders (and this gem's own
+        # format_error_types conversion to InvalidFormatError) hold on
+        # every adapter. Without the json gem the engine error
+        # propagates unchanged.
+        def self.parser_error(error)
+          return error unless defined?(::JSON::ParserError)
+
+          ::JSON::ParserError.new(error.message)
         end
+        private_class_method :parser_error
       end
     end
   end

data/lib/lutaml/key_value/data_model/element.rb

@@ -193,7 +193,11 @@ module Lutaml
         #
         # @return [Hash]
         def children_to_hash
-          @children.map(&:to_hash).reduce({}, :merge)
+          # Built in place: map+reduce(:merge) allocated the mapped
+          # array plus one merged hash per reduce step (TODO.perf 16).
+          h = {}
+          @children.each { |child| h.update(child.to_hash) }
+          h
         end
 
         # Convert children to Array

data/lib/lutaml/key_value/mapping_rule.rb

@@ -83,6 +83,13 @@ module Lutaml
       def root_mapping?
         name == "root_mapping"
       end
+
+      # The writer symbol for this rule's target, interned once per
+      # rule: the interpolation allocated a String and interned it per
+      # row per document in the transform's hot loop.
+      def kv_setter
+        @kv_setter ||= :"#{to}="
+      end
     end
   end
 end

data/lib/lutaml/key_value/transform.rb

@@ -1,5 +1,6 @@
 module Lutaml
   module KeyValue
+    MISSING = Object.new.freeze
     class Transform < Lutaml::Model::Transform
       def data_to_model(data, format, options = {})
         # Use child's own default register if it has one
@@ -55,13 +56,9 @@ module Lutaml
           # transformation_for returns nil for cyclic dependencies or :building sentinel
           # Fall back to legacy approach in these cases
           if transformation.is_a?(Lutaml::KeyValue::Transformation)
-            # Use new Transformation to get KeyValueElement
-            kv_element = transformation.transform(instance, options)
-            # Convert KeyValueElement to hash for backward compatibility with adapters
-            # The to_hash method returns {"__root__" => {actual_hash}}
-            kv_hash = kv_element.to_hash
-            # For root element, return just the content hash
-            return kv_hash["__root__"] || kv_hash
+            # transform emits the output hash directly (TODO.perf 19) —
+            # the __root__ Element tree and its to_hash flatten are gone.
+            return transformation.transform(instance, options)
           end
         end
 
@@ -176,18 +173,21 @@ module Lutaml
             .substitution_for(type).any?
 
           if type.is_a?(Class) && type.include?(Lutaml::Model::Serialize)
-            child_rows = kv_group_plan(type, format,
-                                       Lutaml::Model::Register
-                                         .resolve_for_child(type, register))
+            child_register = Lutaml::Model::Register
+              .resolve_for_child(type, register)
+            child_rows = kv_group_plan(type, format, child_register)
             return false unless child_rows
 
             rows ||= []
-            rows << [rule, attr, :model, type, child_rows]
+            # child_register rides the row: the plan was BUILT under it,
+            # and rebuilding the resolution per model row per document
+            # cost a Concurrent::Map hop each time (TODO.perf 14).
+            rows << [rule, attr, :model, type, child_rows, child_register]
           elsif type.is_a?(Class) && type < Lutaml::Model::Type::Value &&
               !attr.value_policy.whole_value?(type) &&
               !Lutaml::Model::Attribute.custom_from_probe?(type)
             rows ||= []
-            rows << [rule, attr, :scalar, nil, nil]
+            rows << [rule, attr, :scalar, nil, nil, nil]
           else
             return false
           end
@@ -206,21 +206,24 @@ module Lutaml
       def kv_group_build(rows, doc, format, register, options)
         setters = []
         children = []
-        absent = []
-        rows.each do |rule, attr, kind, type, child_rows|
-          unless Lutaml::Model::Utils.string_or_symbol_key?(doc, rule.name)
-            absent << rule
+        # Lazy: the present-key hot path never pushes, and this runs
+        # once per hydrated model instance.
+        absent = nil
+        rows.each do |rule, attr, kind, type, child_rows, child_register|
+          # One hash pass per row: fetch with a sentinel distinguishes
+          # absent keys from present-nil values (both read as nil), so
+          # the separate string_or_symbol_key? probe goes away.
+          v = Lutaml::Model::Utils.fetch_str_or_sym(doc, rule.name, MISSING)
+          if v.equal?(MISSING)
+            (absent ||= []) << rule
             next
           end
-          v = Lutaml::Model::Utils.fetch_str_or_sym(doc, rule.name)
 
           if kind == :scalar
-            setters << [:"#{rule.to}=", v]
+            setters << [rule.kv_setter, v]
             next
           end
 
-          child_register = Lutaml::Model::Register.resolve_for_child(type,
-                                                                     register)
           if attr.collection?
             # A present-but-nil collection reaches the per-rule walk,
             # which owns the sentinel interplay for that edge
@@ -239,7 +242,7 @@ module Lutaml
 
               built << child
             end
-            setters << [:"#{rule.to}=", built]
+            setters << [rule.kv_setter, built]
             children.concat(built)
           else
             return nil unless v.is_a?(::Hash)
@@ -249,13 +252,13 @@ module Lutaml
                                                  options)
             return nil if child.nil?
 
-            setters << [:"#{rule.to}=", child]
+            setters << [rule.kv_setter, child]
             children << child
           end
         end
         instance = model_class.new(lutaml_register: register)
         setters.each { |name, value| instance.public_send(name, value) }
-        absent.each do |rule|
+        absent&.each do |rule|
           process_mapping_rule(doc, instance, format, rule, options, nil)
         end
         children.each do |child|
@@ -342,7 +345,9 @@ options)
       end
 
       def valid_mapping?(rule, options)
-        return false if rule.respond_to?(:serialize?) && !rule.serialize?
+        # rule is always Lutaml::Model::MappingRule; serialize? is
+        # statically defined (the hydrate-only marker).
+        return false unless rule.serialize?
 
         only = options[:only]
         except = options[:except]

data/lib/lutaml/key_value/transformation/collection_serializer.rb

@@ -140,15 +140,11 @@ options = {})
           nil_mapping = to_map[:nil]
 
           if nil_mapping == :empty || converted_from_nil_to_empty
-            element = Lutaml::KeyValue::DataModel::Element.new(
-              rule.serialized_name, []
-            )
-            parent.add_child(element)
+            puts "  HNC-EMPTY #{rule.serialized_name}" if ENV["T19"]
+            parent[rule.serialized_name] = []
           elsif nil_mapping == :nil || converted_from_empty_to_nil
-            element = Lutaml::KeyValue::DataModel::Element.new(
-              rule.serialized_name, nil
-            )
-            parent.add_child(element)
+            puts "  HNC-NIL #{rule.serialized_name}" if ENV["T19"]
+            parent[rule.serialized_name] = nil
           end
           # else: Skip nil collection (default behavior)
         end
@@ -158,8 +154,6 @@ options = {})
           key_attribute = root_mappings.key(:key)
           value_attribute = root_mappings.key(:value)
 
-          parent.value ||= {} if parent.value.nil?
-
           if key_attribute && value_attribute
             handle_root_mappings_key_value(parent, items, key_attribute,
                                            value_attribute, rule, options)
@@ -213,7 +207,7 @@ options = {})
                                else
                                  attr_value
                                end
-            parent.value[key_value.to_s] = serialized_value if serialized_value
+            parent[key_value.to_s] = serialized_value if serialized_value
           end
         end
 
@@ -336,7 +330,7 @@ options)
               end
             end
 
-            parent.value[key_value.to_s] = item_hash unless item_hash.empty?
+            parent[key_value.to_s] = item_hash unless item_hash.empty?
           end
         end
 
@@ -362,10 +356,7 @@ options)
             end
           end
 
-          element = Lutaml::KeyValue::DataModel::Element.new(
-            rule.serialized_name, keyed_hash
-          )
-          parent.add_child(element)
+          parent[rule.serialized_name] = keyed_hash
         end
 
         # Find value attribute from child_mappings.
@@ -490,18 +481,11 @@ options)
 
         # Create an array collection element (default).
         def create_array_collection_element(parent, rule, items, options)
-          coll_element = Lutaml::KeyValue::DataModel::Element.new(rule.serialized_name)
-
-          if items.empty?
-            coll_element.value = []
-          else
-            items.each do |item|
-              child_value = create_value_for_item(rule, item, options)
-              coll_element.add_child(child_value) unless child_value.nil?
-            end
+          built = items.each_with_object([]) do |item, memo|
+            child_value = create_value_for_item(rule, item, options)
+            memo << child_value unless child_value.nil?
           end
-
-          parent.add_child(coll_element)
+          parent[rule.serialized_name] = built
         end
 
         # Create a value element for a collection item.
@@ -554,8 +538,7 @@ options)
         def serialize_nested_model(value, attr_type, options)
           child_transformation = create_transformation(attr_type)
           child_root = child_transformation.transform(value, options)
-          child_hash = child_root.to_hash
-          child_hash["__root__"]
+          child_root.empty? ? nil : child_root
         end
 
         # Serialize a primitive value.

data/lib/lutaml/key_value/transformation/value_serializer.rb

@@ -151,8 +151,19 @@ model_class: nil)
             return value.public_send(:"to_#{format}")
           end
 
-          # Wrap value in type and call to_#{format}
           if rule.attribute_type.is_a?(Class) && rule.attribute_type < Lutaml::Model::Type::Value
+            plan = serialize_plan(rule)
+            # Identity only under identity_fast_type's exact condition
+            # (no custom to AND no custom from): a custom from means the
+            # wrapper constructor normalizes the value, so the raw value
+            # must still pass through it.
+            if plan[:identity_type] && value.instance_of?(plan[:identity_type])
+              return value
+            end
+            if (to_fn = plan[:to_fn])
+              return to_fn.call(rule.attribute_type.new(value))
+            end
+
             wrapped_value = rule.attribute_type.new(value)
             wrapped_value.public_send(:"to_#{format}")
           else
@@ -187,9 +198,23 @@ model_class: nil)
             reference: reference_type?(rule),
             union: ::Lutaml::Model::Type::Union.rule?(rule),
             nested: nested_model?(rule),
+            # The custom to_<format> serializer, resolved once per rule:
+            # the default path is pure identity (to_<format> returns the
+            # wrapped .value), so values without a custom serializer skip
+            # the wrap-allocate-cast ceremony entirely, and custom ones
+            # skip the per-value registry lookup.
+            to_fn: custom_to_fn(rule),
           }
         end
 
+        def custom_to_fn(rule)
+          type = rule.attribute_type
+          return nil unless type.is_a?(::Class) && type < ::Lutaml::Model::Type::Value
+
+          ::Lutaml::Model::Type::Value
+            .format_type_serializer_for(@format, type)&.fetch(:to, nil)
+        end
+
         # Builtin scalars whose type registers no custom to_<format>
         # serializer serialize as the value itself — Type::Value#to_<format>'s
         # default — without the wrap-allocate-cast ceremony per field.
@@ -278,12 +303,10 @@ model_class: nil)
         # @param options [Hash] Serialization options
         # @return [Hash, nil] The serialized hash or nil if empty
         def transform_nested_model(value, child_transformation, options)
+          # transform emits the content hash directly (TODO.perf 19);
+          # nil for empty keeps render_nil: false working.
           child_root = child_transformation.transform(value, options)
-          child_hash = child_root.to_hash
-          result = child_hash["__root__"]
-
-          # Return nil for empty hashes (allows render_nil: false to work)
-          result.nil? || result.empty? ? nil : result
+          child_root.empty? ? nil : child_root
         end
 
         # Check if the rule defines a Reference type.