lutaml-model 0.7.7 → 0.8.1

data/docs/cli_compare.adoc

@@ -0,0 +1,249 @@
+= CLI Compare Feature in Lutaml::Model
+
+The Lutaml::Model CLI provides a powerful `compare` command that allows you to compare two data files of different formats using your Lutaml::Model classes. This feature calculates similarity scores and provides detailed difference reports, making it valuable for data validation, testing, and migration scenarios.
+
+== Overview
+
+The `compare` command:
+
+* Supports multiple file formats: XML, JSON, YAML (.yml/.yaml), TOML, and any other format supported by your Lutaml::Model classes
+* Provides cross-format comparison (e.g., compare XML with JSON)
+* Calculates similarity percentage scores
+* Shows detailed differences between files
+* Uses your existing Lutaml::Model classes for parsing and comparison logic
+
+== Installation
+
+The CLI is available as part of the lutaml-model gem:
+
+[source,bash]
+----
+gem install lutaml-model
+----
+
+Or add to your Gemfile:
+
+[source,ruby]
+----
+gem 'lutaml-model'
+----
+
+== Basic Usage
+
+=== Command Syntax
+
+[source,bash]
+----
+lutaml-model compare PATH_1 PATH_2 -m MODEL_FILE -r ROOT_CLASS
+----
+
+=== Required Parameters
+
+* `PATH_1` - Path to the first file to compare
+* `PATH_2` - Path to the second file to compare
+* `-m, --model-file` - Path to the Ruby file containing your Lutaml::Model class definitions
+* `-r, --root-class` - Name of the root model class to use for parsing both files
+
+== Examples
+
+=== Comparing XML Files
+
+Given a model definition in `models/document.rb`:
+
+[source,ruby]
+----
+require "lutaml/model"
+
+class ExtractLanguage < Lutaml::Model::Serializable
+  attribute :language, :string
+  attribute :order, :integer
+
+  xml do
+    map_attribute "language", to: :language
+    map_attribute "order", to: :order
+  end
+end
+
+class TermiumExtract < Lutaml::Model::Serializable
+  attribute :language, :string
+  attribute :extract_language, ExtractLanguage, collection: true
+
+  xml do
+    root "termium_extract"
+    namespace "http://termium.tpsgc-pwgsc.gc.ca/schemas/2012/06/Termium", "ns2"
+
+    map_attribute "language", to: :language
+    map_element "extractLanguage", to: :extract_language
+  end
+
+  yaml do
+    map "language", to: :language
+    map "extract_language", to: :extract_language
+  end
+
+  json do
+    map "language", to: :language
+    map "extract_language", to: :extract_language
+  end
+end
+----
+
+Compare two XML files:
+
+[source,bash]
+----
+lutaml-model compare data/file1.xml data/file2.xml \
+  -m models/document.rb \
+  -r TermiumExtract
+----
+
+Sample output:
+
+[source,text]
+----
+Differences between data/file1.xml and data/file2.xml:
+  └── TermiumExtract
+      ├── language (Lutaml::Model::Type::String):
+      │   └──  (Lutaml::Model::Type::String):
+      │       ├── - (String) "EN"
+      │       └── + (nil)
+      └── extract_language (collection):
+          ├── [1] (ExtractLanguage)
+          │   ├── language (Lutaml::Model::Type::String):
+          │   │   ├── - (String) "EN"
+          │   │   └── + (String) "EN1"
+          │   └── order (Lutaml::Model::Type::Integer):
+          │       ├── - (Integer) 0
+          │       └── + (Integer) 1
+          └── - [2] (ExtractLanguage)
+              ├── language (Lutaml::Model::Type::String):
+              │   └── (String) "FR"
+              └── order (Lutaml::Model::Type::Integer):
+                  └── (Integer) 1
+Similarity score: 12.5%
+----
+
+=== Cross-Format Comparison
+
+Compare an XML file with a JSON file:
+
+[source,bash]
+----
+lutaml-model compare data/source.xml data/target.json \
+  -m models/document.rb \
+  -r TermiumExtract
+----
+
+Compare YAML with XML:
+
+[source,bash]
+----
+lutaml-model compare config.yml data.xml \
+  -m models/document.rb \
+  -r TermiumExtract
+----
+
+=== Checking for Identical Files
+
+When files are identical (100% similarity):
+
+[source,bash]
+----
+lutaml-model compare data/file1.xml data/file1.xml \
+  -m models/document.rb \
+  -r TermiumExtract
+----
+
+Output:
+[source,text]
+----
+Differences between data/file1.xml and data/file1.xml:
+  └── TermiumExtract
+Similarity score: 100%
+----
+
+== Advanced Use Cases
+
+=== Data Migration Validation
+
+When migrating data between formats:
+
+[source,bash]
+----
+# Validate XML to JSON conversion
+lutaml-model compare original.xml converted.json \
+  -m models/schema.rb \
+  -r MyModel
+
+# Check YAML to TOML migration
+lutaml-model compare config.yml config.toml \
+  -m models/config.rb \
+  -r ConfigModel
+----
+
+=== Testing Data Transformations
+
+[source,bash]
+----
+# Compare before and after data processing
+lutaml-model compare input/raw_data.xml output/processed_data.xml \
+  -m models/data_model.rb \
+  -r DataRecord
+----
+
+=== Configuration Validation
+
+[source,bash]
+----
+# Compare production vs development configs
+lutaml-model compare config/production.yml config/development.yml \
+  -m models/config.rb \
+  -r AppConfig
+----
+
+== Error Handling
+
+=== Common Errors and Solutions
+
+**File not found:**
+[source,text]
+----
+Error: File not found: /path/to/file.xml
+----
+*Solution:* Verify the file path exists and is accessible.
+
+**Model file not found:**
+[source,text]
+----
+Error: Model file not found: models/missing.rb
+----
+*Solution:* Check that the model file path is correct.
+
+**Class not defined:**
+[source,text]
+----
+Error: MyClass not defined in model-file
+----
+*Solution:* Ensure the class name matches exactly and is properly defined in your model file.
+
+**Parsing errors:**
+[source,text]
+----
+Error parsing file data.xml: Invalid XML syntax
+----
+*Solution:* Verify file format and syntax. The CLI will attempt to parse files according to their extension.
+
+== Supported File Formats
+
+The compare command automatically detects file format based on extension:
+
+* `.xml` → XML format
+* `.json` → JSON format
+* `.yml`, `.yaml` → YAML format
+* `.toml` → TOML format
+* Any other format supported by your model's format mappings
+
+== Limitations
+
+* Both files must be parseable by the same model class
+* The model class must define appropriate format mappings for both file types

data/docs/collection_validation.adoc

@@ -0,0 +1,366 @@
+= Collection Validation Examples
+
+This document demonstrates the new collection-level validation functionality in `Lutaml::Model::Collection`.
+
+== Overview
+
+Collections support two types of validations:
+
+. *Instance-level validations*: Applied to each individual item in the collection (existing functionality)
+. *Collection-level validations*: Applied to the collection as a whole (new functionality)
+. *Validation chaining*: Collection-level validations can share state and coordinate with each other
+
+== Instance-level Validations (Existing)
+
+These validations work on each individual item in the collection:
+
+[source,ruby]
+----
+class PublicationCollection < Lutaml::Model::Collection
+  instances(:publications, Publication) do
+    validates :year, numericality: { greater_than: 1900 }
+    validates :title, presence: true
+
+    validate :must_have_author
+
+    def must_have_author(publications)
+      publications.each do |publication|
+        next unless publication.author.nil?
+        errors.add(:author, "`#{publication.title}` must have an author")
+      end
+    end
+  end
+end
+----
+
+== Collection-level Validations (New)
+
+These validations work on the entire collection:
+
+=== 1. Uniqueness Validation
+
+Ensures that a field value is unique across all instances in the collection:
+
+[source,ruby]
+----
+class UniquePublicationCollection < Lutaml::Model::Collection
+  instances :publications, Publication
+  validates_uniqueness_of :id, message: "Publication IDs must be unique"
+  validates_uniqueness_of :title  # Uses default message
+end
+
+# Usage
+collection = UniquePublicationCollection.new([
+  Publication.new(id: "1", title: "Title A"),
+  Publication.new(id: "1", title: "Title B")  # Duplicate ID!
+])
+
+collection.validate!
+# => Raises ValidationError: "Publication IDs must be unique"
+----
+
+=== 2. Count Validations
+
+Ensures the collection has the right number of items:
+
+[source,ruby]
+----
+class SizedPublicationCollection < Lutaml::Model::Collection
+  instances :publications, Publication
+  validates_min_count 2, message: "Must have at least 2 publications"
+  validates_max_count 10, message: "Cannot have more than 10 publications"
+end
+
+# Usage
+collection = SizedPublicationCollection.new([Publication.new(id: "1")])
+collection.validate!
+# => Raises ValidationError: "Must have at least 2 publications"
+----
+
+=== 3. "All Must Have" Validation
+
+Ensures all instances have specific required attributes:
+
+[source,ruby]
+----
+class CompletePublicationCollection < Lutaml::Model::Collection
+  instances :publications, Publication
+  validates_all_present :author, message: "All publications must have an author"
+  validates_all_present :year
+end
+
+# Usage
+collection = CompletePublicationCollection.new([
+  Publication.new(id: "1", title: "Title A", author: "Author A"),
+  Publication.new(id: "2", title: "Title B")  # Missing author!
+])
+
+collection.validate!
+# => Raises ValidationError: "All publications must have an author"
+----
+
+=== 4. Custom Collection Validations
+
+Define custom validation logic that operates on the entire collection:
+
+[source,ruby]
+----
+class CustomValidatedPublicationCollection < Lutaml::Model::Collection
+  instances :publications, Publication
+
+  # Ensure publication years are sequential
+  validate_collection do |publications, errors|
+    return if publications.empty?
+
+    years = publications.map(&:year).compact.sort
+    (1...years.length).each do |i|
+      unless years[i] == years[i-1] + 1
+        errors.add(:collection, "Publication years must be sequential")
+        break
+      end
+    end
+  end
+
+  # Ensure diversity of categories
+  validate_collection do |publications, errors|
+    categories = publications.map(&:category).compact
+    if categories.uniq.length < 2
+      errors.add(:collection, "Collection must have publications from at least 2 different categories")
+    end
+  end
+end
+----
+
+== Validation Chaining
+
+Collection-level validations can share state through a context object, enabling sophisticated validation workflows:
+
+=== The ValidationContext Object
+
+Each validation receives a third parameter (context) that enables chaining:
+
+[source,ruby]
+----
+class PublicationCollection < Lutaml::Model::Collection
+  instances :publications, Publication
+
+  validates_uniqueness_of :id
+
+  validate_collection do |collection, errors, ctx|
+    # ctx stores results from previous validations
+    # ctx[:duplicates_of_id] contains any duplicate IDs found
+
+    if ctx[:duplicates_of_id]&.any?
+      errors.add(:collection, "Cannot proceed with duplicate IDs")
+    end
+  end
+end
+----
+
+=== Context Methods
+
+The context object provides these methods:
+
+* `ctx[:key]` - Get stored value
+* `ctx[:key] = value` - Store a value for downstream validations
+* `ctx.failed?` - Check if any errors have been added
+* `ctx.stopped?` - Check if chain was stopped
+* `ctx.stop!` - Stop the validation chain
+
+=== Built-in Context Values
+
+After validation methods run, these values are automatically stored:
+
+* `ctx[:duplicates_of_<field>]` - Set of duplicate values for uniqueness validations
+* `ctx[:missing_<field>_count]` - Count of items missing a field for `validates_all_present`
+
+=== Conditional Execution
+
+Validations can be conditionally run based on context state:
+
+[source,ruby]
+----
+class PublicationCollection < Lutaml::Model::Collection
+  instances :publications, Publication
+
+  validates_uniqueness_of :id
+
+  # Only run if uniqueness passed
+  validate_collection(if_cond: ->(ctx) { !ctx[:duplicates_of_id]&.any? }) do |collection, errors, ctx|
+    # This expensive validation only runs when IDs are unique
+  end
+
+  # Skip when condition is met
+  validate_collection(unless_cond: ->(ctx) { ctx[:duplicates_of_id]&.any? }) do |collection, errors, ctx|
+    # This also only runs when IDs are unique
+  end
+end
+----
+
+=== Early Exit
+
+Stop the validation chain when a critical validation fails:
+
+[source,ruby]
+----
+class PublicationCollection < Lutaml::Model::Collection
+  instances :publications, Publication
+
+  validate_collection do |collection, errors, ctx|
+    # Critical validation
+    if collection.empty?
+      errors.add(:collection, "Cannot have empty collection")
+      ctx.stop!  # Stop further validations
+    end
+  end
+
+  validate_collection do |collection, errors, ctx|
+    # This will NOT run if the first validation called ctx.stop!
+  end
+end
+----
+
+=== Complete Chaining Example
+
+[source,ruby]
+----
+class BusinessRuleCollection < Lutaml::Model::Collection
+  instances :publications, Publication
+
+  # Step 1: Basic data integrity
+  validates_uniqueness_of :id
+  validates_all_present :author
+  validates_min_count 1
+
+  # Step 2: Cross-validation (only runs if above passed)
+  validate_collection do |collection, errors, ctx|
+    return if ctx[:duplicates_of_id]&.any?
+
+    # Expensive cross-reference check
+    # (safely skipped if duplicates exist)
+  end
+
+  # Step 3: Aggregate business rules
+  validate_collection do |collection, errors, ctx|
+    # Can reference results from any previous validation
+    if ctx[:missing_author_count].to_i > 5
+      errors.add(:collection, "Too many items missing author")
+    end
+  end
+end
+----
+
+== Combining Instance and Collection Validations
+
+You can use both types of validations together:
+
+[source,ruby]
+----
+class MixedValidationPublicationCollection < Lutaml::Model::Collection
+  # Instance-level validations (applied to each item)
+  instances(:publications, Publication) do
+    validates :year, numericality: { greater_than: 1900 }
+    validates :title, presence: true
+  end
+
+  # Collection-level validations (applied to the whole collection)
+  validates_uniqueness_of :id
+  validates_min_count 1
+  validates_all_present :author
+
+  # Custom collection validation
+  validate_collection do |publications, errors|
+    total_pages = publications.sum { |pub| pub.pages || 0 }
+    if total_pages > 10000
+      errors.add(:collection, "Total pages across all publications cannot exceed 10,000")
+    end
+  end
+end
+----
+
+== Available Collection Validation Methods
+
+=== Built-in Validators
+
+* `validates_uniqueness_of(field, message: nil)` - Ensures field values are unique
+* `validates_min_count(count, message: nil)` - Ensures minimum number of items
+* `validates_max_count(count, message: nil)` - Ensures maximum number of items
+* `validates_all_present(field, message: nil)` - Ensures all items have the field
+
+=== Custom Validators
+
+* `validate_collection(&block)` - Define custom validation logic
+* `validate_collection(if_cond: ->(ctx) { ... }, &block)` - Conditional validation
+* `validate_collection(unless_cond: ->(ctx) { ... }, &block)` - Skip on condition
+
+The block can receive 1, 2, or 3 parameters:
+
+* `|collection|` - Just the collection items
+* `|collection, errors|` - Collection and errors object
+* `|collection, errors, ctx|` - Collection, errors, and context for chaining
+
+== Error Handling
+
+Collection validations integrate with the existing error handling system:
+
+[source,ruby]
+----
+collection = UniquePublicationCollection.new([duplicate_items])
+
+# Check for errors without raising
+errors = collection.validate
+if errors.any?
+  puts "Validation failed: #{errors.map(&:message).join(', ')}"
+end
+
+# Raise on validation failure
+begin
+  collection.validate!
+rescue Lutaml::Model::ValidationError => e
+  puts "Validation error: #{e.message}"
+end
+----
+
+== Use Cases
+
+Collection-level validations are ideal for:
+
+. *Uniqueness constraints*: Ensuring IDs, names, or other identifiers are unique
+. *Business rules*: Enforcing domain-specific collection constraints
+. *Data integrity*: Ensuring the collection as a whole makes sense
+. *Size constraints*: Limiting or requiring minimum collection sizes
+. *Cross-item relationships*: Validating relationships between items
+. *Aggregated properties*: Validating sums, averages, or other aggregate values
+
+== Migration Guide
+
+Existing collections will continue to work without changes. To add collection-level validations:
+
+. Keep existing instance validations in the `instances` block
+. Add collection validations using the new class methods or `validate_collection`
+. Both types of validations will run when `validate` or `validate!` is called
+. For chaining, add `ctx` parameter and use context methods
+
+[source,ruby]
+----
+# Old code still works:
+class OldCollection < Lutaml::Model::Collection
+  instances :items, Item do
+    validates :name, presence: true
+  end
+end
+
+# Add collection-level validation:
+class NewCollection < Lutaml::Model::Collection
+  instances :items, Item do
+    validates :name, presence: true
+  end
+
+  validates_uniqueness_of :id
+
+  # Optional: Add chaining
+  validate_collection do |items, errors, ctx|
+    # ctx[:duplicates_of_id] is automatically available
+  end
+end
+----

data/docs/index.adoc

@@ -0,0 +1,108 @@
+---
+title: Home
+nav_order: 1
+---
+:page-layout: home
+
+= Lutaml::Model: Ruby data modeler
+
+image:https://img.shields.io/gem/v/lutaml-model.svg[RubyGems Version]
+image:https://img.shields.io/github/license/lutaml/lutaml-model.svg[License]
+image:https://github.com/lutaml/lutaml-model/actions/workflows/rake.yml/badge.svg["Build", link="https://github.com/lutaml/lutaml-model/actions/workflows/rake.yml"]
+image:https://github.com/lutaml/lutaml-model/actions/workflows/dependent-tests.yml/badge.svg["Dependent tests", link="https://github.com/lutaml/lutaml-model/actions/workflows/dependent-tests.yml"]
+
+Lutaml::Model is a Ruby library for creating information models with attributes and types, providing flexible and comprehensive mechanisms for serializing to and from multiple formats including XML, JSON, YAML, TOML, and Hash.
+
+== Key features
+
+* **Model-driven design** - Define models with attributes and types
+* **Multi-format serialization** - XML, JSON, YAML, TOML, Hash, YAML Stream
+* **XML namespace support** - Full W3C namespace implementation
+* **Validation** - Built-in validation with custom rules
+* **Schema generation** - Generate XSD, JSON Schema, YAML Schema
+* **Polymorphism** - Handle multiple types elegantly
+* **Collections** - Work with groups of models
+* **Custom types** - Extend with your own value types
+
+== Getting started
+
+Choose your path:
+
+link:installation[**Installation**]::
+Install Lutaml::Model and configure adapters
+
+link:quick-start[**Quick Start**]::
+Create your first model in 5 minutes
+
+link:tutorials/index[**Tutorials**]::
+Step-by-step guides for progressive learning
+
+link:guides/index[**Guides**]::
+Task-oriented documentation for specific goals
+
+link:index[**Core Topics**]::
+Complete reference for all features
+
+== Quick example
+
+[source,ruby]
+----
+require 'lutaml/model'
+
+class Person < Lutaml::Model::Serializable
+  attribute :name, :string
+  attribute :age, :integer
+
+  xml do
+    root "person"
+    map_element "name", to: :name
+    map_element "age", to: :age
+  end
+
+  key_value do
+    map "name", to: :name
+    map "age", to: :age
+  end
+end
+
+# Serialize
+person = Person.new(name: "John", age: 30)
+person.to_xml   # => <person><name>John</name><age>30</age></person>
+person.to_json  # => {"name":"John","age":30}
+
+# Deserialize
+Person.from_xml(xml_string)
+Person.from_json(json_string)
+----
+
+== Documentation sections
+
+=== link:index[Core Topics]
+
+Fundamental concepts, configuration, and essential features.
+
+* link:yamls_sequence.adoc[YAMLS Sequence -- Heterogeneous YAML Stream Support]
+
+=== link:tutorials/index[Tutorials]
+
+Progressive, step-by-step learning from basics to advanced topics.
+
+=== link:guides/index[Guides]
+
+Task-oriented guides for accomplishing specific goals.
+
+=== link:references/index[Reference]
+
+Technical reference, architecture, and implementation details.
+
+== Community
+
+* link:https://github.com/lutaml/lutaml-model[GitHub Repository]
+* link:https://github.com/lutaml/lutaml-model/issues[Issue Tracker]
+* link:https://rubygems.org/gems/lutaml-model[RubyGems Page]
+
+== License
+
+Lutaml::Model is released under the BSD 2-Clause License.
+
+Copyright (c) 2025 Ribose Inc.