shale 0.9.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3f880bc8d984c45e7e8a05bd155c1014df237e037263c7bf68a7ae82c0e49ea4
-  data.tar.gz: ba7a2f12bed87e048dd53d43900f1c8203d90f5f89a43b3824475ddda39eb07b
+  metadata.gz: 60a0f059693d5ea371ffca8b475ebb261ee99096509e588d2f1163a96bf8fbc0
+  data.tar.gz: c1af0c61c7d9a93bd601188c50916d4f702228052541adee34456c4d7dac8b9a
 SHA512:
-  metadata.gz: 54023d4f621dae53537f73e0ad79d3736d5d52e605c1a7da2c4bdb06588b348feaae9066ce00b6b3aedc9e84ea23a425884eabf987717ffda6d3c22eccc6325f
-  data.tar.gz: e82cad292dab557891cad4f210bfb4a97d2096a6d2151a5a8aef020358accb5986e527e32a141a9a2e3a01d594e9a3adb42c0f7e8d17464517731136f63289a2
+  metadata.gz: 31bd59c352febaf287eda07a79f56e5fe942fd86623f532e48c25ff5e946bb53fc879c3f8bb13121989268057beb4ee740f8aac7b1275dda8b9a2d21a5f75aaf
+  data.tar.gz: 127b46cdeb0960643aa60410597f7ace1815dd19e3845cd0cb7c2ea3d2d44b4326333656bdf96209effcf17daabbc5a70de9f430acd42e21d961e2c356d9be9f

data/CHANGELOG.md

@@ -1,3 +1,32 @@
+## [1.1.0] - 2024-02-17
+
+### Added
+- [bkjohnson] Add support for JSON Schema validation keywords (#29)
+- Add support for Ruby 3.3
+
+### Changed
+- Drop support for Ruby 2.6 and Ruby 2.7
+
+### Fixed
+- Fix Ox adapter incorrectly handling documents with XML declaration
+
+## [1.0.0] - 2023-07-15
+
+### Added
+- Support for Ruby 3.2
+- Support for delegating fields to nested attributes
+- JSON and XML schema namespace mapping support
+- Allow to set render_nil defaults
+
+### Changed
+- Use `ShaleError` as a base class for exceptions
+- Use model instead of mapper names for schema types
+
+### Fixed
+- Fix compilation error for bundled JSON schemas
+- Fix type inference for XML schema when default namespace is used
+- Fix XML schema handling with a period in the element name
+
 ## [0.9.0] - 2022-10-31
 
 ### Added

data/README.md

@@ -17,7 +17,7 @@ Documentation with interactive examples is available at [Shale website](https://
 
 ## Installation
 
-Shale supports Ruby (MRI) 2.6+
+Shale supports Ruby (MRI) 3.0+
 
 Add this line to your application's Gemfile:
 
@@ -63,6 +63,7 @@ $ gem install shale
 * [Using XML namespaces](#using-xml-namespaces)
 * [Rendering nil values](#rendering-nil-values)
 * [Using methods to extract and generate data](#using-methods-to-extract-and-generate-data)
+* [Delegating fields to child attributes](#delegating-fields-to-child-attributes)
 * [Additional options](#additional-options)
 * [Using custom models](#using-custom-models)
 * [Supported types](#supported-types)
@@ -352,6 +353,25 @@ person.to_xml
 
 ### Converting CSV to object
 
+To use CSV with Shale you have to set adapter.
+Shale comes with adapter for [csv](https://github.com/ruby/csv).
+For details see [Adapters](#adapters) section.
+
+To set it, first make sure CSV gem is installed:
+
+```
+$ gem install csv
+```
+
+then setup adapter:
+
+```ruby
+require 'shale/adapter/csv'
+Shale.csv_adapter = Shale::Adapter::CSV
+```
+
+Now you can use CSV with Shale.
+
 CSV represents a flat data structure, so you can't map properties to complex types directly,
 but you can use methods to map properties to complex types
 (see [Using methods to extract and generate data](#using-methods-to-extract-and-generate-data)
@@ -692,6 +712,49 @@ puts person.to_xml(pretty: true)
 # </person>
 ```
 
+If you want to change how nil values are rendered for all mappings you can use `render_nil` method:
+
+```ruby
+class Base < Shale::Mapper
+  json do
+    # change render_nil default for all JSON mappings inheriting from Base class
+    render_nil true
+  end
+end
+
+class Person < Base
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+  attribute :age, Shale::Type::Integer
+
+  json do
+    # override default from Base class
+    render_nil false
+
+    map 'first_name', to: :first_name
+    map 'last_name', to: :last_name
+    map 'age', to: :age, render_nil: true # override default
+  end
+end
+```
+
+:warning: The default affects only the mappings declared after setting the default value e.g.
+
+```ruby
+class Person < Base
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+
+  json do
+    render_nil false
+    map 'first_name', to: :first_name # render_nil will be false for this mapping
+
+    render_nil true
+    map 'last_name', to: :last_name # render_nil will be true for this mapping
+  end
+end
+```
+
 ### Using methods to extract and generate data
 
 If you need full controll over extracting and generating data from/to document,
@@ -865,6 +928,42 @@ DATA
 # => #<Person:0x00007f9bc3086d60 @name="John Doe">
 ```
 
+### Delegating fields to child attributes
+
+To delegate fields to child complex types you can use `receiver: :child` declaration:
+
+```ruby
+class Address < Shale::Mapper
+  attribute :city, Shale::Type::String
+  attribute :street, Shale::Type::String
+end
+
+class Person < Shale::Mapper
+  attribute :name, Shale::Type::String
+  attribute :address, Address
+
+  json do
+    map 'name', to: :name
+    map 'city', to: :city, receiver: :address
+    map 'street', to: :street, receiver: :address
+  end
+end
+
+person = Person.from_json(<<~DATA)
+{
+  "name": "John Doe",
+  "city": "London",
+  "street": "Oxford Street"
+}
+DATA
+
+# =>
+#
+# #<Person:0x00007f9bc3086d60
+#  @name="John Doe",
+#  @address=#<Address:0x0000000102cbd218 @city="London", @street="Oxford Street">>
+```
+
 ### Additional options
 
 You can control which attributes to render and parse by
@@ -1075,7 +1174,7 @@ end
 ### Adapters
 
 Shale uses adapters for parsing and generating documents.
-By default Ruby's standard JSON, YAML, CSV parsers are used for handling JSON YAML, CSV documents.
+By default Ruby's standard JSON and YAML parsers are used for handling JSON and YAML documents.
 
 You can change it by providing your own adapter. For JSON, YAML, TOML and CSV adapter must
 implement `.load` and `.dump` class methods.
@@ -1103,6 +1202,14 @@ require 'shale/adapter/toml_rb'
 Shale.toml_adapter = Shale::Adapter::TomlRB
 ```
 
+To handle CSV documents you have to set CSV adapter. Shale provides adapter for `csv` parser:
+
+```ruby
+require 'shale'
+require 'shale/adapter/csv'
+Shale.csv_adapter = Shale::Adapter::CSV
+```
+
 To handle XML documents you have to explicitly set XML adapter.
 Shale provides adapters for most popular Ruby XML parsers:
 
@@ -1208,11 +1315,82 @@ end
 Shale::Schema::JSONGenerator.register_json_type(MyEmailType, MyEmailJSONType)
 ```
 
+To add validation keywords to the schema, you can use a custom model and do this:
+
+```ruby
+require 'shale/schema'
+
+class PersonMapper < Shale::Mapper
+  model Person
+
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+  attribute :address, Shale::Type::String
+  attribute :age, Shale::Type::Integer
+
+  json do
+    properties max_properties: 5
+
+    map "first_name", to: :first_name, schema: { required: true }
+    map "last_name", to: :last_name, schema: { required: true }
+    map "address", to: :age, schema: { max_length: 128 }
+    map "age", to: :age, schema: { minimum: 1, maximum: 150 }
+  end
+end
+
+Shale::Schema.to_json(
+  PersonMapper,
+  pretty: true
+)
+
+# =>
+#
+# {
+#   "$schema": "https://json-schema.org/draft/2020-12/schema",
+#   "description": "My description",
+#   "$ref": "#/$defs/Person",
+#   "$defs": {
+#     "Person": {
+#       "type": "object",
+#       "maxProperties": 5,
+#       "properties": {
+#         "first_name": {
+#           "type": "string"
+#         },
+#         "last_name": {
+#           "type": "string"
+#         },
+#         "age": {
+#           "type": [
+#             "integer",
+#             "null"
+#           ],
+#          "minimum": 1,
+#          "maximum": 150
+#        },
+#         "address": {
+#           "type": [
+#             "string",
+#             "null"
+#           ],
+#           "maxLength": 128
+#         }
+#       },
+#       "required": ["first_name", "last_name"]
+#     }
+#   }
+# }
+```
+
+Validation keywords are supported for all types, only the global `enum` and `const` types are not supported.
+
 ### Compiling JSON Schema into Shale model
 
 :warning: Only **[Draft 2020-12](https://json-schema.org/draft/2020-12/schema)** JSON Schema is supported
 
-To generate Shale data model from JSON Schema use:
+To generate Shale data model from JSON Schema use `Shale::Schema.from_json`.
+You can pass `root_name: 'Foobar'` to change the name of the root type and
+`namespace_mapping: {}` to map schemas to Ruby modules:
 
 ```ruby
 require 'shale/schema'
@@ -1223,7 +1401,11 @@ schema = <<~SCHEMA
   "properties": {
     "firstName": { "type": "string" },
     "lastName": { "type": "string" },
-    "address": {
+    "address": { "$ref": "http://bar.com" }
+  },
+  "$defs": {
+    "Address": {
+      "$id": "http://bar.com",
       "type": "object",
       "properties": {
         "street": { "type": "string" },
@@ -1234,38 +1416,53 @@ schema = <<~SCHEMA
 }
 SCHEMA
 
-Shale::Schema.from_json([schema], root_name: 'Person')
+Shale::Schema.from_json(
+  [schema],
+  root_name: 'Person',
+  namespace_mapping: {
+    nil => 'Api::Foo', # default schema (without ID)
+    'http://bar.com' => 'Api::Bar',
+  }
+)
 
 # =>
 #
 # {
-#   "address" => "
+#   "api/bar/address" => "
 #     require 'shale'
 #
-#     class Address < Shale::Mapper
-#       attribute :street, Shale::Type::String
-#       attribute :city, Shale::Type::String
+#     module Api
+#       module Bar
+#         class Address < Shale::Mapper
+#           attribute :street, Shale::Type::String
+#           attribute :city, Shale::Type::String
 #
-#       json do
-#         map 'street', to: :street
-#         map 'city', to: :city
+#           json do
+#             map 'street', to: :street
+#             map 'city', to: :city
+#           end
+#         end
 #       end
 #     end
 #   ",
-#   "person" => "
+#   "api/foo/person" => "
 #     require 'shale'
 #
-#     require_relative 'address'
+#     require_relative '../bar/address'
 #
-#     class Person < Shale::Mapper
-#       attribute :first_name, Shale::Type::String
-#       attribute :last_name, Shale::Type::String
-#       attribute :address, Address
+#     module Api
+#       module Foo
+#         class Person < Shale::Mapper
+#           attribute :first_name, Shale::Type::String
+#           attribute :last_name, Shale::Type::String
+#           attribute :address, Api::Bar::Address
 #
-#       json do
-#         map 'firstName', to: :first_name
-#         map 'lastName', to: :last_name
-#         map 'address', to: :address
+#           json do
+#             map 'firstName', to: :first_name
+#             map 'lastName', to: :last_name
+#             map 'address', to: :address
+#           end
+#         end
 #       end
 #     end
 #   "
@@ -1275,7 +1472,7 @@ Shale::Schema.from_json([schema], root_name: 'Person')
 You can also use a command line tool to do it:
 
 ```
-$ shaleb -c -i schema.json -r Person
+$ shaleb -c -i schema.json -r Person -m http://bar.com=Api::Bar,=Api::Foo
 ```
 
 ### Generating XML Schema
@@ -1346,22 +1543,39 @@ Shale::Schema::XMLGenerator.register_xml_type(MyEmailType, 'myEmailXMLType')
 
 ### Compiling XML Schema into Shale model
 
-To generate Shale data model from XML Schema use:
+To generate Shale data model from XML Schema use `Shale::Schema.from_xml`.
+You can pass `namespace_mapping: {}` to map XML namespaces to Ruby modules:
 
 ```ruby
 require 'shale/schema'
 
-schema = <<~SCHEMA
-<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
+schema1 = <<~SCHEMA
+<xs:schema
+  xmlns:xs="http://www.w3.org/2001/XMLSchema"
+  xmlns:bar="http://bar.com"
+  elementFormDefault="qualified"
+>
+  <xs:import namespace="http://bar.com" />
+
   <xs:element name="Person" type="Person" />
 
   <xs:complexType name="Person">
     <xs:sequence>
-      <xs:element name="FirstName" type="xs:string" />
-      <xs:element name="LastName" type="xs:string" />
-      <xs:element name="Address" type="Address" />
+      <xs:element name="Name" type="xs:string" />
+      <xs:element ref="bar:Address" />
     </xs:sequence>
   </xs:complexType>
+</xs:schema>
+SCHEMA
+
+schema2 = <<~SCHEMA
+<xs:schema
+  xmlns:xs="http://www.w3.org/2001/XMLSchema"
+  xmlns:bar="http://bar.com"
+  targetNamespace="http://bar.com"
+  elementFormDefault="qualified"
+>
+  <xs:element name="Address" type="bar:Address" />
 
   <xs:complexType name="Address">
     <xs:sequence>
@@ -1372,42 +1586,55 @@ schema = <<~SCHEMA
 </xs:schema>
 SCHEMA
 
-Shale::Schema.from_xml([schema])
+Shale::Schema.from_xml(
+  [schema1, schema2],
+  namespace_mapping: {
+    nil => 'Api::Foo', # no namespace
+    'http://bar.com' => 'Api::Bar',
+  }
+)
 
 # =>
 #
 # {
-#   "address" => "
+#   "api/bar/address" => "
 #     require 'shale'
 #
-#     class Address < Shale::Mapper
-#       attribute :street, Shale::Type::String
-#       attribute :city, Shale::Type::String
+#     module Api
+#       module Bar
+#         class Address < Shale::Mapper
+#           attribute :street, Shale::Type::String
+#           attribute :city, Shale::Type::String
 #
-#       xml do
-#         root 'Address'
+#           xml do
+#             root 'Address'
+#             namespace 'http://bar.com', 'bar'
 #
-#         map_element 'Street', to: :street
-#         map_element 'City', to: :city
+#             map_element 'Street', to: :street
+#             map_element 'City', to: :city
+#           end
+#         end
 #       end
 #     end
 #   ",
-#   "person" => "
+#   "api/foo/person" => "
 #     require 'shale'
 #
-#     require_relative 'address'
+#     require_relative '../bar/address'
 #
-#     class Person < Shale::Mapper
-#       attribute :first_name, Shale::Type::String
-#       attribute :last_name, Shale::Type::String
-#       attribute :address, Address
+#     module Api
+#       module Foo
+#         class Person < Shale::Mapper
+#           attribute :name, Shale::Type::String
+#           attribute :address, Api::Bar::Address
 #
-#       xml do
-#         root 'Person'
+#           xml do
+#             root 'Person'
 #
-#         map_element 'FirstName', to: :first_name
-#         map_element 'LastName', to: :last_name
-#         map_element 'Address', to: :address
+#             map_element 'Name', to: :name
+#             map_element 'Address', to: :address, prefix: 'bar', namespace: 'http://bar.com'
+#           end
+#         end
 #       end
 #     end
 #   "
@@ -1417,7 +1644,7 @@ Shale::Schema.from_xml([schema])
 You can also use a command line tool to do it:
 
 ```
-$ shaleb -c -f xml -i schema.xml
+$ shaleb -c -f xml -i schema.xml -m http://bar.com=Api::Bar,=Api::Foo
 ```
 
 ## Contributing

data/exe/shaleb

@@ -36,8 +36,8 @@ ARGV << '-h' if ARGV.empty?
 OptionParser.new do |opts|
   opts.banner = <<~BANNER
     Usage: shaleb [options]
-    example generate schema from Shale model: shaleb -g -i data_model.rb -c MyRoot
-    example generate Shale model from schema: shaleb -c -i schema1.json,schema2.json -c MyRoot
+    example generate schema from Shale model: shaleb -g -i data_model.rb -r MyRoot
+    example generate Shale model from schema: shaleb -c -i schema1.json,schema2.json -r MyRoot
   BANNER
 
   opts.on('-g', '--generate', 'generate schema from Shale model')
@@ -47,6 +47,7 @@ OptionParser.new do |opts|
   opts.on('-r ROOT', '--root ROOT', 'Shale model class name')
   opts.on('-f FORMAT', '--format FORMAT', 'Schema format: JSON (default), XML')
   opts.on('-p', '--pretty', 'Pretty print generated schema')
+  opts.on('-m MAPPING', '--mapping', Array, 'Namespace mapping')
 
   opts.on('-v', '--version', 'Show version') do
     puts "shaleb version #{Shale::VERSION}"
@@ -71,11 +72,24 @@ if params[:compile]
     end
   end
 
+  if params[:mapping]
+    namespace_mapping = params[:mapping]
+      .to_h { |e| [*e.split('='), nil][0, 2] }
+      .transform_keys { |key| key.empty? ? nil : key }
+  end
+
   if params[:format] == 'xml'
     load_xml_parser
-    models = Shale::Schema.from_xml(schemas)
+    models = Shale::Schema.from_xml(
+      schemas,
+      namespace_mapping: namespace_mapping
+    )
   else
-    models = Shale::Schema.from_json(schemas, root_name: params[:root])
+    models = Shale::Schema.from_json(
+      schemas,
+      root_name: params[:root],
+      namespace_mapping: namespace_mapping
+    )
   end
 
   if params[:output]
@@ -84,6 +98,7 @@ if params[:compile]
 
     models.each do |name, model|
       output_path = File.join(dir, "#{name}.rb")
+      FileUtils.mkdir_p(File.dirname(output_path))
       File.write(output_path, model)
     end
   else

data/lib/shale/adapter/nokogiri.rb

@@ -61,7 +61,7 @@ module Shale
         result = doc.to_xml(save_with: save_with)
 
         unless pretty
-          result = result.sub(/\n/, '')
+          result = result.sub("\n", '')
         end
 
         result

data/lib/shale/adapter/ox.rb

@@ -22,7 +22,8 @@ module Shale
       #
       # @api private
       def self.load(xml)
-        Node.new(::Ox.parse(xml))
+        element = ::Ox.parse(xml)
+        Node.new(element.respond_to?(:root) ? element.root : element)
       rescue ::Ox::ParseError => e
         raise ParseError, "Document is invalid: #{e.message}"
       end

data/lib/shale/error.rb

@@ -38,6 +38,18 @@ module Shale
     Shale.xml_adapter = Shale::Adapter::Ox
   MSG
 
+  # Error message displayed when CSV adapter is not set
+  # @api private
+  CSV_ADAPTER_NOT_SET_MESSAGE = <<~MSG
+    CSV Adapter is not set.
+    To use Shale with CSV documents you have to install parser and set adapter.
+
+    # To use csv gem:
+    # Make sure csv is installed eg. execute: gem install csv
+    require 'shale/adapter/csv'
+    Shale.csv_adapter = Shale::Adapter::CSV
+  MSG
+
   # Error for assigning value to not existing attribute
   #
   # @api private
@@ -53,10 +65,16 @@ module Shale
     end
   end
 
+  # Shale base error class
+  #
+  # @api private
+  class ShaleError < StandardError
+  end
+
   # Error for trying to assign not callable object as an attribute's default
   #
   # @api private
-  class DefaultNotCallableError < StandardError
+  class DefaultNotCallableError < ShaleError
     # Initialize error object
     #
     # @param [String] record
@@ -71,36 +89,42 @@ module Shale
   # Error for passing incorrect model type
   #
   # @api private
-  class IncorrectModelError < StandardError
+  class IncorrectModelError < ShaleError
   end
 
   # Error for passing incorrect arguments to map functions
   #
   # @api private
-  class IncorrectMappingArgumentsError < StandardError
+  class IncorrectMappingArgumentsError < ShaleError
+  end
+
+  # Error for using incorrect type
+  #
+  # @api private
+  class NotAShaleMapperError < ShaleError
   end
 
-  # Error for passing incorrect arguments to schema generation function
+  # Raised when receiver attribute is not defined
   #
   # @api private
-  class NotAShaleMapperError < StandardError
+  class AttributeNotDefinedError < ShaleError
   end
 
   # Schema compilation error
   #
   # @api private
-  class SchemaError < StandardError
+  class SchemaError < ShaleError
   end
 
   # Parsing error
   #
   # @api private
-  class ParseError < StandardError
+  class ParseError < ShaleError
   end
 
   # Adapter error
   #
   # @api private
-  class AdapterError < StandardError
+  class AdapterError < ShaleError
   end
 end