shale 0.8.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 76c0022b3d2d42be143362c84c6697628781acdce7694fb9bc0f305d3f854964
-  data.tar.gz: 0b0b1b060eab81ebfae80afa3c31c809280b66ebd77e413c5908c4bbd09a081a
+  metadata.gz: 4e3b4d27ab7e8cc9b4873dfa60b52bdbafe50c2885085c4ec77ac6887351aab1
+  data.tar.gz: 6ce71a10ebf53633930b6d5a92ebb5f736aab44c47d6910a25238209192a4b92
 SHA512:
-  metadata.gz: 43bc9188d1f07c5c367a8ed94e2a3da6b878a7b7b86773d7898697abbd3937d4de6ae0471f9362c72e8a6be2cc5e5100a5206b2f2cc7155ff2ff8784953ef702
-  data.tar.gz: 0e610703cc73809a78007785962d9a322e515b88ab143a55adfeaa1a0da5961f14d8768a3dec4ad797070bbab165c238123702cbc671f522f86e2934968ff78b
+  metadata.gz: 23c3b182c72c0e1d5102eb667541caccfb56e836e636bd083027425a60a12ed39ffce1ab3540adc0659c6b1bcbcf5255d192bbdc2d39bfc10c762ee4bdaaaa1e
+  data.tar.gz: ef7481e6076cc8f426c1dd0fb002d984594cbf3d59743a31450e8989048ffdd6cdc8fc735e90b7529c1935d7251c6751e70f7c3734d5c3f7df4b8124cc4b8256

data/CHANGELOG.md

@@ -1,3 +1,27 @@
+## [1.0.0] - 2023-07-15
+
+### Added
+- Support for Ruby 3.2
+- Support for delegating fields to nested attributes
+- JSON and XML schema namespace mapping support
+- Allow to set render_nil defaults
+
+### Changed
+- Use `ShaleError` as a base class for exceptions
+- Use model instead of mapper names for schema types
+
+### Fixed
+- Fix compilation error for bundled JSON schemas
+- Fix type inference for XML schema when default namespace is used
+- Fix XML schema handling with a period in the element name
+
+## [0.9.0] - 2022-10-31
+
+### Added
+- Support for CSV
+- Allow to specify version and add encoding to XML declaration
+- Support for mapping/generating collections
+
 ## [0.8.0] - 2022-08-30
 
 ### Added

data/README.md

@@ -1,18 +1,18 @@
 # Shale
 
-Shale is a Ruby object mapper and serializer for JSON, YAML, TOML and XML.
-It allows you to parse JSON, YAML, TOML and XML data and convert it into Ruby data structures,
-as well as serialize data structures into JSON, YAML, TOML or XML.
+Shale is a Ruby object mapper and serializer for JSON, YAML, TOML, CSV and XML.
+It allows you to parse JSON, YAML, TOML, CSV and XML data and convert it into Ruby data structures,
+as well as serialize data structures into JSON, YAML, TOML, CSV or XML.
 
 Documentation with interactive examples is available at [Shale website](https://www.shalerb.org)
 
 ## Features
 
-* Convert JSON, YAML, TOML and XML to Ruby data model
-* Convert Ruby data model to JSON, YAML, TOML and XML
+* Convert JSON, YAML, TOML, CSV and XML to Ruby data model
+* Convert Ruby data model to JSON, YAML, TOML, CSV and XML
 * Generate JSON and XML Schema from Ruby models
 * Compile JSON and XML Schema into Ruby models
-* Out of the box support for JSON, YAML, Tomlib, toml-rb, Nokogiri, REXML and Ox parsers
+* Out of the box support for JSON, YAML, Tomlib, toml-rb, CSV, Nokogiri, REXML and Ox parsers
 * Support for custom adapters
 
 ## Installation
@@ -51,14 +51,19 @@ $ gem install shale
 * [Converting object to Hash](#converting-object-to-hash)
 * [Converting XML to object](#converting-xml-to-object)
 * [Converting object to XML](#converting-object-to-xml)
+* [Converting CSV to object](#converting-csv-to-object)
+* [Converting object to CSV](#converting-object-to-csv)
+* [Converting collections](#converting-collections)
 * [Mapping JSON keys to object attributes](#mapping-json-keys-to-object-attributes)
 * [Mapping YAML keys to object attributes](#mapping-yaml-keys-to-object-attributes)
 * [Mapping TOML keys to object attributes](#mapping-toml-keys-to-object-attributes)
+* [Mapping CSV columns to object attributes](#mapping-csv-columns-to-object-attributes)
 * [Mapping Hash keys to object attributes](#mapping-hash-keys-to-object-attributes)
 * [Mapping XML elements and attributes to object attributes](#mapping-xml-elements-and-attributes-to-object-attributes)
 * [Using XML namespaces](#using-xml-namespaces)
 * [Rendering nil values](#rendering-nil-values)
 * [Using methods to extract and generate data](#using-methods-to-extract-and-generate-data)
+* [Delegating fields to child attributes](#delegating-fields-to-child-attributes)
 * [Additional options](#additional-options)
 * [Using custom models](#using-custom-models)
 * [Supported types](#supported-types)
@@ -346,6 +351,70 @@ person.to_xml
 # </person>
 ```
 
+### Converting CSV to object
+
+CSV represents a flat data structure, so you can't map properties to complex types directly,
+but you can use methods to map properties to complex types
+(see [Using methods to extract and generate data](#using-methods-to-extract-and-generate-data)
+section).
+
+`.from_csv` method allways returns an array of records.
+
+```ruby
+people = Person.from_csv(<<~DATA)
+John,Doe,50,false
+DATA
+```
+
+### Converting object to CSV
+
+```ruby
+people[0].to_csv # or Person.to_csv(people) if you want to convert a collection
+
+# =>
+#
+# John,Doe,50,false
+```
+
+### Converting collections
+
+Shale allows converting collections for formats that support it (JSON, YAML and CSV).
+To convert Ruby array to JSON:
+
+```ruby
+person1 = Person.new(name: 'John Doe')
+person2 = Person.new(name: 'Joe Sixpack')
+
+Person.to_json([person1, person2], pretty: true)
+# or Person.to_yaml([person1, person2])
+# or Person.to_csv([person1, person2])
+
+# =>
+#
+# [
+#   { "name": "John Doe" },
+#   { "name": "Joe Sixpack" }
+# ]
+```
+
+To convert JSON array to Ruby:
+
+```ruby
+Person.from_json(<<~JSON)
+[
+  { "name": "John Doe" },
+  { "name": "Joe Sixpack" }
+]
+JSON
+
+# =>
+#
+# [
+#   #<Person:0x00000001033dbce8 @name="John Doe">,
+#   #<Person:0x00000001033db4c8 @name="Joe Sixpack">
+# ]
+```
+
 ### Mapping JSON keys to object attributes
 
 By default keys are named the same as attributes. To use custom keys use:
@@ -392,6 +461,24 @@ class Person < Shale::Mapper
 end
 ```
 
+### Mapping CSV columns to object attributes
+
+For CSV the order of mapping matters, the first argument in the `map` method is only
+used as a label in header row. So, in the example below the first column will be mapped
+to `:first_name` attribute and the second column to `:last_name`.
+
+```ruby
+class Person < Shale::Mapper
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+
+  csv do
+    map 'firstName', to: :first_name
+    map 'lastName', to: :last_name
+  end
+end
+```
+
 ### Mapping Hash keys to object attributes
 
 ```ruby
@@ -561,8 +648,9 @@ DATA
 
 ### Rendering nil values
 
-By default elements with `nil` value are not rendered. You can change this behavior
-by using `render_nil: true` on a mapping.
+For JSON, YAML, TOML and XML by default, elements with `nil` value are not rendered.
+You can change this behavior by using `render_nil: true` on a mapping.
+For CSV the default is to render `nil` elements.
 
 ```ruby
 class Person < Shale::Mapper
@@ -605,6 +693,49 @@ puts person.to_xml(pretty: true)
 # </person>
 ```
 
+If you want to change how nil values are rendered for all mappings you can use `render_nil` method:
+
+```ruby
+class Base < Shale::Mapper
+  json do
+    # change render_nil default for all JSON mappings inheriting from Base class
+    render_nil true
+  end
+end
+
+class Person < Base
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+  attribute :age, Shale::Type::Integer
+
+  json do
+    # override default from Base class
+    render_nil false
+
+    map 'first_name', to: :first_name
+    map 'last_name', to: :last_name
+    map 'age', to: :age, render_nil: true # override default
+  end
+end
+```
+
+:warning: The default affects only the mappings declared after setting the default value e.g.
+
+```ruby
+class Person < Base
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+
+  json do
+    render_nil false
+    map 'first_name', to: :first_name # render_nil will be false for this mapping
+
+    render_nil true
+    map 'last_name', to: :last_name # render_nil will be true for this mapping
+  end
+end
+```
+
 ### Using methods to extract and generate data
 
 If you need full controll over extracting and generating data from/to document,
@@ -778,6 +909,42 @@ DATA
 # => #<Person:0x00007f9bc3086d60 @name="John Doe">
 ```
 
+### Delegating fields to child attributes
+
+To delegate fields to child complex types you can use `receiver: :child` declaration:
+
+```ruby
+class Address < Shale::Mapper
+  attribute :city, Shale::Type::String
+  attribute :street, Shale::Type::String
+end
+
+class Person < Shale::Mapper
+  attribute :name, Shale::Type::String
+  attribute :address, Address
+
+  json do
+    map 'name', to: :name
+    map 'city', to: :city, receiver: :address
+    map 'street', to: :street, receiver: :address
+  end
+end
+
+person = Person.from_json(<<~DATA)
+{
+  "name": "John Doe",
+  "city": "London",
+  "street": "Oxford Street"
+}
+DATA
+
+# =>
+#
+# #<Person:0x00007f9bc3086d60
+#  @name="John Doe",
+#  @address=#<Address:0x0000000102cbd218 @city="London", @street="Oxford Street">>
+```
+
 ### Additional options
 
 You can control which attributes to render and parse by
@@ -853,19 +1020,52 @@ person.to_json(pretty: true)
 # }
 ```
 
-You can also add an XML declaration by passing `declaration: true` to `#to_xml`
+You can also add an XML declaration by passing `declaration: true` and `encoding: true`
+or if you want to spcify version: `declaration: '1.1'` and `encoding: 'ASCII'` to `#to_xml`
 
 ```ruby
-person.to_xml(pretty: true, declaration: true)
+person.to_xml(pretty: true, declaration: true, encoding: true)
 
 # =>
 #
-# <?xml version="1.0"?>
+# <?xml version="1.0" encoding="UTF-8"?>
 # <Person>
 #   <Address city="London"/>
 # </Person>
 ```
 
+For CSV you can pass `headers: true` to indicate that the first row contains column
+names and shouldn't be included in the returned collection. It also accepts all the options that
+[CSV parser](https://ruby-doc.org/stdlib-3.1.2/libdoc/csv/rdoc/CSV.html#class-CSV-label-Options) accepts.
+
+```ruby
+class Person
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+end
+
+people = Person.from_csv(<<~DATA, headers: true, col_sep: '|')
+  first_name|last_name
+  John|Doe
+  James|Sixpack
+DATA
+
+# =>
+#
+# [
+#   #<Person:0x0000000113d7a488 @first_name="John", @last_name="Doe">,
+#   #<Person:0x0000000113d7a488 @first_name="James", @last_name="Sixpack">
+# ]
+
+Person.to_csv(people, headers: true, col_sep: '|')
+
+# =>
+#
+# first_name|last_name
+# John|Doe
+# James|Sixpack
+```
+
 ### Using custom models
 
 By default Shale combines mapper and model into one class. If you want to use your own classes
@@ -955,10 +1155,10 @@ end
 ### Adapters
 
 Shale uses adapters for parsing and generating documents.
-By default Ruby's standard JSON, YAML parsers are used for handling JSON and YAML documents.
+By default Ruby's standard JSON, YAML, CSV parsers are used for handling JSON YAML, CSV documents.
 
-You can change it by providing your own adapter. For JSON, YAML and TOML, adapter must implement
-`.load` and `.dump` class methods.
+You can change it by providing your own adapter. For JSON, YAML, TOML and CSV adapter must
+implement `.load` and `.dump` class methods.
 
 ```ruby
 require 'shale'
@@ -1092,7 +1292,9 @@ Shale::Schema::JSONGenerator.register_json_type(MyEmailType, MyEmailJSONType)
 
 :warning: Only **[Draft 2020-12](https://json-schema.org/draft/2020-12/schema)** JSON Schema is supported
 
-To generate Shale data model from JSON Schema use:
+To generate Shale data model from JSON Schema use `Shale::Schema.from_json`.
+You can pass `root_name: 'Foobar'` to change the name of the root type and
+`namespace_mapping: {}` to map schemas to Ruby modules:
 
 ```ruby
 require 'shale/schema'
@@ -1103,7 +1305,11 @@ schema = <<~SCHEMA
   "properties": {
     "firstName": { "type": "string" },
     "lastName": { "type": "string" },
-    "address": {
+    "address": { "$ref": "http://bar.com" }
+  },
+  "$defs": {
+    "Address": {
+      "$id": "http://bar.com",
       "type": "object",
       "properties": {
         "street": { "type": "string" },
@@ -1114,38 +1320,53 @@ schema = <<~SCHEMA
 }
 SCHEMA
 
-Shale::Schema.from_json([schema], root_name: 'Person')
+Shale::Schema.from_json(
+  [schema],
+  root_name: 'Person',
+  namespace_mapping: {
+    nil => 'Api::Foo', # default schema (without ID)
+    'http://bar.com' => 'Api::Bar',
+  }
+)
 
 # =>
 #
 # {
-#   "address" => "
+#   "api/bar/address" => "
 #     require 'shale'
 #
-#     class Address < Shale::Mapper
-#       attribute :street, Shale::Type::String
-#       attribute :city, Shale::Type::String
+#     module Api
+#       module Bar
+#         class Address < Shale::Mapper
+#           attribute :street, Shale::Type::String
+#           attribute :city, Shale::Type::String
 #
-#       json do
-#         map 'street', to: :street
-#         map 'city', to: :city
+#           json do
+#             map 'street', to: :street
+#             map 'city', to: :city
+#           end
+#         end
 #       end
 #     end
 #   ",
-#   "person" => "
+#   "api/foo/person" => "
 #     require 'shale'
 #
-#     require_relative 'address'
+#     require_relative '../bar/address'
 #
-#     class Person < Shale::Mapper
-#       attribute :first_name, Shale::Type::String
-#       attribute :last_name, Shale::Type::String
-#       attribute :address, Address
+#     module Api
+#       module Foo
+#         class Person < Shale::Mapper
+#           attribute :first_name, Shale::Type::String
+#           attribute :last_name, Shale::Type::String
+#           attribute :address, Api::Bar::Address
 #
-#       json do
-#         map 'firstName', to: :first_name
-#         map 'lastName', to: :last_name
-#         map 'address', to: :address
+#           json do
+#             map 'firstName', to: :first_name
+#             map 'lastName', to: :last_name
+#             map 'address', to: :address
+#           end
+#         end
 #       end
 #     end
 #   "
@@ -1155,7 +1376,7 @@ Shale::Schema.from_json([schema], root_name: 'Person')
 You can also use a command line tool to do it:
 
 ```
-$ shaleb -c -i schema.json -r Person
+$ shaleb -c -i schema.json -r Person -m http://bar.com=Api::Bar,=Api::Foo
 ```
 
 ### Generating XML Schema
@@ -1226,22 +1447,39 @@ Shale::Schema::XMLGenerator.register_xml_type(MyEmailType, 'myEmailXMLType')
 
 ### Compiling XML Schema into Shale model
 
-To generate Shale data model from XML Schema use:
+To generate Shale data model from XML Schema use `Shale::Schema.from_xml`.
+You can pass `namespace_mapping: {}` to map XML namespaces to Ruby modules:
 
 ```ruby
 require 'shale/schema'
 
-schema = <<~SCHEMA
-<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
+schema1 = <<~SCHEMA
+<xs:schema
+  xmlns:xs="http://www.w3.org/2001/XMLSchema"
+  xmlns:bar="http://bar.com"
+  elementFormDefault="qualified"
+>
+  <xs:import namespace="http://bar.com" />
+
   <xs:element name="Person" type="Person" />
 
   <xs:complexType name="Person">
     <xs:sequence>
-      <xs:element name="FirstName" type="xs:string" />
-      <xs:element name="LastName" type="xs:string" />
-      <xs:element name="Address" type="Address" />
+      <xs:element name="Name" type="xs:string" />
+      <xs:element ref="bar:Address" />
     </xs:sequence>
   </xs:complexType>
+</xs:schema>
+SCHEMA
+
+schema2 = <<~SCHEMA
+<xs:schema
+  xmlns:xs="http://www.w3.org/2001/XMLSchema"
+  xmlns:bar="http://bar.com"
+  targetNamespace="http://bar.com"
+  elementFormDefault="qualified"
+>
+  <xs:element name="Address" type="bar:Address" />
 
   <xs:complexType name="Address">
     <xs:sequence>
@@ -1252,42 +1490,55 @@ schema = <<~SCHEMA
 </xs:schema>
 SCHEMA
 
-Shale::Schema.from_xml([schema])
+Shale::Schema.from_xml(
+  [schema1, schema2],
+  namespace_mapping: {
+    nil => 'Api::Foo', # no namespace
+    'http://bar.com' => 'Api::Bar',
+  }
+)
 
 # =>
 #
 # {
-#   "address" => "
+#   "api/bar/address" => "
 #     require 'shale'
 #
-#     class Address < Shale::Mapper
-#       attribute :street, Shale::Type::String
-#       attribute :city, Shale::Type::String
+#     module Api
+#       module Bar
+#         class Address < Shale::Mapper
+#           attribute :street, Shale::Type::String
+#           attribute :city, Shale::Type::String
 #
-#       xml do
-#         root 'Address'
+#           xml do
+#             root 'Address'
+#             namespace 'http://bar.com', 'bar'
 #
-#         map_element 'Street', to: :street
-#         map_element 'City', to: :city
+#             map_element 'Street', to: :street
+#             map_element 'City', to: :city
+#           end
+#         end
 #       end
 #     end
 #   ",
-#   "person" => "
+#   "api/foo/person" => "
 #     require 'shale'
 #
-#     require_relative 'address'
+#     require_relative '../bar/address'
 #
-#     class Person < Shale::Mapper
-#       attribute :first_name, Shale::Type::String
-#       attribute :last_name, Shale::Type::String
-#       attribute :address, Address
+#     module Api
+#       module Foo
+#         class Person < Shale::Mapper
+#           attribute :name, Shale::Type::String
+#           attribute :address, Api::Bar::Address
 #
-#       xml do
-#         root 'Person'
+#           xml do
+#             root 'Person'
 #
-#         map_element 'FirstName', to: :first_name
-#         map_element 'LastName', to: :last_name
-#         map_element 'Address', to: :address
+#             map_element 'Name', to: :name
+#             map_element 'Address', to: :address, prefix: 'bar', namespace: 'http://bar.com'
+#           end
+#         end
 #       end
 #     end
 #   "
@@ -1297,7 +1548,7 @@ Shale::Schema.from_xml([schema])
 You can also use a command line tool to do it:
 
 ```
-$ shaleb -c -f xml -i schema.xml
+$ shaleb -c -f xml -i schema.xml -m http://bar.com=Api::Bar,=Api::Foo
 ```
 
 ## Contributing

data/exe/shaleb

@@ -36,8 +36,8 @@ ARGV << '-h' if ARGV.empty?
 OptionParser.new do |opts|
   opts.banner = <<~BANNER
     Usage: shaleb [options]
-    example generate schema from Shale model: shaleb -g -i data_model.rb -c MyRoot
-    example generate Shale model from schema: shaleb -c -i schema1.json,schema2.json -c MyRoot
+    example generate schema from Shale model: shaleb -g -i data_model.rb -r MyRoot
+    example generate Shale model from schema: shaleb -c -i schema1.json,schema2.json -r MyRoot
   BANNER
 
   opts.on('-g', '--generate', 'generate schema from Shale model')
@@ -47,6 +47,7 @@ OptionParser.new do |opts|
   opts.on('-r ROOT', '--root ROOT', 'Shale model class name')
   opts.on('-f FORMAT', '--format FORMAT', 'Schema format: JSON (default), XML')
   opts.on('-p', '--pretty', 'Pretty print generated schema')
+  opts.on('-m MAPPING', '--mapping', Array, 'Namespace mapping')
 
   opts.on('-v', '--version', 'Show version') do
     puts "shaleb version #{Shale::VERSION}"
@@ -71,11 +72,24 @@ if params[:compile]
     end
   end
 
+  if params[:mapping]
+    namespace_mapping = params[:mapping]
+      .to_h { |e| [*e.split('='), nil][0, 2] }
+      .transform_keys { |key| key.empty? ? nil : key }
+  end
+
   if params[:format] == 'xml'
     load_xml_parser
-    models = Shale::Schema.from_xml(schemas)
+    models = Shale::Schema.from_xml(
+      schemas,
+      namespace_mapping: namespace_mapping
+    )
   else
-    models = Shale::Schema.from_json(schemas, root_name: params[:root])
+    models = Shale::Schema.from_json(
+      schemas,
+      root_name: params[:root],
+      namespace_mapping: namespace_mapping
+    )
   end
 
   if params[:output]
@@ -84,6 +98,7 @@ if params[:compile]
 
     models.each do |name, model|
       output_path = File.join(dir, "#{name}.rb")
+      FileUtils.mkdir_p(File.dirname(output_path))
       File.write(output_path, model)
     end
   else

data/lib/shale/adapter/csv.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require 'csv'
+
+module Shale
+  module Adapter
+    # CSV adapter
+    #
+    # @api public
+    class CSV
+      # Parse CSV into Array<Hash<String, any>>
+      #
+      # @param [String] csv CSV document
+      # @param [Array<String>] headers
+      # @param [Hash] options
+      #
+      # @return [Array<Hash<String, any>>]
+      #
+      # @api private
+      def self.load(csv, headers:, **options)
+        ::CSV.parse(csv, headers: headers, **options).map(&:to_h)
+      end
+
+      # Serialize Array<Hash<String, any>> into CSV
+      #
+      # @param [Array<Hash<String, any>>] obj Array<Hash<String, any>> object
+      # @param [Array<String>] headers
+      # @param [Hash] options
+      #
+      # @return [String]
+      #
+      # @api private
+      def self.dump(obj, headers:, **options)
+        ::CSV.generate(**options) do |csv|
+          obj.each do |row|
+            values = []
+
+            headers.each do |header|
+              values << row[header] if row.key?(header)
+            end
+
+            csv << values
+          end
+        end
+      end
+    end
+  end
+end