shale 0.7.1 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 468fc4f3fa1ccd796958748eea653aaa9aec0dbf223c2e5486ddba00350d79fc
-  data.tar.gz: 4b78746e97bc6fd080b8a6566be2534c3a1c05017acacabd3935f7e2ea81b097
+  metadata.gz: 3f880bc8d984c45e7e8a05bd155c1014df237e037263c7bf68a7ae82c0e49ea4
+  data.tar.gz: ba7a2f12bed87e048dd53d43900f1c8203d90f5f89a43b3824475ddda39eb07b
 SHA512:
-  metadata.gz: 776a30c22e95f8c85e150b64ec7df09e89aeb4e86fb2f6f0fcec67a438b13b73926c3d5419b8194f457a2f09fc8eb64bf491e8b1ea5c5d6957e1e1b5cfab9549
-  data.tar.gz: 0d236bde4f3b5aaf9b3ea5a5b8380b3861b24f5d0b310430329c68d010c368890dedbd7f6fad8649b71cd443362c8c6272ebb9c7a234998fa546ecb99e728f44
+  metadata.gz: 54023d4f621dae53537f73e0ad79d3736d5d52e605c1a7da2c4bdb06588b348feaae9066ce00b6b3aedc9e84ea23a425884eabf987717ffda6d3c22eccc6325f
+  data.tar.gz: e82cad292dab557891cad4f210bfb4a97d2096a6d2151a5a8aef020358accb5986e527e32a141a9a2e3a01d594e9a3adb42c0f7e8d17464517731136f63289a2

data/CHANGELOG.md

@@ -1,9 +1,26 @@
-## [0.7.1] - [2022-08-12]
+## [0.9.0] - 2022-10-31
+
+### Added
+- Support for CSV
+- Allow to specify version and add encoding to XML declaration
+- Support for mapping/generating collections
+
+## [0.8.0] - 2022-08-30
+
+### Added
+- Allow to group mappings using `group` block
+- Bring back Ruby 2.6 support
+
+### Changed
+- Use anonymous module for attributes definition.
+  It allows to override accessors and `super` works as expected.
+
+## [0.7.1] - 2022-08-12
 
 ### Fixed
 - Fix broken handling of Date and Time types
 
-## [0.7.0] - [2022-08-09]
+## [0.7.0] - 2022-08-09
 
 ### Added
 - `only: []` and `except: []` options that allow to controll what attributes are rendered/parsed

data/README.md

@@ -1,23 +1,23 @@
 # Shale
 
-Shale is a Ruby object mapper and serializer for JSON, YAML, TOML and XML.
-It allows you to parse JSON, YAML, TOML and XML data and convert it into Ruby data structures,
-as well as serialize data structures into JSON, YAML, TOML or XML.
+Shale is a Ruby object mapper and serializer for JSON, YAML, TOML, CSV and XML.
+It allows you to parse JSON, YAML, TOML, CSV and XML data and convert it into Ruby data structures,
+as well as serialize data structures into JSON, YAML, TOML, CSV or XML.
 
 Documentation with interactive examples is available at [Shale website](https://www.shalerb.org)
 
 ## Features
 
-* Convert JSON, YAML, TOML and XML to Ruby data model
-* Convert Ruby data model to JSON, YAML, TOML and XML
+* Convert JSON, YAML, TOML, CSV and XML to Ruby data model
+* Convert Ruby data model to JSON, YAML, TOML, CSV and XML
 * Generate JSON and XML Schema from Ruby models
 * Compile JSON and XML Schema into Ruby models
-* Out of the box support for JSON, YAML, Tomlib, toml-rb, Nokogiri, REXML and Ox parsers
+* Out of the box support for JSON, YAML, Tomlib, toml-rb, CSV, Nokogiri, REXML and Ox parsers
 * Support for custom adapters
 
 ## Installation
 
-Shale supports Ruby (MRI) 2.7+
+Shale supports Ruby (MRI) 2.6+
 
 Add this line to your application's Gemfile:
 
@@ -51,9 +51,13 @@ $ gem install shale
 * [Converting object to Hash](#converting-object-to-hash)
 * [Converting XML to object](#converting-xml-to-object)
 * [Converting object to XML](#converting-object-to-xml)
+* [Converting CSV to object](#converting-csv-to-object)
+* [Converting object to CSV](#converting-object-to-csv)
+* [Converting collections](#converting-collections)
 * [Mapping JSON keys to object attributes](#mapping-json-keys-to-object-attributes)
 * [Mapping YAML keys to object attributes](#mapping-yaml-keys-to-object-attributes)
 * [Mapping TOML keys to object attributes](#mapping-toml-keys-to-object-attributes)
+* [Mapping CSV columns to object attributes](#mapping-csv-columns-to-object-attributes)
 * [Mapping Hash keys to object attributes](#mapping-hash-keys-to-object-attributes)
 * [Mapping XML elements and attributes to object attributes](#mapping-xml-elements-and-attributes-to-object-attributes)
 * [Using XML namespaces](#using-xml-namespaces)
@@ -346,6 +350,70 @@ person.to_xml
 # </person>
 ```
 
+### Converting CSV to object
+
+CSV represents a flat data structure, so you can't map properties to complex types directly,
+but you can use methods to map properties to complex types
+(see [Using methods to extract and generate data](#using-methods-to-extract-and-generate-data)
+section).
+
+`.from_csv` method allways returns an array of records.
+
+```ruby
+people = Person.from_csv(<<~DATA)
+John,Doe,50,false
+DATA
+```
+
+### Converting object to CSV
+
+```ruby
+people[0].to_csv # or Person.to_csv(people) if you want to convert a collection
+
+# =>
+#
+# John,Doe,50,false
+```
+
+### Converting collections
+
+Shale allows converting collections for formats that support it (JSON, YAML and CSV).
+To convert Ruby array to JSON:
+
+```ruby
+person1 = Person.new(name: 'John Doe')
+person2 = Person.new(name: 'Joe Sixpack')
+
+Person.to_json([person1, person2], pretty: true)
+# or Person.to_yaml([person1, person2])
+# or Person.to_csv([person1, person2])
+
+# =>
+#
+# [
+#   { "name": "John Doe" },
+#   { "name": "Joe Sixpack" }
+# ]
+```
+
+To convert JSON array to Ruby:
+
+```ruby
+Person.from_json(<<~JSON)
+[
+  { "name": "John Doe" },
+  { "name": "Joe Sixpack" }
+]
+JSON
+
+# =>
+#
+# [
+#   #<Person:0x00000001033dbce8 @name="John Doe">,
+#   #<Person:0x00000001033db4c8 @name="Joe Sixpack">
+# ]
+```
+
 ### Mapping JSON keys to object attributes
 
 By default keys are named the same as attributes. To use custom keys use:
@@ -392,6 +460,24 @@ class Person < Shale::Mapper
 end
 ```
 
+### Mapping CSV columns to object attributes
+
+For CSV the order of mapping matters, the first argument in the `map` method is only
+used as a label in header row. So, in the example below the first column will be mapped
+to `:first_name` attribute and the second column to `:last_name`.
+
+```ruby
+class Person < Shale::Mapper
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+
+  csv do
+    map 'firstName', to: :first_name
+    map 'lastName', to: :last_name
+  end
+end
+```
+
 ### Mapping Hash keys to object attributes
 
 ```ruby
@@ -561,8 +647,9 @@ DATA
 
 ### Rendering nil values
 
-By default elements with `nil` value are not rendered. You can change this behavior
-by using `render_nil: true` on a mapping.
+For JSON, YAML, TOML and XML by default, elements with `nil` value are not rendered.
+You can change this behavior by using `render_nil: true` on a mapping.
+For CSV the default is to render `nil` elements.
 
 ```ruby
 class Person < Shale::Mapper
@@ -729,6 +816,55 @@ end
 Person.new(password: 'secret').to_json(context: current_user)
 ```
 
+If you want to work on multiple elements at a time you can group them using `group` block:
+
+```ruby
+class Person < Shale::Mapper
+  attribute :name, Shale::Type::String
+
+  json do
+    group from: :name_from_json, to: :name_to_json do
+      map 'first_name'
+      map 'last_name'
+    end
+  end
+
+  xml do
+    group from: :name_from_xml, to: :name_to_xml do
+      map_content
+      map_element 'first_name'
+      map_attribute 'last_name'
+    end
+  end
+
+  def name_from_json(model, value)
+    model.name = "#{value['first_name']} #{value['last_name']}"
+  end
+
+  def name_to_json(model, doc)
+    doc['first_name'] = model.name.split(' ')[0]
+    doc['last_name'] = model.name.split(' ')[1]
+  end
+
+  def name_from_xml(model, value)
+    # value => { content: ..., attributes: {}, elements: {} }
+  end
+
+  def name_to_xml(model, element, doc)
+    # ...
+  end
+end
+
+Person.from_json(<<~DATA)
+{
+  "first_name": "John",
+  "last_name": "Doe"
+}
+DATA
+
+# => #<Person:0x00007f9bc3086d60 @name="John Doe">
+```
+
 ### Additional options
 
 You can control which attributes to render and parse by
@@ -804,19 +940,52 @@ person.to_json(pretty: true)
 # }
 ```
 
-You can also add an XML declaration by passing `declaration: true` to `#to_xml`
+You can also add an XML declaration by passing `declaration: true` and `encoding: true`
+or if you want to spcify version: `declaration: '1.1'` and `encoding: 'ASCII'` to `#to_xml`
 
 ```ruby
-person.to_xml(pretty: true, declaration: true)
+person.to_xml(pretty: true, declaration: true, encoding: true)
 
 # =>
 #
-# <?xml version="1.0"?>
+# <?xml version="1.0" encoding="UTF-8"?>
 # <Person>
 #   <Address city="London"/>
 # </Person>
 ```
 
+For CSV you can pass `headers: true` to indicate that the first row contains column
+names and shouldn't be included in the returned collection. It also accepts all the options that
+[CSV parser](https://ruby-doc.org/stdlib-3.1.2/libdoc/csv/rdoc/CSV.html#class-CSV-label-Options) accepts.
+
+```ruby
+class Person
+  attribute :first_name, Shale::Type::String
+  attribute :last_name, Shale::Type::String
+end
+
+people = Person.from_csv(<<~DATA, headers: true, col_sep: '|')
+  first_name|last_name
+  John|Doe
+  James|Sixpack
+DATA
+
+# =>
+#
+# [
+#   #<Person:0x0000000113d7a488 @first_name="John", @last_name="Doe">,
+#   #<Person:0x0000000113d7a488 @first_name="James", @last_name="Sixpack">
+# ]
+
+Person.to_csv(people, headers: true, col_sep: '|')
+
+# =>
+#
+# first_name|last_name
+# John|Doe
+# James|Sixpack
+```
+
 ### Using custom models
 
 By default Shale combines mapper and model into one class. If you want to use your own classes
@@ -906,10 +1075,10 @@ end
 ### Adapters
 
 Shale uses adapters for parsing and generating documents.
-By default Ruby's standard JSON, YAML parsers are used for handling JSON and YAML documents.
+By default Ruby's standard JSON, YAML, CSV parsers are used for handling JSON YAML, CSV documents.
 
-You can change it by providing your own adapter. For JSON, YAML and TOML, adapter must implement
-`.load` and `.dump` class methods.
+You can change it by providing your own adapter. For JSON, YAML, TOML and CSV adapter must
+implement `.load` and `.dump` class methods.
 
 ```ruby
 require 'shale'

data/lib/shale/adapter/csv.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require 'csv'
+
+module Shale
+  module Adapter
+    # CSV adapter
+    #
+    # @api public
+    class CSV
+      # Parse CSV into Array<Hash<String, any>>
+      #
+      # @param [String] csv CSV document
+      # @param [Array<String>] headers
+      # @param [Hash] options
+      #
+      # @return [Array<Hash<String, any>>]
+      #
+      # @api private
+      def self.load(csv, headers:, **options)
+        ::CSV.parse(csv, headers: headers, **options).map(&:to_h)
+      end
+
+      # Serialize Array<Hash<String, any>> into CSV
+      #
+      # @param [Array<Hash<String, any>>] obj Array<Hash<String, any>> object
+      # @param [Array<String>] headers
+      # @param [Hash] options
+      #
+      # @return [String]
+      #
+      # @api private
+      def self.dump(obj, headers:, **options)
+        ::CSV.generate(**options) do |csv|
+          obj.each do |row|
+            values = []
+
+            headers.each do |header|
+              values << row[header] if row.key?(header)
+            end
+
+            csv << values
+          end
+        end
+      end
+    end
+  end
+end

data/lib/shale/adapter/nokogiri/document.rb

@@ -9,9 +9,14 @@ module Shale
       class Document
         # Initialize object
         #
+        # @param [String, nil] version
+        #
         # @api private
-        def initialize
-          @doc = ::Nokogiri::XML::Document.new
+        def initialize(version = nil)
+          ver = nil
+          ver = version if version.is_a?(String)
+
+          @doc = ::Nokogiri::XML::Document.new(ver)
           @namespaces = {}
         end
 

data/lib/shale/adapter/nokogiri.rb

@@ -37,12 +37,13 @@ module Shale
       #
       # @param [::Nokogiri::XML::Document] doc Nokogiri document
       # @param [true, false] pretty
-      # @param [true, false] declaration
+      # @param [true, false, String] declaration
+      # @param [true, false, String] encoding
       #
       # @return [String]
       #
       # @api private
-      def self.dump(doc, pretty: false, declaration: false)
+      def self.dump(doc, pretty: false, declaration: false, encoding: false)
         save_with = ::Nokogiri::XML::Node::SaveOptions::AS_XML
 
         if pretty
@@ -53,6 +54,10 @@ module Shale
           save_with |= ::Nokogiri::XML::Node::SaveOptions::NO_DECLARATION
         end
 
+        if encoding
+          doc.encoding = encoding == true ? 'UTF-8' : encoding
+        end
+
         result = doc.to_xml(save_with: save_with)
 
         unless pretty
@@ -64,9 +69,11 @@ module Shale
 
       # Create Shale::Adapter::Nokogiri::Document instance
       #
+      # @param [true, false, String, nil] declaration
+      #
       # @api private
-      def self.create_document
-        Document.new
+      def self.create_document(version = nil)
+        Document.new(version)
       end
     end
   end

data/lib/shale/adapter/ox.rb

@@ -31,12 +31,13 @@ module Shale
       #
       # @param [::Ox::Document, ::Ox::Element] doc Ox document
       # @param [true, false] pretty
-      # @param [true, false] declaration
+      # @param [true, false, String] declaration
+      # @param [true, false, String] encoding
       #
       # @return [String]
       #
       # @api private
-      def self.dump(doc, pretty: false, declaration: false)
+      def self.dump(doc, pretty: false, declaration: false, encoding: false)
         opts = { indent: -1, with_xml: false }
 
         if pretty
@@ -44,7 +45,12 @@ module Shale
         end
 
         if declaration
-          doc[:version] = '1.0'
+          doc[:version] = declaration == true ? '1.0' : declaration
+
+          if encoding
+            doc[:encoding] = encoding == true ? 'UTF-8' : encoding
+          end
+
           opts[:with_xml] = true
         end
 
@@ -54,7 +60,7 @@ module Shale
       # Create Shale::Adapter::Ox::Document instance
       #
       # @api private
-      def self.create_document
+      def self.create_document(_version = nil)
         Document.new
       end
     end

data/lib/shale/adapter/rexml.rb

@@ -32,14 +32,28 @@ module Shale
       #
       # @param [::REXML::Document] doc REXML document
       # @param [true, false] pretty
-      # @param [true, false] declaration
+      # @param [true, false, String] declaration
+      # @param [true, false, String] encoding
       #
       # @return [String]
       #
       # @api private
-      def self.dump(doc, pretty: false, declaration: false)
+      def self.dump(doc, pretty: false, declaration: false, encoding: false)
         if declaration
-          doc.add(::REXML::XMLDecl.new)
+          ver = nil
+          enc = nil
+
+          if declaration.is_a?(String)
+            ver = declaration
+          end
+
+          if encoding == true
+            enc = 'UTF-8'
+          else
+            enc = encoding || nil
+          end
+
+          doc.add(::REXML::XMLDecl.new(ver, enc))
         end
 
         io = StringIO.new
@@ -58,7 +72,7 @@ module Shale
       # Create Shale::Adapter::REXML::Document instance
       #
       # @api private
-      def self.create_document
+      def self.create_document(_version = nil)
         Document.new
       end
     end

data/lib/shale/mapper.rb

@@ -49,6 +49,7 @@ module Shale
     @json_mapping = Mapping::Dict.new
     @yaml_mapping = Mapping::Dict.new
     @toml_mapping = Mapping::Dict.new
+    @csv_mapping = Mapping::Dict.new(render_nil_default: true)
     @xml_mapping = Mapping::Xml.new
 
     class << self
@@ -87,6 +88,13 @@ module Shale
       # @api public
       attr_reader :toml_mapping
 
+      # Return CSV mapping object
+      #
+      # @return [Shale::Mapping::Dict]
+      #
+      # @api public
+      attr_reader :csv_mapping
+
       # Return XML mapping object
       #
       # @return [Shale::Mapping::XML]
@@ -98,6 +106,10 @@ module Shale
       def inherited(subclass)
         super
 
+        attributes_module = Module.new
+        subclass.instance_variable_set('@attributes_module', attributes_module)
+        subclass.include(attributes_module)
+
         subclass.instance_variable_set('@model', subclass)
         subclass.instance_variable_set('@attributes', @attributes.dup)
 
@@ -105,12 +117,14 @@ module Shale
         subclass.instance_variable_set('@__json_mapping_init', @json_mapping.dup)
         subclass.instance_variable_set('@__yaml_mapping_init', @yaml_mapping.dup)
         subclass.instance_variable_set('@__toml_mapping_init', @toml_mapping.dup)
+        subclass.instance_variable_set('@__csv_mapping_init', @csv_mapping.dup)
         subclass.instance_variable_set('@__xml_mapping_init', @xml_mapping.dup)
 
         subclass.instance_variable_set('@hash_mapping', @hash_mapping.dup)
         subclass.instance_variable_set('@json_mapping', @json_mapping.dup)
         subclass.instance_variable_set('@yaml_mapping', @yaml_mapping.dup)
         subclass.instance_variable_set('@toml_mapping', @toml_mapping.dup)
+        subclass.instance_variable_set('@csv_mapping', @csv_mapping.dup)
 
         xml_mapping = @xml_mapping.dup
         xml_mapping.root(Utils.underscore(subclass.name || ''))
@@ -169,9 +183,10 @@ module Shale
         @json_mapping.map(name.to_s, to: name) unless @json_mapping.finalized?
         @yaml_mapping.map(name.to_s, to: name) unless @yaml_mapping.finalized?
         @toml_mapping.map(name.to_s, to: name) unless @toml_mapping.finalized?
+        @csv_mapping.map(name.to_s, to: name) unless @csv_mapping.finalized?
         @xml_mapping.map_element(name.to_s, to: name) unless @xml_mapping.finalized?
 
-        class_eval(<<-RUBY, __FILE__, __LINE__ + 1)
+        @attributes_module.class_eval(<<-RUBY, __FILE__, __LINE__ + 1)
           attr_reader :#{name}
 
           def #{name}=(val)
@@ -276,6 +291,30 @@ module Shale
         @toml_mapping.instance_eval(&block)
       end
 
+      # Define CSV mapping
+      #
+      # @param [Proc] block
+      #
+      # @example
+      #   calss Person < Shale::Mapper
+      #     attribute :first_name, Shale::Type::String
+      #     attribute :last_name, Shale::Type::String
+      #     attribute :age, Shale::Type::Integer
+      #
+      #     csv do
+      #       map 'first_name', to: :first_name
+      #       map 'last_name', to: :last_name
+      #       map 'age', to: :age
+      #     end
+      #   end
+      #
+      # @api public
+      def csv(&block)
+        @csv_mapping = @__csv_mapping_init.dup
+        @csv_mapping.finalize!
+        @csv_mapping.instance_eval(&block)
+      end
+
       # Define XML mapping
       #
       # @param [Proc] block

data/lib/shale/mapping/descriptor/dict.rb

@@ -35,17 +35,26 @@ module Shale
         # @api private
         attr_reader :method_to
 
+        # Return group name
+        #
+        # @return [String]
+        #
+        # @api private
+        attr_reader :group
+
         # Initialize instance
         #
         # @param [String] name
         # @param [Symbol, nil] attribute
         # @param [Hash, nil] methods
+        # @param [String, nil] group
         # @param [true, false] render_nil
         #
         # @api private
-        def initialize(name:, attribute:, methods:, render_nil:)
+        def initialize(name:, attribute:, methods:, group:, render_nil:)
           @name = name
           @attribute = attribute
+          @group = group
           @render_nil = render_nil
 
           if methods

data/lib/shale/mapping/descriptor/xml.rb

@@ -28,13 +28,21 @@ module Shale
         # @param [String] name
         # @param [Symbol, String] attribute
         # @param [Hash, nil] methods
+        # @param [String, nil] group
         # @param [Shale::Mapping::XmlNamespace] namespace
         # @param [true, false] cdata
         # @param [true, false] render_nil
         #
         # @api private
-        def initialize(name:, attribute:, methods:, namespace:, cdata:, render_nil:)
-          super(name: name, attribute: attribute, methods: methods, render_nil: render_nil)
+        def initialize(name:, attribute:, methods:, group:, namespace:, cdata:, render_nil:)
+          super(
+            name: name,
+            attribute: attribute,
+            methods: methods,
+            group: group,
+            render_nil: render_nil
+          )
+
           @namespace = namespace
           @cdata = cdata
         end
@@ -47,6 +55,15 @@ module Shale
         def prefixed_name
           [namespace.prefix, name].compact.join(':')
         end
+
+        # Return name with XML namespace
+        #
+        # @return [String]
+        #
+        # @api private
+        def namespaced_name
+          [namespace.name, name].compact.join(':')
+        end
       end
     end
   end