shale 0.8.0 → 0.9.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -0
- data/README.md +134 -14
- data/lib/shale/adapter/csv.rb +48 -0
- data/lib/shale/adapter/nokogiri/document.rb +7 -2
- data/lib/shale/adapter/nokogiri.rb +11 -4
- data/lib/shale/adapter/ox.rb +10 -4
- data/lib/shale/adapter/rexml.rb +18 -4
- data/lib/shale/mapper.rb +35 -0
- data/lib/shale/mapping/dict.rb +2 -2
- data/lib/shale/mapping/dict_base.rb +7 -4
- data/lib/shale/type/complex.rb +126 -10
- data/lib/shale/type/date.rb +11 -0
- data/lib/shale/type/time.rb +11 -0
- data/lib/shale/type/value.rb +22 -0
- data/lib/shale/version.rb +1 -1
- data/lib/shale.rb +30 -4
- data/shale.gemspec +2 -2
- metadata +5 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3f880bc8d984c45e7e8a05bd155c1014df237e037263c7bf68a7ae82c0e49ea4
|
4
|
+
data.tar.gz: ba7a2f12bed87e048dd53d43900f1c8203d90f5f89a43b3824475ddda39eb07b
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 54023d4f621dae53537f73e0ad79d3736d5d52e605c1a7da2c4bdb06588b348feaae9066ce00b6b3aedc9e84ea23a425884eabf987717ffda6d3c22eccc6325f
|
7
|
+
data.tar.gz: e82cad292dab557891cad4f210bfb4a97d2096a6d2151a5a8aef020358accb5986e527e32a141a9a2e3a01d594e9a3adb42c0f7e8d17464517731136f63289a2
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
@@ -1,18 +1,18 @@
|
|
1
1
|
# Shale
|
2
2
|
|
3
|
-
Shale is a Ruby object mapper and serializer for JSON, YAML, TOML and XML.
|
4
|
-
It allows you to parse JSON, YAML, TOML and XML data and convert it into Ruby data structures,
|
5
|
-
as well as serialize data structures into JSON, YAML, TOML or XML.
|
3
|
+
Shale is a Ruby object mapper and serializer for JSON, YAML, TOML, CSV and XML.
|
4
|
+
It allows you to parse JSON, YAML, TOML, CSV and XML data and convert it into Ruby data structures,
|
5
|
+
as well as serialize data structures into JSON, YAML, TOML, CSV or XML.
|
6
6
|
|
7
7
|
Documentation with interactive examples is available at [Shale website](https://www.shalerb.org)
|
8
8
|
|
9
9
|
## Features
|
10
10
|
|
11
|
-
* Convert JSON, YAML, TOML and XML to Ruby data model
|
12
|
-
* Convert Ruby data model to JSON, YAML, TOML and XML
|
11
|
+
* Convert JSON, YAML, TOML, CSV and XML to Ruby data model
|
12
|
+
* Convert Ruby data model to JSON, YAML, TOML, CSV and XML
|
13
13
|
* Generate JSON and XML Schema from Ruby models
|
14
14
|
* Compile JSON and XML Schema into Ruby models
|
15
|
-
* Out of the box support for JSON, YAML, Tomlib, toml-rb, Nokogiri, REXML and Ox parsers
|
15
|
+
* Out of the box support for JSON, YAML, Tomlib, toml-rb, CSV, Nokogiri, REXML and Ox parsers
|
16
16
|
* Support for custom adapters
|
17
17
|
|
18
18
|
## Installation
|
@@ -51,9 +51,13 @@ $ gem install shale
|
|
51
51
|
* [Converting object to Hash](#converting-object-to-hash)
|
52
52
|
* [Converting XML to object](#converting-xml-to-object)
|
53
53
|
* [Converting object to XML](#converting-object-to-xml)
|
54
|
+
* [Converting CSV to object](#converting-csv-to-object)
|
55
|
+
* [Converting object to CSV](#converting-object-to-csv)
|
56
|
+
* [Converting collections](#converting-collections)
|
54
57
|
* [Mapping JSON keys to object attributes](#mapping-json-keys-to-object-attributes)
|
55
58
|
* [Mapping YAML keys to object attributes](#mapping-yaml-keys-to-object-attributes)
|
56
59
|
* [Mapping TOML keys to object attributes](#mapping-toml-keys-to-object-attributes)
|
60
|
+
* [Mapping CSV columns to object attributes](#mapping-csv-columns-to-object-attributes)
|
57
61
|
* [Mapping Hash keys to object attributes](#mapping-hash-keys-to-object-attributes)
|
58
62
|
* [Mapping XML elements and attributes to object attributes](#mapping-xml-elements-and-attributes-to-object-attributes)
|
59
63
|
* [Using XML namespaces](#using-xml-namespaces)
|
@@ -346,6 +350,70 @@ person.to_xml
|
|
346
350
|
# </person>
|
347
351
|
```
|
348
352
|
|
353
|
+
### Converting CSV to object
|
354
|
+
|
355
|
+
CSV represents a flat data structure, so you can't map properties to complex types directly,
|
356
|
+
but you can use methods to map properties to complex types
|
357
|
+
(see [Using methods to extract and generate data](#using-methods-to-extract-and-generate-data)
|
358
|
+
section).
|
359
|
+
|
360
|
+
`.from_csv` method allways returns an array of records.
|
361
|
+
|
362
|
+
```ruby
|
363
|
+
people = Person.from_csv(<<~DATA)
|
364
|
+
John,Doe,50,false
|
365
|
+
DATA
|
366
|
+
```
|
367
|
+
|
368
|
+
### Converting object to CSV
|
369
|
+
|
370
|
+
```ruby
|
371
|
+
people[0].to_csv # or Person.to_csv(people) if you want to convert a collection
|
372
|
+
|
373
|
+
# =>
|
374
|
+
#
|
375
|
+
# John,Doe,50,false
|
376
|
+
```
|
377
|
+
|
378
|
+
### Converting collections
|
379
|
+
|
380
|
+
Shale allows converting collections for formats that support it (JSON, YAML and CSV).
|
381
|
+
To convert Ruby array to JSON:
|
382
|
+
|
383
|
+
```ruby
|
384
|
+
person1 = Person.new(name: 'John Doe')
|
385
|
+
person2 = Person.new(name: 'Joe Sixpack')
|
386
|
+
|
387
|
+
Person.to_json([person1, person2], pretty: true)
|
388
|
+
# or Person.to_yaml([person1, person2])
|
389
|
+
# or Person.to_csv([person1, person2])
|
390
|
+
|
391
|
+
# =>
|
392
|
+
#
|
393
|
+
# [
|
394
|
+
# { "name": "John Doe" },
|
395
|
+
# { "name": "Joe Sixpack" }
|
396
|
+
# ]
|
397
|
+
```
|
398
|
+
|
399
|
+
To convert JSON array to Ruby:
|
400
|
+
|
401
|
+
```ruby
|
402
|
+
Person.from_json(<<~JSON)
|
403
|
+
[
|
404
|
+
{ "name": "John Doe" },
|
405
|
+
{ "name": "Joe Sixpack" }
|
406
|
+
]
|
407
|
+
JSON
|
408
|
+
|
409
|
+
# =>
|
410
|
+
#
|
411
|
+
# [
|
412
|
+
# #<Person:0x00000001033dbce8 @name="John Doe">,
|
413
|
+
# #<Person:0x00000001033db4c8 @name="Joe Sixpack">
|
414
|
+
# ]
|
415
|
+
```
|
416
|
+
|
349
417
|
### Mapping JSON keys to object attributes
|
350
418
|
|
351
419
|
By default keys are named the same as attributes. To use custom keys use:
|
@@ -392,6 +460,24 @@ class Person < Shale::Mapper
|
|
392
460
|
end
|
393
461
|
```
|
394
462
|
|
463
|
+
### Mapping CSV columns to object attributes
|
464
|
+
|
465
|
+
For CSV the order of mapping matters, the first argument in the `map` method is only
|
466
|
+
used as a label in header row. So, in the example below the first column will be mapped
|
467
|
+
to `:first_name` attribute and the second column to `:last_name`.
|
468
|
+
|
469
|
+
```ruby
|
470
|
+
class Person < Shale::Mapper
|
471
|
+
attribute :first_name, Shale::Type::String
|
472
|
+
attribute :last_name, Shale::Type::String
|
473
|
+
|
474
|
+
csv do
|
475
|
+
map 'firstName', to: :first_name
|
476
|
+
map 'lastName', to: :last_name
|
477
|
+
end
|
478
|
+
end
|
479
|
+
```
|
480
|
+
|
395
481
|
### Mapping Hash keys to object attributes
|
396
482
|
|
397
483
|
```ruby
|
@@ -561,8 +647,9 @@ DATA
|
|
561
647
|
|
562
648
|
### Rendering nil values
|
563
649
|
|
564
|
-
|
565
|
-
by using `render_nil: true` on a mapping.
|
650
|
+
For JSON, YAML, TOML and XML by default, elements with `nil` value are not rendered.
|
651
|
+
You can change this behavior by using `render_nil: true` on a mapping.
|
652
|
+
For CSV the default is to render `nil` elements.
|
566
653
|
|
567
654
|
```ruby
|
568
655
|
class Person < Shale::Mapper
|
@@ -853,19 +940,52 @@ person.to_json(pretty: true)
|
|
853
940
|
# }
|
854
941
|
```
|
855
942
|
|
856
|
-
You can also add an XML declaration by passing `declaration: true`
|
943
|
+
You can also add an XML declaration by passing `declaration: true` and `encoding: true`
|
944
|
+
or if you want to spcify version: `declaration: '1.1'` and `encoding: 'ASCII'` to `#to_xml`
|
857
945
|
|
858
946
|
```ruby
|
859
|
-
person.to_xml(pretty: true, declaration: true)
|
947
|
+
person.to_xml(pretty: true, declaration: true, encoding: true)
|
860
948
|
|
861
949
|
# =>
|
862
950
|
#
|
863
|
-
# <?xml version="1.0"?>
|
951
|
+
# <?xml version="1.0" encoding="UTF-8"?>
|
864
952
|
# <Person>
|
865
953
|
# <Address city="London"/>
|
866
954
|
# </Person>
|
867
955
|
```
|
868
956
|
|
957
|
+
For CSV you can pass `headers: true` to indicate that the first row contains column
|
958
|
+
names and shouldn't be included in the returned collection. It also accepts all the options that
|
959
|
+
[CSV parser](https://ruby-doc.org/stdlib-3.1.2/libdoc/csv/rdoc/CSV.html#class-CSV-label-Options) accepts.
|
960
|
+
|
961
|
+
```ruby
|
962
|
+
class Person
|
963
|
+
attribute :first_name, Shale::Type::String
|
964
|
+
attribute :last_name, Shale::Type::String
|
965
|
+
end
|
966
|
+
|
967
|
+
people = Person.from_csv(<<~DATA, headers: true, col_sep: '|')
|
968
|
+
first_name|last_name
|
969
|
+
John|Doe
|
970
|
+
James|Sixpack
|
971
|
+
DATA
|
972
|
+
|
973
|
+
# =>
|
974
|
+
#
|
975
|
+
# [
|
976
|
+
# #<Person:0x0000000113d7a488 @first_name="John", @last_name="Doe">,
|
977
|
+
# #<Person:0x0000000113d7a488 @first_name="James", @last_name="Sixpack">
|
978
|
+
# ]
|
979
|
+
|
980
|
+
Person.to_csv(people, headers: true, col_sep: '|')
|
981
|
+
|
982
|
+
# =>
|
983
|
+
#
|
984
|
+
# first_name|last_name
|
985
|
+
# John|Doe
|
986
|
+
# James|Sixpack
|
987
|
+
```
|
988
|
+
|
869
989
|
### Using custom models
|
870
990
|
|
871
991
|
By default Shale combines mapper and model into one class. If you want to use your own classes
|
@@ -955,10 +1075,10 @@ end
|
|
955
1075
|
### Adapters
|
956
1076
|
|
957
1077
|
Shale uses adapters for parsing and generating documents.
|
958
|
-
By default Ruby's standard JSON, YAML parsers are used for handling JSON
|
1078
|
+
By default Ruby's standard JSON, YAML, CSV parsers are used for handling JSON YAML, CSV documents.
|
959
1079
|
|
960
|
-
You can change it by providing your own adapter. For JSON, YAML and
|
961
|
-
`.load` and `.dump` class methods.
|
1080
|
+
You can change it by providing your own adapter. For JSON, YAML, TOML and CSV adapter must
|
1081
|
+
implement `.load` and `.dump` class methods.
|
962
1082
|
|
963
1083
|
```ruby
|
964
1084
|
require 'shale'
|
@@ -0,0 +1,48 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'csv'
|
4
|
+
|
5
|
+
module Shale
|
6
|
+
module Adapter
|
7
|
+
# CSV adapter
|
8
|
+
#
|
9
|
+
# @api public
|
10
|
+
class CSV
|
11
|
+
# Parse CSV into Array<Hash<String, any>>
|
12
|
+
#
|
13
|
+
# @param [String] csv CSV document
|
14
|
+
# @param [Array<String>] headers
|
15
|
+
# @param [Hash] options
|
16
|
+
#
|
17
|
+
# @return [Array<Hash<String, any>>]
|
18
|
+
#
|
19
|
+
# @api private
|
20
|
+
def self.load(csv, headers:, **options)
|
21
|
+
::CSV.parse(csv, headers: headers, **options).map(&:to_h)
|
22
|
+
end
|
23
|
+
|
24
|
+
# Serialize Array<Hash<String, any>> into CSV
|
25
|
+
#
|
26
|
+
# @param [Array<Hash<String, any>>] obj Array<Hash<String, any>> object
|
27
|
+
# @param [Array<String>] headers
|
28
|
+
# @param [Hash] options
|
29
|
+
#
|
30
|
+
# @return [String]
|
31
|
+
#
|
32
|
+
# @api private
|
33
|
+
def self.dump(obj, headers:, **options)
|
34
|
+
::CSV.generate(**options) do |csv|
|
35
|
+
obj.each do |row|
|
36
|
+
values = []
|
37
|
+
|
38
|
+
headers.each do |header|
|
39
|
+
values << row[header] if row.key?(header)
|
40
|
+
end
|
41
|
+
|
42
|
+
csv << values
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
@@ -9,9 +9,14 @@ module Shale
|
|
9
9
|
class Document
|
10
10
|
# Initialize object
|
11
11
|
#
|
12
|
+
# @param [String, nil] version
|
13
|
+
#
|
12
14
|
# @api private
|
13
|
-
def initialize
|
14
|
-
|
15
|
+
def initialize(version = nil)
|
16
|
+
ver = nil
|
17
|
+
ver = version if version.is_a?(String)
|
18
|
+
|
19
|
+
@doc = ::Nokogiri::XML::Document.new(ver)
|
15
20
|
@namespaces = {}
|
16
21
|
end
|
17
22
|
|
@@ -37,12 +37,13 @@ module Shale
|
|
37
37
|
#
|
38
38
|
# @param [::Nokogiri::XML::Document] doc Nokogiri document
|
39
39
|
# @param [true, false] pretty
|
40
|
-
# @param [true, false] declaration
|
40
|
+
# @param [true, false, String] declaration
|
41
|
+
# @param [true, false, String] encoding
|
41
42
|
#
|
42
43
|
# @return [String]
|
43
44
|
#
|
44
45
|
# @api private
|
45
|
-
def self.dump(doc, pretty: false, declaration: false)
|
46
|
+
def self.dump(doc, pretty: false, declaration: false, encoding: false)
|
46
47
|
save_with = ::Nokogiri::XML::Node::SaveOptions::AS_XML
|
47
48
|
|
48
49
|
if pretty
|
@@ -53,6 +54,10 @@ module Shale
|
|
53
54
|
save_with |= ::Nokogiri::XML::Node::SaveOptions::NO_DECLARATION
|
54
55
|
end
|
55
56
|
|
57
|
+
if encoding
|
58
|
+
doc.encoding = encoding == true ? 'UTF-8' : encoding
|
59
|
+
end
|
60
|
+
|
56
61
|
result = doc.to_xml(save_with: save_with)
|
57
62
|
|
58
63
|
unless pretty
|
@@ -64,9 +69,11 @@ module Shale
|
|
64
69
|
|
65
70
|
# Create Shale::Adapter::Nokogiri::Document instance
|
66
71
|
#
|
72
|
+
# @param [true, false, String, nil] declaration
|
73
|
+
#
|
67
74
|
# @api private
|
68
|
-
def self.create_document
|
69
|
-
Document.new
|
75
|
+
def self.create_document(version = nil)
|
76
|
+
Document.new(version)
|
70
77
|
end
|
71
78
|
end
|
72
79
|
end
|
data/lib/shale/adapter/ox.rb
CHANGED
@@ -31,12 +31,13 @@ module Shale
|
|
31
31
|
#
|
32
32
|
# @param [::Ox::Document, ::Ox::Element] doc Ox document
|
33
33
|
# @param [true, false] pretty
|
34
|
-
# @param [true, false] declaration
|
34
|
+
# @param [true, false, String] declaration
|
35
|
+
# @param [true, false, String] encoding
|
35
36
|
#
|
36
37
|
# @return [String]
|
37
38
|
#
|
38
39
|
# @api private
|
39
|
-
def self.dump(doc, pretty: false, declaration: false)
|
40
|
+
def self.dump(doc, pretty: false, declaration: false, encoding: false)
|
40
41
|
opts = { indent: -1, with_xml: false }
|
41
42
|
|
42
43
|
if pretty
|
@@ -44,7 +45,12 @@ module Shale
|
|
44
45
|
end
|
45
46
|
|
46
47
|
if declaration
|
47
|
-
doc[:version] = '1.0'
|
48
|
+
doc[:version] = declaration == true ? '1.0' : declaration
|
49
|
+
|
50
|
+
if encoding
|
51
|
+
doc[:encoding] = encoding == true ? 'UTF-8' : encoding
|
52
|
+
end
|
53
|
+
|
48
54
|
opts[:with_xml] = true
|
49
55
|
end
|
50
56
|
|
@@ -54,7 +60,7 @@ module Shale
|
|
54
60
|
# Create Shale::Adapter::Ox::Document instance
|
55
61
|
#
|
56
62
|
# @api private
|
57
|
-
def self.create_document
|
63
|
+
def self.create_document(_version = nil)
|
58
64
|
Document.new
|
59
65
|
end
|
60
66
|
end
|
data/lib/shale/adapter/rexml.rb
CHANGED
@@ -32,14 +32,28 @@ module Shale
|
|
32
32
|
#
|
33
33
|
# @param [::REXML::Document] doc REXML document
|
34
34
|
# @param [true, false] pretty
|
35
|
-
# @param [true, false] declaration
|
35
|
+
# @param [true, false, String] declaration
|
36
|
+
# @param [true, false, String] encoding
|
36
37
|
#
|
37
38
|
# @return [String]
|
38
39
|
#
|
39
40
|
# @api private
|
40
|
-
def self.dump(doc, pretty: false, declaration: false)
|
41
|
+
def self.dump(doc, pretty: false, declaration: false, encoding: false)
|
41
42
|
if declaration
|
42
|
-
|
43
|
+
ver = nil
|
44
|
+
enc = nil
|
45
|
+
|
46
|
+
if declaration.is_a?(String)
|
47
|
+
ver = declaration
|
48
|
+
end
|
49
|
+
|
50
|
+
if encoding == true
|
51
|
+
enc = 'UTF-8'
|
52
|
+
else
|
53
|
+
enc = encoding || nil
|
54
|
+
end
|
55
|
+
|
56
|
+
doc.add(::REXML::XMLDecl.new(ver, enc))
|
43
57
|
end
|
44
58
|
|
45
59
|
io = StringIO.new
|
@@ -58,7 +72,7 @@ module Shale
|
|
58
72
|
# Create Shale::Adapter::REXML::Document instance
|
59
73
|
#
|
60
74
|
# @api private
|
61
|
-
def self.create_document
|
75
|
+
def self.create_document(_version = nil)
|
62
76
|
Document.new
|
63
77
|
end
|
64
78
|
end
|
data/lib/shale/mapper.rb
CHANGED
@@ -49,6 +49,7 @@ module Shale
|
|
49
49
|
@json_mapping = Mapping::Dict.new
|
50
50
|
@yaml_mapping = Mapping::Dict.new
|
51
51
|
@toml_mapping = Mapping::Dict.new
|
52
|
+
@csv_mapping = Mapping::Dict.new(render_nil_default: true)
|
52
53
|
@xml_mapping = Mapping::Xml.new
|
53
54
|
|
54
55
|
class << self
|
@@ -87,6 +88,13 @@ module Shale
|
|
87
88
|
# @api public
|
88
89
|
attr_reader :toml_mapping
|
89
90
|
|
91
|
+
# Return CSV mapping object
|
92
|
+
#
|
93
|
+
# @return [Shale::Mapping::Dict]
|
94
|
+
#
|
95
|
+
# @api public
|
96
|
+
attr_reader :csv_mapping
|
97
|
+
|
90
98
|
# Return XML mapping object
|
91
99
|
#
|
92
100
|
# @return [Shale::Mapping::XML]
|
@@ -109,12 +117,14 @@ module Shale
|
|
109
117
|
subclass.instance_variable_set('@__json_mapping_init', @json_mapping.dup)
|
110
118
|
subclass.instance_variable_set('@__yaml_mapping_init', @yaml_mapping.dup)
|
111
119
|
subclass.instance_variable_set('@__toml_mapping_init', @toml_mapping.dup)
|
120
|
+
subclass.instance_variable_set('@__csv_mapping_init', @csv_mapping.dup)
|
112
121
|
subclass.instance_variable_set('@__xml_mapping_init', @xml_mapping.dup)
|
113
122
|
|
114
123
|
subclass.instance_variable_set('@hash_mapping', @hash_mapping.dup)
|
115
124
|
subclass.instance_variable_set('@json_mapping', @json_mapping.dup)
|
116
125
|
subclass.instance_variable_set('@yaml_mapping', @yaml_mapping.dup)
|
117
126
|
subclass.instance_variable_set('@toml_mapping', @toml_mapping.dup)
|
127
|
+
subclass.instance_variable_set('@csv_mapping', @csv_mapping.dup)
|
118
128
|
|
119
129
|
xml_mapping = @xml_mapping.dup
|
120
130
|
xml_mapping.root(Utils.underscore(subclass.name || ''))
|
@@ -173,6 +183,7 @@ module Shale
|
|
173
183
|
@json_mapping.map(name.to_s, to: name) unless @json_mapping.finalized?
|
174
184
|
@yaml_mapping.map(name.to_s, to: name) unless @yaml_mapping.finalized?
|
175
185
|
@toml_mapping.map(name.to_s, to: name) unless @toml_mapping.finalized?
|
186
|
+
@csv_mapping.map(name.to_s, to: name) unless @csv_mapping.finalized?
|
176
187
|
@xml_mapping.map_element(name.to_s, to: name) unless @xml_mapping.finalized?
|
177
188
|
|
178
189
|
@attributes_module.class_eval(<<-RUBY, __FILE__, __LINE__ + 1)
|
@@ -280,6 +291,30 @@ module Shale
|
|
280
291
|
@toml_mapping.instance_eval(&block)
|
281
292
|
end
|
282
293
|
|
294
|
+
# Define CSV mapping
|
295
|
+
#
|
296
|
+
# @param [Proc] block
|
297
|
+
#
|
298
|
+
# @example
|
299
|
+
# calss Person < Shale::Mapper
|
300
|
+
# attribute :first_name, Shale::Type::String
|
301
|
+
# attribute :last_name, Shale::Type::String
|
302
|
+
# attribute :age, Shale::Type::Integer
|
303
|
+
#
|
304
|
+
# csv do
|
305
|
+
# map 'first_name', to: :first_name
|
306
|
+
# map 'last_name', to: :last_name
|
307
|
+
# map 'age', to: :age
|
308
|
+
# end
|
309
|
+
# end
|
310
|
+
#
|
311
|
+
# @api public
|
312
|
+
def csv(&block)
|
313
|
+
@csv_mapping = @__csv_mapping_init.dup
|
314
|
+
@csv_mapping.finalize!
|
315
|
+
@csv_mapping.instance_eval(&block)
|
316
|
+
end
|
317
|
+
|
283
318
|
# Define XML mapping
|
284
319
|
#
|
285
320
|
# @param [Proc] block
|
data/lib/shale/mapping/dict.rb
CHANGED
@@ -14,12 +14,12 @@ module Shale
|
|
14
14
|
# @param [String] key Document's key
|
15
15
|
# @param [Symbol, nil] to Object's attribute
|
16
16
|
# @param [Hash, nil] using
|
17
|
-
# @param [true, false] render_nil
|
17
|
+
# @param [true, false, nil] render_nil
|
18
18
|
#
|
19
19
|
# @raise [IncorrectMappingArgumentsError] when arguments are incorrect
|
20
20
|
#
|
21
21
|
# @api private
|
22
|
-
def map(key, to: nil, using: nil, render_nil:
|
22
|
+
def map(key, to: nil, using: nil, render_nil: nil)
|
23
23
|
super(key, to: to, using: using, render_nil: render_nil)
|
24
24
|
end
|
25
25
|
|
@@ -18,10 +18,13 @@ module Shale
|
|
18
18
|
|
19
19
|
# Initialize instance
|
20
20
|
#
|
21
|
+
# @param [true, false] render_nil_default
|
22
|
+
#
|
21
23
|
# @api private
|
22
|
-
def initialize
|
24
|
+
def initialize(render_nil_default: false)
|
23
25
|
@keys = {}
|
24
26
|
@finalized = false
|
27
|
+
@render_nil_default = render_nil_default
|
25
28
|
end
|
26
29
|
|
27
30
|
# Map key to attribute
|
@@ -30,19 +33,19 @@ module Shale
|
|
30
33
|
# @param [Symbol, nil] to
|
31
34
|
# @param [Hash, nil] using
|
32
35
|
# @param [String, nil] group
|
33
|
-
# @param [true, false] render_nil
|
36
|
+
# @param [true, false, nil] render_nil
|
34
37
|
#
|
35
38
|
# @raise [IncorrectMappingArgumentsError] when arguments are incorrect
|
36
39
|
#
|
37
40
|
# @api private
|
38
|
-
def map(key, to: nil, using: nil, group: nil, render_nil:
|
41
|
+
def map(key, to: nil, using: nil, group: nil, render_nil: nil)
|
39
42
|
Validator.validate_arguments(key, to, using)
|
40
43
|
@keys[key] = Descriptor::Dict.new(
|
41
44
|
name: key,
|
42
45
|
attribute: to,
|
43
46
|
methods: using,
|
44
47
|
group: group,
|
45
|
-
render_nil: render_nil
|
48
|
+
render_nil: render_nil.nil? ? @render_nil_default : render_nil
|
46
49
|
)
|
47
50
|
end
|
48
51
|
|
data/lib/shale/type/complex.rb
CHANGED
@@ -13,11 +13,11 @@ module Shale
|
|
13
13
|
# @api private
|
14
14
|
class Complex < Value
|
15
15
|
class << self
|
16
|
-
%i[hash json yaml toml].each do |format|
|
16
|
+
%i[hash json yaml toml csv].each do |format|
|
17
17
|
class_eval(<<-RUBY, __FILE__, __LINE__ + 1)
|
18
18
|
# Convert Hash to Object using Hash/JSON/YAML/TOML mapping
|
19
19
|
#
|
20
|
-
# @param [Hash] hash Hash to convert
|
20
|
+
# @param [Hash, Array] hash Hash to convert
|
21
21
|
# @param [Array<Symbol>] only
|
22
22
|
# @param [Array<Symbol>] except
|
23
23
|
# @param [any] context
|
@@ -26,6 +26,18 @@ module Shale
|
|
26
26
|
#
|
27
27
|
# @api public
|
28
28
|
def of_#{format}(hash, only: nil, except: nil, context: nil)
|
29
|
+
#{
|
30
|
+
if format != :toml
|
31
|
+
<<~CODE
|
32
|
+
if hash.is_a?(Array)
|
33
|
+
return hash.map do |item|
|
34
|
+
of_#{format}(item, only: only, except: except, context: context)
|
35
|
+
end
|
36
|
+
end
|
37
|
+
CODE
|
38
|
+
end
|
39
|
+
}
|
40
|
+
|
29
41
|
instance = model.new
|
30
42
|
|
31
43
|
attributes
|
@@ -109,7 +121,7 @@ module Shale
|
|
109
121
|
|
110
122
|
# Convert Object to Hash using Hash/JSON/YAML/TOML mapping
|
111
123
|
#
|
112
|
-
# @param [any] instance Object to convert
|
124
|
+
# @param [any, Array<any>] instance Object to convert
|
113
125
|
# @param [Array<Symbol>] only
|
114
126
|
# @param [Array<Symbol>] except
|
115
127
|
# @param [any] context
|
@@ -120,6 +132,18 @@ module Shale
|
|
120
132
|
#
|
121
133
|
# @api public
|
122
134
|
def as_#{format}(instance, only: nil, except: nil, context: nil)
|
135
|
+
#{
|
136
|
+
if format != :toml
|
137
|
+
<<~CODE
|
138
|
+
if instance.is_a?(Array)
|
139
|
+
return instance.map do |item|
|
140
|
+
as_#{format}(item, only: only, except: except, context: context)
|
141
|
+
end
|
142
|
+
end
|
143
|
+
CODE
|
144
|
+
end
|
145
|
+
}
|
146
|
+
|
123
147
|
unless instance.is_a?(model)
|
124
148
|
msg = "argument is a '\#{instance.class}' but should be a '\#{model}'"
|
125
149
|
raise IncorrectModelError, msg
|
@@ -312,6 +336,65 @@ module Shale
|
|
312
336
|
)
|
313
337
|
end
|
314
338
|
|
339
|
+
# Convert CSV to Object
|
340
|
+
#
|
341
|
+
# @param [String] csv CSV to convert
|
342
|
+
# @param [Array<Symbol>] only
|
343
|
+
# @param [Array<Symbol>] except
|
344
|
+
# @param [any] context
|
345
|
+
# @param [true, false] headers
|
346
|
+
# @param [Hash] csv_options
|
347
|
+
#
|
348
|
+
# @return [model instance]
|
349
|
+
#
|
350
|
+
# @api public
|
351
|
+
def from_csv(csv, only: nil, except: nil, context: nil, headers: false, **csv_options)
|
352
|
+
data = Shale.csv_adapter.load(csv, **csv_options.merge(headers: csv_mapping.keys.keys))
|
353
|
+
|
354
|
+
data.shift if headers
|
355
|
+
|
356
|
+
of_csv(
|
357
|
+
data,
|
358
|
+
only: only,
|
359
|
+
except: except,
|
360
|
+
context: context
|
361
|
+
)
|
362
|
+
end
|
363
|
+
|
364
|
+
# Convert Object to CSV
|
365
|
+
#
|
366
|
+
# @param [model instance] instance Object to convert
|
367
|
+
# @param [Array<Symbol>] only
|
368
|
+
# @param [Array<Symbol>] except
|
369
|
+
# @param [any] context
|
370
|
+
# @param [true, false] headers
|
371
|
+
# @param [Hash] csv_options
|
372
|
+
#
|
373
|
+
# @return [String]
|
374
|
+
#
|
375
|
+
# @api public
|
376
|
+
def to_csv(instance, only: nil, except: nil, context: nil, headers: false, **csv_options)
|
377
|
+
data = as_csv([*instance], only: only, except: except, context: context)
|
378
|
+
|
379
|
+
cols = csv_mapping.keys.values
|
380
|
+
|
381
|
+
if only
|
382
|
+
cols = cols.select { |e| only.include?(e.attribute) }
|
383
|
+
end
|
384
|
+
|
385
|
+
if except
|
386
|
+
cols = cols.reject { |e| except.include?(e.attribute) }
|
387
|
+
end
|
388
|
+
|
389
|
+
cols = cols.map(&:name)
|
390
|
+
|
391
|
+
if headers
|
392
|
+
data.prepend(cols.to_h { |e| [e, e] })
|
393
|
+
end
|
394
|
+
|
395
|
+
Shale.csv_adapter.dump(data, **csv_options.merge(headers: cols))
|
396
|
+
end
|
397
|
+
|
315
398
|
# Convert XML document to Object
|
316
399
|
#
|
317
400
|
# @param [Shale::Adapter::<XML adapter>::Node] element
|
@@ -495,7 +578,8 @@ module Shale
|
|
495
578
|
_cdata = nil,
|
496
579
|
only: nil,
|
497
580
|
except: nil,
|
498
|
-
context: nil
|
581
|
+
context: nil,
|
582
|
+
version: nil
|
499
583
|
)
|
500
584
|
unless instance.is_a?(model)
|
501
585
|
msg = "argument is a '#{instance.class}' but should be a '#{model}'"
|
@@ -503,7 +587,7 @@ module Shale
|
|
503
587
|
end
|
504
588
|
|
505
589
|
unless doc
|
506
|
-
doc = Shale.xml_adapter.create_document
|
590
|
+
doc = Shale.xml_adapter.create_document(version)
|
507
591
|
|
508
592
|
element = as_xml(
|
509
593
|
instance,
|
@@ -680,6 +764,7 @@ module Shale
|
|
680
764
|
# @param [any] context
|
681
765
|
# @param [true, false] pretty
|
682
766
|
# @param [true, false] declaration
|
767
|
+
# @param [true, false, String] encoding
|
683
768
|
#
|
684
769
|
# @raise [AdapterError]
|
685
770
|
#
|
@@ -692,13 +777,15 @@ module Shale
|
|
692
777
|
except: nil,
|
693
778
|
context: nil,
|
694
779
|
pretty: false,
|
695
|
-
declaration: false
|
780
|
+
declaration: false,
|
781
|
+
encoding: false
|
696
782
|
)
|
697
783
|
validate_xml_adapter
|
698
784
|
Shale.xml_adapter.dump(
|
699
|
-
as_xml(instance, only: only, except: except, context: context),
|
785
|
+
as_xml(instance, only: only, except: except, context: context, version: declaration),
|
700
786
|
pretty: pretty,
|
701
|
-
declaration: declaration
|
787
|
+
declaration: declaration,
|
788
|
+
encoding: encoding
|
702
789
|
)
|
703
790
|
end
|
704
791
|
|
@@ -801,6 +888,26 @@ module Shale
|
|
801
888
|
self.class.to_toml(self, only: only, except: except, context: context)
|
802
889
|
end
|
803
890
|
|
891
|
+
# Convert Object to CSV
|
892
|
+
#
|
893
|
+
# @param [Array<Symbol>] only
|
894
|
+
# @param [Array<Symbol>] except
|
895
|
+
# @param [any] context
|
896
|
+
#
|
897
|
+
# @return [String]
|
898
|
+
#
|
899
|
+
# @api public
|
900
|
+
def to_csv(only: nil, except: nil, context: nil, headers: false, **csv_options)
|
901
|
+
self.class.to_csv(
|
902
|
+
self,
|
903
|
+
only: only,
|
904
|
+
except: except,
|
905
|
+
context: context,
|
906
|
+
headers: headers,
|
907
|
+
**csv_options
|
908
|
+
)
|
909
|
+
end
|
910
|
+
|
804
911
|
# Convert Object to XML
|
805
912
|
#
|
806
913
|
# @param [Array<Symbol>] only
|
@@ -808,18 +915,27 @@ module Shale
|
|
808
915
|
# @param [any] context
|
809
916
|
# @param [true, false] pretty
|
810
917
|
# @param [true, false] declaration
|
918
|
+
# @param [true, false, String] encoding
|
811
919
|
#
|
812
920
|
# @return [String]
|
813
921
|
#
|
814
922
|
# @api public
|
815
|
-
def to_xml(
|
923
|
+
def to_xml(
|
924
|
+
only: nil,
|
925
|
+
except: nil,
|
926
|
+
context: nil,
|
927
|
+
pretty: false,
|
928
|
+
declaration: false,
|
929
|
+
encoding: false
|
930
|
+
)
|
816
931
|
self.class.to_xml(
|
817
932
|
self,
|
818
933
|
only: only,
|
819
934
|
except: except,
|
820
935
|
context: context,
|
821
936
|
pretty: pretty,
|
822
|
-
declaration: declaration
|
937
|
+
declaration: declaration,
|
938
|
+
encoding: encoding
|
823
939
|
)
|
824
940
|
end
|
825
941
|
end
|
data/lib/shale/type/date.rb
CHANGED
@@ -47,6 +47,17 @@ module Shale
|
|
47
47
|
value&.iso8601
|
48
48
|
end
|
49
49
|
|
50
|
+
# Use ISO 8601 format in CSV document
|
51
|
+
#
|
52
|
+
# @param [Date] value
|
53
|
+
#
|
54
|
+
# @return [String]
|
55
|
+
#
|
56
|
+
# @api private
|
57
|
+
def self.as_csv(value, **)
|
58
|
+
value&.iso8601
|
59
|
+
end
|
60
|
+
|
50
61
|
# Use ISO 8601 format in XML document
|
51
62
|
#
|
52
63
|
# @param [Date] value Value to convert to XML
|
data/lib/shale/type/time.rb
CHANGED
@@ -47,6 +47,17 @@ module Shale
|
|
47
47
|
value&.iso8601
|
48
48
|
end
|
49
49
|
|
50
|
+
# Use ISO 8601 format in CSV document
|
51
|
+
#
|
52
|
+
# @param [Time] value
|
53
|
+
#
|
54
|
+
# @return [String]
|
55
|
+
#
|
56
|
+
# @api private
|
57
|
+
def self.as_csv(value, **)
|
58
|
+
value&.iso8601
|
59
|
+
end
|
60
|
+
|
50
61
|
# Use ISO 8601 format in XML document
|
51
62
|
#
|
52
63
|
# @param [Time] value Value to convert to XML
|
data/lib/shale/type/value.rb
CHANGED
@@ -111,6 +111,28 @@ module Shale
|
|
111
111
|
value
|
112
112
|
end
|
113
113
|
|
114
|
+
# Extract value from CSV document
|
115
|
+
#
|
116
|
+
# @param [any] value
|
117
|
+
#
|
118
|
+
# @return [any]
|
119
|
+
#
|
120
|
+
# @api private
|
121
|
+
def of_csv(value, **)
|
122
|
+
value
|
123
|
+
end
|
124
|
+
|
125
|
+
# Convert value to form accepted by CSV document
|
126
|
+
#
|
127
|
+
# @param [any] value
|
128
|
+
#
|
129
|
+
# @return [any]
|
130
|
+
#
|
131
|
+
# @api private
|
132
|
+
def as_csv(value, **)
|
133
|
+
value
|
134
|
+
end
|
135
|
+
|
114
136
|
# Extract value from XML document
|
115
137
|
#
|
116
138
|
# @param [Shale::Adapter::<XML adapter>::Node] value
|
data/lib/shale/version.rb
CHANGED
data/lib/shale.rb
CHANGED
@@ -3,6 +3,7 @@
|
|
3
3
|
require 'yaml'
|
4
4
|
|
5
5
|
require_relative 'shale/mapper'
|
6
|
+
require_relative 'shale/adapter/csv'
|
6
7
|
require_relative 'shale/adapter/json'
|
7
8
|
require_relative 'shale/type/boolean'
|
8
9
|
require_relative 'shale/type/date'
|
@@ -15,7 +16,7 @@ require_relative 'shale/version'
|
|
15
16
|
# Main library namespace
|
16
17
|
#
|
17
18
|
# Shale uses adapters for parsing and serializing documents.
|
18
|
-
# For handling JSON and
|
19
|
+
# For handling JSON, YAML, TOML and CSV, adapter must implement .load and .dump methods, so
|
19
20
|
# e.g for handling JSON, MultiJson works out of the box.
|
20
21
|
#
|
21
22
|
# Adapters for XML handling are more complicated and must conform to [@see shale/adapter/rexml]
|
@@ -29,9 +30,11 @@ require_relative 'shale/version'
|
|
29
30
|
# Shale.json_adapter = MultiJson
|
30
31
|
# Shale.json_adapter # => MultiJson
|
31
32
|
#
|
32
|
-
# @example setting
|
33
|
-
#
|
34
|
-
#
|
33
|
+
# @example setting TOML adapter for handling TOML documents
|
34
|
+
# require 'shale/adapter/toml_rb'
|
35
|
+
#
|
36
|
+
# Shale.toml_adapter = Shale::Adapter::TomlRB
|
37
|
+
# Shale.toml_adapter # => Shale::Adapter::TomlRB
|
35
38
|
#
|
36
39
|
# @example setting REXML adapter for handling XML documents
|
37
40
|
# Shale.xml_adapter = Shale::Adapter::REXML
|
@@ -86,6 +89,16 @@ module Shale
|
|
86
89
|
# @api public
|
87
90
|
attr_accessor :toml_adapter
|
88
91
|
|
92
|
+
# Set CSV adapter
|
93
|
+
#
|
94
|
+
# @param [.load, .dump] adapter
|
95
|
+
#
|
96
|
+
# @example
|
97
|
+
# Shale.csv_adapter = Shale::Adapter::CSV
|
98
|
+
#
|
99
|
+
# @api public
|
100
|
+
attr_writer :csv_adapter
|
101
|
+
|
89
102
|
# XML adapter accessor. Available adapters are Shale::Adapter::REXML,
|
90
103
|
# Shale::Adapter::Nokogiri and Shale::Adapter::Ox
|
91
104
|
#
|
@@ -126,5 +139,18 @@ module Shale
|
|
126
139
|
def yaml_adapter
|
127
140
|
@yaml_adapter || YAML
|
128
141
|
end
|
142
|
+
|
143
|
+
# Return CSV adapter. By default CSV is used
|
144
|
+
#
|
145
|
+
# @return [.load, .dump]
|
146
|
+
#
|
147
|
+
# @example
|
148
|
+
# Shale.csv_adapter
|
149
|
+
# # => Shale::Adapter::CSV
|
150
|
+
#
|
151
|
+
# @api public
|
152
|
+
def csv_adapter
|
153
|
+
@csv_adapter || Adapter::CSV
|
154
|
+
end
|
129
155
|
end
|
130
156
|
end
|
data/shale.gemspec
CHANGED
@@ -8,8 +8,8 @@ Gem::Specification.new do |spec|
|
|
8
8
|
spec.authors = ['Kamil Giszczak']
|
9
9
|
spec.email = ['beerkg@gmail.com']
|
10
10
|
|
11
|
-
spec.summary = 'Ruby object mapper and serializer for XML, JSON, TOML and YAML.'
|
12
|
-
spec.description = 'Ruby object mapper and serializer for XML, JSON, TOML and YAML.'
|
11
|
+
spec.summary = 'Ruby object mapper and serializer for XML, JSON, TOML, CSV and YAML.'
|
12
|
+
spec.description = 'Ruby object mapper and serializer for XML, JSON, TOML, CSV and YAML.'
|
13
13
|
spec.homepage = 'https://shalerb.org'
|
14
14
|
spec.license = 'MIT'
|
15
15
|
|
metadata
CHANGED
@@ -1,16 +1,16 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: shale
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.9.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Kamil Giszczak
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-
|
11
|
+
date: 2022-10-31 00:00:00.000000000 Z
|
12
12
|
dependencies: []
|
13
|
-
description: Ruby object mapper and serializer for XML, JSON, TOML and YAML.
|
13
|
+
description: Ruby object mapper and serializer for XML, JSON, TOML, CSV and YAML.
|
14
14
|
email:
|
15
15
|
- beerkg@gmail.com
|
16
16
|
executables:
|
@@ -23,6 +23,7 @@ files:
|
|
23
23
|
- README.md
|
24
24
|
- exe/shaleb
|
25
25
|
- lib/shale.rb
|
26
|
+
- lib/shale/adapter/csv.rb
|
26
27
|
- lib/shale/adapter/json.rb
|
27
28
|
- lib/shale/adapter/nokogiri.rb
|
28
29
|
- lib/shale/adapter/nokogiri/document.rb
|
@@ -126,5 +127,5 @@ requirements: []
|
|
126
127
|
rubygems_version: 3.3.7
|
127
128
|
signing_key:
|
128
129
|
specification_version: 4
|
129
|
-
summary: Ruby object mapper and serializer for XML, JSON, TOML and YAML.
|
130
|
+
summary: Ruby object mapper and serializer for XML, JSON, TOML, CSV and YAML.
|
130
131
|
test_files: []
|