csv2hash 0.0.1 → 0.0.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +4 -0
- data/Gemfile.lock +1 -1
- data/README.md +132 -5
- data/bin/publish +24 -0
- data/csv2hash.gemspec +1 -1
- data/lib/csv2hash/csv_array.rb +12 -0
- data/lib/csv2hash/definition.rb +16 -8
- data/lib/csv2hash/parser/collection.rb +29 -0
- data/lib/csv2hash/parser/mapping.rb +26 -0
- data/lib/csv2hash/parser.rb +0 -22
- data/lib/csv2hash/validator/collection.rb +17 -0
- data/lib/csv2hash/validator/mapping.rb +14 -0
- data/lib/csv2hash/validator.rb +26 -18
- data/lib/csv2hash/version.rb +1 -1
- data/lib/csv2hash.rb +56 -7
- data/spec/csv2hash/definition_spec.rb +8 -25
- data/spec/csv2hash/parser/collection_spec.rb +49 -0
- data/spec/csv2hash/parser/mapping_spec.rb +38 -0
- data/spec/csv2hash/parser_spec.rb +1 -37
- data/spec/csv2hash/validator/collection_spec.rb +59 -0
- data/spec/csv2hash/validator/mapping_spec.rb +68 -0
- data/spec/csv2hash/validator_spec.rb +17 -18
- metadata +16 -2
- data/lib/csv2hash/definition/mapping.rb +0 -7
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: d4d2f7ba34ef01f65b28e952dcc72dc479ed2891
|
4
|
+
data.tar.gz: 77bfe0698393fedb815d54f631d8e6e7af05f401
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d246e42bd2202619bc8e4ad3ca581c9c5b3fa15be34bcee30817d31030b782d33d1f1e2d89cf95df17059c21e63c185bc4a682ec9a08fbc5c1242f18ed0e8800
|
7
|
+
data.tar.gz: b863db362851632a77e7a157d5a112f67e0e8d3c6ae31dd7a2d610fdc92defb1c59f720656a84e54f2ac207b79b2e695924e91925c99dbc808322cb4c2ceb101
|
data/.gitignore
CHANGED
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -27,6 +27,8 @@ Or install it yourself as:
|
|
27
27
|
|
28
28
|
## Usage
|
29
29
|
|
30
|
+
#### Rules
|
31
|
+
|
30
32
|
You should be declare an definition for you CSV, for each cells you should define what you expect.
|
31
33
|
|
32
34
|
Example :
|
@@ -35,7 +37,7 @@ You want first cell parsed should be string with values are 'yes' or 'no' you mu
|
|
35
37
|
|
36
38
|
{ name: 'aswering', type: 'string', values: ['yes', 'no'], position: [0,0] }
|
37
39
|
|
38
|
-
All keys as default
|
40
|
+
All keys as default values, so you can just define this rule :
|
39
41
|
|
40
42
|
{ name: 'aswering', values: ['yes', 'no'], position: [0,0] }
|
41
43
|
|
@@ -43,23 +45,148 @@ You can define message, default is 'undefined :key on :position'
|
|
43
45
|
|
44
46
|
{ name: 'aswering', values: ['yes', 'no'], position: [0,0], message: 'this value is not supported' }
|
45
47
|
|
48
|
+
You can also define Range
|
49
|
+
|
50
|
+
{ name: 'score', values: 0..5, position: [0,0] }
|
51
|
+
|
46
52
|
if you insert key on you message they will be substituted
|
47
53
|
|
48
54
|
{ ..., message: 'value of :name is not supported, please you one of :values' }
|
49
55
|
|
50
|
-
produce
|
51
|
-
|
56
|
+
produce :
|
57
|
+
|
58
|
+
|
59
|
+
value of aswering is not supported, please you one of [yes, no]
|
60
|
+
|
61
|
+
##### Position
|
62
|
+
|
63
|
+
Position mean [Y, X], where Y is rows, X columns
|
64
|
+
|
65
|
+
#### Definition
|
66
|
+
|
67
|
+
You should provide a definition, you have 2 types of definitions, mapping definition for search on x,y in your data or collection definition for rules apply for all lines in x, so you position rules should be only x value
|
68
|
+
|
69
|
+
### Sample
|
70
|
+
|
71
|
+
#### Mapping
|
72
|
+
|
73
|
+
Consider csv data like that
|
74
|
+
|
75
|
+
| Fields | Person Informations | Optional |
|
76
|
+
|-------------|----------------------|----------|
|
77
|
+
| Nickname | john | no |
|
78
|
+
| First Name | John | yes |
|
79
|
+
| Last Name | Doe | yes |
|
80
|
+
|
81
|
+
|
82
|
+
Mapping sample definition
|
83
|
+
|
84
|
+
class MyParser
|
85
|
+
|
86
|
+
attr_accessor :file_path
|
87
|
+
|
88
|
+
def initialize file_path
|
89
|
+
@file_path = file_path
|
90
|
+
end
|
91
|
+
|
92
|
+
def rules
|
93
|
+
[].tap do |mapping|
|
94
|
+
mapping << { position: [2,1], key: 'first_name' }
|
95
|
+
mapping << { position: [3,1], key: 'last_name' }
|
96
|
+
end
|
97
|
+
end
|
98
|
+
|
99
|
+
def definition
|
100
|
+
Definition.new(rules, type = Definition::MAPPING)
|
101
|
+
end
|
102
|
+
|
103
|
+
def data
|
104
|
+
Csv2hash.new(definition, file_path).tap do |csv2hash|
|
105
|
+
csv2hash.parse
|
106
|
+
end.data
|
107
|
+
end
|
108
|
+
|
109
|
+
end
|
110
|
+
|
111
|
+
#### Collection
|
112
|
+
|
113
|
+
Consider csv data like that
|
114
|
+
|
115
|
+
| Nickname | First Name | Last Name |
|
116
|
+
|----------|------------|-----------|
|
117
|
+
| john | John | Doe |
|
118
|
+
| jane | Jane | Doe |
|
119
|
+
|
120
|
+
Mapping sample definition
|
121
|
+
|
122
|
+
class MyParser
|
123
|
+
|
124
|
+
attr_accessor :file_path
|
125
|
+
|
126
|
+
def initialize file_path
|
127
|
+
@file_path = file_path
|
128
|
+
end
|
129
|
+
|
130
|
+
def rules
|
131
|
+
[].tap do |mapping|
|
132
|
+
mapping << { position: 0, key: 'nickname' }
|
133
|
+
mapping << { position: 1, key: 'first_name' }
|
134
|
+
mapping << { position: 2, key: 'last_name' }
|
135
|
+
end
|
136
|
+
end
|
137
|
+
|
138
|
+
def definition
|
139
|
+
Definition.new(rules, type = Definition::COLLECTION)
|
140
|
+
end
|
141
|
+
|
142
|
+
def data
|
143
|
+
Csv2hash.new(definition, file_path).tap do |csv2hash|
|
144
|
+
csv2hash.parse
|
145
|
+
end.data
|
146
|
+
end
|
147
|
+
|
148
|
+
end
|
149
|
+
|
150
|
+
#### Headers
|
151
|
+
|
152
|
+
You should be define header size
|
153
|
+
|
154
|
+
Definition.new(rules, type, header_size=0)
|
155
|
+
|
156
|
+
#### Exception or CSV mode
|
157
|
+
|
158
|
+
You can choice 2 mode of parsing, either exception mode for raise exception in first breaking rules or csv mode for get csv original data + errors throwing into added columns.
|
159
|
+
|
160
|
+
|
161
|
+
parse return data or csv_with_errors if parse is invalid, you can plug this like that :
|
162
|
+
|
163
|
+
csv2hash = Csv2hash.new(definition, 'file_path').new
|
164
|
+
result = csv2hash.parse
|
165
|
+
return result if csv2hash.valid?
|
166
|
+
|
167
|
+
filename = 'issues_errors.csv'
|
168
|
+
tempfile = Tempfile.new [filename, File.extname(filename)]
|
169
|
+
File.open(tempfile.path, 'wb') { |file| file.write result }
|
170
|
+
|
171
|
+
# Send mail with csv file + errors and free resource
|
172
|
+
|
173
|
+
tempfile.unlink
|
174
|
+
|
52
175
|
|
53
176
|
#### Default values
|
54
177
|
|
178
|
+
only position is require
|
179
|
+
|
180
|
+
* :position
|
181
|
+
|
182
|
+
all remaind keys are optionals
|
183
|
+
|
55
184
|
* message: 'undefined :key on :position'
|
56
185
|
* mappable: true
|
57
186
|
* type: 'string'
|
58
187
|
* values: nil
|
59
188
|
* nested: nil
|
60
189
|
* allow_blank: false
|
61
|
-
* position: nil
|
62
|
-
* maptype: 'cell'
|
63
190
|
|
64
191
|
### Limitations
|
65
192
|
|
data/bin/publish
ADDED
@@ -0,0 +1,24 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
# bin/publish 0.0.1
|
4
|
+
|
5
|
+
class Publish
|
6
|
+
|
7
|
+
def start version
|
8
|
+
system "bundle && bundle exec rake spec"
|
9
|
+
system "gem build csv2hash.gemspec"
|
10
|
+
system "git tag -a v#{version} -m 'version #{version}'"
|
11
|
+
system "git push --tags"
|
12
|
+
system "gem push csv2hash-#{version}.gem"
|
13
|
+
system "git push origin master"
|
14
|
+
end
|
15
|
+
|
16
|
+
end
|
17
|
+
|
18
|
+
if ARGV.length != 1 or !ARGV[0].match(/\d{1,3}.\d{1,3}.\d{1,3}/)
|
19
|
+
puts 'HELP: '
|
20
|
+
puts '$ bin/publish 0.0.1'
|
21
|
+
exit 0
|
22
|
+
end
|
23
|
+
|
24
|
+
Publish.new.start ARGV[0]
|
data/csv2hash.gemspec
CHANGED
data/lib/csv2hash/definition.rb
CHANGED
@@ -5,7 +5,11 @@ class Definition
|
|
5
5
|
|
6
6
|
TYPES = [Definition::MAPPING, Definition::COLLECTION]
|
7
7
|
|
8
|
-
attr_accessor :type, :
|
8
|
+
attr_accessor :rules, :type, :header_size
|
9
|
+
|
10
|
+
def initialize rules, type, header_size=0
|
11
|
+
@rules, @type, @header_size = rules, type, header_size
|
12
|
+
end
|
9
13
|
|
10
14
|
def validate!
|
11
15
|
unless TYPES.include?(type)
|
@@ -17,14 +21,18 @@ class Definition
|
|
17
21
|
def default!
|
18
22
|
rules.each do |rule|
|
19
23
|
default_position rule
|
20
|
-
|
24
|
+
unless rule.has_key? :message
|
25
|
+
if rule.has_key? :values
|
26
|
+
rule.merge! message: ':key not supported, please use one of :values'
|
27
|
+
else
|
28
|
+
rule.merge! message: 'undefined :key on :position'
|
29
|
+
end
|
30
|
+
end
|
21
31
|
rule.merge! mappable: true unless rule.has_key? :mappable
|
22
32
|
rule.merge! type: 'string' unless rule.has_key? :type
|
23
33
|
rule.merge! values: nil unless rule.has_key? :values
|
24
34
|
rule.merge! nested: nil unless rule.has_key? :nested
|
25
35
|
rule.merge! allow_blank: false unless rule.has_key? :allow_blank
|
26
|
-
rule.merge! position: nil unless rule.has_key? :position
|
27
|
-
rule.merge! maptype: 'cell' unless rule.has_key? :maptype
|
28
36
|
end
|
29
37
|
end
|
30
38
|
|
@@ -33,11 +41,11 @@ class Definition
|
|
33
41
|
def default_position rule
|
34
42
|
case type
|
35
43
|
when Definition::MAPPING
|
36
|
-
|
37
|
-
rule.merge! key: "key_#{
|
44
|
+
y, x = rule.fetch(:position, ['undefined', 'undefined'])
|
45
|
+
rule.merge! key: "key_#{y}_#{x}" unless rule.has_key? :key
|
38
46
|
when Definition::COLLECTION
|
39
|
-
|
40
|
-
rule.merge! key: "
|
47
|
+
x = rule.fetch :position
|
48
|
+
rule.merge! key: "key_undefined_#{x}" unless rule.has_key? :key
|
41
49
|
end
|
42
50
|
end
|
43
51
|
|
@@ -0,0 +1,29 @@
|
|
1
|
+
module Parser::Collection extend Parser
|
2
|
+
|
3
|
+
def fill!
|
4
|
+
@data = {}.tap do |data_computed|
|
5
|
+
data_computed[:data] ||= []
|
6
|
+
@data_source.each_with_index do |line, y|
|
7
|
+
next if y < definition.header_size
|
8
|
+
data_computed[:data] << {}.tap do |data_parsed|
|
9
|
+
fill_it data_parsed, line
|
10
|
+
end
|
11
|
+
end
|
12
|
+
end
|
13
|
+
end
|
14
|
+
|
15
|
+
def fill_it parsed_data, source_data
|
16
|
+
definition.rules.each do |rule|
|
17
|
+
if rule.fetch :mappable
|
18
|
+
x = rule.fetch :position
|
19
|
+
if (nested = rule.fetch :nested)
|
20
|
+
parsed_data[nested] ||= {}
|
21
|
+
parsed_data[nested][rule.fetch(:key)] = source_data[x]
|
22
|
+
else
|
23
|
+
parsed_data[rule.fetch(:key)] = source_data[x]
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
end
|
@@ -0,0 +1,26 @@
|
|
1
|
+
module Parser::Mapping extend Parser
|
2
|
+
|
3
|
+
def fill!
|
4
|
+
@data = {}.tap do |data_computed|
|
5
|
+
data_computed[:data] ||= []
|
6
|
+
data_computed[:data] << {}.tap do |data_parsed|
|
7
|
+
fill_it data_parsed, data_source
|
8
|
+
end
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
def fill_it parsed_data, source_data
|
13
|
+
definition.rules.each do |rule|
|
14
|
+
if rule.fetch :mappable
|
15
|
+
y, x = rule.fetch :position
|
16
|
+
if (nested = rule.fetch :nested)
|
17
|
+
parsed_data[nested] ||= {}
|
18
|
+
parsed_data[nested][rule.fetch(:key)] = source_data[y][x]
|
19
|
+
else
|
20
|
+
parsed_data[rule.fetch(:key)] = source_data[y][x]
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
end
|
data/lib/csv2hash/parser.rb
CHANGED
@@ -1,24 +1,2 @@
|
|
1
1
|
module Parser
|
2
|
-
|
3
|
-
def fill!
|
4
|
-
@data = {}.tap do |data_computed|
|
5
|
-
data_computed[:data] ||= []
|
6
|
-
data_computed[:data] << {}.tap do |data_parsed|
|
7
|
-
|
8
|
-
definition.rules.each do |rule|
|
9
|
-
if rule.fetch :mappable
|
10
|
-
x, y = rule.fetch :position
|
11
|
-
if (nested = rule.fetch :nested)
|
12
|
-
data_parsed[nested] ||= {}
|
13
|
-
data_parsed[nested][rule.fetch(:key)] = data_source[x][y]
|
14
|
-
else
|
15
|
-
data_parsed[rule.fetch(:key)] = data_source[x][y]
|
16
|
-
end
|
17
|
-
end
|
18
|
-
end
|
19
|
-
|
20
|
-
end
|
21
|
-
end
|
22
|
-
end
|
23
|
-
|
24
2
|
end
|
@@ -0,0 +1,17 @@
|
|
1
|
+
module Validator::Collection
|
2
|
+
include Validator
|
3
|
+
|
4
|
+
def validate_data!
|
5
|
+
@data_source.each_with_index do |line, y|
|
6
|
+
next if y < definition.header_size
|
7
|
+
validate_rules y
|
8
|
+
end
|
9
|
+
end
|
10
|
+
|
11
|
+
protected
|
12
|
+
|
13
|
+
def position _position
|
14
|
+
[nil, _position]
|
15
|
+
end
|
16
|
+
|
17
|
+
end
|
data/lib/csv2hash/validator.rb
CHANGED
@@ -1,34 +1,42 @@
|
|
1
1
|
module Validator
|
2
|
-
|
3
|
-
|
2
|
+
|
3
|
+
attr_accessor :errors, :exception
|
4
|
+
|
5
|
+
def validate_rules y=nil
|
4
6
|
definition.rules.each do |rule|
|
5
|
-
|
6
|
-
|
7
|
-
|
7
|
+
_y, x = position rule.fetch(:position)
|
8
|
+
begin
|
9
|
+
validate_cell (_y||y), x, rule
|
10
|
+
rescue => e
|
11
|
+
errors << { y: (_y||y), x: x, message: e.message, key: rule.fetch(:key) }
|
12
|
+
raise if exception
|
8
13
|
end
|
9
14
|
end
|
10
15
|
end
|
11
16
|
|
12
|
-
|
17
|
+
def valid?() errors.empty?; end
|
18
|
+
|
19
|
+
protected
|
20
|
+
|
21
|
+
def validate_cell y, x, rule
|
22
|
+
|
23
|
+
value = data_source[y][x] rescue nil
|
13
24
|
|
14
|
-
def validate_rule x, y, rule
|
15
25
|
begin
|
16
|
-
raise unless
|
17
|
-
|
18
|
-
raise unless
|
19
|
-
end
|
20
|
-
if (values = rule.fetch :values)
|
21
|
-
raise unless values.include?(data_source[x][y])
|
26
|
+
raise unless value unless rule.fetch :allow_blank
|
27
|
+
if value && (values = rule.fetch :values)
|
28
|
+
raise unless values.include?(value)
|
22
29
|
end
|
23
30
|
rescue => e
|
24
|
-
raise message(rule)
|
31
|
+
raise message(rule, y, x)
|
25
32
|
end
|
26
33
|
end
|
27
34
|
|
28
|
-
def message rule
|
29
|
-
rule.fetch(:message).tap do |msg|
|
30
|
-
rule.each { |key, value| msg.gsub!
|
35
|
+
def message rule, y, x
|
36
|
+
msg = rule.fetch(:message).tap do |msg|
|
37
|
+
rule.each { |key, value| msg.gsub!(":#{key.to_s}", value.to_s) unless key == :position }
|
31
38
|
end
|
39
|
+
msg.gsub ':position', "[#{y}, #{x}]"
|
32
40
|
end
|
33
|
-
|
41
|
+
|
34
42
|
end
|
data/lib/csv2hash/version.rb
CHANGED
data/lib/csv2hash.rb
CHANGED
@@ -1,24 +1,73 @@
|
|
1
1
|
require 'csv2hash/version'
|
2
2
|
require 'csv2hash/definition'
|
3
|
-
require 'csv2hash/definition/mapping'
|
4
3
|
require 'csv2hash/validator'
|
4
|
+
require 'csv2hash/validator/mapping'
|
5
|
+
require 'csv2hash/validator/collection'
|
5
6
|
require 'csv2hash/parser'
|
7
|
+
require 'csv2hash/parser/mapping'
|
8
|
+
require 'csv2hash/parser/collection'
|
9
|
+
require 'csv2hash/csv_array'
|
10
|
+
require 'csv'
|
6
11
|
|
7
12
|
class Csv2hash
|
8
|
-
include Validator
|
9
|
-
include Parser
|
10
13
|
|
11
|
-
attr_accessor :definition, :
|
14
|
+
attr_accessor :definition, :file_path, :data, :data_source
|
12
15
|
|
13
|
-
def initialize definition,
|
14
|
-
@definition, @
|
16
|
+
def initialize definition, file_path, exception=true
|
17
|
+
@definition, @file_path = definition, file_path
|
18
|
+
dynamic_parser_loading
|
19
|
+
@exception, @errors = exception, []
|
20
|
+
dynamic_validator_loading
|
15
21
|
end
|
16
22
|
|
17
23
|
def parse
|
24
|
+
load_data_source
|
18
25
|
definition.validate!
|
19
26
|
definition.default!
|
20
27
|
validate_data!
|
21
|
-
|
28
|
+
if valid?
|
29
|
+
fill!
|
30
|
+
data
|
31
|
+
else
|
32
|
+
csv_with_errors
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
def csv_with_errors
|
37
|
+
@csv_with_errors ||= begin
|
38
|
+
CsvArray.new.tap do |rows|
|
39
|
+
errors.each do |error|
|
40
|
+
rows << (([data_source[error[:x]][error[:y]]]||[nil]) + [error[:message]])
|
41
|
+
end
|
42
|
+
end.to_csv
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
# protected
|
47
|
+
|
48
|
+
def data_source
|
49
|
+
@data_source ||= CSV.read @file_path
|
50
|
+
end
|
51
|
+
alias_method :load_data_source, :data_source
|
52
|
+
|
53
|
+
private
|
54
|
+
|
55
|
+
def dynamic_validator_loading
|
56
|
+
case definition.type
|
57
|
+
when Definition::MAPPING
|
58
|
+
self.extend Validator::Mapping
|
59
|
+
when Definition::COLLECTION
|
60
|
+
self.extend Validator::Collection
|
61
|
+
end
|
62
|
+
end
|
63
|
+
|
64
|
+
def dynamic_parser_loading
|
65
|
+
case definition.type
|
66
|
+
when Definition::MAPPING
|
67
|
+
self.extend Parser::Mapping
|
68
|
+
when Definition::COLLECTION
|
69
|
+
self.extend Parser::Collection
|
70
|
+
end
|
22
71
|
end
|
23
72
|
|
24
73
|
end
|
@@ -4,14 +4,10 @@ describe Definition do
|
|
4
4
|
|
5
5
|
context 'regular context' do
|
6
6
|
subject do
|
7
|
-
Definition.new
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
{ position: [0,0], key: 'name' }
|
12
|
-
]
|
13
|
-
end
|
14
|
-
end
|
7
|
+
Definition.new(
|
8
|
+
[ { position: [0,0], key: 'name' } ],
|
9
|
+
Definition::MAPPING
|
10
|
+
)
|
15
11
|
end
|
16
12
|
|
17
13
|
it 'variable should be assigned' do
|
@@ -23,9 +19,7 @@ describe Definition do
|
|
23
19
|
describe '#validate!' do
|
24
20
|
context 'rules failling validation' do
|
25
21
|
subject do
|
26
|
-
Definition.new
|
27
|
-
definition.type = 'unsuitable_type'
|
28
|
-
end
|
22
|
+
Definition.new nil, 'unsuitable_type'
|
29
23
|
end
|
30
24
|
it 'should throw exception' do
|
31
25
|
expect {
|
@@ -35,10 +29,7 @@ describe Definition do
|
|
35
29
|
end
|
36
30
|
context 'rules failling validation' do
|
37
31
|
subject do
|
38
|
-
Definition.new
|
39
|
-
definition.type = Definition::MAPPING
|
40
|
-
definition.rules = 'rules'
|
41
|
-
end
|
32
|
+
Definition.new 'rules',Definition::MAPPING
|
42
33
|
end
|
43
34
|
it 'should throw exception' do
|
44
35
|
expect { subject.validate! }.to raise_error 'rules must be an Array of rules'
|
@@ -48,14 +39,7 @@ describe Definition do
|
|
48
39
|
|
49
40
|
describe '#default!' do
|
50
41
|
subject do
|
51
|
-
Definition.new
|
52
|
-
definition.type = Definition::MAPPING
|
53
|
-
definition.rules = begin
|
54
|
-
[
|
55
|
-
{ position: [0,0], key: 'name' }
|
56
|
-
]
|
57
|
-
end
|
58
|
-
end
|
42
|
+
Definition.new [ { position: [0,0], key: 'name' } ], Definition::MAPPING
|
59
43
|
end
|
60
44
|
|
61
45
|
before { subject.default! }
|
@@ -68,8 +52,7 @@ describe Definition do
|
|
68
52
|
type: 'string',
|
69
53
|
values: nil,
|
70
54
|
nested: nil,
|
71
|
-
allow_blank: false
|
72
|
-
maptype: 'cell' }])
|
55
|
+
allow_blank: false }])
|
73
56
|
end
|
74
57
|
end
|
75
58
|
end
|
@@ -0,0 +1,49 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Parser::Collection do
|
4
|
+
|
5
|
+
let(:definition) do
|
6
|
+
Definition.new [ { position: 0, key: 'name' } ], Definition::COLLECTION
|
7
|
+
end
|
8
|
+
|
9
|
+
let(:data_source) { [ [ 'John Doe' ], [ 'Jane Doe' ] ] }
|
10
|
+
|
11
|
+
subject do
|
12
|
+
Csv2hash.new(definition, 'file_path').tap do |csv2hash|
|
13
|
+
csv2hash.data_source = data_source
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
context 'regular way' do
|
18
|
+
it { expect { subject.parse }.to_not raise_error }
|
19
|
+
it {
|
20
|
+
subject.tap do |csv2hash|
|
21
|
+
csv2hash.parse
|
22
|
+
end.data.should eql({ data: [ { 'name' => 'John Doe' }, { 'name' => 'Jane Doe' } ] })
|
23
|
+
}
|
24
|
+
context 'with header' do
|
25
|
+
before { subject.definition.header_size = 1 }
|
26
|
+
let(:data_source) { [ [ 'Name' ], [ 'John Doe' ], [ 'Jane Doe' ] ]}
|
27
|
+
it {
|
28
|
+
subject.tap { |c| c.parse }.data.should eql({ data: [ { 'name' => 'John Doe' }, { 'name' => 'Jane Doe' } ] })
|
29
|
+
}
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
context 'with nested' do
|
34
|
+
let(:data_source) { [ [ 'John Doe', 22 ], [ 'Jane Doe', 19 ] ] }
|
35
|
+
before do
|
36
|
+
definition.rules << { position: 1, key: 'age', nested: 'infos' }
|
37
|
+
end
|
38
|
+
it {
|
39
|
+
subject.tap { |c| c.parse }.data.should eql(
|
40
|
+
{ data: [
|
41
|
+
{ 'name' => 'John Doe', 'infos' => { 'age' => 22 } },
|
42
|
+
{ 'name' => 'Jane Doe', 'infos' => { 'age' => 19 } }
|
43
|
+
]
|
44
|
+
}
|
45
|
+
)
|
46
|
+
}
|
47
|
+
end
|
48
|
+
|
49
|
+
end
|
@@ -0,0 +1,38 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Parser::Mapping do
|
4
|
+
|
5
|
+
let(:definition) do
|
6
|
+
Definition.new [ { position: [0,0], key: 'name' } ], Definition::MAPPING
|
7
|
+
end
|
8
|
+
|
9
|
+
let(:data_source) { [ [ 'John Doe' ] ] }
|
10
|
+
|
11
|
+
subject do
|
12
|
+
Csv2hash.new(definition, 'file_path').tap do |csv2hash|
|
13
|
+
csv2hash.data_source = data_source
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
context 'regular way' do
|
18
|
+
it { expect { subject.parse }.to_not raise_error }
|
19
|
+
it {
|
20
|
+
subject.tap do |csv2hash|
|
21
|
+
csv2hash.parse
|
22
|
+
end.data.should eql({ data: [ { 'name' => 'John Doe' } ] })
|
23
|
+
}
|
24
|
+
end
|
25
|
+
|
26
|
+
context 'with nested' do
|
27
|
+
let(:data_source) { [ [ 'John Doe', 22 ] ] }
|
28
|
+
before do
|
29
|
+
definition.rules << { position: [0,1], key: 'age', nested: 'infos' }
|
30
|
+
end
|
31
|
+
it {
|
32
|
+
subject.tap { |c| c.parse }.data.should eql(
|
33
|
+
{ data: [ { 'name' => 'John Doe', 'infos' => { 'age' => 22 } } ] }
|
34
|
+
)
|
35
|
+
}
|
36
|
+
end
|
37
|
+
|
38
|
+
end
|
@@ -1,39 +1,3 @@
|
|
1
1
|
require 'spec_helper'
|
2
2
|
|
3
|
-
describe Parser
|
4
|
-
|
5
|
-
let(:definition) do
|
6
|
-
Definition.new.tap do |definition|
|
7
|
-
definition.type = Definition::MAPPING
|
8
|
-
definition.rules = [ { position: [0,0], key: 'name' } ]
|
9
|
-
definition.validate!
|
10
|
-
definition.default!
|
11
|
-
end
|
12
|
-
end
|
13
|
-
|
14
|
-
let(:data_source) { [ [ 'John Doe' ] ] }
|
15
|
-
|
16
|
-
subject { Csv2hash.new definition, data_source }
|
17
|
-
|
18
|
-
context 'regular way' do
|
19
|
-
it { expect { subject.parse }.to_not raise_error }
|
20
|
-
it {
|
21
|
-
subject.tap do |csv2hash|
|
22
|
-
csv2hash.parse
|
23
|
-
end.data.should eql({ data: [ { 'name' => 'John Doe' } ] })
|
24
|
-
}
|
25
|
-
end
|
26
|
-
|
27
|
-
context 'with nested' do
|
28
|
-
let(:data_source) { [ [ 'John Doe', 22 ] ] }
|
29
|
-
before do
|
30
|
-
definition.rules << { position: [0,1], key: 'age', nested: 'infos' }
|
31
|
-
end
|
32
|
-
it {
|
33
|
-
subject.tap { |c| c.parse }.data.should eql(
|
34
|
-
{ data: [ { 'name' => 'John Doe', 'infos' => { 'age' => 22 } } ] }
|
35
|
-
)
|
36
|
-
}
|
37
|
-
end
|
38
|
-
|
39
|
-
end
|
3
|
+
describe Parser
|
@@ -0,0 +1,59 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Validator::Collection do
|
4
|
+
|
5
|
+
let(:definition) do
|
6
|
+
Definition.new([ { position: 0, key: 'name' } ], Definition::COLLECTION).tap do |definition|
|
7
|
+
definition.validate!
|
8
|
+
definition.default!
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
subject do
|
13
|
+
Csv2hash.new(definition, 'file_path').tap do |csv2hash|
|
14
|
+
csv2hash.data_source = data_source
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
context 'with valid data' do
|
19
|
+
let(:data_source) { [ [ 'John Doe' ] ]}
|
20
|
+
it { expect { subject.validate_data! }.to_not raise_error }
|
21
|
+
context 'with header' do
|
22
|
+
before { subject.definition.header_size = 1 }
|
23
|
+
let(:data_source) { [ [ 'Name' ], [ 'John Doe' ] ]}
|
24
|
+
it { expect { subject.validate_data! }.to_not raise_error }
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
context 'with invalid data' do
|
29
|
+
let(:data_source) { [ [ ] ]}
|
30
|
+
it { expect { subject.validate_data! }.to raise_error('undefined name on [0, 0]') }
|
31
|
+
context 'with header' do
|
32
|
+
before { subject.definition.header_size = 1 }
|
33
|
+
let(:data_source) { [ [ 'Name' ], [ ] ]}
|
34
|
+
it { expect { subject.validate_data! }.to raise_error('undefined name on [1, 0]') }
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
context 'wihtout exception' do
|
39
|
+
let(:data_source) { [ [ ] ]}
|
40
|
+
before { subject.exception = false }
|
41
|
+
it { subject.parse.should eql ",\"undefined name on [0, 0]\"\n" }
|
42
|
+
|
43
|
+
context 'errors should be filled' do
|
44
|
+
before { subject.parse }
|
45
|
+
its(:errors) { should eql [{x: 0, y: 0, message: 'undefined name on [0, 0]', key: 'name'}] }
|
46
|
+
end
|
47
|
+
|
48
|
+
context 'original csv + errors should returned' do
|
49
|
+
let(:definition) do
|
50
|
+
Definition.new([ { position: 0, key: 'agree', values: ['yes', 'no'] } ], Definition::COLLECTION).tap do |d|
|
51
|
+
d.validate!; d.default!
|
52
|
+
end
|
53
|
+
end
|
54
|
+
let(:data_source) { [ [ 'what?' ], [ 'yes' ], [ 'no' ] ] }
|
55
|
+
it { subject.parse.should eql "what?,\"agree not supported, please use one of [\"\"yes\"\", \"\"no\"\"]\"\n" }
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
59
|
+
end
|
@@ -0,0 +1,68 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Validator::Mapping do
|
4
|
+
|
5
|
+
let(:definition) do
|
6
|
+
Definition.new([ { position: [0,0], key: 'name' } ], Definition::MAPPING).tap do |definition|
|
7
|
+
definition.validate!
|
8
|
+
definition.default!
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
subject do
|
13
|
+
Csv2hash.new(definition, 'file_path').tap do |csv2hash|
|
14
|
+
csv2hash.data_source = data_source
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
context 'with valid data' do
|
19
|
+
let(:data_source) { [ [ 'John Doe' ] ]}
|
20
|
+
it { expect { subject.validate_data! }.to_not raise_error }
|
21
|
+
end
|
22
|
+
|
23
|
+
context 'with invalid data' do
|
24
|
+
let(:data_source) { [ [ ] ]}
|
25
|
+
it { expect { subject.validate_data! }.to raise_error('undefined name on [0, 0]') }
|
26
|
+
end
|
27
|
+
|
28
|
+
context 'wihtout exception' do
|
29
|
+
let(:data_source) { [ [ ] ]}
|
30
|
+
before { subject.exception = false }
|
31
|
+
it { subject.parse.should eql ",\"undefined name on [0, 0]\"\n" }
|
32
|
+
|
33
|
+
context 'errors should be filled' do
|
34
|
+
before { subject.parse }
|
35
|
+
its(:errors) { should eql [{x: 0, y: 0, message: 'undefined name on [0, 0]', key: 'name'}] }
|
36
|
+
end
|
37
|
+
|
38
|
+
context 'original csv + errors should be returned' do
|
39
|
+
let(:definition) do
|
40
|
+
Definition.new(rules, Definition::MAPPING).tap do |d|
|
41
|
+
d.validate!; d.default!
|
42
|
+
end
|
43
|
+
end
|
44
|
+
context 'string values' do
|
45
|
+
let(:rules) do
|
46
|
+
[
|
47
|
+
{ position: [0,0], key: 'agree', values: ['yes', 'no'] },
|
48
|
+
{ position: [1,0], key: 'agree', values: ['yes', 'no'] },
|
49
|
+
{ position: [2,0], key: 'agree', values: ['yes', 'no'] }
|
50
|
+
]
|
51
|
+
end
|
52
|
+
let(:data_source) { [ [ 'what?' ], [ 'yes', 'what?' ], [ 'yes', 'what?', 'no' ] ] }
|
53
|
+
it { subject.parse.should eql "what?,\"agree not supported, please use one of [\"\"yes\"\", \"\"no\"\"]\"\n" }
|
54
|
+
end
|
55
|
+
context 'range values' do
|
56
|
+
let(:rules) do
|
57
|
+
[
|
58
|
+
{ position: [0,0], key: 'score', values: 1..10 },
|
59
|
+
{ position: [1,0], key: 'score', values: 1..10 },
|
60
|
+
{ position: [2,0], key: 'score', values: 1..10 }
|
61
|
+
]
|
62
|
+
end
|
63
|
+
let(:data_source) { [ [ 12 ], [ 2, 12 ], [ 3, 12, 1 ] ] }
|
64
|
+
it { subject.parse.should eql "12,\"score not supported, please use one of 1..10\"\n" }
|
65
|
+
end
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
@@ -3,36 +3,26 @@ require 'spec_helper'
|
|
3
3
|
describe Validator do
|
4
4
|
|
5
5
|
let(:definition) do
|
6
|
-
Definition.new.tap do |definition|
|
7
|
-
definition.type = Definition::MAPPING
|
8
|
-
definition.rules = [ { position: [0,0], key: 'name' } ]
|
6
|
+
Definition.new([ { position: [0,0], key: 'name' } ], Definition::MAPPING).tap do |definition|
|
9
7
|
definition.validate!
|
10
8
|
definition.default!
|
11
9
|
end
|
12
10
|
end
|
13
11
|
|
14
12
|
subject do
|
15
|
-
Csv2hash.new
|
16
|
-
|
17
|
-
|
18
|
-
context 'with valid data' do
|
19
|
-
let(:data_source) { [ [ 'John Doe' ] ]}
|
20
|
-
it { expect { subject.validate_data! }.to_not raise_error }
|
21
|
-
end
|
22
|
-
|
23
|
-
context 'with invalid data' do
|
24
|
-
let(:data_source) { [ [ ] ]}
|
25
|
-
it { expect { subject.validate_data! }.to raise_error('undefined name on [0, 0]') }
|
13
|
+
Csv2hash.new(definition, 'file_path').tap do |csv2hash|
|
14
|
+
csv2hash.data_source = data_source
|
15
|
+
end
|
26
16
|
end
|
27
17
|
|
28
18
|
describe '#message' do
|
29
|
-
subject { Csv2hash.new
|
19
|
+
subject { Csv2hash.new double('definition', type: Definition::COLLECTION), nil }
|
30
20
|
|
31
21
|
context 'string value' do
|
32
22
|
let(:rule) { { foo: 'bar', message: ':foo are value of foo key' } }
|
33
23
|
|
34
24
|
it 'substitue value of key' do
|
35
|
-
subject.send(:message, rule).should eql 'bar are value of foo key'
|
25
|
+
subject.send(:message, rule, nil, nil).should eql 'bar are value of foo key'
|
36
26
|
end
|
37
27
|
end
|
38
28
|
|
@@ -40,8 +30,17 @@ describe Validator do
|
|
40
30
|
let(:rule) { { foo: ['bar', 'zone'], message: ':foo are values of foo key' } }
|
41
31
|
|
42
32
|
it 'substitue value of key' do
|
43
|
-
subject.send(:message, rule).should eql '["bar", "zone"] are values of foo key'
|
33
|
+
subject.send(:message, rule, nil, nil).should eql '["bar", "zone"] are values of foo key'
|
34
|
+
end
|
35
|
+
end
|
36
|
+
|
37
|
+
context 'with position' do
|
38
|
+
let(:rule) { { message: 'value not found on :position' } }
|
39
|
+
|
40
|
+
it 'substitue value of key' do
|
41
|
+
subject.send(:message, rule, 0, 2).should eql 'value not found on [0, 2]'
|
44
42
|
end
|
45
43
|
end
|
46
44
|
end
|
47
|
-
|
45
|
+
|
46
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: csv2hash
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Joel AZEMAR
|
@@ -56,6 +56,7 @@ description: DSL for CSV Ruby Hash mapping
|
|
56
56
|
email: joel.azemar@gmail.com
|
57
57
|
executables:
|
58
58
|
- load_rvm
|
59
|
+
- publish
|
59
60
|
extensions: []
|
60
61
|
extra_rdoc_files: []
|
61
62
|
files:
|
@@ -70,15 +71,24 @@ files:
|
|
70
71
|
- README.md
|
71
72
|
- Rakefile
|
72
73
|
- bin/load_rvm
|
74
|
+
- bin/publish
|
73
75
|
- csv2hash.gemspec
|
74
76
|
- lib/csv2hash.rb
|
77
|
+
- lib/csv2hash/csv_array.rb
|
75
78
|
- lib/csv2hash/definition.rb
|
76
|
-
- lib/csv2hash/definition/mapping.rb
|
77
79
|
- lib/csv2hash/parser.rb
|
80
|
+
- lib/csv2hash/parser/collection.rb
|
81
|
+
- lib/csv2hash/parser/mapping.rb
|
78
82
|
- lib/csv2hash/validator.rb
|
83
|
+
- lib/csv2hash/validator/collection.rb
|
84
|
+
- lib/csv2hash/validator/mapping.rb
|
79
85
|
- lib/csv2hash/version.rb
|
80
86
|
- spec/csv2hash/definition_spec.rb
|
87
|
+
- spec/csv2hash/parser/collection_spec.rb
|
88
|
+
- spec/csv2hash/parser/mapping_spec.rb
|
81
89
|
- spec/csv2hash/parser_spec.rb
|
90
|
+
- spec/csv2hash/validator/collection_spec.rb
|
91
|
+
- spec/csv2hash/validator/mapping_spec.rb
|
82
92
|
- spec/csv2hash/validator_spec.rb
|
83
93
|
- spec/csv2hash_spec.rb
|
84
94
|
- spec/spec_helper.rb
|
@@ -108,7 +118,11 @@ specification_version: 4
|
|
108
118
|
summary: Mapping CSV to Ruby Hash
|
109
119
|
test_files:
|
110
120
|
- spec/csv2hash/definition_spec.rb
|
121
|
+
- spec/csv2hash/parser/collection_spec.rb
|
122
|
+
- spec/csv2hash/parser/mapping_spec.rb
|
111
123
|
- spec/csv2hash/parser_spec.rb
|
124
|
+
- spec/csv2hash/validator/collection_spec.rb
|
125
|
+
- spec/csv2hash/validator/mapping_spec.rb
|
112
126
|
- spec/csv2hash/validator_spec.rb
|
113
127
|
- spec/csv2hash_spec.rb
|
114
128
|
- spec/spec_helper.rb
|