cascade-rb 0.2.3 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.hound.yml +1 -1
- data/.rubocop.yml +11 -0
- data/.travis.yml +11 -0
- data/Gemfile +3 -1
- data/README.md +2 -25
- data/Rakefile +7 -4
- data/cascade.gemspec +22 -21
- data/lib/cascade.rb +6 -7
- data/lib/cascade/columns_matching.rb +14 -13
- data/lib/cascade/complex_fields.rb +5 -3
- data/lib/cascade/complex_fields/array_processor.rb +2 -0
- data/lib/cascade/complex_fields/boolean.rb +3 -1
- data/lib/cascade/complex_fields/currency.rb +8 -4
- data/lib/cascade/concerns/statistics_collectible.rb +3 -1
- data/lib/cascade/data_parser.rb +4 -2
- data/lib/cascade/error_handler.rb +4 -7
- data/lib/cascade/exceptions.rb +5 -3
- data/lib/cascade/exceptions/unknown_presenter_type.rb +2 -0
- data/lib/cascade/exceptions/unsupported_component.rb +3 -1
- data/lib/cascade/exceptions/wrong_mapping_format.rb +2 -0
- data/lib/cascade/helpers/hash.rb +2 -0
- data/lib/cascade/registry.rb +6 -3
- data/lib/cascade/row_processor.rb +23 -22
- data/lib/cascade/statistics.rb +4 -2
- data/lib/cascade/statistics_stores.rb +5 -3
- data/lib/cascade/statistics_stores/abstract_store.rb +2 -0
- data/lib/cascade/statistics_stores/array_store.rb +2 -0
- data/lib/cascade/statistics_stores/counter_store.rb +2 -0
- data/lib/cascade/version.rb +3 -1
- data/spec/integration_spec.rb +70 -0
- data/spec/lib/columns_matching_spec.rb +18 -26
- data/spec/lib/complex_fields/array_processor_spec.rb +6 -4
- data/spec/lib/complex_fields/boolean_spec.rb +8 -6
- data/spec/lib/complex_fields/currency_spec.rb +11 -9
- data/spec/lib/concerns/statistics_collectible_spec.rb +6 -4
- data/spec/lib/data_parser_spec.rb +14 -11
- data/spec/lib/error_handler_spec.rb +21 -15
- data/spec/lib/exceptions/unknown_presenter_type_spec.rb +5 -3
- data/spec/lib/exceptions/unsupported_component_spec.rb +10 -8
- data/spec/lib/exceptions/wrong_mapping_format_spec.rb +5 -3
- data/spec/lib/helpers/hash_spec.rb +7 -5
- data/spec/lib/registry_spec.rb +9 -8
- data/spec/lib/row_processor_spec.rb +33 -33
- data/spec/lib/statistics_spec.rb +14 -12
- data/spec/lib/statistics_stores/abstract_store_spec.rb +7 -5
- data/spec/lib/statistics_stores/array_store_spec.rb +8 -6
- data/spec/lib/statistics_stores/counter_store_spec.rb +8 -6
- data/spec/spec_helper.rb +16 -15
- metadata +28 -28
- data/.ruby-style.yml +0 -1063
- data/lib/cascade/helpers/configuration.rb +0 -32
- data/spec/lib/helpers/configuration_spec.rb +0 -36
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 1fed47a0100266f90223d01c1bb3d44b0051793c
|
4
|
+
data.tar.gz: 8eef3a7b5d3ecd9f4da62a863d0b7c980ab1b828
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 2a61fe4e44402f142faa7b00dbd8c1dcde6296c75232668eb96fe09ed1de6d8f71bee6c87140febf0a969defbfff24dbcdaf833362c19e79a849599f3f98c395
|
7
|
+
data.tar.gz: 8505f5e2f46df476fc9f75bb31aaffd8307a018ffdd6b9658ee89b591cde724a64e67beb2d44d49a159174a62d655d17a9a2df89e1bcbb62c158d6cf4988428c
|
data/.hound.yml
CHANGED
@@ -1,2 +1,2 @@
|
|
1
1
|
ruby:
|
2
|
-
config_file: .
|
2
|
+
config_file: .rubocop.yml
|
data/.rubocop.yml
ADDED
data/.travis.yml
ADDED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# [Cascade]
|
2
2
|
|
3
|
-
[![
|
3
|
+
[![Build Status](https://travis-ci.com/ignat-z/cascade.svg?branch=master)](https://travis-ci.com/ignat-z/cascade) [![Gem Version](https://badge.fury.io/rb/cascade-rb.svg)](http://badge.fury.io/rb/cascade-rb)
|
4
4
|
|
5
5
|
The main goal of this gem is to provide some kind of template for parsing files.
|
6
6
|
Usually, file parsing process consists of the following steps:
|
@@ -15,12 +15,6 @@ Usually, file parsing process consists of the following steps:
|
|
15
15
|
|
16
16
|
Cascade pretends to simplify main part of this step to save your time.
|
17
17
|
|
18
|
-
## Examples
|
19
|
-
|
20
|
-
[Minimal working example](https://github.com/ignat-zakrevsky/cascade-example)
|
21
|
-
|
22
|
-
[More complicated example](https://github.com/ignat-zakrevsky/cascade-example/tree/json-example)
|
23
|
-
|
24
18
|
## Installation
|
25
19
|
Install the cascade-rb package from Rubygems:
|
26
20
|
```
|
@@ -38,20 +32,6 @@ Require gem files
|
|
38
32
|
require 'cascade'
|
39
33
|
```
|
40
34
|
|
41
|
-
Configure it!
|
42
|
-
```ruby
|
43
|
-
Cascade.configuration do
|
44
|
-
# [Object#call] will be used for undefined fields types
|
45
|
-
Cascade::RowProcessor.deafult_presenter
|
46
|
-
# [Boolean] will throw exception in case of unavailable presenter
|
47
|
-
Cascade::RowProcessor.use_default_presenter
|
48
|
-
# [String] filepath with columns mapping, see below
|
49
|
-
Cascade::ColumnsMatching.mapping_file
|
50
|
-
# [Boolean] will raise fields parsing exceptions
|
51
|
-
Cascade::ErrorHandler.raise_parse_errors
|
52
|
-
end
|
53
|
-
```
|
54
|
-
|
55
35
|
Provide enumerable object for parsing and run it!
|
56
36
|
```ruby
|
57
37
|
Cascade::DataParser.new(data_provider: Csv.open("data_test.csv")).call
|
@@ -111,11 +91,8 @@ Provide all this stuff into data parser
|
|
111
91
|
```ruby
|
112
92
|
Cascade::DataParser.new(
|
113
93
|
data_provider: ParserJSON.new.open("data_test.csv"),
|
114
|
-
row_processor: Cascade::RowProcessor.new(date: DateParser.new),
|
94
|
+
row_processor: Cascade::RowProcessor.new(ext_parsers: { date: DateParser.new }),
|
115
95
|
data_saver: PERSON_SAVER
|
116
96
|
).call
|
117
97
|
```
|
118
98
|
And that's all!
|
119
|
-
[Example](https://github.com/ignat-zakrevsky/cascade-example/blob/json-example/main.rb)
|
120
|
-
## Conventions
|
121
|
-
I'm fan of callable object as consequence I prefer #call methods for classes with one responsibility. [Nice video](http://www.rubytapas.com/episodes/35-Callable) that describes benefits of such approach
|
data/Rakefile
CHANGED
@@ -1,9 +1,12 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'rake/testtask'
|
4
|
+
require 'bundler/gem_tasks'
|
2
5
|
|
3
6
|
Rake::TestTask.new do |task|
|
4
|
-
task.libs <<
|
5
|
-
task.libs <<
|
6
|
-
task.pattern =
|
7
|
+
task.libs << 'lib'
|
8
|
+
task.libs << 'spec'
|
9
|
+
task.pattern = 'spec/**/*_spec.rb'
|
7
10
|
end
|
8
11
|
|
9
12
|
task default: [:test]
|
data/cascade.gemspec
CHANGED
@@ -1,31 +1,32 @@
|
|
1
|
-
#
|
2
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
lib = File.expand_path('lib', __dir__)
|
3
4
|
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
-
require
|
5
|
+
require 'cascade/version'
|
5
6
|
|
6
7
|
Gem::Specification.new do |spec|
|
7
|
-
spec.name =
|
8
|
+
spec.name = 'cascade-rb'
|
8
9
|
spec.version = Cascade::VERSION
|
9
|
-
spec.authors = [
|
10
|
-
spec.email = %w
|
11
|
-
spec.summary =
|
12
|
-
spec.description =
|
13
|
-
spec.homepage =
|
14
|
-
spec.license =
|
10
|
+
spec.authors = ['Ignat Zakrevsky']
|
11
|
+
spec.email = %w[iezakrevsky@gmail.com]
|
12
|
+
spec.summary = 'Ruby data parser gem.'
|
13
|
+
spec.description = 'Highly customizable data parser with a lot of DI'
|
14
|
+
spec.homepage = 'https://github.com/ignat-zakrevsky/cascade'
|
15
|
+
spec.license = 'MIT'
|
15
16
|
|
16
17
|
spec.files = `git ls-files -z`.split("\x0")
|
17
18
|
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
18
19
|
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
19
|
-
spec.require_paths = [
|
20
|
+
spec.require_paths = ['lib']
|
20
21
|
|
21
|
-
spec.add_development_dependency
|
22
|
-
spec.add_development_dependency
|
23
|
-
spec.add_development_dependency
|
24
|
-
spec.add_development_dependency
|
25
|
-
spec.add_development_dependency
|
26
|
-
spec.add_development_dependency
|
27
|
-
spec.add_development_dependency
|
28
|
-
spec.add_development_dependency
|
29
|
-
spec.add_development_dependency
|
30
|
-
spec.add_development_dependency
|
22
|
+
spec.add_development_dependency 'bundler'
|
23
|
+
spec.add_development_dependency 'codeclimate-test-reporter'
|
24
|
+
spec.add_development_dependency 'minitest'
|
25
|
+
spec.add_development_dependency 'pry'
|
26
|
+
spec.add_development_dependency 'rake'
|
27
|
+
spec.add_development_dependency 'rr'
|
28
|
+
spec.add_development_dependency 'rubocop'
|
29
|
+
spec.add_development_dependency 'shoulda-matchers', '2.8.0'
|
30
|
+
spec.add_development_dependency 'simplecov'
|
31
|
+
spec.add_development_dependency 'yard'
|
31
32
|
end
|
data/lib/cascade.rb
CHANGED
@@ -1,11 +1,10 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require
|
4
|
-
require
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'cascade/version'
|
4
|
+
require 'cascade/columns_matching'
|
5
|
+
require 'cascade/row_processor'
|
5
6
|
|
6
7
|
# Base gem module
|
7
8
|
module Cascade
|
8
|
-
|
9
|
-
|
10
|
-
autoload :DataParser, "cascade/data_parser"
|
9
|
+
autoload :DataParser, 'cascade/data_parser'
|
11
10
|
end
|
@@ -1,42 +1,43 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'yaml'
|
2
4
|
require 'forwardable'
|
3
|
-
require
|
4
|
-
require "cascade/helpers/configuration"
|
5
|
+
require 'cascade/exceptions'
|
5
6
|
|
6
7
|
module Cascade
|
7
8
|
class ColumnsMatching
|
8
9
|
extend Forwardable
|
9
|
-
extend Configuration
|
10
10
|
|
11
|
-
|
11
|
+
ROOT_KEY = 'mapping'
|
12
12
|
|
13
13
|
def_delegator :supported_keys, :index
|
14
14
|
|
15
15
|
def initialize(options = {})
|
16
|
-
@
|
17
|
-
|
16
|
+
@content = options.fetch(:content) do
|
17
|
+
parse_content_file(options.fetch(:filepath))
|
18
|
+
end
|
18
19
|
end
|
19
20
|
|
20
21
|
# Defines set of possible keys that can be used for iterating through
|
21
|
-
# parsed line
|
22
|
+
# the parsed line
|
22
23
|
#
|
23
|
-
# @return [Array] of supported keys
|
24
|
+
# @return [Array] of the supported keys
|
24
25
|
def supported_keys
|
25
26
|
@supported_keys ||= @content.keys
|
26
27
|
end
|
27
28
|
|
28
29
|
# Presenter for passed key
|
29
30
|
#
|
30
|
-
# @return [Symbol] with
|
31
|
+
# @return [Symbol] with the corresponding value
|
31
32
|
def column_type(key)
|
32
33
|
@content[key].to_sym
|
33
34
|
end
|
34
35
|
|
35
36
|
private
|
36
37
|
|
37
|
-
def parse_content_file
|
38
|
-
content = YAML.load_file(
|
39
|
-
(content && content[
|
38
|
+
def parse_content_file(filepath)
|
39
|
+
content = YAML.load_file(filepath)
|
40
|
+
(content && content[ROOT_KEY]) || raise(Cascade::WrongMappingFormat)
|
40
41
|
end
|
41
42
|
end
|
42
43
|
end
|
@@ -1,3 +1,5 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'cascade/complex_fields/currency'
|
4
|
+
require 'cascade/complex_fields/boolean'
|
5
|
+
require 'cascade/complex_fields/array_processor'
|
@@ -1,5 +1,7 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'bigdecimal'
|
4
|
+
require 'bigdecimal/util'
|
3
5
|
|
4
6
|
module Cascade
|
5
7
|
module ComplexFields
|
@@ -12,11 +14,13 @@ module Cascade
|
|
12
14
|
private
|
13
15
|
|
14
16
|
def normalized_value(value)
|
15
|
-
String(value).tr(
|
17
|
+
String(value).tr(',', '.').tr(' ', '')
|
16
18
|
end
|
17
19
|
|
18
20
|
def valid?(value)
|
19
|
-
true if Float(value)
|
21
|
+
true if Float(value)
|
22
|
+
rescue StandardError
|
23
|
+
false
|
20
24
|
end
|
21
25
|
end
|
22
26
|
end
|
data/lib/cascade/data_parser.rb
CHANGED
@@ -1,12 +1,8 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
2
|
|
3
3
|
module Cascade
|
4
4
|
class ErrorHandler
|
5
|
-
|
6
|
-
|
7
|
-
define_setting :raise_parse_errors, false
|
8
|
-
|
9
|
-
HANDLING_EXCEPTIONS = [IndexError]
|
5
|
+
HANDLING_EXCEPTIONS = [IndexError].freeze
|
10
6
|
DEFAULT_ERROR_STORE = lambda do |row, exception|
|
11
7
|
@errors ||= []
|
12
8
|
@errors << [row, exception.to_s]
|
@@ -14,6 +10,7 @@ module Cascade
|
|
14
10
|
|
15
11
|
def initialize(options = {})
|
16
12
|
@error_store = options.fetch(:error_store) { DEFAULT_ERROR_STORE }
|
13
|
+
@raise_parse_errors = options.fetch(:raise_parse_errors, false)
|
17
14
|
@handling_exceptions = options.fetch(:handling_exceptions) do
|
18
15
|
HANDLING_EXCEPTIONS
|
19
16
|
end
|
@@ -27,7 +24,7 @@ module Cascade
|
|
27
24
|
yield
|
28
25
|
rescue *@handling_exceptions => exception
|
29
26
|
@error_store.call(row, exception)
|
30
|
-
raise exception if
|
27
|
+
raise exception if @raise_parse_errors
|
31
28
|
end
|
32
29
|
end
|
33
30
|
end
|
data/lib/cascade/exceptions.rb
CHANGED
@@ -1,3 +1,5 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'cascade/exceptions/wrong_mapping_format'
|
4
|
+
require 'cascade/exceptions/unknown_presenter_type'
|
5
|
+
require 'cascade/exceptions/unsupported_component'
|
@@ -1,6 +1,8 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module Cascade
|
2
4
|
class UnsupportedComponent < ::StandardError
|
3
|
-
MESSAGE =
|
5
|
+
MESSAGE = 'You should provide data provider that respond to `each` method'
|
4
6
|
|
5
7
|
def initialize(msg = MESSAGE)
|
6
8
|
super
|
data/lib/cascade/helpers/hash.rb
CHANGED
data/lib/cascade/registry.rb
CHANGED
@@ -1,12 +1,15 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'cascade/error_handler'
|
4
|
+
require 'cascade/row_processor'
|
3
5
|
|
4
6
|
module Cascade
|
5
7
|
module Registry
|
6
8
|
PUTS_DATA_SAVER = ->(*args) { p args }
|
9
|
+
SELF_PROCESSOR = ->(row) { row }
|
7
10
|
|
8
11
|
def self.row_processor
|
9
|
-
|
12
|
+
SELF_PROCESSOR
|
10
13
|
end
|
11
14
|
|
12
15
|
def self.error_handler
|
@@ -1,27 +1,27 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require
|
4
|
-
require
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'cascade/complex_fields'
|
4
|
+
require 'cascade/exceptions'
|
5
|
+
require 'cascade/helpers/hash'
|
5
6
|
|
6
7
|
module Cascade
|
7
8
|
class RowProcessor
|
8
|
-
extend Configuration
|
9
9
|
using HashRefinements
|
10
10
|
|
11
|
-
DEFAULT_PROCESSOR = ->(value) { value
|
12
|
-
|
13
|
-
define_setting :use_default_presenter, false
|
14
|
-
define_setting :deafult_presenter, -> { DEFAULT_PROCESSOR }
|
11
|
+
DEFAULT_PROCESSOR = ->(value) { value&.to_s&.strip }
|
15
12
|
|
16
13
|
def initialize(options = {})
|
17
|
-
@options
|
18
|
-
@
|
14
|
+
@options = options
|
15
|
+
@ext_presenters = options[:ext_presenters].to_h
|
16
|
+
@columns_matching = options.fetch(:columns_matching)
|
17
|
+
@use_default_presenter = options.fetch(:use_default_presenter, false)
|
18
|
+
@deafult_presenter = options.fetch(:deafult_presenter, DEFAULT_PROCESSOR)
|
19
19
|
end
|
20
20
|
|
21
21
|
# Iterates through object using columns values supported keys as keys for
|
22
|
-
# iterating, then parse it by
|
22
|
+
# iterating, then parse it by the corresponding parser.
|
23
23
|
#
|
24
|
-
# @param row [Hash] the object retrieved from
|
24
|
+
# @param row [Hash] the object retrieved from data parser
|
25
25
|
# @return [Hash] the object with parsed columns
|
26
26
|
def call(row)
|
27
27
|
@columns_matching.supported_keys.inject({}) do |result, key|
|
@@ -37,27 +37,28 @@ module Cascade
|
|
37
37
|
|
38
38
|
def receive_presenter(column_name)
|
39
39
|
presenter = presenters[@columns_matching.column_type(column_name)]
|
40
|
-
if presenter.nil? &&
|
41
|
-
raise Cascade::UnknownPresenterType
|
40
|
+
if presenter.nil? && !@use_default_presenter
|
41
|
+
raise Cascade::UnknownPresenterType
|
42
42
|
end
|
43
|
-
|
43
|
+
|
44
|
+
presenter || @deafult_presenter
|
44
45
|
end
|
45
46
|
|
46
47
|
def presenters
|
47
|
-
@presenters ||=
|
48
|
+
@presenters ||= @ext_presenters.reverse_merge(predefined_presenters)
|
48
49
|
end
|
49
50
|
|
50
51
|
def self_copy
|
51
52
|
self.class.new(options)
|
52
53
|
end
|
53
54
|
|
54
|
-
def
|
55
|
+
def predefined_presenters
|
55
56
|
{
|
56
|
-
string:
|
57
|
-
currency:
|
58
|
-
boolean:
|
57
|
+
string: DEFAULT_PROCESSOR,
|
58
|
+
currency: ComplexFields::Currency.new,
|
59
|
+
boolean: ComplexFields::Boolean.new,
|
59
60
|
iterable_recursive: ComplexFields::ArrayProcessor.new(self_copy),
|
60
|
-
recursive:
|
61
|
+
recursive: self_copy
|
61
62
|
}
|
62
63
|
end
|
63
64
|
end
|