json-mapping-transform 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: d98b188ca6cf86ba7a386096fe708a4ca2e7986e18471e1e53c85f84baddb98e
4
+ data.tar.gz: ecdf870a41b97c13c574fdc1982e69c239d417422bfcbfb2372b02a78150141a
5
+ SHA512:
6
+ metadata.gz: 45898c72eedfca9afa4c2ba5837fb08b4002ff55da24d475325781a99483d440f5b973b808d5b5c9e9d052a8b058f69ddf876199c93c311f6733594e59cd7ddd
7
+ data.tar.gz: e980815abd5856ee5f0193b672214d113fd543138d711680f3c0aff56756b6cb8e3c62eb2b0c72bd67b71d86afce3ef1481da8e830e0e8821c7abf42c30297b7
@@ -0,0 +1,12 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /_yardoc/
4
+ /coverage/
5
+ /doc/
6
+ /pkg/
7
+ /spec/reports/
8
+ /tmp/
9
+ vendor/*
10
+ # rspec failure tracking
11
+ .rspec_status
12
+ Gemfile.lock
data/.rspec ADDED
@@ -0,0 +1,3 @@
1
+ --format documentation
2
+ --color
3
+ --require spec_helper
@@ -0,0 +1,31 @@
1
+ AllCops:
2
+ Exclude:
3
+ - 'spec/spec_helper.rb'
4
+ - 'json-mapping-transform.gemspec'
5
+ - 'vendor/**/*'
6
+ - 'coverage/**'
7
+ - 'bin/**'
8
+ - '.bundle/**'
9
+ Layout/LineLength:
10
+ Max: 140
11
+
12
+ Metrics/AbcSize:
13
+ Enabled: false
14
+ Metrics/MethodLength:
15
+ Max: 50
16
+ Metrics/ClassLength:
17
+ Max: 200
18
+ Metrics/PerceivedComplexity:
19
+ Max: 15
20
+ Metrics/CyclomaticComplexity:
21
+ Max: 15
22
+ Metrics/BlockLength:
23
+ Exclude:
24
+ - spec/**/*
25
+
26
+ Style/IfUnlessModifier:
27
+ Enabled: false
28
+ Style/FrozenStringLiteralComment:
29
+ Enabled: false
30
+ Style/SingleLineMethods:
31
+ Enabled: false
@@ -0,0 +1,7 @@
1
+ ---
2
+ sudo: false
3
+ language: ruby
4
+ cache: bundler
5
+ rvm:
6
+ - 2.6.3
7
+ before_install: gem install bundler -v 2.0.2
@@ -0,0 +1,74 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as
6
+ contributors and maintainers pledge to making participation in our project and
7
+ our community a harassment-free experience for everyone, regardless of age, body
8
+ size, disability, ethnicity, gender identity and expression, level of experience,
9
+ nationality, personal appearance, race, religion, or sexual identity and
10
+ orientation.
11
+
12
+ ## Our Standards
13
+
14
+ Examples of behavior that contributes to creating a positive environment
15
+ include:
16
+
17
+ * Using welcoming and inclusive language
18
+ * Being respectful of differing viewpoints and experiences
19
+ * Gracefully accepting constructive criticism
20
+ * Focusing on what is best for the community
21
+ * Showing empathy towards other community members
22
+
23
+ Examples of unacceptable behavior by participants include:
24
+
25
+ * The use of sexualized language or imagery and unwelcome sexual attention or
26
+ advances
27
+ * Trolling, insulting/derogatory comments, and personal or political attacks
28
+ * Public or private harassment
29
+ * Publishing others' private information, such as a physical or electronic
30
+ address, without explicit permission
31
+ * Other conduct which could reasonably be considered inappropriate in a
32
+ professional setting
33
+
34
+ ## Our Responsibilities
35
+
36
+ Project maintainers are responsible for clarifying the standards of acceptable
37
+ behavior and are expected to take appropriate and fair corrective action in
38
+ response to any instances of unacceptable behavior.
39
+
40
+ Project maintainers have the right and responsibility to remove, edit, or
41
+ reject comments, commits, code, wiki edits, issues, and other contributions
42
+ that are not aligned to this Code of Conduct, or to ban temporarily or
43
+ permanently any contributor for other behaviors that they deem inappropriate,
44
+ threatening, offensive, or harmful.
45
+
46
+ ## Scope
47
+
48
+ This Code of Conduct applies both within project spaces and in public spaces
49
+ when an individual is representing the project or its community. Examples of
50
+ representing a project or community include using an official project e-mail
51
+ address, posting via an official social media account, or acting as an appointed
52
+ representative at an online or offline event. Representation of a project may be
53
+ further defined and clarified by project maintainers.
54
+
55
+ ## Enforcement
56
+
57
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
+ reported by contacting the project team at parandea1.7@gmail.com. All
59
+ complaints will be reviewed and investigated and will result in a response that
60
+ is deemed necessary and appropriate to the circumstances. The project team is
61
+ obligated to maintain confidentiality with regard to the reporter of an incident.
62
+ Further details of specific enforcement policies may be posted separately.
63
+
64
+ Project maintainers who do not follow or enforce the Code of Conduct in good
65
+ faith may face temporary or permanent repercussions as determined by other
66
+ members of the project's leadership.
67
+
68
+ ## Attribution
69
+
70
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71
+ available at [http://contributor-covenant.org/version/1/4][version]
72
+
73
+ [homepage]: http://contributor-covenant.org
74
+ [version]: http://contributor-covenant.org/version/1/4/
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in json-mapping-transform.gemspec
4
+ gemspec
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2020 Anmol Parande
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
@@ -0,0 +1,227 @@
1
+ # JSON Map Transform
2
+
3
+ ## Overview
4
+ When building data pipelines, it is often useful to extract and transfrom data from an input JSON and output it in a different format. The standard process for doing this in Ruby is to write a series of if-else logic coupled with for-loops. This code ends up being largely redundant, confusing, and difficult to maintain or change. This Gem provides an easy and extensible solution to this problem by allowing you to define your mapping in YAML and apply it to any JSON object in a single line of code.
5
+
6
+ The general format of the transform mapping looks as follows:
7
+ ``` yaml
8
+ ---
9
+ conditions:
10
+ condition_name:
11
+ class: (required)
12
+ predicate: (optional)
13
+ objects:
14
+ - name: (required)
15
+ path: (optional)
16
+ default: (optional)
17
+ attributes: (optional)
18
+ transform: (optional)
19
+ conditions: (optional)
20
+ - name: (required)
21
+ output: (optional)
22
+ field: (optional)
23
+ ```
24
+
25
+ ## Installation
26
+ Add the gem to the Gemfile
27
+ ```ruby
28
+ gem 'json-mapping-transform'
29
+ ```
30
+ Require the mapping in your code
31
+ ```ruby
32
+ require 'json_mapping'
33
+ ```
34
+
35
+ ## Objects
36
+ Objects are the output keys of the mapping. `JsonMapper#map` will output a single Ruby Hash when applied to an input.
37
+ The following rules apply to objects:
38
+ - Each object has a name that translates to its key in the output JSON
39
+ - The **path** specifies the input key in the source JSON that the object corresponds to
40
+ - Paths are defined from the top level of the JSON: `/`
41
+ - When `*` is included in the path, the result will be an array
42
+ - When a path is not found (or not proviided), the object evaluates to `nil`
43
+ - Objects can have a **default** value which is returned if the path evaluates to `nil`
44
+ - Objects can have **attributes** which are a list of more objects (nested JSON objects)
45
+ - **Note:** Paths in nested objects are relative to the path of the top-level object
46
+
47
+ ## Conditions
48
+ Conditions are `if` statements performed on an extracted value. They are defined as a hash in the mapping file.
49
+ - By default, conditions are evaluated against the object path
50
+ - If **field** is specified, the condition is evaluated against the path relative to the object path
51
+ - If the extracted value satisfies the condition, the output will be set to **output** (to the extracted value if output is not specified)
52
+ - If the extracted value does not satisfy the condition, the output will be set to the object's default
53
+ - If the extracted value is `nil`, conditions are not evaluated
54
+ - Conditions are referenced by **name** in the object definition
55
+ - If multiple conditions are defined and satisfied, the output will be an array
56
+
57
+ There are several built-in condition types which can be used for the `class` field of the condition.
58
+ - `InCondition`: Check if an object/Array is in/intersects with the **predicate**, an array
59
+ - `RegexCondition`: Check if a string matches the **predicate**, a regular expression
60
+ - `AnyCondition`: Check if an object/Array is/contains a truthy value
61
+ - `LessThanCondition`: Check if a `Numeric` is less than the **predicate**, another `Numeric`
62
+ - `GreaterThanCondition`: Check if a `Numeric` is less than the **predicate**, another `Numeric`
63
+ - `AndCondition`: Check if an object satisfies all conditions provided in the **predicate**
64
+ - `OrCondition`: Check if an object satisfies at least one condition provided in the **predicate**
65
+ - `NotCondition`: Check if an object does not satisfy the condition provided as the **predicate**
66
+
67
+ Developers can create their own custom conditions by extending `BaseCondition` inside of the `Conditions` module
68
+
69
+ ## Transforms
70
+ - Transforms are arbitrary blocks of code which act on the extracted value for an object
71
+ - Transforms are applied after conditions (i.e they will only be applied if at least one condition is satisfied)
72
+ - If the extracted value is `nil` (or all conditions fail), then transforms are not evaluated
73
+ - Transforms are referenced by name in YAML. You must pass in a hash of them to the `JsonMapper` during initialization
74
+
75
+ ## Failure Cases
76
+ ### Graceful Failures
77
+ The mapping will gracefully fail (fall back on default) when
78
+ 1. Encountering a null object in the original object
79
+ 2. Encountering non-existent paths
80
+ 3. Indexing an array out of bounds
81
+
82
+ ### Exceptions
83
+ The mapping will raise an exception when
84
+ 1. The YAML map is not formatted properly (`JsonMapper::FormatError`)
85
+ 2. A condition is referenced but not defined (`Conditions::ConditionError`)
86
+ 3. Unknown condition type (`NameError`)
87
+ 4. A condition is defined with an incorrect predicate (`Conditions::ConditionError`)
88
+ 5. A condition is given a value it can't compare to the predicate (`Conditions::ConditionError`)
89
+ 6. A provided transform is not callable (`JsonMapper::TransformError`)
90
+ 7. The `*` operator is used on a non-array (`JsonMapper::PathError`)
91
+ 8. An exception is encountered while applying a transform (`StandardError`)
92
+
93
+ ## Examples
94
+ For all the examples provided below, this is the input JSON that is being mapped:
95
+ ```json
96
+ {
97
+ "name": "Trader Joe's",
98
+ "location": "Berkeley, California",
99
+ "weeklyVisitors": 5000,
100
+ "storeId": 1234,
101
+ "employees": [
102
+ { "name": "Jim Shoes" },
103
+ { "name": "Kay Oss" }
104
+ ],
105
+ "inventory": [
106
+ { "itemName": "Apples", "price": 0.5, "unit": "lb" },
107
+ { "itemName": "Oranges", "price": 2, "unit": "lb" },
108
+ { "itemName": "Bag of Carrots", "price": 1.5, "unit": "count" }
109
+ ]
110
+ }
111
+ ```
112
+ ### Basic Example
113
+ An simple example which just converts between two objects
114
+ #### Mapping
115
+ ```yaml
116
+ ---
117
+ objects:
118
+ - name: name
119
+ path: "/name"
120
+ - name: profits
121
+ default: 0
122
+ - name: location
123
+ path: "/location"
124
+ - name: weekly_visitors
125
+ path: "/weeklyVisitors"
126
+ - name: store_id
127
+ path: "/storeId"
128
+ - name: employees
129
+ path: "/employees/*/name"
130
+ - name: inventory
131
+ path: "/inventory/*"
132
+ attributes:
133
+ - name: item_name
134
+ path: /itemName
135
+ - name: price
136
+ path: /price
137
+ - name: unit
138
+ path: /unit
139
+ ```
140
+ #### Output
141
+ ```json
142
+ {
143
+ "name": "Trader Joe\'s",
144
+ "profits": 0,
145
+ "location": "Berkeley, California",
146
+ "weekly_visitors": 5000,
147
+ "store_id": 1234,
148
+ "employees": ["Jim Shoes", "Kay Oss"],
149
+ "inventory": [
150
+ { "item_name": "Apples", "price": 0.5, "unit": "lb" },
151
+ { "item_name": "Oranges", "price": 2, "unit": "lb" },
152
+ { "item_name": "Bag of Carrots", "price": 1.5, "unit": "count" }
153
+ ]
154
+ }
155
+ ```
156
+ ### Transforms Example
157
+ An example of a custom transformation
158
+ #### Mapping
159
+ ```yaml
160
+ ---
161
+ objects:
162
+ - name: name
163
+ path: "/name"
164
+ - name: inventory
165
+ path: "/inventory/*/"
166
+ transform: listing_transform
167
+ ```
168
+ #### Code
169
+ ```ruby
170
+ transforms = {
171
+ 'listing_transform' => ->(list) { list.map { |x| "#{x['itemName']} at $#{x['price']}/#{x['unit']}" } }
172
+ }
173
+ output = JsonMapping.new(path, transforms).map(store_fixture)
174
+ ```
175
+ #### Output
176
+ ```json
177
+ {
178
+ "name": "Trader Joe\'s",
179
+ "inventory": ["Apples at $0.5/lb", "Oranges at $2/lb", "Bag of Carrots at $1.5/count"]
180
+ }
181
+ ```
182
+ ### Conditions Example
183
+ An example using conditions
184
+ ```yaml
185
+ ---
186
+ conditions:
187
+ apple_condition:
188
+ class: AppleCondition
189
+ high_performance_condition:
190
+ class: AndCondition
191
+ predicate:
192
+ - class: LessThanCondition
193
+ predicate: 10000
194
+ - class: GreaterThanCondition
195
+ predicate: 1000
196
+
197
+ objects:
198
+ - name: performance
199
+ path: "/weeklyVisitors"
200
+ conditions:
201
+ - name: high_performance_condition
202
+ output: high
203
+ - name: apple
204
+ path: "/inventory"
205
+ conditions:
206
+ - name: apple_condition
207
+ ```
208
+ #### Code
209
+ ```ruby
210
+ module Conditions
211
+ class AppleCondition < BaseCondition
212
+ def apply(value)
213
+ puts value
214
+ value.is_a?(Hash) && value['itemName'] == 'Apples'
215
+ end
216
+ end
217
+ end
218
+
219
+ output = JsonMapping.new(path).map(store_fixture)
220
+ ```
221
+ #### Output
222
+ ```json
223
+ {
224
+ "performance": "high",
225
+ "apple": [{ "itemName": "Apples", "price": 0.5, "unit": "lb" }]
226
+ }
227
+ ```
@@ -0,0 +1,6 @@
1
+ require 'bundler/gem_tasks'
2
+ require 'rspec/core/rake_task'
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
5
+
6
+ task default: :spec
@@ -0,0 +1,14 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "json_mapping"
5
+
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ # (If you use this, don't forget to add pry to your Gemfile!)
10
+ # require "pry"
11
+ # Pry.start
12
+
13
+ require "irb"
14
+ IRB.start(__FILE__)
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
@@ -0,0 +1,32 @@
1
+ lib = File.expand_path("lib", __dir__)
2
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
3
+
4
+ Gem::Specification.new do |spec|
5
+ spec.name = 'json-mapping-transform'
6
+ spec.version = '0.1.0'
7
+ spec.authors = ['Anmol Parande']
8
+ spec.email = ['parande.anmol@gmail.com']
9
+
10
+ spec.summary = 'Map one JSON format into another JSON format'
11
+ spec.description = 'When building data pipelines, it is often useful to extract and transfrom data from an input JSON and output it in a different format. The standard process for doing this in Ruby is to write a series of if-else logic coupled with for-loops. This code ends up being largely redundant, confusing, and difficult to maintain or change. This Gem provides an easy and extensible solution to this problem by allowing you to define your mapping in YAML and apply it to any JSON object in a single line of code.'
12
+ spec.homepage = 'https://github.com/aparande/json-mapping-transform'
13
+ spec.license = 'MIT'
14
+
15
+ spec.metadata['homepage_uri'] = spec.homepage
16
+ spec.metadata['source_code_uri'] = 'https://github.com/aparande/json-mapping-transform'
17
+ # spec.metadata['changelog_uri'] = "TODO: Put your gem's CHANGELOG.md URL here."
18
+
19
+ # Specify which files should be added to the gem when it is released.
20
+ # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
21
+ spec.files = Dir.chdir(File.expand_path('..', __FILE__)) do
22
+ `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
23
+ end
24
+ spec.require_paths = ['lib']
25
+
26
+ spec.add_development_dependency 'bundler', '~> 2.0'
27
+ spec.add_development_dependency 'rake', '~> 10.0'
28
+ spec.add_development_dependency 'redcarpet'
29
+ spec.add_development_dependency 'rspec', '~> 3.0'
30
+ spec.add_development_dependency 'simplecov'
31
+ spec.add_development_dependency 'yard'
32
+ end
@@ -0,0 +1,178 @@
1
+ ##
2
+ # Stores conditions accessible to +JsonMapper+
3
+ module Conditions
4
+ ##
5
+ # Thrown when a condition encounters an error
6
+ class ConditionError < StandardError; end
7
+
8
+ ##
9
+ # Abstract class from which all conditions inherit
10
+ class BaseCondition
11
+ ##
12
+ # @param [Any] predicate A predicate value which will be compared against input values
13
+ def initialize(predicate)
14
+ @predicate = predicate
15
+ end
16
+
17
+ def apply(_) true; end
18
+ end
19
+
20
+ ##
21
+ # Checks if an element (or elements of an array) belong to an array
22
+ # Can be used as an "equals to" condition
23
+ class InCondition < BaseCondition
24
+ ##
25
+ # @param [Array] predicate An array predicate
26
+ def initialize(predicate)
27
+ raise ConditionError, "In condition value must be an Array, not #{predicate.class}" unless predicate.is_a? Array
28
+
29
+ super(predicate)
30
+ end
31
+
32
+ ##
33
+ # @param [Any, Array] value A value to be checked against the predicate
34
+ # @return [true] if value and predicate have overlapping values
35
+ # @return [false] if value and predicate have no overlapping values
36
+ def apply(value)
37
+ value = [value] unless value.is_a? Array
38
+ (value & @predicate).any?
39
+ end
40
+ end
41
+
42
+ ##
43
+ # Compares a value against a regular expression
44
+ class RegexCondition < BaseCondition
45
+ ##
46
+ # @param [String] predicate A valid regular expression
47
+ def initialize(predicate)
48
+ predicate = Regexp.new(predicate.to_s)
49
+ super(predicate)
50
+ rescue RegexpError => e
51
+ raise ConditionError, e.inspect
52
+ end
53
+
54
+ ##
55
+ # @param [String] value
56
+ # @return [true] if value matches regular expression
57
+ # @return [false] if value does not match regular expression
58
+ def apply(value)
59
+ @predicate.match?(value)
60
+ end
61
+ end
62
+
63
+ ##
64
+ # Checks if any values in the array are true
65
+ class AnyCondition < BaseCondition
66
+ ##
67
+ # @param [Any, Array] value
68
+ # @return [true] if value is truthy (or has at least one truthy value)
69
+ # @return [false] if the value is not truthy (or has no truthy values)
70
+ def apply(value)
71
+ value = [value] unless value.is_a? Array
72
+ value.any?
73
+ end
74
+ end
75
+
76
+ ##
77
+ # Checks if two conditions are true
78
+ class AndCondition < BaseCondition
79
+ ##
80
+ # @param [Array] predicate An array of Hashes representating conditions (Greater than length 2)
81
+ def initialize(predicate)
82
+ raise ConditionError, 'And condition predicate must be an array of conditions' unless predicate.is_a?(Array) && predicate.length > 1
83
+
84
+ predicate.map! { |x| Object.const_get("Conditions::#{x['class']}").new(x['predicate']) }
85
+ super(predicate)
86
+ end
87
+
88
+ ##
89
+ # @param [Any] value
90
+ # @return [true] if +value+ matches all conditions
91
+ # @return [false] if +value+ does not match one condition
92
+ def apply(value)
93
+ @predicate.map { |x| x.apply(value) }.all?
94
+ end
95
+ end
96
+
97
+ ##
98
+ # Checks if either or both of two conditions are true
99
+ class OrCondition < BaseCondition
100
+ ##
101
+ # @param [Array] predicate An array of Hashes representating conditions (Greater than length 2)
102
+ def initialize(predicate)
103
+ raise ConditionError, 'Or condition predicate must be an array of conditions' unless predicate.is_a?(Array) && predicate.length > 1
104
+
105
+ predicate.map! { |x| Object.const_get("Conditions::#{x['class']}").new(x['predicate']) }
106
+ super(predicate)
107
+ end
108
+
109
+ ##
110
+ # @param [Any] value
111
+ # @return [true] if +value+ matches any condition
112
+ # @return [false] if +value+ matches no conditions
113
+ def apply(value)
114
+ @predicate.map { |x| x.apply(value) }.any?
115
+ end
116
+ end
117
+
118
+ ##
119
+ # Checks if a condition is not true
120
+ class NotCondition < BaseCondition
121
+ ##
122
+ # @param [Hash] predicate A hash representing a condition
123
+ def initialize(predicate)
124
+ raise ConditionError, 'Not condition predicate a condition' unless predicate.is_a?(Hash) && predicate.key?('class')
125
+
126
+ predicate = Object.const_get("Conditions::#{predicate['class']}").new(predicate['predicate'])
127
+ super(predicate)
128
+ end
129
+
130
+ ##
131
+ # @param [Any] value
132
+ # @return [true] if +value+ does not satisfy the condition
133
+ # @return [false] if +value+ satisfies the condition
134
+ def apply(value)
135
+ !@predicate.apply(value)
136
+ end
137
+ end
138
+
139
+ ##
140
+ # Checks if value is less than a predicate
141
+ class LessThanCondition < BaseCondition
142
+ ##
143
+ # @param [Numeric] predicate
144
+ def initialize(predicate)
145
+ raise ConditionError, 'LessThan condition predicate must a number' unless predicate.is_a?(Numeric)
146
+
147
+ super(predicate)
148
+ end
149
+
150
+ ##
151
+ # @param [Numeric] value
152
+ # @return [true] if +value < predicate+
153
+ # @return [false] if +value >= predicate+
154
+ def apply(value)
155
+ value < @predicate
156
+ end
157
+ end
158
+
159
+ ##
160
+ # Checks if value is greater than a predicate
161
+ class GreaterThanCondition < BaseCondition
162
+ ##
163
+ # @param [Numeric] predicate
164
+ def initialize(predicate)
165
+ raise ConditionError, 'GreaterThan condition predicate must a number' unless predicate.is_a?(Numeric)
166
+
167
+ super(predicate)
168
+ end
169
+
170
+ ##
171
+ # @param [Numeric] value
172
+ # @return [true] if +value > predicate+
173
+ # @return [false] if +value <= predicate+
174
+ def apply(value)
175
+ value > @predicate
176
+ end
177
+ end
178
+ end
@@ -0,0 +1,184 @@
1
+ require 'yaml'.freeze
2
+ require 'logger'.freeze
3
+ require 'conditions'.freeze
4
+
5
+ ##
6
+ # Stores and applies a mapping to an input ruby Hash
7
+ class JsonMapping
8
+ ##
9
+ # Thrown when a transform is not found or not callable
10
+ class TransformError < StandardError; end
11
+ ##
12
+ # Thrown when parsing an invalid path
13
+ class PathError < StandardError; end
14
+ ##
15
+ # Thrown when the YAML transform is not formatted properly
16
+ class FormatError < StandardError; end
17
+
18
+ ##
19
+ # @param [String] schema_path The path to the YAML schema
20
+ # @param [Hash] transforms A hash of callable objects (Procs/Lambdas). Keys must match transform names specified in YAML
21
+ def initialize(schema_path, transforms = {})
22
+ schema = YAML.safe_load(File.read(schema_path))
23
+
24
+ @conditions = (schema['conditions'] || {}).map do |key, value|
25
+ [key, Object.const_get("Conditions::#{value['class']}").new(value['predicate'])]
26
+ end.to_h
27
+
28
+ @object_schemas = schema['objects']
29
+ @transforms = transforms || {}
30
+ @logger = Logger.new($stdout)
31
+ end
32
+
33
+ ##
34
+ # @param [Hash] input_hash A ruby hash onto which the schema should be applied
35
+ # @return [Array] An array of output hashes representing the mapped objects
36
+ def apply(input_hash)
37
+ raise FormatError, 'Must define objects under the \'objects\' name' if @object_schemas.nil?
38
+
39
+ @object_schemas.map { |schema| parse_object(input_hash, schema) }.reduce(&:merge)
40
+ end
41
+
42
+ private
43
+
44
+ ##
45
+ # Maps an object schema to an object in the output
46
+ # @param [Hash] input_hash The hash onto which the schema should be mapped
47
+ # @param [Hash] schema A hash representing the schema which should be applied to the input
48
+ # Raises +FormatError+ if +schema+ is not a +Hash+ or has no key +name+
49
+ # @return [Hash] The output object
50
+ def parse_object(input_hash, schema)
51
+ raise FormatError, "Object should be a hash: #{schema}" unless schema.is_a? Hash
52
+ raise FormatError, "Object needs a name: #{schema}" unless schema.key?('name')
53
+
54
+ output = {}
55
+ # Its an object
56
+ if schema.key?('attributes')
57
+ output[schema['name']] = schema['default']
58
+
59
+ object_hash = parse_path(input_hash, schema['path'])
60
+ return output if object_hash.nil?
61
+
62
+ unless object_hash.is_a? Array
63
+ object_hash = [object_hash]
64
+ end
65
+
66
+ attrs = []
67
+ object_hash.each do |obj|
68
+ attributes_hash = {}
69
+ schema['attributes'].each do |attribute|
70
+ attr_hash = parse_object(obj, attribute)
71
+ attributes_hash = attributes_hash.merge(attr_hash)
72
+ end
73
+ attrs << attributes_hash
74
+ end
75
+
76
+ output[schema['name']] = attrs.length == 1 && schema['path'][-1] != '*' ? attrs[0] : attrs
77
+ else # Its a value
78
+ output = map_value(input_hash, schema)
79
+ end
80
+
81
+ output
82
+ end
83
+
84
+ ##
85
+ # Maps a schema to a single field in the output schema
86
+ # @param [Hash] input_hash The input hash to be mapped
87
+ # @param [Hash] schema The schema which should be applied
88
+ # @return [Hash] A Hash which represents the applied schema
89
+ def map_value(input_hash, schema)
90
+ raise FormatError, "Schema should be a hash: #{schema}" unless schema.is_a? Hash
91
+
92
+ output = {}
93
+ output[schema['name']] = schema['default']
94
+ return output if schema['path'].nil?
95
+
96
+ value = parse_path(input_hash, schema['path'])
97
+ return output if value.nil?
98
+
99
+ if schema.key?('conditions')
100
+ value = apply_conditions(value, schema['conditions']) || output[schema['name']]
101
+ end
102
+
103
+ if schema.key?('transform') && value != output[schema['name']]
104
+ raise TransformError, "Undefined transform named #{schema['transform']}" unless @transforms.key?(schema['transform'])
105
+ raise TransformError, 'Transforms should respond to the \'call\' method' unless @transforms[schema['transform']].respond_to?(:call)
106
+
107
+ value = @transforms[schema['transform']].call(value)
108
+ end
109
+
110
+ output[schema['name']] = value
111
+ output
112
+ end
113
+
114
+ ##
115
+ # @param [Hash] input_hash The input hash
116
+ # @param [String] path The path at which to grab the value
117
+ # @return [Any] The value at the particular path
118
+ def parse_path(input_hash, path)
119
+ raise ArgumentError, "path must be string, not #{path.class}" unless path.is_a? String
120
+
121
+ parts = path.split('/')
122
+ value = input_hash
123
+
124
+ parts.each_with_index do |part, idx|
125
+ if value.nil?
126
+ @logger.warn("Could not find #{path} in #{input_hash}")
127
+ break
128
+ end
129
+
130
+ if part == '*'
131
+ raise PathError, "#{parts[0, idx].join('/')} in #{input_hash} is not an array" unless value.is_a? Array
132
+
133
+ return value.map { |obj| parse_path(obj, parts[idx + 1..-1].join('/')) }
134
+ else
135
+ next if part.empty?
136
+
137
+ if value.is_a? Array
138
+ part = part.to_i
139
+
140
+ if part >= value.length
141
+ @logger.warn("Index went out of bounds while parsing #{path} in #{input_hash}")
142
+ value = nil
143
+ break
144
+ end
145
+ end
146
+
147
+ value = value[part]
148
+ end
149
+ end
150
+ value
151
+ end
152
+
153
+ ##
154
+ # Applies conditions to a value
155
+ # @param [Any] value A value to compare the condition predicates against
156
+ # @param [Array] conds An array of conditions
157
+ # @return [Array] If multiple conditions are satisfied
158
+ # @return [Any] If one condition is satisfied
159
+ # @return [nil] If no conditions are satisfied
160
+ def apply_conditions(value, conds)
161
+ output = []
162
+ conds.each do |cond|
163
+ input_val = value
164
+ raise FormatError, "Conditions are a hash: #{cond}" unless cond.is_a? Hash
165
+ raise Conditions::ConditionError, "Unknown condition named #{cond['name']}" unless @conditions.key?(cond['name'])
166
+
167
+ condition = @conditions[cond['name']]
168
+
169
+ input_val = [input_val] unless input_val.is_a? Array
170
+ input_val = input_val.select do |x|
171
+ x = parse_path(x, cond['field']) if cond.key?('field')
172
+ condition.apply(x)
173
+ end
174
+
175
+ next if input_val.empty?
176
+
177
+ # Maintain the original data-type of the value (i.e Array or single element)
178
+ input_val = input_val[0] if input_val.length == 1 && !value.is_a?(Array)
179
+ output << (cond['output'] || input_val)
180
+ end
181
+
182
+ return (output.length == 1 ? output[0] : output) unless output.empty?
183
+ end
184
+ end
metadata ADDED
@@ -0,0 +1,149 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: json-mapping-transform
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Anmol Parande
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2020-08-21 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: bundler
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '2.0'
20
+ type: :development
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '2.0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: rake
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '10.0'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '10.0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: redcarpet
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rspec
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '3.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '3.0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: simplecov
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: yard
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ description: When building data pipelines, it is often useful to extract and transfrom
98
+ data from an input JSON and output it in a different format. The standard process
99
+ for doing this in Ruby is to write a series of if-else logic coupled with for-loops.
100
+ This code ends up being largely redundant, confusing, and difficult to maintain
101
+ or change. This Gem provides an easy and extensible solution to this problem by
102
+ allowing you to define your mapping in YAML and apply it to any JSON object in a
103
+ single line of code.
104
+ email:
105
+ - parande.anmol@gmail.com
106
+ executables: []
107
+ extensions: []
108
+ extra_rdoc_files: []
109
+ files:
110
+ - ".gitignore"
111
+ - ".rspec"
112
+ - ".rubocop.yml"
113
+ - ".travis.yml"
114
+ - CODE_OF_CONDUCT.md
115
+ - Gemfile
116
+ - LICENSE.txt
117
+ - README.md
118
+ - Rakefile
119
+ - bin/console
120
+ - bin/setup
121
+ - json-mapping-transform.gemspec
122
+ - lib/conditions.rb
123
+ - lib/json_mapping.rb
124
+ homepage: https://github.com/aparande/json-mapping-transform
125
+ licenses:
126
+ - MIT
127
+ metadata:
128
+ homepage_uri: https://github.com/aparande/json-mapping-transform
129
+ source_code_uri: https://github.com/aparande/json-mapping-transform
130
+ post_install_message:
131
+ rdoc_options: []
132
+ require_paths:
133
+ - lib
134
+ required_ruby_version: !ruby/object:Gem::Requirement
135
+ requirements:
136
+ - - ">="
137
+ - !ruby/object:Gem::Version
138
+ version: '0'
139
+ required_rubygems_version: !ruby/object:Gem::Requirement
140
+ requirements:
141
+ - - ">="
142
+ - !ruby/object:Gem::Version
143
+ version: '0'
144
+ requirements: []
145
+ rubygems_version: 3.0.3
146
+ signing_key:
147
+ specification_version: 4
148
+ summary: Map one JSON format into another JSON format
149
+ test_files: []