json_data_extractor 0.0.2 → 0.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +51 -11
- data/lib/json_data_extractor.rb +9 -1
- data/lib/src/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 101c20f78189fbd300dedf5a03779d19558d65fb4f4d2d4767ec8b1f638aafde
|
|
4
|
+
data.tar.gz: 924d26135a261281c14051ab7aa305debcec61688fd6deda4aeb322da8028ecd
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: df55bd36b0e00ac0bd4f427451dcf69425538dc06382632b261ec03aaa6cb24b0648873645b80c609b1e7329e3f9aac85fa97612afdcc492efb6efa91d062f83
|
|
7
|
+
data.tar.gz: df3a0c646a823792fa1bce461cab8eeb037b6be9906fbb1f5608cf69da4e13133c11f2e71aa90033b3924ea4ccc11f6a9900ca17b263c44702583c6eb3b679fa
|
data/README.md
CHANGED
|
@@ -1,10 +1,12 @@
|
|
|
1
1
|
# JsonDataExtractor
|
|
2
2
|
|
|
3
|
+
NOTE: This is still a very early beta.
|
|
4
|
+
|
|
3
5
|
Transform JSON data structures with the help of a simple schema and JsonPath expressions.
|
|
4
6
|
Use the JsonDataExtractor gem to extract and modify data from complex JSON structures using a straightforward syntax
|
|
5
7
|
and a range of built-in or custom modifiers.
|
|
6
8
|
|
|
7
|
-
_Another try to make something for JSON that is XSLT for XML.
|
|
9
|
+
_Another try to make something for JSON that is XSLT for XML.
|
|
8
10
|
We transform one JSON into another JSON with the help of a third JSON!!!111!!eleventy!!_
|
|
9
11
|
|
|
10
12
|
Remap one JSON structure into another with some basic rules and [jsonpath](https://github.com/joshbuddy/jsonpath).
|
|
@@ -29,7 +31,9 @@ Or install it yourself as:
|
|
|
29
31
|
|
|
30
32
|
## Usage
|
|
31
33
|
|
|
32
|
-
JsonDataExtractor allows you to remap one JSON structure into another with some basic rules
|
|
34
|
+
JsonDataExtractor allows you to remap one JSON structure into another with some basic rules
|
|
35
|
+
and [JSONPath](https://goessner.net/articles/JsonPath/) expressions. The process involves defining a schema that maps
|
|
36
|
+
the input JSON structure to the desired output structure.
|
|
33
37
|
|
|
34
38
|
We'll base our examples on the following source:
|
|
35
39
|
|
|
@@ -74,11 +78,15 @@ We'll base our examples on the following source:
|
|
|
74
78
|
|
|
75
79
|
### Defining a Schema
|
|
76
80
|
|
|
77
|
-
A schema
|
|
81
|
+
A schema consists of one or more mappings that specify how to extract data from the input JSON and where to place it in
|
|
82
|
+
the output JSON.
|
|
78
83
|
|
|
79
|
-
Each mapping has a path field that specifies the JsonPath expression to use for data extraction, and an optional
|
|
84
|
+
Each mapping has a path field that specifies the JsonPath expression to use for data extraction, and an optional
|
|
85
|
+
modifier field that specifies one or more modifiers to apply to the extracted data. Modifiers are used to transform the
|
|
86
|
+
data in some way before placing it in the output JSON.
|
|
80
87
|
|
|
81
|
-
Here's an example schema that extracts the authors and categories from a JSON structure similar to the one used in the
|
|
88
|
+
Here's an example schema that extracts the authors and categories from a JSON structure similar to the one used in the
|
|
89
|
+
previous example (here it's in YAML just for readability):
|
|
82
90
|
|
|
83
91
|
```yaml
|
|
84
92
|
schemas:
|
|
@@ -87,7 +95,9 @@ schemas:
|
|
|
87
95
|
modifier: downcase
|
|
88
96
|
categories: $..category
|
|
89
97
|
```
|
|
98
|
+
|
|
90
99
|
The resulting json will be:
|
|
100
|
+
|
|
91
101
|
```json
|
|
92
102
|
{
|
|
93
103
|
"authors": [
|
|
@@ -106,22 +116,50 @@ The resulting json will be:
|
|
|
106
116
|
|
|
107
117
|
```
|
|
108
118
|
|
|
109
|
-
Modifiers can be supplied on object creation and/or added later by calling `#add_modifier` method.
|
|
119
|
+
Modifiers can be supplied on object creation and/or added later by calling `#add_modifier` method. Please see specs for
|
|
120
|
+
examples.
|
|
121
|
+
|
|
122
|
+
### Nested schemas
|
|
123
|
+
|
|
124
|
+
JDE supports nested schemas. Just provide your element with a type of `array` and add a `schema` key for its data.
|
|
125
|
+
|
|
126
|
+
E.g. this is a valid real-life schema with nested data:
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
{
|
|
130
|
+
name: '$.Name',
|
|
131
|
+
code: '$.Code',
|
|
132
|
+
services: '$.Services[*].Code',
|
|
133
|
+
locations: {
|
|
134
|
+
path: '$.Locations[*]',
|
|
135
|
+
type: 'array',
|
|
136
|
+
schema: {
|
|
137
|
+
name: '$.Name',
|
|
138
|
+
type: '$.Type',
|
|
139
|
+
code: '$.Code'
|
|
140
|
+
}
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
```
|
|
110
144
|
|
|
111
145
|
## TODO
|
|
112
146
|
|
|
113
147
|
Update this readme for better usage cases. Add info on arrays and modifiers.
|
|
114
148
|
|
|
115
|
-
|
|
116
149
|
## Development
|
|
117
150
|
|
|
118
|
-
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can
|
|
151
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can
|
|
152
|
+
also run `bin/console` for an interactive prompt that will allow you to experiment.
|
|
119
153
|
|
|
120
|
-
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
|
|
154
|
+
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
|
|
155
|
+
version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
|
|
156
|
+
push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
121
157
|
|
|
122
158
|
## Contributing
|
|
123
159
|
|
|
124
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/austerlitz/json_data_extractor. This project
|
|
160
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/austerlitz/json_data_extractor. This project
|
|
161
|
+
is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to
|
|
162
|
+
the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
|
|
125
163
|
|
|
126
164
|
## License
|
|
127
165
|
|
|
@@ -129,4 +167,6 @@ The gem is available as open source under the terms of the [MIT License](https:/
|
|
|
129
167
|
|
|
130
168
|
## Code of Conduct
|
|
131
169
|
|
|
132
|
-
Everyone interacting in the JsonDataExtractor project’s codebases, issue trackers, chat rooms and mailing lists is
|
|
170
|
+
Everyone interacting in the JsonDataExtractor project’s codebases, issue trackers, chat rooms and mailing lists is
|
|
171
|
+
expected to follow
|
|
172
|
+
the [code of conduct](https://github.com/austerlitz/json_data_extractor/blob/master/CODE_OF_CONDUCT.md).
|
data/lib/json_data_extractor.rb
CHANGED
|
@@ -23,6 +23,7 @@ class JsonDataExtractor
|
|
|
23
23
|
path = val[:path]
|
|
24
24
|
modifiers = Array(val[:modifiers] || val[:modifier]).map(&:to_sym)
|
|
25
25
|
array_type = 'array' == val[:type]
|
|
26
|
+
nested = val.delete(:schema)
|
|
26
27
|
else
|
|
27
28
|
path = val
|
|
28
29
|
modifiers = []
|
|
@@ -35,8 +36,15 @@ class JsonDataExtractor
|
|
|
35
36
|
results[key] = nil
|
|
36
37
|
else
|
|
37
38
|
results[key] = apply_modifiers(extracted_data, modifiers)
|
|
38
|
-
|
|
39
|
+
|
|
40
|
+
# TODO yeah, this looks ugly. Address it later.
|
|
41
|
+
if !array_type
|
|
39
42
|
results[key] = results[key].first unless 1 < results[key].size
|
|
43
|
+
elsif nested
|
|
44
|
+
results[key] = []
|
|
45
|
+
Array(extracted_data).each do |item|
|
|
46
|
+
results[key] << self.class.new(item).extract(nested)
|
|
47
|
+
end
|
|
40
48
|
end
|
|
41
49
|
end
|
|
42
50
|
end
|
data/lib/src/version.rb
CHANGED