json_data_extractor 0.0.8 → 0.0.10

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ac39771f7b1358e30e26bfdc81b5b4ad5ac493238e84142e31331648f96fa37e
-  data.tar.gz: 69a0d03f65e05dfb292ae01f950986822ec04991771968ede64c63c6f202d99f
+  metadata.gz: be51d8200f061eb267a9c591934a29072729462d242c349c6057e48cb5e77627
+  data.tar.gz: 88213a955399a735f2cd15546d9c0b9ad22ba38de8798e9070d5a35e4a47902f
 SHA512:
-  metadata.gz: 403b5b95e7bfc0b7f530021980a10407f08be499092a80550a2dc55df7aeabc64197f68e6bf44f98c19be4ea8d57884db1205490970a57552ea924cf673c0f8a
-  data.tar.gz: 58260729ea70e88fdd1937cbbac0cffac998ce9dab686dedbbcb110e7618edd72a9bd71e494bfdc7441665a306e3ead800d9558b4c80b2fce525de3c70f90eee
+  metadata.gz: f724f3a5b3542644abe1103d8831bdefa0e6bbf2f073d8bfe928cce6f8d42992ecf570527f4adceba4b8a01f12fe0146511ae2ef80301f1a48a52b7e0f5c697a
+  data.tar.gz: e4318e950ec778eb19558cc1a7da37234f730ba977b42ff19f5be4d5b86c8b7a4b8a7e50a85705fa38c89d41d9cb0f39493d8f6c0bfa1e79215956f2e9817528

data/README.md

@@ -3,13 +3,15 @@
 NOTE: This is still a very early beta.
 
 Transform JSON data structures with the help of a simple schema and JsonPath expressions.
-Use the JsonDataExtractor gem to extract and modify data from complex JSON structures using a straightforward syntax
+Use the JsonDataExtractor gem to extract and modify data from complex JSON structures using a
+straightforward syntax
 and a range of built-in or custom modifiers.
 
 _Another try to make something for JSON that is XSLT for XML.
 We transform one JSON into another JSON with the help of a third JSON!!!111!!eleventy!!_
 
-Remap one JSON structure into another with some basic rules and [jsonpath](https://github.com/joshbuddy/jsonpath).
+Remap one JSON structure into another with some basic rules
+and [jsonpath](https://github.com/joshbuddy/jsonpath).
 
 Heavily inspired by [xml_data_extractor](https://github.com/monde-sistemas/xml_data_extractor).
 
@@ -32,8 +34,8 @@ Or install it yourself as:
 ## Usage
 
 JsonDataExtractor allows you to remap one JSON structure into another with some basic rules
-and [JSONPath](https://goessner.net/articles/JsonPath/) expressions. The process involves defining a schema that maps
-the input JSON structure to the desired output structure.
+and [JSONPath](https://goessner.net/articles/JsonPath/) expressions. The process involves defining a
+schema that maps the input JSON structure to the desired output structure.
 
 We'll base our examples on the following source:
 
@@ -78,15 +80,15 @@ We'll base our examples on the following source:
 
 ### Defining a Schema
 
-A schema consists of one or more mappings that specify how to extract data from the input JSON and where to place it in
-the output JSON.
+A schema consists of one or more mappings that specify how to extract data from the input JSON and
+where to place it in the output JSON.
 
-Each mapping has a path field that specifies the JsonPath expression to use for data extraction, and an optional
-modifier field that specifies one or more modifiers to apply to the extracted data. Modifiers are used to transform the
-data in some way before placing it in the output JSON.
+Each mapping has a path field that specifies the JsonPath expression to use for data extraction, and
+an optional modifier field that specifies one or more modifiers to apply to the extracted data.
+Modifiers are used to transform the data in some way before placing it in the output JSON.
 
-Here's an example schema that extracts the authors and categories from a JSON structure similar to the one used in the
-previous example (here it's in YAML just for readability):
+Here's an example schema that extracts the authors and categories from a JSON structure similar to
+the one used in the previous example (here it's in YAML just for readability):
 
 ```yaml
 schemas:
@@ -116,23 +118,66 @@ The resulting json will be:
 
 ```
 
-Modifiers can be supplied on object creation and/or added later by calling `#add_modifier` method. Please see specs for
-examples.
+### Modifiers
+
+Modifiers can be supplied on object creation and/or added later by calling `#add_modifier` method.
+Please see specs for examples.
+Modifiers allow you to perform transformations on the extracted data before it is returned. You can
+use modifiers to clean up the data, format it, or apply any custom logic you need.
+
+Modifiers can be defined in two ways: by providing a symbol corresponding to the name of the method
+or lambda that should be called on each extracted value, or by providing an anonymous lambda. Here's
+an example schema that uses both types of modifiers:
+
+```ruby
+schema = {
+  name:  '$.name',
+  age:   { path: '$.age', modifier: :to_i },
+  email: { path: '$.contact.email', modifiers: [:downcase, lambda { |email| email.gsub(/\s/, '') }] }
+}
+
+```
+
+In this schema, the name value is simply extracted as-is. The age value is extracted from the JSON,
+but it is modified with the `to_i` method, which converts the value to an integer. The email value
+is extracted from a nested object, and then passed through two modifiers: first `downcase` is called
+to convert the email address to all lowercase letters, and then an anonymous lambda is called to
+remove any whitespace in the email address.
+
+You can also define custom modifiers by passing a lambda to the `add_modifier` method on a
+JsonDataExtractor instance:
+
+```ruby
+extractor = JsonDataExtractor.new(json_data)
+extractor.add_modifier(:remove_newlines) { |value| value.gsub("\n", '') }
+
+schema = {
+  name: 'name',
+  bio:  { path: 'bio', modifiers: [:remove_newlines] }
+}
+
+results = extractor.extract(schema)
+
+```
+
+Modifiers are called in the order in which they are defined, so keep that in mind when defining your
+schema.
 
 ### Nested schemas
 
-JDE supports nested schemas. Just provide your element with a type of `array` and add a `schema` key for its data.
+JDE supports nested schemas. Just provide your element with a type of `array` and add a `schema` key
+for its data.
 
 E.g. this is a valid real-life schema with nested data:
 
 ```json
 {
-  "name":      "$.Name",
-  "code":      "$.Code",
-  "services":  "$.Services[*].Code",
+  "name": "$.Name",
+  "code": "$.Code",
+  "services": "$.Services[*].Code",
   "locations": {
-    "path":   "$.Locations[*]",
-    "type":   "array",
+    "path": "$.Locations[*]",
+    "type": "array",
     "schema": {
       "name": "$.Name",
       "type": "$.Type",
@@ -148,25 +193,32 @@ Update this readme for better usage cases. Add info on arrays and modifiers.
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run
+the tests. You can
 also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
-version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new
+version, update the
+version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag
+for the version,
 push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/austerlitz/json_data_extractor. This project
-is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to
+Bug reports and pull requests are welcome on GitHub
+at https://github.com/austerlitz/json_data_extractor. This project
+is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere
+to
 the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+The gem is available as open source under the terms of
+the [MIT License](https://opensource.org/licenses/MIT).
 
 ## Code of Conduct
 
-Everyone interacting in the JsonDataExtractor project’s codebases, issue trackers, chat rooms and mailing lists is
+Everyone interacting in the JsonDataExtractor project’s codebases, issue trackers, chat rooms and
+mailing lists is
 expected to follow
 the [code of conduct](https://github.com/austerlitz/json_data_extractor/blob/master/CODE_OF_CONDUCT.md).

data/json_data_extractor.gemspec

@@ -26,7 +26,7 @@ transformations. The schema is defined as a simple Ruby hash that maps keys to p
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ['lib']
 
-  spec.add_development_dependency 'bundler', '~> 1.17'
+  spec.add_development_dependency 'bundler'
   spec.add_development_dependency 'rake', '~> 10.0'
   spec.add_development_dependency 'rspec', '~> 3.0'
   spec.add_development_dependency 'pry'

data/lib/json_data_extractor.rb

@@ -22,7 +22,16 @@ class JsonDataExtractor
       if val.is_a?(Hash)
         val.transform_keys!(&:to_sym)
         path       = val[:path]
-        modifiers  = Array(val[:modifiers] || val[:modifier]).map(&:to_sym)
+        modifiers  = Array(val[:modifiers] || val[:modifier]).map do |mod|
+          case mod
+          when Symbol, Proc
+            mod
+          when String
+            mod.to_sym
+          else
+            raise ArgumentError, "Invalid modifier: #{mod.inspect}"
+          end
+        end
         array_type = 'array' == val[:type]
         nested     = val.dup.delete(:schema)
       else
@@ -37,20 +46,17 @@ class JsonDataExtractor
       else
         results[key] = apply_modifiers(extracted_data, modifiers)
 
-        # TODO yeah, this looks ugly. Address it later.
-        if !array_type
-          if nested
-            results[key] = self.class.new(results[key].first, @modifiers).extract(nested)
-            # todo there might be a problem if results[key] yields more than 1 item
-            # Yeah, we'll get back to it later. Or send your PRs
-          else
-            results[key] = results[key].first unless 1 < results[key].size
-          end
-        elsif nested
-          results[key] = []
-          Array(extracted_data).each do |item|
-            results[key] << self.class.new(item, @modifiers).extract(nested)
-          end
+        if array_type && nested
+          results[key] = extract_nested_data(results[key], nested)
+        elsif !array_type && nested
+          results[key] = extract_nested_data(results[key], nested).first
+        elsif !array_type && 1 < results[key].size
+          # TODO: handle case where results[key] has more than one item
+          # do nothing for now
+        elsif array_type && !nested
+          # do nothing, it is already an array
+        else
+          results[key] = results[key].first
         end
       end
     end
@@ -59,6 +65,12 @@ class JsonDataExtractor
 
   private
 
+  def extract_nested_data(data, schema)
+    Array(data).map do |item|
+      self.class.new(item, modifiers).extract(schema)
+    end
+  end
+
   def apply_modifiers(data, modifiers)
     data.map do |value|
       modified_value = value

data/lib/src/version.rb

@@ -1,3 +1,3 @@
 class JsonDataExtractor
-  VERSION = '0.0.8'
+  VERSION = '0.0.10'
 end

metadata

@@ -1,29 +1,29 @@
 --- !ruby/object:Gem::Specification
 name: json_data_extractor
 version: !ruby/object:Gem::Version
-  version: 0.0.8
+  version: 0.0.10
 platform: ruby
 authors:
 - Max Buslaev
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-05-10 00:00:00.000000000 Z
+date: 2023-05-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '1.17'
+        version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '1.17'
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement