json_data_extractor 0.0.1 → 0.0.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a4263e61419466e8b50f31914c91697a628b1a0921c5bd3b4f783f81ee969f62
-  data.tar.gz: f93d27bfca788b17431cabee2e36632e4b3b349592cc1d2e8889b835ac162d9e
+  metadata.gz: 101c20f78189fbd300dedf5a03779d19558d65fb4f4d2d4767ec8b1f638aafde
+  data.tar.gz: 924d26135a261281c14051ab7aa305debcec61688fd6deda4aeb322da8028ecd
 SHA512:
-  metadata.gz: ff5e1c2da47047683df1a108ace950e8d0fed4b0435360dedbf90af0a3d58ac11167da3eff6ce3fcd7fd6f705daef794a7942b96a6fb79175b8322c93218d242
-  data.tar.gz: a6ff91e82661c4adc31a15ec600bd573f783c2b43da06ab17754712a7991047d9ea10c9f20fe8df5385428b9e7d3fef1bfb84fff145d57b5baf551f14d3b79b2
+  metadata.gz: df55bd36b0e00ac0bd4f427451dcf69425538dc06382632b261ec03aaa6cb24b0648873645b80c609b1e7329e3f9aac85fa97612afdcc492efb6efa91d062f83
+  data.tar.gz: df3a0c646a823792fa1bce461cab8eeb037b6be9906fbb1f5608cf69da4e13133c11f2e71aa90033b3924ea4ccc11f6a9900ca17b263c44702583c6eb3b679fa

data/README.md

@@ -1,7 +1,13 @@
 # JsonDataExtractor
 
-Another try to make something for JSON that is XSLT for XML. 
-We transform one JSON into another JSON with the help of a third JSON!!!111!!eleventy!!
+NOTE: This is still a very early beta.
+
+Transform JSON data structures with the help of a simple schema and JsonPath expressions.
+Use the JsonDataExtractor gem to extract and modify data from complex JSON structures using a straightforward syntax
+and a range of built-in or custom modifiers.
+
+_Another try to make something for JSON that is XSLT for XML.
+We transform one JSON into another JSON with the help of a third JSON!!!111!!eleventy!!_
 
 Remap one JSON structure into another with some basic rules and [jsonpath](https://github.com/joshbuddy/jsonpath).
 
@@ -25,9 +31,11 @@ Or install it yourself as:
 
 ## Usage
 
-Assuming you are familiar with [JSONPath](https://goessner.net/articles/JsonPath/), you can write simple mappers that will remap incoming data into the structure you need.
+JsonDataExtractor allows you to remap one JSON structure into another with some basic rules
+and [JSONPath](https://goessner.net/articles/JsonPath/) expressions. The process involves defining a schema that maps
+the input JSON structure to the desired output structure.
 
-With the following source:
+We'll base our examples on the following source:
 
 ```json
 {
@@ -68,7 +76,17 @@ With the following source:
 }
 ```
 
-and the following schema:
+### Defining a Schema
+
+A schema consists of one or more mappings that specify how to extract data from the input JSON and where to place it in
+the output JSON.
+
+Each mapping has a path field that specifies the JsonPath expression to use for data extraction, and an optional
+modifier field that specifies one or more modifiers to apply to the extracted data. Modifiers are used to transform the
+data in some way before placing it in the output JSON.
+
+Here's an example schema that extracts the authors and categories from a JSON structure similar to the one used in the
+previous example (here it's in YAML just for readability):
 
 ```yaml
 schemas:
@@ -77,7 +95,9 @@ schemas:
     modifier: downcase
   categories: $..category
 ```
+
 The resulting json will be:
+
 ```json
 {
   "authors": [
@@ -96,22 +116,50 @@ The resulting json will be:
 
 ```
 
-Modifiers can be supplied on object creation and/or added later by calling `#add_modifier` method.  Please see specs for examples.
+Modifiers can be supplied on object creation and/or added later by calling `#add_modifier` method. Please see specs for
+examples.
 
-## TODO
+### Nested schemas
+
+JDE supports nested schemas. Just provide your element with a type of `array` and add a `schema` key for its data.
 
-Update this readme for better usage cases. 
+E.g. this is a valid real-life schema with nested data:
+
+```ruby
+{
+  name:      '$.Name',
+  code:      '$.Code',
+  services:  '$.Services[*].Code',
+  locations: {
+    path:   '$.Locations[*]',
+    type:   'array',
+    schema: {
+      name: '$.Name',
+      type: '$.Type',
+      code: '$.Code'
+    }
+  }
+}
+```
+
+## TODO
 
+Update this readme for better usage cases. Add info on arrays and modifiers.
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can
+also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
+version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
+push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/austerlitz/json_data_extractor. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+Bug reports and pull requests are welcome on GitHub at https://github.com/austerlitz/json_data_extractor. This project
+is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to
+the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
 ## License
 
@@ -119,4 +167,6 @@ The gem is available as open source under the terms of the [MIT License](https:/
 
 ## Code of Conduct
 
-Everyone interacting in the JsonDataExtractor project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/austerlitz/json_data_extractor/blob/master/CODE_OF_CONDUCT.md).
+Everyone interacting in the JsonDataExtractor project’s codebases, issue trackers, chat rooms and mailing lists is
+expected to follow
+the [code of conduct](https://github.com/austerlitz/json_data_extractor/blob/master/CODE_OF_CONDUCT.md).

data/json_data_extractor.gemspec

@@ -1,33 +1,36 @@
-
-lib = File.expand_path("../lib", __FILE__)
+lib = File.expand_path('../lib', __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
-require "src/version"
+require 'src/version'
 
 Gem::Specification.new do |spec|
-  spec.name          = "json_data_extractor"
-  spec.version       = JsonDataExtractor::VERSION
-  spec.authors       = ["Max Buslaev"]
-  spec.email         = ["max@buslaev.net"]
+  spec.name    = 'json_data_extractor'
+  spec.version = JsonDataExtractor::VERSION
+  spec.authors = ['Max Buslaev']
+  spec.email   = ['max@buslaev.net']
 
-  spec.summary       = %q{Transform JSON data structures with the help of simple schema.}
-  spec.description   = %q{Transform JSON data structures with the help of simple schema.}
-  spec.homepage      = "https://github.com/austerlitz/json_data_extractor"
-  spec.license       = "MIT"
+  spec.summary     = %q{Transform JSON data structures with the help of a simple schema and JsonPath expressions.
+Use the JsonDataExtractor gem to extract and modify data from complex JSON structures using a straightforward syntax
+and a range of built-in or custom modifiers.}
+  spec.description = %q{json_data_extractor makes it easy to extract data from complex JSON structures,
+such as API responses or configuration files, using a schema that defines the path to the data and any necessary
+transformations. The schema is defined as a simple Ruby hash that maps keys to paths and optional modifiers.}
+  spec.homepage    = 'https://github.com/austerlitz/json_data_extractor'
+  spec.license     = 'MIT'
 
-   # Specify which files should be added to the gem when it is released.
+  # Specify which files should be added to the gem when it is released.
   # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
   spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
     `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
   end
-  spec.bindir        = "exe"
+  spec.bindir        = 'exe'
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
-  spec.require_paths = ["lib"]
+  spec.require_paths = ['lib']
 
-  spec.add_development_dependency "bundler", "~> 1.17"
-  spec.add_development_dependency "rake", "~> 10.0"
-  spec.add_development_dependency "rspec", "~> 3.0"
-  spec.add_development_dependency "pry"
-  spec.add_development_dependency "amazing_print"
+  spec.add_development_dependency 'bundler', '~> 1.17'
+  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'rspec', '~> 3.0'
+  spec.add_development_dependency 'pry'
+  spec.add_development_dependency 'amazing_print'
 
   spec.add_dependency 'jsonpath'
 end

data/lib/json_data_extractor.rb

@@ -1,32 +1,50 @@
 require "src/version"
 
 class JsonDataExtractor
-  attr_reader :json_data, :modifiers
+  attr_reader :data, :modifiers
 
   def initialize(json_data, modifiers = {})
-    @json_data = json_data
-    @modifiers = modifiers
+    @data      = json_data
+    @modifiers = modifiers.transform_keys(&:to_sym) # todo address this later
   end
 
+  # @param modifier_name [String, Symbol]
   def add_modifier(modifier_name, &block)
+    modifier_name            = modifier_name.to_sym unless modifier_name.is_a?(Symbol)
     modifiers[modifier_name] = block
   end
 
+  # @param schema [Hash] schema of the expected data mapping
   def extract(schema)
     results = {}
     schema.each do |key, val|
-      path = val.is_a?(Hash) && val['path'] ? val['path'] : val
-      modifiers = val.is_a?(Hash) ? Array(val['modifiers'] || val['modifier']) : []
+      if val.is_a?(Hash)
+        val.transform_keys!(&:to_sym)
+        path       = val[:path]
+        modifiers  = Array(val[:modifiers] || val[:modifier]).map(&:to_sym)
+        array_type = 'array' == val[:type]
+        nested     = val.delete(:schema)
+      else
+        path      = val
+        modifiers = []
+      end
 
-      json_path = JsonPath.new(path)
-      extracted_data = json_path.on(@json_data).flatten.compact
+      json_path      = JsonPath.new(path)
+      extracted_data = json_path.on(@data)
 
       if extracted_data.empty?
         results[key] = nil
       else
         results[key] = apply_modifiers(extracted_data, modifiers)
-        unless val.is_a?(Hash) && val['type'] && 'array' == val['type']
+
+        # TODO yeah, this looks ugly. Address it later.
+        if !array_type
           results[key] = results[key].first unless 1 < results[key].size
+        elsif nested
+          results[key] = []
+          Array(extracted_data).each do |item|
+            results[key] << self.class.new(item).extract(nested)
+          end
         end
       end
     end
@@ -39,11 +57,7 @@ class JsonDataExtractor
     data.map do |value|
       modified_value = value
       modifiers.each do |modifier|
-        if @modifiers.key?(modifier.to_sym)
-          modified_value = @modifiers[modifier.to_sym].call(modified_value)
-        else
-          modified_value = apply_single_modifier(modifier, modified_value)
-        end
+        modified_value = apply_single_modifier(modifier, modified_value)
       end
       modified_value
     end

data/lib/src/version.rb

@@ -1,3 +1,3 @@
 class JsonDataExtractor
-  VERSION = "0.0.1"
+  VERSION = '0.0.3'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: json_data_extractor
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.0.3
 platform: ruby
 authors:
 - Max Buslaev
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-03-25 00:00:00.000000000 Z
+date: 2023-03-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -94,7 +94,10 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description: Transform JSON data structures with the help of simple schema.
+description: |-
+  json_data_extractor makes it easy to extract data from complex JSON structures,
+  such as API responses or configuration files, using a schema that defines the path to the data and any necessary
+  transformations. The schema is defined as a simple Ruby hash that maps keys to paths and optional modifiers.
 email:
 - max@buslaev.net
 executables: []
@@ -136,5 +139,8 @@ requirements: []
 rubygems_version: 3.0.3
 signing_key: 
 specification_version: 4
-summary: Transform JSON data structures with the help of simple schema.
+summary: Transform JSON data structures with the help of a simple schema and JsonPath
+  expressions. Use the JsonDataExtractor gem to extract and modify data from complex
+  JSON structures using a straightforward syntax and a range of built-in or custom
+  modifiers.
 test_files: []