burner 1.9.0.pre.alpha → 1.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: eca471b05b356ad1e96f71c0173673109622e14dc7d7d46bdab0beccc1e03396
-  data.tar.gz: 55f81d7080c1f0eb4a4cc1d51df7534376306362a1d97a8369f3fd85c0107a3b
+  metadata.gz: ce126372d1a0fa7d4873cb58c354372a13a013a56f89f4814950eb497e9fb095
+  data.tar.gz: 0efcd55a83ef5b1c0cc716cafeae2204f4207c894419a3e40450d46e505b8ca7
 SHA512:
-  metadata.gz: bbd76fedb82b0499f7f8913e5530a1c35e2d4fc613ef11e95fb8ad6c759a28a7f4737d5ae29a71e0409dbd8f45b03de5b96dc22b84e3b85384cb256d1742eb9c
-  data.tar.gz: 311f81afb55fdbf5548c9394ec1ee30e020833f205e4e1c53fef87630c2e77d0b6cbd6639689efaa8956ae7bb188555f35ffb1dca7c08c2cedb6881f959cd508
+  metadata.gz: 5ebf524b5794577e5d287a917a52207962241ad98aa68c75069aee65d0bfeb06ae4a20af84b4ce2568449f72c2611922f69cbc130fb91b859c60f3fd28e34465
+  data.tar.gz: 18db46b8876228fec2357672b6abe6cf07a86434ab1595667d18c230122d2dfac3ae03cf0794a097be7fc75c42444a368a7ede5ff8207eca5defec57c6022775

data/CHANGELOG.md

@@ -1,4 +1,26 @@
-# 1.9.0 (TBD)
+# 1.12.0 (August 12th, 2021)
+
+Enhanced Job:
+
+* b/collection/flat_file_parse now supports key mappings.
+
+Added Jobs:
+
+* b/collection/prepend (alias for unshift)
+* b/collection/unshift
+
+# 1.11.0 (May 17th, 2021)
+
+Added Jobs:
+
+* b/collection/flat_file_parse
+
+# 1.10.0 (May 17th, 2021)
+
+Added Jobs:
+
+* b/collection/only_keys
+# 1.9.0 (April 13th, 2021)
 
 Added Jobs:
 

data/README.md

@@ -266,16 +266,20 @@ This library only ships with very basic, rudimentary jobs that are meant to just
 * **b/collection/arrays_to_objects** [mappings, register]: Convert an array of arrays to an array of objects.
 * **b/collection/coalesce** [grouped_register, insensitive, key_mappings, keys, register, separator]: Merge two datasets together based on the key values of one dataset (array) with a grouped dataset (hash).  If insensitive (false by default) is true then each key's value will be converted/coerced to a lowercase string.
 * **b/collection/concatenate** [from_registers, to_register]: Concatenate each from_register's value and place the newly concatenated array into the to_register.  Note: this does not do any deep copying and should be assumed it is shallow copying all objects.
+* **b/collection/flat_file_parse** [keys_register, register, separator, key_mappings]: Map an array of arrays to an array of hashes.  These keys can be realized at run-time as they are pulled from the first entry in the array.  The `keys_register` will also be set to the keys used for mapping. Only keys that are mapped will be included in the `keys_register` array if `key_mappings` are defined. Otherwise all keys that are pulled from the first entry in the `register` will be included in the `keys_register`.
 * **b/collection/graph** [config, key, register]: Use [Hashematics](https://github.com/bluemarblepayroll/hashematics) to turn a flat array of objects into a deeply nested object tree.
 * **b/collection/group** [insensitive, keys, register, separator]: Take a register's value (an array of objects) and group the objects by the specified keys.  If insensitive (false by default) is true then each key's value will be converted/coerced to a lowercase string.
 * **b/collection/nested_aggregate** [register, key_mappings, key, separator]: Traverse a set of objects, resolving key's value for each object, optionally copying down key_mappings to the child records, then merging all the inner records together.
 * **b/collection/number** [key, register, separator, start_at]: This job can iterate over a set of records and sequence them (set the specified key to a sequential index value.)
 * **b/collection/objects_to_arrays** [mappings, register]: Convert an array of objects to an array of arrays.
+* **b/collection/only_keys** [keys_register, register, separator]: Limit an array of objects' keys to a specified set of keys.  These keys can be realized at run-time as they are pulled from another register (`keys_register`) thus making it dynamic.
 * **b/collection/pivot** [unique_keys, insensitive, other_keys, pivot_key, pivot_value_key, register, separator]:
 Take an array of objects and pivot a key into multiple keys.  It essentially takes all the values for a key and creates N number of keys (one per value.) Under the hood it uses HashMath's [Record](https://github.com/bluemarblepayroll/hash_math#record-the-hash-prototype) and [Table](https://github.com/bluemarblepayroll/hash_math#table-the-double-hash-hash-of-hashes) classes.
+* **b/collection/prepend** [from_registers, to_register]: Alias for b\collection\unshift.
 * **b/collection/shift** [amount, register]: Remove the first N number of elements from an array.
 * **b/collection/transform** [attributes, exclusive, separator, register]: Iterate over all objects and transform each key per the attribute transformers specifications.  If exclusive is set to false then the current object will be overridden/merged.  Separator can also be set for key path support.  This job uses [Realize](https://github.com/bluemarblepayroll/realize), which provides its own extendable value-transformation pipeline.  If an attribute is not set with `explicit: true` then it will automatically start from the key's value from the record.  If `explicit: true` is started, then it will start from the record itself.
 * **b/collection/unpivot** [pivot_set, register]: Take an array of objects and unpivot specific sets of keys into rows.  Under the hood it uses [HashMath's Unpivot class](https://github.com/bluemarblepayroll/hash_math#unpivot-hash-key-coalescence-and-row-extrapolation).
+* **b/collection/unshift** [from_registers, register]: Adds the values of the `from_registers` to the `register` array. All existing elements in the `register` array will be shifted upwards.
 * **b/collection/validate** [invalid_register, join_char, message_key, register, separator, validations]: Take an array of objects, run it through each declared validator, and split the objects into two registers.  The valid objects will be split into the current register while the invalid ones will go into the invalid_register as declared.  Optional arguments, join_char and message_key, help determine the compiled error messages.  The separator option can be utilized to use dot-notation for validating keys.  See each validation's options by viewing their classes within the `lib/modeling/validations` directory.
 * **b/collection/values** [include_keys, register]: Take an array of objects and call `#values` on each object. If include_keys is true (it is false by default), then call `#keys` on the first object and inject that as a "header" object.
 * **b/collection/zip** [base_register, register, with_register]: Combines `base_register` and `with_register`s' data to form one single array in `register`.  It will combine each element, positionally in each array to form the final array.  For example:  ['hello', 'bugs'] + ['world', 'bunny'] => [['hello', 'world'], ['bugs', 'bunny']]

data/lib/burner/job_with_dynamic_keys.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'job_with_register'
+
+module Burner
+  # Add on a register attribute to the configuration for a job.  This indicates that a job
+  # either accesses and/or mutates the payload's registers.
+  class JobWithDynamicKeys < JobWithRegister
+    attr_reader :key_mappings,
+                :keys_register,
+                :resolver
+
+    def initialize(
+      keys_register:,
+      name: '',
+      register: DEFAULT_REGISTER,
+      separator: BLANK,
+      key_mappings: []
+    )
+      super(name: name, register: register)
+
+      @key_mappings  = Modeling::KeyMapping.array(key_mappings)
+      @keys_register = keys_register.to_s
+      @resolver      = Objectable.resolver(separator: separator)
+
+      freeze
+    end
+  end
+end

data/lib/burner/job_with_register.rb

@@ -13,6 +13,8 @@ module Burner
   # Add on a register attribute to the configuration for a job.  This indicates that a job
   # either accesses and/or mutates the payload's registers.
   class JobWithRegister < Job
+    BLANK = ''
+
     attr_reader :register
 
     def initialize(name: '', register: DEFAULT_REGISTER)

data/lib/burner/jobs.rb

@@ -25,15 +25,19 @@ module Burner
     register 'b/collection/arrays_to_objects', Library::Collection::ArraysToObjects
     register 'b/collection/coalesce',          Library::Collection::Coalesce
     register 'b/collection/concatenate',       Library::Collection::Concatenate
-    register 'b/collection/graph',             Library::Collection::Graph
     register 'b/collection/group',             Library::Collection::Group
+    register 'b/collection/flat_file_parse',   Library::Collection::FlatFileParse
+    register 'b/collection/graph',             Library::Collection::Graph
     register 'b/collection/nested_aggregate',  Library::Collection::NestedAggregate
     register 'b/collection/number',            Library::Collection::Number
     register 'b/collection/objects_to_arrays', Library::Collection::ObjectsToArrays
+    register 'b/collection/only_keys',         Library::Collection::OnlyKeys
     register 'b/collection/pivot',             Library::Collection::Pivot
+    register 'b/collection/prepend',           Library::Collection::Unshift
     register 'b/collection/shift',             Library::Collection::Shift
     register 'b/collection/transform',         Library::Collection::Transform
     register 'b/collection/unpivot',           Library::Collection::Unpivot
+    register 'b/collection/unshift',           Library::Collection::Unshift
     register 'b/collection/values',            Library::Collection::Values
     register 'b/collection/validate',          Library::Collection::Validate
     register 'b/collection/zip',               Library::Collection::Zip

data/lib/burner/library.rb

@@ -7,7 +7,7 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-require_relative 'job_with_register'
+require_relative 'job_with_dynamic_keys'
 
 require_relative 'library/echo'
 require_relative 'library/nothing'
@@ -16,15 +16,18 @@ require_relative 'library/sleep'
 require_relative 'library/collection/arrays_to_objects'
 require_relative 'library/collection/coalesce'
 require_relative 'library/collection/concatenate'
+require_relative 'library/collection/flat_file_parse'
 require_relative 'library/collection/graph'
 require_relative 'library/collection/group'
 require_relative 'library/collection/nested_aggregate'
 require_relative 'library/collection/number'
 require_relative 'library/collection/objects_to_arrays'
+require_relative 'library/collection/only_keys'
 require_relative 'library/collection/pivot'
 require_relative 'library/collection/shift'
 require_relative 'library/collection/transform'
 require_relative 'library/collection/unpivot'
+require_relative 'library/collection/unshift'
 require_relative 'library/collection/validate'
 require_relative 'library/collection/values'
 require_relative 'library/collection/zip'

data/lib/burner/library/collection/flat_file_parse.rb

@@ -0,0 +1,147 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Burner
+  module Library
+    module Collection
+      # Convert an array of arrays to an array of objects.  The difference between this
+      # job and ArraysToObjects is that this one does not require mappings and instead
+      # will use the first entry as the array of keys to positionally map to.
+      # If key mappings are specified then the keys register will only contain
+      # the keys that are mapped. The register will still contain mapped and unmapped keys.
+      #
+      # For example, if a register had this:
+      #
+      #   [['id', 'first', 'last'], [1, 'frank', 'rizzo']]
+      #
+      # Then executing this job would result in this:
+      #
+      #   [{ 'id' => 1, 'first' => frank, 'last' => rizzo }]
+      #
+      # As a side-effect, the keys_register would result in this, if no key mappings were specified:
+      #
+      #   ['id', 'first', 'last']
+      #
+      # If key mappings are specified:
+      # [
+      #
+      #  {
+      #     from: 'first',
+      #     to: 'first_name'
+      #  }
+      #  {
+      #     from: 'iban',
+      #     to: 'iban_number'
+      #  }
+      # ]
+      # Then the register will now be:
+      #   [{ 'id' => 1, 'first_name' => frank, 'last' => rizzo }]
+      #
+      # And the keys_register will now contain:
+      #   ['first_name']
+      #
+      # Since 'last' did not have a key mapping entry it does not exist in the keys register.
+      # In addition, even though 'iban' does have a key mapping it does not exist in the
+      # register's payload, so it also does not exist in the keys register.
+      #
+      # Expected Payload[register] input: array of arrays.
+      # Payload[register] output: An array of hashes.
+      # Payload[keys_register] output; An array of key names.
+      class FlatFileParse < JobWithDynamicKeys
+        def perform(output, payload)
+          objects     = array(payload[register])
+          keys        = array(objects.shift)
+          count       = objects.length
+          mapped_keys = update_keys_using_mappings(keys, output)
+
+          payload[register]      = objects.map { |object| transform(object, mapped_keys, output) }
+          payload[keys_register] = get_mapped_keys(mapped_keys)
+
+          output.detail("Mapping #{count} array(s)")
+
+          output.detail("Mapping keys register array to: #{payload[keys_register].join(', ')}")
+        end
+
+        private
+
+        def transform(object, keys, output)
+          object.each_with_object({}).with_index do |(value, memo), index|
+            next if index >= keys.length
+
+            key_hash = keys[index]
+
+            key = key_hash[:unmapped_key_name]
+
+            mapped_key = key_hash[:mapped_key_name]
+
+            key = mapped_key if mapped_key
+
+            resolver.set(memo, key, value)
+
+            output.detail("Using key #{key}")
+          end
+        end
+
+        def get_mapped_keys(mapped_keys)
+          mapped_keys.each_with_object([]) do |mapped_key_hash, memo|
+            mapped_key = mapped_key_hash[:mapped_key_name]
+
+            memo << mapped_key if mapped_key
+          end
+        end
+
+        def update_keys_using_mappings(keys, output)
+          if key_mappings.count.positive?
+            populate_key_hashes_from_mappings(keys, output)
+          else
+            populate_key_hashes_without_mappings(keys)
+          end
+        end
+
+        def populate_key_hashes_from_mappings(keys, output)
+          keys.each_with_object([]) do |key, memo|
+            mapped_key_name = find_key_name_to_use_from_mappings(key, output)
+
+            mapped_key_name = nil if key == mapped_key_name
+
+            memo << create_key_hash(key, mapped_key_name)
+          end
+        end
+
+        def populate_key_hashes_without_mappings(keys)
+          keys.each_with_object([]) do |key, memo|
+            memo << create_key_hash(key, key)
+          end
+        end
+
+        def create_key_hash(unmapped_key_name, mapped_key_name)
+          { unmapped_key_name: unmapped_key_name, mapped_key_name: mapped_key_name }
+        end
+
+        def find_key_name_to_use_from_mappings(key, output)
+          key_to_use = key
+
+          key_mappings.each do |key_mapping|
+            next unless key_mapping.from.downcase == key.downcase
+
+            key_to_use = populate_key_to_use(key_mapping)
+            output.detail("Using key_mapping from key #{key} to #{key_to_use}")
+            break
+          end
+
+          key_to_use
+        end
+
+        def populate_key_to_use(key_mapping)
+          key_mapping.to unless key_mapping.to.empty?
+        end
+      end
+    end
+  end
+end

data/lib/burner/library/collection/only_keys.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Burner
+  module Library
+    module Collection
+      # This job knows how to take an array of objects and limit it to a specific set of keys.
+      # The keys are pulled from another register which helps make it dynamic (you can load
+      # up this other register with a dynamic list of keys at run-time.)
+      #
+      # Expected Payload[register] input: array of objects.
+      # Payload[register] output: An array of objects.
+      class OnlyKeys < JobWithDynamicKeys
+        def perform(output, payload)
+          objects = array(payload[register])
+          count   = objects.length
+          keys    = array(payload[keys_register])
+
+          output.detail("Dynamically limiting #{count} object(s) with key(s): #{keys.join(', ')}")
+
+          payload[register] = objects.map { |object| transform(object, keys) }
+        end
+
+        private
+
+        def transform(object, keys)
+          keys.each_with_object({}) do |key, memo|
+            value = resolver.get(object, key)
+
+            resolver.set(memo, key, value)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/burner/library/collection/transform.rb

@@ -21,8 +21,6 @@ module Burner
       # Expected Payload[register] input: array of objects.
       # Payload[register] output: An array of objects.
       class Transform < JobWithRegister
-        BLANK = ''
-
         attr_reader :attribute_renderers,
                     :exclusive,
                     :resolver

data/lib/burner/library/collection/unshift.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Burner
+  module Library
+    module Collection
+      # Adds the values of the from_registers to the to_register array.
+      # All existing elements in the register array will be shifted upwards.
+      #
+      # Expected Payload[from_registers] input: Array containing names of registers
+      # whose values to prepend.
+      # Payload[register] output: An array with the from_registers'
+      # payload added to the beginning.
+      class Unshift < JobWithRegister
+        attr_reader :from_registers
+
+        def initialize(from_registers: [], name: '', register: DEFAULT_REGISTER)
+          super(name: name, register: register)
+
+          @from_registers = Array(from_registers)
+
+          freeze
+        end
+
+        def perform(output, payload)
+          output.detail("Prepending registers: '#{from_registers}' to: '#{register}'")
+
+          payload_register = array(payload[register])
+
+          from_registers.each do |from_register|
+            from_register_value = payload[from_register]
+
+            payload_register.unshift(from_register_value)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/burner/library/param/base.rb

@@ -12,8 +12,6 @@ module Burner
     module Param
       # Common logic shared across Param job subclasses.
       class Base < JobWithRegister
-        BLANK = ''
-
         attr_reader :param_key
 
         def initialize(name: BLANK, param_key: BLANK, register: DEFAULT_REGISTER)

data/lib/burner/version.rb

@@ -8,5 +8,5 @@
 #
 
 module Burner
-  VERSION = '1.9.0-alpha'
+  VERSION = '1.12.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: burner
 version: !ruby/object:Gem::Version
-  version: 1.9.0.pre.alpha
+  version: 1.12.0
 platform: ruby
 authors:
 - Matthew Ruggio
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-04-13 00:00:00.000000000 Z
+date: 2021-08-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: acts_as_hashable
@@ -240,21 +240,25 @@ files:
 - lib/burner/disks/local.rb
 - lib/burner/job.rb
 - lib/burner/job_set.rb
+- lib/burner/job_with_dynamic_keys.rb
 - lib/burner/job_with_register.rb
 - lib/burner/jobs.rb
 - lib/burner/library.rb
 - lib/burner/library/collection/arrays_to_objects.rb
 - lib/burner/library/collection/coalesce.rb
 - lib/burner/library/collection/concatenate.rb
+- lib/burner/library/collection/flat_file_parse.rb
 - lib/burner/library/collection/graph.rb
 - lib/burner/library/collection/group.rb
 - lib/burner/library/collection/nested_aggregate.rb
 - lib/burner/library/collection/number.rb
 - lib/burner/library/collection/objects_to_arrays.rb
+- lib/burner/library/collection/only_keys.rb
 - lib/burner/library/collection/pivot.rb
 - lib/burner/library/collection/shift.rb
 - lib/burner/library/collection/transform.rb
 - lib/burner/library/collection/unpivot.rb
+- lib/burner/library/collection/unshift.rb
 - lib/burner/library/collection/validate.rb
 - lib/burner/library/collection/values.rb
 - lib/burner/library/collection/zip.rb
@@ -321,9 +325,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '2.5'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">"
+  - - ">="
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
 rubygems_version: 3.0.3
 signing_key: