burner 1.0.0.pre.alpha.6 → 1.0.0.pre.alpha.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 73263c161bce669b99286160acb7129d33c529162da964b72fd34321cc7e3724
-  data.tar.gz: 18fd59799a73d9a8a3bf691a9dd3f8a004d442b37862bf83cbe347fbddd7e49a
+  metadata.gz: 8cb851b5c317d3566c1b3b77c9c904e26970de8e0ce4329e2c8b83336c372b02
+  data.tar.gz: 0ad28e56c8b52ceede4ed321812e45edfaef2bcd4000428738c348e5e849092a
 SHA512:
-  metadata.gz: 7db0c2be13a882885b97681ec7c9fd60b6702368d673d97ba0aa484b446db0c77c9f823adf03c8d9e970b2078b43a99d2d159ba97df345124a461497718b7046
-  data.tar.gz: 509382aa7fb48e116b8b3eb145a8b8226c1e4d4282eec1aabb10dcc9f7b333b906a7887d97cee4a96ae730dd7c02a0b0d4cd4a7b06dbfcc90349cfd82767e048
+  metadata.gz: cf5bc810ffa5dba106e6538dbab2f44e6763f5ad073da2b645e46eff3e976e61f9a0d551ae4d5ad9998c5c1d87b5e2913b902901020ad77f20b4c29800321df5
+  data.tar.gz: ef83de8173fbad28c2cc07c44f101175ebe9c60d5a9275d2e7280ba1b621a8c51082504ae5d5c917916d398a7a89d06ca7416dd08ba7148ebebb87cd42dd7b05

data/.rubocop.yml

@@ -31,3 +31,5 @@ Style/TrailingCommaInHashLiteral:
 Style/TrailingCommaInArrayLiteral:
   Enabled: false
 
+Metrics/ParameterLists:
+  CountKeywordArgs: false

data/README.md

@@ -233,39 +233,43 @@ This library only ships with very basic, rudimentary jobs that are meant to just
 
 #### Collection
 
-* **collection/arrays_to_objects** [mappings]: Convert an array of arrays to an array of objects.
-* **collection/graph** [config, key]: Use [Hashematics](https://github.com/bluemarblepayroll/hashematics) to turn a flat array of objects into a deeply nested object tree.
-* **collection/objects_to_arrays** [mappings]: Convert an array of objects to an array of arrays.
-* **collection/shift** [amount]: Remove the first N number of elements from an array.
-* **collection/transform** [attributes, exclusive, separator]: Iterate over all objects and transform each key per the attribute transformers specifications.  If exclusive is set to false then the current object will be overridden/merged.  Separator can also be set for key path support.  This job uses [Realize](https://github.com/bluemarblepayroll/realize), which provides its own extendable value-transformation pipeline.
-* **collection/unpivot** [pivot_set]: Take an array of objects and unpivot specific sets of keys into rows.  Under the hood it uses [HashMath's Unpivot class](https://github.com/bluemarblepayroll/hash_math#unpivot-hash-key-coalescence-and-row-extrapolation).
-* **collection/values** [include_keys]: Take an array of objects and call `#values` on each object. If include_keys is true (it is false by default), then call `#keys` on the first object and inject that as a "header" object.
+* **collection/arrays_to_objects** [mappings, register]: Convert an array of arrays to an array of objects.
+* **collection/graph** [config, key, register]: Use [Hashematics](https://github.com/bluemarblepayroll/hashematics) to turn a flat array of objects into a deeply nested object tree.
+* **collection/objects_to_arrays** [mappings, register]: Convert an array of objects to an array of arrays.
+* **collection/shift** [amount, register]: Remove the first N number of elements from an array.
+* **collection/transform** [attributes, exclusive, separator, register]: Iterate over all objects and transform each key per the attribute transformers specifications.  If exclusive is set to false then the current object will be overridden/merged.  Separator can also be set for key path support.  This job uses [Realize](https://github.com/bluemarblepayroll/realize), which provides its own extendable value-transformation pipeline.
+* **collection/unpivot** [pivot_set, register]: Take an array of objects and unpivot specific sets of keys into rows.  Under the hood it uses [HashMath's Unpivot class](https://github.com/bluemarblepayroll/hash_math#unpivot-hash-key-coalescence-and-row-extrapolation).
+* **collection/validate** [invalid_register, join_char, message_key, register, separator, validations]: Take an array of objects, run it through each declared validator, and split the objects into two registers.  The valid objects will be split into the current register while the invalid ones will go into the invalid_register as declared.  Optional arguments, join_char and message_key, help determine the compiled error messages.  The separator option can be utilized to use dot-notation for validating keys.  See each validation's options by viewing their classes within the `lib/modeling/validations` directory.
+* **collection/values** [include_keys, register]: Take an array of objects and call `#values` on each object. If include_keys is true (it is false by default), then call `#keys` on the first object and inject that as a "header" object.
 
 #### De-serialization
 
-* **deserialize/csv** []: Take a CSV string and de-serialize into object(s).  Currently it will return an array of arrays, with each nested array representing one row.
-* **deserialize/json** []: Treat input as a string and de-serialize it to JSON.
-* **deserialize/yaml** [safe]: Treat input as a string and de-serialize it to YAML.  By default it will try and [safely de-serialize](https://ruby-doc.org/stdlib-2.6.1/libdoc/psych/rdoc/Psych.html#method-c-safe_load) it (only using core classes).  If you wish to de-serialize it to any class type, pass in `safe: false`
+* **deserialize/csv** [register]: Take a CSV string and de-serialize into object(s).  Currently it will return an array of arrays, with each nested array representing one row.
+* **deserialize/json** [register]: Treat input as a string and de-serialize it to JSON.
+* **deserialize/yaml** [register, safe]: Treat input as a string and de-serialize it to YAML.  By default it will try and [safely de-serialize](https://ruby-doc.org/stdlib-2.6.1/libdoc/psych/rdoc/Psych.html#method-c-safe_load) it (only using core classes).  If you wish to de-serialize it to any class type, pass in `safe: false`
 
 #### IO
 
 * **io/exist** [path, short_circuit]: Check to see if a file exists. The path parameter can be interpolated using `Payload#params`.  If short_circuit was set to true (defaults to false) and the file does not exist then the pipeline will be short-circuited.
-* **io/read** [binary, path]: Read in a local file.  The path parameter can be interpolated using `Payload#params`.  If the contents are binary, pass in `binary: true` to open it up in binary+read mode.
-* **io/write** [binary, path]: Write to a local file.  The path parameter can be interpolated using `Payload#params`.  If the contents are binary, pass in `binary: true` to open it up in binary+write mode.
+* **io/read** [binary, path, register]: Read in a local file.  The path parameter can be interpolated using `Payload#params`.  If the contents are binary, pass in `binary: true` to open it up in binary+read mode.
+* **io/write** [binary, path, register]: Write to a local file.  The path parameter can be interpolated using `Payload#params`.  If the contents are binary, pass in `binary: true` to open it up in binary+write mode.
 
 #### Serialization
 
-* **serialize/csv** []: Take an array of arrays and create a CSV.
-* **serialize/json** []: Convert value to JSON.
-* **serialize/yaml** []: Convert value to YAML.
+* **serialize/csv** [register]: Take an array of arrays and create a CSV.
+* **serialize/json** [register]: Convert value to JSON.
+* **serialize/yaml** [register]: Convert value to YAML.
 
 #### General
 
 * **dummy** []: Do nothing
 * **echo** [message]: Write a message to the output.  The message parameter can be interpolated using  `Payload#params`.
-* **set** [value]: Set the value to any arbitrary value.
+* **set** [register, value]: Set the value to any arbitrary value.
 * **sleep** [seconds]: Sleep the thread for X number of seconds.
 
+Notes:
+
+* If you see that a job accepts a 'register' attribute/argument, that indicates a job will access and/or mutate the payload.  The register indicates which part of the payload the job will interact with.  This allows jobs to be placed into 'lanes'.  If register is not specified, then the default register is used.
 
 ### Adding & Registering Jobs
 
@@ -274,9 +278,9 @@ Where this library shines is when additional jobs are plugged in.  Burner uses i
 Let's say we would like to register a job to parse a CSV:
 
 ````ruby
-class ParseCsv < Burner::Job
+class ParseCsv < Burner::JobWithRegister
   def perform(output, payload)
-    payload.value = CSV.parse(payload.value, headers: true).map(&:to_h)
+    payload[register] = CSV.parse(payload[register], headers: true).map(&:to_h)
 
     nil
   end

data/lib/burner/job.rb

@@ -7,11 +7,9 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-require_relative 'string_template'
-
 module Burner
   # Abstract base class for all job subclasses.  The only public method a subclass needs to
-  # implement #perform(params, payload, reporter) and then you can register it for use using
+  # implement #perform(output, payload) and then you can register it for use using
   # the Burner::Jobs factory class method #register.  An example of a registration:
   #   Burner::Jobs.register('your_class', YourClass)
   class Job
@@ -26,7 +24,7 @@ module Burner
       @name = name.to_s
     end
 
-    # There are only two requirements to be considered a valid Burner Job:
+    # There are only a few requirements to be considered a valid Burner Job:
     #   1. The class responds to #name
     #   2. The class responds to #perform(output, payload)
     #
@@ -49,9 +47,9 @@ module Burner
     protected
 
     def job_string_template(expression, output, payload)
-      templatable_params = payload.params.merge(__id: output.id, __value: payload.value)
+      templatable_params = payload.params.merge(__id: output.id, __value: payload[''])
 
-      StringTemplate.instance.evaluate(expression, templatable_params)
+      Util::StringTemplate.instance.evaluate(expression, templatable_params)
     end
   end
 end

data/lib/burner/job_with_register.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'job'
+
+module Burner
+  # Add on a register attribute to the configuration for a job.  This indicates that a job
+  # either accesses and/or mutates the payload's registers.
+  class JobWithRegister < Job
+    attr_reader :register
+
+    def initialize(name:, register: '')
+      super(name: name)
+
+      @register = register.to_s
+    end
+  end
+end

data/lib/burner/library.rb

@@ -7,13 +7,15 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-require_relative 'job'
+require_relative 'job_with_register'
+
 require_relative 'library/collection/arrays_to_objects'
 require_relative 'library/collection/graph'
 require_relative 'library/collection/objects_to_arrays'
 require_relative 'library/collection/shift'
 require_relative 'library/collection/transform'
 require_relative 'library/collection/unpivot'
+require_relative 'library/collection/validate'
 require_relative 'library/collection/values'
 require_relative 'library/deserialize/csv'
 require_relative 'library/deserialize/json'

data/lib/burner/library/collection/arrays_to_objects.rb

@@ -47,11 +47,11 @@ module Burner
       #   }
       #
       #   Burner::Pipeline.make(config).execute
-      class ArraysToObjects < Job
+      class ArraysToObjects < JobWithRegister
         attr_reader :mappings
 
-        def initialize(name:, mappings: [])
-          super(name: name)
+        def initialize(name:, mappings: [], register: '')
+          super(name: name, register: register)
 
           @mappings = Modeling::KeyIndexMapping.array(mappings)
 
@@ -59,9 +59,7 @@ module Burner
         end
 
         def perform(_output, payload)
-          payload.value = array(payload.value).map { |array| index_to_key_map(array) }
-
-          nil
+          payload[register] = array(payload[register]).map { |array| index_to_key_map(array) }
         end
 
         private

data/lib/burner/library/collection/graph.rb

@@ -15,11 +15,11 @@ module Burner
       #
       # Expected Payload#value input: array of objects.
       # Payload#value output: An array of objects.
-      class Graph < Job
+      class Graph < JobWithRegister
         attr_reader :key, :groups
 
-        def initialize(name:, key:, config: Hashematics::Configuration.new)
-          super(name: name)
+        def initialize(name:, key:, config: Hashematics::Configuration.new, register: '')
+          super(name: name, register: register)
 
           raise ArgumentError, 'key is required' if key.to_s.empty?
 
@@ -30,13 +30,11 @@ module Burner
         end
 
         def perform(output, payload)
-          graph = Hashematics::Graph.new(groups).add(array(payload.value))
+          graph = Hashematics::Graph.new(groups).add(array(payload[register]))
 
           output.detail("Graphing: #{key}")
 
-          payload.value = graph.data(key)
-
-          nil
+          payload[register] = graph.data(key)
         end
       end
     end

data/lib/burner/library/collection/objects_to_arrays.rb

@@ -48,7 +48,7 @@ module Burner
       #   }
       #
       #   Burner::Pipeline.make(config).execute
-      class ObjectsToArrays < Job
+      class ObjectsToArrays < JobWithRegister
         attr_reader :mappings
 
         # If you wish to support nested objects you can pass in a string to use as a
@@ -56,8 +56,8 @@ module Burner
         # nested hashes then set separator to '.'.  For more information, see the underlying
         # library that supports this dot-notation concept:
         #   https://github.com/bluemarblepayroll/objectable
-        def initialize(name:, mappings: [], separator: '')
-          super(name: name)
+        def initialize(name:, mappings: [], register: '', separator: '')
+          super(name: name, register: register)
 
           @mappings = Modeling::KeyIndexMapping.array(mappings)
           @resolver = Objectable.resolver(separator: separator.to_s)
@@ -66,9 +66,7 @@ module Burner
         end
 
         def perform(_output, payload)
-          payload.value = array(payload.value).map { |object| key_to_index_map(object) }
-
-          nil
+          payload[register] = array(payload[register]).map { |object| key_to_index_map(object) }
         end
 
         private

data/lib/burner/library/collection/shift.rb

@@ -16,15 +16,15 @@ module Burner
       #
       # Expected Payload#value input: nothing.
       # Payload#value output: An array with N beginning elements removed.
-      class Shift < Job
+      class Shift < JobWithRegister
         DEFAULT_AMOUNT = 0
 
         private_constant :DEFAULT_AMOUNT
 
         attr_reader :amount
 
-        def initialize(name:, amount: DEFAULT_AMOUNT)
-          super(name: name)
+        def initialize(name:, amount: DEFAULT_AMOUNT, register: '')
+          super(name: name, register: register)
 
           @amount = amount.to_i
 
@@ -34,9 +34,7 @@ module Burner
         def perform(output, payload)
           output.detail("Shifting #{amount} entries.")
 
-          payload.value = array(payload.value).slice(amount..-1)
-
-          nil
+          payload[register] = array(payload[register]).slice(amount..-1)
         end
       end
     end

data/lib/burner/library/collection/transform.rb

@@ -20,15 +20,15 @@ module Burner
       #
       # Expected Payload#value input: array of objects.
       # Payload#value output: An array of objects.
-      class Transform < Job
+      class Transform < JobWithRegister
         BLANK = ''
 
         attr_reader :attribute_renderers,
                     :exclusive,
                     :resolver
 
-        def initialize(name:, attributes: [], exclusive: false, separator: BLANK)
-          super(name: name)
+        def initialize(name:, attributes: [], exclusive: false, register: '', separator: BLANK)
+          super(name: name, register: register)
 
           @resolver  = Objectable.resolver(separator: separator)
           @exclusive = exclusive || false
@@ -41,14 +41,12 @@ module Burner
         end
 
         def perform(output, payload)
-          payload.value = array(payload.value).map { |row| transform(row, payload.time) }
+          payload[register] = array(payload[register]).map { |row| transform(row, payload.time) }
 
           attr_count = attribute_renderers.length
-          row_count  = payload.value.length
+          row_count  = payload[register].length
 
           output.detail("Transformed #{attr_count} attributes(s) for #{row_count} row(s)")
-
-          nil
         end
 
         private

data/lib/burner/library/collection/unpivot.rb

@@ -16,11 +16,11 @@ module Burner
       #
       # Expected Payload#value input: array of objects.
       # Payload#value output: An array of objects.
-      class Unpivot < Job
+      class Unpivot < JobWithRegister
         attr_reader :unpivot
 
-        def initialize(name:, pivot_set: HashMath::Unpivot::PivotSet.new)
-          super(name: name)
+        def initialize(name:, pivot_set: HashMath::Unpivot::PivotSet.new, register: '')
+          super(name: name, register: register)
 
           @unpivot = HashMath::Unpivot.new(pivot_set)
 
@@ -28,18 +28,24 @@ module Burner
         end
 
         def perform(output, payload)
-          pivot_count   = unpivot.pivot_set.pivots.length
-          key_count     = unpivot.pivot_set.pivots.map { |p| p.keys.length }.sum
-          payload.value = array(payload.value)
-          object_count  = payload.value.length || 0
+          payload[register] = array(payload[register])
+          object_count      = payload[register].length || 0
 
           message = "#{pivot_count} Pivots, Key(s): #{key_count} key(s), #{object_count} objects(s)"
 
           output.detail(message)
 
-          payload.value = payload.value.flat_map { |object| unpivot.expand(object) }
+          payload[register] = payload[register].flat_map { |object| unpivot.expand(object) }
+        end
+
+        private
+
+        def pivot_count
+          unpivot.pivot_set.pivots.length
+        end
 
-          nil
+        def key_count
+          unpivot.pivot_set.pivots.map { |p| p.keys.length }.sum
         end
       end
     end

data/lib/burner/library/collection/validate.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Burner
+  module Library
+    module Collection
+      # Process each object in an array and see if its attribute values match a given set
+      # of validations.  The main register will include the valid objects and the invalid_register
+      # will contain the invalid objects.
+      #
+      # Expected Payload#value input: array of objects.
+      # Payload#value output: An array of objects.
+      class Validate < JobWithRegister
+        DEFAULT_INVALID_REGISTER = 'invalid'
+        DEFAULT_JOIN_CHAR        = ', '
+        DEFAULT_MESSAGE_KEY      = 'errors'
+
+        attr_reader :invalid_register,
+                    :join_char,
+                    :message_key,
+                    :resolver,
+                    :validations
+
+        def initialize(
+          name:,
+          invalid_register: DEFAULT_INVALID_REGISTER,
+          join_char: DEFAULT_JOIN_CHAR,
+          message_key: DEFAULT_MESSAGE_KEY,
+          register: '',
+          separator: '',
+          validations: []
+        )
+          super(name: name, register: register)
+
+          @invalid_register = invalid_register.to_s
+          @join_char        = join_char.to_s
+          @message_key      = message_key.to_s
+          @resolver         = Objectable.resolver(separator: separator)
+          @validations      = Modeling::Validations.array(validations)
+
+          freeze
+        end
+
+        def perform(output, payload)
+          valid   = []
+          invalid = []
+
+          (payload[register] || []).each do |object|
+            errors = validate(object)
+
+            if errors.empty?
+              valid << object
+            else
+              invalid << make_in_error(object, errors)
+            end
+          end
+
+          output.detail("Valid count: #{valid.length}")
+          output.detail("Invalid count: #{invalid.length}")
+
+          payload[register]         = valid
+          payload[invalid_register] = invalid
+
+          nil
+        end
+
+        private
+
+        def validate(object)
+          validations.each_with_object([]) do |validation, memo|
+            next if validation.valid?(object, resolver)
+
+            memo << validation.message
+          end
+        end
+
+        def make_in_error(object, errors)
+          resolver.set(object, message_key, errors.join(join_char))
+        end
+      end
+    end
+  end
+end

data/lib/burner/library/collection/values.rb

@@ -16,11 +16,11 @@ module Burner
       #
       # Expected Payload#value input: array of objects.
       # Payload#value output: An array of arrays.
-      class Values < Job
+      class Values < JobWithRegister
         attr_reader :include_keys
 
-        def initialize(name:, include_keys: false)
-          super(name: name)
+        def initialize(name:, include_keys: false, register: '')
+          super(name: name, register: register)
 
           @include_keys = include_keys || false
 
@@ -28,12 +28,10 @@ module Burner
         end
 
         def perform(_output, payload)
-          payload.value = array(payload.value)
-          keys          = include_keys ? [keys(payload.value.first)] : []
-          values        = payload.value.map { |object| values(object) }
-          payload.value = keys + values
-
-          nil
+          payload[register] = array(payload[register])
+          keys = include_keys ? [keys(payload[register].first)] : []
+          values = payload[register].map { |object| values(object) }
+          payload[register] = keys + values
         end
 
         private

data/lib/burner/library/deserialize/csv.rb

@@ -14,14 +14,12 @@ module Burner
       #
       # Expected Payload#value input: nothing.
       # Payload#value output: an array of arrays.  Each inner array represents one data row.
-      class Csv < Job
+      class Csv < JobWithRegister
         # This currently only supports returning an array of arrays, including the header row.
         # In the future this could be extended to offer more customizable options, such as
         # making it return an array of hashes with the columns mapped, etc.)
         def perform(_output, payload)
-          payload.value = CSV.new(payload.value, headers: false).to_a
-
-          nil
+          payload[register] = CSV.new(payload[register], headers: false).to_a
         end
       end
     end

data/lib/burner/library/deserialize/json.rb

@@ -14,11 +14,9 @@ module Burner
       #
       # Expected Payload#value input: string of JSON data.
       # Payload#value output: anything, as specified by the JSON de-serializer.
-      class Json < Job
+      class Json < JobWithRegister
         def perform(_output, payload)
-          payload.value = JSON.parse(payload.value)
-
-          nil
+          payload[register] = JSON.parse(payload[register])
         end
       end
     end

data/lib/burner/library/deserialize/yaml.rb

@@ -17,11 +17,11 @@ module Burner
       #
       # Expected Payload#value input: string of YAML data.
       # Payload#value output: anything as specified by the YAML de-serializer.
-      class Yaml < Job
+      class Yaml < JobWithRegister
         attr_reader :safe
 
-        def initialize(name:, safe: true)
-          super(name: name)
+        def initialize(name:, register: '', safe: true)
+          super(name: name, register: register)
 
           @safe = safe
 
@@ -36,9 +36,9 @@ module Burner
         def perform(output, payload)
           output.detail('Warning: loading YAML not using safe_load.') unless safe
 
-          payload.value = safe ? YAML.safe_load(payload.value) : YAML.load(payload.value)
+          value = payload[register]
 
-          nil
+          payload[register] = safe ? YAML.safe_load(value) : YAML.load(value)
         end
         # rubocop:enable Security/YAMLLoad
       end

data/lib/burner/library/dummy.rb

@@ -13,9 +13,7 @@ module Burner
     #
     # Note: this does not use Payload#value.
     class Dummy < Job
-      def perform(_output, _payload)
-        nil
-      end
+      def perform(_output, _payload); end
     end
   end
 end

data/lib/burner/library/echo.rb

@@ -27,8 +27,6 @@ module Burner
         compiled_message = job_string_template(message, output, payload)
 
         output.detail(compiled_message)
-
-        nil
       end
     end
   end

data/lib/burner/library/io/base.rb

@@ -11,11 +11,11 @@ module Burner
   module Library
     module IO
       # Common configuration/code for all IO Job subclasses.
-      class Base < Job
+      class Base < JobWithRegister
         attr_reader :path
 
-        def initialize(name:, path:)
-          super(name: name)
+        def initialize(name:, path:, register: '')
+          super(name: name, register: register)
 
           raise ArgumentError, 'path is required' if path.to_s.empty?
 

data/lib/burner/library/io/exist.rb

@@ -16,15 +16,16 @@ module Burner
       # does not exist then the job will return false and short circuit the pipeline.
       #
       # Note: this does not use Payload#value.
-      class Exist < Base
-        attr_reader :short_circuit
+      class Exist < Job
+        attr_reader :path, :short_circuit
 
         def initialize(name:, path:, short_circuit: false)
-          super(name: name, path: path)
+          super(name: name)
 
-          @short_circuit = short_circuit || false
+          raise ArgumentError, 'path is required' if path.to_s.empty?
 
-          freeze
+          @path          = path.to_s
+          @short_circuit = short_circuit || false
         end
 
         def perform(output, payload)
@@ -35,9 +36,8 @@ module Burner
 
           output.detail("The path: #{compiled_path} #{verb} exist")
 
-          # if anything but false is returned then the pipeline will not short circuit.  So
-          # we need to make sure we explicitly return false.
-          short_circuit && !exists ? false : nil
+          # if anything but false is returned then the pipeline will not short circuit.
+          payload.halt_pipeline if short_circuit && !exists
         end
       end
     end

data/lib/burner/library/io/read.rb

@@ -19,8 +19,8 @@ module Burner
       class Read < Base
         attr_reader :binary
 
-        def initialize(name:, path:, binary: false)
-          super(name: name, path: path)
+        def initialize(name:, path:, binary: false, register: '')
+          super(name: name, path: path, register: register)
 
           @binary = binary || false
 
@@ -32,9 +32,7 @@ module Burner
 
           output.detail("Reading: #{compiled_path}")
 
-          payload.value = File.open(compiled_path, mode, &:read)
-
-          nil
+          payload[register] = File.open(compiled_path, mode, &:read)
         end
 
         private

data/lib/burner/library/io/write.rb

@@ -19,8 +19,8 @@ module Burner
       class Write < Base
         attr_reader :binary
 
-        def initialize(name:, path:, binary: false)
-          super(name: name, path: path)
+        def initialize(name:, path:, binary: false, register: '')
+          super(name: name, path: path, register: register)
 
           @binary = binary || false
 
@@ -35,7 +35,7 @@ module Burner
           output.detail("Writing: #{compiled_path}")
 
           time_in_seconds = Benchmark.measure do
-            File.open(compiled_path, mode) { |io| io.write(payload.value) }
+            File.open(compiled_path, mode) { |io| io.write(payload[register]) }
           end.real
 
           side_effect = SideEffects::WrittenFile.new(
@@ -45,8 +45,6 @@ module Burner
           )
 
           payload.add_side_effect(side_effect)
-
-          nil
         end
 
         private

data/lib/burner/library/serialize/csv.rb

@@ -14,15 +14,13 @@ module Burner
       #
       # Expected Payload#value input: array of arrays.
       # Payload#value output: a serialized CSV string.
-      class Csv < Job
+      class Csv < JobWithRegister
         def perform(_output, payload)
-          payload.value = CSV.generate(options) do |csv|
-            array(payload.value).each do |row|
+          payload[register] = CSV.generate(options) do |csv|
+            array(payload[register]).each do |row|
               csv << row
             end
           end
-
-          nil
         end
 
         private

data/lib/burner/library/serialize/json.rb

@@ -14,11 +14,9 @@ module Burner
       #
       # Expected Payload#value input: anything.
       # Payload#value output: string representing the output of the JSON serializer.
-      class Json < Job
+      class Json < JobWithRegister
         def perform(_output, payload)
-          payload.value = payload.value.to_json
-
-          nil
+          payload[register] = payload[register].to_json
         end
       end
     end

data/lib/burner/library/serialize/yaml.rb

@@ -14,11 +14,9 @@ module Burner
       #
       # Expected Payload#value input: anything.
       # Payload#value output: string representing the output of the YAML serializer.
-      class Yaml < Job
+      class Yaml < JobWithRegister
         def perform(_output, payload)
-          payload.value = payload.value.to_yaml
-
-          nil
+          payload[register] = payload[register].to_yaml
         end
       end
     end

data/lib/burner/library/set_value.rb

@@ -13,11 +13,11 @@ module Burner
     #
     # Expected Payload#value input: anything.
     # Payload#value output: whatever value was specified in this job.
-    class SetValue < Job
+    class SetValue < JobWithRegister
       attr_reader :value
 
-      def initialize(name:, value: nil)
-        super(name: name)
+      def initialize(name:, register: '', value: nil)
+        super(name: name, register: register)
 
         @value = value
 
@@ -25,9 +25,7 @@ module Burner
       end
 
       def perform(_output, payload)
-        payload.value = value
-
-        nil
+        payload[register] = value
       end
     end
   end

data/lib/burner/library/sleep.rb

@@ -27,8 +27,6 @@ module Burner
         output.detail("Going to sleep for #{seconds} second(s)")
 
         Kernel.sleep(seconds)
-
-        nil
       end
     end
   end

data/lib/burner/modeling.rb

@@ -10,3 +10,4 @@
 require_relative 'modeling/attribute'
 require_relative 'modeling/attribute_renderer'
 require_relative 'modeling/key_index_mapping'
+require_relative 'modeling/validations'

data/lib/burner/modeling/validations.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'validations/blank'
+require_relative 'validations/present'
+
+module Burner
+  module Modeling
+    # Factory for building sub-classes that can validate an individual object and field value.
+    class Validations
+      acts_as_hashable_factory
+
+      register 'blank',   Validations::Blank
+      register 'present', Validations::Present
+    end
+  end
+end

data/lib/burner/modeling/validations/base.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Burner
+  module Modeling
+    class Validations
+      # Common logic shared among all Validation subclasses.
+      # This class is an abstract class, make sure to implement:
+      # - #valid?(object, resolver)
+      # - #default_message
+      class Base
+        acts_as_hashable
+
+        attr_reader :key
+
+        def initialize(key:, message: '')
+          raise ArgumentError, 'key is required' if key.to_s.empty?
+
+          @key     = key.to_s
+          @message = message.to_s
+        end
+
+        def message
+          @message.to_s.empty? ? "#{key}#{default_message}" : @message.to_s
+        end
+      end
+    end
+  end
+end

data/lib/burner/modeling/validations/blank.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'base'
+
+module Burner
+  module Modeling
+    class Validations
+      # Check if a value is blank, if it is not blank then it is not valid.
+      class Blank < Base
+        acts_as_hashable
+
+        def valid?(object, resolver)
+          resolver.get(object, key).to_s.empty?
+        end
+
+        private
+
+        def default_message
+          ' must be blank'
+        end
+      end
+    end
+  end
+end

data/lib/burner/modeling/validations/present.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'base'
+
+module Burner
+  module Modeling
+    class Validations
+      # Check if a value is present.  If it is blank (null or empty) then it is invalid.
+      class Present < Base
+        acts_as_hashable
+
+        def valid?(object_value, resolver)
+          !resolver.get(object_value, key).to_s.empty?
+        end
+
+        private
+
+        def default_message
+          ' is required'
+        end
+      end
+    end
+  end
+end

data/lib/burner/payload.rb

@@ -19,26 +19,60 @@ module Burner
   # report back the files it has written in a more structured data way (as opposed to simply
   # writing some information to the output.)
   class Payload
-    attr_accessor :value
-
     attr_reader :params,
+                :registers,
                 :side_effects,
                 :time
 
     def initialize(
       params: {},
+      registers: {},
       side_effects: [],
-      time: Time.now.utc,
-      value: nil
+      time: Time.now.utc
     )
       @params       = params || {}
+      @registers    = {}
       @side_effects = side_effects || []
       @time         = time || Time.now.utc
-      @value        = value
+
+      add_registers(registers)
     end
 
+    # Add a side effect of a job.  This helps to keep track of things jobs do outside of its
+    # register mutations.
     def add_side_effect(side_effect)
       tap { side_effects << side_effect }
     end
+
+    # Set a register's value.
+    def []=(key, value)
+      set(key, value)
+    end
+
+    # Retrieve a register's value.
+    def [](key)
+      registers[key.to_s]
+    end
+
+    # Set halt_pipeline to true.  This will indicate to the pipeline to stop all
+    # subsequent processing.
+    def halt_pipeline
+      @halt_pipeline = true
+    end
+
+    # Check and see if halt_pipeline was called.
+    def halt_pipeline?
+      @halt_pipeline || false
+    end
+
+    private
+
+    def set(key, value)
+      registers[key.to_s] = value
+    end
+
+    def add_registers(registers)
+      (registers || {}).each { |k, v| set(k, v) }
+    end
   end
 end

data/lib/burner/pipeline.rb

@@ -48,10 +48,10 @@ module Burner
 
       time_in_seconds = Benchmark.measure do
         steps.each do |step|
-          return_value = step.perform(output, payload)
+          step.perform(output, payload)
 
-          if return_value.is_a?(FalseClass)
-            output.detail('Job returned false, ending pipeline.')
+          if payload.halt_pipeline?
+            output.detail('Payload was halted, ending pipeline.')
             break
           end
         end

data/lib/burner/step.rb

@@ -29,17 +29,13 @@ module Burner
     end
 
     def perform(output, payload)
-      return_value = nil
-
       output.title("#{job.class.name}#{SEPARATOR}#{job.name}")
 
       time_in_seconds = Benchmark.measure do
-        return_value = job.perform(output, payload)
+        job.perform(output, payload)
       end.real.round(3)
 
       output.complete(time_in_seconds)
-
-      return_value
     end
   end
 end

data/lib/burner/util.rb

@@ -8,3 +8,4 @@
 #
 
 require_relative 'util/arrayable'
+require_relative 'util/string_template'

data/lib/burner/util/string_template.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Burner
+  module Util
+    # Can take in a string and an object and use the object for formatting string interpolations
+    # using tokens of form: {attribute_name}.  This templating class does not understand nested
+    # structures, so input should be a flat object/hash in the form of key-value pairs.
+    # A benefit of using Objectable for resolution is that it can understand almost any type of
+    # object: Hash, Struct, OpenStruct, custom objects, etc.
+    # For more information see underlying libraries:
+    #   * Stringento: https://github.com/bluemarblepayroll/stringento
+    #   * Objectable: https://github.com/bluemarblepayroll/objectable
+    class StringTemplate
+      include Singleton
+
+      attr_reader :resolver
+
+      def initialize
+        @resolver = Objectable.resolver(separator: '')
+
+        freeze
+      end
+
+      # For general consumption
+      def evaluate(expression, input)
+        Stringento.evaluate(expression, input, resolver: self)
+      end
+
+      # For Stringento consumption
+      def resolve(value, input)
+        resolver.get(input, value)
+      end
+    end
+  end
+end

data/lib/burner/version.rb

@@ -8,5 +8,5 @@
 #
 
 module Burner
-  VERSION = '1.0.0-alpha.6'
+  VERSION = '1.0.0-alpha.7'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: burner
 version: !ruby/object:Gem::Version
-  version: 1.0.0.pre.alpha.6
+  version: 1.0.0.pre.alpha.7
 platform: ruby
 authors:
 - Matthew Ruggio
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-10-22 00:00:00.000000000 Z
+date: 2020-10-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: acts_as_hashable
@@ -220,6 +220,7 @@ files:
 - lib/burner.rb
 - lib/burner/cli.rb
 - lib/burner/job.rb
+- lib/burner/job_with_register.rb
 - lib/burner/jobs.rb
 - lib/burner/library.rb
 - lib/burner/library/collection/arrays_to_objects.rb
@@ -228,6 +229,7 @@ files:
 - lib/burner/library/collection/shift.rb
 - lib/burner/library/collection/transform.rb
 - lib/burner/library/collection/unpivot.rb
+- lib/burner/library/collection/validate.rb
 - lib/burner/library/collection/values.rb
 - lib/burner/library/deserialize/csv.rb
 - lib/burner/library/deserialize/json.rb
@@ -247,15 +249,19 @@ files:
 - lib/burner/modeling/attribute.rb
 - lib/burner/modeling/attribute_renderer.rb
 - lib/burner/modeling/key_index_mapping.rb
+- lib/burner/modeling/validations.rb
+- lib/burner/modeling/validations/base.rb
+- lib/burner/modeling/validations/blank.rb
+- lib/burner/modeling/validations/present.rb
 - lib/burner/output.rb
 - lib/burner/payload.rb
 - lib/burner/pipeline.rb
 - lib/burner/side_effects.rb
 - lib/burner/side_effects/written_file.rb
 - lib/burner/step.rb
-- lib/burner/string_template.rb
 - lib/burner/util.rb
 - lib/burner/util/arrayable.rb
+- lib/burner/util/string_template.rb
 - lib/burner/version.rb
 homepage: https://github.com/bluemarblepayroll/burner
 licenses:

data/lib/burner/string_template.rb

@@ -1,40 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-module Burner
-  # Can take in a string and an object and use the object for formatting string interpolations
-  # using tokens of form: {attribute_name}.  This templating class does not understand nested
-  # structures, so input should be a flat object/hash in the form of key-value pairs.  A benefit of
-  # using Objectable for resolution is that it can understand almost any type of
-  # object: Hash, Struct, OpenStruct, custom objects, etc.
-  # For more information see underlying libraries:
-  #   * Stringento: https://github.com/bluemarblepayroll/stringento
-  #   * Objectable: https://github.com/bluemarblepayroll/objectable
-  class StringTemplate
-    include Singleton
-
-    attr_reader :resolver
-
-    def initialize
-      @resolver = Objectable.resolver(separator: '')
-
-      freeze
-    end
-
-    # For general consumption
-    def evaluate(expression, input)
-      Stringento.evaluate(expression, input, resolver: self)
-    end
-
-    # For Stringento consumption
-    def resolve(value, input)
-      resolver.get(input, value)
-    end
-  end
-end