realize 1.0.0.pre.alpha.1 → 1.0.0.pre.alpha.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b982efd1c16b8416b6a1f230d6de7ca3923ddaf2d6986f4b16fa84628f6075ab
-  data.tar.gz: d3f20dc7adb39811a3ab1c065cabff494dded6ff13c7e5bbc5e6398b657f80cc
+  metadata.gz: 289c0fa13f4421249a59e82ce767881f04bd47bea5d0b3a4eb4db503b37add28
+  data.tar.gz: f8575e76151f98fc0e34d8fcf4c50231294ffad0f037eb6f276fd1393f8e6cd4
 SHA512:
-  metadata.gz: 8a2d81b01a096e15ae87e05ec5e303ad68136c758f48df06ead805f0feb5f40e943f42cce4c929ad53ede412a330f94b71091d025de5997c64fe972784b41104
-  data.tar.gz: b94ecac10ff8727cc69b8b67113703e7a623dc640b914fe8f9ad3a6c25050c1fc5310d18e846333b0d1041a712367263f2309999460d9e4cbe7c08f900571386
+  metadata.gz: cf1f9808bdb3870860155d3eecf6187b4bb9a5b3b4ea755552a125e9a96fb34173d3174d2ba98c5da9aeef00dea520e3d2cf2b2f7821daa4b2d3c6558f8cec21
+  data.tar.gz: ec3bc0d03c004d8481b4d1d9b690076f9dd48988e9e957cd61c82aac40e77d60401666057dc9484b4ea8d6d116ed781305af49b0dcdf0d2bdfe77583393fc931

data/README.md

@@ -2,7 +2,11 @@
 
 [![Gem Version](https://badge.fury.io/rb/realize.svg)](https://badge.fury.io/rb/realize) [![Build Status](https://travis-ci.org/bluemarblepayroll/realize.svg?branch=master)](https://travis-ci.org/bluemarblepayroll/realize) [![Maintainability](https://api.codeclimate.com/v1/badges/115f0c5a1d0a4cce7230/maintainability)](https://codeclimate.com/github/bluemarblepayroll/realize/maintainability) [![Test Coverage](https://api.codeclimate.com/v1/badges/115f0c5a1d0a4cce7230/test_coverage)](https://codeclimate.com/github/bluemarblepayroll/realize/test_coverage) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-TODO
+This library provides a pluggable and configurable data transformation framework.  The general use-case is:
+
+> We need to be able to configure the data transformation pipeline, within an application, for a system-to-system integration.
+
+It is currently used in production at Blue Marble to power the transformation pipeline within a larger ETL framework.
 
 ## Installation
 
@@ -20,7 +24,87 @@ bundle add realize
 
 ## Examples
 
-TODO
+### Basic Transformer Example
+
+Here is a simple record we will use for data derivation and transformation:
+
+````ruby
+record = {
+  id: 1,
+  created_at: '2020-03-04T12:34:56Z',
+  first: 'Frank',
+  last: 'Rizzo'
+}
+````
+
+Let's say we wanted to retrieve the created_at formatted as: 'MM/DD/YY', we could write:
+
+````ruby
+transformers = [
+  {
+    type: 'r/value/resolve',
+    key: :created_at
+  },
+  {
+    type: 'r/format/date',
+    output_format: '%D'
+  }
+]
+
+value = Realize.pipeline(transformers).transform(record) # 03/04/20
+````
+
+Notice how all built-in transformers are prefixed with 'r'.  This should help isolate the built-in transformers from potential externally registered transformers.
+
+### Transformer Gallery
+
+Here is a list of each built-in transformer, their options, and what their function is:
+
+#### Collection-oriented Transformers
+
+* **r/collection/sort** [key, direction]:  Takes an array (or coerces value to an array) and sort it either ascending or descending by some defined key's value.
+
+#### Filtering Transformers
+
+* **r/filter/by_key_record_value** [key, value]: Takes an array (or coerces value to an array) and selects only the records that match the key's value.  In this case the value is derived off of the main record.
+* **r/filter/by_key_value_presence** [key]: Takes an array (or coerces value to an array) and selects only the records where the key's value is present (not nil and not empty).
+* **r/filter/by_key_record_value** [key, value]: Takes an array (or coerces value to an array) and selects only the records that match the key's value.  In this case, the value is statically defined.
+* **r/filter/inactive** [key, value]: Takes an array (or coerces value to an array) and selects only the records where the current time is between the start and end times as derived from the record.  Note that *current time* can be passed in but defaults to current UTC time.
+
+#### Format-oriented Transformers
+
+* **r/format/date** [input_format, output_format]: Parses the incoming value into a Time object using the configured input_format and outputs it as formatted by the configured output_format.
+* **r/format/remove_whitespace** []: Removes all whitespace from the incoming value.
+* **r/format/string_replace** [original, replacement]: Replaces all occurrences of the configured original value with the replacement value.
+
+#### Logical Transformers
+
+* **r/logical/switch** [cases, default_transformers, key]: Provides a value-based logic branching.  If a value matches a specific case, the specific cases transformers will be executed.  If it does not match any case then the default_transformers will be executed.
+
+#### Value-oriented Transformers
+
+* **r/logical/blank** []: Always return a blank string.
+* **r/logical/map** [values]: Do a lookup on the value using the `values` hash.  If a value is found, then its corresponding value is returned.  If it isn't then the input is returned.
+* **r/logical/null** []: Always returns null.
+* **r/logical/resolve** [key]: Dynamically resolves a value based on the record's key.  It uses the [Objectable](https://github.com/bluemarblepayroll/objectable) library by default, which provides some goodies like dot-notation for nested objects and type-insensitivity (works for Ruby objects, hashes, structs, etc.)
+* **r/logical/static** [value]: Always returns a hard-coded value.
+* **r/logical/verbatim** []: Default transformer, simply echos back the input.
+
+### Plugging in Transformers
+
+Custom transformers can be externally created and registered as long as it complies with the general transformer interface:
+
+* constructor accepts keyword arguments from which the configuration provides
+* responds to a method called transform with the signature: transform(resolver, value, time, record)
+
+After you have implemented the custom transformer, you can externally register is by:
+
+````ruby
+Realize::Transformers.register('some_custom_transformer', SomeCustomTransformer)
+````
+
+You should now be able to use the type: 'some_custom_transformer' within your transformation configuration.
+
 
 ## Contributing
 

data/lib/realize.rb

@@ -18,11 +18,8 @@ require_relative 'realize/pipeline'
 # Top-level syntactic sugar.
 module Realize
   class << self
-    def pipeline(resolver: Objectable.resolver, transformers: [])
-      Pipeline.new(
-        resolver: resolver,
-        transformers: transformers
-      )
+    def pipeline(transformers = [], resolver: Objectable.resolver)
+      Pipeline.new(transformers, resolver: resolver)
     end
   end
 end

data/lib/realize/collection/sort.rb

@@ -28,10 +28,10 @@ module Realize
         freeze
       end
 
-      def transform(_resolver, value, _time, _record)
+      def transform(resolver, value, _time, _record)
         records = array(value)
 
-        sorted = records.sort_by { |hsh| hsh[key.to_sym] }
+        sorted = records.sort_by { |hsh| resolver.get(hsh, key) }
 
         order == DESCENDING ? sorted.reverse : sorted
       end

data/lib/realize/format/remove_whitespace.rb

@@ -7,8 +7,6 @@ module Realize
     class RemoveWhitespace
       acts_as_hashable
 
-      def initialize(_opts = {}); end
-
       def transform(_resolver, value, _time, _record)
         value.to_s.gsub(/\s+/, ' ')
       end

data/lib/realize/pipeline.rb

@@ -6,7 +6,7 @@ module Realize
   class Pipeline
     attr_reader :resolver, :transformers
 
-    def initialize(resolver: Objectable.resolver, transformers: [])
+    def initialize(transformers = [], resolver: Objectable.resolver)
       raise ArgumentError, 'resolver is required' unless resolver
 
       @resolver     = resolver
@@ -15,11 +15,8 @@ module Realize
       freeze
     end
 
-    def transform(record, opts = {})
-      time  = opts.fetch(:time, Time.now.utc)
-      value = opts.fetch(:value, record)
-
-      transformers.inject(value) do |memo, transformer|
+    def transform(record, time = Time.now.utc)
+      transformers.inject(record) do |memo, transformer|
         transformer.transform(resolver, memo, time, record)
       end
     end

data/lib/realize/value/blank.rb

@@ -6,8 +6,6 @@ module Realize
     class Blank
       acts_as_hashable
 
-      def initialize(_opts = {}); end
-
       def transform(_resolver, _value, _time, _record)
         ''
       end

data/lib/realize/value/null.rb

@@ -6,8 +6,6 @@ module Realize
     class Null
       acts_as_hashable
 
-      def initialize(_opts = {}); end
-
       def transform(_resolver, _value, _time, _record)
         nil
       end

data/lib/realize/value/verbatim.rb

@@ -6,11 +6,6 @@ module Realize
     class Verbatim
       acts_as_hashable
 
-      # This is here to satisfy an underlying issue in acts_as_hashable.
-      # The #make calls in the factory and hashable module should be calling #new with no
-      # args if no keys are detected.
-      def initialize(_opts = {}); end
-
       def transform(_resolver, value, _time, _record)
         value
       end

data/lib/realize/version.rb

@@ -8,5 +8,5 @@
 #
 
 module Realize
-  VERSION = '1.0.0-alpha.1'
+  VERSION = '1.0.0-alpha.2'
 end

data/realize.gemspec

@@ -28,7 +28,7 @@ Gem::Specification.new do |s|
 
   s.required_ruby_version = '>= 2.5'
 
-  s.add_dependency('acts_as_hashable', '~>1')
+  s.add_dependency('acts_as_hashable', '=1.2.0.pre.alpha')
   s.add_dependency('objectable', '~>1')
 
   s.add_development_dependency('guard-rspec', '~>4.7')

metadata

@@ -1,29 +1,29 @@
 --- !ruby/object:Gem::Specification
 name: realize
 version: !ruby/object:Gem::Version
-  version: 1.0.0.pre.alpha.1
+  version: 1.0.0.pre.alpha.2
 platform: ruby
 authors:
 - Matthew Ruggio
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-06-08 00:00:00.000000000 Z
+date: 2020-06-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: acts_as_hashable
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
-        version: '1'
+        version: 1.2.0.pre.alpha
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
-        version: '1'
+        version: 1.2.0.pre.alpha
 - !ruby/object:Gem::Dependency
   name: objectable
   requirement: !ruby/object:Gem::Requirement