db_fuel 1.1.0.pre.alpha → 1.2.1.pre.alpha.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fb563756df642d23e7d268cf0e7b35735300ddfc84da31e0ee11d7e138d41584
-  data.tar.gz: f5720ed0f4649ba82d0c233cf9ccbc4aff5d54d8d9c4267c830467f43f85cdf3
+  metadata.gz: 0a93b59424a5cf6866f1061785f41a0c49c1bcc707ce84a6652f481a283aab64
+  data.tar.gz: 3b7a65c38a5cc24ee844412ef6cb1bc3b2bc2a33c9afd564196e1b1f739108b7
 SHA512:
-  metadata.gz: 16c63b0939d36f32cfc72baeefc20fbb88e7285d5c5e8266d337f55362909a6f751f7f2b1c535f6c231a11adba2693807c68e6af1b4d5eaaf4b14a9213589d01
-  data.tar.gz: a5143b01108dc065d98a5e085da06c8e82a5f0c084a0097128219e8c49542818a8ce754ef1af79be23d6493f259f745657bba4e74051cf856264fbe87a76dfde
+  metadata.gz: f5bc7c2644584e1f8e0412e0215bef34b6431b085e18fba822723e94ab0baea9c7a97a25d4970dc9169a2adc23f7818d618f48f8ab2f9d7ddb9f5a5914ecd356
+  data.tar.gz: 02d4f517420b8aa3331d99a69dadd39faa7b53382e89f153467eaea3d74a52141e75c000cba62c090e8e0aaa2fb2d293908160b016b242e35d483f396e37b542

data/.rubocop.yml

@@ -1,6 +1,7 @@
 AllCops:
   TargetRubyVersion: 2.5
   NewCops: enable
+  SuggestExtensions: false
 
 Layout/LineLength:
   Max: 100
@@ -8,7 +9,7 @@ Layout/LineLength:
     - db_fuel.gemspec
 
 Metrics/BlockLength:
-  ExcludedMethods:
+  IgnoredMethods:
     - let
     - it
     - describe

data/.tool-versions

@@ -0,0 +1 @@
+ruby 2.6.6

data/CHANGELOG.md

@@ -1,7 +1,8 @@
-# 1.1.0 (TBD)
+# 1.1.0 (Decmeber 1st, 2020)
 
 New Jobs:
 
+* db_fuel/active_record/find_or_insert
 * db_fuel/active_record/insert
 * db_fuel/active_record/update
 

data/README.md

@@ -24,13 +24,16 @@ Refer to the [Burner](https://github.com/bluemarblepayroll/burner) library for m
 
 ### ActiveRecord Jobs
 
-* **db_fuel/active_record/insert** [table_name, attributes, debug, primary_key, register, separator, timestamps]: This job can take the objects in a register and insert them into a database table.  Attributes defines which object properties to convert to SQL.  Refer to the class and constructor specification for more detail.
-* **db_fuel/active_record/update** [table_name, attributes, debug, register, separator, timestamps, unique_keys]: This job can take the objects in a register and updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while unique_keys translate to WHERE clauses.  Refer to the class and constructor specification for more detail.
+* **db_fuel/active_record/find_or_insert** [name, table_name, attributes, debug, primary_key, register, separator, timestamps, unique_attributes]: An extension of the `db_fuel/active_record/insert` job that adds an existence check before SQL insertion. The  `unique_attributes` will be converted to WHERE clauses for performing the existence check.
+* **db_fuel/active_record/insert** [name, table_name, attributes, debug, primary_key, register, separator, timestamps]: This job can take the objects in a register and insert them into a database table.  If primary_key is specified then its key will be set to the primary key.  Note that composite primary keys are not supported.  Attributes defines which object properties to convert to SQL.  Refer to the class and constructor specification for more detail.
+* **db_fuel/active_record/update_all** [name, table_name, attributes, debug, register, separator, timestamps, unique_attributes]: This job can take the objects in a register and updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while unique_attributes translate to WHERE clauses. One or more records may be updated at a time.  Refer to the class and constructor specification for more detail.
+* **db_fuel/active_record/update** [name, table_name, attributes, debug, register, primary_key, separator, timestamps, unique_attributes]: This job can take the unique objects in a register and updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while unique_attributes translate to WHERE clauses to find the records to update. The primary_key is used to update the unique record. Only one record will be updated per statement.  Refer to the class and constructor specification for more detail.
+* **db_fuel/active_record/upsert** [name, table_name, attributes, debug, primary_key, register, separator, timestamps, unique_attributes]: This job can take the objects in a register and either inserts or updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while each key in unique_attributes become a WHERE clause in order to check for the existence of a specific record. The updated record will use the primary_key specified to perform the UPDATE operation. Note that composite primary keys are not supported. Refer to the class and constructor specification for more detail.
 
 ### Dbee Jobs
 
-* **db_fuel/dbee/query** [model, query, register]:  Pass in a [Dbee](https://github.com/bluemarblepayroll/dbee) model and query and store the results in the specified register.  Refer to the [Dbee](https://github.com/bluemarblepayroll/dbee) library directly on how to craft a model or query.
-* **db_fuel/dbee/range** [key, key_path, model, query, register, separator]: Similar to `db_fuel/dbee/query` with the addition of being able to grab a list of values from the register to use as a Dbee EQUALS/IN filter.  This helps to dynamically limit the resulting record set.  The key is used to specify where to grab the list of values, while the key_path will be used to craft the [Dbee equal's filter](https://github.com/bluemarblepayroll/dbee/blob/master/lib/dbee/query/filters/equals.rb).  Separator is exposed in case nested object support is necessary.
+* **db_fuel/dbee/query** [model, query, register, debug]:  Pass in a [Dbee](https://github.com/bluemarblepayroll/dbee) model and query and store the results in the specified register.  Refer to the [Dbee](https://github.com/bluemarblepayroll/dbee) library directly on how to craft a model or query.
+* **db_fuel/dbee/range** [key, key_path, model, query, register, separator, debug]: Similar to `db_fuel/dbee/query` with the addition of being able to grab a list of values from the register to use as a Dbee EQUALS/IN filter.  This helps to dynamically limit the resulting record set.  The key is used to specify where to grab the list of values, while the key_path will be used to craft the [Dbee equal's filter](https://github.com/bluemarblepayroll/dbee/blob/master/lib/dbee/query/filters/equals.rb).  Separator is exposed in case nested object support is necessary.
 
 ## Examples
 
@@ -95,6 +98,10 @@ If we were to inspect the contents of `payload` we should see the patient's resu
 payload['patients'] # array in form of: [ { "id" => 1, "first_name" => "Something" }, ... ]
 ````
 
+Notes
+
+* Set `debug: true` to print out SQL statement in the output (not for production use.)
+
 ### Limiting Result Sets
 
 The `db_fuel/dbee/query` does not provide a way to dynamically connect the query to existing data.  You are free to put any Dbee query filters in the query declaration, but what if you would like to further limit this based on the knowledge of a range of values?  The `db_fuel/dbee/range` job is meant to do exactly this.  On the surface it is mainly an extension of the `db_fuel/dbee/query` job.
@@ -148,6 +155,10 @@ payload['patients'] # array in form of: [ { "id" => 1, "first_name" => "Somethin
 
 The only difference between the query and range jobs should be the latter is limited based on the incoming first names.
 
+Notes
+
+* Set `debug: true` to print out SQL statement in the output (not for production use.)
+
 ### Updating the Database
 
 #### Inserting Records
@@ -193,11 +204,55 @@ There should now be two new patients, AB0 and AB1, present in the table `patient
 Notes:
 
 * Since we specified the `primary_key`, the records' `id` attributes should be set to their respective primary key values.
+* Composite primary keys are not currently supported.
 * Set `debug: true` to print out each INSERT statement in the output (not for production use.)
 
+#### Inserting Only New Records
+
+Another job `db_fuel/active_record/find_or_insert` allows for an existence check to performed each insertion.  If a record is found then it will not insert the record.  If `primary_key` is set then the existence check will also still set the primary key on the payload's respective object.  Note that composite primary keys are not currently supported. We can build on the above insert example for only inserting new patients if their chart_number is unique:
+
+````ruby
+pipeline = {
+  jobs: [
+    {
+      name: :load_patients,
+      type: 'b/value/static',
+      register: :patients,
+      value: [
+        { chart_number: 'B0001', first_name: 'Bugs', last_name: 'Bunny' },
+        { chart_number: 'B0002', first_name: 'Babs', last_name: 'Bunny' }
+      ]
+    },
+    {
+      name: 'insert_patients',
+      type: 'db_fuel/active_record/insert',
+      register: :patients,
+      attributes: [
+        { key: :chart_number },
+        { key: :first_name },
+        { key: :last_name }
+      ],
+      table_name: 'patients',
+      primary_key: {
+        key: :id
+      },
+      unique_attributes: [
+        { key: :chart_number }
+      ]
+    }
+  ]
+}
+
+payload = Burner::Payload.new
+
+Burner::Pipeline.make(pipeline).execute(payload: payload)
+````
+
+Now only records where the chart_number does not match an existing record will be inserted.
+
 #### Updating Records
 
-Let's say we now want to update those records' last names:
+Let's say we now want to update these unique records' last names:
 
 ````ruby
 pipeline = {
@@ -219,7 +274,92 @@ pipeline = {
         { key: :last_name }
       ],
       table_name: 'patients',
-      unique_keys: [
+      primary_key: {
+        key: :id
+      },
+      unique_attributes: [
+        { key: :chart_number }
+      ]
+    }
+  ]
+}
+
+payload = Burner::Payload.new
+
+Burner::Pipeline.make(pipeline).execute(payload: payload)
+````
+
+Each database record should have been updated with their new respective last names based on the primary key specified.
+
+#### Updating All Records
+
+Let's say we want to update those records' midddle names:
+
+````ruby
+pipeline = {
+  jobs: [
+    {
+      name: :load_patients,
+      type: 'b/value/static',
+      register: :patients,
+      value: [
+        { chart_number: 'B0001', middle_name: 'Rabbit' },
+        { chart_number: 'C0001', middle_name: 'Elf' }
+      ]
+    },
+    {
+      name: 'update_patients',
+      type: 'db_fuel/active_record/update_all',
+      register: :patients,
+      attributes: [
+        { key: :last_name }
+      ],
+      table_name: 'patients',
+      unique_attributes: [
+        { key: :chart_number }
+      ]
+    }
+  ]
+}
+
+payload = Burner::Payload.new
+
+Burner::Pipeline.make(pipeline).execute(payload: payload)
+````
+
+Each database record should have been updated with their new respective middle names based on chart_number.
+
+#### Upserting Records
+
+Let's say we don't know if these chart_number values already exist or not. 
+So we want db_fuel to either insert a record if the chart_number doesn't exist or update the record if the chart_number already exists.
+
+````ruby
+pipeline = {
+  jobs: [
+    {
+      name: :load_patients,
+      type: 'b/value/static',
+      register: :patients,
+      value: [
+        { chart_number: 'B0002', first_name: 'Babs', last_name: 'Bunny' },
+        { chart_number: 'B0003', first_name: 'Daffy', last_name: 'Duck' }
+      ]
+    },
+    {
+      name: 'update_patients',
+      type: 'db_fuel/active_record/upsert',
+      register: :patients,
+      attributes: [
+        { key: :chart_number },
+        { key: :first_name },
+        { key: :last_name }
+      ],
+      table_name: 'patients',
+      primary_key: {
+        key: :id
+      },
+      unique_attributes: [
         { key: :chart_number }
       ]
     }
@@ -231,11 +371,12 @@ payload = Burner::Payload.new
 Burner::Pipeline.make(pipeline).execute(payload: payload)
 ````
 
-Each database record should have been updated with their new respective last names.
+Each database record should have been either inserted or updated with their corresponding values. In this case Babs' last name
+was switched back to Bunny and a new record was created for Daffy Duck.
 
 Notes:
 
-* The unique_keys translate to WHERE clauses.
+* The `unique_attributes` translate to WHERE clauses.
 * Set `debug: true` to print out each UPDATE statement in the output (not for production use.)
 ## Contributing
 

data/db_fuel.gemspec

@@ -11,8 +11,8 @@ Gem::Specification.new do |s|
     This library adds database-centric jobs to the Burner library.  Burner does not ship with database jobs out of the box.
   DESCRIPTION
 
-  s.authors     = ['Matthew Ruggio']
-  s.email       = ['mruggio@bluemarblepayroll.com']
+  s.authors     = ['Matthew Ruggio', 'John Bosko']
+  s.email       = ['mruggio@bluemarblepayroll.com', 'jbosko@bluemarblepayroll.com']
   s.files       = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
   s.bindir      = 'exe'
   s.executables = %w[]
@@ -51,7 +51,7 @@ Gem::Specification.new do |s|
   s.add_development_dependency('pry', '~>0')
   s.add_development_dependency('rake', '~> 13')
   s.add_development_dependency('rspec', '~> 3.8')
-  s.add_development_dependency('rubocop', '~>0.90.0')
+  s.add_development_dependency('rubocop', '~>1.7.0')
   s.add_development_dependency('simplecov', '~>0.18.5')
   s.add_development_dependency('simplecov-console', '~>0.7.0')
   s.add_development_dependency('sqlite3', '~>1')

data/lib/db_fuel.rb

@@ -17,4 +17,7 @@ require 'objectable'
 # General purpose classes used by the main job classes.
 require_relative 'db_fuel/modeling'
 
+# Internal logic used across jobs.
+require_relative 'db_fuel/db_provider'
+
 require_relative 'db_fuel/library'

data/lib/db_fuel/db_provider.rb

@@ -0,0 +1,82 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module DbFuel
+  # Intermediate internal API for Arel/ActiveRecord.  There is some overlap in job needs when
+  # it comes to the Arel interface so this class condenses down those needs into this class.
+  class DbProvider # :nodoc: all
+    attr_reader :arel_table
+
+    def initialize(table_name)
+      raise ArgumentError, 'table_name is required' if table_name.to_s.empty?
+
+      @arel_table = ::Arel::Table.new(table_name.to_s)
+
+      freeze
+    end
+
+    def first(object)
+      sql = first_sql(object)
+
+      ::ActiveRecord::Base.connection.exec_query(sql).first
+    end
+
+    def first_sql(object)
+      relation = arel_table.project(Arel.star).take(1)
+      manager  = apply_where(object, relation)
+
+      manager.to_sql
+    end
+
+    def insert_sql(object)
+      insert_manager(object).to_sql
+    end
+
+    def insert(object)
+      manager = insert_manager(object)
+
+      ::ActiveRecord::Base.connection.insert(manager)
+    end
+
+    def update(set_object, where_object)
+      manager = update_manager(set_object, where_object)
+
+      ::ActiveRecord::Base.connection.update(manager)
+    end
+
+    def update_sql(set_object, where_object)
+      update_manager(set_object, where_object).to_sql
+    end
+
+    private
+
+    def update_manager(set_object, where_object)
+      arel_row       = make_arel_row(set_object)
+      update_manager = ::Arel::UpdateManager.new.set(arel_row).table(arel_table)
+
+      apply_where(where_object, update_manager)
+    end
+
+    def apply_where(hash, manager)
+      (hash || {}).inject(manager) do |memo, (key, value)|
+        memo.where(arel_table[key].eq(value))
+      end
+    end
+
+    def insert_manager(object)
+      arel_row = make_arel_row(object)
+
+      ::Arel::InsertManager.new.insert(arel_row)
+    end
+
+    def make_arel_row(row)
+      row.map { |key, value| [arel_table[key], value] }
+    end
+  end
+end

data/lib/db_fuel/library.rb

@@ -7,8 +7,11 @@
 # LICENSE file in the root directory of this source tree.
 #
 
+require_relative 'library/active_record/find_or_insert'
 require_relative 'library/active_record/insert'
 require_relative 'library/active_record/update'
+require_relative 'library/active_record/update_all'
+require_relative 'library/active_record/upsert'
 
 require_relative 'library/dbee/query'
 require_relative 'library/dbee/range'
@@ -16,10 +19,13 @@ require_relative 'library/dbee/range'
 module Burner
   # Open up Burner::Jobs and add registrations for this libraries jobs.
   class Jobs
-    register 'db_fuel/active_record/insert', DbFuel::Library::ActiveRecord::Insert
-    register 'db_fuel/active_record/update', DbFuel::Library::ActiveRecord::Update
+    register 'db_fuel/active_record/find_or_insert', DbFuel::Library::ActiveRecord::FindOrInsert
+    register 'db_fuel/active_record/insert',         DbFuel::Library::ActiveRecord::Insert
+    register 'db_fuel/active_record/update',         DbFuel::Library::ActiveRecord::Update
+    register 'db_fuel/active_record/update_all',     DbFuel::Library::ActiveRecord::UpdateAll
+    register 'db_fuel/active_record/upsert',         DbFuel::Library::ActiveRecord::Upsert
 
-    register 'db_fuel/dbee/query', DbFuel::Library::Dbee::Query
-    register 'db_fuel/dbee/range', DbFuel::Library::Dbee::Range
+    register 'db_fuel/dbee/query',                   DbFuel::Library::Dbee::Query
+    register 'db_fuel/dbee/range',                   DbFuel::Library::Dbee::Range
   end
 end

data/lib/db_fuel/library/active_record/base.rb

@@ -19,10 +19,11 @@ module DbFuel
         NOW_TYPE   = 'r/value/now'
         UPDATED_AT = :updated_at
 
-        attr_reader :arel_table,
-                    :attribute_renderers,
+        attr_reader :attribute_renderers,
+                    :db_provider,
                     :debug,
-                    :resolver
+                    :resolver,
+                    :attribute_renderers_set
 
         def initialize(
           name:,
@@ -34,45 +35,20 @@ module DbFuel
         )
           super(name: name, register: register)
 
-          @arel_table = ::Arel::Table.new(table_name.to_s)
-          @debug      = debug || false
-
-          # set resolver first since make_attribute_renderers needs it.
           @resolver = Objectable.resolver(separator: separator)
-
-          @attribute_renderers = Burner::Modeling::Attribute
-                                 .array(attributes)
-                                 .map { |a| Burner::Modeling::AttributeRenderer.new(a, resolver) }
+          @attribute_renderers_set = Modeling::AttributeRendererSet.new(attributes: attributes,
+                                                                        resolver: resolver)
+          @db_provider = DbProvider.new(table_name)
+          @debug = debug || false
         end
 
-        private
-
-        def timestamp_attribute(key)
-          Burner::Modeling::Attribute.make(
-            key: key,
-            transformers: [
-              { type: NOW_TYPE }
-            ]
-          )
-        end
+        protected
 
         def debug_detail(output, message)
           return unless debug
 
           output.detail(message)
         end
-
-        def make_arel_row(row)
-          row.map { |key, value| [arel_table[key], value] }
-        end
-
-        def transform(row, time)
-          attribute_renderers.each_with_object({}) do |attribute_renderer, memo|
-            value = attribute_renderer.transform(row, time)
-
-            resolver.set(memo, attribute_renderer.key, value)
-          end
-        end
       end
     end
   end