db_fuel 2.0.1 → 2.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 81819bf8d353ae3425d90360089da3857aa5ccaeb56ffc499557bd036d318493
-  data.tar.gz: 32ee04a7d5c25f3f6b40dae72809aa8752d00e9736abf03bde7b0a08ef18f86a
+  metadata.gz: fba199a30d6af2387ecafebfbaa2a946a25f6a8630558c838d1967f8d7c41d51
+  data.tar.gz: 80aaa216b30b90477bf50b7f4a3e139e465870c129fa8a1030c34c0b0ce3d7b8
 SHA512:
-  metadata.gz: b9bf6c8852f0da644839fc0883daceae2808c4646e5a26581da878dca9f09c71e7b3de887c0ac4f2107a3a7cdd3e8ccc6e58f70e01a5717bbae5842aa6691387
-  data.tar.gz: c1c2fb22e493cecc947d1f597ba91118c5f62acd1dd4263842b3004aa8d18943ff8910600ab2a8bde39a54eb50de8db220e309941ff439a02d19895fec7d8d97
+  metadata.gz: 87ab66e3f9705fa672f0701cc500df58c218f0332a63809d94823c1eed1a0bd521418110bd95fabe63db1310079e0fba8b9db78d0bbae7a84a85096492afc122
+  data.tar.gz: a6bc96a500ff5dc25728d5ff933239e21536aded8f4ae94a7d375ceef29d5157e12746039838e4358be83a107e8be5fdc88bd049de9b241425c5d34a23974c6b

data/CHANGELOG.md

@@ -1,6 +1,14 @@
+
+# 2.1.0 (May 19th, 2021)
+
+Additions:
+
+* Added keys_register to all db_fuel/active_record jobs.
+
 # 2.0.1 (March 18th, 2021)
 
 Changes:
+
 * Updated attribute_renderer_set to avoid an evaluation time issue with acts_as_hashable.
 
 # 2.0.0 (March 16th, 2021)

data/README.md

@@ -24,11 +24,11 @@ Refer to the [Burner](https://github.com/bluemarblepayroll/burner) library for m
 
 ### ActiveRecord Jobs
 
-* **db_fuel/active_record/find_or_insert** [name, table_name, attributes, debug, primary_keyed_column, register, separator, timestamps, unique_attributes]: An extension of the `db_fuel/active_record/insert` job that adds an existence check before SQL insertion. The  `unique_attributes` will be converted to WHERE clauses for performing the existence check.
-* **db_fuel/active_record/insert** [name, table_name, attributes, debug, primary_keyed_column, register, separator, timestamps]: This job can take the objects in a register and insert them into a database table.  If primary_keyed_column is specified then its key will be set to the primary key.  Note that composite primary keys are not supported.  Attributes defines which object properties to convert to SQL.  Refer to the class and constructor specification for more detail.
-* **db_fuel/active_record/update_all** [name, table_name, attributes, debug, register, separator, timestamps, unique_attributes]: This job can take the objects in a register and updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while unique_attributes translate to WHERE clauses. One or more records may be updated at a time.  Refer to the class and constructor specification for more detail.
-* **db_fuel/active_record/update** [name, table_name, attributes, debug, register, primary_keyed_column, separator, timestamps, unique_attributes]: This job can take the unique objects in a register and updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while unique_attributes translate to WHERE clauses to find the records to update. The primary_keyed_column is used to update the unique record. Only one record will be updated per statement. Note that composite primary keys are not supported.  Refer to the class and constructor specification for more detail.
-* **db_fuel/active_record/upsert** [name, table_name, attributes, debug, primary_keyed_column, register, separator, timestamps, unique_attributes]: This job can take the objects in a register and either inserts or updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while each key in unique_attributes become a WHERE clause in order to check for the existence of a specific record. The updated record will use the primary_keyed_column specified to perform the UPDATE operation. Note that composite primary keys are not supported. Refer to the class and constructor specification for more detail.
+* **db_fuel/active_record/find_or_insert** [name, table_name, attributes, debug, primary_keyed_column, keys_register, register, separator, timestamps, unique_attributes]: An extension of the `db_fuel/active_record/insert` job that adds an existence check before SQL insertion. The  `unique_attributes` will be converted to WHERE clauses for performing the existence check.
+* **db_fuel/active_record/insert** [name, table_name, attributes, debug, primary_keyed_column, keys_register, register, separator, timestamps]: This job can take the objects in a register and insert them into a database table.  If primary_keyed_column is specified then its key will be set to the primary key.  Note that composite primary keys are not supported.  Attributes defines which object properties to convert to SQL.  Refer to the class and constructor specification for more detail.
+* **db_fuel/active_record/update_all** [name, table_name, attributes, debug, keys_register, register, separator, timestamps, unique_attributes]: This job can take the objects in a register and updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while unique_attributes translate to WHERE clauses. One or more records may be updated at a time.  Refer to the class and constructor specification for more detail.
+* **db_fuel/active_record/update** [name, table_name, attributes, debug, keys_register, register, primary_keyed_column, separator, timestamps, unique_attributes]: This job can take the unique objects in a register and updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while unique_attributes translate to WHERE clauses to find the records to update. The primary_keyed_column is used to update the unique record. Only one record will be updated per statement. Note that composite primary keys are not supported.  Refer to the class and constructor specification for more detail.
+* **db_fuel/active_record/upsert** [name, table_name, attributes, debug, primary_keyed_column, keys_register, register, separator, timestamps, unique_attributes]: This job can take the objects in a register and either inserts or updates them within a database table.  Attributes defines which object properties to convert to SQL SET clauses while each key in unique_attributes become a WHERE clause in order to check for the existence of a specific record. The updated record will use the primary_keyed_column specified to perform the UPDATE operation. Note that composite primary keys are not supported. Refer to the class and constructor specification for more detail.
 
 ### Dbee Jobs
 
@@ -331,7 +331,7 @@ Each database record should have been updated with their new respective middle n
 
 #### Upserting Records
 
-Let's say we don't know if these chart_number values already exist or not. 
+Let's say we don't know if these chart_number values already exist or not.
 So we want db_fuel to either insert a record if the chart_number doesn't exist or update the record if the chart_number already exists.
 
 ````ruby
@@ -378,6 +378,10 @@ Notes:
 
 * The `unique_attributes` translate to WHERE clauses.
 * Set `debug: true` to print out each UPDATE statement in the output (not for production use.)
+
+#### Limiting The Columns Being Inserted/Updated
+
+All the db_fuel/active_record/* jobs now feature an optional `keys_register` option.  If this is set, the register will be read and used as a key filter for which fields to set.  For example, if you configure a db_fuel/active_record/insert job to update A, B, C, and D, but your keys_register contains [A,B] then only A and B will be inserted/set.  This is helpful in scenarios where you may want to outline how to update _all_ possible fields but your input set only contains a subset of those fields and you do not wish to set the others to null.
 ## Contributing
 
 ### Development Environment Configuration

data/lib/db_fuel/library/active_record/base.rb

@@ -22,6 +22,7 @@ module DbFuel
         attr_reader :attribute_renderers,
                     :db_provider,
                     :debug,
+                    :keys_register,
                     :resolver,
                     :attribute_renderers_set
 
@@ -30,12 +31,14 @@ module DbFuel
           name: '',
           attributes: [],
           debug: false,
+          keys_register: nil,
           register: Burner::DEFAULT_REGISTER,
           separator: ''
         )
           super(name: name, register: register)
 
-          @resolver = Objectable.resolver(separator: separator)
+          @keys_register           = keys_register.to_s
+          @resolver                = Objectable.resolver(separator: separator)
           @attribute_renderers_set = Modeling::AttributeRendererSet.new(resolver: resolver,
                                                                         attributes: attributes)
           @db_provider = DbProvider.new(table_name)
@@ -49,6 +52,16 @@ module DbFuel
 
           output.detail(message)
         end
+
+        def resolve_key_set(output, payload)
+          return Set.new if keys_register.empty?
+
+          keys = array(payload[keys_register]).map(&:to_s).to_set
+
+          output.detail("Limiting to only keys: #{keys.to_a.join(', ')}") if keys.any?
+
+          keys
+        end
       end
     end
   end

data/lib/db_fuel/library/active_record/find_or_insert.rb

@@ -57,6 +57,7 @@ module DbFuel
           name: '',
           attributes: [],
           debug: false,
+          keys_register: nil,
           primary_keyed_column: nil,
           register: Burner::DEFAULT_REGISTER,
           separator: '',
@@ -66,6 +67,7 @@ module DbFuel
 
           super(
             name: name,
+            keys_register: keys_register,
             table_name: table_name,
             attributes: attributes,
             debug: debug,
@@ -82,6 +84,7 @@ module DbFuel
           total_existed  = 0
 
           payload[register] = array(payload[register])
+          keys              = resolve_key_set(output, payload)
 
           payload[register].each do |row|
             exists = find_record(output, row, payload.time)
@@ -91,7 +94,7 @@ module DbFuel
               next
             end
 
-            insert_record(output, row, payload.time)
+            insert_record(output, row, payload.time, keys)
 
             total_inserted += 1
           end

data/lib/db_fuel/library/active_record/insert.rb

@@ -45,6 +45,7 @@ module DbFuel
         #               to the current UTC timestamp.
         def initialize(
           table_name:,
+          keys_register: nil,
           name: '',
           attributes: [],
           debug: false,
@@ -53,11 +54,11 @@ module DbFuel
           separator: '',
           timestamps: true
         )
-
           attributes = Burner::Modeling::Attribute.array(attributes)
 
           super(
             name: name,
+            keys_register: keys_register,
             table_name: table_name,
             attributes: attributes,
             debug: debug,
@@ -70,8 +71,11 @@ module DbFuel
 
         def perform(output, payload)
           payload[register] = array(payload[register])
+          keys              = resolve_key_set(output, payload)
+
+          output.detail("Inserting #{payload[register].count} record(s)")
 
-          payload[register].each { |row| insert_record(output, row, payload.time) }
+          payload[register].each { |row| insert_record(output, row, payload.time, keys) }
         end
       end
     end

data/lib/db_fuel/library/active_record/update.rb

@@ -57,6 +57,7 @@ module DbFuel
           attributes: [],
           debug: false,
           primary_keyed_column: nil,
+          keys_register: nil,
           register: Burner::DEFAULT_REGISTER,
           separator: '',
           timestamps: true,
@@ -70,6 +71,7 @@ module DbFuel
             table_name: table_name,
             attributes: attributes,
             debug: debug,
+            keys_register: keys_register,
             primary_keyed_column: primary_keyed_column,
             register: register,
             separator: separator,
@@ -81,20 +83,16 @@ module DbFuel
         end
 
         def perform(output, payload)
-          total_rows_affected = 0
-
           payload[register] = array(payload[register])
+          keys              = resolve_key_set(output, payload)
 
-          payload[register].each do |row|
-            rows_affected = 0
-
-            first_record = update_record(output, row, payload.time)
-
-            rows_affected = 1 if first_record
+          total_rows_affected = payload[register].inject(0) do |memo, row|
+            first_record  = update_record(output, row, payload.time, keys)
+            rows_affected = first_record ? 1 : 0
 
             debug_detail(output, "Individual Rows Affected: #{rows_affected}")
 
-            total_rows_affected += rows_affected
+            memo + rows_affected
           end
 
           output.detail("Total Rows Affected: #{total_rows_affected}")

data/lib/db_fuel/library/active_record/update_all.rb

@@ -50,6 +50,7 @@ module DbFuel
           attributes: [],
           debug: false,
           register: Burner::DEFAULT_REGISTER,
+          keys_register: nil,
           separator: '',
           timestamps: true,
           unique_attributes: []
@@ -63,6 +64,7 @@ module DbFuel
             attributes: attributes,
             debug: debug,
             primary_keyed_column: nil,
+            keys_register: keys_register,
             register: register,
             separator: separator,
             timestamps: timestamps,
@@ -73,19 +75,21 @@ module DbFuel
         end
 
         def perform(output, payload)
-          total_rows_affected = 0
-
           payload[register] = array(payload[register])
+          keys              = resolve_key_set(output, payload)
 
-          payload[register].each do |row|
-            where_object = attribute_renderers_set
-                           .transform(unique_attribute_renderers, row, payload.time)
+          total_rows_affected = payload[register].inject(0) do |memo, row|
+            where_object = attribute_renderers_set.transform(
+              unique_attribute_renderers,
+              row,
+              payload.time
+            )
 
-            rows_affected = update(output, row, payload.time, where_object)
+            rows_affected = update(output, row, payload.time, where_object, keys)
 
             debug_detail(output, "Individual Rows Affected: #{rows_affected}")
 
-            total_rows_affected += rows_affected
+            memo + rows_affected
           end
 
           output.detail("Total Rows Affected: #{total_rows_affected}")

data/lib/db_fuel/library/active_record/upsert.rb

@@ -59,6 +59,7 @@ module DbFuel
         def initialize(
           table_name:,
           primary_keyed_column:,
+          keys_register: nil,
           name: '',
           attributes: [],
           debug: false,
@@ -69,6 +70,7 @@ module DbFuel
         )
           super(
             name: name,
+            keys_register: keys_register,
             table_name: table_name,
             attributes: attributes,
             debug: debug,
@@ -93,9 +95,10 @@ module DbFuel
           total_updated  = 0
 
           payload[register] = array(payload[register])
+          keys              = resolve_key_set(output, payload)
 
           payload[register].each do |row|
-            record_updated = insert_or_update(output, row, payload.time)
+            record_updated = insert_or_update(output, row, payload.time, keys)
 
             if record_updated
               total_updated += 1
@@ -128,7 +131,7 @@ module DbFuel
           first_record
         end
 
-        def insert_record(output, row, time)
+        def insert_record(output, row, time, keys = Set.new)
           dynamic_attrs = if timestamps
                             # doing an INSERT and timestamps should be set
                             # set the created_at and updated_at fields
@@ -137,7 +140,7 @@ module DbFuel
                             attribute_renderers_set.attribute_renderers
                           end
 
-          set_object = attribute_renderers_set.transform(dynamic_attrs, row, time)
+          set_object = attribute_renderers_set.transform(dynamic_attrs, row, time, keys)
 
           insert_sql = db_provider.insert_sql(set_object)
 
@@ -152,7 +155,7 @@ module DbFuel
         end
 
         # Updates only a single record. Lookups primary key to update the record.
-        def update_record(output, row, time)
+        def update_record(output, row, time, keys)
           raise ArgumentError, 'primary_keyed_column is required' unless primary_keyed_column
 
           first_record = find_record(output, row, time)
@@ -165,14 +168,14 @@ module DbFuel
             where_object = { primary_keyed_column.column => id }
 
             # update record using the primary key as the WHERE clause
-            update(output, row, time, where_object)
+            update(output, row, time, where_object, keys)
           end
 
           first_record
         end
 
         # Updates one or many records depending on where_object passed
-        def update(output, row, time, where_object)
+        def update(output, row, time, where_object, keys)
           dynamic_attrs = if timestamps
                             # doing an UPDATE and timestamps should be set,
                             # modify the updated_at field, don't modify the created_at field
@@ -181,7 +184,7 @@ module DbFuel
                             attribute_renderers_set.attribute_renderers
                           end
 
-          set_object = attribute_renderers_set.transform(dynamic_attrs, row, time)
+          set_object = attribute_renderers_set.transform(dynamic_attrs, row, time, keys)
 
           update_sql = db_provider.update_sql(set_object, where_object)
 
@@ -194,14 +197,14 @@ module DbFuel
 
         private
 
-        def insert_or_update(output, row, time)
-          first_record = update_record(output, row, time)
+        def insert_or_update(output, row, time, keys)
+          first_record = update_record(output, row, time, keys)
 
           if first_record
             first_record
           else
             # create the record
-            insert_record(output, row, time)
+            insert_record(output, row, time, keys)
             nil
           end
         end

data/lib/db_fuel/modeling/attribute_renderer_set.rb

@@ -52,8 +52,10 @@ module DbFuel
           .map { |a| Burner::Modeling::AttributeRenderer.new(a, resolver) }
       end
 
-      def transform(attribute_renderers, row, time)
+      def transform(attribute_renderers, row, time, keys = Set.new)
         attribute_renderers.each_with_object({}) do |attribute_renderer, memo|
+          next if keys.any? && keys.exclude?(attribute_renderer.key)
+
           value = attribute_renderer.transform(row, time)
 
           resolver.set(memo, attribute_renderer.key, value)

data/lib/db_fuel/version.rb

@@ -8,5 +8,5 @@
 #
 
 module DbFuel
-  VERSION = '2.0.1'
+  VERSION = '2.1.0'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: db_fuel
 version: !ruby/object:Gem::Version
-  version: 2.0.1
+  version: 2.1.0
 platform: ruby
 authors:
 - Matthew Ruggio
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-03-18 00:00:00.000000000 Z
+date: 2021-05-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord