activerecord-import 0.25.0 → 0.26.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 26224d6c0d4c23f3edaf1faea68466f18c03fbbc
-  data.tar.gz: 55107f77211b86e3be77b287d224e488dbb280ac
+SHA256:
+  metadata.gz: ecfc0672f1042ae828bb3125c0675364d1af8e1b4aefb82af633d3656a8d5fc6
+  data.tar.gz: 8b03926293827932a61342fded8e5400fbe8261e92817ce32c09eda7fa16860f
 SHA512:
-  metadata.gz: c0815961332178cf2858b39663f1d9f6ef3d61985cd4cb528cef410714e4bf07c354515b5e06124ad4bb5c7e57e7a179c48d7fb5c5e28886bc9e30c4399e82c9
-  data.tar.gz: df5e8ccd8a658d4981cd26167e09fa8c5e0425c58e4397c48e6f7721221d1de3cc4c6cdf903e3330f07cf6a74b93214cd11cca977f6d75cf6e8b43409807c23b
+  metadata.gz: 874795a3c02d59b9411ab831ba6257c0c4dab7bad2b478e69451b3d5344652f4d2f906f727189170dc8417ae64a5a2f236a61da1b24a8c1112a7e731a087a7ca
+  data.tar.gz: 00abfb0263c84d2c687451bd1344ae73935291b8db074388c64220d1d6e5ee9dccc515f62217cc4a61c5912acd1de60d5a57ac7229b3734fa145bfd7936f5b56

data/CHANGELOG.md

@@ -1,3 +1,16 @@
+## Changes in 0.26.0
+
+### New Features
+
+* Add  on_duplicate_key_update for SQLite. Thanks to @jkowens via \#542.
+* Add option to update all fields on_duplicate_key_update. Thanks to @aimerald, @jkowens via \#543.
+
+### Fixes
+
+* Handle deeply frozen options hashes. Thanks to @jturkel via \#546.
+* Switch from FactoryGirl to FactoryBot. Thanks to @koic via \#547.
+* Allow import to work with ProxySQL. Thanks to @GregFarrell via \#550.
+
 ## Changes in 0.25.0
 
 ### New Features

data/Gemfile

@@ -29,7 +29,7 @@ platforms :jruby do
 end
 
 # Support libs
-gem "factory_girl", "~> 4.2.0"
+gem "factory_bot"
 gem "timecop"
 gem "chronic"
 gem "mocha", "~> 1.3.0"

data/README.markdown

@@ -21,11 +21,53 @@ and then the reviews:
 That would be about 4M SQL insert statements vs 3, which results in vastly improved performance. In our case, it converted
 an 18 hour batch process to <2 hrs.
 
-### More Information : Usage and Examples in Wiki
+## Table of Contents
 
-For more information on activerecord-import please see its wiki: https://github.com/zdennis/activerecord-import/wiki
+* [Callbacks](#callbacks)
+* [Additional Adapters](#additional-adapters)
+* [Load Path Setup](#load-path-setup)
+* [More Information](#more-information)
+
+### Callbacks
+
+ActiveRecord callbacks related to [creating](http://guides.rubyonrails.org/active_record_callbacks.html#creating-an-object), [updating](http://guides.rubyonrails.org/active_record_callbacks.html#updating-an-object), or [destroying](http://guides.rubyonrails.org/active_record_callbacks.html#destroying-an-object) records (other than `before_validation` and `after_validation`) will NOT be called when calling the import method. This is because it is mass importing rows of data and doesn't necessarily have access to in-memory ActiveRecord objects.
+
+If you do have a collection of in-memory ActiveRecord objects you can do something like this:
+
+```
+books.each do |book|
+  book.run_callbacks(:save) { false }
+  book.run_callbacks(:create) { false }
+end
+Book.import(books)
+```
+
+This will run before_create and before_save callbacks on each item. The `false` argument is needed to prevent after_save being run, which wouldn't make sense prior to bulk import. Something to note in this example is that the before_create and before_save callbacks will run before the validation callbacks.
+
+If that is an issue, another possible approach is to loop through your models first to do validations and then only run callbacks on and import the valid models.
+
+```
+valid_books = []
+invalid_books = []
+
+books.each do |book|
+  if book.valid?
+    valid_books << book
+  else
+    invalid_books << book
+  end
+end
+
+valid_books.each do |book|
+  book.run_callbacks(:save) { false }
+  book.run_callbacks(:create) { false }
+end
+
+Book.import valid_books, validate: false
+```
+
+### Additional Adapters
 
-## Additional Adapters
 Additional adapters can be provided by gems external to activerecord-import by providing an adapter that matches the naming convention setup by activerecord-import (and subsequently activerecord) for dynamically loading adapters.  This involves also providing a folder on the load path that follows the activerecord-import naming convention to allow activerecord-import to dynamically load the file.
 
 When `ActiveRecord::Import.require_adapter("fake_name")` is called the require will be:
@@ -63,6 +105,12 @@ activerecord-import-fake_name/
 
 When rubygems pushes the `lib` folder onto the load path a `require` will now find `activerecord-import/active_record/adapters/fake_name_adapter` as it runs through the lookup process for a ruby file under that path in `$LOAD_PATH`
 
+### More Information
+
+For more information on activerecord-import please see its wiki: https://github.com/zdennis/activerecord-import/wiki
+
+To document new information, please add to the README instead of the wiki. See https://github.com/zdennis/activerecord-import/issues/397 for discussion.
+
 # License
 
 This is licensed under the ruby license.

data/Rakefile

@@ -32,7 +32,7 @@ ADAPTERS.each do |adapter|
   namespace :test do
     desc "Runs #{adapter} database tests."
     Rake::TestTask.new(adapter) do |t|
-      # FactoryGirl has an issue with warnings, so turn off, so noisy
+      # FactoryBot has an issue with warnings, so turn off, so noisy
       # t.warning = true
       t.test_files = FileList["test/adapters/#{adapter}.rb", "test/*_test.rb", "test/active_record/*_test.rb", "test/#{adapter}/**/*_test.rb"]
     end

data/lib/activerecord-import/adapters/mysql_adapter.rb

@@ -56,7 +56,7 @@ module ActiveRecord::Import::MysqlAdapter
   # in a single packet
   def max_allowed_packet # :nodoc:
     @max_allowed_packet ||= begin
-      result = execute( "SHOW VARIABLES like 'max_allowed_packet';" )
+      result = execute( "SHOW VARIABLES like 'max_allowed_packet'" )
       # original Mysql gem responds to #fetch_row while Mysql2 responds to #first
       val = result.respond_to?(:fetch_row) ? result.fetch_row[1] : result.first[1]
       val.to_i

data/lib/activerecord-import/adapters/sqlite3_adapter.rb

@@ -1,7 +1,9 @@
 module ActiveRecord::Import::SQLite3Adapter
   include ActiveRecord::Import::ImportSupport
+  include ActiveRecord::Import::OnDuplicateKeyUpdateSupport
 
   MIN_VERSION_FOR_IMPORT = "3.7.11".freeze
+  MIN_VERSION_FOR_UPSERT = "3.24.0".freeze
   SQLITE_LIMIT_COMPOUND_SELECT = 500
 
   # Override our conformance to ActiveRecord::Import::ImportSupport interface
@@ -15,6 +17,10 @@ module ActiveRecord::Import::SQLite3Adapter
     end
   end
 
+  def supports_on_duplicate_key_update?(current_version = sqlite_version)
+    current_version >= MIN_VERSION_FOR_UPSERT
+  end
+
   # +sql+ can be a single string or an array. If it is an array all
   # elements that are in position >= 1 will be appended to the final SQL.
   def insert_many( sql, values, _options = {}, *args ) # :nodoc:
@@ -40,16 +46,133 @@ module ActiveRecord::Import::SQLite3Adapter
     ActiveRecord::Import::Result.new([], number_of_inserts, [], [])
   end
 
-  def pre_sql_statements( options)
+  def pre_sql_statements( options )
     sql = []
     # Options :recursive and :on_duplicate_key_ignore are mutually exclusive
-    if (options[:ignore] || options[:on_duplicate_key_ignore]) && !options[:recursive]
+    if !supports_on_duplicate_key_update? && (options[:ignore] || options[:on_duplicate_key_ignore])
       sql << "OR IGNORE"
     end
     sql + super
   end
 
+  def post_sql_statements( table_name, options ) # :nodoc:
+    sql = []
+
+    if supports_on_duplicate_key_update?
+      # Options :recursive and :on_duplicate_key_ignore are mutually exclusive
+      if (options[:ignore] || options[:on_duplicate_key_ignore]) && !options[:on_duplicate_key_update]
+        sql << sql_for_on_duplicate_key_ignore( options[:on_duplicate_key_ignore] )
+      end
+    end
+
+    sql + super
+  end
+
   def next_value_for_sequence(sequence_name)
     %{nextval('#{sequence_name}')}
   end
+
+  # Add a column to be updated on duplicate key update
+  def add_column_for_on_duplicate_key_update( column, options = {} ) # :nodoc:
+    arg = options[:on_duplicate_key_update]
+    if arg.is_a?( Hash )
+      columns = arg.fetch( :columns ) { arg[:columns] = [] }
+      case columns
+      when Array then columns << column.to_sym unless columns.include?( column.to_sym )
+      when Hash then columns[column.to_sym] = column.to_sym
+      end
+    elsif arg.is_a?( Array )
+      arg << column.to_sym unless arg.include?( column.to_sym )
+    end
+  end
+
+  # Returns a generated ON CONFLICT DO NOTHING statement given the passed
+  # in +args+.
+  def sql_for_on_duplicate_key_ignore( *args ) # :nodoc:
+    arg = args.first
+    conflict_target = sql_for_conflict_target( arg ) if arg.is_a?( Hash )
+    " ON CONFLICT #{conflict_target}DO NOTHING"
+  end
+
+  # Returns a generated ON CONFLICT DO UPDATE statement given the passed
+  # in +args+.
+  def sql_for_on_duplicate_key_update( _table_name, *args ) # :nodoc:
+    arg, primary_key, locking_column = args
+    arg = { columns: arg } if arg.is_a?( Array ) || arg.is_a?( String )
+    return unless arg.is_a?( Hash )
+
+    sql = ' ON CONFLICT '
+    conflict_target = sql_for_conflict_target( arg )
+
+    columns = arg.fetch( :columns, [] )
+    condition = arg[:condition]
+    if columns.respond_to?( :empty? ) && columns.empty?
+      return sql << "#{conflict_target}DO NOTHING"
+    end
+
+    conflict_target ||= sql_for_default_conflict_target( primary_key )
+    unless conflict_target
+      raise ArgumentError, 'Expected :conflict_target to be specified'
+    end
+
+    sql << "#{conflict_target}DO UPDATE SET "
+    if columns.is_a?( Array )
+      sql << sql_for_on_duplicate_key_update_as_array( locking_column, columns )
+    elsif columns.is_a?( Hash )
+      sql << sql_for_on_duplicate_key_update_as_hash( locking_column, columns )
+    elsif columns.is_a?( String )
+      sql << columns
+    else
+      raise ArgumentError, 'Expected :columns to be an Array or Hash'
+    end
+
+    sql << " WHERE #{condition}" if condition.present?
+
+    sql
+  end
+
+  def sql_for_on_duplicate_key_update_as_array( locking_column, arr ) # :nodoc:
+    results = arr.map do |column|
+      qc = quote_column_name( column )
+      "#{qc}=EXCLUDED.#{qc}"
+    end
+    increment_locking_column!(results, locking_column)
+    results.join( ',' )
+  end
+
+  def sql_for_on_duplicate_key_update_as_hash( locking_column, hsh ) # :nodoc:
+    results = hsh.map do |column1, column2|
+      qc1 = quote_column_name( column1 )
+      qc2 = quote_column_name( column2 )
+      "#{qc1}=EXCLUDED.#{qc2}"
+    end
+    increment_locking_column!(results, locking_column)
+    results.join( ',' )
+  end
+
+  def sql_for_conflict_target( args = {} )
+    conflict_target = args[:conflict_target]
+    index_predicate = args[:index_predicate]
+    if conflict_target.present?
+      '(' << Array( conflict_target ).reject( &:blank? ).join( ', ' ) << ') '.tap do |sql|
+        sql << "WHERE #{index_predicate} " if index_predicate
+      end
+    end
+  end
+
+  def sql_for_default_conflict_target( primary_key )
+    conflict_target = Array(primary_key).join(', ')
+    "(#{conflict_target}) " if conflict_target.present?
+  end
+
+  # Return true if the statement is a duplicate key record error
+  def duplicate_key_update_error?(exception) # :nodoc:
+    exception.is_a?(ActiveRecord::StatementInvalid) && exception.to_s.include?('duplicate key')
+  end
+
+  def increment_locking_column!(results, locking_column)
+    if locking_column.present?
+      results << "\"#{locking_column}\"=EXCLUDED.\"#{locking_column}\"+1"
+    end
+  end
 end

data/lib/activerecord-import/import.rb

@@ -298,8 +298,8 @@ class ActiveRecord::Base
     #   recursive import. For database adapters that normally support
     #   setting primary keys on imported objects, this option prevents
     #   that from occurring.
-    # * +on_duplicate_key_update+ - an Array or Hash, tells import to
-    #   use MySQL's ON DUPLICATE KEY UPDATE or Postgres 9.5+ ON CONFLICT
+    # * +on_duplicate_key_update+ - :all, an Array, or Hash, tells import to
+    #   use MySQL's ON DUPLICATE KEY UPDATE or Postgres/SQLite ON CONFLICT
     #   DO UPDATE ability. See On Duplicate Key Update below.
     # * +synchronize+ - an array of ActiveRecord instances for the model
     #   that you are currently importing data into. This synchronizes
@@ -358,7 +358,15 @@ class ActiveRecord::Base
     #
     # == On Duplicate Key Update (MySQL)
     #
-    # The :on_duplicate_key_update option can be either an Array or a Hash.
+    # The :on_duplicate_key_update option can be either :all, an Array, or a Hash.
+    #
+    # ==== Using :all
+    #
+    # The :on_duplicate_key_update option can be set to :all. All columns
+    # other than the primary key are updated. If a list of column names is
+    # supplied, only those columns will be updated. Below is an example:
+    #
+    #   BlogPost.import columns, values, on_duplicate_key_update: :all
     #
     # ==== Using an Array
     #
@@ -377,11 +385,19 @@ class ActiveRecord::Base
     #
     #   BlogPost.import columns, attributes, on_duplicate_key_update: { title: :title }
     #
-    # == On Duplicate Key Update (Postgres 9.5+)
+    # == On Duplicate Key Update (Postgres 9.5+ and SQLite 3.24+)
     #
-    # The :on_duplicate_key_update option can be an Array or a Hash with up to
+    # The :on_duplicate_key_update option can be :all, an Array, or a Hash with up to
     # three attributes, :conflict_target (and optionally :index_predicate) or
-    # :constraint_name, and :columns.
+    # :constraint_name (Postgres), and :columns.
+    #
+    # ==== Using :all
+    #
+    # The :on_duplicate_key_update option can be set to :all. All columns
+    # other than the primary key are updated. If a list of column names is
+    # supplied, only those columns will be updated. Below is an example:
+    #
+    #   BlogPost.import columns, values, on_duplicate_key_update: :all
     #
     # ==== Using an Array
     #
@@ -439,7 +455,15 @@ class ActiveRecord::Base
     #
     # ===== :columns
     #
-    # The :columns attribute can be either an Array or a Hash.
+    # The :columns attribute can be either :all, an Array, or a Hash.
+    #
+    # ===== Using :all
+    #
+    # The :columns attribute can be :all. All columns other than the primary key will be updated.
+    # If a list of column names is supplied, only those columns will be updated.
+    # Below is an example:
+    #
+    #   BlogPost.import columns, values, on_duplicate_key_update: { conflict_target: :slug, columns: :all }
     #
     # ===== Using an Array
     #
@@ -495,16 +519,6 @@ class ActiveRecord::Base
       options[:primary_key] = primary_key
       options[:locking_column] = locking_column if attribute_names.include?(locking_column)
 
-      # Don't modify incoming arguments
-      on_duplicate_key_update = options[:on_duplicate_key_update]
-      if on_duplicate_key_update && on_duplicate_key_update.duplicable?
-        options[:on_duplicate_key_update] = if on_duplicate_key_update.is_a?(Hash)
-          on_duplicate_key_update.each { |k, v| on_duplicate_key_update[k] = v.dup if v.duplicable? }
-        else
-          on_duplicate_key_update.dup
-        end
-      end
-
       is_validating = options[:validate_with_context].present? ? true : options[:validate]
       validator = ActiveRecord::Import::Validator.new(options)
 
@@ -522,8 +536,12 @@ class ActiveRecord::Base
           end
         end
 
-        if models.first.id.nil? && column_names.include?(primary_key) && columns_hash[primary_key].type == :uuid
-          column_names.delete(primary_key)
+        if models.first.id.nil?
+          Array(primary_key).each do |c|
+            if column_names.include?(c) && columns_hash[c].type == :uuid
+              column_names.delete(c)
+            end
+          end
         end
 
         default_values = column_defaults
@@ -611,6 +629,29 @@ class ActiveRecord::Base
         array_of_attributes.each { |a| a.concat(new_fields) }
       end
 
+      # Don't modify incoming arguments
+      on_duplicate_key_update = options[:on_duplicate_key_update]
+      if on_duplicate_key_update
+        updatable_columns = symbolized_column_names.reject { |c| symbolized_primary_key.include? c }
+        options[:on_duplicate_key_update] = if on_duplicate_key_update.is_a?(Hash)
+          on_duplicate_key_update.each_with_object({}) do |(k, v), duped_options|
+            duped_options[k] = if k == :columns && v == :all
+              updatable_columns
+            elsif v.duplicable?
+              v.dup
+            else
+              v
+            end
+          end
+        elsif on_duplicate_key_update == :all
+          updatable_columns
+        elsif on_duplicate_key_update.duplicable?
+          on_duplicate_key_update.dup
+        else
+          on_duplicate_key_update
+        end
+      end
+
       timestamps = {}
 
       # record timestamps unless disabled in ActiveRecord::Base
@@ -646,7 +687,7 @@ class ActiveRecord::Base
       end
 
       if options[:synchronize]
-        sync_keys = options[:synchronize_keys] || [primary_key]
+        sync_keys = options[:synchronize_keys] || Array(primary_key)
         synchronize( options[:synchronize], sync_keys)
       end
       return_obj.num_inserts = 0 if return_obj.num_inserts.nil?
@@ -871,7 +912,7 @@ class ActiveRecord::Base
           column = columns[j]
 
           # be sure to query sequence_name *last*, only if cheaper tests fail, because it's costly
-          if val.nil? && column.name == primary_key && !sequence_name.blank?
+          if val.nil? && Array(primary_key).first == column.name && !sequence_name.blank?
             connection_memo.next_value_for_sequence(sequence_name)
           elsif val.respond_to?(:to_sql)
             "(#{val.to_sql})"

data/lib/activerecord-import/version.rb

@@ -1,5 +1,5 @@
 module ActiveRecord
   module Import
-    VERSION = "0.25.0".freeze
+    VERSION = "0.26.0".freeze
   end
 end

data/test/import_test.rb

@@ -567,7 +567,7 @@ describe "#import" do
 
   context "importing through an association scope" do
     { has_many: :chapters, polymorphic: :discounts }.each do |association_type, association|
-      book   = FactoryGirl.create :book
+      book   = FactoryBot.create :book
       scope  = book.public_send association
       klass  = { chapters: Chapter, discounts: Discount }[association]
       column = { chapters: :title,  discounts: :amount  }[association]
@@ -609,7 +609,7 @@ describe "#import" do
 
   context "importing model with polymorphic belongs_to" do
     it "works without error" do
-      book     = FactoryGirl.create :book
+      book     = FactoryBot.create :book
       discount = Discount.new(discountable: book)
 
       Discount.import([discount])

data/test/schema/sqlite3_schema.rb

@@ -0,0 +1,13 @@
+ActiveRecord::Schema.define do
+  create_table :alarms, force: true do |t|
+    t.column :device_id, :integer, null: false
+    t.column :alarm_type, :integer, null: false
+    t.column :status, :integer, null: false
+    t.column :metadata, :text
+    t.column :secret_key, :binary
+    t.datetime :created_at
+    t.datetime :updated_at
+  end
+
+  add_index :alarms, [:device_id, :alarm_type], unique: true, where: 'status <> 0'
+end

data/test/support/factories.rb

@@ -1,4 +1,4 @@
-FactoryGirl.define do
+FactoryBot.define do
   sequence(:book_title) { |n| "Book #{n}" }
   sequence(:chapter_title) { |n| "Chapter #{n}" }
   sequence(:end_note) { |n| "Endnote #{n}" }
@@ -9,7 +9,7 @@ FactoryGirl.define do
 
   factory :invalid_topic, class: "Topic" do
     sequence(:title) { |n| "Title #{n}" }
-    author_name nil
+    author_name { nil }
   end
 
   factory :topic do
@@ -27,7 +27,7 @@ FactoryGirl.define do
 
     trait :with_rule do
       after(:build) do |question|
-        question.build_rule(FactoryGirl.attributes_for(:rule))
+        question.build_rule(FactoryBot.attributes_for(:rule))
       end
     end
   end
@@ -40,21 +40,21 @@ FactoryGirl.define do
   factory :topic_with_book, parent: :topic do
     after(:build) do |topic|
       2.times do
-        book = topic.books.build(title: FactoryGirl.generate(:book_title), author_name: 'Stephen King')
+        book = topic.books.build(title: FactoryBot.generate(:book_title), author_name: 'Stephen King')
         3.times do
-          book.chapters.build(title: FactoryGirl.generate(:chapter_title))
+          book.chapters.build(title: FactoryBot.generate(:chapter_title))
         end
 
         4.times do
-          book.end_notes.build(note: FactoryGirl.generate(:end_note))
+          book.end_notes.build(note: FactoryBot.generate(:end_note))
         end
       end
     end
   end
 
   factory :book do
-    title 'Tortilla Flat'
-    author_name 'John Steinbeck'
+    title { 'Tortilla Flat' }
+    author_name { 'John Steinbeck' }
   end
 
   factory :car do

data/test/support/generate.rb

@@ -2,28 +2,28 @@ class ActiveSupport::TestCase
   def Build(*args) # rubocop:disable Style/MethodName
     n = args.shift if args.first.is_a?(Numeric)
     factory = args.shift
-    factory_girl_args = args.shift || {}
+    factory_bot_args = args.shift || {}
 
     if n
       [].tap do |collection|
-        n.times.each { collection << FactoryGirl.build(factory.to_s.singularize.to_sym, factory_girl_args) }
+        n.times.each { collection << FactoryBot.build(factory.to_s.singularize.to_sym, factory_bot_args) }
       end
     else
-      FactoryGirl.build(factory.to_s.singularize.to_sym, factory_girl_args)
+      FactoryBot.build(factory.to_s.singularize.to_sym, factory_bot_args)
     end
   end
 
   def Generate(*args) # rubocop:disable Style/MethodName
     n = args.shift if args.first.is_a?(Numeric)
     factory = args.shift
-    factory_girl_args = args.shift || {}
+    factory_bot_args = args.shift || {}
 
     if n
       [].tap do |collection|
-        n.times.each { collection << FactoryGirl.create(factory.to_s.singularize.to_sym, factory_girl_args) }
+        n.times.each { collection << FactoryBot.create(factory.to_s.singularize.to_sym, factory_bot_args) }
       end
     else
-      FactoryGirl.create(factory.to_s.singularize.to_sym, factory_girl_args)
+      FactoryBot.create(factory.to_s.singularize.to_sym, factory_bot_args)
     end
   end
 end

data/test/support/postgresql/import_examples.rb

@@ -295,6 +295,30 @@ def should_support_postgresql_upsert_functionality
       end
 
       context "using a hash" do
+        context "with :columns :all" do
+          let(:columns) { %w( id title author_name author_email_address parent_id ) }
+          let(:updated_values) { [[99, "Book - 2nd Edition", "Jane Doe", "janedoe@example.com", 57]] }
+
+          macro(:perform_import) do |*opts|
+            Topic.import columns, updated_values, opts.extract_options!.merge(on_duplicate_key_update: { conflict_target: :id, columns: :all }, validate: false)
+          end
+
+          setup do
+            values = [[99, "Book", "John Doe", "john@doe.com", 17, 3]]
+            Topic.import columns + ['replies_count'], values, validate: false
+          end
+
+          it "should update all specified columns" do
+            perform_import
+            updated_topic = Topic.find(99)
+            assert_equal 'Book - 2nd Edition', updated_topic.title
+            assert_equal 'Jane Doe', updated_topic.author_name
+            assert_equal 'janedoe@example.com', updated_topic.author_email_address
+            assert_equal 57, updated_topic.parent_id
+            assert_equal 3, updated_topic.replies_count
+          end
+        end
+
         context "with :columns a hash" do
           let(:columns) { %w( id title author_name author_email_address parent_id ) }
           let(:values) { [[99, "Book", "John Doe", "john@doe.com", 17]] }
@@ -312,7 +336,7 @@ def should_support_postgresql_upsert_functionality
           it "should not modify the passed in :on_duplicate_key_update columns array" do
             assert_nothing_raised do
               columns = %w(title author_name).freeze
-              Topic.import columns, [%w(foo, bar)], on_duplicate_key_update: { columns: columns }
+              Topic.import columns, [%w(foo, bar)], { on_duplicate_key_update: { columns: columns }.freeze }.freeze
             end
           end
 

data/test/support/shared_examples/on_duplicate_key_update.rb

@@ -213,6 +213,30 @@ def should_support_basic_on_duplicate_key_update
     end
 
     context "with :on_duplicate_key_update" do
+      describe 'using :all' do
+        let(:columns) { %w( id title author_name author_email_address parent_id ) }
+        let(:updated_values) { [[99, "Book - 2nd Edition", "Jane Doe", "janedoe@example.com", 57]] }
+
+        macro(:perform_import) do |*opts|
+          Topic.import columns, updated_values, opts.extract_options!.merge(on_duplicate_key_update: :all, validate: false)
+        end
+
+        setup do
+          values = [[99, "Book", "John Doe", "john@doe.com", 17, 3]]
+          Topic.import columns + ['replies_count'], values, validate: false
+        end
+
+        it 'updates all specified columns' do
+          perform_import
+          updated_topic = Topic.find(99)
+          assert_equal 'Book - 2nd Edition', updated_topic.title
+          assert_equal 'Jane Doe', updated_topic.author_name
+          assert_equal 'janedoe@example.com', updated_topic.author_email_address
+          assert_equal 57, updated_topic.parent_id
+          assert_equal 3, updated_topic.replies_count
+        end
+      end
+
       describe "argument safety" do
         it "should not modify the passed in :on_duplicate_key_update array" do
           assert_nothing_raised do

data/test/support/shared_examples/recursive_import.rb

@@ -11,7 +11,7 @@ def should_support_recursive_import
     let(:num_chapters) { 18 }
     let(:num_endnotes) { 24 }
 
-    let(:new_question_with_rule) { FactoryGirl.build :question, :with_rule }
+    let(:new_question_with_rule) { FactoryBot.build :question, :with_rule }
 
     it 'imports top level' do
       assert_difference "Topic.count", +num_topics do

data/test/support/sqlite3/import_examples.rb

@@ -1,6 +1,8 @@
 # encoding: UTF-8
 def should_support_sqlite3_import_functionality
-  should_support_on_duplicate_key_ignore
+  if ActiveRecord::Base.connection.supports_on_duplicate_key_update?
+    should_support_sqlite_upsert_functionality
+  end
 
   describe "#supports_imports?" do
     context "and SQLite is 3.7.11 or higher" do
@@ -49,18 +51,193 @@ def should_support_sqlite3_import_functionality
         assert_equal 2500, Topic.count, "Failed to insert all records. Make sure you have a supported version of SQLite3 (3.7.11 or higher) installed"
       end
     end
+  end
+end
+
+def should_support_sqlite_upsert_functionality
+  should_support_basic_on_duplicate_key_update
+  should_support_on_duplicate_key_ignore
+
+  describe "#import" do
+    extend ActiveSupport::TestCase::ImportAssertions
+
+    macro(:perform_import) { raise "supply your own #perform_import in a context below" }
+    macro(:updated_topic) { Topic.find(@topic.id) }
+
+    context "with :on_duplicate_key_ignore and validation checks turned off" do
+      let(:columns) { %w( id title author_name author_email_address parent_id ) }
+      let(:values) { [[99, "Book", "John Doe", "john@doe.com", 17]] }
+      let(:updated_values) { [[99, "Book - 2nd Edition", "Author Should Not Change", "johndoe@example.com", 57]] }
+
+      setup do
+        Topic.import columns, values, validate: false
+      end
+
+      it "should not update any records" do
+        result = Topic.import columns, updated_values, on_duplicate_key_ignore: true, validate: false
+        assert_equal [], result.ids
+      end
+    end
+
+    context "with :on_duplicate_key_update and validation checks turned off" do
+      asssertion_group(:should_support_on_duplicate_key_update) do
+        should_not_update_fields_not_mentioned
+        should_update_foreign_keys
+        should_not_update_created_at_on_timestamp_columns
+        should_update_updated_at_on_timestamp_columns
+      end
+
+      context "using a hash" do
+        context "with :columns a hash" do
+          let(:columns) { %w( id title author_name author_email_address parent_id ) }
+          let(:values) { [[99, "Book", "John Doe", "john@doe.com", 17]] }
+          let(:updated_values) { [[99, "Book - 2nd Edition", "Author Should Not Change", "johndoe@example.com", 57]] }
+
+          macro(:perform_import) do |*opts|
+            Topic.import columns, updated_values, opts.extract_options!.merge(on_duplicate_key_update: { conflict_target: :id, columns: update_columns }, validate: false)
+          end
+
+          setup do
+            Topic.import columns, values, validate: false
+            @topic = Topic.find 99
+          end
+
+          it "should not modify the passed in :on_duplicate_key_update columns array" do
+            assert_nothing_raised do
+              columns = %w(title author_name).freeze
+              Topic.import columns, [%w(foo, bar)], on_duplicate_key_update: { columns: columns }
+            end
+          end
+
+          context "using string hash map" do
+            let(:update_columns) { { "title" => "title", "author_email_address" => "author_email_address", "parent_id" => "parent_id" } }
+            should_support_on_duplicate_key_update
+            should_update_fields_mentioned
+          end
+
+          context "using string hash map, but specifying column mismatches" do
+            let(:update_columns) { { "title" => "author_email_address", "author_email_address" => "title", "parent_id" => "parent_id" } }
+            should_support_on_duplicate_key_update
+            should_update_fields_mentioned_with_hash_mappings
+          end
+
+          context "using symbol hash map" do
+            let(:update_columns) { { title: :title, author_email_address: :author_email_address, parent_id: :parent_id } }
+            should_support_on_duplicate_key_update
+            should_update_fields_mentioned
+          end
+
+          context "using symbol hash map, but specifying column mismatches" do
+            let(:update_columns) { { title: :author_email_address, author_email_address: :title, parent_id: :parent_id } }
+            should_support_on_duplicate_key_update
+            should_update_fields_mentioned_with_hash_mappings
+          end
+        end
+
+        context 'with :index_predicate' do
+          let(:columns) { %w( id device_id alarm_type status metadata ) }
+          let(:values) { [[99, 17, 1, 1, 'foo']] }
+          let(:updated_values) { [[99, 17, 1, 2, 'bar']] }
+
+          macro(:perform_import) do |*opts|
+            Alarm.import columns, updated_values, opts.extract_options!.merge(on_duplicate_key_update: { conflict_target: [:device_id, :alarm_type], index_predicate: 'status <> 0', columns: [:status] }, validate: false)
+          end
+
+          macro(:updated_alarm) { Alarm.find(@alarm.id) }
+
+          setup do
+            Alarm.import columns, values, validate: false
+            @alarm = Alarm.find 99
+          end
+
+          context 'supports on duplicate key update for partial indexes' do
+            it 'should not update created_at timestamp columns' do
+              Timecop.freeze Chronic.parse("5 minutes from now") do
+                perform_import
+                assert_in_delta @alarm.created_at.to_i, updated_alarm.created_at.to_i, 1
+              end
+            end
+
+            it 'should update updated_at timestamp columns' do
+              time = Chronic.parse("5 minutes from now")
+              Timecop.freeze time do
+                perform_import
+                assert_in_delta time.to_i, updated_alarm.updated_at.to_i, 1
+              end
+            end
+
+            it 'should not update fields not mentioned' do
+              perform_import
+              assert_equal 'foo', updated_alarm.metadata
+            end
+
+            it 'should update fields mentioned with hash mappings' do
+              perform_import
+              assert_equal 2, updated_alarm.status
+            end
+          end
+        end
+
+        context 'with :condition' do
+          let(:columns) { %w( id device_id alarm_type status metadata) }
+          let(:values) { [[99, 17, 1, 1, 'foo']] }
+          let(:updated_values) { [[99, 17, 1, 1, 'bar']] }
+
+          macro(:perform_import) do |*opts|
+            Alarm.import(
+              columns,
+              updated_values,
+              opts.extract_options!.merge(
+                on_duplicate_key_update: {
+                  conflict_target: [:id],
+                  condition: "alarms.metadata NOT LIKE '%foo%'",
+                  columns: [:metadata]
+                },
+                validate: false
+              )
+            )
+          end
+
+          macro(:updated_alarm) { Alarm.find(@alarm.id) }
+
+          setup do
+            Alarm.import columns, values, validate: false
+            @alarm = Alarm.find 99
+          end
+
+          it 'should not update fields not matched' do
+            perform_import
+            assert_equal 'foo', updated_alarm.metadata
+          end
+        end
+
+        context "with no :conflict_target" do
+          context "with no primary key" do
+            it "raises ArgumentError" do
+              error = assert_raises ArgumentError do
+                Rule.import Build(3, :rules), on_duplicate_key_update: [:condition_text], validate: false
+              end
+              assert_match(/Expected :conflict_target to be specified/, error.message)
+            end
+          end
+        end
+
+        context "with no :columns" do
+          let(:columns) { %w( id title author_name author_email_address ) }
+          let(:values) { [[100, "Book", "John Doe", "john@doe.com"]] }
+          let(:updated_values) { [[100, "Title Should Not Change", "Author Should Not Change", "john@nogo.com"]] }
 
-    context "with :on_duplicate_key_update" do
-      let(:topics) { Build(1, :topics) }
+          macro(:perform_import) do |*opts|
+            Topic.import columns, updated_values, opts.extract_options!.merge(on_duplicate_key_update: { conflict_target: :id }, validate: false)
+          end
 
-      it "should log a warning message" do
-        log = StringIO.new
-        logger = Logger.new(log)
-        logger.level = Logger::WARN
-        ActiveRecord::Base.connection.stubs(:logger).returns(logger)
+          setup do
+            Topic.import columns, values, validate: false
+            @topic = Topic.find 100
+          end
 
-        Topic.import topics, on_duplicate_key_update: true
-        assert_match(/Ignoring on_duplicate_key_update/, log.string)
+          should_update_updated_at_on_timestamp_columns
+        end
       end
     end
   end

data/test/test_helper.rb

@@ -58,7 +58,7 @@ ActiveSupport::Notifications.subscribe(/active_record.sql/) do |_, _, _, _, hsh|
   ActiveRecord::Base.logger.info hsh[:sql]
 end
 
-require "factory_girl"
+require "factory_bot"
 Dir[File.dirname(__FILE__) + "/support/**/*.rb"].each { |file| require file }
 
 # Load base/generic schema

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: activerecord-import
 version: !ruby/object:Gem::Version
-  version: 0.25.0
+  version: 0.26.0
 platform: ruby
 authors:
 - Zach Dennis
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-07-10 00:00:00.000000000 Z
+date: 2018-10-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -146,6 +146,7 @@ files:
 - test/schema/mysql2_schema.rb
 - test/schema/postgis_schema.rb
 - test/schema/postgresql_schema.rb
+- test/schema/sqlite3_schema.rb
 - test/schema/version.rb
 - test/sqlite3/import_test.rb
 - test/support/active_support/test_case_extensions.rb
@@ -183,7 +184,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.13
+rubygems_version: 2.7.7
 signing_key: 
 specification_version: 4
 summary: Bulk insert extension for ActiveRecord
@@ -236,6 +237,7 @@ test_files:
 - test/schema/mysql2_schema.rb
 - test/schema/postgis_schema.rb
 - test/schema/postgresql_schema.rb
+- test/schema/sqlite3_schema.rb
 - test/schema/version.rb
 - test/sqlite3/import_test.rb
 - test/support/active_support/test_case_extensions.rb