activerecord-import 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.markdown +2 -23
- data/README.textile +62 -0
- data/Rakefile +9 -4
- data/VERSION +1 -1
- data/lib/activerecord-import.rb +16 -0
- data/lib/activerecord-import/active_record/adapters/abstract_adapter.rb +4 -140
- data/lib/activerecord-import/active_record/adapters/mysql2_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/mysql_adapter.rb +3 -44
- data/lib/activerecord-import/active_record/adapters/postgresql_adapter.rb +4 -8
- data/lib/activerecord-import/active_record/adapters/sqlite3_adapter.rb +4 -8
- data/lib/activerecord-import/adapters/abstract_adapter.rb +145 -0
- data/lib/activerecord-import/adapters/mysql_adapter.rb +50 -0
- data/lib/activerecord-import/adapters/postgresql_adapter.rb +7 -0
- data/lib/activerecord-import/adapters/sqlite3_adapter.rb +7 -0
- data/lib/activerecord-import/base.rb +13 -2
- data/lib/activerecord-import/import.rb +2 -2
- data/lib/activerecord-import/mysql.rb +8 -2
- data/lib/activerecord-import/mysql2.rb +8 -0
- data/lib/activerecord-import/postgresql.rb +8 -2
- data/lib/activerecord-import/sqlite3.rb +8 -2
- data/test/adapters/mysql2.rb +1 -0
- data/test/mysql/import_test.rb +4 -116
- data/test/mysql2/import_test.rb +6 -0
- data/test/support/active_support/test_case_extensions.rb +67 -0
- data/test/support/mysql/import_examples.rb +117 -0
- data/test/test_helper.rb +7 -76
- metadata +26 -10
data/README.markdown
CHANGED
@@ -1,29 +1,8 @@
|
|
1
1
|
# activerecord-import
|
2
2
|
|
3
|
-
activerecord-import is a library for bulk inserting data using ActiveRecord.
|
3
|
+
activerecord-import is a library for bulk inserting data using ActiveRecord.
|
4
4
|
|
5
|
-
|
6
|
-
Book.create! :name => "book #{i}"
|
7
|
-
end
|
8
|
-
|
9
|
-
This may work fine if all you have is 10 records, but if you have hundreds, thousands, or millions of records it can turn into a nightmare. This is where activerecord-import comes into play. Here's the equivalent behaviour using the #import method:
|
10
|
-
|
11
|
-
books = []
|
12
|
-
10.times{ |i| books << Book.new(:name => "book #{i}") }
|
13
|
-
Book.import books
|
14
|
-
|
15
|
-
Pretty slick, eh?
|
16
|
-
|
17
|
-
Maybe, just maybe you're thinking, why do I have do instantiate ActiveRecord objects? Will that perform validations? What if I don't want validations? What if I want to take advantage of features like MySQL's on duplicate key update? Well, activerecord-import handles all of these cases and more!
|
18
|
-
|
19
|
-
For more documentation on the matter you can refer to two places:
|
20
|
-
|
21
|
-
1. activerecord-import github wiki: http://wiki.github.com/zdennis/activerecord-import/
|
22
|
-
1. the tests in the code base
|
23
|
-
|
24
|
-
# Upgrading from ar-extensions
|
25
|
-
|
26
|
-
This library replaces the ar-extensions library and is compatible with Rails 3. It provides the exact same API for importing data, but it does not include any additional ar-extensions functionality.
|
5
|
+
For more information on activerecord-import please see its wiki: http://wiki.github.com/zdennis/activerecord-import/
|
27
6
|
|
28
7
|
# License
|
29
8
|
|
data/README.textile
ADDED
@@ -0,0 +1,62 @@
|
|
1
|
+
h1. activerecord-import
|
2
|
+
|
3
|
+
activerecord-import is a library for bulk inserting data using ActiveRecord.
|
4
|
+
|
5
|
+
h2. Why activerecord-import?
|
6
|
+
|
7
|
+
Because plain-vanilla, out-of-the-box ActiveRecord doesn't provide support for inserting large amounts of data efficiently. With vanilla ActiveRecord you would have to perform individual save operations on each model:
|
8
|
+
|
9
|
+
<pre>
|
10
|
+
10.times do |i|
|
11
|
+
Book.create! :name => "book #{i}"
|
12
|
+
end
|
13
|
+
</pre>
|
14
|
+
|
15
|
+
This may work fine if all you have is 10 records, but if you have hundreds, thousands, or millions of records it can turn into a nightmare. This is where activerecord-import comes into play.
|
16
|
+
|
17
|
+
h2. An Introductory Example
|
18
|
+
|
19
|
+
Here's an example with equivalent behaviour using the @#import@ method:
|
20
|
+
|
21
|
+
<pre>
|
22
|
+
books = []
|
23
|
+
10.times do |i|
|
24
|
+
books << Book.new(:name => "book #{i}")
|
25
|
+
end
|
26
|
+
Book.import books
|
27
|
+
</pre>
|
28
|
+
|
29
|
+
This call to import does whatever is most efficient for the underlying database adapter. Pretty slick, eh?
|
30
|
+
|
31
|
+
h2. Features
|
32
|
+
|
33
|
+
Here's a list of some of the high-level features that activerecord-import provides:
|
34
|
+
|
35
|
+
* activerecord-import can work with raw columns and arrays of values (fastest)
|
36
|
+
* activerecord-import works with model objects (faster)
|
37
|
+
* activerecord-import can perform validations (fast)
|
38
|
+
* activerecord-import can perform on duplicate key updates (requires mysql)
|
39
|
+
|
40
|
+
|
41
|
+
h1. Upgrading from ar-extensions
|
42
|
+
|
43
|
+
activerecord-import replaces the ar-extensions library and is compatible with Rails 3. It provides the exact same API for importing data, but it does not include any additional ar-extensions functionality.
|
44
|
+
|
45
|
+
h1. License
|
46
|
+
|
47
|
+
This is licensed under the ruby license.
|
48
|
+
|
49
|
+
h1. Author
|
50
|
+
|
51
|
+
Zach Dennis (zach.dennis@gmail.com)
|
52
|
+
|
53
|
+
h1. Contributors
|
54
|
+
|
55
|
+
* Blythe Dunham
|
56
|
+
* Gabe da Silveira
|
57
|
+
* Henry Work
|
58
|
+
* James Herdman
|
59
|
+
* Marcus Crafter
|
60
|
+
* Thibaud Guillaume-Gentil
|
61
|
+
* Mark Van Holstyn
|
62
|
+
* Victor Costan
|
data/Rakefile
CHANGED
@@ -1,10 +1,11 @@
|
|
1
|
-
require
|
1
|
+
require "bundler"
|
2
|
+
Bundler.setup
|
3
|
+
|
2
4
|
require 'rake'
|
3
5
|
require 'rake/testtask'
|
4
6
|
|
5
7
|
begin
|
6
8
|
require 'jeweler'
|
7
|
-
require 'bundler'
|
8
9
|
Jeweler::Tasks.new do |gem|
|
9
10
|
gem.name = "activerecord-import"
|
10
11
|
gem.summary = %Q{Bulk-loading extension for ActiveRecord}
|
@@ -13,7 +14,11 @@ begin
|
|
13
14
|
gem.homepage = "http://github.com/zdennis/activerecord-import"
|
14
15
|
gem.authors = ["Zach Dennis"]
|
15
16
|
gem.files = FileList["VERSION", "Rakefile", "README*", "lib/**/*"]
|
16
|
-
|
17
|
+
|
18
|
+
bundler = Bundler.load
|
19
|
+
bundler.dependencies_for(:default).each do |dependency|
|
20
|
+
gem.add_dependency dependency.name, *dependency.requirements_list
|
21
|
+
end
|
17
22
|
|
18
23
|
# gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
|
19
24
|
end
|
@@ -31,7 +36,7 @@ namespace :display do
|
|
31
36
|
end
|
32
37
|
task :default => ["display:notice"]
|
33
38
|
|
34
|
-
ADAPTERS = %w(mysql postgresql sqlite3)
|
39
|
+
ADAPTERS = %w(mysql mysql2 postgresql sqlite3)
|
35
40
|
ADAPTERS.each do |adapter|
|
36
41
|
namespace :test do
|
37
42
|
desc "Runs #{adapter} database tests."
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.
|
1
|
+
0.2.0
|
@@ -0,0 +1,16 @@
|
|
1
|
+
class ActiveRecord::Base
|
2
|
+
class << self
|
3
|
+
def establish_connection_with_activerecord_import(*args)
|
4
|
+
establish_connection_without_activerecord_import(*args)
|
5
|
+
ActiveSupport.run_load_hooks(:active_record_connection_established, connection)
|
6
|
+
end
|
7
|
+
alias_method_chain :establish_connection, :activerecord_import
|
8
|
+
end
|
9
|
+
end
|
10
|
+
|
11
|
+
ActiveSupport.on_load(:active_record_connection_established) do |connection|
|
12
|
+
if !ActiveRecord.const_defined?(:Import) || !ActiveRecord::Import.respond_to?(:load_from_connection)
|
13
|
+
require File.join File.dirname(__FILE__), "activerecord-import/base"
|
14
|
+
end
|
15
|
+
ActiveRecord::Import.load_from_connection connection
|
16
|
+
end
|
@@ -1,146 +1,10 @@
|
|
1
|
+
require "activerecord-import/adapters/abstract_adapter"
|
2
|
+
|
1
3
|
module ActiveRecord # :nodoc:
|
2
4
|
module ConnectionAdapters # :nodoc:
|
3
5
|
class AbstractAdapter # :nodoc:
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
def next_value_for_sequence(sequence_name)
|
8
|
-
%{#{sequence_name}.nextval}
|
9
|
-
end
|
10
|
-
|
11
|
-
# +sql+ can be a single string or an array. If it is an array all
|
12
|
-
# elements that are in position >= 1 will be appended to the final SQL.
|
13
|
-
def insert_many( sql, values, *args ) # :nodoc:
|
14
|
-
# the number of inserts default
|
15
|
-
number_of_inserts = 0
|
16
|
-
|
17
|
-
base_sql,post_sql = if sql.is_a?( String )
|
18
|
-
[ sql, '' ]
|
19
|
-
elsif sql.is_a?( Array )
|
20
|
-
[ sql.shift, sql.join( ' ' ) ]
|
21
|
-
end
|
22
|
-
|
23
|
-
sql_size = QUERY_OVERHEAD + base_sql.size + post_sql.size
|
24
|
-
|
25
|
-
# the number of bytes the requested insert statement values will take up
|
26
|
-
values_in_bytes = self.class.sum_sizes( *values )
|
27
|
-
|
28
|
-
# the number of bytes (commas) it will take to comma separate our values
|
29
|
-
comma_separated_bytes = values.size-1
|
30
|
-
|
31
|
-
# the total number of bytes required if this statement is one statement
|
32
|
-
total_bytes = sql_size + values_in_bytes + comma_separated_bytes
|
33
|
-
|
34
|
-
max = max_allowed_packet
|
35
|
-
|
36
|
-
# if we can insert it all as one statement
|
37
|
-
if NO_MAX_PACKET == max or total_bytes < max
|
38
|
-
number_of_inserts += 1
|
39
|
-
sql2insert = base_sql + values.join( ',' ) + post_sql
|
40
|
-
insert( sql2insert, *args )
|
41
|
-
else
|
42
|
-
value_sets = self.class.get_insert_value_sets( values, sql_size, max )
|
43
|
-
value_sets.each do |values|
|
44
|
-
number_of_inserts += 1
|
45
|
-
sql2insert = base_sql + values.join( ',' ) + post_sql
|
46
|
-
insert( sql2insert, *args )
|
47
|
-
end
|
48
|
-
end
|
49
|
-
|
50
|
-
number_of_inserts
|
51
|
-
end
|
52
|
-
|
53
|
-
def pre_sql_statements(options)
|
54
|
-
sql = []
|
55
|
-
sql << options[:pre_sql] if options[:pre_sql]
|
56
|
-
sql << options[:command] if options[:command]
|
57
|
-
sql << "IGNORE" if options[:ignore]
|
58
|
-
|
59
|
-
#add keywords like IGNORE or DELAYED
|
60
|
-
if options[:keywords].is_a?(Array)
|
61
|
-
sql.concat(options[:keywords])
|
62
|
-
elsif options[:keywords]
|
63
|
-
sql << options[:keywords].to_s
|
64
|
-
end
|
65
|
-
|
66
|
-
sql
|
67
|
-
end
|
68
|
-
|
69
|
-
# Synchronizes the passed in ActiveRecord instances with the records in
|
70
|
-
# the database by calling +reload+ on each instance.
|
71
|
-
def after_import_synchronize( instances )
|
72
|
-
instances.each { |e| e.reload }
|
73
|
-
end
|
74
|
-
|
75
|
-
# Returns an array of post SQL statements given the passed in options.
|
76
|
-
def post_sql_statements( table_name, options ) # :nodoc:
|
77
|
-
post_sql_statements = []
|
78
|
-
if options[:on_duplicate_key_update]
|
79
|
-
post_sql_statements << sql_for_on_duplicate_key_update( table_name, options[:on_duplicate_key_update] )
|
80
|
-
end
|
81
|
-
|
82
|
-
#custom user post_sql
|
83
|
-
post_sql_statements << options[:post_sql] if options[:post_sql]
|
84
|
-
|
85
|
-
#with rollup
|
86
|
-
post_sql_statements << rollup_sql if options[:rollup]
|
87
|
-
|
88
|
-
post_sql_statements
|
89
|
-
end
|
90
|
-
|
91
|
-
|
92
|
-
# Generates the INSERT statement used in insert multiple value sets.
|
93
|
-
def multiple_value_sets_insert_sql(table_name, column_names, options) # :nodoc:
|
94
|
-
"INSERT #{options[:ignore] ? 'IGNORE ':''}INTO #{table_name} (#{column_names.join(',')}) VALUES "
|
95
|
-
end
|
96
|
-
|
97
|
-
# Returns SQL the VALUES for an INSERT statement given the passed in +columns+
|
98
|
-
# and +array_of_attributes+.
|
99
|
-
def values_sql_for_column_names_and_attributes( columns, array_of_attributes ) # :nodoc:
|
100
|
-
values = []
|
101
|
-
array_of_attributes.each do |arr|
|
102
|
-
my_values = []
|
103
|
-
arr.each_with_index do |val,j|
|
104
|
-
my_values << quote( val, columns[j] )
|
105
|
-
end
|
106
|
-
values << my_values
|
107
|
-
end
|
108
|
-
values_arr = values.map{ |arr| '(' + arr.join( ',' ) + ')' }
|
109
|
-
end
|
110
|
-
|
111
|
-
# Returns the sum of the sizes of the passed in objects. This should
|
112
|
-
# probably be moved outside this class, but to where?
|
113
|
-
def self.sum_sizes( *objects ) # :nodoc:
|
114
|
-
objects.inject( 0 ){|sum,o| sum += o.size }
|
115
|
-
end
|
116
|
-
|
117
|
-
# Returns the maximum number of bytes that the server will allow
|
118
|
-
# in a single packet
|
119
|
-
def max_allowed_packet
|
120
|
-
NO_MAX_PACKET
|
121
|
-
end
|
122
|
-
|
123
|
-
def self.get_insert_value_sets( values, sql_size, max_bytes ) # :nodoc:
|
124
|
-
value_sets = []
|
125
|
-
arr, current_arr_values_size, current_size = [], 0, 0
|
126
|
-
values.each_with_index do |val,i|
|
127
|
-
comma_bytes = arr.size
|
128
|
-
sql_size_thus_far = sql_size + current_size + val.size + comma_bytes
|
129
|
-
if NO_MAX_PACKET == max_bytes or sql_size_thus_far <= max_bytes
|
130
|
-
current_size += val.size
|
131
|
-
arr << val
|
132
|
-
else
|
133
|
-
value_sets << arr
|
134
|
-
arr = [ val ]
|
135
|
-
current_size = val.size
|
136
|
-
end
|
137
|
-
|
138
|
-
# if we're on the last iteration push whatever we have in arr to value_sets
|
139
|
-
value_sets << arr if i == (values.size-1)
|
140
|
-
end
|
141
|
-
[ *value_sets ]
|
142
|
-
end
|
143
|
-
|
6
|
+
extend ActiveRecord::Import::AbstractAdapter::ClassMethods
|
7
|
+
include ActiveRecord::Import::AbstractAdapter::InstanceMethods
|
144
8
|
end
|
145
9
|
end
|
146
10
|
end
|
@@ -1,47 +1,6 @@
|
|
1
1
|
require "active_record/connection_adapters/mysql_adapter"
|
2
|
+
require "activerecord-import/adapters/mysql_adapter"
|
2
3
|
|
3
4
|
class ActiveRecord::ConnectionAdapters::MysqlAdapter
|
4
|
-
include ActiveRecord::
|
5
|
-
|
6
|
-
|
7
|
-
# Returns a generated ON DUPLICATE KEY UPDATE statement given the passed
|
8
|
-
# in +args+.
|
9
|
-
def sql_for_on_duplicate_key_update( table_name, *args ) # :nodoc:
|
10
|
-
sql = ' ON DUPLICATE KEY UPDATE '
|
11
|
-
arg = args.first
|
12
|
-
if arg.is_a?( Array )
|
13
|
-
sql << sql_for_on_duplicate_key_update_as_array( table_name, arg )
|
14
|
-
elsif arg.is_a?( Hash )
|
15
|
-
sql << sql_for_on_duplicate_key_update_as_hash( table_name, arg )
|
16
|
-
elsif arg.is_a?( String )
|
17
|
-
sql << arg
|
18
|
-
else
|
19
|
-
raise ArgumentError.new( "Expected Array or Hash" )
|
20
|
-
end
|
21
|
-
sql
|
22
|
-
end
|
23
|
-
|
24
|
-
def sql_for_on_duplicate_key_update_as_array( table_name, arr ) # :nodoc:
|
25
|
-
results = arr.map do |column|
|
26
|
-
qc = quote_column_name( column )
|
27
|
-
"#{table_name}.#{qc}=VALUES(#{qc})"
|
28
|
-
end
|
29
|
-
results.join( ',' )
|
30
|
-
end
|
31
|
-
|
32
|
-
def sql_for_on_duplicate_key_update_as_hash( table_name, hsh ) # :nodoc:
|
33
|
-
sql = ' ON DUPLICATE KEY UPDATE '
|
34
|
-
results = hsh.map do |column1, column2|
|
35
|
-
qc1 = quote_column_name( column1 )
|
36
|
-
qc2 = quote_column_name( column2 )
|
37
|
-
"#{table_name}.#{qc1}=VALUES( #{qc2} )"
|
38
|
-
end
|
39
|
-
results.join( ',')
|
40
|
-
end
|
41
|
-
|
42
|
-
#return true if the statement is a duplicate key record error
|
43
|
-
def duplicate_key_update_error?(exception)# :nodoc:
|
44
|
-
exception.is_a?(ActiveRecord::StatementInvalid) && exception.to_s.include?('Duplicate entry')
|
45
|
-
end
|
46
|
-
|
47
|
-
end
|
5
|
+
include ActiveRecord::Import::MysqlAdapter::InstanceMethods
|
6
|
+
end
|
@@ -1,11 +1,7 @@
|
|
1
1
|
require "active_record/connection_adapters/postgresql_adapter"
|
2
|
+
require "activerecord-import/adapters/postgresql_adapter"
|
2
3
|
|
3
|
-
|
4
|
-
|
5
|
-
class PostgreSQLAdapter # :nodoc:
|
6
|
-
def next_value_for_sequence(sequence_name)
|
7
|
-
%{nextval('#{sequence_name}')}
|
8
|
-
end
|
9
|
-
end
|
10
|
-
end
|
4
|
+
class ActiveRecord::ConnectionAdapters::PostgreSQLAdapter
|
5
|
+
include ActiveRecord::Import::PostgreSQLAdapter::InstanceMethods
|
11
6
|
end
|
7
|
+
|
@@ -1,11 +1,7 @@
|
|
1
1
|
require "active_record/connection_adapters/sqlite3_adapter"
|
2
|
+
require "activerecord-import/adapters/sqlite3_adapter"
|
2
3
|
|
3
|
-
|
4
|
-
|
5
|
-
class Sqlite3Adapter # :nodoc:
|
6
|
-
def next_value_for_sequence(sequence_name)
|
7
|
-
%{nextval('#{sequence_name}')}
|
8
|
-
end
|
9
|
-
end
|
10
|
-
end
|
4
|
+
class ActiveRecord::ConnectionAdapters::Sqlite3Adapter
|
5
|
+
include ActiveRecord::Import::Sqlite3Adapter::InstanceMethods
|
11
6
|
end
|
7
|
+
|
@@ -0,0 +1,145 @@
|
|
1
|
+
module ActiveRecord::Import::AbstractAdapter
|
2
|
+
NO_MAX_PACKET = 0
|
3
|
+
QUERY_OVERHEAD = 8 #This was shown to be true for MySQL, but it's not clear where the overhead is from.
|
4
|
+
|
5
|
+
module ClassMethods
|
6
|
+
# Returns the sum of the sizes of the passed in objects. This should
|
7
|
+
# probably be moved outside this class, but to where?
|
8
|
+
def sum_sizes( *objects ) # :nodoc:
|
9
|
+
objects.inject( 0 ){|sum,o| sum += o.size }
|
10
|
+
end
|
11
|
+
|
12
|
+
def get_insert_value_sets( values, sql_size, max_bytes ) # :nodoc:
|
13
|
+
value_sets = []
|
14
|
+
arr, current_arr_values_size, current_size = [], 0, 0
|
15
|
+
values.each_with_index do |val,i|
|
16
|
+
comma_bytes = arr.size
|
17
|
+
sql_size_thus_far = sql_size + current_size + val.size + comma_bytes
|
18
|
+
if NO_MAX_PACKET == max_bytes or sql_size_thus_far <= max_bytes
|
19
|
+
current_size += val.size
|
20
|
+
arr << val
|
21
|
+
else
|
22
|
+
value_sets << arr
|
23
|
+
arr = [ val ]
|
24
|
+
current_size = val.size
|
25
|
+
end
|
26
|
+
|
27
|
+
# if we're on the last iteration push whatever we have in arr to value_sets
|
28
|
+
value_sets << arr if i == (values.size-1)
|
29
|
+
end
|
30
|
+
[ *value_sets ]
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
module InstanceMethods
|
35
|
+
def next_value_for_sequence(sequence_name)
|
36
|
+
%{#{sequence_name}.nextval}
|
37
|
+
end
|
38
|
+
|
39
|
+
# +sql+ can be a single string or an array. If it is an array all
|
40
|
+
# elements that are in position >= 1 will be appended to the final SQL.
|
41
|
+
def insert_many( sql, values, *args ) # :nodoc:
|
42
|
+
# the number of inserts default
|
43
|
+
number_of_inserts = 0
|
44
|
+
|
45
|
+
base_sql,post_sql = if sql.is_a?( String )
|
46
|
+
[ sql, '' ]
|
47
|
+
elsif sql.is_a?( Array )
|
48
|
+
[ sql.shift, sql.join( ' ' ) ]
|
49
|
+
end
|
50
|
+
|
51
|
+
sql_size = QUERY_OVERHEAD + base_sql.size + post_sql.size
|
52
|
+
|
53
|
+
# the number of bytes the requested insert statement values will take up
|
54
|
+
values_in_bytes = self.class.sum_sizes( *values )
|
55
|
+
|
56
|
+
# the number of bytes (commas) it will take to comma separate our values
|
57
|
+
comma_separated_bytes = values.size-1
|
58
|
+
|
59
|
+
# the total number of bytes required if this statement is one statement
|
60
|
+
total_bytes = sql_size + values_in_bytes + comma_separated_bytes
|
61
|
+
|
62
|
+
max = max_allowed_packet
|
63
|
+
|
64
|
+
# if we can insert it all as one statement
|
65
|
+
if NO_MAX_PACKET == max or total_bytes < max
|
66
|
+
number_of_inserts += 1
|
67
|
+
sql2insert = base_sql + values.join( ',' ) + post_sql
|
68
|
+
insert( sql2insert, *args )
|
69
|
+
else
|
70
|
+
value_sets = self.class.get_insert_value_sets( values, sql_size, max )
|
71
|
+
value_sets.each do |values|
|
72
|
+
number_of_inserts += 1
|
73
|
+
sql2insert = base_sql + values.join( ',' ) + post_sql
|
74
|
+
insert( sql2insert, *args )
|
75
|
+
end
|
76
|
+
end
|
77
|
+
|
78
|
+
number_of_inserts
|
79
|
+
end
|
80
|
+
|
81
|
+
def pre_sql_statements(options)
|
82
|
+
sql = []
|
83
|
+
sql << options[:pre_sql] if options[:pre_sql]
|
84
|
+
sql << options[:command] if options[:command]
|
85
|
+
sql << "IGNORE" if options[:ignore]
|
86
|
+
|
87
|
+
#add keywords like IGNORE or DELAYED
|
88
|
+
if options[:keywords].is_a?(Array)
|
89
|
+
sql.concat(options[:keywords])
|
90
|
+
elsif options[:keywords]
|
91
|
+
sql << options[:keywords].to_s
|
92
|
+
end
|
93
|
+
|
94
|
+
sql
|
95
|
+
end
|
96
|
+
|
97
|
+
# Synchronizes the passed in ActiveRecord instances with the records in
|
98
|
+
# the database by calling +reload+ on each instance.
|
99
|
+
def after_import_synchronize( instances )
|
100
|
+
instances.each { |e| e.reload }
|
101
|
+
end
|
102
|
+
|
103
|
+
# Returns an array of post SQL statements given the passed in options.
|
104
|
+
def post_sql_statements( table_name, options ) # :nodoc:
|
105
|
+
post_sql_statements = []
|
106
|
+
if options[:on_duplicate_key_update]
|
107
|
+
post_sql_statements << sql_for_on_duplicate_key_update( table_name, options[:on_duplicate_key_update] )
|
108
|
+
end
|
109
|
+
|
110
|
+
#custom user post_sql
|
111
|
+
post_sql_statements << options[:post_sql] if options[:post_sql]
|
112
|
+
|
113
|
+
#with rollup
|
114
|
+
post_sql_statements << rollup_sql if options[:rollup]
|
115
|
+
|
116
|
+
post_sql_statements
|
117
|
+
end
|
118
|
+
|
119
|
+
|
120
|
+
# Generates the INSERT statement used in insert multiple value sets.
|
121
|
+
def multiple_value_sets_insert_sql(table_name, column_names, options) # :nodoc:
|
122
|
+
"INSERT #{options[:ignore] ? 'IGNORE ':''}INTO #{table_name} (#{column_names.join(',')}) VALUES "
|
123
|
+
end
|
124
|
+
|
125
|
+
# Returns SQL the VALUES for an INSERT statement given the passed in +columns+
|
126
|
+
# and +array_of_attributes+.
|
127
|
+
def values_sql_for_column_names_and_attributes( columns, array_of_attributes ) # :nodoc:
|
128
|
+
values = []
|
129
|
+
array_of_attributes.each do |arr|
|
130
|
+
my_values = []
|
131
|
+
arr.each_with_index do |val,j|
|
132
|
+
my_values << quote( val, columns[j] )
|
133
|
+
end
|
134
|
+
values << my_values
|
135
|
+
end
|
136
|
+
values_arr = values.map{ |arr| '(' + arr.join( ',' ) + ')' }
|
137
|
+
end
|
138
|
+
|
139
|
+
# Returns the maximum number of bytes that the server will allow
|
140
|
+
# in a single packet
|
141
|
+
def max_allowed_packet
|
142
|
+
NO_MAX_PACKET
|
143
|
+
end
|
144
|
+
end
|
145
|
+
end
|
@@ -0,0 +1,50 @@
|
|
1
|
+
module ActiveRecord::Import::MysqlAdapter
|
2
|
+
module InstanceMethods
|
3
|
+
def self.included(klass)
|
4
|
+
klass.instance_eval do
|
5
|
+
include ActiveRecord::Import::ImportSupport
|
6
|
+
include ActiveRecord::Import::OnDuplicateKeyUpdateSupport
|
7
|
+
end
|
8
|
+
end
|
9
|
+
|
10
|
+
# Returns a generated ON DUPLICATE KEY UPDATE statement given the passed
|
11
|
+
# in +args+.
|
12
|
+
def sql_for_on_duplicate_key_update( table_name, *args ) # :nodoc:
|
13
|
+
sql = ' ON DUPLICATE KEY UPDATE '
|
14
|
+
arg = args.first
|
15
|
+
if arg.is_a?( Array )
|
16
|
+
sql << sql_for_on_duplicate_key_update_as_array( table_name, arg )
|
17
|
+
elsif arg.is_a?( Hash )
|
18
|
+
sql << sql_for_on_duplicate_key_update_as_hash( table_name, arg )
|
19
|
+
elsif arg.is_a?( String )
|
20
|
+
sql << arg
|
21
|
+
else
|
22
|
+
raise ArgumentError.new( "Expected Array or Hash" )
|
23
|
+
end
|
24
|
+
sql
|
25
|
+
end
|
26
|
+
|
27
|
+
def sql_for_on_duplicate_key_update_as_array( table_name, arr ) # :nodoc:
|
28
|
+
results = arr.map do |column|
|
29
|
+
qc = quote_column_name( column )
|
30
|
+
"#{table_name}.#{qc}=VALUES(#{qc})"
|
31
|
+
end
|
32
|
+
results.join( ',' )
|
33
|
+
end
|
34
|
+
|
35
|
+
def sql_for_on_duplicate_key_update_as_hash( table_name, hsh ) # :nodoc:
|
36
|
+
sql = ' ON DUPLICATE KEY UPDATE '
|
37
|
+
results = hsh.map do |column1, column2|
|
38
|
+
qc1 = quote_column_name( column1 )
|
39
|
+
qc2 = quote_column_name( column2 )
|
40
|
+
"#{table_name}.#{qc1}=VALUES( #{qc2} )"
|
41
|
+
end
|
42
|
+
results.join( ',')
|
43
|
+
end
|
44
|
+
|
45
|
+
#return true if the statement is a duplicate key record error
|
46
|
+
def duplicate_key_update_error?(exception)# :nodoc:
|
47
|
+
exception.is_a?(ActiveRecord::StatementInvalid) && exception.to_s.include?('Duplicate entry')
|
48
|
+
end
|
49
|
+
end
|
50
|
+
end
|
@@ -2,15 +2,26 @@ require "pathname"
|
|
2
2
|
require "active_record"
|
3
3
|
require "active_record/version"
|
4
4
|
|
5
|
-
module ActiveRecord::
|
5
|
+
module ActiveRecord::Import
|
6
6
|
AdapterPath = File.join File.expand_path(File.dirname(__FILE__)), "/active_record/adapters"
|
7
|
-
|
7
|
+
|
8
|
+
# Loads the import functionality for a specific database adapter
|
8
9
|
def self.require_adapter(adapter)
|
9
10
|
require File.join(AdapterPath,"/abstract_adapter")
|
10
11
|
require File.join(AdapterPath,"/#{adapter}_adapter")
|
11
12
|
end
|
13
|
+
|
14
|
+
# Loads the import functionality for the passed in ActiveRecord connection
|
15
|
+
def self.load_from_connection(connection)
|
16
|
+
import_adapter = "ActiveRecord::Import::#{connection.class.name.demodulize}::InstanceMethods"
|
17
|
+
unless connection.class.ancestors.map(&:name).include?(import_adapter)
|
18
|
+
config = connection.instance_variable_get :@config
|
19
|
+
require_adapter config[:adapter]
|
20
|
+
end
|
21
|
+
end
|
12
22
|
end
|
13
23
|
|
24
|
+
|
14
25
|
this_dir = Pathname.new File.dirname(__FILE__)
|
15
26
|
require this_dir.join("import")
|
16
27
|
require this_dir.join("active_record/adapters/abstract_adapter")
|
@@ -1,8 +1,8 @@
|
|
1
1
|
require "ostruct"
|
2
2
|
|
3
|
-
module ActiveRecord::
|
3
|
+
module ActiveRecord::Import::ConnectionAdapters ; end
|
4
4
|
|
5
|
-
module ActiveRecord::
|
5
|
+
module ActiveRecord::Import #:nodoc:
|
6
6
|
module ImportSupport #:nodoc:
|
7
7
|
def supports_import? #:nodoc:
|
8
8
|
true
|
@@ -1,2 +1,8 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
warn <<-MSG
|
2
|
+
[DEPRECATION] loading activerecord-import via 'require "activerecord-import/<adapter-name>"'
|
3
|
+
is deprecated. Update to autorequire using 'require "activerecord-import"'. See
|
4
|
+
http://github.com/zdennis/activerecord-import/wiki/Requiring for more information
|
5
|
+
MSG
|
6
|
+
|
7
|
+
require File.expand_path(File.join(File.dirname(__FILE__), "/../activerecord-import"))
|
8
|
+
|
@@ -0,0 +1,8 @@
|
|
1
|
+
warn <<-MSG
|
2
|
+
[DEPRECATION] loading activerecord-import via 'require "activerecord-import/<adapter-name>"'
|
3
|
+
is deprecated. Update to autorequire using 'require "activerecord-import"'. See
|
4
|
+
http://github.com/zdennis/activerecord-import/wiki/Requiring for more information
|
5
|
+
MSG
|
6
|
+
|
7
|
+
require File.expand_path(File.join(File.dirname(__FILE__), "/../activerecord-import"))
|
8
|
+
|
@@ -1,2 +1,8 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
warn <<-MSG
|
2
|
+
[DEPRECATION] loading activerecord-import via 'require "activerecord-import/<adapter-name>"'
|
3
|
+
is deprecated. Update to autorequire using 'require "activerecord-import"'. See
|
4
|
+
http://github.com/zdennis/activerecord-import/wiki/Requiring for more information
|
5
|
+
MSG
|
6
|
+
|
7
|
+
require File.expand_path(File.join(File.dirname(__FILE__), "/../activerecord-import"))
|
8
|
+
|
@@ -1,2 +1,8 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
warn <<-MSG
|
2
|
+
[DEPRECATION] loading activerecord-import via 'require "activerecord-import/<adapter-name>"'
|
3
|
+
is deprecated. Update to autorequire using 'require "activerecord-import"'. See
|
4
|
+
http://github.com/zdennis/activerecord-import/wiki/Requiring for more information
|
5
|
+
MSG
|
6
|
+
|
7
|
+
require File.expand_path(File.join(File.dirname(__FILE__), "/../activerecord-import"))
|
8
|
+
|
@@ -0,0 +1 @@
|
|
1
|
+
ENV["ARE_DB"] = "mysql2"
|
data/test/mysql/import_test.rb
CHANGED
@@ -1,118 +1,6 @@
|
|
1
1
|
require File.expand_path(File.dirname(__FILE__) + '/../test_helper')
|
2
|
-
require "activerecord-import/mysql"
|
3
2
|
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
should_not_update_fields_not_mentioned
|
9
|
-
should_update_foreign_keys
|
10
|
-
should_not_update_created_at_on_timestamp_columns
|
11
|
-
should_update_updated_at_on_timestamp_columns
|
12
|
-
end
|
13
|
-
|
14
|
-
macro(:perform_import){ raise "supply your own #perform_import in a context below" }
|
15
|
-
macro(:updated_topic){ Topic.find(@topic) }
|
16
|
-
|
17
|
-
context "given columns and values with :validation checks turned off" do
|
18
|
-
let(:columns){ %w( id title author_name author_email_address parent_id ) }
|
19
|
-
let(:values){ [ [ 99, "Book", "John Doe", "john@doe.com", 17 ] ] }
|
20
|
-
let(:updated_values){ [ [ 99, "Book - 2nd Edition", "Author Should Not Change", "johndoe@example.com", 57 ] ] }
|
21
|
-
|
22
|
-
macro(:perform_import) do |*opts|
|
23
|
-
Topic.import columns, updated_values, opts.extract_options!.merge(:on_duplicate_key_update => update_columns, :validate => false)
|
24
|
-
end
|
25
|
-
|
26
|
-
setup do
|
27
|
-
Topic.import columns, values, :validate => false
|
28
|
-
@topic = Topic.find 99
|
29
|
-
end
|
30
|
-
|
31
|
-
context "using string column names" do
|
32
|
-
let(:update_columns){ [ "title", "author_email_address", "parent_id" ] }
|
33
|
-
should_support_on_duplicate_key_update
|
34
|
-
should_update_fields_mentioned
|
35
|
-
end
|
36
|
-
|
37
|
-
context "using symbol column names" do
|
38
|
-
let(:update_columns){ [ :title, :author_email_address, :parent_id ] }
|
39
|
-
should_support_on_duplicate_key_update
|
40
|
-
should_update_fields_mentioned
|
41
|
-
end
|
42
|
-
|
43
|
-
context "using string hash map" do
|
44
|
-
let(:update_columns){ { "title" => "title", "author_email_address" => "author_email_address", "parent_id" => "parent_id" } }
|
45
|
-
should_support_on_duplicate_key_update
|
46
|
-
should_update_fields_mentioned
|
47
|
-
end
|
48
|
-
|
49
|
-
context "using string hash map, but specifying column mismatches" do
|
50
|
-
let(:update_columns){ { "title" => "author_email_address", "author_email_address" => "title", "parent_id" => "parent_id" } }
|
51
|
-
should_support_on_duplicate_key_update
|
52
|
-
should_update_fields_mentioned_with_hash_mappings
|
53
|
-
end
|
54
|
-
|
55
|
-
context "using symbol hash map" do
|
56
|
-
let(:update_columns){ { :title => :title, :author_email_address => :author_email_address, :parent_id => :parent_id } }
|
57
|
-
should_support_on_duplicate_key_update
|
58
|
-
should_update_fields_mentioned
|
59
|
-
end
|
60
|
-
|
61
|
-
context "using symbol hash map, but specifying column mismatches" do
|
62
|
-
let(:update_columns){ { :title => :author_email_address, :author_email_address => :title, :parent_id => :parent_id } }
|
63
|
-
should_support_on_duplicate_key_update
|
64
|
-
should_update_fields_mentioned_with_hash_mappings
|
65
|
-
end
|
66
|
-
end
|
67
|
-
|
68
|
-
context "given array of model instances with :validation checks turned off" do
|
69
|
-
macro(:perform_import) do |*opts|
|
70
|
-
@topic.title = "Book - 2nd Edition"
|
71
|
-
@topic.author_name = "Author Should Not Change"
|
72
|
-
@topic.author_email_address = "johndoe@example.com"
|
73
|
-
@topic.parent_id = 57
|
74
|
-
Topic.import [@topic], opts.extract_options!.merge(:on_duplicate_key_update => update_columns, :validate => false)
|
75
|
-
end
|
76
|
-
|
77
|
-
setup do
|
78
|
-
@topic = Generate(:topic, :id => 99, :author_name => "John Doe", :parent_id => 17)
|
79
|
-
end
|
80
|
-
|
81
|
-
context "using string column names" do
|
82
|
-
let(:update_columns){ [ "title", "author_email_address", "parent_id" ] }
|
83
|
-
should_support_on_duplicate_key_update
|
84
|
-
should_update_fields_mentioned
|
85
|
-
end
|
86
|
-
|
87
|
-
context "using symbol column names" do
|
88
|
-
let(:update_columns){ [ :title, :author_email_address, :parent_id ] }
|
89
|
-
should_support_on_duplicate_key_update
|
90
|
-
should_update_fields_mentioned
|
91
|
-
end
|
92
|
-
|
93
|
-
context "using string hash map" do
|
94
|
-
let(:update_columns){ { "title" => "title", "author_email_address" => "author_email_address", "parent_id" => "parent_id" } }
|
95
|
-
should_support_on_duplicate_key_update
|
96
|
-
should_update_fields_mentioned
|
97
|
-
end
|
98
|
-
|
99
|
-
context "using string hash map, but specifying column mismatches" do
|
100
|
-
let(:update_columns){ { "title" => "author_email_address", "author_email_address" => "title", "parent_id" => "parent_id" } }
|
101
|
-
should_support_on_duplicate_key_update
|
102
|
-
should_update_fields_mentioned_with_hash_mappings
|
103
|
-
end
|
104
|
-
|
105
|
-
context "using symbol hash map" do
|
106
|
-
let(:update_columns){ { :title => :title, :author_email_address => :author_email_address, :parent_id => :parent_id } }
|
107
|
-
should_support_on_duplicate_key_update
|
108
|
-
should_update_fields_mentioned
|
109
|
-
end
|
110
|
-
|
111
|
-
context "using symbol hash map, but specifying column mismatches" do
|
112
|
-
let(:update_columns){ { :title => :author_email_address, :author_email_address => :title, :parent_id => :parent_id } }
|
113
|
-
should_support_on_duplicate_key_update
|
114
|
-
should_update_fields_mentioned_with_hash_mappings
|
115
|
-
end
|
116
|
-
end
|
117
|
-
|
118
|
-
end
|
3
|
+
require "test/support/mysql/assertions"
|
4
|
+
require "test/support/mysql/import_examples"
|
5
|
+
|
6
|
+
should_support_mysql_import_functionality
|
@@ -0,0 +1,67 @@
|
|
1
|
+
class ActiveSupport::TestCase
|
2
|
+
include ActiveRecord::TestFixtures
|
3
|
+
self.use_transactional_fixtures = true
|
4
|
+
|
5
|
+
class << self
|
6
|
+
def assertion(name, &block)
|
7
|
+
mc = class << self ; self ; end
|
8
|
+
mc.class_eval do
|
9
|
+
define_method(name) do
|
10
|
+
it(name, &block)
|
11
|
+
end
|
12
|
+
end
|
13
|
+
end
|
14
|
+
|
15
|
+
def asssertion_group(name, &block)
|
16
|
+
mc = class << self ; self ; end
|
17
|
+
mc.class_eval do
|
18
|
+
define_method(name, &block)
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
def macro(name, &block)
|
23
|
+
class_eval do
|
24
|
+
define_method(name, &block)
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
def describe(description, toplevel=nil, &blk)
|
29
|
+
text = toplevel ? description : "#{name} #{description}"
|
30
|
+
klass = Class.new(self)
|
31
|
+
|
32
|
+
klass.class_eval <<-RUBY_EVAL
|
33
|
+
def self.name
|
34
|
+
"#{text}"
|
35
|
+
end
|
36
|
+
RUBY_EVAL
|
37
|
+
|
38
|
+
# do not inherit test methods from the superclass
|
39
|
+
klass.class_eval do
|
40
|
+
instance_methods.grep(/^test.+/) do |method|
|
41
|
+
undef_method method
|
42
|
+
end
|
43
|
+
end
|
44
|
+
|
45
|
+
klass.instance_eval &blk
|
46
|
+
end
|
47
|
+
alias_method :context, :describe
|
48
|
+
|
49
|
+
def let(name, &blk)
|
50
|
+
values = {}
|
51
|
+
define_method(name) do
|
52
|
+
return values[name] if values.has_key?(name)
|
53
|
+
values[name] = instance_eval(&blk)
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
def it(description, &blk)
|
58
|
+
define_method("test: #{name} #{description}", &blk)
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
end
|
63
|
+
|
64
|
+
def describe(description, &blk)
|
65
|
+
ActiveSupport::TestCase.describe(description, true, &blk)
|
66
|
+
end
|
67
|
+
|
@@ -0,0 +1,117 @@
|
|
1
|
+
def should_support_mysql_import_functionality
|
2
|
+
describe "#import with :on_duplicate_key_update option (mysql specific functionality)" do
|
3
|
+
extend ActiveSupport::TestCase::MySQLAssertions
|
4
|
+
|
5
|
+
asssertion_group(:should_support_on_duplicate_key_update) do
|
6
|
+
should_not_update_fields_not_mentioned
|
7
|
+
should_update_foreign_keys
|
8
|
+
should_not_update_created_at_on_timestamp_columns
|
9
|
+
should_update_updated_at_on_timestamp_columns
|
10
|
+
end
|
11
|
+
|
12
|
+
macro(:perform_import){ raise "supply your own #perform_import in a context below" }
|
13
|
+
macro(:updated_topic){ Topic.find(@topic) }
|
14
|
+
|
15
|
+
context "given columns and values with :validation checks turned off" do
|
16
|
+
let(:columns){ %w( id title author_name author_email_address parent_id ) }
|
17
|
+
let(:values){ [ [ 99, "Book", "John Doe", "john@doe.com", 17 ] ] }
|
18
|
+
let(:updated_values){ [ [ 99, "Book - 2nd Edition", "Author Should Not Change", "johndoe@example.com", 57 ] ] }
|
19
|
+
|
20
|
+
macro(:perform_import) do |*opts|
|
21
|
+
Topic.import columns, updated_values, opts.extract_options!.merge(:on_duplicate_key_update => update_columns, :validate => false)
|
22
|
+
end
|
23
|
+
|
24
|
+
setup do
|
25
|
+
Topic.import columns, values, :validate => false
|
26
|
+
@topic = Topic.find 99
|
27
|
+
end
|
28
|
+
|
29
|
+
context "using string column names" do
|
30
|
+
let(:update_columns){ [ "title", "author_email_address", "parent_id" ] }
|
31
|
+
should_support_on_duplicate_key_update
|
32
|
+
should_update_fields_mentioned
|
33
|
+
end
|
34
|
+
|
35
|
+
context "using symbol column names" do
|
36
|
+
let(:update_columns){ [ :title, :author_email_address, :parent_id ] }
|
37
|
+
should_support_on_duplicate_key_update
|
38
|
+
should_update_fields_mentioned
|
39
|
+
end
|
40
|
+
|
41
|
+
context "using string hash map" do
|
42
|
+
let(:update_columns){ { "title" => "title", "author_email_address" => "author_email_address", "parent_id" => "parent_id" } }
|
43
|
+
should_support_on_duplicate_key_update
|
44
|
+
should_update_fields_mentioned
|
45
|
+
end
|
46
|
+
|
47
|
+
context "using string hash map, but specifying column mismatches" do
|
48
|
+
let(:update_columns){ { "title" => "author_email_address", "author_email_address" => "title", "parent_id" => "parent_id" } }
|
49
|
+
should_support_on_duplicate_key_update
|
50
|
+
should_update_fields_mentioned_with_hash_mappings
|
51
|
+
end
|
52
|
+
|
53
|
+
context "using symbol hash map" do
|
54
|
+
let(:update_columns){ { :title => :title, :author_email_address => :author_email_address, :parent_id => :parent_id } }
|
55
|
+
should_support_on_duplicate_key_update
|
56
|
+
should_update_fields_mentioned
|
57
|
+
end
|
58
|
+
|
59
|
+
context "using symbol hash map, but specifying column mismatches" do
|
60
|
+
let(:update_columns){ { :title => :author_email_address, :author_email_address => :title, :parent_id => :parent_id } }
|
61
|
+
should_support_on_duplicate_key_update
|
62
|
+
should_update_fields_mentioned_with_hash_mappings
|
63
|
+
end
|
64
|
+
end
|
65
|
+
|
66
|
+
context "given array of model instances with :validation checks turned off" do
|
67
|
+
macro(:perform_import) do |*opts|
|
68
|
+
@topic.title = "Book - 2nd Edition"
|
69
|
+
@topic.author_name = "Author Should Not Change"
|
70
|
+
@topic.author_email_address = "johndoe@example.com"
|
71
|
+
@topic.parent_id = 57
|
72
|
+
Topic.import [@topic], opts.extract_options!.merge(:on_duplicate_key_update => update_columns, :validate => false)
|
73
|
+
end
|
74
|
+
|
75
|
+
setup do
|
76
|
+
@topic = Generate(:topic, :id => 99, :author_name => "John Doe", :parent_id => 17)
|
77
|
+
end
|
78
|
+
|
79
|
+
context "using string column names" do
|
80
|
+
let(:update_columns){ [ "title", "author_email_address", "parent_id" ] }
|
81
|
+
should_support_on_duplicate_key_update
|
82
|
+
should_update_fields_mentioned
|
83
|
+
end
|
84
|
+
|
85
|
+
context "using symbol column names" do
|
86
|
+
let(:update_columns){ [ :title, :author_email_address, :parent_id ] }
|
87
|
+
should_support_on_duplicate_key_update
|
88
|
+
should_update_fields_mentioned
|
89
|
+
end
|
90
|
+
|
91
|
+
context "using string hash map" do
|
92
|
+
let(:update_columns){ { "title" => "title", "author_email_address" => "author_email_address", "parent_id" => "parent_id" } }
|
93
|
+
should_support_on_duplicate_key_update
|
94
|
+
should_update_fields_mentioned
|
95
|
+
end
|
96
|
+
|
97
|
+
context "using string hash map, but specifying column mismatches" do
|
98
|
+
let(:update_columns){ { "title" => "author_email_address", "author_email_address" => "title", "parent_id" => "parent_id" } }
|
99
|
+
should_support_on_duplicate_key_update
|
100
|
+
should_update_fields_mentioned_with_hash_mappings
|
101
|
+
end
|
102
|
+
|
103
|
+
context "using symbol hash map" do
|
104
|
+
let(:update_columns){ { :title => :title, :author_email_address => :author_email_address, :parent_id => :parent_id } }
|
105
|
+
should_support_on_duplicate_key_update
|
106
|
+
should_update_fields_mentioned
|
107
|
+
end
|
108
|
+
|
109
|
+
context "using symbol hash map, but specifying column mismatches" do
|
110
|
+
let(:update_columns){ { :title => :author_email_address, :author_email_address => :title, :parent_id => :parent_id } }
|
111
|
+
should_support_on_duplicate_key_update
|
112
|
+
should_update_fields_mentioned_with_hash_mappings
|
113
|
+
end
|
114
|
+
end
|
115
|
+
|
116
|
+
end
|
117
|
+
end
|
data/test/test_helper.rb
CHANGED
@@ -11,98 +11,29 @@ ENV["RAILS_ENV"] = "test"
|
|
11
11
|
require "bundler"
|
12
12
|
Bundler.setup
|
13
13
|
|
14
|
+
require "logger"
|
14
15
|
require "rails"
|
15
16
|
require "rails/test_help"
|
17
|
+
require "active_record"
|
16
18
|
require "active_record/fixtures"
|
17
19
|
require "active_support/test_case"
|
18
20
|
|
19
21
|
require "delorean"
|
20
|
-
|
21
|
-
require "active_record"
|
22
|
-
require "logger"
|
23
|
-
|
24
22
|
require "ruby-debug"
|
25
23
|
|
26
|
-
# load test helpers
|
27
24
|
class MyApplication < Rails::Application ; end
|
28
25
|
adapter = ENV["ARE_DB"] || "sqlite3"
|
29
26
|
|
30
|
-
|
31
|
-
require "activerecord-import/#{adapter}"
|
32
|
-
|
33
|
-
class ActiveSupport::TestCase
|
34
|
-
include ActiveRecord::TestFixtures
|
35
|
-
self.use_transactional_fixtures = true
|
36
|
-
|
37
|
-
class << self
|
38
|
-
def assertion(name, &block)
|
39
|
-
mc = class << self ; self ; end
|
40
|
-
mc.class_eval do
|
41
|
-
define_method(name) do
|
42
|
-
it(name, &block)
|
43
|
-
end
|
44
|
-
end
|
45
|
-
end
|
46
|
-
|
47
|
-
def asssertion_group(name, &block)
|
48
|
-
mc = class << self ; self ; end
|
49
|
-
mc.class_eval do
|
50
|
-
define_method(name, &block)
|
51
|
-
end
|
52
|
-
end
|
53
|
-
|
54
|
-
def macro(name, &block)
|
55
|
-
class_eval do
|
56
|
-
define_method(name, &block)
|
57
|
-
end
|
58
|
-
end
|
59
|
-
|
60
|
-
def describe(description, toplevel=nil, &blk)
|
61
|
-
text = toplevel ? description : "#{name} #{description}"
|
62
|
-
klass = Class.new(self)
|
63
|
-
|
64
|
-
klass.class_eval <<-RUBY_EVAL
|
65
|
-
def self.name
|
66
|
-
"#{text}"
|
67
|
-
end
|
68
|
-
RUBY_EVAL
|
69
|
-
|
70
|
-
# do not inherit test methods from the superclass
|
71
|
-
klass.class_eval do
|
72
|
-
instance_methods.grep(/^test.+/) do |method|
|
73
|
-
undef_method method
|
74
|
-
end
|
75
|
-
end
|
76
|
-
|
77
|
-
klass.instance_eval &blk
|
78
|
-
end
|
79
|
-
alias_method :context, :describe
|
80
|
-
|
81
|
-
def let(name, &blk)
|
82
|
-
values = {}
|
83
|
-
define_method(name) do
|
84
|
-
return values[name] if values.has_key?(name)
|
85
|
-
values[name] = instance_eval(&blk)
|
86
|
-
end
|
87
|
-
end
|
88
|
-
|
89
|
-
def it(description, &blk)
|
90
|
-
define_method("test: #{name} #{description}", &blk)
|
91
|
-
end
|
92
|
-
end
|
93
|
-
|
94
|
-
end
|
95
|
-
|
96
|
-
def describe(description, &blk)
|
97
|
-
ActiveSupport::TestCase.describe(description, true, &blk)
|
98
|
-
end
|
99
|
-
|
100
|
-
FileUtils.mkdir_p'log'
|
27
|
+
FileUtils.mkdir_p 'log'
|
101
28
|
ActiveRecord::Base.logger = Logger.new("log/test.log")
|
102
29
|
ActiveRecord::Base.logger.level = Logger::DEBUG
|
103
30
|
ActiveRecord::Base.configurations["test"] = YAML.load(test_dir.join("database.yml").open)[adapter]
|
31
|
+
|
32
|
+
|
33
|
+
require "activerecord-import"
|
104
34
|
ActiveRecord::Base.establish_connection "test"
|
105
35
|
|
36
|
+
|
106
37
|
ActiveSupport::Notifications.subscribe(/active_record.sql/) do |event, _, _, _, hsh|
|
107
38
|
ActiveRecord::Base.logger.info hsh[:sql]
|
108
39
|
end
|
metadata
CHANGED
@@ -1,13 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: activerecord-import
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
4
|
+
hash: 23
|
5
5
|
prerelease: false
|
6
6
|
segments:
|
7
7
|
- 0
|
8
|
-
-
|
8
|
+
- 2
|
9
9
|
- 0
|
10
|
-
version: 0.
|
10
|
+
version: 0.2.0
|
11
11
|
platform: ruby
|
12
12
|
authors:
|
13
13
|
- Zach Dennis
|
@@ -15,26 +15,25 @@ autorequire:
|
|
15
15
|
bindir: bin
|
16
16
|
cert_chain: []
|
17
17
|
|
18
|
-
date: 2010-
|
18
|
+
date: 2010-09-26 00:00:00 -04:00
|
19
19
|
default_executable:
|
20
20
|
dependencies:
|
21
21
|
- !ruby/object:Gem::Dependency
|
22
|
+
type: :runtime
|
22
23
|
prerelease: false
|
23
24
|
version_requirements: &id001 !ruby/object:Gem::Requirement
|
24
25
|
none: false
|
25
26
|
requirements:
|
26
27
|
- - ">="
|
27
28
|
- !ruby/object:Gem::Version
|
28
|
-
hash:
|
29
|
+
hash: 416656576
|
29
30
|
segments:
|
30
31
|
- 3
|
31
32
|
- 0
|
32
|
-
-
|
33
|
-
|
34
|
-
version: 3.0.0.rc
|
35
|
-
requirement: *id001
|
33
|
+
- 0rc
|
34
|
+
version: 3.0.0rc
|
36
35
|
name: rails
|
37
|
-
|
36
|
+
requirement: *id001
|
38
37
|
description: Extraction of the ActiveRecord::Base#import functionality from ar-extensions for Rails 3 and beyond
|
39
38
|
email: zach.dennis@gmail.com
|
40
39
|
executables: []
|
@@ -43,21 +42,31 @@ extensions: []
|
|
43
42
|
|
44
43
|
extra_rdoc_files:
|
45
44
|
- README.markdown
|
45
|
+
- README.textile
|
46
46
|
files:
|
47
47
|
- README.markdown
|
48
|
+
- README.textile
|
48
49
|
- Rakefile
|
49
50
|
- VERSION
|
51
|
+
- lib/activerecord-import.rb
|
50
52
|
- lib/activerecord-import/active_record/adapters/abstract_adapter.rb
|
53
|
+
- lib/activerecord-import/active_record/adapters/mysql2_adapter.rb
|
51
54
|
- lib/activerecord-import/active_record/adapters/mysql_adapter.rb
|
52
55
|
- lib/activerecord-import/active_record/adapters/postgresql_adapter.rb
|
53
56
|
- lib/activerecord-import/active_record/adapters/sqlite3_adapter.rb
|
57
|
+
- lib/activerecord-import/adapters/abstract_adapter.rb
|
58
|
+
- lib/activerecord-import/adapters/mysql_adapter.rb
|
59
|
+
- lib/activerecord-import/adapters/postgresql_adapter.rb
|
60
|
+
- lib/activerecord-import/adapters/sqlite3_adapter.rb
|
54
61
|
- lib/activerecord-import/base.rb
|
55
62
|
- lib/activerecord-import/import.rb
|
56
63
|
- lib/activerecord-import/mysql.rb
|
64
|
+
- lib/activerecord-import/mysql2.rb
|
57
65
|
- lib/activerecord-import/postgresql.rb
|
58
66
|
- lib/activerecord-import/sqlite3.rb
|
59
67
|
- test/active_record/connection_adapter_test.rb
|
60
68
|
- test/adapters/mysql.rb
|
69
|
+
- test/adapters/mysql2.rb
|
61
70
|
- test/adapters/postgresql.rb
|
62
71
|
- test/adapters/sqlite3.rb
|
63
72
|
- test/import_test.rb
|
@@ -65,12 +74,15 @@ files:
|
|
65
74
|
- test/models/group.rb
|
66
75
|
- test/models/topic.rb
|
67
76
|
- test/mysql/import_test.rb
|
77
|
+
- test/mysql2/import_test.rb
|
68
78
|
- test/schema/generic_schema.rb
|
69
79
|
- test/schema/mysql_schema.rb
|
70
80
|
- test/schema/version.rb
|
81
|
+
- test/support/active_support/test_case_extensions.rb
|
71
82
|
- test/support/factories.rb
|
72
83
|
- test/support/generate.rb
|
73
84
|
- test/support/mysql/assertions.rb
|
85
|
+
- test/support/mysql/import_examples.rb
|
74
86
|
- test/test_helper.rb
|
75
87
|
has_rdoc: true
|
76
88
|
homepage: http://github.com/zdennis/activerecord-import
|
@@ -109,6 +121,7 @@ summary: Bulk-loading extension for ActiveRecord
|
|
109
121
|
test_files:
|
110
122
|
- test/active_record/connection_adapter_test.rb
|
111
123
|
- test/adapters/mysql.rb
|
124
|
+
- test/adapters/mysql2.rb
|
112
125
|
- test/adapters/postgresql.rb
|
113
126
|
- test/adapters/sqlite3.rb
|
114
127
|
- test/import_test.rb
|
@@ -116,10 +129,13 @@ test_files:
|
|
116
129
|
- test/models/group.rb
|
117
130
|
- test/models/topic.rb
|
118
131
|
- test/mysql/import_test.rb
|
132
|
+
- test/mysql2/import_test.rb
|
119
133
|
- test/schema/generic_schema.rb
|
120
134
|
- test/schema/mysql_schema.rb
|
121
135
|
- test/schema/version.rb
|
136
|
+
- test/support/active_support/test_case_extensions.rb
|
122
137
|
- test/support/factories.rb
|
123
138
|
- test/support/generate.rb
|
124
139
|
- test/support/mysql/assertions.rb
|
140
|
+
- test/support/mysql/import_examples.rb
|
125
141
|
- test/test_helper.rb
|