activerecord-import-uuid 0.1
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +32 -0
- data/.rubocop.yml +49 -0
- data/.rubocop_todo.yml +36 -0
- data/.travis.yml +52 -0
- data/Brewfile +3 -0
- data/CHANGELOG.md +87 -0
- data/Gemfile +54 -0
- data/LICENSE +56 -0
- data/README.markdown +101 -0
- data/Rakefile +66 -0
- data/activerecord-import.gemspec +23 -0
- data/benchmarks/README +32 -0
- data/benchmarks/benchmark.rb +67 -0
- data/benchmarks/lib/base.rb +138 -0
- data/benchmarks/lib/cli_parser.rb +106 -0
- data/benchmarks/lib/float.rb +15 -0
- data/benchmarks/lib/mysql2_benchmark.rb +19 -0
- data/benchmarks/lib/output_to_csv.rb +19 -0
- data/benchmarks/lib/output_to_html.rb +64 -0
- data/benchmarks/models/test_innodb.rb +3 -0
- data/benchmarks/models/test_memory.rb +3 -0
- data/benchmarks/models/test_myisam.rb +3 -0
- data/benchmarks/schema/mysql_schema.rb +16 -0
- data/gemfiles/3.2.gemfile +3 -0
- data/gemfiles/4.0.gemfile +3 -0
- data/gemfiles/4.1.gemfile +3 -0
- data/gemfiles/4.2.gemfile +7 -0
- data/gemfiles/5.0.gemfile +3 -0
- data/lib/activerecord-import.rb +19 -0
- data/lib/activerecord-import/active_record/adapters/abstract_adapter.rb +9 -0
- data/lib/activerecord-import/active_record/adapters/jdbcmysql_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/jdbcpostgresql_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/mysql2_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/postgresql_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/seamless_database_pool_adapter.rb +7 -0
- data/lib/activerecord-import/active_record/adapters/sqlite3_adapter.rb +6 -0
- data/lib/activerecord-import/adapters/abstract_adapter.rb +78 -0
- data/lib/activerecord-import/adapters/em_mysql2_adapter.rb +5 -0
- data/lib/activerecord-import/adapters/mysql2_adapter.rb +5 -0
- data/lib/activerecord-import/adapters/mysql_adapter.rb +114 -0
- data/lib/activerecord-import/adapters/postgresql_adapter.rb +144 -0
- data/lib/activerecord-import/adapters/sqlite3_adapter.rb +51 -0
- data/lib/activerecord-import/base.rb +38 -0
- data/lib/activerecord-import/import.rb +660 -0
- data/lib/activerecord-import/mysql2.rb +7 -0
- data/lib/activerecord-import/postgresql.rb +7 -0
- data/lib/activerecord-import/sqlite3.rb +7 -0
- data/lib/activerecord-import/synchronize.rb +66 -0
- data/lib/activerecord-import/value_sets_parser.rb +55 -0
- data/lib/activerecord-import/version.rb +5 -0
- data/test/adapters/jdbcmysql.rb +1 -0
- data/test/adapters/jdbcpostgresql.rb +1 -0
- data/test/adapters/mysql2.rb +1 -0
- data/test/adapters/mysql2_makara.rb +1 -0
- data/test/adapters/mysql2spatial.rb +1 -0
- data/test/adapters/postgis.rb +1 -0
- data/test/adapters/postgresql.rb +1 -0
- data/test/adapters/postgresql_makara.rb +1 -0
- data/test/adapters/seamless_database_pool.rb +1 -0
- data/test/adapters/spatialite.rb +1 -0
- data/test/adapters/sqlite3.rb +1 -0
- data/test/database.yml.sample +52 -0
- data/test/import_test.rb +574 -0
- data/test/jdbcmysql/import_test.rb +6 -0
- data/test/jdbcpostgresql/import_test.rb +5 -0
- data/test/models/book.rb +7 -0
- data/test/models/chapter.rb +4 -0
- data/test/models/discount.rb +3 -0
- data/test/models/end_note.rb +4 -0
- data/test/models/group.rb +3 -0
- data/test/models/promotion.rb +3 -0
- data/test/models/question.rb +3 -0
- data/test/models/rule.rb +3 -0
- data/test/models/topic.rb +9 -0
- data/test/models/widget.rb +24 -0
- data/test/mysql2/import_test.rb +5 -0
- data/test/mysql2_makara/import_test.rb +6 -0
- data/test/mysqlspatial2/import_test.rb +6 -0
- data/test/postgis/import_test.rb +4 -0
- data/test/postgresql/import_test.rb +8 -0
- data/test/schema/generic_schema.rb +144 -0
- data/test/schema/mysql_schema.rb +16 -0
- data/test/schema/version.rb +10 -0
- data/test/sqlite3/import_test.rb +52 -0
- data/test/support/active_support/test_case_extensions.rb +70 -0
- data/test/support/assertions.rb +73 -0
- data/test/support/factories.rb +57 -0
- data/test/support/generate.rb +29 -0
- data/test/support/mysql/import_examples.rb +85 -0
- data/test/support/postgresql/import_examples.rb +242 -0
- data/test/support/shared_examples/on_duplicate_key_update.rb +103 -0
- data/test/support/shared_examples/recursive_import.rb +122 -0
- data/test/synchronize_test.rb +33 -0
- data/test/test_helper.rb +59 -0
- data/test/travis/database.yml +62 -0
- data/test/value_sets_bytes_parser_test.rb +93 -0
- data/test/value_sets_records_parser_test.rb +32 -0
- metadata +225 -0
data/.gitignore
ADDED
@@ -0,0 +1,32 @@
|
|
1
|
+
## MAC OS
|
2
|
+
.DS_Store
|
3
|
+
|
4
|
+
## TEXTMATE
|
5
|
+
*.tmproj
|
6
|
+
tmtags
|
7
|
+
|
8
|
+
## EMACS
|
9
|
+
*~
|
10
|
+
\#*
|
11
|
+
.\#*
|
12
|
+
|
13
|
+
## VIM
|
14
|
+
*.swp
|
15
|
+
|
16
|
+
## PROJECT::GENERAL
|
17
|
+
coverage
|
18
|
+
rdoc
|
19
|
+
pkg
|
20
|
+
*.gem
|
21
|
+
*.lock
|
22
|
+
|
23
|
+
## PROJECT::SPECIFIC
|
24
|
+
log/*.log
|
25
|
+
test.db
|
26
|
+
test/database.yml
|
27
|
+
|
28
|
+
.ruby-*
|
29
|
+
.bundle/
|
30
|
+
.redcar/
|
31
|
+
.rvmrc
|
32
|
+
docsite/
|
data/.rubocop.yml
ADDED
@@ -0,0 +1,49 @@
|
|
1
|
+
inherit_from: .rubocop_todo.yml
|
2
|
+
|
3
|
+
Lint/EndAlignment:
|
4
|
+
AlignWith: variable
|
5
|
+
|
6
|
+
Metrics/AbcSize:
|
7
|
+
Enabled: false
|
8
|
+
|
9
|
+
Metrics/ClassLength:
|
10
|
+
Enabled: false
|
11
|
+
|
12
|
+
Metrics/CyclomaticComplexity:
|
13
|
+
Enabled: false
|
14
|
+
|
15
|
+
Metrics/LineLength:
|
16
|
+
Enabled: false
|
17
|
+
|
18
|
+
Metrics/MethodLength:
|
19
|
+
Enabled: false
|
20
|
+
|
21
|
+
Metrics/ModuleLength:
|
22
|
+
Enabled: false
|
23
|
+
|
24
|
+
Metrics/PerceivedComplexity:
|
25
|
+
Enabled: false
|
26
|
+
|
27
|
+
Style/AlignParameters:
|
28
|
+
EnforcedStyle: with_fixed_indentation
|
29
|
+
|
30
|
+
Style/ClassAndModuleChildren:
|
31
|
+
Enabled: false
|
32
|
+
|
33
|
+
Style/Documentation:
|
34
|
+
Enabled: false
|
35
|
+
|
36
|
+
Style/ElseAlignment:
|
37
|
+
Enabled: false
|
38
|
+
|
39
|
+
Style/SpaceInsideParens:
|
40
|
+
Enabled: false
|
41
|
+
|
42
|
+
Style/SpecialGlobalVars:
|
43
|
+
Enabled: false
|
44
|
+
|
45
|
+
Style/StringLiterals:
|
46
|
+
Enabled: false
|
47
|
+
|
48
|
+
Style/TrailingCommaInLiteral:
|
49
|
+
Enabled: false
|
data/.rubocop_todo.yml
ADDED
@@ -0,0 +1,36 @@
|
|
1
|
+
# This configuration was generated by
|
2
|
+
# `rubocop --auto-gen-config`
|
3
|
+
# on 2016-03-17 18:14:55 -0700 using RuboCop version 0.38.0.
|
4
|
+
# The point is for the user to remove these configuration records
|
5
|
+
# one by one as the offenses are removed from the code base.
|
6
|
+
# Note that changes in the inspected code, or installation of new
|
7
|
+
# versions of RuboCop, may require this file to be generated again.
|
8
|
+
|
9
|
+
# Offense count: 2
|
10
|
+
Lint/HandleExceptions:
|
11
|
+
Exclude:
|
12
|
+
- 'lib/activerecord-import/base.rb'
|
13
|
+
- 'test/import_test.rb'
|
14
|
+
|
15
|
+
# Offense count: 2
|
16
|
+
Lint/RescueException:
|
17
|
+
Exclude:
|
18
|
+
- 'benchmarks/lib/cli_parser.rb'
|
19
|
+
- 'test/import_test.rb'
|
20
|
+
|
21
|
+
# Offense count: 4
|
22
|
+
# Cop supports --auto-correct.
|
23
|
+
# Configuration parameters: AllowUnusedKeywordArguments, IgnoreEmptyMethods.
|
24
|
+
Lint/UnusedMethodArgument:
|
25
|
+
Exclude:
|
26
|
+
- 'lib/activerecord-import/adapters/postgresql_adapter.rb'
|
27
|
+
- 'lib/activerecord-import/import.rb'
|
28
|
+
|
29
|
+
# Offense count: 2
|
30
|
+
# Cop supports --auto-correct.
|
31
|
+
# Configuration parameters: Keywords.
|
32
|
+
# Keywords: TODO, FIXME, OPTIMIZE, HACK, REVIEW
|
33
|
+
Style/CommentAnnotation:
|
34
|
+
Exclude:
|
35
|
+
- 'benchmarks/lib/cli_parser.rb'
|
36
|
+
- 'lib/activerecord-import/import.rb'
|
data/.travis.yml
ADDED
@@ -0,0 +1,52 @@
|
|
1
|
+
language: ruby
|
2
|
+
cache: bundler
|
3
|
+
rvm:
|
4
|
+
- 2.2.4
|
5
|
+
|
6
|
+
env:
|
7
|
+
global:
|
8
|
+
# https://github.com/discourse/discourse/blob/master/.travis.yml
|
9
|
+
- RUBY_GC_MALLOC_LIMIT=50000000
|
10
|
+
matrix:
|
11
|
+
- AR_VERSION=3.2
|
12
|
+
- AR_VERSION=4.0
|
13
|
+
- AR_VERSION=4.1
|
14
|
+
- AR_VERSION=4.2
|
15
|
+
- AR_VERSION=5.0
|
16
|
+
|
17
|
+
matrix:
|
18
|
+
include:
|
19
|
+
- rvm: jruby-9.0.5.0
|
20
|
+
env: AR_VERSION=4.2
|
21
|
+
script:
|
22
|
+
- bundle exec rake test:jdbcmysql
|
23
|
+
- bundle exec rake test:jdbcpostgresql
|
24
|
+
|
25
|
+
fast_finish: true
|
26
|
+
|
27
|
+
before_script:
|
28
|
+
- mysql -e 'create database activerecord_import_test;'
|
29
|
+
- psql -c 'create database activerecord_import_test;' -U postgres
|
30
|
+
- psql -U postgres -c "create extension postgis"
|
31
|
+
- cp test/travis/database.yml test/database.yml
|
32
|
+
|
33
|
+
addons:
|
34
|
+
apt:
|
35
|
+
sources:
|
36
|
+
- travis-ci/sqlite3
|
37
|
+
packages:
|
38
|
+
- sqlite3
|
39
|
+
|
40
|
+
script:
|
41
|
+
- bundle exec rake test:mysql2
|
42
|
+
- bundle exec rake test:mysql2_makara
|
43
|
+
- bundle exec rake test:mysql2spatial
|
44
|
+
- bundle exec rake test:postgis
|
45
|
+
- bundle exec rake test:postgresql
|
46
|
+
- bundle exec rake test:postgresql_makara
|
47
|
+
- bundle exec rake test:seamless_database_pool
|
48
|
+
- bundle exec rake test:spatialite
|
49
|
+
- bundle exec rake test:sqlite3
|
50
|
+
- bundle exec rubocop
|
51
|
+
|
52
|
+
sudo: false
|
data/Brewfile
ADDED
data/CHANGELOG.md
ADDED
@@ -0,0 +1,87 @@
|
|
1
|
+
## Changes in 0.15.0
|
2
|
+
|
3
|
+
### New Features
|
4
|
+
|
5
|
+
* An ArgumentError is now raised if when no `conflict_target` or `conflict_name` is provided or can be determined when using the `on_duplicate_key_update` option for PostgreSQL. Thanks to @jkowens via \#290
|
6
|
+
* Support for Rails 5.0 final release for all except the JDBC driver which is not yet updated to support Rails 5.0
|
7
|
+
|
8
|
+
### Fixes
|
9
|
+
|
10
|
+
* activerecord-import no longer modifies a value array inside of the given values array when called with `import(columns, values)`. Thanks to @jkowens via \#291
|
11
|
+
|
12
|
+
### Misc
|
13
|
+
|
14
|
+
* `raise_error` is used to raise errors for ActiveRecord 5.0. Thanks to @couragecourag via \#294 `raise_record_invalid` has been
|
15
|
+
|
16
|
+
|
17
|
+
## Changes in 0.14.1
|
18
|
+
|
19
|
+
### Fixes
|
20
|
+
|
21
|
+
* JRuby/JDBCDriver with PostgreSQL will no longer raise a JDBCDriver error when using the :no_returning boolean option. Thanks to @jkowens via \#287
|
22
|
+
|
23
|
+
## Changes in 0.14.0
|
24
|
+
|
25
|
+
### New Features
|
26
|
+
|
27
|
+
* Support for ActiveRecord 3.1 has been dropped. Thanks to @sferik via \#254
|
28
|
+
* SQLite3 has learned the :recursive option. Thanks to @jkowens via \#281
|
29
|
+
* :on_duplicate_key_ignore will be ignored when imports are being done with :recursive. Thanks to @jkowens via \#268
|
30
|
+
* :activerecord-import learned how to tell PostgreSQL to return no data back from the import via the :no_returning boolean option. Thanks to @makaroni4 via \#276
|
31
|
+
|
32
|
+
### Fixes
|
33
|
+
|
34
|
+
* Polymorphic associations will not import the :type column. Thanks to @seanlinsley via \#282 and \#283
|
35
|
+
* ~2X speed increase for importing models with validations. Thanks to @jkowens via \#266
|
36
|
+
|
37
|
+
### Misc
|
38
|
+
|
39
|
+
* Benchmark HTML report has been fixed. Thanks to @jkowens via \#264
|
40
|
+
* seamless_database_pool has been updated to work with AR 5.0. Thanks to @jkowens via \#280
|
41
|
+
* Code cleanup, removal of redundant condition checks. Thanks to @pavlik4k via \#273
|
42
|
+
* Code cleanup, removal of deprecated `alias_method_chain`. Thanks to @codeodor via \#271
|
43
|
+
|
44
|
+
|
45
|
+
## Changes in 0.13.0
|
46
|
+
|
47
|
+
### New Features
|
48
|
+
|
49
|
+
* Addition of :batch_size option to control the number of rows to insert per INSERT statement. The default is the total number of records being inserted so there is a single INSERT statement. Thanks to @jkowens via \#245
|
50
|
+
|
51
|
+
* Addition `import!` which will raise an exception if a validation occurs. It will fail fast. Thanks to @jkowens via \#246
|
52
|
+
|
53
|
+
### Fixes
|
54
|
+
|
55
|
+
* Fixing issue with recursive import when utilizing the `:on_duplicate_key_update` option. The `on_duplicate_key_update` only applies to parent models at this time. Thanks to @yuri-karpovich for reporting and @jkowens for fixing via \#249
|
56
|
+
|
57
|
+
### Misc
|
58
|
+
|
59
|
+
* Refactoring of fetching and assigning attributes. Thanks to @jkownes via \#259
|
60
|
+
* Lots of code cleanup and addition of Rubocop linter. Thanks to @sferik via \#256 and \#250
|
61
|
+
* Resolving errors with the test suite when running against ActiveRecord 4.0 and 4.1. Thanks to @jkowens via \#262
|
62
|
+
* Cleaning up the TravisCI settings and packages. Thanks to @sferik via \#258 and \#251
|
63
|
+
|
64
|
+
## Changes in 0.12.0
|
65
|
+
|
66
|
+
### New Features
|
67
|
+
|
68
|
+
* PostgreSQL UPSERT support has been added. Thanks @jkowens via \#218
|
69
|
+
|
70
|
+
### Fixes
|
71
|
+
|
72
|
+
* has_one and has_many associations will now be recursively imported regardless of :autosave being set. Thanks @sferik, @jkowens via \#243, \#234
|
73
|
+
* Fixing an issue with enum column support for Rails > 4.1. Thanks @aquajach via \#235
|
74
|
+
|
75
|
+
### Removals
|
76
|
+
|
77
|
+
* Support for em-synchrony has been removed since it appears the project has been abandoned. Thanks @sferik, @zdennis via \#239
|
78
|
+
* Support for the mysql gem/adapter has been removed since it has officially been abandoned. Use the mysql2 gem/adapter instead. Thanks @sferik, @zdennis via \#239
|
79
|
+
|
80
|
+
### Misc
|
81
|
+
|
82
|
+
* Cleaned up TravisCI output and removing deprecation warnings. Thanks @jkowens, @zdennis \#242
|
83
|
+
|
84
|
+
|
85
|
+
## Changes before 0.12.0
|
86
|
+
|
87
|
+
> Never look back. What's gone is now history. But in the process make memory of events to help you understand what will help you to make your dream a true story. Mistakes of the past are lessons, success of the past is inspiration. – Dr. Anil Kr Sinha
|
data/Gemfile
ADDED
@@ -0,0 +1,54 @@
|
|
1
|
+
source 'https://rubygems.org'
|
2
|
+
|
3
|
+
gemspec
|
4
|
+
|
5
|
+
group :development, :test do
|
6
|
+
gem 'rubocop', '~> 0.38.0'
|
7
|
+
end
|
8
|
+
|
9
|
+
# Database Adapters
|
10
|
+
platforms :ruby do
|
11
|
+
gem "mysql2", "~> 0.3.0"
|
12
|
+
gem "pg", "~> 0.9"
|
13
|
+
gem "sqlite3", "~> 1.3.10"
|
14
|
+
gem "seamless_database_pool", "~> 1.0.18"
|
15
|
+
end
|
16
|
+
|
17
|
+
platforms :jruby do
|
18
|
+
gem "jdbc-mysql"
|
19
|
+
gem "jdbc-postgres"
|
20
|
+
gem "activerecord-jdbcmysql-adapter"
|
21
|
+
gem "activerecord-jdbcpostgresql-adapter"
|
22
|
+
end
|
23
|
+
|
24
|
+
# Support libs
|
25
|
+
gem "factory_girl", "~> 4.2.0"
|
26
|
+
gem "timecop"
|
27
|
+
gem "chronic"
|
28
|
+
|
29
|
+
# Debugging
|
30
|
+
platforms :jruby do
|
31
|
+
gem "ruby-debug-base", "= 0.10.4"
|
32
|
+
end
|
33
|
+
|
34
|
+
platforms :jruby, :mri_18 do
|
35
|
+
gem "ruby-debug", "= 0.10.4"
|
36
|
+
end
|
37
|
+
|
38
|
+
platforms :mri_19 do
|
39
|
+
gem "debugger"
|
40
|
+
end
|
41
|
+
|
42
|
+
platforms :ruby do
|
43
|
+
gem "pry-byebug"
|
44
|
+
end
|
45
|
+
|
46
|
+
version = ENV['AR_VERSION'] || "4.2"
|
47
|
+
|
48
|
+
if version >= "4.0"
|
49
|
+
gem "minitest"
|
50
|
+
else
|
51
|
+
gem "test-unit"
|
52
|
+
end
|
53
|
+
|
54
|
+
eval_gemfile File.expand_path("../gemfiles/#{version}.gemfile", __FILE__)
|
data/LICENSE
ADDED
@@ -0,0 +1,56 @@
|
|
1
|
+
Ruby is copyrighted free software by Yukihiro Matsumoto <matz@netlab.jp>.
|
2
|
+
You can redistribute it and/or modify it under either the terms of the
|
3
|
+
2-clause BSDL (see the file BSDL), or the conditions below:
|
4
|
+
|
5
|
+
1. You may make and give away verbatim copies of the source form of the
|
6
|
+
software without restriction, provided that you duplicate all of the
|
7
|
+
original copyright notices and associated disclaimers.
|
8
|
+
|
9
|
+
2. You may modify your copy of the software in any way, provided that
|
10
|
+
you do at least ONE of the following:
|
11
|
+
|
12
|
+
a) place your modifications in the Public Domain or otherwise
|
13
|
+
make them Freely Available, such as by posting said
|
14
|
+
modifications to Usenet or an equivalent medium, or by allowing
|
15
|
+
the author to include your modifications in the software.
|
16
|
+
|
17
|
+
b) use the modified software only within your corporation or
|
18
|
+
organization.
|
19
|
+
|
20
|
+
c) give non-standard binaries non-standard names, with
|
21
|
+
instructions on where to get the original software distribution.
|
22
|
+
|
23
|
+
d) make other distribution arrangements with the author.
|
24
|
+
|
25
|
+
3. You may distribute the software in object code or binary form,
|
26
|
+
provided that you do at least ONE of the following:
|
27
|
+
|
28
|
+
a) distribute the binaries and library files of the software,
|
29
|
+
together with instructions (in the manual page or equivalent)
|
30
|
+
on where to get the original distribution.
|
31
|
+
|
32
|
+
b) accompany the distribution with the machine-readable source of
|
33
|
+
the software.
|
34
|
+
|
35
|
+
c) give non-standard binaries non-standard names, with
|
36
|
+
instructions on where to get the original software distribution.
|
37
|
+
|
38
|
+
d) make other distribution arrangements with the author.
|
39
|
+
|
40
|
+
4. You may modify and include the part of the software into any other
|
41
|
+
software (possibly commercial). But some files in the distribution
|
42
|
+
are not written by the author, so that they are not under these terms.
|
43
|
+
|
44
|
+
For the list of those files and their copying conditions, see the
|
45
|
+
file LEGAL.
|
46
|
+
|
47
|
+
5. The scripts and library files supplied as input to or produced as
|
48
|
+
output from the software do not automatically fall under the
|
49
|
+
copyright of the software, but belong to whomever generated them,
|
50
|
+
and may be sold commercially, and may be aggregated with this
|
51
|
+
software.
|
52
|
+
|
53
|
+
6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
|
54
|
+
IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
|
55
|
+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
56
|
+
PURPOSE.
|
data/README.markdown
ADDED
@@ -0,0 +1,101 @@
|
|
1
|
+
# activerecord-import [![Build Status](https://travis-ci.org/zdennis/activerecord-import.svg?branch=master)](https://travis-ci.org/zdennis/activerecord-import)
|
2
|
+
|
3
|
+
activerecord-import is a library for bulk inserting data using ActiveRecord.
|
4
|
+
|
5
|
+
One of its major features is following activerecord associations and generating the minimal
|
6
|
+
number of SQL insert statements required, avoiding the N+1 insert problem. An example probably
|
7
|
+
explains it best. Say you had a schema like this:
|
8
|
+
|
9
|
+
- Publishers have Books
|
10
|
+
- Books have Reviews
|
11
|
+
|
12
|
+
and you wanted to bulk insert 100 new publishers with 10K books and 3 reviews per book. This library will follow the associations
|
13
|
+
down and generate only 3 SQL insert statements - one for the publishers, one for the books, and one for the reviews.
|
14
|
+
|
15
|
+
In contrast, the standard ActiveRecord save would generate
|
16
|
+
100 insert statements for the publishers, then it would visit each publisher and save all the books:
|
17
|
+
100 * 10,000 = 1,000,000 SQL insert statements
|
18
|
+
and then the reviews:
|
19
|
+
100 * 10,000 * 3 = 3M SQL insert statements,
|
20
|
+
|
21
|
+
That would be about 4M SQL insert statements vs 3, which results in vastly improved performance. In our case, it converted
|
22
|
+
an 18 hour batch process to <2 hrs.
|
23
|
+
|
24
|
+
### Rails 5.0
|
25
|
+
|
26
|
+
Use activerecord-import 0.11.0 or higher.
|
27
|
+
|
28
|
+
### Rails 4.0
|
29
|
+
|
30
|
+
Use activerecord-import 0.4.0 or higher.
|
31
|
+
|
32
|
+
### Rails 3.1.x up to, but not including 4.0
|
33
|
+
|
34
|
+
Use the latest in the activerecord-import 0.3.x series.
|
35
|
+
|
36
|
+
### Rails 3.0.x up to, but not including 3.1
|
37
|
+
|
38
|
+
Use activerecord-import 0.2.11. As of activerecord-import 0.3.0 we are relying on functionality that was introduced in Rails 3.1. Since Rails 3.0.x is no longer a supported version of Rails we have decided to drop support as well.
|
39
|
+
|
40
|
+
### For More Information
|
41
|
+
|
42
|
+
For more information on activerecord-import please see its wiki: https://github.com/zdennis/activerecord-import/wiki
|
43
|
+
|
44
|
+
## Additional Adapters
|
45
|
+
Additional adapters can be provided by gems external to activerecord-import by providing an adapter that matches the naming convention setup by activerecord-import (and subsequently activerecord) for dynamically loading adapters. This involves also providing a folder on the load path that follows the activerecord-import naming convention to allow activerecord-import to dynamically load the file.
|
46
|
+
|
47
|
+
When `ActiveRecord::Import.require_adapter("fake_name")` is called the require will be:
|
48
|
+
|
49
|
+
```ruby
|
50
|
+
require 'activerecord-import/active_record/adapters/fake_name_adapter'
|
51
|
+
```
|
52
|
+
|
53
|
+
This allows an external gem to dyanmically add an adapter without the need to add any file/code to the core activerecord-import gem.
|
54
|
+
|
55
|
+
### Load Path Setup
|
56
|
+
To understand how rubygems loads code you can reference the following:
|
57
|
+
|
58
|
+
http://guides.rubygems.org/patterns/#loading_code
|
59
|
+
|
60
|
+
And an example of how active_record dynamically load adapters:
|
61
|
+
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/connection_specification.rb
|
62
|
+
|
63
|
+
In summary, when a gem is loaded rubygems adds the `lib` folder of the gem to the global load path `$LOAD_PATH` so that all `require` lookups will not propegate through all of the folders on the load path. When a `require` is issued each folder on the `$LOAD_PATH` is checked for the file and/or folder referenced. This allows a gem (like activerecord-import) to define push the activerecord-import folder (or namespace) on the `$LOAD_PATH` and any adapters provided by activerecord-import will be found by rubygems when the require is issued.
|
64
|
+
|
65
|
+
If `fake_name` adapter is needed by a gem (potentially called `activerecord-import-fake_name`) then the folder structure should look as follows:
|
66
|
+
|
67
|
+
```bash
|
68
|
+
activerecord-import-fake_name/
|
69
|
+
|-- activerecord-import-fake_name.gemspec
|
70
|
+
|-- lib
|
71
|
+
| |-- activerecord-import-fake_name
|
72
|
+
| | |-- version.rb
|
73
|
+
| |-- activerecord-import
|
74
|
+
| | |-- active_record
|
75
|
+
| | | |-- adapters
|
76
|
+
| | | |-- fake_name_adapter.rb
|
77
|
+
|--activerecord-import-fake_name.rb
|
78
|
+
```
|
79
|
+
|
80
|
+
When rubygems pushes the `lib` folder onto the load path a `require` will now find `activerecord-import/active_record/adapters/fake_name_adapter` as it runs through the lookup process for a ruby file under that path in `$LOAD_PATH`
|
81
|
+
|
82
|
+
# License
|
83
|
+
|
84
|
+
This is licensed under the ruby license.
|
85
|
+
|
86
|
+
# Author
|
87
|
+
|
88
|
+
Zach Dennis (zach.dennis@gmail.com)
|
89
|
+
|
90
|
+
# Contributors
|
91
|
+
|
92
|
+
* Jordan Owens (@jkowens)
|
93
|
+
* Erik Michaels-Ober (@sferik)
|
94
|
+
* Blythe Dunham
|
95
|
+
* Gabe da Silveira
|
96
|
+
* Henry Work
|
97
|
+
* James Herdman
|
98
|
+
* Marcus Crafter
|
99
|
+
* Thibaud Guillaume-Gentil
|
100
|
+
* Mark Van Holstyn
|
101
|
+
* Victor Costan
|