activerecord-import-uuid 0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.gitignore +32 -0
- data/.rubocop.yml +49 -0
- data/.rubocop_todo.yml +36 -0
- data/.travis.yml +52 -0
- data/Brewfile +3 -0
- data/CHANGELOG.md +87 -0
- data/Gemfile +54 -0
- data/LICENSE +56 -0
- data/README.markdown +101 -0
- data/Rakefile +66 -0
- data/activerecord-import.gemspec +23 -0
- data/benchmarks/README +32 -0
- data/benchmarks/benchmark.rb +67 -0
- data/benchmarks/lib/base.rb +138 -0
- data/benchmarks/lib/cli_parser.rb +106 -0
- data/benchmarks/lib/float.rb +15 -0
- data/benchmarks/lib/mysql2_benchmark.rb +19 -0
- data/benchmarks/lib/output_to_csv.rb +19 -0
- data/benchmarks/lib/output_to_html.rb +64 -0
- data/benchmarks/models/test_innodb.rb +3 -0
- data/benchmarks/models/test_memory.rb +3 -0
- data/benchmarks/models/test_myisam.rb +3 -0
- data/benchmarks/schema/mysql_schema.rb +16 -0
- data/gemfiles/3.2.gemfile +3 -0
- data/gemfiles/4.0.gemfile +3 -0
- data/gemfiles/4.1.gemfile +3 -0
- data/gemfiles/4.2.gemfile +7 -0
- data/gemfiles/5.0.gemfile +3 -0
- data/lib/activerecord-import.rb +19 -0
- data/lib/activerecord-import/active_record/adapters/abstract_adapter.rb +9 -0
- data/lib/activerecord-import/active_record/adapters/jdbcmysql_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/jdbcpostgresql_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/mysql2_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/postgresql_adapter.rb +6 -0
- data/lib/activerecord-import/active_record/adapters/seamless_database_pool_adapter.rb +7 -0
- data/lib/activerecord-import/active_record/adapters/sqlite3_adapter.rb +6 -0
- data/lib/activerecord-import/adapters/abstract_adapter.rb +78 -0
- data/lib/activerecord-import/adapters/em_mysql2_adapter.rb +5 -0
- data/lib/activerecord-import/adapters/mysql2_adapter.rb +5 -0
- data/lib/activerecord-import/adapters/mysql_adapter.rb +114 -0
- data/lib/activerecord-import/adapters/postgresql_adapter.rb +144 -0
- data/lib/activerecord-import/adapters/sqlite3_adapter.rb +51 -0
- data/lib/activerecord-import/base.rb +38 -0
- data/lib/activerecord-import/import.rb +660 -0
- data/lib/activerecord-import/mysql2.rb +7 -0
- data/lib/activerecord-import/postgresql.rb +7 -0
- data/lib/activerecord-import/sqlite3.rb +7 -0
- data/lib/activerecord-import/synchronize.rb +66 -0
- data/lib/activerecord-import/value_sets_parser.rb +55 -0
- data/lib/activerecord-import/version.rb +5 -0
- data/test/adapters/jdbcmysql.rb +1 -0
- data/test/adapters/jdbcpostgresql.rb +1 -0
- data/test/adapters/mysql2.rb +1 -0
- data/test/adapters/mysql2_makara.rb +1 -0
- data/test/adapters/mysql2spatial.rb +1 -0
- data/test/adapters/postgis.rb +1 -0
- data/test/adapters/postgresql.rb +1 -0
- data/test/adapters/postgresql_makara.rb +1 -0
- data/test/adapters/seamless_database_pool.rb +1 -0
- data/test/adapters/spatialite.rb +1 -0
- data/test/adapters/sqlite3.rb +1 -0
- data/test/database.yml.sample +52 -0
- data/test/import_test.rb +574 -0
- data/test/jdbcmysql/import_test.rb +6 -0
- data/test/jdbcpostgresql/import_test.rb +5 -0
- data/test/models/book.rb +7 -0
- data/test/models/chapter.rb +4 -0
- data/test/models/discount.rb +3 -0
- data/test/models/end_note.rb +4 -0
- data/test/models/group.rb +3 -0
- data/test/models/promotion.rb +3 -0
- data/test/models/question.rb +3 -0
- data/test/models/rule.rb +3 -0
- data/test/models/topic.rb +9 -0
- data/test/models/widget.rb +24 -0
- data/test/mysql2/import_test.rb +5 -0
- data/test/mysql2_makara/import_test.rb +6 -0
- data/test/mysqlspatial2/import_test.rb +6 -0
- data/test/postgis/import_test.rb +4 -0
- data/test/postgresql/import_test.rb +8 -0
- data/test/schema/generic_schema.rb +144 -0
- data/test/schema/mysql_schema.rb +16 -0
- data/test/schema/version.rb +10 -0
- data/test/sqlite3/import_test.rb +52 -0
- data/test/support/active_support/test_case_extensions.rb +70 -0
- data/test/support/assertions.rb +73 -0
- data/test/support/factories.rb +57 -0
- data/test/support/generate.rb +29 -0
- data/test/support/mysql/import_examples.rb +85 -0
- data/test/support/postgresql/import_examples.rb +242 -0
- data/test/support/shared_examples/on_duplicate_key_update.rb +103 -0
- data/test/support/shared_examples/recursive_import.rb +122 -0
- data/test/synchronize_test.rb +33 -0
- data/test/test_helper.rb +59 -0
- data/test/travis/database.yml +62 -0
- data/test/value_sets_bytes_parser_test.rb +93 -0
- data/test/value_sets_records_parser_test.rb +32 -0
- metadata +225 -0
data/.gitignore
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
## MAC OS
|
|
2
|
+
.DS_Store
|
|
3
|
+
|
|
4
|
+
## TEXTMATE
|
|
5
|
+
*.tmproj
|
|
6
|
+
tmtags
|
|
7
|
+
|
|
8
|
+
## EMACS
|
|
9
|
+
*~
|
|
10
|
+
\#*
|
|
11
|
+
.\#*
|
|
12
|
+
|
|
13
|
+
## VIM
|
|
14
|
+
*.swp
|
|
15
|
+
|
|
16
|
+
## PROJECT::GENERAL
|
|
17
|
+
coverage
|
|
18
|
+
rdoc
|
|
19
|
+
pkg
|
|
20
|
+
*.gem
|
|
21
|
+
*.lock
|
|
22
|
+
|
|
23
|
+
## PROJECT::SPECIFIC
|
|
24
|
+
log/*.log
|
|
25
|
+
test.db
|
|
26
|
+
test/database.yml
|
|
27
|
+
|
|
28
|
+
.ruby-*
|
|
29
|
+
.bundle/
|
|
30
|
+
.redcar/
|
|
31
|
+
.rvmrc
|
|
32
|
+
docsite/
|
data/.rubocop.yml
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
inherit_from: .rubocop_todo.yml
|
|
2
|
+
|
|
3
|
+
Lint/EndAlignment:
|
|
4
|
+
AlignWith: variable
|
|
5
|
+
|
|
6
|
+
Metrics/AbcSize:
|
|
7
|
+
Enabled: false
|
|
8
|
+
|
|
9
|
+
Metrics/ClassLength:
|
|
10
|
+
Enabled: false
|
|
11
|
+
|
|
12
|
+
Metrics/CyclomaticComplexity:
|
|
13
|
+
Enabled: false
|
|
14
|
+
|
|
15
|
+
Metrics/LineLength:
|
|
16
|
+
Enabled: false
|
|
17
|
+
|
|
18
|
+
Metrics/MethodLength:
|
|
19
|
+
Enabled: false
|
|
20
|
+
|
|
21
|
+
Metrics/ModuleLength:
|
|
22
|
+
Enabled: false
|
|
23
|
+
|
|
24
|
+
Metrics/PerceivedComplexity:
|
|
25
|
+
Enabled: false
|
|
26
|
+
|
|
27
|
+
Style/AlignParameters:
|
|
28
|
+
EnforcedStyle: with_fixed_indentation
|
|
29
|
+
|
|
30
|
+
Style/ClassAndModuleChildren:
|
|
31
|
+
Enabled: false
|
|
32
|
+
|
|
33
|
+
Style/Documentation:
|
|
34
|
+
Enabled: false
|
|
35
|
+
|
|
36
|
+
Style/ElseAlignment:
|
|
37
|
+
Enabled: false
|
|
38
|
+
|
|
39
|
+
Style/SpaceInsideParens:
|
|
40
|
+
Enabled: false
|
|
41
|
+
|
|
42
|
+
Style/SpecialGlobalVars:
|
|
43
|
+
Enabled: false
|
|
44
|
+
|
|
45
|
+
Style/StringLiterals:
|
|
46
|
+
Enabled: false
|
|
47
|
+
|
|
48
|
+
Style/TrailingCommaInLiteral:
|
|
49
|
+
Enabled: false
|
data/.rubocop_todo.yml
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# This configuration was generated by
|
|
2
|
+
# `rubocop --auto-gen-config`
|
|
3
|
+
# on 2016-03-17 18:14:55 -0700 using RuboCop version 0.38.0.
|
|
4
|
+
# The point is for the user to remove these configuration records
|
|
5
|
+
# one by one as the offenses are removed from the code base.
|
|
6
|
+
# Note that changes in the inspected code, or installation of new
|
|
7
|
+
# versions of RuboCop, may require this file to be generated again.
|
|
8
|
+
|
|
9
|
+
# Offense count: 2
|
|
10
|
+
Lint/HandleExceptions:
|
|
11
|
+
Exclude:
|
|
12
|
+
- 'lib/activerecord-import/base.rb'
|
|
13
|
+
- 'test/import_test.rb'
|
|
14
|
+
|
|
15
|
+
# Offense count: 2
|
|
16
|
+
Lint/RescueException:
|
|
17
|
+
Exclude:
|
|
18
|
+
- 'benchmarks/lib/cli_parser.rb'
|
|
19
|
+
- 'test/import_test.rb'
|
|
20
|
+
|
|
21
|
+
# Offense count: 4
|
|
22
|
+
# Cop supports --auto-correct.
|
|
23
|
+
# Configuration parameters: AllowUnusedKeywordArguments, IgnoreEmptyMethods.
|
|
24
|
+
Lint/UnusedMethodArgument:
|
|
25
|
+
Exclude:
|
|
26
|
+
- 'lib/activerecord-import/adapters/postgresql_adapter.rb'
|
|
27
|
+
- 'lib/activerecord-import/import.rb'
|
|
28
|
+
|
|
29
|
+
# Offense count: 2
|
|
30
|
+
# Cop supports --auto-correct.
|
|
31
|
+
# Configuration parameters: Keywords.
|
|
32
|
+
# Keywords: TODO, FIXME, OPTIMIZE, HACK, REVIEW
|
|
33
|
+
Style/CommentAnnotation:
|
|
34
|
+
Exclude:
|
|
35
|
+
- 'benchmarks/lib/cli_parser.rb'
|
|
36
|
+
- 'lib/activerecord-import/import.rb'
|
data/.travis.yml
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
language: ruby
|
|
2
|
+
cache: bundler
|
|
3
|
+
rvm:
|
|
4
|
+
- 2.2.4
|
|
5
|
+
|
|
6
|
+
env:
|
|
7
|
+
global:
|
|
8
|
+
# https://github.com/discourse/discourse/blob/master/.travis.yml
|
|
9
|
+
- RUBY_GC_MALLOC_LIMIT=50000000
|
|
10
|
+
matrix:
|
|
11
|
+
- AR_VERSION=3.2
|
|
12
|
+
- AR_VERSION=4.0
|
|
13
|
+
- AR_VERSION=4.1
|
|
14
|
+
- AR_VERSION=4.2
|
|
15
|
+
- AR_VERSION=5.0
|
|
16
|
+
|
|
17
|
+
matrix:
|
|
18
|
+
include:
|
|
19
|
+
- rvm: jruby-9.0.5.0
|
|
20
|
+
env: AR_VERSION=4.2
|
|
21
|
+
script:
|
|
22
|
+
- bundle exec rake test:jdbcmysql
|
|
23
|
+
- bundle exec rake test:jdbcpostgresql
|
|
24
|
+
|
|
25
|
+
fast_finish: true
|
|
26
|
+
|
|
27
|
+
before_script:
|
|
28
|
+
- mysql -e 'create database activerecord_import_test;'
|
|
29
|
+
- psql -c 'create database activerecord_import_test;' -U postgres
|
|
30
|
+
- psql -U postgres -c "create extension postgis"
|
|
31
|
+
- cp test/travis/database.yml test/database.yml
|
|
32
|
+
|
|
33
|
+
addons:
|
|
34
|
+
apt:
|
|
35
|
+
sources:
|
|
36
|
+
- travis-ci/sqlite3
|
|
37
|
+
packages:
|
|
38
|
+
- sqlite3
|
|
39
|
+
|
|
40
|
+
script:
|
|
41
|
+
- bundle exec rake test:mysql2
|
|
42
|
+
- bundle exec rake test:mysql2_makara
|
|
43
|
+
- bundle exec rake test:mysql2spatial
|
|
44
|
+
- bundle exec rake test:postgis
|
|
45
|
+
- bundle exec rake test:postgresql
|
|
46
|
+
- bundle exec rake test:postgresql_makara
|
|
47
|
+
- bundle exec rake test:seamless_database_pool
|
|
48
|
+
- bundle exec rake test:spatialite
|
|
49
|
+
- bundle exec rake test:sqlite3
|
|
50
|
+
- bundle exec rubocop
|
|
51
|
+
|
|
52
|
+
sudo: false
|
data/Brewfile
ADDED
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
## Changes in 0.15.0
|
|
2
|
+
|
|
3
|
+
### New Features
|
|
4
|
+
|
|
5
|
+
* An ArgumentError is now raised if when no `conflict_target` or `conflict_name` is provided or can be determined when using the `on_duplicate_key_update` option for PostgreSQL. Thanks to @jkowens via \#290
|
|
6
|
+
* Support for Rails 5.0 final release for all except the JDBC driver which is not yet updated to support Rails 5.0
|
|
7
|
+
|
|
8
|
+
### Fixes
|
|
9
|
+
|
|
10
|
+
* activerecord-import no longer modifies a value array inside of the given values array when called with `import(columns, values)`. Thanks to @jkowens via \#291
|
|
11
|
+
|
|
12
|
+
### Misc
|
|
13
|
+
|
|
14
|
+
* `raise_error` is used to raise errors for ActiveRecord 5.0. Thanks to @couragecourag via \#294 `raise_record_invalid` has been
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
## Changes in 0.14.1
|
|
18
|
+
|
|
19
|
+
### Fixes
|
|
20
|
+
|
|
21
|
+
* JRuby/JDBCDriver with PostgreSQL will no longer raise a JDBCDriver error when using the :no_returning boolean option. Thanks to @jkowens via \#287
|
|
22
|
+
|
|
23
|
+
## Changes in 0.14.0
|
|
24
|
+
|
|
25
|
+
### New Features
|
|
26
|
+
|
|
27
|
+
* Support for ActiveRecord 3.1 has been dropped. Thanks to @sferik via \#254
|
|
28
|
+
* SQLite3 has learned the :recursive option. Thanks to @jkowens via \#281
|
|
29
|
+
* :on_duplicate_key_ignore will be ignored when imports are being done with :recursive. Thanks to @jkowens via \#268
|
|
30
|
+
* :activerecord-import learned how to tell PostgreSQL to return no data back from the import via the :no_returning boolean option. Thanks to @makaroni4 via \#276
|
|
31
|
+
|
|
32
|
+
### Fixes
|
|
33
|
+
|
|
34
|
+
* Polymorphic associations will not import the :type column. Thanks to @seanlinsley via \#282 and \#283
|
|
35
|
+
* ~2X speed increase for importing models with validations. Thanks to @jkowens via \#266
|
|
36
|
+
|
|
37
|
+
### Misc
|
|
38
|
+
|
|
39
|
+
* Benchmark HTML report has been fixed. Thanks to @jkowens via \#264
|
|
40
|
+
* seamless_database_pool has been updated to work with AR 5.0. Thanks to @jkowens via \#280
|
|
41
|
+
* Code cleanup, removal of redundant condition checks. Thanks to @pavlik4k via \#273
|
|
42
|
+
* Code cleanup, removal of deprecated `alias_method_chain`. Thanks to @codeodor via \#271
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
## Changes in 0.13.0
|
|
46
|
+
|
|
47
|
+
### New Features
|
|
48
|
+
|
|
49
|
+
* Addition of :batch_size option to control the number of rows to insert per INSERT statement. The default is the total number of records being inserted so there is a single INSERT statement. Thanks to @jkowens via \#245
|
|
50
|
+
|
|
51
|
+
* Addition `import!` which will raise an exception if a validation occurs. It will fail fast. Thanks to @jkowens via \#246
|
|
52
|
+
|
|
53
|
+
### Fixes
|
|
54
|
+
|
|
55
|
+
* Fixing issue with recursive import when utilizing the `:on_duplicate_key_update` option. The `on_duplicate_key_update` only applies to parent models at this time. Thanks to @yuri-karpovich for reporting and @jkowens for fixing via \#249
|
|
56
|
+
|
|
57
|
+
### Misc
|
|
58
|
+
|
|
59
|
+
* Refactoring of fetching and assigning attributes. Thanks to @jkownes via \#259
|
|
60
|
+
* Lots of code cleanup and addition of Rubocop linter. Thanks to @sferik via \#256 and \#250
|
|
61
|
+
* Resolving errors with the test suite when running against ActiveRecord 4.0 and 4.1. Thanks to @jkowens via \#262
|
|
62
|
+
* Cleaning up the TravisCI settings and packages. Thanks to @sferik via \#258 and \#251
|
|
63
|
+
|
|
64
|
+
## Changes in 0.12.0
|
|
65
|
+
|
|
66
|
+
### New Features
|
|
67
|
+
|
|
68
|
+
* PostgreSQL UPSERT support has been added. Thanks @jkowens via \#218
|
|
69
|
+
|
|
70
|
+
### Fixes
|
|
71
|
+
|
|
72
|
+
* has_one and has_many associations will now be recursively imported regardless of :autosave being set. Thanks @sferik, @jkowens via \#243, \#234
|
|
73
|
+
* Fixing an issue with enum column support for Rails > 4.1. Thanks @aquajach via \#235
|
|
74
|
+
|
|
75
|
+
### Removals
|
|
76
|
+
|
|
77
|
+
* Support for em-synchrony has been removed since it appears the project has been abandoned. Thanks @sferik, @zdennis via \#239
|
|
78
|
+
* Support for the mysql gem/adapter has been removed since it has officially been abandoned. Use the mysql2 gem/adapter instead. Thanks @sferik, @zdennis via \#239
|
|
79
|
+
|
|
80
|
+
### Misc
|
|
81
|
+
|
|
82
|
+
* Cleaned up TravisCI output and removing deprecation warnings. Thanks @jkowens, @zdennis \#242
|
|
83
|
+
|
|
84
|
+
|
|
85
|
+
## Changes before 0.12.0
|
|
86
|
+
|
|
87
|
+
> Never look back. What's gone is now history. But in the process make memory of events to help you understand what will help you to make your dream a true story. Mistakes of the past are lessons, success of the past is inspiration. – Dr. Anil Kr Sinha
|
data/Gemfile
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
source 'https://rubygems.org'
|
|
2
|
+
|
|
3
|
+
gemspec
|
|
4
|
+
|
|
5
|
+
group :development, :test do
|
|
6
|
+
gem 'rubocop', '~> 0.38.0'
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
# Database Adapters
|
|
10
|
+
platforms :ruby do
|
|
11
|
+
gem "mysql2", "~> 0.3.0"
|
|
12
|
+
gem "pg", "~> 0.9"
|
|
13
|
+
gem "sqlite3", "~> 1.3.10"
|
|
14
|
+
gem "seamless_database_pool", "~> 1.0.18"
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
platforms :jruby do
|
|
18
|
+
gem "jdbc-mysql"
|
|
19
|
+
gem "jdbc-postgres"
|
|
20
|
+
gem "activerecord-jdbcmysql-adapter"
|
|
21
|
+
gem "activerecord-jdbcpostgresql-adapter"
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
# Support libs
|
|
25
|
+
gem "factory_girl", "~> 4.2.0"
|
|
26
|
+
gem "timecop"
|
|
27
|
+
gem "chronic"
|
|
28
|
+
|
|
29
|
+
# Debugging
|
|
30
|
+
platforms :jruby do
|
|
31
|
+
gem "ruby-debug-base", "= 0.10.4"
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
platforms :jruby, :mri_18 do
|
|
35
|
+
gem "ruby-debug", "= 0.10.4"
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
platforms :mri_19 do
|
|
39
|
+
gem "debugger"
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
platforms :ruby do
|
|
43
|
+
gem "pry-byebug"
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
version = ENV['AR_VERSION'] || "4.2"
|
|
47
|
+
|
|
48
|
+
if version >= "4.0"
|
|
49
|
+
gem "minitest"
|
|
50
|
+
else
|
|
51
|
+
gem "test-unit"
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
eval_gemfile File.expand_path("../gemfiles/#{version}.gemfile", __FILE__)
|
data/LICENSE
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
Ruby is copyrighted free software by Yukihiro Matsumoto <matz@netlab.jp>.
|
|
2
|
+
You can redistribute it and/or modify it under either the terms of the
|
|
3
|
+
2-clause BSDL (see the file BSDL), or the conditions below:
|
|
4
|
+
|
|
5
|
+
1. You may make and give away verbatim copies of the source form of the
|
|
6
|
+
software without restriction, provided that you duplicate all of the
|
|
7
|
+
original copyright notices and associated disclaimers.
|
|
8
|
+
|
|
9
|
+
2. You may modify your copy of the software in any way, provided that
|
|
10
|
+
you do at least ONE of the following:
|
|
11
|
+
|
|
12
|
+
a) place your modifications in the Public Domain or otherwise
|
|
13
|
+
make them Freely Available, such as by posting said
|
|
14
|
+
modifications to Usenet or an equivalent medium, or by allowing
|
|
15
|
+
the author to include your modifications in the software.
|
|
16
|
+
|
|
17
|
+
b) use the modified software only within your corporation or
|
|
18
|
+
organization.
|
|
19
|
+
|
|
20
|
+
c) give non-standard binaries non-standard names, with
|
|
21
|
+
instructions on where to get the original software distribution.
|
|
22
|
+
|
|
23
|
+
d) make other distribution arrangements with the author.
|
|
24
|
+
|
|
25
|
+
3. You may distribute the software in object code or binary form,
|
|
26
|
+
provided that you do at least ONE of the following:
|
|
27
|
+
|
|
28
|
+
a) distribute the binaries and library files of the software,
|
|
29
|
+
together with instructions (in the manual page or equivalent)
|
|
30
|
+
on where to get the original distribution.
|
|
31
|
+
|
|
32
|
+
b) accompany the distribution with the machine-readable source of
|
|
33
|
+
the software.
|
|
34
|
+
|
|
35
|
+
c) give non-standard binaries non-standard names, with
|
|
36
|
+
instructions on where to get the original software distribution.
|
|
37
|
+
|
|
38
|
+
d) make other distribution arrangements with the author.
|
|
39
|
+
|
|
40
|
+
4. You may modify and include the part of the software into any other
|
|
41
|
+
software (possibly commercial). But some files in the distribution
|
|
42
|
+
are not written by the author, so that they are not under these terms.
|
|
43
|
+
|
|
44
|
+
For the list of those files and their copying conditions, see the
|
|
45
|
+
file LEGAL.
|
|
46
|
+
|
|
47
|
+
5. The scripts and library files supplied as input to or produced as
|
|
48
|
+
output from the software do not automatically fall under the
|
|
49
|
+
copyright of the software, but belong to whomever generated them,
|
|
50
|
+
and may be sold commercially, and may be aggregated with this
|
|
51
|
+
software.
|
|
52
|
+
|
|
53
|
+
6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
|
|
54
|
+
IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
|
|
55
|
+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
|
56
|
+
PURPOSE.
|
data/README.markdown
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
# activerecord-import [](https://travis-ci.org/zdennis/activerecord-import)
|
|
2
|
+
|
|
3
|
+
activerecord-import is a library for bulk inserting data using ActiveRecord.
|
|
4
|
+
|
|
5
|
+
One of its major features is following activerecord associations and generating the minimal
|
|
6
|
+
number of SQL insert statements required, avoiding the N+1 insert problem. An example probably
|
|
7
|
+
explains it best. Say you had a schema like this:
|
|
8
|
+
|
|
9
|
+
- Publishers have Books
|
|
10
|
+
- Books have Reviews
|
|
11
|
+
|
|
12
|
+
and you wanted to bulk insert 100 new publishers with 10K books and 3 reviews per book. This library will follow the associations
|
|
13
|
+
down and generate only 3 SQL insert statements - one for the publishers, one for the books, and one for the reviews.
|
|
14
|
+
|
|
15
|
+
In contrast, the standard ActiveRecord save would generate
|
|
16
|
+
100 insert statements for the publishers, then it would visit each publisher and save all the books:
|
|
17
|
+
100 * 10,000 = 1,000,000 SQL insert statements
|
|
18
|
+
and then the reviews:
|
|
19
|
+
100 * 10,000 * 3 = 3M SQL insert statements,
|
|
20
|
+
|
|
21
|
+
That would be about 4M SQL insert statements vs 3, which results in vastly improved performance. In our case, it converted
|
|
22
|
+
an 18 hour batch process to <2 hrs.
|
|
23
|
+
|
|
24
|
+
### Rails 5.0
|
|
25
|
+
|
|
26
|
+
Use activerecord-import 0.11.0 or higher.
|
|
27
|
+
|
|
28
|
+
### Rails 4.0
|
|
29
|
+
|
|
30
|
+
Use activerecord-import 0.4.0 or higher.
|
|
31
|
+
|
|
32
|
+
### Rails 3.1.x up to, but not including 4.0
|
|
33
|
+
|
|
34
|
+
Use the latest in the activerecord-import 0.3.x series.
|
|
35
|
+
|
|
36
|
+
### Rails 3.0.x up to, but not including 3.1
|
|
37
|
+
|
|
38
|
+
Use activerecord-import 0.2.11. As of activerecord-import 0.3.0 we are relying on functionality that was introduced in Rails 3.1. Since Rails 3.0.x is no longer a supported version of Rails we have decided to drop support as well.
|
|
39
|
+
|
|
40
|
+
### For More Information
|
|
41
|
+
|
|
42
|
+
For more information on activerecord-import please see its wiki: https://github.com/zdennis/activerecord-import/wiki
|
|
43
|
+
|
|
44
|
+
## Additional Adapters
|
|
45
|
+
Additional adapters can be provided by gems external to activerecord-import by providing an adapter that matches the naming convention setup by activerecord-import (and subsequently activerecord) for dynamically loading adapters. This involves also providing a folder on the load path that follows the activerecord-import naming convention to allow activerecord-import to dynamically load the file.
|
|
46
|
+
|
|
47
|
+
When `ActiveRecord::Import.require_adapter("fake_name")` is called the require will be:
|
|
48
|
+
|
|
49
|
+
```ruby
|
|
50
|
+
require 'activerecord-import/active_record/adapters/fake_name_adapter'
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
This allows an external gem to dyanmically add an adapter without the need to add any file/code to the core activerecord-import gem.
|
|
54
|
+
|
|
55
|
+
### Load Path Setup
|
|
56
|
+
To understand how rubygems loads code you can reference the following:
|
|
57
|
+
|
|
58
|
+
http://guides.rubygems.org/patterns/#loading_code
|
|
59
|
+
|
|
60
|
+
And an example of how active_record dynamically load adapters:
|
|
61
|
+
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/connection_specification.rb
|
|
62
|
+
|
|
63
|
+
In summary, when a gem is loaded rubygems adds the `lib` folder of the gem to the global load path `$LOAD_PATH` so that all `require` lookups will not propegate through all of the folders on the load path. When a `require` is issued each folder on the `$LOAD_PATH` is checked for the file and/or folder referenced. This allows a gem (like activerecord-import) to define push the activerecord-import folder (or namespace) on the `$LOAD_PATH` and any adapters provided by activerecord-import will be found by rubygems when the require is issued.
|
|
64
|
+
|
|
65
|
+
If `fake_name` adapter is needed by a gem (potentially called `activerecord-import-fake_name`) then the folder structure should look as follows:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
activerecord-import-fake_name/
|
|
69
|
+
|-- activerecord-import-fake_name.gemspec
|
|
70
|
+
|-- lib
|
|
71
|
+
| |-- activerecord-import-fake_name
|
|
72
|
+
| | |-- version.rb
|
|
73
|
+
| |-- activerecord-import
|
|
74
|
+
| | |-- active_record
|
|
75
|
+
| | | |-- adapters
|
|
76
|
+
| | | |-- fake_name_adapter.rb
|
|
77
|
+
|--activerecord-import-fake_name.rb
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
When rubygems pushes the `lib` folder onto the load path a `require` will now find `activerecord-import/active_record/adapters/fake_name_adapter` as it runs through the lookup process for a ruby file under that path in `$LOAD_PATH`
|
|
81
|
+
|
|
82
|
+
# License
|
|
83
|
+
|
|
84
|
+
This is licensed under the ruby license.
|
|
85
|
+
|
|
86
|
+
# Author
|
|
87
|
+
|
|
88
|
+
Zach Dennis (zach.dennis@gmail.com)
|
|
89
|
+
|
|
90
|
+
# Contributors
|
|
91
|
+
|
|
92
|
+
* Jordan Owens (@jkowens)
|
|
93
|
+
* Erik Michaels-Ober (@sferik)
|
|
94
|
+
* Blythe Dunham
|
|
95
|
+
* Gabe da Silveira
|
|
96
|
+
* Henry Work
|
|
97
|
+
* James Herdman
|
|
98
|
+
* Marcus Crafter
|
|
99
|
+
* Thibaud Guillaume-Gentil
|
|
100
|
+
* Mark Van Holstyn
|
|
101
|
+
* Victor Costan
|