activerecord-import 1.0.1 → 1.0.7
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/.travis.yml +17 -14
- data/CHANGELOG.md +68 -2
- data/Gemfile +4 -1
- data/LICENSE +21 -56
- data/README.markdown +66 -70
- data/activerecord-import.gemspec +3 -3
- data/benchmarks/schema/{mysql_schema.rb → mysql2_schema.rb} +0 -0
- data/gemfiles/6.0.gemfile +1 -0
- data/gemfiles/6.1.gemfile +1 -0
- data/lib/activerecord-import/adapters/abstract_adapter.rb +7 -1
- data/lib/activerecord-import/adapters/mysql_adapter.rb +6 -6
- data/lib/activerecord-import/adapters/postgresql_adapter.rb +9 -9
- data/lib/activerecord-import/adapters/sqlite3_adapter.rb +15 -19
- data/lib/activerecord-import/base.rb +8 -1
- data/lib/activerecord-import/import.rb +45 -20
- data/lib/activerecord-import/synchronize.rb +1 -1
- data/lib/activerecord-import/version.rb +1 -1
- data/test/import_test.rb +39 -0
- data/test/models/animal.rb +6 -0
- data/test/schema/postgresql_schema.rb +14 -0
- data/test/support/postgresql/import_examples.rb +31 -0
- data/test/support/shared_examples/on_duplicate_key_update.rb +10 -0
- data/test/support/shared_examples/recursive_import.rb +9 -0
- data/test/support/sqlite3/import_examples.rb +2 -15
- metadata +14 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: fdf26ba513a4d3365d24cf9b2bf1fc814db9918d06ccdb91c5f32dec026fb7b7
|
4
|
+
data.tar.gz: 35ba31f95ef4b585cbe8ca4b0819d8ddc1ba50b12db6a70d3be2352669fd9289
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3ef741e92b76c9705c63145b58351e82813eb71b003993a73cff5db2d0779d6dc14b48d4dc43a354ad8ba6a24a6acd2aa54bdf167775dee8e0404db4905c03f2
|
7
|
+
data.tar.gz: f8634e6ae1e9345ca9b1d68bca8fe4e67c04b1ae51101fdf97608a609d1fc349737d424ce23f809548d4e6a42931eb38d246d6be354acf49282fcaf20d827fa0
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
@@ -1,30 +1,29 @@
|
|
1
1
|
language: ruby
|
2
2
|
cache: bundler
|
3
3
|
rvm:
|
4
|
-
- 2.
|
4
|
+
- 2.5.5
|
5
5
|
|
6
6
|
env:
|
7
7
|
global:
|
8
8
|
# https://github.com/discourse/discourse/blob/master/.travis.yml
|
9
9
|
- RUBY_GC_MALLOC_LIMIT=50000000
|
10
10
|
matrix:
|
11
|
-
- AR_VERSION=3.2
|
12
|
-
- AR_VERSION=4.0
|
13
|
-
- AR_VERSION=4.1
|
14
|
-
- AR_VERSION=4.2
|
15
|
-
- AR_VERSION=5.0
|
16
11
|
- AR_VERSION=5.1
|
17
12
|
- AR_VERSION=5.2
|
13
|
+
- AR_VERSION=6.0
|
18
14
|
|
19
15
|
matrix:
|
20
16
|
include:
|
21
|
-
- rvm:
|
17
|
+
- rvm: 2.3.8
|
18
|
+
env: AR_VERSION=3.2
|
19
|
+
- rvm: 2.3.8
|
20
|
+
env: AR_VERSION=4.0
|
21
|
+
- rvm: 2.3.8
|
22
|
+
env: AR_VERSION=4.1
|
23
|
+
- rvm: 2.3.8
|
22
24
|
env: AR_VERSION=4.2
|
23
|
-
|
24
|
-
|
25
|
-
- bundle exec rake test:jdbcsqlite3
|
26
|
-
- bundle exec rake test:jdbcmysql
|
27
|
-
- bundle exec rake test:jdbcpostgresql
|
25
|
+
- rvm: 2.3.8
|
26
|
+
env: AR_VERSION=5.0
|
28
27
|
|
29
28
|
fast_finish: true
|
30
29
|
|
@@ -38,7 +37,7 @@ addons:
|
|
38
37
|
- sqlite3
|
39
38
|
- mysql-server
|
40
39
|
- mysql-client
|
41
|
-
- postgresql-9.5-postgis-2.
|
40
|
+
- postgresql-9.5-postgis-2.4
|
42
41
|
|
43
42
|
before_install:
|
44
43
|
- gem update --system
|
@@ -66,6 +65,10 @@ script:
|
|
66
65
|
- bundle exec rake test:sqlite3
|
67
66
|
- bundle exec rubocop
|
68
67
|
|
69
|
-
dist:
|
68
|
+
dist: xenial
|
69
|
+
|
70
|
+
services:
|
71
|
+
- mysql
|
72
|
+
- postgresql
|
70
73
|
|
71
74
|
sudo: required
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,69 @@
|
|
1
|
+
## Changes in 1.0.7
|
2
|
+
|
3
|
+
* Use @@max_allowed_packet session variable instead of querying SHOW VARIABLES. Thanks to @diclophis via \#706.
|
4
|
+
* Add option :track_validation_failures. When this is set to true, failed_instances will be an array of arrays, with each inner array having the form [:index_in_dataset, :object_with_errors]. Thanks to @rorymckinley via \#684.
|
5
|
+
|
6
|
+
### Fixes
|
7
|
+
|
8
|
+
* Prevent mass-assignment errors in Rails strict mode. Thanks to @diclophis via \##709.
|
9
|
+
|
10
|
+
## Changes in 1.0.6
|
11
|
+
|
12
|
+
### Fixes
|
13
|
+
|
14
|
+
* Handle after_initialize callbacks. Thanks to @AhMohsen46 via \#691 and
|
15
|
+
\#692.
|
16
|
+
* Fix regression introduced in 1.0.4. Explicity allow adapters to
|
17
|
+
support on duplicate key update. Thanks to @dsobiera, @jkowens via \#696.
|
18
|
+
|
19
|
+
## Changes in 1.0.5
|
20
|
+
|
21
|
+
### Fixes
|
22
|
+
|
23
|
+
* Allow serialized attributes to be returned from import. Thanks to @timanovsky, @jkowens via \#660.
|
24
|
+
* Return ActiveRecord::Connection from
|
25
|
+
ActiveREcord::Base#establish_connection. Thanks to @reverentF via
|
26
|
+
\#663.
|
27
|
+
* Support PostgreSQL array. Thanks to @ujihisa via \#669.
|
28
|
+
* Skip loading association ids when column changed. Thanks to @Aristat
|
29
|
+
via \#673.
|
30
|
+
|
31
|
+
## Changes in 1.0.4
|
32
|
+
|
33
|
+
### Fixes
|
34
|
+
|
35
|
+
* Use prepend pattern for ActiveRecord::Base#establish_connection patching. Thanks to @dombesz via \#648.
|
36
|
+
* Fix NoMethodError when using PostgreSQL ENUM types. Thanks to @sebcoetzee via \#651.
|
37
|
+
* Fix issue updating optimistic lock in Postgres. Thanks to @timanovsky
|
38
|
+
via \#656.
|
39
|
+
|
40
|
+
## Changes in 1.0.3
|
41
|
+
|
42
|
+
### New Features
|
43
|
+
|
44
|
+
* Add support for ActiveRecord 6.1.0.alpha. Thanks to @imtayadeway via
|
45
|
+
\#642.
|
46
|
+
|
47
|
+
### Fixes
|
48
|
+
|
49
|
+
* Return an empty array for results instead of nil when importing empty
|
50
|
+
array. Thanks to @gyfis via \#636.
|
51
|
+
|
52
|
+
## Changes in 1.0.2
|
53
|
+
|
54
|
+
### New Features
|
55
|
+
|
56
|
+
* Add support for CockroachDB adapter. Thanks to @willie via \#605.
|
57
|
+
* Add support for ActiveRecord 6.0.0.rc1. Thanks to @madeindjs, @bill-filler,
|
58
|
+
@jkowens via \#619, \#623.
|
59
|
+
|
60
|
+
### Fixes
|
61
|
+
|
62
|
+
* Fixes NoMethodError when attempting to use nil logger. Thanks to @MattMecel,
|
63
|
+
@khiav22357.
|
64
|
+
* Fix issue validating STI models. Thanks to @thejbsmith, @jkowens via
|
65
|
+
\#626.
|
66
|
+
|
1
67
|
## Changes in 1.0.1
|
2
68
|
|
3
69
|
### Fixes
|
@@ -226,7 +292,7 @@
|
|
226
292
|
Thanks to @jkowens via \#301.
|
227
293
|
* Allow for custom timestamp columns. Thanks to @mojidabckuu, @jkowens
|
228
294
|
via \#401.
|
229
|
-
|
295
|
+
|
230
296
|
### Fixes
|
231
297
|
|
232
298
|
* Fix ActiveRecord 5 issue coercing boolean values when serializing
|
@@ -238,7 +304,7 @@
|
|
238
304
|
|
239
305
|
* Fix issue where PostgreSQL cannot recognize columns if names
|
240
306
|
include mixed case characters. Thanks to @hugobgranja via \#379.
|
241
|
-
* Fix an issue for ActiveRecord 5 where serialized fields with
|
307
|
+
* Fix an issue for ActiveRecord 5 where serialized fields with
|
242
308
|
default values were not being typecast. Thanks to @whistlerbrk,
|
243
309
|
@jkowens via \#386.
|
244
310
|
* Add option :force_single_insert for MySQL to make sure a single
|
data/Gemfile
CHANGED
@@ -6,6 +6,8 @@ version = ENV['AR_VERSION'].to_f
|
|
6
6
|
|
7
7
|
mysql2_version = '0.3.0'
|
8
8
|
mysql2_version = '0.4.0' if version >= 4.2
|
9
|
+
sqlite3_version = '1.3.0'
|
10
|
+
sqlite3_version = '1.4.0' if version >= 6.0
|
9
11
|
|
10
12
|
group :development, :test do
|
11
13
|
gem 'rubocop', '~> 0.40.0'
|
@@ -16,7 +18,7 @@ end
|
|
16
18
|
platforms :ruby do
|
17
19
|
gem "mysql2", "~> #{mysql2_version}"
|
18
20
|
gem "pg", "~> 0.9"
|
19
|
-
gem "sqlite3", "~>
|
21
|
+
gem "sqlite3", "~> #{sqlite3_version}"
|
20
22
|
gem "seamless_database_pool", "~> 1.0.20"
|
21
23
|
end
|
22
24
|
|
@@ -45,6 +47,7 @@ end
|
|
45
47
|
|
46
48
|
platforms :ruby do
|
47
49
|
gem "pry-byebug"
|
50
|
+
gem "pry", "~> 0.12.0"
|
48
51
|
gem "rb-readline"
|
49
52
|
end
|
50
53
|
|
data/LICENSE
CHANGED
@@ -1,56 +1,21 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
d) make other distribution arrangements with the author.
|
24
|
-
|
25
|
-
3. You may distribute the software in object code or binary form,
|
26
|
-
provided that you do at least ONE of the following:
|
27
|
-
|
28
|
-
a) distribute the binaries and library files of the software,
|
29
|
-
together with instructions (in the manual page or equivalent)
|
30
|
-
on where to get the original distribution.
|
31
|
-
|
32
|
-
b) accompany the distribution with the machine-readable source of
|
33
|
-
the software.
|
34
|
-
|
35
|
-
c) give non-standard binaries non-standard names, with
|
36
|
-
instructions on where to get the original software distribution.
|
37
|
-
|
38
|
-
d) make other distribution arrangements with the author.
|
39
|
-
|
40
|
-
4. You may modify and include the part of the software into any other
|
41
|
-
software (possibly commercial). But some files in the distribution
|
42
|
-
are not written by the author, so that they are not under these terms.
|
43
|
-
|
44
|
-
For the list of those files and their copying conditions, see the
|
45
|
-
file LEGAL.
|
46
|
-
|
47
|
-
5. The scripts and library files supplied as input to or produced as
|
48
|
-
output from the software do not automatically fall under the
|
49
|
-
copyright of the software, but belong to whomever generated them,
|
50
|
-
and may be sold commercially, and may be aggregated with this
|
51
|
-
software.
|
52
|
-
|
53
|
-
6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
|
54
|
-
IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
|
55
|
-
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
56
|
-
PURPOSE.
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2020 Zach Dennis <zach.dennis@gmail.com>
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.markdown
CHANGED
@@ -1,6 +1,6 @@
|
|
1
|
-
#
|
1
|
+
# Activerecord-Import [![Build Status](https://travis-ci.org/zdennis/activerecord-import.svg?branch=master)](https://travis-ci.org/zdennis/activerecord-import)
|
2
2
|
|
3
|
-
|
3
|
+
Activerecord-Import is a library for bulk inserting data using ActiveRecord.
|
4
4
|
|
5
5
|
One of its major features is following activerecord associations and generating the minimal
|
6
6
|
number of SQL insert statements required, avoiding the N+1 insert problem. An example probably
|
@@ -23,10 +23,10 @@ an 18 hour batch process to <2 hrs.
|
|
23
23
|
|
24
24
|
The gem provides the following high-level features:
|
25
25
|
|
26
|
-
*
|
27
|
-
*
|
28
|
-
*
|
29
|
-
*
|
26
|
+
* Works with raw columns and arrays of values (fastest)
|
27
|
+
* Works with model objects (faster)
|
28
|
+
* Performs validations (fast)
|
29
|
+
* Performs on duplicate key updates (requires MySQL, SQLite 3.24.0+, or Postgres 9.5+)
|
30
30
|
|
31
31
|
## Table of Contents
|
32
32
|
|
@@ -54,16 +54,19 @@ The gem provides the following high-level features:
|
|
54
54
|
* [More Information](#more-information)
|
55
55
|
* [Contributing](#contributing)
|
56
56
|
* [Running Tests](#running-tests)
|
57
|
+
* [Issue Triage](#issue-triage)
|
57
58
|
|
58
59
|
### Examples
|
59
60
|
|
60
61
|
#### Introduction
|
61
62
|
|
63
|
+
This gem adds an `import` method (or `bulk_import`, for compatibility with gems like `elasticsearch-model`; see [Conflicts With Other Gems](#conflicts-with-other-gems)) to ActiveRecord classes.
|
64
|
+
|
62
65
|
Without `activerecord-import`, you'd write something like this:
|
63
66
|
|
64
67
|
```ruby
|
65
68
|
10.times do |i|
|
66
|
-
Book.create! :
|
69
|
+
Book.create! name: "book #{i}"
|
67
70
|
end
|
68
71
|
```
|
69
72
|
|
@@ -72,7 +75,7 @@ This would end up making 10 SQL calls. YUCK! With `activerecord-import`, you ca
|
|
72
75
|
```ruby
|
73
76
|
books = []
|
74
77
|
10.times do |i|
|
75
|
-
books << Book.new(:
|
78
|
+
books << Book.new(name: "book #{i}")
|
76
79
|
end
|
77
80
|
Book.import books # or use import!
|
78
81
|
```
|
@@ -85,13 +88,13 @@ The `import` method can take an array of column names (string or symbols) and an
|
|
85
88
|
|
86
89
|
```ruby
|
87
90
|
columns = [ :title, :author ]
|
88
|
-
values = [ ['Book1', '
|
91
|
+
values = [ ['Book1', 'George Orwell'], ['Book2', 'Bob Jones'] ]
|
89
92
|
|
90
93
|
# Importing without model validations
|
91
|
-
Book.import columns, values, :
|
94
|
+
Book.import columns, values, validate: false
|
92
95
|
|
93
96
|
# Import with model validations
|
94
|
-
Book.import columns, values, :
|
97
|
+
Book.import columns, values, validate: true
|
95
98
|
|
96
99
|
# when not specified :validate defaults to true
|
97
100
|
Book.import columns, values
|
@@ -102,7 +105,7 @@ Book.import columns, values
|
|
102
105
|
The `import` method can take an array of hashes. The keys map to the column names in the database.
|
103
106
|
|
104
107
|
```ruby
|
105
|
-
values = [{ title: 'Book1', author: '
|
108
|
+
values = [{ title: 'Book1', author: 'George Orwell' }, { title: 'Book2', author: 'Bob Jones'}]
|
106
109
|
|
107
110
|
# Importing without model validations
|
108
111
|
Book.import values, validate: false
|
@@ -119,7 +122,7 @@ The `import` method can take an array of column names and an array of hash objec
|
|
119
122
|
|
120
123
|
```ruby
|
121
124
|
books = [
|
122
|
-
{ title: "Book 1", author: "
|
125
|
+
{ title: "Book 1", author: "George Orwell" },
|
123
126
|
{ title: "Book 2", author: "Bob Jones" }
|
124
127
|
]
|
125
128
|
columns = [ :title ]
|
@@ -171,15 +174,15 @@ The `import` method can take an array of models. The attributes will be pulled o
|
|
171
174
|
|
172
175
|
```ruby
|
173
176
|
books = [
|
174
|
-
Book.new(:
|
175
|
-
Book.new(:
|
177
|
+
Book.new(title: "Book 1", author: "George Orwell"),
|
178
|
+
Book.new(title: "Book 2", author: "Bob Jones")
|
176
179
|
]
|
177
180
|
|
178
181
|
# without validations
|
179
|
-
Book.import books, :
|
182
|
+
Book.import books, validate: false
|
180
183
|
|
181
184
|
# with validations
|
182
|
-
Book.import books, :
|
185
|
+
Book.import books, validate: true
|
183
186
|
|
184
187
|
# when not specified :validate defaults to true
|
185
188
|
Book.import books
|
@@ -189,16 +192,16 @@ The `import` method can take an array of column names and an array of models. Th
|
|
189
192
|
|
190
193
|
```ruby
|
191
194
|
books = [
|
192
|
-
Book.new(:
|
193
|
-
Book.new(:
|
195
|
+
Book.new(title: "Book 1", author: "George Orwell"),
|
196
|
+
Book.new(title: "Book 2", author: "Bob Jones")
|
194
197
|
]
|
195
198
|
columns = [ :title ]
|
196
199
|
|
197
200
|
# without validations
|
198
|
-
Book.import columns, books, :
|
201
|
+
Book.import columns, books, validate: false
|
199
202
|
|
200
203
|
# with validations
|
201
|
-
Book.import columns, books, :
|
204
|
+
Book.import columns, books, validate: true
|
202
205
|
|
203
206
|
# when not specified :validate defaults to true
|
204
207
|
Book.import columns, books
|
@@ -217,28 +220,29 @@ The `import` method can take a `batch_size` option to control the number of rows
|
|
217
220
|
|
218
221
|
```ruby
|
219
222
|
books = [
|
220
|
-
Book.new(:
|
221
|
-
Book.new(:
|
222
|
-
Book.new(:
|
223
|
-
Book.new(:
|
223
|
+
Book.new(title: "Book 1", author: "George Orwell"),
|
224
|
+
Book.new(title: "Book 2", author: "Bob Jones"),
|
225
|
+
Book.new(title: "Book 1", author: "John Doe"),
|
226
|
+
Book.new(title: "Book 2", author: "Richard Wright")
|
224
227
|
]
|
225
228
|
columns = [ :title ]
|
226
229
|
|
227
230
|
# 2 INSERT statements for 4 records
|
228
|
-
Book.import columns, books, :
|
231
|
+
Book.import columns, books, batch_size: 2
|
229
232
|
```
|
230
233
|
|
231
234
|
#### Recursive
|
232
235
|
|
233
|
-
NOTE: This only works with PostgreSQL.
|
236
|
+
NOTE: This only works with PostgreSQL and ActiveRecord objects. This won't work with
|
237
|
+
hashes or arrays as recursive inputs.
|
234
238
|
|
235
239
|
Assume that Books <code>has_many</code> Reviews.
|
236
240
|
|
237
241
|
```ruby
|
238
242
|
books = []
|
239
243
|
10.times do |i|
|
240
|
-
book = Book.new(:
|
241
|
-
book.reviews.build(:
|
244
|
+
book = Book.new(name: "book #{i}")
|
245
|
+
book.reviews.build(title: "Excellent")
|
242
246
|
books << book
|
243
247
|
end
|
244
248
|
Book.import books, recursive: true
|
@@ -246,19 +250,21 @@ Book.import books, recursive: true
|
|
246
250
|
|
247
251
|
### Options
|
248
252
|
|
249
|
-
Key
|
250
|
-
|
251
|
-
:validate
|
252
|
-
:validate_uniqueness
|
253
|
-
:
|
254
|
-
:
|
255
|
-
:
|
256
|
-
:
|
257
|
-
:
|
258
|
-
:
|
259
|
-
:
|
260
|
-
:
|
261
|
-
|
253
|
+
Key | Options | Default | Description
|
254
|
+
------------------------- | --------------------- | ------------------ | -----------
|
255
|
+
:validate | `true`/`false` | `true` | Whether or not to run `ActiveRecord` validations (uniqueness skipped). This option will always be true when using `import!`.
|
256
|
+
:validate_uniqueness | `true`/`false` | `false` | Whether or not to run uniqueness validations, has potential pitfalls, use with caution (requires `>= v0.27.0`).
|
257
|
+
:validate_with_context | `Symbol` |`:create`/`:update` | Allows passing an ActiveModel validation context for each model. Default is `:create` for new records and `:update` for existing ones.
|
258
|
+
:track_validation_failures| `true`/`false` | `false` | When this is set to true, `failed_instances` will be an array of arrays, with each inner array having the form `[:index_in_dataset, :object_with_errors]`
|
259
|
+
:on_duplicate_key_ignore | `true`/`false` | `false` | Allows skipping records with duplicate keys. See [here](https://github.com/zdennis/activerecord-import/#duplicate-key-ignore) for more details.
|
260
|
+
:ignore | `true`/`false` | `false` | Alias for :on_duplicate_key_ignore.
|
261
|
+
:on_duplicate_key_update | :all, `Array`, `Hash` | N/A | Allows upsert logic to be used. See [here](https://github.com/zdennis/activerecord-import/#duplicate-key-update) for more details.
|
262
|
+
:synchronize | `Array` | N/A | An array of ActiveRecord instances. This synchronizes existing instances in memory with updates from the import.
|
263
|
+
:timestamps | `true`/`false` | `true` | Enables/disables timestamps on imported records.
|
264
|
+
:recursive | `true`/`false` | `false` | Imports has_many/has_one associations (PostgreSQL only).
|
265
|
+
:batch_size | `Integer` | total # of records | Max number of records to insert per import
|
266
|
+
:raise_error | `true`/`false` | `false` | Raises an exception at the first invalid record. This means there will not be a result object returned. The `import!` method is a shortcut for this.
|
267
|
+
:all_or_none | `true`/`false` | `false` | Will not import any records if there is a record with validation errors.
|
262
268
|
|
263
269
|
#### Duplicate Key Ignore
|
264
270
|
|
@@ -267,14 +273,14 @@ Key | Options | Default | Descripti
|
|
267
273
|
For Postgres 9.5+ it adds `ON CONFLICT DO NOTHING`, for MySQL it uses `INSERT IGNORE`, and for SQLite it uses `INSERT OR IGNORE`. Cannot be enabled on a recursive import. For database adapters that normally support setting primary keys on imported objects, this option prevents that from occurring.
|
268
274
|
|
269
275
|
```ruby
|
270
|
-
book = Book.create! title: "Book1", author: "
|
276
|
+
book = Book.create! title: "Book1", author: "George Orwell"
|
271
277
|
book.title = "Updated Book Title"
|
272
278
|
book.author = "Bob Barker"
|
273
279
|
|
274
280
|
Book.import [book], on_duplicate_key_ignore: true
|
275
281
|
|
276
282
|
book.reload.title # => "Book1" (stayed the same)
|
277
|
-
book.reload.author # => "
|
283
|
+
book.reload.author # => "George Orwell" (stayed the same)
|
278
284
|
```
|
279
285
|
|
280
286
|
The option `:on_duplicate_key_ignore` is bypassed when `:recursive` is enabled for [PostgreSQL imports](https://github.com/zdennis/activerecord-import/wiki#recursive-example-postgresql-only).
|
@@ -290,7 +296,7 @@ This will use MySQL's `ON DUPLICATE KEY UPDATE` or Postgres/SQLite `ON CONFLICT
|
|
290
296
|
Basic Update
|
291
297
|
|
292
298
|
```ruby
|
293
|
-
book = Book.create! title: "Book1", author: "
|
299
|
+
book = Book.create! title: "Book1", author: "George Orwell"
|
294
300
|
book.title = "Updated Book Title"
|
295
301
|
book.author = "Bob Barker"
|
296
302
|
|
@@ -304,13 +310,13 @@ Book.import [book], on_duplicate_key_update: {conflict_target: [:id], columns: [
|
|
304
310
|
Book.import [book], on_duplicate_key_update: [:title]
|
305
311
|
|
306
312
|
book.reload.title # => "Updated Book Title" (changed)
|
307
|
-
book.reload.author # => "
|
313
|
+
book.reload.author # => "George Orwell" (stayed the same)
|
308
314
|
```
|
309
315
|
|
310
316
|
Using the value from another column
|
311
317
|
|
312
318
|
```ruby
|
313
|
-
book = Book.create! title: "Book1", author: "
|
319
|
+
book = Book.create! title: "Book1", author: "George Orwell"
|
314
320
|
book.title = "Updated Book Title"
|
315
321
|
|
316
322
|
# MySQL version
|
@@ -328,7 +334,7 @@ book.reload.author # => "Updated Book Title" (changed)
|
|
328
334
|
Using Custom SQL
|
329
335
|
|
330
336
|
```ruby
|
331
|
-
book = Book.create! title: "Book1", author: "
|
337
|
+
book = Book.create! title: "Book1", author: "George Orwell"
|
332
338
|
book.author = "Bob Barker"
|
333
339
|
|
334
340
|
# MySQL version
|
@@ -349,7 +355,7 @@ book.reload.author # => "Bob Barker" (changed)
|
|
349
355
|
PostgreSQL Using constraints
|
350
356
|
|
351
357
|
```ruby
|
352
|
-
book = Book.create! title: "Book1", author: "
|
358
|
+
book = Book.create! title: "Book1", author: "George Orwell", edition: 3, published_at: nil
|
353
359
|
book.published_at = Time.now
|
354
360
|
|
355
361
|
# in migration
|
@@ -363,7 +369,7 @@ Book.import [book], on_duplicate_key_update: {constraint_name: :for_upsert, colu
|
|
363
369
|
|
364
370
|
|
365
371
|
book.reload.title # => "Book1" (stayed the same)
|
366
|
-
book.reload.author # => "
|
372
|
+
book.reload.author # => "George Orwell" (stayed the same)
|
367
373
|
book.reload.edition # => 3 (stayed the same)
|
368
374
|
book.reload.published_at # => 2017-10-09 (changed)
|
369
375
|
```
|
@@ -383,7 +389,7 @@ articles = [
|
|
383
389
|
Article.new(author_id: 3, content: '')
|
384
390
|
]
|
385
391
|
|
386
|
-
demo = Article.import(articles
|
392
|
+
demo = Article.import(articles, returning: :title) # => #<struct ActiveRecord::Import::Result
|
387
393
|
|
388
394
|
demo.failed_instances
|
389
395
|
=> [#<Article id: 3, author_id: 3, title: nil, content: "", created_at: nil, updated_at: nil>]
|
@@ -397,7 +403,7 @@ demo.ids
|
|
397
403
|
|
398
404
|
demo.results
|
399
405
|
=> ["First Article", "Second Article"] # for Postgres
|
400
|
-
=> [] for other DBs
|
406
|
+
=> [] # for other DBs
|
401
407
|
```
|
402
408
|
|
403
409
|
### Counter Cache
|
@@ -524,7 +530,7 @@ require 'activerecord-import'
|
|
524
530
|
### Load Path Setup
|
525
531
|
To understand how rubygems loads code you can reference the following:
|
526
532
|
|
527
|
-
http://guides.rubygems.org/patterns/#
|
533
|
+
http://guides.rubygems.org/patterns/#loading-code
|
528
534
|
|
529
535
|
And an example of how active_record dynamically load adapters:
|
530
536
|
|
@@ -552,7 +558,7 @@ When rubygems pushes the `lib` folder onto the load path a `require` will now fi
|
|
552
558
|
|
553
559
|
### Conflicts With Other Gems
|
554
560
|
|
555
|
-
|
561
|
+
Activerecord-Import adds the `.import` method onto `ActiveRecord::Base`. There are other gems, such as `elasticsearch-rails`, that do the same thing. In conflicts such as this, there is an aliased method named `.bulk_import` that can be used interchangeably.
|
556
562
|
|
557
563
|
If you are using the `apartment` gem, there is a weird triple interaction between that gem, `activerecord-import`, and `activerecord` involving caching of the `sequence_name` of a model. This can be worked around by explcitly setting this value within the model. For example:
|
558
564
|
|
@@ -579,7 +585,7 @@ See https://github.com/zdennis/activerecord-import/issues/233 for further discus
|
|
579
585
|
|
580
586
|
### More Information
|
581
587
|
|
582
|
-
For more information on
|
588
|
+
For more information on Activerecord-Import please see its wiki: https://github.com/zdennis/activerecord-import/wiki
|
583
589
|
|
584
590
|
To document new information, please add to the README instead of the wiki. See https://github.com/zdennis/activerecord-import/issues/397 for discussion.
|
585
591
|
|
@@ -605,24 +611,14 @@ AR_VERSION=4.2 bundle exec rake test:postgresql test:sqlite3 test:mysql2
|
|
605
611
|
|
606
612
|
Once you have pushed up your changes, you can find your CI results [here](https://travis-ci.org/zdennis/activerecord-import/).
|
607
613
|
|
614
|
+
## Issue Triage [![Open Source Helpers](https://www.codetriage.com/zdennis/activerecord-import/badges/users.svg)](https://www.codetriage.com/zdennis/activerecord-import)
|
615
|
+
|
616
|
+
You can triage issues which may include reproducing bug reports or asking for vital information, such as version numbers or reproduction instructions. If you would like to start triaging issues, one easy way to get started is to [subscribe to activerecord-import on CodeTriage](https://www.codetriage.com/zdennis/activerecord-import).
|
617
|
+
|
608
618
|
# License
|
609
619
|
|
610
|
-
This is licensed under the
|
620
|
+
This is licensed under the MIT license.
|
611
621
|
|
612
622
|
# Author
|
613
623
|
|
614
624
|
Zach Dennis (zach.dennis@gmail.com)
|
615
|
-
|
616
|
-
# Contributors
|
617
|
-
|
618
|
-
* Jordan Owens (@jkowens)
|
619
|
-
* Erik Michaels-Ober (@sferik)
|
620
|
-
* Blythe Dunham
|
621
|
-
* Gabe da Silveira
|
622
|
-
* Henry Work
|
623
|
-
* James Herdman
|
624
|
-
* Marcus Crafter
|
625
|
-
* Thibaud Guillaume-Gentil
|
626
|
-
* Mark Van Holstyn
|
627
|
-
* Victor Costan
|
628
|
-
* Dillon Welch
|