rails_admin_import 2.0.0 → 2.3.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/CHANGELOG.md +25 -0
- data/README.md +162 -19
- data/app/views/rails_admin/main/import.html.haml +18 -4
- data/config/locales/README.md +3 -0
- data/config/locales/import.en.yml +47 -45
- data/lib/rails_admin_import/config/sections/import.rb +1 -1
- data/lib/rails_admin_import/config.rb +6 -0
- data/lib/rails_admin_import/formats/csv_importer.rb +8 -5
- data/lib/rails_admin_import/formats/json_importer.rb +1 -0
- data/lib/rails_admin_import/formats/xlsx_importer.rb +4 -2
- data/lib/rails_admin_import/import_logger.rb +1 -1
- data/lib/rails_admin_import/importer.rb +33 -7
- data/lib/rails_admin_import/version.rb +1 -1
- metadata +12 -19
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: fed24bf8ccc152d3b45b68e56600ba77160a707794228cd8828ce7e289003056
|
4
|
+
data.tar.gz: 3135d1461ba3b1e6ac4a3eb0c13cc85c1c3cda432aeaad1759bfa5e49114bdeb
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 669291db9c194aaaf47a5f7f75e5d6fb252615a5d0e01b87160f1afc3bbf8150ed8f223fcfbb33629f78ae47c2cb0034bbf8548f4fa80cec30c92d25eb31eac5
|
7
|
+
data.tar.gz: 297247a8340a59c5364b6ab01208eaad4275aea4f2290362037117010c250063e2748a6d1fb9bfc3583ec89cf4d645597ec45acf57da5b07add65471832e0244
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,30 @@
|
|
1
1
|
# Change Log
|
2
2
|
|
3
|
+
# 2.3.1 / 2022-01-29
|
4
|
+
|
5
|
+
- Mark compatibility to Rails Admin < 3.0
|
6
|
+
- Bump Nokogiri in tests
|
7
|
+
|
8
|
+
# 2.3.0 / 2021-06-11
|
9
|
+
|
10
|
+
- Added an option to pass filename for each record. Thanks @waheedi
|
11
|
+
- Enumeration translation. Thanks @zaknafain
|
12
|
+
|
13
|
+
# 2.2.0 / 2019-02-24
|
14
|
+
|
15
|
+
- Switch to using CharlockHolmes for character detection. Thanks @codealchemy
|
16
|
+
- Readme improvements. Thanks @olleolleolle
|
17
|
+
|
18
|
+
# 2.1.0 / 2017-11-18
|
19
|
+
|
20
|
+
- French translation. Thanks @rodinux
|
21
|
+
- Check for blank headers. Thanks @JuandGirald
|
22
|
+
- Italian translation. Thanks @aprofiti
|
23
|
+
- Remove haml dependency. Thanks @prem-prakash
|
24
|
+
- Japanese translation. Thanks @higumachan
|
25
|
+
- Multiple mapping keys. Thanks @dmitrypol
|
26
|
+
- Add more hooks during import
|
27
|
+
|
3
28
|
# 2.0.0 / 2016-05-04
|
4
29
|
|
5
30
|
- Pull in the encoding list from the Encoding module instead of RailsAdmin. Thanks @baldursson, @patricklewis and @lucasff
|
data/README.md
CHANGED
@@ -1,17 +1,15 @@
|
|
1
1
|
# Rails Admin Import
|
2
2
|
|
3
|
-
[![Build Status](https://
|
3
|
+
[![Build Status](https://github.com/monkbroc/rails_admin_import/actions/workflows/ruby.yml/badge.svg)](https://github.com/monkbroc/rails_admin_import/actions/workflows/ruby.yml)
|
4
4
|
|
5
5
|
Plugin functionality to add generic import to Rails Admin from CSV, JSON and XLSX files
|
6
6
|
|
7
|
-
*This Readme is for version 1.x. If you are still using version 0.1.x, see [this branch](https://github.com/stephskardal/rails_admin_import/tree/legacy)*
|
8
|
-
|
9
7
|
## Installation
|
10
8
|
|
11
9
|
* First, add to Gemfile:
|
12
10
|
|
13
|
-
```
|
14
|
-
gem "rails_admin_import", "~>
|
11
|
+
```ruby
|
12
|
+
gem "rails_admin_import", "~> 2.3"
|
15
13
|
```
|
16
14
|
|
17
15
|
* Define configuration in `config/initializers/rails_admin_import.rb`:
|
@@ -28,8 +26,10 @@ RailsAdmin.config do |config|
|
|
28
26
|
|
29
27
|
# Optional:
|
30
28
|
# Configure global RailsAdminImport options
|
29
|
+
# Configure pass filename to records hashes
|
31
30
|
config.configure_with(:import) do |config|
|
32
31
|
config.logging = true
|
32
|
+
config.pass_filename = true
|
33
33
|
end
|
34
34
|
|
35
35
|
# Optional:
|
@@ -62,9 +62,10 @@ Both updating existing records and associating records requires the use of
|
|
62
62
|
### Mapping Keys
|
63
63
|
|
64
64
|
Every importable class has a mapping key that uniquely identifies its
|
65
|
-
instances. The
|
66
|
-
|
67
|
-
|
65
|
+
instances. The mapping key can be one or more fields. The value for
|
66
|
+
these fields can then be provided in import data, either to update the
|
67
|
+
existing record or to attach it through an association to another model.
|
68
|
+
This concept exists because `id`s are often not constant when moving
|
68
69
|
records between data stores.
|
69
70
|
|
70
71
|
For example, a `User` model may have an `email` field. When uploading a set
|
@@ -73,14 +74,19 @@ our mapping key and then provide that field on each record in our data,
|
|
73
74
|
allowing us to update existing records with matching emails.
|
74
75
|
|
75
76
|
Using a csv formatted example:
|
77
|
+
|
76
78
|
```
|
77
79
|
Email,First name,Last name
|
78
80
|
peter.gibbons@initech.com,Peter,Gibbons
|
79
81
|
michael.bolton@initech.com,Michael,Bolton
|
80
82
|
```
|
83
|
+
|
81
84
|
would look for existing users with those emails. If one was found, its name
|
82
85
|
fields would be updated. Otherwise, a new one would be created.
|
83
86
|
|
87
|
+
For updating building owners, the mapping key could be `street_address` and
|
88
|
+
`zip_code`.
|
89
|
+
|
84
90
|
Similarly, if each user has favorite books, we could set the mapping key
|
85
91
|
for `Book` to be `isbn` and then include the isbn for their books within each
|
86
92
|
user record. The syntax for this is to use the name of the associated model as
|
@@ -89,28 +95,36 @@ a user record would have one or more fields named "Book" that include each
|
|
89
95
|
associated book's ISBN.
|
90
96
|
|
91
97
|
Again using a csv formatted example:
|
98
|
+
|
92
99
|
```
|
93
100
|
Email, Book, Book, Book
|
94
101
|
peter.gibbons@initech.com, 9781119997870, 9780671027032
|
95
102
|
michael.bolton@initech.com, 9780446677479
|
96
103
|
```
|
104
|
+
|
97
105
|
would look up books with those ISBNs and attach them to those users.
|
98
106
|
|
99
107
|
Mapping keys can be selected on the import page. Their defaults can also be
|
100
108
|
globally configured in the config file:
|
101
109
|
|
102
|
-
```
|
110
|
+
```ruby
|
103
111
|
RailsAdmin.config do |config|
|
104
112
|
config.model 'User' do
|
105
113
|
import do
|
106
114
|
mapping_key :email
|
115
|
+
# for multiple values, use mapping_key [:first_name, :last_name]
|
116
|
+
mapping_key_list [:email, :some_other_id]
|
107
117
|
end
|
108
118
|
end
|
109
119
|
end
|
110
120
|
```
|
111
121
|
|
122
|
+
Since in models with large number of fields it doesn't make sense to use
|
123
|
+
most of them as mapping values, you can add `mapping_key_list` to
|
124
|
+
restrict which fields can be selected as mapping key in the UI during import.
|
125
|
+
|
112
126
|
Note that a matched record must exist when attaching associated models, or the
|
113
|
-
|
127
|
+
imported record will fail and be skipped.
|
114
128
|
|
115
129
|
Complex associations (`has_one ..., :through` or polymorphic associations)
|
116
130
|
need to be dealt with via custom logic called by one of the import hooks
|
@@ -120,7 +134,7 @@ existed through an intermediary model called `ServiceProvider`, we could
|
|
120
134
|
provide a `user_email` field in our records and handle the actual
|
121
135
|
association with an import hook:
|
122
136
|
|
123
|
-
```
|
137
|
+
```ruby
|
124
138
|
class Service < ActiveRecord::Base
|
125
139
|
belongs_to :service_provider
|
126
140
|
has_one :user, through: :service_provider
|
@@ -133,6 +147,8 @@ class Service < ActiveRecord::Base
|
|
133
147
|
end
|
134
148
|
```
|
135
149
|
|
150
|
+
Importing new records by id is not recommended since it ignores the sequences of ids in database. That will lead to `ERROR: duplicate key value violates unique constraint` in future. You can work around this issue by adding an `import_id` column to your model, renaming the `id` column in your CSV to `import_id` and using `import_id` as the update lookup field.
|
151
|
+
|
136
152
|
### File format
|
137
153
|
|
138
154
|
The format is inferred by the extension (.csv, .json or .xlsx).
|
@@ -153,6 +169,8 @@ Peter,Gibbons,IT,Management
|
|
153
169
|
Michael,Bolton,IT,
|
154
170
|
```
|
155
171
|
|
172
|
+
Blank lines will be skipped.
|
173
|
+
|
156
174
|
#### JSON
|
157
175
|
|
158
176
|
The file must be an array or an object with a root key the same name as the plural model name, i.e. the default Rails JSON output format with include_root_in_json on or off.
|
@@ -163,19 +181,46 @@ The Microsoft Excel XLM format (XLSX) is supported, but not the old binary Micro
|
|
163
181
|
|
164
182
|
The expected rows and columns are the same as for the CSV format (first line contains headers, multiple columns for "many" associations).
|
165
183
|
|
184
|
+
Blank lines will be skipped.
|
185
|
+
|
166
186
|
## Configuration
|
167
187
|
|
168
188
|
### Global configuration options
|
169
189
|
|
190
|
+
```ruby
|
191
|
+
RailsAdmin.config do |config|
|
192
|
+
config.actions do
|
193
|
+
all
|
194
|
+
import
|
195
|
+
end
|
196
|
+
|
197
|
+
# Default global RailsAdminImport options
|
198
|
+
config.configure_with(:import) do |config|
|
199
|
+
config.logging = false
|
200
|
+
config.line_item_limit = 1000
|
201
|
+
config.update_if_exists = false
|
202
|
+
config.pass_filename = false
|
203
|
+
config.rollback_on_error = false
|
204
|
+
config.header_converter = lambda do |header|
|
205
|
+
header.parameterize.underscore if header.present?
|
206
|
+
end
|
207
|
+
config.csv_options = {}
|
208
|
+
end
|
209
|
+
end
|
210
|
+
```
|
211
|
+
|
170
212
|
* __logging__ (default `false`): Save a copy of each imported file to log/import and a detailed import log to log/rails_admin_import.log
|
171
213
|
|
172
214
|
* __line_item_limit__ (default `1000`): max number of items that can be imported at one time.
|
173
215
|
|
216
|
+
* __update_if_exists__ (default `false`): default value for the "Update if exists" checkbox on the import page.
|
217
|
+
|
174
218
|
* __rollback_on_error__ (default `false`): import records in a transaction and rollback if there is one error. Only for ActiveRecord, not Mongoid.
|
175
219
|
|
176
220
|
* __header_converter__ (default `lambda { ... }`): a lambda to convert each CSV header text string to a model attribute name. The default header converter converts to lowercase and replaces spaces with underscores.
|
177
221
|
|
178
222
|
* __csv_options__ (default `{}`): a hash of options that will be passed to a new [CSV](http://ruby-doc.org/stdlib-2.0.0/libdoc/csv/rdoc/CSV.html) instance
|
223
|
+
* __pass_filename__ (default `false`): Access the uploaded file name in your model actions, for example if set to true, inside each record, there will be an addtional property `record[:filename_importer]` which contains the file name of the currently uploaded file.
|
179
224
|
|
180
225
|
### Model-specific configuration
|
181
226
|
|
@@ -239,7 +284,7 @@ end
|
|
239
284
|
RailsAdmin.config do |config|
|
240
285
|
config.model 'User' do
|
241
286
|
import do
|
242
|
-
default_excluded_fields [:created_at, :updated_at]
|
287
|
+
default_excluded_fields [:created_at, :updated_at, :deleted_at, :c_at, :u_at]
|
243
288
|
end
|
244
289
|
end
|
245
290
|
end
|
@@ -247,18 +292,56 @@ end
|
|
247
292
|
|
248
293
|
## Import hooks
|
249
294
|
|
250
|
-
|
251
|
-
Define instance methods on your models to be hooked into the import process, if special/additional processing is required on the data:
|
295
|
+
Define methods on your models to be hooked into the import process, if special/additional processing is required on the data:
|
252
296
|
|
253
297
|
```ruby
|
254
298
|
# some model
|
255
299
|
class User < ActiveRecord::Base
|
300
|
+
def self.before_import
|
301
|
+
# called on the model class once before importing any individual records
|
302
|
+
end
|
303
|
+
|
304
|
+
def self.before_import_find(record)
|
305
|
+
# called on the model class before finding or creating the new record
|
306
|
+
# maybe modify the import record that will be used to find the model
|
307
|
+
# throw :skip to skip importing this record
|
308
|
+
throw :skip unless record[:email].ends_with? "@mycompany.com"
|
309
|
+
end
|
310
|
+
|
311
|
+
def before_import_attributes(record)
|
312
|
+
# called on the blank new model or the found model before fields are imported
|
313
|
+
# maybe delete fields from the import record that you don't need
|
314
|
+
# throw :skip to skip importing this record
|
315
|
+
end
|
316
|
+
|
317
|
+
def before_import_associations(record)
|
318
|
+
# called on the model with attributes but before associations are imported
|
319
|
+
# do custom import of associations
|
320
|
+
# make sure to delete association fields from the import record to avoid double import
|
321
|
+
record.delete(:my_association)
|
322
|
+
# throw :skip to skip importing this record
|
323
|
+
end
|
324
|
+
|
256
325
|
def before_import_save(record)
|
257
|
-
#
|
326
|
+
# called on the model before it is saved but after all fields and associations have been imported
|
327
|
+
# make final modifications to the record
|
328
|
+
# throw :skip to skip importing this record
|
258
329
|
end
|
259
330
|
|
260
331
|
def after_import_save(record)
|
261
|
-
#
|
332
|
+
# called on the model after it is saved
|
333
|
+
end
|
334
|
+
|
335
|
+
def after_import_association_error(record)
|
336
|
+
# called on the model when an association cannot be found
|
337
|
+
end
|
338
|
+
|
339
|
+
def after_import_error(record)
|
340
|
+
# called on the model when save fails
|
341
|
+
end
|
342
|
+
|
343
|
+
def self.after_import
|
344
|
+
# called once on the model class after importing all individual records
|
262
345
|
end
|
263
346
|
end
|
264
347
|
```
|
@@ -269,12 +352,35 @@ For example, you could
|
|
269
352
|
|
270
353
|
* Import an image into Carrierwave via a URL provided in the CSV.
|
271
354
|
|
272
|
-
```
|
355
|
+
```ruby
|
273
356
|
def before_import_save(record)
|
274
357
|
self.remote_image_url = record[:image] if record[:image].present?
|
275
358
|
end
|
276
359
|
```
|
277
360
|
|
361
|
+
* Skip some validations when importing.
|
362
|
+
|
363
|
+
```ruby
|
364
|
+
class User < ActiveRecord::Base
|
365
|
+
# Non-persistent attribute to allow creating a new user without a password
|
366
|
+
# Password will be set by the user by following a link in the invitation email
|
367
|
+
attr_accessor :allow_blank_password
|
368
|
+
|
369
|
+
devise :validatable
|
370
|
+
|
371
|
+
# Called by Devise to enable/disable password presence validation
|
372
|
+
def password_required?
|
373
|
+
allow_blank_password ? false : super
|
374
|
+
end
|
375
|
+
|
376
|
+
# Don't require a password when importing users
|
377
|
+
def before_import_save(record)
|
378
|
+
self.allow_blank_password = true
|
379
|
+
end
|
380
|
+
end
|
381
|
+
```
|
382
|
+
|
383
|
+
|
278
384
|
## ORM: ActiveRecord and Mongoid
|
279
385
|
|
280
386
|
The gem is tested to work with ActiveRecord and Mongoid.
|
@@ -288,10 +394,32 @@ Since the import functionality is rarely used in many applications, some gems ar
|
|
288
394
|
|
289
395
|
If you prefer to eager load all dependecies at boot, use this line in your `Gemfile`.
|
290
396
|
|
291
|
-
```
|
397
|
+
```ruby
|
292
398
|
gem "rails_admin_import", "~> 1.2.0", require: "rails_admin_import/eager_load"
|
293
399
|
```
|
294
400
|
|
401
|
+
## Import error due to Rails class reloading
|
402
|
+
|
403
|
+
![error due to class reloading](https://user-images.githubusercontent.com/2566348/51355874-0f83ad00-1a7e-11e9-8e58-46bc4699f2e6.jpg)
|
404
|
+
|
405
|
+
If you get an error like `Error during import: MyModel(#70286054976500) expected, got MyModel(#70286114743280)`, you need restart the rails server and redo the import. This is due to the fact that Rails reloads the ActiveRecord model classes in development when you make changes to them and Rails Admin is still using the old class.
|
406
|
+
|
407
|
+
Another suggestion is to set `config.cache_classes = true` to true in your `development.rb` for Rails Admin Import to work around the ActiveRecord model class reloading issue. See [this comment](https://github.com/stephskardal/rails_admin_import/issues/88#issuecomment-455374671) for more information.
|
408
|
+
|
409
|
+
## Customize the UI
|
410
|
+
|
411
|
+
If you want to hide all the advanced fields from the import UI, you can copy [`app/views/rails_admin/main/import.html.haml`](app/views/rails_admin/main/import.html.haml) to your project at the same path. Add `.hidden` at the end of lines you want to hide.
|
412
|
+
|
413
|
+
For example:
|
414
|
+
|
415
|
+
```haml
|
416
|
+
.form-group.control-group.hidden
|
417
|
+
%label.col-sm-2.control-label= t("admin.import.update_if_exists")
|
418
|
+
.col-sm-10.controls
|
419
|
+
= check_box_tag :update_if_exists, '1', true, :class => "form-control"
|
420
|
+
%p.help-block= t('admin.import.help.update_if_exists')
|
421
|
+
```
|
422
|
+
|
295
423
|
## Upgrading
|
296
424
|
|
297
425
|
* Move global config to `config.configure_with(:import)` in `config/initializers/rails_admin_import.rb`.
|
@@ -303,11 +431,19 @@ gem "rails_admin_import", "~> 1.2.0", require: "rails_admin_import/eager_load"
|
|
303
431
|
* Update model import hooks to take 1 hash argument instead of 2 arrays with values and headers.
|
304
432
|
|
305
433
|
* Support for importing file attributes was removed since I couldn't understand how it works. It should be possible to reimplement it yourself using post import hooks. Open an issue to discuss how to put back support for importing files into the gem.
|
306
|
-
|
434
|
+
|
307
435
|
## Community-contributed translations
|
308
436
|
|
309
437
|
* [Spanish translation](https://gist.github.com/yovasx2/dc0e9512e6c6243f840c) by Giovanni Alberto
|
310
438
|
|
439
|
+
* [French translation](https://github.com/rodinux/rails_admin_import.fr-MX.yml) by Rodolphe Robles. (I suggest to translate also rails admin.fr and your locales.fr to resolve an issue with DatePicker)
|
440
|
+
|
441
|
+
* [Italian translation](https://gist.github.com/aprofiti/ec3dc452898c8c48534b59eeb2701765) by Alessandro Profiti
|
442
|
+
|
443
|
+
* [Japanese translation](https://gist.github.com/higumachan/c4bf669d6446ec509386229f916ba5fc) by Yuta Hinokuma
|
444
|
+
|
445
|
+
* [Brazilian Portuguese translation](https://gist.github.com/tteurs/5a87ff4bc5f24692dab05b3cde0ca9df) by Matheo Gracia Pegoraro
|
446
|
+
|
311
447
|
## Run tests
|
312
448
|
|
313
449
|
1. Clone the repository to your machine
|
@@ -326,6 +462,13 @@ Original author: [Steph Skardal](https://github.com/stephskardal)
|
|
326
462
|
Maintainer (since May 2015): [Julien Vanier](https://github.com/monkbroc)
|
327
463
|
|
328
464
|
|
465
|
+
## Release
|
466
|
+
|
467
|
+
- Update `lib/rails_admin_import/version.rb`
|
468
|
+
- Update the install instructions at [the top of the readme](#installation)
|
469
|
+
- Commit to git
|
470
|
+
- `rake release`
|
471
|
+
|
329
472
|
## Contributing
|
330
473
|
|
331
474
|
Everyone is encouraged to help improve this project. Here are a few ways you can help:
|
@@ -1,3 +1,14 @@
|
|
1
|
+
:ruby
|
2
|
+
translations = {
|
3
|
+
add: t('admin.import.enumeration.add'),
|
4
|
+
chooseAll: t('admin.import.enumeration.choose_all'),
|
5
|
+
clearAll: t('admin.import.enumeration.clear_all'),
|
6
|
+
down: t('admin.import.enumeration.down'),
|
7
|
+
remove: t('admin.import.enumeration.remove'),
|
8
|
+
search: t('admin.import.enumeration.search'),
|
9
|
+
up: t('admin.import.enumeration.up')
|
10
|
+
}
|
11
|
+
|
1
12
|
= render "results"
|
2
13
|
|
3
14
|
= form_tag import_path(@abstract_model), :multipart => true, class: 'form-horizontal denser' do
|
@@ -25,19 +36,21 @@
|
|
25
36
|
.col-sm-10.controls
|
26
37
|
= select_tag 'encoding',
|
27
38
|
options_for_select(Encoding.name_list.sort),
|
28
|
-
include_blank: true, data: { enumeration: true }
|
39
|
+
include_blank: true, data: { enumeration: true, options: { regional: translations } }
|
29
40
|
%p.help-block= t('admin.import.help.encoding', name: 'UTF-8')
|
30
41
|
.form-group.control-group
|
31
42
|
%label.col-sm-2.control-label= t("admin.import.update_if_exists")
|
32
43
|
.col-sm-10.controls
|
33
|
-
= check_box_tag :update_if_exists, '1',
|
44
|
+
= check_box_tag :update_if_exists, '1', RailsAdminImport.config.update_if_exists, :class => "form-control"
|
34
45
|
%p.help-block= t('admin.import.help.update_if_exists')
|
35
46
|
.form-group.control-group
|
36
47
|
%label.col-sm-2.control-label{for: "update_lookup"}= t("admin.import.update_lookup")
|
37
48
|
.col-sm-10.controls
|
38
49
|
= select_tag 'update_lookup',
|
39
50
|
options_for_select(@import_model.update_lookup_field_names,
|
40
|
-
|
51
|
+
Array.wrap(@import_model.config.mapping_key).map(&:to_s)),
|
52
|
+
multiple: true,
|
53
|
+
data: { enumeration: true, options: { regional: translations } }
|
41
54
|
|
42
55
|
- unless @import_model.association_fields.empty?
|
43
56
|
%fieldset
|
@@ -53,7 +66,8 @@
|
|
53
66
|
.col-sm-10.controls
|
54
67
|
= select_tag "associations[#{field.name}]",
|
55
68
|
options_for_select(@import_model.associated_model_fields(field),
|
56
|
-
|
69
|
+
Array.wrap(@import_model.associated_config(field).mapping_key).first.to_s),
|
70
|
+
data: { enumeration: true, options: { regional: translations } }
|
57
71
|
|
58
72
|
%br
|
59
73
|
.form-actions
|
@@ -0,0 +1,3 @@
|
|
1
|
+
See the [community contributed translation section of the README](https://github.com/stephskardal/rails_admin_import#community-contributed-translations) for more languages.
|
2
|
+
|
3
|
+
To contribute a new translation, please put it in a gist and submit a pull request to link your translation in the README.
|
@@ -1,51 +1,53 @@
|
|
1
|
-
|
2
1
|
en:
|
3
2
|
admin:
|
4
3
|
actions:
|
5
4
|
import:
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
link:
|
10
|
-
|
11
|
-
|
5
|
+
breadcrumb: Import
|
6
|
+
bulk_link: Import
|
7
|
+
done: Imported
|
8
|
+
link: Import
|
9
|
+
menu: Import
|
10
|
+
title: Import
|
12
11
|
import:
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
legend:
|
27
|
-
fields: "Fields to import"
|
28
|
-
upload: "Upload file"
|
29
|
-
mapping: "Related fields mapping"
|
30
|
-
import_success:
|
31
|
-
create: "Created %{name}"
|
32
|
-
update: "Updated %{name}"
|
33
|
-
import_error:
|
34
|
-
create: "Failed to create %{name}: %{error}"
|
35
|
-
update: "Failed to update %{name}: %{error}"
|
36
|
-
general: "Error during import: %{error}"
|
37
|
-
line_item_limit: "Please limit upload file to %{limit} line items."
|
38
|
-
old_import_hook: >
|
39
|
-
The import hook %{model}.%{method} should take only 1 argument.
|
40
|
-
Data may not imported correctly.
|
41
|
-
See Upgrading section readme in Rails Admin Import.
|
42
|
-
association_not_found: "Association not found. %{error}"
|
12
|
+
association_fields: Association fields
|
13
|
+
association_not_found: 'Association not found. %{error}'
|
14
|
+
encoding: Encoding
|
15
|
+
enumeration:
|
16
|
+
add: Add new
|
17
|
+
choose_all: Choose all
|
18
|
+
clear_all: Clear all
|
19
|
+
down: Down
|
20
|
+
remove: Remove
|
21
|
+
search: Search
|
22
|
+
up: Up
|
23
|
+
file: Data file
|
24
|
+
format: File format
|
43
25
|
help:
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
|
51
|
-
|
26
|
+
association_fields: |
|
27
|
+
These fields map to other tables in the database, lookup via attribute selected below. For "many" associations, you may include multiple columns with the same header in the CSV file.
|
28
|
+
encoding: Choose file encoding. Leave empty to auto-detect. Ignored for JSON.
|
29
|
+
file_limit: 'Please limit upload file to %{limit} line items.'
|
30
|
+
model_fields: The fields above may be included in the import file.
|
31
|
+
update_if_exists: Update records found with the lookup field below instead of creating new records
|
32
|
+
import_error:
|
33
|
+
create: 'Failed to create %{name}: %{error}'
|
34
|
+
general: 'Error during import: %{error}'
|
35
|
+
line_item_limit: 'Please limit upload file to %{limit} line items.'
|
36
|
+
old_import_hook: |
|
37
|
+
The import hook %{model}.%{method} should take only 1 argument. Data may not imported correctly. See Upgrading section readme in Rails Admin Import.
|
38
|
+
update: 'Failed to update %{name}: %{error}'
|
39
|
+
import_success:
|
40
|
+
create: 'Created %{name}'
|
41
|
+
update: 'Updated %{name}'
|
42
|
+
invalid_format: Invalid import format.
|
43
|
+
invalid_json: 'The JSON data should be an array of records or an object with a key ''%{root_key}'' set to an array of records'
|
44
|
+
legend:
|
45
|
+
fields: Fields to import
|
46
|
+
mapping: Related fields mapping
|
47
|
+
upload: Upload file
|
48
|
+
mapping: mapping
|
49
|
+
missing_file: You must select a file
|
50
|
+
missing_update_lookup: Your file must contain a column for the 'Update lookup field' you selected.
|
51
|
+
model_fields: Model fields
|
52
|
+
update_if_exists: Update if exists
|
53
|
+
update_lookup: Update lookup field(s)
|
@@ -6,11 +6,15 @@ module RailsAdminImport
|
|
6
6
|
attr_accessor :logging
|
7
7
|
attr_accessor :line_item_limit
|
8
8
|
attr_accessor :rollback_on_error
|
9
|
+
attr_accessor :update_if_exists
|
9
10
|
attr_accessor :header_converter
|
10
11
|
attr_accessor :csv_options
|
12
|
+
attr_accessor :pass_filename
|
11
13
|
|
12
14
|
# Default is to downcase headers and add underscores to convert into attribute names
|
13
15
|
HEADER_CONVERTER = lambda do |header|
|
16
|
+
# check for nil/blank headers
|
17
|
+
next if header.blank?
|
14
18
|
header.parameterize.underscore
|
15
19
|
end
|
16
20
|
|
@@ -31,7 +35,9 @@ module RailsAdminImport
|
|
31
35
|
@logging = false
|
32
36
|
@line_item_limit = 1000
|
33
37
|
@rollback_on_error = false
|
38
|
+
@update_if_exists = false
|
34
39
|
@header_converter = HEADER_CONVERTER
|
40
|
+
@pass_filename = false
|
35
41
|
@csv_options = {}
|
36
42
|
end
|
37
43
|
end
|
@@ -1,9 +1,11 @@
|
|
1
1
|
require "csv"
|
2
|
+
require "charlock_holmes"
|
2
3
|
|
3
4
|
module RailsAdminImport
|
4
5
|
module Formats
|
5
6
|
class CSVImporter < FileImporter
|
6
7
|
Formats.register(:csv, self)
|
8
|
+
Formats.register(:CSV, self)
|
7
9
|
|
8
10
|
autoload :CharDet, "rchardet"
|
9
11
|
|
@@ -15,8 +17,9 @@ module RailsAdminImport
|
|
15
17
|
|
16
18
|
# A method that yields a hash of attributes for each record to import
|
17
19
|
def each_record
|
18
|
-
CSV.foreach(filename, csv_options) do |row|
|
19
|
-
|
20
|
+
CSV.foreach(filename, **csv_options) do |row|
|
21
|
+
attr = convert_to_attributes(row)
|
22
|
+
yield attr unless attr.all? { |field, value| value.blank? }
|
20
23
|
end
|
21
24
|
end
|
22
25
|
|
@@ -51,9 +54,9 @@ module RailsAdminImport
|
|
51
54
|
end
|
52
55
|
|
53
56
|
def detect_encoding
|
54
|
-
charset =
|
55
|
-
if charset[
|
56
|
-
from_encoding = charset[
|
57
|
+
charset = CharlockHolmes::EncodingDetector.detect File.read(filename)
|
58
|
+
if charset[:confidence] > 0.6
|
59
|
+
from_encoding = charset[:encoding]
|
57
60
|
from_encoding = "UTF-8" if from_encoding == "ascii"
|
58
61
|
end
|
59
62
|
from_encoding
|
@@ -4,6 +4,7 @@ module RailsAdminImport
|
|
4
4
|
module Formats
|
5
5
|
class XLSXImporter < FileImporter
|
6
6
|
Formats.register(:xlsx, self)
|
7
|
+
Formats.register(:XLSX, self)
|
7
8
|
|
8
9
|
autoload :SimpleXlsxReader, "simple_xlsx_reader"
|
9
10
|
|
@@ -18,7 +19,8 @@ module RailsAdminImport
|
|
18
19
|
sheet = doc.sheets.first
|
19
20
|
@headers = convert_headers(sheet.headers)
|
20
21
|
sheet.data.each do |row|
|
21
|
-
|
22
|
+
attr = convert_to_attributes(row)
|
23
|
+
yield attr unless attr.all? { |field, value| value.blank? }
|
22
24
|
end
|
23
25
|
end
|
24
26
|
|
@@ -33,7 +35,7 @@ module RailsAdminImport
|
|
33
35
|
def convert_to_attributes(row)
|
34
36
|
row_with_headers = @headers.zip(row)
|
35
37
|
row_with_headers.each_with_object({}) do |(field, value), record|
|
36
|
-
|
38
|
+
next if field.nil?
|
37
39
|
field = field.to_sym
|
38
40
|
if import_model.has_multiple_values?(field)
|
39
41
|
field = import_model.pluralize_field(field)
|
@@ -16,7 +16,6 @@ module RailsAdminImport
|
|
16
16
|
begin
|
17
17
|
init_results
|
18
18
|
|
19
|
-
|
20
19
|
if records.count > RailsAdminImport.config.line_item_limit
|
21
20
|
return results = {
|
22
21
|
success: [],
|
@@ -24,13 +23,19 @@ module RailsAdminImport
|
|
24
23
|
}
|
25
24
|
end
|
26
25
|
|
26
|
+
perform_global_callback(:before_import)
|
27
|
+
|
27
28
|
with_transaction do
|
28
29
|
records.each do |record|
|
29
|
-
|
30
|
+
catch :skip do
|
31
|
+
import_record(record)
|
32
|
+
end
|
30
33
|
end
|
31
34
|
|
32
35
|
rollback_if_error
|
33
36
|
end
|
37
|
+
|
38
|
+
perform_global_callback(:after_import)
|
34
39
|
rescue Exception => e
|
35
40
|
report_general_error("#{e} (#{e.backtrace.first})")
|
36
41
|
end
|
@@ -65,19 +70,28 @@ module RailsAdminImport
|
|
65
70
|
end
|
66
71
|
|
67
72
|
def import_record(record)
|
68
|
-
if
|
73
|
+
if params["file"] && RailsAdminImport.config.pass_filename
|
74
|
+
record.merge!({:filename_importer => params[:file].original_filename})
|
75
|
+
end
|
76
|
+
|
77
|
+
perform_model_callback(import_model.model, :before_import_find, record)
|
78
|
+
|
79
|
+
if update_lookup && !(update_lookup - record.keys).empty?
|
69
80
|
raise UpdateLookupError, I18n.t("admin.import.missing_update_lookup")
|
70
81
|
end
|
71
82
|
|
72
83
|
object = find_or_create_object(record, update_lookup)
|
84
|
+
return if object.nil?
|
73
85
|
action = object.new_record? ? :create : :update
|
74
86
|
|
75
87
|
begin
|
88
|
+
perform_model_callback(object, :before_import_associations, record)
|
76
89
|
import_single_association_data(object, record)
|
77
90
|
import_many_association_data(object, record)
|
78
91
|
rescue AssociationNotFound => e
|
79
92
|
error = I18n.t("admin.import.association_not_found", :error => e.to_s)
|
80
93
|
report_error(object, action, error)
|
94
|
+
perform_model_callback(object, :after_import_association_error, record)
|
81
95
|
return
|
82
96
|
end
|
83
97
|
|
@@ -88,12 +102,13 @@ module RailsAdminImport
|
|
88
102
|
perform_model_callback(object, :after_import_save, record)
|
89
103
|
else
|
90
104
|
report_error(object, action, object.errors.full_messages.join(", "))
|
105
|
+
perform_model_callback(object, :after_import_error, record)
|
91
106
|
end
|
92
107
|
end
|
93
108
|
|
94
109
|
def update_lookup
|
95
110
|
@update_lookup ||= if params[:update_if_exists] == "1"
|
96
|
-
params[:update_lookup].to_sym
|
111
|
+
params[:update_lookup].map(&:to_sym)
|
97
112
|
end
|
98
113
|
end
|
99
114
|
|
@@ -169,6 +184,11 @@ module RailsAdminImport
|
|
169
184
|
end
|
170
185
|
end
|
171
186
|
|
187
|
+
def perform_global_callback(method_name)
|
188
|
+
object = import_model.model
|
189
|
+
object.send(method_name) if object.respond_to?(method_name)
|
190
|
+
end
|
191
|
+
|
172
192
|
def find_or_create_object(record, update)
|
173
193
|
field_names = import_model.model_fields.map(&:name)
|
174
194
|
new_attrs = record.select do |field_name, value|
|
@@ -177,13 +197,19 @@ module RailsAdminImport
|
|
177
197
|
|
178
198
|
model = import_model.model
|
179
199
|
object = if update.present?
|
180
|
-
|
200
|
+
query = update.each_with_object({}) do
|
201
|
+
|field, query| query[field] = record[field]
|
202
|
+
end
|
203
|
+
model.where(query).first
|
181
204
|
end
|
182
205
|
|
183
206
|
if object.nil?
|
184
|
-
object = model.new
|
207
|
+
object = model.new
|
208
|
+
perform_model_callback(object, :before_import_attributes, record)
|
209
|
+
object.attributes = new_attrs
|
185
210
|
else
|
186
|
-
object
|
211
|
+
perform_model_callback(object, :before_import_attributes, record)
|
212
|
+
object.attributes = new_attrs.except(update.map(&:to_sym))
|
187
213
|
end
|
188
214
|
object
|
189
215
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: rails_admin_import
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.
|
4
|
+
version: 2.3.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Steph Skardal
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2022-01-29 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: rails
|
@@ -32,6 +32,9 @@ dependencies:
|
|
32
32
|
- - ">="
|
33
33
|
- !ruby/object:Gem::Version
|
34
34
|
version: 0.6.6
|
35
|
+
- - "<"
|
36
|
+
- !ruby/object:Gem::Version
|
37
|
+
version: 3.0.0.beta
|
35
38
|
type: :runtime
|
36
39
|
prerelease: false
|
37
40
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -39,34 +42,23 @@ dependencies:
|
|
39
42
|
- - ">="
|
40
43
|
- !ruby/object:Gem::Version
|
41
44
|
version: 0.6.6
|
42
|
-
-
|
43
|
-
name: haml
|
44
|
-
requirement: !ruby/object:Gem::Requirement
|
45
|
-
requirements:
|
46
|
-
- - "~>"
|
47
|
-
- !ruby/object:Gem::Version
|
48
|
-
version: '4.0'
|
49
|
-
type: :runtime
|
50
|
-
prerelease: false
|
51
|
-
version_requirements: !ruby/object:Gem::Requirement
|
52
|
-
requirements:
|
53
|
-
- - "~>"
|
45
|
+
- - "<"
|
54
46
|
- !ruby/object:Gem::Version
|
55
|
-
version:
|
47
|
+
version: 3.0.0.beta
|
56
48
|
- !ruby/object:Gem::Dependency
|
57
|
-
name:
|
49
|
+
name: charlock_holmes
|
58
50
|
requirement: !ruby/object:Gem::Requirement
|
59
51
|
requirements:
|
60
52
|
- - "~>"
|
61
53
|
- !ruby/object:Gem::Version
|
62
|
-
version: '
|
54
|
+
version: '0.7'
|
63
55
|
type: :runtime
|
64
56
|
prerelease: false
|
65
57
|
version_requirements: !ruby/object:Gem::Requirement
|
66
58
|
requirements:
|
67
59
|
- - "~>"
|
68
60
|
- !ruby/object:Gem::Version
|
69
|
-
version: '
|
61
|
+
version: '0.7'
|
70
62
|
- !ruby/object:Gem::Dependency
|
71
63
|
name: simple_xlsx_reader
|
72
64
|
requirement: !ruby/object:Gem::Requirement
|
@@ -96,6 +88,7 @@ files:
|
|
96
88
|
- app/views/rails_admin/main/_results.html.haml
|
97
89
|
- app/views/rails_admin/main/_section.html.haml
|
98
90
|
- app/views/rails_admin/main/import.html.haml
|
91
|
+
- config/locales/README.md
|
99
92
|
- config/locales/import.en.yml
|
100
93
|
- lib/rails_admin_import.rb
|
101
94
|
- lib/rails_admin_import/action.rb
|
@@ -135,7 +128,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
135
128
|
version: '0'
|
136
129
|
requirements: []
|
137
130
|
rubyforge_project:
|
138
|
-
rubygems_version: 2.
|
131
|
+
rubygems_version: 2.7.6
|
139
132
|
signing_key:
|
140
133
|
specification_version: 4
|
141
134
|
summary: Import functionality for Rails Admin
|