rails_admin_import 2.0.0 → 2.3.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 741d56674d36948aadddfa2951204d3fd3ece179
4
- data.tar.gz: 4535fb361151b7f3dbb615c380a0d9eeb93a6478
2
+ SHA256:
3
+ metadata.gz: fed24bf8ccc152d3b45b68e56600ba77160a707794228cd8828ce7e289003056
4
+ data.tar.gz: 3135d1461ba3b1e6ac4a3eb0c13cc85c1c3cda432aeaad1759bfa5e49114bdeb
5
5
  SHA512:
6
- metadata.gz: f364e695d7e1eac68a4bc821cc08f34c1dc1ea340a8d8c26a317aba5fcbcff7aded642fc7c9657e6c4671e3306b4dc1522ab39ef4c9ae449850e5a53806b5ec8
7
- data.tar.gz: dd57d2e866208dfeb6da5acf65d118902c87d650d0cf42828266a901e726734b8d1a159b4cde9569f36fd7260a952792b63d58aeec11bfa9ebff7960418b11c4
6
+ metadata.gz: 669291db9c194aaaf47a5f7f75e5d6fb252615a5d0e01b87160f1afc3bbf8150ed8f223fcfbb33629f78ae47c2cb0034bbf8548f4fa80cec30c92d25eb31eac5
7
+ data.tar.gz: 297247a8340a59c5364b6ab01208eaad4275aea4f2290362037117010c250063e2748a6d1fb9bfc3583ec89cf4d645597ec45acf57da5b07add65471832e0244
data/CHANGELOG.md CHANGED
@@ -1,5 +1,30 @@
1
1
  # Change Log
2
2
 
3
+ # 2.3.1 / 2022-01-29
4
+
5
+ - Mark compatibility to Rails Admin < 3.0
6
+ - Bump Nokogiri in tests
7
+
8
+ # 2.3.0 / 2021-06-11
9
+
10
+ - Added an option to pass filename for each record. Thanks @waheedi
11
+ - Enumeration translation. Thanks @zaknafain
12
+
13
+ # 2.2.0 / 2019-02-24
14
+
15
+ - Switch to using CharlockHolmes for character detection. Thanks @codealchemy
16
+ - Readme improvements. Thanks @olleolleolle
17
+
18
+ # 2.1.0 / 2017-11-18
19
+
20
+ - French translation. Thanks @rodinux
21
+ - Check for blank headers. Thanks @JuandGirald
22
+ - Italian translation. Thanks @aprofiti
23
+ - Remove haml dependency. Thanks @prem-prakash
24
+ - Japanese translation. Thanks @higumachan
25
+ - Multiple mapping keys. Thanks @dmitrypol
26
+ - Add more hooks during import
27
+
3
28
  # 2.0.0 / 2016-05-04
4
29
 
5
30
  - Pull in the encoding list from the Encoding module instead of RailsAdmin. Thanks @baldursson, @patricklewis and @lucasff
data/README.md CHANGED
@@ -1,17 +1,15 @@
1
1
  # Rails Admin Import
2
2
 
3
- [![Build Status](https://travis-ci.org/monkbroc/rails_admin_import.svg?branch=master)](https://travis-ci.org/monkbroc/rails_admin_import)
3
+ [![Build Status](https://github.com/monkbroc/rails_admin_import/actions/workflows/ruby.yml/badge.svg)](https://github.com/monkbroc/rails_admin_import/actions/workflows/ruby.yml)
4
4
 
5
5
  Plugin functionality to add generic import to Rails Admin from CSV, JSON and XLSX files
6
6
 
7
- *This Readme is for version 1.x. If you are still using version 0.1.x, see [this branch](https://github.com/stephskardal/rails_admin_import/tree/legacy)*
8
-
9
7
  ## Installation
10
8
 
11
9
  * First, add to Gemfile:
12
10
 
13
- ```
14
- gem "rails_admin_import", "~> 1.2"
11
+ ```ruby
12
+ gem "rails_admin_import", "~> 2.3"
15
13
  ```
16
14
 
17
15
  * Define configuration in `config/initializers/rails_admin_import.rb`:
@@ -28,8 +26,10 @@ RailsAdmin.config do |config|
28
26
 
29
27
  # Optional:
30
28
  # Configure global RailsAdminImport options
29
+ # Configure pass filename to records hashes
31
30
  config.configure_with(:import) do |config|
32
31
  config.logging = true
32
+ config.pass_filename = true
33
33
  end
34
34
 
35
35
  # Optional:
@@ -62,9 +62,10 @@ Both updating existing records and associating records requires the use of
62
62
  ### Mapping Keys
63
63
 
64
64
  Every importable class has a mapping key that uniquely identifies its
65
- instances. The value for this field can then be provided in import data, either
66
- to update the existing record or to attach it through an association to another
67
- model. This concept exists because `id`s are often not constant when moving
65
+ instances. The mapping key can be one or more fields. The value for
66
+ these fields can then be provided in import data, either to update the
67
+ existing record or to attach it through an association to another model.
68
+ This concept exists because `id`s are often not constant when moving
68
69
  records between data stores.
69
70
 
70
71
  For example, a `User` model may have an `email` field. When uploading a set
@@ -73,14 +74,19 @@ our mapping key and then provide that field on each record in our data,
73
74
  allowing us to update existing records with matching emails.
74
75
 
75
76
  Using a csv formatted example:
77
+
76
78
  ```
77
79
  Email,First name,Last name
78
80
  peter.gibbons@initech.com,Peter,Gibbons
79
81
  michael.bolton@initech.com,Michael,Bolton
80
82
  ```
83
+
81
84
  would look for existing users with those emails. If one was found, its name
82
85
  fields would be updated. Otherwise, a new one would be created.
83
86
 
87
+ For updating building owners, the mapping key could be `street_address` and
88
+ `zip_code`.
89
+
84
90
  Similarly, if each user has favorite books, we could set the mapping key
85
91
  for `Book` to be `isbn` and then include the isbn for their books within each
86
92
  user record. The syntax for this is to use the name of the associated model as
@@ -89,28 +95,36 @@ a user record would have one or more fields named "Book" that include each
89
95
  associated book's ISBN.
90
96
 
91
97
  Again using a csv formatted example:
98
+
92
99
  ```
93
100
  Email, Book, Book, Book
94
101
  peter.gibbons@initech.com, 9781119997870, 9780671027032
95
102
  michael.bolton@initech.com, 9780446677479
96
103
  ```
104
+
97
105
  would look up books with those ISBNs and attach them to those users.
98
106
 
99
107
  Mapping keys can be selected on the import page. Their defaults can also be
100
108
  globally configured in the config file:
101
109
 
102
- ```
110
+ ```ruby
103
111
  RailsAdmin.config do |config|
104
112
  config.model 'User' do
105
113
  import do
106
114
  mapping_key :email
115
+ # for multiple values, use mapping_key [:first_name, :last_name]
116
+ mapping_key_list [:email, :some_other_id]
107
117
  end
108
118
  end
109
119
  end
110
120
  ```
111
121
 
122
+ Since in models with large number of fields it doesn't make sense to use
123
+ most of them as mapping values, you can add `mapping_key_list` to
124
+ restrict which fields can be selected as mapping key in the UI during import.
125
+
112
126
  Note that a matched record must exist when attaching associated models, or the
113
- imported record will fail and be skipped.
127
+ imported record will fail and be skipped.
114
128
 
115
129
  Complex associations (`has_one ..., :through` or polymorphic associations)
116
130
  need to be dealt with via custom logic called by one of the import hooks
@@ -120,7 +134,7 @@ existed through an intermediary model called `ServiceProvider`, we could
120
134
  provide a `user_email` field in our records and handle the actual
121
135
  association with an import hook:
122
136
 
123
- ```
137
+ ```ruby
124
138
  class Service < ActiveRecord::Base
125
139
  belongs_to :service_provider
126
140
  has_one :user, through: :service_provider
@@ -133,6 +147,8 @@ class Service < ActiveRecord::Base
133
147
  end
134
148
  ```
135
149
 
150
+ Importing new records by id is not recommended since it ignores the sequences of ids in database. That will lead to `ERROR: duplicate key value violates unique constraint` in future. You can work around this issue by adding an `import_id` column to your model, renaming the `id` column in your CSV to `import_id` and using `import_id` as the update lookup field.
151
+
136
152
  ### File format
137
153
 
138
154
  The format is inferred by the extension (.csv, .json or .xlsx).
@@ -153,6 +169,8 @@ Peter,Gibbons,IT,Management
153
169
  Michael,Bolton,IT,
154
170
  ```
155
171
 
172
+ Blank lines will be skipped.
173
+
156
174
  #### JSON
157
175
 
158
176
  The file must be an array or an object with a root key the same name as the plural model name, i.e. the default Rails JSON output format with include_root_in_json on or off.
@@ -163,19 +181,46 @@ The Microsoft Excel XLM format (XLSX) is supported, but not the old binary Micro
163
181
 
164
182
  The expected rows and columns are the same as for the CSV format (first line contains headers, multiple columns for "many" associations).
165
183
 
184
+ Blank lines will be skipped.
185
+
166
186
  ## Configuration
167
187
 
168
188
  ### Global configuration options
169
189
 
190
+ ```ruby
191
+ RailsAdmin.config do |config|
192
+ config.actions do
193
+ all
194
+ import
195
+ end
196
+
197
+ # Default global RailsAdminImport options
198
+ config.configure_with(:import) do |config|
199
+ config.logging = false
200
+ config.line_item_limit = 1000
201
+ config.update_if_exists = false
202
+ config.pass_filename = false
203
+ config.rollback_on_error = false
204
+ config.header_converter = lambda do |header|
205
+ header.parameterize.underscore if header.present?
206
+ end
207
+ config.csv_options = {}
208
+ end
209
+ end
210
+ ```
211
+
170
212
  * __logging__ (default `false`): Save a copy of each imported file to log/import and a detailed import log to log/rails_admin_import.log
171
213
 
172
214
  * __line_item_limit__ (default `1000`): max number of items that can be imported at one time.
173
215
 
216
+ * __update_if_exists__ (default `false`): default value for the "Update if exists" checkbox on the import page.
217
+
174
218
  * __rollback_on_error__ (default `false`): import records in a transaction and rollback if there is one error. Only for ActiveRecord, not Mongoid.
175
219
 
176
220
  * __header_converter__ (default `lambda { ... }`): a lambda to convert each CSV header text string to a model attribute name. The default header converter converts to lowercase and replaces spaces with underscores.
177
221
 
178
222
  * __csv_options__ (default `{}`): a hash of options that will be passed to a new [CSV](http://ruby-doc.org/stdlib-2.0.0/libdoc/csv/rdoc/CSV.html) instance
223
+ * __pass_filename__ (default `false`): Access the uploaded file name in your model actions, for example if set to true, inside each record, there will be an addtional property `record[:filename_importer]` which contains the file name of the currently uploaded file.
179
224
 
180
225
  ### Model-specific configuration
181
226
 
@@ -239,7 +284,7 @@ end
239
284
  RailsAdmin.config do |config|
240
285
  config.model 'User' do
241
286
  import do
242
- default_excluded_fields [:created_at, :updated_at]
287
+ default_excluded_fields [:created_at, :updated_at, :deleted_at, :c_at, :u_at]
243
288
  end
244
289
  end
245
290
  end
@@ -247,18 +292,56 @@ end
247
292
 
248
293
  ## Import hooks
249
294
 
250
-
251
- Define instance methods on your models to be hooked into the import process, if special/additional processing is required on the data:
295
+ Define methods on your models to be hooked into the import process, if special/additional processing is required on the data:
252
296
 
253
297
  ```ruby
254
298
  # some model
255
299
  class User < ActiveRecord::Base
300
+ def self.before_import
301
+ # called on the model class once before importing any individual records
302
+ end
303
+
304
+ def self.before_import_find(record)
305
+ # called on the model class before finding or creating the new record
306
+ # maybe modify the import record that will be used to find the model
307
+ # throw :skip to skip importing this record
308
+ throw :skip unless record[:email].ends_with? "@mycompany.com"
309
+ end
310
+
311
+ def before_import_attributes(record)
312
+ # called on the blank new model or the found model before fields are imported
313
+ # maybe delete fields from the import record that you don't need
314
+ # throw :skip to skip importing this record
315
+ end
316
+
317
+ def before_import_associations(record)
318
+ # called on the model with attributes but before associations are imported
319
+ # do custom import of associations
320
+ # make sure to delete association fields from the import record to avoid double import
321
+ record.delete(:my_association)
322
+ # throw :skip to skip importing this record
323
+ end
324
+
256
325
  def before_import_save(record)
257
- # Your custom special sauce
326
+ # called on the model before it is saved but after all fields and associations have been imported
327
+ # make final modifications to the record
328
+ # throw :skip to skip importing this record
258
329
  end
259
330
 
260
331
  def after_import_save(record)
261
- # Your custom special sauce
332
+ # called on the model after it is saved
333
+ end
334
+
335
+ def after_import_association_error(record)
336
+ # called on the model when an association cannot be found
337
+ end
338
+
339
+ def after_import_error(record)
340
+ # called on the model when save fails
341
+ end
342
+
343
+ def self.after_import
344
+ # called once on the model class after importing all individual records
262
345
  end
263
346
  end
264
347
  ```
@@ -269,12 +352,35 @@ For example, you could
269
352
 
270
353
  * Import an image into Carrierwave via a URL provided in the CSV.
271
354
 
272
- ```
355
+ ```ruby
273
356
  def before_import_save(record)
274
357
  self.remote_image_url = record[:image] if record[:image].present?
275
358
  end
276
359
  ```
277
360
 
361
+ * Skip some validations when importing.
362
+
363
+ ```ruby
364
+ class User < ActiveRecord::Base
365
+ # Non-persistent attribute to allow creating a new user without a password
366
+ # Password will be set by the user by following a link in the invitation email
367
+ attr_accessor :allow_blank_password
368
+
369
+ devise :validatable
370
+
371
+ # Called by Devise to enable/disable password presence validation
372
+ def password_required?
373
+ allow_blank_password ? false : super
374
+ end
375
+
376
+ # Don't require a password when importing users
377
+ def before_import_save(record)
378
+ self.allow_blank_password = true
379
+ end
380
+ end
381
+ ```
382
+
383
+
278
384
  ## ORM: ActiveRecord and Mongoid
279
385
 
280
386
  The gem is tested to work with ActiveRecord and Mongoid.
@@ -288,10 +394,32 @@ Since the import functionality is rarely used in many applications, some gems ar
288
394
 
289
395
  If you prefer to eager load all dependecies at boot, use this line in your `Gemfile`.
290
396
 
291
- ```
397
+ ```ruby
292
398
  gem "rails_admin_import", "~> 1.2.0", require: "rails_admin_import/eager_load"
293
399
  ```
294
400
 
401
+ ## Import error due to Rails class reloading
402
+
403
+ ![error due to class reloading](https://user-images.githubusercontent.com/2566348/51355874-0f83ad00-1a7e-11e9-8e58-46bc4699f2e6.jpg)
404
+
405
+ If you get an error like `Error during import: MyModel(#70286054976500) expected, got MyModel(#70286114743280)`, you need restart the rails server and redo the import. This is due to the fact that Rails reloads the ActiveRecord model classes in development when you make changes to them and Rails Admin is still using the old class.
406
+
407
+ Another suggestion is to set `config.cache_classes = true` to true in your `development.rb` for Rails Admin Import to work around the ActiveRecord model class reloading issue. See [this comment](https://github.com/stephskardal/rails_admin_import/issues/88#issuecomment-455374671) for more information.
408
+
409
+ ## Customize the UI
410
+
411
+ If you want to hide all the advanced fields from the import UI, you can copy [`app/views/rails_admin/main/import.html.haml`](app/views/rails_admin/main/import.html.haml) to your project at the same path. Add `.hidden` at the end of lines you want to hide.
412
+
413
+ For example:
414
+
415
+ ```haml
416
+ .form-group.control-group.hidden
417
+ %label.col-sm-2.control-label= t("admin.import.update_if_exists")
418
+ .col-sm-10.controls
419
+ = check_box_tag :update_if_exists, '1', true, :class => "form-control"
420
+ %p.help-block= t('admin.import.help.update_if_exists')
421
+ ```
422
+
295
423
  ## Upgrading
296
424
 
297
425
  * Move global config to `config.configure_with(:import)` in `config/initializers/rails_admin_import.rb`.
@@ -303,11 +431,19 @@ gem "rails_admin_import", "~> 1.2.0", require: "rails_admin_import/eager_load"
303
431
  * Update model import hooks to take 1 hash argument instead of 2 arrays with values and headers.
304
432
 
305
433
  * Support for importing file attributes was removed since I couldn't understand how it works. It should be possible to reimplement it yourself using post import hooks. Open an issue to discuss how to put back support for importing files into the gem.
306
-
434
+
307
435
  ## Community-contributed translations
308
436
 
309
437
  * [Spanish translation](https://gist.github.com/yovasx2/dc0e9512e6c6243f840c) by Giovanni Alberto
310
438
 
439
+ * [French translation](https://github.com/rodinux/rails_admin_import.fr-MX.yml) by Rodolphe Robles. (I suggest to translate also rails admin.fr and your locales.fr to resolve an issue with DatePicker)
440
+
441
+ * [Italian translation](https://gist.github.com/aprofiti/ec3dc452898c8c48534b59eeb2701765) by Alessandro Profiti
442
+
443
+ * [Japanese translation](https://gist.github.com/higumachan/c4bf669d6446ec509386229f916ba5fc) by Yuta Hinokuma
444
+
445
+ * [Brazilian Portuguese translation](https://gist.github.com/tteurs/5a87ff4bc5f24692dab05b3cde0ca9df) by Matheo Gracia Pegoraro
446
+
311
447
  ## Run tests
312
448
 
313
449
  1. Clone the repository to your machine
@@ -326,6 +462,13 @@ Original author: [Steph Skardal](https://github.com/stephskardal)
326
462
  Maintainer (since May 2015): [Julien Vanier](https://github.com/monkbroc)
327
463
 
328
464
 
465
+ ## Release
466
+
467
+ - Update `lib/rails_admin_import/version.rb`
468
+ - Update the install instructions at [the top of the readme](#installation)
469
+ - Commit to git
470
+ - `rake release`
471
+
329
472
  ## Contributing
330
473
 
331
474
  Everyone is encouraged to help improve this project. Here are a few ways you can help:
@@ -1,3 +1,14 @@
1
+ :ruby
2
+ translations = {
3
+ add: t('admin.import.enumeration.add'),
4
+ chooseAll: t('admin.import.enumeration.choose_all'),
5
+ clearAll: t('admin.import.enumeration.clear_all'),
6
+ down: t('admin.import.enumeration.down'),
7
+ remove: t('admin.import.enumeration.remove'),
8
+ search: t('admin.import.enumeration.search'),
9
+ up: t('admin.import.enumeration.up')
10
+ }
11
+
1
12
  = render "results"
2
13
 
3
14
  = form_tag import_path(@abstract_model), :multipart => true, class: 'form-horizontal denser' do
@@ -25,19 +36,21 @@
25
36
  .col-sm-10.controls
26
37
  = select_tag 'encoding',
27
38
  options_for_select(Encoding.name_list.sort),
28
- include_blank: true, data: { enumeration: true }
39
+ include_blank: true, data: { enumeration: true, options: { regional: translations } }
29
40
  %p.help-block= t('admin.import.help.encoding', name: 'UTF-8')
30
41
  .form-group.control-group
31
42
  %label.col-sm-2.control-label= t("admin.import.update_if_exists")
32
43
  .col-sm-10.controls
33
- = check_box_tag :update_if_exists, '1', false, :class => "form-control"
44
+ = check_box_tag :update_if_exists, '1', RailsAdminImport.config.update_if_exists, :class => "form-control"
34
45
  %p.help-block= t('admin.import.help.update_if_exists')
35
46
  .form-group.control-group
36
47
  %label.col-sm-2.control-label{for: "update_lookup"}= t("admin.import.update_lookup")
37
48
  .col-sm-10.controls
38
49
  = select_tag 'update_lookup',
39
50
  options_for_select(@import_model.update_lookup_field_names,
40
- @import_model.config.mapping_key.to_s), data: { enumeration: true }
51
+ Array.wrap(@import_model.config.mapping_key).map(&:to_s)),
52
+ multiple: true,
53
+ data: { enumeration: true, options: { regional: translations } }
41
54
 
42
55
  - unless @import_model.association_fields.empty?
43
56
  %fieldset
@@ -53,7 +66,8 @@
53
66
  .col-sm-10.controls
54
67
  = select_tag "associations[#{field.name}]",
55
68
  options_for_select(@import_model.associated_model_fields(field),
56
- @import_model.associated_config(field).mapping_key.to_s), data: { enumeration: true }
69
+ Array.wrap(@import_model.associated_config(field).mapping_key).first.to_s),
70
+ data: { enumeration: true, options: { regional: translations } }
57
71
 
58
72
  %br
59
73
  .form-actions
@@ -0,0 +1,3 @@
1
+ See the [community contributed translation section of the README](https://github.com/stephskardal/rails_admin_import#community-contributed-translations) for more languages.
2
+
3
+ To contribute a new translation, please put it in a gist and submit a pull request to link your translation in the README.
@@ -1,51 +1,53 @@
1
-
2
1
  en:
3
2
  admin:
4
3
  actions:
5
4
  import:
6
- title: "Import"
7
- menu: "Import"
8
- breadcrumb: "Import"
9
- link: "Import"
10
- bulk_link: "Import"
11
- done: "Imported"
5
+ breadcrumb: Import
6
+ bulk_link: Import
7
+ done: Imported
8
+ link: Import
9
+ menu: Import
10
+ title: Import
12
11
  import:
13
- model_fields: "Model fields"
14
- association_fields: "Association fields"
15
-
16
- file: "Data file"
17
- missing_file: "You must select a file"
18
- format: "File format"
19
- invalid_format: "Invalid import format."
20
- missing_update_lookup: "Your file must contain a column for the 'Update lookup field' you selected."
21
- invalid_json: "The JSON data should be an array of records or an object with a key '%{root_key}' set to an array of records"
22
- update_if_exists: "Update if exists"
23
- update_lookup: "Update lookup field"
24
- mapping: "mapping"
25
- encoding: "Encoding"
26
- legend:
27
- fields: "Fields to import"
28
- upload: "Upload file"
29
- mapping: "Related fields mapping"
30
- import_success:
31
- create: "Created %{name}"
32
- update: "Updated %{name}"
33
- import_error:
34
- create: "Failed to create %{name}: %{error}"
35
- update: "Failed to update %{name}: %{error}"
36
- general: "Error during import: %{error}"
37
- line_item_limit: "Please limit upload file to %{limit} line items."
38
- old_import_hook: >
39
- The import hook %{model}.%{method} should take only 1 argument.
40
- Data may not imported correctly.
41
- See Upgrading section readme in Rails Admin Import.
42
- association_not_found: "Association not found. %{error}"
12
+ association_fields: Association fields
13
+ association_not_found: 'Association not found. %{error}'
14
+ encoding: Encoding
15
+ enumeration:
16
+ add: Add new
17
+ choose_all: Choose all
18
+ clear_all: Clear all
19
+ down: Down
20
+ remove: Remove
21
+ search: Search
22
+ up: Up
23
+ file: Data file
24
+ format: File format
43
25
  help:
44
- model_fields: "The fields above may be included in the import file."
45
- association_fields: >
46
- These fields map to other tables in the database, lookup via attribute selected below.
47
- For "many" associations, you may include multiple columns with the same header in the CSV file.
48
- update_if_exists: "Update records found with the lookup field below instead of creating new records"
49
- file_limit: "Please limit upload file to %{limit} line items."
50
- encoding: "Choose file encoding. Leave empty to auto-detect. Ignored for JSON."
51
-
26
+ association_fields: |
27
+ These fields map to other tables in the database, lookup via attribute selected below. For "many" associations, you may include multiple columns with the same header in the CSV file.
28
+ encoding: Choose file encoding. Leave empty to auto-detect. Ignored for JSON.
29
+ file_limit: 'Please limit upload file to %{limit} line items.'
30
+ model_fields: The fields above may be included in the import file.
31
+ update_if_exists: Update records found with the lookup field below instead of creating new records
32
+ import_error:
33
+ create: 'Failed to create %{name}: %{error}'
34
+ general: 'Error during import: %{error}'
35
+ line_item_limit: 'Please limit upload file to %{limit} line items.'
36
+ old_import_hook: |
37
+ The import hook %{model}.%{method} should take only 1 argument. Data may not imported correctly. See Upgrading section readme in Rails Admin Import.
38
+ update: 'Failed to update %{name}: %{error}'
39
+ import_success:
40
+ create: 'Created %{name}'
41
+ update: 'Updated %{name}'
42
+ invalid_format: Invalid import format.
43
+ invalid_json: 'The JSON data should be an array of records or an object with a key ''%{root_key}'' set to an array of records'
44
+ legend:
45
+ fields: Fields to import
46
+ mapping: Related fields mapping
47
+ upload: Upload file
48
+ mapping: mapping
49
+ missing_file: You must select a file
50
+ missing_update_lookup: Your file must contain a column for the 'Update lookup field' you selected.
51
+ model_fields: Model fields
52
+ update_if_exists: Update if exists
53
+ update_lookup: Update lookup field(s)
@@ -14,7 +14,7 @@ module RailsAdmin
14
14
  end
15
15
 
16
16
  register_instance_option(:default_excluded_fields) do
17
- [:id, :_id, :created_at, :updated_at, :c_at, :u_at]
17
+ [:id, :_id, :created_at, :updated_at, :c_at, :u_at, :deleted_at]
18
18
  end
19
19
  end
20
20
  end
@@ -6,11 +6,15 @@ module RailsAdminImport
6
6
  attr_accessor :logging
7
7
  attr_accessor :line_item_limit
8
8
  attr_accessor :rollback_on_error
9
+ attr_accessor :update_if_exists
9
10
  attr_accessor :header_converter
10
11
  attr_accessor :csv_options
12
+ attr_accessor :pass_filename
11
13
 
12
14
  # Default is to downcase headers and add underscores to convert into attribute names
13
15
  HEADER_CONVERTER = lambda do |header|
16
+ # check for nil/blank headers
17
+ next if header.blank?
14
18
  header.parameterize.underscore
15
19
  end
16
20
 
@@ -31,7 +35,9 @@ module RailsAdminImport
31
35
  @logging = false
32
36
  @line_item_limit = 1000
33
37
  @rollback_on_error = false
38
+ @update_if_exists = false
34
39
  @header_converter = HEADER_CONVERTER
40
+ @pass_filename = false
35
41
  @csv_options = {}
36
42
  end
37
43
  end
@@ -1,9 +1,11 @@
1
1
  require "csv"
2
+ require "charlock_holmes"
2
3
 
3
4
  module RailsAdminImport
4
5
  module Formats
5
6
  class CSVImporter < FileImporter
6
7
  Formats.register(:csv, self)
8
+ Formats.register(:CSV, self)
7
9
 
8
10
  autoload :CharDet, "rchardet"
9
11
 
@@ -15,8 +17,9 @@ module RailsAdminImport
15
17
 
16
18
  # A method that yields a hash of attributes for each record to import
17
19
  def each_record
18
- CSV.foreach(filename, csv_options) do |row|
19
- yield convert_to_attributes(row)
20
+ CSV.foreach(filename, **csv_options) do |row|
21
+ attr = convert_to_attributes(row)
22
+ yield attr unless attr.all? { |field, value| value.blank? }
20
23
  end
21
24
  end
22
25
 
@@ -51,9 +54,9 @@ module RailsAdminImport
51
54
  end
52
55
 
53
56
  def detect_encoding
54
- charset = CharDet.detect File.read(filename)
55
- if charset["confidence"] > 0.6
56
- from_encoding = charset["encoding"]
57
+ charset = CharlockHolmes::EncodingDetector.detect File.read(filename)
58
+ if charset[:confidence] > 0.6
59
+ from_encoding = charset[:encoding]
57
60
  from_encoding = "UTF-8" if from_encoding == "ascii"
58
61
  end
59
62
  from_encoding
@@ -2,6 +2,7 @@ module RailsAdminImport
2
2
  module Formats
3
3
  class JSONImporter < FileImporter
4
4
  Formats.register(:json, self)
5
+ Formats.register(:JSON, self)
5
6
 
6
7
  # A method that yields a hash of attributes for each record to import
7
8
  def each_record
@@ -4,6 +4,7 @@ module RailsAdminImport
4
4
  module Formats
5
5
  class XLSXImporter < FileImporter
6
6
  Formats.register(:xlsx, self)
7
+ Formats.register(:XLSX, self)
7
8
 
8
9
  autoload :SimpleXlsxReader, "simple_xlsx_reader"
9
10
 
@@ -18,7 +19,8 @@ module RailsAdminImport
18
19
  sheet = doc.sheets.first
19
20
  @headers = convert_headers(sheet.headers)
20
21
  sheet.data.each do |row|
21
- yield convert_to_attributes(row)
22
+ attr = convert_to_attributes(row)
23
+ yield attr unless attr.all? { |field, value| value.blank? }
22
24
  end
23
25
  end
24
26
 
@@ -33,7 +35,7 @@ module RailsAdminImport
33
35
  def convert_to_attributes(row)
34
36
  row_with_headers = @headers.zip(row)
35
37
  row_with_headers.each_with_object({}) do |(field, value), record|
36
- break if field.nil?
38
+ next if field.nil?
37
39
  field = field.to_sym
38
40
  if import_model.has_multiple_values?(field)
39
41
  field = import_model.pluralize_field(field)
@@ -7,7 +7,7 @@ module RailsAdminImport
7
7
  @logger = Logger.new(File.join(Rails.root, "log", log_file_name))
8
8
  end
9
9
  end
10
-
10
+
11
11
  def info(message)
12
12
  if RailsAdminImport.config.logging
13
13
  @logger.info message
@@ -16,7 +16,6 @@ module RailsAdminImport
16
16
  begin
17
17
  init_results
18
18
 
19
-
20
19
  if records.count > RailsAdminImport.config.line_item_limit
21
20
  return results = {
22
21
  success: [],
@@ -24,13 +23,19 @@ module RailsAdminImport
24
23
  }
25
24
  end
26
25
 
26
+ perform_global_callback(:before_import)
27
+
27
28
  with_transaction do
28
29
  records.each do |record|
29
- import_record(record)
30
+ catch :skip do
31
+ import_record(record)
32
+ end
30
33
  end
31
34
 
32
35
  rollback_if_error
33
36
  end
37
+
38
+ perform_global_callback(:after_import)
34
39
  rescue Exception => e
35
40
  report_general_error("#{e} (#{e.backtrace.first})")
36
41
  end
@@ -65,19 +70,28 @@ module RailsAdminImport
65
70
  end
66
71
 
67
72
  def import_record(record)
68
- if update_lookup && !record.has_key?(update_lookup)
73
+ if params["file"] && RailsAdminImport.config.pass_filename
74
+ record.merge!({:filename_importer => params[:file].original_filename})
75
+ end
76
+
77
+ perform_model_callback(import_model.model, :before_import_find, record)
78
+
79
+ if update_lookup && !(update_lookup - record.keys).empty?
69
80
  raise UpdateLookupError, I18n.t("admin.import.missing_update_lookup")
70
81
  end
71
82
 
72
83
  object = find_or_create_object(record, update_lookup)
84
+ return if object.nil?
73
85
  action = object.new_record? ? :create : :update
74
86
 
75
87
  begin
88
+ perform_model_callback(object, :before_import_associations, record)
76
89
  import_single_association_data(object, record)
77
90
  import_many_association_data(object, record)
78
91
  rescue AssociationNotFound => e
79
92
  error = I18n.t("admin.import.association_not_found", :error => e.to_s)
80
93
  report_error(object, action, error)
94
+ perform_model_callback(object, :after_import_association_error, record)
81
95
  return
82
96
  end
83
97
 
@@ -88,12 +102,13 @@ module RailsAdminImport
88
102
  perform_model_callback(object, :after_import_save, record)
89
103
  else
90
104
  report_error(object, action, object.errors.full_messages.join(", "))
105
+ perform_model_callback(object, :after_import_error, record)
91
106
  end
92
107
  end
93
108
 
94
109
  def update_lookup
95
110
  @update_lookup ||= if params[:update_if_exists] == "1"
96
- params[:update_lookup].to_sym
111
+ params[:update_lookup].map(&:to_sym)
97
112
  end
98
113
  end
99
114
 
@@ -169,6 +184,11 @@ module RailsAdminImport
169
184
  end
170
185
  end
171
186
 
187
+ def perform_global_callback(method_name)
188
+ object = import_model.model
189
+ object.send(method_name) if object.respond_to?(method_name)
190
+ end
191
+
172
192
  def find_or_create_object(record, update)
173
193
  field_names = import_model.model_fields.map(&:name)
174
194
  new_attrs = record.select do |field_name, value|
@@ -177,13 +197,19 @@ module RailsAdminImport
177
197
 
178
198
  model = import_model.model
179
199
  object = if update.present?
180
- model.where(update => record[update]).first
200
+ query = update.each_with_object({}) do
201
+ |field, query| query[field] = record[field]
202
+ end
203
+ model.where(query).first
181
204
  end
182
205
 
183
206
  if object.nil?
184
- object = model.new(new_attrs)
207
+ object = model.new
208
+ perform_model_callback(object, :before_import_attributes, record)
209
+ object.attributes = new_attrs
185
210
  else
186
- object.attributes = new_attrs.except(update.to_sym)
211
+ perform_model_callback(object, :before_import_attributes, record)
212
+ object.attributes = new_attrs.except(update.map(&:to_sym))
187
213
  end
188
214
  object
189
215
  end
@@ -1,3 +1,3 @@
1
1
  module RailsAdminImport
2
- VERSION = "2.0.0"
2
+ VERSION = "2.3.1"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rails_admin_import
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.0
4
+ version: 2.3.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Steph Skardal
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2016-05-05 00:00:00.000000000 Z
12
+ date: 2022-01-29 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: rails
@@ -32,6 +32,9 @@ dependencies:
32
32
  - - ">="
33
33
  - !ruby/object:Gem::Version
34
34
  version: 0.6.6
35
+ - - "<"
36
+ - !ruby/object:Gem::Version
37
+ version: 3.0.0.beta
35
38
  type: :runtime
36
39
  prerelease: false
37
40
  version_requirements: !ruby/object:Gem::Requirement
@@ -39,34 +42,23 @@ dependencies:
39
42
  - - ">="
40
43
  - !ruby/object:Gem::Version
41
44
  version: 0.6.6
42
- - !ruby/object:Gem::Dependency
43
- name: haml
44
- requirement: !ruby/object:Gem::Requirement
45
- requirements:
46
- - - "~>"
47
- - !ruby/object:Gem::Version
48
- version: '4.0'
49
- type: :runtime
50
- prerelease: false
51
- version_requirements: !ruby/object:Gem::Requirement
52
- requirements:
53
- - - "~>"
45
+ - - "<"
54
46
  - !ruby/object:Gem::Version
55
- version: '4.0'
47
+ version: 3.0.0.beta
56
48
  - !ruby/object:Gem::Dependency
57
- name: rchardet
49
+ name: charlock_holmes
58
50
  requirement: !ruby/object:Gem::Requirement
59
51
  requirements:
60
52
  - - "~>"
61
53
  - !ruby/object:Gem::Version
62
- version: '1.6'
54
+ version: '0.7'
63
55
  type: :runtime
64
56
  prerelease: false
65
57
  version_requirements: !ruby/object:Gem::Requirement
66
58
  requirements:
67
59
  - - "~>"
68
60
  - !ruby/object:Gem::Version
69
- version: '1.6'
61
+ version: '0.7'
70
62
  - !ruby/object:Gem::Dependency
71
63
  name: simple_xlsx_reader
72
64
  requirement: !ruby/object:Gem::Requirement
@@ -96,6 +88,7 @@ files:
96
88
  - app/views/rails_admin/main/_results.html.haml
97
89
  - app/views/rails_admin/main/_section.html.haml
98
90
  - app/views/rails_admin/main/import.html.haml
91
+ - config/locales/README.md
99
92
  - config/locales/import.en.yml
100
93
  - lib/rails_admin_import.rb
101
94
  - lib/rails_admin_import/action.rb
@@ -135,7 +128,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
135
128
  version: '0'
136
129
  requirements: []
137
130
  rubyforge_project:
138
- rubygems_version: 2.4.5
131
+ rubygems_version: 2.7.6
139
132
  signing_key:
140
133
  specification_version: 4
141
134
  summary: Import functionality for Rails Admin