rails_admin_import 0.1.9 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b11e6300c09b071ce9b711b6b3fd3cda25aa5520
-  data.tar.gz: a462c3e1016ca867f092d0e9219188f693af4c4c
+  metadata.gz: 7926c16cbb6726c90f684e87620b811157065162
+  data.tar.gz: 99dbbe4c5a95dc163a870ceeff25bcba0801e97d
 SHA512:
-  metadata.gz: d38fd170f28cd50af1b5bfb632bb32c3dd8c98a5b170e4c11e7bb4a614730872abd9f79592e40c401d4398283dffc013daab13f4a322dc384d42c0a3da68527e
-  data.tar.gz: a13f5db4ae247c17ee7a67e8db890356d62b75d473da8ba93662382b97c2677e593f9a19b75ec4fa8fc31f087dfcc2e09f0822fa81f6760397d394aa17356f94
+  metadata.gz: 620e0bf33570d47b6b93ea5b8b41f96524153d57b302bd08af316733cb936869f92cb64013009bdb228317b2478961a48a05e4ce90a15bcc729fe2e74b1dff7a
+  data.tar.gz: bd7a26c0ee8300e915ba4cf95146220e24b6b56998b4c43bb19b8d68a5237561ebfb78551059ba10a156571972f69bb809c47b5a50e1ff34723015508289fafb

data/MIT-LICENSE

@@ -1,4 +1,4 @@
-Copyright 2012 YOURNAME
+Copyright 2012 Steph Skardal
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

data/README.md

@@ -1,74 +1,217 @@
-Request for Contributors / Core Contributors
-========
+# Rails Admin Import
 
-I apologize for my extreme lack of attention to this repository since it was created. I see that several users have forked this gem and applied updates. I'd be interested in giving access to this main repository for any interested in maintaining it and/or adding features. Please contact me if you are interested at steph at endpoint dot com. 
+[![Build Status](https://travis-ci.org/monkbroc/rails_admin_import.svg?branch=master)](https://travis-ci.org/monkbroc/rails_admin_import)
 
+Plugin functionality to add generic import to Rails Admin from CSV and JSON files
 
-Rails Admin Import functionality
-========
+*This Readme is for version 1.0. If you are still using version 0.1.x, see [this branch](https://github.com/stephskardal/rails_admin_import/tree/legacy)*
 
-Plugin functionality to add generic import to Rails Admin interface
-
-Installation
-========
+## Installation
 
 * First, add to Gemfile:
-    
-        gem "rails_admin_import"
 
-* Next, mount in your application by adding following line to your config/routes.rb:
+```
+gem "rails_admin_import", "~> 1.0.0"
+```
+
+* Define configuration in `config/initializers/rails_admin_import.rb`:
+
+```ruby
+RailsAdmin.config do |config|
+  # REQUIRED:
+  # Include the import action
+  # See https://github.com/sferik/rails_admin/wiki/Actions
+  config.actions do
+    all
+    import
+  end
+
+  # Optional:
+  # Configure global RailsAdminImport options
+  config.configure_with(:import) do |config|
+    config.logging = true
+  end
+
+  # Optional:
+  # Configure model-specific options using standard RailsAdmin DSL
+  # See https://github.com/sferik/rails_admin/wiki/Railsadmin-DSL
+  config.model 'User' do
+    import do
+      include_all_fields
+      exclude_fields :secret_token
+    end
+  end
+end
+```
+
+* If you are using CanCanCan for authorization, add to ability.rb to specify which models can be imported:
+
+```ruby
+can :import, [User, Model1, Model2]
+```
+
+## File format
+
+### CSV
+
+The first line must contain attribute names. They will be converted to lowercase and underscored (First Name ==> first_name).
+
+For "many" associations, you may include multiple columns with the same header in the CSV file.
+
+The repeated header may be singular or plural. For example, for a "children" association, you may have multiple "child" columns or multiple "children" column, each containing one lookup value for an associated record. Blank values are ignored.
+
+Example
+
+```
+First name,Last name,Team,Team
+Peter,Gibbons,IT,Management
+Michael,Bolton,IT,
+```
+
+### JSON
+
+The file must be an array or an object with a root key the same name as the plural model name, i.e. the default Rails JSON output format with include_root_in_json on or off.
+
+
+## Configuration
+
+### Global configuration options
+
+* __logging__ (default `false`): Save a copy of each imported file to log/import and a detailed import log to log/rails_admin_import.log
+
+* __line_item_limit__ (default `1000`): max number of items that can be imported at one time. TODO: Currently this is suggested but not enforced.
+
+* __rollback_on_error__ (default `false`): import records in a transaction and rollback if there is one error. Only for ActiveRecord, not Mongoid.
+
+* __header_converter__ (default `nil`): a lambda to convert each CSV header text string to a model attribute name. The default header converter converts to lowercase and replaces spaces with underscores.
+
+### Model-specific configuration
 
-        mount RailsAdminImport::Engine => '/rails_admin_import', :as => 'rails_admin_import'
+Use [standard RailsAdmin DSL](https://github.com/sferik/rails_admin/wiki/Railsadmin-DSL) to add or remove fields.
 
-* If you are using cancan, add to ability.rb to specify which models can be imported:
+* To change the default attribute that will be used to find associations on import, set `mapping_key` (default attribute is `name`)
 
-        can :import, [User, Model1, Model2]
+```ruby
+RailsAdmin.config do |config|
+  config.model 'Ball' do
+    import do
+      mapping_key :color
+    end
+  end
+end
+```
 
-* Define configuration in config/initializers/rails_admin_import.rb:
+* To include a specific list of fields:
 
-        RailsAdminImport.config do |config| 
-          config.model User do
-            excluded_fields do
-              [:field1, :field2, :field3]
-            end
-            label :name
-            extra_fields do
-              [:field3, :field4, :field5]
-            end
-          end
-        end
+```ruby
+RailsAdmin.config do |config|
+  config.model 'User' do
+    import do
+      field :first_name
+      field :last_name
+      field :email
+    end
+  end
+end
+```
 
-* (Optional) Define instance methods to be hooked into the import process, if special/additional processing is required on the data:
+* To exclude specific fields:
 
-        # some model
-        def before_import_save(row, map)
-          # Your custom special sauce          
-        end
+```ruby
+RailsAdmin.config do |config|
+  config.model 'User' do
+    import do
+      include_all_fields
+      exclude_fields :secret_token
+    end
+  end
+end
+```
 
-        def after_import_save(row, map)
-          # Your custom special sauce          
-        end
+* To add extra fields that will be set as attributes on the model and that will be passed to the import hook methods:
+
+```ruby
+RailsAdmin.config do |config|
+  config.model 'User' do
+    import do
+      include_all_fields
+      fields :special_import_token
+    end
+  end
+end
+```
+
+## Import hooks
+
+
+Define instance methods on your models to be hooked into the import process, if special/additional processing is required on the data:
+
+```ruby
+# some model
+class User < ActiveRecord::Base
+  def before_import_save(record)
+    # Your custom special sauce          
+  end
+
+  def after_import_save(record)
+    # Your custom special sauce          
+  end
+end
+```
+
+For example, you could
+
+* Set an attribute on a Devise User model to skip checking for a password when importing a new model.
+
+* Download a file based on a URL from the import file and set a Paperclip file attribute on the model.
+
+
+## ORM: ActiveRecord and Mongoid
+
+The gem is tested to work with ActiveRecord and Mongoid.
+
+Support for Mongoid is early, so if you can suggest improvements (especially around importing embedded models), open an issue.
+
+
+## Upgrading
+
+* Move global config to `config.configure_with(:import)` in `config/initializers/rails_admin_import.rb`.
+
+* Move the field definitions to `config.model 'User' do; import do; // ...` in `config/initializers/rails_admin_import.rb`.
+
+* No need to mount RailsAdminImport in `config/routes.rb` (RailsAdmin must still be mounted).
+
+* Update model import hooks to take 1 hash argument instead of 2 arrays with values and headers.
+
+* Support for importing file attributes was removed since I couldn't understand how it works. It should be possible to reimplement it yourself using post import hooks. Open an issue to discuss how to put back support for importing files into the gem.
+
+## Run tests
+
+1. Clone the repository to your machine
+
+    git clone https://github.com/stephskardal/rails_admin_import
+    
+2. Run `bundle install`
+3. Run `rspec`
+  
+The structure of the tests is taken from the Rails Admin gem.
 
-* "import" action must be added inside config.actions block in main application RailsAdmin configuration: config/initializers/rails_admin.rb.
+## Authors
 
-        config.actions do
-          ...
-          import
-          ...
-        end
+Original author: [Steph Skardal](https://github.com/stephskardal)
 
-  Refer to [RailAdmin documentation on custom actions](https://github.com/sferik/rails_admin/wiki/Actions) that must be present in this block.
+Maintainer (since May 2015): [Julien Vanier](https://github.com/monkbroc)
 
 
-* TODO: Right now, import doesn't work for fields ending in s, because inflector fails in models ending in s singularly. Belongs_to and many
-  mapping needs to be updated to use klasses instead of symbols
+## Contributing
 
-TODO
-========
+Everyone is encouraged to help improve this project. Here are a few ways you can help:
 
-* Testing
+- [Report bugs](https://github.com/ankane/blazer/issues)
+- Fix bugs and [submit pull requests](https://github.com/ankane/blazer/pulls)
+- Write, clarify, or fix documentation
+- Suggest or add new features
 
-Copyright
-========
+## Copyright
 
-Copyright (c) 2014 End Point & Steph Skardal. See LICENSE.txt for further details.
+Copyright (c) 2015 End Point, Steph Skardal and contributors. See LICENSE.txt for further details.

data/Rakefile

@@ -1,39 +1,15 @@
 #!/usr/bin/env rake
-begin
-  require 'bundler/setup'
-rescue LoadError
-  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
-end
-begin
-  require 'rdoc/task'
-rescue LoadError
-  require 'rdoc/rdoc'
-  require 'rake/rdoctask'
-  RDoc::Task = Rake::RDocTask
-end
-
-RDoc::Task.new(:rdoc) do |rdoc|
-  rdoc.rdoc_dir = 'rdoc'
-  rdoc.title    = 'RailsAdminImport'
-  rdoc.options << '--line-numbers'
-  rdoc.rdoc_files.include('README.rdoc')
-  rdoc.rdoc_files.include('lib/**/*.rb')
-end
-
-APP_RAKEFILE = File.expand_path("../test/dummy/Rakefile", __FILE__)
-
-
 
+require 'bundler'
 Bundler::GemHelper.install_tasks
 
-require 'rake/testtask'
+# Piggyback off the Rails Admin rake tasks to set up the CI environment
+spec = Gem::Specification.find_by_name 'rails_admin'
+Dir["#{spec.gem_dir}/lib/tasks/*.rake"].each { |rake| load rake }
 
-Rake::TestTask.new(:test) do |t|
-  t.libs << 'lib'
-  t.libs << 'test'
-  t.pattern = 'test/**/*_test.rb'
-  t.verbose = false
-end
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec)
 
+task test: :spec
 
-task :default => :test
+task default: :spec

data/app/views/rails_admin/main/_results.html.haml

@@ -0,0 +1,18 @@
+- if @results && @results[:success].any?
+  .alert.alert-success.form-horizontal.denser
+    %fieldset
+      %legend
+        %i.icon-chevron-right
+        = @results[:success_message]
+      %ul.control-group{style: 'display: none'}
+        - @results[:success].each do |message|
+          %li= message
+- if @results && @results[:error].any?
+  .alert.alert-danger.form-horizontal.denser
+    %fieldset
+      %legend
+        %i.icon-chevron-down
+        = @results[:error_message]
+      %ul.control-group
+        - @results[:error].each do |message|
+          %li= message

data/app/views/rails_admin/main/_section.html.haml

@@ -0,0 +1,10 @@
+- if fields.any?
+  .form-group.control-group
+    %label.col-sm-2.control-label= t("admin.import.#{section}")
+    .col-sm-10.controls
+      %ul.list-unstyled
+        - fields.each do |field|
+          %li
+            %label= capitalize_first_letter(field.label)
+      %p.help-block= t("admin.import.help.#{section}")
+

data/app/views/rails_admin/main/import.html.haml

@@ -0,0 +1,72 @@
+= render "results"
+
+= form_tag import_path(@abstract_model), :multipart => true, class: 'form-horizontal denser' do
+
+  %input{name: "send_data", type: "hidden", value: "true"}/
+  %fieldset
+    %legend
+      %i.icon-chevron-down
+      = t('admin.import.legend.fields')
+
+    = render "section", section: "model_fields", fields: @import_model.model_fields
+    = render "section", section: "association_fields", fields: @import_model.association_fields
+
+  %fieldset
+    %legend
+      %i.icon-chevron-down
+      = t('admin.import.legend.upload')
+    .form-group.control-group
+      %label.col-sm-2.control-label{for: "file"}= t("admin.import.file")
+      .col-sm-10.controls
+        = file_field_tag :file, :class => "form-control"
+        %p.help-block= t('admin.import.help.file_limit', limit: RailsAdminImport.config.line_item_limit)
+    .form-group.control-group
+      %label.col-sm-2.control-label{for: "import_format"}= t("admin.import.format")
+      .col-sm-10.controls
+        = select_tag 'import_format',
+          options_for_select(RailsAdminImport::Formats.all),
+          data: { enumeration: true }
+    .form-group.control-group
+      %label.col-sm-2.control-label{for: "encoding"}= t("admin.import.encoding")
+      .col-sm-10.controls
+        = select_tag 'encoding',
+          options_for_select(RailsAdmin::CSVConverter::TARGET_ENCODINGS),
+          include_blank: true, data: { enumeration: true }
+        %p.help-block= t('admin.import.help.encoding', name: 'UTF-8')
+    .form-group.control-group
+      %label.col-sm-2.control-label= t("admin.import.update_if_exists")
+      .col-sm-10.controls
+        = check_box_tag :update_if_exists, '1', false, :class => "form-control"
+        %p.help-block= t('admin.import.help.update_if_exists')
+    .form-group.control-group
+      %label.col-sm-2.control-label{for: "update_lookup"}= t("admin.import.update_lookup")
+      .col-sm-10.controls
+        = select_tag 'update_lookup',
+          options_for_select(@import_model.model_fields.map(&:name),
+          @import_model.config.mapping_key.to_s), data: { enumeration: true }
+
+  - unless @import_model.association_fields.empty?
+    %fieldset
+      %legend
+        %i.icon-chevron-down
+        = t('admin.import.legend.mapping')
+
+      - @import_model.association_fields.each do |field|
+        .form-group.control-group
+          %label.col-sm-2.control-label
+            = capitalize_first_letter(field.label)
+            = t("admin.import.mapping")
+          .col-sm-10.controls
+            = select_tag "associations[#{field.name}]",
+              options_for_select(@import_model.associated_model_fields(field).map(&:name),
+              @import_model.associated_config(field).mapping_key.to_s), data: { enumeration: true }
+
+  %br
+  .form-actions
+    %input{type: :hidden, name: 'return_to', value: (request.params[:return_to].presence || request.referer)}
+    %button.btn.btn-primary{type: "submit", name: "commit", data: {disable_with: "Uploading..."} }
+      %i.icon-white.icon-ok
+      = t("admin.form.save")
+    %button.btn{type: "submit", name: "_continue"}
+      %i.icon-remove
+      = t("admin.form.cancel")

data/config/locales/import.en.yml

@@ -9,3 +9,42 @@ en:
         link: "Import"
         bulk_link: "Import"
         done: "Imported"
+    import:
+      model_fields: "Model fields"
+      association_fields: "Association fields"
+
+      file: "Data file"
+      missing_file: "You must select a file"
+      format: "File format"
+      invalid_format: "Invalid import format."
+      missing_update_lookup: "Your file must contain a column for the 'Update lookup field' you selected."
+      invalid_json: "The JSON data should be an array of records or an object with a key '%{root_key}' set to an array of records"
+      update_if_exists: "Update if exists"
+      update_lookup: "Update lookup field"
+      mapping: "mapping"
+      encoding: "Encoding"
+      legend:
+        fields: "Fields to import"
+        upload: "Upload file"
+        mapping: "Related fields mapping"
+      import_success:
+        create: "Created %{name}"
+        update: "Updated %{name}"
+      import_error:
+        create: "Failed to create %{name}: %{error}"
+        update: "Failed to update %{name}: %{error}"
+        general: "Error during import: %{error}"
+        old_import_hook: >
+          The import hook %{model}.%{method} should take only 1 argument.
+          Data may not imported correctly.
+          See Upgrading section readme in Rails Admin Import.
+      association_not_found: "Association not found. %{error}"
+      help:
+        model_fields: "The fields above may be included in the import file."
+        association_fields: >
+          These fields map to other tables in the database, lookup via attribute selected below.
+          For "many" associations, you may include multiple columns with the same header in the CSV file.
+        update_if_exists: "Update records found with the lookup field below instead of creating new records"
+        file_limit: "Please limit upload file to %{limit} line items."
+        encoding: "Choose file encoding. Leave empty to auto-detect. Ignored for JSON."
+

data/lib/rails_admin_import/action.rb

@@ -0,0 +1,45 @@
+require "rails_admin/config/actions"
+
+module RailsAdmin
+  module Config
+    module Actions
+      class Import < Base
+        RailsAdmin::Config::Actions.register(self)
+
+        register_instance_option :collection do
+          true
+        end
+
+        register_instance_option :http_methods do
+          [:get, :post]
+        end
+
+        register_instance_option :controller do
+          proc do
+            @import_model = RailsAdminImport::ImportModel.new(@abstract_model)
+
+            if request.post?
+              record_importer = RailsAdminImport::Formats.for(
+                params[:import_format] || "csv", @import_model, params)
+
+              if record_importer.valid?
+                importer = RailsAdminImport::Importer.new(
+                  @import_model, params)
+
+                @results = importer.import(record_importer.each)
+              else
+                flash[:error] = record_importer.error
+              end
+            end
+
+            render action: @action.template_name
+          end
+        end
+
+        register_instance_option :link_icon do
+          "icon-folder-open"
+        end
+      end
+    end
+  end
+end

data/lib/rails_admin_import/config/legacy_model.rb

@@ -0,0 +1,42 @@
+module RailsAdminImport
+  module Config
+    class LegacyModel
+      attr_reader :model_name
+      def initialize(model_name)
+        @model_name = model_name
+      end
+
+      def label(_value)
+        # Ignored now
+        # RailsAdmin object_label_method will be used
+      end
+
+      def mapping_key(value)
+        config = RailsAdmin.config(model_name)
+        config.mapping_key(value)
+      end
+
+      def excluded_fields(values)
+        config = RailsAdmin.config(model_name)
+
+        # Call appropriate Rails Admin field list methods
+        config.import do
+          include_all_fields
+          exclude_fields *values
+        end
+      end
+
+      def extra_fields(values)
+        config = RailsAdmin.config(model_name)
+
+        # Call appropriate Rails Admin field list methods
+        config.import do
+          include_all_fields
+          values.each do |value|
+            field value
+          end
+        end
+      end
+    end
+  end
+end

data/lib/rails_admin_import/config/sections/import.rb

@@ -0,0 +1,30 @@
+require "rails_admin/config/sections/base"
+
+module RailsAdmin
+  module Config
+    module Sections
+      # Configuration of the navigation view
+      class Import < RailsAdmin::Config::Sections::Base
+        register_instance_option(:mapping_key) do
+          :name
+        end
+
+        register_instance_option(:default_excluded_fields) do
+          [:id, :_id, :created_at, :updated_at]
+        end
+      end
+    end
+  end
+end
+
+section = RailsAdmin::Config::Sections::Import
+name = :import
+
+# Manually add to Rails Admin as a model configuration section until
+# there is a better API to do this
+RailsAdmin::Config::Model.send(:define_method, name) do |&block|
+  @sections = {} unless @sections
+  @sections[name] = section.new(self) unless @sections[name]
+  @sections[name].instance_eval(&block) if block
+  @sections[name]
+end

data/lib/rails_admin_import/config.rb

@@ -1,52 +1,31 @@
-require 'rails_admin_import/config/model'
-# require 'active_support/core_ext/class/attribute_accessors'
+require "rails_admin_import/config/legacy_model"
 
 module RailsAdminImport
   module Config
     class << self
-      # Stores model configuration objects in a hash identified by model's class
-      # name.
-      #
-      # @see RailsAdminImport::Config.model
-      attr_reader :registry
       attr_accessor :logging
       attr_accessor :line_item_limit
+      attr_accessor :rollback_on_error
+      attr_accessor :header_converter
 
-      # Loads a model configuration instance from the registry or registers
-      # a new one if one is yet to be added.
-      #
-      # First argument can be an instance of requested model, its class object,
-      # its class name as a string or symbol or a RailsAdminImport::AbstractModel
-      # instance.
-      #
-      # If a block is given it is evaluated in the context of configuration instance.
-      #
-      # Returns given model's configuration
-      #
-      # @see RailsAdminImport::Config.registry
-      def model(entity, &block)
-        key = entity.name.to_sym
-
-        config = @registry[key] ||= RailsAdminImport::Config::Model.new(entity)
-        config.instance_eval(&block) if block
-        config
+      def model(model_name, &block)
+        unless @deprecation_shown
+          warn "RailsAdminImport::Config#model is deprecated. " \
+            "Add a import section for your model inside the rails_admin " \
+            "config block. See the Readme.md for more details"
+          @deprecation_shown = true
+        end
+        legacy_config = RailsAdminImport::Config::LegacyModel.new(model_name)
+        legacy_config.instance_eval(&block) if block
+        legacy_config
       end
-            
+
       # Reset all configurations to defaults.
-      #
-      # @see RailsAdminImport::Config.registry
       def reset
-        @registry = {}
         @logging = false
         @line_item_limit = 1000
-      end
-
-      # Reset a provided model's configuration.
-      #
-      # @see RailsAdminImport::Config.registry
-      def reset_model(model)
-        key = model.kind_of?(Class) ? model.name.to_sym : model.to_sym
-        @registry.delete(key)
+        @rollback_on_error = false
+        @header_converter = nil
       end
     end