gunwale 0.5.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 83044c55ce9d087739769f71c931c017cacd6d99b41bf60ee55015702c9204bb
+  data.tar.gz: aef66c8ba50fe24a8cf184d5c2d08a58e2d42af39f287aa3ca88c049316f55ca
+SHA512:
+  metadata.gz: 9e091778396ddb89c50142472c4aa2b4bbfb0a81158f76db13b2a57ab30e4cc930279e7b8806c63f0b8e434ab6d5356a1994e0bb136e9b734332f95e31b26085
+  data.tar.gz: 7a223936945200c527840a0f472d23364360da51bbcf69fac890f1a49f733ecb6b38447498c8f99bc7f07a8ac0f9ec00dab7f73b9df7693eadc9d8ac1a060821

data/.gitignore

@@ -0,0 +1,12 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+
+# rspec failure tracking
+.rspec_status

data/.rspec

@@ -0,0 +1,2 @@
+--format progress
+--color

data/.rubocop.yml

@@ -0,0 +1,63 @@
+AllCops:
+  Include:
+    - "**/Rakefile"
+  Exclude:
+    - db/**/*
+    - gemfiles/**/*
+    - vendor/**/*
+Lint/SpaceBeforeFirstArg:
+  Enabled: false
+Lint/UnusedBlockArgument:
+  Enabled: false
+Lint/UnusedMethodArgument:
+  Enabled: false
+Metrics/AbcSize:
+  Enabled: false
+Metrics/ClassLength:
+  Enabled: false
+Metrics/CyclomaticComplexity:
+  Enabled: false
+Metrics/LineLength:
+  Enabled: false
+Metrics/MethodLength:
+  Enabled: false
+Metrics/ModuleLength:
+  Enabled: false
+Metrics/PerceivedComplexity:
+  Enabled: false
+Metrics/BlockLength:
+  Enabled: false
+Security/YAMLLoad:
+  Enabled: false
+Style/AlignParameters:
+  Enabled: false
+Style/ClassAndModuleChildren:
+  Enabled: false
+Style/ClassVars:
+  Enabled: false
+Style/Documentation:
+  Enabled: false
+Style/DoubleNegation:
+  Enabled: false
+Style/FileName:
+  Enabled: false
+Style/GuardClause:
+  Enabled: false
+Style/IndentHash:
+  Enabled: false
+Style/NilComparison:
+  Enabled: false
+Style/OpMethod:
+  Enabled: false
+Style/RescueModifier:
+  Enabled: false
+Style/SignalException:
+  Enabled: false
+Style/SingleLineMethods:
+  Enabled: false
+Style/SpaceAroundEqualsInParameterDefault:
+  Enabled: false
+Style/StringLiterals:
+  EnforcedStyle: double_quotes
+Performance/EndWith:
+  Enabled: false

data/.ruby-version

@@ -0,0 +1 @@
+3.2.2

data/.travis.yml

@@ -0,0 +1,18 @@
+dist: trusty
+sudo: false
+language: ruby
+rvm:
+  - 2.3.3
+  - 2.4.1
+addons:
+  postgresql: '9.6'
+before_install:
+  - gem update --system
+  - gem --version
+  - gem install bundler -v 1.14.6
+before_script:
+  - bundle exec rake db:create
+  - bundle exec rake db:migrate
+cache: bundler
+gemfile:
+  - gemfiles/csv_import_1.1_and_0.18.gemfile

data/API.md

@@ -0,0 +1,387 @@
+# RowBoat API
+
+This is really more of a summary of what you can do with `RowBoat::Base` since you subclass it to do everything :)
+
+## Contents
+
+- [Basic Usage](#basic-usage)
+- [`.import`](#import)
+- [`initialize`](#initialize)
+- [`import`](#import-1)
+- [`import_into`](#import_into)
+- [`csv_source`](#csv_source)
+- [`column_mapping`](#column_mapping)
+- [`preprocess_row`](#preprocess_row)
+- [`preprocess_rows`](#preprocess_rows)
+- [`options`](#options)
+- [`handle_failed_row`](#handle_failed_row)
+- [`handle_failed_rows`](#handle_failed_rows)
+- [`value_converters`](#value_converters)
+- [`rollback_transaction?`](#rollback_transaction)
+
+## Basic Usage
+
+Just subclass `RowBoat::Base` and define the [`import_into`](#import_into) and [`column_mapping`](#column_mapping) methods to get started (They're the only methods that you're required to implement).
+
+```ruby
+class ImportProduct < RowBoat::Base
+  def import_into
+    Product
+  end
+
+  def column_mapping
+    {
+      downcased_csv_column_header: :model_attribute_name,
+      another_downcased_csv_column_header: :another_model_attribute_name
+    }
+  end
+end
+```
+
+## `.import`
+
+### Description
+Imports database records form the given CSV-like object. The CSV-like object can be anything that can be passed to [`SmarterCSV.process`](https://github.com/tilo/smarter_csv#documentation) (string paths to files, files, tempfiles, instances of StringIO, etc).
+
+It returns a hash containing
+
+- `:invalid_records` - an array of all records that failed to import since they were invalid. If you've configured the `:validate` option to be `false` it will be an empty array.
+- `:total_inserts` - the total number of database inserts that were run.
+- `:inserted_ids` - an array of all of the ids of records inserted into the database.
+- `:skipped_rows` - every row skipped by returning `nil` from [`preprocess_row`](#preprocess_row).
+
+If you want to pass additional information to help import CSVs, *don't override this method*. It just passes through to [`initialize`](#initialize) so override that :)
+
+### Example
+
+#### Basic Use
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # required configuration omitted for brevity
+end
+
+ImportProduct.import("path/to/my.csv")
+```
+
+#### Advanced Use
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # required configuration omitted for brevity
+  def intitialize(csv_source, my_options)
+    super(csv_source)
+    @my_options = my_options
+  end
+end
+
+ImportProduct.import("path/to/my.csv", foo: "bar")
+```
+
+## `initialize`
+
+### Description
+
+Makes a new instance with the given CSV-like object. See [`.import`](#import) for more details around when and how to override this method.
+
+## `import`
+
+### Description
+
+The instance method that actually parses and imports the CSV. Generally, you wouldn't call this directly and would instead call [`.import`](#import).
+
+## `import_into`
+
+### Description
+
+It is required that you override this method to return whatever ActiveRecord class you want your CSV imported into.
+
+### Example
+
+#### Basic Use
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # other required configuration omitted for brevity
+  def import_into
+    Product
+  end
+end
+```
+
+#### Advanced Use
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # other required configuration omitted for brevity
+  def import_into
+    if csv_source.is_a?(String) && csv_source.match(/category/i)
+      ProductCategory
+    else
+      Product
+    end
+  end
+end
+
+ImportProduct.import("path/to/category.csv")
+ImportProduct.import("path/to/product.csv")
+```
+
+
+## `csv_source`
+
+### Description
+
+Whatever you originally passed in as the CSV source.
+
+### Example
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # other required configuration omitted for brevity
+  def import_into
+    # `csv_source` is available in any of our instance methods
+    if csv_source.is_a?(String) && csv_source.match(/category/i)
+      ProductCategory
+    else
+      Product
+    end
+  end
+end
+
+ImportProduct.import("path/to/category.csv")
+ImportProduct.import("path/to/product.csv")
+```
+
+## `column_mapping`
+
+### Description
+
+It is required that you override this method with either a hash that maps columns in your CSV to their preferred names or an array of your preferred column names.
+
+By default when using a hash
+- CSV column names are downcased symbols of what they look like in the CSV.
+- CSV columns that are not mapped are ignored when processing the CSV.
+
+If you're familiar with [SmarterCSV](https://github.com/tilo/smarter_csv#documentation), this method essentially defines your `:key_mapping` with the `:remove_unmapped_keys` setting set to `true` when provided with a hash. When given an array it is the `:user_provided_headers` option.
+
+You can change these defaults by overriding the [`options`](#options) method.
+
+### Example
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # other required configuration omitted for brevity
+  def column_mapping
+    {
+      prdct_nm: :name,
+      "price/cost_amnt": :price_in_cents
+    }
+  end
+end
+
+# or...
+
+class ImportProduct < RowBoat::Base
+  # other required configuration omitted for brevity
+  def column_mapping
+    [:name, :price_in_cents]
+  end
+end
+```
+
+## `preprocess_row`
+
+### Description
+
+Implement this method if you need to do some work on the row before the record is inserted/updated.
+
+If you return `nil` from this method, the row will be skipped in the import. You can access these rows in the return value from [`.import`](#import) (the `:skipped_rows` key).
+
+You also have access to `row_number` here.
+
+If the work you intend to do with the row only requires changing one attribute, it is recommended that you override [`value_converters`](#value_converters) instead of this.
+
+### Example
+
+```ruby
+  class ImportProduct < RowBoat::Base
+    # required configuration omitted for brevity
+    def preprocess_row(row)
+      { position: row_number }.merge(row)
+    end
+    # or...
+    def preprocess_row(row)
+      if row[:name] && row[:price]
+        row
+      else
+        nil
+      end
+    end
+  end
+```
+
+## `preprocess_rows`
+
+### Description
+
+Override this method if you need to do something with a chunk of rows (the chunk size is determined by the `:chunk_size` option in the [`options`](#options) method).
+
+If you need to filter particular rows, it's better to just implement [`preprocess_row`](#preprocess_row) and return `nil` for the rows you want to ignore.
+
+### Example
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # required configuration omitted for brevity
+  def preprocess_rows(rows)
+    if skip_batch?(rows)
+      super([])
+    else
+      super
+    end
+  end
+
+  def skip_batch?(rows)
+    # decide whether or not to skip the batch
+  end
+end
+```
+
+## `options`
+
+### Description
+
+Implement this to configure RowBoat, [SmarterCSV](https://github.com/tilo/smarter_csv), and [activerecord-import](https://github.com/zdennis/activerecord-import).
+
+Except for `:wrap_in_transaction`, all options pass through to SmarterCSV and activerecord-import.
+
+`:wrap_in_transaction` simply tells RowBoat whether or not you want your whole import wrapped in a database transaction.
+
+Whatever you define in this method will be merged into the defaults:
+
+  - `:chunk_size` - `500`
+  - `:key_mapping` - `column_mapping`
+  - `:recursive` - `false`
+  - `:remove_unmapped_keys` - `true`
+  - `:validate` - `true`
+  - `:value_converters` - `csv_value_converters`
+  - `:wrap_in_transaction` - `true`
+
+Don't provide `value_converters` or `key_mapping` options here. Implement the [`value_converters`](#value_converters) and [`column_mapping`](#column_mapping) respectively.
+
+### Example
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # required configuration omitted for brevity
+  def options
+    {
+      chunk_size: 1000,
+      validate: false,
+      wrap_in_transaction: false
+    }
+  end
+end
+```
+
+## `handle_failed_row`
+
+### Description
+
+Implement this to do some work with a row that has failed to import.
+
+It's important to note that
+- This happens after the import has completed.
+- The given row is an instance of whatever class was returned by [`import_into`](#import_into).
+
+These records are also available in the return value of [`.import`](#import).
+
+### Example
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # required configuration omitted for brevity
+  def handle_failed_row(row)
+    puts row.errors.full_messages.join(", ")
+  end
+end
+```
+
+## `handle_failed_rows`
+
+### Description
+
+Override this method to do some work will all of the rows that failed to import.
+
+### Example
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # required configuration omitted for brevity
+  def handle_failed_rows(rows)
+    puts "Failed to import #{rows.size} rows :("
+    super
+  end
+end
+```
+
+## `value_converters`
+
+### Description
+
+Implement to specify how to translate values from the CSV into whatever sorts of objects you need.
+
+Simply return a hash that has the mapped column name (ie, what you mapped it to in the [`column_mapping`](#column_mapping) method) as a key pointing to either
+- a method name as a symbol
+- a proc or lambda
+- an object that implements `convert`
+
+Regardless of which one you choose, it takes a value and returns a converted value.
+
+This is essentially a sugared up version of `:value_converters` option in [SmarterCSV](https://github.com/tilo/smarter_csv#documentation).
+
+### Example
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # required configuration omitted for brevity
+  def value_converters
+    {
+      sell_by: :convert_date,
+      name: -> (value) { value.titlelize },
+      price: proc { |value| value.to_i },
+      description: DescriptionConverter
+    }
+  end
+
+  def convert_date(value)
+    Date.parse(value) rescue nil
+  end
+end
+
+module DescriptionConverter
+  def self.convert(value)
+    value.present? ? value : "default description :("
+  end
+end
+```
+
+## `rollback_transaction?`
+
+### Description
+
+Implement this method if you'd like to rollback the transaction after it otherwise has completed.
+
+Note: imports are only wrapped in a transaction if the `wrap_in_transaction` option is `true`. It defaults to `true` but this can be configured in [`options`](#options)
+
+### Example
+
+```ruby
+class ImportProduct < RowBoat::Base
+  # required configuration omitted for brevity
+  def rollback_transaction?
+    CsvService.already_imported?(csv_source)
+  end
+end
+```

data/Appraisals

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+appraise "csv_import_1.1_and_0.18" do
+  gem "activerecord-import", "~> 0.18.2"
+  gem "smarter_csv", "~> 1.1.0"
+end

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at michael.crismali@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in row_boat.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2017 Michael Crismali
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.