row_boat 0.1.0.alpha.3 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 5341bb8c702726b4d6dd4ed105339121815f41ab
4
- data.tar.gz: 8bc96cb40cc2f160df283acc73d6e9b767212cad
3
+ metadata.gz: 42b9a926cbca073ac25906f8f3ca224d2d665b62
4
+ data.tar.gz: 28e138c7db5dfa6e8ef4139c82033551da7b6dfe
5
5
  SHA512:
6
- metadata.gz: 4c75ea55754040da725c9919e70f4d84efecb261db688d9f00142d454d3931bb8e81f8c99903297029349a75f627d68b2d9847ed32663f97901334f00f1baccf
7
- data.tar.gz: f12351b6332d266d841ad07941fe415c0a5a9d23a35cc1e54996cafda6c91cd5ef4c629ea7fffb48419122fa38137cbe16cd2d0711a76222218b8b8dad29d9bc
6
+ metadata.gz: cfc989f812ad764feae56ab730d64fe5f6f5e57306eef8cf46c3940f5ba22394916bd9bd7372d2468a4f06141d9348fa82eae5a7a49f8dc47f165f0e6bf1b5ab
7
+ data.tar.gz: 7007c0b60355e42523f5f6b4158c7ae5c82c7d80e012f557dda2093841db42be017f8c54f7e142ee761764ef5f2476d0073af1d35b58b636be393e9b8ea24afc
data/.travis.yml CHANGED
@@ -1,5 +1,18 @@
1
+ dist: trusty
1
2
  sudo: false
2
3
  language: ruby
3
4
  rvm:
4
5
  - 2.3.3
5
- before_install: gem install bundler -v 1.14.6
6
+ - 2.4.1
7
+ addons:
8
+ postgresql: '9.6'
9
+ before_install:
10
+ - gem update --system
11
+ - gem --version
12
+ - gem install bundler -v 1.14.6
13
+ before_script:
14
+ - bundle exec rake db:create
15
+ - bundle exec rake db:migrate
16
+ cache: bundler
17
+ gemfile:
18
+ - gemfiles/csv_import_1.1_and_0.18.gemfile
data/API.md ADDED
@@ -0,0 +1,355 @@
1
+ # RowBoat API
2
+
3
+ This is really more of a summary of what you can do with `RowBoat::Base` since you subclass it to do everything :)
4
+
5
+ ## Contents
6
+
7
+ - [Basic Usage](#basic-usage)
8
+ - [`.import`](#import)
9
+ - [`initialize`](#initialize)
10
+ - [`import`](#import-1)
11
+ - [`import_into`](#import_into)
12
+ - [`csv_source`](#csv_source)
13
+ - [`column_mapping`](#column_mapping)
14
+ - [`preprocess_row`](#preprocess_row)
15
+ - [`preprocess_rows`](#preprocess_rows)
16
+ - [`options`](#options)
17
+ - [`handle_failed_row`](#handle_failed_row)
18
+ - [`handle_failed_rows`](#handle_failed_rows)
19
+ - [`value_converters`](#value_converters)
20
+
21
+ ## Basic Usage
22
+
23
+ Just subclass `RowBoat::Base` and define the [`import_into`](#import_into) and [`column_mapping`](#column_mapping) methods to get started (They're the only methods that you're required to implement).
24
+
25
+ ```ruby
26
+ class ImportProduct < RowBoat::Base
27
+ def import_into
28
+ Product
29
+ end
30
+
31
+ def column_mapping
32
+ {
33
+ downcased_csv_column_header: :model_attribute_name,
34
+ another_downcased_csv_column_header: :another_model_attribute_name
35
+ }
36
+ end
37
+ end
38
+ ```
39
+
40
+ ## `.import`
41
+
42
+ ### Description
43
+ Imports database records form the given CSV-like object. The CSV-like object can be anything that can be passed to [`SmarterCSV.process`](https://github.com/tilo/smarter_csv#documentation) (string paths to files, files, tempfiles, instances of StringIO, etc).
44
+
45
+ It returns a hash containing
46
+
47
+ - `:invalid_records` - an array of all records that failed to import since they were invalid. If you've configured the `:validate` option to be `false` it will be an empty array.
48
+ - `:total_inserted` - the total number of records inserted into the database.
49
+ - `:inserted_ids` - an array of all of the ids of records inserted into the database.
50
+
51
+ If you want to pass additional information to help import CSVs, *don't override this method*. It just passes through to [`initialize`](#initialize) so override that :)
52
+
53
+ ### Example
54
+
55
+ #### Basic Use
56
+
57
+ ```ruby
58
+ class ImportProduct < RowBoat::Base
59
+ # required configuration omitted for brevity
60
+ end
61
+
62
+ ImportProduct.import("path/to/my.csv")
63
+ ```
64
+
65
+ #### Advanced Use
66
+
67
+ ```ruby
68
+ class ImportProduct < RowBoat::Base
69
+ # required configuration omitted for brevity
70
+ def intitialize(csv_source, my_options)
71
+ super(csv_source)
72
+ @my_options = my_options
73
+ end
74
+ end
75
+
76
+ ImportProduct.import("path/to/my.csv", foo: "bar")
77
+ ```
78
+
79
+ ## `initialize`
80
+
81
+ ### Description
82
+
83
+ Makes a new instance with the given CSV-like object. See [`.import`](#import) for more details around when and how to override this method.
84
+
85
+ ## `import`
86
+
87
+ ### Description
88
+
89
+ The instance method that actually parses and imports the CSV. Generally, you wouldn't call this directly and would instead call [`.import`](#import).
90
+
91
+ ## `import_into`
92
+
93
+ ### Description
94
+
95
+ It is required that you override this method to return whatever ActiveRecord class you want your CSV imported into.
96
+
97
+ ### Example
98
+
99
+ #### Basic Use
100
+
101
+ ```ruby
102
+ class ImportProduct < RowBoat::Base
103
+ # other required configuration omitted for brevity
104
+ def import_into
105
+ Product
106
+ end
107
+ end
108
+ ```
109
+
110
+ #### Advanced Use
111
+
112
+ ```ruby
113
+ class ImportProduct < RowBoat::Base
114
+ # other required configuration omitted for brevity
115
+ def import_into
116
+ if csv_source.is_a?(String) && csv_source.match(/category/i)
117
+ ProductCategory
118
+ else
119
+ Product
120
+ end
121
+ end
122
+ end
123
+
124
+ ImportProduct.import("path/to/category.csv")
125
+ ImportProduct.import("path/to/product.csv")
126
+ ```
127
+
128
+
129
+ ## `csv_source`
130
+
131
+ ### Description
132
+
133
+ Whatever you originally passed in as the CSV source.
134
+
135
+ ### Example
136
+
137
+ ```ruby
138
+ class ImportProduct < RowBoat::Base
139
+ # other required configuration omitted for brevity
140
+ def import_into
141
+ # `csv_source` is available in any of our instance methods
142
+ if csv_source.is_a?(String) && csv_source.match(/category/i)
143
+ ProductCategory
144
+ else
145
+ Product
146
+ end
147
+ end
148
+ end
149
+
150
+ ImportProduct.import("path/to/category.csv")
151
+ ImportProduct.import("path/to/product.csv")
152
+ ```
153
+
154
+ ## `column_mapping`
155
+
156
+ ### Description
157
+
158
+ It is required that you override this method with a hash that maps columns in your CSV to their preferred names.
159
+
160
+ By default
161
+ - CSV column names are downcased symbols of what they look like in the CSV.
162
+ - CSV columns that are not mapped are ignored when processing the CSV.
163
+
164
+ If you're familiar with [SmarterCSV](https://github.com/tilo/smarter_csv#documentation), this method essentially defines your `:key_mapping` and with the `:remove_unmapped_keys` setting set to `true`.
165
+
166
+ You can change these defaults by overriding the [`options`](#options) method.
167
+
168
+ ### Example
169
+
170
+ ```ruby
171
+ class ImportProduct < RowBoat::Base
172
+ # other required configuration omitted for brevity
173
+ def column_mapping
174
+ {
175
+ prdct_nm: :name,
176
+ "price/cost_amnt": :price_in_cents
177
+ }
178
+ end
179
+ end
180
+ ```
181
+
182
+ ## `preprocess_row`
183
+
184
+ ### Description
185
+
186
+ Implement this method if you need to do some work on the row before the record is inserted/updated.
187
+
188
+ If you return `nil` from this method, the row will be skipped in the import.
189
+
190
+ If the work you intend to do with the row only requires changing one attribute, it is recommended that you override [`value_converters`](#value_converters) instead of this.
191
+
192
+ ### Example
193
+
194
+ ```ruby
195
+ class ImportProduct < RowBoat::Base
196
+ # required configuration omitted for brevity
197
+ def preprocess_row(row)
198
+ { default: :value }.merge(row)
199
+ end
200
+ # or...
201
+ def preprocess_row(row)
202
+ if row[:name] && row[:price]
203
+ row
204
+ else
205
+ nil
206
+ end
207
+ end
208
+ end
209
+ ```
210
+
211
+ ## `preprocess_rows`
212
+
213
+ ### Description
214
+
215
+ Override this method if you need to do something with a chunk of rows (the chunk size is determined by the `:chunk_size` option in the [`options`](#options) method).
216
+
217
+ If you need to filter particular rows, it's better to just implement [`preprocess_row`](#preprocess_row) and return `nil` for the rows you want to ignore.
218
+
219
+ ### Example
220
+
221
+ ```ruby
222
+ class ImportProduct < RowBoat::Base
223
+ # required configuration omitted for brevity
224
+ def preprocess_rows(rows)
225
+ if skip_batch?(rows)
226
+ super([])
227
+ else
228
+ super
229
+ end
230
+ end
231
+
232
+ def skip_batch?(rows)
233
+ # decide whether or not to skip the batch
234
+ end
235
+ end
236
+ ```
237
+
238
+ ## `options`
239
+
240
+ ### Description
241
+
242
+ Implement this to configure RowBoat, [SmarterCSV](https://github.com/tilo/smarter_csv), and [activerecord-import](https://github.com/zdennis/activerecord-import).
243
+
244
+ Except for `:wrap_in_transaction`, all options pass through to SmarterCSV and activerecord-import.
245
+
246
+ `:wrap_in_transaction` simply tells RowBoat whether or not you want your whole import wrapped in a database transaction.
247
+
248
+ Whatever you define in this method will be merged into the defaults:
249
+
250
+ - `:chunk_size` - `500`
251
+ - `:key_mapping` - `column_mapping`
252
+ - `:recursive` - `true`
253
+ - `:remove_unmapped_keys` - `true`
254
+ - `:validate` - `true`
255
+ - `:value_converters` - `csv_value_converters`
256
+ - `:wrap_in_transaction` - `true`
257
+
258
+ Don't provide `value_converters` or `key_mapping` options here. Implement the [`value_converters`](#value_converters) and [`column_mapping`](#column_mapping) respectively.
259
+
260
+ ### Example
261
+
262
+ ```ruby
263
+ class ImportProduct < RowBoat::Base
264
+ # required configuration omitted for brevity
265
+ def options
266
+ {
267
+ chunk_size: 1000,
268
+ validate: false,
269
+ wrap_in_transaction: false
270
+ }
271
+ end
272
+ end
273
+ ```
274
+
275
+ ## `handle_failed_row`
276
+
277
+ ### Description
278
+
279
+ Implement this to do some work with a row that has failed to import.
280
+
281
+ It's important to note that
282
+ - This happens after the import has completed.
283
+ - The given row is an instance of whatever class was returned by [`import_into`](#import_into).
284
+
285
+ These records are also available in the return value of [`.import`](#import).
286
+
287
+ ### Example
288
+
289
+ ```ruby
290
+ class ImportProduct < RowBoat::Base
291
+ # required configuration omitted for brevity
292
+ def handle_failed_row(row)
293
+ puts row.errors.full_messages.join(", ")
294
+ end
295
+ end
296
+ ```
297
+
298
+ ## `handle_failed_rows`
299
+
300
+ ### Description
301
+
302
+ Override this method to do some work will all of the rows that failed to import.
303
+
304
+ ### Example
305
+
306
+ ```ruby
307
+ class ImportProduct < RowBoat::Base
308
+ # required configuration omitted for brevity
309
+ def handle_failed_rows(rows)
310
+ puts "Failed to import #{rows.size} rows :("
311
+ super
312
+ end
313
+ end
314
+ ```
315
+
316
+ ## `value_converters`
317
+
318
+ ### Description
319
+
320
+ Implement to specify how to translate values from the CSV into whatever sorts of objects you need.
321
+
322
+ Simply return a hash that has the mapped column name (ie, what you mapped it to in the [`column_mapping`](#column_mapping) method) as a key pointing to either
323
+ - a method name as a symbol
324
+ - a proc or lambda
325
+ - an object that implements `convert`
326
+
327
+ Regardless of which one you choose, it takes a value and returns a converted value.
328
+
329
+ This is essentially a sugared up version of `:value_converters` option in [SmarterCSV](https://github.com/tilo/smarter_csv#documentation).
330
+
331
+ ### Example
332
+
333
+ ```ruby
334
+ class ImportProduct < RowBoat::Base
335
+ # required configuration omitted for brevity
336
+ def value_converters
337
+ {
338
+ sell_by: :convert_date,
339
+ name: -> (value) { value.titlelize },
340
+ price: proc { |value| value.to_i },
341
+ description: DescriptionConverter
342
+ }
343
+ end
344
+
345
+ def convert_date(value)
346
+ Date.parse(value) rescue nil
347
+ end
348
+ end
349
+
350
+ module DescriptionConverter
351
+ def self.convert(value)
352
+ value.present? ? value : "default description :("
353
+ end
354
+ end
355
+ ```
data/Appraisals ADDED
@@ -0,0 +1,6 @@
1
+ # frozen_string_literal: true
2
+
3
+ appraise "csv_import_1.1_and_0.18" do
4
+ gem "activerecord-import", "~> 0.18.2"
5
+ gem "smarter_csv", "~> 1.1.0"
6
+ end
data/README.md CHANGED
@@ -1,13 +1,29 @@
1
1
  # RowBoat
2
2
 
3
+ Created by &nbsp;&nbsp;&nbsp; [<img src="https://raw.githubusercontent.com/devmynd/row_boat/master/devmynd-logo.png" alt="DevMynd Logo" />](https://www.devmynd.com/)
4
+
5
+ [![Gem Version](https://badge.fury.io/rb/row_boat.svg)](http://badge.fury.io/rb/row_boat) &nbsp;&nbsp;&nbsp;[![Build Status](https://travis-ci.org/devmynd/row_boat.svg?branch=master)](https://travis-ci.org/devmynd/row_boat)
6
+
3
7
  A simple gem to help you import CSVs into your ActiveRecord models.
4
8
 
9
+ [Check out the documentation!](/API.md#rowboat-api)
10
+
11
+ It uses [SmarterCSV](https://github.com/tilo/smarter_csv) and [`activerecord-import`](https://github.com/zdennis/activerecord-import) to import database records from your CSVs.
12
+
13
+ ## Contents
14
+
15
+ - [Installation](#installation)
16
+ - [Basic Usage](#basic-usage)
17
+ - [Development](#development)
18
+ - [Contributing](#contributing)
19
+ - [License](#license)
20
+
5
21
  ## Installation
6
22
 
7
23
  Add this line to your application's Gemfile:
8
24
 
9
25
  ```ruby
10
- gem "row_boat"
26
+ gem "row_boat", "~> 0.1"
11
27
  ```
12
28
 
13
29
  And then execute:
@@ -18,45 +34,72 @@ Or install it yourself as:
18
34
 
19
35
  $ gem install row_boat
20
36
 
21
- ## Usage
37
+ ## Basic Usage
38
+
39
+ #### [Full documentation can be found here.](/API.md#rowboat-api)
40
+
41
+ Below we're defining the required methods ([`import_into`](/API.md#import_into) and [`column_mapping`](/API.md#column_mapping)) and a few additional options as well (via [`value_converters`](/API.md#value_converters) and [`options`](/API.md#options)). Checkout [API.md](/API.md#rowboat-api) for the full documentation for more :)
22
42
 
23
43
  ```ruby
24
- class ProductBoat < RowBoat::Base
44
+ class ImportProduct
25
45
  # required
26
46
  def import_into
27
- Product
47
+ Product # The ActiveRecord class we want to import records into.
28
48
  end
29
49
 
30
50
  # required
31
51
  def column_mapping
32
52
  {
33
- downcased_csv_column_header: :model_attribute_name,
34
- another_downcased_csv_column_header: :another_model_attribute_name
53
+ # `:prdct_name` is the downcased and symbolized version
54
+ # of our column header, while `:name` is the attribute
55
+ # of our model we want to receive `:prdct_name`'s value
56
+ prdct_name: :name,
57
+ dllr_amnt: :price_in_cents,
58
+ desc: :description
35
59
  }
36
60
  end
37
61
 
38
62
  # optional
39
- def options
40
- super.merge(validate: false)
63
+ def value_converters
64
+ {
65
+ # Allows us to change values we want to import
66
+ # before we import them
67
+ price_in_cents: -> (value) { value * 1000 }
68
+ }
41
69
  end
42
- end
43
70
 
44
- # Later...
71
+ # optional
72
+ def preprocess_row(row)
73
+ if row[:name] && row[:description] && row[:price]
74
+ row
75
+ else
76
+ nil # return nil to skip a row
77
+ end
78
+ # we could also remove some attributes or do any
79
+ # other kind of work we want with the given row.
80
+ end
45
81
 
46
- ProductBoat.import(path_to_csv)
82
+ #optional
83
+ def options
84
+ {
85
+ # These are additional configurations that
86
+ # are generally passed through to SmarterCSV
87
+ # and activerecord-import
88
+ validate: false, # this defaults to `true`
89
+ wrap_in_transaction: false # this defaults to `true`
90
+ }
91
+ end
92
+ end
47
93
  ```
48
94
 
49
95
  ## Development
50
96
 
51
- After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
52
-
53
- To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
97
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake` to run the tests (run `appraisal install` and `appraisal rake` to run the tests against different combinations of dependencies). You can also run `bin/console` for an interactive prompt that will allow you to experiment.
54
98
 
55
99
  ## Contributing
56
100
 
57
101
  Bug reports and pull requests are welcome on GitHub at https://github.com/devmynd/row_boat. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
58
102
 
59
-
60
103
  ## License
61
104
 
62
105
  The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
data/bin/setup CHANGED
@@ -6,3 +6,5 @@ set -vx
6
6
  bundle install
7
7
 
8
8
  # Do any other automated setup that you need to do here
9
+ bundle exec rake db:create
10
+ bundle exec rake db:migrate
data/devmynd-logo.png ADDED
Binary file
@@ -0,0 +1,8 @@
1
+ # This file was generated by Appraisal
2
+
3
+ source "https://rubygems.org"
4
+
5
+ gem "activerecord-import", "~> 0.18.2"
6
+ gem "smarter_csv", "~> 1.1.0"
7
+
8
+ gemspec path: "../"
@@ -0,0 +1,144 @@
1
+ PATH
2
+ remote: ..
3
+ specs:
4
+ row_boat (0.1.0)
5
+ activerecord (>= 5.0.0)
6
+ activerecord-import (~> 0.18.2)
7
+ smarter_csv (~> 1.1)
8
+
9
+ GEM
10
+ remote: https://rubygems.org/
11
+ specs:
12
+ actionpack (5.1.0)
13
+ actionview (= 5.1.0)
14
+ activesupport (= 5.1.0)
15
+ rack (~> 2.0)
16
+ rack-test (~> 0.6.3)
17
+ rails-dom-testing (~> 2.0)
18
+ rails-html-sanitizer (~> 1.0, >= 1.0.2)
19
+ actionview (5.1.0)
20
+ activesupport (= 5.1.0)
21
+ builder (~> 3.1)
22
+ erubi (~> 1.4)
23
+ rails-dom-testing (~> 2.0)
24
+ rails-html-sanitizer (~> 1.0, >= 1.0.3)
25
+ activemodel (5.1.0)
26
+ activesupport (= 5.1.0)
27
+ activerecord (5.1.0)
28
+ activemodel (= 5.1.0)
29
+ activesupport (= 5.1.0)
30
+ arel (~> 8.0)
31
+ activerecord-import (0.18.3)
32
+ activerecord (>= 3.2)
33
+ activesupport (5.1.0)
34
+ concurrent-ruby (~> 1.0, >= 1.0.2)
35
+ i18n (~> 0.7)
36
+ minitest (~> 5.1)
37
+ tzinfo (~> 1.1)
38
+ appraisal (2.2.0)
39
+ bundler
40
+ rake
41
+ thor (>= 0.14.0)
42
+ arel (8.0.0)
43
+ ast (2.3.0)
44
+ awesome_print (1.7.0)
45
+ builder (3.2.3)
46
+ coderay (1.1.1)
47
+ concurrent-ruby (1.0.5)
48
+ database_cleaner (1.6.1)
49
+ diff-lcs (1.3)
50
+ erubi (1.6.0)
51
+ i18n (0.8.1)
52
+ loofah (2.0.3)
53
+ nokogiri (>= 1.5.9)
54
+ method_source (0.8.2)
55
+ mini_portile2 (2.1.0)
56
+ minitest (5.10.2)
57
+ nokogiri (1.7.2)
58
+ mini_portile2 (~> 2.1.0)
59
+ parser (2.4.0.0)
60
+ ast (~> 2.2)
61
+ pg (0.20.0)
62
+ powerpack (0.1.1)
63
+ pry (0.10.4)
64
+ coderay (~> 1.1.0)
65
+ method_source (~> 0.8.1)
66
+ slop (~> 3.4)
67
+ pry-doc (0.10.0)
68
+ pry (~> 0.9)
69
+ yard (~> 0.9)
70
+ pry-nav (0.2.4)
71
+ pry (>= 0.9.10, < 0.11.0)
72
+ rack (2.0.2)
73
+ rack-test (0.6.3)
74
+ rack (>= 1.0)
75
+ rails-dom-testing (2.0.3)
76
+ activesupport (>= 4.2.0)
77
+ nokogiri (>= 1.6)
78
+ rails-html-sanitizer (1.0.3)
79
+ loofah (~> 2.0)
80
+ railties (5.1.0)
81
+ actionpack (= 5.1.0)
82
+ activesupport (= 5.1.0)
83
+ method_source
84
+ rake (>= 0.8.7)
85
+ thor (>= 0.18.1, < 2.0)
86
+ rainbow (2.2.2)
87
+ rake
88
+ rake (10.5.0)
89
+ rspec (3.6.0)
90
+ rspec-core (~> 3.6.0)
91
+ rspec-expectations (~> 3.6.0)
92
+ rspec-mocks (~> 3.6.0)
93
+ rspec-core (3.6.0)
94
+ rspec-support (~> 3.6.0)
95
+ rspec-expectations (3.6.0)
96
+ diff-lcs (>= 1.2.0, < 2.0)
97
+ rspec-support (~> 3.6.0)
98
+ rspec-mocks (3.6.0)
99
+ diff-lcs (>= 1.2.0, < 2.0)
100
+ rspec-support (~> 3.6.0)
101
+ rspec-support (3.6.0)
102
+ rubocop (0.48.1)
103
+ parser (>= 2.3.3.1, < 3.0)
104
+ powerpack (~> 0.1)
105
+ rainbow (>= 1.99.1, < 3.0)
106
+ ruby-progressbar (~> 1.7)
107
+ unicode-display_width (~> 1.0, >= 1.0.1)
108
+ ruby-progressbar (1.8.1)
109
+ slop (3.6.0)
110
+ smarter_csv (1.1.4)
111
+ standalone_migrations (5.2.1)
112
+ activerecord (>= 4.2.7, < 5.2.0)
113
+ railties (>= 4.2.7, < 5.2.0)
114
+ rake (~> 10.0)
115
+ thor (0.19.4)
116
+ thread_safe (0.3.6)
117
+ tzinfo (1.2.3)
118
+ thread_safe (~> 0.1)
119
+ unicode-display_width (1.2.1)
120
+ yard (0.9.9)
121
+
122
+ PLATFORMS
123
+ ruby
124
+
125
+ DEPENDENCIES
126
+ activerecord-import (~> 0.18.2)
127
+ appraisal (~> 2.2.0)
128
+ awesome_print
129
+ bundler (~> 1.14)
130
+ database_cleaner (~> 1.6.0)
131
+ pg
132
+ pry
133
+ pry-doc
134
+ pry-nav
135
+ rake (~> 10.0)
136
+ row_boat!
137
+ rspec (~> 3.0)
138
+ rubocop (~> 0.48.1)
139
+ smarter_csv (~> 1.1.0)
140
+ standalone_migrations (~> 5.2.0)
141
+ yard (~> 0.9.9)
142
+
143
+ BUNDLED WITH
144
+ 1.14.6
data/lib/row_boat/base.rb CHANGED
@@ -9,15 +9,36 @@ module RowBoat
9
9
  attr_reader :csv_source
10
10
 
11
11
  class << self
12
+ # Imports database records from the given CSV-like object.
13
+ #
14
+ # @overload import(csv_source)
15
+ # @param csv_source [String, #read] a CSV-like object that SmarterCSV can read.
16
+ #
17
+ # @return [Hash] a hash with +:invalid_records+, +:total_inserted+ and +:inserted_ids+.
18
+ #
19
+ # @see https://github.com/tilo/smarter_csv#documentation SmarterCSV Docs
12
20
  def import(*args, &block)
13
21
  new(*args, &block).import
14
22
  end
15
23
  end
16
24
 
25
+ # Makes a new instance with the given +csv_source+.
26
+ #
27
+ # @abstract Override this method if you need additional arguments to process your CSV (like defaults).
28
+ #
29
+ # @example
30
+ # def initialize(csv_source, default_name)
31
+ # super(csv_source)
32
+ # @default_name = default_name
33
+ # end
17
34
  def initialize(csv_source)
18
35
  @csv_source = csv_source
19
36
  end
20
37
 
38
+ # Parses the csv and inserts/updates the database. You probably won't call this method directly,
39
+ # instead you would call {RowBoat::Base.import}.
40
+ #
41
+ # @return [Hash] a hash with +:invalid_records+, +:total_inserted+ and +:inserted_ids+.
21
42
  def import
22
43
  import_results = []
23
44
 
@@ -32,24 +53,71 @@ module RowBoat
32
53
  end
33
54
  end
34
55
 
56
+ # Override with the ActiveRecord class that the CSV should be imported into.
57
+ #
58
+ # @abstract
59
+ #
60
+ # @note You must implement this method.
61
+ #
62
+ # @example
63
+ # def import_into
64
+ # Product
65
+ # end
35
66
  def import_into
36
67
  raise NotImplementedError, not_implemented_error_message(__method__)
37
68
  end
38
69
 
70
+ # Override with a hash that maps CSV column names to their preferred names.
71
+ # Oftentimes these are the names of the attributes on the model class from {#import_into}.
72
+ #
73
+ # @abstract
74
+ #
75
+ # @note You must implement this method.
76
+ #
77
+ # @example
78
+ # def column_mapping
79
+ # {
80
+ # prdct_name: :name,
81
+ # price: :price,
82
+ # sl_exp: :sale_expires_at
83
+ # }
84
+ # end
85
+ #
86
+ # @see #import_into
39
87
  def column_mapping
40
88
  raise NotImplementedError, not_implemented_error_message(__method__)
41
89
  end
42
90
 
91
+ # Override this method if you need to do some work on the row before the record is
92
+ # inserted/updated or want to skip the row in the import. Simply return +nil+ to skip the row.
93
+ #
94
+ # @abstract
95
+ #
96
+ # @note If you only need to manipulate one attribute (ie parse a date from a string, etc), then
97
+ # you should probably use {#value_converters}
98
+ #
99
+ # @return [Hash,NilClass] a hash of attributes, +nil+, or even and instance of the class returned
100
+ # in {#import_into}.
101
+ #
102
+ # @see #import_into
43
103
  def preprocess_row(row)
44
104
  row
45
105
  end
46
106
 
107
+ # @api private
47
108
  def import_rows(rows)
48
109
  import_options = ::RowBoat::Helpers.extract_import_options(merged_options)
49
110
  preprocessed_rows = preprocess_rows(rows)
50
111
  import_into.import(preprocessed_rows, import_options)
51
112
  end
52
113
 
114
+ # Override this method if you need to do something with a chunk of rows.
115
+ #
116
+ # @abstract
117
+ #
118
+ # @note If you want to filter out a row, you can just return +nil+ from {#preprocess_row}.
119
+ #
120
+ # @see #preprocess_row
53
121
  def preprocess_rows(rows)
54
122
  rows.each_with_object([]) do |row, preprocessed_rows|
55
123
  preprocessed_row = preprocess_row(row)
@@ -57,10 +125,31 @@ module RowBoat
57
125
  end
58
126
  end
59
127
 
128
+ # Override this method to specify CSV parsing and importing options.
129
+ # All SmarterCSV and activerecord-import options can be listed here along with
130
+ # +:wrap_in_transaction+. The defaults provided by RowBoat can be found in {#default_options}
131
+ #
132
+ # @abstract
133
+ #
134
+ # @note If you want to use the +:value_converters+ option provided by SmarterCSV
135
+ # just override {#value_converters}.
136
+ #
137
+ # @return [Hash] a hash of configuration options.
138
+ #
139
+ # @see https://github.com/tilo/smarter_csv#documentation SmarterCSV docs
140
+ # @see https://github.com/zdennis/activerecord-import/wiki activerecord-import docs
141
+ # @see #value_converters
60
142
  def options
61
143
  {}
62
144
  end
63
145
 
146
+ # Default options provided by RowBoat for CSV parsing and importing.
147
+ #
148
+ # @note Do not override.
149
+ #
150
+ # @return [Hash] a hash of configuration options.
151
+ #
152
+ # @api private
64
153
  def default_options
65
154
  {
66
155
  chunk_size: 500,
@@ -73,22 +162,56 @@ module RowBoat
73
162
  }
74
163
  end
75
164
 
165
+ # @api private
76
166
  def merged_options
77
167
  default_options.merge(options)
78
168
  end
79
169
 
170
+ # Override this method to do some work with a row that has failed to import.
171
+ #
172
+ # @abstract
173
+ #
174
+ # @note +row+ here is actually an instance of the class returned in {#import_into}
175
+ #
176
+ # @see #import_into
80
177
  def handle_failed_row(row)
81
178
  row
82
179
  end
83
180
 
181
+ # Override this method to do some work will all of the rows that failed to import.
182
+ #
183
+ # @abstract
184
+ #
185
+ # @note If you override this method and {#handle_failed_row}, be sure to call +super+.
84
186
  def handle_failed_rows(rows)
85
187
  rows.each { |row| handle_failed_row(row) }
86
188
  end
87
189
 
190
+ # Override this method to specify how to translate values from the CSV
191
+ # into ruby objects.
192
+ #
193
+ # You can provide an object that implements +convert+, a proc or lambda, or the
194
+ # the name of a method as a Symbol
195
+ #
196
+ # @abstract
197
+ #
198
+ # @example
199
+ # def value_converters
200
+ # {
201
+ # name: -> (value) { value.titleize }
202
+ # price: :convert_price,
203
+ # expires_at: CustomDateConverter
204
+ # }
205
+ # end
206
+ #
207
+ # def convert_price(value)
208
+ # value || 0
209
+ # end
88
210
  def value_converters
89
211
  {}
90
212
  end
91
213
 
214
+ # @api private
92
215
  def csv_value_converters
93
216
  value_converters.each_with_object({}) do |(key, potential_converter), converters_hash|
94
217
  case potential_converter
@@ -106,19 +229,27 @@ module RowBoat
106
229
 
107
230
  private
108
231
 
232
+ # @api private
233
+ # @private
109
234
  def not_implemented_error_message(method_name)
110
235
  "Subclasses of #{self.class.name} must implement `#{method_name}`"
111
236
  end
112
237
 
238
+ # @api private
239
+ # @private
113
240
  def parse_rows(&block)
114
241
  csv_options = ::RowBoat::Helpers.extract_csv_options(merged_options)
115
242
  ::SmarterCSV.process(csv_source, csv_options, &block)
116
243
  end
117
244
 
245
+ # @api private
246
+ # @private
118
247
  def transaction_if_needed(&block)
119
248
  merged_options[:wrap_in_transaction] ? import_into.transaction(&block) : yield
120
249
  end
121
250
 
251
+ # @api private
252
+ # @private
122
253
  def process_import_results(import_results)
123
254
  import_results.each_with_object(
124
255
  invalid_records: [],
@@ -1,6 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module RowBoat
4
+ # @api private
4
5
  module Helpers
5
6
  CSV_OPTION_KEYS = %i[
6
7
  chunk_size
@@ -1,6 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module RowBoat
4
+ # @api private
4
5
  class ValueConverter
5
6
  attr_reader :converter
6
7
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module RowBoat
4
- VERSION = "0.1.0.alpha.3"
4
+ VERSION = "0.1.0"
5
5
  end
data/row_boat.gemspec CHANGED
@@ -27,6 +27,7 @@ Gem::Specification.new do |spec|
27
27
  spec.add_dependency "activerecord-import", "~> 0.18.2"
28
28
  spec.add_dependency "smarter_csv", "~> 1.1"
29
29
 
30
+ spec.add_development_dependency "appraisal", "~> 2.2.0"
30
31
  spec.add_development_dependency "awesome_print"
31
32
  spec.add_development_dependency "bundler", "~> 1.14"
32
33
  spec.add_development_dependency "database_cleaner", "~> 1.6.0"
@@ -38,4 +39,5 @@ Gem::Specification.new do |spec|
38
39
  spec.add_development_dependency "rspec", "~> 3.0"
39
40
  spec.add_development_dependency "rubocop", "~> 0.48.1"
40
41
  spec.add_development_dependency "standalone_migrations", "~> 5.2.0"
42
+ spec.add_development_dependency "yard", "~> 0.9.9"
41
43
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: row_boat
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0.alpha.3
4
+ version: 0.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Michael Crismali
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2017-05-12 00:00:00.000000000 Z
11
+ date: 2017-05-16 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activerecord
@@ -52,6 +52,20 @@ dependencies:
52
52
  - - "~>"
53
53
  - !ruby/object:Gem::Version
54
54
  version: '1.1'
55
+ - !ruby/object:Gem::Dependency
56
+ name: appraisal
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: 2.2.0
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: 2.2.0
55
69
  - !ruby/object:Gem::Dependency
56
70
  name: awesome_print
57
71
  requirement: !ruby/object:Gem::Requirement
@@ -206,6 +220,20 @@ dependencies:
206
220
  - - "~>"
207
221
  - !ruby/object:Gem::Version
208
222
  version: 5.2.0
223
+ - !ruby/object:Gem::Dependency
224
+ name: yard
225
+ requirement: !ruby/object:Gem::Requirement
226
+ requirements:
227
+ - - "~>"
228
+ - !ruby/object:Gem::Version
229
+ version: 0.9.9
230
+ type: :development
231
+ prerelease: false
232
+ version_requirements: !ruby/object:Gem::Requirement
233
+ requirements:
234
+ - - "~>"
235
+ - !ruby/object:Gem::Version
236
+ version: 0.9.9
209
237
  description: Turn the rows of your CSV into rows in your database
210
238
  email:
211
239
  - michael@crismali.com
@@ -218,6 +246,8 @@ files:
218
246
  - ".rubocop.yml"
219
247
  - ".ruby-version"
220
248
  - ".travis.yml"
249
+ - API.md
250
+ - Appraisals
221
251
  - CODE_OF_CONDUCT.md
222
252
  - Gemfile
223
253
  - LICENSE.txt
@@ -228,6 +258,9 @@ files:
228
258
  - db/config.yml
229
259
  - db/migrate/20170511160154_create_products.rb
230
260
  - db/schema.rb
261
+ - devmynd-logo.png
262
+ - gemfiles/csv_import_1.1_and_0.18.gemfile
263
+ - gemfiles/csv_import_1.1_and_0.18.gemfile.lock
231
264
  - lib/row_boat.rb
232
265
  - lib/row_boat/base.rb
233
266
  - lib/row_boat/helpers.rb
@@ -249,9 +282,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
249
282
  version: '0'
250
283
  required_rubygems_version: !ruby/object:Gem::Requirement
251
284
  requirements:
252
- - - ">"
285
+ - - ">="
253
286
  - !ruby/object:Gem::Version
254
- version: 1.3.1
287
+ version: '0'
255
288
  requirements: []
256
289
  rubyforge_project:
257
290
  rubygems_version: 2.6.11