simple_csv 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 7f379384bc15e489a0bbb1a8d7a28026e58162a3
-  data.tar.gz: c239e1ebccf33ed4fe0f9051f7c1a5a65a44f6da
+  metadata.gz: accda7d430cbe5c71e9a775bac95f9f8f7c01038
+  data.tar.gz: c06a504dbdeaed0f91ee652655ce871a8ef5b18b
 SHA512:
-  metadata.gz: 1404a5abbf6bbc700ddcc53ad3edc8e3428c1316c547d516d146202a50b98fde7951a853bb4c21449b090512423ce36644e34e44c545b1234466bf089c7bcad3
-  data.tar.gz: 432d6aac54b96f6e2775f869484b4ac61ffc1b39927a30c0e334f2f7e3b87068e9bea1ec9267e9219ade9771666552b2f87a1c03b8cd55f6cacddaddc77a311f
+  metadata.gz: a979d33a0aab1573a75f7e8f8c98f4fea21b7bd4752b3ef80945f7ccaccb06f289efc947e537929e22d05c3db081ae9b267fb4ba89a562b8067133d07875a2a4
+  data.tar.gz: d217b4ccea541d1136d5b2a6381720eb9defdf4d7232445b5d24c5870c3530a2cf3b84fab8266955a1c18cdba59786a6ab3d513a45f4b9fb9e7c61400061772f

data/.gitignore

@@ -7,5 +7,5 @@
 /pkg/
 /spec/reports/
 /tmp/
-output.csv
-input.csv
+/**/output.csv
+/**/input.csv

data/.rubocop.yml

@@ -0,0 +1,2 @@
+AllCops:
+  TargetRubyVersion: 2.3

data/.travis.yml

@@ -1,6 +1,10 @@
 sudo: false
 language: ruby
+cache: bundler
 rvm:
+  - 2.1.0
+  - 2.2.0
   - 2.2.2
   - 2.3.0
+  - 2.3.3
 before_install: gem install bundler -v 1.13.6

data/README.md

@@ -29,17 +29,84 @@ Or install it yourself as:
 
 ## Usage
 
-### Generating a CSV file
+### General
+
+By default, the settings used will be those of `CSV::DEFAULT_OPTIONS` generally.
+`SimpleCsv` sets the `:headers` property to true by default, this is due to the nature of `SimpleCsv`.
+
+Headers have to be defined either before reading or generating a CSV.
+Since `:headers` is now `true` by default, `SimpleCsv` will allow `CSV` to parse the first line as headers.
+These headers are then converted in method calls to use within an `SimpleCsv::Reader#each_row` loop.
+
+If however, your file lacks headers, you have the ability to set `:has_headers` to false and supply headers manually before calling `SimpleCsv::Reader#each_row`.
+The headers will be picked up and used instead of the first line.
+
+#### SimpleCsv default settings
+
+These are the settings that will be merged with settings passed throogh either `SimpleCsv#generate` or `SimpleCsv#read`
+
+|        setting       |                   value               |
+|----------------------|---------------------------------------|
+|`:col_sep`            | `","`                                |
+|`:row_sep`            | `:auto`                               |
+|`:quote_char`         | `"\"`                               |
+|`:field_size_limit`    | `nil`                                 |
+|`:converters`         | `[:all, :blank_to_nil, :null_to_nil]` |
+|`:unconverted_fields`  | `nil`                                 |
+|`:headers`            | `true`                                |
+|`:return_headers`     | `false`                               |
+|`:header_converters`  | `nil`                                 |
+|`:skip_blanks`        | `false`                               |
+|`:force_quotes`       | `true`                                |
+|`:skip_lines`         | `nil`                                 |
+
+The following settings differ from the `CSV::DEFAULT_OPTIONS`
+
+* `:converters` is set to `[:all, :blank_to_nil, :null_to_nil]`
+* `:headers` is `true` by default
+* `:force_quotes` is `true` by default
+
+This essentially means that when reading a CSV file, headers are required otherwise a `SimpleCsv::HeadersNotSet` exception will be thrown.
+Also, when reading a CSV, all values will be parsed to their respective types, so `"1"` would become `1` as end value.
+
+#### SimpleCsv::Writer additional default settings
+
+Additionally, `SimpleCsv::Writer` has one additional default setting that ensures an entire row is written before being able to write another one.
+This setting enforces you to call each method once before calling one of them again, if this condition is not met a `SimpleCsv::RowNotComplete` exception will be thrown
 
-The `SimpleCsv#generate` method takes a path and options to use for generating the file.
-By default, a CSV is generated using comma's (`,`) as seperator and all fields are quoted.
+|         setting        |                   value               |
+|------------------------|---------------------------------------|
+|`:force_row_completion` | `true`                                |
 
-When supplying options they will be merged into the existing defaults rather than overwriting them completely.
+#### Setting aliasses
+
+An _alias_ can be used instead of it's respective _setting_.
+
+|    setting    |     alias     |
+|---------------|---------------|
+| `:col_sep`    | `:seperator`  |
+| `headers`     | `has_headers` |
+
+#### Converters
+
+The above `:converters` option is set to include `:all` converters and additionally, `:blank_to_nil` and `:null_to_nil`
+The code for these two can be summed up in two lines:
 
 ```ruby
-SimpleCsv.generate(path, options = { seperator: ',', force_quotes: true })
+CSV::Converters[:blank_to_nil] = ->(f) { f && f.empty? ? nil : f }
+CSV::Converters[:null_to_nil] = ->(f) { f && f == 'NULL' ? nil : f }
 ```
 
+What they do replace empty values or the string `'NULL'` by nil within a column when it's being parsed.
+For now, these are the default two and they are always used unless the `:converters` option is set to `nil` within `SimpleCsv#generate` or `SimpleCsv#read`
+
+### Generating a CSV file
+
+```ruby
+SimpleCsv.generate path, options = { ... }, &block
+```
+
+The `SimpleCsv#generate` method takes a (required) path, an (optional) hash of options and a (required) block to start building a CSV file.
 To generate a CSV file we use `SimpleCsv#generate` (using the [faker](https://github.com/stympy/faker) gem to provide fake data)
 
 ```ruby
@@ -63,23 +130,34 @@ end
 
 ### Reading a CSV file
 
-The `SimpleCsv#read` method takes a path and options to use for reading the file.
-By default, the expected seperator is a comma (`,`), `:headers` is set to `true` so that `CSV` parses the file expecting headers.
-The `:converters` option ensures values are converted to a proper type if possible, by default (if this is not set) all values are returned as strings
-
-When supplying options they will be merged into the existing defaults rather than overwriting them completely.
-
 ```ruby
-SimpleCsv.read(path, options = { headers: true, seperator: ',',
-                                 converters: [:all, :blank_to_nil, :null_to_nil] })
+SimpleCsv.read path, options = { ... }, &block
 ```
 
+The `SimpleCsv#generate` method takes a (required) path, an (optional) hash of options and a (required) block to start building a CSV file.
+
 To read a CSV file we use `SimpleCsv#read`, we will pass it a file path and a block as arguments.
 Within the block we define the headers present in the file, these will be transformed into methods you can call within `SimpleCsv::Reader#each_row` to get that property's current value
 
 ```ruby
 SimpleCsv.read('input.csv') do
-  # first define the headers in the file manually
+  # assumes headers are set, they will be read and callable within each_row
+
+  each_row do
+    puts [first_name, last_name, birth_date, employed_at].compact.join ', '
+  end
+end
+```
+
+### Reading a CSV file without headers
+
+Last but not least, if we have a CSV file that does not contain headers we can use the following setup.
+Setting `:has_headers` to `false` means we do not expect the first line to be headers.
+Therefore we have to explicitly define the headers before looping the CSV.
+
+```ruby
+SimpleCsv.read('headerless.csv', has_headers: false) do
+  # first define the headers in the file manually if the file does not have them
   headers :first_name, :last_name, :birth_date, :employed_at
 
   each_row do
@@ -89,18 +167,18 @@ SimpleCsv.read('input.csv') do
 end
 ```
 
-If we have a large CSV we might want to batch operations (say, if we are inserting this data into a database).
+### Batch operations
+
+If we have a large CSV we might want to batch operations (say, if we are inserting this data into a database or through an API).
 For this we can use `SimpleCsv::Reader#in_groups_of` and pass the size of the group.
 Within that we call `SimpleCsv::Reader#each_row` as usual
 
 ```ruby
 SimpleCsv.read('input.csv') do
-  # first define the headers in the file manually
-  headers :first_name, :last_name, :birth_date, :employed_at
+  # assumes headers are set, they will be read and callable within each_row
 
   in_groups_of(100) do
     each_row do
-      # print each field defined in headers (that is not nil)
       puts [first_name, last_name, birth_date, employed_at].compact.join ', '
     end
     # execute after every 100 rows
@@ -109,15 +187,36 @@ SimpleCsv.read('input.csv') do
 end
 ```
 
+### Aliassing existing headers
+
+Should you want to map existing headers to different names, this is possible by passing a hash at the end with key value pairs.
+When generating a CSV file, aliasses are ignored and therefore should not be passed.
+
+When defining columns manually using `headers` for a file without headers, ALL columns must be named before defining aliasses.
+This means that if your CSV exists of 3 columns, 3 headers must be defined before aliassing any of those to something shorter or more concise.
+
+To create an alias `date_of_birth` of `birth_date` *(In a CSV file without headers)* we would write *(notice `:birth_date` is present twice, once as column entry, and once more as key for an alias)*:
+
+```ruby
+headers :first_name, :last_name, :employed_at, :birth_date, birth_date: :date_of_birth
+```
+
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bundle` to install dependencies. Then, run `rspec` to run the tests. You can also use the `bin/console` file to play around and debug.
+
+To install this gem onto your local machine, run `rake install`.
+
+To release a new version:
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+* Run CI in dev branch, if tests pass, merge into master
+* Update version number in _lib/simple_csv/version.rb_ according to [symver](http://semver.org/)
+* Update _README.md_ to reflect your changes
+* run `rake release` to push commits, create a tag for the current commit and push the `.gem` file to RubyGems
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/simple_csv. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+Bug reports and pull requests are welcome on GitHub at https://github.com/sidofc/simple_csv. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
 
 ## License

data/bin/console

@@ -5,27 +5,9 @@ require 'pry'
 require 'faker'
 require 'simple_csv'
 
-# CSV generation test
-SimpleCsv.generate('spec/files/output.csv') do
-  headers :first_name, :last_name, :birth_date, :employed_at
-
-  100.times do
-    first_name Faker::Name.first_name
-    last_name Faker::Name.last_name
-    birth_date Faker::Date.between(Date.today << 1000, Date.today << 200)
-    employed_at [Faker::Company.name, nil].sample
-  end
+SimpleCsv.read('spec/files/headerless.csv', has_headers: false) do
+  headers :first_name, :last_name, :birth_date, :employed_at, employed_at: :job
+  each_row { p job }
 end
 
-# CSV read test
-SimpleCsv.read('spec/files/output.csv') do
-  headers :first_name, :last_name, :birth_date, :employed_at
-
-  in_groups_of(10) do
-    each_row :with_index do
-      puts [index, first_name, last_name,
-            birth_date, employed_at].compact.join ', '
-    end
-    sleep 1
-  end
-end
+# Pry.start

data/lib/simple_csv/base.rb

@@ -2,28 +2,38 @@ module SimpleCsv
   class Base
     attr_reader :index
 
-    COMMON_DELIMITERS = %w(, ; |)
-    DEFAULTS = { headers: true, col_sep: ',', seperator: ',',
-                 converters: [:all, :blank_to_nil, :null_to_nil] }.freeze
+    COMMON_DELIMITERS = %w(, ; |).freeze
+
+    def debug_headers
+      p "@headers is now: #{@headers.join ','}"
+      p "@col_map is now #{@col_map}"
+    end
 
     private
 
     def settings(**opts)
-      @settings ||= DEFAULTS.dup
-      @settings = @settings.merge opts[:merge] if opts[:merge]
+      return @settings.merge opts if opts.any? && @settings
+      @settings ||= Settings.new opts
+    end
+
+    def headers(*cols, **col_map)
+      @headers ||= []
+      @headers.concat cols.map(&:to_s) if cols.any?
+
+      @col_map ||= {}
+      @col_map.merge! stringify_col_map(col_map) if col_map.any?
 
-      @settings[:col_sep] = @settings.delete :seperator if @settings[:seperator]
+      @headers_set ||= @headers.any?
 
-      @settings
+      @headers
     end
 
-    def headers(*column_names)
-      return @headers if column_names.empty?
-      @headers = column_names.map(&:to_s)
+    def stringify_col_map(col_map)
+      col_map.to_a.map { |m| m.reverse.map(&:to_s) }.to_h
     end
 
     def respond_to_missing?(mtd, include_private = false)
-      @headers.include?(mtd) || super
+      @headers.include?(mtd) || @col_map.key?(mtd.to_s) || super
     end
   end
 end

data/lib/simple_csv/reader.rb

@@ -3,15 +3,13 @@ module SimpleCsv
     attr_reader :index
 
     def initialize(path, **opts, &block)
-      settings merge: opts
+      @csv_path = File.expand_path path
 
-      unless opts[:seperator]
-        settings merge: { seperator: detect_delimiter(path) }
-      end
+      opts[:seperator] ||= detect_delimiter
+      settings.apply opts
 
-      headers(*find_headers(path)) if settings[:headers]
+      load_csv_with_auto_headers if settings.for_csv[:headers]
 
-      @csv = @original = CSV.open File.expand_path(path), settings
       instance_eval(&block)
     end
 
@@ -27,6 +25,8 @@ module SimpleCsv
     def each_row(*arr_opts, &block)
       @index ||= 0 if arr_opts.include?(:with_index)
 
+      load_csv_with_manual_headers unless @csv
+
       @csv.each do |record|
         @record = record
         instance_eval(&block)
@@ -34,30 +34,47 @@ module SimpleCsv
       end
     end
 
-    def respond_to_missing?(mtd, include_private = false)
-      super
-    end
+    private
 
-    def method_missing(mtd, *args, &block)
-      @headers.include?(mtd.to_s) ? @record[mtd.to_s] : super
+    def load_csv_with_auto_headers
+      headers(*find_headers)
+
+      @csv = @original = CSV.open @csv_path, settings.for_csv
     end
 
-    private
+    def load_csv_with_manual_headers
+      SimpleCsv.csv_manually_set_headers! unless @headers_set
+      csv_arr = CSV.open(@csv_path).to_a
 
-    def find_headers(csv_path)
-      sep = detect_delimiter(csv_path)
-      first_line(csv_path).split(sep).map { |h| h.gsub(/^"*|"*$/, '') }
+      if csv_arr.first.size == headers.size
+        csv_str = csv_arr.unshift(headers).map(&:to_csv).join
+        settings.apply(headers: true)
+        @csv = @original = CSV.new csv_str, settings.for_csv
+      else
+        SimpleCsv.csv_not_enough_headers!
+      end
+    end
+
+    def find_headers
+      first_line.split(detect_delimiter).map { |h| h.gsub(/^"*|"*$/, '') }
     end
 
-    def detect_delimiter(csv_path)
-      line = first_line(csv_path)
+    def detect_delimiter
+      line = first_line
       @delimiters = COMMON_DELIMITERS.map { |sep| [sep, line.scan(sep).length] }
                                      .sort { |a, b| b[1] <=> a[1] }
       @delimiter ||= @delimiters[0][0]
     end
 
-    def first_line(csv_path)
-      @first_line ||= File.open(File.expand_path(csv_path), &:readline)
+    def first_line
+      @first_line ||= File.open @csv_path, &:readline
+    end
+
+    def method_missing(mtd, *args, &block)
+      m = mtd.to_s
+      return @record[m] if headers.include?(m)
+      return @record[@col_map[m]] if @col_map[m]
+      super
     end
   end
 end

data/lib/simple_csv/settings.rb

@@ -0,0 +1,59 @@
+module SimpleCsv
+  class Settings
+    DEFAULTS = { headers: true, col_sep: ',', seperator: ',',
+                 force_quotes: true,
+                 converters: [:all, :blank_to_nil, :null_to_nil] }.freeze
+    ALIASSED = { seperator: :col_sep, has_headers: :headers }.freeze
+    INVERTED_ALIASSES = ALIASSED.to_a.map(&:reverse).to_h
+
+    def initialize(**opts)
+      @settings = DEFAULTS.dup.merge opts
+    end
+
+    def [](setting)
+      send setting
+    end
+
+    def []=(m, val)
+      @settings[m] = val
+      return @settings[ALIASSED[m]] = val if ALIASSED.key? m
+      return @settings[INVERTED_ALIASSES[m]] = val if INVERTED_ALIASSES.key? m
+      val
+    end
+
+    def for_csv
+      settings = @settings.dup
+
+      ALIASSED.each do |opt_alias, opt|
+        settings[opt] = settings.delete(opt_alias) if settings.key? opt_alias
+      end
+
+      CSV::DEFAULT_OPTIONS.each_with_object({}) do |(prop, default), csv_hash|
+        csv_hash[prop] = settings.key?(prop) ? settings[prop] : default
+      end
+    end
+
+    def apply(*hashes)
+      hashes.each { |opts| opts.each { |k, v| self[k] = v } } && @settings
+    end
+
+    def any?
+      @settings && @settings.any?
+    end
+
+    def respond_to_missing?(mtd, include_private = false)
+      accepted_method? mtd || super
+    end
+
+    def method_missing(mtd, *args, &block)
+      return super unless accepted_method? mtd
+      mtd = ALIASSED[mtd] if ALIASSED.key? mtd
+      mtd = INVERTED_ALIASSES[mtd] if INVERTED_ALIASSES.key? mtd
+      @settings[mtd]
+    end
+
+    def accepted_method?(mtd)
+      @settings.key?(mtd) || ALIASSED.key?(mtd) || INVERTED_ALIASSES.key?(mtd)
+    end
+  end
+end

data/lib/simple_csv/version.rb

@@ -1,3 +1,3 @@
 module SimpleCsv
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

data/lib/simple_csv/writer.rb

@@ -1,12 +1,10 @@
 module SimpleCsv
   class Writer < Base
-    DEFAULTS = { force_quotes: true }.freeze
-
     def initialize(path, **opts, &block)
-      settings merge: DEFAULTS.merge(opts)
-      CSV.open(File.expand_path(path), 'w', @settings) do |csv|
+      settings.apply({force_row_completion: true}, opts)
+      CSV.open(File.expand_path(path), 'w', settings.for_csv) do |csv|
         @csv = csv
-        @current_row = []
+        @current_row = {}
         instance_eval(&block)
       end
     end
@@ -18,17 +16,25 @@ module SimpleCsv
     end
 
     def method_missing(mtd, *args, &block)
-      super unless @headers.include? mtd.to_s
+      SimpleCsv.csv_manually_set_headers! unless @headers_written
+      super unless headers.include? mtd.to_s
+
+      if settings.force_row_completion && @current_row.key?(mtd)
+        SimpleCsv.row_not_complete!(mtd, args.first)
+      end
 
-      @current_row << args[0]
-      row_complete = @current_row.count == @headers.count
-      (@csv << @current_row) && @current_row = [] if row_complete
+      @current_row[mtd] = args.first || ''
+
+      return unless @current_row.size == headers.size
+
+      @csv << @current_row.values
+      @current_row = {}
     end
 
-    def headers(*column_names)
-      super
-      @csv << @headers if @csv
-      @headers
+    def headers(*col_names)
+      return @headers if col_names.empty?
+      super(*col_names)
+      (@csv << @headers) && @headers_written = true if @csv
     end
   end
 end

data/lib/simple_csv.rb

@@ -1,6 +1,7 @@
 require 'csv'
 
 require 'simple_csv/version'
+require 'simple_csv/settings'
 require 'simple_csv/base'
 require 'simple_csv/reader'
 require 'simple_csv/writer'
@@ -8,6 +9,26 @@ require 'simple_csv/writer'
 module SimpleCsv
   @converters_initialized = false
 
+  class RowNotComplete < StandardError; end
+  class UnparseableCsv < StandardError; end
+  class HeadersNotSet < UnparseableCsv; end
+  class NotEnoughHeaders < UnparseableCsv; end
+
+  def self.row_not_complete!(mtd, value)
+    raise RowNotComplete,
+          "Row not complete! #{mtd} called twice with value #{value}"
+  end
+
+  def self.csv_not_enough_headers!
+    raise NotEnoughHeaders, 'Not enough headers defined!'
+  end
+
+  def self.csv_manually_set_headers!
+    raise HeadersNotSet,
+          ['CSV does not contain headers',
+           'please add headers in them manually or in the file'].join(', ')
+  end
+
   def self.read(path, **options, &block)
     initialize_converters unless converters_initialized
     Reader.new path, options, &block

data/simple_csv.gemspec

@@ -22,6 +22,8 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ['lib']
 
+  spec.required_ruby_version = '>= 2.1'
+
   spec.add_development_dependency 'bundler', '~> 1.13'
   spec.add_development_dependency 'rake', '~> 10.0'
   spec.add_development_dependency 'rspec', '~> 3.0'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: simple_csv
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Sidney Liebrand
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-12-19 00:00:00.000000000 Z
+date: 2016-12-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -117,6 +117,7 @@ extra_rdoc_files: []
 files:
 - ".gitignore"
 - ".rspec"
+- ".rubocop.yml"
 - ".travis.yml"
 - CODE_OF_CONDUCT.md
 - Gemfile
@@ -128,6 +129,7 @@ files:
 - lib/simple_csv.rb
 - lib/simple_csv/base.rb
 - lib/simple_csv/reader.rb
+- lib/simple_csv/settings.rb
 - lib/simple_csv/version.rb
 - lib/simple_csv/writer.rb
 - simple_csv.gemspec
@@ -143,7 +145,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: '2.1'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="