honey_format 0.12.0 → 0.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6d236d3a0eeee26825fc307504a6e65c66dbd30c5cbc8b4d190f5af391e73fc0
-  data.tar.gz: e5989adcda923101ccff44d34b15adffe7db4cbd1cabdfad3229dace94076c6b
+  metadata.gz: 0c4d5fc0d404dc7820a766b51307451f8900497495e51b42414498c569dbf782
+  data.tar.gz: 96faa29782adeb254a1a8c3e9a4a2a2f19bd88d261bce4a96675ad9f234d6ae4
 SHA512:
-  metadata.gz: 5fa617674f689a27707c15a6a9d307181a8bda8662da9f40e5a3a689821c1269ca34762c4609511a7265cf75c04e551c1e1de8d5e2d0825826ff6a422aa77398
-  data.tar.gz: de36bd4aa3bd05e3e9f99d703b0ffe4674f894edaa85aecc3b51114f434dcdf9d6ea3b23c9b3b2b0c0c5319d63849c49532d46bcccb044ebc4190c77c9212eeb
+  metadata.gz: 6dd3cd150abd94f14b9f4416b2dc5d42756b1901f963144e8850cafe344ce657443f188d32d4cc008437a3d42430cda9b02a0d25f741f0417ede3b838cc848a8
+  data.tar.gz: 0ec2641fc7346b44376460a8587a70248f7f61d249d3277e6a9a50f7d2f2f8b7ad8225f4f1bb4e1d797ee6adc85242026acb71ec9c8526499f403b86bd608d6c

data/CHANGELOG.md

@@ -1,3 +1,19 @@
+# HEAD
+
+# v0.13.0
+
+:warning: This release contains some backwards compatible changes.
+
+* Extract `Matrix` super class from `CSV`
+* Add `Header#empty?` and `Rows#empty?`
+* Value converters [[#PR15](https://github.com/buren/honey_format/pull/15)]
+  + Convert column value to number, date, etc..
+  + Additional converters in [[#PR20](https://github.com/buren/honey_format/pull/20)]
+* Add support for CSV row delimiter and quote character [[#PR15](https://github.com/buren/honey_format/pull/15)]
+* :warning: `CSV#header` now returns an instance of `Header` instead of an array of the original header columns [[#PR15](https://github.com/buren/honey_format/pull/15)]
+* Add `--[no-]rows-only` CLI option
+* Rename `--[no-]only-header` CLI option to `--[no-]header-only`
+
 # v0.12.0
 
 * Add `--[no-]only-header` option to CLI

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2015 Jacob Burenstam Linder
+Copyright (c) 2018 Jacob Burenstam Linder
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,28 +1,40 @@
-# HoneyFormat [![Build Status](https://travis-ci.org/buren/honey_format.svg)](https://travis-ci.org/buren/honey_format) [![Code Climate](https://codeclimate.com/github/buren/honey_format/badges/gpa.svg)](https://codeclimate.com/github/buren/honey_format) [![Inline docs](http://inch-ci.org/github/buren/honey_format.svg)](http://inch-ci.org/github/buren/honey_format)
+# HoneyFormat [![Build Status](https://travis-ci.org/buren/honey_format.svg)](https://travis-ci.org/buren/honey_format) [![Code Climate](https://codeclimate.com/github/buren/honey_format/badges/gpa.svg)](https://codeclimate.com/github/buren/honey_format) [![Inline docs](http://inch-ci.org/github/buren/honey_format.svg)](https://www.rubydoc.info/gems/honey_format/)
 
-Convert CSV to an array of objects with with ease.
+> Makes working with CSVs as smooth as honey.
+
+Proper objects for CSV headers and rows, convert column values, filter columns and rows, small(-ish) perfomance overhead, no dependencies other than Ruby stdlib.
 
 ## Features
 
 - Proper objects for CSV header and rows
-- Convert column values with custom row builder
+- Convert column values
+- Pass your own custom row builder
 - Convert header column names
-- Customize what columns and rows are included in CSV output
-- [CLI](#cli)
+- Filter what columns and rows are included in CSV output
+- [CLI](#cli) - Simple command line interface
+- Only ~5-10% overhead from using Ruby CSV, see [benchmarks](#benchmark)
 - Has no dependencies other than Ruby stdlib
 - Supports Ruby >= 2.3
 
-## Examples
+Read the [usage section](#usage),  [RubyDoc](https://www.rubydoc.info/gems/honey_format/) or [examples/ directory](https://github.com/buren/honey_format/tree/master/examples)  for how to use this gem.
+
+## Quick use
 
-See [examples/](https://github.com/buren/honey_format/tree/master/examples) for more examples.
 
 ```ruby
-csv_string = "Id,Username\n1,buren"
-csv = HoneyFormat::CSV.new(csv_string)
-csv.header      # => ["Id", "Username"]
+csv_string = <<-CSV
+Id,Username,Email
+1,buren,buren@example.com
+2,jacob,jacob@example.com
+CSV
+csv = HoneyFormat::CSV.new(csv_string, type_map: { id: :integer })
+csv.columns     # => [:id, :username]
 user = csv.rows # => [#<struct id="1", username="buren">]
-user.id         # => "1"
+user.id         # => 1
 user.username   # => "buren"
+
+csv.to_csv(columns: [:id, :username]) { |row| row.id < 2 }
+# => "id,username\n1,buren\n"
 ```
 
 ## Installation
@@ -45,21 +57,73 @@ $ gem install honey_format
 
 ## Usage
 
-By default assumes a header in the CSV file.
+By default assumes a header in the CSV file
 
 ```ruby
 csv_string = "Id,Username\n1,buren"
 csv = HoneyFormat::CSV.new(csv_string)
-csv.header # => ["Id", "Username"]
-csv.columns # => [:id, :username]
 
+# Header
+header = csv.header
+header.original # => ["Id", "Username"]
+header.columns # => [:id, :username]
+
+
+# Rows
 rows = csv.rows # => [#<struct id="1", username="buren">]
 user = rows.first
 user.id         # => "1"
 user.username   # => "buren"
 ```
 
-Minimal custom row builder
+Set delimiter & quote character
+```ruby
+csv_string = "name;id|'John Doe';42"
+csv = HoneyFormat::CSV.new(
+  csv_string,
+  delimiter: ';',
+  row_delimiter: '|',
+  quote_character: "'",
+)
+```
+
+__Type converters__
+
+> Type converters are great if you want to convert column values, like numbers and dates.
+
+There are a few default type converters
+```ruby
+csv_string = "Id,Username\n1,buren"
+type_map = { id: :integer }
+csv = HoneyFormat::CSV.new(csv_string, type_map: type_map)
+csv.rows.first.id # => 1
+```
+
+Add your own converter
+```ruby
+HoneyFormat.configure do |config|
+  config.converter.register :upcased, proc { |v| v.upcase }
+end
+
+csv_string = "Id,Username\n1,buren"
+type_map = { username: :upcased }
+csv = HoneyFormat::CSV.new(csv_string, type_map: type_map)
+csv.rows.first.username # => "BUREN"
+```
+
+Access registered converters
+```ruby
+decimal_converter = HoneyFormat.value_converter[:decimal]
+decimal_converter.call('1.1') # => 1.1
+```
+
+See [`ValueConverter::DEFAULT_CONVERTERS`](https://github.com/buren/honey_format/tree/master/lib/honey_format/value_converter.rb) for a complete list of the default ones.
+
+__Row builder__
+
+> Pass your own row builder if you want more control of the entire row or if you want to return your own row object.
+
+Custom row builder
 ```ruby
 csv_string = "Id,Username\n1,buren"
 upcaser = ->(row) { row.tap { |r| r.username.upcase! } }
@@ -67,26 +131,38 @@ csv = HoneyFormat::CSV.new(csv_string, row_builder: upcaser)
 csv.rows # => [#<struct id="1", username="BUREN">]
 ```
 
-Complete custom row builder
+As long as the row builder responds to `#call` you can pass anything you like
 ```ruby
 class Anonymizer
-  def self.call(row)
+  def call(row)
+    @cache ||= {}
     # Return an object you want to represent the row
     row.tap do |r|
-      r.name = '<anon>'
-      r.email = '<anon>'
-      r.ssn = '<anon>'
+      # given the same value make sure to return the same anonymized value every time
+      @cache[r.email] ||= "#{SecureRandom.hex(6)}@example.com"
+      r.email = @cache[r.email]
       r.payment_id = '<scrubbed>'
     end
   end
 end
 
-csv_string = "Id,Username\n1,buren"
-csv = HoneyFormat::CSV.new(csv_string, row_builder: Anonymizer)
-csv.rows # => [#<struct id="1", username="BUREN">]
+csv_string = <<~CSV
+Email,Payment ID
+buren@example.com,123
+buren@example.com,998
+CSV
+csv = HoneyFormat::CSV.new(csv_string, row_builder: Anonymizer.new)
+csv.rows.to_csv(columns: [:email])
+# => 8f6ed70a7f98@example.com
+#    8f6ed70a7f98@example.com
+#    0db96f350cea@example.com
 ```
 
-Output CSV
+__Output CSV__
+
+> Makes it super easy to output a subset of columns/rows.
+
+Manipulate the rows before output
 ```ruby
 csv_string = "Id,Username\n1,buren"
 csv = HoneyFormat::CSV.new(csv_string)
@@ -94,35 +170,33 @@ csv.rows.each { |row| row.id = nil }
 csv.to_csv # => "id,username\n,buren\n"
 ```
 
-Output a subset of columns to CSV
+Output a subset of columns
 ```ruby
 csv_string = "Id, Username, Country\n1,buren,Sweden"
 csv = HoneyFormat::CSV.new(csv_string)
 csv.to_csv(columns: [:id, :country]) # => "id,country\nburen,Sweden\n"
 ```
 
-Output a subset of rows to CSV
+Output a subset of rows
 ```ruby
 csv_string = "Name, Country\nburen,Sweden\njacob,Denmark"
 csv = HoneyFormat::CSV.new(csv_string)
 csv.to_csv { |row| row.country == 'Sweden' } # => "name,country\nburen,Sweden\n"
 ```
 
-You can of course set the delimiter
-```ruby
-HoneyFormat::CSV.new(csv_string, delimiter: ';')
-```
+__Headers__
 
-Validate CSV header
+> By default generates method-like names for each header column, but also gives you full control: define them or convert them.
+
+By default assumes a header in the CSV file.
 ```ruby
 csv_string = "Id,Username\n1,buren"
-# Invalid
-HoneyFormat::CSV.new(csv_string, valid_columns: [:something, :username])
-# => HoneyFormat::UnknownHeaderColumnError (column :id not in [:something, :username])
+csv = HoneyFormat::CSV.new(csv_string)
 
-# Valid
-csv = HoneyFormat::CSV.new(csv_string, valid_columns: [:id, :username])
-csv.rows.first.username # => "buren"
+# Header
+header = csv.header
+header.original # => ["Id", "Username"]
+header.columns # => [:id, :username]
 ```
 
 Define header
@@ -132,32 +206,33 @@ csv = HoneyFormat::CSV.new(csv_string, header: ['Id', 'Username'])
 csv.rows.first.username # => "buren"
 ```
 
-If your header contains special chars and/or chars that can't be part of Ruby method names,
-things can get a little awkward..
+Set default header converter
 ```ruby
-csv_string = "ÅÄÖ\nSwedish characters"
-user = HoneyFormat::CSV.new(csv_string).rows.first
-# Note that these chars aren't "downcased" in Ruby 2.3 and older versions of Ruby,
-# "ÅÄÖ".downcase # => "ÅÄÖ"
-user.ÅÄÖ # => "Swedish characters"
-# while on Ruby > 2.3
-user.åäö
+HoneyFormat.configure do |config|
+  config.header_converter = proc { |v| v.downcase }
+end
 
-csv_string = "First^Name\nJacob"
-user = HoneyFormat::CSV.new(csv_string).rows.first
-user.public_send(:"first^name") # => "Jacob"
-# or
-user['first^name'] # => "Jacob"
+# you can get the default one with
+header_converter = HoneyFormat.value_converter[:header_column]
+header_converter.call('First name') # => "first_name"
+```
+
+Use any value converter as the header converter
+```ruby
+csv_string = "Id,Username\n1,buren"
+csv = HoneyFormat::CSV.new(csv_string, header_converter: :upcase)
+csv.columns # => [:ID, :USERNAME]
 ```
 
 Pass your own header converter
 ```ruby
 map = { 'First^Name' => :first_name }
-converter = ->(column) { map.fetch(column, column) }
+converter = ->(column) { map.fetch(column, column.downcase) }
 
-csv_string = "First^Name\nJacob"
+csv_string = "ID,First^Name\n1,Jacob"
 user = HoneyFormat::CSV.new(csv_string, header_converter: converter).rows.first
 user.first_name # => "Jacob"
+user.id # => "1"
 ```
 
 Missing header values
@@ -168,9 +243,30 @@ user = csv.rows.first
 user.column1 # => "val1"
 ```
 
-Errors
+If your header contains special chars and/or chars that can't be part of Ruby method names,
+things can get a little awkward..
+```ruby
+csv_string = "ÅÄÖ\nSwedish characters"
+user = HoneyFormat::CSV.new(csv_string).rows.first
+# Note that these chars aren't "downcased" in Ruby 2.3 and older versions of Ruby,
+# "ÅÄÖ".downcase # => "ÅÄÖ"
+user.ÅÄÖ # => "Swedish characters"
+# while on Ruby > 2.3
+user.åäö
+
+csv_string = "First^Name\nJacob"
+user = HoneyFormat::CSV.new(csv_string).rows.first
+user.public_send(:"first^name") # => "Jacob"
+# or
+user['first^name'] # => "Jacob"
+```
+
+__Errors__
+
+> When you need that some extra safety.
+
+If you want to there are some errors you can rescue
 ```ruby
-# there are two error super classes
 begin
   HoneyFormat::CSV.new(csv_string)
 rescue HoneyFormat::HeaderError => e
@@ -184,13 +280,17 @@ end
 
 You can see all [available errors here](https://www.rubydoc.info/gems/honey_format/HoneyFormat/Errors).
 
-If you want to see more usage examples check out the `spec/` directory.
+If you want to see more usage examples check out the [`examples/`](https://github.com/buren/honey_format/tree/master/examples) and [`spec/`](https://github.com/buren/honey_format/tree/master/spec) directories.
 
 ## CLI
 
+> Perfect when you want to get something simple done quickly.
+
 ```
 Usage: honey_format [file.csv] [options]
         --csv=input.csv              CSV file
+        --[no-]header-only           Print only the header
+        --[no-]rows-only             Print only the rows
         --columns=id,name            Select columns.
         --output=output.csv          CSV output (STDOUT otherwise)
         --delimiter=,                CSV delimiter (default: ,)
@@ -198,30 +298,48 @@ Usage: honey_format [file.csv] [options]
         --version                    Show version
 ```
 
-## Benchmark
-
-_Note_: This gem, adds some overhead to parsing a CSV string. I've included some benchmarks below, your mileage may vary..
-
-You can run the benchmarks yourself:
+Output a subset of columns to a new file
+```
+# input.csv
+id,name,username
+1,jacob,buren
+```
 
 ```
-$ bin/benchmark file.csv
+$ honey_format input.csv --columns=id,username > output.csv
 ```
 
+
+## Benchmark
+
+_Note_: This gem, adds some overhead to parsing a CSV string, typically ~5-10%. I've included some benchmarks below, your mileage may vary..
+
 204KB (1k lines)
 
 ```
-      stdlib CSV:       51.9 i/s
-HoneyFormat::CSV:       49.6 i/s - 1.05x  slower
+ CSV no options:       51.0 i/s
+ CSV with header:      36.1 i/s - 1.41x  slower
+HoneyFormat::CSV:      48.7 i/s - 1.05x  slower
 ```
 
 2MB (10k lines)
 
 ```
-      stdlib CSV:        4.6 i/s
-HoneyFormat::CSV:        4.2 i/s - 1.08x  slower
+  CSV no options:        5.1 i/s
+ CSV with header:        3.6 i/s - 1.42x  slower
+HoneyFormat::CSV:        4.9 i/s - 1.05x  slower
 ```
 
+You can run the benchmarks yourself
+```
+Usage: bin/benchmark [file.csv] [options]
+        --csv=[file1.csv]            CSV file(s)
+        --[no-]verbose               Verbose output
+        --lines-multipliers=[1,2,10] Multiply the rows in the CSV file (default: 1)
+        --time=[30]                  Benchmark time (default: 30)
+        --warmup=[30]                Benchmark warmup (default: 30)
+    -h, --help                       How to use
+```
 
 ## Development
 

data/bin/benchmark

@@ -4,72 +4,43 @@ require 'honey_format'
 
 require 'benchmark/ips'
 require 'csv'
-require 'optparse'
 
-input_path = nil
-benchmark_time = 30
-benchmark_warmup = 5
-lines_multipliers = [1]
+require 'honey_format/cli/benchmark_cli'
 
-OptionParser.new do |parser|
-  parser.banner = "Usage: bin/benchmark [file.csv] [options]"
-  parser.default_argv = ARGV
+cli = HoneyFormat::BenchmarkCLI.new
+Writer = cli.writer
+options = cli.options
 
-  parser.on("--csv=file1.csv", String, "CSV file(s)") do |value|
-    input_path = value
-  end
-
-  parser.on("--lines-multipliers=[1,10,50]", Array, "Multiply the rows in the CSV file (default: 1)") do |value|
-    lines_multipliers = value.map do |v|
-      Integer(v).tap do |int|
-        unless int >= 1
-          raise(ArgumentError, '--lines-multiplier must be 1 or greater')
-        end
-      end
-    end
-  end
-
-  parser.on("--time=[30]", String, "Benchmark time (default: 30)") do |value|
-    benchmark_time = Integer(value)
-  end
-
-  parser.on("--warmup=[30]", String, "Benchmark warmup (default: 30)") do |value|
-    benchmark_warmup = Integer(value)
-  end
-
-  parser.on("-h", "--help", "How to use") do
-    puts parser
-    exit
-  end
+input_path = options[:input_path]
+benchmark_time = options[:benchmark_time]
+benchmark_warmup = options[:benchmark_warmup]
+lines_multipliers = options[:lines_multipliers]
 
-  # No argument, shows at tail. This will print an options summary.
-  parser.on_tail("-h", "--help", "Show this message") do
-    puts parser
-    exit
-  end
-end.parse!
+original_csv = input_path ? File.read(input_path) : cli.fetch_default_benchmark_csv
+original_csv_lines = original_csv.lines
 
-csv = File.read(input_path)
+runtime_seconds = cli.expected_runtime_seconds(report_count: 3)
+Writer.puts "Expected runtime: ~#{runtime_seconds} seconds.", verbose: true
 
-lines_multipliers.each do |lines_multiplier|
-  if lines_multiplier > 1
-    orignial_csv_lines = csv.lines
-    rows = orignial_csv_lines[1..-1] * lines_multiplier
-    csv = orignial_csv_lines.first + rows.join
-  end
+lines_multipliers.each_with_index do |lines_multiplier, index|
+  rows = original_csv_lines[1..-1] * lines_multiplier
+  csv =  original_csv_lines.first + rows.join
 
   line_count = csv.lines.length
 
-  puts "== [START] Benchmark for #{line_count} lines =="
+  Writer.puts "== Benchmark #{index + 1} of #{lines_multipliers.length} =="
+  Writer.puts "path           #{cli.used_input_path}"
+  Writer.puts "lines          #{line_count}"
+  Writer.puts "multiplier     #{lines_multiplier}"
+
   Benchmark.ips do |x|
     x.time = benchmark_time
     x.warmup = benchmark_warmup
 
-    x.report('stdlib CSV no options')  { CSV.parse(csv) }
-    x.report('stdlib CSV with header') { CSV.parse(csv, headers: true) }
-    x.report('HoneyFormat::CSV')       { HoneyFormat::CSV.new(csv).rows }
+    x.report('CSV no options')   { CSV.parse(csv) }
+    x.report('CSV with header')  { CSV.parse(csv, headers: true) }
+    x.report('HoneyFormat::CSV') { HoneyFormat::CSV.new(csv).rows }
 
     x.compare!
   end
-  puts "== [END] Benchmark for #{line_count} lines =="
 end