smarter_csv 1.13.0 → 1.13.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (9) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +4 -1
  3. data/README.md +2 -2
  4. data/docs/header_transformations.md +5 -5
  5. data/docs/options.md +1 -3
  6. data/lib/smarter_csv/options.rb +0 -1
  7. data/lib/smarter_csv/version.rb +1 -1
  8. data/lib/smarter_csv/writer.rb +1 -0
  9. metadata +3 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c28d21a143e743de4b21d8ad93d860b1d51424e525e9ec0a73bb640b170d9823
4
- data.tar.gz: e7a16ae8494196b85d9a196d071f09b302583c9ef3a414f09f5ad6ae1f11c29b
3
+ metadata.gz: 2e785bafb4281cfadba23ef7db89a50a1a22f642ca4dd03cd2e97323a8cfa761
4
+ data.tar.gz: 2d3b7f87540a0982582859fc804cb50088d9b4906aa75765fa3bf8d4ac535a23
5
5
  SHA512:
6
- metadata.gz: 319ee5aed33630e9670a1c95cc8da6fd57df9d1d7db57a00af79c1e5c10de56b4e9054c86b6b462ebdc693513a79aff2c881a1ede00ad28e5da58768b4a6f2cf
7
- data.tar.gz: 6b36378d3a15ed9065c697f56f2cafc359d2c746e5796780a276c1a87c6a04be38616205f31ec9341412f3c9a4f52d150ce4ef95c0e76f340368d5683b1452e6
6
+ metadata.gz: 0d3e04d0ec26174a179d42f577c412483956e4ecf8af0e3b4b27234ed52027b984e26afbadfdb8fc1ee95f77780c6856f824dc1ca6e8eb91a87e7a81d46d40ed
7
+ data.tar.gz: 379e3a0daa40afb0158fb546cf33fbb33d52eea77847f587349e69e4f9c6f51233b9953d60c5c3cf0a06e4a30115174cbefb81f42f9c5b2ac495d22a1fa7efa1
data/CHANGELOG.md CHANGED
@@ -1,6 +1,9 @@
1
1
 
2
2
  # SmarterCSV 1.x Change Log
3
3
 
4
+ ## 1.13.1 (2024-12-12)
5
+ * fix bug with SmarterCSV.generate with `force_quotes: true` ([issue 294](https://github.com/tilo/smarter_csv/issues/294))
6
+
4
7
  ## 1.13.0 (2024-11-06) ⚡ POTENTIALLY BREAKING ⚡
5
8
 
6
9
  CHANGED DEFAULT BEHAVIOR
@@ -38,7 +41,7 @@
38
41
 
39
42
  IF you set `user_provided_headers` and the file has a header, then provide `headers_in_file: true` to avoid getting that extra record.
40
43
 
41
- * handling of numeric columns with leading zeroes, e.g. ZIP codes. ([issue #151](https://github.com/tilo/smarter_csv/issues/151) thanks to David Moles). `convert_values_to_numeric: { except: [:zip] }` will now return a string for that column instead.
44
+ * improved documentation for handling of numeric columns with leading zeroes, e.g. ZIP codes. ([issue #151](https://github.com/tilo/smarter_csv/issues/151) thanks to David Moles). `convert_values_to_numeric: { except: [:zip] }` will return a string for that column instead (since version 1.10.x)
42
45
 
43
46
  ## 1.12.1 (2024-07-10)
44
47
  * Improved column separator detection by ignoring quoted sections [#276](https://github.com/tilo/smarter_csv/pull/276) (thanks to Nicolas Castellanos)
data/README.md CHANGED
@@ -1,7 +1,7 @@
1
1
 
2
2
  # SmarterCSV
3
3
 
4
- [![codecov](https://codecov.io/gh/tilo/smarter_csv/branch/main/graph/badge.svg?token=1L7OD80182)](https://codecov.io/gh/tilo/smarter_csv) [![Gem Version](https://badge.fury.io/rb/smarter_csv.svg)](http://badge.fury.io/rb/smarter_csv)
4
+ [![codecov](https://codecov.io/gh/tilo/smarter_csv/branch/main/graph/badge.svg?token=1L7OD80182)](https://codecov.io/gh/tilo/smarter_csv) ![Gem Version](https://img.shields.io/gem/v/smarter_csv) [View on RubyGems](https://rubygems.org/gems/smarter_csv) [View on RubyToolbox](https://www.ruby-toolbox.com/search?q=smarter_csv)
5
5
 
6
6
  SmarterCSV provides a convenient interface for reading and writing CSV files and data.
7
7
 
@@ -47,7 +47,7 @@ Or install it yourself as:
47
47
  # Articles
48
48
  * [Parsing CSV Files in Ruby with SmarterCSV](https://tilo-sloboda.medium.com/parsing-csv-files-in-ruby-with-smartercsv-6ce66fb6cf38)
49
49
  * [Processing 1.4 Million CSV Records in Ruby, fast ](https://lcx.wien/blog/processing-14-million-csv-records-in-ruby/)
50
- * [Speeding up CSV parsing with parallel processing](http://xjlin0.github.io/tech/2015/05/25/faster-parsing-csv-with-parallel-processing)
50
+ * [Faster Parsing CSV with Parallel Processing](http://xjlin0.github.io/tech/2015/05/25/faster-parsing-csv-with-parallel-processing) by [Jack lin](https://github.com/xjlin0/)
51
51
  * [The original post](http://www.unixgods.org/Ruby/process_csv_as_hashes.html) that started SmarterCSV
52
52
 
53
53
  # [ChangeLog](./CHANGELOG.md)
data/docs/header_transformations.md CHANGED
@@ -81,8 +81,8 @@ There is an additional option `remove_unmapped_keys` which can be enabled to onl
81
81
 
82
82
  ## CSV Files without Headers
83
83
 
84
- If you have CSV files without headers, it is important to set `headers_in_file: false`, otherwise you'll lose the first data line in your file.
85
- You then have to provide `user_provided_headers`, which takes an array of either symbols or strings.
84
+ If you have CSV files without headers, it is important to set `headers_in_file: false`, otherwise you'll lose the first data line in your file.
85
+ You then have to provide `user_provided_headers`, which takes an array of either symbols or strings. Versions >1.13 now automatically set `headers_in_file: false` if you provide `user_provided_headers`. Also see next paragraph.
86
86
 
87
87
 
88
88
  ## CSV Files with Headers
@@ -93,6 +93,7 @@ For CSV files with headers, you can either:
93
93
  * map one or more headers into whatever you chose using the `map_headers` option.
94
94
  (if you map a header to `nil`, it will remove that column from the resulting row hash).
95
95
  * completely replace the headers using `user_provided_headers` (please be careful with this powerful option, as it is not robust against changes in input format).
96
+ When you use `user_provided_headers`, versions >1.13 will set `headers_in_file: false` -- so if you replace the headers for a file that has headers, you must set `headers_in_file: true` to override this and ignore the header row.
96
97
  * use the original unmodified headers from the CSV file, using `keep_original_headers`. This results in hash keys that are strings, and may be padded with spaces.
97
98
 
98
99
 
@@ -104,11 +105,10 @@ For CSV files with headers, you can either:
104
105
  * any occurences of :comment_regexp or :row_sep will be stripped from the first line with the CSV header
105
106
  * any of the keys in the header line will be downcased, spaces replaced by underscore, and converted to Ruby symbols before being used as keys in the returned Hashes
106
107
  * you can not combine the :user_provided_headers and :key_mapping options
107
- * if the incorrect number of headers are provided via :user_provided_headers, exception SmarterCSV::HeaderSizeMismatch is raised
108
+ * if the incorrect number of headers are provided via :user_provided_headers, versions >1.13 will automatically add column names `column_N` for additional unexpected columns. If you want to raise an error instead, add option `strict: true`, and it will raise `SmarterCSV::HeaderSizeMismatch`.
108
109
 
109
110
  ### NOTES on improper quotation and unwanted characters in headers:
110
- * some CSV files use un-escaped quotation characters inside fields. This can cause the import to break. To get around this, use the `:force_simple_split => true` option in combination with `:strip_chars_from_headers => /[\-"]/` . This will also significantly speed up the import.
111
- If you would force a different :quote_char instead (setting it to a non-used character), then the import would be up to 5-times slower than using `:force_simple_split`.
111
+ * some CSV files use un-escaped quotation characters inside fields. This can cause the import to break. To get around this, set the `quote_char` to something different, e.g. `quote_char: "%"`, or try setting `:strip_chars_from_headers => /[\-"]/`
112
112
 
113
113
  ---------------
114
114
  PREVIOUS: [Row and Column Separators](./row_col_sep.md) | NEXT: [Header Validations](./header_validations.md)
data/docs/options.md CHANGED
@@ -41,9 +41,7 @@
41
41
  | :skip_lines | nil | how many lines to skip before the first line or header line is processed |
42
42
  | :comment_regexp | nil | regular expression to ignore comment lines (see NOTE on CSV header), e.g./\A#/ |
43
43
  ---------------------------------------------------------------------------------------------------------------------------------
44
- | :col_sep | :auto | column separator (default was ',') |
45
- | :force_simple_split | false | force simple splitting on :col_sep character for non-standard CSV-files. |
46
- | | | e.g. when :quote_char is not properly escaped |
44
+ | :col_sep | :auto | column separator (default was ',') |
47
45
  | :row_sep | :auto | row separator or record separator (previous default was system's $/ , which defaulted to "\n") |
48
46
  | | | This can also be set to :auto, but will process the whole cvs file first (slow!) |
49
47
  | :auto_row_sep_chars | 500 | How many characters to analyze when using `:row_sep => :auto`. nil or 0 means whole file. |
data/lib/smarter_csv/options.rb CHANGED
@@ -20,7 +20,6 @@ module SmarterCSV
20
20
  downcase_header: true,
21
21
  duplicate_header_suffix: '', # was: nil,
22
22
  file_encoding: 'utf-8',
23
- force_simple_split: false,
24
23
  force_utf8: false,
25
24
  headers_in_file: true,
26
25
  invalid_byte_sequence: '',
data/lib/smarter_csv/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SmarterCSV
4
- VERSION = "1.13.0"
4
+ VERSION = "1.13.1"
5
5
  end
data/lib/smarter_csv/writer.rb CHANGED
@@ -81,6 +81,7 @@ module SmarterCSV
81
81
  def finalize
82
82
  # Map headers if :map_headers option is provided
83
83
  mapped_headers = @headers.map { |header| @map_headers[header] || header }
84
+ mapped_headers = mapped_headers.map{|x| escape_csv_field(x)} if @force_quotes
84
85
 
85
86
  @temp_file.rewind
86
87
  @output_file.write(mapped_headers.join(@col_sep) + @row_sep)
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: smarter_csv
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.13.0
4
+ version: 1.13.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Tilo Sloboda
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2024-11-05 00:00:00.000000000 Z
11
+ date: 2024-12-12 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: awesome_print
@@ -165,7 +165,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
165
165
  - !ruby/object:Gem::Version
166
166
  version: '0'
167
167
  requirements: []
168
- rubygems_version: 3.2.3
168
+ rubygems_version: 3.5.4
169
169
  signing_key:
170
170
  specification_version: 4
171
171
  summary: Convenient CSV Reading and Writing