smarter_csv 1.13.0 → 1.13.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -1
- data/README.md +2 -2
- data/docs/header_transformations.md +5 -5
- data/docs/options.md +1 -3
- data/lib/smarter_csv/options.rb +0 -1
- data/lib/smarter_csv/version.rb +1 -1
- data/lib/smarter_csv/writer.rb +1 -0
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2e785bafb4281cfadba23ef7db89a50a1a22f642ca4dd03cd2e97323a8cfa761
|
|
4
|
+
data.tar.gz: 2d3b7f87540a0982582859fc804cb50088d9b4906aa75765fa3bf8d4ac535a23
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 0d3e04d0ec26174a179d42f577c412483956e4ecf8af0e3b4b27234ed52027b984e26afbadfdb8fc1ee95f77780c6856f824dc1ca6e8eb91a87e7a81d46d40ed
|
|
7
|
+
data.tar.gz: 379e3a0daa40afb0158fb546cf33fbb33d52eea77847f587349e69e4f9c6f51233b9953d60c5c3cf0a06e4a30115174cbefb81f42f9c5b2ac495d22a1fa7efa1
|
data/CHANGELOG.md
CHANGED
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
|
|
2
2
|
# SmarterCSV 1.x Change Log
|
|
3
3
|
|
|
4
|
+
## 1.13.1 (2024-12-12)
|
|
5
|
+
* fix bug with SmarterCSV.generate with `force_quotes: true` ([issue 294](https://github.com/tilo/smarter_csv/issues/294))
|
|
6
|
+
|
|
4
7
|
## 1.13.0 (2024-11-06) ⚡ POTENTIALLY BREAKING ⚡
|
|
5
8
|
|
|
6
9
|
CHANGED DEFAULT BEHAVIOR
|
|
@@ -38,7 +41,7 @@
|
|
|
38
41
|
|
|
39
42
|
IF you set `user_provided_headers` and the file has a header, then provide `headers_in_file: true` to avoid getting that extra record.
|
|
40
43
|
|
|
41
|
-
* handling of numeric columns with leading zeroes, e.g. ZIP codes. ([issue #151](https://github.com/tilo/smarter_csv/issues/151) thanks to David Moles). `convert_values_to_numeric: { except: [:zip] }` will
|
|
44
|
+
* improved documentation for handling of numeric columns with leading zeroes, e.g. ZIP codes. ([issue #151](https://github.com/tilo/smarter_csv/issues/151) thanks to David Moles). `convert_values_to_numeric: { except: [:zip] }` will return a string for that column instead (since version 1.10.x)
|
|
42
45
|
|
|
43
46
|
## 1.12.1 (2024-07-10)
|
|
44
47
|
* Improved column separator detection by ignoring quoted sections [#276](https://github.com/tilo/smarter_csv/pull/276) (thanks to Nicolas Castellanos)
|
data/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
|
|
2
2
|
# SmarterCSV
|
|
3
3
|
|
|
4
|
-
[](https://codecov.io/gh/tilo/smarter_csv)
|
|
4
|
+
[](https://codecov.io/gh/tilo/smarter_csv)  [View on RubyGems](https://rubygems.org/gems/smarter_csv) [View on RubyToolbox](https://www.ruby-toolbox.com/search?q=smarter_csv)
|
|
5
5
|
|
|
6
6
|
SmarterCSV provides a convenient interface for reading and writing CSV files and data.
|
|
7
7
|
|
|
@@ -47,7 +47,7 @@ Or install it yourself as:
|
|
|
47
47
|
# Articles
|
|
48
48
|
* [Parsing CSV Files in Ruby with SmarterCSV](https://tilo-sloboda.medium.com/parsing-csv-files-in-ruby-with-smartercsv-6ce66fb6cf38)
|
|
49
49
|
* [Processing 1.4 Million CSV Records in Ruby, fast ](https://lcx.wien/blog/processing-14-million-csv-records-in-ruby/)
|
|
50
|
-
* [
|
|
50
|
+
* [Faster Parsing CSV with Parallel Processing](http://xjlin0.github.io/tech/2015/05/25/faster-parsing-csv-with-parallel-processing) by [Jack lin](https://github.com/xjlin0/)
|
|
51
51
|
* [The original post](http://www.unixgods.org/Ruby/process_csv_as_hashes.html) that started SmarterCSV
|
|
52
52
|
|
|
53
53
|
# [ChangeLog](./CHANGELOG.md)
|
|
@@ -81,8 +81,8 @@ There is an additional option `remove_unmapped_keys` which can be enabled to onl
|
|
|
81
81
|
|
|
82
82
|
## CSV Files without Headers
|
|
83
83
|
|
|
84
|
-
If you have CSV files without headers, it is important to set `headers_in_file: false`, otherwise you'll lose the first data line in your file.
|
|
85
|
-
You then have to provide `user_provided_headers`, which takes an array of either symbols or strings.
|
|
84
|
+
If you have CSV files without headers, it is important to set `headers_in_file: false`, otherwise you'll lose the first data line in your file.
|
|
85
|
+
You then have to provide `user_provided_headers`, which takes an array of either symbols or strings. Versions >1.13 now automatically set `headers_in_file: false` if you provide `user_provided_headers`. Also see next paragraph.
|
|
86
86
|
|
|
87
87
|
|
|
88
88
|
## CSV Files with Headers
|
|
@@ -93,6 +93,7 @@ For CSV files with headers, you can either:
|
|
|
93
93
|
* map one or more headers into whatever you chose using the `map_headers` option.
|
|
94
94
|
(if you map a header to `nil`, it will remove that column from the resulting row hash).
|
|
95
95
|
* completely replace the headers using `user_provided_headers` (please be careful with this powerful option, as it is not robust against changes in input format).
|
|
96
|
+
When you use `user_provided_headers`, versions >1.13 will set `headers_in_file: false` -- so if you replace the headers for a file that has headers, you must set `headers_in_file: true` to override this and ignore the header row.
|
|
96
97
|
* use the original unmodified headers from the CSV file, using `keep_original_headers`. This results in hash keys that are strings, and may be padded with spaces.
|
|
97
98
|
|
|
98
99
|
|
|
@@ -104,11 +105,10 @@ For CSV files with headers, you can either:
|
|
|
104
105
|
* any occurences of :comment_regexp or :row_sep will be stripped from the first line with the CSV header
|
|
105
106
|
* any of the keys in the header line will be downcased, spaces replaced by underscore, and converted to Ruby symbols before being used as keys in the returned Hashes
|
|
106
107
|
* you can not combine the :user_provided_headers and :key_mapping options
|
|
107
|
-
* if the incorrect number of headers are provided via :user_provided_headers,
|
|
108
|
+
* if the incorrect number of headers are provided via :user_provided_headers, versions >1.13 will automatically add column names `column_N` for additional unexpected columns. If you want to raise an error instead, add option `strict: true`, and it will raise `SmarterCSV::HeaderSizeMismatch`.
|
|
108
109
|
|
|
109
110
|
### NOTES on improper quotation and unwanted characters in headers:
|
|
110
|
-
* some CSV files use un-escaped quotation characters inside fields. This can cause the import to break. To get around this,
|
|
111
|
-
If you would force a different :quote_char instead (setting it to a non-used character), then the import would be up to 5-times slower than using `:force_simple_split`.
|
|
111
|
+
* some CSV files use un-escaped quotation characters inside fields. This can cause the import to break. To get around this, set the `quote_char` to something different, e.g. `quote_char: "%"`, or try setting `:strip_chars_from_headers => /[\-"]/`
|
|
112
112
|
|
|
113
113
|
---------------
|
|
114
114
|
PREVIOUS: [Row and Column Separators](./row_col_sep.md) | NEXT: [Header Validations](./header_validations.md)
|
data/docs/options.md
CHANGED
|
@@ -41,9 +41,7 @@
|
|
|
41
41
|
| :skip_lines | nil | how many lines to skip before the first line or header line is processed |
|
|
42
42
|
| :comment_regexp | nil | regular expression to ignore comment lines (see NOTE on CSV header), e.g./\A#/ |
|
|
43
43
|
---------------------------------------------------------------------------------------------------------------------------------
|
|
44
|
-
| :col_sep |
|
|
45
|
-
| :force_simple_split | false | force simple splitting on :col_sep character for non-standard CSV-files. |
|
|
46
|
-
| | | e.g. when :quote_char is not properly escaped |
|
|
44
|
+
| :col_sep | :auto | column separator (default was ',') |
|
|
47
45
|
| :row_sep | :auto | row separator or record separator (previous default was system's $/ , which defaulted to "\n") |
|
|
48
46
|
| | | This can also be set to :auto, but will process the whole cvs file first (slow!) |
|
|
49
47
|
| :auto_row_sep_chars | 500 | How many characters to analyze when using `:row_sep => :auto`. nil or 0 means whole file. |
|
data/lib/smarter_csv/options.rb
CHANGED
data/lib/smarter_csv/version.rb
CHANGED
data/lib/smarter_csv/writer.rb
CHANGED
|
@@ -81,6 +81,7 @@ module SmarterCSV
|
|
|
81
81
|
def finalize
|
|
82
82
|
# Map headers if :map_headers option is provided
|
|
83
83
|
mapped_headers = @headers.map { |header| @map_headers[header] || header }
|
|
84
|
+
mapped_headers = mapped_headers.map{|x| escape_csv_field(x)} if @force_quotes
|
|
84
85
|
|
|
85
86
|
@temp_file.rewind
|
|
86
87
|
@output_file.write(mapped_headers.join(@col_sep) + @row_sep)
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: smarter_csv
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.13.
|
|
4
|
+
version: 1.13.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Tilo Sloboda
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2024-
|
|
11
|
+
date: 2024-12-12 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: awesome_print
|
|
@@ -165,7 +165,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
165
165
|
- !ruby/object:Gem::Version
|
|
166
166
|
version: '0'
|
|
167
167
|
requirements: []
|
|
168
|
-
rubygems_version: 3.
|
|
168
|
+
rubygems_version: 3.5.4
|
|
169
169
|
signing_key:
|
|
170
170
|
specification_version: 4
|
|
171
171
|
summary: Convenient CSV Reading and Writing
|