smarter_csv 1.2.7 → 1.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +2 -0
- data/.travis.yml +8 -4
- data/CHANGELOG.md +169 -0
- data/CONTRIBUTORS.md +45 -0
- data/LICENSE.txt +21 -0
- data/README.md +47 -219
- data/Rakefile +8 -15
- data/lib/smarter_csv/smarter_csv.rb +112 -35
- data/lib/smarter_csv/version.rb +1 -1
- data/lib/smarter_csv.rb +8 -0
- data/smarter_csv.gemspec +20 -16
- data/spec/fixtures/empty_columns_1.csv +2 -0
- data/spec/fixtures/empty_columns_2.csv +2 -0
- data/spec/fixtures/key_mapping.csv +2 -0
- data/spec/fixtures/numeric.csv +1 -1
- data/spec/fixtures/separator_colon.csv +4 -0
- data/spec/fixtures/separator_comma.csv +4 -0
- data/spec/fixtures/separator_pipe.csv +4 -0
- data/spec/fixtures/{separator.csv → separator_semi.csv} +0 -0
- data/spec/fixtures/separator_tab.csv +4 -0
- data/spec/smarter_csv/blank_spec.rb +55 -0
- data/spec/smarter_csv/carriage_return_spec.rb +27 -7
- data/spec/smarter_csv/column_separator_spec.rb +89 -5
- data/spec/smarter_csv/empty_columns_spec.rb +74 -0
- data/spec/smarter_csv/key_mapping_spec.rb +31 -0
- data/spec/smarter_csv/malformed_spec.rb +0 -4
- metadata +44 -13
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3be724101d41326ff480bcb723c1b40a3cabd879eb55e0c2f044372f8e5a57d0
|
4
|
+
data.tar.gz: 657db1421352f449bf042f8df4d5178167af048ad37836e4f2f2f8a6aea3ece0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3430649df35ac8139d35b04b85e8691ca5fc3d98b7b15f0d3987855f571987bdb742e0ed6f807ddb7a2e61e61d696d529ac311bc58e30188325f1c4bb78098a4
|
7
|
+
data.tar.gz: 1b386af7cc7c39bc7ea934875e16f6641a2cc0c2bb5dfaa3b1f298739b1b355b2f41570e42998a2d7790a17f96feb07118b69c23d913acc634aae5901f0c9229
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
@@ -8,10 +8,14 @@ matrix:
|
|
8
8
|
include:
|
9
9
|
- rvm: 2.2.10
|
10
10
|
- rvm: 2.3.8
|
11
|
-
- rvm: 2.4.
|
12
|
-
- rvm: 2.5.
|
13
|
-
- rvm: 2.6.
|
14
|
-
- rvm:
|
11
|
+
- rvm: 2.4.10
|
12
|
+
- rvm: 2.5.8
|
13
|
+
- rvm: 2.6.9
|
14
|
+
- rvm: 2.7.5
|
15
|
+
- rvm: 3.0.3
|
16
|
+
- rvm: 3.1.0
|
17
|
+
- rvm: jruby-9.2.19.0
|
18
|
+
- rvm: jruby-9.3.3.0
|
15
19
|
env:
|
16
20
|
- JRUBY_OPTS="--server -Xcompile.invokedynamic=false -J-XX:+TieredCompilation -J-XX:TieredStopAtLevel=1 -J-noverify -J-Xms512m -J-Xmx1024m"
|
17
21
|
- rvm: ruby-head
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,169 @@
|
|
1
|
+
|
2
|
+
# SmarterCSV 1.x Change Log
|
3
|
+
|
4
|
+
## 1.4.1 (2022-02-12)
|
5
|
+
* minor fix: also support `col_sep: :auto`
|
6
|
+
* added simplecov
|
7
|
+
|
8
|
+
## 1.4.0 (2022-02-11)
|
9
|
+
* dropped GPL license, smarter_csv is now only using the MIT License
|
10
|
+
* added experimental option `col_sep: 'auto` to auto-detect the column separator (issue #183)
|
11
|
+
The default behavior is still to assume `,` is the column separator.
|
12
|
+
* fixed buggy behavior when using `remove_empty_values: false` (issue #168)
|
13
|
+
* fixed Ruby 3.0 deprecation
|
14
|
+
|
15
|
+
## 1.3.0 (2022-02-06) Breaking code change if you used `--key_mappings`
|
16
|
+
* fix bug for key_mappings (issue #181)
|
17
|
+
The values of the `key_mappings` hash will now be used "as is", and no longer forced to be symbols
|
18
|
+
|
19
|
+
**Users with existing code with `--key_mappings` need to change their code** to
|
20
|
+
* either use symbols in the `key_mapping` hash
|
21
|
+
* or change the expected keys from symbols to strings
|
22
|
+
|
23
|
+
## 1.2.9 (2021-11-22) (PULLED)
|
24
|
+
* fix bug for key_mappings (issue #181)
|
25
|
+
The values of the `key_mappings` hash will now be used "as is", and no longer forced to be symbols
|
26
|
+
|
27
|
+
## 1.2.8 (2020-02-04)
|
28
|
+
* fix deprecation warnings on Ruby 2.7 (thank to Diego Salido)
|
29
|
+
|
30
|
+
## 1.2.7 (2020-02-03)
|
31
|
+
|
32
|
+
## 1.2.6 (2018-11-13)
|
33
|
+
* fixing error caused by calling f.close when we do not hand in a file
|
34
|
+
|
35
|
+
## 1.2.5 (2018-09-16)
|
36
|
+
* fixing issue #136 with comments in CSV files
|
37
|
+
* fixing error class hierarchy
|
38
|
+
|
39
|
+
## 1.2.4 (2018-08-06)
|
40
|
+
* using Rails blank? if it's available
|
41
|
+
|
42
|
+
## 1.2.3 (2018-01-27)
|
43
|
+
* fixed regression / test
|
44
|
+
* fuxed quote_char interpolation for headers, but not data (thanks to Colin Petruno)
|
45
|
+
* bugfix (thanks to Joshua Smith for reporting)
|
46
|
+
|
47
|
+
## 1.2.0 (2018-01-20)
|
48
|
+
* add default validation that a header can only appear once
|
49
|
+
* add option `required_headers`
|
50
|
+
|
51
|
+
## 1.1.5 (2017-11-05)
|
52
|
+
* fix issue with invalid byte sequences in header (issue #103, thanks to Dave Myron)
|
53
|
+
* fix issue with invalid byte sequences in multi-line data (thanks to Ivan Ushakov)
|
54
|
+
* analyze only 500 characters by default when `:row_sep => :auto` is used.
|
55
|
+
added option `row_sep_auto_chars` to change the default if necessary. (thanks to Matthieu Paret)
|
56
|
+
|
57
|
+
## 1.1.4 (2017-01-16)
|
58
|
+
* fixing UTF-8 related bug which was introduced in 1.1.2 (thanks to Tirdad C.)
|
59
|
+
|
60
|
+
## 1.1.3 (2016-12-30)
|
61
|
+
* added warning when options indicate UTF-8 processing, but input filehandle is not opened with r:UTF-8 option
|
62
|
+
|
63
|
+
## 1.1.2 (2016-12-29)
|
64
|
+
* added option `invalid_byte_sequence` (thanks to polycarpou)
|
65
|
+
* added comments on handling of UTF-8 encoding when opening from File vs. OpenURI (thanks to KevinColemanInc)
|
66
|
+
|
67
|
+
## 1.1.1 (2016-11-26)
|
68
|
+
* added option to `skip_lines` (thanks to wal)
|
69
|
+
* added option to `force_utf8` encoding (thanks to jordangraft)
|
70
|
+
* bugfix if no headers in input data (thanks to esBeee)
|
71
|
+
* ensure input file is closed (thanks to waldyr)
|
72
|
+
* improved verbose output (thankd to benmaher)
|
73
|
+
* improved documentation
|
74
|
+
|
75
|
+
## 1.1.0 (2015-07-26)
|
76
|
+
* added feature :value_converters, which allows parsing of dates, money, and other things (thanks to Raphaël Bleuse, Lucas Camargo de Almeida, Alejandro)
|
77
|
+
* added error if :headers_in_file is set to false, and no :user_provided_headers are given (thanks to innhyu)
|
78
|
+
* added support to convert dashes to underscore characters in headers (thanks to César Camacho)
|
79
|
+
* fixing automatic detection of \r\n line-endings (thanks to feens)
|
80
|
+
|
81
|
+
## 1.0.19 (2014-10-29)
|
82
|
+
* added option :keep_original_headers to keep CSV-headers as-is (thanks to Benjamin Thouret)
|
83
|
+
|
84
|
+
## 1.0.18 (2014-10-27)
|
85
|
+
* added support for multi-line fields / csv fields containing CR (thanks to Chris Hilton) (issue #31)
|
86
|
+
|
87
|
+
## 1.0.17 (2014-01-13)
|
88
|
+
* added option to set :row_sep to :auto , for automatic detection of the row-separator (issue #22)
|
89
|
+
|
90
|
+
## 1.0.16 (2014-01-13)
|
91
|
+
* :convert_values_to_numeric option can now be qualified with :except or :only (thanks to Hugo Lepetit)
|
92
|
+
* removed deprecated `process_csv` method
|
93
|
+
|
94
|
+
## 1.0.15 (2013-12-07)
|
95
|
+
* new option:
|
96
|
+
* :remove_unmapped_keys to completely ignore columns which were not mapped with :key_mapping (thanks to Dave Sanders)
|
97
|
+
|
98
|
+
## 1.0.14 (2013-11-01)
|
99
|
+
* added GPL-2 and MIT license to GEM spec file; if you need another license contact me
|
100
|
+
|
101
|
+
## 1.0.12 (2013-10-15)
|
102
|
+
* added RSpec tests
|
103
|
+
|
104
|
+
## 1.0.11 (2013-09-28)
|
105
|
+
* bugfix : fixed issue #18 - fixing issue with last chunk not being properly returned (thanks to Jordan Running)
|
106
|
+
* added RSpec tests
|
107
|
+
|
108
|
+
## 1.0.10 (2013-06-26)
|
109
|
+
* bugfix : fixed issue #14 - passing options along to CSV.parse (thanks to Marcos Zimmermann)
|
110
|
+
|
111
|
+
## 1.0.9 (2013-06-19)
|
112
|
+
* bugfix : fixed issue #13 with negative integers and floats not being correctly converted (thanks to Graham Wetzler)
|
113
|
+
|
114
|
+
## 1.0.8 (2013-06-01)
|
115
|
+
|
116
|
+
* bugfix : fixed issue with nil values in inputs with quote-char (thanks to Félix Bellanger)
|
117
|
+
* new options:
|
118
|
+
* :force_simple_split : to force simiple splitting on :col_sep character for non-standard CSV-files. e.g. without properly escaped :quote_char
|
119
|
+
* :verbose : print out line number while processing (to track down problems in input files)
|
120
|
+
|
121
|
+
## 1.0.7 (2013-05-20)
|
122
|
+
|
123
|
+
* allowing process to work with objects with a 'readline' method (thanks to taq)
|
124
|
+
* added options:
|
125
|
+
* :file_encoding : defaults to utf8 (thanks to MrTin, Paxa)
|
126
|
+
|
127
|
+
## 1.0.6 (2013-05-19)
|
128
|
+
|
129
|
+
* bugfix : quoted fields are now correctly parsed
|
130
|
+
|
131
|
+
## 1.0.5 (2013-05-08)
|
132
|
+
|
133
|
+
* bugfix : for :headers_in_file option
|
134
|
+
|
135
|
+
## 1.0.4 (2012-08-17)
|
136
|
+
|
137
|
+
* renamed the following options:
|
138
|
+
* :strip_whitepace_from_values => :strip_whitespace - removes leading/trailing whitespace from headers and values
|
139
|
+
|
140
|
+
## 1.0.3 (2012-08-16)
|
141
|
+
|
142
|
+
* added the following options:
|
143
|
+
* :strip_whitepace_from_values - removes leading/trailing whitespace from values
|
144
|
+
|
145
|
+
## 1.0.2 (2012-08-02)
|
146
|
+
|
147
|
+
* added more options for dealing with headers:
|
148
|
+
* :user_provided_headers ,user provided Array with header strings or symbols, to precisely define what the headers should be, overriding any in-file headers (default: nil)
|
149
|
+
* :headers_in_file , if the file contains headers as the first line (default: true)
|
150
|
+
|
151
|
+
## 1.0.1 (2012-07-30)
|
152
|
+
|
153
|
+
* added the following options:
|
154
|
+
* :downcase_header
|
155
|
+
* :strings_as_keys
|
156
|
+
* :remove_zero_values
|
157
|
+
* :remove_values_matching
|
158
|
+
* :remove_empty_hashes
|
159
|
+
* :convert_values_to_numeric
|
160
|
+
|
161
|
+
* renamed the following options:
|
162
|
+
* :remove_empty_fields => :remove_empty_values
|
163
|
+
|
164
|
+
|
165
|
+
## 1.0.0 (2012-07-29)
|
166
|
+
|
167
|
+
* renamed `SmarterCSV.process_csv` to `SmarterCSV.process`.
|
168
|
+
|
169
|
+
## 1.0.0.pre1 (2012-07-29)
|
data/CONTRIBUTORS.md
ADDED
@@ -0,0 +1,45 @@
|
|
1
|
+
# A Big Thank You to all the Contributors!!
|
2
|
+
|
3
|
+
|
4
|
+
A Big Thank you to everyone who filed issues, sent comments, and who contributed with pull requests:
|
5
|
+
|
6
|
+
* [Jack 0](https://github.com/xjlin0)
|
7
|
+
* [Alejandro](https://github.com/agaviria)
|
8
|
+
* [Lucas Camargo de Almeida](https://github.com/lcalmeida)
|
9
|
+
* [Raphaël Bleuse](https://github.com/bleuse)
|
10
|
+
* [feens](https://github.com/feens)
|
11
|
+
* [César Camacho](https://github.com/chanko)
|
12
|
+
* [innhyu](https://github.com/innhyu)
|
13
|
+
* [Benjamin Thouret](https://github.com/benichu)
|
14
|
+
* [Chris Hilton](https://github.com/chrismhilton)
|
15
|
+
* [Sean Duckett](http://github.com/sduckett)
|
16
|
+
* [Alex Ong](http://github.com/khaong)
|
17
|
+
* [Martin Nilsson](http://github.com/MrTin)
|
18
|
+
* [Eustáquio Rangel](http://github.com/taq)
|
19
|
+
* [Pavel](http://github.com/paxa)
|
20
|
+
* [Félix Bellanger](https://github.com/Keeguon)
|
21
|
+
* [Graham Wetzler](https://github.com/grahamwetzler)
|
22
|
+
* [Marcos G. Zimmermann](https://github.com/marcosgz)
|
23
|
+
* [Jordan Running](https://github.com/jrunning)
|
24
|
+
* [Dave Sanders](https://github.com/DaveSanders)
|
25
|
+
* [Hugo Lepetit](https://github.com/giglemad)
|
26
|
+
* [esBeee](https://github.com/esBeee)
|
27
|
+
* [Waldyr de Souza](https://github.com/waldyr)
|
28
|
+
* [Ben Maher](https://github.com/benmaher)
|
29
|
+
* [Wal McConnell](https://github.com/wal)
|
30
|
+
* [Jordan Graft](https://github.com/jordangraft)
|
31
|
+
* [Michael](https://github.com/polycarpou)
|
32
|
+
* [Kevin Coleman](https://github.com/KevinColemanInc)
|
33
|
+
* [Tirdad C.](https://github.com/tridadc)
|
34
|
+
* [Dave Myron](https://github.com/contentfree)
|
35
|
+
* [Ivan Ushakov](https://github.com/IvanUshakov)
|
36
|
+
* [Matthieu Paret](https://github.com/mtparet)
|
37
|
+
* [Rohit Amarnath](https://github.com/ramarnat)
|
38
|
+
* [Joshua Smith](https://github.com/enviable)
|
39
|
+
* [Colin Petruno](https://github.com/colinpetruno)
|
40
|
+
* [Diego Salido](https://github.com/salidux)
|
41
|
+
* [Elie](https://github.com/elieteyssedou)
|
42
|
+
* [Chris Wong](https://github.com/lightwave)
|
43
|
+
* [Olle Jonsson](https://github.com/olleolleolle)
|
44
|
+
* [Nicolas Guillemain](https://github.com/Viiruus)
|
45
|
+
* [Sp6](https://github.com/sp6)
|
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2012..2022 Tilo Sloboda
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|