csvlint 0.3.3 → 0.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/ISSUE_TEMPLATE.md +24 -0
- data/.github/PULL_REQUEST_TEMPLATE.md +7 -0
- data/.ruby-version +1 -1
- data/.travis.yml +5 -4
- data/CHANGELOG.md +47 -130
- data/CODE_OF_CONDUCT.md +74 -0
- data/CONTRIBUTING.md +32 -0
- data/Gemfile +0 -3
- data/README.md +50 -46
- data/csvlint.gemspec +5 -3
- data/features/fixtures/white space in filename.csv +0 -0
- data/lib/csvlint/cli.rb +9 -6
- data/lib/csvlint/schema.rb +15 -3
- data/lib/csvlint/validate.rb +1 -1
- data/lib/csvlint/version.rb +1 -1
- data/spec/validator_spec.rb +26 -8
- metadata +20 -14
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5d5ee384ce684c04b09f786edab7aaba1598f651
|
4
|
+
data.tar.gz: 7e473e918dea5c6578c5f8d5870633c006ee7c80
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e2dbc5910aca216330618cf5095950a8b227b412ccc7f6b997fa401b2e02d19895c4c18b0a4d2b01cb76c0415089a4793429f10c09a1c7eb4273d4dfa5cff0db
|
7
|
+
data.tar.gz: dd5bc5ff3d8b31b52a342bbba7d9b8984dca267cfa618b514c1c3995139190e275e5c8ee6f5ac0b965910458a49891a13bf60fd46d4c329a99e4fc51d75d28ed
|
@@ -0,0 +1,24 @@
|
|
1
|
+
> Please provide a general summary of the issue in the Issue Title above
|
2
|
+
> fill out the headings below as applicable to the issue you are reporting,
|
3
|
+
> deleting as appropriate but offering us as much detail as you can to help us resolve the issue
|
4
|
+
|
5
|
+
### Expected Behaviour
|
6
|
+
> What should happen?
|
7
|
+
|
8
|
+
### Desired Behaviour (for improvement suggestions only)
|
9
|
+
> if relevant include images or hyperlinks to other resources that clarify the enhancement you're seeking
|
10
|
+
|
11
|
+
### Current Behaviour (for problems)
|
12
|
+
> What currently happens that isn't expected behaviour?
|
13
|
+
|
14
|
+
### Steps to Reproduce (for problems)
|
15
|
+
> Provide a link to a live example, or an unambiguous set of steps to reproduce this bug. Include code to reproduce, if relevant
|
16
|
+
1.
|
17
|
+
2.
|
18
|
+
3.
|
19
|
+
4.
|
20
|
+
|
21
|
+
### Your Environment
|
22
|
+
> Include as many relevant details about the environment you experienced the bug in - this will help us resolve the bug more expediently
|
23
|
+
* Environment name and version (e.g. Chrome 39, node.js 5.4):
|
24
|
+
* Operating System and version (desktop or mobile):
|
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
2.1
|
1
|
+
2.4.1
|
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,21 +1,63 @@
|
|
1
1
|
# Change Log
|
2
2
|
|
3
|
-
## [0.
|
3
|
+
## [0.4.0](https://github.com/theodi/csvlint.rb/tree/0.4.0) (2017-xx-xx)
|
4
|
+
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.3.3...0.4.0)
|
5
|
+
|
6
|
+
- Support for Ruby 2.4
|
7
|
+
- Ruby 2.4 improves detections of unclosed quotes
|
8
|
+
- Support Rails ~> 5.0
|
9
|
+
- Added `--werror` flag to command line, to treat warnings as errors
|
10
|
+
- Deprecated `Schema#load_from_json` and replaced with `Schema#load_from_uri`. Method will be removed in 1.0.0.
|
11
|
+
- Added `Schema#load_from_string` to load from a string instead of reading a URI
|
12
|
+
|
13
|
+
**Closed issues:**
|
14
|
+
|
15
|
+
- CLI doesn't handle filenames with spaces [\#182](https://github.com/theodi/csvlint.rb/issues/182)
|
16
|
+
|
17
|
+
## [0.3.3](https://github.com/theodi/csvlint.rb/tree/0.3.3) (2016-11-10)
|
18
|
+
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.3.2...0.3.3)
|
19
|
+
|
20
|
+
**Closed issues:**
|
21
|
+
|
22
|
+
- testing issue alerts, sorry [\#186](https://github.com/theodi/csvlint.rb/issues/186)
|
23
|
+
|
24
|
+
**Merged pull requests:**
|
25
|
+
|
26
|
+
- Add row + col to foreign key & duplicate key errors [\#188](https://github.com/theodi/csvlint.rb/pull/188) ([nickzoic](https://github.com/nickzoic))
|
27
|
+
- Trap-and-bin this [\#185](https://github.com/theodi/csvlint.rb/pull/185) ([pikesley](https://github.com/pikesley))
|
28
|
+
- csvw: common property names can be URLs [\#181](https://github.com/theodi/csvlint.rb/pull/181) ([JeniT](https://github.com/JeniT))
|
29
|
+
- force UTF-8 if encoding is ASCII-8BIT [\#180](https://github.com/theodi/csvlint.rb/pull/180) ([JeniT](https://github.com/JeniT))
|
30
|
+
|
31
|
+
## [0.3.2](https://github.com/theodi/csvlint.rb/tree/0.3.2) (2016-05-24)
|
32
|
+
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.3.1...0.3.2)
|
33
|
+
|
34
|
+
**Merged pull requests:**
|
35
|
+
|
36
|
+
- Add schema errors to cli json [\#184](https://github.com/theodi/csvlint.rb/pull/184) ([pezholio](https://github.com/pezholio))
|
37
|
+
|
38
|
+
## [0.3.1](https://github.com/theodi/csvlint.rb/tree/0.3.1) (2016-05-23)
|
39
|
+
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.3.0...0.3.1)
|
40
|
+
|
41
|
+
**Closed issues:**
|
4
42
|
|
43
|
+
- Error installing on Windows because of \*escape\_utils\* dependency [\#175](https://github.com/theodi/csvlint.rb/issues/175)
|
44
|
+
|
45
|
+
**Merged pull requests:**
|
46
|
+
|
47
|
+
- Add CLI option to output JSON [\#183](https://github.com/theodi/csvlint.rb/pull/183) ([pezholio](https://github.com/pezholio))
|
48
|
+
|
49
|
+
## [0.3.0](https://github.com/theodi/csvlint.rb/tree/0.3.0) (2016-01-12)
|
5
50
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.2.6...0.3.0)
|
6
51
|
|
7
52
|
**Merged pull requests:**
|
8
53
|
|
9
54
|
- still increment current\_line after invalid\_encoding error [\#174](https://github.com/theodi/csvlint.rb/pull/174) ([wjordan213](https://github.com/wjordan213))
|
10
|
-
|
11
55
|
- Support for CSV on the Web transformations [\#173](https://github.com/theodi/csvlint.rb/pull/173) ([JeniT](https://github.com/JeniT))
|
12
56
|
|
13
57
|
## [0.2.6](https://github.com/theodi/csvlint.rb/tree/0.2.6) (2015-11-16)
|
14
|
-
|
15
58
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.2.5...0.2.6)
|
16
59
|
|
17
60
|
## [0.2.5](https://github.com/theodi/csvlint.rb/tree/0.2.5) (2015-11-16)
|
18
|
-
|
19
61
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.2.4...0.2.5)
|
20
62
|
|
21
63
|
**Merged pull requests:**
|
@@ -23,7 +65,6 @@
|
|
23
65
|
- Use STDIN instead of ARGF [\#169](https://github.com/theodi/csvlint.rb/pull/169) ([pezholio](https://github.com/pezholio))
|
24
66
|
|
25
67
|
## [0.2.4](https://github.com/theodi/csvlint.rb/tree/0.2.4) (2015-10-20)
|
26
|
-
|
27
68
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.2.3...0.2.4)
|
28
69
|
|
29
70
|
**Merged pull requests:**
|
@@ -31,23 +72,19 @@
|
|
31
72
|
- Fixes for CLI [\#164](https://github.com/theodi/csvlint.rb/pull/164) ([pezholio](https://github.com/pezholio))
|
32
73
|
|
33
74
|
## [0.2.3](https://github.com/theodi/csvlint.rb/tree/0.2.3) (2015-10-20)
|
34
|
-
|
35
75
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.2.2...0.2.3)
|
36
76
|
|
37
77
|
**Closed issues:**
|
38
78
|
|
39
79
|
- Include field name with error [\#161](https://github.com/theodi/csvlint.rb/issues/161)
|
40
|
-
|
41
80
|
- Refactor the binary [\#150](https://github.com/theodi/csvlint.rb/issues/150)
|
42
81
|
|
43
82
|
**Merged pull requests:**
|
44
83
|
|
45
84
|
- Refactor CLI [\#163](https://github.com/theodi/csvlint.rb/pull/163) ([pezholio](https://github.com/pezholio))
|
46
|
-
|
47
85
|
- Update schema file example to clarify type [\#162](https://github.com/theodi/csvlint.rb/pull/162) ([wachunga](https://github.com/wachunga))
|
48
86
|
|
49
87
|
## [0.2.2](https://github.com/theodi/csvlint.rb/tree/0.2.2) (2015-10-09)
|
50
|
-
|
51
88
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.2.1...0.2.2)
|
52
89
|
|
53
90
|
**Closed issues:**
|
@@ -57,17 +94,12 @@
|
|
57
94
|
**Merged pull requests:**
|
58
95
|
|
59
96
|
- Check characters in validate\_line method [\#160](https://github.com/theodi/csvlint.rb/pull/160) ([pezholio](https://github.com/pezholio))
|
60
|
-
|
61
97
|
- Further optimisations [\#159](https://github.com/theodi/csvlint.rb/pull/159) ([pezholio](https://github.com/pezholio))
|
62
|
-
|
63
98
|
- More optimizations after \#157 [\#158](https://github.com/theodi/csvlint.rb/pull/158) ([jpmckinney](https://github.com/jpmckinney))
|
64
|
-
|
65
99
|
- Memoize the result of CSV\#encode\_re [\#157](https://github.com/theodi/csvlint.rb/pull/157) ([jpmckinney](https://github.com/jpmckinney))
|
66
|
-
|
67
100
|
- Don't pass leading string to parse\_line [\#155](https://github.com/theodi/csvlint.rb/pull/155) ([pezholio](https://github.com/pezholio))
|
68
101
|
|
69
102
|
## [0.2.1](https://github.com/theodi/csvlint.rb/tree/0.2.1) (2015-10-07)
|
70
|
-
|
71
103
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.2.0...0.2.1)
|
72
104
|
|
73
105
|
**Implemented enhancements:**
|
@@ -81,11 +113,9 @@
|
|
81
113
|
**Merged pull requests:**
|
82
114
|
|
83
115
|
- Add `row\_count` method [\#153](https://github.com/theodi/csvlint.rb/pull/153) ([pezholio](https://github.com/pezholio))
|
84
|
-
|
85
116
|
- Streaming validation [\#146](https://github.com/theodi/csvlint.rb/pull/146) ([pezholio](https://github.com/pezholio))
|
86
117
|
|
87
118
|
## [0.2.0](https://github.com/theodi/csvlint.rb/tree/0.2.0) (2015-10-05)
|
88
|
-
|
89
119
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.1.4...0.2.0)
|
90
120
|
|
91
121
|
**Closed issues:**
|
@@ -95,15 +125,11 @@
|
|
95
125
|
**Merged pull requests:**
|
96
126
|
|
97
127
|
- Recover from `ArgumentError`s when attempting to locate a schema and detect bad schema when JSON is malformed [\#152](https://github.com/theodi/csvlint.rb/pull/152) ([pezholio](https://github.com/pezholio))
|
98
|
-
|
99
128
|
- Catch errors if link headers are don't have particular values [\#151](https://github.com/theodi/csvlint.rb/pull/151) ([pezholio](https://github.com/pezholio))
|
100
|
-
|
101
129
|
- Rescue excel warning [\#149](https://github.com/theodi/csvlint.rb/pull/149) ([quadrophobiac](https://github.com/quadrophobiac))
|
102
|
-
|
103
130
|
- CSVW-based validation! [\#142](https://github.com/theodi/csvlint.rb/pull/142) ([JeniT](https://github.com/JeniT))
|
104
131
|
|
105
132
|
## [0.1.4](https://github.com/theodi/csvlint.rb/tree/0.1.4) (2015-08-06)
|
106
|
-
|
107
133
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.1.3...0.1.4)
|
108
134
|
|
109
135
|
**Merged pull requests:**
|
@@ -111,19 +137,15 @@
|
|
111
137
|
- change made to the constraint parameter in order that it is more cons… [\#140](https://github.com/theodi/csvlint.rb/pull/140) ([quadrophobiac](https://github.com/quadrophobiac))
|
112
138
|
|
113
139
|
## [0.1.3](https://github.com/theodi/csvlint.rb/tree/0.1.3) (2015-07-24)
|
114
|
-
|
115
140
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.1.2...0.1.3)
|
116
141
|
|
117
142
|
**Merged pull requests:**
|
118
143
|
|
119
144
|
- Error reporting schema expanded test suite [\#138](https://github.com/theodi/csvlint.rb/pull/138) ([quadrophobiac](https://github.com/quadrophobiac))
|
120
|
-
|
121
145
|
- Validate header size improvement [\#137](https://github.com/theodi/csvlint.rb/pull/137) ([adamc00](https://github.com/adamc00))
|
122
|
-
|
123
146
|
- Invalid schema [\#132](https://github.com/theodi/csvlint.rb/pull/132) ([bcouston](https://github.com/bcouston))
|
124
147
|
|
125
148
|
## [0.1.2](https://github.com/theodi/csvlint.rb/tree/0.1.2) (2015-07-15)
|
126
|
-
|
127
149
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.1.1...0.1.2)
|
128
150
|
|
129
151
|
**Closed issues:**
|
@@ -133,243 +155,138 @@
|
|
133
155
|
**Merged pull requests:**
|
134
156
|
|
135
157
|
- Catch invalid URIs [\#133](https://github.com/theodi/csvlint.rb/pull/133) ([pezholio](https://github.com/pezholio))
|
136
|
-
|
137
158
|
- Emit a warning when the CSV header does not match the supplied schema [\#127](https://github.com/theodi/csvlint.rb/pull/127) ([adamc00](https://github.com/adamc00))
|
138
159
|
|
139
160
|
## [0.1.1](https://github.com/theodi/csvlint.rb/tree/0.1.1) (2015-07-13)
|
140
|
-
|
141
161
|
[Full Changelog](https://github.com/theodi/csvlint.rb/compare/0.1.0...0.1.1)
|
142
162
|
|
143
163
|
**Closed issues:**
|
144
164
|
|
145
165
|
- Add Command Line Support [\#128](https://github.com/theodi/csvlint.rb/issues/128)
|
146
|
-
|
147
166
|
- BUG: Incorrect inconsistent\_values error on numeric columns [\#106](https://github.com/theodi/csvlint.rb/issues/106)
|
148
167
|
|
149
168
|
**Merged pull requests:**
|
150
169
|
|
151
170
|
- Fixes line content incorrectly being put into the row column field when there is an encoding error. [\#130](https://github.com/theodi/csvlint.rb/pull/130) ([glacier](https://github.com/glacier))
|
152
|
-
|
153
171
|
- Add command line help [\#129](https://github.com/theodi/csvlint.rb/pull/129) ([pezholio](https://github.com/pezholio))
|
154
|
-
|
155
172
|
- Remove stray q character. [\#125](https://github.com/theodi/csvlint.rb/pull/125) ([adamc00](https://github.com/adamc00))
|
156
|
-
|
157
173
|
- csvlint utility can take arguments to specify a schema and pp errors [\#124](https://github.com/theodi/csvlint.rb/pull/124) ([adamc00](https://github.com/adamc00))
|
158
|
-
|
159
174
|
- Fixed warning - use expect\( \) rather than .should [\#123](https://github.com/theodi/csvlint.rb/pull/123) ([jezhiggins](https://github.com/jezhiggins))
|
160
|
-
|
161
175
|
- Fixed spelling mistake [\#121](https://github.com/theodi/csvlint.rb/pull/121) ([jezhiggins](https://github.com/jezhiggins))
|
162
|
-
|
163
176
|
- Avoid using \#blank? if unnecessary [\#120](https://github.com/theodi/csvlint.rb/pull/120) ([jpmckinney](https://github.com/jpmckinney))
|
164
|
-
|
165
177
|
- eliminate some date and time formats, related \#105 [\#119](https://github.com/theodi/csvlint.rb/pull/119) ([jpmckinney](https://github.com/jpmckinney))
|
166
|
-
|
167
178
|
- Match another CSV error about line endings [\#118](https://github.com/theodi/csvlint.rb/pull/118) ([jpmckinney](https://github.com/jpmckinney))
|
168
|
-
|
169
179
|
- fixed typo mistake in README [\#117](https://github.com/theodi/csvlint.rb/pull/117) ([railsfactory-kumaresan](https://github.com/railsfactory-kumaresan))
|
170
|
-
|
171
180
|
- Integrate @jpmickinney's build\_formats improvements [\#112](https://github.com/theodi/csvlint.rb/pull/112) ([Floppy](https://github.com/Floppy))
|
172
|
-
|
173
181
|
- make limit\_lines into a non-dialect option [\#110](https://github.com/theodi/csvlint.rb/pull/110) ([Floppy](https://github.com/Floppy))
|
174
|
-
|
175
182
|
- fix coveralls stats [\#109](https://github.com/theodi/csvlint.rb/pull/109) ([Floppy](https://github.com/Floppy))
|
176
|
-
|
177
183
|
- Limit lines [\#101](https://github.com/theodi/csvlint.rb/pull/101) ([Hoedic](https://github.com/Hoedic))
|
178
184
|
|
179
185
|
## [0.1.0](https://github.com/theodi/csvlint.rb/tree/0.1.0) (2014-11-27)
|
180
|
-
|
181
186
|
**Implemented enhancements:**
|
182
187
|
|
183
188
|
- Blank values shouldn't count as inconsistencies [\#90](https://github.com/theodi/csvlint.rb/issues/90)
|
184
|
-
|
185
189
|
- Make sure we don't check schema column count and ragged row count together [\#66](https://github.com/theodi/csvlint.rb/issues/66)
|
186
|
-
|
187
190
|
- Include the failed constraints in error message when doing field validation [\#64](https://github.com/theodi/csvlint.rb/issues/64)
|
188
|
-
|
189
191
|
- Include the column value in error message when field validation fails [\#63](https://github.com/theodi/csvlint.rb/issues/63)
|
190
|
-
|
191
192
|
- Expose optional JSON table schema fields [\#55](https://github.com/theodi/csvlint.rb/issues/55)
|
192
|
-
|
193
193
|
- Ensure header rows are properly handled and validated [\#48](https://github.com/theodi/csvlint.rb/issues/48)
|
194
|
-
|
195
194
|
- Support zipped CSV? [\#30](https://github.com/theodi/csvlint.rb/issues/30)
|
196
|
-
|
197
195
|
- Improve feedback on inconsistent values [\#29](https://github.com/theodi/csvlint.rb/issues/29)
|
198
|
-
|
199
196
|
- Reported error positions are not massively useful [\#15](https://github.com/theodi/csvlint.rb/issues/15)
|
200
197
|
|
201
198
|
**Fixed bugs:**
|
202
199
|
|
203
200
|
- undefined method `\[\]' for nil:NilClass from fetch\_error [\#71](https://github.com/theodi/csvlint.rb/issues/71)
|
204
|
-
|
205
201
|
- Inconsistent column bases [\#69](https://github.com/theodi/csvlint.rb/issues/69)
|
206
|
-
|
207
202
|
- Improve error handling in Schema loading [\#42](https://github.com/theodi/csvlint.rb/issues/42)
|
208
|
-
|
209
203
|
- Recover from some line ending problems [\#41](https://github.com/theodi/csvlint.rb/issues/41)
|
210
|
-
|
211
204
|
- Inconsistent values due to number format differences [\#32](https://github.com/theodi/csvlint.rb/issues/32)
|
212
|
-
|
213
205
|
- New lines in quoted fields are valid [\#31](https://github.com/theodi/csvlint.rb/issues/31)
|
214
|
-
|
215
206
|
- Wrongly reporting incorrect file extension [\#23](https://github.com/theodi/csvlint.rb/issues/23)
|
216
|
-
|
217
207
|
- Incorrect extension reported when URL has query options at the end [\#14](https://github.com/theodi/csvlint.rb/issues/14)
|
218
208
|
|
219
209
|
**Closed issues:**
|
220
210
|
|
221
211
|
- Get gem continuously deploying [\#93](https://github.com/theodi/csvlint.rb/issues/93)
|
222
|
-
|
223
212
|
- Publish on rubygems.org [\#92](https://github.com/theodi/csvlint.rb/issues/92)
|
224
|
-
|
225
213
|
- Duplicate column names [\#87](https://github.com/theodi/csvlint.rb/issues/87)
|
226
|
-
|
227
214
|
- Return code is always 0 \(except when it isn't\) [\#85](https://github.com/theodi/csvlint.rb/issues/85)
|
228
|
-
|
229
215
|
- Can't pipe data to csvlint [\#84](https://github.com/theodi/csvlint.rb/issues/84)
|
230
|
-
|
231
216
|
- They have some validator running if someone wants to inspect it for "inspiration" [\#27](https://github.com/theodi/csvlint.rb/issues/27)
|
232
|
-
|
233
217
|
- Allow CSV parsing options to be configured as a parameter [\#6](https://github.com/theodi/csvlint.rb/issues/6)
|
234
|
-
|
235
218
|
- Use explicit CSV parsing options [\#5](https://github.com/theodi/csvlint.rb/issues/5)
|
236
|
-
|
237
219
|
- Improving encoding detection [\#2](https://github.com/theodi/csvlint.rb/issues/2)
|
238
220
|
|
239
221
|
**Merged pull requests:**
|
240
222
|
|
241
223
|
- Speed up \#build\_formats \(changes its API\) [\#103](https://github.com/theodi/csvlint.rb/pull/103) ([jpmckinney](https://github.com/jpmckinney))
|
242
|
-
|
243
224
|
- Continuously deploy gem [\#102](https://github.com/theodi/csvlint.rb/pull/102) ([pezholio](https://github.com/pezholio))
|
244
|
-
|
245
225
|
- Make csvlint way faster [\#99](https://github.com/theodi/csvlint.rb/pull/99) ([jpmckinney](https://github.com/jpmckinney))
|
246
|
-
|
247
226
|
- Update README.md [\#98](https://github.com/theodi/csvlint.rb/pull/98) ([rmalecky](https://github.com/rmalecky))
|
248
|
-
|
249
227
|
- Undeclared header error [\#95](https://github.com/theodi/csvlint.rb/pull/95) ([Floppy](https://github.com/Floppy))
|
250
|
-
|
251
228
|
- Blank values shouldn't count as inconsistencies [\#91](https://github.com/theodi/csvlint.rb/pull/91) ([pezholio](https://github.com/pezholio))
|
252
|
-
|
253
229
|
- Use `reject` instead of `delete\_if` [\#89](https://github.com/theodi/csvlint.rb/pull/89) ([pezholio](https://github.com/pezholio))
|
254
|
-
|
255
230
|
- Raise a warning if a title row is found [\#88](https://github.com/theodi/csvlint.rb/pull/88) ([pezholio](https://github.com/pezholio))
|
256
|
-
|
257
231
|
- Improve executable [\#86](https://github.com/theodi/csvlint.rb/pull/86) ([pezholio](https://github.com/pezholio))
|
258
|
-
|
259
232
|
- Feature undeclared header [\#83](https://github.com/theodi/csvlint.rb/pull/83) ([ldodds](https://github.com/ldodds))
|
260
|
-
|
261
233
|
- Support xsd:integer [\#82](https://github.com/theodi/csvlint.rb/pull/82) ([ldodds](https://github.com/ldodds))
|
262
|
-
|
263
234
|
- Downgrade header errors [\#81](https://github.com/theodi/csvlint.rb/pull/81) ([ldodds](https://github.com/ldodds))
|
264
|
-
|
265
235
|
- Go home, pry [\#78](https://github.com/theodi/csvlint.rb/pull/78) ([pikesley](https://github.com/pikesley))
|
266
|
-
|
267
236
|
- Use type validations to check consistency [\#77](https://github.com/theodi/csvlint.rb/pull/77) ([pezholio](https://github.com/pezholio))
|
268
|
-
|
269
237
|
- Add data accessor [\#76](https://github.com/theodi/csvlint.rb/pull/76) ([Floppy](https://github.com/Floppy))
|
270
|
-
|
271
238
|
- Add failed constraints to schema errors [\#75](https://github.com/theodi/csvlint.rb/pull/75) ([ldodds](https://github.com/ldodds))
|
272
|
-
|
273
239
|
- Only perform ragged row check if there's no schema [\#74](https://github.com/theodi/csvlint.rb/pull/74) ([ldodds](https://github.com/ldodds))
|
274
|
-
|
275
240
|
- Handle tempfiles [\#73](https://github.com/theodi/csvlint.rb/pull/73) ([pezholio](https://github.com/pezholio))
|
276
|
-
|
277
241
|
- Catch errors if regex doesn't match [\#72](https://github.com/theodi/csvlint.rb/pull/72) ([pezholio](https://github.com/pezholio))
|
278
|
-
|
279
242
|
- Inconsistent column base [\#70](https://github.com/theodi/csvlint.rb/pull/70) ([ldodds](https://github.com/ldodds))
|
280
|
-
|
281
243
|
- include column name in :header\_name message [\#68](https://github.com/theodi/csvlint.rb/pull/68) ([Floppy](https://github.com/Floppy))
|
282
|
-
|
283
244
|
- Record default dialect [\#67](https://github.com/theodi/csvlint.rb/pull/67) ([pezholio](https://github.com/pezholio))
|
284
|
-
|
285
245
|
- Schema validation message improvements [\#65](https://github.com/theodi/csvlint.rb/pull/65) ([Floppy](https://github.com/Floppy))
|
286
|
-
|
287
246
|
- Fix ignore empty fields [\#62](https://github.com/theodi/csvlint.rb/pull/62) ([ldodds](https://github.com/ldodds))
|
288
|
-
|
289
247
|
- Create stub schema from existing CSV file [\#61](https://github.com/theodi/csvlint.rb/pull/61) ([ldodds](https://github.com/ldodds))
|
290
|
-
|
291
248
|
- Validate dates [\#59](https://github.com/theodi/csvlint.rb/pull/59) ([ldodds](https://github.com/ldodds))
|
292
|
-
|
293
249
|
- add schema access from validator [\#58](https://github.com/theodi/csvlint.rb/pull/58) ([Floppy](https://github.com/Floppy))
|
294
|
-
|
295
250
|
- Allow schema and fields to have title and description [\#57](https://github.com/theodi/csvlint.rb/pull/57) ([ldodds](https://github.com/ldodds))
|
296
|
-
|
297
251
|
- Feature min max ranges [\#56](https://github.com/theodi/csvlint.rb/pull/56) ([ldodds](https://github.com/ldodds))
|
298
|
-
|
299
252
|
- Check header without schema [\#54](https://github.com/theodi/csvlint.rb/pull/54) ([ldodds](https://github.com/ldodds))
|
300
|
-
|
301
253
|
- Validate types [\#53](https://github.com/theodi/csvlint.rb/pull/53) ([pikesley](https://github.com/pikesley))
|
302
|
-
|
303
254
|
- Added open\_uri\_redirections to allow HTTP/HTTPS transfers [\#52](https://github.com/theodi/csvlint.rb/pull/52) ([ldodds](https://github.com/ldodds))
|
304
|
-
|
305
255
|
- Added docs on CSV options and header error/warning messages [\#51](https://github.com/theodi/csvlint.rb/pull/51) ([ldodds](https://github.com/ldodds))
|
306
|
-
|
307
256
|
- Feature header validation [\#50](https://github.com/theodi/csvlint.rb/pull/50) ([ldodds](https://github.com/ldodds))
|
308
|
-
|
309
257
|
- Handle unique columns [\#49](https://github.com/theodi/csvlint.rb/pull/49) ([pikesley](https://github.com/pikesley))
|
310
|
-
|
311
258
|
- Validate all the fields [\#47](https://github.com/theodi/csvlint.rb/pull/47) ([ldodds](https://github.com/ldodds))
|
312
|
-
|
313
259
|
- Tolerate incomplete schemas [\#46](https://github.com/theodi/csvlint.rb/pull/46) ([ldodds](https://github.com/ldodds))
|
314
|
-
|
315
260
|
- Add accessor for line breaks [\#45](https://github.com/theodi/csvlint.rb/pull/45) ([Floppy](https://github.com/Floppy))
|
316
|
-
|
317
261
|
- update README for info messages and new error types [\#44](https://github.com/theodi/csvlint.rb/pull/44) ([Floppy](https://github.com/Floppy))
|
318
|
-
|
319
262
|
- Info messages for line breaks [\#43](https://github.com/theodi/csvlint.rb/pull/43) ([Floppy](https://github.com/Floppy))
|
320
|
-
|
321
263
|
- Add category to messages [\#40](https://github.com/theodi/csvlint.rb/pull/40) ([ldodds](https://github.com/ldodds))
|
322
|
-
|
323
264
|
- Badges [\#39](https://github.com/theodi/csvlint.rb/pull/39) ([pikesley](https://github.com/pikesley))
|
324
|
-
|
325
265
|
- Generic field validation using JSON Table Schema [\#38](https://github.com/theodi/csvlint.rb/pull/38) ([ldodds](https://github.com/ldodds))
|
326
|
-
|
327
266
|
- Feature validate strings and files [\#37](https://github.com/theodi/csvlint.rb/pull/37) ([ldodds](https://github.com/ldodds))
|
328
|
-
|
329
267
|
- Support reporting of column number in errors [\#36](https://github.com/theodi/csvlint.rb/pull/36) ([ldodds](https://github.com/ldodds))
|
330
|
-
|
331
268
|
- Fix up casing of keys in CSV DDF options [\#35](https://github.com/theodi/csvlint.rb/pull/35) ([ldodds](https://github.com/ldodds))
|
332
|
-
|
333
269
|
- Add errors for incorrect newlines [\#34](https://github.com/theodi/csvlint.rb/pull/34) ([pezholio](https://github.com/pezholio))
|
334
|
-
|
335
270
|
- Change from parsing CSV line by line to using CSV.new and trapping errors [\#33](https://github.com/theodi/csvlint.rb/pull/33) ([ldodds](https://github.com/ldodds))
|
336
|
-
|
337
271
|
- Improved the README, tweaked LICENSE [\#28](https://github.com/theodi/csvlint.rb/pull/28) ([ldodds](https://github.com/ldodds))
|
338
|
-
|
339
272
|
- Handle 404s [\#26](https://github.com/theodi/csvlint.rb/pull/26) ([pezholio](https://github.com/pezholio))
|
340
|
-
|
341
273
|
- Create more fine-grained errors and warnings for content type issues [\#25](https://github.com/theodi/csvlint.rb/pull/25) ([ldodds](https://github.com/ldodds))
|
342
|
-
|
343
274
|
- Report trailing empty rows as an error. Previously threw exception [\#24](https://github.com/theodi/csvlint.rb/pull/24) ([ldodds](https://github.com/ldodds))
|
344
|
-
|
345
275
|
- Simplify the guessing of column types [\#22](https://github.com/theodi/csvlint.rb/pull/22) ([ldodds](https://github.com/ldodds))
|
346
|
-
|
347
276
|
- Class-ify error messages [\#21](https://github.com/theodi/csvlint.rb/pull/21) ([pezholio](https://github.com/pezholio))
|
348
|
-
|
349
277
|
- Error extracts [\#20](https://github.com/theodi/csvlint.rb/pull/20) ([Floppy](https://github.com/Floppy))
|
350
|
-
|
351
278
|
- Return headers [\#19](https://github.com/theodi/csvlint.rb/pull/19) ([pezholio](https://github.com/pezholio))
|
352
|
-
|
353
279
|
- Return a warning if no character set specified [\#18](https://github.com/theodi/csvlint.rb/pull/18) ([pezholio](https://github.com/pezholio))
|
354
|
-
|
355
280
|
- Ignore query params [\#17](https://github.com/theodi/csvlint.rb/pull/17) ([Floppy](https://github.com/Floppy))
|
356
|
-
|
357
281
|
- Add invalid\_encoding error for invalid byte sequences [\#16](https://github.com/theodi/csvlint.rb/pull/16) ([ldodds](https://github.com/ldodds))
|
358
|
-
|
359
282
|
- Check inconsistent values [\#13](https://github.com/theodi/csvlint.rb/pull/13) ([pezholio](https://github.com/pezholio))
|
360
|
-
|
361
283
|
- Add CSV dialect options [\#11](https://github.com/theodi/csvlint.rb/pull/11) ([pezholio](https://github.com/pezholio))
|
362
|
-
|
363
284
|
- Return warning if extension doesn't match content type [\#10](https://github.com/theodi/csvlint.rb/pull/10) ([pezholio](https://github.com/pezholio))
|
364
|
-
|
365
285
|
- Return warnings for file extension [\#8](https://github.com/theodi/csvlint.rb/pull/8) ([pezholio](https://github.com/pezholio))
|
366
|
-
|
367
286
|
- Detect blank rows [\#7](https://github.com/theodi/csvlint.rb/pull/7) ([pezholio](https://github.com/pezholio))
|
368
|
-
|
369
287
|
- Detect bad content type [\#3](https://github.com/theodi/csvlint.rb/pull/3) ([pezholio](https://github.com/pezholio))
|
370
|
-
|
371
288
|
- Return information about CSV [\#1](https://github.com/theodi/csvlint.rb/pull/1) ([pezholio](https://github.com/pezholio))
|
372
289
|
|
373
290
|
|
374
291
|
|
375
|
-
\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)*
|
292
|
+
\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)*
|
data/CODE_OF_CONDUCT.md
ADDED
@@ -0,0 +1,74 @@
|
|
1
|
+
## Code of Conduct
|
2
|
+
|
3
|
+
### Our Pledge
|
4
|
+
|
5
|
+
In the interest of fostering an open and welcoming environment, we as
|
6
|
+
contributors and maintainers pledge to making participation in our project and
|
7
|
+
our community a harassment-free experience for everyone, regardless of age, body
|
8
|
+
size, disability, ethnicity, gender identity and expression, level of experience,
|
9
|
+
nationality, personal appearance, race, religion, or sexual identity and
|
10
|
+
orientation.
|
11
|
+
|
12
|
+
### Our Standards
|
13
|
+
|
14
|
+
Examples of behavior that contributes to creating a positive environment
|
15
|
+
include:
|
16
|
+
|
17
|
+
* Using welcoming and inclusive language
|
18
|
+
* Being respectful of differing viewpoints and experiences
|
19
|
+
* Gracefully accepting constructive criticism
|
20
|
+
* Focusing on what is best for the community
|
21
|
+
* Showing empathy towards other community members
|
22
|
+
|
23
|
+
Examples of unacceptable behavior by participants include:
|
24
|
+
|
25
|
+
* The use of sexualized language or imagery and unwelcome sexual attention or
|
26
|
+
advances
|
27
|
+
* Trolling, insulting/derogatory comments, and personal or political attacks
|
28
|
+
* Public or private harassment
|
29
|
+
* Publishing others' private information, such as a physical or electronic
|
30
|
+
address, without explicit permission
|
31
|
+
* Other conduct which could reasonably be considered inappropriate in a
|
32
|
+
professional setting
|
33
|
+
|
34
|
+
### Our Responsibilities
|
35
|
+
|
36
|
+
Project maintainers are responsible for clarifying the standards of acceptable
|
37
|
+
behavior and are expected to take appropriate and fair corrective action in
|
38
|
+
response to any instances of unacceptable behavior.
|
39
|
+
|
40
|
+
Project maintainers have the right and responsibility to remove, edit, or
|
41
|
+
reject comments, commits, code, wiki edits, issues, and other contributions
|
42
|
+
that are not aligned to this Code of Conduct, or to ban temporarily or
|
43
|
+
permanently any contributor for other behaviors that they deem inappropriate,
|
44
|
+
threatening, offensive, or harmful.
|
45
|
+
|
46
|
+
### Scope
|
47
|
+
|
48
|
+
This Code of Conduct applies both within project spaces and in public spaces
|
49
|
+
when an individual is representing the project or its community. Examples of
|
50
|
+
representing a project or community include using an official project e-mail
|
51
|
+
address, posting via an official social media account, or acting as an appointed
|
52
|
+
representative at an online or offline event. Representation of a project may be
|
53
|
+
further defined and clarified by project maintainers.
|
54
|
+
|
55
|
+
### Enforcement
|
56
|
+
|
57
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
58
|
+
reported by contacting the project team at [labs@theodi.org](mailto:labs@theodi.org). All
|
59
|
+
complaints will be reviewed and investigated and will result in a response that
|
60
|
+
is deemed necessary and appropriate to the circumstances. The project team is
|
61
|
+
obligated to maintain confidentiality with regard to the reporter of an incident.
|
62
|
+
Further details of specific enforcement policies may be posted separately.
|
63
|
+
|
64
|
+
Project maintainers who do not follow or enforce the Code of Conduct in good
|
65
|
+
faith may face temporary or permanent repercussions as determined by other
|
66
|
+
members of the project's leadership.
|
67
|
+
|
68
|
+
### Attribution
|
69
|
+
|
70
|
+
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
|
71
|
+
available at [http://contributor-covenant.org/version/1/4][version]
|
72
|
+
|
73
|
+
[homepage]: http://contributor-covenant.org
|
74
|
+
[version]: http://contributor-covenant.org/version/1/4/
|
data/CONTRIBUTING.md
ADDED
@@ -0,0 +1,32 @@
|
|
1
|
+
# Contributing to CSVlint.rb
|
2
|
+
|
3
|
+
The CSVlint library is open source, and contributions are gratefully accepted!
|
4
|
+
Details on how to contribute are below. By participating in this project, you agree to abide by our [Code of Conduct](https://github.com/theodi/csvlint.rb/blob/CODE_OF_CONDUCT.md).
|
5
|
+
|
6
|
+
Before you start coding, please reach out to us either on our [gitter channel](https://gitter.im/theodi/toolbox) or by tagging a repository administrator on the issue ticket you are interested in contributing towards to indicate your interest in helping.
|
7
|
+
|
8
|
+
If this is your first time contributing to the ODI’s codebase you will need to [create a fork of this repository](https://help.github.com/articles/fork-a-repo/).
|
9
|
+
|
10
|
+
Consult our [Getting Started Guide](https://github.com/theodi/toolbox/wiki/Developers-Guide:-Getting-Started) (if necessary) and then follow the [readme instructions](https://github.com/theodi/csvlint.rb/blob/master/README.md#development) to get your Development environment running locally
|
11
|
+
|
12
|
+
Ensure that the [tests](https://github.com/theodi/csvlint.rb/blob/master/README.md#tests) pass before working on your contribution
|
13
|
+
|
14
|
+
## Code Review Process
|
15
|
+
|
16
|
+
All contributions to the codebase - whether fork or pull request - will be reviewed per the below criteria.
|
17
|
+
To increase your chances of your push being accepted please be aware of the following
|
18
|
+
- Write [well formed commit messages](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html)
|
19
|
+
- Follow our [style guide recommendations](https://github.com/theodi/toolbox/blob/README.md#code-style-guide)
|
20
|
+
- Write tests for all changes (additions or refactors of existing code).
|
21
|
+
- Of the github integrations we use two will be utilised to check appraise your contribution. In order of priority these are
|
22
|
+
- Travis ensures that all tests (existing and additions) pass
|
23
|
+
- Travis/Coveralls ensures that overall test coverage for lines of code meets a certain threshold. If this metric dips below what it previously was for the repository you’re pushing to then your PR will be rejected
|
24
|
+
- Gemnasium ensures dependencies are up to date
|
25
|
+
- Once your PR is published and passes the above checks a repository administrator will review your contribution. Where appropriate comments may be provided and amendments suggested before your PR is merged into Master.
|
26
|
+
- Once your PR is accepted you will be granted push access to the repository you have contributed to! Congratulations on joining our community, you’ll no longer need to work from forks.
|
27
|
+
|
28
|
+
If you make a contribution to another repository in the Toolbox you will be expected to repeat this process. Read more about that [here](https://github.com/theodi/toolbox/blob/master/README.md#push-access).
|
29
|
+
|
30
|
+
## Code Style Guide
|
31
|
+
|
32
|
+
We follow the same code style conventions as detailed in Github’s [Ruby Style Guide](https://github.com/github/rubocop-github/blob/master/STYLEGUIDE.md)
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -6,7 +6,46 @@
|
|
6
6
|
|
7
7
|
# CSV Lint
|
8
8
|
|
9
|
-
A ruby gem to support validating CSV files to check their syntax and contents.
|
9
|
+
A ruby gem to support validating CSV files to check their syntax and contents. You can either use this gem within your own Ruby code, or as a standolone command line application
|
10
|
+
|
11
|
+
## Summary of features
|
12
|
+
|
13
|
+
* Validation that checks the structural formatting of a CSV file
|
14
|
+
* Validation of a delimiter-separated values (dsv) file accesible via URL, File, or an IO-style object (e.g. StringIO)
|
15
|
+
* Validation against [CSV dialects](http://dataprotocols.org/csv-dialect/)
|
16
|
+
* Validation against multiple schema standards; [JSON Table Schema](https://github.com/theodi/csvlint.rb/blob/master/README.md#json-table-schema-support) and [CSV on the Web](https://github.com/theodi/csvlint.rb/blob/master/README.md#csv-on-the-web-validation-support)
|
17
|
+
|
18
|
+
## Development
|
19
|
+
|
20
|
+
`ruby version 2.1.4`
|
21
|
+
|
22
|
+
### Tests
|
23
|
+
|
24
|
+
The codebase includes both rspec and cucumber tests, which can be run together using:
|
25
|
+
|
26
|
+
$ rake
|
27
|
+
|
28
|
+
or separately:
|
29
|
+
|
30
|
+
$ rake spec
|
31
|
+
$ rake features
|
32
|
+
|
33
|
+
When the cucumber tests are first run, a script will create tests based on the latest version of the [CSV on the Web test suite](http://w3c.github.io/csvw/tests/), including creating a local cache of the test files. This requires an internet connection and some patience. Following that download, the tests will run locally; there's also a batch script:
|
34
|
+
|
35
|
+
$ bin/run-csvw-tests
|
36
|
+
|
37
|
+
which will run the tests from the command line.
|
38
|
+
|
39
|
+
If you need to refresh the CSV on the Web tests:
|
40
|
+
|
41
|
+
$ rm bin/run-csvw-tests
|
42
|
+
$ rm features/csvw_validation_tests.feature
|
43
|
+
$ rm -r features/fixtures/csvw
|
44
|
+
|
45
|
+
and then run the cucumber tests again or:
|
46
|
+
|
47
|
+
$ ruby features/support/load_tests.rb
|
48
|
+
|
10
49
|
|
11
50
|
## Installation
|
12
51
|
|
@@ -265,7 +304,7 @@ Schema validation provides some additional types of error and warning messages:
|
|
265
304
|
* `:below_minimum` (error) -- a column with a `minimum` constraint contains a value that is below the minimum
|
266
305
|
* `:above_maximum` (error) -- a column with a `maximum` constraint contains a value that is above the maximum
|
267
306
|
|
268
|
-
|
307
|
+
### Other validation options
|
269
308
|
|
270
309
|
You can also provide an optional options hash as the fourth argument to Validator#new. Supported options are:
|
271
310
|
|
@@ -281,48 +320,13 @@ validator = Csvlint::Validator.new( "http://example.org/data.csv", nil, nil, opt
|
|
281
320
|
* :lambda -- Pass a block of code to be called when each line is validated, this will give you access to the `Validator` object. For example, this will return the current line number for every line validated:
|
282
321
|
|
283
322
|
```
|
284
|
-
options = {
|
285
|
-
|
286
|
-
}
|
287
|
-
validator = Csvlint::Validator.new( "http://example.org/data.csv", nil, nil, options )
|
288
|
-
=> 1
|
289
|
-
2
|
290
|
-
3
|
291
|
-
4
|
292
|
-
.....
|
323
|
+
options = {
|
324
|
+
lambda: ->(validator) { puts validator.current_line }
|
325
|
+
}
|
326
|
+
validator = Csvlint::Validator.new( "http://example.org/data.csv", nil, nil, options )
|
327
|
+
=> 1
|
328
|
+
2
|
329
|
+
3
|
330
|
+
4
|
331
|
+
.....
|
293
332
|
```
|
294
|
-
|
295
|
-
## Contributing
|
296
|
-
|
297
|
-
1. Fork it
|
298
|
-
2. Create your feature branch (`git checkout -b my-new-feature`)
|
299
|
-
3. Commit your changes (`git commit -am 'Add some feature'`)
|
300
|
-
4. Push to the branch (`git push origin my-new-feature`)
|
301
|
-
5. Create new Pull Request
|
302
|
-
|
303
|
-
### Testing
|
304
|
-
|
305
|
-
The codebase includes both rspec and cucumber tests, which can be run together using:
|
306
|
-
|
307
|
-
$ rake
|
308
|
-
|
309
|
-
or separately:
|
310
|
-
|
311
|
-
$ rake spec
|
312
|
-
$ rake features
|
313
|
-
|
314
|
-
When the cucumber tests are first run, a script will create tests based on the latest version of the [CSV on the Web test suite](http://w3c.github.io/csvw/tests/), including creating a local cache of the test files. This requires an internet connection and some patience. Following that download, the tests will run locally; there's also a batch script:
|
315
|
-
|
316
|
-
$ bin/run-csvw-tests
|
317
|
-
|
318
|
-
which will run the tests from the command line.
|
319
|
-
|
320
|
-
If you need to refresh the CSV on the Web tests:
|
321
|
-
|
322
|
-
$ rm bin/run-csvw-tests
|
323
|
-
$ rm features/csvw_validation_tests.feature
|
324
|
-
$ rm -r features/fixtures/csvw
|
325
|
-
|
326
|
-
and then run the cucumber tests again or:
|
327
|
-
|
328
|
-
$ ruby features/support/load_tests.rb
|
data/csvlint.gemspec
CHANGED
@@ -10,7 +10,7 @@ Gem::Specification.new do |spec|
|
|
10
10
|
spec.email = ["pezholio@gmail.com"]
|
11
11
|
spec.description = %q{CSV Validator}
|
12
12
|
spec.summary = %q{CSV Validator}
|
13
|
-
spec.homepage = ""
|
13
|
+
spec.homepage = "https://github.com/theodi/csvlint.rb"
|
14
14
|
spec.license = "MIT"
|
15
15
|
|
16
16
|
spec.files = `git ls-files`.split($/)
|
@@ -18,16 +18,18 @@ Gem::Specification.new do |spec|
|
|
18
18
|
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
19
19
|
spec.require_paths = ["lib"]
|
20
20
|
|
21
|
+
spec.required_ruby_version = ['~> 2.0']
|
22
|
+
|
21
23
|
spec.add_dependency "mime-types"
|
22
24
|
spec.add_dependency "colorize"
|
23
25
|
spec.add_dependency "open_uri_redirections"
|
24
|
-
spec.add_dependency "activesupport"
|
26
|
+
spec.add_dependency "activesupport"
|
25
27
|
spec.add_dependency "addressable"
|
26
28
|
spec.add_dependency "typhoeus"
|
27
29
|
spec.add_dependency "escape_utils"
|
28
30
|
spec.add_dependency "uri_template"
|
29
31
|
spec.add_dependency "thor"
|
30
|
-
spec.add_dependency "rack"
|
32
|
+
spec.add_dependency "rack"
|
31
33
|
spec.add_dependency "net-http-persistent", "< 3.0"
|
32
34
|
|
33
35
|
spec.add_development_dependency "bundler", "~> 1.3"
|
Binary file
|
data/lib/csvlint/cli.rb
CHANGED
@@ -10,15 +10,18 @@ module Csvlint
|
|
10
10
|
class Cli < Thor
|
11
11
|
|
12
12
|
desc "myfile.csv OR csvlint http://example.com/myfile.csv", "Supports validating CSV files to check their syntax and contents"
|
13
|
+
|
13
14
|
option :dump_errors, desc: "Pretty print error and warning objects.", type: :boolean, aliases: :d
|
14
15
|
option :schema, banner: "FILENAME OR URL", desc: "Schema file", aliases: :s
|
15
16
|
option :json, desc: "Output errors as JSON", type: :boolean, aliases: :j
|
17
|
+
option :werror, desc: "Make all warnings into errors", type: :boolean, aliases: :w
|
18
|
+
|
16
19
|
def validate(source = nil)
|
17
20
|
source = read_source(source)
|
18
21
|
@schema = get_schema(options[:schema]) if options[:schema]
|
19
22
|
fetch_schema_tables(@schema, options) if source.nil?
|
20
23
|
|
21
|
-
valid = validate_csv(source, @schema, options[:
|
24
|
+
valid = validate_csv(source, @schema, options[:dump_errors], options[:json], options[:werror])
|
22
25
|
exit 1 unless valid
|
23
26
|
end
|
24
27
|
|
@@ -80,7 +83,7 @@ module Csvlint
|
|
80
83
|
rescue Errno::ENOENT
|
81
84
|
return_error "#{source} not found"
|
82
85
|
end unless source =~ /^http(s)?/
|
83
|
-
valid &= validate_csv(source, schema, options[:
|
86
|
+
valid &= validate_csv(source, schema, options[:dump_errors], nil, options[:werror])
|
84
87
|
end
|
85
88
|
|
86
89
|
exit 1 unless valid
|
@@ -129,7 +132,7 @@ module Csvlint
|
|
129
132
|
exit 1
|
130
133
|
end
|
131
134
|
|
132
|
-
def validate_csv(source, schema, dump, json)
|
135
|
+
def validate_csv(source, schema, dump, json, werror)
|
133
136
|
@error_count = 0
|
134
137
|
|
135
138
|
if json === true
|
@@ -162,11 +165,11 @@ module Csvlint
|
|
162
165
|
else
|
163
166
|
puts "\r\n#{csv} is #{validator.valid? ? "VALID" : "INVALID"}"
|
164
167
|
end
|
165
|
-
|
166
|
-
print_errors(validator.errors, dump)
|
168
|
+
print_errors(validator.errors, dump)
|
167
169
|
print_errors(validator.warnings, dump)
|
168
170
|
end
|
169
|
-
|
171
|
+
|
172
|
+
return false if werror && validator.warnings.size > 0
|
170
173
|
return validator.valid?
|
171
174
|
end
|
172
175
|
|
data/lib/csvlint/schema.rb
CHANGED
@@ -16,6 +16,8 @@ module Csvlint
|
|
16
16
|
|
17
17
|
class << self
|
18
18
|
|
19
|
+
extend Gem::Deprecate
|
20
|
+
|
19
21
|
def from_json_table(uri, json)
|
20
22
|
fields = []
|
21
23
|
json["fields"].each do |field_desc|
|
@@ -29,9 +31,21 @@ module Csvlint
|
|
29
31
|
return Csvlint::Csvw::TableGroup.from_json(uri, json)
|
30
32
|
end
|
31
33
|
|
34
|
+
# Deprecated method signature
|
32
35
|
def load_from_json(uri, output_errors = true)
|
36
|
+
load_from_uri(uri, output_errors)
|
37
|
+
end
|
38
|
+
deprecate :load_from_json, :load_from_uri, 2018, 1
|
39
|
+
|
40
|
+
def load_from_uri(uri, output_errors = true)
|
41
|
+
load_from_string(uri, open(uri).read, output_errors)
|
42
|
+
rescue OpenURI::HTTPError, Errno::ENOENT => e
|
43
|
+
raise e
|
44
|
+
end
|
45
|
+
|
46
|
+
def load_from_string(uri, string, output_errors = true)
|
33
47
|
begin
|
34
|
-
json = JSON.parse(
|
48
|
+
json = JSON.parse( string )
|
35
49
|
if json["@context"]
|
36
50
|
uri = "file:#{File.expand_path(uri)}" unless uri.to_s =~ /^http(s)?/
|
37
51
|
return Schema.from_csvw_metadata(uri,json)
|
@@ -43,8 +57,6 @@ module Csvlint
|
|
43
57
|
|
44
58
|
rescue Csvlint::Csvw::MetadataError => e
|
45
59
|
raise e
|
46
|
-
rescue OpenURI::HTTPError, Errno::ENOENT => e
|
47
|
-
raise e
|
48
60
|
rescue => e
|
49
61
|
if output_errors === true
|
50
62
|
STDERR.puts e.class
|
data/lib/csvlint/validate.rb
CHANGED
data/lib/csvlint/version.rb
CHANGED
data/spec/validator_spec.rb
CHANGED
@@ -28,6 +28,12 @@ describe Csvlint::Validator do
|
|
28
28
|
expect(validator.data.size).to eql(3)
|
29
29
|
end
|
30
30
|
|
31
|
+
it "should validate from a file path including whitespace" do
|
32
|
+
validator = Csvlint::Validator.new(File.new(File.join(File.dirname(__FILE__),'..','features','fixtures','white space in filename.csv')))
|
33
|
+
|
34
|
+
expect(validator.valid?).to eql(true)
|
35
|
+
end
|
36
|
+
|
31
37
|
context "multi line CSV validation with included schema" do
|
32
38
|
|
33
39
|
end
|
@@ -46,7 +52,7 @@ describe Csvlint::Validator do
|
|
46
52
|
|
47
53
|
it ".each() -> parse_contents method validates a well formed CSV" do
|
48
54
|
# when invoking parse contents
|
49
|
-
data = StringIO.new("
|
55
|
+
data = StringIO.new(%Q{"Foo","Bar","Baz"\r\n"1","2","3"\r\n"1","2","3"\r\n"3","2","1"})
|
50
56
|
|
51
57
|
validator = Csvlint::Validator.new(data)
|
52
58
|
|
@@ -61,20 +67,32 @@ describe Csvlint::Validator do
|
|
61
67
|
|
62
68
|
it ".each() -> `parse_contents` parses malformed CSV and catches unclosed quote" do
|
63
69
|
# doesn't build warnings because check_consistency isn't invoked
|
64
|
-
|
65
|
-
data = StringIO.new("\"Foo\",\"Bar\",\"Baz\"\r\n\"1\",\"2\",\"3\"\r\n\"1\",\"2\",\"3\"\r\n\"3\",\"2\",\"1\" ")
|
70
|
+
data = StringIO.new(%Q{"Foo","Bar","Baz"\r\n"1","2","3"\r\n"1","2","3"\r\n"3","2","1})
|
66
71
|
|
67
72
|
validator = Csvlint::Validator.new(data)
|
68
73
|
|
69
74
|
expect(validator.valid?).to eql(false)
|
75
|
+
expect(validator.errors.count).to eql(1)
|
70
76
|
expect(validator.errors.first.type).to eql(:unclosed_quote)
|
77
|
+
end
|
78
|
+
|
79
|
+
it ".each() -> `parse_contents` parses malformed CSV and catches stray quote" do
|
80
|
+
pending "cannot make Ruby generate a stray quote error"
|
81
|
+
# doesn't build warnings because check_consistency isn't invoked
|
82
|
+
# TODO below is trailing whitespace but is interpreted as a stray quote
|
83
|
+
data = StringIO.new(%Q{"Foo","Bar","Baz"\r\n"1","2","3"\r\n"1","2","3"\r\n"3","2","1""})
|
84
|
+
|
85
|
+
validator = Csvlint::Validator.new(data)
|
86
|
+
|
87
|
+
expect(validator.valid?).to eql(false)
|
88
|
+
expect(validator.errors.first.type).to eql(:stray_quote)
|
71
89
|
expect(validator.errors.count).to eql(1)
|
72
90
|
end
|
73
91
|
|
74
92
|
it ".each() -> `parse_contents` parses malformed CSV and catches whitespace and edge case" do
|
75
93
|
# when this data gets passed the header it rescues a whitespace error, resulting in the header row being discarded
|
76
94
|
# TODO - check if this is an edge case, currently passing because it requires advice on how to specify
|
77
|
-
data = StringIO.new(
|
95
|
+
data = StringIO.new(%Q{ "Foo","Bar","Baz"\r\n"1","2","3"\r\n"1","2","3"\r\n"3","2","1" })
|
78
96
|
|
79
97
|
validator = Csvlint::Validator.new(data)
|
80
98
|
|
@@ -84,13 +102,13 @@ describe Csvlint::Validator do
|
|
84
102
|
end
|
85
103
|
|
86
104
|
it "handles line breaks within a cell" do
|
87
|
-
data = StringIO.new("
|
105
|
+
data = StringIO.new(%Q{"a","b","c"\r\n"d","e","this is\r\nvalid"\r\n"a","b","c"})
|
88
106
|
validator = Csvlint::Validator.new(data)
|
89
107
|
expect(validator.valid?).to eql(true)
|
90
108
|
end
|
91
109
|
|
92
110
|
it "handles multiple line breaks within a cell" do
|
93
|
-
data = StringIO.new("
|
111
|
+
data = StringIO.new(%Q{"a","b","c"\r\n"d","this is\r\n valid","as is this\r\n too"})
|
94
112
|
validator = Csvlint::Validator.new(data)
|
95
113
|
expect(validator.valid?).to eql(true)
|
96
114
|
end
|
@@ -125,7 +143,7 @@ describe Csvlint::Validator do
|
|
125
143
|
|
126
144
|
it ".each() -> `validate` to pass input in streaming fashion" do
|
127
145
|
# warnings are built when validate is used to call all three methods
|
128
|
-
data = StringIO.new("
|
146
|
+
data = StringIO.new(%Q{"Foo","Bar","Baz"\r\n"1","2","3"\r\n"1","2","3"\r\n"3","2","1"})
|
129
147
|
validator = Csvlint::Validator.new(data)
|
130
148
|
|
131
149
|
expect(validator.valid?).to eql(true)
|
@@ -136,7 +154,7 @@ describe Csvlint::Validator do
|
|
136
154
|
end
|
137
155
|
|
138
156
|
it ".each() -> `validate` parses malformed CSV, populates errors, warnings & info_msgs,invokes finish()" do
|
139
|
-
data = StringIO.new("
|
157
|
+
data = StringIO.new(%Q{"Foo","Bar","Baz"\r\n"1","2","3"\r\n"1","2","3"\r\n"1","two","3"\r\n"3","2", "1"})
|
140
158
|
|
141
159
|
validator = Csvlint::Validator.new(data)
|
142
160
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: csvlint
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- pezholio
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2017-10-05 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: mime-types
|
@@ -56,16 +56,16 @@ dependencies:
|
|
56
56
|
name: activesupport
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
58
58
|
requirements:
|
59
|
-
- - "
|
59
|
+
- - ">="
|
60
60
|
- !ruby/object:Gem::Version
|
61
|
-
version: '
|
61
|
+
version: '0'
|
62
62
|
type: :runtime
|
63
63
|
prerelease: false
|
64
64
|
version_requirements: !ruby/object:Gem::Requirement
|
65
65
|
requirements:
|
66
|
-
- - "
|
66
|
+
- - ">="
|
67
67
|
- !ruby/object:Gem::Version
|
68
|
-
version: '
|
68
|
+
version: '0'
|
69
69
|
- !ruby/object:Gem::Dependency
|
70
70
|
name: addressable
|
71
71
|
requirement: !ruby/object:Gem::Requirement
|
@@ -140,16 +140,16 @@ dependencies:
|
|
140
140
|
name: rack
|
141
141
|
requirement: !ruby/object:Gem::Requirement
|
142
142
|
requirements:
|
143
|
-
- - "
|
143
|
+
- - ">="
|
144
144
|
- !ruby/object:Gem::Version
|
145
|
-
version: '
|
145
|
+
version: '0'
|
146
146
|
type: :runtime
|
147
147
|
prerelease: false
|
148
148
|
version_requirements: !ruby/object:Gem::Requirement
|
149
149
|
requirements:
|
150
|
-
- - "
|
150
|
+
- - ">="
|
151
151
|
- !ruby/object:Gem::Version
|
152
|
-
version: '
|
152
|
+
version: '0'
|
153
153
|
- !ruby/object:Gem::Dependency
|
154
154
|
name: net-http-persistent
|
155
155
|
requirement: !ruby/object:Gem::Requirement
|
@@ -413,10 +413,14 @@ extra_rdoc_files: []
|
|
413
413
|
files:
|
414
414
|
- ".coveralls.yml"
|
415
415
|
- ".gitattributes"
|
416
|
+
- ".github/ISSUE_TEMPLATE.md"
|
417
|
+
- ".github/PULL_REQUEST_TEMPLATE.md"
|
416
418
|
- ".gitignore"
|
417
419
|
- ".ruby-version"
|
418
420
|
- ".travis.yml"
|
419
421
|
- CHANGELOG.md
|
422
|
+
- CODE_OF_CONDUCT.md
|
423
|
+
- CONTRIBUTING.md
|
420
424
|
- Gemfile
|
421
425
|
- LICENSE.md
|
422
426
|
- README.md
|
@@ -442,6 +446,7 @@ files:
|
|
442
446
|
- features/fixtures/valid.csv
|
443
447
|
- features/fixtures/valid_many_rows.csv
|
444
448
|
- features/fixtures/w3.org/.well-known/csvm
|
449
|
+
- features/fixtures/white space in filename.csv
|
445
450
|
- features/fixtures/windows-line-endings.csv
|
446
451
|
- features/information.feature
|
447
452
|
- features/parse_csv.feature
|
@@ -488,7 +493,7 @@ files:
|
|
488
493
|
- spec/schema_spec.rb
|
489
494
|
- spec/spec_helper.rb
|
490
495
|
- spec/validator_spec.rb
|
491
|
-
homepage:
|
496
|
+
homepage: https://github.com/theodi/csvlint.rb
|
492
497
|
licenses:
|
493
498
|
- MIT
|
494
499
|
metadata: {}
|
@@ -498,9 +503,9 @@ require_paths:
|
|
498
503
|
- lib
|
499
504
|
required_ruby_version: !ruby/object:Gem::Requirement
|
500
505
|
requirements:
|
501
|
-
- - "
|
506
|
+
- - "~>"
|
502
507
|
- !ruby/object:Gem::Version
|
503
|
-
version: '0'
|
508
|
+
version: '2.0'
|
504
509
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
505
510
|
requirements:
|
506
511
|
- - ">="
|
@@ -508,7 +513,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
508
513
|
version: '0'
|
509
514
|
requirements: []
|
510
515
|
rubyforge_project:
|
511
|
-
rubygems_version: 2.
|
516
|
+
rubygems_version: 2.6.13
|
512
517
|
signing_key:
|
513
518
|
specification_version: 4
|
514
519
|
summary: CSV Validator
|
@@ -531,6 +536,7 @@ test_files:
|
|
531
536
|
- features/fixtures/valid.csv
|
532
537
|
- features/fixtures/valid_many_rows.csv
|
533
538
|
- features/fixtures/w3.org/.well-known/csvm
|
539
|
+
- features/fixtures/white space in filename.csv
|
534
540
|
- features/fixtures/windows-line-endings.csv
|
535
541
|
- features/information.feature
|
536
542
|
- features/parse_csv.feature
|