fastercsv 1.2.3 → 1.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/CHANGELOG CHANGED
@@ -2,6 +2,22 @@
2
2
 
3
3
  Below is a complete listing of changes for each revision of FasterCSV.
4
4
 
5
+ == 1.4.0
6
+
7
+ * Added encoding support patch from Michael Reinsch.
8
+ * Improved inspect() messages for better IRb support.
9
+ * Fixed header writing bug reported by Dov Murik.
10
+ * Use custom separators in parsing header Strings as suggested by Shmulik Regev.
11
+ * Added a <tt>:write_headers</tt> option for outputting headers.
12
+ * Handle open() calls in binary mode whenever we can to workaround a Windows
13
+ issue where line-ending translation can cause an off-by-one error in seeking
14
+ back to a non-zero starting position after auto-discovery for
15
+ <tt>:row_sep</tt> as suggested by Robert Battle.
16
+ * Improved the parser to fail faster when fed some forms of invalid CSV that can
17
+ be detected without reading ahead.
18
+ * Added a <tt>:field_size_limit</tt> option to control FasterCSV's lookahead and
19
+ prevent the parser from biting off more data than it can chew.
20
+
5
21
  == 1.2.3
6
22
 
7
23
  * Default to the system line ending when passed a GzipWriter object to wrap.
@@ -75,7 +75,7 @@ require "stringio"
75
75
  #
76
76
  class FasterCSV
77
77
  # The version of the installed library.
78
- VERSION = "1.2.3".freeze
78
+ VERSION = "1.4.0".freeze
79
79
 
80
80
  #
81
81
  # A FasterCSV::Row is part Array and part Hash. It retains an order for the
@@ -363,6 +363,16 @@ class FasterCSV
363
363
  fields.to_csv(options)
364
364
  end
365
365
  alias_method :to_s, :to_csv
366
+
367
+ # A summary of fields, by header.
368
+ def inspect
369
+ str = "#<#{self.class}"
370
+ each do |header, field|
371
+ str << " #{header.is_a?(Symbol) ? header.to_s : header.inspect}:" <<
372
+ field.inspect
373
+ end
374
+ str << ">"
375
+ end
366
376
  end
367
377
 
368
378
  #
@@ -695,6 +705,10 @@ class FasterCSV
695
705
  end.join
696
706
  end
697
707
  alias_method :to_s, :to_csv
708
+
709
+ def inspect
710
+ "#<#{self.class} mode:#{@mode} row_count:#{to_a.size}>"
711
+ end
698
712
  end
699
713
 
700
714
  # The error thrown when the parser encounters illegal CSV formatting.
@@ -999,7 +1013,7 @@ class FasterCSV
999
1013
  # The +options+ parameter can be anything FasterCSV::new() understands.
1000
1014
  #
1001
1015
  def self.foreach(path, options = Hash.new, &block)
1002
- open(path, options) do |csv|
1016
+ open(path, "rb", options) do |csv|
1003
1017
  csv.each(&block)
1004
1018
  end
1005
1019
  end
@@ -1120,8 +1134,8 @@ class FasterCSV
1120
1134
 
1121
1135
  #
1122
1136
  # :call-seq:
1123
- # open( filename, mode="r", options = Hash.new ) { |faster_csv| ... }
1124
- # open( filename, mode="r", options = Hash.new )
1137
+ # open( filename, mode="rb", options = Hash.new ) { |faster_csv| ... }
1138
+ # open( filename, mode="rb", options = Hash.new )
1125
1139
  #
1126
1140
  # This method opens an IO object, and wraps that with FasterCSV. This is
1127
1141
  # intended as the primary interface for writing a CSV file.
@@ -1166,6 +1180,8 @@ class FasterCSV
1166
1180
  def self.open(*args)
1167
1181
  # find the +options+ Hash
1168
1182
  options = if args.last.is_a? Hash then args.pop else Hash.new end
1183
+ # default to a binary open mode
1184
+ args << "rb" if args.size == 1
1169
1185
  # wrap a File opened with the remaining +args+
1170
1186
  csv = new(File.open(*args), options)
1171
1187
 
@@ -1222,7 +1238,7 @@ class FasterCSV
1222
1238
  # file and any +options+ FasterCSV::new() understands.
1223
1239
  #
1224
1240
  def self.read(path, options = Hash.new)
1225
- open(path, options) { |csv| csv.read }
1241
+ open(path, "rb", options) { |csv| csv.read }
1226
1242
  end
1227
1243
 
1228
1244
  # Alias for FasterCSV::read().
@@ -1279,7 +1295,14 @@ class FasterCSV
1279
1295
  # <tt>$INPUT_RECORD_SEPARATOR</tt>
1280
1296
  # (<tt>$/</tt>) is used. Obviously,
1281
1297
  # discovery takes a little time. Set
1282
- # manually if speed is important.
1298
+ # manually if speed is important. Also
1299
+ # note that IO objects should be opened
1300
+ # in binary mode on Windows if this
1301
+ # feature will be used as the
1302
+ # line-ending translation can cause
1303
+ # problems with resetting the document
1304
+ # position to where it was before the
1305
+ # read ahead.
1283
1306
  # <b><tt>:quote_char</tt></b>:: The character used to quote fields.
1284
1307
  # This has to be a single character
1285
1308
  # String. This is useful for
@@ -1289,6 +1312,27 @@ class FasterCSV
1289
1312
  # FasterCSV will always consider a
1290
1313
  # double sequence this character to be
1291
1314
  # an escaped quote.
1315
+ # <b><tt>:encoding</tt></b>:: The encoding to use when parsing the
1316
+ # file. Defaults to your <tt>$KDOCE</tt>
1317
+ # setting. Valid values: <tt>`n’</tt> or
1318
+ # <tt>`N’</tt> for none, <tt>`e’</tt> or
1319
+ # <tt>`E’</tt> for EUC, <tt>`s’</tt> or
1320
+ # <tt>`S’</tt> for SJIS, and
1321
+ # <tt>`u’</tt> or <tt>`U’</tt> for UTF-8
1322
+ # (see Regexp.new()).
1323
+ # <b><tt>:field_size_limit</tt></b>:: This is a maximum size FasterCSV will
1324
+ # read ahead looking for the closing
1325
+ # quote for a field. (In truth, it
1326
+ # reads to the first line ending beyond
1327
+ # this size.) If a quote cannot be
1328
+ # found within the limit FasterCSV will
1329
+ # raise a MalformedCSVError, assuming
1330
+ # the data is faulty. You can use this
1331
+ # limit to prevent what are effectively
1332
+ # DoS attacks on the parser. However,
1333
+ # this limit can cause a legitimate
1334
+ # parse to fail and thus is set to
1335
+ # +nil+, or off, by default.
1292
1336
  # <b><tt>:converters</tt></b>:: An Array of names from the Converters
1293
1337
  # Hash and/or lambdas that handle custom
1294
1338
  # conversion. A single converter
@@ -1309,8 +1353,11 @@ class FasterCSV
1309
1353
  # contents will be used as the headers.
1310
1354
  # If set to a String, the String is run
1311
1355
  # through a call of
1312
- # FasterCSV::parse_line() to produce an
1313
- # Array of headers. This setting causes
1356
+ # FasterCSV::parse_line() with the same
1357
+ # <tt>:col_sep</tt>, <tt>:row_sep</tt>,
1358
+ # and <tt>:quote_char</tt> as this
1359
+ # instance to produce an Array of
1360
+ # headers. This setting causes
1314
1361
  # FasterCSV.shift() to return rows as
1315
1362
  # FasterCSV::Row objects instead of
1316
1363
  # Arrays and FasterCSV.read() to return
@@ -1322,6 +1369,9 @@ class FasterCSV
1322
1369
  # object with identical headers and
1323
1370
  # fields (save that the fields do not go
1324
1371
  # through the converters).
1372
+ # <b><tt>:write_headers</tt></b>:: When +true+ and <tt>:headers</tt> is
1373
+ # set, a header row will be added to the
1374
+ # output.
1325
1375
  # <b><tt>:header_converters</tt></b>:: Identical in functionality to
1326
1376
  # <tt>:converters</tt> save that the
1327
1377
  # conversions are only made to header
@@ -1390,12 +1440,18 @@ class FasterCSV
1390
1440
  # The data source must be open for writing.
1391
1441
  #
1392
1442
  def <<(row)
1443
+ # make sure headers have been assigned
1444
+ if header_row? and [Array, String].include? @use_headers.class
1445
+ parse_headers # won't read data for Array or String
1446
+ self << @headers if @write_headers
1447
+ end
1448
+
1393
1449
  # Handle FasterCSV::Row objects and Hashes
1394
1450
  row = case row
1395
- when self.class::Row then row.fields
1396
- when Hash then @headers.map { |header| row[header] }
1397
- else row
1398
- end
1451
+ when self.class::Row then row.fields
1452
+ when Hash then @headers.map { |header| row[header] }
1453
+ else row
1454
+ end
1399
1455
 
1400
1456
  @headers = row if header_row?
1401
1457
  @lineno += 1
@@ -1513,7 +1569,7 @@ class FasterCSV
1513
1569
  # add another read to the line
1514
1570
  line += @io.gets(@row_sep) rescue return nil
1515
1571
  # copy the line so we can chop it up in parsing
1516
- parse = line.dup
1572
+ parse = line.dup
1517
1573
  parse.sub!(@parsers[:line_end], "")
1518
1574
 
1519
1575
  #
@@ -1590,6 +1646,10 @@ class FasterCSV
1590
1646
  # if we're not empty?() but at eof?(), a quoted field wasn't closed...
1591
1647
  if @io.eof?
1592
1648
  raise MalformedCSVError, "Unclosed quoted field on line #{lineno + 1}."
1649
+ elsif parse =~ @parsers[:bad_field]
1650
+ raise MalformedCSVError, "Illegal quoting on line #{lineno + 1}."
1651
+ elsif @field_size_limit and parse.length >= @field_size_limit
1652
+ raise MalformedCSVError, "Field size exceeded on line #{lineno + 1}."
1593
1653
  end
1594
1654
  # otherwise, we need to loop and pull some more data to complete the row
1595
1655
  end
@@ -1597,6 +1657,32 @@ class FasterCSV
1597
1657
  alias_method :gets, :shift
1598
1658
  alias_method :readline, :shift
1599
1659
 
1660
+ # Returns a simplified description of the key FasterCSV attributes.
1661
+ def inspect
1662
+ str = "<##{self.class} io_type:"
1663
+ # show type of wrapped IO
1664
+ if @io == $stdout then str << "$stdout"
1665
+ elsif @io == $stdin then str << "$stdin"
1666
+ elsif @io == $stderr then str << "$stderr"
1667
+ else str << @io.class.to_s
1668
+ end
1669
+ # show IO.path(), if available
1670
+ if @io.respond_to?(:path) and (p = @io.path)
1671
+ str << " io_path:#{p.inspect}"
1672
+ end
1673
+ # show other attributes
1674
+ %w[ lineno col_sep row_sep
1675
+ quote_char skip_blanks encoding ].each do |attr_name|
1676
+ if a = instance_variable_get("@#{attr_name}")
1677
+ str << " #{attr_name}:#{a.inspect}"
1678
+ end
1679
+ end
1680
+ if @use_headers
1681
+ str << " headers:#{(@headers || true).inspect}"
1682
+ end
1683
+ str << ">"
1684
+ end
1685
+
1600
1686
  private
1601
1687
 
1602
1688
  #
@@ -1690,27 +1776,42 @@ class FasterCSV
1690
1776
  # Pre-compiles parsers and stores them by name for access during reads.
1691
1777
  def init_parsers(options)
1692
1778
  # store the parser behaviors
1693
- @skip_blanks = options.delete(:skip_blanks)
1694
-
1779
+ @skip_blanks = options.delete(:skip_blanks)
1780
+ @encoding = options.delete(:encoding) # nil will use $KCODE
1781
+ @field_size_limit = options.delete(:field_size_limit)
1782
+
1695
1783
  # prebuild Regexps for faster parsing
1696
1784
  esc_col_sep = Regexp.escape(@col_sep)
1697
1785
  esc_row_sep = Regexp.escape(@row_sep)
1698
1786
  esc_quote = Regexp.escape(@quote_char)
1699
1787
  @parsers = {
1700
- :leading_fields =>
1701
- /\A(?:#{esc_col_sep})+/, # for empty leading fields
1702
- :csv_row =>
1703
- ### The Primary Parser ###
1704
- / \G(?:^|#{esc_col_sep}) # anchor the match
1705
- (?: #{esc_quote}( (?>[^#{esc_quote}]*) # find quoted fields
1706
- (?> #{esc_quote*2}
1707
- [^#{esc_quote}]* )* )#{esc_quote}
1708
- | # ... or ...
1709
- ([^#{esc_quote}#{esc_col_sep}]*) # unquoted fields
1710
- )/x,
1711
- ### End Primary Parser ###
1712
- :line_end =>
1713
- /#{esc_row_sep}\z/ # safer than chomp!()
1788
+ # for empty leading fields
1789
+ :leading_fields => Regexp.new("\\A(?:#{esc_col_sep})+", nil, @encoding),
1790
+ # The Primary Parser
1791
+ :csv_row => Regexp.new(<<-END_PARSER, Regexp::EXTENDED, @encoding),
1792
+ \\G(?:\\A|#{esc_col_sep}) # anchor the match
1793
+ (?: #{esc_quote}( (?>[^#{esc_quote}]*) # find quoted fields
1794
+ (?> #{esc_quote*2}
1795
+ [^#{esc_quote}]* )* )#{esc_quote}
1796
+ | # ... or ...
1797
+ ([^#{esc_quote}#{esc_col_sep}]*) # unquoted fields
1798
+ )
1799
+ (?=#{esc_col_sep}|\\z) # ensure we are at field's end
1800
+ END_PARSER
1801
+ # a test for unescaped quotes
1802
+ :bad_field => Regexp.new(<<-END_BAD, Regexp::EXTENDED, @encoding),
1803
+ \\A#{esc_col_sep}? # starts with an optional comma
1804
+ (?: #{esc_quote} (?>[^#{esc_quote}]*) # an extra quote
1805
+ (?> #{esc_quote*2}
1806
+ [^#{esc_quote}]* )*
1807
+ #{esc_quote}[^#{esc_quote}]
1808
+ | # ... or ...
1809
+ [^#{esc_quote}#{esc_col_sep}]+
1810
+ #{esc_quote} # unescaped quote
1811
+ )
1812
+ END_BAD
1813
+ # safer than chomp!()
1814
+ :line_end => Regexp.new("#{esc_row_sep}\\z", nil, @encoding)
1714
1815
  }
1715
1816
  end
1716
1817
 
@@ -1757,6 +1858,7 @@ class FasterCSV
1757
1858
  def init_headers(options)
1758
1859
  @use_headers = options.delete(:headers)
1759
1860
  @return_headers = options.delete(:return_headers)
1861
+ @write_headers = options.delete(:write_headers)
1760
1862
 
1761
1863
  # headers must be delayed until shift(), in case they need a row of content
1762
1864
  @headers = nil
@@ -1827,10 +1929,17 @@ class FasterCSV
1827
1929
  def parse_headers(row = nil)
1828
1930
  if @headers.nil? # header row
1829
1931
  @headers = case @use_headers # save headers
1830
- when Array then @use_headers # Array of headers
1831
- when String then self.class.parse_line(@use_headers) # CSV header String
1832
- else row # first row headers
1833
- end
1932
+ # Array of headers
1933
+ when Array then @use_headers
1934
+ # CSV header String
1935
+ when String
1936
+ self.class.parse_line( @use_headers,
1937
+ :col_sep => @col_sep,
1938
+ :row_sep => @row_sep,
1939
+ :quote_char => @quote_char )
1940
+ # first row is headers
1941
+ else row
1942
+ end
1834
1943
 
1835
1944
  # prepare converted and unconverted copies
1836
1945
  row = @headers if row.nil?
@@ -158,7 +158,7 @@ class TestCSVParsing < Test::Unit::TestCase
158
158
  assert_send([csv.lineno, :<, 4])
159
159
  end
160
160
  rescue FasterCSV::MalformedCSVError
161
- assert_equal("Unclosed quoted field on line 4.", $!.message)
161
+ assert_equal("Illegal quoting on line 4.", $!.message)
162
162
  end
163
163
  end
164
164
  end
@@ -0,0 +1,23 @@
1
+ #!/usr/local/bin/ruby -w
2
+
3
+ # tc_encodings.rb
4
+ #
5
+ # Created by Michael Reinsch.
6
+ # Copyright (c) 2008 Ubiquitous Business Technology, Inc.
7
+
8
+ require "test/unit"
9
+
10
+ require "faster_csv"
11
+
12
+ class TestEncodings < Test::Unit::TestCase
13
+ def test_with_shift_jis_encoding
14
+ $KCODE = 'u' # make sure $KCODE != Shift_JIS
15
+ # this test data will not work with UTF-8 encoding
16
+ shift_jis_data = [ "82D082E782AA82C82094E0",
17
+ "82D082E7826082AA825C",
18
+ "82D082E7826082AA82C8" ].map { |f| [f].pack("H*") }
19
+ fields = FCSV.parse_line( shift_jis_data.map { |f| %Q{"#{f}"} }.join(","),
20
+ :encoding => "s" )
21
+ assert_equal(shift_jis_data, fields)
22
+ end
23
+ end
@@ -174,6 +174,35 @@ class TestFasterCSVFeatures < Test::Unit::TestCase
174
174
  File.unlink(file)
175
175
  end
176
176
 
177
+ def test_inspect_is_smart_about_io_types
178
+ str = FasterCSV.new("string,data").inspect
179
+ assert(str.include?("io_type:StringIO"), "IO type not detected.")
180
+
181
+ str = FasterCSV.new($stderr).inspect
182
+ assert(str.include?("io_type:$stderr"), "IO type not detected.")
183
+
184
+ str = FasterCSV.open( File.join( File.dirname(__FILE__),
185
+ "test_data.csv" ) ) { |csv| csv.inspect }
186
+ assert(str.include?("io_type:File"), "IO type not detected.")
187
+ end
188
+
189
+ def test_inspect_shows_key_attributes
190
+ str = @csv.inspect
191
+ %w[lineno col_sep row_sep quote_char].each do |attr_name|
192
+ assert_match(/\b#{attr_name}:[^\s>]+/, str)
193
+ end
194
+ end
195
+
196
+ def test_inspect_shows_headers_when_available
197
+ FasterCSV.open( File.join( File.dirname(__FILE__),
198
+ "test_data.csv" ),
199
+ :headers => true ) do |csv|
200
+ assert(csv.inspect.include?("headers:true"), "Header hint not shown.")
201
+ csv.shift # load headers
202
+ assert_match(/headers:\[[^\]]+\]/, csv.inspect)
203
+ end
204
+ end
205
+
177
206
  def test_version
178
207
  assert_not_nil(FasterCSV::VERSION)
179
208
  assert_instance_of(String, FasterCSV::VERSION)
@@ -130,6 +130,21 @@ class TestFasterCSVHeaders < Test::Unit::TestCase
130
130
  assert(!row.field_row?)
131
131
  end
132
132
 
133
+ def test_csv_header_string_inherits_separators
134
+ # parse with custom col_sep
135
+ csv = nil
136
+ assert_nothing_raised(Exception) do
137
+ csv = FasterCSV.parse( @data.tr(",", "|"), :col_sep => "|",
138
+ :headers => "my|new|headers" )
139
+ end
140
+
141
+ # verify headers were recognized
142
+ row = csv[0]
143
+ assert_not_nil(row)
144
+ assert_instance_of(FasterCSV::Row, row)
145
+ assert_equal([%w{my first}, %w{new second}, %w{headers third}], row.to_a)
146
+ end
147
+
133
148
  def test_return_headers
134
149
  # activate headers and request they are returned
135
150
  csv = nil
@@ -161,7 +161,6 @@ class TestFasterCSVInterface < Test::Unit::TestCase
161
161
 
162
162
  lines = [{:a => 1, :b => 2, :c => 3}, {:a => 4, :b => 5, :c => 6}]
163
163
  FasterCSV.open( @path, "w", :headers => true,
164
- :converters => :all,
165
164
  :header_converters => :symbol ) do |csv|
166
165
  csv << lines.first.keys
167
166
  lines.each { |line| csv << line }
@@ -172,6 +171,75 @@ class TestFasterCSVInterface < Test::Unit::TestCase
172
171
  csv.each { |line| assert_equal(lines.shift, line.to_hash) }
173
172
  end
174
173
  end
174
+
175
+ def test_write_hash_with_headers_array
176
+ File.unlink(@path)
177
+
178
+ lines = [{:a => 1, :b => 2, :c => 3}, {:a => 4, :b => 5, :c => 6}]
179
+ FasterCSV.open(@path, "w", :headers => [:b, :a, :c]) do |csv|
180
+ lines.each { |line| csv << line }
181
+ end
182
+
183
+ # test writing fields in the correct order
184
+ File.open(@path, "r") do |f|
185
+ assert_equal("2,1,3", f.gets.strip)
186
+ assert_equal("5,4,6", f.gets.strip)
187
+ end
188
+
189
+ # test reading CSV with headers
190
+ FasterCSV.open( @path, "r", :headers => [:b, :a, :c],
191
+ :converters => :all ) do |csv|
192
+ csv.each { |line| assert_equal(lines.shift, line.to_hash) }
193
+ end
194
+ end
195
+
196
+ def test_write_hash_with_headers_string
197
+ File.unlink(@path)
198
+
199
+ lines = [{"a" => 1, "b" => 2, "c" => 3}, {"a" => 4, "b" => 5, "c" => 6}]
200
+ FasterCSV.open( @path, "w", :headers => "b|a|c",
201
+ :col_sep => "|" ) do |csv|
202
+ lines.each { |line| csv << line }
203
+ end
204
+
205
+ # test writing fields in the correct order
206
+ File.open(@path, "r") do |f|
207
+ assert_equal("2|1|3", f.gets.strip)
208
+ assert_equal("5|4|6", f.gets.strip)
209
+ end
210
+
211
+ # test reading CSV with headers
212
+ FasterCSV.open( @path, "r", :headers => "b|a|c",
213
+ :col_sep => "|",
214
+ :converters => :all ) do |csv|
215
+ csv.each { |line| assert_equal(lines.shift, line.to_hash) }
216
+ end
217
+ end
218
+
219
+ def test_write_headers
220
+ File.unlink(@path)
221
+
222
+ lines = [{"a" => 1, "b" => 2, "c" => 3}, {"a" => 4, "b" => 5, "c" => 6}]
223
+ FasterCSV.open( @path, "w", :headers => "b|a|c",
224
+ :write_headers => true,
225
+ :col_sep => "|" ) do |csv|
226
+ lines.each { |line| csv << line }
227
+ end
228
+
229
+ # test writing fields in the correct order
230
+ File.open(@path, "r") do |f|
231
+ assert_equal("b|a|c", f.gets.strip)
232
+ assert_equal("2|1|3", f.gets.strip)
233
+ assert_equal("5|4|6", f.gets.strip)
234
+ end
235
+
236
+ # test reading CSV with headers
237
+ FasterCSV.open( @path, "r", :headers => true,
238
+ :col_sep => "|",
239
+ :converters => :all ) do |csv|
240
+ csv.each { |line| assert_equal(lines.shift, line.to_hash) }
241
+ end
242
+ end
175
243
 
176
244
  def test_append # aliased add_row() and puts()
177
245
  File.unlink(@path)
@@ -285,4 +285,21 @@ class TestFasterCSVRow < Test::Unit::TestCase
285
285
 
286
286
  assert_equal([@row.headers.size, @row.fields.size].max, @row.size)
287
287
  end
288
+
289
+ def test_inspect_shows_header_field_pairs
290
+ str = @row.inspect
291
+ @row.each do |header, field|
292
+ assert( str.include?("#{header.inspect}:#{field.inspect}"),
293
+ "Header field pair not found." )
294
+ end
295
+ end
296
+
297
+ def test_inspect_shows_symbol_headers_as_bare_attributes
298
+ str = FasterCSV::Row.new( @row.headers.map { |h| h.to_sym },
299
+ @row.fields ).inspect
300
+ @row.each do |header, field|
301
+ assert( str.include?("#{header}:#{field.inspect}"),
302
+ "Header field pair not found." )
303
+ end
304
+ end
288
305
  end
@@ -6,12 +6,14 @@
6
6
  # Copyright 2005 Gray Productions. All rights reserved.
7
7
 
8
8
  require "test/unit"
9
+ require "timeout"
9
10
 
10
11
  require "faster_csv"
11
12
  require "csv"
12
13
 
13
14
  class TestFasterCSVSpeed < Test::Unit::TestCase
14
- PATH = File.join(File.dirname(__FILE__), "test_data.csv")
15
+ PATH = File.join(File.dirname(__FILE__), "test_data.csv")
16
+ BIG_DATA = "123456789\n" * 1024
15
17
 
16
18
  def test_that_we_are_doing_the_same_work
17
19
  FasterCSV.open(PATH) do |csv|
@@ -36,4 +38,28 @@ class TestFasterCSVSpeed < Test::Unit::TestCase
36
38
 
37
39
  assert(faster_csv_time < csv_time / 3)
38
40
  end
41
+
42
+ def test_the_parse_fails_fast_when_it_can_for_unquoted_fields
43
+ assert_parse_errors_out('valid,fields,bad start"' + BIG_DATA)
44
+ end
45
+
46
+ def test_the_parse_fails_fast_when_it_can_for_unescaped_quotes
47
+ assert_parse_errors_out('valid,fields,"bad start"unescaped' + BIG_DATA)
48
+ end
49
+
50
+ def test_field_size_limit_controls_lookahead
51
+ assert_parse_errors_out( 'valid,fields,"' + BIG_DATA + '"',
52
+ :field_size_limit => 2048 )
53
+ end
54
+
55
+ private
56
+
57
+ def assert_parse_errors_out(*args)
58
+ assert_raise(FasterCSV::MalformedCSVError) do
59
+ Timeout.timeout(0.2) do
60
+ FasterCSV.parse(*args)
61
+ fail("Parse didn't error out")
62
+ end
63
+ end
64
+ end
39
65
  end
@@ -388,4 +388,13 @@ class TestFasterCSVTable < Test::Unit::TestCase
388
388
 
389
389
  assert_equal(@rows.size, @table.size)
390
390
  end
391
+
392
+ def test_inspect_shows_current_mode
393
+ str = @table.inspect
394
+ assert(str.include?("mode:#{@table.mode}"), "Mode not shown.")
395
+
396
+ @table.by_col!
397
+ str = @table.inspect
398
+ assert(str.include?("mode:#{@table.mode}"), "Mode not shown.")
399
+ end
391
400
  end
@@ -17,3 +17,4 @@ require "tc_row"
17
17
  require "tc_table"
18
18
  require "tc_headers"
19
19
  require "tc_serialization"
20
+ require "tc_encodings"
metadata CHANGED
@@ -1,39 +1,39 @@
1
1
  --- !ruby/object:Gem::Specification
2
- rubygems_version: 0.9.4
3
- specification_version: 1
4
2
  name: fastercsv
5
3
  version: !ruby/object:Gem::Version
6
- version: 1.2.3
7
- date: 2007-12-02 00:00:00 -06:00
8
- summary: FasterCSV is CSV, but faster, smaller, and cleaner.
9
- require_paths:
10
- - lib
11
- email: james@grayproductions.net
12
- homepage: http://fastercsv.rubyforge.org
13
- rubyforge_project: fastercsv
14
- description: FasterCSV is intended as a complete replacement to the CSV standard library. It is significantly faster and smaller while still being pure Ruby code. It also strives for a better interface.
15
- autorequire:
16
- default_executable:
17
- bindir: bin
18
- has_rdoc: true
19
- required_ruby_version: !ruby/object:Gem::Version::Requirement
20
- requirements:
21
- - - ">"
22
- - !ruby/object:Gem::Version
23
- version: 0.0.0
24
- version:
4
+ version: 1.4.0
25
5
  platform: ruby
26
- signing_key:
27
- cert_chain:
28
- post_install_message:
29
6
  authors:
30
7
  - James Edward Gray II
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+
12
+ date: 2008-09-10 00:00:00 -05:00
13
+ default_executable:
14
+ dependencies: []
15
+
16
+ description: FasterCSV is intended as a complete replacement to the CSV standard library. It is significantly faster and smaller while still being pure Ruby code. It also strives for a better interface.
17
+ email: james@grayproductions.net
18
+ executables: []
19
+
20
+ extensions: []
21
+
22
+ extra_rdoc_files:
23
+ - AUTHORS
24
+ - COPYING
25
+ - README
26
+ - INSTALL
27
+ - TODO
28
+ - CHANGELOG
29
+ - LICENSE
31
30
  files:
32
31
  - lib/faster_csv.rb
33
32
  - lib/fastercsv.rb
34
33
  - test/tc_csv_parsing.rb
35
34
  - test/tc_csv_writing.rb
36
35
  - test/tc_data_converters.rb
36
+ - test/tc_encodings.rb
37
37
  - test/tc_features.rb
38
38
  - test/tc_headers.rb
39
39
  - test/tc_interface.rb
@@ -59,26 +59,34 @@ files:
59
59
  - TODO
60
60
  - CHANGELOG
61
61
  - LICENSE
62
- test_files:
63
- - test/ts_all.rb
62
+ has_rdoc: true
63
+ homepage: http://fastercsv.rubyforge.org
64
+ post_install_message:
64
65
  rdoc_options:
65
66
  - --title
66
67
  - FasterCSV Documentation
67
68
  - --main
68
69
  - README
69
- extra_rdoc_files:
70
- - AUTHORS
71
- - COPYING
72
- - README
73
- - INSTALL
74
- - TODO
75
- - CHANGELOG
76
- - LICENSE
77
- executables: []
78
-
79
- extensions: []
80
-
70
+ require_paths:
71
+ - lib
72
+ required_ruby_version: !ruby/object:Gem::Requirement
73
+ requirements:
74
+ - - ">="
75
+ - !ruby/object:Gem::Version
76
+ version: "0"
77
+ version:
78
+ required_rubygems_version: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: "0"
83
+ version:
81
84
  requirements: []
82
85
 
83
- dependencies: []
84
-
86
+ rubyforge_project: fastercsv
87
+ rubygems_version: 1.2.0
88
+ signing_key:
89
+ specification_version: 2
90
+ summary: FasterCSV is CSV, but faster, smaller, and cleaner.
91
+ test_files:
92
+ - test/ts_all.rb