dsv 0.12.10 → 0.12.12

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f88116d5530f25575975779af46933ddd7c67af651b2c61a3894b237b000950a
-  data.tar.gz: cd03ed4c4b47c12044387544346d8be13ac3f256f1263f135302134bf1ce429d
+  metadata.gz: d83c84d7b20fd4f67811f8259313a7c9f7c52cb48430feab0316c96137c3bbd0
+  data.tar.gz: 26bf580f0b29c4886796f267a415548bec6354009bede7a12b45b8fbbe3668a5
 SHA512:
-  metadata.gz: 0a6e2477a6ff620bba7e5317af9ec1482f0effa853f3ac6f6074e1460170f201b0649db16c1ba67b87621546b986948678d4f6610f2623e29766cb9d948c227d
-  data.tar.gz: b9ad0b92bcf1ec1caee1604217005250411ac318014ceafb9384ceb70ae276f5d2e457e097a287dcbb5c9d9b668144e8a2c35194d9e7e2a9d28377c5aa3bf8b9
+  metadata.gz: 9ee707193d1e740d503ec877f0c079a2868ab3ba848459771760cf96992b002763d6e337e3a9b9ac6f97428c98cf4700b5b32866e00a5729ab3fa50c5ba24094
+  data.tar.gz: 007524b8e1cffeb88f7d12b6ebe195b07aaafe85cc060bb1b74c6b372869e944505983ac3629ecabbce84f7f28fecc7e72537133e14ac19e5040dde8020f6344

data/CHANGELOG

@@ -2,6 +2,25 @@
 
 ## 20261103
 
+0.12.12: as_array: with a column selection gives the columns selected.
+
+1. ~ parse_fields(): under as_array:, a selection gives the selected fields; + positions(). Each column not selected had come back as nil, and a selection by position had lost its values.
+2. ~ DSV#each: under as_array:, selects by position. A selection by name had raised TypeError.
+3. ~ test/dsv_test.rb: + selections under as_array:, by name, by position, by a repeated name, and from rows already read.
+4. ~ README.md: + as_array:, and a selection under it.
+5. ~ TODO: - 1.
+6. ~ lib/DSV/VERSION.rb: /0.12.11/0.12.12/.
+
+0.12.11: the class methods close the files they open, and return an Enumerator without a block.
+
+1. ~ DSV.each, .read, .write, .header_row, .first_row, .attributes, .columns: + opened(), so that a path or text given is closed after the call, an early exit included; an IO given is left open. Each had left its file open.
+2. ~ DSV.each: without a block, returns an Enumerator which opens the source afresh on each pass.
+3. ~ DSV.collect, .select, .reject, .detect: by way of the Enumerator. Without a block each had raised NoMethodError.
+4. ~ test/dsv_test.rb: + files_opened(); + the class methods closing what they open, on an early exit too, and leaving an IO open; + the Enumerators.
+5. ~ README.md: Enumerating from the class, closing and the Enumerator.
+6. ~ TODO: - 1.
+7. ~ lib/DSV/VERSION.rb: /0.12.10/0.12.11/.
+
 0.12.10: ~ TODO, now grouped by release; + ROADMAP.md.
 
 1. ~ TODO: the jobs decided, by release: the fixes left for 0.12.x; modes as declarations in 0.13.0; streaming in 0.14.0; what 1.0 needs; speed and new options in 1.x.

data/README.md

@@ -40,6 +40,13 @@ DSV.read("1,2,3\n4,5,6\n")
 # => [{0 => "1", 1 => "2", 2 => "3"}, {0 => "4", 1 => "5", 2 => "6"}]
 ```
 
+With `as_array: true` each row is an Array of its values in column order, the form stdlib CSV gives:
+
+```ruby
+DSV.read("1,2,3\n4,5,6\n", as_array: true)
+# => [["1", "2", "3"], ["4", "5", "6"]]
+```
+
 A String that names an existing file is read as a file; any other String is read as text.
 
 ### Selecting columns
@@ -53,6 +60,8 @@ DSV.read('people.csv', 'name', headers: true)
 DSV.read('people.csv', headers: true, selected_columns: ['name'])
 ```
 
+Under `as_array:` the Arrays hold the columns selected, in column order.
+
 ### Enumerating from the class
 
 The Enumerable calls are available on a source directly, each taking the same options:
@@ -64,7 +73,7 @@ DSV.detect('people.csv', headers: true){|row| row['name'] == 'Ada'}
 DSV.collect('people.csv', headers: true){|row| row['name'].upcase}
 ```
 
-`foreach`, `map`, `find_all` and `find` are aliases.
+`foreach`, `map`, `find_all` and `find` are aliases. Each closes the file it opens. Without a block, each returns an Enumerator, which opens the file again on each pass, so `DSV.each('people.csv', headers: true).each_slice(100)` works as it would on any Enumerable.
 
 ### Any delimiter
 

data/TODO

@@ -2,45 +2,43 @@
 
 ## 0.12.x, fixes
 
-1. The class methods close what they open, an early exit included, and without a block return an Enumerator which opens the source afresh on each pass, as CSV.foreach does; collect, select, reject and detect without a block no longer raise.
-2. as_array: with a column selection: each('name') no longer raises TypeError, and read('name', as_array: true) gives only the columns selected. as_array: in the README, as the way to get Arrays as CSV gives them.
-3. Reading honours quote: given as a String, as writing does; DSV.parse_line honours quote:.
-4. A space as the column separator splits on each space, not as String#split(' ') does.
-5. :double back to 0.12.4's reading, undoing 0.12.5 item 2: a row not starting with a quote split plainly, one starting with a quote split on quote-separator-quote, and one which leaves a lone quote in a field read in full as wholly quoted. The README back to 0.12.3's wording.
-6. :single read, as :double is with a single quote.
-7. filter, a class-level alias of select.
-8. A String under a write mode taken as a path, so that a file not yet there can be written by name.
-9. The internals private.
+1. Reading honours quote: given as a String, as writing does; DSV.parse_line honours quote:.
+2. A space as the column separator splits on each space, not as String#split(' ') does.
+3. :double back to 0.12.4's reading, undoing 0.12.5 item 2: a row not starting with a quote split plainly, one starting with a quote split on quote-separator-quote, and one which leaves a lone quote in a field read in full as wholly quoted. The README back to 0.12.3's wording.
+4. :single read, as :double is with a single quote.
+5. filter, a class-level alias of select.
+6. A String under a write mode taken as a path, so that a file not yet there can be written by name.
+7. The internals private.
 
 ## 0.13.0, modes as declarations
 
-10. An unknown quote mode, file mode or option raises ArgumentError, where each is ignored today.
-11. :mixed, the name of the default.
-12. A mode raises on a contradiction which costs nothing to see, every row alike, header included: under :double, a row not starting with a quote, and a bare field in a row read in full. Declaring :mixed reads such a row.
+8. An unknown quote mode, file mode or option raises ArgumentError, where each is ignored today.
+9. :mixed, the name of the default.
+10. A mode raises on a contradiction which costs nothing to see, every row alike, header included: under :double, a row not starting with a quote, and a bare field in a row read in full. Declaring :mixed reads such a row.
 
 ## 0.14.0, streaming
 
-13. each yields each row as it is read and returns its receiver, so that detect and break stop reading and memory does not grow with the file; rows is no longer filled by each. Chunks come from each_slice. A source read once, start to end, with no rewind, so that a pipe or $stdin can be read: the header from the first row read, and the line of an error found by seeking only where the source can; a rewind left to r+ rewriting and first_row called alone.
+11. each yields each row as it is read and returns its receiver, so that detect and break stop reading and memory does not grow with the file; rows is no longer filled by each. Chunks come from each_slice. A source read once, start to end, with no rewind, so that a pipe or $stdin can be read: the header from the first row read, and the line of an error found by seeking only where the source can; a rewind left to r+ rewriting and first_row called alone.
 
 ## 1.0
 
-14. A corpus in the tests, a day of NEMWEB's dispatch files trimmed and the RFC 4180 cases, each checked against stdlib CSV's rows.
-15. A benchmark script against stdlib CSV, and the README's speed sentence written from its output.
-16. README: every mode by example, and a table matching each to stdlib CSV's and Python's settings.
+12. A corpus in the tests, a day of NEMWEB's dispatch files trimmed and the RFC 4180 cases, each checked against stdlib CSV's rows.
+13. A benchmark script against stdlib CSV, and the README's speed sentence written from its output.
+14. README: every mode by example, and a table matching each to stdlib CSV's and Python's settings.
 
 ## 1.x, speed, results unchanged
 
-17. A per-row fast path for the default: split, the fields bare or wrapped in two quotes unwrapped, any other row read in full.
-18. as_array: returning the split fields, rather than building a Hash and then an Array from it.
-19. The splitter chosen once, at construction, as row_renderer is for writing.
-20. Two string predicates in place of four regex matches per piece; rows built by zip.to_h; the scanner-based parser, at the tag scanner-parser-20261006, judged against the rest.
+15. A per-row fast path for the default: split, the fields bare or wrapped in two quotes unwrapped, any other row read in full.
+16. as_array: returning the split fields, rather than building a Hash and then an Array from it.
+17. The splitter chosen once, at construction, as row_renderer is for writing.
+18. Two string predicates in place of four regex matches per piece; rows built by zip.to_h; the scanner-based parser, at the tag scanner-parser-20261006, judged against the rest.
 
 ## 1.x, features
 
-21. :rfc_4180, a quote mode reading what RFC 4180 allows as :mixed does and raising on what it forbids.
-22. field_count:, :header or an Integer, off by default: a row of another width raises.
-23. Writing: minimal quoting, a field quoted only where it holds a separator, a quote or a row terminator; a row terminator inside a field kept, quoted, or folded to a space; the encoding declared and a transcode offered with any loss marked.
-24. A line number in each parse error, found only when the error is raised, as UnterminatedQuote finds its, where the source can seek. line_numbers:, off by default, counts lines as it reads instead, for any source a pipe included: the read loop chosen once, at construction, as row_renderer is for writing, so that without it no row pays for the count.
-25. The instance-level header_row renamed headers?, kept as an alias; << to append a row and generate to build a string.
-26. Symbols and Strings interchangeable for column names, in a selection, a columns: list and a row's keys.
-27. The duplicate-header modes as an option: keep all as an Array, the default; index, a, a.1; first; last; error.
+19. :rfc_4180, a quote mode reading what RFC 4180 allows as :mixed does and raising on what it forbids.
+20. field_count:, :header or an Integer, off by default: a row of another width raises.
+21. Writing: minimal quoting, a field quoted only where it holds a separator, a quote or a row terminator; a row terminator inside a field kept, quoted, or folded to a space; the encoding declared and a transcode offered with any loss marked.
+22. A line number in each parse error, found only when the error is raised, as UnterminatedQuote finds its, where the source can seek. line_numbers:, off by default, counts lines as it reads instead, for any source a pipe included: the read loop chosen once, at construction, as row_renderer is for writing, so that without it no row pays for the count.
+23. The instance-level header_row renamed headers?, kept as an alias; << to append a row and generate to build a string.
+24. Symbols and Strings interchangeable for column names, in a selection, a columns: list and a row's keys.
+25. The duplicate-header modes as an option: keep all as an Array, the default; index, a, a.1; first; last; error.

data/lib/DSV/VERSION.rb

@@ -2,5 +2,5 @@
 # DSV::VERSION
 
 class DSV
-  VERSION = '0.12.10'
+  VERSION = '0.12.12'
 end

data/lib/dsv.rb

@@ -53,36 +53,30 @@ class DSV
       end
     end
 
-    # The class-level read, parse and each take a column selection before the options, as the instance-level ones take it as arguments.
+    # The class-level read, parse and each take a column selection before the options, as the instance-level ones take it as arguments. Each closes what it opens; without a block, each returns an Enumerator which opens the source afresh on each pass.
     def each(source, *args, &block)
+      return to_enum(:each, source, *args) unless block
       options = DSV.extract_options(args)
-      new(source, options).each(*args, &block)
+      opened(source, options){|dsv| dsv.each(*args, &block)}
     end
     alias_method :foreach, :each
 
     def collect(source, *args, &block)
-      new_collection = []
-      each(source, *args){|row| new_collection << block.call(row)}
-      new_collection
+      each(source, *args).collect(&block)
     end
     alias_method :map, :collect
 
     def select(source, *args, &block)
-      new_collection = []
-      each(source, *args){|row| new_collection << row if block.call(row)}
-      new_collection
+      each(source, *args).select(&block)
     end
     alias_method :find_all, :select
 
     def reject(source, *args, &block)
-      new_collection = []
-      each(source, *args){|row| new_collection << row unless block.call(row)}
-      new_collection
+      each(source, *args).reject(&block)
     end
 
     def detect(source, *args, &block)
-      each(source, *args){|row| return row if block.call(row)}
-      nil
+      each(source, *args).detect(&block)
     end
     alias_method :find, :detect
 
@@ -91,7 +85,7 @@ class DSV
         parse(source, *args, &block)
       else
         options = DSV.extract_options(args)
-        new(source, options).read_csv(*args)
+        opened(source, options){|dsv| dsv.read_csv(*args)}
       end
     end
     alias_method :read_csv, :read
@@ -106,25 +100,25 @@ class DSV
     alias_method :parse_csv, :parse
 
     def write(source, *args)
-      new(source, *args).write_csv
+      opened(source, *args, &:write_csv)
     end
     alias_method :write_csv, :write
 
     # The header row as names, or nil where there is none; first_row is the raw line.
     def header_row(source, *args)
-      new(source, *args).attributes
+      opened(source, *args, &:attributes)
     end
 
     def first_row(source, *args)
-      new(source, *args).first_row
+      opened(source, *args, &:first_row)
     end
 
     def attributes(source, *args)
-      new(source, *args).attributes
+      opened(source, *args, &:attributes)
     end
 
     def columns(source, *args)
-      new(source, *args).columns
+      opened(source, *args, &:columns)
     end
 
     def parse_line(raw_row, *args) # For FasterCSV compatibility.
@@ -135,6 +129,16 @@ class DSV
       sc.parse_row(raw_row)
     end
 
+    private
+
+    # A DSV on the source, given to the block and closed after it, an early exit included, where the source is a path or text and so was opened here; an IO given is the caller's to close.
+    def opened(source, *args)
+      dsv = new(source, *args)
+      yield dsv
+    ensure
+      dsv.close if dsv && source.is_a?(::String)
+    end
+
   end # class << self
 
   include Enumerable
@@ -384,6 +388,8 @@ class DSV
   end
 
   def parse_fields(fields, *selected_columns)
+    selected_columns.flatten!
+    return fields.values_at(*positions(selected_columns)) if @as_array && !selected_columns.empty?
     parsed_row = {}
     i = -1
     if selected_columns.empty?
@@ -395,7 +401,6 @@ class DSV
         fields.each{|column_value| store_field(parsed_row, attributes[i += 1], column_value)}
       end
     else
-      selected_columns.flatten!
       case selected_columns[0]
       when Integer
         fields.each{|column_value| parsed_row[i] = column_value unless !selected_columns.include?(i += 1)}
@@ -414,6 +419,11 @@ class DSV
     end
   end
 
+# The positions of the columns selected, in column order, a repeated name giving each of its own.
+def positions(selected_columns)
+  selected_columns.flat_map{|column| column.is_a?(::Integer) ? column : Array((columns || {})[column])}.sort
+end
+
   # With no columns defined, rows keyed by name supply them, and the header, from the first row's keys; rows keyed by position write positionally and have no header to write.
   def write(*selected_columns)
     prepare_to_rewrite if @mode == 'r+'
@@ -488,14 +498,16 @@ class DSV
     if @rows[0]
       if selected_columns.empty?
         @rows.each{|row| yield row}
+      elsif @as_array
+        @rows.each{|row| yield row.values_at(*positions(selected_columns))}
       else
         @rows.each do |row|
           yield selected_columns.inject({}){|hash, column_name| hash[column_name] = row[column_name]; hash}
         end
       end
     else
-      if selected_columns.empty?
-        read_csv.each{|row| yield row}
+      if selected_columns.empty? || @as_array
+        read_csv(selected_columns).each{|row| yield row}
       else
         read_csv(selected_columns).each do |row|
           yield selected_columns.inject({}){|hash, column_name| hash[column_name] = row[column_name]; hash}

data/test/dsv_test.rb

@@ -60,6 +60,14 @@ describe DSV do
     count
   end
 
+  # The files opened while the block runs.
+  def files_opened
+    files = []
+    original = File.method(:new)
+    File.stub(:new, ->(*arguments){original.call(*arguments).tap{|file| files << file}}){yield}
+    files
+  end
+
   describe ".read" do
     it "reads rows keyed by position when there is no header row" do
       _(DSV.read("1,2,3\n4,5,6\n")).must_equal POSITIONAL
@@ -110,6 +118,15 @@ describe DSV do
       _(yielded(:each, DATA, 'a', headers: true)).must_equal [{'a' => '1'}, {'a' => '4'}]
       _(yielded(:parse, DATA, 'b', headers: true)).must_equal [{'b' => '2'}, {'b' => '5'}]
     end
+
+    it "gives only the columns selected under as_array:, in column order" do
+      _(DSV.read(DATA, 'a', headers: true, as_array: true)).must_equal [['1'], ['4']]
+      _(DSV.read(DATA, ['c', 'a'], headers: true, as_array: true)).must_equal [['1', '3'], ['4', '6']]
+      _(DSV.read(DATA, headers: true, as_array: true, selected_columns: ['b'])).must_equal [['2'], ['5']]
+      _(DSV.read("1,2,3\n4,5,6\n", 1, as_array: true)).must_equal [['2'], ['5']]
+      _(DSV.read("a,a,b\n1,2,3\n", 'a', headers: true, as_array: true)).must_equal [['1', '2']]
+      _(yielded(:each, DATA, 'a', headers: true, as_array: true)).must_equal [['1'], ['4']]
+    end
   end
 
   describe ".parse" do
@@ -163,6 +180,51 @@ describe DSV do
     it "detects nil when no row matches" do
       _(DSV.detect(DATA, headers: true){|row| false}).must_be_nil
     end
+
+    it "returns an Enumerator without a block" do
+      [:collect, :select, :reject, :detect].each{|method| _(DSV.send(method, DATA, headers: true)).must_be_instance_of Enumerator}
+      _(DSV.select(DATA, headers: true).each{|row| row['a'] == '4'}).must_equal [KEYED.last]
+      _(DSV.detect(DATA, headers: true).each{|row| row['a'] == '4'}).must_equal KEYED.last
+    end
+  end
+
+  describe "the class methods, given a path" do
+    it "close the file they open" do
+      with_file do |path|
+        [:each, :collect, :select, :reject, :detect, :read].each do |method|
+          _(files_opened{DSV.send(method, path, headers: true){|row| true}}.map(&:closed?)).must_equal [true]
+        end
+        [:read, :header_row, :first_row, :attributes, :columns].each do |method|
+          _(files_opened{DSV.send(method, path, headers: true)}.map(&:closed?)).must_equal [true]
+        end
+        _(files_opened{DSV.write(path, mode: 'r+', headers: true)}.map(&:closed?)).must_equal [true]
+      end
+    end
+
+    it "close the file they open where the block leaves early" do
+      with_file do |path|
+        _(files_opened{DSV.each(path, headers: true){|row| break}}.map(&:closed?)).must_equal [true]
+      end
+    end
+
+    it "return an Enumerator without a block, opening the file afresh on each pass and closing it" do
+      with_file do |path|
+        rows = nil
+        files = files_opened do
+          rows = DSV.each(path, headers: true)
+          _(rows).must_be_instance_of Enumerator
+          _(rows.to_a).must_equal KEYED
+          _(rows.first).must_equal KEYED.first
+        end
+        _(files.map(&:closed?)).must_equal [true, true]
+      end
+    end
+
+    it "leave open an IO given rather than a path" do
+      io = StringIO.new(DATA)
+      DSV.each(io, headers: true){|row| }
+      _(io.closed?).must_equal false
+    end
   end
 
   describe ".header_row, .first_row, .attributes and .columns" do
@@ -362,6 +424,12 @@ describe DSV do
   end
 
   describe "#each" do
+    it "selects from rows already read under as_array:, in column order" do
+      csv = DSV.new(DATA, headers: true, as_array: true)
+      csv.read
+      _(csv.each(['c', 'a']).to_a).must_equal [['1', '3'], ['4', '6']]
+    end
+
     it "yields each row, reading the source first if it has not been read" do
       rows = []
       DSV.new(DATA, headers: true).each{|row| rows << row}

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dsv
 version: !ruby/object:Gem::Version
-  version: 0.12.10
+  version: 0.12.12
 platform: ruby
 authors:
 - thoran