dsv 0.12.9 → 0.12.11

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7171a2d2de4dba26f3e304ca3e0c216e9bb093a36cef1420629e9f936e644889
-  data.tar.gz: 4ccf92f6000c7e5b9d82315babc17c816edd3df7587583eb79f2c16eb19125cb
+  metadata.gz: 12c41912ef12924ca7177b91aa6d532d4acff23b68073fea18172990fc41cd54
+  data.tar.gz: 553305fef6d5b78bea65c39b99f8e72e6dbab070352978c333ce9c277d63dfdc
 SHA512:
-  metadata.gz: 4727efe3e9571299914d4b6731169ac5a70eabd475176f06cb084dba35cfcc038420faf4740ad01bbedb9caea1b13ba21ed40a058f2699ca6f4fedf9ba47a0b1
-  data.tar.gz: 6b6143f85e9c6106344fed384228433057eff0641a38eff9e6f823a2d905cca416284b7068f80ff26529feb02045fe09f323e689614012bd267d0783aec469ba
+  metadata.gz: d89e956eebebc2e15f6e3374bd39e344de12b9999a551db07e15201aefe0921ec4aa62fde5f8f259f5b3b79a6e3a60b9833c9641380913bc1c202036a832ff2a
+  data.tar.gz: 2e2f17f71e7651025b05378e90436567ff402870ff815e286c9dd777c11c773d84f249383ff8e4a1b0f817b6d70281c1d90de169970bba2a28b10c5493227af6

data/CHANGELOG

@@ -1,5 +1,25 @@
 # dsv/CHANGELOG
 
+## 20261103
+
+0.12.11: the class methods close the files they open, and return an Enumerator without a block.
+
+1. ~ DSV.each, .read, .write, .header_row, .first_row, .attributes, .columns: + opened(), so that a path or text given is closed after the call, an early exit included; an IO given is left open. Each had left its file open.
+2. ~ DSV.each: without a block, returns an Enumerator which opens the source afresh on each pass.
+3. ~ DSV.collect, .select, .reject, .detect: by way of the Enumerator. Without a block each had raised NoMethodError.
+4. ~ test/dsv_test.rb: + files_opened(); + the class methods closing what they open, on an early exit too, and leaving an IO open; + the Enumerators.
+5. ~ README.md: Enumerating from the class, closing and the Enumerator.
+6. ~ TODO: - 1.
+7. ~ lib/DSV/VERSION.rb: /0.12.10/0.12.11/.
+
+0.12.10: ~ TODO, now grouped by release; + ROADMAP.md.
+
+1. ~ TODO: the jobs decided, by release: the fixes left for 0.12.x; modes as declarations in 0.13.0; streaming in 0.14.0; what 1.0 needs; speed and new options in 1.x.
+2. ~ TODO: 11, strict:, and 15, the malformed-quoting option, superseded by field_count: and :rfc_4180; 13, chunks, by a streaming each; 1, 6, 7 and the undecided parts of 2, to ROADMAP.md.
+3. + ROADMAP.md: the design, the boundary for 1.0, the version rule, the release sequence, and the themes after 1.0 with what is undecided in each.
+4. ~ lib/DSV/VERSION.rb: /0.12.9/0.12.10/.
+
+
 ## 20261028
 
 0.12.9: + DSV::UnterminatedQuote, raised where a quoted field does not close.

data/README.md

@@ -64,7 +64,7 @@ DSV.detect('people.csv', headers: true){|row| row['name'] == 'Ada'}
 DSV.collect('people.csv', headers: true){|row| row['name'].upcase}
 ```
 
-`foreach`, `map`, `find_all` and `find` are aliases.
+`foreach`, `map`, `find_all` and `find` are aliases. Each closes the file it opens. Without a block, each returns an Enumerator, which opens the file again on each pass, so `DSV.each('people.csv', headers: true).each_slice(100)` works as it would on any Enumerable.
 
 ### Any delimiter
 

data/TODO

@@ -1,25 +1,45 @@
 # TODO
 
-## 0.12.x, before the gem is published
+## 0.12.x, fixes
 
-1. A path is told from content by the call, not by a guess: .read, .foreach, .open and the other path-taking class methods take a path, .parse takes text, new keeps the guess with a switch to say which, and a path under a read mode that names no file raises rather than being read as text.
-2. Selection as a keyword on the calls, only: or select:, columns: being taken; the positional form kept or retired; selected_columns: renamed to match.
-3. The instance-level header_row accessor renamed headers?, the class-level header_row now being the row.
-4. << to append a row and generate to build a string, the two writing calls a CSV user reaches for first.
+1. as_array: with a column selection: each('name') no longer raises TypeError, and read('name', as_array: true) gives only the columns selected. as_array: in the README, as the way to get Arrays as CSV gives them.
+2. Reading honours quote: given as a String, as writing does; DSV.parse_line honours quote:.
+3. A space as the column separator splits on each space, not as String#split(' ') does.
+4. :double back to 0.12.4's reading, undoing 0.12.5 item 2: a row not starting with a quote split plainly, one starting with a quote split on quote-separator-quote, and one which leaves a lone quote in a field read in full as wholly quoted. The README back to 0.12.3's wording.
+5. :single read, as :double is with a single quote.
+6. filter, a class-level alias of select.
+7. A String under a write mode taken as a path, so that a file not yet there can be written by name.
+8. The internals private.
 
-## 0.13.0, speed
+## 0.13.0, modes as declarations
 
-5. The saved-up speed change, with the evidence in the reconciliation: two string predicates in place of four regex matches per piece in split_csv; rows built by zip.to_h; a benchmark script in the repository against stdlib CSV, and the README's speed sentence written from its output. The scanner-based parser on the branch scanner-parser is judged here.
+9. An unknown quote mode, file mode or option raises ArgumentError, where each is ignored today.
+10. :mixed, the name of the default.
+11. A mode raises on a contradiction which costs nothing to see, every row alike, header included: under :double, a row not starting with a quote, and a bare field in a row read in full. Declaring :mixed reads such a row.
 
-## Features, after
+## 0.14.0, streaming
 
-6. Guessing, as a mode that runs once at construction on a sample of the first rows and feeds the same chooser as an explicit option, scanner only: the column separator, the one whose count is the same on every sampled row; whether the first row is a header, its columns' types differing from the rows beneath in the same way across columns; and, where there is no header, a header made from the data, each column named for what its values look like, integer, decimal, date, email, URL, phone, with a positional fallback, column_0, for text and a suffix where two columns share a class. Names come from what values look like, not what they mean; anything more is a schema, and another library's.
-7. Templated ingestion: a template: option taking a whole-row Regexp with a capture per field, the group names being the column names, one match per row; an Array of separators as its writable special case, compiled into such a Regexp at construction and invertible for writing where a general template is not.
-8. The duplicate-header modes as an option: keep all as an Array, the default; index, a, a.1; first; last; error. All derivations of the same columns map at header time, none costing anything per row.
-9. Writing options: minimal quoting, a field quoted only where it holds a separator, a quote or a row terminator; a row terminator inside a field kept, quoted, or folded to a space; the encoding declared and a transcode offered with any loss marked.
-10. Symbols and Strings interchangeable for column names, in a selection, a columns: list and a row's keys; today a String is required.
-11. A strict: option, off by default, that raises on a row with more or fewer fields than the header; today a short row gets only the columns it has and a long row's excess lands under nil.
-12. A line number in each parse error, found only when the error is raised, from the source's position at that moment, as UnterminatedQuote finds its; never counted as it goes, since counting costs time on every row for the sake of the rare one.
-13. Chunked processing of a large file: a bounded number of rows at a time handed to a block, so that nothing needs the whole file in memory and a partial result can be acted on before the rest is read.
-14. Checks, all of them after the above: strict: (13), column types, and whatever else proves wanted; correct and published comes first.
-15. The rule for malformed quoting as an option, its name to be settled, perhaps one with strict: (11): the default, as now; a stricter one, under which a quote that does not open a field raises too, and :double raises on a row which is not what it promises rather than reading it as unspecified quoting does; and a lenient one, never raising, text after a closing quote kept and a field which does not close running to the end of the source.
+12. each yields each row as it is read and returns its receiver, so that detect and break stop reading and memory does not grow with the file; rows is no longer filled by each. Chunks come from each_slice. A source read once, start to end, with no rewind, so that a pipe or $stdin can be read: the header from the first row read, and the line of an error found by seeking only where the source can; a rewind left to r+ rewriting and first_row called alone.
+
+## 1.0
+
+13. A corpus in the tests, a day of NEMWEB's dispatch files trimmed and the RFC 4180 cases, each checked against stdlib CSV's rows.
+14. A benchmark script against stdlib CSV, and the README's speed sentence written from its output.
+15. README: every mode by example, and a table matching each to stdlib CSV's and Python's settings.
+
+## 1.x, speed, results unchanged
+
+16. A per-row fast path for the default: split, the fields bare or wrapped in two quotes unwrapped, any other row read in full.
+17. as_array: returning the split fields, rather than building a Hash and then an Array from it.
+18. The splitter chosen once, at construction, as row_renderer is for writing.
+19. Two string predicates in place of four regex matches per piece; rows built by zip.to_h; the scanner-based parser, at the tag scanner-parser-20261006, judged against the rest.
+
+## 1.x, features
+
+20. :rfc_4180, a quote mode reading what RFC 4180 allows as :mixed does and raising on what it forbids.
+21. field_count:, :header or an Integer, off by default: a row of another width raises.
+22. Writing: minimal quoting, a field quoted only where it holds a separator, a quote or a row terminator; a row terminator inside a field kept, quoted, or folded to a space; the encoding declared and a transcode offered with any loss marked.
+23. A line number in each parse error, found only when the error is raised, as UnterminatedQuote finds its, where the source can seek. line_numbers:, off by default, counts lines as it reads instead, for any source a pipe included: the read loop chosen once, at construction, as row_renderer is for writing, so that without it no row pays for the count.
+24. The instance-level header_row renamed headers?, kept as an alias; << to append a row and generate to build a string.
+25. Symbols and Strings interchangeable for column names, in a selection, a columns: list and a row's keys.
+26. The duplicate-header modes as an option: keep all as an Array, the default; index, a, a.1; first; last; error.

data/lib/DSV/VERSION.rb

@@ -2,5 +2,5 @@
 # DSV::VERSION
 
 class DSV
-  VERSION = '0.12.9'
+  VERSION = '0.12.11'
 end

data/lib/dsv.rb

@@ -53,36 +53,30 @@ class DSV
       end
     end
 
-    # The class-level read, parse and each take a column selection before the options, as the instance-level ones take it as arguments.
+    # The class-level read, parse and each take a column selection before the options, as the instance-level ones take it as arguments. Each closes what it opens; without a block, each returns an Enumerator which opens the source afresh on each pass.
     def each(source, *args, &block)
+      return to_enum(:each, source, *args) unless block
       options = DSV.extract_options(args)
-      new(source, options).each(*args, &block)
+      opened(source, options){|dsv| dsv.each(*args, &block)}
     end
     alias_method :foreach, :each
 
     def collect(source, *args, &block)
-      new_collection = []
-      each(source, *args){|row| new_collection << block.call(row)}
-      new_collection
+      each(source, *args).collect(&block)
     end
     alias_method :map, :collect
 
     def select(source, *args, &block)
-      new_collection = []
-      each(source, *args){|row| new_collection << row if block.call(row)}
-      new_collection
+      each(source, *args).select(&block)
     end
     alias_method :find_all, :select
 
     def reject(source, *args, &block)
-      new_collection = []
-      each(source, *args){|row| new_collection << row unless block.call(row)}
-      new_collection
+      each(source, *args).reject(&block)
     end
 
     def detect(source, *args, &block)
-      each(source, *args){|row| return row if block.call(row)}
-      nil
+      each(source, *args).detect(&block)
     end
     alias_method :find, :detect
 
@@ -91,7 +85,7 @@ class DSV
         parse(source, *args, &block)
       else
         options = DSV.extract_options(args)
-        new(source, options).read_csv(*args)
+        opened(source, options){|dsv| dsv.read_csv(*args)}
       end
     end
     alias_method :read_csv, :read
@@ -106,25 +100,25 @@ class DSV
     alias_method :parse_csv, :parse
 
     def write(source, *args)
-      new(source, *args).write_csv
+      opened(source, *args, &:write_csv)
     end
     alias_method :write_csv, :write
 
     # The header row as names, or nil where there is none; first_row is the raw line.
     def header_row(source, *args)
-      new(source, *args).attributes
+      opened(source, *args, &:attributes)
     end
 
     def first_row(source, *args)
-      new(source, *args).first_row
+      opened(source, *args, &:first_row)
     end
 
     def attributes(source, *args)
-      new(source, *args).attributes
+      opened(source, *args, &:attributes)
     end
 
     def columns(source, *args)
-      new(source, *args).columns
+      opened(source, *args, &:columns)
     end
 
     def parse_line(raw_row, *args) # For FasterCSV compatibility.
@@ -135,6 +129,16 @@ class DSV
       sc.parse_row(raw_row)
     end
 
+    private
+
+    # A DSV on the source, given to the block and closed after it, an early exit included, where the source is a path or text and so was opened here; an IO given is the caller's to close.
+    def opened(source, *args)
+      dsv = new(source, *args)
+      yield dsv
+    ensure
+      dsv.close if dsv && source.is_a?(::String)
+    end
+
   end # class << self
 
   include Enumerable

data/test/dsv_test.rb

@@ -60,6 +60,14 @@ describe DSV do
     count
   end
 
+  # The files opened while the block runs.
+  def files_opened
+    files = []
+    original = File.method(:new)
+    File.stub(:new, ->(*arguments){original.call(*arguments).tap{|file| files << file}}){yield}
+    files
+  end
+
   describe ".read" do
     it "reads rows keyed by position when there is no header row" do
       _(DSV.read("1,2,3\n4,5,6\n")).must_equal POSITIONAL
@@ -163,6 +171,51 @@ describe DSV do
     it "detects nil when no row matches" do
       _(DSV.detect(DATA, headers: true){|row| false}).must_be_nil
     end
+
+    it "returns an Enumerator without a block" do
+      [:collect, :select, :reject, :detect].each{|method| _(DSV.send(method, DATA, headers: true)).must_be_instance_of Enumerator}
+      _(DSV.select(DATA, headers: true).each{|row| row['a'] == '4'}).must_equal [KEYED.last]
+      _(DSV.detect(DATA, headers: true).each{|row| row['a'] == '4'}).must_equal KEYED.last
+    end
+  end
+
+  describe "the class methods, given a path" do
+    it "close the file they open" do
+      with_file do |path|
+        [:each, :collect, :select, :reject, :detect, :read].each do |method|
+          _(files_opened{DSV.send(method, path, headers: true){|row| true}}.map(&:closed?)).must_equal [true]
+        end
+        [:read, :header_row, :first_row, :attributes, :columns].each do |method|
+          _(files_opened{DSV.send(method, path, headers: true)}.map(&:closed?)).must_equal [true]
+        end
+        _(files_opened{DSV.write(path, mode: 'r+', headers: true)}.map(&:closed?)).must_equal [true]
+      end
+    end
+
+    it "close the file they open where the block leaves early" do
+      with_file do |path|
+        _(files_opened{DSV.each(path, headers: true){|row| break}}.map(&:closed?)).must_equal [true]
+      end
+    end
+
+    it "return an Enumerator without a block, opening the file afresh on each pass and closing it" do
+      with_file do |path|
+        rows = nil
+        files = files_opened do
+          rows = DSV.each(path, headers: true)
+          _(rows).must_be_instance_of Enumerator
+          _(rows.to_a).must_equal KEYED
+          _(rows.first).must_equal KEYED.first
+        end
+        _(files.map(&:closed?)).must_equal [true, true]
+      end
+    end
+
+    it "leave open an IO given rather than a path" do
+      io = StringIO.new(DATA)
+      DSV.each(io, headers: true){|row| }
+      _(io.closed?).must_equal false
+    end
   end
 
   describe ".header_row, .first_row, .attributes and .columns" do

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dsv
 version: !ruby/object:Gem::Version
-  version: 0.12.9
+  version: 0.12.11
 platform: ruby
 authors:
 - thoran