tabreader 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 9f60d622c5ed07603e3ce719cbf1bca92093a1ca
+  data.tar.gz: a751de430625b35d0bd6b9343d2f9b6cd3571f40
+SHA512:
+  metadata.gz: bb921628bb052ae9e5f9ac3ff02db1fe3e578491fab8d222a4b2a6f99a761ba3dbcac0b262a8e91af4a071b99351935b28d670fe6f787614160a314724b872f7
+  data.tar.gz: ab8a161b0d8b72e1eaa7ab226fd31dcd1e719a030de7e7e93fd3547a29dd09c1bca3acb19881f20e282387a41cbe29a76dd29431dd1c90f79c686e068ebe3f57

data/HISTORY.md

@@ -0,0 +1,3 @@
+### 0.0.1 / 2018-08-17
+
+* Everything is new. First release.

data/LICENSE.md

@@ -0,0 +1,116 @@
+CC0 1.0 Universal
+
+Statement of Purpose
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator and
+subsequent owner(s) (each and all, an "owner") of an original work of
+authorship and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for the
+purpose of contributing to a commons of creative, cultural and scientific
+works ("Commons") that the public can reliably and without fear of later
+claims of infringement build upon, modify, incorporate in other works, reuse
+and redistribute as freely as possible in any form whatsoever and for any
+purposes, including without limitation commercial purposes. These owners may
+contribute to the Commons to promote the ideal of a free culture and the
+further production of creative, cultural and scientific works, or to gain
+reputation or greater distribution for their Work in part through the use and
+efforts of others.
+
+For these and/or other purposes and motivations, and without any expectation
+of additional consideration or compensation, the person associating CC0 with a
+Work (the "Affirmer"), to the extent that he or she is an owner of Copyright
+and Related Rights in the Work, voluntarily elects to apply CC0 to the Work
+and publicly distribute the Work under its terms, with knowledge of his or her
+Copyright and Related Rights in the Work and the meaning and intended legal
+effect of CC0 on those rights.
+
+1. Copyright and Related Rights. A Work made available under CC0 may be
+protected by copyright and related or neighboring rights ("Copyright and
+Related Rights"). Copyright and Related Rights include, but are not limited
+to, the following:
+
+  i. the right to reproduce, adapt, distribute, perform, display, communicate,
+  and translate a Work;
+
+  ii. moral rights retained by the original author(s) and/or performer(s);
+
+  iii. publicity and privacy rights pertaining to a person's image or likeness
+  depicted in a Work;
+
+  iv. rights protecting against unfair competition in regards to a Work,
+  subject to the limitations in paragraph 4(a), below;
+
+  v. rights protecting the extraction, dissemination, use and reuse of data in
+  a Work;
+
+  vi. database rights (such as those arising under Directive 96/9/EC of the
+  European Parliament and of the Council of 11 March 1996 on the legal
+  protection of databases, and under any national implementation thereof,
+  including any amended or successor version of such directive); and
+
+  vii. other similar, equivalent or corresponding rights throughout the world
+  based on applicable law or treaty, and any national implementations thereof.
+
+2. Waiver. To the greatest extent permitted by, but not in contravention of,
+applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and
+unconditionally waives, abandons, and surrenders all of Affirmer's Copyright
+and Related Rights and associated claims and causes of action, whether now
+known or unknown (including existing as well as future claims and causes of
+action), in the Work (i) in all territories worldwide, (ii) for the maximum
+duration provided by applicable law or treaty (including future time
+extensions), (iii) in any current or future medium and for any number of
+copies, and (iv) for any purpose whatsoever, including without limitation
+commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes
+the Waiver for the benefit of each member of the public at large and to the
+detriment of Affirmer's heirs and successors, fully intending that such Waiver
+shall not be subject to revocation, rescission, cancellation, termination, or
+any other legal or equitable action to disrupt the quiet enjoyment of the Work
+by the public as contemplated by Affirmer's express Statement of Purpose.
+
+3. Public License Fallback. Should any part of the Waiver for any reason be
+judged legally invalid or ineffective under applicable law, then the Waiver
+shall be preserved to the maximum extent permitted taking into account
+Affirmer's express Statement of Purpose. In addition, to the extent the Waiver
+is so judged Affirmer hereby grants to each affected person a royalty-free,
+non transferable, non sublicensable, non exclusive, irrevocable and
+unconditional license to exercise Affirmer's Copyright and Related Rights in
+the Work (i) in all territories worldwide, (ii) for the maximum duration
+provided by applicable law or treaty (including future time extensions), (iii)
+in any current or future medium and for any number of copies, and (iv) for any
+purpose whatsoever, including without limitation commercial, advertising or
+promotional purposes (the "License"). The License shall be deemed effective as
+of the date CC0 was applied by Affirmer to the Work. Should any part of the
+License for any reason be judged legally invalid or ineffective under
+applicable law, such partial invalidity or ineffectiveness shall not
+invalidate the remainder of the License, and in such case Affirmer hereby
+affirms that he or she will not (i) exercise any of his or her remaining
+Copyright and Related Rights in the Work or (ii) assert any associated claims
+and causes of action with respect to the Work, in either case contrary to
+Affirmer's express Statement of Purpose.
+
+4. Limitations and Disclaimers.
+
+  a. No trademark or patent rights held by Affirmer are waived, abandoned,
+  surrendered, licensed or otherwise affected by this document.
+
+  b. Affirmer offers the Work as-is and makes no representations or warranties
+  of any kind concerning the Work, express, implied, statutory or otherwise,
+  including without limitation warranties of title, merchantability, fitness
+  for a particular purpose, non infringement, or the absence of latent or
+  other defects, accuracy, or the present or absence of errors, whether or not
+  discoverable, all to the greatest extent permissible under applicable law.
+
+  c. Affirmer disclaims responsibility for clearing rights of other persons
+  that may apply to the Work or any use thereof, including without limitation
+  any person's Copyright and Related Rights in the Work. Further, Affirmer
+  disclaims responsibility for obtaining any necessary consents, permissions
+  or other rights required for any use of the Work.
+
+  d. Affirmer understands and acknowledges that Creative Commons is not a
+  party to this document and has no duty or obligation with respect to this
+  CC0 or use of the Work.
+
+For more information, please see
+<http://creativecommons.org/publicdomain/zero/1.0/>

data/Manifest.txt

@@ -0,0 +1,12 @@
+HISTORY.md
+LICENSE.md
+Manifest.txt
+README.md
+Rakefile
+lib/tabreader.rb
+lib/tabreader/reader.rb
+lib/tabreader/version.rb
+test/data/empty.tab
+test/data/test.tab
+test/helper.rb
+test/test_reader.rb

data/README.md

@@ -0,0 +1,237 @@
+# tabreader - read in tabular datafiles in text in the tab format
+
+* home  :: [github.com/datatxt/tabreader](https://github.com/datatxt/tabreader)
+* bugs  :: [github.com/datatxt/tabreader/issues](https://github.com/datatxt/tabreader/issues)
+* gem   :: [rubygems.org/gems/tabreader](https://rubygems.org/gems/tabreader)
+* rdoc  :: [rubydoc.info/gems/tabreader](http://rubydoc.info/gems/tabreader)
+* forum :: [wwwmake](http://groups.google.com/group/wwwmake)
+
+
+## Usage
+
+Yes, it's as simple as:
+
+``` ruby
+line = "1\t2\t3"
+values = line.split( "\t" )
+pp values
+# => ["1","2","3"]
+```
+
+or the "magic" packaged up in `TabReader`:
+
+``` ruby
+line = "1\t2\t3"
+values = TabReader.parse_line( line )
+pp values
+# => ["1","2","3"]
+```
+
+or use the convenience helpers:
+
+``` ruby
+txt <<=TAB
+1\t2\t3
+4\t5\t6
+TAB
+
+records = TabReader.parse( txt )
+pp records
+# => [["1","2","3"],
+#     ["5","6","7"]]
+
+# -or-
+
+records = TabReader.read( "values.tab" )
+pp records
+# => [["1","2","3"],
+#     ["5","6","7"]]
+
+# -or-
+
+TabReader.foreach( "values.tab" ) do |rec|
+  pp rec
+end
+## => ["1","2","3"]
+## => ["5","6","7"]
+```
+
+
+### What about headers?
+
+Use the `TabHashReader`
+if the first line is a header (or if missing pass in the headers
+as an array) and you want your records as hashes instead of arrays of values.
+Example:
+
+``` ruby
+txt <<=TAB
+A\tB\tC
+1\t2\t3
+4\t5\t6
+TAB
+
+records = TabHashReader.parse( txt )
+pp records
+
+# -or-
+
+txt2 <<=TAB
+1\t2\t3
+4\t5\t6
+TAB
+
+records = TabHashReader.parse( txt2, headers: ["A","B","C"] )
+pp records
+
+# => [{"A": "1", "B": "2", "C": "3"},
+#     {"A": "4", "B": "5", "C": "6"}]
+
+# -or-
+
+records = TabHashReader.read( "hash.tab" )
+pp records
+# => [{"A": "1", "B": "2", "C": "3"},
+#     {"A": "4", "B": "5", "C": "6"}]
+
+# -or-
+
+TabHashReader.foreach( "hash.tab" ) do |rec|
+  pp rec
+end
+# => {"A": "1", "B": "2", "C": "3"}
+# => {"A": "4", "B": "5", "C": "6"}
+```
+
+
+
+
+
+## Frequently Asked Questions (FAQ) and Answers
+
+### Q: Why NOT use `CSV.read( col_sep: "\t", quote_char: "∅" )`?
+
+Tab != CSV
+
+The tab format is an (even) simpler format than
+the comma-separated values (CSV) classic format. How?
+
+The tab format has NO escape rules.
+A double quote (`"`) is a double quote (`"`). Example:
+
+```
+"1"→"2"→"3"
+4→5→6
+```
+vs
+
+```
+"1","2","3"
+4,5,6
+```
+
+Turns into `"1", "2", "3"` and `4, 5, 6`
+in tab and `1, 2, 3` and `4, 5, 6` in CSV.
+Note: The surrounding double quotes get stripped in CSV.
+You have to double up double quotes (e.g. `""`)
+for adding "literal" double quotes in CSV:
+
+```
+"""1""","""2""","""3"""
+4,5,"Six says, ""Hello, World!"""
+```
+
+vs
+
+```
+"1"→"2"→"3"
+4→5→Six says, "Hello, World!"
+```
+
+Thus, to avoid any surprises, do NOT use
+`CSV.read( col_sep: "\t", quote_char: "∅" )` and friends for tab.
+Note: Simpler also equals faster :-).
+
+
+
+### Q: What's the tab format?
+
+Let's reprint the (complete) tab spec(ification) right here
+(in an edited simpler version):
+
+A tab file encodes a number of records that may contain multiple fields.
+Each record is represented as a single line.
+Each field value is represented as text.
+Fields in a record are separated from each other by a tab character.
+
+Note that fields that contain tabs are not allowable in this encoding.
+
+Here is a quick grammar in Backus-Naur Form (BNF):
+
+```
+field    ::= [character]+             # multiple characters
+record   ::= field [TAB field]+ EOL   # at least one field, or more
+tab      ::= record+
+```
+
+Example:
+
+```
+Name→Age→Address
+Paul→23→1115 W Franklin
+Bessy the Cow→5→Big Farm Way
+Zeke→45→W Main St
+```
+
+(Source: [Tab Spec @ IANA Media Types](https://www.iana.org/assignments/media-types/text/tab-separated-values))
+
+
+
+
+
+### Q: Why tab?
+
+The tab format works great with
+command line text wrangling / processing tools
+because you can split lines / records on tab
+(with no exceptions or extra escape rules).
+Use classics such as `cut`, `paste`, `sort`, `uniq`, `grep`, `sed`, `awk`  
+and many more.
+
+
+
+
+### Q: Why NOT tab?
+
+Let's quote from the Awesome CSV page:
+
+In theory the tab (`\t`) separator is perfect. Values never use tabs, don't they?  So why hasn't the tab separator taken off?
+
+In practice tab separators are invisible or look like spaces and often you cannot tell if a space is a tab or not.
+
+Thus, tab works great only and only (like space) if your values do NOT use spaces and you treat a tab like a space.
+
+(Source: [Awesome CSV @ CSV v1.1](https://github.com/csv11/awesome-csv))
+
+
+
+## Best of Both Worlds
+
+Use the `csv2tab` tool to convert comma-separated values (CSV) datafiles
+to tab and use the `tab2csv` tool to convert tab datafiles to
+comma-separated values (CSV).
+
+
+
+
+## License
+
+![](https://publicdomainworks.github.io/buttons/zero88x31.png)
+
+The `tabreader` scripts are dedicated to the public domain.
+Use it as you please with no restrictions whatsoever.
+
+## Questions? Comments?
+
+Send them along to the [wwwmake forum](http://groups.google.com/group/wwwmake).
+Thanks!

data/Rakefile

@@ -0,0 +1,26 @@
+require 'hoe'
+require './lib/tabreader/version.rb'
+
+Hoe.spec 'tabreader' do
+
+  self.version = TabReader::VERSION
+
+  self.summary = "tabreader - read in tabular datafiles in text in the tab format"
+  self.description = summary
+
+  self.urls = ['https://github.com/datatext/tabreader']
+
+  self.author = 'Gerald Bauer'
+  self.email = 'wwwmake@googlegroups.com'
+
+  # switch extension to .markdown for gihub formatting
+  self.readme_file = 'README.md'
+  self.history_file = 'HISTORY.md'
+
+  self.licenses = ['Public Domain']
+
+  self.spec_extras = {
+   :required_ruby_version => '>= 2.2.2'
+  }
+
+end

data/lib/tabreader/reader.rb

@@ -0,0 +1,114 @@
+# encoding: utf-8
+
+
+class TabReader
+
+##
+## add more configs - why? why not?
+##   add comments and blanks (skip blank lines) - why? why not?
+##   e.g. comments = '#'
+##        blanks (skip blank lines)
+##        rtrim,ltrim,trim
+
+##
+##  todo: add  converters: e.g. strip (akk trim / ltrim / rtrim )
+
+
+def self.read( path, headers: false )
+  txt = File.open( path, 'r:utf-8' ).read
+	## puts "#{path}:"
+  ## pp txt
+  parse( txt, headers: headers )
+end
+
+def self.parse( txt, headers: false )   ## use parse_rows or parse_lines for array or array results
+   rows = []
+
+   if headers.is_a?( Array )
+     columns = headers
+   else
+     columns = nil    ## header row a.k.a. columns / fields
+   end
+
+   txt.each_line do |line|
+     values = parse_line( line )
+     if headers  ## add values as name/value pairs e.g. array of hashes
+       if columns.nil?
+         columns = values    ## first row is header row
+       else
+         ## note: will cut-off values if values.size > columns.size
+         ##   add warning/error - why? why not?
+         ##  if values.size <= columns.size will get filled-up with nil
+         pairs = columns.zip(values)
+         ## pp pairs
+         h = pairs.to_h
+         ## pp h
+
+         rows << h
+       end
+     else       ## add values as is e.g. array of array
+       rows << values
+     end
+   end
+   rows
+end
+
+def self.foreach( path, headers: false )
+  if headers.is_a?( Array )
+    columns = headers
+  else
+    columns = nil    ## header row a.k.a. columns / fields
+  end
+
+	File.open( path, 'r:utf-8' ).each_line do |line|
+		pp line
+		values = parse_line( line )
+		if headers  ## add values as name/value pairs e.g. array of hashes
+			if columns.nil?
+				columns = values    ## first row is header row
+			else
+				pairs = columns.zip(values)
+				h = pairs.to_h
+				yield( h )
+			end
+		else       ## add values as is e.g. array of array
+			yield( values )
+		end
+	end
+
+	# return nil
+	nil
+end
+
+
+def self.parse_line( line )
+  ## check - can handle comments and blank lines too - why? why not?
+  ## remove trailing newlines
+
+  ##  note: chomp('') if is an empty string,
+  ##    it will remove all trailing newlines from the string.
+	##    use line.sub(/[\n\r]*$/, '') or similar instead - why? why not?
+  line = line.chomp('')
+
+  values = line.split("\t")
+  values
+end
+
+def self.header( path )
+  line =  File.open( path, 'r:utf-8' ) do |f|
+     if f.eof?
+			 ## handle empty file; return empty string; no readline call possible
+			 ##  todo/check: return nil from header is no header or [] - why? why not?
+			 ##   or throw exception  end of file reached (EOFError) - why? why not?
+			  ""
+		 else
+			 f.readline
+		 end
+  end
+
+  ## note: line includes \n or \r\n at the end
+  ## pp line
+  parse_line( line )
+end
+
+end # class TabReader

data/lib/tabreader/version.rb

@@ -0,0 +1,24 @@
+# encoding: utf-8
+
+
+## note: for now TabReader is a class!!! NOT a module - change - why? why not?
+class TabReader
+
+  MAJOR = 0    ## todo: namespace inside version or something - why? why not??
+  MINOR = 0
+  PATCH = 1
+  VERSION = [MAJOR,MINOR,PATCH].join('.')
+
+  def self.version
+    VERSION
+  end
+
+  def self.banner
+    "tabreader/#{VERSION} on Ruby #{RUBY_VERSION} (#{RUBY_RELEASE_DATE}) [#{RUBY_PLATFORM}]"
+  end
+
+  def self.root
+    File.expand_path( File.dirname(File.dirname(File.dirname(__FILE__))) )
+  end
+
+end # class TabReader

data/lib/tabreader.rb

@@ -0,0 +1,16 @@
+# encoding: utf-8
+
+
+require 'pp'
+
+
+###
+# our own code
+require 'tabreader/version' # let version always go first
+require 'tabreader/reader'
+
+
+
+
+
+puts TabReader.banner   # say hello

data/test/data/test.tab

@@ -0,0 +1,5 @@
+a	b	c
+1	2	3
+4	5	6
+{"one":1,"two":2,[3,4,5],null}	7	8
+test newline \n and tab \t literals	9	10

data/test/helper.rb

@@ -0,0 +1,16 @@
+## $:.unshift(File.dirname(__FILE__))
+
+## minitest setup
+
+require 'minitest/autorun'
+
+
+## our own code
+require 'tabreader'
+
+## add test_data_dir helper
+class TabReader
+  def self.test_data_dir
+    "#{root}/test/data"
+  end
+end

data/test/test_reader.rb

@@ -0,0 +1,80 @@
+# encoding: utf-8
+
+###
+#  to run use
+#     ruby -I ./lib -I ./test test/test_reader.rb
+
+
+require 'helper'
+
+class TestReader < MiniTest::Test
+
+
+def test_parse
+
+txt1 = <<TXT
+a\tb\tc
+1\t2\t3
+4\t5\t6
+TXT
+
+txt2 = <<TXT
+a	b	c	d
+1	2	3	4
+5	6	7	8
+TXT
+
+puts "== parse:"
+pp TabReader.parse( txt1 )
+pp TabReader.parse( txt1, headers: true )
+
+puts "== parse:"
+pp TabReader.parse( txt2 )
+pp TabReader.parse( txt2, headers: true )
+
+puts "== parse_line:"
+pp TabReader.parse_line( "1\t2\t3" )
+
+puts "== parse_line:"
+pp TabReader.parse_line( "1	2	3	4" )
+
+puts "== parse_line:"
+pp TabReader.parse_line( "1\t2\t3\r\n" )
+
+  assert true
+end
+
+
+def test_read
+
+puts "== read:"
+pp TabReader.read( "#{TabReader.test_data_dir}/test.tab" )
+pp TabReader.read( "#{TabReader.test_data_dir}/test.tab", headers: true )
+puts "== header:"
+pp TabReader.header( "#{TabReader.test_data_dir}/test.tab" )
+puts "== foreach:"
+TabReader.foreach( "#{TabReader.test_data_dir}/test.tab" ) do |row|
+  pp row
+end
+TabReader.foreach( "#{TabReader.test_data_dir}/test.tab", headers: true ) do |row|
+  pp row
+end
+end
+
+
+def test_read_empty
+
+puts "== read (empty):"
+pp TabReader.read( "#{TabReader.test_data_dir}/empty.tab" )
+puts "== header (empty):"
+pp TabReader.header( "#{TabReader.test_data_dir}/empty.tab" )
+puts "== foreach (empty):"
+TabReader.foreach( "#{TabReader.test_data_dir}/empty.tab" ) do |row|
+	pp row
+end
+puts "== parse (empty):"
+pp TabReader.parse( "" )
+pp TabReader.parse_line( "" )
+end
+
+end

metadata

@@ -0,0 +1,89 @@
+--- !ruby/object:Gem::Specification
+name: tabreader
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+platform: ruby
+authors:
+- Gerald Bauer
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2018-08-17 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rdoc
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.0'
+- !ruby/object:Gem::Dependency
+  name: hoe
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.16'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.16'
+description: tabreader - read in tabular datafiles in text in the tab format
+email: wwwmake@googlegroups.com
+executables: []
+extensions: []
+extra_rdoc_files:
+- HISTORY.md
+- LICENSE.md
+- Manifest.txt
+- README.md
+files:
+- HISTORY.md
+- LICENSE.md
+- Manifest.txt
+- README.md
+- Rakefile
+- lib/tabreader.rb
+- lib/tabreader/reader.rb
+- lib/tabreader/version.rb
+- test/data/empty.tab
+- test/data/test.tab
+- test/helper.rb
+- test/test_reader.rb
+homepage: https://github.com/datatext/tabreader
+licenses:
+- Public Domain
+metadata: {}
+post_install_message: 
+rdoc_options:
+- "--main"
+- README.md
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.2.2
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.5.2
+signing_key: 
+specification_version: 4
+summary: tabreader - read in tabular datafiles in text in the tab format
+test_files: []