fat_table 0.5.4 → 0.5.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0ecb8cb3fb29acb95b0fe11e65c5b587379a5a3070ab68acbc6a2c1785c8f229
-  data.tar.gz: f3cc9e7242ded9fd0970159eee8da4208faade0e6e4c37a324a45e1596c4eeae
+  metadata.gz: 2f391d5e4ad9d7a4dcb303098b90f6fb42379d9df382192c4bacaf169523dab1
+  data.tar.gz: ea75f906fcd164752a3ca2220324ad10d8438562bb31c4846bc6575723e834fa
 SHA512:
-  metadata.gz: 8d404bd69ba3a1c3c9774f634e4f0935987ff4827803d59e89b9adbdc4806b25ef87e5dbcb9f22d7e3475c86dceebf097cf73e0ad1287f91d21415e5b0a5a3b3
-  data.tar.gz: 8d8df3651b9cdcdeaefa0527cd4fc1987c1ec491bca2a88651b255032eea16ee29c82285b9d7f9bae2ecb9b6d90c6fc4d766980af5362dc89045d553e66dec03
+  metadata.gz: 2914c40a497cf24bcdba8868c4a4607ea1bdb0a94fba8caa3d6e31407ff6832621df2a0b4a8a2bb98c3b40080fed6d752f8e936f66a96270fe675e93d5a852d6
+  data.tar.gz: dbc43f4d2bac4a2a19bc7cc90d1fbfbf739a12f0f280a543670af588cfcf2bb0df74d50cfb7ded76b851f5470fd30d4f971659cead5131f9ea15e24b492e5823

data/README.org

@@ -32,7 +32,7 @@ The following is for org.
 #+end_src
 
 #+begin_EXAMPLE
-Current version is: 0.5.3
+Current version is: 0.5.4
 #+end_EXAMPLE
 
 * Introduction
@@ -442,11 +442,26 @@ or nil. There are only five permissible types for a ~Column~:
 5. *NilClass* (for the undetermined column type).
 
 When a ~Table~ is constructed from an external source, all ~Columns~ start out
-having a type of ~NilClass~, that is, their type is as yet undetermined. When a
-string or object of one of the four determined types is added to a ~Column~, it
-fixes the type of the column and all further items added to the ~Column~ must
-either be ~nil~ (indicating no value) or be capable of being coerced to the
-column's type. Otherwise, ~FatTable~ raises an exception.
+having a type of ~NilClass~, that is, their type is as yet undetermined. When
+a string or object of one of the four determined types is added to a ~Column~
+and it can be converted into one of the permissible types, it fixes the type
+of the column, and all further items added to the ~Column~ must either be
+~nil~ (indicating no value) or be capable of being coerced to the column's
+type. Otherwise, ~FatTable~ raises an exception.
+
+The strictness of requiring all items to be of the same type can be relaxed by
+declaring a column to be "tolerant."  You can do so when you create the table
+by adding a tolerant_columns keyword parameter.  If a Column is tolerant,
+~FatTable~ tries to convert new items into a type other than a ~String~ and,
+if it can do so, sets /that/ as the Column's type.  Any later items that
+cannot be converted into the Column's type are converted to strings.  These
+interloper strings are treated like nils for purposes of sorting and
+evaluation, but are displayed according to any string formatting on output.
+See [[*Designating "Tolerant" Columns][Designating "Tolerant" Columns]] below.
+
+It is also possible to force ~FatTable~ to treat a column as a String type,
+even its items look like one of the other types.  See [[*Forcing String Type][Forcing String Type]]
+below.
 
 Items of input must be either one of the permissible ruby objects or strings. If
 they are strings, ~FatTable~ attempts to parse them as one of the permissible
@@ -603,10 +618,18 @@ columns to be created:
 **** Forcing String Type
 Occasionally, ~FatTable~'s automatic type detection can get in the way and you
 just want it to treat one or more columns as Strings regardless of their
-appearance.  Think, for example, of zip codes.  At any time after creating a
-table, you can have it force the String type on any number of columns with the
-~force_string!~ method.  When you do so, all exisiting items in the column are
-converted to strings with the #to_s method.
+appearance.  Think, for example, of zip codes.  If headers are given when a
+table is contructed, you can designate a forced-string column by appending a
+~!~ to the end of the header.  It will not become part of the header, it will
+just mark it as a forced-string Column.
+
+#+begin_SRC emacs-lisp :wrap EXAMPLE
+  tab = FatTable.new(:a, 'b', 'C!', :d, :zip!)
+#+end_SRC
+
+In addition, at any time after creating a table, you can force the String type
+on any number of columns with the ~force_string!~ method.  When you do so, all
+exisiting items in the column are converted to strings with the #to_s method.
 
 #+begin_src ruby :wrap EXAMPLE
   tab = FatTable.new(:a, 'b', 'C', :d, :zip)

data/TODO.org

@@ -1,10 +1,37 @@
+
+* TODO Specify Column Widths
+Allow a formatter to specify column widths.  This could be a number of
+characters, which would be interpreted as a number of "ems" for LaTeX.
+Cell content larger than the width would be truncated.  Any column without a
+width specified would be set at the width of the longest value in that cell,
+after initial formatting.
+
+#+begin_SRC ruby
+  tab.to_text do |f|
+    f.widths(a: 13, b: 30)
+  end
+#+end_SRC
+
+Possible enhancements:
+- specify an overall width and column widths as decimal or fractions, so that
+  a column's width would be that fraction of the overall width.
+- specify a Range for a width, so that the column would at least min and at
+  most max, otherwise the width of its largest cell.
+
 * TODO Conversion to Spreadsheets
 - State "TODO"       from              [2017-04-21 Fri 10:36]
 This is a [[https://github.com/westonganger/spreadsheet_architect][gem]] that I can include into the Table model to convert a table into
 a spread-sheet, or even a sheet in a multi-sheet spreadsheet file.
 
-* TODO Add from_yql for fetching from Yahoo
+* TODO Add Quandl or EODDATA Queries
+Possible replacements for YQL.
+
+* CNCL Add from_yql for fetching from Yahoo
+CLOSED: [2022-01-30 Sun 06:03]
 - State "TODO"       from              [2017-04-21 Fri 10:35]
+
+Cancelled because Yahoo shut down the YQL api service.
+
 Add a constructor to allow fetching stock data from yql.  Perhaps grab all
 available fields, then allow a select of those of interest.
 

data/lib/fat_table/column.rb

@@ -191,8 +191,11 @@ module FatTable
 
     # :category: Aggregates
 
-    # Return the first non-nil item in the Column.  Works with any Column type.
+    # Return the first non-nil item in the Column, or nil if all items are
+    # nil.  Works with any Column type.
     def first
+      return nil if items.all?(&:nil?)
+
       if type == 'String'
         items.reject(&:blank?).first
       else
@@ -204,6 +207,8 @@ module FatTable
 
     # Return the last non-nil item in the Column.  Works with any Column type.
     def last
+      return nil if items.all?(&:nil?)
+
       if type == 'String'
         items.reject(&:blank?).last
       else
@@ -213,9 +218,11 @@ module FatTable
 
     # :category: Aggregates
 
-    # Return a count of the non-nil items in the Column.  Works with any Column
-    # type.
+    # Return a count of the non-nil items in the Column, or the size of the
+    # column if all items are nil.  Works with any Column type.
     def count
+      return items.size if items.all?(&:nil?)
+
       if type == 'String'
         items.reject(&:blank?).count.to_d
       else
@@ -225,8 +232,8 @@ module FatTable
 
     # :category: Aggregates
 
-    # Return the smallest non-nil, non-blank item in the Column.  Works with
-    # numeric, string, and datetime Columns.
+    # Return the smallest non-nil, non-blank item in the Column, or nil if all
+    # items are nil.  Works with numeric, string, and datetime Columns.
     def min
       only_with('min', 'NilClass', 'Numeric', 'String', 'DateTime')
       if type == 'String'
@@ -238,8 +245,8 @@ module FatTable
 
     # :category: Aggregates
 
-    # Return the largest non-nil, non-blank item in the Column.  Works with
-    # numeric, string, and datetime Columns.
+    # Return the largest non-nil, non-blank item in the Column, or nil if all
+    # items are nil.  Works with numeric, string, and datetime Columns.
     def max
       only_with('max', 'NilClass', 'Numeric', 'String', 'DateTime')
       if type == 'String'
@@ -251,19 +258,24 @@ module FatTable
 
     # :category: Aggregates
 
-    # Return a Range object for the smallest to largest value in the column.
-    # Works with numeric, string, and datetime Columns.
+    # Return a Range object for the smallest to largest value in the column,
+    # or nil if all items are nil.  Works with numeric, string, and datetime
+    # Columns.
     def range
       only_with('range', 'NilClass', 'Numeric', 'String', 'DateTime')
+      return nil if items.all?(&:nil?)
+
       Range.new(min, max)
     end
 
     # :category: Aggregates
 
-    # Return the sum of the non-nil items in the Column.  Works with numeric and
-    # string Columns. For a string Column, it will return the concatenation of
-    # the non-nil items.
+    # Return the sum of the non-nil items in the Column, or 0 if all items are
+    # nil.  Works with numeric and string Columns. For a string Column, it
+    # will return the concatenation of the non-nil items.
     def sum
+      return 0 if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('sum', 'Numeric', 'String')
       if type == 'String'
         items.reject(&:blank?).join(' ')
@@ -274,11 +286,13 @@ module FatTable
 
     # :category: Aggregates
 
-    # Return the average value of the non-nil items in the Column.  Works with
-    # numeric and datetime Columns.  For datetime Columns, it converts each date
-    # to its Julian day number, computes the average, and then converts the
-    # average back to a DateTime.
+    # Return the average value of the non-nil items in the Column, or 0 if all
+    # items are nil.  Works with numeric and datetime Columns.  For datetime
+    # Columns, it converts each date to its Julian day number, computes the
+    # average, and then converts the average back to a DateTime.
     def avg
+      return 0 if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('avg', 'DateTime', 'Numeric')
       itms = items.filter_to_type(type)
       size = itms.size.to_d
@@ -293,11 +307,14 @@ module FatTable
     # :category: Aggregates
 
     # Return the sample variance (the unbiased estimator of the population
-    # variance using a divisor of N-1) as the average squared deviation from the
-    # mean, of the non-nil items in the Column. Works with numeric and datetime
-    # Columns. For datetime Columns, it converts each date to its Julian day
-    # number and computes the variance of those numbers.
+    # variance using a divisor of N-1) as the average squared deviation from
+    # the mean, of the non-nil items in the Column, or 0 if all items are
+    # nil. Works with numeric and datetime Columns. For datetime Columns, it
+    # converts each date to its Julian day number and computes the variance of
+    # those numbers.
     def var
+      return 0 if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('var', 'DateTime', 'Numeric')
       all_items =
         if type == 'DateTime'
@@ -319,10 +336,13 @@ module FatTable
 
     # Return the population variance (the biased estimator of the population
     # variance using a divisor of N) as the average squared deviation from the
-    # mean, of the non-nil items in the Column. Works with numeric and datetime
-    # Columns. For datetime Columns, it converts each date to its Julian day
-    # number and computes the variance of those numbers.
+    # mean, of the non-nil items in the Column, or 0 if all items are
+    # nil. Works with numeric and datetime Columns. For datetime Columns, it
+    # converts each date to its Julian day number and computes the variance of
+    # those numbers.
     def pvar
+      return 0 if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('var', 'DateTime', 'Numeric')
       n = items.filter_to_type(type).size.to_d
       return BigDecimal('0.0') if n <= 1
@@ -333,11 +353,13 @@ module FatTable
 
     # Return the sample standard deviation (the unbiased estimator of the
     # population standard deviation using a divisor of N-1) as the square root
-    # of the sample variance, of the non-nil items in the Column. Works with
-    # numeric and datetime Columns. For datetime Columns, it converts each date
-    # to its Julian day number and computes the standard deviation of those
-    # numbers.
+    # of the sample variance, of the non-nil items in the Column, or 0 if all
+    # items are nil. Works with numeric and datetime Columns. For datetime
+    # Columns, it converts each date to its Julian day number and computes the
+    # standard deviation of those numbers.
     def dev
+      return 0 if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('dev', 'DateTime', 'Numeric')
       var.sqrt(20)
     end
@@ -345,12 +367,14 @@ module FatTable
     # :category: Aggregates
 
     # Return the population standard deviation (the biased estimator of the
-    # population standard deviation using a divisor of N) as the square root of
-    # the population variance, of the non-nil items in the Column. Works with
-    # numeric and datetime Columns. For datetime Columns, it converts each date
-    # to its Julian day number and computes the standard deviation of those
-    # numbers.
+    # population standard deviation using a divisor of N) as the square root
+    # of the population variance, of the non-nil items in the Column, or 0 if
+    # all items are nil. Works with numeric and datetime Columns. For datetime
+    # Columns, it converts each date to its Julian day number and computes the
+    # standard deviation of those numbers.
     def pdev
+      return 0 if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('dev', 'DateTime', 'Numeric')
       Math.sqrt(pvar)
     end
@@ -358,8 +382,10 @@ module FatTable
     # :category: Aggregates
 
     # Return true if any of the items in the Column are true; otherwise return
-    # false.  Works only with boolean Columns.
+    # false, or false if all items are nil.  Works only with boolean Columns.
     def any?
+      return false if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('any?', 'Boolean')
       items.filter_to_type(type).any?
     end
@@ -367,17 +393,22 @@ module FatTable
     # :category: Aggregates
 
     # Return true if all of the items in the Column are true; otherwise return
-    # false.  Works only with boolean Columns.
+    # false, or false if all items are nil.  Works only with boolean Columns.
     def all?
+      return false if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('all?', 'Boolean')
       items.filter_to_type(type).all?
     end
 
     # :category: Aggregates
 
-    # Return true if none of the items in the Column are true; otherwise return
-    # false.  Works only with boolean Columns.
+    # Return true if none of the items in the Column are true; otherwise
+    # return false, or true if all items are nil.  Works only with boolean
+    # Columns.
     def none?
+      return true if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('none?', 'Boolean')
       items.filter_to_type(type).none?
     end
@@ -387,6 +418,8 @@ module FatTable
     # Return true if precisely one of the items in the Column is true;
     # otherwise return false.  Works only with boolean Columns.
     def one?
+      return false if type == 'NilClass' || items.all?(&:nil?)
+
       only_with('one?', 'Boolean')
       items.filter_to_type(type).one?
     end
@@ -395,6 +428,7 @@ module FatTable
 
     def only_with(agg, *valid_types)
       return self if valid_types.include?(type)
+
       msg = "aggregate '#{agg}' cannot be applied to a #{type} column"
       raise UserError, msg
     end
@@ -436,7 +470,12 @@ module FatTable
     private
 
     def convert_and_set_type(val)
-      new_val = Convert.convert_to_type(val, type, tolerant: tolerant?)
+      begin
+        new_val = Convert.convert_to_type(val, type, tolerant: tolerant?)
+      rescue IncompatibleTypeError
+        err_msg = "attempt to add '#{val}' to column '#{header}' already typed as #{type}"
+        raise IncompatibleTypeError, err_msg
+      end
       if new_val && (type == 'NilClass' || type == 'String')
         @type =
           if [true, false].include?(new_val)

data/lib/fat_table/convert.rb

@@ -36,8 +36,7 @@ module FatTable
         else
           new_val = convert_to_boolean(val)
           if new_val.nil?
-            msg = "attempt to add '#{val}' to a column already typed as #{type}"
-            raise IncompatibleTypeError, msg
+            raise IncompatibleTypeError
           end
           new_val
         end
@@ -47,8 +46,7 @@ module FatTable
         else
           new_val = convert_to_date_time(val)
           if new_val.nil?
-            msg = "attempt to add '#{val}' to a column already typed as #{type}"
-            raise IncompatibleTypeError, msg
+            raise IncompatibleTypeError
           end
           new_val
         end
@@ -58,8 +56,7 @@ module FatTable
         else
           new_val = convert_to_numeric(val)
           if new_val.nil?
-            msg = "attempt to add '#{val}' to a column already typed as #{type}"
-            raise IncompatibleTypeError, msg
+            raise IncompatibleTypeError
           end
           new_val
         end
@@ -82,8 +79,7 @@ module FatTable
         else
           new_val = convert_to_string(val)
           if new_val.nil?
-            msg = "attempt to add '#{val}' to a column already typed as #{type}"
-            raise IncompatibleTypeError, msg
+            raise IncompatibleTypeError
           end
           new_val
         end

data/lib/fat_table/footer.rb

@@ -166,7 +166,7 @@ module FatTable
       when String
         begin
           converted_val = Convert.convert_to_type(agg, column.type)
-        rescue UserError
+        rescue UserError, IncompatibleTypeError
           converted_val = false
         end
         if converted_val

data/lib/fat_table/formatters/formatter.rb

@@ -532,7 +532,7 @@ module FatTable
       valid_keys = table.headers + %i[string numeric datetime boolean nil]
       invalid_keys = (fmts.keys - valid_keys).uniq
       unless invalid_keys.empty?
-        msg = "invalid #{location} column or type: #{invalid_keys.join(',')}"
+        msg = "invalid #{location} column or type: #{invalid_keys.join(', ')}"
         raise UserError, msg
       end
 
@@ -1004,9 +1004,14 @@ module FatTable
     # converted to strings formatted according to the Formatter's formatting
     # directives given in Formatter.format_for or Formatter.format.
     def output
-      # This results in a hash of two-element arrays. The key is the header and
-      # the value is an array of the header and formatted header. We do the
-      # latter so the structure parallels the structure for rows explained next.
+      # If there are neither headers nor any rows in the table, return an
+      # empty string.
+      return '' if table.empty? && table.headers.empty?
+
+      # This results in a hash of two-element arrays. The key
+      # is the header and the value is an array of the header and formatted
+      # header. We do the latter so the structure parallels the structure for
+      # rows explained next.
       formatted_headers = build_formatted_headers
 
       # These produce an array with each element representing a row of the

data/lib/fat_table/formatters/org_formatter.rb

@@ -20,7 +20,7 @@ module FatTable
     end
 
     def pre_header(widths)
-      result = '|'
+      result = +'|'
       widths.each_value do |w|
         result += '-' * (w + 2) + '+'
       end
@@ -53,7 +53,7 @@ module FatTable
     end
 
     def hline(widths)
-      result = '|'
+      result = +'|'
       widths.each_value do |w|
         result += '-' * (w + 2) + '+'
       end
@@ -62,7 +62,7 @@ module FatTable
     end
 
     def post_footers(widths)
-      result = '|'
+      result = +'|'
       widths.each_value do |w|
         result += '-' * (w + 2) + '+'
       end

data/lib/fat_table/formatters/term_formatter.rb

@@ -221,7 +221,7 @@ module FatTable
     end
 
     def pre_header(widths)
-      result = upper_left
+      result = +upper_left
       widths.each_value do |w|
         result += double_rule * (w + 2) + upper_tee
       end
@@ -255,7 +255,7 @@ module FatTable
     end
 
     def hline(widths)
-      result = left_tee
+      result = +left_tee
       widths.each_value do |w|
         result += horizontal_rule * (w + 2) + single_cross
       end
@@ -289,7 +289,7 @@ module FatTable
     end
 
     def post_footers(widths)
-      result = lower_left
+      result = +lower_left
       widths.each_value do |w|
         result += double_rule * (w + 2) + lower_tee
       end

data/lib/fat_table/formatters/text_formatter.rb

@@ -16,7 +16,7 @@ module FatTable
     end
 
     def pre_header(widths)
-      result = '+'
+      result = +'+'
       widths.each_value do |w|
         result += '=' * (w + 2) + '+'
       end
@@ -49,7 +49,7 @@ module FatTable
     end
 
     def hline(widths)
-      result = '+'
+      result = +'+'
       widths.each_value do |w|
         result += '-' * (w + 2) + '+'
       end
@@ -82,7 +82,7 @@ module FatTable
     end
 
     def post_footers(widths)
-      result = '+'
+      result = +'+'
       widths.each_value do |w|
         result += '=' * (w + 2) + '+'
       end

data/lib/fat_table/table.rb

@@ -105,7 +105,7 @@ module FatTable
       unless heads.empty?
         heads.each do |h|
           if h.to_s.end_with?('!') || @tolerant_columns.include?(h)
-            @columns << Column.new(header: h.to_s.sub(/!\s*\z/, ''), tolerant: true)
+            @columns << Column.new(header: h.to_s.sub(/!\s*\z/, ''), type: 'String')
           else
             @columns << Column.new(header: h)
           end
@@ -120,7 +120,7 @@ module FatTable
     # though FatTable::Table objects have no instance variables, a class that
     # inherits from it might.
     def empty_dup
-      self.dup.__empty!
+      dup.__empty!
     end
 
     def __empty!

data/lib/fat_table/version.rb

@@ -2,5 +2,5 @@
 
 module FatTable
   # The current version of FatTable
-  VERSION = '0.5.4'
+  VERSION = '0.5.5'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fat_table
 version: !ruby/object:Gem::Version
-  version: 0.5.4
+  version: 0.5.5
 platform: ruby
 authors:
 - Daniel E. Doherty
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2022-01-27 00:00:00.000000000 Z
+date: 2022-03-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -332,7 +332,7 @@ licenses: []
 metadata:
   allowed_push_host: https://rubygems.org
   yard.run: yri
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -348,7 +348,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubygems_version: 3.3.3
-signing_key: 
+signing_key:
 specification_version: 4
 summary: Provides tools for working with tables as a data type.
 test_files: []