fat_table 0.6.6 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b9661335d0ae8ec4d87b9e16c3055552af05a354c5222b9838d51f8109ddd749
-  data.tar.gz: fe23c745a61b2fff7d3380221ee133b5e38c3bdd9d4ff429bdce2094cbe47755
+  metadata.gz: 4c16c57274c9f0f67abf924eeae6bce192f82844366199361ac68a7d9e1fdcf4
+  data.tar.gz: 42ea485003f76a56f66988a2bf4df159580dbbed022b1f637260ff8e1cd5888b
 SHA512:
-  metadata.gz: 201d3df5ba9820b96e22b1a035013c128c2f8cf2efd054cd16b9b6f372c13381334db9cc11b268daa0537d25e2d0475c35cd73f1749d08d8a008a2498010ead1
-  data.tar.gz: e067fc5bedae6541e0ce88e41025fffbff64a16a3545fccd89a6b171f2b6d648fd8e1716227ceafd770db509417361781c3c6d4cc718824ec8b9098a266eafab
+  metadata.gz: 32a6d03a5effa045ef6db54654aef86d96e90846cd0002ada80eaf8218c573f8a430067119562b8369226f11bc45d1bd0c144ddb8c1947c7fc9256fca55389b0
+  data.tar.gz: f0a033ff19d04160b4137f1204df322174926be2a0d7f2c442388a358a8ddc4321f5edcc1670346b9b366aa8a55acbe6a23b6b44f0fa9a69bb038e9eee7d24ac

data/README.org

@@ -445,27 +445,49 @@ or nil. There are only five permissible types for a ~Column~:
 4. *String* (for ruby ~String~ objects), or
 5. *NilClass* (for the undetermined column type).
 
-When a ~Table~ is constructed from an external source, all ~Columns~ start out
-having a type of ~NilClass~, that is, their type is as yet undetermined. When
-a string or object of one of the four determined types is added to a ~Column~
-and it can be converted into one of the permissible types, it fixes the type
-of the column, and all further items added to the ~Column~ must either be
-~nil~ (indicating no value) or be capable of being coerced to the column's
-type. Otherwise, ~FatTable~ raises an exception.
+By default, when a ~Table~ is constructed from an external source, all
+~Columns~ start out having a type of ~NilClass~, that is, their type is as yet
+undetermined. When a string or object is added to a ~Column~ and it can be
+converted into one of the permissible types, it fixes the type of the column,
+and all further items added to the ~Column~ must either be ~nil~ (indicating
+no value) or be capable of being coerced to the column's type. Otherwise,
+~FatTable~ raises an ~IncompatibleTypeError~ exception.
+
+*** Type Keywords Arguments
+All of the table constructors allow you to set the type for a column in
+advance by adding keyword arguments to the end of the contructor arguments
+where the keyword is a header symbol and the value is a string designating one
+of the types.  For example, suppose we are constructing a table from a CSV
+file, and we know that one of the columns is labeled 'Start' and another
+'Price'.  We want to require the items in the 'Start' column to be a valid
+date and the items in the 'Price' column to be valid numbers:
+
+#+begin_example
+  FatTable.from_csv_file('data.csv', start: 'date', price: 'num')
+#+end_example
+
+The type string can be anything that starts with 'dat', 'num', 'boo', or
+'str', regardless of case, to designate ~DateTime~, ~Numeric~, ~Boolean~, or
+~String~ types, respectively.  Any other string keeps the type as NilClass,
+that is, it remains open for automatic typing.
 
 The strictness of requiring all items to be of the same type can be relaxed by
-declaring a column to be "tolerant."  You can do so when you create the table
-by adding a tolerant_columns keyword parameter.  If a Column is tolerant,
-~FatTable~ tries to convert new items into a type other than a ~String~ and,
-if it can do so, sets /that/ as the Column's type.  Any later items that
-cannot be converted into the Column's type are converted to strings.  These
-interloper strings are treated like nils for purposes of sorting and
-evaluation, but are displayed according to any string formatting on output.
-See [[*Designating "Tolerant" Columns][Designating "Tolerant" Columns]] below.
-
-It is also possible to force ~FatTable~ to treat a column as a String type,
-even its items look like one of the other types.  See [[*Forcing String Type][Forcing String Type]]
-below.
+declaring a column to be "tolerant."  You can do so by adding a '~' to the end
+of a keyword type specifier in the table constructor.  In the above example,
+if we wanted to allow strings to be mixed up with the numeric prices, we would
+use the following:
+
+#+begin_example
+  FatTable.from_csv_file('data.csv', start: 'date', price: 'num~')
+#+end_example
+
+If a Column is tolerant, ~FatTable~ tries to convert new items into the
+column's specified type, or if the type is still open, to one of ~DateTime~,
+~Numeric~, or ~Boolean~ and then fixing the column's type, or, if it cannot do
+so converts the item into a ~String~ but does not raise an
+~IncompatibleTypeError~ exception.  These interloper strings are treated like
+nils for purposes of sorting and evaluation, but are displayed according to
+any string formatting on output.  See [[*Designating "Tolerant" Columns][Designating "Tolerant" Columns]] below.
 
 Items of input must be either one of the permissible ruby objects or strings. If
 they are strings, ~FatTable~ attempts to parse them as one of the permissible
@@ -626,14 +648,32 @@ columns to be created:
 
 Occasionally, ~FatTable~'s automatic type detection can get in the way and you
 just want it to treat one or more columns as Strings regardless of their
-appearance.  Think, for example, of zip codes.  If headers are given when a
-table is contructed, you can designate a forced-string column by appending a
-~!~ to the end of the header.  It will not become part of the header, it will
-just mark it as a forced-string Column.
+appearance.  Think, for example, of zip codes.  As mentioned above, when a
+table is contructed, you can designate a 'String' type for a column by
+using a keyword parameter.
 
-#+begin_SRC emacs-lisp :wrap EXAMPLE
-  tab = FatTable.new(:a, 'b', 'C!', :d, :zip!)
-#+end_SRC
+#+begin_src ruby :wrap EXAMPLE
+  require 'fat_table'
+  tab = FatTable.new(:a, 'b!', 'C', :d, :zip, zip: 'str')
+  tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '', zip: 18552 }
+  tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
+  tab << { zip: '01879--7884' }
+  tab << { zip: '66210', b: 'Not a Number' }
+  tab << { zip: '90210' }
+  tab.to_text
+#+end_src
+
+#+begin_EXAMPLE
++===+===+============+===+=============+===+
+| A | B | C          | D | Zip         | E |
++---+---+------------+---+-------------+---+
+| 1 | 2 | 2017-01-21 | F | 18552       |   |
+| 3 | 2 | 2016-01-21 | T |             |   |
+|   |   |            |   | 01879--7884 |   |
+|   |   |            |   | 90210       |   |
+|   |   |            |   |             |   |
++===+===+============+===+=============+===+
+#+end_EXAMPLE
 
 In addition, at any time after creating a table, you can force the String type
 on any number of columns with the ~force_string!~ method.  When you do so, all
@@ -662,84 +702,9 @@ exisiting items in the column are converted to strings with the #to_s method.
 +======+======+============+===+=======+===+
 #+end_EXAMPLE
 
-**** Designating "Tolerant" Columns
-
-Related to the problem just discussed is the problem of reading files in from
-the wild where a column may get typed as, say Numeric, but then contain
-something that can't be parsed as a Numeric.  ~FatTable~ raises an exception
-is such cases, and that may be what you want if you can control the input.
-But, especially when you cannot do so, it can be helpful to designate one or
-more columns as "tolerant."  This means that when a conversion problem occurs,
-the column item is retained as a string type in a column that is otherwise of
-one of the types Numeric, DateTime, or Boolean.  Those string items are
-treated as nils for purposes of sorting or evaluation in a ~select~ method.
-When formatted, they participate in string formatting directive, but not those
-for other types.
-
-All of the table construction methods, allow a keyword parameter,
-~tolerant_columns~, where you can designate what columns should be convert to
-String type when conversion to the auto-typed column type is not possible.
-The parameter should be an array of headers, in either string or symbol form,
-for which this behavior is desired.  In addition, it can be set to the special
-string '*' or symbol ~:*~ to indicate that all the columns should be made
-tolerant.
-
-#+begin_src ruby :wrap EXAMPLE
-  require 'fat_table'
-  tab = FatTable.new(:a, 'b', 'C', :d, :zip, tolerant_columns: [:zip])
-  tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '', zip: 18552 }
-  tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
-  tab << { zip: '01879--7884' }
-  tab << { zip: '66210' }
-  tab << { zip: '90210' }
-  tab.to_text
-#+end_src
-
-#+RESULTS:
-#+begin_EXAMPLE
-+======+======+============+===+=============+===+
-| A    | B    | C          | D | Zip         | E |
-+------+------+------------+---+-------------+---+
-| 1    | 2    | 2017-01-21 | F | 18552       |   |
-| 3.14 | 2.17 | 2016-01-21 | T |             |   |
-|      |      |            |   | 01879--7884 |   |
-|      |      |            |   | 66210       |   |
-|      |      |            |   | 90210       |   |
-+======+======+============+===+=============+===+
-#+end_EXAMPLE
-
-Another way to designate a column as tolerant is to end a column you want to
-designate as tolerant with a ~!~.  The ~!~ will be stripped from the header,
-but it will be marked as tolerant.
-#+begin_src ruby :wrap EXAMPLE
-  require 'fat_table'
-  tab = FatTable.new(:a, 'b!', 'C', :d, :zip!)
-  tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '', zip: 18552 }
-  tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
-  tab << { zip: '01879--7884' }
-  tab << { zip: '66210', b: 'Not a Number' }
-  tab << { zip: '90210' }
-  tab.to_text
-#+end_src
-
-#+RESULTS:
-#+begin_EXAMPLE
-+======+==============+============+===+=============+===+
-| A    | B            | C          | D | Zip         | E |
-+------+--------------+------------+---+-------------+---+
-| 1    | 2            | 2017-01-21 | F | 18552       |   |
-| 3.14 | 2.17         | 2016-01-21 | T |             |   |
-|      |              |            |   | 01879--7884 |   |
-|      | Not a Number |            |   | 66210       |   |
-|      |              |            |   | 90210       |   |
-+======+==============+============+===+=============+===+
-#+end_EXAMPLE
-
 *** From CSV or Org Mode files or strings
 Tables can also be read from ~.csv~ files or files containing ~org-mode~
-tables.  Remember that you can make any column tolerant with a
-~tolerant_columns:~ keyword argument or make them all tolerant by designating
-the pseudo-column ~:*~ as tolerant.
+tables.
 
 In the case of org-mode files, ~FatTable~ skips through the file until it finds
 a line that look like a table, that is, it begins with any number of spaces
@@ -799,10 +764,10 @@ header row, and the headers are converted to symbols as described above.
 
 You can also initialize a table directly from ruby data structures. You can,
 for example, build a table from an array of arrays.  Remember that you can
-make any column tolerant with a ~tolerant_columns:~ keyword argument or make
-them all tolerant by designating the pseudo-column ~:*~ as tolerant.
+make any column tolerant with a keyword argument for the column symbol and
+ending it with a '~'.
 
-#+BEGIN_SRC ruby :eval never
+#+BEGIN_SRC ruby
   aoa = [
     ['Ref', 'Date', 'Code', 'Raw', 'Shares', 'Price', 'Info', 'Bool'],
     [1, '2013-05-02', 'P', 795_546.20, 795_546.2, 1.1850, 'ENTITY1', 'T'],
@@ -816,11 +781,27 @@ them all tolerant by designating the pseudo-column ~:*~ as tolerant.
     [13, '2013-05-29', 'S', 13_459.00, 5659.51, 24.7464, 'ENTITY3', 'T'],
     [14, '2013-05-29', 'S', 15_700.00, 6601.85, 24.7790, 'ENTITY3', 'F'],
     [15, '2013-05-29', 'S', 15_900.00, 6685.95, 24.5802, 'ENTITY3', 'T'],
-    [16, '2013-05-30', 'S', 6_679.00, 2808.52, 25.0471, 'ENTITY3', 'T']
-  ]
-  tab = FatTable.from_aoa(aoa)
+    [16, '2013-05-30', 'S', 6_679.00, 2808.52, 25.0471, 'ENTITY3', 'T'] ]
+
+  tab = FatTable.from_aoa(aoa).to_aoa
 #+END_SRC
 
+#+RESULTS:
+| Ref | Date       | Code |    Raw | Shares | Price | Info    | Bool |
+|-----+------------+------+--------+--------+-------+---------+------|
+|   1 | 2013-05-02 | P    | 795546 | 795546 |     1 | ENTITY1 | T    |
+|   2 | 2013-05-02 | P    | 118186 | 118186 |    12 | ENTITY1 | T    |
+|   7 | 2013-05-20 | S    |  12000 |   5046 |    28 | ENTITY3 | F    |
+|   8 | 2013-05-20 | S    |  85000 |  35743 |    28 | ENTITY3 | T    |
+|   9 | 2013-05-20 | S    |  33302 |  14003 |    29 | ENTITY3 | T    |
+|  10 | 2013-05-23 | S    |   8000 |   3364 |    27 | ENTITY3 | T    |
+|  11 | 2013-05-23 | S    |  23054 |   9694 |    27 | ENTITY3 | F    |
+|  12 | 2013-05-23 | S    |  39906 |  16780 |    25 | ENTITY3 | T    |
+|  13 | 2013-05-29 | S    |  13459 |   5660 |    25 | ENTITY3 | T    |
+|  14 | 2013-05-29 | S    |  15700 |   6602 |    25 | ENTITY3 | F    |
+|  15 | 2013-05-29 | S    |  15900 |   6686 |    25 | ENTITY3 | T    |
+|  16 | 2013-05-30 | S    |   6679 |   2809 |    25 | ENTITY3 | T    |
+
 Notice that the values can either be ruby objects, such as the Integer ~85_000~,
 or strings that can be parsed into one of the permissible column types.
 
@@ -898,9 +879,7 @@ This example illustrates several things:
 A second ruby data structure that can be used to initialize a ~FatTable~ table
 is an array of ruby Hashes. Each hash represents a row of the table, and the
 headers of the table are taken from the keys of the hashes. Accordingly, all
-the hashes must have the same keys.  Remember that you can make any column
-tolerant with a ~tolerant_columns:~ keyword argument or make them all tolerant
-by designating the pseudo-column ~:*~ as tolerant.
+the hashes must have the same keys.
 
 This same method can in fact take an array of any objects that can be converted
 to a Hash with the ~#to_h~ method, so you can use an array of your own objects
@@ -989,10 +968,6 @@ The ~.connect~ function need only be called once, and the database handle it
 creates will be used for all subsequent ~.from_sql~ calls until ~.connect~ is
 called again.
 
-Remember that you can make any column tolerant with a ~tolerant_columns:~
-keyword argument or make them all tolerant by designating the pseudo-column
-~:*~ as tolerant.
-
 *** Marking Groups in Input
 **** Manually
 

data/lib/fat_table/convert.rb

@@ -63,19 +63,6 @@ module FatTable
       when 'String'
         if val.nil?
           nil
-        elsif tolerant
-          # Allow String to upgrade to one of Numeric, DateTime, or Boolean if
-          # possible.
-          if (new_val = convert_to_numeric(val))
-            new_val
-          elsif (new_val = convert_to_date_time(val))
-            new_val
-          elsif (new_val = convert_to_boolean(val))
-            new_val
-          else
-            new_val = convert_to_string(val)
-          end
-          new_val
         else
           new_val = convert_to_string(val)
           if new_val.nil?

data/lib/fat_table/formatters/formatter.rb

@@ -53,35 +53,38 @@ module FatTable
     # 'Average'.
     attr_reader :gfooters
 
-    class_attribute :default_format
-    self.default_format = {
-      nil_text: '',
-      case: :none,
-      alignment: :left,
-      bold: false,
-      italic: false,
-      color: 'none',
-      bgcolor: 'none',
-      hms: false,
-      pre_digits: 0,
-      post_digits: 0,
-      commas: false,
-      currency: false,
-      datetime_fmt: '%F %H:%M:%S',
-      date_fmt: '%F',
-      true_text: 'T',
-      false_text: 'F',
-      true_color: 'none',
-      true_bgcolor: 'none',
-      false_color: 'none',
-      false_bgcolor: 'none',
-      underline: false,
-      blink: false,
-    }
-
     class_attribute :valid_colors
     self.valid_colors = ['none']
 
+    def self.default_format
+      {
+        nil_text: '',
+        case: :none,
+        alignment: :left,
+        bold: false,
+        italic: false,
+        color: 'none',
+        bgcolor: 'none',
+        hms: false,
+        pre_digits: 0,
+        post_digits: 0,
+        commas: false,
+        currency: false,
+        datetime_fmt: '%F %H:%M:%S',
+        date_fmt: '%F',
+        true_text: 'T',
+        false_text: 'F',
+        true_color: 'none',
+        true_bgcolor: 'none',
+        false_color: 'none',
+        false_bgcolor: 'none',
+        underline: false,
+        blink: false,
+        _h: nil,
+        _location: nil
+      }
+    end
+
     # :category: Constructors
 
     # Return a new Formatter for the given +table+ which must be of the class
@@ -659,7 +662,7 @@ module FatTable
         # hash.
         format_h =
           if format_at[location][h].empty?
-            default_format.dup
+            default_format
           else
             format_at[location][h].to_h
           end
@@ -700,17 +703,17 @@ module FatTable
         # format_for call with those locations.
         if location == :body
           format_h.each_pair do |k, v|
-            if format_at[:bfirst][h].send(k) == default_format[k]
+            if format_at[:bfirst][h].send(k) == self.class.default_format[k]
               format_at[:bfirst][h].send("#{k}=", v)
             end
-            if format_at[:gfirst][h].send(k) == default_format[k]
+            if format_at[:gfirst][h].send(k) == self.class.default_format[k]
               format_at[:gfirst][h].send("#{k}=", v)
             end
           end
         elsif location == :gfirst
           # Copy :gfirst formatting to :bfirst if it is still the default
           format_h.each_pair do |k, v|
-            if format_at[:bfirst][h].send(k) == default_format[k]
+            if format_at[:bfirst][h].send(k) == self.class.default_format[k]
               format_at[:bfirst][h].send("#{k}=", v)
             end
           end
@@ -942,9 +945,10 @@ module FatTable
     # Convert a value to a string based on the instructions in istruct,
     # depending on the type of val. "Formatting," which changes the content of
     # the string, such as adding commas, is always performed, except alignment
-    # which is only performed when the width parameter is non-nil. "Decorating",
-    # which changes the appearance without changing the content, is performed
-    # only if the decorate parameter is true.
+    # which is only performed when the width parameter is
+    # non-nil. "Decorating", which changes the appearance without changing the
+    # content, is performed only if the decorate parameter is true.  Priority:
+    # lowest to highest: type, location, column_name
     def format_cell(val, istruct, width: nil, decorate: false)
       case val
       when Numeric
@@ -982,8 +986,6 @@ module FatTable
       end
     end
 
-    private
-
     # Add LaTeX control sequences, ANSI terminal escape codes, or other
     # decorations to string to decorate it with the given attributes. None of
     # the decorations may affect the displayed width of the string. Return the
@@ -992,6 +994,8 @@ module FatTable
       str
     end
 
+    private
+
     # Convert a boolean to a string according to instructions in istruct, which
     # is assumed to be the result of parsing a formatting instruction string as
     # above. Only device-independent formatting is done here. Device dependent
@@ -1099,7 +1103,7 @@ module FatTable
           val
         end
       if width && aligned?
-        pad = width - width(val)
+        pad = [width - width(val), 0].max
         case istruct.alignment
         when :left
           val += ' ' * pad

data/lib/fat_table/formatters/latex_formatter.rb

@@ -95,18 +95,6 @@ module FatTable
       result
     end
 
-    private
-
-    def color_valid?(clr)
-      valid_colors.include?(clr)
-    end
-
-    def invalid_color_msg(clr)
-      valid_colors_list = valid_colors.join(' ').wrap
-      "LaTeXFormatter invalid color '#{clr}'. Valid colors are:\n" +
-        valid_colors_list
-    end
-
     # Add LaTeX control sequences. Ignore background color, underline, and
     # blink. Alignment needs to be done by LaTeX, so we have to take it into
     # account unless it's the same as the body alignment, since that is the
@@ -119,15 +107,29 @@ module FatTable
         result = "{\\textcolor{#{istruct.color}}{#{result}}}"
       end
       if istruct.bgcolor && istruct.bgcolor != 'none'
-        result = "\\cellcolor{#{istruct.bgcolor}}#{result}"
+        result = "{\\cellcolor{#{istruct.bgcolor}}{#{result}}}"
       end
-      unless istruct.alignment == format_at[:body][istruct._h].alignment
+      if (istruct._h && format_at[:body][istruct._h] &&
+         istruct.alignment != format_at[:body][istruct._h].alignment) ||
+         (istruct._h.nil? && istruct.alignment.to_sym != :left)
         ac = alignment_code(istruct.alignment)
         result = "\\multicolumn{1}{#{ac}}{#{result}}"
       end
       result
     end
 
+    private
+
+    def color_valid?(clr)
+      valid_colors.include?(clr)
+    end
+
+    def invalid_color_msg(clr)
+      valid_colors_list = valid_colors.join(' ').wrap
+      "LaTeXFormatter invalid color '#{clr}'. Valid colors are:\n" +
+        valid_colors_list
+    end
+
     # Return +str+ with quote marks oriented and special TeX characters quoted.
     def quote(str)
       # Replace single and double quotes with TeX oriented quotes.
@@ -159,7 +161,7 @@ module FatTable
     end
 
     def alignment_code(al_sym)
-      case al_sym
+      case al_sym.to_sym
       when :center
         'c'
       when :right

data/lib/fat_table/formatters/org_formatter.rb

@@ -6,9 +6,12 @@ module FatTable
   # timestamps and the connector at the beginning of hlines is a '|' rather than
   # a '+' as for text tables.
   class OrgFormatter < Formatter
-    self.default_format = default_format.dup
-    default_format[:date_fmt] = '[%F %a]'
-    default_format[:datetime_fmt] = '[%F %a %H:%M:%S]'
+    def self.default_format
+      fmt = super
+      fmt[:date_fmt] = '[%F %a]'
+      fmt[:datetime_fmt] = '[%F %a %H:%M:%S]'
+      fmt
+    end
 
     private
 

data/lib/fat_table/formatters/term_formatter.rb

@@ -38,6 +38,17 @@ module FatTable
     self.valid_colors = ['none'] +
                         ::Rainbow::X11ColorNames::NAMES.keys.map(&:to_s).sort
 
+    # Add ANSI codes to string to implement the given decorations
+    def decorate_string(str, istruct)
+      result = Rainbow(str)
+      result = colorize(result, istruct.color, istruct.bgcolor)
+      result = result.bold if istruct.bold
+      result = result.italic if istruct.italic
+      result = result.underline if istruct.underline
+      result = result.blink if istruct.blink
+      result
+    end
+
     private
 
     def color_valid?(clr)
@@ -66,17 +77,6 @@ module FatTable
       str.gsub(/\e\[[0-9;]+m/, '')
     end
 
-    # Add ANSI codes to string to implement the given decorations
-    def decorate_string(str, istruct)
-      result = Rainbow(str)
-      result = colorize(result, istruct.color, istruct.bgcolor)
-      result = result.bold if istruct.bold
-      result = result.italic if istruct.italic
-      result = result.underline if istruct.underline
-      result = result.blink if istruct.blink
-      result
-    end
-
     def colorize(str, fg_color, bg_color)
       fg_color = nil if fg_color == 'none'
       bg_color = nil if bg_color == 'none'

data/lib/fat_table/table.rb

@@ -54,6 +54,10 @@ module FatTable
     # An Array of FatTable::Columns that constitute the table.
     attr_reader :columns
 
+    # Headers of columns that are to be tolerant when they are built.
+    attr_accessor :tolerant_cols
+    attr_reader :omni_typ, :omni_tol
+
     # Record boundaries set explicitly with mark_boundaries or from reading
     # hlines from input.  When we want to access boundaries, however, we want
     # to add an implict boundary at the last row of the table.  Since, as the
@@ -62,55 +66,58 @@ module FatTable
     # method call.
     attr_accessor :explicit_boundaries
 
-    # An Array of FatTable::Columns that should be tolerant.
-    attr_reader :tolerant_columns
-
     ###########################################################################
     # Constructors
-    ###########################################################################
-
+    #
+    #
     # :category: Constructors
-
+    #
     # Return an empty FatTable::Table object.  Specifying headers is optional.
-    # Any headers ending with a ! are marked as tolerant, in that, if an
-    # incompatible type is added to it, the column is re-typed as a String
-    # column, and construction proceeds.  The ! is stripped from the header to
-    # form the column key, though.  You can also provide the names of columns
-    # that should be tolerant by using the +tolerant_columns key-word to
-    # provide an array of headers that should be tolerant.  The special string
-    # '*' or the symbol :* indicates that all columns should be created
-    # tolerant.
-    def initialize(*heads, tolerant_columns: [])
+    # By default, all columns start our as having an "open" type and get
+    # assigned a type based on their contents.  For example, if a column
+    # contains items that can be interpreted as dates, the column gets
+    # assigned a DateTime type.  Other types are Numeric, Boolean, and String.
+    # Once a type is assigned to a column, any non-conforming vaules in that
+    # column raise an IncompatibleType error.  If a column is marked
+    # "tolerant", however, the incompatible item is converted to a string and
+    # allowed to remain in the column without raising an error.  They count as
+    # nils when calculations are performed on the column and paricipate only
+    # in string formatting directives on output.
+    #
+    # Rather than have a column's type determined by content, you can also
+    # specify a column type by providing a type hash, where the key is the
+    # header's name and the value is the desired type.  In that case, any
+    # incompatible type raises an an IncompatibleTypeError unless the column
+    # is also marked tolerant, in which case it gets converted to a string as
+    # discussed above.  If the type name in the types hash ends in a '~', it
+    # is treated as a specifying the given type but marking it as tolerant as
+    # well.  The values in the type hash can be any string or sybol that
+    # starts with  'num', 'dat', 'bool', or 'str' to specify Numeric,
+    # DateTime, Boolean, or String types respectively.
+    def initialize(*heads, **types)
       @columns = []
-      @explicit_boundaries = []
-      @tolerant_columns =
-        case tolerant_columns
-        when Array
-          tolerant_columns.map { |h| h.to_s.as_sym }
-        when String
-          if tolerant_columns.strip == '*'
-            ['*'.to_sym]
-          else
-            [tolerant_columns.as_sym]
-          end
-        when Symbol
-          if tolerant_columns.to_s.strip == '*'
-            ['*'.to_sym]
-          else
-            [tolerant_columns.to_s.as_sym]
-          end
-        else
-          raise ArgumentError, "set tolerant_columns to String, Symbol, or an Array of either"
-        end
-      unless heads.empty?
-        heads.each do |h|
-          if h.to_s.end_with?('!') || @tolerant_columns.include?(h)
-            @columns << Column.new(header: h.to_s.sub(/!\s*\z/, ''), type: 'String')
-          else
-            @columns << Column.new(header: h)
-          end
-        end
+      @tolerant_cols = []
+      @headers = []
+      # Check for the special 'omni' key
+      @omni_type = 'NilClass'
+      @omni_tol = false
+      if types.keys.map(&:to_s).include?('omni')
+        # All columns not otherwise included in types should have the type and
+        # tolerance of omni.
+        omni_val = (types['omni'] || types[:omni])
+        @omni_type, @omni_tol = Table.typ_tol(omni_val)
+        # Remove omni from types.
+        types.delete(:omni)
+        types.delete('omni')
+      end
+      heads += types.keys
+      heads.uniq.each do |h|
+        typ, tol = Table.typ_tol(types[h])
+        @tolerant_cols << h.to_s.as_sym if tol
+        @columns << Column.new(header: h.to_s.sub(/~\s*\z/, ''), type: typ,
+                               tolerant: tol)
       end
+      @explicit_boundaries = []
     end
 
     # :category: Constructors
@@ -133,9 +140,9 @@ module FatTable
 
     # Construct a Table from the contents of a CSV file named +fname+. Headers
     # will be taken from the first CSV row and converted to symbols.
-    def self.from_csv_file(fname, tolerant_columns: [])
+    def self.from_csv_file(fname, **types)
       File.open(fname, 'r') do |io|
-        from_csv_io(io, tolerant_columns: tolerant_columns)
+        from_csv_io(io, **types)
       end
     end
 
@@ -143,8 +150,8 @@ module FatTable
 
     # Construct a Table from a CSV string +str+, treated in the same manner as
     # the input from a CSV file in ::from_org_file.
-    def self.from_csv_string(str, tolerant_columns: [])
-      from_csv_io(StringIO.new(str), tolerant_columns: tolerant_columns)
+    def self.from_csv_string(str, **types)
+      from_csv_io(StringIO.new(str), **types)
     end
 
     # :category: Constructors
@@ -153,9 +160,9 @@ module FatTable
     # file named +fname+. Headers are taken from the first row if the second row
     # is an hrule. Otherwise, synthetic headers of the form +:col_1+, +:col_2+,
     # etc. are created.
-    def self.from_org_file(fname, tolerant_columns: [])
+    def self.from_org_file(fname, **types)
       File.open(fname, 'r') do |io|
-        from_org_io(io, tolerant_columns: tolerant_columns)
+        from_org_io(io, **types)
       end
     end
 
@@ -163,8 +170,8 @@ module FatTable
 
     # Construct a Table from a string +str+, treated in the same manner as the
     # contents of an org-mode file in ::from_org_file.
-    def self.from_org_string(str, tolerant_columns: [])
-      from_org_io(StringIO.new(str), tolerant_columns: tolerant_columns)
+    def self.from_org_string(str, **types)
+      from_org_io(StringIO.new(str), **types)
     end
 
     # :category: Constructors
@@ -183,8 +190,8 @@ module FatTable
     # :hlines no +) org-mode strips all hrules from the table; otherwise (+
     # HEADER: :hlines yes +) they are indicated with nil elements in the outer
     # array.
-    def self.from_aoa(aoa, hlines: false, tolerant_columns: [])
-      from_array_of_arrays(aoa, hlines: hlines, tolerant_columns: tolerant_columns)
+    def self.from_aoa(aoa, hlines: false, **types)
+      from_array_of_arrays(aoa, hlines: hlines, **types)
     end
 
     # :category: Constructors
@@ -194,9 +201,9 @@ module FatTable
     # keys, which, when converted to symbols will become the headers for the
     # Table. If hlines is set true, mark a group boundary whenever a nil, rather
     # than a hash appears in the outer array.
-    def self.from_aoh(aoh, hlines: false, tolerant_columns: [])
+    def self.from_aoh(aoh, hlines: false, **types)
       if aoh.first.respond_to?(:to_h)
-        from_array_of_hashes(aoh, hlines: hlines, tolerant_columns: tolerant_columns)
+        from_array_of_hashes(aoh, hlines: hlines, **types)
       else
         raise UserError,
               "Cannot initialize Table with an array of #{input[0].class}"
@@ -215,7 +222,7 @@ module FatTable
 
     # Construct a Table by running a SQL +query+ against the database set up
     # with FatTable.connect, with the rows of the query result as rows.
-    def self.from_sql(query, tolerant_columns: [])
+    def self.from_sql(query, **types)
       msg = 'FatTable.db must be set with FatTable.connect'
       raise UserError, msg if FatTable.db.nil?
 
@@ -232,13 +239,32 @@ module FatTable
     ############################################################################
 
     class << self
+      # Return [typ, tol] based on the type string, str.
+      def typ_tol(str)
+        tol = str ? str.match?(/~\s*\Z/) : false
+        typ =
+          case str
+          when /\A\s*num/i
+            'Numeric'
+          when /\A\s*boo/i
+            'Boolean'
+          when /\A\s*dat/i
+            'DateTime'
+          when /\A\s*str/i
+            'String'
+          else
+            'NilClass'
+          end
+        [typ, tol]
+      end
+
       private
 
       # Construct table from an array of hashes or an array of any object that
       # can respond to #to_h. If an array element is a nil, mark it as a group
       # boundary in the Table.
-      def from_array_of_hashes(hashes, hlines: false, tolerant_columns: [])
-        result = new(tolerant_columns: tolerant_columns)
+      def from_array_of_hashes(hashes, hlines: false, **types)
+        result = new(**types)
         hashes.each do |hsh|
           if hsh.nil?
             unless hlines
@@ -266,8 +292,8 @@ module FatTable
       # hlines are stripped from the table, otherwise (:hlines yes) they are
       # indicated with nil elements in the outer array as expected by this
       # method when hlines is set true.
-      def from_array_of_arrays(rows, hlines: false, tolerant_columns: [])
-        result = new(tolerant_columns: tolerant_columns)
+      def from_array_of_arrays(rows, hlines: false, **types)
+        result = new(**types)
         headers = []
         if !hlines
           # Take the first row as headers
@@ -303,8 +329,8 @@ module FatTable
         result
       end
 
-      def from_csv_io(io, tolerant_columns: [])
-        result = new(tolerant_columns: tolerant_columns)
+      def from_csv_io(io, **types)
+        result = new(**types)
         ::CSV.new(io, headers: true, header_converters: :symbol,
                   skip_blanks: true).each do |row|
           result << row.to_h
@@ -317,7 +343,7 @@ module FatTable
       # header row must be marked with an hline (i.e, a row that looks like
       # '|---+--...--|') and groups of rows may be marked with hlines to
       # indicate group boundaries.
-      def from_org_io(io, tolerant_columns: [])
+      def from_org_io(io, **types)
         table_re = /\A\s*\|/
         hrule_re = /\A\s*\|[-+]+/
         rows = []
@@ -352,7 +378,7 @@ module FatTable
             rows << line.split('|').map(&:clean)
           end
         end
-        from_array_of_arrays(rows, hlines: true, tolerant_columns: tolerant_columns)
+        from_array_of_arrays(rows, hlines: true, **types)
       end
     end
 
@@ -377,6 +403,16 @@ module FatTable
       column(key).type
     end
 
+    # Return the type of the Column with the given +key+ as its
+    # header as a String.
+    def types
+      result = {}
+      headers.each do |h|
+        result[h] = type(h)
+      end
+      result
+    end
+
     # :category: Attributes
 
     # Set the column type for Column with the given +key+ as a String type.
@@ -428,7 +464,7 @@ module FatTable
     # :category: Attributes
 
     # Return a Hash of the Table's Column header symbols to type strings.
-    def types
+    def col_types
       result = {}
       columns.each do |c|
         result[c.header] = c.type
@@ -445,11 +481,11 @@ module FatTable
 
     # :category: Attributes
 
-    # Return whether the column with the given head should be made tolerant.
+    # Return whether the column with the given head is supposed to be
+    # tolerant.  We can't just look up the Column because it may not be build
+    # yet, as when we do a row-by-row add.
     def tolerant_col?(h)
-      return true if tolerant_columns.include?(:'*')
-
-      tolerant_columns.include?(h)
+      tolerant_cols.include?(h.to_s.as_sym) || self.omni_tol
     end
 
     # :category: Attributes
@@ -994,11 +1030,11 @@ module FatTable
       result = empty_dup
       headers.each do |h|
         col =
-        if tolerant_col?(h)
-          Column.new(header: h, tolerant: true)
-        else
-          Column.new(header: h)
-        end
+          if tolerant_col?(h)
+            Column.new(header: h, tolerant: true)
+          else
+            Column.new(header: h)
+          end
         result.add_column(col)
       end
       ev = Evaluator.new(ivars: { row: 0, group: 0 })

data/lib/fat_table/version.rb

@@ -2,5 +2,5 @@
 
 module FatTable
   # The current version of FatTable
-  VERSION = '0.6.6'
+  VERSION = '0.8.0'
 end

data/lib/fat_table.rb

@@ -61,22 +61,22 @@ module FatTable
 
   # Return an empty FatTable::Table object. You can use FatTable::Table#add_row
   # or FatTable::Table#add_column to populate the table with data.
-  def self.new(*args, tolerant_columns: [])
-    Table.new(*args, tolerant_columns: tolerant_columns)
+  def self.new(*args, **types)
+    Table.new(*args, **types)
   end
 
   # Construct a FatTable::Table from the contents of a CSV file given by the
   # file name +fname+. Headers will be taken from the first row and converted to
   # symbols.
-  def self.from_csv_file(fname, tolerant_columns: [])
-    Table.from_csv_file(fname, tolerant_columns: tolerant_columns)
+  def self.from_csv_file(fname, **types)
+    Table.from_csv_file(fname, **types)
   end
 
   # Construct a FatTable::Table from the string +str+, treated in the same
   # manner as if read the input from a CSV file. Headers will be taken from the
   # first row and converted to symbols.
-  def self.from_csv_string(str, tolerant_columns: [])
-    Table.from_csv_string(str, tolerant_columns: tolerant_columns)
+  def self.from_csv_string(str, **types)
+    Table.from_csv_string(str, **types)
   end
 
   # Construct a FatTable::Table from the first table found in the Emacs org-mode
@@ -84,8 +84,8 @@ module FatTable
   # is an hline. Otherwise, synthetic headers of the form +:col_1+, +:col_2+,
   # etc. are created. Any other hlines will be treated as marking a boundary in
   # the table.
-  def self.from_org_file(fname, tolerant_columns: [])
-    Table.from_org_file(fname, tolerant_columns: tolerant_columns)
+  def self.from_org_file(fname, **types)
+    Table.from_org_file(fname, **types)
   end
 
   # Construct a FatTable::Table from the first table found in the string +str+,
@@ -93,8 +93,8 @@ module FatTable
   # are taken from the first row if the second row is an hrule. Otherwise,
   # synthetic headers of the form :col_1, :col_2, etc. are created. Any other
   # hlines will be treated as marking a boundary in the table.
-  def self.from_org_string(str, tolerant_columns: [])
-    Table.from_org_string(str, tolerant_columns: tolerant_columns)
+  def self.from_org_string(str, **types)
+    Table.from_org_string(str, **types)
   end
 
   # Construct a FatTable::Table from the array of arrays +aoa+. By default, with
@@ -108,8 +108,8 @@ module FatTable
   # org-mode code blocks, by default (+:hlines no+) all hlines are stripped from
   # the table, otherwise (+:hlines yes+) they are indicated with nil elements in
   # the outer array.
-  def self.from_aoa(aoa, hlines: false, tolerant_columns: [])
-    Table.from_aoa(aoa, hlines: hlines, tolerant_columns: tolerant_columns)
+  def self.from_aoa(aoa, hlines: false, **types)
+    Table.from_aoa(aoa, hlines: hlines, **types)
   end
 
   # Construct a FatTable::Table from the array of hashes +aoh+, which can be an
@@ -117,8 +117,8 @@ module FatTable
   # interpret nil separators as marking boundaries in the new Table. All hashes
   # must have the same keys, which, converted to symbols, become the headers for
   # the new Table.
-  def self.from_aoh(aoh, hlines: false, tolerant_columns: [])
-    Table.from_aoh(aoh, hlines: hlines, tolerant_columns: tolerant_columns)
+  def self.from_aoh(aoh, hlines: false, **types)
+    Table.from_aoh(aoh, hlines: hlines, **types)
   end
 
   # Construct a FatTable::Table from another FatTable::Table. Inherit any group
@@ -130,8 +130,8 @@ module FatTable
   # Construct a Table by running a SQL query against the database set up with
   # FatTable.connect. Return the Table with the query results as rows and the
   # headers from the query, converted to symbols, as headers.
-  def self.from_sql(query, tolerant_columns: [])
-    Table.from_sql(query, tolerant_columns: tolerant_columns)
+  def self.from_sql(query, **types)
+    Table.from_sql(query, **types)
   end
 
   ########################################################################

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fat_table
 version: !ruby/object:Gem::Version
-  version: 0.6.6
+  version: 0.8.0
 platform: ruby
 authors:
 - Daniel E. Doherty
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2023-01-09 00:00:00.000000000 Z
+date: 2023-04-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler