fat_table 0.3.4 → 0.5.2

Sign up to get free protection for your applications and to get access to all the features.
data/README.org CHANGED
@@ -1,5 +1,19 @@
1
- #+OPTIONS: :toc
1
+ #+TITLE: FatTable User Guide
2
+ #+OPTIONS: toc:4
2
3
  #+LATEX_HEADER: \usepackage[margin=0.75in]{geometry}
4
+ #+PROPERTY: header-args:ruby :colnames no :session readme :hlines yes :exports both
5
+ #+PROPERTY: header-args:sh :exports code
6
+ #+STARTUP: inlineimages
7
+
8
+ #+begin_comment
9
+ Notes on producing this README.
10
+
11
+ 1. Result blocks won't be rendered on Github if the #+RESULTS: tag is left
12
+ over the results block, so manually delete them.
13
+
14
+ 2. Make sure that the current version of the gem is geting loaded when running
15
+ the code blocks. rbenv-use-corresponding helps here.
16
+ #+end_comment
3
17
 
4
18
  #+BEGIN_COMMENT
5
19
  This is for markdown output:
@@ -33,37 +47,139 @@ they makes sense for the output medium and treats other formatting directives as
33
47
  no-ops.
34
48
 
35
49
  ~FatTable~ can be used to perform operations on data that are naturally best
36
- conceived of as tables, which in my experience is quite often. It can also serve
37
- as a foundation for providing reporting functions where flexibility about the
38
- output medium can be useful. Finally ~FatTable~ can be used within Emacs
39
- ~org-mode~ files in code blocks targeting the Ruby language. Org mode tables are
40
- presented to a ruby code block as an array of arrays, so ~FatTable~ can read
41
- them in with its ~.from_aoa~ constructor. A ~FatTable~ table output as an array
42
- of arrays with its ~.to_aoa~ output function will be rendered in an org-mode
43
- buffer as an org-table, ready for processing by other code blocks.
50
+ conceived of as tables, which in my experience is quite often. It can also
51
+ serve as a foundation for providing reporting functions where flexibility
52
+ about the output medium can be useful. Finally ~FatTable~ can be used within
53
+ Emacs ~org-mode~ files in code blocks targeting the Ruby language. Org mode
54
+ tables are presented to a ruby code block as an array of arrays, so ~FatTable~
55
+ can read them in with its ~.from_aoa~ constructor. A ~FatTable~ table output as an
56
+ array of arrays with its ~.to_aoa~ output function will be rendered in an
57
+ org-mode buffer as an org-table, ready for processing by other code blocks.
58
+
59
+ * Table of Contents :toc:noexport:
60
+ - [[#introduction][Introduction]]
61
+ - [[#installation][Installation]]
62
+ - [[#using-in-a-gem][Using in a gem]]
63
+ - [[#manually-install][Manually install]]
64
+ - [[#require][Require]]
65
+ - [[#usage][Usage]]
66
+ - [[#quick-start][Quick Start]]
67
+ - [[#a-word-about-the-examples][A Word About the Examples]]
68
+ - [[#anatomy-of-a-table][Anatomy of a Table]]
69
+ - [[#columns][Columns]]
70
+ - [[#headers][Headers]]
71
+ - [[#groups][Groups]]
72
+ - [[#constructing-tables][Constructing Tables]]
73
+ - [[#empty-tables][Empty Tables]]
74
+ - [[#without-headers][Without Headers]]
75
+ - [[#with-headers][With Headers]]
76
+ - [[#forcing-string-type][Forcing String Type]]
77
+ - [[#from-csv-or-org-mode-files-or-strings][From CSV or Org Mode files or strings]]
78
+ - [[#from-arrays-of-arrays][From Arrays of Arrays]]
79
+ - [[#in-ruby-code][In Ruby Code]]
80
+ - [[#in-emacs-org-files][In Emacs Org Files]]
81
+ - [[#from-arrays-of-hashes][From Arrays of Hashes]]
82
+ - [[#from-sql-queries][From SQL queries]]
83
+ - [[#marking-groups-in-input][Marking Groups in Input]]
84
+ - [[#manually][Manually]]
85
+ - [[#when-reading-in-tables][When Reading in Tables]]
86
+ - [[#accessing-parts-of-tables][Accessing Parts of Tables]]
87
+ - [[#rows][Rows]]
88
+ - [[#columns-1][Columns]]
89
+ - [[#cells][Cells]]
90
+ - [[#other-table-attributes][Other table attributes]]
91
+ - [[#operations-on-tables][Operations on Tables]]
92
+ - [[#example-input-tables][Example Input Tables]]
93
+ - [[#select][Select]]
94
+ - [[#selecting-existing-columns-also-of-omni][Selecting Existing Columns (Also of :omni)]]
95
+ - [[#copying-and-renaming-existing-columns][Copying and Renaming Existing Columns]]
96
+ - [[#adding-new-columns][Adding New Columns]]
97
+ - [[#custom-instance-variables-and-hooks][Custom Instance Variables and Hooks]]
98
+ - [[#argument-order-and-boundaries][Argument Order and Boundaries]]
99
+ - [[#where][Where]]
100
+ - [[#order_by][Order_by]]
101
+ - [[#order_with][Order_with]]
102
+ - [[#group_by][Group_by]]
103
+ - [[#join][Join]]
104
+ - [[#join-types][Join Types]]
105
+ - [[#join-expressions][Join Expressions]]
106
+ - [[#join-examples][Join Examples]]
107
+ - [[#inner-joins][Inner Joins]]
108
+ - [[#left-and-right-joins][Left and Right Joins]]
109
+ - [[#full-join][Full Join]]
110
+ - [[#cross-join][Cross Join]]
111
+ - [[#set-operations][Set Operations]]
112
+ - [[#unions][Unions]]
113
+ - [[#intersections][Intersections]]
114
+ - [[#set-differences-with-except][Set Differences with Except]]
115
+ - [[#uniq-aka-distinct][Uniq (aka Distinct)]]
116
+ - [[#remove-groups-with-degroup][Remove groups with degroup!]]
117
+ - [[#formatting-tables][Formatting Tables]]
118
+ - [[#available-formatter-output-targets][Available Formatter Output Targets]]
119
+ - [[#output-media][Output Media]]
120
+ - [[#examples][Examples]]
121
+ - [[#to-text][To Text]]
122
+ - [[#to-org][To Org]]
123
+ - [[#to-term][To Term]]
124
+ - [[#to-latex][To LaTeX]]
125
+ - [[#to-aoa-array-of-arrays][To AoA (Array of Arrays)]]
126
+ - [[#to-aoh-array-of-hashes][To AoH (Array of Hashes)]]
127
+ - [[#formatting-directives][Formatting Directives]]
128
+ - [[#string][String]]
129
+ - [[#numeric][Numeric]]
130
+ - [[#datetime][DateTime]]
131
+ - [[#boolean][Boolean]]
132
+ - [[#nilclass][NilClass]]
133
+ - [[#the-format-and-format_for-methods][The ~format~ and ~format_for~ methods]]
134
+ - [[#table-locations][Table Locations]]
135
+ - [[#location-priority][Location priority]]
136
+ - [[#type-and-column-priority][Type and Column priority]]
137
+ - [[#footers][Footers]]
138
+ - [[#adding-footers][Adding Footers]]
139
+ - [[#aggregators][Aggregators]]
140
+ - [[#footer-objects][Footer objects]]
141
+ - [[#footer-examples][Footer Examples]]
142
+ - [[#built-in-aggregators][Built-in Aggregators]]
143
+ - [[#string-aggregators][String Aggregators]]
144
+ - [[#ruby-objects][Ruby Objects]]
145
+ - [[#lambdas][Lambdas]]
146
+ - [[#invoking-formatters][Invoking Formatters]]
147
+ - [[#by-instantiating-a-formatter][By Instantiating a Formatter]]
148
+ - [[#by-using-fattable-module-level-method-calls][By Using ~FatTable~ module-level method calls]]
149
+ - [[#by-calling-methods-on-table-objects][By Calling Methods on Table Objects]]
150
+ - [[#development][Development]]
151
+ - [[#contributing][Contributing]]
44
152
 
45
153
  * Installation
46
-
47
- ** Installing the gem
48
-
154
+ ** Using in a gem
49
155
  Add this line to your application's Gemfile:
50
-
51
- #+BEGIN_SRC ruby
156
+ #+BEGIN_SRC ruby :exports code
52
157
  gem 'fat_table'
53
158
  #+END_SRC
54
159
 
55
- And then execute:
160
+ Or, something like this in your gemspec file:
161
+ #+begin_SRC ruby :exports code
162
+ gem.add_runtime_dependency 'fat_table'
163
+ #+end_SRC
56
164
 
165
+ And then execute:
57
166
  #+BEGIN_SRC sh
58
167
  $ bundle
59
168
  #+END_SRC
60
169
 
170
+ ** Manually install
61
171
  Or install it yourself as:
62
172
 
63
173
  #+BEGIN_SRC sh
64
174
  $ gem install fat_table
65
175
  #+END_SRC
66
176
 
177
+ ** Require
178
+ Somewhere in your code, make sure that =FatTable= is required:
179
+ #+begin_src ruby :exports code :results silent
180
+ require 'fat_table'
181
+ #+end_src
182
+
67
183
  * Usage
68
184
  ** Quick Start
69
185
 
@@ -71,10 +187,10 @@ Or install it yourself as:
71
187
  operated on in a number of ways. Here's a quick example to illustrate the use of
72
188
  ~FatTable~. See the detailed explanations further on down.
73
189
 
74
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
75
- #+BEGIN_SRC ruby
76
- require 'fat_table'
190
+ Here is a set of data that records some kind of stock activity. It's an array
191
+ of arrays with the first inner array being the headings.
77
192
 
193
+ #+BEGIN_SRC ruby :exports code :results silent
78
194
  data =
79
195
  [['Date', 'Code', 'Raw', 'Shares', 'Price', 'Info', 'Ok'],
80
196
  ['2013-05-29', 'S', 15_700.00, 6601.85, 24.7790, 'ENTITY3', 'F'],
@@ -89,9 +205,17 @@ operated on in a number of ways. Here's a quick example to illustrate the use of
89
205
  ['2013-05-29', 'S', 15_900.00, 6685.95, 24.5802, 'ENTITY3', 'T'],
90
206
  ['2013-05-30', 'S', 6_679.00, 2808.52, 25.0471, 'ENTITY3', 'T'],
91
207
  ['2013-05-23', 'S', 23_054.00, 9694.21, 26.8015, 'ENTITY3', 'F']]
208
+ #+END_SRC
209
+
210
+ Use FatTable to read the data and convert in into a table object. Note that
211
+ the headings within the table are all converted to symbols, lower-cased and
212
+ any spaces replaced with underscores.
92
213
 
93
- # Build the Table and then perform chained operations on it
214
+ Below, we select only those rows having more than 2000 shares, sort by a
215
+ compund key, select all columns but add a column, :ref, for the row number,
216
+ and finally re-order the columns with a final select.
94
217
 
218
+ #+BEGIN_SRC ruby :results silent :exports code
95
219
  table = FatTable.from_aoa(data) \
96
220
  .where('shares > 2000') \
97
221
  .order_by(:date, :code) \
@@ -99,27 +223,51 @@ operated on in a number of ways. Here's a quick example to illustrate the use of
99
223
  :price, :ok, ref: '@row') \
100
224
  .select(:ref, :date, :code,
101
225
  :shares, :price, :ok)
226
+ #+END_SRC
102
227
 
103
- # Convert the table to an ASCII text string
104
-
228
+ You can use the resulting table in other operations, such as performing joins
229
+ or set operations with other tables, etc. The world's your oyster. But
230
+ eventually you will want to present the table in some format, and that is
231
+ where the formatting methods come in. They let you add footers, including
232
+ groups footers, as well as styling the various elements with very simple
233
+ formatting directives that can apply to various "locations" in the table. Any
234
+ formatting directives that are beyond the capabilities of the output medium
235
+ are simply ignored.
236
+
237
+ We can format the table constructed above.
238
+ #+BEGIN_SRC ruby :exports both
105
239
  table.to_text do |fmt|
106
- # Add some table footers
240
+ # Add a group footer at the bottom of each group that results from sorting
241
+ # with the order_by method.
242
+ fmt.gfooter('Avg', shares: :avg, price: :avg)
243
+ # Add some table footers. Averages for the price and shares columns. The
244
+ # avg_footer method applies the avg aggregate to all the named columns with
245
+ # an "Average" label.
107
246
  fmt.avg_footer(:price, :shares)
247
+ # And a second footer that shows the sum for the shares column.
108
248
  fmt.sum_footer(:shares)
109
- # Add a group footer
110
- fmt.gfooter('Avg', shares: :avg, price: :avg)
111
- # Formats for all locations
249
+ # Formats for all locations, :ref column is centered and bold, all numerics
250
+ # are right-aligned, and all booleans are centered and printed with 'Y' or
251
+ # 'N'
112
252
  fmt.format(ref: 'CB', numeric: 'R', boolean: 'CY')
113
- # Formats for different "locations" in the table
253
+ # Formats for different "locations" in the table:
254
+ # The headers are all centered and bold.
114
255
  fmt.format_for(:header, string: 'CB')
256
+ # In the body rows (i.e., not the headers or footers), the code column is
257
+ # centered, shares have grouping commas applied and are rounded to one
258
+ # decimal place, but the price column is rounded to 4 places with no
259
+ # grouping commas.
115
260
  fmt.format_for(:body, code: 'C', shares: ',0.1', price: '0.4', )
261
+ # But the price column in the first row of the body (:bfirst location) will
262
+ # also be formatted with a currency symbol.
116
263
  fmt.format_for(:bfirst, price: '$0.4', )
117
- fmt.format_for(:footer, shares: 'B,0.1', price: '$B0.4', )
264
+ # In the footers, apply the same rounding rules, but make the results bold.
118
265
  fmt.format_for(:gfooter, shares: 'B,0.1', price: 'B0.4', )
266
+ fmt.format_for(:footer, shares: 'B,0.1', price: '$B0.4', )
119
267
  end
120
268
  #+END_SRC
121
269
 
122
- #+BEGIN_EXAMPLE
270
+ #+begin_example
123
271
  +=========+============+======+=============+==========+====+
124
272
  | Ref | Date | Code | Shares | Price | Ok |
125
273
  +---------+------------+------+-------------+----------+----+
@@ -154,23 +302,64 @@ operated on in a number of ways. Here's a quick example to illustrate the use of
154
302
  +---------+------------+------+-------------+----------+----+
155
303
  | Total | | | 1,020,119.1 | | |
156
304
  +=========+============+======+=============+==========+====+
157
- #+END_EXAMPLE
305
+ #+end_example
158
306
 
159
- ** A Word About the Examples
307
+ For the text format above, we were wasting our breath specifying bold styling
308
+ since there is no way to make that happen in plain ASCII text. But with
309
+ LaTeX, bold is doable. The output of the following code block is being
310
+ written to a file =examples/quicktable.tex= which is then =\included=-ed in a
311
+ simple wrapper file, =examples/quick.tex= so it can be compiled by LaTeX.
312
+
313
+ #+BEGIN_SRC ruby :results file :file "examples/quicktable.tex"
314
+ table.to_latex do |fmt|
315
+ fmt.gfooter('Avg', shares: :avg, price: :avg)
316
+ fmt.avg_footer(:price, :shares)
317
+ fmt.sum_footer(:shares)
318
+ fmt.format(ref: 'CB', numeric: 'R', boolean: 'CY')
319
+ fmt.format_for(:header, string: 'CB')
320
+ fmt.format_for(:body, code: 'C', shares: ',0.1c[blue.lightgray]', price: '0.4', )
321
+ fmt.format_for(:bfirst, price: '$0.4', )
322
+ fmt.format_for(:gfooter, shares: 'B,0.1', price: 'B0.4', )
323
+ fmt.format_for(:footer, shares: 'B,0.1', price: '$B0.4', )
324
+ end
325
+ #+END_SRC
326
+
327
+ #+begin_EXAMPLE
328
+ [[file:examples/quicktable.tex]]
329
+ #+end_EXAMPLE
330
+
331
+ These commands run pdflatex on the result twice to get the table aligned
332
+ properly.
333
+ #+begin_src sh :results silent
334
+ cd examples
335
+ pdflatex quick.tex
336
+ pdflatex quick.tex
337
+ #+end_src
338
+
339
+ And we convert the =PDF= into a smaller image for display:
340
+ #+begin_src sh :results verbatim
341
+ cd examples
342
+ pdftoppm -png quick.pdf >quick.png
343
+ convert quick.png -resize 600x800 quick_small.png
344
+ #+end_src
345
+
346
+ [[file:examples/quick_small.png]]
160
347
 
348
+ ** A Word About the Examples
161
349
  When you install the ~fat_table~ gem, you have access to a program ~ft_console~,
162
350
  which opens a ~pry~ session with ~fat_table~ loaded and the tables used in the
163
351
  examples in this ~README~ defined as instance variables so you can experiment
164
352
  with them. Because they are defined as instance variables, you have to write
165
- ~tab1~ as ~@tab1~ in ~ft_console~, but otherwise the examples should work as
166
- shown in this ~README~.
353
+ ~tab1~ as ~@tab1~ in ~ft_console~, but otherwise the examples should work as shown
354
+ in this ~README~.
167
355
 
168
- The examples in this ~README~ file are executed as code blocks within the
169
- ~README.org~ file, so they typically end with a call to ~.to_aoa~. That causes
170
- the table to be inserted into the file and formatted as a table. With
171
- ~ft_console~, you should instead display your tables with ~.to_text~ or
172
- ~.to_term~. These will return a string that you can print to the terminal with
173
- ~puts~.
356
+ The examples in this ~README~ file are executed in Emacs org-mode as code
357
+ blocks within the ~README.org~ file, so they typically end with a call to
358
+ ~.to_aoa~. That causes Emacs to insert the "Array of Array" ruby data
359
+ structure into the file and format it as a table, which is the convention for
360
+ Emacs org-mode. With ~ft_console~, you should instead display your tables with
361
+ ~.to_text~ or ~.to_term~. These will return a string that you can print to the
362
+ terminal with ~puts~.
174
363
 
175
364
  To read in the table used in the Quick Start section above, you might do the
176
365
  following:
@@ -230,7 +419,6 @@ directives.
230
419
 
231
420
  ** Anatomy of a Table
232
421
  *** Columns
233
-
234
422
  ~FatTable::Table~ objects consist of an array of ~FatTable::Column~ objects.
235
423
  Each ~Column~ has a header, a type, and an array of items, all of the given type
236
424
  or nil. There are only five permissible types for a ~Column~:
@@ -252,39 +440,38 @@ Items of input must be either one of the permissible ruby objects or strings. If
252
440
  they are strings, ~FatTable~ attempts to parse them as one of the permissible
253
441
  types as follows:
254
442
 
255
- - Boolean :: the strings, ~'t'~, ~'true'~, ~'yes'~, or ~'y'~, regardless of
256
- case, are interpreted as ~TrueClass~ and the strings, ~'f'~, ~'false'~,
257
- ~'no'~, or ~'n'~, regardless of case, are interpreted as ~FalseClass~, in
443
+ - Boolean :: The strings, ~t~, ~true~, ~yes~, or ~y~, regardless of
444
+ case, are interpreted as ~TrueClass~ and the strings, ~f~, ~false~,
445
+ ~no~, or ~n~, regardless of case, are interpreted as ~FalseClass~, in
258
446
  either case resulting in a Boolean column. Empty strings in a column
259
447
  already having a Boolean type are converted to ~nil~.
260
- - DateTime :: strings that contain patterns of ~'yyyy-mm-dd'~ or ~'yyyy/mm/dd'~
261
- or ~'mm-dd-yyy'~ or ~'mm/dd/yyyy'~ or any of the foregoing with an added
262
- ~'Thh:mm:ss'~ or ~'Thh:mm'~ will be interpreted as a ~DateTime~ or a ~Date~
448
+ - DateTime :: Strings that contain patterns of ~yyyy-mm-dd~ or ~yyyy/mm/dd~
449
+ or ~mm-dd-yyy~ or ~mm/dd/yyyy~ or any of the foregoing with an added
450
+ ~Thh:mm:ss~ or ~Thh:mm~ will be interpreted as a ~DateTime~ or a ~Date~
263
451
  (if there are no sub-day time components present). The number of digits in
264
452
  the month and day can be one or two, but the year component must be four
265
453
  digits. Any time components are valid if they can be properly interpreted
266
454
  by ~DateTime.parse~. Org mode timestamps (any of the foregoing surrounded
267
- by square '~[]~' or pointy '~<>~' brackets), active or inactive, are valid
455
+ by square ~[]~ or pointy ~<>~ brackets), active or inactive, are valid
268
456
  input strings for ~DateTime~ columns. Empty strings in a column already
269
457
  having the ~DateTime~ type are converted to ~nil~.
270
- - Numeric :: all commas ~','~, underscores, ~'_'~, and ~'$'~ dollar signs (or
271
- other currency symbol as set by ~FatTable.currency_symbol~ are removed from
272
- the string and if the remaining string can be interpreted as a ~Numeric~,
273
- it will be. It is interpreted as an ~Integer~ if there are no decimal
274
- places in the remaining string, as a ~Rational~ if the string has the form
275
- '~<number>:<number>~' or '~<number>/<number>~', or as a ~BigDecimal~ if
276
- there is a decimal point in the remaining string. Empty strings in a column
277
- already having the Numeric type are converted to nil.
278
- - String :: if all else fails, ~FatTable~ applies ~#to_s~ to the input value
458
+ - Numeric :: All commas (~,~) underscores (~_~) and (~$~) dollar signs (or
459
+ other currency symbol as set by ~FatTable.currency_symbol~ are removed from
460
+ the string and if the remaining string can be interpreted as a ~Numeric~, it
461
+ will be. It is interpreted as an ~Integer~ if there are no decimal places in
462
+ the remaining string, as a ~Rational~ if the string has the form
463
+ ~<number>:<number>~ or ~<number>/<number>~, or as a ~BigDecimal~ if there is
464
+ a decimal point in the remaining string. Empty strings in a column already
465
+ having the Numeric type are converted to nil.
466
+ - String :: If all else fails, ~FatTable~ applies ~#to_s~ to the input value
279
467
  and, treats it as an item of type ~String~. Empty strings in a column
280
468
  already having the ~String~ type are kept as empty strings.
281
- - NilClass :: until the input contains a non-blank string that can be parsed as
469
+ - NilClass :: Until the input contains a non-blank string that can be parsed as
282
470
  one of the other types, it has this type, meaning that the type is still
283
471
  open. A column comprised completely of blank strings or ~nils~ will retain
284
472
  the ~NilClass~ type.
285
473
 
286
474
  *** Headers
287
-
288
475
  Headers for the columns are formed from the input. No two columns in a table can
289
476
  have the same header. Headers in the input are converted to symbols by
290
477
 
@@ -293,33 +480,36 @@ have the same header. Headers in the input are converted to symbols by
293
480
  - removing any characters that are not letters, numbers, or underscores, and
294
481
  - lowercasing all remaining letters
295
482
 
296
- Thus, a header of ~'Date'~ becomes ~:date~, a header of ~'Id Number'~ becomes,
483
+ Thus, a header of ~Date~ becomes ~:date~, a header of ~Id Number~ becomes,
297
484
  ~:id_number~, etc. When referring to a column in code, you must use the symbol
298
485
  form of the header.
299
486
 
300
487
  If no sensible headers can be discerned from the input, headers of the form
301
488
  ~:col_1~, ~:col_2~, etc., are synthesized.
302
489
 
303
- *** Groups
490
+ You should avoid the use of the column names ~:omni~ and ~:sort_key~ because
491
+ they have special meanings in the ~select~ and ~order_with~ commands,
492
+ respectively.
304
493
 
305
- The rows of a ~FatTable~ table can be sub-divided into groups, either from
306
- markers in the input or as a result of certain operations. There is only one
307
- level of grouping, so ~FatTable~ has no concept of sub-groups. Groups can be
308
- shown on output with rules or "hlines" that underline the last row in each
309
- group, and you can decorate the output with group footers that summarize the
310
- columns in each group.
494
+ *** Groups
495
+ The rows of a ~FatTable~ table can be divided into groups, either from markers
496
+ in the input or as a result of certain operations. There is only one level of
497
+ grouping, so ~FatTable~ has no concept of sub-groups. Groups can be shown on
498
+ output with rules or "hlines" that underline the last row in each group, and
499
+ you can decorate the output with group footers that summarize the rows in
500
+ each group.
311
501
 
312
502
  ** Constructing Tables
313
503
  *** Empty Tables
504
+ **** Without Headers
505
+ You can create an empty table with ~FatTable::Table.new~ or, the shorter form,
506
+ ~FatTable.new~, and then add rows with the ~<<~ operator and a Hash. The keys
507
+ in the added rows determine the names of the headers:
314
508
 
315
- You can create an empty table with ~FatTable.new~, and then add rows with the
316
- ~<<~ operator and a Hash:
317
-
318
- #+BEGIN_SRC ruby
509
+ #+BEGIN_SRC ruby :results silent
319
510
  tab = FatTable.new
320
- tab << { a: 1, b: 2, c: "<2017-01-21>', d: 'f', e: '' }
511
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '' }
321
512
  tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
322
- tab.to_aoa
323
513
  #+END_SRC
324
514
 
325
515
  After this, the table will have column headers ~:a~, ~:b~, ~:c~, ~:d~, and ~:e~.
@@ -328,14 +518,115 @@ Column, ~:a~ and ~:b~ will have type Numeric, column ~:c~ will have type
328
518
  have an open type. Notice that dates in the input can be wrapped in brackets as
329
519
  in org-mode time stamps.
330
520
 
331
- *** From CSV or Org Mode files or strings
521
+ #+BEGIN_SRC ruby :wrap EXAMPLE
522
+ tab.to_text
523
+ #+END_SRC
524
+
525
+ #+begin_EXAMPLE
526
+ +======+======+============+===+===+
527
+ | A | B | C | D | E |
528
+ +------+------+------------+---+---+
529
+ | 1 | 2 | 2017-01-21 | F | |
530
+ | 3.14 | 2.17 | 2016-01-21 | T | |
531
+ +======+======+============+===+===+
532
+ #+end_EXAMPLE
533
+
534
+ You can continue to add rows to the table:
535
+ #+BEGIN_SRC ruby :results silent
536
+ tab << { 'F' => '335:113', a: Rational(3, 5) }
537
+ #+END_SRC
538
+
539
+ This last ~<<~ operation adds a new column headed ~:f~ to the table and makes
540
+ the value of =:f= in all prior rows ~nil~. Also, the values for the new row
541
+ for which no key was give are assigned ~nil~ as well:
542
+
543
+ #+BEGIN_SRC ruby
544
+ tab.to_text
545
+ #+END_SRC
546
+
547
+ #+begin_EXAMPLE
548
+ +======+======+============+===+===+=========+
549
+ | A | B | C | D | E | F |
550
+ +------+------+------------+---+---+---------+
551
+ | 1 | 2 | 2017-01-21 | F | | |
552
+ | 3.14 | 2.17 | 2016-01-21 | T | | |
553
+ +------+------+------------+---+---+---------+
554
+ | 3/5 | | | | | 335/113 |
555
+ +======+======+============+===+===+=========+
556
+ #+end_EXAMPLE
557
+
558
+ **** With Headers
559
+ Alternatively, you can specify the headers at the outset, in which case,
560
+ headers in added rows that do not match any of the initial headers cause new
561
+ columns to be created:
562
+
563
+ #+BEGIN_SRC ruby :wrap EXAMPLE :results raw
564
+ require 'fat_table'
565
+ tab = FatTable.new(:a, 'b', 'C', :d)
566
+ tab.headers
567
+ #+END_SRC
568
+
569
+ #+begin_EXAMPLE
570
+ [:a, :b, :c, :d]
571
+ #+end_EXAMPLE
572
+
573
+ #+begin_src ruby :wrap EXAMPLE
574
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '' }
575
+ tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
576
+ tab.to_text
577
+ #+end_src
578
+
579
+ #+begin_EXAMPLE
580
+ +======+======+============+===+===+
581
+ | A | B | C | D | E |
582
+ +------+------+------------+---+---+
583
+ | 1 | 2 | 2017-01-21 | F | |
584
+ | 3.14 | 2.17 | 2016-01-21 | T | |
585
+ +------+------+------------+---+---+
586
+ | 1 | 2 | 2017-01-21 | F | |
587
+ | 3.14 | 2.17 | 2016-01-21 | T | |
588
+ +======+======+============+===+===+
589
+ #+end_EXAMPLE
590
+
591
+ **** Forcing String Type
592
+ Occasionally, ~FatTable~'s automatic type detection can get in the way and you
593
+ just want it to treat one or more columns as Strings regardless of their
594
+ appearance. Think, for example, of zip codes. At any time after creating a
595
+ table, you can have it force the String type on any number of columns with the
596
+ ~force_string!~ method. When you do so, all exisiting items in the column are
597
+ converted to strings with the #to_s method.
598
+
599
+ #+begin_src ruby :wrap EXAMPLE
600
+ tab = FatTable.new(:a, 'b', 'C', :d, :zip)
601
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '', zip: 18552 }
602
+ tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
603
+ tab.force_string!(:zip, :c)
604
+ tab << { zip: '01879' }
605
+ tab << { zip: '66210' }
606
+ tab << { zip: '90210' }
607
+ tab.to_text
608
+ #+end_src
609
+
610
+ #+begin_EXAMPLE
611
+ +======+======+============+===+=======+===+
612
+ | A | B | C | D | Zip | E |
613
+ +------+------+------------+---+-------+---+
614
+ | 1 | 2 | 2017-01-21 | F | 18552 | |
615
+ | 3.14 | 2.17 | 2016-01-21 | T | | |
616
+ | | | | | 01879 | |
617
+ | | | | | 66210 | |
618
+ | | | | | 90210 | |
619
+ +======+======+============+===+=======+===+
620
+ #+end_EXAMPLE
332
621
 
622
+ *** From CSV or Org Mode files or strings
333
623
  Tables can also be read from ~.csv~ files or files containing ~org-mode~ tables.
624
+
334
625
  In the case of org-mode files, ~FatTable~ skips through the file until it finds
335
626
  a line that look like a table, that is, it begins with any number of spaces
336
627
  followed by ~|-~. Only the first table in an ~.org~ file is read.
337
628
 
338
- For both ~.csv~ and ~.org~ files, the first row in the tables is taken as the
629
+ For both ~.csv~ and ~.org~ files, the first row in the table is taken as the
339
630
  header row, and the headers are converted to symbols as described above.
340
631
 
341
632
  #+BEGIN_SRC ruby
@@ -385,7 +676,7 @@ header row, and the headers are converted to symbols as described above.
385
676
  #+END_SRC
386
677
 
387
678
  *** From Arrays of Arrays
388
-
679
+ **** In Ruby Code
389
680
  You can also initialize a table directly from ruby data structures. You can, for
390
681
  example, build a table from an array of arrays:
391
682
 
@@ -411,6 +702,7 @@ example, build a table from an array of arrays:
411
702
  Notice that the values can either be ruby objects, such as the Integer ~85_000~,
412
703
  or strings that can be parsed into one of the permissible column types.
413
704
 
705
+ **** In Emacs Org Files
414
706
  This method of building a table, ~.from_aoa~, is particularly useful in dealing
415
707
  with Emacs org-mode code blocks. Tables in org-mode are passed to code blocks as
416
708
  arrays of arrays. Likewise, a result of a code block in the form of an array of
@@ -488,7 +780,7 @@ to a Hash with the ~#to_h~ method, so you can use an array of your own objects
488
780
  to initialize a table, provided that you define a suitable ~#to_h~ method for
489
781
  the objects' class.
490
782
 
491
- #+BEGIN_SRC ruby
783
+ #+BEGIN_SRC ruby :results silent
492
784
  aoh = [
493
785
  { ref: 'T001', date: '2016-11-01', code: 'P', price: '7.7000', shares: 100 },
494
786
  { ref: 'T002', date: '2016-11-01', code: 'P', price: 7.7500, shares: 200 },
@@ -514,7 +806,6 @@ Notice, again, that the values can either be ruby objects, such as ~Date.today~,
514
806
  or strings that can be parsed into one of the permissible column types.
515
807
 
516
808
  *** From SQL queries
517
-
518
809
  Another way to initialize a ~FatTable~ table is with the results of a SQL
519
810
  query. Before you can connect to a database, you need to make sure that the required
520
811
  adapter for your database is installed. ~FatTable~ uses the ~sequel~ gem
@@ -529,15 +820,30 @@ You must first set the database parameters to be used for the queries.
529
820
 
530
821
  #+BEGIN_SRC ruby
531
822
  # This automatically requires sequel.
532
- require 'fat_table'
533
- FatTable.connect(adapter: 'postgres',
534
- database: 'XXX_development',
535
- user: 'ken',
536
- password: 'imsecret',
537
- host: 'db.lan')
538
- tab = FatTable.from_sql('select * from trades;')
823
+ FatTable.connect(adapter: 'sqlite',
824
+ database: 'examples/trades.db')
825
+ tab = FatTable.from_sql('select * from trans;').to_text
539
826
  #+END_SRC
540
827
 
828
+ #+begin_example
829
+ +============+======+==========+==========+=========+=========+====+
830
+ | Date | Code | Raw | Shares | Price | Info | Ok |
831
+ +------------+------+----------+----------+---------+---------+----+
832
+ | 2013-05-29 | S | 15700.0 | 6601.85 | 24.779 | ENTITY3 | F |
833
+ | 2013-05-02 | P | 118186.4 | 118186.4 | 11.85 | ENTITY1 | T |
834
+ | 2013-05-20 | S | 12000.0 | 5046.0 | 28.2804 | ENTITY3 | F |
835
+ | 2013-05-23 | S | 8000.0 | 3364.0 | 27.1083 | ENTITY3 | T |
836
+ | 2013-05-23 | S | 39906.0 | 16780.47 | 25.1749 | ENTITY3 | T |
837
+ | 2013-05-20 | S | 85000.0 | 35742.5 | 28.3224 | ENTITY3 | T |
838
+ | 2013-05-02 | P | 795546.2 | 795546.2 | 1.185 | ENTITY1 | T |
839
+ | 2013-05-29 | S | 13459.0 | 5659.51 | 24.7464 | ENTITY3 | T |
840
+ | 2013-05-20 | S | 33302.0 | 14003.49 | 28.6383 | ENTITY3 | T |
841
+ | 2013-05-29 | S | 15900.0 | 6685.95 | 24.5802 | ENTITY3 | T |
842
+ | 2013-05-30 | S | 6679.0 | 2808.52 | 25.0471 | ENTITY3 | T |
843
+ | 2013-05-23 | S | 23054.0 | 9694.21 | 26.8015 | ENTITY3 | F |
844
+ +============+======+==========+==========+=========+=========+====+
845
+ #+end_example
846
+
541
847
  The arguments to ~connect~ are simply passed on to ~sequel~'s connect method, so
542
848
  any set of arguments that work for it should work for ~connect~. Alternatively,
543
849
  you can build the ~Sequel~ connection directly with ~Sequel.connect~ or with
@@ -545,7 +851,6 @@ adapter-specific ~Sequel~ connection methods and let ~FatTable~ know to use that
545
851
  connection:
546
852
 
547
853
  #+BEGIN_SRC ruby
548
- require 'fat_table'
549
854
  FatTable.db = Sequel.connect('postgres://user:password@localhost/dbname')
550
855
  FatTable.db = Sequel.ado(conn_string: 'Provider=Microsoft.ACE.OLEDB.12.0;Data Source=drive:\path\filename.accdb')
551
856
  #+END_SRC
@@ -558,6 +863,13 @@ creates will be used for all subsequent ~.from_sql~ calls until ~.connect~ is
558
863
  called again.
559
864
 
560
865
  *** Marking Groups in Input
866
+ **** Manually
867
+ At any point, you can add a boundary to a table by invokong the
868
+ ~mark_boundary~ method. Without an argument, it adds the boundary to the end
869
+ of the table; with a numeric argument, ~n~, it adds the boundary after row
870
+ ~n~.
871
+
872
+ **** When Reading in Tables
561
873
 
562
874
  ~FatTable~ tables has a concept of "groups" of rows that play a role in many of
563
875
  the methods for operating on them as explained [[Groups][below]].
@@ -577,13 +889,12 @@ the form ~:col_1~, ~:col_2~, ... ~:col_n~.
577
889
  In org mode table text passed to ~.from_org_file~ and ~.from_org_string~, you
578
890
  /must/ mark the header row by following it with an hrule and you /may/ mark
579
891
  group boundaries with an hrule. In org mode tables, hlines are table rows
580
- beginning with something like '~|---~'. The ~.from_org_...~ functions always
892
+ beginning with something like ~|---~. The ~.from_org_...~ functions always
581
893
  recognizes hlines in the input, so it takes no ~hlines:~ keyword parameter.
582
894
 
583
895
  ** Accessing Parts of Tables
584
896
 
585
897
  *** Rows
586
-
587
898
  A ~FatTable~ table is an Enumerable, yielding each row of the table as a Hash
588
899
  keyed on the header symbols. The method ~Table#rows~ returns an Array of the
589
900
  rows as Hashes as well.
@@ -594,15 +905,13 @@ data row of the table, while ~tab[0]~ returns the first row and tab[-1] returns
594
905
  the last row.
595
906
 
596
907
  *** Columns
597
-
598
908
  If the index provided to ~[]~ is a string or a symbol, it returns an Array of
599
909
  the items of the column with that header. Thus, ~tab[:ref]~ returns an Array of
600
910
  all the items of the table's ~:ref~ column.
601
911
 
602
912
  *** Cells
603
-
604
- The two forms of indexing can be combined to access individual cells of the
605
- table:
913
+ The two forms of indexing can be combined, in either order, to access
914
+ individual cells of the table:
606
915
 
607
916
  #+BEGIN_SRC ruby
608
917
  tab[13] # => Hash of the 14th row
@@ -612,53 +921,76 @@ table:
612
921
  #+END_SRC
613
922
 
614
923
  *** Other table attributes
924
+ Here is a quick rundown of other table attributes that you can access:
615
925
 
616
926
  #+BEGIN_SRC ruby
617
927
  tab.headers # => an Array of the headers in symbol form
618
928
  tab.types # => a Hash mapping headers to column types
929
+ tab.type(head) # => return the type of the column for the given head
619
930
  tab.size # => the number of rows in the table
620
931
  tab.width # => the number of columns in the table
621
932
  tab.empty? # => is the table empty?
622
- tab.column?(head) # => does the table have a column with the given header?
933
+ tab.column(head) # => return the FatTable::Column object for the given head
934
+ tab.column?(head) # => does the table have a column with the given head?
623
935
  tab.groups # => return an Array of the table's groups as Arrays of row Hashes.
624
936
  #+END_SRC
625
937
 
626
- ** Operations on Tables
938
+ You should note that what the ~.types~ and ~.type(head)~ methods return is a
939
+ string naming the "type" assigned by ~FatTable~. All of them are also the
940
+ names of Ruby classes except to 'Boolean' a class that doesn't exist in Ruby.
941
+ The value ~true~ is a member of the ~TrueClass~ and ~false~ a member of the
942
+ ~FalseClass~. So for ~FatTable~ to provide a column of type 'Boolean'
943
+ requires it to synthesize the type from these Ruby classes.
944
+
945
+ #+begin_src ruby :wrap EXAMPLE :results raw
946
+ tab.types
947
+ #+end_src
948
+
949
+ #+begin_EXAMPLE
950
+ {:a=>"Numeric", :b=>"Numeric", :c=>"DateTime", :d=>"Boolean", :e=>"NilClass", :f=>"Numeric"}
951
+ #+end_EXAMPLE
952
+
953
+ #+begin_src ruby :wrap EXAMPLE :results output
954
+ puts "Column :d says its type is '#{tab.type(:d)}' and that is a #{tab.type(:d).class}"
955
+ #+end_src
956
+
957
+ #+begin_EXAMPLE
958
+ Column :d says its type is 'Boolean' and that is a String
959
+ #+end_EXAMPLE
627
960
 
961
+ ** Operations on Tables
628
962
  Once you have one or more tables, you will likely want to perform operations on
629
963
  them. The operations provided by ~FatTable~ are the subject of this section.
630
964
  Before getting into the operations, though, there are a couple of issues that
631
965
  cut across all or many of the operations.
632
966
 
633
967
  First, tables are by and large immutable objects. Each operation creates a new
634
- table without affecting the input tables. The only exception is the ~degroup!~
635
- operation, which mutates the receiver table by removing its group boundaries.
968
+ table without affecting the input tables. The only exceptions are the
969
+ ~degroup!~ operation, which mutates the receiver table by removing its group
970
+ boundaries, and ~force_string!~ (explained above at [[*Forcing String Type][Forcing String Type]]),
971
+ which forces columns to have the String type despite what the automatic typing
972
+ rules determine.
636
973
 
637
974
  Second, because each operation returns a ~FatTable::Table~ object, the
638
975
  operations are chainable.
639
976
 
640
- <<Groups>>
641
977
  Third, ~FatTable::Table~ objects can have "groups" of rows within the table.
642
- These can be decorated with hlines and group footers on output. Some of these
643
- operations result in marking group boundaries in the result table, others remove
644
- group boundaries that may have existed in the input table. Operations that
645
- either create or remove groups will be noted below.
978
+ These can be decorated with hlines and group footers on output. Some
979
+ operations result in marking group boundaries in the result table, others
980
+ remove group boundaries that may have existed in the input table. Operations
981
+ that either create or remove groups will be noted below.
646
982
 
647
983
  Finally, the operations are for the most part patterned on SQL table operations,
648
984
  but when expressions play a role, you write them using ruby syntax rather than
649
985
  SQL.
650
986
 
651
- *** Example Input Table
652
-
987
+ *** Example Input Tables
653
988
  For illustration purposes assume that the following tables are read into ruby
654
- variables called '~tab1~' and '~tab2~. We have given the table groups, marked by
989
+ variables called ~tab1~ and ~tab2~. We have given the table groups, marked by
655
990
  the hlines below, and included some duplicate rows to illustrate the effect of
656
991
  certain operations on groups and duplicates.
657
992
 
658
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
659
- #+BEGIN_SRC ruby
660
- require 'fat_table'
661
-
993
+ #+BEGIN_SRC ruby :results silent
662
994
  tab1_str = <<-EOS
663
995
  | Ref | Date | Code | Price | G10 | QP10 | Shares | LP | QP | IPLP | IPQP |
664
996
  |------+------------------+------+--------+-----+------+--------+------+-------+--------+--------|
@@ -708,26 +1040,31 @@ tab2_str = <<-EOS
708
1040
  | T021 | [2017-01-23 Mon] | P | 7.16 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
709
1041
  | T021 | [2017-01-23 Mon] | P | 7.16 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
710
1042
  EOS
1043
+ #+END_SRC
711
1044
 
712
- tab1 = FatTable.from_org_string(tab1_str)
713
- tab2 = FatTable.from_org_string(tab2_str)
1045
+ Rendering ~tab1~ into Emacs org-mode:
1046
+ #+BEGIN_SRC ruby :wrap EXAMPLE :results silent
1047
+ tab1 = FatTable.from_org_string(tab1_str)
714
1048
  #+END_SRC
715
1049
 
716
- *** Select
1050
+ Rendering ~tab2~ into Emacs org-mode:
717
1051
 
718
- With the ~select~ method, you can select which existing columns should appear in
719
- the output table and create new columns in the output table that are a function
720
- of existing and new columns.
1052
+ #+BEGIN_SRC ruby :wrap EXAMPLE :results silent
1053
+ tab2 = FatTable.from_org_string(tab2_str)
1054
+ #+END_SRC
721
1055
 
722
- **** Selecting Existing Columns
1056
+ *** Select
1057
+ With the ~select~ method, you can select columns to appear in the output
1058
+ table, rearrange their order, and create new columns that are a function of
1059
+ other columns.
723
1060
 
1061
+ **** Selecting Existing Columns (Also of :omni)
724
1062
  Here we select three existing columns by simply passing header symbols in the
725
1063
  order we want them to appear in the output. Thus, one use of =select= is to
726
1064
  filter and permute the order of existing columns. The =select= method preserves
727
1065
  any group boundaries present in the input table.
728
1066
 
729
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
730
- #+BEGIN_SRC ruby
1067
+ #+BEGIN_SRC ruby :wrap EXAMPLE
731
1068
  tab1.select(:price, :ref, :shares).to_aoa
732
1069
  #+END_SRC
733
1070
 
@@ -758,15 +1095,91 @@ any group boundaries present in the input table.
758
1095
  | 8.25 | T016 | 100 |
759
1096
  #+END_EXAMPLE
760
1097
 
761
- **** Adding New Columns
1098
+ It can be tedious to type the names of all the columns in a ~select~
1099
+ statement, so ~FatTable~ recognizes the special column name ~:omni~. If the
1100
+ ~select~'s first and only column argument is ~:omni~, it will expand to the
1101
+ names of all the existing columns in the table. Use of ~:omni~ otherwise is
1102
+ not interpreted specially, so you will get an error complaining about a
1103
+ non-existent column unless you happen to have a column named ~:omni~ in your
1104
+ table, which is not advisable. You can add hash arguments after ~:omni~ but
1105
+ you cannot add additional column names:
1106
+
1107
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1108
+ tab1.select(:omni, cost: 'shares * price').to_aoa
1109
+ #+END_SRC
1110
+
1111
+ #+begin_EXAMPLE
1112
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp | Cost |
1113
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1114
+ | T001 | 2016-11-01 | P | 7.7 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 770.0 |
1115
+ | T002 | 2016-11-01 | P | 7.75 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1550.0 |
1116
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0 |
1117
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0 |
1118
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1119
+ | T004 | 2016-11-01 | S | 7.55 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 | 51423.05 |
1120
+ | T005 | 2016-11-01 | S | 7.5 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 | 30000.0 |
1121
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0 |
1122
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0 |
1123
+ | T007 | 2016-11-01 | S | 7.65 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1530.0 |
1124
+ | T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 | 21198.15 |
1125
+ | T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 | 72580.0 |
1126
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1127
+ | T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 | 23971.25 |
1128
+ | T011 | 2016-11-02 | P | 7.425 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 742.5 |
1129
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0 |
1130
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0 |
1131
+ | T013 | 2016-11-02 | P | 7.35 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 | 390285.0 |
1132
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1133
+ | T014 | 2016-11-02 | P | 7.45 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 | 43560.15 |
1134
+ | T015 | 2016-11-02 | P | 7.75 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 | 3875.0 |
1135
+ | T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 | 825.0 |
1136
+ #+end_EXAMPLE
1137
+
1138
+ **** Copying and Renaming Existing Columns
1139
+ After the list of selected column names in the call to ~select~, you can add
1140
+ any number of hash-like arguments. You can use these to add a copy of an
1141
+ existing column. By calling select again, you can include only the copied
1142
+ column, in effect renaming it. For example, if you want ~tab1~ but with ~:ref~
1143
+ changed to ~:id~, just add an argument to define the new ~:id~ column:
1144
+
1145
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1146
+ tab1.select(:omni, id: :ref).
1147
+ select(:id, :date, :code, :price, :shares).to_aoa
1148
+ #+END_SRC
1149
+
1150
+ #+begin_EXAMPLE
1151
+ | Id | Date | Code | Price | Shares |
1152
+ |------+------------+------+-------+--------|
1153
+ | T001 | 2016-11-01 | P | 7.7 | 100 |
1154
+ | T002 | 2016-11-01 | P | 7.75 | 200 |
1155
+ | T003 | 2016-11-01 | P | 7.5 | 800 |
1156
+ | T003 | 2016-11-01 | P | 7.5 | 800 |
1157
+ |------+------------+------+-------+--------|
1158
+ | T004 | 2016-11-01 | S | 7.55 | 6811 |
1159
+ | T005 | 2016-11-01 | S | 7.5 | 4000 |
1160
+ | T006 | 2016-11-01 | S | 7.6 | 1000 |
1161
+ | T006 | 2016-11-01 | S | 7.6 | 1000 |
1162
+ | T007 | 2016-11-01 | S | 7.65 | 200 |
1163
+ | T008 | 2016-11-01 | P | 7.65 | 2771 |
1164
+ | T009 | 2016-11-01 | P | 7.6 | 9550 |
1165
+ |------+------------+------+-------+--------|
1166
+ | T010 | 2016-11-01 | P | 7.55 | 3175 |
1167
+ | T011 | 2016-11-02 | P | 7.425 | 100 |
1168
+ | T012 | 2016-11-02 | P | 7.55 | 4700 |
1169
+ | T012 | 2016-11-02 | P | 7.55 | 4700 |
1170
+ | T013 | 2016-11-02 | P | 7.35 | 53100 |
1171
+ |------+------------+------+-------+--------|
1172
+ | T014 | 2016-11-02 | P | 7.45 | 5847 |
1173
+ | T015 | 2016-11-02 | P | 7.75 | 500 |
1174
+ | T016 | 2016-11-02 | P | 8.25 | 100 |
1175
+ #+end_EXAMPLE
762
1176
 
763
- More interesting is that ~select~ can take hash-like keyword arguments after the
764
- symbol arguments to create new columns in the output as functions of other
765
- columns. For each hash-like parameter, the keyword given must be a symbol, which
766
- becomes the header for the new column, and the value must be either: (1) a
767
- symbol representing an existing column, which has the effect of renaming an
768
- existing column, or (2) a string representing a ruby expression for the value of
769
- a new column.
1177
+ **** Adding New Columns
1178
+ More interesting is that ~select~ can take hash-like keyword arguments after
1179
+ the symbol arguments to create new columns in the output as functions of other
1180
+ columns. For each hash-like parameter, the keyword given must be a symbol,
1181
+ which becomes the header for the new column, and the value can be a string
1182
+ representing a ruby expression for the value of a new column.
770
1183
 
771
1184
  Within the string expression, the names of existing or already-specified
772
1185
  columns are available as local variables. In addition the instance variables
@@ -781,8 +1194,7 @@ variables are set the number of the current row and group number respectively.
781
1194
  For example, if we want to rename the ~traded_on~ column to ~:date~ and add a
782
1195
  new column to compute the cost of shares, we could do the following:
783
1196
 
784
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
785
- #+BEGIN_SRC ruby
1197
+ #+BEGIN_SRC ruby :wrap EXAMPLE
786
1198
  tab1.select(:ref, :price, :shares, traded_on: :date, cost: 'price * shares').to_aoa
787
1199
  #+END_SRC
788
1200
 
@@ -813,8 +1225,8 @@ new column to compute the cost of shares, we could do the following:
813
1225
  | T016 | 8.25 | 100 | 2016-11-02 | 825.0 |
814
1226
  #+END_EXAMPLE
815
1227
 
816
- The parameter '~traded_on: :date~' caused the ~:date~ column of the input table
817
- to be renamed '~:traded_on~, and the parameter ~cost: 'price * shares'~ created
1228
+ The parameter ~traded_on: :date~ caused the ~:date~ column of the input table
1229
+ to be renamed ~:traded_on~, and the parameter ~cost: 'price * shares'~ created
818
1230
  a new column, ~:cost~, as the product of values in the ~:price~ and ~:shares~
819
1231
  columns.
820
1232
 
@@ -822,14 +1234,12 @@ The order of the columns in the result tables is the same as the order of the
822
1234
  parameters to the ~select~ method. So, you can re-order the columns with a
823
1235
  second, chained call to ~select~:
824
1236
 
825
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
826
- #+BEGIN_SRC ruby
827
- tab1.select(:ref, :price, :shares, traded_on: :date, cost: 'price * shares') \
828
- .select(:ref, :traded_on, :price, :shares, :cost) \
829
- .to_aoa
1237
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1238
+ tab1.select(:ref, :price, :shares, traded_on: :date, cost: 'price * shares').
1239
+ select(:ref, :traded_on, :price, :shares, :cost).to_aoa
830
1240
  #+END_SRC
831
1241
 
832
- #+BEGIN_EXAMPLE
1242
+ #+begin_EXAMPLE
833
1243
  | Ref | Traded On | Price | Shares | Cost |
834
1244
  |------+------------+-------+--------+----------|
835
1245
  | T001 | 2016-11-01 | 7.7 | 100 | 770.0 |
@@ -854,10 +1264,9 @@ second, chained call to ~select~:
854
1264
  | T014 | 2016-11-02 | 7.45 | 5847 | 43560.15 |
855
1265
  | T015 | 2016-11-02 | 7.75 | 500 | 3875.0 |
856
1266
  | T016 | 2016-11-02 | 8.25 | 100 | 825.0 |
857
- #+END_EXAMPLE
1267
+ #+end_EXAMPLE
858
1268
 
859
1269
  **** Custom Instance Variables and Hooks
860
-
861
1270
  As the above examples demonstrate, the instance variables ~@row~ and ~@group~
862
1271
  are available when evaluating expressions that add new columns. You can also set
863
1272
  up your own instance variables as well for keeping track of things that cross
@@ -878,8 +1287,7 @@ shows the cumulative cost after each transaction in our example table. The
878
1287
  following example uses the ~ivars:~ and ~before_hook:~ parameters to keep track
879
1288
  of the running cost of shares, then formats the table.
880
1289
 
881
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
882
- #+BEGIN_SRC ruby
1290
+ #+BEGIN_SRC ruby :wrap EXAMPLE
883
1291
  tab = tab1.select(:ref, :price, :shares, traded_on: :date, \
884
1292
  cost: 'price * shares', cumulative: '@total_cost', \
885
1293
  ivars: { total_cost: 0 }, \
@@ -917,7 +1325,6 @@ of the running cost of shares, then formats the table.
917
1325
  #+END_EXAMPLE
918
1326
 
919
1327
  **** Argument Order and Boundaries
920
-
921
1328
  Notice that ~select~ can take any number of arguments but all the symbol
922
1329
  arguments must come first followed by all the hash-like keyword arguments,
923
1330
  including the special arguments for instance variables and hooks.
@@ -926,21 +1333,21 @@ As the example illustrates, ~.select~ transmits any group boundaries in its
926
1333
  input table to the result table.
927
1334
 
928
1335
  *** Where
929
-
930
1336
  You can filter the rows of the result table with the ~.where~ method. It takes a
931
1337
  single string expression as an argument which is evaluated in a manner similar
932
1338
  to ~.select~ in which the value of the cells in each column are available as
933
1339
  local variables and the instance variables ~@row~ and ~@group~ are available for
934
1340
  testing. The expression is evaluated for each row, and if the expression
935
1341
  evaluates to a truthy value, the row is included in the output, otherwise it is
936
- not. The ~.where~ method obliterates any group boundaries in the input, so the
937
- output table has only a single group.
1342
+ not.
1343
+
1344
+ The ~.where~ method removes any group boundaries in the input, so the output
1345
+ table has only a single group.
938
1346
 
939
1347
  Here we select only those even-numbered rows where either of the two boolean
940
1348
  fields is true:
941
1349
 
942
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
943
- #+BEGIN_SRC ruby
1350
+ #+BEGIN_SRC ruby :wrap EXAMPLE
944
1351
  tab1.where('@row.even? && (g10 || qp10)') \
945
1352
  .to_aoa
946
1353
  #+END_SRC
@@ -956,7 +1363,6 @@ fields is true:
956
1363
  #+END_EXAMPLE
957
1364
 
958
1365
  *** Order_by
959
-
960
1366
  You can sort a table on any number of columns with ~order_by~. The ~order_by~
961
1367
  method takes any number of symbol arguments for the columns to sort on. If you
962
1368
  specify more than one column, the sort is performed on the first column, then
@@ -967,8 +1373,7 @@ All columns of the input table are included in the output.
967
1373
 
968
1374
  Let's sort our table first by ~:code~, then in reverse order of ~:date~.
969
1375
 
970
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
971
- #+BEGIN_SRC ruby
1376
+ #+BEGIN_SRC ruby :wrap EXAMPLE
972
1377
  tab1.order_by(:code, :date!) \
973
1378
  .to_aoa
974
1379
  #+END_SRC
@@ -1004,8 +1409,59 @@ input, it adds group boundaries in the output table at those rows where the sort
1004
1409
  keys change. Thus, in each group, ~:code~ and ~:date~ are the same, and when
1005
1410
  either changes, ~order_by~ inserts a group boundary.
1006
1411
 
1007
- *** Group_by
1412
+ *** Order_with
1413
+ The ~order_with~ method is a convenient combination of ~select~ and
1414
+ ~order_by~. It takes a single string expression as an argument to serve as a
1415
+ sort key---one that would be valid as a select expression---but with an
1416
+ optional trailing ~!~ to indicate reverse sort. The resulting table has an
1417
+ additional column called ~:sort_key~ with the expression evaluated for each
1418
+ row, and the table is sorted as with ~order_by~ on that column.
1419
+
1420
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1421
+ tab1.order_with('price * shares').to_aoa
1422
+ #+END_SRC
1423
+
1424
+ #+begin_EXAMPLE
1425
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp | Sort Key |
1426
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1427
+ | T011 | 2016-11-02 | P | 7.425 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 742.5 |
1428
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1429
+ | T001 | 2016-11-01 | P | 7.7 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 770.0 |
1430
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1431
+ | T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 | 825.0 |
1432
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1433
+ | T007 | 2016-11-01 | S | 7.65 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1530.0 |
1434
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1435
+ | T002 | 2016-11-01 | P | 7.75 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1550.0 |
1436
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1437
+ | T015 | 2016-11-02 | P | 7.75 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 | 3875.0 |
1438
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1439
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0 |
1440
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0 |
1441
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1442
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0 |
1443
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0 |
1444
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1445
+ | T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 | 21198.15 |
1446
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1447
+ | T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 | 23971.25 |
1448
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1449
+ | T005 | 2016-11-01 | S | 7.5 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 | 30000.0 |
1450
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1451
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0 |
1452
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0 |
1453
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1454
+ | T014 | 2016-11-02 | P | 7.45 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 | 43560.15 |
1455
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1456
+ | T004 | 2016-11-01 | S | 7.55 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 | 51423.05 |
1457
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1458
+ | T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 | 72580.0 |
1459
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1460
+ | T013 | 2016-11-02 | P | 7.35 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 | 390285.0 |
1461
+ #+end_EXAMPLE
1008
1462
 
1463
+
1464
+ *** Group_by
1009
1465
  Like ~order_by~, ~group_by~ takes a set of parameters of column header symbols,
1010
1466
  the "grouping parameters", by which to sort the table into a set of groups that
1011
1467
  are equal with respect to values in those columns. In addition, those parameters
@@ -1019,8 +1475,7 @@ For example, let's summarize the ~trades~ table by ~:code~ and ~:price~ again,
1019
1475
  and determine total shares, average price, and a few other features of each
1020
1476
  group:
1021
1477
 
1022
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1023
- #+BEGIN_SRC ruby
1478
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1024
1479
  tab1.group_by(:code, :date, price: :avg,
1025
1480
  shares: :sum, lp: :sum, qp: :sum,
1026
1481
  qp10: :all?) \
@@ -1040,7 +1495,7 @@ hash-like "aggregating" parameters where the key is the column to aggregate and
1040
1495
  the value is a symbol for one of several aggregating methods that
1041
1496
  ~FatTable::Column~ objects understand. For example, the ~:avg~ method is applied
1042
1497
  to the :price column so that the output shows the average price in each group.
1043
- The ~:shares~, ~:lp~, and ~:qp~ columns are summed, and the ~:any?~ aggregate is
1498
+ The ~:shares~, ~:lp~, and ~:qp~ columns are summed, and the ~:all?~ aggregate is
1044
1499
  applied to one of the boolean fields, that is, it is ~true~ if any of the values
1045
1500
  in that column are ~true~.
1046
1501
 
@@ -1053,15 +1508,15 @@ will raise an exception.
1053
1508
 
1054
1509
  - ~first~ :: the first non-nil item in the column,
1055
1510
  - ~last~ :: the last non-nil item in the column,
1056
- - ~rng~ :: form a string of the form ~"#{first}..#{last}"~ to show the range of
1057
- values in the column,
1058
- - ~sum~ :: for ~Numeric~ and ~String~ columns, apply '+' to all the non-nil
1059
- values,
1511
+ - ~range~ :: form a Range ~~{min}..{max}~ to show the range of values in the
1512
+ column,
1513
+ - ~sum~ :: for ~Numeric~ columns, apply '+' to all the non-nil
1514
+ values; for ~String~ columns, join the elements with a single space,
1060
1515
  - ~count~ :: the number of non-nil values in the column,
1061
1516
  - ~min~ :: for ~Numeric~, ~String~, and ~DateTime~ columns, return the smallest
1062
- non-nil value in the column,
1517
+ non-nil, non-blank value in the column,
1063
1518
  - ~max~ :: for ~Numeric~, ~String~, and ~DateTime~ columns, return the largest
1064
- non-nil value in the column,
1519
+ non-nil, non-blank value in the column,
1065
1520
  - ~avg~ :: for ~Numeric~ and ~DateTime~ columns, return the arithmetic mean of
1066
1521
  the non-nil values in the column; with respect to ~Date~ or ~DateTime~
1067
1522
  objects, each is converted to a numeric Julian date, the average is
@@ -1093,7 +1548,6 @@ implicit ~order_by~ on the grouping columns is collapsed into a single row.
1093
1548
 
1094
1549
  *** Join
1095
1550
  **** Join Types
1096
-
1097
1551
  So far, all the operations have operated on a single table. ~FatTable~ provides
1098
1552
  several ~join~ methods for combining two tables, each of which takes as
1099
1553
  parameters (1) a second table and (2) except in the case of ~cross_join~, zero
@@ -1130,7 +1584,6 @@ for inclusion in the joined output table.
1130
1584
  M~ rows.
1131
1585
 
1132
1586
  **** Join Expressions
1133
-
1134
1587
  For each of the join types, if no join expressions are given, the tables will be
1135
1588
  joined on columns having the same column header in both tables, and the join
1136
1589
  condition is satisfied when all the values in those columns are equal. If the
@@ -1158,48 +1611,74 @@ local variables within the expression, but the instance variables ~@row~ and
1158
1611
  ~@group~ are not.
1159
1612
 
1160
1613
  **** Join Examples
1161
-
1162
1614
  The following examples are taken from the [[https://www.tutorialspoint.com/postgresql/postgresql_using_joins.htm][Postgresql tutorial]], with some slight
1163
1615
  modifications. The examples will use the following two tables, which are also
1164
1616
  available in ~ft_console~ as ~@tab_a~ and ~@tab_b~:
1165
1617
 
1166
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1167
- #+BEGIN_SRC ruby
1168
- require 'fat_table'
1169
-
1170
- tab_a_str = <<-EOS
1171
- | Id | Name | Age | Address | Salary | Join Date |
1172
- |----+-------+-----+------------+--------+------------|
1173
- | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
1174
- | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
1175
- | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
1176
- | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
1177
- | 2 | Allen | 25 | Texas | | 2005-07-13 |
1178
- | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
1179
- | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
1180
- | 10 | James | 45 | Texas | 5000 | |
1181
- EOS
1618
+ #+BEGIN_SRC ruby :wrap EXAMPLE :results silent
1619
+ tab_a_str = <<-EOS
1620
+ | Id | Name | Age | Address | Salary | Join Date |
1621
+ |----+-------+-----+------------+--------+------------|
1622
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
1623
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
1624
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
1625
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
1626
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
1627
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
1628
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
1629
+ | 10 | James | 45 | Texas | 5000 | |
1630
+ EOS
1182
1631
 
1183
- tab_b_str = <<-EOS
1184
- | Id | Dept | Emp Id |
1185
- |----+-------------+--------|
1186
- | 1 | IT Billing | 1 |
1187
- | 2 | Engineering | 2 |
1188
- | 3 | Finance | 7 |
1189
- EOS
1632
+ tab_b_str = <<-EOS
1633
+ | Id | Dept | Emp Id |
1634
+ |----+-------------+--------|
1635
+ | 1 | IT Billing | 1 |
1636
+ | 2 | Engineering | 2 |
1637
+ | 3 | Finance | 7 |
1638
+ EOS
1190
1639
 
1191
- tab_a = FatTable.from_org_string(tab_a_str)
1192
- tab_b = FatTable.from_org_string(tab_b_str)
1640
+ tab_b = FatTable.from_org_string(tab_b_str)
1193
1641
  #+END_SRC
1194
1642
 
1195
- ***** Inner Joins
1643
+ Here is ~tab_a~:
1644
+ #+begin_src ruby :wrap EXAMPLE
1645
+ tab_a = FatTable.from_org_string(tab_a_str)
1646
+ tab_a.to_aoa
1647
+ #+end_src
1648
+
1649
+ #+begin_EXAMPLE
1650
+ | Id | Name | Age | Address | Salary | Join Date |
1651
+ |----+-------+-----+------------+--------+------------|
1652
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
1653
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
1654
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
1655
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
1656
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
1657
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
1658
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
1659
+ | 10 | James | 45 | Texas | 5000 | |
1660
+ #+end_EXAMPLE
1661
+
1662
+ And ~tab_b~:
1663
+ #+begin_src ruby :wrap EXAMPLE
1664
+ tab_b = FatTable.from_org_string(tab_b_str)
1665
+ tab_b.to_aoa
1666
+ #+end_src
1667
+
1668
+ #+begin_EXAMPLE
1669
+ | Id | Dept | Emp Id |
1670
+ |----+-------------+--------|
1671
+ | 1 | IT Billing | 1 |
1672
+ | 2 | Engineering | 2 |
1673
+ | 3 | Finance | 7 |
1674
+ #+end_EXAMPLE
1196
1675
 
1676
+ ***** Inner Joins
1197
1677
  With no join expression arguments, the tables are joined when their sole common
1198
1678
  field, ~:id~, is equal in both tables. The result is the natural join of the
1199
1679
  two tables.
1200
1680
 
1201
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1202
- #+BEGIN_SRC ruby
1681
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1203
1682
  tab_a.join(tab_b).to_aoa
1204
1683
  #+END_SRC
1205
1684
 
@@ -1216,8 +1695,7 @@ in the second table. To correct this, we need to explicitly state the columns we
1216
1695
  want to join on in each table by disambiguating them with ~_a~ and ~_b~
1217
1696
  suffixes:
1218
1697
 
1219
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1220
- #+BEGIN_SRC ruby
1698
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1221
1699
  tab_a.join(tab_b, :id_a, :emp_id_b).to_aoa
1222
1700
  #+END_SRC
1223
1701
 
@@ -1232,8 +1710,7 @@ Instead of using the disambiguated column names as symbols, we could also use a
1232
1710
  string containing a ruby expression. Within the expression, the column names
1233
1711
  should be treated as local variables:
1234
1712
 
1235
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1236
- #+BEGIN_SRC ruby
1713
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1237
1714
  tab_a.join(tab_b, 'id_a == emp_id_b').to_aoa
1238
1715
  #+END_SRC
1239
1716
 
@@ -1245,12 +1722,10 @@ should be treated as local variables:
1245
1722
  #+END_EXAMPLE
1246
1723
 
1247
1724
  ***** Left and Right Joins
1248
-
1249
1725
  In left join, all the rows of ~tab_a~ are included in the output, augmented by
1250
1726
  the matching columns of ~tab_b~ and augmented with nils where there is no match:
1251
1727
 
1252
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1253
- #+BEGIN_SRC ruby
1728
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1254
1729
  tab_a.left_join(tab_b, 'id_a == emp_id_b').to_aoa
1255
1730
  #+END_SRC
1256
1731
 
@@ -1271,8 +1746,7 @@ In a right join, all the rows of ~tab_b~ are included in the output, augmented
1271
1746
  by the matching columns of ~tab_a~ and augmented with nils where there is no
1272
1747
  match:
1273
1748
 
1274
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1275
- #+BEGIN_SRC ruby
1749
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1276
1750
  tab_a.right_join(tab_b, 'id_a == emp_id_b').to_aoa
1277
1751
  #+END_SRC
1278
1752
 
@@ -1285,13 +1759,11 @@ match:
1285
1759
  #+END_EXAMPLE
1286
1760
 
1287
1761
  ***** Full Join
1288
-
1289
1762
  A full join combines the effects of a left join and a right join. All the rows
1290
1763
  from both tables are included in the output augmented by columns of the other
1291
1764
  table where the join expression is satisfied and augmented with nils otherwise.
1292
1765
 
1293
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1294
- #+BEGIN_SRC ruby
1766
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1295
1767
  tab_a.full_join(tab_b, 'id_a == emp_id_b').to_aoa
1296
1768
  #+END_SRC
1297
1769
 
@@ -1310,14 +1782,12 @@ table where the join expression is satisfied and augmented with nils otherwise.
1310
1782
  #+END_EXAMPLE
1311
1783
 
1312
1784
  ***** Cross Join
1313
-
1314
1785
  Finally, a cross join outputs every row of ~tab_a~ augmented with every row of
1315
1786
  ~tab_b~, in other words, the Cartesian product of the two tables. If ~tab_a~ has
1316
1787
  ~N~ rows and ~tab_b~ has ~M~ rows, the output table will have ~N * M~ rows.
1317
1788
  So be careful lest you consume all your computer's memory.
1318
1789
 
1319
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1320
- #+BEGIN_SRC ruby
1790
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1321
1791
  tab_a.cross_join(tab_b).to_aoa
1322
1792
  #+END_SRC
1323
1793
 
@@ -1351,7 +1821,6 @@ So be careful lest you consume all your computer's memory.
1351
1821
  #+END_EXAMPLE
1352
1822
 
1353
1823
  *** Set Operations
1354
-
1355
1824
  ~FatTable~ can perform several set operations on pairs of tables. In order for
1356
1825
  two tables to be used this way, they must have the same number of columns with
1357
1826
  the same types or an exception will be raised. We'll call two tables that
@@ -1361,8 +1830,7 @@ We'll use the following two set-compatible tables in the examples. They each
1361
1830
  have some duplicates and some group boundaries so you can see the effect of the
1362
1831
  set operations on duplicates and groups.
1363
1832
 
1364
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1365
- #+BEGIN_SRC ruby
1833
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1366
1834
  tab1.to_aoa
1367
1835
  #+END_SRC
1368
1836
 
@@ -1393,8 +1861,7 @@ set operations on duplicates and groups.
1393
1861
  | T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1394
1862
  #+END_EXAMPLE
1395
1863
 
1396
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1397
- #+BEGIN_SRC ruby
1864
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1398
1865
  tab2.to_aoa
1399
1866
  #+END_SRC
1400
1867
 
@@ -1422,7 +1889,6 @@ set operations on duplicates and groups.
1422
1889
  #+END_EXAMPLE
1423
1890
 
1424
1891
  **** Unions
1425
-
1426
1892
  Two tables that are set-compatible can be combined with the ~union~ or
1427
1893
  ~union_all~ methods so that the rows of both tables appear in the output. In the
1428
1894
  output table, the headers of the receiver table are used. You can use ~select~
@@ -1433,8 +1899,7 @@ Any group boundaries in the input tables are destroyed by ~union~ but are
1433
1899
  preserved by ~union_all~. In addition, ~union_all~ (but not ~union~) adds a
1434
1900
  group boundary between the rows of the two input tables.
1435
1901
 
1436
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1437
- #+BEGIN_SRC ruby
1902
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1438
1903
  tab1.union(tab2).to_aoa
1439
1904
  #+END_SRC
1440
1905
 
@@ -1464,8 +1929,7 @@ group boundary between the rows of the two input tables.
1464
1929
  | T021 | 2017-01-23 | P | 7.16 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1465
1930
  #+END_EXAMPLE
1466
1931
 
1467
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1468
- #+BEGIN_SRC ruby
1932
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1469
1933
  tab1.union_all(tab2).to_aoa
1470
1934
  #+END_SRC
1471
1935
 
@@ -1516,12 +1980,10 @@ group boundary between the rows of the two input tables.
1516
1980
  #+END_EXAMPLE
1517
1981
 
1518
1982
  **** Intersections
1519
-
1520
1983
  The ~intersect~ method returns a table having only rows common to both tables,
1521
1984
  eliminating any duplicate rows in the result.
1522
1985
 
1523
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1524
- #+BEGIN_SRC ruby
1986
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1525
1987
  tab1.intersect(tab2).to_aoa
1526
1988
  #+END_SRC
1527
1989
 
@@ -1540,8 +2002,7 @@ With ~intersect_all~, all the rows of the first table, including duplicates, are
1540
2002
  included in the result if they also occur in the second table. However,
1541
2003
  duplicates in the second table do not appear.
1542
2004
 
1543
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1544
- #+BEGIN_SRC ruby
2005
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1545
2006
  tab1.intersect_all(tab2).to_aoa
1546
2007
  #+END_SRC
1547
2008
 
@@ -1562,8 +2023,7 @@ As a result, it makes a difference which table is the receiver of the
1562
2023
  ~intersect_all~ method call and which is the argument. In other words, order of
1563
2024
  operation matters.
1564
2025
 
1565
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1566
- #+BEGIN_SRC ruby
2026
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1567
2027
  tab2.intersect_all(tab1).to_aoa
1568
2028
  #+END_SRC
1569
2029
 
@@ -1580,13 +2040,11 @@ operation matters.
1580
2040
  | T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1581
2041
  #+END_EXAMPLE
1582
2042
 
1583
- **** Differences with Except
1584
-
2043
+ **** Set Differences with Except
1585
2044
  You can use the ~except~ method to delete from a table any rows that occur in
1586
2045
  another table, that is, compute the set difference between the tables.
1587
2046
 
1588
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1589
- #+BEGIN_SRC ruby
2047
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1590
2048
  tab1.except(tab2).to_aoa
1591
2049
  #+END_SRC
1592
2050
 
@@ -1608,8 +2066,7 @@ another table, that is, compute the set difference between the tables.
1608
2066
  Like subtraction, though, the order of operands matters with set difference
1609
2067
  computed by ~except~.
1610
2068
 
1611
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1612
- #+BEGIN_SRC ruby
2069
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1613
2070
  tab2.except(tab1).to_aoa
1614
2071
  #+END_SRC
1615
2072
 
@@ -1626,8 +2083,7 @@ computed by ~except~.
1626
2083
  As with ~intersect_all~, ~except_all~ includes any duplicates in the first,
1627
2084
  receiver table, but not those in the second, argument table.
1628
2085
 
1629
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1630
- #+BEGIN_SRC ruby
2086
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1631
2087
  tab1.except_all(tab2).to_aoa
1632
2088
  #+END_SRC
1633
2089
 
@@ -1649,8 +2105,7 @@ receiver table, but not those in the second, argument table.
1649
2105
 
1650
2106
  And, of course, the order of operands matters here as well.
1651
2107
 
1652
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1653
- #+BEGIN_SRC ruby
2108
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1654
2109
  tab2.except_all(tab1).to_aoa
1655
2110
  #+END_SRC
1656
2111
 
@@ -1667,13 +2122,11 @@ And, of course, the order of operands matters here as well.
1667
2122
  #+END_EXAMPLE
1668
2123
 
1669
2124
  *** Uniq (aka Distinct)
1670
-
1671
2125
  The ~uniq~ method takes no arguments and simply removes any duplicate rows from
1672
2126
  the input table. The ~distinct~ method is an alias for ~uniq~. Any groups in
1673
2127
  the input table are lost.
1674
2128
 
1675
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1676
- #+BEGIN_SRC ruby
2129
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1677
2130
  tab1.uniq.to_aoa
1678
2131
  #+END_SRC
1679
2132
 
@@ -1699,13 +2152,11 @@ the input table are lost.
1699
2152
  #+END_EXAMPLE
1700
2153
 
1701
2154
  *** Remove groups with degroup!
1702
-
1703
2155
  Finally, it is sometimes helpful to remove any group boundaries from a table.
1704
- You can do this with ~.degroup!~, which is the only operation that mutates its
1705
- receiver table by removing its groups.
2156
+ You can do this with ~.degroup!~, which, together with ~force_string!~, are
2157
+ the only operations that mutate their receiver tables.
1706
2158
 
1707
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1708
- #+BEGIN_SRC ruby
2159
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1709
2160
  tab1.degroup!.to_aoa
1710
2161
  #+END_SRC
1711
2162
 
@@ -1734,7 +2185,6 @@ receiver table by removing its groups.
1734
2185
  #+END_EXAMPLE
1735
2186
 
1736
2187
  ** Formatting Tables
1737
-
1738
2188
  Besides creating and operating on tables, you may want to display the resulting
1739
2189
  table. ~FatTable~ seeks to provide a set of formatting directives that are the
1740
2190
  most common across many output media. It provides directives for alignment, for
@@ -1751,8 +2201,8 @@ an ~org-mode~ table as a String, and since ~org-mode~ does not support colors,
1751
2201
  any color directives are ignored. Some of the output targets are not strings,
1752
2202
  but ruby data structures, and for them, things such as alignment are irrelevant.
1753
2203
 
1754
- *** Available Formatters
1755
-
2204
+ *** Available Formatter Output Targets
2205
+ **** Output Media
1756
2206
  ~FatTable~ supports the following output targets for its tables:
1757
2207
 
1758
2208
  - Text :: form the table with ACSII characters,
@@ -1771,62 +2221,172 @@ class by defining about a dozen methods that get called at various places
1771
2221
  during the construction of the output table. The idea is that more output
1772
2222
  formats can be defined by adding additional classes.
1773
2223
 
1774
- *** Table Locations
2224
+ **** Examples
2225
+ ***** To Text
2226
+ This formatter uses nothing by ASCII characters to draw the table. Notice
2227
+ that, unlike to ~to_org~ formatter shown below, the intersections of lines are
2228
+ represented by a ~+~ character. Embelishments such as color, bold, and so
2229
+ forth are ignored.
1775
2230
 
1776
- In the formatting methods, the table is divided into several "locations" for
1777
- which separate formatting directives may be given. These locations are
1778
- identified with the following symbols:
2231
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2232
+ tab_a.to_text
2233
+ #+end_SRC
1779
2234
 
1780
- - :header :: the first row of the output table containing the headers,
1781
- - :footer :: all rows of the table's footers,
1782
- - :gfooter :: all rows of the table's group footers,
1783
- - :body :: all the data rows of the table, that is, those that are neither part
1784
- of the header, footers, or gfooters,
1785
- - :bfirst :: the first row of the table's body, and
1786
- - :gfirst :: the first row in each group in the table's body.
2235
+ #+BEGIN_EXAMPLE
2236
+ +====+=======+=====+============+========+============+
2237
+ | Id | Name | Age | Address | Salary | Join Date |
2238
+ +----+-------+-----+------------+--------+------------+
2239
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2240
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2241
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2242
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2243
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2244
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2245
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2246
+ | 10 | James | 45 | Texas | 5000 | |
2247
+ +====+=======+=====+============+========+============+
2248
+ #+END_EXAMPLE
1787
2249
 
1788
- *** Formatting Directives
2250
+ ***** To Org
2251
+ This formatter is designed to format tables in a manner consistent with the
2252
+ way tables are drawn within Emacs Org Mode. It also uses nothing by ASCII
2253
+ characters to draw the table, but, the intersections of lines are represented
2254
+ by a ~|~ character. Embelishments such as color, bold, and so forth are
2255
+ ignored. When working in Org Mode, note that Emacs will convert an Array of
2256
+ Arrays into an Org Mode table, so when constructing tables programmatically,
2257
+ it may be better to use that formatter as shown below.
1789
2258
 
1790
- The formatting methods explained in the next section all take formatting
1791
- directives as strings in which letters and other characters signify what
1792
- formatting applies. For example, we may apply the formatting directive ~'R,$'~
1793
- to numbers in a certain part of the table. Each of those characters, and in
1794
- some cases a whole substring, is a single directive. They can appear in any
1795
- order, so ~'$R,'~ and ~',$R'~ are equivalent.
2259
+ #+begin_SRC ruby :wrap EXAMPLE
2260
+ tab_a.to_org
2261
+ #+end_SRC
1796
2262
 
1797
- Here is a list of all the formatting directives that apply to each cell type:
2263
+ #+begin_EXAMPLE
2264
+ |----+-------+-----+------------+--------+--------------|
2265
+ | Id | Name | Age | Address | Salary | Join Date |
2266
+ |----+-------+-----+------------+--------+--------------|
2267
+ | 1 | Paul | 32 | California | 20000 | [2001-07-13] |
2268
+ | 3 | Teddy | 23 | Norway | 20000 | [2007-12-13] |
2269
+ | 4 | Mark | 25 | Rich-Mond | 65000 | [2007-12-13] |
2270
+ | 5 | David | 27 | Texas | 85000 | [2007-12-13] |
2271
+ | 2 | Allen | 25 | Texas | | [2005-07-13] |
2272
+ | 8 | Paul | 24 | Houston | 20000 | [2005-07-13] |
2273
+ | 9 | James | 44 | Norway | 5000 | [2005-07-13] |
2274
+ | 10 | James | 45 | Texas | 5000 | |
2275
+ |----+-------+-----+------------+--------+--------------|
2276
+ #+end_EXAMPLE
1798
2277
 
1799
- **** String
2278
+ ***** To Term
2279
+ When outputting to a terminal or other device that can interpret ANSI
2280
+ characters and escape codes, you can use this formatter to get a prettier
2281
+ table. It also allows embelishments such as color and text styles to the
2282
+ extent the device supports it.
1800
2283
 
1801
- For a string element, the following instructions are valid. Note that these can
1802
- also be applied to all the other cell types as well since they are all converted
1803
- to a string in forming the output.
2284
+ #+begin_SRC ruby :wrap EXAMPLE
2285
+ tab_a.to_term
2286
+ #+end_SRC
1804
2287
 
1805
- - u :: convert the element to all lowercase,
1806
- - U :: convert the element to all uppercase,
1807
- - t :: title case the element, that is, upcase the initial letter in
1808
- each word and lower case the other letters
1809
- - B ~B :: make the element bold, or turn off bold
2288
+ #+begin_EXAMPLE
2289
+ ╒════╤═══════╤═════╤════════════╤════════╤════════════╕
2290
+ Id Name │ Age Address │ Salary Join Date │
2291
+ ├────┼───────┼─────┼────────────┼────────┼────────────┤
2292
+ 1 │ Paul │ 32 │ California 20000 │ 2001-07-13
2293
+ │ 3 │ Teddy │ 23 │ Norway │ 20000 │ 2007-12-13 │
2294
+ │ 4 │ Mark │ 25 │ Rich-Mond │ 65000 │ 2007-12-13 │
2295
+ │ 5 │ David │ 27 │ Texas │ 85000 │ 2007-12-13 │
2296
+ │ 2 │ Allen │ 25 │ Texas │ │ 2005-07-13 │
2297
+ │ 8 │ Paul │ 24 │ Houston │ 20000 │ 2005-07-13 │
2298
+ │ 9 │ James │ 44 │ Norway │ 5000 │ 2005-07-13 │
2299
+ │ 10 │ James │ 45 │ Texas │ 5000 │ │
2300
+ ╘════╧═══════╧═════╧════════════╧════════╧════════════╛
2301
+ #+end_EXAMPLE
2302
+
2303
+ ***** To LaTeX
2304
+ This formatter outputs a table in the form suitable for inclusion in a LaTeX
2305
+ document using the ~logtable~ package. Natualy it allows embelishments such
2306
+ as color and text styles to the full extent of LaTeX's formatting prowess.
2307
+
2308
+ #+begin_SRC ruby :wrap EXAMPLE
2309
+ tab_b.to_latex
2310
+ #+end_SRC
2311
+
2312
+ #+begin_EXAMPLE
2313
+ \begin{longtable}{lll}
2314
+ Id&
2315
+ Dept&
2316
+ Emp Id\\
2317
+ \endhead
2318
+ 1&
2319
+ IT Billing&
2320
+ 1\\
2321
+ 2&
2322
+ Engineering&
2323
+ 2\\
2324
+ 3&
2325
+ Finance&
2326
+ 7\\
2327
+ \end{longtable}
2328
+ #+end_EXAMPLE
2329
+
2330
+ ***** To AoA (Array of Arrays)
2331
+ #+begin_SRC ruby :wrap EXAMPLE
2332
+ tab_b.to_aoa
2333
+ #+end_SRC
2334
+
2335
+ #+begin_EXAMPLE
2336
+ [["Id", "Dept", "Emp Id"], nil, ["1", "IT Billing", "1"], ["2", "Engineering", "2"],
2337
+ ["3", "Finance", "7"]]
2338
+ #+end_EXAMPLE
2339
+
2340
+ ***** To AoH (Array of Hashes)
2341
+ #+begin_SRC ruby :wrap EXAMPLE
2342
+ tab_b.to_aoh
2343
+ #+end_SRC
2344
+
2345
+ #+begin_EXAMPLE
2346
+ [{:id=>"1", :dept=>"IT Billing", :emp_id=>"1"}, {:id=>"2", :dept=>"Engineering", :emp_id=>"2"},
2347
+ {:id=>"3", :dept=>"Finance", :emp_id=>"7"}]
2348
+ #+end_EXAMPLE
2349
+
2350
+
2351
+ *** Formatting Directives
2352
+ The formatting methods explained in the next section all take formatting
2353
+ directives as strings in which letters and other characters signify what
2354
+ formatting applies. For example, we may apply the formatting directive 'R,$'
2355
+ to numbers in a certain part of the table. Each of those characters, and in
2356
+ some cases a whole substring, is a single directive. They can appear in any
2357
+ order, so '$R,' and ',$R' are equivalent.
2358
+
2359
+ Here is a list of all the formatting directives that apply to each cell type:
2360
+
2361
+ **** String
2362
+ For a string element, the following instructions are valid. Note that these can
2363
+ also be applied to all the other cell types as well since they are all converted
2364
+ to a string in forming the output.
2365
+
2366
+ - u :: convert the element to all lowercase,
2367
+ - U :: convert the element to all uppercase,
2368
+ - t :: title case the element, that is, upcase the initial letter in
2369
+ each word and lower case the other letters
2370
+ - B ~B :: make the element bold, or turn off bold
1810
2371
  - I ~I :: make the element italic, or turn off italic
1811
2372
  - R :: align the element on the right of the column
1812
2373
  - L :: align the element on the left of the column
1813
2374
  - C :: align the element in the center of the column
1814
- - c[color] :: render the element in the given color; the color can have
1815
- the form fgcolor, fgcolor.bgcolor, or .bgcolor, to set the
1816
- foreground or background colors respectively, and each of those can
1817
- be an ANSI or X11 color name in addition to the special color,
1818
- 'none', which keeps the terminal's default color.
2375
+ - c[<color_spec>] :: render the element in the given color; the <color_spec>
2376
+ can have the form fgcolor, fgcolor.bgcolor, or .bgcolor, to set the
2377
+ foreground or background colors respectively, and each of those can be an
2378
+ ANSI or X11 color name in addition to the special color, 'none', which keeps
2379
+ the output's default color.
1819
2380
  - _ ~_ :: underline the element, or turn off underline
1820
2381
  - * ~* :: cause the element to blink, or turn off blink
1821
2382
 
1822
- For example, the directive ~'tCc[red.yellow]'~ would title-case the element,
2383
+ For example, the directive 'tCc[red.yellow]' would title-case the element,
1823
2384
  center it, and color it red on a yellow background. The directives that are
1824
2385
  boolean have negating forms so that, for example, if bold is turned on for all
1825
2386
  columns of a given type, it can be countermanded in formatting directives for
1826
2387
  particular columns.
1827
2388
 
1828
2389
  **** Numeric
1829
-
1830
2390
  For a numeric element, all the instructions valid for string are available, in
1831
2391
  addition to the following:
1832
2392
 
@@ -1841,11 +2401,10 @@ addition to the following:
1841
2401
  will result in a :numeric expressed as seconds and can be displayed in
1842
2402
  hours, minutes, and seconds with this formatting instruction.
1843
2403
 
1844
- For example, the directive ~'R5.0c[blue]'~ would right-align the numeric
2404
+ For example, the directive 'R5.0c[blue]' would right-align the numeric
1845
2405
  element, pad it on the left with zeros, and color it blue.
1846
2406
 
1847
2407
  **** DateTime
1848
-
1849
2408
  For a ~DateTime~, all the instructions valid for string are available, in
1850
2409
  addition to the following:
1851
2410
 
@@ -1857,61 +2416,216 @@ addition to the following:
1857
2416
  component where fmt is a valid format string for Date#strftime, otherwise,
1858
2417
  the datetime will be formatted as an ISO 8601 string, YYYY-MM-DD.
1859
2418
 
1860
- For example, ~'c[pink]d[%b %-d, %Y]C'~, would format a date element like 'Sep
2419
+ For example, 'c[pink]d[%b %-d, %Y]C', would format a date element like 'Sep
1861
2420
  22, 1957', center it, and color it pink.
1862
2421
 
1863
2422
  **** Boolean
1864
-
1865
2423
  For a boolean cell, all the instructions valid for string are available, in
1866
2424
  addition to the following:
1867
2425
 
1868
- - Y :: print true as '~Y~' and false as '~N~',
1869
- - T :: print true as '~T~' and false as '~F~',
1870
- - X :: print true as '~X~' and false as an empty string '',
2426
+ - Y :: print true as ~Y~ and false as ~N~,
2427
+ - T :: print true as ~T~ and false as ~F~,
2428
+ - X :: print true as ~X~ and false as an empty string '',
1871
2429
  - b[xxx,yyy] :: print true as the string given as ~xxx~ and false as the string
1872
2430
  given as ~yyy~,
1873
2431
  - c[tcolor,fcolor] :: color a true element with ~tcolor~ and a false element
1874
2432
  with ~fcolor~. Each of the colors may be specified in the same manner as
1875
2433
  colors for strings described above.
1876
2434
 
1877
- For example, the directive '~b[Yeppers,Nope]c[green.pink,red.pink]~' would
1878
- render a true boolean as '~Yeppers~' colored green on pink and render a false
1879
- boolean as '~Nope~' colored red on pink. See [[https://www.youtube.com/watch?v=oLdFFD8II8U][Yeppers]] for additional information.
2435
+ For example, the directive 'b[Yeppers,Nope]c[green.pink,red.pink]' would
2436
+ render a true boolean as ~Yeppers~ colored green on pink and render a false
2437
+ boolean as ~Nope~ colored red on pink. See [[https://www.youtube.com/watch?v=oLdFFD8II8U][Yeppers]] for additional information.
1880
2438
 
1881
2439
  **** NilClass
1882
-
1883
2440
  By default, ~nil~ elements are rendered as blank cells, but you can make them
1884
2441
  visible with the following, and in that case, all the formatting instructions
1885
2442
  valid for strings are also available:
1886
2443
 
1887
2444
  - n[niltext] :: render a ~nil~ item with the given niltext.
1888
2445
 
1889
- For example, you might want to use ~'n[-]Cc[purple]'~ to make nils visible as a
2446
+ For example, you might want to use 'n[-]Cc[purple]' to make nils visible as a
1890
2447
  centered purple hyphen.
1891
2448
 
1892
- *** Footers Methods
2449
+ *** The ~format~ and ~format_for~ methods
2450
+ Formatters take only two kinds of methods, those that attach footers to a
2451
+ table, which are discussed in the next section, and those that specify
2452
+ formatting for table cells, which are the subject of this section.
2453
+
2454
+ To set formatting directives for all locations in a table at once, use the
2455
+ ~format~ method; to set formatting directives for a particular location in the
2456
+ table, use the ~format_for~ method, giving the location as the first
2457
+ parameter. See below at [[*Table Locations][Table Locations]] for an explanation of all the
2458
+ locations available.
2459
+
2460
+ Other than that first parameter, the two methods take the same types of
2461
+ parameters. The remaining parameters are hash-like parameters that use either a
2462
+ column name or a type as the key and a string with the formatting directives to
2463
+ apply as the value. The following example says to set the formatting for all
2464
+ locations in the table and to format all numeric fields as strings that are
2465
+ rounded to whole numbers (the '0.0' part), that are right-aligned (the 'R'
2466
+ part), and have grouping commas inserted (the ',' part). But the ~:id~ column is
2467
+ numeric, and the second parameter overrides the formatting for numerics in
2468
+ general and calls for the ~:id~ column to be padded to three digits with zeros
2469
+ on the left (the '3.0' part) and to be centered (the 'C' part).
2470
+
2471
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2472
+ tab_a.to_text do |f|
2473
+ f.format(numeric: '0.0,R', id: '3.0C')
2474
+ f.format_for(:body, string: 'R')
2475
+ f.format_for(:header, string: 'C')
2476
+ end
2477
+ #+END_SRC
2478
+
2479
+ #+begin_EXAMPLE
2480
+ +=====+=======+=====+============+========+============+
2481
+ | Id | Name | Age | Address | Salary | Join Date |
2482
+ +-----+-------+-----+------------+--------+------------+
2483
+ | 001 | Paul | 32 | California | 20,000 | 2001-07-13 |
2484
+ | 003 | Teddy | 23 | Norway | 20,000 | 2007-12-13 |
2485
+ | 004 | Mark | 25 | Rich-Mond | 65,000 | 2007-12-13 |
2486
+ | 005 | David | 27 | Texas | 85,000 | 2007-12-13 |
2487
+ | 002 | Allen | 25 | Texas | | 2005-07-13 |
2488
+ | 008 | Paul | 24 | Houston | 20,000 | 2005-07-13 |
2489
+ | 009 | James | 44 | Norway | 5,000 | 2005-07-13 |
2490
+ | 010 | James | 45 | Texas | 5,000 | |
2491
+ +=====+=======+=====+============+========+============+
2492
+ #+end_EXAMPLE
1893
2493
 
1894
- You can call the ~footer~ and ~gfooter~ methods on ~Formatter~ objects to add
1895
- footers and group footers. Their signatures are:
2494
+ In the example, the ~format~ method affects the whole table. Its ~numeric:~
2495
+ directive affected the ~:age~ and ~:salary~ columns because their types are
2496
+ Numeric. The ~id:~ column is also Numeric, but it's more specific directive
2497
+ takes precedence and it is formatted accordingly.
2498
+
2499
+ But the ~format_for~ methods affected two "locations": the "body" and the
2500
+ "header". Within the body, the ~:string~ directive calls for all strings to
2501
+ be right-aligned, but the headers are unaffected by it. The ~format_for~ the
2502
+ ~:header~ location caused all the headers to be centered.
2503
+
2504
+ All the other cells in the table, namely the cells in the ~:join_date~ column,
2505
+ had the default formatting applied.
2506
+
2507
+ **** Table Locations
2508
+ In the ~format_for~ formatting method, the first argument names a "location."
2509
+ The table is divided into several locations for which separate formatting
2510
+ directives may be given. These locations are identified by the following
2511
+ symbols:
2512
+
2513
+ - :header :: the first row of the output table containing the headers,
2514
+ - :footer :: all rows of the table's footers,
2515
+ - :gfooter :: all rows of the table's group footers,
2516
+ - :body :: all the data rows of the table, that is, those that are neither part
2517
+ of the header, footers, or gfooters,
2518
+ - :bfirst :: the first row of the table's body, and
2519
+ - :gfirst :: the first row in each group in the table's body.
2520
+
2521
+ **** Location priority
2522
+ Formatting for any given cell depends on its location in the table. The
2523
+ ~format_for~ method takes a location to which its formatting directive are
2524
+ restricted as the first argument. It can be one of the following:
2525
+
2526
+ - ~:header~ :: The directives apply only to the header row, that is the first
2527
+ row, of the output table; before the directives are applied, the header's
2528
+ symbol form is converted back into a string and capitalized as is a book
2529
+ title. Thus, only directives applicable to the String type have any effect.
2530
+
2531
+ - ~:body~ :: The directives apply to all rows in the body of the table.
2532
+
2533
+ - ~:gfirst~ :: directives apply to the first row in each group in the body of
2534
+ the table, unless the row is also the first row in the table as a whole, in
2535
+ which case the ~:bfirst~ directives apply,
2536
+
2537
+ - ~:bfirst~ :: The directives apply to the first row in the body of the table,
2538
+ taking precedence over those directives that apply to the body generally or
2539
+ the ~:gfirst~ directives that apply to the first row in each group.
2540
+
2541
+ - ~:footer~ :: The directives apply to all the footer rows of the output
2542
+ table, regardless of how many there are.
2543
+
2544
+ - ~gfooter~ :: The directives apply to all group footer rows of the output
2545
+ tables, regardless of how many there are.
2546
+
2547
+ Directives given to the ~format~ method apply the directives to all locations in
2548
+ the table, but they can be overridden by more specific directives given in a
2549
+ ~format_for~ directive.
2550
+
2551
+ **** Type and Column priority
2552
+ A directive based on type applies to all columns having that type unless
2553
+ overridden by a directive specific to a named column; a directive based on a
2554
+ column name applies only to cells in that column.
2555
+
2556
+ However, there is a twist. Since the end result of formatting is to convert
2557
+ all columns to strings, the formatting directives for the ~String~ type can
2558
+ be applied to all column types. Likewise, since all columns may contain nils,
2559
+ the ~NilClass:~ type applies to nils in all columns regardless of the column's
2560
+ type.
2561
+
2562
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2563
+ tab_a.to_text do |f|
2564
+ f.format(string: 'R', id: '3.0C', nil: 'Cn[-]', salary: 'n[N/A]')
2565
+ end
2566
+ #+END_SRC
2567
+
2568
+ #+BEGIN_EXAMPLE
2569
+ +=====+=======+=====+============+========+============+
2570
+ | Id | Name | Age | Address | Salary | Join Date |
2571
+ +-----+-------+-----+------------+--------+------------+
2572
+ | 001 | Paul | 32 | California | 20000 | 2001-07-13 |
2573
+ | 003 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2574
+ | 004 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2575
+ | 005 | David | 27 | Texas | 85000 | 2007-12-13 |
2576
+ | 002 | Allen | 25 | Texas | N/A | 2005-07-13 |
2577
+ | 008 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2578
+ | 009 | James | 44 | Norway | 5000 | 2005-07-13 |
2579
+ | 010 | James | 45 | Texas | 5000 | |
2580
+ +=====+=======+=====+============+========+============+
2581
+ #+END_EXAMPLE
2582
+
2583
+ The ~string: 'R'~ directive causes all the cells to be right-aligned except
2584
+ ~:id~ which specifies centering for the ~:id~ column only. The ~n[N/A]~
2585
+ directive for specifies how nil are displayed in the numeric column, ~:salary~,
2586
+ but not for other nils, such as in the last row of the ~:join_date~ column.
2587
+
2588
+ *** Footers
2589
+ **** Adding Footers
2590
+ You can call the ~footer,~ ~gfooter, foot, and gfoot~ methods on ~Formatter~
2591
+ objects to add footers and group footers. Note that all of these methods
2592
+ return a ~Footer~ object that can be accessed to extract the computed values.
2593
+ All of these methods return the ~FatTable::Footer~ object so constructed. It
2594
+ can be used to access the values and other attributes of the footer
2595
+ computed. Their signatures are:
1896
2596
 
1897
2597
  - ~footer(label, *sum_cols, **agg_cols)~ :: where ~label~ is a label to be
1898
- placed in the first cell of the footer (unless that column is named as one
1899
- of the ~sum_cols~ or ~agg_cols~, in which case the label is ignored),
1900
- ~*sum_cols~ are zero or more symbols for columns to be summed, and
1901
- ~**agg_cols~ is zero or more hash-like parameters with a column symbol as a
1902
- key and a symbol for an aggregate method as the value. This causes a
1903
- table-wide header to be added at the bottom of the table applying the
1904
- ~:sum~ aggregate to the ~sum_cols~ and the named aggregate method to the
1905
- ~agg_cols~. A table can have any number of footers attached, and they will
1906
- appear at the bottom of the output table in the order they are given.
1907
-
1908
- - ~gfooter(label, *sum_cols, **agg_cols)~ :: where the parameters have the same
1909
- meaning as for the ~footer~ method, but result in a footer for each group
1910
- in the table rather than the table as a whole. These will appear in the
1911
- output table just below each group.
2598
+ placed in the first cell of the footer (unless that column is named as one
2599
+ of the ~sum_cols~ or ~agg_cols~, in which case the label is ignored),
2600
+ ~*sum_cols~ are zero or more symbols for columns to be summed, and
2601
+ ~**agg_cols~ is zero or more hash-like parameters with a column symbol as a
2602
+ key and a valid aggregate as the value. This causes a table-wide header to
2603
+ be added at the bottom of the table applying the ~:sum~ aggregate to the
2604
+ ~sum_cols~ and the named aggregate to the ~agg_cols~. A table can have any
2605
+ number of footers attached, and they will appear at the bottom of the output
2606
+ table in the order they are given.
2607
+
2608
+ - ~foot(label, label_col, **agg_cols)~ :: where ~label~ is a label to be
2609
+ placed in the column with header ~label_col~, or, if ommitted, in the first
2610
+ cell of the footer (unless that column is named as one of the ~agg_cols~, in
2611
+ which case the label is ignored), and ~**agg_cols~ is zero or more hash-like
2612
+ parameters with a column symbol as a key and a valid aggregate as the
2613
+ value. This causes a table-wide header to be added at the bottom of the
2614
+ table applying ~agg~, to the ~agg_cols~. A table can have any number of
2615
+ footers attached, and they will appear at the bottom of the output table in
2616
+ the order they are given.
2617
+
2618
+ - ~gfooter(label, *sum_cols, **agg_cols)~ :: where the parameters have the
2619
+ same meaning as for the ~footer~ method, but results in a footer for each
2620
+ group in the table rather than the table as a whole. These will appear in
2621
+ the output table just below each group.
2622
+
2623
+ - ~gfoot(label, label_col, **agg_cols)~ :: where the parameters have the same
2624
+ meaning as for the ~foot~ method, but results in a footer for each group in
2625
+ the table rather than the table as a whole. These will appear in the output
2626
+ table just below each group.
1912
2627
 
1913
2628
  There are also a number of convenience methods for adding common footers:
1914
-
1915
2629
  - ~sum_footer(*cols)~ :: Add a footer summing the given columns with the label
1916
2630
  'Total'.
1917
2631
  - ~sum_gfooter(*cols)~ :: Add a group footer summing the given columns with the
@@ -1929,22 +2643,324 @@ There are also a number of convenience methods for adding common footers:
1929
2643
  - ~max_gfooter(*cols)~ :: Add a group footer showing the maximum for the given
1930
2644
  columns with the label 'Group Maximum'.
1931
2645
 
1932
- *** Formatting Methods
2646
+ **** Aggregators
2647
+ When adding a footer with the above methods, you can specify an aggregator for
2648
+ each column named in the ~agg_cols~ parameter. There are several candidates
2649
+ for what you can use for an aggregator:
2650
+
2651
+ - Symbol :: one of the following built-in aggregators: :first, :last, :range,
2652
+ :sum, :count, :min, :max, :avg, :var, :pvar, :dev, :pdev, :any?, :all?,
2653
+ :none?, and :one?.
2654
+ + The symbols ending in a question mark are valid only for boolean columns;
2655
+ + :count, :first, and :last work with any column type,
2656
+ + :min, :max, and :range work with all types except boolean;
2657
+ + :sum, works only with numeric columns, and
2658
+ + :avg, :var, :dev, :pvar, and :pdev work with numeric or datetime columns.
2659
+ In the case of datetime columns, these aggrgators convert the dates to
2660
+ julian date numbers, perform the calculation, then convert the result back
2661
+ to a datetime object.
2662
+ - String :: using a string as an aggrgegator can result in:
2663
+ + the string being converted to an object matching the type of the column
2664
+ (for example, using '$1,888' in a numeric column puts the constant number
2665
+ 1888 in the footer field, using '1957-09-22' puts the fixed date in the
2666
+ field, etc.)
2667
+ + if the string cannot be parsed as a valid object matching the column's
2668
+ type, it is placed literally in the footer field (for example, using
2669
+ '(estimated)' can be used to add additional information to the footer)
2670
+ - Ruby object :: you can put a number in a numeric footer field, a DateTime
2671
+ object in a datetime footer field, or a true or false in a boolean footer
2672
+ field;
2673
+ - A Lambda :: finally, you can provide a lambda for performing arbitrary
2674
+ calculations and placing the result in the footer field. The number of
2675
+ arguments the lambda takes can vary:
2676
+ * If the lambda is used in a group footer, it must take a single integer
2677
+ argument that is set to the group number being calculated and /can/ take a
2678
+ second argument for the column symbol in which it appears, or
2679
+ * If the lambda is used in an ordinary footer, it either takes no arguments,
2680
+ or a single argument for the column symbol in which it appears.
2681
+
2682
+ **** Footer objects
2683
+ Each of the methods for adding a footer to a ~Formatter~ returns a ~Footer~ object
2684
+ that you can query for attributes of the generated footer, including accessing
2685
+ their computed values. Here are the accessors available on a
2686
+ ~FatTable::Footer~ object:
2687
+
2688
+ - ~[h]~ :: Return the value of under the ~h~ header, or if this is a group
2689
+ footer, return an array of the values for all the groups under the ~h~
2690
+ header.
2691
+ - .<header> :: like, ~[h]~ but makes the values available in method-call form.
2692
+ - ~number_of_groups~ :: Return the total number of groups in the table to
2693
+ which this footer belongs. Note that if the table has both group footers
2694
+ and normal footers, this will return the number of groups even for a normal
2695
+ footer.
2696
+ - ~column(h)~, ~column(h, k)~ :: Return a FatTable::Column object for the
2697
+ header h and, if the footer is a group footer, the kth group.
2698
+ - ~items(h)~, ~items(h, k)~ :: Return an Array of the values for the header
2699
+ ~h~ and, if a group, for the ~k~th group.
2700
+ - ~to_h~, ~to_h(k)~ :: Return a Hash with a key for each column header mapped
2701
+ to the footer value for that column, nil for unused columns. Use the index
2702
+ ~k~ to specify which group to access in the case of a group footer.
2703
+
2704
+
2705
+ **** Footer Examples
2706
+ As a reminder, here is the table, ~tab_a~ defined earlier:
2707
+
2708
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2709
+ tab_a.to_aoa
2710
+ #+END_SRC
2711
+
2712
+ #+begin_EXAMPLE
2713
+ | Id | Name | Age | Address | Salary | Join Date |
2714
+ |----+-------+-----+------------+--------+------------|
2715
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2716
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2717
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2718
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2719
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2720
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2721
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2722
+ | 10 | James | 45 | Texas | 5000 | |
2723
+ #+end_EXAMPLE
2724
+
2725
+ ***** Built-in Aggregators
2726
+ You can add a footer compute the average of the given columns. You may be
2727
+ surprised that you can average a set of dates, but ~:avg~ simply converts the
2728
+ dates to Julian numbers, averages that, then converts the result back to a
2729
+ date.
2730
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2731
+ tab_a.to_text do |f|
2732
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2733
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2734
+ f.footer('Tally', age: :count)
2735
+ end
2736
+ #+END_SRC
1933
2737
 
1934
- You can call methods on ~Formatter~ objects to specify formatting directives
1935
- for specific columns or types. There are two methods for doing so, ~format_for~
1936
- and ~format~.
2738
+ #+begin_EXAMPLE
2739
+ +=========+=======+=====+============+========+=============+
2740
+ | Id | Name | Age | Address | Salary | Join Date |
2741
+ +---------+-------+-----+------------+--------+-------------+
2742
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2743
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2744
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2745
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2746
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2747
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2748
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2749
+ | 10 | James | 45 | Texas | 5,000 | |
2750
+ +---------+-------+-----+------------+--------+-------------+
2751
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2752
+ +---------+-------+-----+------------+--------+-------------+
2753
+ | Tally | | 8 | | | |
2754
+ +=========+=======+=====+============+========+=============+
2755
+ #+end_EXAMPLE
2756
+
2757
+ ***** String Aggregators
2758
+ If the string is convertible into its columns's type, it will be converted to
2759
+ that type; otherwise, it will be placed in the footer literally. This example
2760
+ also shows how the values from one footer might be used in composing another
2761
+ footer.
2762
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2763
+ tab_a.to_text do |f|
2764
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2765
+ avg_ft = f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2766
+ f.footer('Tally', age: :count)
2767
+ if avg_ft[:salary] < 30000
2768
+ cmt = "We're saving"
2769
+ else
2770
+ cmt = "We're overspending"
2771
+ end
2772
+ f.footer('Pay', join_date: "We have #{avg_ft.number_of_groups} grp")
2773
+ f.footer('Group count', join_date: "We have #{avg_ft.number_of_groups} grp")
2774
+ f.footer('Comment', join_date: cmt)
2775
+ end
2776
+ #+END_SRC
1937
2777
 
1938
- **** Instantiating a Formatter
2778
+ #+begin_EXAMPLE
2779
+ +=============+=======+=====+============+========+====================+
2780
+ | Id | Name | Age | Address | Salary | Join Date |
2781
+ +-------------+-------+-----+------------+--------+--------------------+
2782
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2783
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2784
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2785
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2786
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2787
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2788
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2789
+ | 10 | James | 45 | Texas | 5,000 | |
2790
+ +-------------+-------+-----+------------+--------+--------------------+
2791
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2792
+ +-------------+-------+-----+------------+--------+--------------------+
2793
+ | Tally | | 8 | | | |
2794
+ +-------------+-------+-----+------------+--------+--------------------+
2795
+ | Pay | | | | | We have 1 grp |
2796
+ +-------------+-------+-----+------------+--------+--------------------+
2797
+ | Group count | | | | | We have 1 grp |
2798
+ +-------------+-------+-----+------------+--------+--------------------+
2799
+ | Comment | | | | | We're overspending |
2800
+ +=============+=======+=====+============+========+====================+
2801
+ #+end_EXAMPLE
1939
2802
 
1940
- There are several ways to invoke the formatting methods on a table. First, you
1941
- can instantiate a ~XXXFormatter~ object and feed it a table as a parameter.
1942
- There is a Formatter subclass for each target output medium, for example,
1943
- ~AoaFormatter~ will produce a ruby array of arrays. You can then call the
1944
- ~output~ method on the ~XXXFormatter~.
2803
+ ***** Ruby Objects
2804
+ You can make the aggregator an normal ruby object, in which case it is just
2805
+ inserted into the footer at the requested location. If its type is the same
2806
+ as the column type, it participates in the formatting for that type and
2807
+ column.
2808
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2809
+ tab_a.to_text do |f|
2810
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2811
+ f.footer('Report Date', age: :count, join_date: Date.today)
2812
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2813
+ end
2814
+ #+END_SRC
1945
2815
 
1946
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1947
- #+BEGIN_SRC ruby
2816
+ #+begin_EXAMPLE
2817
+ +=============+=======+=====+============+========+=============+
2818
+ | Id | Name | Age | Address | Salary | Join Date |
2819
+ +-------------+-------+-----+------------+--------+-------------+
2820
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2821
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2822
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2823
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2824
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2825
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2826
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2827
+ | 10 | James | 45 | Texas | 5,000 | |
2828
+ +-------------+-------+-----+------------+--------+-------------+
2829
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2830
+ +-------------+-------+-----+------------+--------+-------------+
2831
+ | Report Date | | 8 | | | 20-JAN-2022 |
2832
+ +=============+=======+=====+============+========+=============+
2833
+ #+end_EXAMPLE
2834
+
2835
+ But it can be any type. Here we pick a lottery winner from the employee ids.
2836
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2837
+ tab_a.to_text do |f|
2838
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2839
+ winner_id = tab_a.column(:id).items.sample
2840
+ f.footer('Lottery Winner', age: :count, join_date: winner_id)
2841
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2842
+ end
2843
+ #+END_SRC
2844
+
2845
+ #+begin_EXAMPLE
2846
+ +================+=======+=====+============+========+=============+
2847
+ | Id | Name | Age | Address | Salary | Join Date |
2848
+ +----------------+-------+-----+------------+--------+-------------+
2849
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2850
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2851
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2852
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2853
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2854
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2855
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2856
+ | 10 | James | 45 | Texas | 5,000 | |
2857
+ +----------------+-------+-----+------------+--------+-------------+
2858
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2859
+ +----------------+-------+-----+------------+--------+-------------+
2860
+ | Lottery Winner | | 8 | | | 4 |
2861
+ +================+=======+=====+============+========+=============+
2862
+ #+end_EXAMPLE
2863
+
2864
+ ***** Lambdas
2865
+ Perhaps the most flexible form of aggregator is a lambda form. They require 2
2866
+ or 3 parameters in non-group and group footers, respectively:
2867
+
2868
+ - ~->(f, c) {...}~ :: in a normal, non-group footer, you must provide for two
2869
+ paramters: the first, ~f~, will be bound to the footer in which the lambda
2870
+ appears and the second, ~c~, will be bound to the column header to which the
2871
+ lambda is attached.
2872
+ - ~->(f, c, k)~ :: in a group footer, you must provide for three paramters:
2873
+ the first, ~f~, will be bound to the footer in which the lambda appears, the
2874
+ second, ~c~, will be bound to the column header to which the lambda is
2875
+ attached, and the third, ~k~ will be bound to the group number of the group
2876
+ being evaluated.
2877
+
2878
+ With the first argument, the footer itself becomes available and with it all
2879
+ the things accessible with the footers, including the items in the current
2880
+ column, through the ~f.items(c)~ accessor.
2881
+
2882
+ Compute the summ of the squares if the items in the ~:age~ column:
2883
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2884
+ tab_a.to_text do |f|
2885
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2886
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2887
+ f.footer('SSQ', age: ->(f, c) { sa = f.items(c).map {|x| x * x}.sum; Math.sqrt(sa) })
2888
+ end
2889
+ #+END_SRC
2890
+
2891
+ #+begin_EXAMPLE
2892
+ +=========+=======+=====+============+========+=============+
2893
+ | Id | Name | Age | Address | Salary | Join Date |
2894
+ +---------+-------+-----+------------+--------+-------------+
2895
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2896
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2897
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2898
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2899
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2900
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2901
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2902
+ | 10 | James | 45 | Texas | 5,000 | |
2903
+ +---------+-------+-----+------------+--------+-------------+
2904
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2905
+ +---------+-------+-----+------------+--------+-------------+
2906
+ | SSQ | | 90 | | | |
2907
+ +=========+=======+=====+============+========+=============+
2908
+ #+end_EXAMPLE
2909
+
2910
+ Group the table according to the employee's year of joining, then compute the
2911
+ summ of the squares if the ages in each group:
2912
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2913
+ tab_a.order_with('join_date.year').to_text do |f|
2914
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]', sort_key: '0.0~,')
2915
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2916
+ f.gfooter('Group SSQ', age: ->(f, c, k) { sa = f.items(c, k).map {|x| x * x}.sum; Math.sqrt(sa) })
2917
+ f.footer('Total SSQ', age: ->(f, c) { sa = f.items(c).map {|x| x * x}.sum; Math.sqrt(sa) })
2918
+ end
2919
+ #+END_SRC
2920
+
2921
+ #+begin_EXAMPLE
2922
+ +===========+=======+=====+============+========+=============+==========+
2923
+ | Id | Name | Age | Address | Salary | Join Date | Sort Key |
2924
+ +-----------+-------+-----+------------+--------+-------------+----------+
2925
+ | 10 | James | 45 | Texas | 5,000 | | |
2926
+ +-----------+-------+-----+------------+--------+-------------+----------+
2927
+ | Group SSQ | | 45 | | | | |
2928
+ +-----------+-------+-----+------------+--------+-------------+----------+
2929
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 | 2001 |
2930
+ +-----------+-------+-----+------------+--------+-------------+----------+
2931
+ | Group SSQ | | 32 | | | | |
2932
+ +-----------+-------+-----+------------+--------+-------------+----------+
2933
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 | 2005 |
2934
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 | 2005 |
2935
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 | 2005 |
2936
+ +-----------+-------+-----+------------+--------+-------------+----------+
2937
+ | Group SSQ | | 56 | | | | |
2938
+ +-----------+-------+-----+------------+--------+-------------+----------+
2939
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 | 2007 |
2940
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 | 2007 |
2941
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 | 2007 |
2942
+ +-----------+-------+-----+------------+--------+-------------+----------+
2943
+ | Group SSQ | | 43 | | | | |
2944
+ +-----------+-------+-----+------------+--------+-------------+----------+
2945
+ | Average | | 31 | | 31,429 | 29-DEC-2005 | |
2946
+ +-----------+-------+-----+------------+--------+-------------+----------+
2947
+ | Total SSQ | | 90 | | | | |
2948
+ +===========+=======+=====+============+========+=============+==========+
2949
+ #+end_EXAMPLE
2950
+
2951
+ *** Invoking Formatters
2952
+ As the examples show, one way to invoke the formatting methods is simply to
2953
+ call one of the ~to_xxx~ methods directly on a table, which will yield a
2954
+ ~FatTable::Formatter~ object to the block, and that is often the most
2955
+ convenient way to do it. But there are a few other ways.
2956
+
2957
+ **** By Instantiating a Formatter
2958
+ You can instantiate a ~XXXFormatter~ object and feed it a table as a
2959
+ parameter. There is a Formatter subclass for each target output medium, for
2960
+ example, ~AoaFormatter~ will produce a ruby array of arrays. You can then call
2961
+ the ~output~ method on the ~XXXFormatter~.
2962
+
2963
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1948
2964
  FatTable::AoaFormatter.new(tab_a).output
1949
2965
  #+END_SRC
1950
2966
 
@@ -1964,8 +2980,7 @@ There is a Formatter subclass for each target output medium, for example,
1964
2980
  The ~XXXFormatter.new~ method yields the new instance to any block given, and
1965
2981
  you can call methods on it to affect the formatting of the output:
1966
2982
 
1967
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1968
- #+BEGIN_SRC ruby
2983
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1969
2984
  FatTable::AoaFormatter.new(tab_a) do |f|
1970
2985
  f.format(numeric: '0.0,R', id: '3.0C')
1971
2986
  end.output
@@ -1984,15 +2999,13 @@ you can call methods on it to affect the formatting of the output:
1984
2999
  | 010 | James | 45 | Texas | 5,000 | |
1985
3000
  #+END_EXAMPLE
1986
3001
 
1987
- **** ~FatTable~ module-level method calls
1988
-
3002
+ **** By Using ~FatTable~ module-level method calls
1989
3003
  The ~FatTable~ module provides a set of methods of the form ~to_aoa~, ~to_text~,
1990
3004
  etc., to access a ~Formatter~ without having to create an instance yourself.
1991
3005
  Without a block, they apply the default formatting to the table and call the
1992
3006
  ~.output~ method automatically:
1993
3007
 
1994
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1995
- #+BEGIN_SRC ruby
3008
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1996
3009
  FatTable.to_aoa(tab_a)
1997
3010
  #+END_SRC
1998
3011
 
@@ -2013,8 +3026,7 @@ With a block, these methods yield a ~Formatter~ instance on which you can call
2013
3026
  formatting and footer methods. The ~.output~ method is called on the ~Formatter~
2014
3027
  automatically after the block:
2015
3028
 
2016
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2017
- #+BEGIN_SRC ruby
3029
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2018
3030
  FatTable.to_aoa(tab_a) do |f|
2019
3031
  f.format(numeric: '0.0,R', id: '3.0C')
2020
3032
  end
@@ -2033,13 +3045,11 @@ automatically after the block:
2033
3045
  | 010 | James | 45 | Texas | 5,000 | |
2034
3046
  #+END_EXAMPLE
2035
3047
 
2036
- **** Calling methods on Table objects
2037
-
2038
- Finally, you can call methods such as ~to_aoa~, ~to_text~, etc., directly on a
2039
- Table:
3048
+ **** By Calling Methods on Table Objects
3049
+ Finally, as in many of the examples, you can call methods such as ~to_aoa~,
3050
+ ~to_text~, etc., directly on a Table:
2040
3051
 
2041
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2042
- #+BEGIN_SRC ruby
3052
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2043
3053
  tab_a.to_aoa
2044
3054
  #+END_SRC
2045
3055
 
@@ -2058,8 +3068,7 @@ Table:
2058
3068
 
2059
3069
  And you can supply a block to them as well to specify formatting or footers:
2060
3070
 
2061
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2062
- #+BEGIN_SRC ruby
3071
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2063
3072
  tab_a.to_aoa do |f|
2064
3073
  f.format(numeric: '0.0,R', id: '3.0C')
2065
3074
  f.sum_footer(:salary, :age)
@@ -2081,135 +3090,13 @@ And you can supply a block to them as well to specify formatting or footers:
2081
3090
  | Total | | 245 | | 220,000 | |
2082
3091
  #+END_EXAMPLE
2083
3092
 
2084
- *** The ~format~ and ~format_for~ methods
2085
-
2086
- Formatters take only two kinds of methods, those that attach footers to a
2087
- table, which are discussed in the next section, and those that specify
2088
- formatting for table cells, which are the subject of this section.
2089
-
2090
- To set formatting directives for all locations in a table at once, use the
2091
- ~format~ method; to set formatting directives for a particular location in the
2092
- table, use the ~format_for~ method, giving the location as the first parameter.
2093
-
2094
- Other than that first parameter, the two methods take the same types of
2095
- parameters. The remaining parameters are hash-like parameters that use either a
2096
- column name or a type as the key and a string with the formatting directives to
2097
- apply as the value. The following example says to set the formatting for all
2098
- locations in the table and to format all numeric fields as strings that are
2099
- rounded to whole numbers (the '0.0' part), that are right-aligned (the 'R'
2100
- part), and have grouping commas inserted (the ',' part). But the ~:id~ column is
2101
- numeric, and the second parameter overrides the formatting for numerics in
2102
- general and calls for the ~:id~ column to be padded to three digits with zeros
2103
- on the left (the '3.0' part) and to be centered (the 'C' part).
2104
-
2105
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2106
- #+BEGIN_SRC ruby
2107
- tab_a.to_aoa do |f|
2108
- f.format(numeric: '0.0,R', id: '3.0C')
2109
- end
2110
- #+END_SRC
2111
-
2112
- #+BEGIN_EXAMPLE
2113
- | Id | Name | Age | Address | Salary | Join Date |
2114
- |-----+-------+-----+------------+--------+------------|
2115
- | 001 | Paul | 32 | California | 20,000 | 2001-07-13 |
2116
- | 003 | Teddy | 23 | Norway | 20,000 | 2007-12-13 |
2117
- | 004 | Mark | 25 | Rich-Mond | 65,000 | 2007-12-13 |
2118
- | 005 | David | 27 | Texas | 85,000 | 2007-12-13 |
2119
- | 002 | Allen | 25 | Texas | | 2005-07-13 |
2120
- | 008 | Paul | 24 | Houston | 20,000 | 2005-07-13 |
2121
- | 009 | James | 44 | Norway | 5,000 | 2005-07-13 |
2122
- | 010 | James | 45 | Texas | 5,000 | |
2123
- #+END_EXAMPLE
2124
-
2125
- The ~numeric:~ directive affected the ~:age~ and ~:salary~ columns and the ~id:~
2126
- directive affected only the ~:id~ column. All the other cells in the table had
2127
- the default formatting applied.
2128
-
2129
- **** Location priority
2130
-
2131
- Formatting for any given cell depends on its location in the table. The
2132
- ~format_for~ method takes a location to which its formatting directive are
2133
- restricted as the first argument. It can be one of the following:
2134
-
2135
- - ~:header~ :: directive apply only to the header row, that is the first row, of
2136
- the output table,
2137
-
2138
- - ~:footer~ :: directives apply to all the footer rows of the output table,
2139
- regardless of how many there are,
2140
-
2141
- - ~gfooter~ :: directives apply to all group footer rows of the output tables,
2142
- regardless of how many there are,
2143
-
2144
- - ~:body~ :: directives apply to all rows in the body of the table unless the
2145
- row is the first row in the table or in a group and separate directives for
2146
- those have been given, in which case those directives apply,
2147
-
2148
- - ~:gfirst~ :: directives apply to the first row in each group in the body of
2149
- the table, unless the row is also the first row in the table as a whole, in
2150
- which case the ~:bfirst~ directives apply,
2151
-
2152
- - ~:bfirst~ :: directives apply to the first row in the body of the table.
2153
-
2154
- If you give directives for ~:body~, they are copied to ~:bfirst~ and ~:gfirst~
2155
- as well and can be overridden by directives for those locations.
2156
-
2157
- Directives given to the ~format~ method apply the directives to all locations in
2158
- the table, but they can be overridden by more specific directives given in a
2159
- ~format_for~ directive.
2160
-
2161
- **** Type and Column priority
2162
-
2163
- A directive based on type applies to all columns having that type unless
2164
- overridden by a directive specific to a named column; a directive based on a
2165
- column name applies only to cells in that column.
2166
-
2167
- However, there is a twist. Since the end result of formatting is to convert all
2168
- columns to strings, the formatting directives for the ~:string~ type applies to
2169
- all columns. Likewise, since all columns may contain nils, the ~nil:~ type
2170
- applies to nils in all columns regardless of the column's type.
2171
-
2172
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2173
- #+BEGIN_SRC ruby
2174
- require 'fat_table'
2175
- tab_a.to_text do |f|
2176
- f.format(string: 'R', id: '3.0C', salary: 'n[N/A]')
2177
- end
2178
- #+END_SRC
2179
-
2180
- #+BEGIN_EXAMPLE
2181
- +=====+=======+=====+============+========+============+
2182
- | Id | Name | Age | Address | Salary | Join Date |
2183
- +-----+-------+-----+------------+--------+------------+
2184
- | 001 | Paul | 32 | California | 20000 | 2001-07-13 |
2185
- | 003 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2186
- | 004 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2187
- | 005 | David | 27 | Texas | 85000 | 2007-12-13 |
2188
- | 002 | Allen | 25 | Texas | N/A | 2005-07-13 |
2189
- | 008 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2190
- | 009 | James | 44 | Norway | 5000 | 2005-07-13 |
2191
- | 010 | James | 45 | Texas | 5000 | |
2192
- +=====+=======+=====+============+========+============+
2193
- #+END_EXAMPLE
2194
-
2195
- The ~string: 'R'~ directive causes all the cells to be right-aligned except
2196
- ~:id~ which specifies centering for the ~:id~ column only. The ~n[N/A]~
2197
- directive for specifies how nil are displayed in the numeric column, ~:salary~,
2198
- but not for other nils, such as in the last row of the ~:join_date~ column.
2199
-
2200
3093
  * Development
2201
-
2202
3094
  After checking out the repo, run `bin/setup` to install dependencies. Then, run
2203
3095
  `rake spec` to run the tests. You can also run `bin/console` for an interactive
2204
3096
  prompt that will allow you to experiment.
2205
3097
 
2206
- To install this gem onto your local machine, run `bundle exec rake install`. To
2207
- release a new version, update the version number in `version.rb`, and then run
2208
- `bundle exec rake release`, which will create a git tag for the version, push
2209
- git commits and tags, and push the `.gem` file to
2210
- [rubygems.org](https://rubygems.org).
3098
+ To install this gem onto your local machine, run `bundle exec rake install`.
2211
3099
 
2212
3100
  * Contributing
2213
-
2214
3101
  Bug reports and pull requests are welcome on GitHub at
2215
3102
  https://github.com/ddoherty03/fat_table.