fat_table 0.4.2 → 0.5.1

Sign up to get free protection for your applications and to get access to all the features.
data/README.org CHANGED
@@ -1,5 +1,19 @@
1
- #+OPTIONS: :toc
1
+ #+TITLE: FatTable User Guide
2
+ #+OPTIONS: toc:4
2
3
  #+LATEX_HEADER: \usepackage[margin=0.75in]{geometry}
4
+ #+PROPERTY: header-args:ruby :colnames no :session readme :hlines yes :exports both
5
+ #+PROPERTY: header-args:sh :exports code
6
+ #+STARTUP: inlineimages
7
+
8
+ #+begin_comment
9
+ Notes on producing this README.
10
+
11
+ 1. Result blocks won't be rendered on Github if the #+RESULTS: tag is left
12
+ over the results block, so manually delete them.
13
+
14
+ 2. Make sure that the current version of the gem is geting loaded when running
15
+ the code blocks. rbenv-use-corresponding helps here.
16
+ #+end_comment
3
17
 
4
18
  #+BEGIN_COMMENT
5
19
  This is for markdown output:
@@ -33,37 +47,137 @@ they makes sense for the output medium and treats other formatting directives as
33
47
  no-ops.
34
48
 
35
49
  ~FatTable~ can be used to perform operations on data that are naturally best
36
- conceived of as tables, which in my experience is quite often. It can also serve
37
- as a foundation for providing reporting functions where flexibility about the
38
- output medium can be useful. Finally ~FatTable~ can be used within Emacs
39
- ~org-mode~ files in code blocks targeting the Ruby language. Org mode tables are
40
- presented to a ruby code block as an array of arrays, so ~FatTable~ can read
41
- them in with its ~.from_aoa~ constructor. A ~FatTable~ table output as an array
42
- of arrays with its ~.to_aoa~ output function will be rendered in an org-mode
43
- buffer as an org-table, ready for processing by other code blocks.
50
+ conceived of as tables, which in my experience is quite often. It can also
51
+ serve as a foundation for providing reporting functions where flexibility
52
+ about the output medium can be useful. Finally ~FatTable~ can be used within
53
+ Emacs ~org-mode~ files in code blocks targeting the Ruby language. Org mode
54
+ tables are presented to a ruby code block as an array of arrays, so ~FatTable~
55
+ can read them in with its ~.from_aoa~ constructor. A ~FatTable~ table output as an
56
+ array of arrays with its ~.to_aoa~ output function will be rendered in an
57
+ org-mode buffer as an org-table, ready for processing by other code blocks.
58
+
59
+ * Table of Contents :toc:noexport:
60
+ - [[#introduction][Introduction]]
61
+ - [[#installation][Installation]]
62
+ - [[#using-in-a-gem][Using in a gem]]
63
+ - [[#manually-install][Manually install]]
64
+ - [[#require][Require]]
65
+ - [[#usage][Usage]]
66
+ - [[#quick-start][Quick Start]]
67
+ - [[#a-word-about-the-examples][A Word About the Examples]]
68
+ - [[#anatomy-of-a-table][Anatomy of a Table]]
69
+ - [[#columns][Columns]]
70
+ - [[#headers][Headers]]
71
+ - [[#groups][Groups]]
72
+ - [[#constructing-tables][Constructing Tables]]
73
+ - [[#empty-tables][Empty Tables]]
74
+ - [[#without-headers][Without Headers]]
75
+ - [[#with-headers][With Headers]]
76
+ - [[#forcing-string-type][Forcing String Type]]
77
+ - [[#from-csv-or-org-mode-files-or-strings][From CSV or Org Mode files or strings]]
78
+ - [[#from-arrays-of-arrays][From Arrays of Arrays]]
79
+ - [[#in-ruby-code][In Ruby Code]]
80
+ - [[#in-emacs-org-files][In Emacs Org Files]]
81
+ - [[#from-arrays-of-hashes][From Arrays of Hashes]]
82
+ - [[#from-sql-queries][From SQL queries]]
83
+ - [[#marking-groups-in-input][Marking Groups in Input]]
84
+ - [[#accessing-parts-of-tables][Accessing Parts of Tables]]
85
+ - [[#rows][Rows]]
86
+ - [[#columns-1][Columns]]
87
+ - [[#cells][Cells]]
88
+ - [[#other-table-attributes][Other table attributes]]
89
+ - [[#operations-on-tables][Operations on Tables]]
90
+ - [[#example-input-tables][Example Input Tables]]
91
+ - [[#select][Select]]
92
+ - [[#selecting-existing-columns-also-of-omni][Selecting Existing Columns (Also of :omni)]]
93
+ - [[#copying-and-renaming-existing-columns][Copying and Renaming Existing Columns]]
94
+ - [[#adding-new-columns][Adding New Columns]]
95
+ - [[#custom-instance-variables-and-hooks][Custom Instance Variables and Hooks]]
96
+ - [[#argument-order-and-boundaries][Argument Order and Boundaries]]
97
+ - [[#where][Where]]
98
+ - [[#order_by][Order_by]]
99
+ - [[#order_with][Order_with]]
100
+ - [[#group_by][Group_by]]
101
+ - [[#join][Join]]
102
+ - [[#join-types][Join Types]]
103
+ - [[#join-expressions][Join Expressions]]
104
+ - [[#join-examples][Join Examples]]
105
+ - [[#inner-joins][Inner Joins]]
106
+ - [[#left-and-right-joins][Left and Right Joins]]
107
+ - [[#full-join][Full Join]]
108
+ - [[#cross-join][Cross Join]]
109
+ - [[#set-operations][Set Operations]]
110
+ - [[#unions][Unions]]
111
+ - [[#intersections][Intersections]]
112
+ - [[#set-differences-with-except][Set Differences with Except]]
113
+ - [[#uniq-aka-distinct][Uniq (aka Distinct)]]
114
+ - [[#remove-groups-with-degroup][Remove groups with degroup!]]
115
+ - [[#formatting-tables][Formatting Tables]]
116
+ - [[#available-formatter-output-targets][Available Formatter Output Targets]]
117
+ - [[#output-media][Output Media]]
118
+ - [[#examples][Examples]]
119
+ - [[#to-text][To Text]]
120
+ - [[#to-org][To Org]]
121
+ - [[#to-term][To Term]]
122
+ - [[#to-latex][To LaTeX]]
123
+ - [[#to-aoa-array-of-arrays][To AoA (Array of Arrays)]]
124
+ - [[#to-aoh-array-of-hashes][To AoH (Array of Hashes)]]
125
+ - [[#formatting-directives][Formatting Directives]]
126
+ - [[#string][String]]
127
+ - [[#numeric][Numeric]]
128
+ - [[#datetime][DateTime]]
129
+ - [[#boolean][Boolean]]
130
+ - [[#nilclass][NilClass]]
131
+ - [[#the-format-and-format_for-methods][The ~format~ and ~format_for~ methods]]
132
+ - [[#table-locations][Table Locations]]
133
+ - [[#location-priority][Location priority]]
134
+ - [[#type-and-column-priority][Type and Column priority]]
135
+ - [[#footers][Footers]]
136
+ - [[#adding-footers][Adding Footers]]
137
+ - [[#aggregators][Aggregators]]
138
+ - [[#footer-objects][Footer objects]]
139
+ - [[#footer-examples][Footer Examples]]
140
+ - [[#built-in-aggregators][Built-in Aggregators]]
141
+ - [[#string-aggregators][String Aggregators]]
142
+ - [[#ruby-objects][Ruby Objects]]
143
+ - [[#lambdas][Lambdas]]
144
+ - [[#invoking-formatters][Invoking Formatters]]
145
+ - [[#by-instantiating-a-formatter][By Instantiating a Formatter]]
146
+ - [[#by-using-fattable-module-level-method-calls][By Using ~FatTable~ module-level method calls]]
147
+ - [[#by-calling-methods-on-table-objects][By Calling Methods on Table Objects]]
148
+ - [[#development][Development]]
149
+ - [[#contributing][Contributing]]
44
150
 
45
151
  * Installation
46
-
47
- ** Installing the gem
48
-
152
+ ** Using in a gem
49
153
  Add this line to your application's Gemfile:
50
-
51
- #+BEGIN_SRC ruby
154
+ #+BEGIN_SRC ruby :exports code
52
155
  gem 'fat_table'
53
156
  #+END_SRC
54
157
 
55
- And then execute:
158
+ Or, something like this in your gemspec file:
159
+ #+begin_SRC ruby :exports code
160
+ gem.add_runtime_dependency 'fat_table'
161
+ #+end_SRC
56
162
 
163
+ And then execute:
57
164
  #+BEGIN_SRC sh
58
165
  $ bundle
59
166
  #+END_SRC
60
167
 
168
+ ** Manually install
61
169
  Or install it yourself as:
62
170
 
63
171
  #+BEGIN_SRC sh
64
172
  $ gem install fat_table
65
173
  #+END_SRC
66
174
 
175
+ ** Require
176
+ Somewhere in your code, make sure that =FatTable= is required:
177
+ #+begin_src ruby :exports code :results silent
178
+ require 'fat_table'
179
+ #+end_src
180
+
67
181
  * Usage
68
182
  ** Quick Start
69
183
 
@@ -71,10 +185,10 @@ Or install it yourself as:
71
185
  operated on in a number of ways. Here's a quick example to illustrate the use of
72
186
  ~FatTable~. See the detailed explanations further on down.
73
187
 
74
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
75
- #+BEGIN_SRC ruby
76
- require 'fat_table'
188
+ Here is a set of data that records some kind of stock activity. It's an array
189
+ of arrays with the first inner array being the headings.
77
190
 
191
+ #+BEGIN_SRC ruby :exports code :results silent
78
192
  data =
79
193
  [['Date', 'Code', 'Raw', 'Shares', 'Price', 'Info', 'Ok'],
80
194
  ['2013-05-29', 'S', 15_700.00, 6601.85, 24.7790, 'ENTITY3', 'F'],
@@ -89,9 +203,17 @@ operated on in a number of ways. Here's a quick example to illustrate the use of
89
203
  ['2013-05-29', 'S', 15_900.00, 6685.95, 24.5802, 'ENTITY3', 'T'],
90
204
  ['2013-05-30', 'S', 6_679.00, 2808.52, 25.0471, 'ENTITY3', 'T'],
91
205
  ['2013-05-23', 'S', 23_054.00, 9694.21, 26.8015, 'ENTITY3', 'F']]
206
+ #+END_SRC
207
+
208
+ Use FatTable to read the data and convert in into a table object. Note that
209
+ the headings within the table are all converted to symbols, lower-cased and
210
+ any spaces replaced with underscores.
92
211
 
93
- # Build the Table and then perform chained operations on it
212
+ Below, we select only those rows having more than 2000 shares, sort by a
213
+ compund key, select all columns but add a column, :ref, for the row number,
214
+ and finally re-order the columns with a final select.
94
215
 
216
+ #+BEGIN_SRC ruby :results silent :exports code
95
217
  table = FatTable.from_aoa(data) \
96
218
  .where('shares > 2000') \
97
219
  .order_by(:date, :code) \
@@ -99,27 +221,51 @@ operated on in a number of ways. Here's a quick example to illustrate the use of
99
221
  :price, :ok, ref: '@row') \
100
222
  .select(:ref, :date, :code,
101
223
  :shares, :price, :ok)
224
+ #+END_SRC
102
225
 
103
- # Convert the table to an ASCII text string
104
-
226
+ You can use the resulting table in other operations, such as performing joins
227
+ or set operations with other tables, etc. The world's your oyster. But
228
+ eventually you will want to present the table in some format, and that is
229
+ where the formatting methods come in. They let you add footers, including
230
+ groups footers, as well as styling the various elements with very simple
231
+ formatting directives that can apply to various "locations" in the table. Any
232
+ formatting directives that are beyond the capabilities of the output medium
233
+ are simply ignored.
234
+
235
+ We can format the table constructed above.
236
+ #+BEGIN_SRC ruby :exports both
105
237
  table.to_text do |fmt|
106
- # Add some table footers
238
+ # Add a group footer at the bottom of each group that results from sorting
239
+ # with the order_by method.
240
+ fmt.gfooter('Avg', shares: :avg, price: :avg)
241
+ # Add some table footers. Averages for the price and shares columns. The
242
+ # avg_footer method applies the avg aggregate to all the named columns with
243
+ # an "Average" label.
107
244
  fmt.avg_footer(:price, :shares)
245
+ # And a second footer that shows the sum for the shares column.
108
246
  fmt.sum_footer(:shares)
109
- # Add a group footer
110
- fmt.gfooter('Avg', shares: :avg, price: :avg)
111
- # Formats for all locations
247
+ # Formats for all locations, :ref column is centered and bold, all numerics
248
+ # are right-aligned, and all booleans are centered and printed with 'Y' or
249
+ # 'N'
112
250
  fmt.format(ref: 'CB', numeric: 'R', boolean: 'CY')
113
- # Formats for different "locations" in the table
251
+ # Formats for different "locations" in the table:
252
+ # The headers are all centered and bold.
114
253
  fmt.format_for(:header, string: 'CB')
254
+ # In the body rows (i.e., not the headers or footers), the code column is
255
+ # centered, shares have grouping commas applied and are rounded to one
256
+ # decimal place, but the price column is rounded to 4 places with no
257
+ # grouping commas.
115
258
  fmt.format_for(:body, code: 'C', shares: ',0.1', price: '0.4', )
259
+ # But the price column in the first row of the body (:bfirst location) will
260
+ # also be formatted with a currency symbol.
116
261
  fmt.format_for(:bfirst, price: '$0.4', )
117
- fmt.format_for(:footer, shares: 'B,0.1', price: '$B0.4', )
262
+ # In the footers, apply the same rounding rules, but make the results bold.
118
263
  fmt.format_for(:gfooter, shares: 'B,0.1', price: 'B0.4', )
264
+ fmt.format_for(:footer, shares: 'B,0.1', price: '$B0.4', )
119
265
  end
120
266
  #+END_SRC
121
267
 
122
- #+BEGIN_EXAMPLE
268
+ #+begin_example
123
269
  +=========+============+======+=============+==========+====+
124
270
  | Ref | Date | Code | Shares | Price | Ok |
125
271
  +---------+------------+------+-------------+----------+----+
@@ -154,23 +300,64 @@ operated on in a number of ways. Here's a quick example to illustrate the use of
154
300
  +---------+------------+------+-------------+----------+----+
155
301
  | Total | | | 1,020,119.1 | | |
156
302
  +=========+============+======+=============+==========+====+
157
- #+END_EXAMPLE
303
+ #+end_example
158
304
 
159
- ** A Word About the Examples
305
+ For the text format above, we were wasting our breath specifying bold styling
306
+ since there is no way to make that happen in plain ASCII text. But with
307
+ LaTeX, bold is doable. The output of the following code block is being
308
+ written to a file =examples/quicktable.tex= which is then =\included=-ed in a
309
+ simple wrapper file, =examples/quick.tex= so it can be compiled by LaTeX.
310
+
311
+ #+BEGIN_SRC ruby :results file :file "examples/quicktable.tex"
312
+ table.to_latex do |fmt|
313
+ fmt.gfooter('Avg', shares: :avg, price: :avg)
314
+ fmt.avg_footer(:price, :shares)
315
+ fmt.sum_footer(:shares)
316
+ fmt.format(ref: 'CB', numeric: 'R', boolean: 'CY')
317
+ fmt.format_for(:header, string: 'CB')
318
+ fmt.format_for(:body, code: 'C', shares: ',0.1c[blue.lightgray]', price: '0.4', )
319
+ fmt.format_for(:bfirst, price: '$0.4', )
320
+ fmt.format_for(:gfooter, shares: 'B,0.1', price: 'B0.4', )
321
+ fmt.format_for(:footer, shares: 'B,0.1', price: '$B0.4', )
322
+ end
323
+ #+END_SRC
324
+
325
+ #+begin_EXAMPLE
326
+ [[file:examples/quicktable.tex]]
327
+ #+end_EXAMPLE
328
+
329
+ These commands run pdflatex on the result twice to get the table aligned
330
+ properly.
331
+ #+begin_src sh :results silent
332
+ cd examples
333
+ pdflatex quick.tex
334
+ pdflatex quick.tex
335
+ #+end_src
336
+
337
+ And we convert the =PDF= into a smaller image for display:
338
+ #+begin_src sh :results verbatim
339
+ cd examples
340
+ pdftoppm -png quick.pdf >quick.png
341
+ convert quick.png -resize 600x800 quick_small.png
342
+ #+end_src
343
+
344
+ [[file:examples/quick_small.png]]
160
345
 
346
+ ** A Word About the Examples
161
347
  When you install the ~fat_table~ gem, you have access to a program ~ft_console~,
162
348
  which opens a ~pry~ session with ~fat_table~ loaded and the tables used in the
163
349
  examples in this ~README~ defined as instance variables so you can experiment
164
350
  with them. Because they are defined as instance variables, you have to write
165
- ~tab1~ as ~@tab1~ in ~ft_console~, but otherwise the examples should work as
166
- shown in this ~README~.
351
+ ~tab1~ as ~@tab1~ in ~ft_console~, but otherwise the examples should work as shown
352
+ in this ~README~.
167
353
 
168
- The examples in this ~README~ file are executed as code blocks within the
169
- ~README.org~ file, so they typically end with a call to ~.to_aoa~. That causes
170
- the table to be inserted into the file and formatted as a table. With
171
- ~ft_console~, you should instead display your tables with ~.to_text~ or
172
- ~.to_term~. These will return a string that you can print to the terminal with
173
- ~puts~.
354
+ The examples in this ~README~ file are executed in Emacs org-mode as code
355
+ blocks within the ~README.org~ file, so they typically end with a call to
356
+ ~.to_aoa~. That causes Emacs to insert the "Array of Array" ruby data
357
+ structure into the file and format it as a table, which is the convention for
358
+ Emacs org-mode. With ~ft_console~, you should instead display your tables with
359
+ ~.to_text~ or ~.to_term~. These will return a string that you can print to the
360
+ terminal with ~puts~.
174
361
 
175
362
  To read in the table used in the Quick Start section above, you might do the
176
363
  following:
@@ -230,7 +417,6 @@ directives.
230
417
 
231
418
  ** Anatomy of a Table
232
419
  *** Columns
233
-
234
420
  ~FatTable::Table~ objects consist of an array of ~FatTable::Column~ objects.
235
421
  Each ~Column~ has a header, a type, and an array of items, all of the given type
236
422
  or nil. There are only five permissible types for a ~Column~:
@@ -252,39 +438,38 @@ Items of input must be either one of the permissible ruby objects or strings. If
252
438
  they are strings, ~FatTable~ attempts to parse them as one of the permissible
253
439
  types as follows:
254
440
 
255
- - Boolean :: the strings, ~'t'~, ~'true'~, ~'yes'~, or ~'y'~, regardless of
256
- case, are interpreted as ~TrueClass~ and the strings, ~'f'~, ~'false'~,
257
- ~'no'~, or ~'n'~, regardless of case, are interpreted as ~FalseClass~, in
441
+ - Boolean :: The strings, ~t~, ~true~, ~yes~, or ~y~, regardless of
442
+ case, are interpreted as ~TrueClass~ and the strings, ~f~, ~false~,
443
+ ~no~, or ~n~, regardless of case, are interpreted as ~FalseClass~, in
258
444
  either case resulting in a Boolean column. Empty strings in a column
259
445
  already having a Boolean type are converted to ~nil~.
260
- - DateTime :: strings that contain patterns of ~'yyyy-mm-dd'~ or ~'yyyy/mm/dd'~
261
- or ~'mm-dd-yyy'~ or ~'mm/dd/yyyy'~ or any of the foregoing with an added
262
- ~'Thh:mm:ss'~ or ~'Thh:mm'~ will be interpreted as a ~DateTime~ or a ~Date~
446
+ - DateTime :: Strings that contain patterns of ~yyyy-mm-dd~ or ~yyyy/mm/dd~
447
+ or ~mm-dd-yyy~ or ~mm/dd/yyyy~ or any of the foregoing with an added
448
+ ~Thh:mm:ss~ or ~Thh:mm~ will be interpreted as a ~DateTime~ or a ~Date~
263
449
  (if there are no sub-day time components present). The number of digits in
264
450
  the month and day can be one or two, but the year component must be four
265
451
  digits. Any time components are valid if they can be properly interpreted
266
452
  by ~DateTime.parse~. Org mode timestamps (any of the foregoing surrounded
267
- by square '~[]~' or pointy '~<>~' brackets), active or inactive, are valid
453
+ by square ~[]~ or pointy ~<>~ brackets), active or inactive, are valid
268
454
  input strings for ~DateTime~ columns. Empty strings in a column already
269
455
  having the ~DateTime~ type are converted to ~nil~.
270
- - Numeric :: all commas ~','~, underscores, ~'_'~, and ~'$'~ dollar signs (or
271
- other currency symbol as set by ~FatTable.currency_symbol~ are removed from
272
- the string and if the remaining string can be interpreted as a ~Numeric~,
273
- it will be. It is interpreted as an ~Integer~ if there are no decimal
274
- places in the remaining string, as a ~Rational~ if the string has the form
275
- '~<number>:<number>~' or '~<number>/<number>~', or as a ~BigDecimal~ if
276
- there is a decimal point in the remaining string. Empty strings in a column
277
- already having the Numeric type are converted to nil.
278
- - String :: if all else fails, ~FatTable~ applies ~#to_s~ to the input value
456
+ - Numeric :: All commas (~,~) underscores (~_~) and (~$~) dollar signs (or
457
+ other currency symbol as set by ~FatTable.currency_symbol~ are removed from
458
+ the string and if the remaining string can be interpreted as a ~Numeric~, it
459
+ will be. It is interpreted as an ~Integer~ if there are no decimal places in
460
+ the remaining string, as a ~Rational~ if the string has the form
461
+ ~<number>:<number>~ or ~<number>/<number>~, or as a ~BigDecimal~ if there is
462
+ a decimal point in the remaining string. Empty strings in a column already
463
+ having the Numeric type are converted to nil.
464
+ - String :: If all else fails, ~FatTable~ applies ~#to_s~ to the input value
279
465
  and, treats it as an item of type ~String~. Empty strings in a column
280
466
  already having the ~String~ type are kept as empty strings.
281
- - NilClass :: until the input contains a non-blank string that can be parsed as
467
+ - NilClass :: Until the input contains a non-blank string that can be parsed as
282
468
  one of the other types, it has this type, meaning that the type is still
283
469
  open. A column comprised completely of blank strings or ~nils~ will retain
284
470
  the ~NilClass~ type.
285
471
 
286
472
  *** Headers
287
-
288
473
  Headers for the columns are formed from the input. No two columns in a table can
289
474
  have the same header. Headers in the input are converted to symbols by
290
475
 
@@ -293,33 +478,36 @@ have the same header. Headers in the input are converted to symbols by
293
478
  - removing any characters that are not letters, numbers, or underscores, and
294
479
  - lowercasing all remaining letters
295
480
 
296
- Thus, a header of ~'Date'~ becomes ~:date~, a header of ~'Id Number'~ becomes,
481
+ Thus, a header of ~Date~ becomes ~:date~, a header of ~Id Number~ becomes,
297
482
  ~:id_number~, etc. When referring to a column in code, you must use the symbol
298
483
  form of the header.
299
484
 
300
485
  If no sensible headers can be discerned from the input, headers of the form
301
486
  ~:col_1~, ~:col_2~, etc., are synthesized.
302
487
 
303
- *** Groups
488
+ You should avoid the use of the column names ~:omni~ and ~:sort_key~ because
489
+ they have special meanings in the ~select~ and ~order_with~ commands,
490
+ respectively.
304
491
 
305
- The rows of a ~FatTable~ table can be sub-divided into groups, either from
306
- markers in the input or as a result of certain operations. There is only one
307
- level of grouping, so ~FatTable~ has no concept of sub-groups. Groups can be
308
- shown on output with rules or "hlines" that underline the last row in each
309
- group, and you can decorate the output with group footers that summarize the
310
- columns in each group.
492
+ *** Groups
493
+ The rows of a ~FatTable~ table can be divided into groups, either from markers
494
+ in the input or as a result of certain operations. There is only one level of
495
+ grouping, so ~FatTable~ has no concept of sub-groups. Groups can be shown on
496
+ output with rules or "hlines" that underline the last row in each group, and
497
+ you can decorate the output with group footers that summarize the rows in
498
+ each group.
311
499
 
312
500
  ** Constructing Tables
313
501
  *** Empty Tables
502
+ **** Without Headers
503
+ You can create an empty table with ~FatTable::Table.new~ or, the shorter form,
504
+ ~FatTable.new~, and then add rows with the ~<<~ operator and a Hash. The keys
505
+ in the added rows determine the names of the headers:
314
506
 
315
- You can create an empty table with ~FatTable.new~, and then add rows with the
316
- ~<<~ operator and a Hash:
317
-
318
- #+BEGIN_SRC ruby
507
+ #+BEGIN_SRC ruby :results silent
319
508
  tab = FatTable.new
320
- tab << { a: 1, b: 2, c: "<2017-01-21>', d: 'f', e: '' }
509
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '' }
321
510
  tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
322
- tab.to_aoa
323
511
  #+END_SRC
324
512
 
325
513
  After this, the table will have column headers ~:a~, ~:b~, ~:c~, ~:d~, and ~:e~.
@@ -328,14 +516,115 @@ Column, ~:a~ and ~:b~ will have type Numeric, column ~:c~ will have type
328
516
  have an open type. Notice that dates in the input can be wrapped in brackets as
329
517
  in org-mode time stamps.
330
518
 
331
- *** From CSV or Org Mode files or strings
519
+ #+BEGIN_SRC ruby :wrap EXAMPLE
520
+ tab.to_text
521
+ #+END_SRC
522
+
523
+ #+begin_EXAMPLE
524
+ +======+======+============+===+===+
525
+ | A | B | C | D | E |
526
+ +------+------+------------+---+---+
527
+ | 1 | 2 | 2017-01-21 | F | |
528
+ | 3.14 | 2.17 | 2016-01-21 | T | |
529
+ +======+======+============+===+===+
530
+ #+end_EXAMPLE
531
+
532
+ You can continue to add rows to the table:
533
+ #+BEGIN_SRC ruby :results silent
534
+ tab << { 'F' => '335:113', a: Rational(3, 5) }
535
+ #+END_SRC
536
+
537
+ This last ~<<~ operation adds a new column headed ~:f~ to the table and makes
538
+ the value of =:f= in all prior rows ~nil~. Also, the values for the new row
539
+ for which no key was give are assigned ~nil~ as well:
540
+
541
+ #+BEGIN_SRC ruby
542
+ tab.to_text
543
+ #+END_SRC
544
+
545
+ #+begin_EXAMPLE
546
+ +======+======+============+===+===+=========+
547
+ | A | B | C | D | E | F |
548
+ +------+------+------------+---+---+---------+
549
+ | 1 | 2 | 2017-01-21 | F | | |
550
+ | 3.14 | 2.17 | 2016-01-21 | T | | |
551
+ +------+------+------------+---+---+---------+
552
+ | 3/5 | | | | | 335/113 |
553
+ +======+======+============+===+===+=========+
554
+ #+end_EXAMPLE
555
+
556
+ **** With Headers
557
+ Alternatively, you can specify the headers at the outset, in which case,
558
+ headers in added rows that do not match any of the initial headers cause new
559
+ columns to be created:
560
+
561
+ #+BEGIN_SRC ruby :wrap EXAMPLE :results raw
562
+ require 'fat_table'
563
+ tab = FatTable.new(:a, 'b', 'C', :d)
564
+ tab.headers
565
+ #+END_SRC
566
+
567
+ #+begin_EXAMPLE
568
+ [:a, :b, :c, :d]
569
+ #+end_EXAMPLE
570
+
571
+ #+begin_src ruby :wrap EXAMPLE
572
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '' }
573
+ tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
574
+ tab.to_text
575
+ #+end_src
576
+
577
+ #+begin_EXAMPLE
578
+ +======+======+============+===+===+
579
+ | A | B | C | D | E |
580
+ +------+------+------------+---+---+
581
+ | 1 | 2 | 2017-01-21 | F | |
582
+ | 3.14 | 2.17 | 2016-01-21 | T | |
583
+ +------+------+------------+---+---+
584
+ | 1 | 2 | 2017-01-21 | F | |
585
+ | 3.14 | 2.17 | 2016-01-21 | T | |
586
+ +======+======+============+===+===+
587
+ #+end_EXAMPLE
588
+
589
+ **** Forcing String Type
590
+ Occasionally, ~FatTable~'s automatic type detection can get in the way and you
591
+ just want it to treat one or more columns as Strings regardless of their
592
+ appearance. Think, for example, of zip codes. At any time after creating a
593
+ table, you can have it force the String type on any number of columns with the
594
+ ~force_string!~ method. When you do so, all exisiting items in the column are
595
+ converted to strings with the #to_s method.
596
+
597
+ #+begin_src ruby :wrap EXAMPLE
598
+ tab = FatTable.new(:a, 'b', 'C', :d, :zip)
599
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '', zip: 18552 }
600
+ tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
601
+ tab.force_string!(:zip, :c)
602
+ tab << { zip: '01879' }
603
+ tab << { zip: '66210' }
604
+ tab << { zip: '90210' }
605
+ tab.to_text
606
+ #+end_src
607
+
608
+ #+begin_EXAMPLE
609
+ +======+======+============+===+=======+===+
610
+ | A | B | C | D | Zip | E |
611
+ +------+------+------------+---+-------+---+
612
+ | 1 | 2 | 2017-01-21 | F | 18552 | |
613
+ | 3.14 | 2.17 | 2016-01-21 | T | | |
614
+ | | | | | 01879 | |
615
+ | | | | | 66210 | |
616
+ | | | | | 90210 | |
617
+ +======+======+============+===+=======+===+
618
+ #+end_EXAMPLE
332
619
 
620
+ *** From CSV or Org Mode files or strings
333
621
  Tables can also be read from ~.csv~ files or files containing ~org-mode~ tables.
622
+
334
623
  In the case of org-mode files, ~FatTable~ skips through the file until it finds
335
624
  a line that look like a table, that is, it begins with any number of spaces
336
625
  followed by ~|-~. Only the first table in an ~.org~ file is read.
337
626
 
338
- For both ~.csv~ and ~.org~ files, the first row in the tables is taken as the
627
+ For both ~.csv~ and ~.org~ files, the first row in the table is taken as the
339
628
  header row, and the headers are converted to symbols as described above.
340
629
 
341
630
  #+BEGIN_SRC ruby
@@ -385,7 +674,7 @@ header row, and the headers are converted to symbols as described above.
385
674
  #+END_SRC
386
675
 
387
676
  *** From Arrays of Arrays
388
-
677
+ **** In Ruby Code
389
678
  You can also initialize a table directly from ruby data structures. You can, for
390
679
  example, build a table from an array of arrays:
391
680
 
@@ -411,6 +700,7 @@ example, build a table from an array of arrays:
411
700
  Notice that the values can either be ruby objects, such as the Integer ~85_000~,
412
701
  or strings that can be parsed into one of the permissible column types.
413
702
 
703
+ **** In Emacs Org Files
414
704
  This method of building a table, ~.from_aoa~, is particularly useful in dealing
415
705
  with Emacs org-mode code blocks. Tables in org-mode are passed to code blocks as
416
706
  arrays of arrays. Likewise, a result of a code block in the form of an array of
@@ -488,7 +778,7 @@ to a Hash with the ~#to_h~ method, so you can use an array of your own objects
488
778
  to initialize a table, provided that you define a suitable ~#to_h~ method for
489
779
  the objects' class.
490
780
 
491
- #+BEGIN_SRC ruby
781
+ #+BEGIN_SRC ruby :results silent
492
782
  aoh = [
493
783
  { ref: 'T001', date: '2016-11-01', code: 'P', price: '7.7000', shares: 100 },
494
784
  { ref: 'T002', date: '2016-11-01', code: 'P', price: 7.7500, shares: 200 },
@@ -514,7 +804,6 @@ Notice, again, that the values can either be ruby objects, such as ~Date.today~,
514
804
  or strings that can be parsed into one of the permissible column types.
515
805
 
516
806
  *** From SQL queries
517
-
518
807
  Another way to initialize a ~FatTable~ table is with the results of a SQL
519
808
  query. Before you can connect to a database, you need to make sure that the required
520
809
  adapter for your database is installed. ~FatTable~ uses the ~sequel~ gem
@@ -529,15 +818,30 @@ You must first set the database parameters to be used for the queries.
529
818
 
530
819
  #+BEGIN_SRC ruby
531
820
  # This automatically requires sequel.
532
- require 'fat_table'
533
- FatTable.connect(adapter: 'postgres',
534
- database: 'XXX_development',
535
- user: 'ken',
536
- password: 'imsecret',
537
- host: 'db.lan')
538
- tab = FatTable.from_sql('select * from trades;')
821
+ FatTable.connect(adapter: 'sqlite',
822
+ database: 'examples/trades.db')
823
+ tab = FatTable.from_sql('select * from trans;').to_text
539
824
  #+END_SRC
540
825
 
826
+ #+begin_example
827
+ +============+======+==========+==========+=========+=========+====+
828
+ | Date | Code | Raw | Shares | Price | Info | Ok |
829
+ +------------+------+----------+----------+---------+---------+----+
830
+ | 2013-05-29 | S | 15700.0 | 6601.85 | 24.779 | ENTITY3 | F |
831
+ | 2013-05-02 | P | 118186.4 | 118186.4 | 11.85 | ENTITY1 | T |
832
+ | 2013-05-20 | S | 12000.0 | 5046.0 | 28.2804 | ENTITY3 | F |
833
+ | 2013-05-23 | S | 8000.0 | 3364.0 | 27.1083 | ENTITY3 | T |
834
+ | 2013-05-23 | S | 39906.0 | 16780.47 | 25.1749 | ENTITY3 | T |
835
+ | 2013-05-20 | S | 85000.0 | 35742.5 | 28.3224 | ENTITY3 | T |
836
+ | 2013-05-02 | P | 795546.2 | 795546.2 | 1.185 | ENTITY1 | T |
837
+ | 2013-05-29 | S | 13459.0 | 5659.51 | 24.7464 | ENTITY3 | T |
838
+ | 2013-05-20 | S | 33302.0 | 14003.49 | 28.6383 | ENTITY3 | T |
839
+ | 2013-05-29 | S | 15900.0 | 6685.95 | 24.5802 | ENTITY3 | T |
840
+ | 2013-05-30 | S | 6679.0 | 2808.52 | 25.0471 | ENTITY3 | T |
841
+ | 2013-05-23 | S | 23054.0 | 9694.21 | 26.8015 | ENTITY3 | F |
842
+ +============+======+==========+==========+=========+=========+====+
843
+ #+end_example
844
+
541
845
  The arguments to ~connect~ are simply passed on to ~sequel~'s connect method, so
542
846
  any set of arguments that work for it should work for ~connect~. Alternatively,
543
847
  you can build the ~Sequel~ connection directly with ~Sequel.connect~ or with
@@ -545,7 +849,6 @@ adapter-specific ~Sequel~ connection methods and let ~FatTable~ know to use that
545
849
  connection:
546
850
 
547
851
  #+BEGIN_SRC ruby
548
- require 'fat_table'
549
852
  FatTable.db = Sequel.connect('postgres://user:password@localhost/dbname')
550
853
  FatTable.db = Sequel.ado(conn_string: 'Provider=Microsoft.ACE.OLEDB.12.0;Data Source=drive:\path\filename.accdb')
551
854
  #+END_SRC
@@ -558,7 +861,6 @@ creates will be used for all subsequent ~.from_sql~ calls until ~.connect~ is
558
861
  called again.
559
862
 
560
863
  *** Marking Groups in Input
561
-
562
864
  ~FatTable~ tables has a concept of "groups" of rows that play a role in many of
563
865
  the methods for operating on them as explained [[Groups][below]].
564
866
 
@@ -577,13 +879,12 @@ the form ~:col_1~, ~:col_2~, ... ~:col_n~.
577
879
  In org mode table text passed to ~.from_org_file~ and ~.from_org_string~, you
578
880
  /must/ mark the header row by following it with an hrule and you /may/ mark
579
881
  group boundaries with an hrule. In org mode tables, hlines are table rows
580
- beginning with something like '~|---~'. The ~.from_org_...~ functions always
882
+ beginning with something like ~|---~. The ~.from_org_...~ functions always
581
883
  recognizes hlines in the input, so it takes no ~hlines:~ keyword parameter.
582
884
 
583
885
  ** Accessing Parts of Tables
584
886
 
585
887
  *** Rows
586
-
587
888
  A ~FatTable~ table is an Enumerable, yielding each row of the table as a Hash
588
889
  keyed on the header symbols. The method ~Table#rows~ returns an Array of the
589
890
  rows as Hashes as well.
@@ -594,15 +895,13 @@ data row of the table, while ~tab[0]~ returns the first row and tab[-1] returns
594
895
  the last row.
595
896
 
596
897
  *** Columns
597
-
598
898
  If the index provided to ~[]~ is a string or a symbol, it returns an Array of
599
899
  the items of the column with that header. Thus, ~tab[:ref]~ returns an Array of
600
900
  all the items of the table's ~:ref~ column.
601
901
 
602
902
  *** Cells
603
-
604
- The two forms of indexing can be combined to access individual cells of the
605
- table:
903
+ The two forms of indexing can be combined, in either order, to access
904
+ individual cells of the table:
606
905
 
607
906
  #+BEGIN_SRC ruby
608
907
  tab[13] # => Hash of the 14th row
@@ -612,53 +911,76 @@ table:
612
911
  #+END_SRC
613
912
 
614
913
  *** Other table attributes
914
+ Here is a quick rundown of other table attributes that you can access:
615
915
 
616
916
  #+BEGIN_SRC ruby
617
917
  tab.headers # => an Array of the headers in symbol form
618
918
  tab.types # => a Hash mapping headers to column types
919
+ tab.type(head) # => return the type of the column for the given head
619
920
  tab.size # => the number of rows in the table
620
921
  tab.width # => the number of columns in the table
621
922
  tab.empty? # => is the table empty?
622
- tab.column?(head) # => does the table have a column with the given header?
923
+ tab.column(head) # => return the FatTable::Column object for the given head
924
+ tab.column?(head) # => does the table have a column with the given head?
623
925
  tab.groups # => return an Array of the table's groups as Arrays of row Hashes.
624
926
  #+END_SRC
625
927
 
626
- ** Operations on Tables
928
+ You should note that what the ~.types~ and ~.type(head)~ methods return is a
929
+ string naming the "type" assigned by ~FatTable~. All of them are also the
930
+ names of Ruby classes except to 'Boolean' a class that doesn't exist in Ruby.
931
+ The value ~true~ is a member of the ~TrueClass~ and ~false~ a member of the
932
+ ~FalseClass~. So for ~FatTable~ to provide a column of type 'Boolean'
933
+ requires it to synthesize the type from these Ruby classes.
934
+
935
+ #+begin_src ruby :wrap EXAMPLE :results raw
936
+ tab.types
937
+ #+end_src
938
+
939
+ #+begin_EXAMPLE
940
+ {:a=>"Numeric", :b=>"Numeric", :c=>"DateTime", :d=>"Boolean", :e=>"NilClass", :f=>"Numeric"}
941
+ #+end_EXAMPLE
942
+
943
+ #+begin_src ruby :wrap EXAMPLE :results output
944
+ puts "Column :d says its type is '#{tab.type(:d)}' and that is a #{tab.type(:d).class}"
945
+ #+end_src
946
+
947
+ #+begin_EXAMPLE
948
+ Column :d says its type is 'Boolean' and that is a String
949
+ #+end_EXAMPLE
627
950
 
951
+ ** Operations on Tables
628
952
  Once you have one or more tables, you will likely want to perform operations on
629
953
  them. The operations provided by ~FatTable~ are the subject of this section.
630
954
  Before getting into the operations, though, there are a couple of issues that
631
955
  cut across all or many of the operations.
632
956
 
633
957
  First, tables are by and large immutable objects. Each operation creates a new
634
- table without affecting the input tables. The only exception is the ~degroup!~
635
- operation, which mutates the receiver table by removing its group boundaries.
958
+ table without affecting the input tables. The only exceptions are the
959
+ ~degroup!~ operation, which mutates the receiver table by removing its group
960
+ boundaries, and ~force_string!~ (explained above at [[*Forcing String Type][Forcing String Type]]),
961
+ which forces columns to have the String type despite what the automatic typing
962
+ rules determine.
636
963
 
637
964
  Second, because each operation returns a ~FatTable::Table~ object, the
638
965
  operations are chainable.
639
966
 
640
- <<Groups>>
641
967
  Third, ~FatTable::Table~ objects can have "groups" of rows within the table.
642
- These can be decorated with hlines and group footers on output. Some of these
643
- operations result in marking group boundaries in the result table, others remove
644
- group boundaries that may have existed in the input table. Operations that
645
- either create or remove groups will be noted below.
968
+ These can be decorated with hlines and group footers on output. Some
969
+ operations result in marking group boundaries in the result table, others
970
+ remove group boundaries that may have existed in the input table. Operations
971
+ that either create or remove groups will be noted below.
646
972
 
647
973
  Finally, the operations are for the most part patterned on SQL table operations,
648
974
  but when expressions play a role, you write them using ruby syntax rather than
649
975
  SQL.
650
976
 
651
- *** Example Input Table
652
-
977
+ *** Example Input Tables
653
978
  For illustration purposes assume that the following tables are read into ruby
654
- variables called '~tab1~' and '~tab2~. We have given the table groups, marked by
979
+ variables called ~tab1~ and ~tab2~. We have given the table groups, marked by
655
980
  the hlines below, and included some duplicate rows to illustrate the effect of
656
981
  certain operations on groups and duplicates.
657
982
 
658
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
659
- #+BEGIN_SRC ruby
660
- require 'fat_table'
661
-
983
+ #+BEGIN_SRC ruby :results silent
662
984
  tab1_str = <<-EOS
663
985
  | Ref | Date | Code | Price | G10 | QP10 | Shares | LP | QP | IPLP | IPQP |
664
986
  |------+------------------+------+--------+-----+------+--------+------+-------+--------+--------|
@@ -708,26 +1030,31 @@ tab2_str = <<-EOS
708
1030
  | T021 | [2017-01-23 Mon] | P | 7.16 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
709
1031
  | T021 | [2017-01-23 Mon] | P | 7.16 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
710
1032
  EOS
1033
+ #+END_SRC
711
1034
 
712
- tab1 = FatTable.from_org_string(tab1_str)
713
- tab2 = FatTable.from_org_string(tab2_str)
1035
+ Rendering ~tab1~ into Emacs org-mode:
1036
+ #+BEGIN_SRC ruby :wrap EXAMPLE :results silent
1037
+ tab1 = FatTable.from_org_string(tab1_str)
714
1038
  #+END_SRC
715
1039
 
716
- *** Select
1040
+ Rendering ~tab2~ into Emacs org-mode:
717
1041
 
718
- With the ~select~ method, you can select which existing columns should appear in
719
- the output table and create new columns in the output table that are a function
720
- of existing and new columns.
1042
+ #+BEGIN_SRC ruby :wrap EXAMPLE :results silent
1043
+ tab2 = FatTable.from_org_string(tab2_str)
1044
+ #+END_SRC
721
1045
 
722
- **** Selecting Existing Columns
1046
+ *** Select
1047
+ With the ~select~ method, you can select columns to appear in the output
1048
+ table, rearrange their order, and create new columns that are a function of
1049
+ other columns.
723
1050
 
1051
+ **** Selecting Existing Columns (Also of :omni)
724
1052
  Here we select three existing columns by simply passing header symbols in the
725
1053
  order we want them to appear in the output. Thus, one use of =select= is to
726
1054
  filter and permute the order of existing columns. The =select= method preserves
727
1055
  any group boundaries present in the input table.
728
1056
 
729
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
730
- #+BEGIN_SRC ruby
1057
+ #+BEGIN_SRC ruby :wrap EXAMPLE
731
1058
  tab1.select(:price, :ref, :shares).to_aoa
732
1059
  #+END_SRC
733
1060
 
@@ -758,15 +1085,91 @@ any group boundaries present in the input table.
758
1085
  | 8.25 | T016 | 100 |
759
1086
  #+END_EXAMPLE
760
1087
 
761
- **** Adding New Columns
1088
+ It can be tedious to type the names of all the columns in a ~select~
1089
+ statement, so ~FatTable~ recognizes the special column name ~:omni~. If the
1090
+ ~select~'s first and only column argument is ~:omni~, it will expand to the
1091
+ names of all the existing columns in the table. Use of ~:omni~ otherwise is
1092
+ not interpreted specially, so you will get an error complaining about a
1093
+ non-existent column unless you happen to have a column named ~:omni~ in your
1094
+ table, which is not advisable. You can add hash arguments after ~:omni~ but
1095
+ you cannot add additional column names:
1096
+
1097
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1098
+ tab1.select(:omni, cost: 'shares * price').to_aoa
1099
+ #+END_SRC
1100
+
1101
+ #+begin_EXAMPLE
1102
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp | Cost |
1103
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1104
+ | T001 | 2016-11-01 | P | 7.7 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 770.0 |
1105
+ | T002 | 2016-11-01 | P | 7.75 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1550.0 |
1106
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0 |
1107
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0 |
1108
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1109
+ | T004 | 2016-11-01 | S | 7.55 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 | 51423.05 |
1110
+ | T005 | 2016-11-01 | S | 7.5 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 | 30000.0 |
1111
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0 |
1112
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0 |
1113
+ | T007 | 2016-11-01 | S | 7.65 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1530.0 |
1114
+ | T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 | 21198.15 |
1115
+ | T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 | 72580.0 |
1116
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1117
+ | T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 | 23971.25 |
1118
+ | T011 | 2016-11-02 | P | 7.425 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 742.5 |
1119
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0 |
1120
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0 |
1121
+ | T013 | 2016-11-02 | P | 7.35 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 | 390285.0 |
1122
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1123
+ | T014 | 2016-11-02 | P | 7.45 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 | 43560.15 |
1124
+ | T015 | 2016-11-02 | P | 7.75 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 | 3875.0 |
1125
+ | T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 | 825.0 |
1126
+ #+end_EXAMPLE
1127
+
1128
+ **** Copying and Renaming Existing Columns
1129
+ After the list of selected column names in the call to ~select~, you can add
1130
+ any number of hash-like arguments. You can use these to add a copy of an
1131
+ existing column. By calling select again, you can include only the copied
1132
+ column, in effect renaming it. For example, if you want ~tab1~ but with ~:ref~
1133
+ changed to ~:id~, just add an argument to define the new ~:id~ column:
1134
+
1135
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1136
+ tab1.select(:omni, id: :ref).
1137
+ select(:id, :date, :code, :price, :shares).to_aoa
1138
+ #+END_SRC
1139
+
1140
+ #+begin_EXAMPLE
1141
+ | Id | Date | Code | Price | Shares |
1142
+ |------+------------+------+-------+--------|
1143
+ | T001 | 2016-11-01 | P | 7.7 | 100 |
1144
+ | T002 | 2016-11-01 | P | 7.75 | 200 |
1145
+ | T003 | 2016-11-01 | P | 7.5 | 800 |
1146
+ | T003 | 2016-11-01 | P | 7.5 | 800 |
1147
+ |------+------------+------+-------+--------|
1148
+ | T004 | 2016-11-01 | S | 7.55 | 6811 |
1149
+ | T005 | 2016-11-01 | S | 7.5 | 4000 |
1150
+ | T006 | 2016-11-01 | S | 7.6 | 1000 |
1151
+ | T006 | 2016-11-01 | S | 7.6 | 1000 |
1152
+ | T007 | 2016-11-01 | S | 7.65 | 200 |
1153
+ | T008 | 2016-11-01 | P | 7.65 | 2771 |
1154
+ | T009 | 2016-11-01 | P | 7.6 | 9550 |
1155
+ |------+------------+------+-------+--------|
1156
+ | T010 | 2016-11-01 | P | 7.55 | 3175 |
1157
+ | T011 | 2016-11-02 | P | 7.425 | 100 |
1158
+ | T012 | 2016-11-02 | P | 7.55 | 4700 |
1159
+ | T012 | 2016-11-02 | P | 7.55 | 4700 |
1160
+ | T013 | 2016-11-02 | P | 7.35 | 53100 |
1161
+ |------+------------+------+-------+--------|
1162
+ | T014 | 2016-11-02 | P | 7.45 | 5847 |
1163
+ | T015 | 2016-11-02 | P | 7.75 | 500 |
1164
+ | T016 | 2016-11-02 | P | 8.25 | 100 |
1165
+ #+end_EXAMPLE
762
1166
 
763
- More interesting is that ~select~ can take hash-like keyword arguments after the
764
- symbol arguments to create new columns in the output as functions of other
765
- columns. For each hash-like parameter, the keyword given must be a symbol, which
766
- becomes the header for the new column, and the value must be either: (1) a
767
- symbol representing an existing column, which has the effect of renaming an
768
- existing column, or (2) a string representing a ruby expression for the value of
769
- a new column.
1167
+ **** Adding New Columns
1168
+ More interesting is that ~select~ can take hash-like keyword arguments after
1169
+ the symbol arguments to create new columns in the output as functions of other
1170
+ columns. For each hash-like parameter, the keyword given must be a symbol,
1171
+ which becomes the header for the new column, and the value can be a string
1172
+ representing a ruby expression for the value of a new column.
770
1173
 
771
1174
  Within the string expression, the names of existing or already-specified
772
1175
  columns are available as local variables. In addition the instance variables
@@ -781,8 +1184,7 @@ variables are set the number of the current row and group number respectively.
781
1184
  For example, if we want to rename the ~traded_on~ column to ~:date~ and add a
782
1185
  new column to compute the cost of shares, we could do the following:
783
1186
 
784
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
785
- #+BEGIN_SRC ruby
1187
+ #+BEGIN_SRC ruby :wrap EXAMPLE
786
1188
  tab1.select(:ref, :price, :shares, traded_on: :date, cost: 'price * shares').to_aoa
787
1189
  #+END_SRC
788
1190
 
@@ -813,8 +1215,8 @@ new column to compute the cost of shares, we could do the following:
813
1215
  | T016 | 8.25 | 100 | 2016-11-02 | 825.0 |
814
1216
  #+END_EXAMPLE
815
1217
 
816
- The parameter '~traded_on: :date~' caused the ~:date~ column of the input table
817
- to be renamed '~:traded_on~, and the parameter ~cost: 'price * shares'~ created
1218
+ The parameter ~traded_on: :date~ caused the ~:date~ column of the input table
1219
+ to be renamed ~:traded_on~, and the parameter ~cost: 'price * shares'~ created
818
1220
  a new column, ~:cost~, as the product of values in the ~:price~ and ~:shares~
819
1221
  columns.
820
1222
 
@@ -822,14 +1224,12 @@ The order of the columns in the result tables is the same as the order of the
822
1224
  parameters to the ~select~ method. So, you can re-order the columns with a
823
1225
  second, chained call to ~select~:
824
1226
 
825
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
826
- #+BEGIN_SRC ruby
827
- tab1.select(:ref, :price, :shares, traded_on: :date, cost: 'price * shares') \
828
- .select(:ref, :traded_on, :price, :shares, :cost) \
829
- .to_aoa
1227
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1228
+ tab1.select(:ref, :price, :shares, traded_on: :date, cost: 'price * shares').
1229
+ select(:ref, :traded_on, :price, :shares, :cost).to_aoa
830
1230
  #+END_SRC
831
1231
 
832
- #+BEGIN_EXAMPLE
1232
+ #+begin_EXAMPLE
833
1233
  | Ref | Traded On | Price | Shares | Cost |
834
1234
  |------+------------+-------+--------+----------|
835
1235
  | T001 | 2016-11-01 | 7.7 | 100 | 770.0 |
@@ -854,10 +1254,9 @@ second, chained call to ~select~:
854
1254
  | T014 | 2016-11-02 | 7.45 | 5847 | 43560.15 |
855
1255
  | T015 | 2016-11-02 | 7.75 | 500 | 3875.0 |
856
1256
  | T016 | 2016-11-02 | 8.25 | 100 | 825.0 |
857
- #+END_EXAMPLE
1257
+ #+end_EXAMPLE
858
1258
 
859
1259
  **** Custom Instance Variables and Hooks
860
-
861
1260
  As the above examples demonstrate, the instance variables ~@row~ and ~@group~
862
1261
  are available when evaluating expressions that add new columns. You can also set
863
1262
  up your own instance variables as well for keeping track of things that cross
@@ -878,8 +1277,7 @@ shows the cumulative cost after each transaction in our example table. The
878
1277
  following example uses the ~ivars:~ and ~before_hook:~ parameters to keep track
879
1278
  of the running cost of shares, then formats the table.
880
1279
 
881
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
882
- #+BEGIN_SRC ruby
1280
+ #+BEGIN_SRC ruby :wrap EXAMPLE
883
1281
  tab = tab1.select(:ref, :price, :shares, traded_on: :date, \
884
1282
  cost: 'price * shares', cumulative: '@total_cost', \
885
1283
  ivars: { total_cost: 0 }, \
@@ -917,7 +1315,6 @@ of the running cost of shares, then formats the table.
917
1315
  #+END_EXAMPLE
918
1316
 
919
1317
  **** Argument Order and Boundaries
920
-
921
1318
  Notice that ~select~ can take any number of arguments but all the symbol
922
1319
  arguments must come first followed by all the hash-like keyword arguments,
923
1320
  including the special arguments for instance variables and hooks.
@@ -926,21 +1323,21 @@ As the example illustrates, ~.select~ transmits any group boundaries in its
926
1323
  input table to the result table.
927
1324
 
928
1325
  *** Where
929
-
930
1326
  You can filter the rows of the result table with the ~.where~ method. It takes a
931
1327
  single string expression as an argument which is evaluated in a manner similar
932
1328
  to ~.select~ in which the value of the cells in each column are available as
933
1329
  local variables and the instance variables ~@row~ and ~@group~ are available for
934
1330
  testing. The expression is evaluated for each row, and if the expression
935
1331
  evaluates to a truthy value, the row is included in the output, otherwise it is
936
- not. The ~.where~ method obliterates any group boundaries in the input, so the
937
- output table has only a single group.
1332
+ not.
1333
+
1334
+ The ~.where~ method removes any group boundaries in the input, so the output
1335
+ table has only a single group.
938
1336
 
939
1337
  Here we select only those even-numbered rows where either of the two boolean
940
1338
  fields is true:
941
1339
 
942
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
943
- #+BEGIN_SRC ruby
1340
+ #+BEGIN_SRC ruby :wrap EXAMPLE
944
1341
  tab1.where('@row.even? && (g10 || qp10)') \
945
1342
  .to_aoa
946
1343
  #+END_SRC
@@ -956,7 +1353,6 @@ fields is true:
956
1353
  #+END_EXAMPLE
957
1354
 
958
1355
  *** Order_by
959
-
960
1356
  You can sort a table on any number of columns with ~order_by~. The ~order_by~
961
1357
  method takes any number of symbol arguments for the columns to sort on. If you
962
1358
  specify more than one column, the sort is performed on the first column, then
@@ -967,8 +1363,7 @@ All columns of the input table are included in the output.
967
1363
 
968
1364
  Let's sort our table first by ~:code~, then in reverse order of ~:date~.
969
1365
 
970
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
971
- #+BEGIN_SRC ruby
1366
+ #+BEGIN_SRC ruby :wrap EXAMPLE
972
1367
  tab1.order_by(:code, :date!) \
973
1368
  .to_aoa
974
1369
  #+END_SRC
@@ -1004,8 +1399,59 @@ input, it adds group boundaries in the output table at those rows where the sort
1004
1399
  keys change. Thus, in each group, ~:code~ and ~:date~ are the same, and when
1005
1400
  either changes, ~order_by~ inserts a group boundary.
1006
1401
 
1007
- *** Group_by
1402
+ *** Order_with
1403
+ The ~order_with~ method is a convenient combination of ~select~ and
1404
+ ~order_by~. It takes a single string expression as an argument to serve as a
1405
+ sort key---one that would be valid as a select expression---but with an
1406
+ optional trailing ~!~ to indicate reverse sort. The resulting table has an
1407
+ additional column called ~:sort_key~ with the expression evaluated for each
1408
+ row, and the table is sorted as with ~order_by~ on that column.
1409
+
1410
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1411
+ tab1.order_with('price * shares').to_aoa
1412
+ #+END_SRC
1413
+
1414
+ #+begin_EXAMPLE
1415
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp | Sort Key |
1416
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1417
+ | T011 | 2016-11-02 | P | 7.425 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 742.5 |
1418
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1419
+ | T001 | 2016-11-01 | P | 7.7 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 770.0 |
1420
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1421
+ | T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 | 825.0 |
1422
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1423
+ | T007 | 2016-11-01 | S | 7.65 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1530.0 |
1424
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1425
+ | T002 | 2016-11-01 | P | 7.75 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1550.0 |
1426
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1427
+ | T015 | 2016-11-02 | P | 7.75 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 | 3875.0 |
1428
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1429
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0 |
1430
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0 |
1431
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1432
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0 |
1433
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0 |
1434
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1435
+ | T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 | 21198.15 |
1436
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1437
+ | T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 | 23971.25 |
1438
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1439
+ | T005 | 2016-11-01 | S | 7.5 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 | 30000.0 |
1440
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1441
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0 |
1442
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0 |
1443
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1444
+ | T014 | 2016-11-02 | P | 7.45 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 | 43560.15 |
1445
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1446
+ | T004 | 2016-11-01 | S | 7.55 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 | 51423.05 |
1447
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1448
+ | T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 | 72580.0 |
1449
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------+----------|
1450
+ | T013 | 2016-11-02 | P | 7.35 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 | 390285.0 |
1451
+ #+end_EXAMPLE
1008
1452
 
1453
+
1454
+ *** Group_by
1009
1455
  Like ~order_by~, ~group_by~ takes a set of parameters of column header symbols,
1010
1456
  the "grouping parameters", by which to sort the table into a set of groups that
1011
1457
  are equal with respect to values in those columns. In addition, those parameters
@@ -1019,8 +1465,7 @@ For example, let's summarize the ~trades~ table by ~:code~ and ~:price~ again,
1019
1465
  and determine total shares, average price, and a few other features of each
1020
1466
  group:
1021
1467
 
1022
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1023
- #+BEGIN_SRC ruby
1468
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1024
1469
  tab1.group_by(:code, :date, price: :avg,
1025
1470
  shares: :sum, lp: :sum, qp: :sum,
1026
1471
  qp10: :all?) \
@@ -1040,7 +1485,7 @@ hash-like "aggregating" parameters where the key is the column to aggregate and
1040
1485
  the value is a symbol for one of several aggregating methods that
1041
1486
  ~FatTable::Column~ objects understand. For example, the ~:avg~ method is applied
1042
1487
  to the :price column so that the output shows the average price in each group.
1043
- The ~:shares~, ~:lp~, and ~:qp~ columns are summed, and the ~:any?~ aggregate is
1488
+ The ~:shares~, ~:lp~, and ~:qp~ columns are summed, and the ~:all?~ aggregate is
1044
1489
  applied to one of the boolean fields, that is, it is ~true~ if any of the values
1045
1490
  in that column are ~true~.
1046
1491
 
@@ -1053,15 +1498,15 @@ will raise an exception.
1053
1498
 
1054
1499
  - ~first~ :: the first non-nil item in the column,
1055
1500
  - ~last~ :: the last non-nil item in the column,
1056
- - ~rng~ :: form a string of the form ~"#{first}..#{last}"~ to show the range of
1057
- values in the column,
1058
- - ~sum~ :: for ~Numeric~ and ~String~ columns, apply '+' to all the non-nil
1059
- values,
1501
+ - ~range~ :: form a Range ~~{min}..{max}~ to show the range of values in the
1502
+ column,
1503
+ - ~sum~ :: for ~Numeric~ columns, apply '+' to all the non-nil
1504
+ values; for ~String~ columns, join the elements with a single space,
1060
1505
  - ~count~ :: the number of non-nil values in the column,
1061
1506
  - ~min~ :: for ~Numeric~, ~String~, and ~DateTime~ columns, return the smallest
1062
- non-nil value in the column,
1507
+ non-nil, non-blank value in the column,
1063
1508
  - ~max~ :: for ~Numeric~, ~String~, and ~DateTime~ columns, return the largest
1064
- non-nil value in the column,
1509
+ non-nil, non-blank value in the column,
1065
1510
  - ~avg~ :: for ~Numeric~ and ~DateTime~ columns, return the arithmetic mean of
1066
1511
  the non-nil values in the column; with respect to ~Date~ or ~DateTime~
1067
1512
  objects, each is converted to a numeric Julian date, the average is
@@ -1093,7 +1538,6 @@ implicit ~order_by~ on the grouping columns is collapsed into a single row.
1093
1538
 
1094
1539
  *** Join
1095
1540
  **** Join Types
1096
-
1097
1541
  So far, all the operations have operated on a single table. ~FatTable~ provides
1098
1542
  several ~join~ methods for combining two tables, each of which takes as
1099
1543
  parameters (1) a second table and (2) except in the case of ~cross_join~, zero
@@ -1130,7 +1574,6 @@ for inclusion in the joined output table.
1130
1574
  M~ rows.
1131
1575
 
1132
1576
  **** Join Expressions
1133
-
1134
1577
  For each of the join types, if no join expressions are given, the tables will be
1135
1578
  joined on columns having the same column header in both tables, and the join
1136
1579
  condition is satisfied when all the values in those columns are equal. If the
@@ -1158,48 +1601,74 @@ local variables within the expression, but the instance variables ~@row~ and
1158
1601
  ~@group~ are not.
1159
1602
 
1160
1603
  **** Join Examples
1161
-
1162
1604
  The following examples are taken from the [[https://www.tutorialspoint.com/postgresql/postgresql_using_joins.htm][Postgresql tutorial]], with some slight
1163
1605
  modifications. The examples will use the following two tables, which are also
1164
1606
  available in ~ft_console~ as ~@tab_a~ and ~@tab_b~:
1165
1607
 
1166
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1167
- #+BEGIN_SRC ruby
1168
- require 'fat_table'
1169
-
1170
- tab_a_str = <<-EOS
1171
- | Id | Name | Age | Address | Salary | Join Date |
1172
- |----+-------+-----+------------+--------+------------|
1173
- | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
1174
- | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
1175
- | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
1176
- | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
1177
- | 2 | Allen | 25 | Texas | | 2005-07-13 |
1178
- | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
1179
- | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
1180
- | 10 | James | 45 | Texas | 5000 | |
1181
- EOS
1608
+ #+BEGIN_SRC ruby :wrap EXAMPLE :results silent
1609
+ tab_a_str = <<-EOS
1610
+ | Id | Name | Age | Address | Salary | Join Date |
1611
+ |----+-------+-----+------------+--------+------------|
1612
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
1613
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
1614
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
1615
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
1616
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
1617
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
1618
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
1619
+ | 10 | James | 45 | Texas | 5000 | |
1620
+ EOS
1182
1621
 
1183
- tab_b_str = <<-EOS
1184
- | Id | Dept | Emp Id |
1185
- |----+-------------+--------|
1186
- | 1 | IT Billing | 1 |
1187
- | 2 | Engineering | 2 |
1188
- | 3 | Finance | 7 |
1189
- EOS
1622
+ tab_b_str = <<-EOS
1623
+ | Id | Dept | Emp Id |
1624
+ |----+-------------+--------|
1625
+ | 1 | IT Billing | 1 |
1626
+ | 2 | Engineering | 2 |
1627
+ | 3 | Finance | 7 |
1628
+ EOS
1190
1629
 
1191
- tab_a = FatTable.from_org_string(tab_a_str)
1192
- tab_b = FatTable.from_org_string(tab_b_str)
1630
+ tab_b = FatTable.from_org_string(tab_b_str)
1193
1631
  #+END_SRC
1194
1632
 
1195
- ***** Inner Joins
1633
+ Here is ~tab_a~:
1634
+ #+begin_src ruby :wrap EXAMPLE
1635
+ tab_a = FatTable.from_org_string(tab_a_str)
1636
+ tab_a.to_aoa
1637
+ #+end_src
1638
+
1639
+ #+begin_EXAMPLE
1640
+ | Id | Name | Age | Address | Salary | Join Date |
1641
+ |----+-------+-----+------------+--------+------------|
1642
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
1643
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
1644
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
1645
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
1646
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
1647
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
1648
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
1649
+ | 10 | James | 45 | Texas | 5000 | |
1650
+ #+end_EXAMPLE
1651
+
1652
+ And ~tab_b~:
1653
+ #+begin_src ruby :wrap EXAMPLE
1654
+ tab_b = FatTable.from_org_string(tab_b_str)
1655
+ tab_b.to_aoa
1656
+ #+end_src
1657
+
1658
+ #+begin_EXAMPLE
1659
+ | Id | Dept | Emp Id |
1660
+ |----+-------------+--------|
1661
+ | 1 | IT Billing | 1 |
1662
+ | 2 | Engineering | 2 |
1663
+ | 3 | Finance | 7 |
1664
+ #+end_EXAMPLE
1196
1665
 
1666
+ ***** Inner Joins
1197
1667
  With no join expression arguments, the tables are joined when their sole common
1198
1668
  field, ~:id~, is equal in both tables. The result is the natural join of the
1199
1669
  two tables.
1200
1670
 
1201
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1202
- #+BEGIN_SRC ruby
1671
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1203
1672
  tab_a.join(tab_b).to_aoa
1204
1673
  #+END_SRC
1205
1674
 
@@ -1216,8 +1685,7 @@ in the second table. To correct this, we need to explicitly state the columns we
1216
1685
  want to join on in each table by disambiguating them with ~_a~ and ~_b~
1217
1686
  suffixes:
1218
1687
 
1219
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1220
- #+BEGIN_SRC ruby
1688
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1221
1689
  tab_a.join(tab_b, :id_a, :emp_id_b).to_aoa
1222
1690
  #+END_SRC
1223
1691
 
@@ -1232,8 +1700,7 @@ Instead of using the disambiguated column names as symbols, we could also use a
1232
1700
  string containing a ruby expression. Within the expression, the column names
1233
1701
  should be treated as local variables:
1234
1702
 
1235
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1236
- #+BEGIN_SRC ruby
1703
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1237
1704
  tab_a.join(tab_b, 'id_a == emp_id_b').to_aoa
1238
1705
  #+END_SRC
1239
1706
 
@@ -1245,12 +1712,10 @@ should be treated as local variables:
1245
1712
  #+END_EXAMPLE
1246
1713
 
1247
1714
  ***** Left and Right Joins
1248
-
1249
1715
  In left join, all the rows of ~tab_a~ are included in the output, augmented by
1250
1716
  the matching columns of ~tab_b~ and augmented with nils where there is no match:
1251
1717
 
1252
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1253
- #+BEGIN_SRC ruby
1718
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1254
1719
  tab_a.left_join(tab_b, 'id_a == emp_id_b').to_aoa
1255
1720
  #+END_SRC
1256
1721
 
@@ -1271,8 +1736,7 @@ In a right join, all the rows of ~tab_b~ are included in the output, augmented
1271
1736
  by the matching columns of ~tab_a~ and augmented with nils where there is no
1272
1737
  match:
1273
1738
 
1274
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1275
- #+BEGIN_SRC ruby
1739
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1276
1740
  tab_a.right_join(tab_b, 'id_a == emp_id_b').to_aoa
1277
1741
  #+END_SRC
1278
1742
 
@@ -1285,13 +1749,11 @@ match:
1285
1749
  #+END_EXAMPLE
1286
1750
 
1287
1751
  ***** Full Join
1288
-
1289
1752
  A full join combines the effects of a left join and a right join. All the rows
1290
1753
  from both tables are included in the output augmented by columns of the other
1291
1754
  table where the join expression is satisfied and augmented with nils otherwise.
1292
1755
 
1293
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1294
- #+BEGIN_SRC ruby
1756
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1295
1757
  tab_a.full_join(tab_b, 'id_a == emp_id_b').to_aoa
1296
1758
  #+END_SRC
1297
1759
 
@@ -1310,14 +1772,12 @@ table where the join expression is satisfied and augmented with nils otherwise.
1310
1772
  #+END_EXAMPLE
1311
1773
 
1312
1774
  ***** Cross Join
1313
-
1314
1775
  Finally, a cross join outputs every row of ~tab_a~ augmented with every row of
1315
1776
  ~tab_b~, in other words, the Cartesian product of the two tables. If ~tab_a~ has
1316
1777
  ~N~ rows and ~tab_b~ has ~M~ rows, the output table will have ~N * M~ rows.
1317
1778
  So be careful lest you consume all your computer's memory.
1318
1779
 
1319
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1320
- #+BEGIN_SRC ruby
1780
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1321
1781
  tab_a.cross_join(tab_b).to_aoa
1322
1782
  #+END_SRC
1323
1783
 
@@ -1351,7 +1811,6 @@ So be careful lest you consume all your computer's memory.
1351
1811
  #+END_EXAMPLE
1352
1812
 
1353
1813
  *** Set Operations
1354
-
1355
1814
  ~FatTable~ can perform several set operations on pairs of tables. In order for
1356
1815
  two tables to be used this way, they must have the same number of columns with
1357
1816
  the same types or an exception will be raised. We'll call two tables that
@@ -1361,8 +1820,7 @@ We'll use the following two set-compatible tables in the examples. They each
1361
1820
  have some duplicates and some group boundaries so you can see the effect of the
1362
1821
  set operations on duplicates and groups.
1363
1822
 
1364
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1365
- #+BEGIN_SRC ruby
1823
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1366
1824
  tab1.to_aoa
1367
1825
  #+END_SRC
1368
1826
 
@@ -1393,8 +1851,7 @@ set operations on duplicates and groups.
1393
1851
  | T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1394
1852
  #+END_EXAMPLE
1395
1853
 
1396
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1397
- #+BEGIN_SRC ruby
1854
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1398
1855
  tab2.to_aoa
1399
1856
  #+END_SRC
1400
1857
 
@@ -1422,7 +1879,6 @@ set operations on duplicates and groups.
1422
1879
  #+END_EXAMPLE
1423
1880
 
1424
1881
  **** Unions
1425
-
1426
1882
  Two tables that are set-compatible can be combined with the ~union~ or
1427
1883
  ~union_all~ methods so that the rows of both tables appear in the output. In the
1428
1884
  output table, the headers of the receiver table are used. You can use ~select~
@@ -1433,8 +1889,7 @@ Any group boundaries in the input tables are destroyed by ~union~ but are
1433
1889
  preserved by ~union_all~. In addition, ~union_all~ (but not ~union~) adds a
1434
1890
  group boundary between the rows of the two input tables.
1435
1891
 
1436
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1437
- #+BEGIN_SRC ruby
1892
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1438
1893
  tab1.union(tab2).to_aoa
1439
1894
  #+END_SRC
1440
1895
 
@@ -1464,8 +1919,7 @@ group boundary between the rows of the two input tables.
1464
1919
  | T021 | 2017-01-23 | P | 7.16 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1465
1920
  #+END_EXAMPLE
1466
1921
 
1467
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1468
- #+BEGIN_SRC ruby
1922
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1469
1923
  tab1.union_all(tab2).to_aoa
1470
1924
  #+END_SRC
1471
1925
 
@@ -1516,12 +1970,10 @@ group boundary between the rows of the two input tables.
1516
1970
  #+END_EXAMPLE
1517
1971
 
1518
1972
  **** Intersections
1519
-
1520
1973
  The ~intersect~ method returns a table having only rows common to both tables,
1521
1974
  eliminating any duplicate rows in the result.
1522
1975
 
1523
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1524
- #+BEGIN_SRC ruby
1976
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1525
1977
  tab1.intersect(tab2).to_aoa
1526
1978
  #+END_SRC
1527
1979
 
@@ -1540,8 +1992,7 @@ With ~intersect_all~, all the rows of the first table, including duplicates, are
1540
1992
  included in the result if they also occur in the second table. However,
1541
1993
  duplicates in the second table do not appear.
1542
1994
 
1543
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1544
- #+BEGIN_SRC ruby
1995
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1545
1996
  tab1.intersect_all(tab2).to_aoa
1546
1997
  #+END_SRC
1547
1998
 
@@ -1562,8 +2013,7 @@ As a result, it makes a difference which table is the receiver of the
1562
2013
  ~intersect_all~ method call and which is the argument. In other words, order of
1563
2014
  operation matters.
1564
2015
 
1565
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1566
- #+BEGIN_SRC ruby
2016
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1567
2017
  tab2.intersect_all(tab1).to_aoa
1568
2018
  #+END_SRC
1569
2019
 
@@ -1580,13 +2030,11 @@ operation matters.
1580
2030
  | T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1581
2031
  #+END_EXAMPLE
1582
2032
 
1583
- **** Differences with Except
1584
-
2033
+ **** Set Differences with Except
1585
2034
  You can use the ~except~ method to delete from a table any rows that occur in
1586
2035
  another table, that is, compute the set difference between the tables.
1587
2036
 
1588
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1589
- #+BEGIN_SRC ruby
2037
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1590
2038
  tab1.except(tab2).to_aoa
1591
2039
  #+END_SRC
1592
2040
 
@@ -1608,8 +2056,7 @@ another table, that is, compute the set difference between the tables.
1608
2056
  Like subtraction, though, the order of operands matters with set difference
1609
2057
  computed by ~except~.
1610
2058
 
1611
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1612
- #+BEGIN_SRC ruby
2059
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1613
2060
  tab2.except(tab1).to_aoa
1614
2061
  #+END_SRC
1615
2062
 
@@ -1626,8 +2073,7 @@ computed by ~except~.
1626
2073
  As with ~intersect_all~, ~except_all~ includes any duplicates in the first,
1627
2074
  receiver table, but not those in the second, argument table.
1628
2075
 
1629
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1630
- #+BEGIN_SRC ruby
2076
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1631
2077
  tab1.except_all(tab2).to_aoa
1632
2078
  #+END_SRC
1633
2079
 
@@ -1649,8 +2095,7 @@ receiver table, but not those in the second, argument table.
1649
2095
 
1650
2096
  And, of course, the order of operands matters here as well.
1651
2097
 
1652
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1653
- #+BEGIN_SRC ruby
2098
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1654
2099
  tab2.except_all(tab1).to_aoa
1655
2100
  #+END_SRC
1656
2101
 
@@ -1667,13 +2112,11 @@ And, of course, the order of operands matters here as well.
1667
2112
  #+END_EXAMPLE
1668
2113
 
1669
2114
  *** Uniq (aka Distinct)
1670
-
1671
2115
  The ~uniq~ method takes no arguments and simply removes any duplicate rows from
1672
2116
  the input table. The ~distinct~ method is an alias for ~uniq~. Any groups in
1673
2117
  the input table are lost.
1674
2118
 
1675
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1676
- #+BEGIN_SRC ruby
2119
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1677
2120
  tab1.uniq.to_aoa
1678
2121
  #+END_SRC
1679
2122
 
@@ -1699,13 +2142,11 @@ the input table are lost.
1699
2142
  #+END_EXAMPLE
1700
2143
 
1701
2144
  *** Remove groups with degroup!
1702
-
1703
2145
  Finally, it is sometimes helpful to remove any group boundaries from a table.
1704
- You can do this with ~.degroup!~, which is the only operation that mutates its
1705
- receiver table by removing its groups.
2146
+ You can do this with ~.degroup!~, which, together with ~force_string!~, are
2147
+ the only operations that mutate their receiver tables.
1706
2148
 
1707
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1708
- #+BEGIN_SRC ruby
2149
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1709
2150
  tab1.degroup!.to_aoa
1710
2151
  #+END_SRC
1711
2152
 
@@ -1734,7 +2175,6 @@ receiver table by removing its groups.
1734
2175
  #+END_EXAMPLE
1735
2176
 
1736
2177
  ** Formatting Tables
1737
-
1738
2178
  Besides creating and operating on tables, you may want to display the resulting
1739
2179
  table. ~FatTable~ seeks to provide a set of formatting directives that are the
1740
2180
  most common across many output media. It provides directives for alignment, for
@@ -1751,8 +2191,8 @@ an ~org-mode~ table as a String, and since ~org-mode~ does not support colors,
1751
2191
  any color directives are ignored. Some of the output targets are not strings,
1752
2192
  but ruby data structures, and for them, things such as alignment are irrelevant.
1753
2193
 
1754
- *** Available Formatters
1755
-
2194
+ *** Available Formatter Output Targets
2195
+ **** Output Media
1756
2196
  ~FatTable~ supports the following output targets for its tables:
1757
2197
 
1758
2198
  - Text :: form the table with ACSII characters,
@@ -1771,62 +2211,172 @@ class by defining about a dozen methods that get called at various places
1771
2211
  during the construction of the output table. The idea is that more output
1772
2212
  formats can be defined by adding additional classes.
1773
2213
 
1774
- *** Table Locations
2214
+ **** Examples
2215
+ ***** To Text
2216
+ This formatter uses nothing by ASCII characters to draw the table. Notice
2217
+ that, unlike to ~to_org~ formatter shown below, the intersections of lines are
2218
+ represented by a ~+~ character. Embelishments such as color, bold, and so
2219
+ forth are ignored.
1775
2220
 
1776
- In the formatting methods, the table is divided into several "locations" for
1777
- which separate formatting directives may be given. These locations are
1778
- identified with the following symbols:
2221
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2222
+ tab_a.to_text
2223
+ #+end_SRC
1779
2224
 
1780
- - :header :: the first row of the output table containing the headers,
1781
- - :footer :: all rows of the table's footers,
1782
- - :gfooter :: all rows of the table's group footers,
1783
- - :body :: all the data rows of the table, that is, those that are neither part
1784
- of the header, footers, or gfooters,
1785
- - :bfirst :: the first row of the table's body, and
1786
- - :gfirst :: the first row in each group in the table's body.
2225
+ #+BEGIN_EXAMPLE
2226
+ +====+=======+=====+============+========+============+
2227
+ | Id | Name | Age | Address | Salary | Join Date |
2228
+ +----+-------+-----+------------+--------+------------+
2229
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2230
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2231
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2232
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2233
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2234
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2235
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2236
+ | 10 | James | 45 | Texas | 5000 | |
2237
+ +====+=======+=====+============+========+============+
2238
+ #+END_EXAMPLE
1787
2239
 
1788
- *** Formatting Directives
2240
+ ***** To Org
2241
+ This formatter is designed to format tables in a manner consistent with the
2242
+ way tables are drawn within Emacs Org Mode. It also uses nothing by ASCII
2243
+ characters to draw the table, but, the intersections of lines are represented
2244
+ by a ~|~ character. Embelishments such as color, bold, and so forth are
2245
+ ignored. When working in Org Mode, note that Emacs will convert an Array of
2246
+ Arrays into an Org Mode table, so when constructing tables programmatically,
2247
+ it may be better to use that formatter as shown below.
1789
2248
 
1790
- The formatting methods explained in the next section all take formatting
1791
- directives as strings in which letters and other characters signify what
1792
- formatting applies. For example, we may apply the formatting directive ~'R,$'~
1793
- to numbers in a certain part of the table. Each of those characters, and in
1794
- some cases a whole substring, is a single directive. They can appear in any
1795
- order, so ~'$R,'~ and ~',$R'~ are equivalent.
2249
+ #+begin_SRC ruby :wrap EXAMPLE
2250
+ tab_a.to_org
2251
+ #+end_SRC
1796
2252
 
1797
- Here is a list of all the formatting directives that apply to each cell type:
2253
+ #+begin_EXAMPLE
2254
+ |----+-------+-----+------------+--------+--------------|
2255
+ | Id | Name | Age | Address | Salary | Join Date |
2256
+ |----+-------+-----+------------+--------+--------------|
2257
+ | 1 | Paul | 32 | California | 20000 | [2001-07-13] |
2258
+ | 3 | Teddy | 23 | Norway | 20000 | [2007-12-13] |
2259
+ | 4 | Mark | 25 | Rich-Mond | 65000 | [2007-12-13] |
2260
+ | 5 | David | 27 | Texas | 85000 | [2007-12-13] |
2261
+ | 2 | Allen | 25 | Texas | | [2005-07-13] |
2262
+ | 8 | Paul | 24 | Houston | 20000 | [2005-07-13] |
2263
+ | 9 | James | 44 | Norway | 5000 | [2005-07-13] |
2264
+ | 10 | James | 45 | Texas | 5000 | |
2265
+ |----+-------+-----+------------+--------+--------------|
2266
+ #+end_EXAMPLE
1798
2267
 
1799
- **** String
2268
+ ***** To Term
2269
+ When outputting to a terminal or other device that can interpret ANSI
2270
+ characters and escape codes, you can use this formatter to get a prettier
2271
+ table. It also allows embelishments such as color and text styles to the
2272
+ extent the device supports it.
1800
2273
 
1801
- For a string element, the following instructions are valid. Note that these can
1802
- also be applied to all the other cell types as well since they are all converted
1803
- to a string in forming the output.
2274
+ #+begin_SRC ruby :wrap EXAMPLE
2275
+ tab_a.to_term
2276
+ #+end_SRC
1804
2277
 
1805
- - u :: convert the element to all lowercase,
1806
- - U :: convert the element to all uppercase,
1807
- - t :: title case the element, that is, upcase the initial letter in
1808
- each word and lower case the other letters
1809
- - B ~B :: make the element bold, or turn off bold
2278
+ #+begin_EXAMPLE
2279
+ ╒════╤═══════╤═════╤════════════╤════════╤════════════╕
2280
+ Id Name │ Age Address │ Salary Join Date │
2281
+ ├────┼───────┼─────┼────────────┼────────┼────────────┤
2282
+ 1 │ Paul │ 32 │ California 20000 │ 2001-07-13
2283
+ │ 3 │ Teddy │ 23 │ Norway │ 20000 │ 2007-12-13 │
2284
+ │ 4 │ Mark │ 25 │ Rich-Mond │ 65000 │ 2007-12-13 │
2285
+ │ 5 │ David │ 27 │ Texas │ 85000 │ 2007-12-13 │
2286
+ │ 2 │ Allen │ 25 │ Texas │ │ 2005-07-13 │
2287
+ │ 8 │ Paul │ 24 │ Houston │ 20000 │ 2005-07-13 │
2288
+ │ 9 │ James │ 44 │ Norway │ 5000 │ 2005-07-13 │
2289
+ │ 10 │ James │ 45 │ Texas │ 5000 │ │
2290
+ ╘════╧═══════╧═════╧════════════╧════════╧════════════╛
2291
+ #+end_EXAMPLE
2292
+
2293
+ ***** To LaTeX
2294
+ This formatter outputs a table in the form suitable for inclusion in a LaTeX
2295
+ document using the ~logtable~ package. Natualy it allows embelishments such
2296
+ as color and text styles to the full extent of LaTeX's formatting prowess.
2297
+
2298
+ #+begin_SRC ruby :wrap EXAMPLE
2299
+ tab_b.to_latex
2300
+ #+end_SRC
2301
+
2302
+ #+begin_EXAMPLE
2303
+ \begin{longtable}{lll}
2304
+ Id&
2305
+ Dept&
2306
+ Emp Id\\
2307
+ \endhead
2308
+ 1&
2309
+ IT Billing&
2310
+ 1\\
2311
+ 2&
2312
+ Engineering&
2313
+ 2\\
2314
+ 3&
2315
+ Finance&
2316
+ 7\\
2317
+ \end{longtable}
2318
+ #+end_EXAMPLE
2319
+
2320
+ ***** To AoA (Array of Arrays)
2321
+ #+begin_SRC ruby :wrap EXAMPLE
2322
+ tab_b.to_aoa
2323
+ #+end_SRC
2324
+
2325
+ #+begin_EXAMPLE
2326
+ [["Id", "Dept", "Emp Id"], nil, ["1", "IT Billing", "1"], ["2", "Engineering", "2"],
2327
+ ["3", "Finance", "7"]]
2328
+ #+end_EXAMPLE
2329
+
2330
+ ***** To AoH (Array of Hashes)
2331
+ #+begin_SRC ruby :wrap EXAMPLE
2332
+ tab_b.to_aoh
2333
+ #+end_SRC
2334
+
2335
+ #+begin_EXAMPLE
2336
+ [{:id=>"1", :dept=>"IT Billing", :emp_id=>"1"}, {:id=>"2", :dept=>"Engineering", :emp_id=>"2"},
2337
+ {:id=>"3", :dept=>"Finance", :emp_id=>"7"}]
2338
+ #+end_EXAMPLE
2339
+
2340
+
2341
+ *** Formatting Directives
2342
+ The formatting methods explained in the next section all take formatting
2343
+ directives as strings in which letters and other characters signify what
2344
+ formatting applies. For example, we may apply the formatting directive 'R,$'
2345
+ to numbers in a certain part of the table. Each of those characters, and in
2346
+ some cases a whole substring, is a single directive. They can appear in any
2347
+ order, so '$R,' and ',$R' are equivalent.
2348
+
2349
+ Here is a list of all the formatting directives that apply to each cell type:
2350
+
2351
+ **** String
2352
+ For a string element, the following instructions are valid. Note that these can
2353
+ also be applied to all the other cell types as well since they are all converted
2354
+ to a string in forming the output.
2355
+
2356
+ - u :: convert the element to all lowercase,
2357
+ - U :: convert the element to all uppercase,
2358
+ - t :: title case the element, that is, upcase the initial letter in
2359
+ each word and lower case the other letters
2360
+ - B ~B :: make the element bold, or turn off bold
1810
2361
  - I ~I :: make the element italic, or turn off italic
1811
2362
  - R :: align the element on the right of the column
1812
2363
  - L :: align the element on the left of the column
1813
2364
  - C :: align the element in the center of the column
1814
- - c[color] :: render the element in the given color; the color can have
1815
- the form fgcolor, fgcolor.bgcolor, or .bgcolor, to set the
1816
- foreground or background colors respectively, and each of those can
1817
- be an ANSI or X11 color name in addition to the special color,
1818
- 'none', which keeps the terminal's default color.
2365
+ - c[<color_spec>] :: render the element in the given color; the <color_spec>
2366
+ can have the form fgcolor, fgcolor.bgcolor, or .bgcolor, to set the
2367
+ foreground or background colors respectively, and each of those can be an
2368
+ ANSI or X11 color name in addition to the special color, 'none', which keeps
2369
+ the output's default color.
1819
2370
  - _ ~_ :: underline the element, or turn off underline
1820
2371
  - * ~* :: cause the element to blink, or turn off blink
1821
2372
 
1822
- For example, the directive ~'tCc[red.yellow]'~ would title-case the element,
2373
+ For example, the directive 'tCc[red.yellow]' would title-case the element,
1823
2374
  center it, and color it red on a yellow background. The directives that are
1824
2375
  boolean have negating forms so that, for example, if bold is turned on for all
1825
2376
  columns of a given type, it can be countermanded in formatting directives for
1826
2377
  particular columns.
1827
2378
 
1828
2379
  **** Numeric
1829
-
1830
2380
  For a numeric element, all the instructions valid for string are available, in
1831
2381
  addition to the following:
1832
2382
 
@@ -1841,11 +2391,10 @@ addition to the following:
1841
2391
  will result in a :numeric expressed as seconds and can be displayed in
1842
2392
  hours, minutes, and seconds with this formatting instruction.
1843
2393
 
1844
- For example, the directive ~'R5.0c[blue]'~ would right-align the numeric
2394
+ For example, the directive 'R5.0c[blue]' would right-align the numeric
1845
2395
  element, pad it on the left with zeros, and color it blue.
1846
2396
 
1847
2397
  **** DateTime
1848
-
1849
2398
  For a ~DateTime~, all the instructions valid for string are available, in
1850
2399
  addition to the following:
1851
2400
 
@@ -1857,61 +2406,216 @@ addition to the following:
1857
2406
  component where fmt is a valid format string for Date#strftime, otherwise,
1858
2407
  the datetime will be formatted as an ISO 8601 string, YYYY-MM-DD.
1859
2408
 
1860
- For example, ~'c[pink]d[%b %-d, %Y]C'~, would format a date element like 'Sep
2409
+ For example, 'c[pink]d[%b %-d, %Y]C', would format a date element like 'Sep
1861
2410
  22, 1957', center it, and color it pink.
1862
2411
 
1863
2412
  **** Boolean
1864
-
1865
2413
  For a boolean cell, all the instructions valid for string are available, in
1866
2414
  addition to the following:
1867
2415
 
1868
- - Y :: print true as '~Y~' and false as '~N~',
1869
- - T :: print true as '~T~' and false as '~F~',
1870
- - X :: print true as '~X~' and false as an empty string '',
2416
+ - Y :: print true as ~Y~ and false as ~N~,
2417
+ - T :: print true as ~T~ and false as ~F~,
2418
+ - X :: print true as ~X~ and false as an empty string '',
1871
2419
  - b[xxx,yyy] :: print true as the string given as ~xxx~ and false as the string
1872
2420
  given as ~yyy~,
1873
2421
  - c[tcolor,fcolor] :: color a true element with ~tcolor~ and a false element
1874
2422
  with ~fcolor~. Each of the colors may be specified in the same manner as
1875
2423
  colors for strings described above.
1876
2424
 
1877
- For example, the directive '~b[Yeppers,Nope]c[green.pink,red.pink]~' would
1878
- render a true boolean as '~Yeppers~' colored green on pink and render a false
1879
- boolean as '~Nope~' colored red on pink. See [[https://www.youtube.com/watch?v=oLdFFD8II8U][Yeppers]] for additional information.
2425
+ For example, the directive 'b[Yeppers,Nope]c[green.pink,red.pink]' would
2426
+ render a true boolean as ~Yeppers~ colored green on pink and render a false
2427
+ boolean as ~Nope~ colored red on pink. See [[https://www.youtube.com/watch?v=oLdFFD8II8U][Yeppers]] for additional information.
1880
2428
 
1881
2429
  **** NilClass
1882
-
1883
2430
  By default, ~nil~ elements are rendered as blank cells, but you can make them
1884
2431
  visible with the following, and in that case, all the formatting instructions
1885
2432
  valid for strings are also available:
1886
2433
 
1887
2434
  - n[niltext] :: render a ~nil~ item with the given niltext.
1888
2435
 
1889
- For example, you might want to use ~'n[-]Cc[purple]'~ to make nils visible as a
2436
+ For example, you might want to use 'n[-]Cc[purple]' to make nils visible as a
1890
2437
  centered purple hyphen.
1891
2438
 
1892
- *** Footers Methods
2439
+ *** The ~format~ and ~format_for~ methods
2440
+ Formatters take only two kinds of methods, those that attach footers to a
2441
+ table, which are discussed in the next section, and those that specify
2442
+ formatting for table cells, which are the subject of this section.
2443
+
2444
+ To set formatting directives for all locations in a table at once, use the
2445
+ ~format~ method; to set formatting directives for a particular location in the
2446
+ table, use the ~format_for~ method, giving the location as the first
2447
+ parameter. See below at [[*Table Locations][Table Locations]] for an explanation of all the
2448
+ locations available.
2449
+
2450
+ Other than that first parameter, the two methods take the same types of
2451
+ parameters. The remaining parameters are hash-like parameters that use either a
2452
+ column name or a type as the key and a string with the formatting directives to
2453
+ apply as the value. The following example says to set the formatting for all
2454
+ locations in the table and to format all numeric fields as strings that are
2455
+ rounded to whole numbers (the '0.0' part), that are right-aligned (the 'R'
2456
+ part), and have grouping commas inserted (the ',' part). But the ~:id~ column is
2457
+ numeric, and the second parameter overrides the formatting for numerics in
2458
+ general and calls for the ~:id~ column to be padded to three digits with zeros
2459
+ on the left (the '3.0' part) and to be centered (the 'C' part).
2460
+
2461
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2462
+ tab_a.to_text do |f|
2463
+ f.format(numeric: '0.0,R', id: '3.0C')
2464
+ f.format_for(:body, string: 'R')
2465
+ f.format_for(:header, string: 'C')
2466
+ end
2467
+ #+END_SRC
2468
+
2469
+ #+begin_EXAMPLE
2470
+ +=====+=======+=====+============+========+============+
2471
+ | Id | Name | Age | Address | Salary | Join Date |
2472
+ +-----+-------+-----+------------+--------+------------+
2473
+ | 001 | Paul | 32 | California | 20,000 | 2001-07-13 |
2474
+ | 003 | Teddy | 23 | Norway | 20,000 | 2007-12-13 |
2475
+ | 004 | Mark | 25 | Rich-Mond | 65,000 | 2007-12-13 |
2476
+ | 005 | David | 27 | Texas | 85,000 | 2007-12-13 |
2477
+ | 002 | Allen | 25 | Texas | | 2005-07-13 |
2478
+ | 008 | Paul | 24 | Houston | 20,000 | 2005-07-13 |
2479
+ | 009 | James | 44 | Norway | 5,000 | 2005-07-13 |
2480
+ | 010 | James | 45 | Texas | 5,000 | |
2481
+ +=====+=======+=====+============+========+============+
2482
+ #+end_EXAMPLE
1893
2483
 
1894
- You can call the ~footer~ and ~gfooter~ methods on ~Formatter~ objects to add
1895
- footers and group footers. Their signatures are:
2484
+ In the example, the ~format~ method affects the whole table. Its ~numeric:~
2485
+ directive affected the ~:age~ and ~:salary~ columns because their types are
2486
+ Numeric. The ~id:~ column is also Numeric, but it's more specific directive
2487
+ takes precedence and it is formatted accordingly.
2488
+
2489
+ But the ~format_for~ methods affected two "locations": the "body" and the
2490
+ "header". Within the body, the ~:string~ directive calls for all strings to
2491
+ be right-aligned, but the headers are unaffected by it. The ~format_for~ the
2492
+ ~:header~ location caused all the headers to be centered.
2493
+
2494
+ All the other cells in the table, namely the cells in the ~:join_date~ column,
2495
+ had the default formatting applied.
2496
+
2497
+ **** Table Locations
2498
+ In the ~format_for~ formatting method, the first argument names a "location."
2499
+ The table is divided into several locations for which separate formatting
2500
+ directives may be given. These locations are identified by the following
2501
+ symbols:
2502
+
2503
+ - :header :: the first row of the output table containing the headers,
2504
+ - :footer :: all rows of the table's footers,
2505
+ - :gfooter :: all rows of the table's group footers,
2506
+ - :body :: all the data rows of the table, that is, those that are neither part
2507
+ of the header, footers, or gfooters,
2508
+ - :bfirst :: the first row of the table's body, and
2509
+ - :gfirst :: the first row in each group in the table's body.
2510
+
2511
+ **** Location priority
2512
+ Formatting for any given cell depends on its location in the table. The
2513
+ ~format_for~ method takes a location to which its formatting directive are
2514
+ restricted as the first argument. It can be one of the following:
2515
+
2516
+ - ~:header~ :: The directives apply only to the header row, that is the first
2517
+ row, of the output table; before the directives are applied, the header's
2518
+ symbol form is converted back into a string and capitalized as is a book
2519
+ title. Thus, only directives applicable to the String type have any effect.
2520
+
2521
+ - ~:body~ :: The directives apply to all rows in the body of the table.
2522
+
2523
+ - ~:gfirst~ :: directives apply to the first row in each group in the body of
2524
+ the table, unless the row is also the first row in the table as a whole, in
2525
+ which case the ~:bfirst~ directives apply,
2526
+
2527
+ - ~:bfirst~ :: The directives apply to the first row in the body of the table,
2528
+ taking precedence over those directives that apply to the body generally or
2529
+ the ~:gfirst~ directives that apply to the first row in each group.
2530
+
2531
+ - ~:footer~ :: The directives apply to all the footer rows of the output
2532
+ table, regardless of how many there are.
2533
+
2534
+ - ~gfooter~ :: The directives apply to all group footer rows of the output
2535
+ tables, regardless of how many there are.
2536
+
2537
+ Directives given to the ~format~ method apply the directives to all locations in
2538
+ the table, but they can be overridden by more specific directives given in a
2539
+ ~format_for~ directive.
2540
+
2541
+ **** Type and Column priority
2542
+ A directive based on type applies to all columns having that type unless
2543
+ overridden by a directive specific to a named column; a directive based on a
2544
+ column name applies only to cells in that column.
2545
+
2546
+ However, there is a twist. Since the end result of formatting is to convert
2547
+ all columns to strings, the formatting directives for the ~String~ type can
2548
+ be applied to all column types. Likewise, since all columns may contain nils,
2549
+ the ~NilClass:~ type applies to nils in all columns regardless of the column's
2550
+ type.
2551
+
2552
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2553
+ tab_a.to_text do |f|
2554
+ f.format(string: 'R', id: '3.0C', nil: 'Cn[-]', salary: 'n[N/A]')
2555
+ end
2556
+ #+END_SRC
2557
+
2558
+ #+BEGIN_EXAMPLE
2559
+ +=====+=======+=====+============+========+============+
2560
+ | Id | Name | Age | Address | Salary | Join Date |
2561
+ +-----+-------+-----+------------+--------+------------+
2562
+ | 001 | Paul | 32 | California | 20000 | 2001-07-13 |
2563
+ | 003 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2564
+ | 004 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2565
+ | 005 | David | 27 | Texas | 85000 | 2007-12-13 |
2566
+ | 002 | Allen | 25 | Texas | N/A | 2005-07-13 |
2567
+ | 008 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2568
+ | 009 | James | 44 | Norway | 5000 | 2005-07-13 |
2569
+ | 010 | James | 45 | Texas | 5000 | |
2570
+ +=====+=======+=====+============+========+============+
2571
+ #+END_EXAMPLE
2572
+
2573
+ The ~string: 'R'~ directive causes all the cells to be right-aligned except
2574
+ ~:id~ which specifies centering for the ~:id~ column only. The ~n[N/A]~
2575
+ directive for specifies how nil are displayed in the numeric column, ~:salary~,
2576
+ but not for other nils, such as in the last row of the ~:join_date~ column.
2577
+
2578
+ *** Footers
2579
+ **** Adding Footers
2580
+ You can call the ~footer,~ ~gfooter, foot, and gfoot~ methods on ~Formatter~
2581
+ objects to add footers and group footers. Note that all of these methods
2582
+ return a ~Footer~ object that can be accessed to extract the computed values.
2583
+ All of these methods return the ~FatTable::Footer~ object so constructed. It
2584
+ can be used to access the values and other attributes of the footer
2585
+ computed. Their signatures are:
1896
2586
 
1897
2587
  - ~footer(label, *sum_cols, **agg_cols)~ :: where ~label~ is a label to be
1898
- placed in the first cell of the footer (unless that column is named as one
1899
- of the ~sum_cols~ or ~agg_cols~, in which case the label is ignored),
1900
- ~*sum_cols~ are zero or more symbols for columns to be summed, and
1901
- ~**agg_cols~ is zero or more hash-like parameters with a column symbol as a
1902
- key and a symbol for an aggregate method as the value. This causes a
1903
- table-wide header to be added at the bottom of the table applying the
1904
- ~:sum~ aggregate to the ~sum_cols~ and the named aggregate method to the
1905
- ~agg_cols~. A table can have any number of footers attached, and they will
1906
- appear at the bottom of the output table in the order they are given.
1907
-
1908
- - ~gfooter(label, *sum_cols, **agg_cols)~ :: where the parameters have the same
1909
- meaning as for the ~footer~ method, but result in a footer for each group
1910
- in the table rather than the table as a whole. These will appear in the
1911
- output table just below each group.
2588
+ placed in the first cell of the footer (unless that column is named as one
2589
+ of the ~sum_cols~ or ~agg_cols~, in which case the label is ignored),
2590
+ ~*sum_cols~ are zero or more symbols for columns to be summed, and
2591
+ ~**agg_cols~ is zero or more hash-like parameters with a column symbol as a
2592
+ key and a valid aggregate as the value. This causes a table-wide header to
2593
+ be added at the bottom of the table applying the ~:sum~ aggregate to the
2594
+ ~sum_cols~ and the named aggregate to the ~agg_cols~. A table can have any
2595
+ number of footers attached, and they will appear at the bottom of the output
2596
+ table in the order they are given.
2597
+
2598
+ - ~foot(label, label_col, **agg_cols)~ :: where ~label~ is a label to be
2599
+ placed in the column with header ~label_col~, or, if ommitted, in the first
2600
+ cell of the footer (unless that column is named as one of the ~agg_cols~, in
2601
+ which case the label is ignored), and ~**agg_cols~ is zero or more hash-like
2602
+ parameters with a column symbol as a key and a valid aggregate as the
2603
+ value. This causes a table-wide header to be added at the bottom of the
2604
+ table applying ~agg~, to the ~agg_cols~. A table can have any number of
2605
+ footers attached, and they will appear at the bottom of the output table in
2606
+ the order they are given.
2607
+
2608
+ - ~gfooter(label, *sum_cols, **agg_cols)~ :: where the parameters have the
2609
+ same meaning as for the ~footer~ method, but results in a footer for each
2610
+ group in the table rather than the table as a whole. These will appear in
2611
+ the output table just below each group.
2612
+
2613
+ - ~gfoot(label, label_col, **agg_cols)~ :: where the parameters have the same
2614
+ meaning as for the ~foot~ method, but results in a footer for each group in
2615
+ the table rather than the table as a whole. These will appear in the output
2616
+ table just below each group.
1912
2617
 
1913
2618
  There are also a number of convenience methods for adding common footers:
1914
-
1915
2619
  - ~sum_footer(*cols)~ :: Add a footer summing the given columns with the label
1916
2620
  'Total'.
1917
2621
  - ~sum_gfooter(*cols)~ :: Add a group footer summing the given columns with the
@@ -1929,22 +2633,324 @@ There are also a number of convenience methods for adding common footers:
1929
2633
  - ~max_gfooter(*cols)~ :: Add a group footer showing the maximum for the given
1930
2634
  columns with the label 'Group Maximum'.
1931
2635
 
1932
- *** Formatting Methods
2636
+ **** Aggregators
2637
+ When adding a footer with the above methods, you can specify an aggregator for
2638
+ each column named in the ~agg_cols~ parameter. There are several candidates
2639
+ for what you can use for an aggregator:
2640
+
2641
+ - Symbol :: one of the following built-in aggregators: :first, :last, :range,
2642
+ :sum, :count, :min, :max, :avg, :var, :pvar, :dev, :pdev, :any?, :all?,
2643
+ :none?, and :one?.
2644
+ + The symbols ending in a question mark are valid only for boolean columns;
2645
+ + :count, :first, and :last work with any column type,
2646
+ + :min, :max, and :range work with all types except boolean;
2647
+ + :sum, works only with numeric columns, and
2648
+ + :avg, :var, :dev, :pvar, and :pdev work with numeric or datetime columns.
2649
+ In the case of datetime columns, these aggrgators convert the dates to
2650
+ julian date numbers, perform the calculation, then convert the result back
2651
+ to a datetime object.
2652
+ - String :: using a string as an aggrgegator can result in:
2653
+ + the string being converted to an object matching the type of the column
2654
+ (for example, using '$1,888' in a numeric column puts the constant number
2655
+ 1888 in the footer field, using '1957-09-22' puts the fixed date in the
2656
+ field, etc.)
2657
+ + if the string cannot be parsed as a valid object matching the column's
2658
+ type, it is placed literally in the footer field (for example, using
2659
+ '(estimated)' can be used to add additional information to the footer)
2660
+ - Ruby object :: you can put a number in a numeric footer field, a DateTime
2661
+ object in a datetime footer field, or a true or false in a boolean footer
2662
+ field;
2663
+ - A Lambda :: finally, you can provide a lambda for performing arbitrary
2664
+ calculations and placing the result in the footer field. The number of
2665
+ arguments the lambda takes can vary:
2666
+ * If the lambda is used in a group footer, it must take a single integer
2667
+ argument that is set to the group number being calculated and /can/ take a
2668
+ second argument for the column symbol in which it appears, or
2669
+ * If the lambda is used in an ordinary footer, it either takes no arguments,
2670
+ or a single argument for the column symbol in which it appears.
2671
+
2672
+ **** Footer objects
2673
+ Each of the methods for adding a footer to a ~Formatter~ returns a ~Footer~ object
2674
+ that you can query for attributes of the generated footer, including accessing
2675
+ their computed values. Here are the accessors available on a
2676
+ ~FatTable::Footer~ object:
2677
+
2678
+ - ~[h]~ :: Return the value of under the ~h~ header, or if this is a group
2679
+ footer, return an array of the values for all the groups under the ~h~
2680
+ header.
2681
+ - .<header> :: like, ~[h]~ but makes the values available in method-call form.
2682
+ - ~number_of_groups~ :: Return the total number of groups in the table to
2683
+ which this footer belongs. Note that if the table has both group footers
2684
+ and normal footers, this will return the number of groups even for a normal
2685
+ footer.
2686
+ - ~column(h)~, ~column(h, k)~ :: Return a FatTable::Column object for the
2687
+ header h and, if the footer is a group footer, the kth group.
2688
+ - ~items(h)~, ~items(h, k)~ :: Return an Array of the values for the header
2689
+ ~h~ and, if a group, for the ~k~th group.
2690
+ - ~to_h~, ~to_h(k)~ :: Return a Hash with a key for each column header mapped
2691
+ to the footer value for that column, nil for unused columns. Use the index
2692
+ ~k~ to specify which group to access in the case of a group footer.
2693
+
2694
+
2695
+ **** Footer Examples
2696
+ As a reminder, here is the table, ~tab_a~ defined earlier:
2697
+
2698
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2699
+ tab_a.to_aoa
2700
+ #+END_SRC
2701
+
2702
+ #+begin_EXAMPLE
2703
+ | Id | Name | Age | Address | Salary | Join Date |
2704
+ |----+-------+-----+------------+--------+------------|
2705
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2706
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2707
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2708
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2709
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2710
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2711
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2712
+ | 10 | James | 45 | Texas | 5000 | |
2713
+ #+end_EXAMPLE
2714
+
2715
+ ***** Built-in Aggregators
2716
+ You can add a footer compute the average of the given columns. You may be
2717
+ surprised that you can average a set of dates, but ~:avg~ simply converts the
2718
+ dates to Julian numbers, averages that, then converts the result back to a
2719
+ date.
2720
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2721
+ tab_a.to_text do |f|
2722
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2723
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2724
+ f.footer('Tally', age: :count)
2725
+ end
2726
+ #+END_SRC
1933
2727
 
1934
- You can call methods on ~Formatter~ objects to specify formatting directives
1935
- for specific columns or types. There are two methods for doing so, ~format_for~
1936
- and ~format~.
2728
+ #+begin_EXAMPLE
2729
+ +=========+=======+=====+============+========+=============+
2730
+ | Id | Name | Age | Address | Salary | Join Date |
2731
+ +---------+-------+-----+------------+--------+-------------+
2732
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2733
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2734
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2735
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2736
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2737
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2738
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2739
+ | 10 | James | 45 | Texas | 5,000 | |
2740
+ +---------+-------+-----+------------+--------+-------------+
2741
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2742
+ +---------+-------+-----+------------+--------+-------------+
2743
+ | Tally | | 8 | | | |
2744
+ +=========+=======+=====+============+========+=============+
2745
+ #+end_EXAMPLE
2746
+
2747
+ ***** String Aggregators
2748
+ If the string is convertible into its columns's type, it will be converted to
2749
+ that type; otherwise, it will be placed in the footer literally. This example
2750
+ also shows how the values from one footer might be used in composing another
2751
+ footer.
2752
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2753
+ tab_a.to_text do |f|
2754
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2755
+ avg_ft = f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2756
+ f.footer('Tally', age: :count)
2757
+ if avg_ft[:salary] < 30000
2758
+ cmt = "We're saving"
2759
+ else
2760
+ cmt = "We're overspending"
2761
+ end
2762
+ f.footer('Pay', join_date: "We have #{avg_ft.number_of_groups} grp")
2763
+ f.footer('Group count', join_date: "We have #{avg_ft.number_of_groups} grp")
2764
+ f.footer('Comment', join_date: cmt)
2765
+ end
2766
+ #+END_SRC
1937
2767
 
1938
- **** Instantiating a Formatter
2768
+ #+begin_EXAMPLE
2769
+ +=============+=======+=====+============+========+====================+
2770
+ | Id | Name | Age | Address | Salary | Join Date |
2771
+ +-------------+-------+-----+------------+--------+--------------------+
2772
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2773
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2774
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2775
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2776
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2777
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2778
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2779
+ | 10 | James | 45 | Texas | 5,000 | |
2780
+ +-------------+-------+-----+------------+--------+--------------------+
2781
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2782
+ +-------------+-------+-----+------------+--------+--------------------+
2783
+ | Tally | | 8 | | | |
2784
+ +-------------+-------+-----+------------+--------+--------------------+
2785
+ | Pay | | | | | We have 1 grp |
2786
+ +-------------+-------+-----+------------+--------+--------------------+
2787
+ | Group count | | | | | We have 1 grp |
2788
+ +-------------+-------+-----+------------+--------+--------------------+
2789
+ | Comment | | | | | We're overspending |
2790
+ +=============+=======+=====+============+========+====================+
2791
+ #+end_EXAMPLE
1939
2792
 
1940
- There are several ways to invoke the formatting methods on a table. First, you
1941
- can instantiate a ~XXXFormatter~ object and feed it a table as a parameter.
1942
- There is a Formatter subclass for each target output medium, for example,
1943
- ~AoaFormatter~ will produce a ruby array of arrays. You can then call the
1944
- ~output~ method on the ~XXXFormatter~.
2793
+ ***** Ruby Objects
2794
+ You can make the aggregator an normal ruby object, in which case it is just
2795
+ inserted into the footer at the requested location. If its type is the same
2796
+ as the column type, it participates in the formatting for that type and
2797
+ column.
2798
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2799
+ tab_a.to_text do |f|
2800
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2801
+ f.footer('Report Date', age: :count, join_date: Date.today)
2802
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2803
+ end
2804
+ #+END_SRC
1945
2805
 
1946
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1947
- #+BEGIN_SRC ruby
2806
+ #+begin_EXAMPLE
2807
+ +=============+=======+=====+============+========+=============+
2808
+ | Id | Name | Age | Address | Salary | Join Date |
2809
+ +-------------+-------+-----+------------+--------+-------------+
2810
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2811
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2812
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2813
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2814
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2815
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2816
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2817
+ | 10 | James | 45 | Texas | 5,000 | |
2818
+ +-------------+-------+-----+------------+--------+-------------+
2819
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2820
+ +-------------+-------+-----+------------+--------+-------------+
2821
+ | Report Date | | 8 | | | 20-JAN-2022 |
2822
+ +=============+=======+=====+============+========+=============+
2823
+ #+end_EXAMPLE
2824
+
2825
+ But it can be any type. Here we pick a lottery winner from the employee ids.
2826
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2827
+ tab_a.to_text do |f|
2828
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2829
+ winner_id = tab_a.column(:id).items.sample
2830
+ f.footer('Lottery Winner', age: :count, join_date: winner_id)
2831
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2832
+ end
2833
+ #+END_SRC
2834
+
2835
+ #+begin_EXAMPLE
2836
+ +================+=======+=====+============+========+=============+
2837
+ | Id | Name | Age | Address | Salary | Join Date |
2838
+ +----------------+-------+-----+------------+--------+-------------+
2839
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2840
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2841
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2842
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2843
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2844
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2845
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2846
+ | 10 | James | 45 | Texas | 5,000 | |
2847
+ +----------------+-------+-----+------------+--------+-------------+
2848
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2849
+ +----------------+-------+-----+------------+--------+-------------+
2850
+ | Lottery Winner | | 8 | | | 4 |
2851
+ +================+=======+=====+============+========+=============+
2852
+ #+end_EXAMPLE
2853
+
2854
+ ***** Lambdas
2855
+ Perhaps the most flexible form of aggregator is a lambda form. They require 2
2856
+ or 3 parameters in non-group and group footers, respectively:
2857
+
2858
+ - ~->(f, c) {...}~ :: in a normal, non-group footer, you must provide for two
2859
+ paramters: the first, ~f~, will be bound to the footer in which the lambda
2860
+ appears and the second, ~c~, will be bound to the column header to which the
2861
+ lambda is attached.
2862
+ - ~->(f, c, k)~ :: in a group footer, you must provide for three paramters:
2863
+ the first, ~f~, will be bound to the footer in which the lambda appears, the
2864
+ second, ~c~, will be bound to the column header to which the lambda is
2865
+ attached, and the third, ~k~ will be bound to the group number of the group
2866
+ being evaluated.
2867
+
2868
+ With the first argument, the footer itself becomes available and with it all
2869
+ the things accessible with the footers, including the items in the current
2870
+ column, through the ~f.items(c)~ accessor.
2871
+
2872
+ Compute the summ of the squares if the items in the ~:age~ column:
2873
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2874
+ tab_a.to_text do |f|
2875
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2876
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2877
+ f.footer('SSQ', age: ->(f, c) { sa = f.items(c).map {|x| x * x}.sum; Math.sqrt(sa) })
2878
+ end
2879
+ #+END_SRC
2880
+
2881
+ #+begin_EXAMPLE
2882
+ +=========+=======+=====+============+========+=============+
2883
+ | Id | Name | Age | Address | Salary | Join Date |
2884
+ +---------+-------+-----+------------+--------+-------------+
2885
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2886
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2887
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2888
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2889
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2890
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2891
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2892
+ | 10 | James | 45 | Texas | 5,000 | |
2893
+ +---------+-------+-----+------------+--------+-------------+
2894
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2895
+ +---------+-------+-----+------------+--------+-------------+
2896
+ | SSQ | | 90 | | | |
2897
+ +=========+=======+=====+============+========+=============+
2898
+ #+end_EXAMPLE
2899
+
2900
+ Group the table according to the employee's year of joining, then compute the
2901
+ summ of the squares if the ages in each group:
2902
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2903
+ tab_a.order_with('join_date.year').to_text do |f|
2904
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]', sort_key: '0.0~,')
2905
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2906
+ f.gfooter('Group SSQ', age: ->(f, c, k) { sa = f.items(c, k).map {|x| x * x}.sum; Math.sqrt(sa) })
2907
+ f.footer('Total SSQ', age: ->(f, c) { sa = f.items(c).map {|x| x * x}.sum; Math.sqrt(sa) })
2908
+ end
2909
+ #+END_SRC
2910
+
2911
+ #+begin_EXAMPLE
2912
+ +===========+=======+=====+============+========+=============+==========+
2913
+ | Id | Name | Age | Address | Salary | Join Date | Sort Key |
2914
+ +-----------+-------+-----+------------+--------+-------------+----------+
2915
+ | 10 | James | 45 | Texas | 5,000 | | |
2916
+ +-----------+-------+-----+------------+--------+-------------+----------+
2917
+ | Group SSQ | | 45 | | | | |
2918
+ +-----------+-------+-----+------------+--------+-------------+----------+
2919
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 | 2001 |
2920
+ +-----------+-------+-----+------------+--------+-------------+----------+
2921
+ | Group SSQ | | 32 | | | | |
2922
+ +-----------+-------+-----+------------+--------+-------------+----------+
2923
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 | 2005 |
2924
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 | 2005 |
2925
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 | 2005 |
2926
+ +-----------+-------+-----+------------+--------+-------------+----------+
2927
+ | Group SSQ | | 56 | | | | |
2928
+ +-----------+-------+-----+------------+--------+-------------+----------+
2929
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 | 2007 |
2930
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 | 2007 |
2931
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 | 2007 |
2932
+ +-----------+-------+-----+------------+--------+-------------+----------+
2933
+ | Group SSQ | | 43 | | | | |
2934
+ +-----------+-------+-----+------------+--------+-------------+----------+
2935
+ | Average | | 31 | | 31,429 | 29-DEC-2005 | |
2936
+ +-----------+-------+-----+------------+--------+-------------+----------+
2937
+ | Total SSQ | | 90 | | | | |
2938
+ +===========+=======+=====+============+========+=============+==========+
2939
+ #+end_EXAMPLE
2940
+
2941
+ *** Invoking Formatters
2942
+ As the examples show, one way to invoke the formatting methods is simply to
2943
+ call one of the ~to_xxx~ methods directly on a table, which will yield a
2944
+ ~FatTable::Formatter~ object to the block, and that is often the most
2945
+ convenient way to do it. But there are a few other ways.
2946
+
2947
+ **** By Instantiating a Formatter
2948
+ You can instantiate a ~XXXFormatter~ object and feed it a table as a
2949
+ parameter. There is a Formatter subclass for each target output medium, for
2950
+ example, ~AoaFormatter~ will produce a ruby array of arrays. You can then call
2951
+ the ~output~ method on the ~XXXFormatter~.
2952
+
2953
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1948
2954
  FatTable::AoaFormatter.new(tab_a).output
1949
2955
  #+END_SRC
1950
2956
 
@@ -1964,8 +2970,7 @@ There is a Formatter subclass for each target output medium, for example,
1964
2970
  The ~XXXFormatter.new~ method yields the new instance to any block given, and
1965
2971
  you can call methods on it to affect the formatting of the output:
1966
2972
 
1967
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1968
- #+BEGIN_SRC ruby
2973
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1969
2974
  FatTable::AoaFormatter.new(tab_a) do |f|
1970
2975
  f.format(numeric: '0.0,R', id: '3.0C')
1971
2976
  end.output
@@ -1984,15 +2989,13 @@ you can call methods on it to affect the formatting of the output:
1984
2989
  | 010 | James | 45 | Texas | 5,000 | |
1985
2990
  #+END_EXAMPLE
1986
2991
 
1987
- **** ~FatTable~ module-level method calls
1988
-
2992
+ **** By Using ~FatTable~ module-level method calls
1989
2993
  The ~FatTable~ module provides a set of methods of the form ~to_aoa~, ~to_text~,
1990
2994
  etc., to access a ~Formatter~ without having to create an instance yourself.
1991
2995
  Without a block, they apply the default formatting to the table and call the
1992
2996
  ~.output~ method automatically:
1993
2997
 
1994
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
1995
- #+BEGIN_SRC ruby
2998
+ #+BEGIN_SRC ruby :wrap EXAMPLE
1996
2999
  FatTable.to_aoa(tab_a)
1997
3000
  #+END_SRC
1998
3001
 
@@ -2013,8 +3016,7 @@ With a block, these methods yield a ~Formatter~ instance on which you can call
2013
3016
  formatting and footer methods. The ~.output~ method is called on the ~Formatter~
2014
3017
  automatically after the block:
2015
3018
 
2016
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2017
- #+BEGIN_SRC ruby
3019
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2018
3020
  FatTable.to_aoa(tab_a) do |f|
2019
3021
  f.format(numeric: '0.0,R', id: '3.0C')
2020
3022
  end
@@ -2033,13 +3035,11 @@ automatically after the block:
2033
3035
  | 010 | James | 45 | Texas | 5,000 | |
2034
3036
  #+END_EXAMPLE
2035
3037
 
2036
- **** Calling methods on Table objects
2037
-
2038
- Finally, you can call methods such as ~to_aoa~, ~to_text~, etc., directly on a
2039
- Table:
3038
+ **** By Calling Methods on Table Objects
3039
+ Finally, as in many of the examples, you can call methods such as ~to_aoa~,
3040
+ ~to_text~, etc., directly on a Table:
2040
3041
 
2041
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2042
- #+BEGIN_SRC ruby
3042
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2043
3043
  tab_a.to_aoa
2044
3044
  #+END_SRC
2045
3045
 
@@ -2058,8 +3058,7 @@ Table:
2058
3058
 
2059
3059
  And you can supply a block to them as well to specify formatting or footers:
2060
3060
 
2061
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2062
- #+BEGIN_SRC ruby
3061
+ #+BEGIN_SRC ruby :wrap EXAMPLE
2063
3062
  tab_a.to_aoa do |f|
2064
3063
  f.format(numeric: '0.0,R', id: '3.0C')
2065
3064
  f.sum_footer(:salary, :age)
@@ -2081,135 +3080,13 @@ And you can supply a block to them as well to specify formatting or footers:
2081
3080
  | Total | | 245 | | 220,000 | |
2082
3081
  #+END_EXAMPLE
2083
3082
 
2084
- *** The ~format~ and ~format_for~ methods
2085
-
2086
- Formatters take only two kinds of methods, those that attach footers to a
2087
- table, which are discussed in the next section, and those that specify
2088
- formatting for table cells, which are the subject of this section.
2089
-
2090
- To set formatting directives for all locations in a table at once, use the
2091
- ~format~ method; to set formatting directives for a particular location in the
2092
- table, use the ~format_for~ method, giving the location as the first parameter.
2093
-
2094
- Other than that first parameter, the two methods take the same types of
2095
- parameters. The remaining parameters are hash-like parameters that use either a
2096
- column name or a type as the key and a string with the formatting directives to
2097
- apply as the value. The following example says to set the formatting for all
2098
- locations in the table and to format all numeric fields as strings that are
2099
- rounded to whole numbers (the '0.0' part), that are right-aligned (the 'R'
2100
- part), and have grouping commas inserted (the ',' part). But the ~:id~ column is
2101
- numeric, and the second parameter overrides the formatting for numerics in
2102
- general and calls for the ~:id~ column to be padded to three digits with zeros
2103
- on the left (the '3.0' part) and to be centered (the 'C' part).
2104
-
2105
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2106
- #+BEGIN_SRC ruby
2107
- tab_a.to_aoa do |f|
2108
- f.format(numeric: '0.0,R', id: '3.0C')
2109
- end
2110
- #+END_SRC
2111
-
2112
- #+BEGIN_EXAMPLE
2113
- | Id | Name | Age | Address | Salary | Join Date |
2114
- |-----+-------+-----+------------+--------+------------|
2115
- | 001 | Paul | 32 | California | 20,000 | 2001-07-13 |
2116
- | 003 | Teddy | 23 | Norway | 20,000 | 2007-12-13 |
2117
- | 004 | Mark | 25 | Rich-Mond | 65,000 | 2007-12-13 |
2118
- | 005 | David | 27 | Texas | 85,000 | 2007-12-13 |
2119
- | 002 | Allen | 25 | Texas | | 2005-07-13 |
2120
- | 008 | Paul | 24 | Houston | 20,000 | 2005-07-13 |
2121
- | 009 | James | 44 | Norway | 5,000 | 2005-07-13 |
2122
- | 010 | James | 45 | Texas | 5,000 | |
2123
- #+END_EXAMPLE
2124
-
2125
- The ~numeric:~ directive affected the ~:age~ and ~:salary~ columns and the ~id:~
2126
- directive affected only the ~:id~ column. All the other cells in the table had
2127
- the default formatting applied.
2128
-
2129
- **** Location priority
2130
-
2131
- Formatting for any given cell depends on its location in the table. The
2132
- ~format_for~ method takes a location to which its formatting directive are
2133
- restricted as the first argument. It can be one of the following:
2134
-
2135
- - ~:header~ :: directive apply only to the header row, that is the first row, of
2136
- the output table,
2137
-
2138
- - ~:footer~ :: directives apply to all the footer rows of the output table,
2139
- regardless of how many there are,
2140
-
2141
- - ~gfooter~ :: directives apply to all group footer rows of the output tables,
2142
- regardless of how many there are,
2143
-
2144
- - ~:body~ :: directives apply to all rows in the body of the table unless the
2145
- row is the first row in the table or in a group and separate directives for
2146
- those have been given, in which case those directives apply,
2147
-
2148
- - ~:gfirst~ :: directives apply to the first row in each group in the body of
2149
- the table, unless the row is also the first row in the table as a whole, in
2150
- which case the ~:bfirst~ directives apply,
2151
-
2152
- - ~:bfirst~ :: directives apply to the first row in the body of the table.
2153
-
2154
- If you give directives for ~:body~, they are copied to ~:bfirst~ and ~:gfirst~
2155
- as well and can be overridden by directives for those locations.
2156
-
2157
- Directives given to the ~format~ method apply the directives to all locations in
2158
- the table, but they can be overridden by more specific directives given in a
2159
- ~format_for~ directive.
2160
-
2161
- **** Type and Column priority
2162
-
2163
- A directive based on type applies to all columns having that type unless
2164
- overridden by a directive specific to a named column; a directive based on a
2165
- column name applies only to cells in that column.
2166
-
2167
- However, there is a twist. Since the end result of formatting is to convert all
2168
- columns to strings, the formatting directives for the ~:string~ type applies to
2169
- all columns. Likewise, since all columns may contain nils, the ~nil:~ type
2170
- applies to nils in all columns regardless of the column's type.
2171
-
2172
- #+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
2173
- #+BEGIN_SRC ruby
2174
- require 'fat_table'
2175
- tab_a.to_text do |f|
2176
- f.format(string: 'R', id: '3.0C', salary: 'n[N/A]')
2177
- end
2178
- #+END_SRC
2179
-
2180
- #+BEGIN_EXAMPLE
2181
- +=====+=======+=====+============+========+============+
2182
- | Id | Name | Age | Address | Salary | Join Date |
2183
- +-----+-------+-----+------------+--------+------------+
2184
- | 001 | Paul | 32 | California | 20000 | 2001-07-13 |
2185
- | 003 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2186
- | 004 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2187
- | 005 | David | 27 | Texas | 85000 | 2007-12-13 |
2188
- | 002 | Allen | 25 | Texas | N/A | 2005-07-13 |
2189
- | 008 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2190
- | 009 | James | 44 | Norway | 5000 | 2005-07-13 |
2191
- | 010 | James | 45 | Texas | 5000 | |
2192
- +=====+=======+=====+============+========+============+
2193
- #+END_EXAMPLE
2194
-
2195
- The ~string: 'R'~ directive causes all the cells to be right-aligned except
2196
- ~:id~ which specifies centering for the ~:id~ column only. The ~n[N/A]~
2197
- directive for specifies how nil are displayed in the numeric column, ~:salary~,
2198
- but not for other nils, such as in the last row of the ~:join_date~ column.
2199
-
2200
3083
  * Development
2201
-
2202
3084
  After checking out the repo, run `bin/setup` to install dependencies. Then, run
2203
3085
  `rake spec` to run the tests. You can also run `bin/console` for an interactive
2204
3086
  prompt that will allow you to experiment.
2205
3087
 
2206
- To install this gem onto your local machine, run `bundle exec rake install`. To
2207
- release a new version, update the version number in `version.rb`, and then run
2208
- `bundle exec rake release`, which will create a git tag for the version, push
2209
- git commits and tags, and push the `.gem` file to
2210
- [rubygems.org](https://rubygems.org).
3088
+ To install this gem onto your local machine, run `bundle exec rake install`.
2211
3089
 
2212
3090
  * Contributing
2213
-
2214
3091
  Bug reports and pull requests are welcome on GitHub at
2215
3092
  https://github.com/ddoherty03/fat_table.