fat_table 0.9.9 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md ADDED
@@ -0,0 +1,2961 @@
1
+ - [Version](#org5334859)
2
+ - [Introduction](#orgb65aad9)
3
+ - [Installation](#org17c4677)
4
+ - [Using in a gem](#org8a17717)
5
+ - [Manually install](#org530598c)
6
+ - [Require](#org57aae27)
7
+ - [Usage](#orgf09f39e)
8
+ - [Quick Start](#orge842ad0)
9
+ - [A Word About the Examples](#org763a25b)
10
+ - [Anatomy of a Table](#orga84269a)
11
+ - [Columns](#org9e64593)
12
+ - [Type Keywords Arguments](#org390f785)
13
+ - [Headers](#orgb4b83f3)
14
+ - [Groups](#orgc73107b)
15
+ - [Constructing Tables](#orgb1c6fef)
16
+ - [Empty Tables](#orgdf8f1e2)
17
+ - [From CSV or Org Mode files or strings](#org97b4744)
18
+ - [From Arrays of Arrays](#org8fddae3)
19
+ - [From Arrays of Hashes](#org2079ffb)
20
+ - [From SQL queries](#org1f3b4b6)
21
+ - [Marking Groups in Input](#org4f6203d)
22
+ - [Accessing Parts of Tables](#orge15c95e)
23
+ - [Rows](#org298a474)
24
+ - [Columns](#org5782179)
25
+ - [Cells](#org326ae38)
26
+ - [Other table attributes](#orgc525931)
27
+ - [Operations on Tables](#orgf11c014)
28
+ - [Example Input Tables](#org4a978ae)
29
+ - [Select](#org3f88455)
30
+ - [Where](#orgb70a9cb)
31
+ - [Order\_by](#org54a98c1)
32
+ - [Order\_with](#org6562f48)
33
+ - [Group\_by](#orgbd3e61e)
34
+ - [Join](#orgebcd7d8)
35
+ - [Set Operations](#org654f7a8)
36
+ - [Uniq (aka Distinct)](#orgd68c265)
37
+ - [Remove groups with degroup!](#org8b63b11)
38
+ - [Formatting Tables](#org767e09e)
39
+ - [Available Formatter Output Targets](#org945b4a7)
40
+ - [Formatting Directives](#orga5d8638)
41
+ - [The `format` and `format_for` methods](#org11bef41)
42
+ - [Footers](#org53ea8ff)
43
+ - [Invoking Formatters](#org102d235)
44
+ - [Development](#orga37c582)
45
+ - [Contributing](#org6d38bff)
46
+
47
+ [![CI](https://github.com/ddoherty03/fat_table/actions/workflows/ruby-with-dbs.yml/badge.svg)](
48
+ https://github.com/ddoherty03/fat_table/actions/workflows/ruby-with-dbs.yml
49
+ )
50
+
51
+
52
+ <a id="org5334859"></a>
53
+
54
+ # Version
55
+
56
+ ```ruby
57
+ "Current version is: #{FatTable::VERSION}"
58
+ ```
59
+
60
+ ```
61
+ Current version is: 1.0.0
62
+ ```
63
+
64
+
65
+ <a id="orgb65aad9"></a>
66
+
67
+ # Introduction
68
+
69
+ `FatTable` is a gem that treats tables as a data type. It provides methods for constructing tables from a variety of sources, building them row-by-row, extracting rows, columns, and cells, and performing aggregate operations on columns. It also provides a set of SQL-esque methods for manipulating table objects: `select` for filtering by columns or for creating new columns, `where` for filtering by rows, `order_by` for sorting rows, `distinct` for eliminating duplicate rows, `group_by` for aggregating multiple rows into single rows and applying column aggregate methods to ungrouped columns, a collection of `join` methods for combining tables, and more.
70
+
71
+ Furthermore, `FatTable` provides methods for formatting tables and producing output that targets various output media: text, ANSI terminals, ruby data structures, LaTeX tables, Emacs org-mode tables, and more. The formatting methods can specify cell formatting in a way that is uniform across all the output methods and can also decorate the output with any number of footers, including group footers. `FatTable` applies formatting directives to the extent they makes sense for the output medium and treats other formatting directives as no-ops.
72
+
73
+ `FatTable` can be used to perform operations on data that are naturally best conceived of as tables, which in my experience is quite often. It can also serve as a foundation for providing reporting functions where flexibility about the output medium can be useful. Finally `FatTable` can be used within Emacs `org-mode` files in code blocks targeting the Ruby language. Org mode tables are presented to a ruby code block as an array of arrays, so `FatTable` can read them in with its `.from_aoa` constructor. A `FatTable` table output as an array of arrays with its `.to_aoa` output function will be rendered in an org-mode buffer as an org-table, ready for processing by other code blocks.
74
+
75
+
76
+ <a id="org17c4677"></a>
77
+
78
+ # Installation
79
+
80
+
81
+ <a id="org8a17717"></a>
82
+
83
+ ## Using in a gem
84
+
85
+ Add this line to your application's Gemfile:
86
+
87
+ ```ruby
88
+ gem 'fat_table'
89
+ ```
90
+
91
+ ```
92
+ true
93
+ ```
94
+
95
+ Or, something like this in your gemspec file:
96
+
97
+ ```ruby
98
+ gem.add_runtime_dependency 'fat_table'
99
+ ```
100
+
101
+ ```
102
+ false
103
+ ```
104
+
105
+ And then execute:
106
+
107
+ ```sh
108
+ $ bundle
109
+ ```
110
+
111
+
112
+ <a id="org530598c"></a>
113
+
114
+ ## Manually install
115
+
116
+ Or install it yourself as:
117
+
118
+ ```sh
119
+ $ gem install fat_table
120
+ ```
121
+
122
+
123
+ <a id="org57aae27"></a>
124
+
125
+ ## Require
126
+
127
+ Somewhere in your code, make sure that `FatTable` is required:
128
+
129
+ ```ruby
130
+ require 'fat_table'
131
+ ```
132
+
133
+
134
+ <a id="orgf09f39e"></a>
135
+
136
+ # Usage
137
+
138
+
139
+ <a id="orge842ad0"></a>
140
+
141
+ ## Quick Start
142
+
143
+ `FatTable` provides table objects as a data type that can be constructed and operated on in a number of ways. Here's a quick example to illustrate the use of `FatTable`. See the detailed explanations further on down.
144
+
145
+ Here is a set of data that records some kind of stock activity. It's an array of arrays with the first inner array being the headings.
146
+
147
+ ```ruby
148
+ data =
149
+ [['Date', 'Code', 'Raw', 'Shares', 'Price', 'Info', 'Ok'],
150
+ ['2013-05-29', 'S', 15_700.00, 6601.85, 24.7790, 'ENTITY3', 'F'],
151
+ ['2013-05-02', 'P', 118_186.40, 118_186.4, 11.8500, 'ENTITY1', 'T'],
152
+ ['2013-05-20', 'S', 12_000.00, 5046.00, 28.2804, 'ENTITY3', 'F'],
153
+ ['2013-05-23', 'S', 8000.00, 3364.00, 27.1083, 'ENTITY3', 'T'],
154
+ ['2013-05-23', 'S', 39_906.00, 16_780.47, 25.1749, 'ENTITY3', 'T'],
155
+ ['2013-05-20', 'S', 85_000.00, 35_742.50, 28.3224, 'ENTITY3', 'T'],
156
+ ['2013-05-02', 'P', 795_546.20, 795_546.2, 1.1850, 'ENTITY1', 'T'],
157
+ ['2013-05-29', 'S', 13_459.00, 5659.51, 24.7464, 'ENTITY3', 'T'],
158
+ ['2013-05-20', 'S', 33_302.00, 14_003.49, 28.6383, 'ENTITY3', 'T'],
159
+ ['2013-05-29', 'S', 15_900.00, 6685.95, 24.5802, 'ENTITY3', 'T'],
160
+ ['2013-05-30', 'S', 6_679.00, 2808.52, 25.0471, 'ENTITY3', 'T'],
161
+ ['2013-05-23', 'S', 23_054.00, 9694.21, 26.8015, 'ENTITY3', 'F']]
162
+ ```
163
+
164
+ Use FatTable to read the data and convert in into a table object. Note that the headings within the table are all converted to symbols, lower-cased and any spaces replaced with underscores.
165
+
166
+ Below, we select only those rows having more than 2000 shares, sort by a compund key, select all columns but add a column, :ref, for the row number, and finally re-order the columns with a final select.
167
+
168
+ ```ruby
169
+ table = FatTable.from_aoa(data)
170
+ .where('shares > 2000')
171
+ .order_by(:date, :code)
172
+ .select(:date, :code, :shares,
173
+ :price, :ok, ref: '@row')
174
+ .select(:ref, :date, :code,
175
+ :shares, :price, :ok)
176
+ ```
177
+
178
+ You can use the resulting table in other operations, such as performing joins or set operations with other tables, etc. The world's your oyster. But eventually you will want to present the table in some format, and that is where the formatting methods come in. They let you add footers, including groups footers, as well as styling the various elements with very simple formatting directives that can apply to various "locations" in the table. Any formatting directives that are beyond the capabilities of the output medium are simply ignored.
179
+
180
+ We can format the table constructed above.
181
+
182
+ ```ruby
183
+ table.to_text do |fmt|
184
+ # Add a group footer at the bottom of each group that results from sorting
185
+ # with the order_by method.
186
+ fmt.gfooter('Avg', shares: :avg, price: :avg)
187
+ # Add some table footers. Averages for the price and shares columns. The
188
+ # avg_footer method applies the avg aggregate to all the named columns with
189
+ # an "Average" label.
190
+ fmt.avg_footer(:price, :shares)
191
+ # And a second footer that shows the sum for the shares column.
192
+ fmt.sum_footer(:shares)
193
+ # Formats for all locations, :ref column is centered and bold, all numerics
194
+ # are right-aligned, and all booleans are centered and printed with 'Y' or
195
+ # 'N'
196
+ fmt.format(ref: 'CB', numeric: 'R', boolean: 'CY')
197
+ # Formats for different "locations" in the table:
198
+ # The headers are all centered and bold.
199
+ fmt.format_for(:header, string: 'CB')
200
+ # In the body rows (i.e., not the headers or footers), the code column is
201
+ # centered, shares have grouping commas applied and are rounded to one
202
+ # decimal place, but the price column is rounded to 4 places with no
203
+ # grouping commas.
204
+ fmt.format_for(:body, code: 'C', shares: ',0.1', price: '0.4', )
205
+ # But the price column in the first row of the body (:bfirst location) will
206
+ # also be formatted with a currency symbol.
207
+ fmt.format_for(:bfirst, price: '$0.4', )
208
+ # In the footers, apply the same rounding rules, but make the results bold.
209
+ fmt.format_for(:gfooter, shares: 'B,0.1', price: 'B0.4', )
210
+ fmt.format_for(:footer, shares: 'B,0.1', price: '$B0.4', )
211
+ end
212
+ ```
213
+
214
+ ```
215
+ +============+======+=============+=============+==========+=========+====+
216
+ | Date | Code | Raw | Shares | Price | Info | Ok |
217
+ +------------+------+-------------+-------------+----------+---------+----+
218
+ | 2013-05-29 | S | 15700.0000 | 6,601.9 | $24.7790 | ENTITY3 | N |
219
+ | 2013-05-02 | P | 118186.4000 | 118,186.4 | 11.8500 | ENTITY1 | Y |
220
+ | 2013-05-20 | S | 12000.0000 | 5,046.0 | 28.2804 | ENTITY3 | N |
221
+ | 2013-05-23 | S | 8000.0000 | 3,364.0 | 27.1083 | ENTITY3 | Y |
222
+ | 2013-05-23 | S | 39906.0000 | 16,780.5 | 25.1749 | ENTITY3 | Y |
223
+ | 2013-05-20 | S | 85000.0000 | 35,742.5 | 28.3224 | ENTITY3 | Y |
224
+ | 2013-05-02 | P | 795546.2000 | 795,546.2 | 1.1850 | ENTITY1 | Y |
225
+ | 2013-05-29 | S | 13459.0000 | 5,659.5 | 24.7464 | ENTITY3 | Y |
226
+ | 2013-05-20 | S | 33302.0000 | 14,003.5 | 28.6383 | ENTITY3 | Y |
227
+ | 2013-05-29 | S | 15900.0000 | 6,685.10 | 24.5802 | ENTITY3 | Y |
228
+ | 2013-05-30 | S | 6679.0000 | 2,808.5 | 25.0471 | ENTITY3 | Y |
229
+ | 2013-05-23 | S | 23054.0000 | 9,694.2 | 26.8015 | ENTITY3 | N |
230
+ +------------+------+-------------+-------------+----------+---------+----+
231
+ | Avg | | | 85,009.9 | 23.0428 | | |
232
+ +------------+------+-------------+-------------+----------+---------+----+
233
+ | Average | | | 85,009.9 | $23.0428 | | |
234
+ +------------+------+-------------+-------------+----------+---------+----+
235
+ | Total | | | 1,020,119.1 | | | |
236
+ +============+======+=============+=============+==========+=========+====+
237
+ ```
238
+
239
+ For the text format above, we were wasting our breath specifying bold styling since there is no way to make that happen in plain ASCII text. But with LaTeX, bold is doable. The output of the following code block is being written to a file `spec/example_files/quicktable.tex` which is then `\included`-ed in a simple wrapper file, `spec/example_files/quick.tex` so it can be compiled by LaTeX.
240
+
241
+ ```ruby
242
+ table.to_latex do |fmt|
243
+ fmt.gfooter('Avg', shares: :avg, price: :avg)
244
+ fmt.avg_footer(:price, :shares)
245
+ fmt.sum_footer(:shares)
246
+ fmt.format(ref: 'CB', numeric: 'R', boolean: 'CY')
247
+ fmt.format_for(:header, string: 'CB')
248
+ fmt.format_for(:body, code: 'C', shares: ',0.1c[blue.lightgray]', price: '0.4', )
249
+ fmt.format_for(:bfirst, price: '$0.4', )
250
+ fmt.format_for(:gfooter, shares: 'B,0.1', price: 'B0.4', )
251
+ fmt.format_for(:footer, shares: 'B,0.1', price: '$B0.4', )
252
+ end
253
+ ```
254
+
255
+ ```
256
+ [[file:spec/example_files/quicktable.tex]]
257
+ ```
258
+
259
+ <spec/example_files/quicktable.tex>
260
+
261
+ These commands run pdflatex on the result twice to get the table aligned properly.
262
+
263
+ ```sh
264
+ cd spec/example_files
265
+ pdflatex quick.tex
266
+ pdflatex quick.tex
267
+ ```
268
+
269
+ And we convert the `PDF` into a smaller image for display:
270
+
271
+ ```sh
272
+ cd spec/example_files
273
+ pdftoppm -png quick.pdf >quick.png
274
+ magick quick.png -resize 600x800 quick_small.png
275
+ ```
276
+
277
+ ![img](spec/example_files/quick_small.png)
278
+
279
+
280
+ <a id="org763a25b"></a>
281
+
282
+ ## A Word About the Examples
283
+
284
+ When you install the `fat_table` gem, you have access to a program `ft_console`, which opens a `pry` session with `fat_table` loaded and the tables used in the examples in this `README` defined as instance variables so you can experiment with them. Because they are defined as instance variables, you have to write `tab1` as `@tab1` in `ft_console`, but otherwise the examples should work as shown in this `README`.
285
+
286
+ The examples in this `README` file are executed in Emacs org-mode as code blocks within the `README.org` file, so they typically end with a call to `.to_aoa`. That causes Emacs to insert the "Array of Array" ruby data structure into the file and format it as a table, which is the convention for Emacs org-mode. With `ft_console`, you should instead display your tables with `.to_text` or `.to_term`. These will return a string that you can print to the terminal with `puts`.
287
+
288
+ To read in the table used in the Quick Start section above, you might do the following:
289
+
290
+ ```
291
+ $ ft_console[1] pry(main)> ls
292
+ ActiveSupport::ToJsonWithActiveSupportEncoder#methods: to_json
293
+ self.methods: inspect to_s
294
+ instance variables:
295
+ @aoa @tab1 @tab2 @tab_a @tab_b @tt
296
+ @data @tab1_str @tab2_str @tab_a_str @tab_b_str
297
+ locals: _ __ _dir_ _ex_ _file_ _in_ _out_ _pry_ lib str version
298
+ [2] pry(main)> table = FatTable.from_aoa(@data)
299
+ => #<FatTable::Table:0x0055b40e6cd870
300
+ @boundaries=[],
301
+ @columns=
302
+ [#<FatTable::Column:0x0055b40e6cc948
303
+ @header=:date,
304
+ @items=
305
+ [Wed, 29 May 2013,
306
+ Thu, 02 May 2013,
307
+ Mon, 20 May 2013,
308
+ Thu, 23 May 2013,
309
+ Thu, 23 May 2013,
310
+ Mon, 20 May 2013,
311
+ Thu, 02 May 2013,
312
+ Wed, 29 May 2013,
313
+ Mon, 20 May 2013,
314
+ ...
315
+ @items=["ENTITY3", "ENTITY1", "ENTITY3", "ENTITY3", "ENTITY3", "ENTITY3", "ENTITY1", "ENTITY3", "ENTITY3", "ENTITY3", "ENTITY3", "ENTITY3"],
316
+ @raw_header=:info,
317
+ @type="String">,
318
+ #<FatTable::Column:0x0055b40e6d2668 @header=:ok, @items=[false, true, false, true, true, true, true, true, true, true, true, false], @raw_header=:ok, @type="Boolean">]>
319
+ [3] pry(main)> puts table.to_text
320
+ +============+======+==========+==========+=========+=========+====+
321
+ | Date | Code | Raw | Shares | Price | Info | Ok |
322
+ +------------+------+----------+----------+---------+---------+----+
323
+ | 2013-05-29 | S | 15700.0 | 6601.85 | 24.779 | ENTITY3 | F |
324
+ | 2013-05-02 | P | 118186.4 | 118186.4 | 11.85 | ENTITY1 | T |
325
+ | 2013-05-20 | S | 12000.0 | 5046.0 | 28.2804 | ENTITY3 | F |
326
+ | 2013-05-23 | S | 8000.0 | 3364.0 | 27.1083 | ENTITY3 | T |
327
+ | 2013-05-23 | S | 39906.0 | 16780.47 | 25.1749 | ENTITY3 | T |
328
+ | 2013-05-20 | S | 85000.0 | 35742.5 | 28.3224 | ENTITY3 | T |
329
+ | 2013-05-02 | P | 795546.2 | 795546.2 | 1.185 | ENTITY1 | T |
330
+ | 2013-05-29 | S | 13459.0 | 5659.51 | 24.7464 | ENTITY3 | T |
331
+ | 2013-05-20 | S | 33302.0 | 14003.49 | 28.6383 | ENTITY3 | T |
332
+ | 2013-05-29 | S | 15900.0 | 6685.95 | 24.5802 | ENTITY3 | T |
333
+ | 2013-05-30 | S | 6679.0 | 2808.52 | 25.0471 | ENTITY3 | T |
334
+ | 2013-05-23 | S | 23054.0 | 9694.21 | 26.8015 | ENTITY3 | F |
335
+ +============+======+==========+==========+=========+=========+====+
336
+ => nil
337
+ [4] pry(main)>
338
+ ```
339
+
340
+ If you use `puts table.to_term`, you can see the effect of the color formatting directives.
341
+
342
+
343
+ <a id="orga84269a"></a>
344
+
345
+ ## Anatomy of a Table
346
+
347
+
348
+ <a id="org9e64593"></a>
349
+
350
+ ### Columns
351
+
352
+ `FatTable::Table` objects consist of an array of `FatTable::Column` objects. Each `Column` has a header, a type, and an array of items, all of the given type or nil. There are only five permissible types for a `Column`:
353
+
354
+ 1. **Boolean** (for holding ruby `TrueClass` and `FalseClass` objects),
355
+ 2. **DateTime** (for holding ruby `DateTime` or `Date` objects),
356
+ 3. **Numeric** (for holding ruby `Integer`, `Rational`, or `BigDecimal` objects),
357
+ 4. **String** (for ruby `String` objects), or
358
+ 5. **NilClass** (for the undetermined column type).
359
+
360
+ By default, when a `Table` is constructed from an external source, all `Columns` start out having a type of `NilClass`, that is, their type is as yet undetermined. When a string or object is added to a `Column` and it can be converted into one of the permissible types, it fixes the type of the column, and all further items added to the `Column` must either be `nil` (indicating no value) or be capable of being coerced to the column's type. Otherwise, `FatTable` raises an `IncompatibleTypeError` exception.
361
+
362
+
363
+ <a id="org390f785"></a>
364
+
365
+ ### Type Keywords Arguments
366
+
367
+ All of the table constructors allow you to set the type for a column in advance by adding keyword arguments to the end of the contructor arguments where the keyword is a header symbol and the value is a string designating one of the types. For example, suppose we are constructing a table from a CSV file, and we know that one of the columns is labeled 'Start' and another 'Price'. We want to require the items in the 'Start' column to be a valid date and the items in the 'Price' column to be valid numbers:
368
+
369
+ ```
370
+ FatTable.from_csv_file('data.csv', start: 'date', price: 'num')
371
+ ```
372
+
373
+ The type string can be anything that starts with 'dat', 'num', 'boo', or 'str', regardless of case, to designate `DateTime`, `Numeric`, `Boolean`, or `String` types, respectively. Any other string keeps the type as NilClass, that is, it remains open for automatic typing.
374
+
375
+ The strictness of requiring all items to be of the same type can be relaxed by declaring a column to be "tolerant." You can do so by adding a `~` to the end of a keyword type specifier in the table constructor. In the above example, if we wanted to allow strings to be mixed up with the numeric prices, we would use the following:
376
+
377
+ ```
378
+ FatTable.from_csv_file('data.csv', start: 'date', price: 'num~')
379
+ ```
380
+
381
+ If a Column is tolerant, `FatTable` tries to convert new items into the column's specified type, or if the type is still open, to one of `DateTime`, `Numeric`, or `Boolean` and then fixing the column's type, or, if it cannot do so converts the item into a `String` but does not raise an `IncompatibleTypeError` exception. These interloper strings are treated like nils for purposes of sorting and evaluation, but are displayed according to any string formatting on output. See below.
382
+
383
+ Items of input must be either one of the permissible ruby objects or strings. If they are strings, `FatTable` attempts to parse them as one of the permissible types as follows:
384
+
385
+ - **Boolean:** The strings, `t`, `true`, `yes`, or `y`, regardless of case, are interpreted as `TrueClass` and the strings, `f`, `false`, `no`, or `n`, regardless of case, are interpreted as `FalseClass`, in either case resulting in a Boolean column. Empty strings in a column already having a Boolean type are converted to `nil`.
386
+ - **DateTime:** Strings that contain patterns of `yyyy-mm-dd` or `yyyy/mm/dd` or `mm-dd-yyy` or `mm/dd/yyyy` or any of the foregoing with an added `Thh:mm:ss` or `Thh:mm` will be interpreted as a `DateTime` or a `Date` (if there are no sub-day time components present). The number of digits in the month and day can be one or two, but the year component must be four digits. Any time components are valid if they can be properly interpreted by `DateTime.parse`. Org mode timestamps (any of the foregoing surrounded by square `[]` or pointy `<>` brackets), active or inactive, are valid input strings for `DateTime` columns. Empty strings in a column already having the `DateTime` type are converted to `nil`.
387
+ - **Numeric:** All commas (`,`) underscores (`_`) and (`$`) dollar signs (or other currency symbol as set by `FatTable.currency_symbol` are removed from the string and if the remaining string can be interpreted as a `Numeric`, it will be. It is interpreted as an `Integer` if there are no decimal places in the remaining string, as a `Rational` if the string has the form `<number>:<number>` or `<number>/<number>`, or as a `BigDecimal` if there is a decimal point in the remaining string. Empty strings in a column already having the Numeric type are converted to nil.
388
+ - **String:** If all else fails, `FatTable` applies `#to_s` to the input value and, treats it as an item of type `String`. Empty strings in a column already having the `String` type are kept as empty strings.
389
+ - **NilClass:** Until the input contains a non-blank string that can be parsed as one of the other types, it has this type, meaning that the type is still open. A column comprised completely of blank strings or `nils` will retain the `NilClass` type.
390
+
391
+
392
+ <a id="orgb4b83f3"></a>
393
+
394
+ ### Headers
395
+
396
+ Headers for the columns are formed from the input. No two columns in a table can have the same header. Headers in the input are converted to symbols by
397
+
398
+ - converting the header to a string with `#to_s`,
399
+ - converting any run of blanks to an underscore `_`,
400
+ - removing any characters that are not letters, numbers, or underscores, and
401
+ - lowercasing all remaining letters
402
+
403
+ Thus, a header of `Date` becomes `:date`, a header of `Id Number` becomes, `:id_number`, etc. When referring to a column in code, you must use the symbol form of the header.
404
+
405
+ If no sensible headers can be discerned from the input, headers of the form `:col_1`, `:col_2`, etc., are synthesized.
406
+
407
+ You should avoid the use of the column names `:omni` and `:sort_key` because they have special meanings in the `select` and `order_with` commands, respectively.
408
+
409
+
410
+ <a id="orgc73107b"></a>
411
+
412
+ ### Groups
413
+
414
+ The rows of a `FatTable` table can be divided into groups, either from markers in the input or as a result of certain operations. There is only one level of grouping, so `FatTable` has no concept of sub-groups. Groups can be shown on output with rules or "hlines" that underline the last row in each group, and you can decorate the output with group footers that summarize the rows in each group.
415
+
416
+
417
+ <a id="orgb1c6fef"></a>
418
+
419
+ ## Constructing Tables
420
+
421
+
422
+ <a id="orgdf8f1e2"></a>
423
+
424
+ ### Empty Tables
425
+
426
+ 1. Without Headers
427
+
428
+ You can create an empty table with `FatTable::Table.new` or, the shorter form, `FatTable.new`, and then add rows with the `<<` operator and a Hash. The keys in the added rows determine the names of the headers:
429
+
430
+ ```ruby
431
+ require 'fat_table'
432
+ tab = FatTable.new
433
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '' }
434
+ tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
435
+ ```
436
+
437
+ After this, the table will have column headers `:a`, `:b`, `:c`, `:d`, and `:e`. Column, `:a` and `:b` will have type Numeric, column `:c` will have type `DateTime`, and column `:d` will have type `Boolean`. Column `:e` will still have an open type. Notice that dates in the input can be wrapped in brackets as in org-mode time stamps.
438
+
439
+ ```ruby
440
+ tab.to_text
441
+ ```
442
+
443
+ ```
444
+ +========+========+============+===+===+
445
+ | A | B | C | D | E |
446
+ +--------+--------+------------+---+---+
447
+ | 1 | 2 | 2017-01-21 | F | |
448
+ | 3.1400 | 2.1700 | 2016-01-21 | T | |
449
+ +========+========+============+===+===+
450
+ ```
451
+
452
+ ```
453
+ +======+======+============+===+===+
454
+ | A | B | C | D | E |
455
+ +------+------+------------+---+---+
456
+ | 1 | 2 | 2017-01-21 | F | |
457
+ | 3.14 | 2.17 | 2016-01-21 | T | |
458
+ +======+======+============+===+===+
459
+ ```
460
+
461
+ You can continue to add rows to the table:
462
+
463
+ ```ruby
464
+ tab << { 'F' => '335:113', a: Rational(3, 5) }
465
+ ```
466
+
467
+ This last `<<` operation adds a new column headed `:f` to the table and makes the value of `:f` in all prior rows `nil`. Also, the values for the new row for which no key was give are assigned `nil` as well:
468
+
469
+ ```ruby
470
+ tab.to_text
471
+ ```
472
+
473
+ ```
474
+ +========+========+============+===+===+=========+
475
+ | A | B | C | D | E | F |
476
+ +--------+--------+------------+---+---+---------+
477
+ | 1 | 2 | 2017-01-21 | F | | |
478
+ | 3.1400 | 2.1700 | 2016-01-21 | T | | |
479
+ | 3/5 | | | | | 335/113 |
480
+ +========+========+============+===+===+=========+
481
+ ```
482
+
483
+ ```
484
+ +======+======+============+===+===+=========+
485
+ | A | B | C | D | E | F |
486
+ +------+------+------------+---+---+---------+
487
+ | 1 | 2 | 2017-01-21 | F | | |
488
+ | 3.14 | 2.17 | 2016-01-21 | T | | |
489
+ +------+------+------------+---+---+---------+
490
+ | 3/5 | | | | | 335/113 |
491
+ +======+======+============+===+===+=========+
492
+ ```
493
+
494
+ 2. With Headers
495
+
496
+ Alternatively, you can specify the headers at the outset, in which case, headers in added rows that do not match any of the initial headers cause new columns to be created:
497
+
498
+ ```ruby
499
+ require 'fat_table'
500
+ tab = FatTable.new(:a, 'b', 'C', :d)
501
+ tab.headers
502
+ ```
503
+
504
+ ```
505
+ [:a, :b, :c, :d]
506
+ ```
507
+
508
+ ```
509
+ [:a, :b, :c, :d]
510
+ ```
511
+
512
+ ```ruby
513
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '' }
514
+ tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
515
+ tab.to_text
516
+ ```
517
+
518
+ ```
519
+ +========+========+============+===+===+
520
+ | A | B | C | D | E |
521
+ +--------+--------+------------+---+---+
522
+ | 1 | 2 | 2017-01-21 | F | |
523
+ | 3.1400 | 2.1700 | 2016-01-21 | T | |
524
+ +========+========+============+===+===+
525
+ ```
526
+
527
+ ```
528
+ +======+======+============+===+===+
529
+ | A | B | C | D | E |
530
+ +------+------+------------+---+---+
531
+ | 1 | 2 | 2017-01-21 | F | |
532
+ | 3.14 | 2.17 | 2016-01-21 | T | |
533
+ +------+------+------------+---+---+
534
+ | 1 | 2 | 2017-01-21 | F | |
535
+ | 3.14 | 2.17 | 2016-01-21 | T | |
536
+ +======+======+============+===+===+
537
+ ```
538
+
539
+ 3. Forcing String Type
540
+
541
+ Occasionally, `FatTable`'s automatic type detection can get in the way and you just want it to treat one or more columns as Strings regardless of their appearance. Think, for example, of zip codes. As mentioned above, when a table is contructed, you can designate a 'String' type for a column by using a keyword parameter.
542
+
543
+ ```ruby
544
+ require 'fat_table'
545
+ tab = FatTable.new(:a, 'b!', 'C', :d, :zip, zip: 'str')
546
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '', zip: 18552 }
547
+ tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
548
+ tab << { zip: '01879--7884' }
549
+ tab << { zip: '66210', b: 'Not a Number' }
550
+ tab << { zip: '90210' }
551
+ tab.to_text
552
+ ```
553
+
554
+ ```
555
+ +========+========+============+===+=============+===+
556
+ | A | B | C | D | Zip | E |
557
+ +--------+--------+------------+---+-------------+---+
558
+ | 1 | 2 | 2017-01-21 | F | 18552 | |
559
+ | 3.1400 | 2.1700 | 2016-01-21 | T | | |
560
+ | | | | | 01879--7884 | |
561
+ | | | | | 90210 | |
562
+ | | | | | | |
563
+ +========+========+============+===+=============+===+
564
+ ```
565
+
566
+ ```
567
+ +===+===+============+===+=============+===+
568
+ | A | B | C | D | Zip | E |
569
+ +---+---+------------+---+-------------+---+
570
+ | 1 | 2 | 2017-01-21 | F | 18552 | |
571
+ | 3 | 2 | 2016-01-21 | T | | |
572
+ | | | | | 01879--7884 | |
573
+ | | | | | 90210 | |
574
+ | | | | | | |
575
+ +===+===+============+===+=============+===+
576
+ ```
577
+
578
+ In addition, at any time after creating a table, you can force the String type on any number of columns with the `force_string!` method. When you do so, all exisiting items in the column are converted to strings with the #to\_s method.
579
+
580
+ ```ruby
581
+ tab = FatTable.new(:a, 'b', 'C', :d, :zip)
582
+ tab << { a: 1, b: 2, c: "<2017-01-21>", d: 'f', e: '', zip: 18552 }
583
+ tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
584
+ tab.force_string!(:zip, :c)
585
+ tab << { zip: '01879' }
586
+ tab << { zip: '66210' }
587
+ tab << { zip: '90210' }
588
+ tab.to_text
589
+ ```
590
+
591
+ ```
592
+ +========+========+============+===+=======+===+
593
+ | A | B | C | D | Zip | E |
594
+ +--------+--------+------------+---+-------+---+
595
+ | 1 | 2 | 2017-01-21 | F | 18552 | |
596
+ | 3.1400 | 2.1700 | 2016-01-21 | T | | |
597
+ | | | | | 01879 | |
598
+ | | | | | 66210 | |
599
+ | | | | | 90210 | |
600
+ +========+========+============+===+=======+===+
601
+ ```
602
+
603
+ ```
604
+ +======+======+============+===+=======+===+
605
+ | A | B | C | D | Zip | E |
606
+ +------+------+------------+---+-------+---+
607
+ | 1 | 2 | 2017-01-21 | F | 18552 | |
608
+ | 3.14 | 2.17 | 2016-01-21 | T | | |
609
+ | | | | | 01879 | |
610
+ | | | | | 66210 | |
611
+ | | | | | 90210 | |
612
+ +======+======+============+===+=======+===+
613
+ ```
614
+
615
+
616
+ <a id="org97b4744"></a>
617
+
618
+ ### From CSV or Org Mode files or strings
619
+
620
+ Tables can also be read from `.csv` files or files containing `org-mode` tables.
621
+
622
+ In the case of org-mode files, `FatTable` skips through the file until it finds a line that look like a table, that is, it begins with any number of spaces followed by `|-`. Only the first table in an `.org` file is read.
623
+
624
+ For both `.csv` and `.org` files, the first row in the table is taken as the header row, and the headers are converted to symbols as described above.
625
+
626
+ ```ruby
627
+ tab1 = FatTable.from_csv_file('~/data.csv')
628
+ tab2 = FatTable.from_org_file('~/project.org')
629
+
630
+ csv_body = <<-EOS
631
+ Ref,Date,Code,RawShares,Shares,Price,Info
632
+ 1,2006-05-02,P,5000,5000,8.6000,2006-08-09-1-I
633
+ 2,2006-05-03,P,5000,5000,8.4200,2006-08-09-1-I
634
+ 3,2006-05-04,P,5000,5000,8.4000,2006-08-09-1-I
635
+ 4,2006-05-10,P,8600,8600,8.0200,2006-08-09-1-D
636
+ 5,2006-05-12,P,10000,10000,7.2500,2006-08-09-1-D
637
+ 6,2006-05-12,P,2000,2000,6.7400,2006-08-09-1-I
638
+ EOS
639
+
640
+ tab3 = FatTable.from_csv_string(csv_body)
641
+
642
+ org_body = <<-EOS
643
+ .* Smith Transactions
644
+ :PROPERTIES:
645
+ :TABLE_EXPORT_FILE: smith.csv
646
+ :END:
647
+
648
+ #+TBLNAME: smith_tab
649
+ | Ref | Date | Code | Raw | Shares | Price | Info |
650
+ |-----+------------+------+---------+--------+----------+---------|
651
+ | 29 | 2013-05-02 | P | 795,546 | 2,609 | 1.18500 | ENTITY1 |
652
+ | 30 | 2013-05-02 | P | 118,186 | 388 | 11.85000 | ENTITY1 |
653
+ | 31 | 2013-05-02 | P | 340,948 | 1,926 | 1.18500 | ENTITY2 |
654
+ | 32 | 2013-05-02 | P | 50,651 | 286 | 11.85000 | ENTITY2 |
655
+ | 33 | 2013-05-20 | S | 12,000 | 32 | 28.28040 | ENTITY3 |
656
+ | 34 | 2013-05-20 | S | 85,000 | 226 | 28.32240 | ENTITY3 |
657
+ | 35 | 2013-05-20 | S | 33,302 | 88 | 28.63830 | ENTITY3 |
658
+ | 36 | 2013-05-23 | S | 8,000 | 21 | 27.10830 | ENTITY3 |
659
+ | 37 | 2013-05-23 | S | 23,054 | 61 | 26.80150 | ENTITY3 |
660
+ | 38 | 2013-05-23 | S | 39,906 | 106 | 25.17490 | ENTITY3 |
661
+ | 39 | 2013-05-29 | S | 13,459 | 36 | 24.74640 | ENTITY3 |
662
+ | 40 | 2013-05-29 | S | 15,700 | 42 | 24.77900 | ENTITY3 |
663
+ | 41 | 2013-05-29 | S | 15,900 | 42 | 24.58020 | ENTITY3 |
664
+ | 42 | 2013-05-30 | S | 6,679 | 18 | 25.04710 | ENTITY3 |
665
+
666
+ .* Another Heading
667
+ EOS
668
+
669
+ tab4 = FatTable.from_org_string(org_body)
670
+ ```
671
+
672
+
673
+ <a id="org8fddae3"></a>
674
+
675
+ ### From Arrays of Arrays
676
+
677
+ 1. In Ruby Code
678
+
679
+ You can also initialize a table directly from ruby data structures. You can, for example, build a table from an array of arrays. Remember that you can make any column tolerant with a keyword argument for the column symbol and ending it with a '~'.
680
+
681
+ ```ruby
682
+ aoa = [
683
+ ['Ref', 'Date', 'Code', 'Raw', 'Shares', 'Price', 'Info', 'Bool'],
684
+ [1, '2013-05-02', 'P', 795_546.20, 795_546.2, 1.1850, 'ENTITY1', 'T'],
685
+ [2, '2013-05-02', 'P', '118_186.40', 118_186.4, 11.8500, 'ENTITY1', 'T'],
686
+ [7, '2013-05-20', 'S', 12_000.00, 5046.00, 28.2804, 'ENTITY3', 'F'],
687
+ [8, '2013-05-20', 'S', 85_000.00, 35_742.50, 28.3224, 'ENTITY3', 'T'],
688
+ [9, '2013-05-20', 'S', 33_302.00, 14_003.49, 28.6383, 'ENTITY3', 'T'],
689
+ [10, Date.parse('2013-05-23'), 'S', 8000.00, 3364.00, 27.1083, 'ENTITY3', true],
690
+ [11, '2013-05-23', 'S', 23_054.00, 9694.21, 26.8015, 'ENTITY3', 'F'],
691
+ ['12', '2013-05-23', 'S', 39_906.00, 16_780.47, 25.1749, 'ENTITY3', 'T'],
692
+ [13, '2013-05-29', 'S', 13_459.00, 5659.51, 24.7464, 'ENTITY3', 'T'],
693
+ [14, '2013-05-29', 'S', 15_700.00, 6601.85, 24.7790, 'ENTITY3', false],
694
+ [15, '2013-05-29', 'S', 15_900.00, 6685.95, 24.5802, 'ENTITY3', 'T'],
695
+ [16, '2013-05-30', 'S', 6_679.00, 2808.52, 25.0471, 'ENTITY3', 'T'] ]
696
+
697
+ tab = FatTable.from_aoa(aoa).to_aoa
698
+ ```
699
+
700
+ ```
701
+ | Ref | Date | Code | Raw | Shares | Price | Info | Bool |
702
+ |-----+------------+------+-------------+-------------+---------+---------+------|
703
+ | 1 | 2013-05-02 | P | 795546.2000 | 795546.2000 | 1.1850 | ENTITY1 | T |
704
+ | 2 | 2013-05-02 | P | 118186.4000 | 118186.4000 | 11.8500 | ENTITY1 | T |
705
+ | 7 | 2013-05-20 | S | 12000.0000 | 5046.0000 | 28.2804 | ENTITY3 | F |
706
+ | 8 | 2013-05-20 | S | 85000.0000 | 35742.5000 | 28.3224 | ENTITY3 | T |
707
+ | 9 | 2013-05-20 | S | 33302.0000 | 14003.4900 | 28.6383 | ENTITY3 | T |
708
+ | 10 | 2013-05-23 | S | 8000.0000 | 3364.0000 | 27.1083 | ENTITY3 | T |
709
+ | 11 | 2013-05-23 | S | 23054.0000 | 9694.2100 | 26.8015 | ENTITY3 | F |
710
+ | 12 | 2013-05-23 | S | 39906.0000 | 16780.4700 | 25.1749 | ENTITY3 | T |
711
+ | 13 | 2013-05-29 | S | 13459.0000 | 5659.5100 | 24.7464 | ENTITY3 | T |
712
+ | 14 | 2013-05-29 | S | 15700.0000 | 6601.8500 | 24.7790 | ENTITY3 | F |
713
+ | 15 | 2013-05-29 | S | 15900.0000 | 6685.9500 | 24.5802 | ENTITY3 | T |
714
+ | 16 | 2013-05-30 | S | 6679.0000 | 2808.5200 | 25.0471 | ENTITY3 | T |
715
+ ```
716
+
717
+ Notice that the values can either be ruby objects, such as the Integer `85_000`, or strings that can be parsed into one of the permissible column types.
718
+
719
+ 2. In Emacs Org Files
720
+
721
+ This method of building a table, `.from_aoa`, is particularly useful in dealing with Emacs org-mode code blocks. Tables in org-mode are passed to code blocks as arrays of arrays. Likewise, a result of a code block in the form of an array of arrays is displayed as an org-mode table:
722
+
723
+ ```
724
+ #+NAME: trades1
725
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | LP | QP | IPLP | IPQP |
726
+ |------+------------+------+--------+-----+------+--------+-------+--------+--------+--------|
727
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
728
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
729
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
730
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
731
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
732
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
733
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
734
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
735
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
736
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
737
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
738
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
739
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
740
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
741
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
742
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
743
+
744
+ #+HEADER: :colnames no
745
+ :#+BEGIN_SRC ruby :var tab=trades1
746
+ require 'fat_table'
747
+ tab = FatTable.from_aoa(tab).where('shares > 500')
748
+ tab.to_aoa
749
+ :#+END_SRC
750
+
751
+ #+RESULTS:
752
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
753
+ |------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
754
+ | T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
755
+ | T004 | 2016-11-01 | S | 7.55 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
756
+ | T005 | 2016-11-01 | S | 7.5 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
757
+ | T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
758
+ | T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
759
+ | T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
760
+ | T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
761
+ | T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
762
+ | T013 | 2016-11-02 | P | 7.35 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
763
+ | T014 | 2016-11-02 | P | 7.45 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
764
+ ```
765
+
766
+ This example illustrates several things:
767
+
768
+ 1. The named org-mode table, `trades1`, can be passed into a ruby code block using the `:var tab=trades1` header argument to the code block; that makes the variable `tab` available to the code block as an array of arrays, which `FatTable` then uses to initialize the table.
769
+ 2. The code block requires that you set `:colnames no` in the header arguments. This suppresses org-mode's own processing of the header line so that `FatTable` can see the headers. Failure to do this will cause an error.
770
+ 3. The table is subjected to some processing, in this case selecting those rows where the number of shares is greater than 500. More on that later.
771
+ 4. `FatTable` passes back to org-mode an array of arrays using the `.to_aoa` method. In an `org-mode` buffer, these are rendered as tables. We'll often apply `.to_aoa` at the end of example blocks in this `README` to render the results as a table inside this file. As we'll see below, `.to_aoa` can also take a block to which formatting and footer directives can be attached.
772
+
773
+
774
+ <a id="org2079ffb"></a>
775
+
776
+ ### From Arrays of Hashes
777
+
778
+ A second ruby data structure that can be used to initialize a `FatTable` table is an array of ruby Hashes. Each hash represents a row of the table, and the headers of the table are taken from the keys of the hashes. Accordingly, all the hashes must have the same keys.
779
+
780
+ This same method can in fact take an array of any objects that can be converted to a Hash with the `#to_h` method, so you can use an array of your own objects to initialize a table, provided that you define a suitable `#to_h` method for the objects' class.
781
+
782
+ ```ruby
783
+ aoh = [
784
+ { ref: 'T001', date: '2016-11-01', code: 'P', price: '7.7000', shares: 100 },
785
+ { ref: 'T002', date: '2016-11-01', code: 'P', price: 7.7500, shares: 200 },
786
+ { ref: 'T003', date: '2016-11-01', code: 'P', price: 7.5000, shares: 800 },
787
+ { ref: 'T004', date: '2016-11-01', code: 'S', price: 7.5500, shares: 6811 },
788
+ { ref: 'T005', date: Date.today, code: 'S', price: 7.5000, shares: 4000 },
789
+ { ref: 'T006', date: '2016-11-01', code: 'S', price: 7.6000, shares: 1000 },
790
+ { ref: 'T007', date: '2016-11-01', code: 'S', price: 7.6500, shares: 200 },
791
+ { ref: 'T008', date: '2016-11-01', code: 'P', price: 7.6500, shares: 2771 },
792
+ { ref: 'T009', date: '2016-11-01', code: 'P', price: 7.6000, shares: 9550 },
793
+ { ref: 'T010', date: '2016-11-01', code: 'P', price: 7.5500, shares: 3175 },
794
+ { ref: 'T011', date: '2016-11-02', code: 'P', price: 7.4250, shares: 100 },
795
+ { ref: 'T012', date: '2016-11-02', code: 'P', price: 7.5500, shares: 4700 },
796
+ { ref: 'T013', date: '2016-11-02', code: 'P', price: 7.3500, shares: 53100 },
797
+ { ref: 'T014', date: '2016-11-02', code: 'P', price: 7.4500, shares: 5847 },
798
+ { ref: 'T015', date: '2016-11-02', code: 'P', price: 7.7500, shares: 500 },
799
+ { ref: 'T016', date: '2016-11-02', code: 'P', price: 8.2500, shares: 100 }
800
+ ]
801
+ tab = FatTable.from_aoh(aoh).to_aoa
802
+ ```
803
+
804
+ ```
805
+ | Ref | Date | Code | Price | Shares |
806
+ |------+------------+------+--------+--------|
807
+ | T001 | 2016-11-01 | P | 7.7000 | 100 |
808
+ | T002 | 2016-11-01 | P | 7.7500 | 200 |
809
+ | T003 | 2016-11-01 | P | 7.5000 | 800 |
810
+ | T004 | 2016-11-01 | S | 7.5500 | 6811 |
811
+ | T005 | 2025-12-28 | S | 7.5000 | 4000 |
812
+ | T006 | 2016-11-01 | S | 7.6000 | 1000 |
813
+ | T007 | 2016-11-01 | S | 7.6500 | 200 |
814
+ | T008 | 2016-11-01 | P | 7.6500 | 2771 |
815
+ | T009 | 2016-11-01 | P | 7.6000 | 9550 |
816
+ | T010 | 2016-11-01 | P | 7.5500 | 3175 |
817
+ | T011 | 2016-11-02 | P | 7.4250 | 100 |
818
+ | T012 | 2016-11-02 | P | 7.5500 | 4700 |
819
+ | T013 | 2016-11-02 | P | 7.3500 | 53100 |
820
+ | T014 | 2016-11-02 | P | 7.4500 | 5847 |
821
+ | T015 | 2016-11-02 | P | 7.7500 | 500 |
822
+ | T016 | 2016-11-02 | P | 8.2500 | 100 |
823
+ ```
824
+
825
+ Notice, again, that the values can either be ruby objects, such as `Date.today`, or strings that can be parsed into one of the permissible column types.
826
+
827
+
828
+ <a id="org1f3b4b6"></a>
829
+
830
+ ### From SQL queries
831
+
832
+ Another way to initialize a `FatTable` table is with the results of a SQL query. Before you can connect to a database, you need to make sure that the required adapter for your database is installed. `FatTable` uses the `sequel` gem under the hood, so any database that it supports can be used. For example, if you are accessing a Postgres database, you must install the `pg` gem with
833
+
834
+ ```sh
835
+ $ gem install pg
836
+ ```
837
+
838
+ You must first call `FatTable.connect` to set the database parameters to be used for the queries.
839
+
840
+ The arguments to `connect` are simply passed on to `sequel`'s connect method, so any set of arguments that work for it should work for `connect`. Alternatively, you can build the `Sequel` connection directly with `Sequel.connect` or with adapter-specific `Sequel` connection methods and let `FatTable` know to use that connection:
841
+
842
+ ```ruby
843
+ # This automatically requires sequel.
844
+ FatTable.connect(adapter: 'sqlite',
845
+ database: 'spec/example_files/trades2.db')
846
+ tab = FatTable.from_sql('select * from trans;').to_text
847
+ ```
848
+
849
+ ```
850
+ +============+======+=============+=============+=========+=========+====+
851
+ | Date | Code | Raw | Shares | Price | Info | Ok |
852
+ +------------+------+-------------+-------------+---------+---------+----+
853
+ | 2013-05-29 | S | 15700.0000 | 6601.8500 | 24.7790 | ENTITY3 | F |
854
+ | 2013-05-02 | P | 118186.4000 | 118186.4000 | 11.8500 | ENTITY1 | T |
855
+ | 2013-05-20 | S | 12000.0000 | 5046.0000 | 28.2804 | ENTITY3 | F |
856
+ | 2013-05-23 | S | 8000.0000 | 3364.0000 | 27.1083 | ENTITY3 | T |
857
+ | 2013-05-23 | S | 39906.0000 | 16780.4700 | 25.1749 | ENTITY3 | T |
858
+ | 2013-05-20 | S | 85000.0000 | 35742.5000 | 28.3224 | ENTITY3 | T |
859
+ | 2013-05-02 | P | 795546.2000 | 795546.2000 | 1.1850 | ENTITY1 | T |
860
+ | 2013-05-29 | S | 13459.0000 | 5659.5100 | 24.7464 | ENTITY3 | T |
861
+ | 2013-05-20 | S | 33302.0000 | 14003.4900 | 28.6383 | ENTITY3 | T |
862
+ | 2013-05-29 | S | 15900.0000 | 6685.9500 | 24.5802 | ENTITY3 | T |
863
+ | 2013-05-30 | S | 6679.0000 | 2808.5200 | 25.0471 | ENTITY3 | T |
864
+ | 2013-05-23 | S | 23054.0000 | 9694.2100 | 26.8015 | ENTITY3 | F |
865
+ +============+======+=============+=============+=========+=========+====+
866
+ ```
867
+
868
+ The same connection can be queries multiple times:
869
+
870
+ ```ruby
871
+ tab = FatTable.from_sql('select * from trans where info = "ENTITY1";')
872
+ tab.to_text
873
+ ```
874
+
875
+ ```
876
+ +============+======+=============+=============+=========+=========+====+
877
+ | Date | Code | Raw | Shares | Price | Info | Ok |
878
+ +------------+------+-------------+-------------+---------+---------+----+
879
+ | 2013-05-02 | P | 118186.4000 | 118186.4000 | 11.8500 | ENTITY1 | T |
880
+ | 2013-05-02 | P | 795546.2000 | 795546.2000 | 1.1850 | ENTITY1 | T |
881
+ +============+======+=============+=============+=========+=========+====+
882
+ ```
883
+
884
+ ```ruby
885
+ FatTable.db = Sequel.connect('postgres://user:password@localhost/dbname')
886
+ FatTable.db = Sequel.ado(conn_string: 'Provider=Microsoft.ACE.OLEDB.12.0;Data Source=drive:\path\filename.accdb')
887
+ ```
888
+
889
+ Consult `Sequel's` documentation for details on its connection methods. <http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html>
890
+
891
+ The `.connect` function need only be called once, and the database handle it creates will be used for all subsequent `.from_sql` calls until `.connect` is called again.
892
+
893
+
894
+ <a id="org4f6203d"></a>
895
+
896
+ ### Marking Groups in Input
897
+
898
+ 1. Manually
899
+
900
+ At any point, you can add a boundary to a table by invokong the `mark_boundary` method. Without an argument, it adds the boundary to the end of the table; with a numeric argument, `n`, it adds the boundary after row `n`.
901
+
902
+ 2. When Reading in Tables
903
+
904
+ `FatTable` tables has a concept of "groups" of rows that play a role in many of the methods for operating on them as explained [below](#orgc73107b).
905
+
906
+ The `.from_aoa` and `.from_aoh` functions take an optional keyword parameter `hlines:` that, if set to `true`, causes them to mark group boundaries in the table wherever a row Array (for `.from_aoa`) or Hash (for `.from_aoh`) is followed by a `nil`. Each boundary means that the rows above it and after the header or prior group boundary all belong to a group. By default `hlines` is false for both functions so neither expects hlines in its input.
907
+
908
+ In the case of `.from_aoa`, if `hlines:` is set true, the input must also include a `nil` in the second element of the outer array to indicate that the first row is to be used as headers. Otherwise, it will synthesize headers of the form `:col_1`, `:col_2`, &#x2026; `:col_n`.
909
+
910
+ In org mode table text passed to `.from_org_file` and `.from_org_string`, you *must* mark the header row by following it with an hrule and you *may* mark group boundaries with an hrule. In org mode tables, hlines are table rows beginning with something like `|---`. The `.from_org_...` functions always recognizes hlines in the input, so it takes no `hlines:` keyword parameter.
911
+
912
+
913
+ <a id="orge15c95e"></a>
914
+
915
+ ## Accessing Parts of Tables
916
+
917
+
918
+ <a id="org298a474"></a>
919
+
920
+ ### Rows
921
+
922
+ A `FatTable` table is an Enumerable, yielding each row of the table as a Hash keyed on the header symbols. The method `Table#rows` returns an Array of the rows as Hashes as well.
923
+
924
+ You can also use indexing to access a row of the table by number. Using an integer index returns a Hash of the given row. Thus, `tab[20]` returns the 21st data row of the table, while `tab[0]` returns the first row and tab[-1] returns the last row.
925
+
926
+
927
+ <a id="org5782179"></a>
928
+
929
+ ### Columns
930
+
931
+ If the index provided to `[]` is a string or a symbol, it returns an Array of the items of the column with that header. Thus, `tab[:ref]` returns an Array of all the items of the table's `:ref` column.
932
+
933
+
934
+ <a id="org326ae38"></a>
935
+
936
+ ### Cells
937
+
938
+ The two forms of indexing can be combined, in either order, to access individual cells of the table:
939
+
940
+ ```ruby
941
+ tab[13] # => Hash of the 14th row
942
+ tab[:date] # => Array of all Dates in the :date column
943
+ tab[13][:date] # => The Date in the 14th row
944
+ tab[:date][13] # => The Date in the 14th row; indexes can be in either order.
945
+ ```
946
+
947
+
948
+ <a id="orgc525931"></a>
949
+
950
+ ### Other table attributes
951
+
952
+ Here is a quick rundown of other table attributes that you can access:
953
+
954
+ ```ruby
955
+ tab.headers # => an Array of the headers in symbol form
956
+ tab.types # => a Hash mapping headers to column types
957
+ tab.type(head) # => return the type of the column for the given head
958
+ tab.size # => the number of rows in the table
959
+ tab.width # => the number of columns in the table
960
+ tab.empty? # => is the table empty?
961
+ tab.column(head) # => return the FatTable::Column object for the given head
962
+ tab.column?(head) # => does the table have a column with the given head?
963
+ tab.groups # => return an Array of the table's groups as Arrays of row Hashes.
964
+ ```
965
+
966
+ You should note that what the `.types` and `.type(head)` methods return is a string naming the "type" assigned by `FatTable`. All of them are also the names of Ruby classes except for 'Boolean' a class that doesn't exist in Ruby. The value `true` is a member of the `TrueClass` and `false` a member of the `FalseClass`. So for `FatTable` to provide a column of type 'Boolean' requires it to synthesize the type from these Ruby classes.
967
+
968
+ ```ruby
969
+ tab.types
970
+ ```
971
+
972
+ ```
973
+ {date: "DateTime", code: "String", raw: "Numeric", shares: "Numeric", price: "Numeric", info: "String", ok: "Boolean"}
974
+ ```
975
+
976
+ ```ruby
977
+ puts "Column :shares says its type is '#{tab.type(:shares)}' and that is a #{tab[:shares][0].class}"
978
+ ```
979
+
980
+ ```
981
+ => false
982
+ Column :shares says its type is 'Numeric' and that is a BigDecimal
983
+ => nil
984
+ :org_babel_ruby_eoe
985
+ ```
986
+
987
+
988
+ <a id="orgf11c014"></a>
989
+
990
+ ## Operations on Tables
991
+
992
+ Once you have one or more tables, you will likely want to perform operations on them. The operations provided by `FatTable` are the subject of this section. Before getting into the operations, though, there are a couple of issues that cut across all or many of the operations.
993
+
994
+ First, tables are by and large immutable objects. Each operation creates a new table without affecting the input tables. The only exceptions are the `degroup!` operation, which mutates the receiver table by removing its group boundaries, and `force_string!` (explained above at [Forcing String Type](#orgfb642e6)), which forces columns to have the String type despite what the automatic typing rules determine.
995
+
996
+ Second, because each operation returns a `FatTable::Table` object, the operations are chainable.
997
+
998
+ Third, `FatTable::Table` objects can have "groups" of rows within the table. These can be decorated with hlines and group footers on output. Some operations result in marking group boundaries in the result table, others remove group boundaries that may have existed in the input table. Operations that either create or remove groups will be noted below.
999
+
1000
+ Finally, the operations are for the most part patterned on SQL table operations, but when expressions play a role, you write them using ruby syntax rather than SQL.
1001
+
1002
+
1003
+ <a id="org4a978ae"></a>
1004
+
1005
+ ### Example Input Tables
1006
+
1007
+ For illustration purposes assume that the following tables are read into ruby variables called `tab1` and `tab2`. We have given the table groups, marked by the hlines below, and included some duplicate rows to illustrate the effect of certain operations on groups and duplicates.
1008
+
1009
+ ```ruby
1010
+ tab1_str = <<-EOS
1011
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | LP | QP | IPLP | IPQP |
1012
+ |------+------------------+------+--------+-----+------+--------+------+-------+--------+--------|
1013
+ | T001 | [2016-11-01 Tue] | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1014
+ | T002 | [2016-11-01 Tue] | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1015
+ | T003 | [2016-11-01 Tue] | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1016
+ | T003 | [2016-11-01 Tue] | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1017
+ |------+------------------+------+--------+-----+------+--------+------+-------+--------+--------|
1018
+ | T004 | [2016-11-01 Tue] | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
1019
+ | T005 | [2016-11-01 Tue] | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
1020
+ | T006 | [2016-11-01 Tue] | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1021
+ | T006 | [2016-11-01 Tue] | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1022
+ | T007 | [2016-11-01 Tue] | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1023
+ | T008 | [2016-11-01 Tue] | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
1024
+ | T009 | [2016-11-01 Tue] | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
1025
+ |------+------------------+------+--------+-----+------+--------+------+-------+--------+--------|
1026
+ | T010 | [2016-11-01 Tue] | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
1027
+ | T011 | [2016-11-02 Wed] | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1028
+ | T012 | [2016-11-02 Wed] | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1029
+ | T012 | [2016-11-02 Wed] | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1030
+ | T013 | [2016-11-02 Wed] | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
1031
+ |------+------------------+------+--------+-----+------+--------+------+-------+--------+--------|
1032
+ | T014 | [2016-11-02 Wed] | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1033
+ | T015 | [2016-11-02 Wed] | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1034
+ | T016 | [2016-11-02 Wed] | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1035
+ EOS
1036
+
1037
+ tab2_str = <<-EOS
1038
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | LP | QP | IPLP | IPQP |
1039
+ |------+------------------+------+--------+-----+------+--------+-------+------+--------+--------|
1040
+ | T003 | [2016-11-01 Tue] | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1041
+ | T003 | [2016-11-01 Tue] | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1042
+ | T017 | [2016-11-01 Tue] | P | 8.3 | F | T | 1801 | 1201 | 600 | 0.2453 | 0.1924 |
1043
+ |------+------------------+------+--------+-----+------+--------+-------+------+--------+--------|
1044
+ | T018 | [2016-11-01 Tue] | S | 7.152 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1045
+ | T018 | [2016-11-01 Tue] | S | 7.152 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1046
+ | T006 | [2016-11-01 Tue] | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1047
+ | T007 | [2016-11-01 Tue] | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1048
+ |------+------------------+------+--------+-----+------+--------+-------+------+--------+--------|
1049
+ | T014 | [2016-11-02 Wed] | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1050
+ | T015 | [2016-11-02 Wed] | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1051
+ | T015 | [2016-11-02 Wed] | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1052
+ | T016 | [2016-11-02 Wed] | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1053
+ |------+------------------+------+--------+-----+------+--------+-------+------+--------+--------|
1054
+ | T019 | [2017-01-15 Sun] | S | 8.75 | T | F | 300 | 175 | 125 | 0.2453 | 0.1924 |
1055
+ | T020 | [2017-01-19 Thu] | S | 8.25 | F | T | 700 | 615 | 85 | 0.2453 | 0.1924 |
1056
+ | T021 | [2017-01-23 Mon] | P | 7.16 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1057
+ | T021 | [2017-01-23 Mon] | P | 7.16 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1058
+ EOS
1059
+ ```
1060
+
1061
+ Rendering `tab1` into Emacs org-mode:
1062
+
1063
+ ```ruby
1064
+ tab1 = FatTable.from_org_string(tab1_str)
1065
+ tab1.to_aoa
1066
+ ```
1067
+
1068
+ ```
1069
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
1070
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1071
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1072
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1073
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1074
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1075
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1076
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
1077
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
1078
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1079
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1080
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1081
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
1082
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
1083
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1084
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
1085
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1086
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1087
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1088
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
1089
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1090
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1091
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1092
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1093
+ ```
1094
+
1095
+ Rendering `tab2` into Emacs org-mode:
1096
+
1097
+ ```ruby
1098
+ tab2 = FatTable.from_org_string(tab2_str)
1099
+ tab2.to_aoa
1100
+ ```
1101
+
1102
+ ```
1103
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
1104
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
1105
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1106
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1107
+ | T017 | 2016-11-01 | P | 8.3000 | F | T | 1801 | 1201 | 600 | 0.2453 | 0.1924 |
1108
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
1109
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1110
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1111
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1112
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1113
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
1114
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1115
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1116
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1117
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1118
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
1119
+ | T019 | 2017-01-15 | S | 8.7500 | T | F | 300 | 175 | 125 | 0.2453 | 0.1924 |
1120
+ | T020 | 2017-01-19 | S | 8.2500 | F | T | 700 | 615 | 85 | 0.2453 | 0.1924 |
1121
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1122
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1123
+ ```
1124
+
1125
+
1126
+ <a id="org3f88455"></a>
1127
+
1128
+ ### Select
1129
+
1130
+ With the `select` method, you can select columns to appear in the output table, rearrange their order, and create new columns that are a function of other columns.
1131
+
1132
+ 1. Selecting Existing Columns (Also of :omni)
1133
+
1134
+ Here we select three existing columns by simply passing header symbols in the order we want them to appear in the output. Thus, one use of `select` is to filter and permute the order of existing columns. The `select` method preserves any group boundaries present in the input table.
1135
+
1136
+ ```ruby
1137
+ tab1.select(:price, :ref, :shares).to_aoa
1138
+ ```
1139
+
1140
+ ```
1141
+ | Price | Ref | Shares |
1142
+ |--------+------+--------|
1143
+ | 7.7000 | T001 | 100 |
1144
+ | 7.7500 | T002 | 200 |
1145
+ | 7.5000 | T003 | 800 |
1146
+ | 7.5000 | T003 | 800 |
1147
+ |--------+------+--------|
1148
+ | 7.5500 | T004 | 6811 |
1149
+ | 7.5000 | T005 | 4000 |
1150
+ | 7.6000 | T006 | 1000 |
1151
+ | 7.6000 | T006 | 1000 |
1152
+ | 7.6500 | T007 | 200 |
1153
+ | 7.6500 | T008 | 2771 |
1154
+ | 7.6000 | T009 | 9550 |
1155
+ |--------+------+--------|
1156
+ | 7.5500 | T010 | 3175 |
1157
+ | 7.4250 | T011 | 100 |
1158
+ | 7.5500 | T012 | 4700 |
1159
+ | 7.5500 | T012 | 4700 |
1160
+ | 7.3500 | T013 | 53100 |
1161
+ |--------+------+--------|
1162
+ | 7.4500 | T014 | 5847 |
1163
+ | 7.7500 | T015 | 500 |
1164
+ | 8.2500 | T016 | 100 |
1165
+ ```
1166
+
1167
+ It can be tedious to type the names of all the columns in a `select` statement, so `FatTable` recognizes the special column name `:omni`. If the `select`'s first and only column argument is `:omni`, it will expand to the names of all the existing columns in the table. Use of `:omni` otherwise is not interpreted specially, so you will get an error complaining about a non-existent column unless you happen to have a column named `:omni` in your table, which is not advisable. You can add hash arguments after `:omni` but you cannot add additional column names:
1168
+
1169
+ ```ruby
1170
+ tab1.select(:omni, cost: 'shares * price').to_aoa
1171
+ ```
1172
+
1173
+ ```
1174
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp | Cost |
1175
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1176
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 770.0000 |
1177
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1550.0000 |
1178
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0000 |
1179
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0000 |
1180
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1181
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 | 51423.0500 |
1182
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 | 30000.0000 |
1183
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0000 |
1184
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0000 |
1185
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1530.0000 |
1186
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 | 21198.1500 |
1187
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 | 72580.0000 |
1188
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1189
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 | 23971.2500 |
1190
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 742.5000 |
1191
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0000 |
1192
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0000 |
1193
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 | 390285.0000 |
1194
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1195
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 | 43560.1500 |
1196
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 | 3875.0000 |
1197
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 | 825.0000 |
1198
+ ```
1199
+
1200
+ 2. Copying and Renaming Existing Columns.
1201
+
1202
+ After the list of selected column names in the call to `select`, you can add any number of hash-like arguments. You can use these to add a copy of an existing column. By calling select again, you can include only the copied column, in effect renaming it. For example, if you want `tab1` but with `:ref` changed to `:id`, just add an argument to define the new `:id` column:
1203
+
1204
+ ```ruby
1205
+ tab1.select(:omni, id: :ref).
1206
+ select(:id, :date, :code, :price, :shares).to_aoa
1207
+ ```
1208
+
1209
+ ```
1210
+ | Id | Date | Code | Price | Shares |
1211
+ |------+------------+------+--------+--------|
1212
+ | T001 | 2016-11-01 | P | 7.7000 | 100 |
1213
+ | T002 | 2016-11-01 | P | 7.7500 | 200 |
1214
+ | T003 | 2016-11-01 | P | 7.5000 | 800 |
1215
+ | T003 | 2016-11-01 | P | 7.5000 | 800 |
1216
+ |------+------------+------+--------+--------|
1217
+ | T004 | 2016-11-01 | S | 7.5500 | 6811 |
1218
+ | T005 | 2016-11-01 | S | 7.5000 | 4000 |
1219
+ | T006 | 2016-11-01 | S | 7.6000 | 1000 |
1220
+ | T006 | 2016-11-01 | S | 7.6000 | 1000 |
1221
+ | T007 | 2016-11-01 | S | 7.6500 | 200 |
1222
+ | T008 | 2016-11-01 | P | 7.6500 | 2771 |
1223
+ | T009 | 2016-11-01 | P | 7.6000 | 9550 |
1224
+ |------+------------+------+--------+--------|
1225
+ | T010 | 2016-11-01 | P | 7.5500 | 3175 |
1226
+ | T011 | 2016-11-02 | P | 7.4250 | 100 |
1227
+ | T012 | 2016-11-02 | P | 7.5500 | 4700 |
1228
+ | T012 | 2016-11-02 | P | 7.5500 | 4700 |
1229
+ | T013 | 2016-11-02 | P | 7.3500 | 53100 |
1230
+ |------+------------+------+--------+--------|
1231
+ | T014 | 2016-11-02 | P | 7.4500 | 5847 |
1232
+ | T015 | 2016-11-02 | P | 7.7500 | 500 |
1233
+ | T016 | 2016-11-02 | P | 8.2500 | 100 |
1234
+ ```
1235
+
1236
+ 3. Adding New Computed Columns
1237
+
1238
+ More interesting is that `select` can take hash-like keyword arguments after the symbol arguments to create new columns in the output as functions of other columns. For each hash-like parameter, the keyword given must be a symbol, which becomes the header for the new column, and the value can be a string representing a ruby expression for the value of a new column.
1239
+
1240
+ Within the string expression, the names of existing or already-specified columns are available as local variables. In addition the instance variables '@row' and '@group' are available as the row number and group number of the new value. So for our example table, the string expressions for new columns have access to local variables `ref`, `date`, `code`, `price`, `g10`, `qp10`, `shares`, `lp`, `qp`, `iplp`, and `ipqp` as well as the instance variables `@row` and `@group`. The local variables are set to the values of the cell in their respective columns for each row in the input table, and the instance variables are set the number of the current row and group number respectively.
1241
+
1242
+ For example, if we want to rename the `traded_on` column to `:date` and add a new column to compute the cost of shares, we could do the following:
1243
+
1244
+ ```ruby
1245
+ tab1.select(:ref, :price, :shares, traded_on: :date, cost: 'price * shares').to_aoa
1246
+ ```
1247
+
1248
+ ```
1249
+ | Ref | Price | Shares | Traded On | Cost |
1250
+ |------+--------+--------+------------+-------------|
1251
+ | T001 | 7.7000 | 100 | 2016-11-01 | 770.0000 |
1252
+ | T002 | 7.7500 | 200 | 2016-11-01 | 1550.0000 |
1253
+ | T003 | 7.5000 | 800 | 2016-11-01 | 6000.0000 |
1254
+ | T003 | 7.5000 | 800 | 2016-11-01 | 6000.0000 |
1255
+ |------+--------+--------+------------+-------------|
1256
+ | T004 | 7.5500 | 6811 | 2016-11-01 | 51423.0500 |
1257
+ | T005 | 7.5000 | 4000 | 2016-11-01 | 30000.0000 |
1258
+ | T006 | 7.6000 | 1000 | 2016-11-01 | 7600.0000 |
1259
+ | T006 | 7.6000 | 1000 | 2016-11-01 | 7600.0000 |
1260
+ | T007 | 7.6500 | 200 | 2016-11-01 | 1530.0000 |
1261
+ | T008 | 7.6500 | 2771 | 2016-11-01 | 21198.1500 |
1262
+ | T009 | 7.6000 | 9550 | 2016-11-01 | 72580.0000 |
1263
+ |------+--------+--------+------------+-------------|
1264
+ | T010 | 7.5500 | 3175 | 2016-11-01 | 23971.2500 |
1265
+ | T011 | 7.4250 | 100 | 2016-11-02 | 742.5000 |
1266
+ | T012 | 7.5500 | 4700 | 2016-11-02 | 35485.0000 |
1267
+ | T012 | 7.5500 | 4700 | 2016-11-02 | 35485.0000 |
1268
+ | T013 | 7.3500 | 53100 | 2016-11-02 | 390285.0000 |
1269
+ |------+--------+--------+------------+-------------|
1270
+ | T014 | 7.4500 | 5847 | 2016-11-02 | 43560.1500 |
1271
+ | T015 | 7.7500 | 500 | 2016-11-02 | 3875.0000 |
1272
+ | T016 | 8.2500 | 100 | 2016-11-02 | 825.0000 |
1273
+ ```
1274
+
1275
+ The parameter `traded_on: :date` caused the `:date` column of the input table to be renamed `:traded_on`, and the parameter `cost: 'price * shares'` created a new computed column, `:cost`, as the product of values in the `:price` and `:shares` columns.
1276
+
1277
+ The order of the columns in the result tables is the same as the order of the parameters to the `select` method. So, you can re-order the columns with a second, chained call to `select`:
1278
+
1279
+ ```ruby
1280
+ tab1.select(:ref, :price, :shares, traded_on: :date, cost: 'price * shares').
1281
+ select(:ref, :traded_on, :price, :shares, :cost).to_aoa
1282
+ ```
1283
+
1284
+ ```
1285
+ | Ref | Traded On | Price | Shares | Cost |
1286
+ |------+------------+--------+--------+-------------|
1287
+ | T001 | 2016-11-01 | 7.7000 | 100 | 770.0000 |
1288
+ | T002 | 2016-11-01 | 7.7500 | 200 | 1550.0000 |
1289
+ | T003 | 2016-11-01 | 7.5000 | 800 | 6000.0000 |
1290
+ | T003 | 2016-11-01 | 7.5000 | 800 | 6000.0000 |
1291
+ |------+------------+--------+--------+-------------|
1292
+ | T004 | 2016-11-01 | 7.5500 | 6811 | 51423.0500 |
1293
+ | T005 | 2016-11-01 | 7.5000 | 4000 | 30000.0000 |
1294
+ | T006 | 2016-11-01 | 7.6000 | 1000 | 7600.0000 |
1295
+ | T006 | 2016-11-01 | 7.6000 | 1000 | 7600.0000 |
1296
+ | T007 | 2016-11-01 | 7.6500 | 200 | 1530.0000 |
1297
+ | T008 | 2016-11-01 | 7.6500 | 2771 | 21198.1500 |
1298
+ | T009 | 2016-11-01 | 7.6000 | 9550 | 72580.0000 |
1299
+ |------+------------+--------+--------+-------------|
1300
+ | T010 | 2016-11-01 | 7.5500 | 3175 | 23971.2500 |
1301
+ | T011 | 2016-11-02 | 7.4250 | 100 | 742.5000 |
1302
+ | T012 | 2016-11-02 | 7.5500 | 4700 | 35485.0000 |
1303
+ | T012 | 2016-11-02 | 7.5500 | 4700 | 35485.0000 |
1304
+ | T013 | 2016-11-02 | 7.3500 | 53100 | 390285.0000 |
1305
+ |------+------------+--------+--------+-------------|
1306
+ | T014 | 2016-11-02 | 7.4500 | 5847 | 43560.1500 |
1307
+ | T015 | 2016-11-02 | 7.7500 | 500 | 3875.0000 |
1308
+ | T016 | 2016-11-02 | 8.2500 | 100 | 825.0000 |
1309
+ ```
1310
+
1311
+ 4. Adding Constant Strings and Other Types in select
1312
+
1313
+ Because `select`'s hash-like parameters evaluate a string as a ruby expression, as just described, it must provide a way to set a new column to a string literal. To indicate that a string should be inserted literally, add a `:` as the first non-blank character in the string. This will supress evaluation and insert the remainder of the string in the named column.
1314
+
1315
+ ```ruby
1316
+ tab1.select(:ref, :price, :shares, traded_on: :date, cost: ':the price of freedom').
1317
+ select(:ref, :traded_on, :price, :shares, :cost).to_aoa
1318
+ ```
1319
+
1320
+ ```
1321
+ | Ref | Traded On | Price | Shares | Cost |
1322
+ |------+------------+--------+--------+----------------------|
1323
+ | T001 | 2016-11-01 | 7.7000 | 100 | the price of freedom |
1324
+ | T002 | 2016-11-01 | 7.7500 | 200 | the price of freedom |
1325
+ | T003 | 2016-11-01 | 7.5000 | 800 | the price of freedom |
1326
+ | T003 | 2016-11-01 | 7.5000 | 800 | the price of freedom |
1327
+ |------+------------+--------+--------+----------------------|
1328
+ | T004 | 2016-11-01 | 7.5500 | 6811 | the price of freedom |
1329
+ | T005 | 2016-11-01 | 7.5000 | 4000 | the price of freedom |
1330
+ | T006 | 2016-11-01 | 7.6000 | 1000 | the price of freedom |
1331
+ | T006 | 2016-11-01 | 7.6000 | 1000 | the price of freedom |
1332
+ | T007 | 2016-11-01 | 7.6500 | 200 | the price of freedom |
1333
+ | T008 | 2016-11-01 | 7.6500 | 2771 | the price of freedom |
1334
+ | T009 | 2016-11-01 | 7.6000 | 9550 | the price of freedom |
1335
+ |------+------------+--------+--------+----------------------|
1336
+ | T010 | 2016-11-01 | 7.5500 | 3175 | the price of freedom |
1337
+ | T011 | 2016-11-02 | 7.4250 | 100 | the price of freedom |
1338
+ | T012 | 2016-11-02 | 7.5500 | 4700 | the price of freedom |
1339
+ | T012 | 2016-11-02 | 7.5500 | 4700 | the price of freedom |
1340
+ | T013 | 2016-11-02 | 7.3500 | 53100 | the price of freedom |
1341
+ |------+------------+--------+--------+----------------------|
1342
+ | T014 | 2016-11-02 | 7.4500 | 5847 | the price of freedom |
1343
+ | T015 | 2016-11-02 | 7.7500 | 500 | the price of freedom |
1344
+ | T016 | 2016-11-02 | 8.2500 | 100 | the price of freedom |
1345
+ ```
1346
+
1347
+ This sets the `:cost` column to the string constant 'the price of freedom' for the whole table.
1348
+
1349
+ You can set a column to a constant of any of the acceptable types, `Numeric`, `Date`, `DateTime`, true, false, or nil.
1350
+
1351
+ ```ruby
1352
+ tab1.select(:ref, :price, :shares, traded_on: :date, cost: Math::PI, today: Date.today).
1353
+ select(:ref, :traded_on, :price, :shares, :cost, :today).to_aoa
1354
+ ```
1355
+
1356
+ ```
1357
+ | Ref | Traded On | Price | Shares | Cost | Today |
1358
+ |------+------------+--------+--------+--------+------------|
1359
+ | T001 | 2016-11-01 | 7.7000 | 100 | 3.1416 | 2025-12-28 |
1360
+ | T002 | 2016-11-01 | 7.7500 | 200 | 3.1416 | 2025-12-28 |
1361
+ | T003 | 2016-11-01 | 7.5000 | 800 | 3.1416 | 2025-12-28 |
1362
+ | T003 | 2016-11-01 | 7.5000 | 800 | 3.1416 | 2025-12-28 |
1363
+ |------+------------+--------+--------+--------+------------|
1364
+ | T004 | 2016-11-01 | 7.5500 | 6811 | 3.1416 | 2025-12-28 |
1365
+ | T005 | 2016-11-01 | 7.5000 | 4000 | 3.1416 | 2025-12-28 |
1366
+ | T006 | 2016-11-01 | 7.6000 | 1000 | 3.1416 | 2025-12-28 |
1367
+ | T006 | 2016-11-01 | 7.6000 | 1000 | 3.1416 | 2025-12-28 |
1368
+ | T007 | 2016-11-01 | 7.6500 | 200 | 3.1416 | 2025-12-28 |
1369
+ | T008 | 2016-11-01 | 7.6500 | 2771 | 3.1416 | 2025-12-28 |
1370
+ | T009 | 2016-11-01 | 7.6000 | 9550 | 3.1416 | 2025-12-28 |
1371
+ |------+------------+--------+--------+--------+------------|
1372
+ | T010 | 2016-11-01 | 7.5500 | 3175 | 3.1416 | 2025-12-28 |
1373
+ | T011 | 2016-11-02 | 7.4250 | 100 | 3.1416 | 2025-12-28 |
1374
+ | T012 | 2016-11-02 | 7.5500 | 4700 | 3.1416 | 2025-12-28 |
1375
+ | T012 | 2016-11-02 | 7.5500 | 4700 | 3.1416 | 2025-12-28 |
1376
+ | T013 | 2016-11-02 | 7.3500 | 53100 | 3.1416 | 2025-12-28 |
1377
+ |------+------------+--------+--------+--------+------------|
1378
+ | T014 | 2016-11-02 | 7.4500 | 5847 | 3.1416 | 2025-12-28 |
1379
+ | T015 | 2016-11-02 | 7.7500 | 500 | 3.1416 | 2025-12-28 |
1380
+ | T016 | 2016-11-02 | 8.2500 | 100 | 3.1416 | 2025-12-28 |
1381
+ ```
1382
+
1383
+ 5. Custom Instance Variables and Hooks
1384
+
1385
+ As the above examples demonstrate, the instance variables `@row` and `@group` are available when evaluating expressions that add new columns. You can also set up your own instance variables as well for keeping track of things that cross row boundaries, such as running sums.
1386
+
1387
+ To declare instance variables, you can use the `ivars:` hash parameter to `select`. Each key of the hash becomes an instance variable and each value becomes its initial value before any rows are evaluated.
1388
+
1389
+ In addition, you can provide `before_hook:` and `after_hook:` parameters to `select` as strings that are evaluated as ruby expressions before and after each row is processed. You can use these to update instance variables. The values set in the `before_hook:` can be used in expressions for adding new columns by referencing them with the '@' prefix.
1390
+
1391
+ For example, suppose we wanted to not only add a cost column, but a column that shows the cumulative cost after each transaction in our example table. The following example uses the `ivars:` and `before_hook:` parameters to keep track of the running cost of shares, then formats the table.
1392
+
1393
+ ```ruby
1394
+ tab = tab1.select(:ref, :price, :shares, traded_on: :date,
1395
+ cost: 'price * shares', cumulative: '@total_cost',
1396
+ ivars: { total_cost: 0 },
1397
+ before_hook: '@total_cost += price * shares')
1398
+ FatTable.to_aoa(tab) do |f|
1399
+ f.format(price: '0.4', shares: '0.0,', cost: '0.2,', cumulative: '0.2,')
1400
+ end
1401
+ ```
1402
+
1403
+ ```
1404
+ | Ref | Price | Shares | Traded On | Cost | Cumulative |
1405
+ |------+--------+--------+------------+------------+------------|
1406
+ | T001 | 7.7000 | 100 | 2016-11-01 | 770.00 | 770.00 |
1407
+ | T002 | 7.7500 | 200 | 2016-11-01 | 1,550.00 | 2,320.00 |
1408
+ | T003 | 7.5000 | 800 | 2016-11-01 | 6,000.00 | 8,320.00 |
1409
+ | T003 | 7.5000 | 800 | 2016-11-01 | 6,000.00 | 14,320.00 |
1410
+ |------+--------+--------+------------+------------+------------|
1411
+ | T004 | 7.5500 | 6,811 | 2016-11-01 | 51,423.05 | 65,743.05 |
1412
+ | T005 | 7.5000 | 4,000 | 2016-11-01 | 30,000.00 | 95,743.05 |
1413
+ | T006 | 7.6000 | 1,000 | 2016-11-01 | 7,600.00 | 103,343.05 |
1414
+ | T006 | 7.6000 | 1,000 | 2016-11-01 | 7,600.00 | 110,943.05 |
1415
+ | T007 | 7.6500 | 200 | 2016-11-01 | 1,530.00 | 112,473.05 |
1416
+ | T008 | 7.6500 | 2,771 | 2016-11-01 | 21,198.15 | 133,671.20 |
1417
+ | T009 | 7.6000 | 9,550 | 2016-11-01 | 72,580.00 | 206,251.20 |
1418
+ |------+--------+--------+------------+------------+------------|
1419
+ | T010 | 7.5500 | 3,175 | 2016-11-01 | 23,971.25 | 230,222.45 |
1420
+ | T011 | 7.4250 | 100 | 2016-11-02 | 742.50 | 230,964.95 |
1421
+ | T012 | 7.5500 | 4,700 | 2016-11-02 | 35,485.00 | 266,449.95 |
1422
+ | T012 | 7.5500 | 4,700 | 2016-11-02 | 35,485.00 | 301,934.95 |
1423
+ | T013 | 7.3500 | 53,100 | 2016-11-02 | 390,285.00 | 692,219.95 |
1424
+ |------+--------+--------+------------+------------+------------|
1425
+ | T014 | 7.4500 | 5,847 | 2016-11-02 | 43,560.15 | 735,780.10 |
1426
+ | T015 | 7.7500 | 500 | 2016-11-02 | 3,875.00 | 739,655.10 |
1427
+ | T016 | 8.2500 | 100 | 2016-11-02 | 825.00 | 740,480.10 |
1428
+ ```
1429
+
1430
+ 6. Argument Order and Boundaries
1431
+
1432
+ Notice that `select` can take any number of arguments but all the symbol arguments must come first followed by all the hash-like keyword arguments, including the special arguments for instance variables and hooks.
1433
+
1434
+ As the example illustrates, `.select` transmits any group boundaries in its input table to the result table.
1435
+
1436
+
1437
+ <a id="orgb70a9cb"></a>
1438
+
1439
+ ### Where
1440
+
1441
+ You can filter the rows of the result table with the `.where` method. It takes a single string expression as an argument which is evaluated in a manner similar to `.select` in which the value of the cells in each column are available as local variables and the instance variables `@row` and `@group` are available for testing. The expression is evaluated for each row, and if the expression evaluates to a truthy value, the row is included in the output, otherwise it is not.
1442
+
1443
+ The `.where` method removes any group boundaries in the input, so the output table has only a single group.
1444
+
1445
+ Here we select only those even-numbered rows where either of the two boolean fields is true:
1446
+
1447
+ ```ruby
1448
+ tab1.where('@row.even? && (g10 || qp10)').to_term
1449
+ ```
1450
+
1451
+ ```
1452
+ ╒══════╤════════════╤══════╤════════╤═════╤══════╤════════╤══════╤═══════╤════════╤════════╕
1453
+ │ Ref │ Date │ Code │ Price │ G10 │ QP10 │ Shares │ Lp │ Qp │ Iplp │ Ipqp │
1454
+ ├──────┼────────────┼──────┼────────┼─────┼──────┼────────┼──────┼───────┼────────┼────────┤
1455
+ │ T002 │ 2016-11-01 │ P │ 7.7500 │ T │ F │ 200 │ 28 │ 172 │ 0.2453 │ 0.1924 │
1456
+ │ T003 │ 2016-11-01 │ P │ 7.5000 │ F │ T │ 800 │ 112 │ 688 │ 0.2453 │ 0.1924 │
1457
+ │ T006 │ 2016-11-01 │ S │ 7.6000 │ F │ T │ 1000 │ 143 │ 857 │ 0.2453 │ 0.1924 │
1458
+ │ T010 │ 2016-11-01 │ P │ 7.5500 │ F │ T │ 3175 │ 451 │ 2724 │ 0.2453 │ 0.1924 │
1459
+ │ T013 │ 2016-11-02 │ P │ 7.3500 │ T │ T │ 53100 │ 7656 │ 45444 │ 0.2453 │ 0.1924 │
1460
+ ╘══════╧════════════╧══════╧════════╧═════╧══════╧════════╧══════╧═══════╧════════╧════════╛
1461
+ ```
1462
+
1463
+
1464
+ <a id="org54a98c1"></a>
1465
+
1466
+ ### Order\_by
1467
+
1468
+ You can sort a table on any number of columns with `order_by`. The `order_by` method takes any number of symbol arguments for the columns to sort on. If you specify more than one column, the sort is performed on the first column, then all columns that are equal with respect to the first column are sorted by the second column, and so on. Ordering is done is ascending order for each of the columns, but can be reversed by adding a '!' to the end a symbol argument. All columns of the input table are included in the output.
1469
+
1470
+ Let's sort our table first by `:code`, then in reverse order of `:date`.
1471
+
1472
+ ```ruby
1473
+ tab1.order_by(:code, :date!).to_aoa
1474
+ ```
1475
+
1476
+ ```
1477
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
1478
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1479
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1480
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1481
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1482
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
1483
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1484
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1485
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1486
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1487
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1488
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1489
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1490
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1491
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
1492
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
1493
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
1494
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1495
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
1496
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
1497
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1498
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1499
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1500
+ ```
1501
+
1502
+ The interesting thing about `order_by` is that, while it ignores groups in its input, it adds group boundaries in the output table at those rows where the sort keys change. Thus, in each group, `:code` and `:date` are the same, and when either changes, `order_by` inserts a group boundary.
1503
+
1504
+
1505
+ <a id="org6562f48"></a>
1506
+
1507
+ ### Order\_with
1508
+
1509
+ The `order_with` method is a convenient combination of `select` and `order_by`. It takes a single string expression as an argument to serve as a sort key&#x2014;one that would be valid as a select expression&#x2014;but with an optional trailing `!` to indicate reverse sort. The resulting table has an additional column called `:sort_key` with the expression evaluated for each row, and the table is sorted as with `order_by` on that column.
1510
+
1511
+ ```ruby
1512
+ tab1.order_with('price * shares').to_aoa
1513
+ ```
1514
+
1515
+ ```
1516
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp | Sort Key |
1517
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1518
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 742.5000 |
1519
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1520
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 | 770.0000 |
1521
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1522
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 | 825.0000 |
1523
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1524
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1530.0000 |
1525
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1526
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 | 1550.0000 |
1527
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1528
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 | 3875.0000 |
1529
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1530
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0000 |
1531
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 | 6000.0000 |
1532
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1533
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0000 |
1534
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 | 7600.0000 |
1535
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1536
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 | 21198.1500 |
1537
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1538
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 | 23971.2500 |
1539
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1540
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 | 30000.0000 |
1541
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1542
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0000 |
1543
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 | 35485.0000 |
1544
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1545
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 | 43560.1500 |
1546
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1547
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 | 51423.0500 |
1548
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1549
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 | 72580.0000 |
1550
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------+-------------|
1551
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 | 390285.0000 |
1552
+ ```
1553
+
1554
+
1555
+ <a id="orgbd3e61e"></a>
1556
+
1557
+ ### Group\_by
1558
+
1559
+ Like `order_by`, `group_by` takes a set of parameters of column header symbols, the "grouping parameters", by which to sort the table into a set of groups that are equal with respect to values in those columns. In addition, those parameters can be followed by a series of hash-like parameters, the "aggregating parameters", that indicate how any of the remaining, non-group columns are to be aggregated into a single value. The output table has one row for each group for which the grouping parameters are equal containing those columns and an aggregate column for each of the aggregating parameters.
1560
+
1561
+ For example, let's summarize the `trades` table by `:code` and `:price` again, and determine total shares, average price, and a few other features of each group:
1562
+
1563
+ ```ruby
1564
+ tab1.group_by(:code, :date, price: :avg,
1565
+ shares: :sum, lp: :sum, qp: :sum,
1566
+ qp10: :all?).to_aoa { |f| f.format(avg_price: '0.5R') }
1567
+ ```
1568
+
1569
+ ```
1570
+ | Code | Date | Avg Price | Sum Shares | Sum Lp | Sum Qp | All QP10 |
1571
+ |------+------------+-----------+------------+--------+--------+----------|
1572
+ | P | 2016-11-01 | 7.60714 | 17396 | 2473 | 14923 | F |
1573
+ | P | 2016-11-02 | 7.61786 | 69047 | 9945 | 59102 | F |
1574
+ | S | 2016-11-01 | 7.58000 | 13011 | 1852 | 11159 | F |
1575
+ ```
1576
+
1577
+ After the grouping column parameters, `:code` and `:date`, there are several hash-like "aggregating" parameters where the key is the column to aggregate and the value is a symbol for one of several aggregating methods that `FatTable::Column` objects understand. For example, the `:avg` method is applied to the :price column so that the output shows the average price in each group. The `:shares`, `:lp`, and `:qp` columns are summed, and the `:all?` aggregate is applied to one of the boolean fields, that is, it is `true` if all of the values in that column are `true`.
1578
+
1579
+ Note that the column names in the output of the aggregated columns have the name of the aggregating method pre-pended to the column name.
1580
+
1581
+ Here is a list of all the aggregate methods available. If the description restricts the aggregate to particular column types, applying it to other types will raise an exception.
1582
+
1583
+ - **`first`:** the first non-nil item in the column,
1584
+ - **`last`:** the last non-nil item in the column,
1585
+ - **`range`:** form a Range `~{min}..{max}` to show the range of values in the column,
1586
+ - **`sum`:** for `Numeric` columns, apply '+' to all the non-nil values; for `String` columns, join the elements with a single space,
1587
+ - **`count`:** the number of non-nil values in the column,
1588
+ - **`min`:** for `Numeric`, `String`, and `DateTime` columns, return the smallest non-nil, non-blank value in the column,
1589
+ - **`max`:** for `Numeric`, `String`, and `DateTime` columns, return the largest non-nil, non-blank value in the column,
1590
+ - **`avg`:** for `Numeric` and `DateTime` columns, return the arithmetic mean of the non-nil values in the column; with respect to `Date` or `DateTime` objects, each is converted to a numeric Julian date, the average is calculated, and the result converted back to a `Date` or `DateTime` object,
1591
+ - **`var`:** for `Numeric` and `DateTime` columns, compute the sample variance of the non-nil values in the column, dates are converted to Julian date numbers as for the `:avg` aggregate,
1592
+ - **`pvar`:** for `Numeric` and `DateTime` columns, compute the population variance of the non-nil values in the column, dates are converted to Julian date numbers as for the `:avg` aggregate,
1593
+ - **`dev`:** for `Numeric` and `DateTime` columns, compute the sample standard deviation of the non-nil values in the column, dates are converted to Julian date numbers as for the `:avg` aggregate,
1594
+ - **`pdev`:** for `Numeric` and `DateTime` columns, compute the population standard deviation of the non-nil values in the column, dates are converted to numbers as for the `:avg` aggregate,
1595
+ - **`all?`:** for `Boolean` columns only, return true if all of the non-nil values in the column are true,
1596
+ - **`any?`:** for `Boolean` columns only, return true if any non-nil value in the column is true,
1597
+ - **`none?`:** for `Boolean` columns only, return true if no non-nil value in the column is true,
1598
+ - **`one?`:** for `Boolean` columns only, return true if exactly one non-nil value in the column is true,
1599
+
1600
+ Perhaps surprisingly, the `group_by` method ignores any groups in its input and results in no group boundaries in the output since each group formed by the implicit `order_by` on the grouping columns is collapsed into a single row.
1601
+
1602
+
1603
+ <a id="orgebcd7d8"></a>
1604
+
1605
+ ### Join
1606
+
1607
+ 1. Join Types
1608
+
1609
+ So far, all the operations have operated on a single table. `FatTable` provides several `join` methods for combining two tables, each of which takes as parameters (1) a second table and (2) except in the case of `cross_join`, zero or more "join expressions". In the descriptions below, `T1` is the table on which the method is called, `T2` is the table supplied as the first parameter `other`, and `R1` and `R2` are rows in their respective tables being considered for inclusion in the joined output table.
1610
+
1611
+ - **`join(other, *jexps)`:** Performs an "inner join" on the tables. For each row `R1` of `T1`, the joined table has a row for each row in `T2` that satisfies the join condition with `R1`.
1612
+
1613
+ - **`left_join(other, *jexps)`:** First, an inner join is performed. Then, for each row in `T1` that does not satisfy the join condition with any row in `T2`, a joined row is added with null values in columns of `T2`. Thus, the joined table always has at least one row for each row in `T1`.
1614
+
1615
+ - **`right_join(other, *jexps)`:** First, an inner join is performed. Then, for each row in `T2` that does not satisfy the join condition with any row in `T1`, a joined row is added with null values in columns of `T1`. This is the converse of a left join: the result table will always have a row for each row in `T2`.
1616
+
1617
+ - **`full_join(other, *jexps)`:** First, an inner join is performed. Then, for each row in `T1` that does not satisfy the join condition with any row in `T2`, a joined row is added with null values in columns of `T2`. Also, for each row of `T2` that does not satisfy the join condition with any row in `T1`, a joined row with null values in the columns of `T1` is added.
1618
+
1619
+ - **`cross_join(other)`:** For every possible combination of rows from `T1` and `T2` (i.e., a Cartesian product), the joined table will contain a row consisting of all columns in `T1` followed by all columns in `T2`. If the tables have `N` and `M` rows respectively, the joined table will have `N * M` rows.
1620
+
1621
+ 2. Join Expressions
1622
+
1623
+ For each of the join types, if no join expressions are given, the tables will be joined on columns having the same column header in both tables, and the join condition is satisfied when all the values in those columns are equal. If the join type is an inner join, this is a so-called "natural" join.
1624
+
1625
+ If the join expressions are one or more symbols, the join condition requires that the values of both tables are equal for all columns named by the symbols. A column that appears in both tables can be given without modification and will be assumed to require equality on that column. If an unmodified symbol is not a name that appears in both tables, an exception will be raised. Column names that are unique to the first table must have a `_a` appended to the column name and column names that are unique to the other table must have a `_b` appended to the column name. These disambiguated column names must come in pairs, one for the first table and one for the second, and they will imply a join condition that the columns must be equal on those columns. Several such symbol expressions will require that all such implied pairs are equal in order for the join condition to be met.
1626
+
1627
+ Finally, a join expression can be a string that contains an arbitrary ruby expression that will be evaluated for truthiness. Within the string, *all* column names must be disambiguated with the `_a` or `_b` modifiers whether they are common to both tables or not. As with `select` and `where` methods, the names of the columns in both tables (albeit disambiguated) are available as local variables within the expression, but the instance variables `@row` and `@group` are not.
1628
+
1629
+ 3. Join Examples
1630
+
1631
+ The following examples are taken from the [Postgresql tutorial](https://www.tutorialspoint.com/postgresql/postgresql_using_joins.htm), with some slight modifications. The examples will use the following two tables, which are also available in `ft_console` as `@tab_a` and `@tab_b`:
1632
+
1633
+ ```ruby
1634
+ tab_a_str = <<-EOS
1635
+ | Id | Name | Age | Address | Salary | Join Date |
1636
+ |----+-------+-----+------------+--------+------------|
1637
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
1638
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
1639
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
1640
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
1641
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
1642
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
1643
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
1644
+ | 10 | James | 45 | Texas | 5000 | |
1645
+ EOS
1646
+
1647
+ tab_b_str = <<-EOS
1648
+ | Id | Dept | Emp Id |
1649
+ |----+-------------+--------|
1650
+ | 1 | IT Billing | 1 |
1651
+ | 2 | Engineering | 2 |
1652
+ | 3 | Finance | 7 |
1653
+ EOS
1654
+ ```
1655
+
1656
+ Here is `tab_a`:
1657
+
1658
+ ```ruby
1659
+ tab_a = FatTable.from_org_string(tab_a_str)
1660
+ tab_a.to_aoa
1661
+ ```
1662
+
1663
+ ```
1664
+ | Id | Name | Age | Address | Salary | Join Date |
1665
+ |----+-------+-----+------------+--------+------------|
1666
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
1667
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
1668
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
1669
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
1670
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
1671
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
1672
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
1673
+ | 10 | James | 45 | Texas | 5000 | |
1674
+ ```
1675
+
1676
+ And `tab_b`:
1677
+
1678
+ ```ruby
1679
+ tab_b = FatTable.from_org_string(tab_b_str)
1680
+ tab_b.to_aoa
1681
+ ```
1682
+
1683
+ ```
1684
+ | Id | Dept | Emp Id |
1685
+ |----+-------------+--------|
1686
+ | 1 | IT Billing | 1 |
1687
+ | 2 | Engineering | 2 |
1688
+ | 3 | Finance | 7 |
1689
+ ```
1690
+
1691
+ 1. Inner Joins
1692
+
1693
+ With no join expression arguments, the tables are joined when their sole common field, `:id`, is equal in both tables. The result is the natural join of the two tables.
1694
+
1695
+ ```ruby
1696
+ tab_a.join(tab_b).to_aoa
1697
+ ```
1698
+
1699
+ ```
1700
+ | Id | Name | Age | Address | Salary | Join Date | Dept | Emp Id |
1701
+ |----+-------+-----+------------+--------+------------+-------------+--------|
1702
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | IT Billing | 1 |
1703
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 | Finance | 7 |
1704
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | Engineering | 2 |
1705
+ ```
1706
+
1707
+ But the natural join joined employee IDs in the first table and department IDs in the second table. To correct this, we need to explicitly state the columns we want to join on in each table by disambiguating them with `_a` and `_b` suffixes:
1708
+
1709
+ ```ruby
1710
+ tab_a.join(tab_b, :id_a, :emp_id_b).to_aoa
1711
+ ```
1712
+
1713
+ ```
1714
+ | Id | Name | Age | Address | Salary | Join Date | Id B | Dept |
1715
+ |----+-------+-----+------------+--------+------------+------+-------------|
1716
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | 1 | IT Billing |
1717
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | 2 | Engineering |
1718
+ ```
1719
+
1720
+ Instead of using the disambiguated column names as symbols, we could also use a string containing a ruby expression. Within the expression, the column names should be treated as local variables:
1721
+
1722
+ ```ruby
1723
+ tab_a.join(tab_b, 'id_a == emp_id_b').to_aoa
1724
+ ```
1725
+
1726
+ ```
1727
+ | Id | Name | Age | Address | Salary | Join Date | Id B | Dept | Emp Id |
1728
+ |----+-------+-----+------------+--------+------------+------+-------------+--------|
1729
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | 1 | IT Billing | 1 |
1730
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | 2 | Engineering | 2 |
1731
+ ```
1732
+
1733
+ 2. Left and Right Joins
1734
+
1735
+ In left join, all the rows of `tab_a` are included in the output, augmented by the matching columns of `tab_b` and augmented with nils where there is no match:
1736
+
1737
+ ```ruby
1738
+ tab_a.left_join(tab_b, 'id_a == emp_id_b').to_aoa
1739
+ ```
1740
+
1741
+ ```
1742
+ | Id | Name | Age | Address | Salary | Join Date | Id B | Dept | Emp Id |
1743
+ |----+-------+-----+------------+--------+------------+------+-------------+--------|
1744
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | 1 | IT Billing | 1 |
1745
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 | | | |
1746
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 | | | |
1747
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 | | | |
1748
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | 2 | Engineering | 2 |
1749
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 | | | |
1750
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 | | | |
1751
+ | 10 | James | 45 | Texas | 5000 | | | | |
1752
+ ```
1753
+
1754
+ In a right join, all the rows of `tab_b` are included in the output, augmented by the matching columns of `tab_a` and augmented with nils where there is no match:
1755
+
1756
+ ```ruby
1757
+ tab_a.right_join(tab_b, 'id_a == emp_id_b').to_aoa
1758
+ ```
1759
+
1760
+ ```
1761
+ | Id | Name | Age | Address | Salary | Join Date | Id B | Dept | Emp Id |
1762
+ |----+-------+-----+------------+--------+------------+------+-------------+--------|
1763
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | 1 | IT Billing | 1 |
1764
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | 2 | Engineering | 2 |
1765
+ | | | | | | | 3 | Finance | 7 |
1766
+ ```
1767
+
1768
+ 3. Full Join
1769
+
1770
+ A full join combines the effects of a left join and a right join. All the rows from both tables are included in the output augmented by columns of the other table where the join expression is satisfied and augmented with nils otherwise.
1771
+
1772
+ ```ruby
1773
+ tab_a.full_join(tab_b, 'id_a == emp_id_b').to_aoa
1774
+ ```
1775
+
1776
+ ```
1777
+ | Id | Name | Age | Address | Salary | Join Date | Id B | Dept | Emp Id |
1778
+ |----+-------+-----+------------+--------+------------+------+-------------+--------|
1779
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | 1 | IT Billing | 1 |
1780
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 | | | |
1781
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 | | | |
1782
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 | | | |
1783
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | 2 | Engineering | 2 |
1784
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 | | | |
1785
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 | | | |
1786
+ | 10 | James | 45 | Texas | 5000 | | | | |
1787
+ | | | | | | | 3 | Finance | 7 |
1788
+ ```
1789
+
1790
+ 4. Cross Join
1791
+
1792
+ Finally, a cross join outputs every row of `tab_a` augmented with every row of `tab_b`, in other words, the Cartesian product of the two tables. If `tab_a` has `N` rows and `tab_b` has `M` rows, the output table will have `N * M` rows. So be careful lest you consume all your computer's memory.
1793
+
1794
+ ```ruby
1795
+ tab_a.cross_join(tab_b).to_aoa
1796
+ ```
1797
+
1798
+ ```
1799
+ | Id | Name | Age | Address | Salary | Join Date | Id B | Dept | Emp Id |
1800
+ |----+-------+-----+------------+--------+------------+------+-------------+--------|
1801
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | 1 | IT Billing | 1 |
1802
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | 2 | Engineering | 2 |
1803
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 | 3 | Finance | 7 |
1804
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 | 1 | IT Billing | 1 |
1805
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 | 2 | Engineering | 2 |
1806
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 | 3 | Finance | 7 |
1807
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 | 1 | IT Billing | 1 |
1808
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 | 2 | Engineering | 2 |
1809
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 | 3 | Finance | 7 |
1810
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 | 1 | IT Billing | 1 |
1811
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 | 2 | Engineering | 2 |
1812
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 | 3 | Finance | 7 |
1813
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | 1 | IT Billing | 1 |
1814
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | 2 | Engineering | 2 |
1815
+ | 2 | Allen | 25 | Texas | | 2005-07-13 | 3 | Finance | 7 |
1816
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 | 1 | IT Billing | 1 |
1817
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 | 2 | Engineering | 2 |
1818
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 | 3 | Finance | 7 |
1819
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 | 1 | IT Billing | 1 |
1820
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 | 2 | Engineering | 2 |
1821
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 | 3 | Finance | 7 |
1822
+ | 10 | James | 45 | Texas | 5000 | | 1 | IT Billing | 1 |
1823
+ | 10 | James | 45 | Texas | 5000 | | 2 | Engineering | 2 |
1824
+ | 10 | James | 45 | Texas | 5000 | | 3 | Finance | 7 |
1825
+ ```
1826
+
1827
+
1828
+ <a id="org654f7a8"></a>
1829
+
1830
+ ### Set Operations
1831
+
1832
+ `FatTable` can perform several set operations on pairs of tables. In order for two tables to be used this way, they must have the same number of columns with the same types or an exception will be raised. We'll call two tables that qualify for combining with set operations "set-compatible."
1833
+
1834
+ We'll use the following two set-compatible tables in the examples. They each have some duplicates and some group boundaries so you can see the effect of the set operations on duplicates and groups.
1835
+
1836
+ ```ruby
1837
+ tab1.to_aoa
1838
+ ```
1839
+
1840
+ ```
1841
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
1842
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1843
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1844
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1845
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1846
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1847
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1848
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
1849
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
1850
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1851
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1852
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1853
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
1854
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
1855
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1856
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
1857
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1858
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1859
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1860
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
1861
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
1862
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1863
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1864
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1865
+ ```
1866
+
1867
+ ```ruby
1868
+ tab2.to_aoa
1869
+ ```
1870
+
1871
+ ```
1872
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
1873
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
1874
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1875
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1876
+ | T017 | 2016-11-01 | P | 8.3000 | F | T | 1801 | 1201 | 600 | 0.2453 | 0.1924 |
1877
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
1878
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1879
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1880
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1881
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1882
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
1883
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1884
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1885
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1886
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1887
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
1888
+ | T019 | 2017-01-15 | S | 8.7500 | T | F | 300 | 175 | 125 | 0.2453 | 0.1924 |
1889
+ | T020 | 2017-01-19 | S | 8.2500 | F | T | 700 | 615 | 85 | 0.2453 | 0.1924 |
1890
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1891
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1892
+ ```
1893
+
1894
+ 1. Unions
1895
+
1896
+ Two tables that are set-compatible can be combined with the `union` or `union_all` methods so that the rows of both tables appear in the output. In the output table, the headers of the receiver table are used. You can use `select` to change or re-order the headers if you prefer. The `union` method eliminates duplicate rows in the result table, the `union_all` method does not.
1897
+
1898
+ Any group boundaries in the input tables are destroyed by `union` but are preserved by `union_all`. In addition, `union_all` (but not `union`) adds a group boundary between the rows of the two input tables.
1899
+
1900
+ ```ruby
1901
+ tab1.union(tab2).to_aoa
1902
+ ```
1903
+
1904
+ ```
1905
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
1906
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1907
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1908
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1909
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1910
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
1911
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
1912
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1913
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1914
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
1915
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
1916
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
1917
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1918
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1919
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
1920
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1921
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1922
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1923
+ | T017 | 2016-11-01 | P | 8.3000 | F | T | 1801 | 1201 | 600 | 0.2453 | 0.1924 |
1924
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1925
+ | T019 | 2017-01-15 | S | 8.7500 | T | F | 300 | 175 | 125 | 0.2453 | 0.1924 |
1926
+ | T020 | 2017-01-19 | S | 8.2500 | F | T | 700 | 615 | 85 | 0.2453 | 0.1924 |
1927
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1928
+ ```
1929
+
1930
+ ```ruby
1931
+ tab1.union_all(tab2).to_aoa
1932
+ ```
1933
+
1934
+ ```
1935
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
1936
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1937
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1938
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1939
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1940
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1941
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1942
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
1943
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
1944
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1945
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1946
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1947
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
1948
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
1949
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1950
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
1951
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
1952
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1953
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
1954
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
1955
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1956
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1957
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1958
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1959
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1960
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1961
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1962
+ | T017 | 2016-11-01 | P | 8.3000 | F | T | 1801 | 1201 | 600 | 0.2453 | 0.1924 |
1963
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1964
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1965
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
1966
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1967
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1968
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1969
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1970
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1971
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1972
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1973
+ |------+------------+------+--------+-----+------+--------+-------+-------+--------+--------|
1974
+ | T019 | 2017-01-15 | S | 8.7500 | T | F | 300 | 175 | 125 | 0.2453 | 0.1924 |
1975
+ | T020 | 2017-01-19 | S | 8.2500 | F | T | 700 | 615 | 85 | 0.2453 | 0.1924 |
1976
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1977
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
1978
+ ```
1979
+
1980
+ 2. Intersections
1981
+
1982
+ The `intersect` method returns a table having only rows common to both tables, eliminating any duplicate rows in the result.
1983
+
1984
+ ```ruby
1985
+ tab1.intersect(tab2).to_aoa
1986
+ ```
1987
+
1988
+ ```
1989
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
1990
+ |------+------------+------+--------+-----+------+--------+-----+------+--------+--------|
1991
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
1992
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
1993
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
1994
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
1995
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
1996
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
1997
+ ```
1998
+
1999
+ With `intersect_all`, all the rows of the first table, including duplicates, are included in the result if they also occur in the second table. However, duplicates in the second table do not appear.
2000
+
2001
+ ```ruby
2002
+ tab1.intersect_all(tab2).to_aoa
2003
+ ```
2004
+
2005
+ ```
2006
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
2007
+ |------+------------+------+--------+-----+------+--------+-----+------+--------+--------|
2008
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
2009
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
2010
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
2011
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
2012
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
2013
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
2014
+ ```
2015
+
2016
+ As a result, it makes a difference which table is the receiver of the `intersect_all` method call and which is the argument. In other words, order of operation matters.
2017
+
2018
+ ```ruby
2019
+ tab2.intersect_all(tab1).to_aoa
2020
+ ```
2021
+
2022
+ ```
2023
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
2024
+ |------+------------+------+--------+-----+------+--------+-----+------+--------+--------|
2025
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
2026
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
2027
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
2028
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
2029
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
2030
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
2031
+ ```
2032
+
2033
+ 3. Set Differences with Except
2034
+
2035
+ You can use the `except` method to delete from a table any rows that occur in another table, that is, compute the set difference between the tables.
2036
+
2037
+ ```ruby
2038
+ tab1.except(tab2).to_aoa
2039
+ ```
2040
+
2041
+ ```
2042
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
2043
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
2044
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
2045
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
2046
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
2047
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
2048
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
2049
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
2050
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
2051
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
2052
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
2053
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
2054
+ ```
2055
+
2056
+ Like subtraction, though, the order of operands matters with set difference computed by `except`.
2057
+
2058
+ ```ruby
2059
+ tab2.except(tab1).to_aoa
2060
+ ```
2061
+
2062
+ ```
2063
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
2064
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
2065
+ | T017 | 2016-11-01 | P | 8.3000 | F | T | 1801 | 1201 | 600 | 0.2453 | 0.1924 |
2066
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
2067
+ | T019 | 2017-01-15 | S | 8.7500 | T | F | 300 | 175 | 125 | 0.2453 | 0.1924 |
2068
+ | T020 | 2017-01-19 | S | 8.2500 | F | T | 700 | 615 | 85 | 0.2453 | 0.1924 |
2069
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
2070
+ ```
2071
+
2072
+ As with `intersect_all`, `except_all` includes any duplicates in the first, receiver table, but not those in the second, argument table.
2073
+
2074
+ ```ruby
2075
+ tab1.except_all(tab2).to_aoa
2076
+ ```
2077
+
2078
+ ```
2079
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
2080
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
2081
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
2082
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
2083
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
2084
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
2085
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
2086
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
2087
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
2088
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
2089
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
2090
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
2091
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
2092
+ ```
2093
+
2094
+ And, of course, the order of operands matters here as well.
2095
+
2096
+ ```ruby
2097
+ tab2.except_all(tab1).to_aoa
2098
+ ```
2099
+
2100
+ ```
2101
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
2102
+ |------+------------+------+--------+-----+------+--------+-------+------+--------+--------|
2103
+ | T017 | 2016-11-01 | P | 8.3000 | F | T | 1801 | 1201 | 600 | 0.2453 | 0.1924 |
2104
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
2105
+ | T018 | 2016-11-01 | S | 7.1520 | T | F | 2516 | 2400 | 116 | 0.2453 | 0.1924 |
2106
+ | T019 | 2017-01-15 | S | 8.7500 | T | F | 300 | 175 | 125 | 0.2453 | 0.1924 |
2107
+ | T020 | 2017-01-19 | S | 8.2500 | F | T | 700 | 615 | 85 | 0.2453 | 0.1924 |
2108
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
2109
+ | T021 | 2017-01-23 | P | 7.1600 | T | T | 12100 | 11050 | 1050 | 0.2453 | 0.1924 |
2110
+ ```
2111
+
2112
+
2113
+ <a id="orgd68c265"></a>
2114
+
2115
+ ### Uniq (aka Distinct)
2116
+
2117
+ The `uniq` method takes no arguments and simply removes any duplicate rows from the input table. The `distinct` method is an alias for `uniq`. Any groups in the input table are lost.
2118
+
2119
+ ```ruby
2120
+ tab1.uniq.to_aoa
2121
+ ```
2122
+
2123
+ ```
2124
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
2125
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
2126
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
2127
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
2128
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
2129
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
2130
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
2131
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
2132
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
2133
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
2134
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
2135
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
2136
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
2137
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
2138
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
2139
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
2140
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
2141
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
2142
+ ```
2143
+
2144
+
2145
+ <a id="org8b63b11"></a>
2146
+
2147
+ ### Remove groups with degroup!
2148
+
2149
+ Finally, it is sometimes helpful to remove any group boundaries from a table. You can do this with `.degroup!`, which, together with `force_string!`, are the only operations that mutate their receiver tables.
2150
+
2151
+ ```ruby
2152
+ tab1.degroup!.to_aoa
2153
+ ```
2154
+
2155
+ ```
2156
+ | Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
2157
+ |------+------------+------+--------+-----+------+--------+------+-------+--------+--------|
2158
+ | T001 | 2016-11-01 | P | 7.7000 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
2159
+ | T002 | 2016-11-01 | P | 7.7500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
2160
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
2161
+ | T003 | 2016-11-01 | P | 7.5000 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
2162
+ | T004 | 2016-11-01 | S | 7.5500 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
2163
+ | T005 | 2016-11-01 | S | 7.5000 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
2164
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
2165
+ | T006 | 2016-11-01 | S | 7.6000 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
2166
+ | T007 | 2016-11-01 | S | 7.6500 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
2167
+ | T008 | 2016-11-01 | P | 7.6500 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
2168
+ | T009 | 2016-11-01 | P | 7.6000 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
2169
+ | T010 | 2016-11-01 | P | 7.5500 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
2170
+ | T011 | 2016-11-02 | P | 7.4250 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
2171
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
2172
+ | T012 | 2016-11-02 | P | 7.5500 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
2173
+ | T013 | 2016-11-02 | P | 7.3500 | T | T | 53100 | 7656 | 45444 | 0.2453 | 0.1924 |
2174
+ | T014 | 2016-11-02 | P | 7.4500 | F | T | 5847 | 835 | 5012 | 0.2453 | 0.1924 |
2175
+ | T015 | 2016-11-02 | P | 7.7500 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
2176
+ | T016 | 2016-11-02 | P | 8.2500 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
2177
+ ```
2178
+
2179
+
2180
+ <a id="org767e09e"></a>
2181
+
2182
+ ## Formatting Tables
2183
+
2184
+ Besides creating and operating on tables, you may want to display the resulting table. `FatTable` seeks to provide a set of formatting directives that are the most common across many output media. It provides directives for alignment, for color, for adding currency symbols and grouping commas to numbers, for padding numbers, and for formatting dates and booleans.
2185
+
2186
+ In addition, you can add any number of footers to a table, which appear at the end of the table, and any number of group footers, which appear after each group in the table. These can be formatted independently of the table body.
2187
+
2188
+ If the target output medium does not support a formatting directive or the directive does not make sense, it is simply ignored. For example, you can output an `org-mode` table as a String, and since `org-mode` does not support colors, any color directives are ignored. Some of the output targets are not strings, but ruby data structures, and for them, things such as alignment are irrelevant.
2189
+
2190
+
2191
+ <a id="org945b4a7"></a>
2192
+
2193
+ ### Available Formatter Output Targets
2194
+
2195
+ 1. Output Media
2196
+
2197
+ `FatTable` supports the following output targets for its tables:
2198
+
2199
+ - **Text:** form the table with ACSII characters,
2200
+ - **Org:** form the table with ASCII characters but in the form used by Emacs org-mode for constructing tables,
2201
+ - **Term:** form the table with ANSI terminal codes and unicode characters, possibly including colored text and cell backgrounds,
2202
+ - **LaTeX:** form the table as input for LaTeX's longtable environment,
2203
+ - **Aoh:** output the table as a ruby data structure, building the table as an array of hashes, and
2204
+ - **Aoa:** output the table as a ruby data structure, building the table as an array of array,
2205
+
2206
+ These are all implemented by classes that inherit from `FatTable::Formatter` class by defining about a dozen methods that get called at various places during the construction of the output table. The idea is that more output formats can be defined by adding additional classes.
2207
+
2208
+ 2. Examples
2209
+
2210
+ 1. To Text
2211
+
2212
+ This formatter uses nothing but ASCII characters to draw the table. Notice that, unlike to `to_org` formatter shown below, the intersections of lines are represented by a `+` character. Embelishments such as color, bold, and so forth are ignored.
2213
+
2214
+ ```ruby
2215
+ tab_a.to_text
2216
+ ```
2217
+
2218
+ ```
2219
+ +====+=======+=====+============+========+============+
2220
+ | Id | Name | Age | Address | Salary | Join Date |
2221
+ +----+-------+-----+------------+--------+------------+
2222
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2223
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2224
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2225
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2226
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2227
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2228
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2229
+ | 10 | James | 45 | Texas | 5000 | |
2230
+ +====+=======+=====+============+========+============+
2231
+ ```
2232
+
2233
+ 2. To Org
2234
+
2235
+ This formatter is designed to format tables in a manner consistent with the way tables are drawn within Emacs Org Mode. It also uses nothing by ASCII characters to draw the table, but, the intersections of lines are represented by a `|` character. Embelishments such as color, bold, and so forth are ignored. When working in Org Mode, note that Emacs will convert an Array of Arrays into an Org Mode table, so when constructing tables programmatically, it may be better to use the `to_aoa` formatter shown below.
2236
+
2237
+ ```ruby
2238
+ tab_a.to_org
2239
+ ```
2240
+
2241
+ ```
2242
+ |----+-------+-----+------------+--------+------------------|
2243
+ | Id | Name | Age | Address | Salary | Join Date |
2244
+ |----+-------+-----+------------+--------+------------------|
2245
+ | 1 | Paul | 32 | California | 20000 | [2001-07-13 Fri] |
2246
+ | 3 | Teddy | 23 | Norway | 20000 | [2007-12-13 Thu] |
2247
+ | 4 | Mark | 25 | Rich-Mond | 65000 | [2007-12-13 Thu] |
2248
+ | 5 | David | 27 | Texas | 85000 | [2007-12-13 Thu] |
2249
+ | 2 | Allen | 25 | Texas | | [2005-07-13 Wed] |
2250
+ | 8 | Paul | 24 | Houston | 20000 | [2005-07-13 Wed] |
2251
+ | 9 | James | 44 | Norway | 5000 | [2005-07-13 Wed] |
2252
+ | 10 | James | 45 | Texas | 5000 | |
2253
+ |----+-------+-----+------------+--------+------------------|
2254
+ ```
2255
+
2256
+ 3. To Term
2257
+
2258
+ When outputting to a terminal or other device that can interpret ANSI characters and escape codes, you can use this formatter to get a prettier table. It also allows embelishments such as color and text styles to the extent the device supports it.
2259
+
2260
+ ```ruby
2261
+ tab_a.to_term
2262
+ ```
2263
+
2264
+ ```
2265
+ ╒════╤═══════╤═════╤════════════╤════════╤════════════╕
2266
+ │ Id │ Name │ Age │ Address │ Salary │ Join Date │
2267
+ ├────┼───────┼─────┼────────────┼────────┼────────────┤
2268
+ │ 1 │ Paul │ 32 │ California │ 20000 │ 2001-07-13 │
2269
+ │ 3 │ Teddy │ 23 │ Norway │ 20000 │ 2007-12-13 │
2270
+ │ 4 │ Mark │ 25 │ Rich-Mond │ 65000 │ 2007-12-13 │
2271
+ │ 5 │ David │ 27 │ Texas │ 85000 │ 2007-12-13 │
2272
+ │ 2 │ Allen │ 25 │ Texas │ │ 2005-07-13 │
2273
+ │ 8 │ Paul │ 24 │ Houston │ 20000 │ 2005-07-13 │
2274
+ │ 9 │ James │ 44 │ Norway │ 5000 │ 2005-07-13 │
2275
+ │ 10 │ James │ 45 │ Texas │ 5000 │ │
2276
+ ╘════╧═══════╧═════╧════════════╧════════╧════════════╛
2277
+ ```
2278
+
2279
+ 4. To LaTeX
2280
+
2281
+ This formatter outputs a table in the form suitable for inclusion in a LaTeX document using the `logtable` package. Natualy it allows embelishments such as color and text styles to the full extent of LaTeX's formatting prowess.
2282
+
2283
+ ```ruby
2284
+ tab_b.to_latex
2285
+ ```
2286
+
2287
+ ```
2288
+ \begin{longtable}{lll}
2289
+ Id&
2290
+ Dept&
2291
+ Emp Id\\
2292
+ \endhead
2293
+ 1&
2294
+ IT Billing&
2295
+ 1\\
2296
+ 2&
2297
+ Engineering&
2298
+ 2\\
2299
+ 3&
2300
+ Finance&
2301
+ 7\\
2302
+ \end{longtable}
2303
+ ```
2304
+
2305
+ 5. To AoA (Array of Arrays)
2306
+
2307
+ ```ruby
2308
+ tab_b.to_aoa
2309
+ ```
2310
+
2311
+ ```
2312
+ [["Id", "Dept", "Emp Id"], nil, ["1", "IT Billing", "1"], ["2", "Engineering", "2"], ["3", "Finance", "7"]]
2313
+ ```
2314
+
2315
+ 6. To AoH (Array of Hashes)
2316
+
2317
+ ```ruby
2318
+ tab_b.to_aoh
2319
+ ```
2320
+
2321
+ ```
2322
+ [{id: "1", dept: "IT Billing", emp_id: "1"}, {id: "2", dept: "Engineering", emp_id: "2"}, {id: "3", dept: "Finance", emp_id: "7"}]
2323
+ ```
2324
+
2325
+
2326
+ <a id="orga5d8638"></a>
2327
+
2328
+ ### Formatting Directives
2329
+
2330
+ The formatting methods explained in the next section all take formatting directives as strings in which letters and other characters signify what formatting applies. For example, we may apply the formatting directive 'R,$' to numbers in a certain part of the table. Each of those characters, and in some cases a whole substring, is a single directive. They can appear in any order, so '$R,' and ',$R' are equivalent.
2331
+
2332
+ Here is a list of all the formatting directives that apply to each cell type:
2333
+
2334
+ 1. All Types as Strings
2335
+
2336
+ For a string element, or any an element of any type (since these are applied after the element has been converted to a String), the following instructions are valid.
2337
+
2338
+ - **u:** convert the element to all lowercase [default false],
2339
+ - **U:** convert the element to all uppercase [default false],
2340
+ - **t:** title case the element, that is, upcase the initial letter in each word and lower case the other letters [default false],
2341
+ - **B ~B:** make the element bold, or turn off bold [default ~B]
2342
+ - **I ~I:** make the element italic, or turn off italic [default ~I]
2343
+ - **R:** align the element on the right of the column [default off]
2344
+ - **L:** align the element on the left of the column [default on]
2345
+ - **C:** align the element in the center of the column [default off]
2346
+ - **c[<color\_spec>]:** render the element in the given color; the <color\_spec> can have the form fgcolor, fgcolor.bgcolor, or .bgcolor, to set the foreground or background colors respectively, and each of those can be an ANSI or X11 color name in addition to the special color, 'none', which keeps the output's default color [default none].
2347
+ - **\_ ~\_:** underline the element, or turn off underline [default off]
2348
+ - **\* ~\*:** cause the element to blink, or turn off blink [default off]
2349
+
2350
+ For example, the directive 'tCc[red.yellow]' would title-case the element, center it, and color it red on a yellow background. The directives that are boolean have negating forms so that, for example, if bold is turned on for all columns of a given type, it can be countermanded in formatting directives for particular columns.
2351
+
2352
+ 2. Numeric
2353
+
2354
+ For a numeric element, all the instructions valid for string are available, in addition to the following:
2355
+
2356
+ - **, ~,:** insert grouping commas, or do not insert grouping commas [default ~,],
2357
+ - **$ ~$:** format the number as currency according to the locale, or not [default ~$],
2358
+ - **m.n:** include at least m digits before the decimal point, padding on the left with zeroes as needed, and round the number to the n decimal places and include n digits after the decimal point, padding on the right with zeroes as needed. If n is negative, the value will be rounded to the left of the decimal point: e.g., if n is -2, the number will be rounded to the nearest hundred, if -3, to the nearest thousand, etc. [default 0.0]
2359
+ - **H:** convert the number (assumed to be in units of seconds) to `HH:MM:SS.ss` form. So a column that is the result of subtracting two :datetime forms will result in a :numeric expressed as seconds and can be displayed in hours, minutes, and seconds with this formatting instruction. If this directive is included, all other numeric directives will be ignored. [default off]
2360
+
2361
+ For example, the directive 'R5.0c[blue]' would right-align the numeric element, pad it on the left with zeros, and color it blue.
2362
+
2363
+ 3. DateTime
2364
+
2365
+ For a `DateTime`, all the instructions valid for string are available, in addition to the following:
2366
+
2367
+ - **d[fmt]:** apply the format to a `Date` or a `DateTime` that is a whole day, that is that has no or zero hour, minute, and second components, where fmt is a valid format string for `Date#strftime`, otherwise, the datetime will be formatted as an ISO 8601 string, YYYY-MM-DD.
2368
+ - **D[fmt]:** apply the format to a datetime that has at least a non-zero hour component where fmt is a valid format string for Date#strftime, otherwise, the datetime will be formatted as an ISO 8601 string, YYYY-MM-DD HH:MM:SS.
2369
+
2370
+ For example, 'c[pink]d[%b %-d, %Y]C', would format a date element like 'Sep 22, 1957', center it, and color it pink.
2371
+
2372
+ 4. Boolean
2373
+
2374
+ For a boolean cell, all the instructions valid for string are available, in addition to the following:
2375
+
2376
+ - **Y:** print true as `Y` and false as `N`,
2377
+ - **T:** print true as `T` and false as `F` [this is the default],
2378
+ - **X:** print true as `X` and false as an empty string '',
2379
+ - **b[xxx,yyy]:** print true as the string given as `xxx` and false as the string given as `yyy`,
2380
+ - **c[tcolor,fcolor]:** color a true element with `tcolor` and a false element with `fcolor`. Each of the colors may be specified in the same manner as colors for strings described above.
2381
+
2382
+ For example, the directive 'b[Yeppers,Nope]c[green.pink,red.pink]' would render a true boolean as `Yeppers` colored green on pink and render a false boolean as `Nope` colored red on pink. See [Yeppers](https://www.youtube.com/watch?v=oLdFFD8II8U) for additional information.
2383
+
2384
+ 5. NilClass
2385
+
2386
+ By default, `nil` elements are rendered as blank cells, but you can make them visible with the following, and in that case, all the formatting instructions valid for strings are also available:
2387
+
2388
+ - **n[niltext]:** render a `nil` item with the given niltext [default ''].
2389
+
2390
+ For example, you might want to use 'n[-]Cc[purple]' to make nils visible as a centered purple hyphen.
2391
+
2392
+
2393
+ <a id="org11bef41"></a>
2394
+
2395
+ ### The `format` and `format_for` methods
2396
+
2397
+ Formatters take only two kinds of methods, those that attach footers to a table, which are discussed in the next section, and those that specify formatting for table cells, which are the subject of this section.
2398
+
2399
+ To set formatting directives for all locations in a table at once, use the `format` method; to set formatting directives for a particular location in the table, use the `format_for` method, giving the location as the first parameter. See below at [Table Locations](#org810c2fb) for an explanation of all the locations available.
2400
+
2401
+ Other than that first parameter, the two methods take the same types of parameters. The remaining parameters are hash-like parameters that use either a column name or a type as the key and a string with the formatting directives to apply as the value. If a key represents neither a column name nor a valid type, it is silently ignored. The following example says to set the formatting for all locations in the table and to format all numeric fields as strings that are rounded to whole numbers (the '0.0' part), that are right-aligned (the 'R' part), and have grouping commas inserted (the ',' part). But the `:id` column is numeric, and the second parameter overrides the formatting for numerics in general and calls for the `:id` column to be padded to three digits with zeros on the left (the '3.0' part) and to be centered (the 'C' part).
2402
+
2403
+ ```ruby
2404
+ tab_a.to_text do |f|
2405
+ # Note: blat: is silently ignored
2406
+ f.format(numeric: '0.0,R', id: '3.0C', blat: 'B')
2407
+ f.format_for(:body, string: 'R')
2408
+ f.format_for(:header, string: 'C')
2409
+ end
2410
+ ```
2411
+
2412
+ ```
2413
+ +=====+=======+=====+============+========+============+
2414
+ | Id | Name | Age | Address | Salary | Join Date |
2415
+ +-----+-------+-----+------------+--------+------------+
2416
+ | 001 | Paul | 32 | California | 20,000 | 2001-07-13 |
2417
+ | 003 | Teddy | 23 | Norway | 20,000 | 2007-12-13 |
2418
+ | 004 | Mark | 25 | Rich-Mond | 65,000 | 2007-12-13 |
2419
+ | 005 | David | 27 | Texas | 85,000 | 2007-12-13 |
2420
+ | 002 | Allen | 25 | Texas | | 2005-07-13 |
2421
+ | 008 | Paul | 24 | Houston | 20,000 | 2005-07-13 |
2422
+ | 009 | James | 44 | Norway | 5,000 | 2005-07-13 |
2423
+ | 010 | James | 45 | Texas | 5,000 | |
2424
+ +=====+=======+=====+============+========+============+
2425
+ ```
2426
+
2427
+ In the example, the `format` method affects the whole table. Its `numeric:` directive affected the `:age` and `:salary` columns because their types are Numeric. The `id:` column is also Numeric, but it's more specific directive takes precedence and it is formatted accordingly.
2428
+
2429
+ But the `format_for` methods affected two "locations": the "body" and the "header". Within the body, the `:string` directive calls for all strings to be right-aligned, but the headers are unaffected by it. The `format_for` the `:header` location caused all the headers to be centered.
2430
+
2431
+ All the other cells in the table, namely the cells in the `:join_date` column, had the default formatting applied.
2432
+
2433
+ 1. Table Locations
2434
+
2435
+ In the `format_for` formatting method, the first argument names a "location." The table is divided into several locations for which separate formatting directives may be given. These locations are identified by the following symbols:
2436
+
2437
+ - **:header:** the first row of the output table containing the headers,
2438
+ - **:footer:** all rows of the table's footers,
2439
+ - **:gfooter:** all rows of the table's group footers,
2440
+ - **:body:** all the data rows of the table, that is, those that are neither part of the header, footers, or gfooters,
2441
+ - **:bfirst:** the first row of the table's body, and
2442
+ - **:gfirst:** the first row in each group in the table's body.
2443
+
2444
+ 2. Location priority
2445
+
2446
+ Formatting for any given cell depends on its location in the table. The `format_for` method takes a location to which its formatting directive are restricted as the first argument. It can be one of the following:
2447
+
2448
+ - **`:header`:** The directives apply only to the header row, that is the first row, of the output table; before the directives are applied, the header's symbol form is converted back into a string and capitalized as is a book title. Thus, only directives applicable to the String type have any effect.
2449
+
2450
+ - **`:body`:** The directives apply to all rows in the body of the table.
2451
+
2452
+ - **`:gfirst`:** directives apply to the first row in each group in the body of the table, unless the row is also the first row in the table as a whole, in which case the `:bfirst` directives apply,
2453
+
2454
+ - **`:bfirst`:** The directives apply to the first row in the body of the table, taking precedence over those directives that apply to the body generally or the `:gfirst` directives that apply to the first row in each group.
2455
+
2456
+ - **`:footer`:** The directives apply to all the footer rows of the output table, regardless of how many there are.
2457
+
2458
+ - **`gfooter`:** The directives apply to all group footer rows of the output tables, regardless of how many there are.
2459
+
2460
+ Directives given to the `format` method apply the directives to all locations in the table, but they can be overridden by more specific directives given in a `format_for` directive.
2461
+
2462
+ 3. Type and Column priority
2463
+
2464
+ A directive based the column name overrides any directive based on type. If any cell has both a type-based formatting and column-based, the column instructions prevail. In earlier versions the instuctions were "merged" but that is no longer the case.
2465
+
2466
+ However, there is a twist. Since the end result of formatting is to convert all columns to strings, the formatting directives for the `String` type can be applied to all column types. Likewise, since all columns may contain nils, the `NilClass:` type applies to nils in all columns regardless of the column's type.
2467
+
2468
+ ```ruby
2469
+ tab_a.to_text do |f|
2470
+ f.format(string: 'R', id: '3.0C', nil: 'Cn[-]', salary: 'n[N/A]')
2471
+ end
2472
+ ```
2473
+
2474
+ ```
2475
+ +=====+=======+=====+============+========+============+
2476
+ | Id | Name | Age | Address | Salary | Join Date |
2477
+ +-----+-------+-----+------------+--------+------------+
2478
+ | 001 | Paul | 32 | California | 20000 | 2001-07-13 |
2479
+ | 003 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2480
+ | 004 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2481
+ | 005 | David | 27 | Texas | 85000 | 2007-12-13 |
2482
+ | 002 | Allen | 25 | Texas | N/A | 2005-07-13 |
2483
+ | 008 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2484
+ | 009 | James | 44 | Norway | 5000 | 2005-07-13 |
2485
+ | 010 | James | 45 | Texas | 5000 | - |
2486
+ +=====+=======+=====+============+========+============+
2487
+ ```
2488
+
2489
+ The `string: 'R'` directive causes all the cells to be right-aligned except `:id` which specifies centering for the `:id` column only. The `n[N/A]` directive for specifies how nil are displayed in the numeric column, `:salary`, but not for other nils, such as in the last row of the `:join_date` column.
2490
+
2491
+
2492
+ <a id="org53ea8ff"></a>
2493
+
2494
+ ### Footers
2495
+
2496
+ 1. Adding Footers
2497
+
2498
+ You can call the `foot`, `gfoot`, `footer,` or `gfooter`, methods on `Formatter` objects to add footers and group footers. Note that all of these methods return a `Footer` object that can be accessed to extract the computed values. All of these methods return the `FatTable::Footer` object so constructed. It can be used to access the values and other attributes of the footer computed. Their signatures are:
2499
+
2500
+ - **`foot(label: label, label_col: nil, **agg_cols)`:** where `label` is a label to be placed in the column with header `label_col`, or, if ommitted, in the first cell of the footer (unless that column is named as one of the `agg_cols`, in which case the label is ignored), and `**agg_cols` is zero or more hash-like parameters with a column symbol as a key and a valid aggregate as the value. This causes a table-wide header to be added at the bottom of the table applying `agg`, to the `agg_cols`. A table can have any number of footers attached, and they will appear at the bottom of the output table in the order they are given.
2501
+
2502
+ - **`gfoot(label: 'Group Total', label_col: nil, **agg_cols)`:** where the parameters have the same meaning as for the `foot` method, but results in a footer for each group in the table rather than the table as a whole. These will appear in the output table just below each group.
2503
+
2504
+ - **`footer(label, *sum_cols, **agg_cols)`:** where `label` is a label to be placed in the first cell of the footer (unless that column is named as one of the `sum_cols` or `agg_cols`, in which case the label is ignored), `*sum_cols` are zero or more symbols for columns to be summed, and `**agg_cols` is zero or more hash-like parameters with a column symbol as a key and a valid aggregate as the value. This causes a table-wide header to be added at the bottom of the table applying the `:sum` aggregate to the `sum_cols` and the named aggregate to the `agg_cols`. A table can have any number of footers attached, and they will appear at the bottom of the output table in the order they are given.
2505
+
2506
+ - **`gfooter(label, *sum_cols, **agg_cols)`:** where the parameters have the same meaning as for the `footer` method, but results in a footer for each group in the table rather than the table as a whole. These will appear in the output table just below each group.
2507
+
2508
+ There are also a number of convenience methods for adding common footers:
2509
+
2510
+ - **`sum_footer(*cols)`:** Add a footer summing the given columns with the label 'Total'.
2511
+ - **`sum_gfooter(*cols)`:** Add a group footer summing the given columns with the label 'Group Total'.
2512
+ - **`avg_footer(*cols)`:** Add a footer averaging the given columns with the label 'Average'.
2513
+ - **`avg_gfooter(*cols)`:** Add a group footer averaging the given columns with the label 'Group Average'.
2514
+ - **`min_footer(*cols)`:** Add a footer showing the minimum for the given columns with the label 'Minimum'.
2515
+ - **`min_gfooter(*cols)`:** Add a group footer showing the minumum for the given columns with the label 'Group Minimum'.
2516
+ - **`max_footer(*cols)`:** Add a footer showing the maximum for the given columns with the label 'Maximum'.
2517
+ - **`max_gfooter(*cols)`:** Add a group footer showing the maximum for the given columns with the label 'Group Maximum'.
2518
+
2519
+ 2. Dynamic Labels
2520
+
2521
+ Most of the time, you will want a fixed string as the label. However, especially in the case of a group footer, you might want a dynamically contructed label. You can use a proc or lambda for a label, and it will be computed for you. In the case of non-group footers, the proc takes a single parameter, the footer object itself. This allows you to make the label a function of other footer values, for example, you could make the label include the most recent year from the date column:
2522
+
2523
+ ```ruby
2524
+ fmtr.foot(label: -> (f) { "Average (latest year #{f.column(:date).max.year})" },
2525
+ temp: :avg)
2526
+ ```
2527
+
2528
+ In the case of a group footer, the lambda or proc may take either one or two parameters. If it takes one, the parameter is simply the 0-based number of the group:
2529
+
2530
+ ```ruby
2531
+ fmtr.gfoot(label: -> (k) { "Group #{(k+1).to_roman} Average" }, temp: :avg)
2532
+ ```
2533
+
2534
+ This would format the label with a roman numeral (assuming you defined a method to do so) for the group number.
2535
+
2536
+ If it takes two arguments, the second argument is the footer itself, as with non-group footers:
2537
+
2538
+ ```ruby
2539
+ fmtr.gfoot(label: -> (k, f) { "Year #{f.column(:date, k).max.year} Group #{(k+1).to_roman} Average" },
2540
+ temp: :avg)
2541
+ ```
2542
+
2543
+ This would add the group's year to label, assuming the :date column of the footer's table had the same year for each item in the group.
2544
+
2545
+ 3. Aggregators
2546
+
2547
+ When adding a footer with the above methods, you can specify an aggregator for each column named in the `agg_cols` parameter. There are several candidates for what you can use for an aggregator:
2548
+
2549
+ - **Symbol:** one of the following built-in aggregators: :first, :last, :range, :sum, :count, :min, :max, :avg, :var, :pvar, :dev, :pdev, :any?, :all?, :none?, and :one?.
2550
+ - The symbols ending in a question mark are valid only for boolean columns;
2551
+ - :count, :first, and :last work with any column type,
2552
+ - :min, :max, and :range work with all types except boolean;
2553
+ - :sum, works only with numeric columns, and
2554
+ - :avg, :var, :dev, :pvar, and :pdev work with numeric or datetime columns. In the case of datetime columns, these aggrgators convert the dates to julian date numbers, perform the calculation, then convert the result back to a datetime object. Apart from the built-in aggregators, you could define your own by opening the FatTable::Column class and adding a suitable instance method. In that case, the symbol could also refer to the method you defined.
2555
+ - **String:** using a string as an aggrgegator can result in:
2556
+ - the string being converted to an object matching the type of the column (for example, using '$1,888' in a numeric column puts the constant number 1888 in the footer field, using '1957-09-22' puts the fixed date in the field, etc.)
2557
+ - if the string cannot be parsed as a valid object matching the column's type, it is placed literally in the footer field (for example, using '(estimated)' can be used to add additional information to the footer)
2558
+ - **Ruby object:** you can put a number in a numeric footer field, a DateTime object in a datetime footer field, or a true or false in a boolean footer field;
2559
+ - **A Lambda:** finally, you can provide a lambda for performing arbitrary calculations and placing the result in the footer field. The number of arguments the lambda takes can vary:
2560
+ - If the lambda is used in an ordinary footer column, it can take 0, 1, or 2 arguments: (1) the first argument, if given, will be set to the FatTable::Column object for that column and (2) the second argument, if given, will be set to the Footer object itself.
2561
+ - If the lambda is used in a group footer column, it can 0, 1, 2, or 3 arguments: (1) the first argument, if given, will be set to the group's 0-based index number, (2) the second argument, if given, will be set to a FatTable::Column object consisting of those items in the group's column, and (3) the third argument, if given, will be set to the Footer object itself.
2562
+
2563
+ 4. Footer objects
2564
+
2565
+ Each of the methods for adding a footer to a `Formatter` returns a `Footer` object that you can query for attributes of the generated footer, including accessing their computed values. Here are the accessors available on a `FatTable::Footer` object:
2566
+
2567
+ - **`[h]`:** Return the value of under the `h` header, or if this is a group footer, return an array of the values for all the groups under the `h` header.
2568
+ - **.<header>:** like, `[h]` but makes the values available in method-call form.
2569
+ - **`number_of_groups`:** Return the total number of groups in the table to which this footer belongs. Note that if the table has both group footers and normal footers, this will return the number of groups even for a normal footer.
2570
+ - **`column(h)`, `column(h, k)`:** Return a FatTable::Column object for the header h and, if the footer is a group footer, the kth group.
2571
+ - **`items(h)`, `items(h, k)`:** Return an Array of the values for the header `h` and, if a group, for the ~k~th group.
2572
+ - **`to_h`, `to_h(k)`:** Return a Hash with a key for each column header mapped to the footer value for that column, nil for unused columns. Use the index `k` to specify which group to access in the case of a group footer.
2573
+
2574
+ 5. Footer Examples
2575
+
2576
+ As a reminder, here is the table, `tab_a` defined earlier:
2577
+
2578
+ ```ruby
2579
+ tab_a.to_aoa
2580
+ ```
2581
+
2582
+ ```
2583
+ | Id | Name | Age | Address | Salary | Join Date |
2584
+ |----+-------+-----+------------+--------+------------|
2585
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2586
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2587
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2588
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2589
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2590
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2591
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2592
+ | 10 | James | 45 | Texas | 5000 | |
2593
+ ```
2594
+
2595
+ 1. Built-in Aggregators
2596
+
2597
+ You can add a footer compute the average of the given columns. You may be surprised that you can average a set of dates, but `:avg` simply converts the dates to Julian numbers, averages that, then converts the result back to a date.
2598
+
2599
+ ```ruby
2600
+ tab_a.to_text do |f|
2601
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2602
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2603
+ f.footer('Tally', age: :count)
2604
+ end
2605
+ ```
2606
+
2607
+ ```
2608
+ +=========+=======+=====+============+========+=============+
2609
+ | Id | Name | Age | Address | Salary | Join Date |
2610
+ +---------+-------+-----+------------+--------+-------------+
2611
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2612
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2613
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2614
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2615
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2616
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2617
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2618
+ | 10 | James | 45 | Texas | 5,000 | |
2619
+ +---------+-------+-----+------------+--------+-------------+
2620
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2621
+ +---------+-------+-----+------------+--------+-------------+
2622
+ | Tally | | 8 | | | |
2623
+ +=========+=======+=====+============+========+=============+
2624
+ ```
2625
+
2626
+ 2. String Aggregators
2627
+
2628
+ If the string is convertible into its columns's type, it will be converted to that type; otherwise, it will be placed in the footer literally. This example also shows how the values from one footer might be used in composing another footer.
2629
+
2630
+ ```ruby
2631
+ tab_a.to_text do |f|
2632
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2633
+ avg_ft = f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2634
+ f.footer('Tally', age: :count)
2635
+ if avg_ft[:salary] < 30000
2636
+ cmt = "We're saving"
2637
+ else
2638
+ cmt = "We're overspending"
2639
+ end
2640
+ f.footer('Pay', join_date: "We have #{avg_ft.number_of_groups} grp")
2641
+ f.footer('Group count', join_date: "We have #{avg_ft.number_of_groups} grp")
2642
+ f.footer('Comment', join_date: cmt)
2643
+ end
2644
+ ```
2645
+
2646
+ ```
2647
+ +=============+=======+=====+============+========+====================+
2648
+ | Id | Name | Age | Address | Salary | Join Date |
2649
+ +-------------+-------+-----+------------+--------+--------------------+
2650
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2651
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2652
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2653
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2654
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2655
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2656
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2657
+ | 10 | James | 45 | Texas | 5,000 | |
2658
+ +-------------+-------+-----+------------+--------+--------------------+
2659
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2660
+ +-------------+-------+-----+------------+--------+--------------------+
2661
+ | Tally | | 8 | | | |
2662
+ +-------------+-------+-----+------------+--------+--------------------+
2663
+ | Pay | | | | | We have 1 grp |
2664
+ +-------------+-------+-----+------------+--------+--------------------+
2665
+ | Group count | | | | | We have 1 grp |
2666
+ +-------------+-------+-----+------------+--------+--------------------+
2667
+ | Comment | | | | | We're overspending |
2668
+ +=============+=======+=====+============+========+====================+
2669
+ ```
2670
+
2671
+ 3. Ruby Objects
2672
+
2673
+ You can make the aggregator an normal ruby object, in which case it is just inserted into the footer at the requested location. If its type is the same as the column type, it participates in the formatting for that type and column.
2674
+
2675
+ ```ruby
2676
+ tab_a.to_text do |f|
2677
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2678
+ f.footer('Report Date', age: :count, join_date: Date.today)
2679
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2680
+ end
2681
+ ```
2682
+
2683
+ ```
2684
+ +=============+=======+=====+============+========+=============+
2685
+ | Id | Name | Age | Address | Salary | Join Date |
2686
+ +-------------+-------+-----+------------+--------+-------------+
2687
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2688
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2689
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2690
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2691
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2692
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2693
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2694
+ | 10 | James | 45 | Texas | 5,000 | |
2695
+ +-------------+-------+-----+------------+--------+-------------+
2696
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2697
+ +-------------+-------+-----+------------+--------+-------------+
2698
+ | Report Date | | 8 | | | 28-DEC-2025 |
2699
+ +=============+=======+=====+============+========+=============+
2700
+ ```
2701
+
2702
+ But it can be any type. Here we pick a lottery winner from the employee ids.
2703
+
2704
+ ```ruby
2705
+ tab_a.to_text do |f|
2706
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2707
+ winner_id = tab_a.column(:id).items.sample
2708
+ f.footer('Lottery Winner', age: :count, join_date: winner_id)
2709
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2710
+ end
2711
+ ```
2712
+
2713
+ ```
2714
+ +================+=======+=====+============+========+=============+
2715
+ | Id | Name | Age | Address | Salary | Join Date |
2716
+ +----------------+-------+-----+------------+--------+-------------+
2717
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2718
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2719
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2720
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2721
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2722
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2723
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2724
+ | 10 | James | 45 | Texas | 5,000 | |
2725
+ +----------------+-------+-----+------------+--------+-------------+
2726
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2727
+ +----------------+-------+-----+------------+--------+-------------+
2728
+ | Lottery Winner | | 8 | | | 9 |
2729
+ +================+=======+=====+============+========+=============+
2730
+ ```
2731
+
2732
+ 4. Lambdas
2733
+
2734
+ Perhaps the most flexible form of aggregator is a lambda form. They can take up to 2 or up to 3 parameters in non-group and group footers, respectively:
2735
+
2736
+ - **`->(c, f) {...}`:** in a normal, non-group footer, you may provide for up to two paramters: the first, `c`, if given, will be bound to the column header to which the lambda is attached and and the second, `f`, if given will be bound to the footer in which the lambda appears. A lambda with no parameters can be provided as well if none are needed.
2737
+ - **`->(k, c, f)`:** in a group footer, you may provide for up to three paramters: the the first, `k`, if provided, will be bound to the group number of the group being evaluated, the second, `c`, if provided, will be bound to the column header to which the lambda is attached, and the third, `f`, will be bound to the footer in which the lambda appears. A lambda with no parameters can be provided as well if none are needed.
2738
+
2739
+ With the first argument, the footer itself becomes available and with it all the things accessible with the footers, including the items in the current column, through the `f.items(c)` accessor.
2740
+
2741
+ Compute the summ of the squares if the items in the `:age` column:
2742
+
2743
+ ```ruby
2744
+ tab_a.to_text do |f|
2745
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]')
2746
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2747
+ f.footer('SSQ', age: ->(c) { sa = c.items.map {|x| x * x}.sum; Math.sqrt(sa) })
2748
+ end
2749
+ ```
2750
+
2751
+ ```
2752
+ +=========+=======+=====+============+========+=============+
2753
+ | Id | Name | Age | Address | Salary | Join Date |
2754
+ +---------+-------+-----+------------+--------+-------------+
2755
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 |
2756
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 |
2757
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 |
2758
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 |
2759
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 |
2760
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 |
2761
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 |
2762
+ | 10 | James | 45 | Texas | 5,000 | |
2763
+ +---------+-------+-----+------------+--------+-------------+
2764
+ | Average | | 31 | | 31,429 | 29-DEC-2005 |
2765
+ +---------+-------+-----+------------+--------+-------------+
2766
+ | SSQ | | 810 | | | |
2767
+ +=========+=======+=====+============+========+=============+
2768
+ ```
2769
+
2770
+ Group the table according to the employee's year of joining, then compute the summ of the squares if the ages in each group:
2771
+
2772
+ ```ruby
2773
+ tab_a.order_with('join_date.year').to_text do |f|
2774
+ f.format(numeric: '0.0R,', datetime: 'd[%v]D[%v]', sort_key: '0.0~,')
2775
+ f.footer('Average', age: :avg, salary: :avg, join_date: :avg)
2776
+ f.gfooter('Group SSQ', age: ->(k, c, f) { sa = c.items.map {|x| x * x}.sum; Math.sqrt(sa) })
2777
+ f.footer('Total SSQ', age: ->(c, f) { sa = c.items.map {|x| x * x}.sum; Math.sqrt(sa) })
2778
+ end
2779
+ ```
2780
+
2781
+ ```
2782
+ +===========+=======+=====+============+========+=============+==========+
2783
+ | Id | Name | Age | Address | Salary | Join Date | Sort Key |
2784
+ +-----------+-------+-----+------------+--------+-------------+----------+
2785
+ | 10 | James | 45 | Texas | 5,000 | | |
2786
+ +-----------+-------+-----+------------+--------+-------------+----------+
2787
+ | Group SSQ | | 45 | | | | |
2788
+ +-----------+-------+-----+------------+--------+-------------+----------+
2789
+ | 1 | Paul | 32 | California | 20,000 | 13-JUL-2001 | 2001 |
2790
+ +-----------+-------+-----+------------+--------+-------------+----------+
2791
+ | Group SSQ | | 32 | | | | |
2792
+ +-----------+-------+-----+------------+--------+-------------+----------+
2793
+ | 2 | Allen | 25 | Texas | | 13-JUL-2005 | 2005 |
2794
+ | 8 | Paul | 24 | Houston | 20,000 | 13-JUL-2005 | 2005 |
2795
+ | 9 | James | 44 | Norway | 5,000 | 13-JUL-2005 | 2005 |
2796
+ +-----------+-------+-----+------------+--------+-------------+----------+
2797
+ | Group SSQ | | 56 | | | | |
2798
+ +-----------+-------+-----+------------+--------+-------------+----------+
2799
+ | 3 | Teddy | 23 | Norway | 20,000 | 13-DEC-2007 | 2007 |
2800
+ | 4 | Mark | 25 | Rich-Mond | 65,000 | 13-DEC-2007 | 2007 |
2801
+ | 5 | David | 27 | Texas | 85,000 | 13-DEC-2007 | 2007 |
2802
+ +-----------+-------+-----+------------+--------+-------------+----------+
2803
+ | Group SSQ | | 43 | | | | |
2804
+ +-----------+-------+-----+------------+--------+-------------+----------+
2805
+ | Average | | 31 | | 31,429 | 29-DEC-2005 | |
2806
+ +-----------+-------+-----+------------+--------+-------------+----------+
2807
+ | Total SSQ | | 810 | | | | |
2808
+ +===========+=======+=====+============+========+=============+==========+
2809
+ ```
2810
+
2811
+
2812
+ <a id="org102d235"></a>
2813
+
2814
+ ### Invoking Formatters
2815
+
2816
+ As the examples show, one way to invoke the formatting methods is simply to call one of the `to_xxx` methods directly on a table, which will yield a `FatTable::Formatter` object to the block, and that is often the most convenient way to do it. But there are a few other ways.
2817
+
2818
+ 1. By Instantiating a Formatter
2819
+
2820
+ You can instantiate a `XXXFormatter` object and feed it a table as a parameter. There is a Formatter subclass for each target output medium, for example, `AoaFormatter` will produce a ruby array of arrays. You can then call the `output` method on the `XXXFormatter`.
2821
+
2822
+ ```ruby
2823
+ FatTable::AoaFormatter.new(tab_a).output
2824
+ ```
2825
+
2826
+ ```
2827
+ | Id | Name | Age | Address | Salary | Join Date |
2828
+ |----+-------+-----+------------+--------+------------|
2829
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2830
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2831
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2832
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2833
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2834
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2835
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2836
+ | 10 | James | 45 | Texas | 5000 | |
2837
+ ```
2838
+
2839
+ The `XXXFormatter.new` method yields the new instance to any block given, and you can call methods on it to affect the formatting of the output:
2840
+
2841
+ ```ruby
2842
+ FatTable::AoaFormatter.new(tab_a) do |f|
2843
+ f.format(numeric: '0.0,R', id: '3.0C')
2844
+ end.output
2845
+ ```
2846
+
2847
+ ```
2848
+ | Id | Name | Age | Address | Salary | Join Date |
2849
+ |-----+-------+-----+------------+--------+------------|
2850
+ | 001 | Paul | 32 | California | 20,000 | 2001-07-13 |
2851
+ | 003 | Teddy | 23 | Norway | 20,000 | 2007-12-13 |
2852
+ | 004 | Mark | 25 | Rich-Mond | 65,000 | 2007-12-13 |
2853
+ | 005 | David | 27 | Texas | 85,000 | 2007-12-13 |
2854
+ | 002 | Allen | 25 | Texas | | 2005-07-13 |
2855
+ | 008 | Paul | 24 | Houston | 20,000 | 2005-07-13 |
2856
+ | 009 | James | 44 | Norway | 5,000 | 2005-07-13 |
2857
+ | 010 | James | 45 | Texas | 5,000 | |
2858
+ ```
2859
+
2860
+ 2. By Using `FatTable` module-level method calls
2861
+
2862
+ The `FatTable` module provides a set of methods of the form `to_aoa`, `to_text`, etc., to access a `Formatter` without having to create an instance yourself. Without a block, they apply the default formatting to the table and call the `.output` method automatically:
2863
+
2864
+ ```ruby
2865
+ FatTable.to_aoa(tab_a)
2866
+ ```
2867
+
2868
+ ```
2869
+ | Id | Name | Age | Address | Salary | Join Date |
2870
+ |----+-------+-----+------------+--------+------------|
2871
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2872
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2873
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2874
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2875
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2876
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2877
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2878
+ | 10 | James | 45 | Texas | 5000 | |
2879
+ ```
2880
+
2881
+ With a block, these methods yield a `Formatter` instance on which you can call formatting and footer methods. The `.output` method is called on the `Formatter` automatically after the block:
2882
+
2883
+ ```ruby
2884
+ FatTable.to_aoa(tab_a) do |f|
2885
+ f.format(numeric: '0.0,R', id: '3.0C')
2886
+ end
2887
+ ```
2888
+
2889
+ ```
2890
+ | Id | Name | Age | Address | Salary | Join Date |
2891
+ |-----+-------+-----+------------+--------+------------|
2892
+ | 001 | Paul | 32 | California | 20,000 | 2001-07-13 |
2893
+ | 003 | Teddy | 23 | Norway | 20,000 | 2007-12-13 |
2894
+ | 004 | Mark | 25 | Rich-Mond | 65,000 | 2007-12-13 |
2895
+ | 005 | David | 27 | Texas | 85,000 | 2007-12-13 |
2896
+ | 002 | Allen | 25 | Texas | | 2005-07-13 |
2897
+ | 008 | Paul | 24 | Houston | 20,000 | 2005-07-13 |
2898
+ | 009 | James | 44 | Norway | 5,000 | 2005-07-13 |
2899
+ | 010 | James | 45 | Texas | 5,000 | |
2900
+ ```
2901
+
2902
+ 3. By Calling Methods on Table Objects
2903
+
2904
+ Finally, as in many of the examples, you can call methods such as `to_aoa`, `to_text`, etc., directly on a Table:
2905
+
2906
+ ```ruby
2907
+ tab_a.to_aoa
2908
+ ```
2909
+
2910
+ ```
2911
+ | Id | Name | Age | Address | Salary | Join Date |
2912
+ |----+-------+-----+------------+--------+------------|
2913
+ | 1 | Paul | 32 | California | 20000 | 2001-07-13 |
2914
+ | 3 | Teddy | 23 | Norway | 20000 | 2007-12-13 |
2915
+ | 4 | Mark | 25 | Rich-Mond | 65000 | 2007-12-13 |
2916
+ | 5 | David | 27 | Texas | 85000 | 2007-12-13 |
2917
+ | 2 | Allen | 25 | Texas | | 2005-07-13 |
2918
+ | 8 | Paul | 24 | Houston | 20000 | 2005-07-13 |
2919
+ | 9 | James | 44 | Norway | 5000 | 2005-07-13 |
2920
+ | 10 | James | 45 | Texas | 5000 | |
2921
+ ```
2922
+
2923
+ And you can supply a block to them to specify formatting or footers:
2924
+
2925
+ ```ruby
2926
+ tab_a.to_aoa do |f|
2927
+ f.format(numeric: '$0.0,R', id: '3.0C')
2928
+ f.sum_footer(:salary, :age)
2929
+ end
2930
+ ```
2931
+
2932
+ ```
2933
+ | Id | Name | Age | Address | Salary | Join Date |
2934
+ |-------+-------+------+------------+----------+------------|
2935
+ | 001 | Paul | $32 | California | $20,000 | 2001-07-13 |
2936
+ | 003 | Teddy | $23 | Norway | $20,000 | 2007-12-13 |
2937
+ | 004 | Mark | $25 | Rich-Mond | $65,000 | 2007-12-13 |
2938
+ | 005 | David | $27 | Texas | $85,000 | 2007-12-13 |
2939
+ | 002 | Allen | $25 | Texas | | 2005-07-13 |
2940
+ | 008 | Paul | $24 | Houston | $20,000 | 2005-07-13 |
2941
+ | 009 | James | $44 | Norway | $5,000 | 2005-07-13 |
2942
+ | 010 | James | $45 | Texas | $5,000 | |
2943
+ |-------+-------+------+------------+----------+------------|
2944
+ | Total | | $245 | | $220,000 | |
2945
+ ```
2946
+
2947
+
2948
+ <a id="orga37c582"></a>
2949
+
2950
+ # Development
2951
+
2952
+ After checking out the repo, run \`bin/setup\` to install dependencies. Then, run \`rake spec\` to run the tests. You can also run \`bin/console\` for an interactive prompt that will allow you to experiment.
2953
+
2954
+ To install this gem onto your local machine, run \`bundle exec rake install\`.
2955
+
2956
+
2957
+ <a id="org6d38bff"></a>
2958
+
2959
+ # Contributing
2960
+
2961
+ Bug reports and pull requests are welcome on GitHub at <https://github.com/ddoherty03/fat_table>.