fat_table 0.2.6 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.gitignore +3 -0
- data/.rubocop.yml +3 -0
- data/.travis.yml +7 -2
- data/.yardopts +5 -1
- data/README.org +271 -251
- data/README.rdoc +4 -4
- data/TODO.org +7 -0
- data/bin/ft_console +82 -79
- data/fat_table.gemspec +47 -46
- data/lib/fat_table.rb +11 -2
- data/lib/fat_table/column.rb +41 -29
- data/lib/fat_table/db_handle.rb +31 -50
- data/lib/fat_table/evaluator.rb +26 -24
- data/lib/fat_table/formatters/aoa_formatter.rb +5 -6
- data/lib/fat_table/formatters/aoh_formatter.rb +6 -7
- data/lib/fat_table/formatters/formatter.rb +67 -48
- data/lib/fat_table/formatters/latex_formatter.rb +9 -7
- data/lib/fat_table/formatters/org_formatter.rb +8 -9
- data/lib/fat_table/formatters/term_formatter.rb +19 -18
- data/lib/fat_table/formatters/text_formatter.rb +6 -7
- data/lib/fat_table/patches.rb +30 -1
- data/lib/fat_table/table.rb +143 -117
- data/lib/fat_table/version.rb +1 -1
- data/md/README.md +2167 -0
- metadata +78 -84
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: f7870805173324438b3a4c2416e4f59fc3830cd7dc9272b30237cfa1a9ac9d20
|
4
|
+
data.tar.gz: 31e1d6f2c8fdabe9459850e8ca1c570028c63eec1c0c2f61a030421e943d9d66
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: fd60f7879416e544c64c2e1d0efdc4cd349bf496a9c907bf7814d0ddc62c2b170ec46cc9bdb1d241fde7f50c6434233b0b2f4609adce5ec8c2073f32f52fd36e
|
7
|
+
data.tar.gz: f493a05c1b17640a41fffa5bceda6ff01047391a5f9ccb15abca1508e54a2701193912d2a6f5fc40d707b80209c93cf3c1524fd60f9cc55bdb9518b9383a3fec
|
data/.gitignore
CHANGED
data/.rubocop.yml
ADDED
data/.travis.yml
CHANGED
@@ -2,6 +2,9 @@ language: ruby
|
|
2
2
|
before_install:
|
3
3
|
- sudo apt-get -qq update
|
4
4
|
- sudo apt-get install -y texlive-latex-base texlive-latex-recommended
|
5
|
+
- gem install pg
|
6
|
+
# before_script:
|
7
|
+
# - createdb -U travis travis
|
5
8
|
after_failure:
|
6
9
|
- "pwd"
|
7
10
|
- "cat ./spec/tmp/latex.err"
|
@@ -11,7 +14,9 @@ bundler_args: --without debug
|
|
11
14
|
services:
|
12
15
|
- postgresql
|
13
16
|
rvm:
|
14
|
-
- 2.2.2
|
15
|
-
- 2.3
|
16
17
|
- 2.4
|
18
|
+
- 2.5
|
19
|
+
- 2.6
|
20
|
+
- 2.7
|
17
21
|
- ruby-head
|
22
|
+
- truffleruby
|
data/.yardopts
CHANGED
data/README.org
CHANGED
@@ -2,7 +2,11 @@
|
|
2
2
|
#+LATEX_HEADER: \usepackage[margin=0.75in]{geometry}
|
3
3
|
|
4
4
|
#+BEGIN_COMMENT
|
5
|
+
This is for markdown output:
|
6
|
+
|
5
7
|
[![Build Status](https://travis-ci.org/ddoherty03/fat_table.svg?branch=master)](https://travis-ci.org/ddoherty03/fat_table)
|
8
|
+
|
9
|
+
The following is for org.
|
6
10
|
#+END_COMMENT
|
7
11
|
|
8
12
|
[[https://travis-ci.org/ddoherty03/fat_table.svg?branch=master]]
|
@@ -12,7 +16,7 @@
|
|
12
16
|
~FatTable~ is a gem that treats tables as a data type. It provides methods for
|
13
17
|
constructing tables from a variety of sources, building them row-by-row,
|
14
18
|
extracting rows, columns, and cells, and performing aggregate operations on
|
15
|
-
columns. It also provides
|
19
|
+
columns. It also provides a set of SQL-esque methods for manipulating table
|
16
20
|
objects: ~select~ for filtering by columns or for creating new columns, ~where~
|
17
21
|
for filtering by rows, ~order_by~ for sorting rows, ~distinct~ for eliminating
|
18
22
|
duplicate rows, ~group_by~ for aggregating multiple rows into single rows and
|
@@ -31,7 +35,7 @@ no-ops.
|
|
31
35
|
~FatTable~ can be used to perform operations on data that are naturally best
|
32
36
|
conceived of as tables, which in my experience is quite often. It can also serve
|
33
37
|
as a foundation for providing reporting functions where flexibility about the
|
34
|
-
output medium can be
|
38
|
+
output medium can be useful. Finally ~FatTable~ can be used within Emacs
|
35
39
|
~org-mode~ files in code blocks targeting the Ruby language. Org mode tables are
|
36
40
|
presented to a ruby code block as an array of arrays, so ~FatTable~ can read
|
37
41
|
them in with its ~.from_aoa~ constructor. A ~FatTable~ table output as an array
|
@@ -40,6 +44,8 @@ buffer as an org-table, ready for processing by other code blocks.
|
|
40
44
|
|
41
45
|
* Installation
|
42
46
|
|
47
|
+
** Installing the gem
|
48
|
+
|
43
49
|
Add this line to your application's Gemfile:
|
44
50
|
|
45
51
|
#+BEGIN_SRC ruby
|
@@ -63,7 +69,7 @@ Or install it yourself as:
|
|
63
69
|
|
64
70
|
~FatTable~ provides table objects as a data type that can be constructed and
|
65
71
|
operated on in a number of ways. Here's a quick example to illustrate the use of
|
66
|
-
|
72
|
+
~FatTable~. See the detailed explanations further on down.
|
67
73
|
|
68
74
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
69
75
|
#+BEGIN_SRC ruby
|
@@ -152,11 +158,12 @@ the main features of ~FatTable~. See the detailed explanations further on down.
|
|
152
158
|
|
153
159
|
** A Word About the Examples
|
154
160
|
|
155
|
-
When you install the fat_table gem, you have access to a program ~ft_console
|
161
|
+
When you install the ~fat_table~ gem, you have access to a program ~ft_console~,
|
156
162
|
which opens a ~pry~ session with ~fat_table~ loaded and the tables used in the
|
157
|
-
examples in this README defined as instance variables so you can experiment
|
158
|
-
them.
|
159
|
-
as ~@tab1~ in ~ft_console~, but otherwise the examples should work
|
163
|
+
examples in this ~README~ defined as instance variables so you can experiment
|
164
|
+
with them. Because they are defined as instance variables, you have to write
|
165
|
+
~tab1~ as ~@tab1~ in ~ft_console~, but otherwise the examples should work as
|
166
|
+
shown in this ~README~.
|
160
167
|
|
161
168
|
The examples in this ~README~ file are executed as code blocks within the
|
162
169
|
~README.org~ file, so they typically end with a call to ~.to_aoa~. That causes
|
@@ -218,7 +225,7 @@ locals: _ __ _dir_ _ex_ _file_ _in_ _out_ _pry_ lib str version
|
|
218
225
|
[4] pry(main)>
|
219
226
|
#+END_EXAMPLE
|
220
227
|
|
221
|
-
|
228
|
+
If you use ~puts table.to_term~, you can see the effect of the color formatting
|
222
229
|
directives.
|
223
230
|
|
224
231
|
** Anatomy of a Table
|
@@ -228,52 +235,52 @@ directives.
|
|
228
235
|
Each ~Column~ has a header, a type, and an array of items, all of the given type
|
229
236
|
or nil. There are only five permissible types for a ~Column~:
|
230
237
|
|
231
|
-
1. Boolean (for holding ruby ~TrueClass~ and ~FalseClass~ objects),
|
232
|
-
2. DateTime (for holding ruby ~DateTime~ or ~Date~ objects),
|
233
|
-
3. Numeric (for holding ruby ~Integer~, ~Rational~, or ~BigDecimal~ objects),
|
234
|
-
4. String (for ruby String objects), or
|
235
|
-
5. NilClass (for the undetermined column type).
|
238
|
+
1. *Boolean* (for holding ruby ~TrueClass~ and ~FalseClass~ objects),
|
239
|
+
2. *DateTime* (for holding ruby ~DateTime~ or ~Date~ objects),
|
240
|
+
3. *Numeric* (for holding ruby ~Integer~, ~Rational~, or ~BigDecimal~ objects),
|
241
|
+
4. *String* (for ruby ~String~ objects), or
|
242
|
+
5. *NilClass* (for the undetermined column type).
|
236
243
|
|
237
244
|
When a ~Table~ is constructed from an external source, all ~Columns~ start out
|
238
245
|
having a type of ~NilClass~, that is, their type is as yet undetermined. When a
|
239
246
|
string or object of one of the four determined types is added to a ~Column~, it
|
240
247
|
fixes the type of the column and all further items added to the ~Column~ must
|
241
|
-
either be nil (indicating no value) or be capable of being coerced to the
|
248
|
+
either be ~nil~ (indicating no value) or be capable of being coerced to the
|
242
249
|
column's type. Otherwise, ~FatTable~ raises an exception.
|
243
250
|
|
244
251
|
Items of input must be either one of the permissible ruby objects or strings. If
|
245
252
|
they are strings, ~FatTable~ attempts to parse them as one of the permissible
|
246
253
|
types as follows:
|
247
254
|
|
248
|
-
- Boolean :: the strings, 't'
|
249
|
-
interpreted as ~TrueClass~ and the strings, 'f'
|
250
|
-
regardless of case, are interpreted as ~FalseClass~, in
|
251
|
-
resulting in a Boolean column. Empty strings in a column
|
252
|
-
Boolean type are converted to nil
|
253
|
-
- DateTime :: strings that contain patterns of 'yyyy-mm-dd' or 'yyyy/mm/dd'
|
254
|
-
'mm-dd-yyy' or 'mm/dd/yyyy' or any of the foregoing with an added
|
255
|
-
'Thh:mm:ss' or 'Thh:mm' will be interpreted as a ~DateTime~ or a ~Date~
|
256
|
-
there are no sub-day time components present). The number of digits in
|
257
|
-
month and day can be one or two, but the year component must be four
|
255
|
+
- Boolean :: the strings, ~'t'~, ~'true'~, ~'yes'~, or ~'y'~, regardless of
|
256
|
+
case, are interpreted as ~TrueClass~ and the strings, ~'f'~, ~'false'~,
|
257
|
+
~'no'~, or ~'n'~, regardless of case, are interpreted as ~FalseClass~, in
|
258
|
+
either case resulting in a Boolean column. Empty strings in a column
|
259
|
+
already having a Boolean type are converted to ~nil~.
|
260
|
+
- DateTime :: strings that contain patterns of ~'yyyy-mm-dd'~ or ~'yyyy/mm/dd'~
|
261
|
+
or ~'mm-dd-yyy'~ or ~'mm/dd/yyyy'~ or any of the foregoing with an added
|
262
|
+
~'Thh:mm:ss'~ or ~'Thh:mm'~ will be interpreted as a ~DateTime~ or a ~Date~
|
263
|
+
(if there are no sub-day time components present). The number of digits in
|
264
|
+
the month and day can be one or two, but the year component must be four
|
258
265
|
digits. Any time components are valid if they can be properly interpreted
|
259
266
|
by ~DateTime.parse~. Org mode timestamps (any of the foregoing surrounded
|
260
|
-
by square '[]' or pointy '
|
261
|
-
input strings for DateTime columns. Empty strings in a column already
|
262
|
-
having the DateTime type are converted to nil
|
263
|
-
- Numeric :: all commas ','
|
264
|
-
currency symbol as set by ~FatTable.currency_symbol~ are removed from
|
265
|
-
string and if the remaining string can be interpreted as a ~Numeric~,
|
266
|
-
will be. It is interpreted as an ~Integer~ if there are no decimal
|
267
|
-
in the remaining string, as a ~Rational~ if the string has the form
|
268
|
-
'
|
269
|
-
is a decimal point in the remaining string. Empty strings in a column
|
267
|
+
by square '~[]~' or pointy '~<>~' brackets), active or inactive, are valid
|
268
|
+
input strings for ~DateTime~ columns. Empty strings in a column already
|
269
|
+
having the ~DateTime~ type are converted to ~nil~.
|
270
|
+
- Numeric :: all commas ~','~, underscores, ~'_'~, and ~'$'~ dollar signs (or
|
271
|
+
other currency symbol as set by ~FatTable.currency_symbol~ are removed from
|
272
|
+
the string and if the remaining string can be interpreted as a ~Numeric~,
|
273
|
+
it will be. It is interpreted as an ~Integer~ if there are no decimal
|
274
|
+
places in the remaining string, as a ~Rational~ if the string has the form
|
275
|
+
'~<number>:<number>~' or '~<number>/<number>~', or as a ~BigDecimal~ if
|
276
|
+
there is a decimal point in the remaining string. Empty strings in a column
|
270
277
|
already having the Numeric type are converted to nil.
|
271
278
|
- String :: if all else fails, ~FatTable~ applies ~#to_s~ to the input value
|
272
|
-
and, treats it as an item of type ~String~.
|
273
|
-
already having the String type are kept as empty strings.
|
279
|
+
and, treats it as an item of type ~String~. Empty strings in a column
|
280
|
+
already having the ~String~ type are kept as empty strings.
|
274
281
|
- NilClass :: until the input contains a non-blank string that can be parsed as
|
275
282
|
one of the other types, it has this type, meaning that the type is still
|
276
|
-
open. A column comprised completely of blank strings or nils will retain
|
283
|
+
open. A column comprised completely of blank strings or ~nils~ will retain
|
277
284
|
the ~NilClass~ type.
|
278
285
|
|
279
286
|
*** Headers
|
@@ -282,23 +289,23 @@ Headers for the columns are formed from the input. No two columns in a table can
|
|
282
289
|
have the same header. Headers in the input are converted to symbols by
|
283
290
|
|
284
291
|
- converting the header to a string with ~#to_s~,
|
285
|
-
- converting any run of blanks to an underscore
|
292
|
+
- converting any run of blanks to an underscore ~_~,
|
286
293
|
- removing any characters that are not letters, numbers, or underscores, and
|
287
294
|
- lowercasing all remaining letters
|
288
295
|
|
289
|
-
Thus, a header of 'Date' becomes ~:date~, a header of 'Id Number' becomes,
|
296
|
+
Thus, a header of ~'Date'~ becomes ~:date~, a header of ~'Id Number'~ becomes,
|
290
297
|
~:id_number~, etc. When referring to a column in code, you must use the symbol
|
291
298
|
form of the header.
|
292
299
|
|
293
300
|
If no sensible headers can be discerned from the input, headers of the form
|
294
|
-
|
301
|
+
~:col_1~, ~:col_2~, etc., are synthesized.
|
295
302
|
|
296
303
|
*** Groups
|
297
304
|
|
298
305
|
The rows of a ~FatTable~ table can be sub-divided into groups, either from
|
299
306
|
markers in the input or as a result of certain operations. There is only one
|
300
307
|
level of grouping, so ~FatTable~ has no concept of sub-groups. Groups can be
|
301
|
-
shown on output with rules or
|
308
|
+
shown on output with rules or "hlines" that underline the last row in each
|
302
309
|
group, and you can decorate the output with group footers that summarize the
|
303
310
|
columns in each group.
|
304
311
|
|
@@ -310,7 +317,7 @@ You can create an empty table with ~FatTable.new~, and then add rows with the
|
|
310
317
|
|
311
318
|
#+BEGIN_SRC ruby
|
312
319
|
tab = FatTable.new
|
313
|
-
tab << { a: 1, b: 2, c:
|
320
|
+
tab << { a: 1, b: 2, c: "<2017-01-21>', d: 'f', e: '' }
|
314
321
|
tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
|
315
322
|
tab.to_aoa
|
316
323
|
#+END_SRC
|
@@ -325,7 +332,7 @@ in org-mode time stamps.
|
|
325
332
|
|
326
333
|
Tables can also be read from ~.csv~ files or files containing ~org-mode~ tables.
|
327
334
|
In the case of org-mode files, ~FatTable~ skips through the file until it finds
|
328
|
-
a line that look like a table, that is it begins with any number of spaces
|
335
|
+
a line that look like a table, that is, it begins with any number of spaces
|
329
336
|
followed by ~|-~. Only the first table in an ~.org~ file is read.
|
330
337
|
|
331
338
|
For both ~.csv~ and ~.org~ files, the first row in the tables is taken as the
|
@@ -383,21 +390,22 @@ You can also initialize a table directly from ruby data structures. You can, for
|
|
383
390
|
example, build a table from an array of arrays:
|
384
391
|
|
385
392
|
#+BEGIN_SRC ruby
|
386
|
-
aoa =
|
387
|
-
[
|
388
|
-
|
389
|
-
|
390
|
-
|
391
|
-
|
392
|
-
|
393
|
-
|
394
|
-
|
395
|
-
|
396
|
-
|
397
|
-
|
398
|
-
|
399
|
-
|
400
|
-
|
393
|
+
aoa = [
|
394
|
+
['Ref', 'Date', 'Code', 'Raw', 'Shares', 'Price', 'Info', 'Bool'],
|
395
|
+
[1, '2013-05-02', 'P', 795_546.20, 795_546.2, 1.1850, 'ENTITY1', 'T'],
|
396
|
+
[2, '2013-05-02', 'P', 118_186.40, 118_186.4, 11.8500, 'ENTITY1', 'T'],
|
397
|
+
[7, '2013-05-20', 'S', 12_000.00, 5046.00, 28.2804, 'ENTITY3', 'F'],
|
398
|
+
[8, '2013-05-20', 'S', 85_000.00, 35_742.50, 28.3224, 'ENTITY3', 'T'],
|
399
|
+
[9, '2013-05-20', 'S', 33_302.00, 14_003.49, 28.6383, 'ENTITY3', 'T'],
|
400
|
+
[10, '2013-05-23', 'S', 8000.00, 3364.00, 27.1083, 'ENTITY3', 'T'],
|
401
|
+
[11, '2013-05-23', 'S', 23_054.00, 9694.21, 26.8015, 'ENTITY3', 'F'],
|
402
|
+
[12, '2013-05-23', 'S', 39_906.00, 16_780.47, 25.1749, 'ENTITY3', 'T'],
|
403
|
+
[13, '2013-05-29', 'S', 13_459.00, 5659.51, 24.7464, 'ENTITY3', 'T'],
|
404
|
+
[14, '2013-05-29', 'S', 15_700.00, 6601.85, 24.7790, 'ENTITY3', 'F'],
|
405
|
+
[15, '2013-05-29', 'S', 15_900.00, 6685.95, 24.5802, 'ENTITY3', 'T'],
|
406
|
+
[16, '2013-05-30', 'S', 6_679.00, 2808.52, 25.0471, 'ENTITY3', 'T']
|
407
|
+
]
|
408
|
+
tab = FatTable.from_aoa(aoa)
|
401
409
|
#+END_SRC
|
402
410
|
|
403
411
|
Notice that the values can either be ruby objects, such as the Integer ~85_000~,
|
@@ -453,7 +461,7 @@ arrays is displayed as an org-mode table:
|
|
453
461
|
|
454
462
|
This example illustrates several things:
|
455
463
|
|
456
|
-
1. The named org-mode table,
|
464
|
+
1. The named org-mode table, ~trades1~, can be passed into a ruby code block
|
457
465
|
using the ~:var tab=trades1~ header argument to the code block; that makes
|
458
466
|
the variable ~tab~ available to the code block as an array of arrays, which
|
459
467
|
~FatTable~ then uses to initialize the table.
|
@@ -464,19 +472,21 @@ This example illustrates several things:
|
|
464
472
|
where the number of shares is greater than 500. More on that later.
|
465
473
|
4. ~FatTable~ passes back to org-mode an array of arrays using the ~.to_aoa~
|
466
474
|
method. In an ~org-mode~ buffer, these are rendered as tables. We'll often
|
467
|
-
apply ~.to_aoa~ at the end of example blocks to render the
|
468
|
-
this
|
469
|
-
to which formatting
|
475
|
+
apply ~.to_aoa~ at the end of example blocks in this ~README~ to render the
|
476
|
+
results as a table inside this file. As we'll see below, ~.to_aoa~ can also
|
477
|
+
take a block to which formatting and footer directives can be attached.
|
470
478
|
|
471
479
|
*** From Arrays of Hashes
|
472
480
|
|
473
481
|
A second ruby data structure that can be used to initialize a ~FatTable~ table
|
474
482
|
is an array of ruby Hashes. Each hash represents a row of the table, and the
|
475
|
-
headers of the table are
|
476
|
-
hashes should have the same keys.
|
477
|
-
|
478
|
-
|
479
|
-
a
|
483
|
+
headers of the table are taken from the keys of the hashes. Accordingly, all the
|
484
|
+
hashes should have the same keys.
|
485
|
+
|
486
|
+
This same method can in fact take an array of any objects that can be converted
|
487
|
+
to a Hash with the ~#to_h~ method, so you can use an array of your own objects
|
488
|
+
to initialize a table, provided that you define a suitable ~#to_h~ method for
|
489
|
+
the objects' class.
|
480
490
|
|
481
491
|
#+BEGIN_SRC ruby
|
482
492
|
aoh = [
|
@@ -501,40 +511,38 @@ tab = FatTable.from_aoh(aoh)
|
|
501
511
|
#+END_SRC
|
502
512
|
|
503
513
|
Notice, again, that the values can either be ruby objects, such as ~Date.today~,
|
504
|
-
or strings that can parsed into one of the permissible column types.
|
514
|
+
or strings that can be parsed into one of the permissible column types.
|
505
515
|
|
506
516
|
*** From SQL queries
|
507
517
|
|
508
|
-
Another way to initialize a ~FatTable~ table is with the results of a SQL
|
509
|
-
|
510
|
-
|
518
|
+
Another way to initialize a ~FatTable~ table is with the results of a SQL
|
519
|
+
query. Before you can connect to a database, you need to make sure that the required
|
520
|
+
adapter for your database is installed. ~FatTable~ uses the ~sequel~ gem
|
521
|
+
under the hood, so any database that it supports can be used. For example, if
|
522
|
+
you are accessing a Postgres database, you must install the ~pg~ gem with
|
523
|
+
|
524
|
+
#+begin_src sh
|
525
|
+
$ gem install pg
|
526
|
+
#+end_src
|
527
|
+
|
528
|
+
You must first set the database parameters to be used for the queries.
|
511
529
|
|
512
530
|
#+BEGIN_SRC ruby
|
513
531
|
# This automatically requires sequel.
|
514
532
|
require 'fat_table'
|
515
|
-
FatTable.
|
533
|
+
FatTable.connect(adapter: 'postgres',
|
516
534
|
database: 'XXX_development',
|
517
|
-
user: '
|
518
|
-
password: '
|
519
|
-
host: '
|
520
|
-
socket: '/tmp/.s.PGSQL.5432')
|
535
|
+
user: 'ken',
|
536
|
+
password: 'imsecret',
|
537
|
+
host: 'db.lan')
|
521
538
|
tab = FatTable.from_sql('select * from trades;')
|
522
539
|
#+END_SRC
|
523
540
|
|
524
|
-
|
525
|
-
|
526
|
-
|
527
|
-
|
528
|
-
|
529
|
-
required.
|
530
|
-
|
531
|
-
The ~.set_db~ function need only be called once, and the database handle it
|
532
|
-
creates will be used for all subsequent ~.from_sql~ calls until ~.set_db~ is
|
533
|
-
called again.
|
534
|
-
|
535
|
-
Alternatively, you can build the ~Sequel~ connection with ~Sequel.connect~ or
|
536
|
-
with adapter-specific ~Sequel~ connection methods and let ~FatTable~ know to use
|
537
|
-
that connection:
|
541
|
+
The arguments to ~connect~ are simply passed on to ~sequel~'s connect method, so
|
542
|
+
any set of arguments that work for it should work for ~connect~. Alternatively,
|
543
|
+
you can build the ~Sequel~ connection directly with ~Sequel.connect~ or with
|
544
|
+
adapter-specific ~Sequel~ connection methods and let ~FatTable~ know to use that
|
545
|
+
connection:
|
538
546
|
|
539
547
|
#+BEGIN_SRC ruby
|
540
548
|
require 'fat_table'
|
@@ -545,8 +553,15 @@ that connection:
|
|
545
553
|
Consult ~Sequel's~ documentation for details on its connection methods.
|
546
554
|
[[http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html]]
|
547
555
|
|
556
|
+
The ~.connect~ function need only be called once, and the database handle it
|
557
|
+
creates will be used for all subsequent ~.from_sql~ calls until ~.connect~ is
|
558
|
+
called again.
|
559
|
+
|
548
560
|
*** Marking Groups in Input
|
549
561
|
|
562
|
+
~FatTable~ tables has a concept of "groups" of rows that play a role in many of
|
563
|
+
the methods for operating on them as explained [[Groups][below]].
|
564
|
+
|
550
565
|
The ~.from_aoa~ and ~.from_aoh~ functions take an optional keyword parameter
|
551
566
|
~hlines:~ that, if set to ~true~, causes them to mark group boundaries in the
|
552
567
|
table wherever a row Array (for ~.from_aoa~) or Hash (for ~.from_aoh~) is
|
@@ -566,11 +581,12 @@ beginning with something like '~|---~'. The ~.from_org_...~ functions always
|
|
566
581
|
recognizes hlines in the input, so it takes no ~hlines:~ keyword parameter.
|
567
582
|
|
568
583
|
** Accessing Parts of Tables
|
584
|
+
|
569
585
|
*** Rows
|
570
586
|
|
571
587
|
A ~FatTable~ table is an Enumerable, yielding each row of the table as a Hash
|
572
|
-
keyed on the header symbols.
|
573
|
-
Hashes as well.
|
588
|
+
keyed on the header symbols. The method ~Table#rows~ returns an Array of the
|
589
|
+
rows as Hashes as well.
|
574
590
|
|
575
591
|
You can also use indexing to access a row of the table by number. Using an
|
576
592
|
integer index returns a Hash of the given row. Thus, ~tab[20]~ returns the 21st
|
@@ -621,6 +637,7 @@ operation, which mutates the receiver table by removing its group boundaries.
|
|
621
637
|
Second, because each operation returns a ~FatTable::Table~ object, the
|
622
638
|
operations are chainable.
|
623
639
|
|
640
|
+
<<Groups>>
|
624
641
|
Third, ~FatTable::Table~ objects can have "groups" of rows within the table.
|
625
642
|
These can be decorated with hlines and group footers on output. Some of these
|
626
643
|
operations result in marking group boundaries in the result table, others remove
|
@@ -635,8 +652,8 @@ SQL.
|
|
635
652
|
|
636
653
|
For illustration purposes assume that the following tables are read into ruby
|
637
654
|
variables called '~tab1~' and '~tab2~. We have given the table groups, marked by
|
638
|
-
the hlines below, and some duplicate rows to illustrate the effect of
|
639
|
-
operations on groups and duplicates.
|
655
|
+
the hlines below, and included some duplicate rows to illustrate the effect of
|
656
|
+
certain operations on groups and duplicates.
|
640
657
|
|
641
658
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
642
659
|
#+BEGIN_SRC ruby
|
@@ -743,12 +760,13 @@ any group boundaries present in the input table.
|
|
743
760
|
|
744
761
|
**** Adding New Columns
|
745
762
|
|
746
|
-
More interesting is that ~select~ can take hash-like keyword arguments
|
747
|
-
|
748
|
-
|
749
|
-
|
750
|
-
symbol representing an existing column
|
751
|
-
expression for the value of
|
763
|
+
More interesting is that ~select~ can take hash-like keyword arguments after the
|
764
|
+
symbol arguments to create new columns in the output as functions of other
|
765
|
+
columns. For each hash-like parameter, the keyword given must be a symbol, which
|
766
|
+
becomes the header for the new column, and the value must be either: (1) a
|
767
|
+
symbol representing an existing column, which has the effect of renaming an
|
768
|
+
existing column, or (2) a string representing a ruby expression for the value of
|
769
|
+
a new column.
|
752
770
|
|
753
771
|
Within the string expression, the names of existing or already-specified columns
|
754
772
|
are available as local variables, as well as the instance variables '@row' and
|
@@ -759,8 +777,8 @@ access to local variables ~ref~, ~date~, ~code~, ~price~, ~g10~, ~qp10~,
|
|
759
777
|
their respective columns for each row in the input table and the instance
|
760
778
|
variables are set the number of the current row and group respectively.
|
761
779
|
|
762
|
-
For example, if we want to rename the
|
763
|
-
shares, we could do the following:
|
780
|
+
For example, if we want to rename the ~:date~ column and add a new column to
|
781
|
+
compute the cost of shares, we could do the following:
|
764
782
|
|
765
783
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
766
784
|
#+BEGIN_SRC ruby
|
@@ -840,18 +858,18 @@ second, chained call to ~select~:
|
|
840
858
|
**** Custom Instance Variables and Hooks
|
841
859
|
|
842
860
|
As the above examples demonstrate, the instance variables ~@row~ and ~@group~
|
843
|
-
are available when evaluating expressions that add new columns.
|
844
|
-
your own instance variables as well for keeping track of things that cross
|
845
|
-
boundaries, such as running sums.
|
861
|
+
are available when evaluating expressions that add new columns. You can also set
|
862
|
+
up your own instance variables as well for keeping track of things that cross
|
863
|
+
row boundaries, such as running sums.
|
846
864
|
|
847
865
|
To declare instance variables, you can use the ~ivars:~ hash parameter to
|
848
866
|
~select~. Each key of the hash becomes an instance variable and each value
|
849
867
|
becomes its initial value before any rows are evaluated.
|
850
868
|
|
851
|
-
In addition, you can provide ~before_hook:~ and ~after_hook:~ parameters
|
852
|
-
strings that are evaluated as ruby expressions before and after each
|
853
|
-
processed.
|
854
|
-
the ~before_hook:~ can be used in expressions for adding new columns by
|
869
|
+
In addition, you can provide ~before_hook:~ and ~after_hook:~ parameters to
|
870
|
+
~select~ as strings that are evaluated as ruby expressions before and after each
|
871
|
+
row is processed. You can use these to update instance variables. The values set
|
872
|
+
in the ~before_hook:~ can be used in expressions for adding new columns by
|
855
873
|
referencing them with the '@' prefix.
|
856
874
|
|
857
875
|
For example, suppose we wanted to not only add a cost column, but a column that
|
@@ -942,28 +960,21 @@ You can sort a table on any number of columns with ~order_by~. The ~order_by~
|
|
942
960
|
method takes any number of symbol arguments for the columns to sort on. If you
|
943
961
|
specify more than one column, the sort is performed on the first column, then
|
944
962
|
all columns that are equal with respect to the first column are sorted by the
|
945
|
-
second column, and so on.
|
946
|
-
|
963
|
+
second column, and so on. Ordering is done is ascending order for each of the
|
964
|
+
columns, but can be reversed by adding a '!' to the end a symbol argument.
|
965
|
+
All columns of the input table are included in the output.
|
947
966
|
|
948
|
-
Let's sort our table first by ~:code~, then
|
967
|
+
Let's sort our table first by ~:code~, then in reverse order of ~:date~.
|
949
968
|
|
950
969
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
951
970
|
#+BEGIN_SRC ruby
|
952
|
-
tab1.order_by(:code, :date) \
|
971
|
+
tab1.order_by(:code, :date!) \
|
953
972
|
.to_aoa
|
954
973
|
#+END_SRC
|
955
974
|
|
956
|
-
#+
|
975
|
+
#+begin_EXAMPLE
|
957
976
|
| Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
|
958
977
|
|------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
|
959
|
-
| T001 | 2016-11-01 | P | 7.7 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
|
960
|
-
| T002 | 2016-11-01 | P | 7.75 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
|
961
|
-
| T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
|
962
|
-
| T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
|
963
|
-
| T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
|
964
|
-
| T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
|
965
|
-
| T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
|
966
|
-
|------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
|
967
978
|
| T011 | 2016-11-02 | P | 7.425 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
|
968
979
|
| T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
|
969
980
|
| T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
|
@@ -972,12 +983,20 @@ Let's sort our table first by ~:code~, then by ~:date~.
|
|
972
983
|
| T015 | 2016-11-02 | P | 7.75 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
|
973
984
|
| T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
|
974
985
|
|------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
|
986
|
+
| T001 | 2016-11-01 | P | 7.7 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
|
987
|
+
| T002 | 2016-11-01 | P | 7.75 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
|
988
|
+
| T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
|
989
|
+
| T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
|
990
|
+
| T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
|
991
|
+
| T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
|
992
|
+
| T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
|
993
|
+
|------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
|
975
994
|
| T004 | 2016-11-01 | S | 7.55 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
|
976
995
|
| T005 | 2016-11-01 | S | 7.5 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
|
977
996
|
| T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
|
978
997
|
| T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
|
979
998
|
| T007 | 2016-11-01 | S | 7.65 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
|
980
|
-
#+
|
999
|
+
#+end_EXAMPLE
|
981
1000
|
|
982
1001
|
The interesting thing about ~order_by~ is that, while it ignores groups in its
|
983
1002
|
input, it adds group boundaries in the output table at those rows where the sort
|
@@ -996,7 +1015,8 @@ which the grouping parameters are equal containing those columns and an
|
|
996
1015
|
aggregate column for each of the aggregating parameters.
|
997
1016
|
|
998
1017
|
For example, let's summarize the ~trades~ table by ~:code~ and ~:price~ again,
|
999
|
-
and determine total shares, average price, and other features of each
|
1018
|
+
and determine total shares, average price, and a few other features of each
|
1019
|
+
group:
|
1000
1020
|
|
1001
1021
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
1002
1022
|
#+BEGIN_SRC ruby
|
@@ -1030,38 +1050,39 @@ will raise an exception.
|
|
1030
1050
|
|
1031
1051
|
- ~first~ :: the first non-nil item in the column,
|
1032
1052
|
- ~last~ :: the last non-nil item in the column,
|
1033
|
-
- ~rng~ :: form a string of the form "#{first}..#{last}" to show the range of
|
1053
|
+
- ~rng~ :: form a string of the form ~"#{first}..#{last}"~ to show the range of
|
1034
1054
|
values in the column,
|
1035
|
-
- ~sum~ :: for Numeric and String columns, apply '+' to all the non-nil
|
1055
|
+
- ~sum~ :: for ~Numeric~ and ~String~ columns, apply '+' to all the non-nil
|
1056
|
+
values,
|
1036
1057
|
- ~count~ :: the number of non-nil values in the column,
|
1037
|
-
- ~min~ :: for Numeric
|
1038
|
-
value in the column,
|
1039
|
-
- ~max~ :: for Numeric
|
1040
|
-
value in the column,
|
1041
|
-
- ~avg~ :: for Numeric and DateTime columns, return the arithmetic mean of
|
1042
|
-
non-nil values in the column; with respect to
|
1043
|
-
converted to a numeric Julian date, the average is
|
1044
|
-
result converted back to a Date or DateTime object,
|
1045
|
-
- ~var~ :: for Numeric and DateTime columns, compute the sample variance of
|
1046
|
-
non-nil values in the column, dates are converted to
|
1047
|
-
|
1048
|
-
- ~pvar~ :: for Numeric and DateTime columns, compute the population
|
1049
|
-
the non-nil values in the column, dates are converted to
|
1050
|
-
|
1051
|
-
- ~dev~ :: for Numeric and DateTime columns, compute the sample standard
|
1052
|
-
deviation of the non-nil values in the column, dates are converted to
|
1053
|
-
numbers as for the :avg aggregate,
|
1054
|
-
- ~pdev~ :: for Numeric and DateTime columns, compute the population standard
|
1058
|
+
- ~min~ :: for ~Numeric~, ~String~, and ~DateTime~ columns, return the smallest
|
1059
|
+
non-nil value in the column,
|
1060
|
+
- ~max~ :: for ~Numeric~, ~String~, and ~DateTime~ columns, return the largest
|
1061
|
+
non-nil value in the column,
|
1062
|
+
- ~avg~ :: for ~Numeric~ and ~DateTime~ columns, return the arithmetic mean of
|
1063
|
+
the non-nil values in the column; with respect to ~Date~ or ~DateTime~
|
1064
|
+
objects, each is converted to a numeric Julian date, the average is
|
1065
|
+
calculated, and the result converted back to a ~Date~ or ~DateTime~ object,
|
1066
|
+
- ~var~ :: for ~Numeric~ and ~DateTime~ columns, compute the sample variance of
|
1067
|
+
the non-nil values in the column, dates are converted to Julian date
|
1068
|
+
numbers as for the ~:avg~ aggregate,
|
1069
|
+
- ~pvar~ :: for ~Numeric~ and ~DateTime~ columns, compute the population
|
1070
|
+
variance of the non-nil values in the column, dates are converted to Julian
|
1071
|
+
date numbers as for the ~:avg~ aggregate,
|
1072
|
+
- ~dev~ :: for ~Numeric~ and ~DateTime~ columns, compute the sample standard
|
1055
1073
|
deviation of the non-nil values in the column, dates are converted to
|
1056
|
-
numbers as for the
|
1057
|
-
- ~
|
1074
|
+
Julian date numbers as for the ~:avg~ aggregate,
|
1075
|
+
- ~pdev~ :: for ~Numeric~ and ~DateTime~ columns, compute the population
|
1076
|
+
standard deviation of the non-nil values in the column, dates are converted
|
1077
|
+
to numbers as for the ~:avg~ aggregate,
|
1078
|
+
- ~all?~ :: for ~Boolean~ columns only, return true if all of the non-nil values
|
1058
1079
|
in the column are true,
|
1059
|
-
- ~any?~ :: for Boolean columns only, return true if any non-nil value in the
|
1080
|
+
- ~any?~ :: for ~Boolean~ columns only, return true if any non-nil value in the
|
1060
1081
|
column is true,
|
1061
|
-
- ~none?~ :: for Boolean columns only, return true if no non-nil value in the
|
1082
|
+
- ~none?~ :: for ~Boolean~ columns only, return true if no non-nil value in the
|
1062
1083
|
column is true,
|
1063
|
-
- ~one?~ :: for Boolean columns only, return true if exactly one non-nil value
|
1064
|
-
the column is true,
|
1084
|
+
- ~one?~ :: for ~Boolean~ columns only, return true if exactly one non-nil value
|
1085
|
+
in the column is true,
|
1065
1086
|
|
1066
1087
|
Perhaps surprisingly, the ~group_by~ method ignores any groups in its input and
|
1067
1088
|
results in no group boundaries in the output since each group formed by the
|
@@ -1073,36 +1094,37 @@ implicit ~order_by~ on the grouping columns is collapsed into a single row.
|
|
1073
1094
|
So far, all the operations have operated on a single table. ~FatTable~ provides
|
1074
1095
|
several ~join~ methods for combining two tables, each of which takes as
|
1075
1096
|
parameters (1) a second table and (2) except in the case of ~cross_join~, zero
|
1076
|
-
or more "join expressions".
|
1077
|
-
the method is called, ~T2~ is the table supplied as the first parameter
|
1078
|
-
and ~R1~ and ~R2~ are rows in their respective tables being considered
|
1079
|
-
inclusion in the joined output table.
|
1097
|
+
or more "join expressions". In the descriptions below, ~T1~ is the table on
|
1098
|
+
which the method is called, ~T2~ is the table supplied as the first parameter
|
1099
|
+
~other~, and ~R1~ and ~R2~ are rows in their respective tables being considered
|
1100
|
+
for inclusion in the joined output table.
|
1080
1101
|
|
1081
1102
|
- ~join(other, *jexps)~ :: Performs an "inner join" on the tables. For each row
|
1082
|
-
R1 of T1
|
1083
|
-
join condition with R1
|
1103
|
+
~R1~ of ~T1~, the joined table has a row for each row in ~T2~ that
|
1104
|
+
satisfies the join condition with ~R1~.
|
1084
1105
|
|
1085
1106
|
- ~left_join(other, *jexps)~ :: First, an inner join is performed. Then, for
|
1086
|
-
each row in T1 that does not satisfy the join condition with any row in
|
1087
|
-
a joined row is added with null values in columns of T2
|
1088
|
-
table always has at least one row for each row in T1
|
1107
|
+
each row in ~T1~ that does not satisfy the join condition with any row in
|
1108
|
+
~T2~, a joined row is added with null values in columns of ~T2~. Thus, the
|
1109
|
+
joined table always has at least one row for each row in ~T1~.
|
1089
1110
|
|
1090
1111
|
- ~right_join(other, *jexps)~ :: First, an inner join is performed. Then, for
|
1091
|
-
each row in T2 that does not satisfy the join condition with any row in
|
1092
|
-
a joined row is added with null values in columns of T1
|
1093
|
-
converse of a left join: the result table will always have a row for
|
1094
|
-
row in T2
|
1112
|
+
each row in ~T2~ that does not satisfy the join condition with any row in
|
1113
|
+
~T1~, a joined row is added with null values in columns of ~T1~. This is
|
1114
|
+
the converse of a left join: the result table will always have a row for
|
1115
|
+
each row in ~T2~.
|
1095
1116
|
|
1096
1117
|
- ~full_join(other, *jexps)~ :: First, an inner join is performed. Then, for
|
1097
|
-
each row in T1 that does not satisfy the join condition with any row in
|
1098
|
-
a joined row is added with null values in columns of T2
|
1099
|
-
of T2 that does not satisfy the join condition with any row in
|
1100
|
-
row with null values in the columns of T1 is added.
|
1118
|
+
each row in ~T1~ that does not satisfy the join condition with any row in
|
1119
|
+
~T2~, a joined row is added with null values in columns of ~T2~. Also, for
|
1120
|
+
each row of ~T2~ that does not satisfy the join condition with any row in
|
1121
|
+
~T1~, a joined row with null values in the columns of ~T1~ is added.
|
1101
1122
|
|
1102
|
-
- ~cross_join(other)~ :: For every possible combination of rows from T1 and
|
1103
|
-
(i.e., a Cartesian product), the joined table will contain a row
|
1104
|
-
of all columns in T1 followed by all columns in T2
|
1105
|
-
and M rows respectively, the joined table will have N *
|
1123
|
+
- ~cross_join(other)~ :: For every possible combination of rows from ~T1~ and
|
1124
|
+
~T2~ (i.e., a Cartesian product), the joined table will contain a row
|
1125
|
+
consisting of all columns in ~T1~ followed by all columns in ~T2~. If the
|
1126
|
+
tables have ~N~ and ~M~ rows respectively, the joined table will have ~N *
|
1127
|
+
M~ rows.
|
1106
1128
|
|
1107
1129
|
**** Join Expressions
|
1108
1130
|
|
@@ -1116,8 +1138,8 @@ that the values of both tables are equal for all columns named by the symbols. A
|
|
1116
1138
|
column that appears in both tables can be given without modification and will be
|
1117
1139
|
assumed to require equality on that column. If an unmodified symbol is not a
|
1118
1140
|
name that appears in both tables, an exception will be raised. Column names that
|
1119
|
-
are unique to the first table must have a
|
1120
|
-
column names that are unique to the other table must have a
|
1141
|
+
are unique to the first table must have a ~_a~ appended to the column name and
|
1142
|
+
column names that are unique to the other table must have a ~_b~ appended to the
|
1121
1143
|
column name. These disambiguated column names must come in pairs, one for the
|
1122
1144
|
first table and one for the second, and they will imply a join condition that
|
1123
1145
|
the columns must be equal on those columns. Several such symbol expressions will
|
@@ -1126,7 +1148,7 @@ be met.
|
|
1126
1148
|
|
1127
1149
|
Finally, a join expression can be a string that contains an arbitrary ruby
|
1128
1150
|
expression that will be evaluated for truthiness. Within the string, /all/
|
1129
|
-
column names must be disambiguated with the
|
1151
|
+
column names must be disambiguated with the ~_a~ or ~_b~ modifiers whether they
|
1130
1152
|
are common to both tables or not. As with ~select~ and ~where~ methods, the
|
1131
1153
|
names of the columns in both tables (albeit disambiguated) are available as
|
1132
1154
|
local variables within the expression, but the instance variables ~@row~ and
|
@@ -1134,9 +1156,9 @@ local variables within the expression, but the instance variables ~@row~ and
|
|
1134
1156
|
|
1135
1157
|
**** Join Examples
|
1136
1158
|
|
1137
|
-
The following examples are taken from
|
1138
|
-
|
1139
|
-
|
1159
|
+
The following examples are taken from the [[https://www.tutorialspoint.com/postgresql/postgresql_using_joins.htm][Postgresql tutorial]], with some slight
|
1160
|
+
modifications. The examples will use the following two tables, which are also
|
1161
|
+
available in ~ft_console~ as ~@tab_a~ and ~@tab_b~:
|
1140
1162
|
|
1141
1163
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
1142
1164
|
#+BEGIN_SRC ruby
|
@@ -1326,10 +1348,10 @@ Finally, a cross join outputs every row of ~tab_a~ augmented with every row of
|
|
1326
1348
|
|
1327
1349
|
*** Set Operations
|
1328
1350
|
|
1329
|
-
~FatTable~ can perform several set operations on tables.
|
1330
|
-
|
1331
|
-
|
1332
|
-
|
1351
|
+
~FatTable~ can perform several set operations on tables. In order for two tables
|
1352
|
+
to be used this way, they must have the same number of columns with the same
|
1353
|
+
types or an exception will be raised. We'll call two tables that qualify for
|
1354
|
+
combining with set operations "set-compatible."
|
1333
1355
|
|
1334
1356
|
We'll use the following two set-compatible tables in the examples. They each
|
1335
1357
|
have some duplicates and some group boundaries so you can see the effect of the
|
@@ -1400,7 +1422,7 @@ set operations on duplicates and groups.
|
|
1400
1422
|
Two tables that are set-compatible can be combined with the ~union~ or
|
1401
1423
|
~union_all~ methods so that the rows of both tables appear in the output. In the
|
1402
1424
|
output table, the headers of the receiver table are used. You can use ~select~
|
1403
|
-
to change or re-order the headers if you prefer.
|
1425
|
+
to change or re-order the headers if you prefer. The ~union~ method eliminates
|
1404
1426
|
duplicate rows in the result table, the ~union_all~ method does not.
|
1405
1427
|
|
1406
1428
|
Any group boundaries in the input tables are destroyed by ~union~ but are
|
@@ -1511,7 +1533,7 @@ eliminating any duplicate rows in the result.
|
|
1511
1533
|
#+END_EXAMPLE
|
1512
1534
|
|
1513
1535
|
With ~intersect_all~, all the rows of the first table, including duplicates, are
|
1514
|
-
included in the result if they also occur in the second table.
|
1536
|
+
included in the result if they also occur in the second table. However,
|
1515
1537
|
duplicates in the second table do not appear.
|
1516
1538
|
|
1517
1539
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
@@ -1710,21 +1732,20 @@ receiver table by removing its groups.
|
|
1710
1732
|
** Formatting Tables
|
1711
1733
|
|
1712
1734
|
Besides creating and operating on tables, you may want to display the resulting
|
1713
|
-
table.
|
1714
|
-
most common across many output media.
|
1735
|
+
table. ~FatTable~ seeks to provide a set of formatting directives that are the
|
1736
|
+
most common across many output media. It provides directives for alignment, for
|
1715
1737
|
color, for adding currency symbols and grouping commas to numbers, for padding
|
1716
1738
|
numbers, and for formatting dates and booleans.
|
1717
1739
|
|
1718
1740
|
In addition, you can add any number of footers to a table, which appear at the
|
1719
1741
|
end of the table, and any number of group footers, which appear after each group
|
1720
|
-
in the table.
|
1742
|
+
in the table. These can be formatted independently of the table body.
|
1721
1743
|
|
1722
1744
|
If the target output medium does not support a formatting directive or the
|
1723
|
-
directive
|
1724
|
-
|
1725
|
-
|
1726
|
-
|
1727
|
-
alignment are simply irrelevant.
|
1745
|
+
directive does not make sense, it is simply ignored. For example, you can output
|
1746
|
+
an ~org-mode~ table as a String, and since ~org-mode~ does not support colors,
|
1747
|
+
any color directives are ignored. Some of the output targets are not strings,
|
1748
|
+
but ruby data structures, and for them, things such as alignment are irrelevant.
|
1728
1749
|
|
1729
1750
|
*** Available Formatters
|
1730
1751
|
|
@@ -1743,13 +1764,13 @@ alignment are simply irrelevant.
|
|
1743
1764
|
|
1744
1765
|
These are all implemented by classes that inherit from ~FatTable::Formatter~
|
1745
1766
|
class by defining about a dozen methods that get called at various places during
|
1746
|
-
the construction of the output table.
|
1767
|
+
the construction of the output table. The idea is that more classes can be
|
1747
1768
|
defined by adding additional classes.
|
1748
1769
|
|
1749
1770
|
*** Table Locations
|
1750
1771
|
|
1751
1772
|
In the formatting methods, the table is divided into several "locations" for
|
1752
|
-
which separate formatting directives may be given.
|
1773
|
+
which separate formatting directives may be given. These locations are
|
1753
1774
|
identified with the following symbols:
|
1754
1775
|
|
1755
1776
|
- :header :: the first row of the output table containing the headers,
|
@@ -1794,8 +1815,8 @@ to a string in forming the output.
|
|
1794
1815
|
- _ ~_ :: underline the element, or turn off underline
|
1795
1816
|
- * ~* :: cause the element to blink, or turn off blink
|
1796
1817
|
|
1797
|
-
For example, the directive 'tCc[red.yellow]' would title-case the element,
|
1798
|
-
center it, and color it red on a yellow background.
|
1818
|
+
For example, the directive ~'tCc[red.yellow]'~ would title-case the element,
|
1819
|
+
center it, and color it red on a yellow background. The directives that are
|
1799
1820
|
boolean have negating forms so that, for example, if bold is turned on for all
|
1800
1821
|
columns of a given type, it can be countermanded in formatting directives for
|
1801
1822
|
particular columns.
|
@@ -1811,65 +1832,63 @@ addition to the following:
|
|
1811
1832
|
the left with zeroes as needed, and round the number to the n
|
1812
1833
|
decimal places and include n digits after the decimal point,
|
1813
1834
|
padding on the right with zeroes as needed,
|
1814
|
-
- H :: convert the number (assumed to be in units of seconds) to
|
1815
|
-
|
1816
|
-
|
1817
|
-
|
1818
|
-
formatting instruction.
|
1835
|
+
- H :: convert the number (assumed to be in units of seconds) to ~HH:MM:SS.ss~
|
1836
|
+
form. So a column that is the result of subtracting two :datetime forms
|
1837
|
+
will result in a :numeric expressed as seconds and can be displayed in
|
1838
|
+
hours, minutes, and seconds with this formatting instruction.
|
1819
1839
|
|
1820
|
-
For example, the directive 'R5.0c[blue]' would right-align the numeric
|
1821
|
-
pad it on the left with zeros, and color it blue.
|
1840
|
+
For example, the directive ~'R5.0c[blue]'~ would right-align the numeric
|
1841
|
+
element, pad it on the left with zeros, and color it blue.
|
1822
1842
|
|
1823
1843
|
**** DateTime
|
1824
1844
|
|
1825
|
-
For a
|
1826
|
-
to the following:
|
1845
|
+
For a ~DateTime~, all the instructions valid for string are available, in
|
1846
|
+
addition to the following:
|
1827
1847
|
|
1828
|
-
- d[fmt] :: apply the format to a
|
1829
|
-
no or zero hour, minute, and second components, where fmt
|
1830
|
-
string for Date#strftime
|
1831
|
-
ISO 8601 string, YYYY-MM-DD.
|
1832
|
-
- D[fmt] :: apply the format to a datetime that has at least a non-zero
|
1833
|
-
|
1834
|
-
|
1835
|
-
8601 string, YYYY-MM-DD.
|
1848
|
+
- d[fmt] :: apply the format to a ~Date~ or a ~DateTime~ that is a whole day,
|
1849
|
+
that is that has no or zero hour, minute, and second components, where fmt
|
1850
|
+
is a valid format string for ~Date#strftime~, otherwise, the datetime will
|
1851
|
+
be formatted as an ISO 8601 string, YYYY-MM-DD.
|
1852
|
+
- D[fmt] :: apply the format to a datetime that has at least a non-zero hour
|
1853
|
+
component where fmt is a valid format string for Date#strftime, otherwise,
|
1854
|
+
the datetime will be formatted as an ISO 8601 string, YYYY-MM-DD.
|
1836
1855
|
|
1837
|
-
For example, 'c[pink]d[%b %-d, %Y]C'
|
1838
|
-
1957', center it, and color it pink.
|
1856
|
+
For example, ~'c[pink]d[%b %-d, %Y]C'~, would format a date element like 'Sep
|
1857
|
+
22, 1957', center it, and color it pink.
|
1839
1858
|
|
1840
1859
|
**** Boolean
|
1841
1860
|
|
1842
1861
|
For a boolean cell, all the instructions valid for string are available, in
|
1843
|
-
|
1862
|
+
addition to the following:
|
1844
1863
|
|
1845
|
-
- Y :: print true as 'Y' and false as 'N',
|
1846
|
-
- T :: print true as 'T' and false as 'F',
|
1847
|
-
- X :: print true as 'X' and false as '',
|
1848
|
-
- b[xxx,yyy] :: print true as the string given as xxx and false as the
|
1849
|
-
|
1850
|
-
- c[tcolor,fcolor] :: color a true element with tcolor and a false
|
1851
|
-
|
1852
|
-
|
1864
|
+
- Y :: print true as '~Y~' and false as '~N~',
|
1865
|
+
- T :: print true as '~T~' and false as '~F~',
|
1866
|
+
- X :: print true as '~X~' and false as an empty string '',
|
1867
|
+
- b[xxx,yyy] :: print true as the string given as ~xxx~ and false as the string
|
1868
|
+
given as ~yyy~,
|
1869
|
+
- c[tcolor,fcolor] :: color a true element with ~tcolor~ and a false element
|
1870
|
+
with ~fcolor~. Each of the colors may be specified in the same manner as
|
1871
|
+
colors for strings described above.
|
1853
1872
|
|
1854
|
-
For example, the directive 'b[Yeppers,Nope]c[green.pink,red.pink]' would
|
1855
|
-
a true boolean as 'Yeppers' colored green on pink and render a false
|
1856
|
-
'Nope' colored red on pink. See [[https://www.youtube.com/watch?v=oLdFFD8II8U][Yeppers]].
|
1873
|
+
For example, the directive '~b[Yeppers,Nope]c[green.pink,red.pink]~' would
|
1874
|
+
render a true boolean as '~Yeppers~' colored green on pink and render a false
|
1875
|
+
boolean as '~Nope~' colored red on pink. See [[https://www.youtube.com/watch?v=oLdFFD8II8U][Yeppers]] for additional information.
|
1857
1876
|
|
1858
1877
|
**** NilClass
|
1859
1878
|
|
1860
|
-
By default, nil elements are rendered as blank cells, but you can make them
|
1879
|
+
By default, ~nil~ elements are rendered as blank cells, but you can make them
|
1861
1880
|
visible with the following, and in that case, all the formatting instructions
|
1862
1881
|
valid for strings are also available:
|
1863
1882
|
|
1864
|
-
- n[niltext] :: render a nil item with the given
|
1883
|
+
- n[niltext] :: render a ~nil~ item with the given niltext.
|
1865
1884
|
|
1866
|
-
For example, you might want to use 'n[-]Cc[purple]' to make nils visible as a
|
1867
|
-
centered hyphen.
|
1885
|
+
For example, you might want to use ~'n[-]Cc[purple]'~ to make nils visible as a
|
1886
|
+
centered purple hyphen.
|
1868
1887
|
|
1869
1888
|
*** Footers Methods
|
1870
1889
|
|
1871
|
-
You can call methods on
|
1872
|
-
Their signatures are:
|
1890
|
+
You can call the ~footer~ and ~gfooter~ methods on ~Formatter~ objects to add
|
1891
|
+
footers and group footers. Their signatures are:
|
1873
1892
|
|
1874
1893
|
- ~footer(label, *sum_cols, **agg_cols)~ :: where ~label~ is a label to be
|
1875
1894
|
placed in the first cell of the footer (unless that column is named as one
|
@@ -1877,8 +1896,8 @@ Their signatures are:
|
|
1877
1896
|
~*sum_cols~ are zero or more symbols for columns to be summed, and
|
1878
1897
|
~**agg_cols~ is zero or more hash-like parameters with a column symbol as a
|
1879
1898
|
key and a symbol for an aggregate method as the value. This causes a
|
1880
|
-
table-wide header to be added at the bottom of the table applying the
|
1881
|
-
aggregate to the ~sum_cols~ and the named aggregate method to the
|
1899
|
+
table-wide header to be added at the bottom of the table applying the
|
1900
|
+
~:sum~ aggregate to the ~sum_cols~ and the named aggregate method to the
|
1882
1901
|
~agg_cols~. A table can have any number of footers attached, and they will
|
1883
1902
|
appear at the bottom of the output table in the order they are given.
|
1884
1903
|
|
@@ -1915,10 +1934,10 @@ and ~format~.
|
|
1915
1934
|
**** Instantiating a Formatter
|
1916
1935
|
|
1917
1936
|
There are several ways to invoke the formatting methods on a table. First, you
|
1918
|
-
can instantiate a ~XXXFormatter~ object
|
1919
|
-
subclass for each target output medium, for example,
|
1920
|
-
a ruby array of arrays. You can then call the
|
1921
|
-
~XXXFormatter~.
|
1937
|
+
can instantiate a ~XXXFormatter~ object and feed it a table as a parameter.
|
1938
|
+
There is a Formatter subclass for each target output medium, for example,
|
1939
|
+
~AoaFormatter~ will produce a ruby array of arrays. You can then call the
|
1940
|
+
~output~ method on the ~XXXFormatter~.
|
1922
1941
|
|
1923
1942
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
1924
1943
|
#+BEGIN_SRC ruby
|
@@ -2072,12 +2091,12 @@ Other than that first parameter, the two methods take the same types of
|
|
2072
2091
|
parameters. The remaining parameters are hash-like parameters that use either a
|
2073
2092
|
column name or a type as the key and a string with the formatting directives to
|
2074
2093
|
apply as the value. The following example says to set the formatting for all
|
2075
|
-
locations in the table and to
|
2076
|
-
numbers (the '0.0' part), that are right-aligned (the 'R'
|
2077
|
-
grouping commas inserted (the ',' part). But the ~:id~ column is
|
2078
|
-
the second parameter overrides the formatting for numerics in
|
2079
|
-
for the ~:id~ column to be padded to three digits with zeros
|
2080
|
-
'3.0' part) and to be centered (the 'C' part).
|
2094
|
+
locations in the table and to format all numeric fields as strings that are
|
2095
|
+
rounded to whole numbers (the '0.0' part), that are right-aligned (the 'R'
|
2096
|
+
part), and have grouping commas inserted (the ',' part). But the ~:id~ column is
|
2097
|
+
numeric, and the second parameter overrides the formatting for numerics in
|
2098
|
+
general and calls for the ~:id~ column to be padded to three digits with zeros
|
2099
|
+
on the left (the '3.0' part) and to be centered (the 'C' part).
|
2081
2100
|
|
2082
2101
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
2083
2102
|
#+BEGIN_SRC ruby
|
@@ -2100,14 +2119,14 @@ for the ~:id~ column to be padded to three digits with zeros on the left (the
|
|
2100
2119
|
#+END_EXAMPLE
|
2101
2120
|
|
2102
2121
|
The ~numeric:~ directive affected the ~:age~ and ~:salary~ columns and the ~id:~
|
2103
|
-
directive affected only the ~:id~ column.
|
2122
|
+
directive affected only the ~:id~ column. All the other cells in the table had
|
2104
2123
|
the default formatting applied.
|
2105
2124
|
|
2106
2125
|
**** Location priority
|
2107
2126
|
|
2108
|
-
Formatting for any given cell depends on its location in the table.
|
2127
|
+
Formatting for any given cell depends on its location in the table. The
|
2109
2128
|
~format_for~ method takes a location to which its formatting directive are
|
2110
|
-
restricted as the first argument.
|
2129
|
+
restricted as the first argument. It can be one of the following:
|
2111
2130
|
|
2112
2131
|
- ~:header~ :: directive apply only to the header row, that is the first row, of
|
2113
2132
|
the output table,
|
@@ -2122,11 +2141,11 @@ restricted as the first argument. It can be one of the following:
|
|
2122
2141
|
row is the first row in the table or in a group and separate directives for
|
2123
2142
|
those have been given, in which case those directives apply,
|
2124
2143
|
|
2125
|
-
- ~:gfirst~ :: directives apply to the first row in each group in the
|
2126
|
-
table, unless the row is also the first row in the table as a whole, in
|
2144
|
+
- ~:gfirst~ :: directives apply to the first row in each group in the body of
|
2145
|
+
the table, unless the row is also the first row in the table as a whole, in
|
2127
2146
|
which case the ~:bfirst~ directives apply,
|
2128
2147
|
|
2129
|
-
- ~:bfirst~ :: directives apply to the first row in the table.
|
2148
|
+
- ~:bfirst~ :: directives apply to the first row in the body of the table.
|
2130
2149
|
|
2131
2150
|
If you give directives for ~:body~, they are copied to ~:bfirst~ and ~:gfirst~
|
2132
2151
|
as well and can be overridden by directives for those locations.
|
@@ -2171,7 +2190,8 @@ require 'fat_table'
|
|
2171
2190
|
|
2172
2191
|
The ~string: 'R'~ directive causes all the cells to be right-aligned except
|
2173
2192
|
~:id~ which specifies centering for the ~:id~ column only. The ~n[N/A]~
|
2174
|
-
directive for
|
2193
|
+
directive for specifies how nil are displayed in the numeric column, ~:salary~,
|
2194
|
+
but not for other nils, such as in the last row of the ~:join_date~ column.
|
2175
2195
|
|
2176
2196
|
* Development
|
2177
2197
|
|