fat_table 0.2.6 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/.gitignore +3 -0
- data/.rubocop.yml +3 -0
- data/.travis.yml +7 -2
- data/.yardopts +5 -1
- data/README.org +271 -251
- data/README.rdoc +4 -4
- data/TODO.org +7 -0
- data/bin/ft_console +82 -79
- data/fat_table.gemspec +47 -46
- data/lib/fat_table.rb +11 -2
- data/lib/fat_table/column.rb +41 -29
- data/lib/fat_table/db_handle.rb +31 -50
- data/lib/fat_table/evaluator.rb +26 -24
- data/lib/fat_table/formatters/aoa_formatter.rb +5 -6
- data/lib/fat_table/formatters/aoh_formatter.rb +6 -7
- data/lib/fat_table/formatters/formatter.rb +67 -48
- data/lib/fat_table/formatters/latex_formatter.rb +9 -7
- data/lib/fat_table/formatters/org_formatter.rb +8 -9
- data/lib/fat_table/formatters/term_formatter.rb +19 -18
- data/lib/fat_table/formatters/text_formatter.rb +6 -7
- data/lib/fat_table/patches.rb +30 -1
- data/lib/fat_table/table.rb +143 -117
- data/lib/fat_table/version.rb +1 -1
- data/md/README.md +2167 -0
- metadata +78 -84
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: f7870805173324438b3a4c2416e4f59fc3830cd7dc9272b30237cfa1a9ac9d20
|
4
|
+
data.tar.gz: 31e1d6f2c8fdabe9459850e8ca1c570028c63eec1c0c2f61a030421e943d9d66
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: fd60f7879416e544c64c2e1d0efdc4cd349bf496a9c907bf7814d0ddc62c2b170ec46cc9bdb1d241fde7f50c6434233b0b2f4609adce5ec8c2073f32f52fd36e
|
7
|
+
data.tar.gz: f493a05c1b17640a41fffa5bceda6ff01047391a5f9ccb15abca1508e54a2701193912d2a6f5fc40d707b80209c93cf3c1524fd60f9cc55bdb9518b9383a3fec
|
data/.gitignore
CHANGED
data/.rubocop.yml
ADDED
data/.travis.yml
CHANGED
@@ -2,6 +2,9 @@ language: ruby
|
|
2
2
|
before_install:
|
3
3
|
- sudo apt-get -qq update
|
4
4
|
- sudo apt-get install -y texlive-latex-base texlive-latex-recommended
|
5
|
+
- gem install pg
|
6
|
+
# before_script:
|
7
|
+
# - createdb -U travis travis
|
5
8
|
after_failure:
|
6
9
|
- "pwd"
|
7
10
|
- "cat ./spec/tmp/latex.err"
|
@@ -11,7 +14,9 @@ bundler_args: --without debug
|
|
11
14
|
services:
|
12
15
|
- postgresql
|
13
16
|
rvm:
|
14
|
-
- 2.2.2
|
15
|
-
- 2.3
|
16
17
|
- 2.4
|
18
|
+
- 2.5
|
19
|
+
- 2.6
|
20
|
+
- 2.7
|
17
21
|
- ruby-head
|
22
|
+
- truffleruby
|
data/.yardopts
CHANGED
data/README.org
CHANGED
@@ -2,7 +2,11 @@
|
|
2
2
|
#+LATEX_HEADER: \usepackage[margin=0.75in]{geometry}
|
3
3
|
|
4
4
|
#+BEGIN_COMMENT
|
5
|
+
This is for markdown output:
|
6
|
+
|
5
7
|
[](https://travis-ci.org/ddoherty03/fat_table)
|
8
|
+
|
9
|
+
The following is for org.
|
6
10
|
#+END_COMMENT
|
7
11
|
|
8
12
|
[[https://travis-ci.org/ddoherty03/fat_table.svg?branch=master]]
|
@@ -12,7 +16,7 @@
|
|
12
16
|
~FatTable~ is a gem that treats tables as a data type. It provides methods for
|
13
17
|
constructing tables from a variety of sources, building them row-by-row,
|
14
18
|
extracting rows, columns, and cells, and performing aggregate operations on
|
15
|
-
columns. It also provides
|
19
|
+
columns. It also provides a set of SQL-esque methods for manipulating table
|
16
20
|
objects: ~select~ for filtering by columns or for creating new columns, ~where~
|
17
21
|
for filtering by rows, ~order_by~ for sorting rows, ~distinct~ for eliminating
|
18
22
|
duplicate rows, ~group_by~ for aggregating multiple rows into single rows and
|
@@ -31,7 +35,7 @@ no-ops.
|
|
31
35
|
~FatTable~ can be used to perform operations on data that are naturally best
|
32
36
|
conceived of as tables, which in my experience is quite often. It can also serve
|
33
37
|
as a foundation for providing reporting functions where flexibility about the
|
34
|
-
output medium can be
|
38
|
+
output medium can be useful. Finally ~FatTable~ can be used within Emacs
|
35
39
|
~org-mode~ files in code blocks targeting the Ruby language. Org mode tables are
|
36
40
|
presented to a ruby code block as an array of arrays, so ~FatTable~ can read
|
37
41
|
them in with its ~.from_aoa~ constructor. A ~FatTable~ table output as an array
|
@@ -40,6 +44,8 @@ buffer as an org-table, ready for processing by other code blocks.
|
|
40
44
|
|
41
45
|
* Installation
|
42
46
|
|
47
|
+
** Installing the gem
|
48
|
+
|
43
49
|
Add this line to your application's Gemfile:
|
44
50
|
|
45
51
|
#+BEGIN_SRC ruby
|
@@ -63,7 +69,7 @@ Or install it yourself as:
|
|
63
69
|
|
64
70
|
~FatTable~ provides table objects as a data type that can be constructed and
|
65
71
|
operated on in a number of ways. Here's a quick example to illustrate the use of
|
66
|
-
|
72
|
+
~FatTable~. See the detailed explanations further on down.
|
67
73
|
|
68
74
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
69
75
|
#+BEGIN_SRC ruby
|
@@ -152,11 +158,12 @@ the main features of ~FatTable~. See the detailed explanations further on down.
|
|
152
158
|
|
153
159
|
** A Word About the Examples
|
154
160
|
|
155
|
-
When you install the fat_table gem, you have access to a program ~ft_console
|
161
|
+
When you install the ~fat_table~ gem, you have access to a program ~ft_console~,
|
156
162
|
which opens a ~pry~ session with ~fat_table~ loaded and the tables used in the
|
157
|
-
examples in this README defined as instance variables so you can experiment
|
158
|
-
them.
|
159
|
-
as ~@tab1~ in ~ft_console~, but otherwise the examples should work
|
163
|
+
examples in this ~README~ defined as instance variables so you can experiment
|
164
|
+
with them. Because they are defined as instance variables, you have to write
|
165
|
+
~tab1~ as ~@tab1~ in ~ft_console~, but otherwise the examples should work as
|
166
|
+
shown in this ~README~.
|
160
167
|
|
161
168
|
The examples in this ~README~ file are executed as code blocks within the
|
162
169
|
~README.org~ file, so they typically end with a call to ~.to_aoa~. That causes
|
@@ -218,7 +225,7 @@ locals: _ __ _dir_ _ex_ _file_ _in_ _out_ _pry_ lib str version
|
|
218
225
|
[4] pry(main)>
|
219
226
|
#+END_EXAMPLE
|
220
227
|
|
221
|
-
|
228
|
+
If you use ~puts table.to_term~, you can see the effect of the color formatting
|
222
229
|
directives.
|
223
230
|
|
224
231
|
** Anatomy of a Table
|
@@ -228,52 +235,52 @@ directives.
|
|
228
235
|
Each ~Column~ has a header, a type, and an array of items, all of the given type
|
229
236
|
or nil. There are only five permissible types for a ~Column~:
|
230
237
|
|
231
|
-
1. Boolean (for holding ruby ~TrueClass~ and ~FalseClass~ objects),
|
232
|
-
2. DateTime (for holding ruby ~DateTime~ or ~Date~ objects),
|
233
|
-
3. Numeric (for holding ruby ~Integer~, ~Rational~, or ~BigDecimal~ objects),
|
234
|
-
4. String (for ruby String objects), or
|
235
|
-
5. NilClass (for the undetermined column type).
|
238
|
+
1. *Boolean* (for holding ruby ~TrueClass~ and ~FalseClass~ objects),
|
239
|
+
2. *DateTime* (for holding ruby ~DateTime~ or ~Date~ objects),
|
240
|
+
3. *Numeric* (for holding ruby ~Integer~, ~Rational~, or ~BigDecimal~ objects),
|
241
|
+
4. *String* (for ruby ~String~ objects), or
|
242
|
+
5. *NilClass* (for the undetermined column type).
|
236
243
|
|
237
244
|
When a ~Table~ is constructed from an external source, all ~Columns~ start out
|
238
245
|
having a type of ~NilClass~, that is, their type is as yet undetermined. When a
|
239
246
|
string or object of one of the four determined types is added to a ~Column~, it
|
240
247
|
fixes the type of the column and all further items added to the ~Column~ must
|
241
|
-
either be nil (indicating no value) or be capable of being coerced to the
|
248
|
+
either be ~nil~ (indicating no value) or be capable of being coerced to the
|
242
249
|
column's type. Otherwise, ~FatTable~ raises an exception.
|
243
250
|
|
244
251
|
Items of input must be either one of the permissible ruby objects or strings. If
|
245
252
|
they are strings, ~FatTable~ attempts to parse them as one of the permissible
|
246
253
|
types as follows:
|
247
254
|
|
248
|
-
- Boolean :: the strings, 't'
|
249
|
-
interpreted as ~TrueClass~ and the strings, 'f'
|
250
|
-
regardless of case, are interpreted as ~FalseClass~, in
|
251
|
-
resulting in a Boolean column. Empty strings in a column
|
252
|
-
Boolean type are converted to nil
|
253
|
-
- DateTime :: strings that contain patterns of 'yyyy-mm-dd' or 'yyyy/mm/dd'
|
254
|
-
'mm-dd-yyy' or 'mm/dd/yyyy' or any of the foregoing with an added
|
255
|
-
'Thh:mm:ss' or 'Thh:mm' will be interpreted as a ~DateTime~ or a ~Date~
|
256
|
-
there are no sub-day time components present). The number of digits in
|
257
|
-
month and day can be one or two, but the year component must be four
|
255
|
+
- Boolean :: the strings, ~'t'~, ~'true'~, ~'yes'~, or ~'y'~, regardless of
|
256
|
+
case, are interpreted as ~TrueClass~ and the strings, ~'f'~, ~'false'~,
|
257
|
+
~'no'~, or ~'n'~, regardless of case, are interpreted as ~FalseClass~, in
|
258
|
+
either case resulting in a Boolean column. Empty strings in a column
|
259
|
+
already having a Boolean type are converted to ~nil~.
|
260
|
+
- DateTime :: strings that contain patterns of ~'yyyy-mm-dd'~ or ~'yyyy/mm/dd'~
|
261
|
+
or ~'mm-dd-yyy'~ or ~'mm/dd/yyyy'~ or any of the foregoing with an added
|
262
|
+
~'Thh:mm:ss'~ or ~'Thh:mm'~ will be interpreted as a ~DateTime~ or a ~Date~
|
263
|
+
(if there are no sub-day time components present). The number of digits in
|
264
|
+
the month and day can be one or two, but the year component must be four
|
258
265
|
digits. Any time components are valid if they can be properly interpreted
|
259
266
|
by ~DateTime.parse~. Org mode timestamps (any of the foregoing surrounded
|
260
|
-
by square '[]' or pointy '
|
261
|
-
input strings for DateTime columns. Empty strings in a column already
|
262
|
-
having the DateTime type are converted to nil
|
263
|
-
- Numeric :: all commas ','
|
264
|
-
currency symbol as set by ~FatTable.currency_symbol~ are removed from
|
265
|
-
string and if the remaining string can be interpreted as a ~Numeric~,
|
266
|
-
will be. It is interpreted as an ~Integer~ if there are no decimal
|
267
|
-
in the remaining string, as a ~Rational~ if the string has the form
|
268
|
-
'
|
269
|
-
is a decimal point in the remaining string. Empty strings in a column
|
267
|
+
by square '~[]~' or pointy '~<>~' brackets), active or inactive, are valid
|
268
|
+
input strings for ~DateTime~ columns. Empty strings in a column already
|
269
|
+
having the ~DateTime~ type are converted to ~nil~.
|
270
|
+
- Numeric :: all commas ~','~, underscores, ~'_'~, and ~'$'~ dollar signs (or
|
271
|
+
other currency symbol as set by ~FatTable.currency_symbol~ are removed from
|
272
|
+
the string and if the remaining string can be interpreted as a ~Numeric~,
|
273
|
+
it will be. It is interpreted as an ~Integer~ if there are no decimal
|
274
|
+
places in the remaining string, as a ~Rational~ if the string has the form
|
275
|
+
'~<number>:<number>~' or '~<number>/<number>~', or as a ~BigDecimal~ if
|
276
|
+
there is a decimal point in the remaining string. Empty strings in a column
|
270
277
|
already having the Numeric type are converted to nil.
|
271
278
|
- String :: if all else fails, ~FatTable~ applies ~#to_s~ to the input value
|
272
|
-
and, treats it as an item of type ~String~.
|
273
|
-
already having the String type are kept as empty strings.
|
279
|
+
and, treats it as an item of type ~String~. Empty strings in a column
|
280
|
+
already having the ~String~ type are kept as empty strings.
|
274
281
|
- NilClass :: until the input contains a non-blank string that can be parsed as
|
275
282
|
one of the other types, it has this type, meaning that the type is still
|
276
|
-
open. A column comprised completely of blank strings or nils will retain
|
283
|
+
open. A column comprised completely of blank strings or ~nils~ will retain
|
277
284
|
the ~NilClass~ type.
|
278
285
|
|
279
286
|
*** Headers
|
@@ -282,23 +289,23 @@ Headers for the columns are formed from the input. No two columns in a table can
|
|
282
289
|
have the same header. Headers in the input are converted to symbols by
|
283
290
|
|
284
291
|
- converting the header to a string with ~#to_s~,
|
285
|
-
- converting any run of blanks to an underscore
|
292
|
+
- converting any run of blanks to an underscore ~_~,
|
286
293
|
- removing any characters that are not letters, numbers, or underscores, and
|
287
294
|
- lowercasing all remaining letters
|
288
295
|
|
289
|
-
Thus, a header of 'Date' becomes ~:date~, a header of 'Id Number' becomes,
|
296
|
+
Thus, a header of ~'Date'~ becomes ~:date~, a header of ~'Id Number'~ becomes,
|
290
297
|
~:id_number~, etc. When referring to a column in code, you must use the symbol
|
291
298
|
form of the header.
|
292
299
|
|
293
300
|
If no sensible headers can be discerned from the input, headers of the form
|
294
|
-
|
301
|
+
~:col_1~, ~:col_2~, etc., are synthesized.
|
295
302
|
|
296
303
|
*** Groups
|
297
304
|
|
298
305
|
The rows of a ~FatTable~ table can be sub-divided into groups, either from
|
299
306
|
markers in the input or as a result of certain operations. There is only one
|
300
307
|
level of grouping, so ~FatTable~ has no concept of sub-groups. Groups can be
|
301
|
-
shown on output with rules or
|
308
|
+
shown on output with rules or "hlines" that underline the last row in each
|
302
309
|
group, and you can decorate the output with group footers that summarize the
|
303
310
|
columns in each group.
|
304
311
|
|
@@ -310,7 +317,7 @@ You can create an empty table with ~FatTable.new~, and then add rows with the
|
|
310
317
|
|
311
318
|
#+BEGIN_SRC ruby
|
312
319
|
tab = FatTable.new
|
313
|
-
tab << { a: 1, b: 2, c:
|
320
|
+
tab << { a: 1, b: 2, c: "<2017-01-21>', d: 'f', e: '' }
|
314
321
|
tab << { a: 3.14, b: 2.17, c: '[2016-01-21 Thu]', d: 'Y', e: nil }
|
315
322
|
tab.to_aoa
|
316
323
|
#+END_SRC
|
@@ -325,7 +332,7 @@ in org-mode time stamps.
|
|
325
332
|
|
326
333
|
Tables can also be read from ~.csv~ files or files containing ~org-mode~ tables.
|
327
334
|
In the case of org-mode files, ~FatTable~ skips through the file until it finds
|
328
|
-
a line that look like a table, that is it begins with any number of spaces
|
335
|
+
a line that look like a table, that is, it begins with any number of spaces
|
329
336
|
followed by ~|-~. Only the first table in an ~.org~ file is read.
|
330
337
|
|
331
338
|
For both ~.csv~ and ~.org~ files, the first row in the tables is taken as the
|
@@ -383,21 +390,22 @@ You can also initialize a table directly from ruby data structures. You can, for
|
|
383
390
|
example, build a table from an array of arrays:
|
384
391
|
|
385
392
|
#+BEGIN_SRC ruby
|
386
|
-
aoa =
|
387
|
-
[
|
388
|
-
|
389
|
-
|
390
|
-
|
391
|
-
|
392
|
-
|
393
|
-
|
394
|
-
|
395
|
-
|
396
|
-
|
397
|
-
|
398
|
-
|
399
|
-
|
400
|
-
|
393
|
+
aoa = [
|
394
|
+
['Ref', 'Date', 'Code', 'Raw', 'Shares', 'Price', 'Info', 'Bool'],
|
395
|
+
[1, '2013-05-02', 'P', 795_546.20, 795_546.2, 1.1850, 'ENTITY1', 'T'],
|
396
|
+
[2, '2013-05-02', 'P', 118_186.40, 118_186.4, 11.8500, 'ENTITY1', 'T'],
|
397
|
+
[7, '2013-05-20', 'S', 12_000.00, 5046.00, 28.2804, 'ENTITY3', 'F'],
|
398
|
+
[8, '2013-05-20', 'S', 85_000.00, 35_742.50, 28.3224, 'ENTITY3', 'T'],
|
399
|
+
[9, '2013-05-20', 'S', 33_302.00, 14_003.49, 28.6383, 'ENTITY3', 'T'],
|
400
|
+
[10, '2013-05-23', 'S', 8000.00, 3364.00, 27.1083, 'ENTITY3', 'T'],
|
401
|
+
[11, '2013-05-23', 'S', 23_054.00, 9694.21, 26.8015, 'ENTITY3', 'F'],
|
402
|
+
[12, '2013-05-23', 'S', 39_906.00, 16_780.47, 25.1749, 'ENTITY3', 'T'],
|
403
|
+
[13, '2013-05-29', 'S', 13_459.00, 5659.51, 24.7464, 'ENTITY3', 'T'],
|
404
|
+
[14, '2013-05-29', 'S', 15_700.00, 6601.85, 24.7790, 'ENTITY3', 'F'],
|
405
|
+
[15, '2013-05-29', 'S', 15_900.00, 6685.95, 24.5802, 'ENTITY3', 'T'],
|
406
|
+
[16, '2013-05-30', 'S', 6_679.00, 2808.52, 25.0471, 'ENTITY3', 'T']
|
407
|
+
]
|
408
|
+
tab = FatTable.from_aoa(aoa)
|
401
409
|
#+END_SRC
|
402
410
|
|
403
411
|
Notice that the values can either be ruby objects, such as the Integer ~85_000~,
|
@@ -453,7 +461,7 @@ arrays is displayed as an org-mode table:
|
|
453
461
|
|
454
462
|
This example illustrates several things:
|
455
463
|
|
456
|
-
1. The named org-mode table,
|
464
|
+
1. The named org-mode table, ~trades1~, can be passed into a ruby code block
|
457
465
|
using the ~:var tab=trades1~ header argument to the code block; that makes
|
458
466
|
the variable ~tab~ available to the code block as an array of arrays, which
|
459
467
|
~FatTable~ then uses to initialize the table.
|
@@ -464,19 +472,21 @@ This example illustrates several things:
|
|
464
472
|
where the number of shares is greater than 500. More on that later.
|
465
473
|
4. ~FatTable~ passes back to org-mode an array of arrays using the ~.to_aoa~
|
466
474
|
method. In an ~org-mode~ buffer, these are rendered as tables. We'll often
|
467
|
-
apply ~.to_aoa~ at the end of example blocks to render the
|
468
|
-
this
|
469
|
-
to which formatting
|
475
|
+
apply ~.to_aoa~ at the end of example blocks in this ~README~ to render the
|
476
|
+
results as a table inside this file. As we'll see below, ~.to_aoa~ can also
|
477
|
+
take a block to which formatting and footer directives can be attached.
|
470
478
|
|
471
479
|
*** From Arrays of Hashes
|
472
480
|
|
473
481
|
A second ruby data structure that can be used to initialize a ~FatTable~ table
|
474
482
|
is an array of ruby Hashes. Each hash represents a row of the table, and the
|
475
|
-
headers of the table are
|
476
|
-
hashes should have the same keys.
|
477
|
-
|
478
|
-
|
479
|
-
a
|
483
|
+
headers of the table are taken from the keys of the hashes. Accordingly, all the
|
484
|
+
hashes should have the same keys.
|
485
|
+
|
486
|
+
This same method can in fact take an array of any objects that can be converted
|
487
|
+
to a Hash with the ~#to_h~ method, so you can use an array of your own objects
|
488
|
+
to initialize a table, provided that you define a suitable ~#to_h~ method for
|
489
|
+
the objects' class.
|
480
490
|
|
481
491
|
#+BEGIN_SRC ruby
|
482
492
|
aoh = [
|
@@ -501,40 +511,38 @@ tab = FatTable.from_aoh(aoh)
|
|
501
511
|
#+END_SRC
|
502
512
|
|
503
513
|
Notice, again, that the values can either be ruby objects, such as ~Date.today~,
|
504
|
-
or strings that can parsed into one of the permissible column types.
|
514
|
+
or strings that can be parsed into one of the permissible column types.
|
505
515
|
|
506
516
|
*** From SQL queries
|
507
517
|
|
508
|
-
Another way to initialize a ~FatTable~ table is with the results of a SQL
|
509
|
-
|
510
|
-
|
518
|
+
Another way to initialize a ~FatTable~ table is with the results of a SQL
|
519
|
+
query. Before you can connect to a database, you need to make sure that the required
|
520
|
+
adapter for your database is installed. ~FatTable~ uses the ~sequel~ gem
|
521
|
+
under the hood, so any database that it supports can be used. For example, if
|
522
|
+
you are accessing a Postgres database, you must install the ~pg~ gem with
|
523
|
+
|
524
|
+
#+begin_src sh
|
525
|
+
$ gem install pg
|
526
|
+
#+end_src
|
527
|
+
|
528
|
+
You must first set the database parameters to be used for the queries.
|
511
529
|
|
512
530
|
#+BEGIN_SRC ruby
|
513
531
|
# This automatically requires sequel.
|
514
532
|
require 'fat_table'
|
515
|
-
FatTable.
|
533
|
+
FatTable.connect(adapter: 'postgres',
|
516
534
|
database: 'XXX_development',
|
517
|
-
user: '
|
518
|
-
password: '
|
519
|
-
host: '
|
520
|
-
socket: '/tmp/.s.PGSQL.5432')
|
535
|
+
user: 'ken',
|
536
|
+
password: 'imsecret',
|
537
|
+
host: 'db.lan')
|
521
538
|
tab = FatTable.from_sql('select * from trades;')
|
522
539
|
#+END_SRC
|
523
540
|
|
524
|
-
|
525
|
-
|
526
|
-
|
527
|
-
|
528
|
-
|
529
|
-
required.
|
530
|
-
|
531
|
-
The ~.set_db~ function need only be called once, and the database handle it
|
532
|
-
creates will be used for all subsequent ~.from_sql~ calls until ~.set_db~ is
|
533
|
-
called again.
|
534
|
-
|
535
|
-
Alternatively, you can build the ~Sequel~ connection with ~Sequel.connect~ or
|
536
|
-
with adapter-specific ~Sequel~ connection methods and let ~FatTable~ know to use
|
537
|
-
that connection:
|
541
|
+
The arguments to ~connect~ are simply passed on to ~sequel~'s connect method, so
|
542
|
+
any set of arguments that work for it should work for ~connect~. Alternatively,
|
543
|
+
you can build the ~Sequel~ connection directly with ~Sequel.connect~ or with
|
544
|
+
adapter-specific ~Sequel~ connection methods and let ~FatTable~ know to use that
|
545
|
+
connection:
|
538
546
|
|
539
547
|
#+BEGIN_SRC ruby
|
540
548
|
require 'fat_table'
|
@@ -545,8 +553,15 @@ that connection:
|
|
545
553
|
Consult ~Sequel's~ documentation for details on its connection methods.
|
546
554
|
[[http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html]]
|
547
555
|
|
556
|
+
The ~.connect~ function need only be called once, and the database handle it
|
557
|
+
creates will be used for all subsequent ~.from_sql~ calls until ~.connect~ is
|
558
|
+
called again.
|
559
|
+
|
548
560
|
*** Marking Groups in Input
|
549
561
|
|
562
|
+
~FatTable~ tables has a concept of "groups" of rows that play a role in many of
|
563
|
+
the methods for operating on them as explained [[Groups][below]].
|
564
|
+
|
550
565
|
The ~.from_aoa~ and ~.from_aoh~ functions take an optional keyword parameter
|
551
566
|
~hlines:~ that, if set to ~true~, causes them to mark group boundaries in the
|
552
567
|
table wherever a row Array (for ~.from_aoa~) or Hash (for ~.from_aoh~) is
|
@@ -566,11 +581,12 @@ beginning with something like '~|---~'. The ~.from_org_...~ functions always
|
|
566
581
|
recognizes hlines in the input, so it takes no ~hlines:~ keyword parameter.
|
567
582
|
|
568
583
|
** Accessing Parts of Tables
|
584
|
+
|
569
585
|
*** Rows
|
570
586
|
|
571
587
|
A ~FatTable~ table is an Enumerable, yielding each row of the table as a Hash
|
572
|
-
keyed on the header symbols.
|
573
|
-
Hashes as well.
|
588
|
+
keyed on the header symbols. The method ~Table#rows~ returns an Array of the
|
589
|
+
rows as Hashes as well.
|
574
590
|
|
575
591
|
You can also use indexing to access a row of the table by number. Using an
|
576
592
|
integer index returns a Hash of the given row. Thus, ~tab[20]~ returns the 21st
|
@@ -621,6 +637,7 @@ operation, which mutates the receiver table by removing its group boundaries.
|
|
621
637
|
Second, because each operation returns a ~FatTable::Table~ object, the
|
622
638
|
operations are chainable.
|
623
639
|
|
640
|
+
<<Groups>>
|
624
641
|
Third, ~FatTable::Table~ objects can have "groups" of rows within the table.
|
625
642
|
These can be decorated with hlines and group footers on output. Some of these
|
626
643
|
operations result in marking group boundaries in the result table, others remove
|
@@ -635,8 +652,8 @@ SQL.
|
|
635
652
|
|
636
653
|
For illustration purposes assume that the following tables are read into ruby
|
637
654
|
variables called '~tab1~' and '~tab2~. We have given the table groups, marked by
|
638
|
-
the hlines below, and some duplicate rows to illustrate the effect of
|
639
|
-
operations on groups and duplicates.
|
655
|
+
the hlines below, and included some duplicate rows to illustrate the effect of
|
656
|
+
certain operations on groups and duplicates.
|
640
657
|
|
641
658
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
642
659
|
#+BEGIN_SRC ruby
|
@@ -743,12 +760,13 @@ any group boundaries present in the input table.
|
|
743
760
|
|
744
761
|
**** Adding New Columns
|
745
762
|
|
746
|
-
More interesting is that ~select~ can take hash-like keyword arguments
|
747
|
-
|
748
|
-
|
749
|
-
|
750
|
-
symbol representing an existing column
|
751
|
-
expression for the value of
|
763
|
+
More interesting is that ~select~ can take hash-like keyword arguments after the
|
764
|
+
symbol arguments to create new columns in the output as functions of other
|
765
|
+
columns. For each hash-like parameter, the keyword given must be a symbol, which
|
766
|
+
becomes the header for the new column, and the value must be either: (1) a
|
767
|
+
symbol representing an existing column, which has the effect of renaming an
|
768
|
+
existing column, or (2) a string representing a ruby expression for the value of
|
769
|
+
a new column.
|
752
770
|
|
753
771
|
Within the string expression, the names of existing or already-specified columns
|
754
772
|
are available as local variables, as well as the instance variables '@row' and
|
@@ -759,8 +777,8 @@ access to local variables ~ref~, ~date~, ~code~, ~price~, ~g10~, ~qp10~,
|
|
759
777
|
their respective columns for each row in the input table and the instance
|
760
778
|
variables are set the number of the current row and group respectively.
|
761
779
|
|
762
|
-
For example, if we want to rename the
|
763
|
-
shares, we could do the following:
|
780
|
+
For example, if we want to rename the ~:date~ column and add a new column to
|
781
|
+
compute the cost of shares, we could do the following:
|
764
782
|
|
765
783
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
766
784
|
#+BEGIN_SRC ruby
|
@@ -840,18 +858,18 @@ second, chained call to ~select~:
|
|
840
858
|
**** Custom Instance Variables and Hooks
|
841
859
|
|
842
860
|
As the above examples demonstrate, the instance variables ~@row~ and ~@group~
|
843
|
-
are available when evaluating expressions that add new columns.
|
844
|
-
your own instance variables as well for keeping track of things that cross
|
845
|
-
boundaries, such as running sums.
|
861
|
+
are available when evaluating expressions that add new columns. You can also set
|
862
|
+
up your own instance variables as well for keeping track of things that cross
|
863
|
+
row boundaries, such as running sums.
|
846
864
|
|
847
865
|
To declare instance variables, you can use the ~ivars:~ hash parameter to
|
848
866
|
~select~. Each key of the hash becomes an instance variable and each value
|
849
867
|
becomes its initial value before any rows are evaluated.
|
850
868
|
|
851
|
-
In addition, you can provide ~before_hook:~ and ~after_hook:~ parameters
|
852
|
-
strings that are evaluated as ruby expressions before and after each
|
853
|
-
processed.
|
854
|
-
the ~before_hook:~ can be used in expressions for adding new columns by
|
869
|
+
In addition, you can provide ~before_hook:~ and ~after_hook:~ parameters to
|
870
|
+
~select~ as strings that are evaluated as ruby expressions before and after each
|
871
|
+
row is processed. You can use these to update instance variables. The values set
|
872
|
+
in the ~before_hook:~ can be used in expressions for adding new columns by
|
855
873
|
referencing them with the '@' prefix.
|
856
874
|
|
857
875
|
For example, suppose we wanted to not only add a cost column, but a column that
|
@@ -942,28 +960,21 @@ You can sort a table on any number of columns with ~order_by~. The ~order_by~
|
|
942
960
|
method takes any number of symbol arguments for the columns to sort on. If you
|
943
961
|
specify more than one column, the sort is performed on the first column, then
|
944
962
|
all columns that are equal with respect to the first column are sorted by the
|
945
|
-
second column, and so on.
|
946
|
-
|
963
|
+
second column, and so on. Ordering is done is ascending order for each of the
|
964
|
+
columns, but can be reversed by adding a '!' to the end a symbol argument.
|
965
|
+
All columns of the input table are included in the output.
|
947
966
|
|
948
|
-
Let's sort our table first by ~:code~, then
|
967
|
+
Let's sort our table first by ~:code~, then in reverse order of ~:date~.
|
949
968
|
|
950
969
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
951
970
|
#+BEGIN_SRC ruby
|
952
|
-
tab1.order_by(:code, :date) \
|
971
|
+
tab1.order_by(:code, :date!) \
|
953
972
|
.to_aoa
|
954
973
|
#+END_SRC
|
955
974
|
|
956
|
-
#+
|
975
|
+
#+begin_EXAMPLE
|
957
976
|
| Ref | Date | Code | Price | G10 | QP10 | Shares | Lp | Qp | Iplp | Ipqp |
|
958
977
|
|------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
|
959
|
-
| T001 | 2016-11-01 | P | 7.7 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
|
960
|
-
| T002 | 2016-11-01 | P | 7.75 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
|
961
|
-
| T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
|
962
|
-
| T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
|
963
|
-
| T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
|
964
|
-
| T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
|
965
|
-
| T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
|
966
|
-
|------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
|
967
978
|
| T011 | 2016-11-02 | P | 7.425 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
|
968
979
|
| T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
|
969
980
|
| T012 | 2016-11-02 | P | 7.55 | F | F | 4700 | 677 | 4023 | 0.2453 | 0.1924 |
|
@@ -972,12 +983,20 @@ Let's sort our table first by ~:code~, then by ~:date~.
|
|
972
983
|
| T015 | 2016-11-02 | P | 7.75 | F | F | 500 | 72 | 428 | 0.2453 | 0.1924 |
|
973
984
|
| T016 | 2016-11-02 | P | 8.25 | T | T | 100 | 14 | 86 | 0.2453 | 0.1924 |
|
974
985
|
|------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
|
986
|
+
| T001 | 2016-11-01 | P | 7.7 | T | F | 100 | 14 | 86 | 0.2453 | 0.1924 |
|
987
|
+
| T002 | 2016-11-01 | P | 7.75 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
|
988
|
+
| T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
|
989
|
+
| T003 | 2016-11-01 | P | 7.5 | F | T | 800 | 112 | 688 | 0.2453 | 0.1924 |
|
990
|
+
| T008 | 2016-11-01 | P | 7.65 | F | F | 2771 | 393 | 2378 | 0.2453 | 0.1924 |
|
991
|
+
| T009 | 2016-11-01 | P | 7.6 | F | F | 9550 | 1363 | 8187 | 0.2453 | 0.1924 |
|
992
|
+
| T010 | 2016-11-01 | P | 7.55 | F | T | 3175 | 451 | 2724 | 0.2453 | 0.1924 |
|
993
|
+
|------+------------+------+-------+-----+------+--------+------+-------+--------+--------|
|
975
994
|
| T004 | 2016-11-01 | S | 7.55 | T | F | 6811 | 966 | 5845 | 0.2453 | 0.1924 |
|
976
995
|
| T005 | 2016-11-01 | S | 7.5 | F | F | 4000 | 572 | 3428 | 0.2453 | 0.1924 |
|
977
996
|
| T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
|
978
997
|
| T006 | 2016-11-01 | S | 7.6 | F | T | 1000 | 143 | 857 | 0.2453 | 0.1924 |
|
979
998
|
| T007 | 2016-11-01 | S | 7.65 | T | F | 200 | 28 | 172 | 0.2453 | 0.1924 |
|
980
|
-
#+
|
999
|
+
#+end_EXAMPLE
|
981
1000
|
|
982
1001
|
The interesting thing about ~order_by~ is that, while it ignores groups in its
|
983
1002
|
input, it adds group boundaries in the output table at those rows where the sort
|
@@ -996,7 +1015,8 @@ which the grouping parameters are equal containing those columns and an
|
|
996
1015
|
aggregate column for each of the aggregating parameters.
|
997
1016
|
|
998
1017
|
For example, let's summarize the ~trades~ table by ~:code~ and ~:price~ again,
|
999
|
-
and determine total shares, average price, and other features of each
|
1018
|
+
and determine total shares, average price, and a few other features of each
|
1019
|
+
group:
|
1000
1020
|
|
1001
1021
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
1002
1022
|
#+BEGIN_SRC ruby
|
@@ -1030,38 +1050,39 @@ will raise an exception.
|
|
1030
1050
|
|
1031
1051
|
- ~first~ :: the first non-nil item in the column,
|
1032
1052
|
- ~last~ :: the last non-nil item in the column,
|
1033
|
-
- ~rng~ :: form a string of the form "#{first}..#{last}" to show the range of
|
1053
|
+
- ~rng~ :: form a string of the form ~"#{first}..#{last}"~ to show the range of
|
1034
1054
|
values in the column,
|
1035
|
-
- ~sum~ :: for Numeric and String columns, apply '+' to all the non-nil
|
1055
|
+
- ~sum~ :: for ~Numeric~ and ~String~ columns, apply '+' to all the non-nil
|
1056
|
+
values,
|
1036
1057
|
- ~count~ :: the number of non-nil values in the column,
|
1037
|
-
- ~min~ :: for Numeric
|
1038
|
-
value in the column,
|
1039
|
-
- ~max~ :: for Numeric
|
1040
|
-
value in the column,
|
1041
|
-
- ~avg~ :: for Numeric and DateTime columns, return the arithmetic mean of
|
1042
|
-
non-nil values in the column; with respect to
|
1043
|
-
converted to a numeric Julian date, the average is
|
1044
|
-
result converted back to a Date or DateTime object,
|
1045
|
-
- ~var~ :: for Numeric and DateTime columns, compute the sample variance of
|
1046
|
-
non-nil values in the column, dates are converted to
|
1047
|
-
|
1048
|
-
- ~pvar~ :: for Numeric and DateTime columns, compute the population
|
1049
|
-
the non-nil values in the column, dates are converted to
|
1050
|
-
|
1051
|
-
- ~dev~ :: for Numeric and DateTime columns, compute the sample standard
|
1052
|
-
deviation of the non-nil values in the column, dates are converted to
|
1053
|
-
numbers as for the :avg aggregate,
|
1054
|
-
- ~pdev~ :: for Numeric and DateTime columns, compute the population standard
|
1058
|
+
- ~min~ :: for ~Numeric~, ~String~, and ~DateTime~ columns, return the smallest
|
1059
|
+
non-nil value in the column,
|
1060
|
+
- ~max~ :: for ~Numeric~, ~String~, and ~DateTime~ columns, return the largest
|
1061
|
+
non-nil value in the column,
|
1062
|
+
- ~avg~ :: for ~Numeric~ and ~DateTime~ columns, return the arithmetic mean of
|
1063
|
+
the non-nil values in the column; with respect to ~Date~ or ~DateTime~
|
1064
|
+
objects, each is converted to a numeric Julian date, the average is
|
1065
|
+
calculated, and the result converted back to a ~Date~ or ~DateTime~ object,
|
1066
|
+
- ~var~ :: for ~Numeric~ and ~DateTime~ columns, compute the sample variance of
|
1067
|
+
the non-nil values in the column, dates are converted to Julian date
|
1068
|
+
numbers as for the ~:avg~ aggregate,
|
1069
|
+
- ~pvar~ :: for ~Numeric~ and ~DateTime~ columns, compute the population
|
1070
|
+
variance of the non-nil values in the column, dates are converted to Julian
|
1071
|
+
date numbers as for the ~:avg~ aggregate,
|
1072
|
+
- ~dev~ :: for ~Numeric~ and ~DateTime~ columns, compute the sample standard
|
1055
1073
|
deviation of the non-nil values in the column, dates are converted to
|
1056
|
-
numbers as for the
|
1057
|
-
- ~
|
1074
|
+
Julian date numbers as for the ~:avg~ aggregate,
|
1075
|
+
- ~pdev~ :: for ~Numeric~ and ~DateTime~ columns, compute the population
|
1076
|
+
standard deviation of the non-nil values in the column, dates are converted
|
1077
|
+
to numbers as for the ~:avg~ aggregate,
|
1078
|
+
- ~all?~ :: for ~Boolean~ columns only, return true if all of the non-nil values
|
1058
1079
|
in the column are true,
|
1059
|
-
- ~any?~ :: for Boolean columns only, return true if any non-nil value in the
|
1080
|
+
- ~any?~ :: for ~Boolean~ columns only, return true if any non-nil value in the
|
1060
1081
|
column is true,
|
1061
|
-
- ~none?~ :: for Boolean columns only, return true if no non-nil value in the
|
1082
|
+
- ~none?~ :: for ~Boolean~ columns only, return true if no non-nil value in the
|
1062
1083
|
column is true,
|
1063
|
-
- ~one?~ :: for Boolean columns only, return true if exactly one non-nil value
|
1064
|
-
the column is true,
|
1084
|
+
- ~one?~ :: for ~Boolean~ columns only, return true if exactly one non-nil value
|
1085
|
+
in the column is true,
|
1065
1086
|
|
1066
1087
|
Perhaps surprisingly, the ~group_by~ method ignores any groups in its input and
|
1067
1088
|
results in no group boundaries in the output since each group formed by the
|
@@ -1073,36 +1094,37 @@ implicit ~order_by~ on the grouping columns is collapsed into a single row.
|
|
1073
1094
|
So far, all the operations have operated on a single table. ~FatTable~ provides
|
1074
1095
|
several ~join~ methods for combining two tables, each of which takes as
|
1075
1096
|
parameters (1) a second table and (2) except in the case of ~cross_join~, zero
|
1076
|
-
or more "join expressions".
|
1077
|
-
the method is called, ~T2~ is the table supplied as the first parameter
|
1078
|
-
and ~R1~ and ~R2~ are rows in their respective tables being considered
|
1079
|
-
inclusion in the joined output table.
|
1097
|
+
or more "join expressions". In the descriptions below, ~T1~ is the table on
|
1098
|
+
which the method is called, ~T2~ is the table supplied as the first parameter
|
1099
|
+
~other~, and ~R1~ and ~R2~ are rows in their respective tables being considered
|
1100
|
+
for inclusion in the joined output table.
|
1080
1101
|
|
1081
1102
|
- ~join(other, *jexps)~ :: Performs an "inner join" on the tables. For each row
|
1082
|
-
R1 of T1
|
1083
|
-
join condition with R1
|
1103
|
+
~R1~ of ~T1~, the joined table has a row for each row in ~T2~ that
|
1104
|
+
satisfies the join condition with ~R1~.
|
1084
1105
|
|
1085
1106
|
- ~left_join(other, *jexps)~ :: First, an inner join is performed. Then, for
|
1086
|
-
each row in T1 that does not satisfy the join condition with any row in
|
1087
|
-
a joined row is added with null values in columns of T2
|
1088
|
-
table always has at least one row for each row in T1
|
1107
|
+
each row in ~T1~ that does not satisfy the join condition with any row in
|
1108
|
+
~T2~, a joined row is added with null values in columns of ~T2~. Thus, the
|
1109
|
+
joined table always has at least one row for each row in ~T1~.
|
1089
1110
|
|
1090
1111
|
- ~right_join(other, *jexps)~ :: First, an inner join is performed. Then, for
|
1091
|
-
each row in T2 that does not satisfy the join condition with any row in
|
1092
|
-
a joined row is added with null values in columns of T1
|
1093
|
-
converse of a left join: the result table will always have a row for
|
1094
|
-
row in T2
|
1112
|
+
each row in ~T2~ that does not satisfy the join condition with any row in
|
1113
|
+
~T1~, a joined row is added with null values in columns of ~T1~. This is
|
1114
|
+
the converse of a left join: the result table will always have a row for
|
1115
|
+
each row in ~T2~.
|
1095
1116
|
|
1096
1117
|
- ~full_join(other, *jexps)~ :: First, an inner join is performed. Then, for
|
1097
|
-
each row in T1 that does not satisfy the join condition with any row in
|
1098
|
-
a joined row is added with null values in columns of T2
|
1099
|
-
of T2 that does not satisfy the join condition with any row in
|
1100
|
-
row with null values in the columns of T1 is added.
|
1118
|
+
each row in ~T1~ that does not satisfy the join condition with any row in
|
1119
|
+
~T2~, a joined row is added with null values in columns of ~T2~. Also, for
|
1120
|
+
each row of ~T2~ that does not satisfy the join condition with any row in
|
1121
|
+
~T1~, a joined row with null values in the columns of ~T1~ is added.
|
1101
1122
|
|
1102
|
-
- ~cross_join(other)~ :: For every possible combination of rows from T1 and
|
1103
|
-
(i.e., a Cartesian product), the joined table will contain a row
|
1104
|
-
of all columns in T1 followed by all columns in T2
|
1105
|
-
and M rows respectively, the joined table will have N *
|
1123
|
+
- ~cross_join(other)~ :: For every possible combination of rows from ~T1~ and
|
1124
|
+
~T2~ (i.e., a Cartesian product), the joined table will contain a row
|
1125
|
+
consisting of all columns in ~T1~ followed by all columns in ~T2~. If the
|
1126
|
+
tables have ~N~ and ~M~ rows respectively, the joined table will have ~N *
|
1127
|
+
M~ rows.
|
1106
1128
|
|
1107
1129
|
**** Join Expressions
|
1108
1130
|
|
@@ -1116,8 +1138,8 @@ that the values of both tables are equal for all columns named by the symbols. A
|
|
1116
1138
|
column that appears in both tables can be given without modification and will be
|
1117
1139
|
assumed to require equality on that column. If an unmodified symbol is not a
|
1118
1140
|
name that appears in both tables, an exception will be raised. Column names that
|
1119
|
-
are unique to the first table must have a
|
1120
|
-
column names that are unique to the other table must have a
|
1141
|
+
are unique to the first table must have a ~_a~ appended to the column name and
|
1142
|
+
column names that are unique to the other table must have a ~_b~ appended to the
|
1121
1143
|
column name. These disambiguated column names must come in pairs, one for the
|
1122
1144
|
first table and one for the second, and they will imply a join condition that
|
1123
1145
|
the columns must be equal on those columns. Several such symbol expressions will
|
@@ -1126,7 +1148,7 @@ be met.
|
|
1126
1148
|
|
1127
1149
|
Finally, a join expression can be a string that contains an arbitrary ruby
|
1128
1150
|
expression that will be evaluated for truthiness. Within the string, /all/
|
1129
|
-
column names must be disambiguated with the
|
1151
|
+
column names must be disambiguated with the ~_a~ or ~_b~ modifiers whether they
|
1130
1152
|
are common to both tables or not. As with ~select~ and ~where~ methods, the
|
1131
1153
|
names of the columns in both tables (albeit disambiguated) are available as
|
1132
1154
|
local variables within the expression, but the instance variables ~@row~ and
|
@@ -1134,9 +1156,9 @@ local variables within the expression, but the instance variables ~@row~ and
|
|
1134
1156
|
|
1135
1157
|
**** Join Examples
|
1136
1158
|
|
1137
|
-
The following examples are taken from
|
1138
|
-
|
1139
|
-
|
1159
|
+
The following examples are taken from the [[https://www.tutorialspoint.com/postgresql/postgresql_using_joins.htm][Postgresql tutorial]], with some slight
|
1160
|
+
modifications. The examples will use the following two tables, which are also
|
1161
|
+
available in ~ft_console~ as ~@tab_a~ and ~@tab_b~:
|
1140
1162
|
|
1141
1163
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
1142
1164
|
#+BEGIN_SRC ruby
|
@@ -1326,10 +1348,10 @@ Finally, a cross join outputs every row of ~tab_a~ augmented with every row of
|
|
1326
1348
|
|
1327
1349
|
*** Set Operations
|
1328
1350
|
|
1329
|
-
~FatTable~ can perform several set operations on tables.
|
1330
|
-
|
1331
|
-
|
1332
|
-
|
1351
|
+
~FatTable~ can perform several set operations on tables. In order for two tables
|
1352
|
+
to be used this way, they must have the same number of columns with the same
|
1353
|
+
types or an exception will be raised. We'll call two tables that qualify for
|
1354
|
+
combining with set operations "set-compatible."
|
1333
1355
|
|
1334
1356
|
We'll use the following two set-compatible tables in the examples. They each
|
1335
1357
|
have some duplicates and some group boundaries so you can see the effect of the
|
@@ -1400,7 +1422,7 @@ set operations on duplicates and groups.
|
|
1400
1422
|
Two tables that are set-compatible can be combined with the ~union~ or
|
1401
1423
|
~union_all~ methods so that the rows of both tables appear in the output. In the
|
1402
1424
|
output table, the headers of the receiver table are used. You can use ~select~
|
1403
|
-
to change or re-order the headers if you prefer.
|
1425
|
+
to change or re-order the headers if you prefer. The ~union~ method eliminates
|
1404
1426
|
duplicate rows in the result table, the ~union_all~ method does not.
|
1405
1427
|
|
1406
1428
|
Any group boundaries in the input tables are destroyed by ~union~ but are
|
@@ -1511,7 +1533,7 @@ eliminating any duplicate rows in the result.
|
|
1511
1533
|
#+END_EXAMPLE
|
1512
1534
|
|
1513
1535
|
With ~intersect_all~, all the rows of the first table, including duplicates, are
|
1514
|
-
included in the result if they also occur in the second table.
|
1536
|
+
included in the result if they also occur in the second table. However,
|
1515
1537
|
duplicates in the second table do not appear.
|
1516
1538
|
|
1517
1539
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
@@ -1710,21 +1732,20 @@ receiver table by removing its groups.
|
|
1710
1732
|
** Formatting Tables
|
1711
1733
|
|
1712
1734
|
Besides creating and operating on tables, you may want to display the resulting
|
1713
|
-
table.
|
1714
|
-
most common across many output media.
|
1735
|
+
table. ~FatTable~ seeks to provide a set of formatting directives that are the
|
1736
|
+
most common across many output media. It provides directives for alignment, for
|
1715
1737
|
color, for adding currency symbols and grouping commas to numbers, for padding
|
1716
1738
|
numbers, and for formatting dates and booleans.
|
1717
1739
|
|
1718
1740
|
In addition, you can add any number of footers to a table, which appear at the
|
1719
1741
|
end of the table, and any number of group footers, which appear after each group
|
1720
|
-
in the table.
|
1742
|
+
in the table. These can be formatted independently of the table body.
|
1721
1743
|
|
1722
1744
|
If the target output medium does not support a formatting directive or the
|
1723
|
-
directive
|
1724
|
-
|
1725
|
-
|
1726
|
-
|
1727
|
-
alignment are simply irrelevant.
|
1745
|
+
directive does not make sense, it is simply ignored. For example, you can output
|
1746
|
+
an ~org-mode~ table as a String, and since ~org-mode~ does not support colors,
|
1747
|
+
any color directives are ignored. Some of the output targets are not strings,
|
1748
|
+
but ruby data structures, and for them, things such as alignment are irrelevant.
|
1728
1749
|
|
1729
1750
|
*** Available Formatters
|
1730
1751
|
|
@@ -1743,13 +1764,13 @@ alignment are simply irrelevant.
|
|
1743
1764
|
|
1744
1765
|
These are all implemented by classes that inherit from ~FatTable::Formatter~
|
1745
1766
|
class by defining about a dozen methods that get called at various places during
|
1746
|
-
the construction of the output table.
|
1767
|
+
the construction of the output table. The idea is that more classes can be
|
1747
1768
|
defined by adding additional classes.
|
1748
1769
|
|
1749
1770
|
*** Table Locations
|
1750
1771
|
|
1751
1772
|
In the formatting methods, the table is divided into several "locations" for
|
1752
|
-
which separate formatting directives may be given.
|
1773
|
+
which separate formatting directives may be given. These locations are
|
1753
1774
|
identified with the following symbols:
|
1754
1775
|
|
1755
1776
|
- :header :: the first row of the output table containing the headers,
|
@@ -1794,8 +1815,8 @@ to a string in forming the output.
|
|
1794
1815
|
- _ ~_ :: underline the element, or turn off underline
|
1795
1816
|
- * ~* :: cause the element to blink, or turn off blink
|
1796
1817
|
|
1797
|
-
For example, the directive 'tCc[red.yellow]' would title-case the element,
|
1798
|
-
center it, and color it red on a yellow background.
|
1818
|
+
For example, the directive ~'tCc[red.yellow]'~ would title-case the element,
|
1819
|
+
center it, and color it red on a yellow background. The directives that are
|
1799
1820
|
boolean have negating forms so that, for example, if bold is turned on for all
|
1800
1821
|
columns of a given type, it can be countermanded in formatting directives for
|
1801
1822
|
particular columns.
|
@@ -1811,65 +1832,63 @@ addition to the following:
|
|
1811
1832
|
the left with zeroes as needed, and round the number to the n
|
1812
1833
|
decimal places and include n digits after the decimal point,
|
1813
1834
|
padding on the right with zeroes as needed,
|
1814
|
-
- H :: convert the number (assumed to be in units of seconds) to
|
1815
|
-
|
1816
|
-
|
1817
|
-
|
1818
|
-
formatting instruction.
|
1835
|
+
- H :: convert the number (assumed to be in units of seconds) to ~HH:MM:SS.ss~
|
1836
|
+
form. So a column that is the result of subtracting two :datetime forms
|
1837
|
+
will result in a :numeric expressed as seconds and can be displayed in
|
1838
|
+
hours, minutes, and seconds with this formatting instruction.
|
1819
1839
|
|
1820
|
-
For example, the directive 'R5.0c[blue]' would right-align the numeric
|
1821
|
-
pad it on the left with zeros, and color it blue.
|
1840
|
+
For example, the directive ~'R5.0c[blue]'~ would right-align the numeric
|
1841
|
+
element, pad it on the left with zeros, and color it blue.
|
1822
1842
|
|
1823
1843
|
**** DateTime
|
1824
1844
|
|
1825
|
-
For a
|
1826
|
-
to the following:
|
1845
|
+
For a ~DateTime~, all the instructions valid for string are available, in
|
1846
|
+
addition to the following:
|
1827
1847
|
|
1828
|
-
- d[fmt] :: apply the format to a
|
1829
|
-
no or zero hour, minute, and second components, where fmt
|
1830
|
-
string for Date#strftime
|
1831
|
-
ISO 8601 string, YYYY-MM-DD.
|
1832
|
-
- D[fmt] :: apply the format to a datetime that has at least a non-zero
|
1833
|
-
|
1834
|
-
|
1835
|
-
8601 string, YYYY-MM-DD.
|
1848
|
+
- d[fmt] :: apply the format to a ~Date~ or a ~DateTime~ that is a whole day,
|
1849
|
+
that is that has no or zero hour, minute, and second components, where fmt
|
1850
|
+
is a valid format string for ~Date#strftime~, otherwise, the datetime will
|
1851
|
+
be formatted as an ISO 8601 string, YYYY-MM-DD.
|
1852
|
+
- D[fmt] :: apply the format to a datetime that has at least a non-zero hour
|
1853
|
+
component where fmt is a valid format string for Date#strftime, otherwise,
|
1854
|
+
the datetime will be formatted as an ISO 8601 string, YYYY-MM-DD.
|
1836
1855
|
|
1837
|
-
For example, 'c[pink]d[%b %-d, %Y]C'
|
1838
|
-
1957', center it, and color it pink.
|
1856
|
+
For example, ~'c[pink]d[%b %-d, %Y]C'~, would format a date element like 'Sep
|
1857
|
+
22, 1957', center it, and color it pink.
|
1839
1858
|
|
1840
1859
|
**** Boolean
|
1841
1860
|
|
1842
1861
|
For a boolean cell, all the instructions valid for string are available, in
|
1843
|
-
|
1862
|
+
addition to the following:
|
1844
1863
|
|
1845
|
-
- Y :: print true as 'Y' and false as 'N',
|
1846
|
-
- T :: print true as 'T' and false as 'F',
|
1847
|
-
- X :: print true as 'X' and false as '',
|
1848
|
-
- b[xxx,yyy] :: print true as the string given as xxx and false as the
|
1849
|
-
|
1850
|
-
- c[tcolor,fcolor] :: color a true element with tcolor and a false
|
1851
|
-
|
1852
|
-
|
1864
|
+
- Y :: print true as '~Y~' and false as '~N~',
|
1865
|
+
- T :: print true as '~T~' and false as '~F~',
|
1866
|
+
- X :: print true as '~X~' and false as an empty string '',
|
1867
|
+
- b[xxx,yyy] :: print true as the string given as ~xxx~ and false as the string
|
1868
|
+
given as ~yyy~,
|
1869
|
+
- c[tcolor,fcolor] :: color a true element with ~tcolor~ and a false element
|
1870
|
+
with ~fcolor~. Each of the colors may be specified in the same manner as
|
1871
|
+
colors for strings described above.
|
1853
1872
|
|
1854
|
-
For example, the directive 'b[Yeppers,Nope]c[green.pink,red.pink]' would
|
1855
|
-
a true boolean as 'Yeppers' colored green on pink and render a false
|
1856
|
-
'Nope' colored red on pink. See [[https://www.youtube.com/watch?v=oLdFFD8II8U][Yeppers]].
|
1873
|
+
For example, the directive '~b[Yeppers,Nope]c[green.pink,red.pink]~' would
|
1874
|
+
render a true boolean as '~Yeppers~' colored green on pink and render a false
|
1875
|
+
boolean as '~Nope~' colored red on pink. See [[https://www.youtube.com/watch?v=oLdFFD8II8U][Yeppers]] for additional information.
|
1857
1876
|
|
1858
1877
|
**** NilClass
|
1859
1878
|
|
1860
|
-
By default, nil elements are rendered as blank cells, but you can make them
|
1879
|
+
By default, ~nil~ elements are rendered as blank cells, but you can make them
|
1861
1880
|
visible with the following, and in that case, all the formatting instructions
|
1862
1881
|
valid for strings are also available:
|
1863
1882
|
|
1864
|
-
- n[niltext] :: render a nil item with the given
|
1883
|
+
- n[niltext] :: render a ~nil~ item with the given niltext.
|
1865
1884
|
|
1866
|
-
For example, you might want to use 'n[-]Cc[purple]' to make nils visible as a
|
1867
|
-
centered hyphen.
|
1885
|
+
For example, you might want to use ~'n[-]Cc[purple]'~ to make nils visible as a
|
1886
|
+
centered purple hyphen.
|
1868
1887
|
|
1869
1888
|
*** Footers Methods
|
1870
1889
|
|
1871
|
-
You can call methods on
|
1872
|
-
Their signatures are:
|
1890
|
+
You can call the ~footer~ and ~gfooter~ methods on ~Formatter~ objects to add
|
1891
|
+
footers and group footers. Their signatures are:
|
1873
1892
|
|
1874
1893
|
- ~footer(label, *sum_cols, **agg_cols)~ :: where ~label~ is a label to be
|
1875
1894
|
placed in the first cell of the footer (unless that column is named as one
|
@@ -1877,8 +1896,8 @@ Their signatures are:
|
|
1877
1896
|
~*sum_cols~ are zero or more symbols for columns to be summed, and
|
1878
1897
|
~**agg_cols~ is zero or more hash-like parameters with a column symbol as a
|
1879
1898
|
key and a symbol for an aggregate method as the value. This causes a
|
1880
|
-
table-wide header to be added at the bottom of the table applying the
|
1881
|
-
aggregate to the ~sum_cols~ and the named aggregate method to the
|
1899
|
+
table-wide header to be added at the bottom of the table applying the
|
1900
|
+
~:sum~ aggregate to the ~sum_cols~ and the named aggregate method to the
|
1882
1901
|
~agg_cols~. A table can have any number of footers attached, and they will
|
1883
1902
|
appear at the bottom of the output table in the order they are given.
|
1884
1903
|
|
@@ -1915,10 +1934,10 @@ and ~format~.
|
|
1915
1934
|
**** Instantiating a Formatter
|
1916
1935
|
|
1917
1936
|
There are several ways to invoke the formatting methods on a table. First, you
|
1918
|
-
can instantiate a ~XXXFormatter~ object
|
1919
|
-
subclass for each target output medium, for example,
|
1920
|
-
a ruby array of arrays. You can then call the
|
1921
|
-
~XXXFormatter~.
|
1937
|
+
can instantiate a ~XXXFormatter~ object and feed it a table as a parameter.
|
1938
|
+
There is a Formatter subclass for each target output medium, for example,
|
1939
|
+
~AoaFormatter~ will produce a ruby array of arrays. You can then call the
|
1940
|
+
~output~ method on the ~XXXFormatter~.
|
1922
1941
|
|
1923
1942
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
1924
1943
|
#+BEGIN_SRC ruby
|
@@ -2072,12 +2091,12 @@ Other than that first parameter, the two methods take the same types of
|
|
2072
2091
|
parameters. The remaining parameters are hash-like parameters that use either a
|
2073
2092
|
column name or a type as the key and a string with the formatting directives to
|
2074
2093
|
apply as the value. The following example says to set the formatting for all
|
2075
|
-
locations in the table and to
|
2076
|
-
numbers (the '0.0' part), that are right-aligned (the 'R'
|
2077
|
-
grouping commas inserted (the ',' part). But the ~:id~ column is
|
2078
|
-
the second parameter overrides the formatting for numerics in
|
2079
|
-
for the ~:id~ column to be padded to three digits with zeros
|
2080
|
-
'3.0' part) and to be centered (the 'C' part).
|
2094
|
+
locations in the table and to format all numeric fields as strings that are
|
2095
|
+
rounded to whole numbers (the '0.0' part), that are right-aligned (the 'R'
|
2096
|
+
part), and have grouping commas inserted (the ',' part). But the ~:id~ column is
|
2097
|
+
numeric, and the second parameter overrides the formatting for numerics in
|
2098
|
+
general and calls for the ~:id~ column to be padded to three digits with zeros
|
2099
|
+
on the left (the '3.0' part) and to be centered (the 'C' part).
|
2081
2100
|
|
2082
2101
|
#+HEADER: :colnames no :session readme :hlines yes :wrap EXAMPLE :exports both
|
2083
2102
|
#+BEGIN_SRC ruby
|
@@ -2100,14 +2119,14 @@ for the ~:id~ column to be padded to three digits with zeros on the left (the
|
|
2100
2119
|
#+END_EXAMPLE
|
2101
2120
|
|
2102
2121
|
The ~numeric:~ directive affected the ~:age~ and ~:salary~ columns and the ~id:~
|
2103
|
-
directive affected only the ~:id~ column.
|
2122
|
+
directive affected only the ~:id~ column. All the other cells in the table had
|
2104
2123
|
the default formatting applied.
|
2105
2124
|
|
2106
2125
|
**** Location priority
|
2107
2126
|
|
2108
|
-
Formatting for any given cell depends on its location in the table.
|
2127
|
+
Formatting for any given cell depends on its location in the table. The
|
2109
2128
|
~format_for~ method takes a location to which its formatting directive are
|
2110
|
-
restricted as the first argument.
|
2129
|
+
restricted as the first argument. It can be one of the following:
|
2111
2130
|
|
2112
2131
|
- ~:header~ :: directive apply only to the header row, that is the first row, of
|
2113
2132
|
the output table,
|
@@ -2122,11 +2141,11 @@ restricted as the first argument. It can be one of the following:
|
|
2122
2141
|
row is the first row in the table or in a group and separate directives for
|
2123
2142
|
those have been given, in which case those directives apply,
|
2124
2143
|
|
2125
|
-
- ~:gfirst~ :: directives apply to the first row in each group in the
|
2126
|
-
table, unless the row is also the first row in the table as a whole, in
|
2144
|
+
- ~:gfirst~ :: directives apply to the first row in each group in the body of
|
2145
|
+
the table, unless the row is also the first row in the table as a whole, in
|
2127
2146
|
which case the ~:bfirst~ directives apply,
|
2128
2147
|
|
2129
|
-
- ~:bfirst~ :: directives apply to the first row in the table.
|
2148
|
+
- ~:bfirst~ :: directives apply to the first row in the body of the table.
|
2130
2149
|
|
2131
2150
|
If you give directives for ~:body~, they are copied to ~:bfirst~ and ~:gfirst~
|
2132
2151
|
as well and can be overridden by directives for those locations.
|
@@ -2171,7 +2190,8 @@ require 'fat_table'
|
|
2171
2190
|
|
2172
2191
|
The ~string: 'R'~ directive causes all the cells to be right-aligned except
|
2173
2192
|
~:id~ which specifies centering for the ~:id~ column only. The ~n[N/A]~
|
2174
|
-
directive for
|
2193
|
+
directive for specifies how nil are displayed in the numeric column, ~:salary~,
|
2194
|
+
but not for other nils, such as in the last row of the ~:join_date~ column.
|
2175
2195
|
|
2176
2196
|
* Development
|
2177
2197
|
|