red_amber 0.1.2 → 0.1.5

data/doc/Vector.md

@@ -0,0 +1,317 @@
+# Vector
+
+Class `RedAmber::Vector` represents a series of data in the DataFrame.
+
+## Constructor
+
+### Create from a column in a DataFrame
+  
+  ```ruby
+  df = RedAmber::DataFrame.new(x: [1, 2, 3])
+  df[:x]
+  # =>
+  #<RedAmber::Vector(:uint8, size=3):0x000000000000f4ec>
+  [1, 2, 3]
+  ```
+
+### New from an Array
+
+  ```ruby
+  vector = RedAmber::Vector.new([1, 2, 3])
+  # =>
+  #<RedAmber::Vector(:uint8, size=3):0x000000000000f514>
+  [1, 2, 3]
+  ```
+
+## Properties
+
+### `to_s`
+
+### `values`, `to_a`, `entries`
+
+### `size`, `length`, `n_rows`, `nrow`
+
+### `type`
+
+### `boolean?`, `numeric?`, `string?`, `temporal?`
+
+### `type_class`
+
+### [ ] `each` (not impremented yet)
+
+### [ ] `chunked?` (not impremented yet)
+
+### [ ] `n_chunks` (not impremented yet)
+
+### [ ] `each_chunk` (not impremented yet)
+
+### `n_nils`, `n_nans`
+
+  - `n_nulls` is an alias of `n_nils`
+
+### `inspect(limit: 80)`
+
+  - `limit` sets size limit to display long array.
+
+    ```ruby
+    vector = RedAmber::Vector.new((1..50).to_a)
+    # =>
+    #<RedAmber::Vector(:uint8, size=50):0x000000000000f528>
+    [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, ... ]
+    ```
+
+## Functions
+
+### Unary aggregations: `vector.func => scalar`
+
+  ![unary aggregation](doc/image/../../image/vector/unary_aggregation_w_option.png)
+
+| Method    |Boolean|Numeric|String|Options|Remarks|
+| ----------- | --- | --- | --- | --- | --- |
+| ✓ `all`     |  ✓  |     |     | ✓ ScalarAggregate|     |
+| ✓ `any`     |  ✓  |     |     | ✓ ScalarAggregate|     |
+| ✓ `approximate_median`|  |✓|  | ✓ ScalarAggregate| alias `median`|
+| ✓ `count`   |  ✓  |  ✓  |  ✓  | ✓  Count  |     |
+| ✓ `count_distinct`| ✓ | ✓ | ✓ | ✓  Count  |alias `count_uniq`|
+|[ ]`index`   | [ ] | [ ] | [ ] |[ ] Index  |     |
+| ✓ `max`     |  ✓  |  ✓  |  ✓  | ✓ ScalarAggregate|     |
+| ✓ `mean`    |  ✓  |  ✓  |     | ✓ ScalarAggregate|     |
+| ✓ `min`     |  ✓  |  ✓  |  ✓  | ✓ ScalarAggregate|     |
+| ✓ `min_max` |  ✓  |  ✓  |  ✓  | ✓ ScalarAggregate|     |
+|[ ]`mode`    |     | [ ] |     |[ ] Mode    |     |
+| ✓ `product` |  ✓  |  ✓  |     | ✓ ScalarAggregate|     |
+|[ ]`quantile`|     | [ ] |     |[ ] Quantile|     |
+| ✓ `sd    `  |     |  ✓  |     |          |ddof: 1 at `stddev`|
+| ✓ `stddev`  |     |  ✓  |     | ✓ Variance|ddof: 0 by default|
+| ✓ `sum`     |  ✓  |  ✓  |     | ✓ ScalarAggregate|     |
+|[ ]`tdigest` |     | [ ] |     |[ ] TDigest |     |
+| ✓ `var    `|     |  ✓  |     |   |ddof: 1 at `variance`<br>alias `unbiased_variance`|
+| ✓ `variance`|     |  ✓  |     | ✓ Variance|ddof: 0 by default|
+
+
+Options can be used as follows.
+See the [document of C++ function](https://arrow.apache.org/docs/cpp/compute.html) for detail.
+
+```ruby
+double = RedAmber::Vector.new([1, 0/0.0, -1/0.0, 1/0.0, nil, ""])
+#=>
+#<RedAmber::Vector(:double, size=6):0x000000000000f910>
+[1.0, NaN, -Infinity, Infinity, nil, 0.0]
+
+double.count #=> 5
+double.count(opts: {mode: :only_valid}) #=> 5, default
+double.count(opts: {mode: :only_null}) #=> 1
+double.count(opts: {mode: :all}) #=> 6
+
+boolean = RedAmber::Vector.new([true, true, nil])
+#=>
+#<RedAmber::Vector(:boolean, size=3):0x000000000000f924>
+[true, true, nil]
+
+boolean.all #=> true
+boolean.all(opts: {skip_nulls: true}) #=> true
+boolean.all(opts: {skip_nulls: false}) #=> false
+```
+
+### Unary element-wise: `vector.func => vector`
+
+  ![unary element-wise](doc/image/../../image/vector/unary_element_wise.png)
+
+| Method    |Boolean|Numeric|String|Options|Remarks|
+| ------------ | --- | --- | --- | --- | ----- |
+| ✓ `-@`       |     |  ✓  |     |     |as `-vector`|
+| ✓ `negate`   |     |  ✓  |     |     |`-@`   |
+| ✓ `abs`      |     |  ✓  |     |     |       |
+|[ ]`acos`     |     | [ ] |     |     |       |
+|[ ]`asin`     |     | [ ] |     |     |       |
+| ✓ `atan`     |     |  ✓  |     |     |       |
+| ✓ `bit_wise_not`|  | (✓) |     |     |integer only|
+| ✓ `ceil`     |     |  ✓  |     |     |       |
+| ✓ `cos`      |     |  ✓  |     |     |       |
+| ✓`fill_nil_backward`| ✓ | ✓ | ✓ |    |       |
+| ✓`fill_nil_forward` | ✓ | ✓ | ✓ |    |       |
+| ✓ `floor`    |     |  ✓  |     |     |       |
+| ✓ `invert`   |  ✓  |     |     |     |`!`, alias `not`|
+|[ ]`ln`       |     | [ ] |     |     |       |
+|[ ]`log10`    |     | [ ] |     |     |       |
+|[ ]`log1p`    |     | [ ] |     |     |       |
+|[ ]`log2`     |     | [ ] |     |     |       |
+| ✓ `round`    |     |  ✓  |     | ✓ Round (:mode, :n_digits)|    |
+| ✓ `round_to_multiple`| | ✓ |   | ✓ RoundToMultiple :mode, :multiple| multiple must be an Arrow::Scalar|
+| ✓ `sign`     |     |  ✓  |     |     |       |
+| ✓ `sin`      |     |  ✓  |     |     |       |
+| ✓`sort_indexes`| ✓  | ✓  | ✓  |:order|alias `sort_indices`|
+| ✓ `tan`      |     |  ✓  |     |     |       |
+| ✓ `trunc`    |     |  ✓  |     |     |       |
+
+### Binary element-wise: `vector.func(vector) => vector`
+
+  ![binary element-wise](doc/image/../../image/vector/binary_element_wise.png)
+
+| Method       |Boolean|Numeric|String|Options|Remarks|
+| ----------------- | --- | --- | --- | --- | ----- |
+| ✓ `add`           |     |  ✓  |     |     | `+`   |
+| ✓ `atan2`         |     |  ✓  |     |     |       |
+| ✓ `and_kleene`    |  ✓  |     |     |     | `&`   |
+| ✓ `and_org   `    |  ✓  |     |     |     |`and` in Red Arrow|
+| ✓ `and_not`       |  ✓  |     |     |     |       |
+| ✓ `and_not_kleene`|  ✓  |     |     |     |       |
+| ✓ `bit_wise_and`  |     | (✓) |     |     |integer only|
+| ✓ `bit_wise_or`   |     | (✓) |     |     |integer only|
+| ✓ `bit_wise_xor`  |     | (✓) |     |     |integer only|
+| ✓ `divide`        |     |  ✓  |     |     | `/`   |
+| ✓ `equal`         |  ✓  |  ✓  |  ✓  |     |`==`, alias `eq`|
+| ✓ `greater`       |  ✓  |  ✓  |  ✓  |     |`>`, alias `gt`|
+| ✓ `greater_equal` |  ✓  |  ✓  |  ✓  |     |`>=`, alias `ge`|
+| ✓ `is_finite`     |     |  ✓  |     |     |       |
+| ✓ `is_inf`        |     |  ✓  |     |     |       |
+| ✓ `is_na`         |  ✓  |  ✓  |  ✓  |     |       |
+| ✓ `is_nan`        |     |  ✓  |     |     |       |
+|[ ]`is_nil`        |  ✓  |  ✓  |  ✓  |[ ] Null|alias `is_null`|
+| ✓ `is_valid`      |  ✓  |  ✓  |  ✓  |     |       |
+| ✓ `less`          |  ✓  |  ✓  |  ✓  |     |`<`, alias `lt`|
+| ✓ `less_equal`    |  ✓  |  ✓  |  ✓  |     |`<=`, alias `le`|
+|[ ]`logb`          |     | [ ] |     |     |       |
+|[ ]`mod`           |     | [ ] |     |     | `%`   |
+| ✓ `multiply`      |     |  ✓  |     |     | `*`   |
+| ✓ `not_equal`     |  ✓  |  ✓  |  ✓  |     |`!=`, alias `ne`|
+| ✓ `or_kleene`     |  ✓  |     |     |     | `\|`  |
+| ✓ `or_org`        |  ✓  |     |     |     |`or` in Red Arrow|
+| ✓ `power`         |     |  ✓  |     |     | `**`  |
+| ✓ `subtract`      |     |  ✓  |     |     | `-`   |
+| ✓ `shift_left`    |     | (✓) |     |     |`<<`, integer only|
+| ✓ `shift_right`   |     | (✓) |     |     |`>>`, integer only|
+| ✓ `xor`           |  ✓  |     |     |     | `^`   |
+
+### `uniq`
+
+  Returns a new array with distinct elements.
+
+(Not impremented functions)
+
+### `tally` and `value_counts`
+
+  Compute counts of unique elements and return a Hash.
+
+  It returns almost same result as Ruby's tally. These methods consider NaNs are same.
+
+  ```ruby
+  array = [0.0/0, Float::NAN]
+  array.tally #=> {NaN=>1, NaN=>1}
+
+  vector = RedAmber::Vector.new(array)
+  vector.tally #=> {NaN=>2}
+  vector.value_counts #=> {NaN=>2}
+  ```
+
+### `sort_indexes`, `sort_indices`, `array_sort_indices`
+
+### [ ] `sort`, `sort_by`
+### [ ] argmin, argmax
+### [ ] (array functions)
+### [ ] (strings functions)
+### [ ] (temporal functions)
+### [ ] (conditional functions)
+### [ ] (index functions)
+### [ ] (other functions)
+
+## Coerce (not impremented)
+
+## Update vector's value
+### `replace_with(booleans, replacements)` => vector
+
+- Accepts Vector, Array, Arrow::Array for booleans and replacements.
+  - Replacements can accept scalar
+- Booleans specifies the position of replacement in true.
+- Replacements specifies the vaues to be replaced.
+  - The number of true in booleans must be equal to the length of replacement
+
+```ruby
+vector = RedAmber::Vector.new([1, 2, 3])
+booleans = [true, false, true]
+replacemants = [4, 5]
+vector.replace_with(booleans, replacemants)
+# => 
+#<RedAmber::Vector(:uint8, size=3):0x000000000001ee10>
+[4, 2, 5] 
+```
+
+- Scalar value in replacements can be broadcasted.
+
+```ruby
+replacemant = 0
+vector.replace_with(booleans, replacement)
+# => 
+#<RedAmber::Vector(:uint8, size=3):0x000000000001ee10>
+[0, 2, 0] 
+```
+
+- Returned data type is automatically up-casted by replacement.
+
+```ruby
+replacement = 1.0
+vector.replace_with(booleans, replacement)
+# => 
+#<RedAmber::Vector(:double, size=3):0x0000000000025d78>
+[1.0, 2.0, 1.0]
+```
+
+- Position of nil in booleans is replaced with nil.
+
+```ruby
+booleans = [true, false, nil]
+replacemant = -1
+vec.replace_with(booleans, replacement)
+=> 
+#<RedAmber::Vector(:int8, size=3):0x00000000000304d0>
+[-1, 2, nil]
+```
+
+- Replacemants can have nil in it.
+
+```ruby
+booleans = [true, false, true]
+replacemants = [nil]
+vec.replace_with(booleans, replacemants)
+=> 
+#<RedAmber::Vector(:int8, size=3):0x00000000000304d0>
+[nil, 2, nil]
+```
+
+- If no replacemants specified, it is same as to specify nil.
+
+```ruby
+booleans = [true, false, true]
+vec.replace_with(booleans)
+=> 
+#<RedAmber::Vector(:int8, size=3):0x00000000000304d0>
+[nil, 2, nil]
+```
+
+- An example to replace 'NA' to nil.
+
+```ruby
+vector = RedAmber::Vector.new(['A', 'B', 'NA'])
+vector.replace_with(vector == 'NA', nil)
+# =>
+#<RedAmber::Vector(:string, size=3):0x000000000000f8ac>
+["A", "B", nil]
+```
+
+### `fill_nil_forward`, `fill_nil_backward` => vector
+
+Propagate the last valid observation forward (or backward).
+Or preserve nil if all previous values are nil or at the end.
+
+```ruby
+integer = RedAmber::Vector.new([0, 1, nil, 3, nil])
+integer.fill_nil_forward
+# =>
+#<RedAmber::Vector(:uint8, size=5):0x000000000000f960>
+[0, 1, 1, 3, 3]
+
+integer.fill_nil_backward
+# =>
+#<RedAmber::Vector(:uint8, size=5):0x000000000000f974>
+[0, 1, 3, 3, nil]
+```

data/doc/tdr.md

@@ -0,0 +1,56 @@
+# TDR (Transposed DataFrame Representation)
+
+([Japanese version](tdr_ja.md) of this document is available)
+
+TDR is a presentation style of 2D data. It shows columnar vector values in *row Vector* and observations in *column* just like a **transposed** table.
+
+![TDR Image](image/tdr.png) 
+
+Row-oriented data table (1) and columnar data table (2) have different data allocation in memory within a context of Arrow Columnar Format. But they have the same data placement (in rows and columns) in our brain.
+
+TDR (3) is a logical concept of data placement to transpose rows and columns in a columnar table (2).
+
+![TDR and Table Image](image/tdr_and_table.png)
+
+TDR is not an implementation in software but a logical image in our mind.
+
+TDR is consistent with the 'transposed' tidy data concept. The only thing we should do is not to use the positional words 'row' and 'column'.
+
+![tidy data in TDR](image/tidy_data_in_TDR.png)
+
+TDR is one of a simple way to create DataFrame object in many libraries. For example, we can initalize Arrow::Table in Red Arrow like the right below and get table as left.
+
+![Arrow Table New](image/arrow_table_new.png)
+
+We are using TDR style code naturally. For other example:
+  - Ruby: Daru::DataFrame, Rover::DataFrame accept same arguments.
+  - Python: similar style in Pandas for pd.DataFrame(data_in_dict)
+  - R: similar style in tidyr for tibble(x = 1:3, y = c("A", "B", "C"))
+
+There are other ways to initialize data frame, but they are not intuitive.
+
+## Table and TDR API
+
+The API based on TDR is draft and RedAmber is a small experiment to test the TDR concept. The following is a comparison of Table and TDR (draft).
+
+|     |Basic Table|Transposed DataFrame|Comment for TDR|
+|-----------|---------|------------|---|
+|name in TDR|`Table`|`TDR`|**T**ransposed **D**ataFrame **R**epresentation|
+|variable   |located in a column|a key and a `Vector` in lateral|select by keys|
+|observation|located in a row|sliced in vertical|select by indices|
+|number of variables|n_columns etc. |`n_keys`  |`n_cols` is available as an alias|
+|number of observations|n_rows etc. |`size` |`n_rows` is available as an alias|
+|shape      |[n_rows, n_columns]  |`shape`=`[size, n_keys]` |same order as Table|
+|Select variables|select, filter, [ ], etc.|`pick` or `[keys]`  |accepts arguments or a block|
+|Reject variables|drop, etc.|`drop`  |accepts arguments or a block|
+|Select observations|slice, [ ], iloc, etc.|`slice` or `[indices]` |accepts arguments or a block|
+|Reject observations|drop, etc.|`remove`  |accepts arguments or a block|
+|Add variables|mutate, assign, etc.|`assign`  |accepts arguments or a block|
+|update variables|transmute, [ ]=, etc.|`assign`  |accepts arguments or a block|
+|inner join| inner_join(a,b)<br>merge(a, b, how='inner')|`a.inner_join(b)` |with a option on:|
+|left join| left_join(a,b)<br>merge(a, b, how='left')|`a.join(b)` |naturally join from bottom<br>with a option on:|
+|right join| right_join(a,b))<br>merge(a, b, how='right')|`b.join(a)` |naturally join from bottom<br>with a option on:|
+
+## Q and A for TDR
+
+（Not prepared yet)

data/doc/tdr_ja.md

@@ -0,0 +1,56 @@
+# TDR (Transposed DataFrame Representation)
+
+([英語版](tdr.md) もあります)
+
+TDR は、２次元のデータの表現方法につけた名前です。TDR では下の図のように同じ型のデータに key というラベルをつけて横に並べ、それらを縦に積み重ねてデータを表現します。
+
+![TDR Image](image/tdr.png) 
+
+Arrow Columnar Format では、csv のような従来の行指向データ(1)に対して、列方向に連続したデータ(2)を取り扱います。この行、列という言葉は私たちの脳内イメージを規定していて、データフレームの構造といえば(1)または(2)のような形を思い浮かべることでしょう。しかし、本質は連続したデータの配置にあるので、我々の頭の中では(3)のように行と列を入れ替えて考えてもいいはずです。
+
+![TDR and Table Image](image/tdr_and_table.png)
+
+大事なことは、TDR は頭の中の論理的なイメージであって、実装上のアーキテクチャではないということです。
+
+TDR は、整然データ(tidy data)の考え方とも矛盾しません。TDR における整然データは行と列を入れ替えた形で全く同じデータを表しています。一つだけ気をつけることは、混乱を避けるため、位置や方向に関するワードである行(row)や列(column)を避けるべきであるということです。
+
+![tidy data in TDR](image/tidy_data_in_TDR.png)
+
+TDR は、現時点でも2次元データを楽に初期化できる記法で、ごく自然に使われています。例えば、Red Arrow ではArrow::Table を初期化する際に下の図の右のように書けます。
+
+![Arrow Table New](image/arrow_table_new.png)
+
+これはごく自然な書き方ですが、この形は TDR の形と一致しています。その他の例として:
+  - Ruby: Daru::DataFrame, Rover::DataFrame でも上と同じように書けます。
+  - Python: Pandas で pd.DataFrame(data_in_dict) のように dict を使う場合が同じです。
+  - R: tidyr で tibble(x = 1:3, y = c("A", "B", "C")) のように書けます。
+
+それぞれのライブラリーで、データフレームを初期化するやり方はこれだけではありませんが、他の方法は少し回りくどいような印象があります。
+
+TDR で考えた方がちょっぴりうまくいくというのは単なる仮説ですが、その理由は「この惑星では横書きでコードを書く」からではないかと私は考えています。
+
+## Table and TDR API
+
+TDR に基づいた API はまだ暫定板の段階であり、RedAmber は TDR の実験の場であると考えています。下記の表に TDR と行x列形式の Table のAPIの比較を示します（暫定版）。
+
+|     |従来の Table|Transposed DataFrame|TDRに対するコメント|
+|-----------|---------|------------|---|
+|TDRでの呼称|`Table`|`TDR`|**T**ransposed **D**ataFrame **R**epresentationの略|
+|変数 |列に配置|`variables`<br>key と `Vector` として横方向に配置|key で選択|
+|観測 |行に配置|`observations`<br>縦方向に切った一つ一つはslice|index や `slice` メソッドで選択|
+|変数(列)の数|ncol, n_columns など |`n_keys`  |`n_cols` をエイリアスとして設定|
+|観測(行)の数|nrow, n_rows など |`size` |`n_rows` をエイリアスとして設定|
+|形状      |[nrow, ncol]  |`shape`=`[size, n_keys]` |行, 列の順番は同じ|
+|変数(列)の選択|select, filter, [ ], など|`pick` or `[keys]`  |引数またはブロックで指定|
+|変数(列)の削除|drop, など|`drop`  |引数またはブロックで指定|
+|観測(行)の選択|slice, [ ], iloc, など|`slice` or `[indices]` |引数またはブロックで指定|
+|観測(行)の削除|drop, など|`remove`  |引数またはブロックで指定|
+|変数(列)の追加|mutate, assign, など|`assign`  |引数またはブロックで指定|
+|変数(列)の更新|transmute, [ ]=, など|`assign`  |引数またはブロックで指定|
+|内部結合| inner_join(a,b)<br>merge(a, b, how='inner')|`a.inner_join(b)` |オプション on:|
+|左結合| left_join(a,b)<br>merge(a, b, how='left')|`a.join(b)` |自然に下にくっつける<br>オプション on:|
+|右結合| right_join(a,b))<br>merge(a, b, how='right')|`b.join(a)` |自然に下にくっつける<br>オプション on:|
+
+## Q and A for TDR
+
+（作成中)

data/lib/red_amber/data_frame.rb

@@ -5,19 +5,23 @@ module RedAmber
   #   @table   : holds Arrow::Table object
   class DataFrame
     # mix-in
+    include DataFrameDisplayable
+    include DataFrameHelper
+    include DataFrameIndexable
     include DataFrameSelectable
-    include DataFrameOutput
+    include DataFrameObservationOperation
+    include DataFrameVariableOperation
 
     def initialize(*args)
-      # DataFrame.new, DataFrame.new([]), DataFrame.new({}), DataFrame.new(nil)
-      #   returns empty DataFrame
-      @table = Arrow::Table.new({}, [])
+      @variables = @keys = @vectors = @types = @data_types = nil
       # bug in gobject-introspection: ruby-gnome/ruby-gnome#1472
       #  [Arrow::Table] == [nil] shows ArgumentError
       #  temporary use yoda condition to workaround
-      return if args.empty? || args == [[]] || args == [{}] || [nil] == args
-
-      if args.size > 1
+      if args.empty? || args == [[]] || args == [{}] || [nil] == args
+        # DataFrame.new, DataFrame.new([]), DataFrame.new({}), DataFrame.new(nil)
+        #   returns empty DataFrame
+        @table = Arrow::Table.new({}, [])
+      elsif args.size > 1
         @table = Arrow::Table.new(*args)
       else
         arg = args[0]
@@ -39,56 +43,71 @@ module RedAmber
 
     attr_reader :table
 
+    def to_arrow
+      table
+    end
+
     def save(output, options = {})
       @table.save(output, options)
     end
 
-    # Properties ===
-    def n_rows
+    def size
       @table.n_rows
     end
-    alias_method :nrow, :n_rows
-    alias_method :size, :n_rows
-    alias_method :length, :n_rows
+    alias_method :n_rows, :size
+    alias_method :n_obs, :size
 
-    def n_columns
+    def n_keys
       @table.n_columns
     end
-    alias_method :ncol, :n_columns
-    alias_method :width, :n_columns
+    alias_method :n_cols, :n_keys
+    alias_method :n_vars, :n_keys
 
     def shape
-      [n_rows, n_columns]
+      [size, n_keys]
+    end
+
+    def variables
+      @variables || @variables = init_instance_vars(:variables)
+    end
+    alias_method :vars, :variables
+
+    def keys
+      @keys || @keys = init_instance_vars(:keys)
+    end
+    alias_method :column_names, :keys
+    alias_method :var_names, :keys
+
+    def key?(key)
+      keys.include?(key.to_sym)
     end
+    alias_method :has_key?, :key?
 
-    def column_names
-      @table.columns.map { |column| column.name.to_sym }
+    def key_index(key)
+      keys.find_index(key.to_sym)
     end
-    alias_method :keys, :column_names
-    alias_method :header, :column_names
+    alias_method :find_index, :key_index
+    alias_method :index, :key_index
 
     def types
-      @table.columns.map do |column|
-        column.data_type.to_s.to_sym
-      end
+      @types || @types = @table.columns.map { |column| column.data.value_type.nick.to_sym }
     end
 
-    def data_types
-      @table.columns.map do |column|
-        column.data_type.class
-      end
+    def type_classes
+      @data_types || @data_types = @table.columns.map { |column| column.data_type.class }
     end
 
     def vectors
-      @table.columns.map do |column|
-        Vector.new(column.data)
-      end
+      @vectors || @vectors = init_instance_vars(:vectors)
+    end
+
+    def indexes
+      0...size
     end
+    alias_method :indices, :indexes
 
     def to_h
-      @table.columns.each_with_object({}) do |column, result|
-        result[column.name.to_sym] = column.entries
-      end
+      variables.transform_values(&:to_a)
     end
 
     def to_a
@@ -107,13 +126,27 @@ module RedAmber
     end
 
     def empty?
-      @table.columns.empty?
+      variables.empty?
     end
 
     def to_rover
       Rover::DataFrame.new(to_h)
     end
 
-    # def to_parquet() end
+    private
+
+    # initialize @variable, @keys, @vectors and return one of them
+    def init_instance_vars(var)
+      ary = @table.columns.each_with_object([{}, [], []]) do |column, (variables, keys, vectors)|
+        v = Vector.new(column.data)
+        k = column.name.to_sym
+        v.key = k
+        variables[k] = v
+        keys << k
+        vectors << v
+      end
+      @variables, @keys, @vectors = ary
+      ary[%i[variables keys vectors].index(var)]
+    end
   end
 end