galaaz 0.4.10 → 0.5.0

data/blogs/nse_dplyr/nse_dplyr.md

@@ -24,30 +24,59 @@ fontsize: 11pt
 
 # Introduction
 
-In this post we will see how to program with _dplyr_ in Galaaz.
+According to Steven Sagaert answer on Quora about "Is programming language R overrated?":
 
-## But first, what is Galaaz??
+> R is a sophisticated language with an unusual (i.e. non-mainstream) set of features. It‘s 
+> an impure functional programming language with sophisticated metaprogramming and 3 
+> different OO systems.
+
+> Just like common lisp you can completely customise how things work via metaprogramming. 
+> The biggest example is the tidyverse: by creating it’s own evaluation system (tidyeval)
+> was able to create a custom syntax for dplyr.
+
+> Mastering R (the language) and its ecosystem is not a matter of weeks or months but 
+> takes years. The rabbit hole goes pretty deep…
+
+Although having a highly configurable language might give extreme power to the programmer,
+it can also be, as stated above, a question of years to master it. Programming with _dplyr_
+for instance, requires learning a set of complex concepts and rules that are not easily 
+accessible for casual users or _unsofisticated_ programmers as many users of R are. Being
+_unsofisticated_ is NOT used here in a negative sense, as R was build for statitians and
+not programmers, that need to solve real problems, often in a short time spam and are not
+concerned about creating complex computer systems.
+
+Unfortunatelly, if this _unsofisticated_ programmer decides to move unto more sofisticated
+coding, the learning curve might become a serious impediment. 
+
+In this post we will see how to program with _dplyr_ in Galaaz and how Ruby can simplify
+the learning curve of mastering _dplyr_ coding.
+
+# But first, what is Galaaz??
 
 Galaaz is a system for tightly coupling Ruby and R.  Ruby is a powerful language, with
-a large community, a very large set of libraries and great for web development.  However,
+a large community, a very large set of libraries and great for web development.  It is also
+easy to learn. However,
 it lacks libraries for data science, statistics, scientific plotting and machine learning.
 On the other hand, R is considered one of the most powerful languages for solving all of the
 above problems.  Maybe the strongest competitor to R is Python with libraries such as NumPy,
-Pandas, SciPy, SciKit-Learn and many more.
+Pandas, SciPy, SciKit-Learn and many more.  We will not get here in the discussion on R
+versus Python, both are excellent languages with powerful features, benefits and drawbacks.
+Our interest is to bring to yet another excellent language, Ruby, the data science libraries
+that it lacks.
 
 With Galaaz we do not intend to re-implement any of the scientific libraries in R. However, we 
 allow for very tight coupling between the two languages to the point that the Ruby 
 developer does not need to know that there is an R engine running.  Also, from the point of
-view of the R user/developer Galaaz looks a lot like R, with just minor syntactic difference,
-so there is almost no learning courve for the R developer. And as we will see in this
-post, programming with _dplyr_ is easier in Galaaz than in R.
+view of the R user/developer, Galaaz looks a lot like R, with just minor syntactic difference,
+so there is almost no learning curve for the R developer. And as we will see in this
+post that programming with _dplyr_ is easier in Galaaz than in R.
 
-R users are probably quite knowledgeable about _dplyr_, for the Ruby developer, _dplyr_ and
+R users are probably quite knowledgeable about _dplyr_. For the Ruby developer, _dplyr_ and
 the _tidyverse_ libraries are a set of libraries for data manipulation in R, developed by
 Hardley Wickham, chief scientis at RStudio and a prolific R coder and writer.
 
-For the coupling of Ruby and R we use new technologies provided by Oracle: GraalVM, 
-TruffleRuby and FastR:
+For the coupling of Ruby and R, we use new technologies provided by Oracle: GraalVM, 
+TruffleRuby and FastR. GraalVM home page had the following definition:
 
      GraalVM is a universal virtual machine for running applications
      written in JavaScript, Python 3, Ruby, R, JVM-based languages like Java,
@@ -76,22 +105,39 @@ TruffleRuby and FastR:
     With GraalVM we aim to allow developers to freely choose the right language
     for the task at hand without making compromises.
 
-Interested readers should also check out the following sites:
 
-* [GraalVM Home](https://www.graalvm.org/)
-* [TruffleRuby](https://github.com/oracle/truffleruby)
-* [FastR](https://github.com/oracle/fastr)
-* [Faster R with FastR](https://medium.com/graalvm/faster-r-with-fastr-4b8db0e0dceb)
-* [How to make Beautiful Ruby Plots with Galaaz](https://medium.freecodecamp.org/how-to-make-beautiful-ruby-plots-with-galaaz-320848058857)
-* [Ruby Plotting with Galaaz: An example of tightly coupling Ruby and R in GraalVM](https://towardsdatascience.com/ruby-plotting-with-galaaz-an-example-of-tightly-coupling-ruby-and-r-in-graalvm-520b69e21021)
-* [How to do reproducible research in Ruby with gKnit](https://towardsdatascience.com/how-to-do-reproducible-research-in-ruby-with-gknit-c26d2684d64e)
-* [R for Data Science](https://r4ds.had.co.nz/)
-* [Advanced R](https://adv-r.hadley.nz/)
+# Tidyverse and dplyr
+
+In [What is the tidyverse?](https://rviews.rstudio.com/2017/06/08/what-is-the-tidyverse/) the 
+tidyverse is explained as follows:
+
+> The tidyverse is a coherent system of packages for data manipulation, exploration and 
+> visualization that share a common design philosophy. These were mostly developed by 
+> Hadley Wickham himself, but they are now being expanded by several contributors. Tidyverse 
+> packages are intended to make statisticians and data scientists more productive by 
+> guiding them through workflows that facilitate communication, and result in reproducible 
+> work products. Fundamentally, the tidyverse is about the connections between the tools 
+> that make the workflow possible.
+
+_dplyr_ is one of the many packages that are part of the tidyverse.  It is:
+
+> a grammar of data manipulation, providing a consistent set of verbs that help you solve 
+> the most common data manipulation challenges:
 
-## Programming with dplyr
+> 1. mutate() adds new variables that are functions of existing variables
+> 2. select() picks variables based on their names.
+> 3. filter() picks cases based on their values.
+> 4. summarise() reduces multiple values down to a single summary.
+> 5. arrange() changes the ordering of the rows.
 
-This post will follow closely the work done in https://dplyr.tidyverse.org/articles/programming.html,
-by Hardley Wickham. In it, Hardley states:
+Very often R is used interactively and users use _dplyr_ to manipulate a single dataset
+without programming. When users want to replicate their work for
+multiple datasets, programming becomes necessary.
+
+# Programming with dplyr
+
+In the vignette ["Programming with dplyr"](https://dplyr.tidyverse.org/articles/programming.html), 
+Hardley Wickham states:
 
 > Most dplyr functions use non-standard evaluation (NSE). This is a catch-all term that 
 > means they don’t follow the usual R rules of evaluation. Instead, they capture the 
@@ -106,6 +152,8 @@ by Hardley Wickham. In it, Hardley states:
 > database backends because dplyr itself doesn’t do any work, but instead generates the SQL 
 > that tells the database what to do.
 
+But then he goes on:
+
 > Unfortunately these benefits do not come for free. There are two main drawbacks:
 
 > Most dplyr arguments are not referentially transparent. That means you can’t replace a value 
@@ -131,8 +179,27 @@ filter(df, my_var == 1)
 ```
 > This makes it hard to create functions with arguments that change how dplyr verbs are computed.
 
-In this post we will see that programming with _dplyr_ in Galaaz does not require knowledge of
-non-standard evaluation in R and can be accomplished by utilizing normal Ruby constructs.
+As a result of this, programming with _dplyr_ requires learning a set of new ideas and concepts.
+In this vignette Hardley goes on showing how to program ever more difficult problems with _dplyr_, 
+showing the problems it faces and the new concepts needed to solve them.
+
+In this blog, we will look at all the problems presented by Harley on the vignette and show how
+those same problems can be solved using Galaaz and the Ruby language. 
+
+This blog is organized as follows: first we show how to write expressions using Galaaz.  
+Expressions are a fundamental concept in _dplyr_ and are not part of basic Ruby.  We extend
+the Ruby language create a manipulate expressions that will be used by _dplyr_ functions. 
+
+Then we show very succintly how Ruby and R can be integrated and how R functions are 
+transparently called from Ruby. Galaaz [user manual](https://github.com/rbotafogo/galaaz/wiki)
+(still in development) goes in much deeper detail about this integration.
+
+Next in section "Data manipulation wiht _dplyr_" we go through all the problems on the 
+_dplyr_ vignette and look at how they are solved in Galaaz.  We then discuss why programming
+with Galaaz and _dplyr_ is easier than programming with _dplyr_ in plain R.
+
+The following section looks at another more advanced problem and shows that Galaaz can still
+handle it without any difficulty.  We then provide further reading and concluding remarks.
 
 # Writing Expressions in Galaaz
 
@@ -141,12 +208,24 @@ Galaaz extends Ruby to work with expressions, similar to R's expressions build w
 formulae.  For instance, in mathematics, the expression $y = sin(x)$ describes a function but cannot
 be computed unless the value of $x$ is bound to some value. 
 
-Let's take a look at some of those expressions in Ruby:
+Expressions are fundamental in _dplyr_ programming as they are the input to _dplyr_ functions,
+for instance, as we will see shortly, if a data frame has a column named 'x' and we want 
+to add another column, y, to this dataframe that has the values of 'x' times 2, then we would
+call a _dplyr_ function with the expression 'y = x * 2'.
+
+## A note on notation
+
+This blog was written in Rmarkdown and automatically converted to HTML or PDF (depending on
+where you are reading this blog) with gKnit (a tool provided by Galaaz). In Rmarkdown, it is
+possible to write text and code blocks that are executed to generate the final report. Code
+blocks appear inside a 'box' and the result of their execution appear either in another type 
+of 'box' with a different background (HTML) or as normal text (PDF).  Every output line from
+the code execution is preceeded by '##'.
 
 ## Expressions from operators
 
-The code bellow creates an expression summing two symbols. Note that :a and :b are Ruby symbols and
-are not bound to any value at the time of expression definition:
+The code below creates an expression summing two symbols. Note that :a and :b are Ruby symbols and
+are not bound to any values at the time of expression definition:
 
 
 ```ruby
@@ -157,7 +236,7 @@ puts exp1
 ```
 ## a + b
 ```
-We can build any complex mathematical expression such as:
+In Galaaz, we can build any complex mathematical expression such as:
 
 
 ```ruby
@@ -168,7 +247,13 @@ puts exp2
 ```
 ## (a + b) * 2 + c^2L/z
 ```
-The 'L' after two indicates that 2 is an integer.
+Expressions are printed with the same format as the equivalent R expressions. The 'L' after 
+2 indicates that 2 is an integer. 
+
+The R developer should note that in R, if she writes the
+number '2', the R interpreter will convert it to float. In order to get an interger she
+should write '2L'. Galaaz follows Ruby notation and '2' is an integer, while '2.0' is a
+float.
 
 It is also possible to use inequality operators in building expressions:
 
@@ -186,13 +271,13 @@ Expressions' definition can also make use of normal Ruby variables without any p
 
 ```ruby
 x = 20
-y = 30
+y = 30.0
 exp_var = (:a + :b) * x <= :z - y
 puts exp_var
 ```
 
 ```
-## (a + b) * 20L <= z - 30L
+## (a + b) * 20L <= z - 30
 ```
 
 Galaaz provides both symbolic representations for operators, such as (>, <, !=) as functional 
@@ -209,10 +294,12 @@ puts exp4
 ## a + b >= z
 ```
 
-Two type of expression, however, can only be created with the functional representation 
-of the operators,  those are expressions involving '==', and '='.  In order to write an 
-expression involving '==' we
-need to use the method '.eq' and for '=' we need the function '.assign'
+Two types of expressions, however, can only be created with the functional representation 
+of the operators.  Those are expressions involving '==', and '='. This is the case since
+those symbols have special meaning in Ruby and should not be redefined.
+
+In order to write an expression involving '==' we
+need to use the method '.eq' and for '=' we need the function '.assign':
 
 
 ```ruby
@@ -233,8 +320,9 @@ puts exp6
 ```
 ## y <- a + b
 ```
-In general we think that using the functional notation is preferable to using the 
-symbolic notation as otherwise, we end up writing invalid expressions such as
+Users should be careful when writing expressions not to inadvertently use '==' or '=' as 
+this will generate an error, that might be a bit cryptic (in future releases of Galaza, we 
+plan to improve the error message).
 
 
 ```ruby
@@ -248,17 +336,16 @@ puts exp_wrong
 ##   object 'a' not found (RError)
 ## Translated to internal error
 ```
-and it might be difficult to understand what is going on here.  The problem lies with the fact that
-when using '==' we are comparing expression (:a + :b) to expression :z with '=='.  When the 
-comparison is executed, the system tries to evaluate :a, :b and :z, and those symbols at 
-this time are not bound to anything and we get a "object 'a' not found" message.
-If we only use functional notation, this type of error will not occur.
+The problem lies with the fact that
+when using '==' we are comparing expression (:a + :b) to expression :z with '=='.  When this 
+comparison is executed, the system tries to evaluate :a, :b and :z, and those symbols, at 
+this time, are not bound to anything giving the "object 'a' not found" message.
 
 ## Expressions with R methods
 
 It is often necessary to create an expression that uses a method or function.  For instance, in
 mathematics, it's quite natural to write an expressin such as $y = sin(x)$. In this case, the
-'sin' function is part of the expression and should not immediately be executed. When we want 
+'sin' function is part of the expression and should not be immediately executed. When we want 
 the function to be part of the expression, we call the function preceeding it
 by the letter E, such as 'E.sin(x)'
 
@@ -271,8 +358,7 @@ puts exp7
 ```
 ## y <- sin(x)
 ```
-
-Expressions can also be written using '.' notation:
+Function expressions can also be written using '.' notation:
 
 
 ```ruby
@@ -283,8 +369,9 @@ puts exp8
 ```
 ## y <- sin(x)
 ```
-
-When a function has multiple arguments, the first one can be used before the '.':
+When a function has multiple arguments, the first one can be used before the '.'.  For instance,
+the R concatenate function 'c', that concatenates two or more arguments can be part of 
+an expression as:
 
 
 ```ruby
@@ -295,11 +382,16 @@ puts exp9
 ```
 ## c(x, y)
 ```
+Note that this gives an OO feeling to the code, as if we were saying 'x' concatenates 'y'. As a
+side note, '.' notation can be used as the R pipe operator '%>%', but is more general than the
+pipe.
 
 ## Evaluating an Expression
 
-Expressions can be evaluated by calling function 'eval' with a binding. A binding can be provided
-with a list:
+Although we are mainly focusing on expressions to pass them to _dplyr_ functions, expressions 
+can be evaluated by calling function 'eval' with a binding. 
+
+A binding can be provided with a list or a data frame as shown below:
 
 
 ```ruby
@@ -311,7 +403,7 @@ puts exp.eval(R.list(a: 10, b: 20, c: 30, z: 40))
 ## [1] 82.5
 ```
 
-... with a data frame:
+with a data frame:
 
 
 ```ruby
@@ -336,8 +428,8 @@ this post, we do not have enough space to write a complete manual on Galaaz
 (a short manual can be found at: https://www.rubydoc.info/gems/galaaz/0.4.9), so we will 
 present only a few examples scripts using Galaaz.
 
-Basically, to call an R function from Ruby with Galaaz, one only needs to preceed the function
-with 'R.'.  For instance, to create a vector in R, the 'c' function is used.  From Galaaz, a
+Basically, to call an R function from Ruby with Galaaz, one only needs to preced the function
+with 'R.'.  For instance, to create a vector in R, the 'c' function is used. In Galaaz, a
 vector can be created by using 'R.c':
 
 
@@ -399,11 +491,103 @@ puts exp10
 ## y <- 0.850903524534118
 ```
 
-# Filtering using expressions
+# Data manipulation wiht _dplyr_
+
+In this section we will give a brief tour _dplyr_'s usage in Galaaz and how to manipulate 
+data in Ruby with it.  This section will follow [_dplyr_'s vignette](https://dplyr.tidyverse.org/articles/dplyr.html) that explores the nycflights13 data set. This dataset contains all 336776 
+flights that departed from New York City in 2013. The data comes from the US Bureau of 
+Transportation Statistics.
+
+Let's start by taking a look at this dataset:
+
+
+```ruby
+R.library('nycflights13')
+# check it's dimension
+puts ~:flights.dim
+# and the structure
+~:flights.str
+```
+
+```
+## Message:
+##  Method ~ not found in R environment
+```
+
+```
+## Message:
+##  /home/rbotafogo/desenv/galaaz/lib/R_interface/rsupport.rb:90:in `eval'
+## /home/rbotafogo/desenv/galaaz/lib/R_interface/rsupport.rb:270:in `exec_function_name'
+## /home/rbotafogo/desenv/galaaz/lib/R_interface/robject.rb:166:in `method_missing'
+## /home/rbotafogo/desenv/galaaz/lib/util/exec_ruby.rb:105:in `get_binding'
+## /home/rbotafogo/desenv/galaaz/lib/util/exec_ruby.rb:102:in `eval'
+## /home/rbotafogo/desenv/galaaz/lib/util/exec_ruby.rb:102:in `exec_ruby'
+## /home/rbotafogo/desenv/galaaz/lib/gknit/knitr_engine.rb:650:in `block in initialize'
+## /home/rbotafogo/desenv/galaaz/lib/R_interface/ruby_callback.rb:77:in `call'
+## /home/rbotafogo/desenv/galaaz/lib/R_interface/ruby_callback.rb:77:in `callback'
+## (eval):3:in `function(...) {\n          rb_method(...)'
+## unknown.r:1:in `in_dir'
+## unknown.r:1:in `block_exec'
+## /usr/local/lib/graalvm-ce-java11-20.0.0/languages/R/library/knitr/R/block.R:92:in `call_block'
+## /usr/local/lib/graalvm-ce-java11-20.0.0/languages/R/library/knitr/R/block.R:6:in `process_group.block'
+## /usr/local/lib/graalvm-ce-java11-20.0.0/languages/R/library/knitr/R/block.R:3:in `<no source>'
+## unknown.r:1:in `withCallingHandlers'
+## unknown.r:1:in `process_file'
+## unknown.r:1:in `<no source>'
+## unknown.r:1:in `<no source>'
+## <REPL>:4:in `<repl wrapper>'
+## <REPL>:1
+```
+
+Now, let's use a first verb of _dplyr_: 'filter'. This verb, obviously, will filter the data
+by the given expression. In the next block, we filter by columns 'month' and 'day'. The
+first argument to the filter function is symbol ':flights'. A Ruby symbol, when given to
+an R function will convert to the R variable of the same name, in this case 'flights', that
+holds the nycflights13 data frame. 
+
+The second and third arguments are expressions that will be used by the filter function to
+filter by columns, looking for entries in which the month and day are equal to 1.
+
+
+```ruby
+puts R.filter(:flights, (:month.eq 1), (:day.eq 1))
+```
+
+```
+## # A tibble: 842 x 19
+##     year month   day dep_time sched_dep_time dep_delay arr_time
+##    <int> <int> <int>    <int>          <int>     <dbl>    <int>
+##  1  2013     1     1      517            515         2      830
+##  2  2013     1     1      533            529         4      850
+##  3  2013     1     1      542            540         2      923
+##  4  2013     1     1      544            545        -1     1004
+##  5  2013     1     1      554            600        -6      812
+##  6  2013     1     1      554            558        -4      740
+##  7  2013     1     1      555            600        -5      913
+##  8  2013     1     1      557            600        -3      709
+##  9  2013     1     1      557            600        -3      838
+## 10  2013     1     1      558            600        -2      753
+## # … with 832 more rows, and 12 more variables: sched_arr_time <int>,
+## #   arr_delay <dbl>, carrier <chr>, flight <int>, tailnum <chr>,
+## #   origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
+## #   minute <dbl>, time_hour <dttm>
+```
+
+
+## Programming with _dplyr_: problems and how to solve them in Galaaz
+
+In this section we look at the list of problems that Hardley describes in the "Programming with dplyr"
+vignette and show how those problems are solved and coded with Galaaz. Readers interested in
+how those problems are treated in _dplyr_ should read the vignette and use it as a comparison with
+this blog.
 
-Now that we know how to write expression and call R functions let's do some data manipulation in
-Galaaz.  Let's first start by creating the same data frame that we created previously in section
-"Programming with dplyr":
+## Filtering using expressions
+
+Now that we know how to write expressions and call R functions, let's do some data manipulation in
+Galaaz.  Let's first start by creating a data frame. In R, the 'data.frame' function creates a
+data frame.  In Ruby, writing 'data.frame' will not parse as a single object. To call R 
+functions that have a '.' in them, we need to substitute the '.' with '__'. So, method
+'data.frame' in R, is called in Galaaz as 'R.data\_\_frame':
 
 
 ```ruby
@@ -417,9 +601,15 @@ puts df
 ## 2 2 2
 ## 3 3 1
 ```
-The 'filter' function can be called on this data frame either by using 'R.filter(df, ...)' or 
-by using dot notation.  We prefer to use dot notation as shown bellow.  The argument to 'filter'
-in Galaaz should be an expression. Note that if we gave to filter a Ruby expression such as
+
+_dplyr_ provides the 'filter' function, that filters data in a data brame. The 'filter' 
+function can be called on this data frame either by using 'R.filter(df, ...)' or 
+by using dot notation.  
+
+-------FIX---------
+
+We prefer to use dot notation as shown bellow.  The argument to 'filter' should be an 
+expression. Note that if we gave to filter a Ruby expression such as
 'x == 1', we would get an error, since there is no variable 'x' defined and if 'x' was a variable
 then 'x == 1' would either be 'true' or 'false'. Our goal is to filter our data frame returning
 all rows in which the 'x' value is equal to 1. To express this we want: ':x.eq 1', where :x will
@@ -434,7 +624,6 @@ puts df.filter(:x.eq 1)
 ##   x y
 ## 1 1 3
 ```
-
 In R, and when coding with 'tidyverse', arguments to a function are usually not 
 *referencially transparent*. That is, you can’t replace a value with a seemingly equivalent 
 object that you’ve defined elsewhere. In other words, this code
@@ -504,7 +693,7 @@ puts df.filter(:x.eq y)
 ##   x y
 ## 1 1 3
 ```
-# Writing a function that applies to different data sets
+## Writing a function that applies to different data sets
 
 Let's suppose that we want to write a function that receives as the first argument a data frame
 and as second argument an expression that adds a column to the data frame that is equal to the
@@ -571,7 +760,7 @@ mutate_y(df1)
 ##   mismatched protect/unprotect (unprotect with empty protect stack) (RError)
 ## Translated to internal error
 ```
-# Different expressions
+## Different expressions
 
 Let's move to the next problem as presented by Hardley where trying to write a function in R 
 that will receive two argumens, the first a variable and the second an expression is not trivial.
@@ -589,16 +778,16 @@ df <- data.frame(
   b = sample(5)
 )
 
-as.data.frame(df) 
+as.data.frame(df)
 ```
 
 ```
 ##   g1 g2 a b
-## 1  1  1 2 1
-## 2  1  2 4 3
-## 3  2  1 5 4
-## 4  2  2 3 2
-## 5  2  1 1 5
+## 1  1  1 3 3
+## 2  1  2 2 1
+## 3  2  1 5 2
+## 4  2  2 4 5
+## 5  2  1 1 4
 ```
 
 ```r
@@ -606,13 +795,13 @@ d2 <- df %>%
   group_by(g1) %>%
   summarise(a = mean(a))
 	       
-as.data.frame(d2)	       
+as.data.frame(d2)
 ```
 
 ```
-##   g1 a
-## 1  1 3
-## 2  2 3
+##   g1        a
+## 1  1 2.500000
+## 2  2 3.333333
 ```
 
 ```r
@@ -624,9 +813,9 @@ as.data.frame(d2)
 ```
 
 ```
-##   g2        a
-## 1  1 2.666667
-## 2  2 3.500000
+##   g2 a
+## 1  1 3
+## 2  2 3
 ```
 
 As shown by Hardley, one might expect this function to do the trick:
@@ -662,38 +851,42 @@ def my_summarize(df, group_var)
     summarize(a: :a.mean)
 end
 
-puts my_summarize(:df, :g1).as__data__frame
+puts my_summarize(:df, :g1)
 ```
 
 ```
 ##   g1 g2 a b
-## 1  1  1 2 1
-## 2  1  2 4 3
-## 3  2  1 5 4
-## 4  2  2 3 2
-## 5  2  1 1 5
+## 1  1  1 3 3
+## 2  1  2 2 1
+## 3  2  1 5 2
+## 4  2  2 4 5
+## 5  2  1 1 4
 ## 
-##   g1 a
-## 1  1 3
-## 2  2 3
+## # A tibble: 2 x 2
+##      g1     a
+##   <dbl> <dbl>
+## 1     1  2.5 
+## 2     2  3.33
 ```
 It works!!! Well, let's make sure this was not just some coincidence
 
 
 ```ruby
-puts my_summarize(:df, :g2).as__data__frame
+puts my_summarize(:df, :g2)
 ```
 
 ```
-##   g2        a
-## 1  1 2.666667
-## 2  2 3.500000
+## # A tibble: 2 x 2
+##      g2     a
+##   <dbl> <dbl>
+## 1     1     3
+## 2     2     3
 ```
 
 Great, everything is fine! No magic, no new functions, no complexities, just normal, standard Ruby
 code.  If you've ever done NSE in R, this certainly feels much safer and easy to implement.
 
-# Different input variables
+## Different input variables
 
 In the previous section we've managed to get rid of all NSE formulation for a simple example, but
 does this remain true for more complex examples, or will the Galaaz way prove inpractical for
@@ -743,7 +936,7 @@ puts my_summarise2((~:df), :a * :b)
 Once again, there is no need to use any special theory or functions.  The only point to be 
 careful about is the use of 'E' to build expressions from functions 'mean', 'sum' and 'n'.
 
-# Different input and output variable
+## Different input and output variable
 
 Now the next challenge presented by Hardley is to vary the name of the output variables based on 
 the received expression.  So, if the input expression is 'a', we want our data frame columns to
@@ -792,17 +985,17 @@ puts my_mutate((~:df), :b)
 
 ```
 ##   g1 g2 a b mean_a sum_a
-## 1  1  1 2 1      3    15
-## 2  1  2 4 3      3    15
-## 3  2  1 5 4      3    15
-## 4  2  2 3 2      3    15
-## 5  2  1 1 5      3    15
+## 1  1  1 3 3      3    15
+## 2  1  2 2 1      3    15
+## 3  2  1 5 2      3    15
+## 4  2  2 4 5      3    15
+## 5  2  1 1 4      3    15
 ##   g1 g2 a b mean_b sum_b
-## 1  1  1 2 1      3    15
-## 2  1  2 4 3      3    15
-## 3  2  1 5 4      3    15
-## 4  2  2 3 2      3    15
-## 5  2  1 1 5      3    15
+## 1  1  1 3 3      3    15
+## 2  1  2 2 1      3    15
+## 3  2  1 5 2      3    15
+## 4  2  2 4 5      3    15
+## 5  2  1 1 4      3    15
 ```
 It really seems that "Non Standard Evaluation" is actually quite standard in Galaaz! But, you 
 might have noticed a small change in the way the arguments to the mutate method were called.
@@ -812,7 +1005,7 @@ and variable mean\_name is not followed by ':' but by '=>'.  This is standard Ru
 
 [explain....]
 
-# Capturing multiple variables
+## Capturing multiple variables
 
 Moving on with new complexities, Hardley proposes us to solve the problem in which the 
 summarise function will receive any number of grouping variables.
@@ -827,15 +1020,18 @@ def my_summarise3(df, *group_vars)
     summarise(a: E.mean(:a))
 end
 
-puts my_summarise3((~:df), :g1, :g2).as__data__frame
+puts my_summarise3((~:df), :g1, :g2)
 ```
 
 ```
-##   g1 g2 a
-## 1  1  1 2
-## 2  1  2 4
-## 3  2  1 3
-## 4  2  2 3
+## # A tibble: 4 x 3
+## # Groups:   g1 [?]
+##      g1    g2     a
+##   <dbl> <dbl> <dbl>
+## 1     1     1     3
+## 2     1     2     2
+## 3     2     1     3
+## 4     2     2     4
 ```
 
 # Why does R require NSE and Galaaz does not?
@@ -868,7 +1064,7 @@ call the function passing the expression ':a.eq :b'.
 
 # Advanced dplyr features
 
-In the blog: Programming with dplyr by using dplyr (https://www.r-bloggers.com/programming-with-dplyr-by-using-dplyr/) Iñaki Úcar shows surprise that some R users are trying to code in dplyr avoiding
+In the blog: [Programming with dplyr by using dplyr](https://www.r-bloggers.com/programming-with-dplyr-by-using-dplyr/) Iñaki Úcar shows surprise that some R users are trying to code in dplyr avoiding
 the use of NSE.  For instance he says:
 
 > Take the example of seplyr. It stands for standard evaluation dplyr, and enables us to 
@@ -887,38 +1083,21 @@ features of characters in the Starwars movies:
 
 
 ```ruby
-puts (~:starwars).head.as__data__frame
-```
-
-```
-##             name height mass  hair_color  skin_color eye_color birth_year
-## 1 Luke Skywalker    172   77       blond        fair      blue       19.0
-## 2          C-3PO    167   75        <NA>        gold    yellow      112.0
-## 3          R2-D2     96   32        <NA> white, blue       red       33.0
-## 4    Darth Vader    202  136        none       white    yellow       41.9
-## 5    Leia Organa    150   49       brown       light     brown       19.0
-## 6      Owen Lars    178  120 brown, grey       light      blue       52.0
-##   gender homeworld species
-## 1   male  Tatooine   Human
-## 2   <NA>  Tatooine   Droid
-## 3   <NA>     Naboo   Droid
-## 4   male  Tatooine   Human
-## 5 female  Alderaan   Human
-## 6   male  Tatooine   Human
-##                                                                                                                                       films
-## 1                                           Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
-## 2                    Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope
-## 3 Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
-## 4                                                              Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope
-## 5                                           Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
-## 6                                                                                     Attack of the Clones, Revenge of the Sith, A New Hope
-##                             vehicles                starships
-## 1 Snowspeeder, Imperial Speeder Bike X-wing, Imperial shuttle
-## 2                                                            
-## 3                                                            
-## 4                                             TIE Advanced x1
-## 5              Imperial Speeder Bike                         
-## 6
+puts (~:starwars).head
+```
+
+```
+## # A tibble: 6 x 13
+##   name  height  mass hair_color skin_color eye_color birth_year gender
+##   <chr>  <int> <dbl> <chr>      <chr>      <chr>          <dbl> <chr> 
+## 1 Luke…    172    77 blond      fair       blue            19   male  
+## 2 C-3PO    167    75 <NA>       gold       yellow         112   <NA>  
+## 3 R2-D2     96    32 <NA>       white, bl… red             33   <NA>  
+## 4 Dart…    202   136 none       white      yellow          41.9 male  
+## 5 Leia…    150    49 brown      light      brown           19   female
+## 6 Owen…    178   120 brown, gr… light      blue            52   male  
+## # … with 5 more variables: homeworld <chr>, species <chr>, films <list>,
+## #   vehicles <list>, starships <list>
 ```
 The grouped_mean function bellow will receive a grouping variable and calculate summaries for
 the value\_variables given:
@@ -970,28 +1149,44 @@ def grouped_mean(data, grouping_variables, value_variables)
     rename_at(value_variables, E.funs(E.paste0("mean_", value_variables)))
 end
 
-puts grouped_mean((~:starwars), "eye_color", E.c("mass", "birth_year")).as__data__frame
+puts grouped_mean((~:starwars), "eye_color", E.c("mass", "birth_year"))
 ```
 
 ```
-##        eye_color mean_mass mean_birth_year count
-## 1          black  76.28571        33.00000    10
-## 2           blue  86.51667        67.06923    19
-## 3      blue-gray  77.00000        57.00000     1
-## 4          brown  66.09231       108.96429    21
-## 5           dark       NaN             NaN     1
-## 6           gold       NaN             NaN     1
-## 7  green, yellow 159.00000             NaN     1
-## 8          hazel  66.00000        34.50000     3
-## 9         orange 282.33333       231.00000     8
-## 10          pink       NaN             NaN     1
-## 11           red  81.40000        33.66667     5
-## 12     red, blue       NaN             NaN     1
-## 13       unknown  31.50000             NaN     3
-## 14         white  48.00000             NaN     1
-## 15        yellow  81.11111        76.38000    11
+## # A tibble: 15 x 4
+##    eye_color     mean_mass mean_birth_year count
+##    <chr>             <dbl>           <dbl> <dbl>
+##  1 black              76.3            33      10
+##  2 blue               86.5            67.1    19
+##  3 blue-gray          77              57       1
+##  4 brown              66.1           109.     21
+##  5 dark              NaN             NaN       1
+##  6 gold              NaN             NaN       1
+##  7 green, yellow     159             NaN       1
+##  8 hazel              66              34.5     3
+##  9 orange            282.            231       8
+## 10 pink              NaN             NaN       1
+## 11 red                81.4            33.7     5
+## 12 red, blue         NaN             NaN       1
+## 13 unknown            31.5           NaN       3
+## 14 white              48             NaN       1
+## 15 yellow             81.1            76.4    11
 ```
 
+# Further reading
+
+For more information on GraalVM, TruffleRuby, fastR, R and Galaaz check out the following sites/posts:
+
+* [GraalVM Home](https://www.graalvm.org/)
+* [TruffleRuby](https://github.com/oracle/truffleruby)
+* [FastR](https://github.com/oracle/fastr)
+* [Faster R with FastR](https://medium.com/graalvm/faster-r-with-fastr-4b8db0e0dceb)
+* [How to make Beautiful Ruby Plots with Galaaz](https://medium.freecodecamp.org/how-to-make-beautiful-ruby-plots-with-galaaz-320848058857)
+* [Ruby Plotting with Galaaz: An example of tightly coupling Ruby and R in GraalVM](https://towardsdatascience.com/ruby-plotting-with-galaaz-an-example-of-tightly-coupling-ruby-and-r-in-graalvm-520b69e21021)
+* [How to do reproducible research in Ruby with gKnit](https://towardsdatascience.com/how-to-do-reproducible-research-in-ruby-with-gknit-c26d2684d64e)
+* [R for Data Science](https://r4ds.had.co.nz/)
+* [Advanced R](https://adv-r.hadley.nz/)
+
 # Conclusion
 
 Ruby and Galaaz provide a nice framework for developing code that uses R functions. Although R is