galaaz 0.4.1 → 0.4.2

data/blogs/galaaz_ggplot/midwest.Rmd

@@ -4,7 +4,7 @@ screen, bellow, we generate an 'svg' image and then include it in this document.
 generate and image, the R.svg device is used.  To generate the plot on the screen, use the R.awt
 device, as commented on the code.
 
-```{galaaz midwest_rb, warning=FALSE}
+```{ruby midwest_rb, warning=FALSE}
 require 'galaaz'
 require 'ggplot'
 

data/blogs/galaaz_ggplot/midwest_external_png

@@ -0,0 +1,13 @@
+![Midwest Plot](https://user-images.githubusercontent.com/3999729/46742999-87bc2480-cc7e-11e8-9f16-31c3437e4a58.PNG){width=70%}
+
+![Midwest Plot with 'glm' function and modified theme](https://user-images.githubusercontent.com/3999729/47120345-a903ae80-d244-11e8-9be3-a0db13cf51ab.PNG)
+
+
+    Sir Galahad (/ˈɡæləhæd/; sometimes referred to as Galeas /ɡəˈliːəs/ or Galath /ˈɡæləθ/),
+    in Arthurian legend, is a knight of King Arthur's Round Table and one of the three
+    achievers of the Holy Grail. He is the illegitimate son of Sir Lancelot and Elaine of
+    Corbenic, and is renowned for his gallantry and purity as the most perfect of all knights.
+    Emerging quite late in the medieval Arthurian tradition, Sir Galahad first appears in the
+    Lancelot–Grail cycle, and his story is taken up in later works such as the Post-Vulgate
+    Cycle and Sir Thomas Malory's Le Morte d'Arthur. His name should not be mistaken with
+    Galehaut, a different knight from Arthurian legend. 

data/blogs/galaaz_ggplot/midwest_external_png~

@@ -0,0 +1 @@
+![Midwest Plot with 'glm' function and modified theme](https://user-images.githubusercontent.com/3999729/47120345-a903ae80-d244-11e8-9be3-a0db13cf51ab.PNG)

data/blogs/gknit/gknit.Rmd

@@ -0,0 +1,500 @@
+---
+title: "gKnit - Ruby and R Knitting with Galaaz in GraalVM"
+author: "Rodrigo Botafogo"
+tags: [Galaaz, Ruby, R, TruffleRuby, FastR, GraalVM, knitr]
+date: "19 October 2018"
+output:
+  html_document:
+    self_contained: true
+    keep_md: true
+  pdf_document:
+    includes:
+      in_header: ["../../sty/galaaz.sty"]
+    number_sections: yes
+---
+
+```{r setup, echo=FALSE}
+
+```
+
+# Introduction
+
+The idea of "literate programming" was first introduced by Donald Knuth in the 1980's.
+The main intention of this approach was to develop software interspersing macro snippets,
+traditional source code, and a natural language such as English that could be compiled into
+executable code and at the same time easily read by a human developer. According to Knuth
+"The practitioner of 
+literate programming can be regarded as an essayist, whose main concern is with exposition 
+and excellence of style."
+
+The idea of literate programming evolved into the idea of reproducible research, in which
+all the data, software code, documentation, graphics etc. needed to reproduce the research
+and its reports could be included in a
+single document or set of documents that when distributed to peers could be rerun generating
+the same output and reports.
+
+The R community has put a great deal of effort in reproducible research.  In 2002, Sweave was
+introduced and it allowed mixing R code with Latex generating high quality PDF documents.  Those
+documents could include the code, the result of executing the code, graphics and text.  This
+contained the whole narrative to reproduce the research.  But Sweave had many problems and in
+2012, Knitr, developed by Yihui Xie from RStudio was released, solving many of the long lasting
+problems from Sweave and including in one single package many extensions and add-on packages that
+were necessary for Sweave.
+
+With Knitr, R markdown was also developed, an extension the the
+Markdown format.  With R markdown and Knitr it is possible to generate reports in a multitude
+of formats such as HTML, markdown, Latex, PDF, dvi, etc.  R markdown also allows the use of
+multiple programming languages in the same document.  In R markdown text is interspersed with
+code chunks that can be executed and both the code as the result of executing the code can become
+part of the final report.  Although R markdown allows multiple programming languages in the
+same document, only R and Python (with
+the reticulate package) can persist variables between chunks.  For other languages, such as
+Ruby, every chunk will start a new process and thus all data is lost between chunks, unless it
+is somehow stored in a data file that is read by the next chunk.
+
+Being able to persist data
+between chunks is critical for literate programming otherwise the flow of the narrative is lost
+by all the effort of having to save data and then reload it. Probably, because of this impossibility,
+it is very rare to see any R markdown document document in the Ruby community.  
+
+In the Python community, the same effort to have code and text in an integrated environment
+started also on the first decade of 2000. In 2006 iPython 0.7.2 was released.  In 2014,
+Fernando Pérez, spun off project Jupyter from iPython creating a web-based interactive
+computation environment.  Jupyter can now be used with many languages, including Ruby with the
+iruby gem (https://github.com/SciRuby/iruby).  I am not sure if multiple languages can be used
+in a Jupyter notebook.
+
+# gKnitting a Document
+
+This document describes gKnit.  gKnit uses Knitr and R markdown to knit a document in Ruby or R
+and output it in any of the
+available formats for R markdown.  The only difference between gKnit and normal Knitr documents
+is that gKnit runs atop of GraalVM, and Galaaz (an integration library between Ruby and R).
+Another blog post on Galaaz and its integration with ggplot2 can be found at:
+https://towardsdatascience.com/ruby-plotting-with-galaaz-an-example-of-tightly-coupling-ruby-and-r-in-graalvm-520b69e21021.  With Galaaz, gKnit can knit documents in Ruby and R and both
+Ruby and R execute on the same process and memory, variables, classes, etc.
+will be preserved between chunks of code.
+
+This is not a blog post on rmarkdown, and the interested user is directed to
+
+* https://rmarkdown.rstudio.com/ or
+* https://bookdown.org/yihui/rmarkdown/ for detailed information on its capabilities and use.
+
+Here, we will describe quickly the main aspects of R markdown, so the user can start gKnitting
+Ruby and R documents quickly.
+
+## The Yaml header
+
+An R markdown document should start with a Yaml header and be stored in a file with '.Rmd' extension.
+This document has the following header for gKitting an HTML document.
+
+```
+---
+title: "gKnit - Ruby and R Knitting with Galaaz in GraalVM"
+author: "Rodrigo Botafogo"
+tags: [Galaaz, Ruby, R, TruffleRuby, FastR, GraalVM, knitr, gknit]
+date: "29 October 2018"
+output:
+  html_document:
+    keep_md: true
+---
+```
+
+For more information on the options in the Yaml header, check https://bookdown.org/yihui/rmarkdown/html-document.html.
+
+## R Markdown formatting
+
+Document formatting can be done with simple markups such as:
+
+### Headers
+
+```
+# Header 1
+
+## Header 2
+
+### Header 3
+
+```
+
+### Lists
+
+```
+Unordered lists:
+
+* Item 1
+* Item 2
+    + Item 2a
+    + Item 2b
+```
+
+```
+Ordered Lists
+
+1. Item 1
+2. Item 2
+3. Item 3
+    + Item 3a
+    + Item 3b
+```
+
+Please, go to https://rmarkdown.rstudio.com/authoring_basics.html, for more R markdown formatting.
+
+## Code Chunks
+
+Running and executing Ruby and R code is actually what really interests us is this blog.  Inserting
+a code chunk is done by adding code in a block delimited by three back ticks followed by a
+block with the engine name (r, ruby, rb, include, others), an optional chunk_label and optional
+options, as shown bellow:
+
+````
+```{engine_name [chunk_label], [chunk_options]}`r ''`
+```
+````
+
+for instance, let's add an R chunk to the document labeled 'first_r_chunk'.  In this case, the
+code should not be shown in the document, so the option 'echo=FALSE' was added.
+
+````
+```{r first_r_chunk, echo = FALSE}`r ''`
+```
+````
+
+A description of the available chunk options can be found in the documentation cited above.
+
+For including a Ruby chunk, just change the name of the engine to ruby as follows:
+
+````
+```{ruby first_ruby_chunk}`r ''`
+```
+````
+
+In this example, the ruby chunk is called 'first_ruby_chunk'.  One important aspect of chunk
+labels is that they cannot be duplicate.  If a chunk label is duplicate, the knitting will
+stop with an error.
+
+### R chunks
+
+Let's now add an R chunk to this document.  In this example, a vector 'r_vec' is created and
+a new function 'redef_sum' is defined.  The chunk specification is
+
+````
+```{r data_creation}`r ''`
+r_vec <- c(1, 2, 3, 4, 5)
+
+redef_sum <- function(...) {
+  Reduce(sum, as.list(...))
+}
+```
+````
+
+and this is how it will look like once executed.  From now on, we will not show the chunk
+definition any longer.
+
+
+```{r data_creation}
+r_vec <- c(1, 2, 3, 4, 5)
+
+redef_sum <- function(...) {
+  Reduce(sum, as.list(...))
+}
+```
+
+We can, possibly in another chunk, access the vector and call the function as follows:
+
+```{r using_previous}
+print(r_vec)
+print(redef_sum(r_vec))
+```
+
+```{r bubble}
+# load package and data
+library(ggplot2)
+data(mpg, package="ggplot2")
+# mpg <- read.csv("http://goo.gl/uEeRGu")
+
+mpg_select <- mpg[mpg$manufacturer %in% c("audi", "ford", "honda", "hyundai"), ]
+
+# Scatterplot
+theme_set(theme_bw())  # pre-set the bw theme.
+g <- ggplot(mpg_select, aes(displ, cty)) + 
+  labs(subtitle="mpg: Displacement vs City Mileage",
+       title="Bubble chart")
+
+g + geom_jitter(aes(col=manufacturer, size=hwy)) + 
+  geom_smooth(aes(col=manufacturer), method="lm", se=F)
+```
+
+### Ruby chunks
+
+In the same way that an R chunk was created, let's now create a Ruby chunk.  One important aspect
+of Ruby is that in Ruby every evaluation of a chunk occurs on its own local scope, so, creating
+a variable in a chunk will be out of scope in the next chunk.  To make sure that variables are
+available between chunks, they should be made global.
+
+In this chunk, variable '\$a', '\$b' and '\$c' are standard Ruby variables and '\$vec' and '\$vec2'
+are two vectors created by a call to FastR.  It should be clear that there is no requirement
+in gknit to call or use R functions.  gKnit will knit standard Ruby code, or even general
+text without code.
+
+```{ruby split_data}
+$a = [1, 2, 3]
+$b = "US$ 250.000"
+$c = "Inline text in a Heading"
+
+$vec = R.c(1, 2, 3)
+$vec2 = R.c(10, 20, 30)
+```
+
+In this next block, variables '\$a', '\$vec' and '\$vec2' are used and printed.
+
+```{ruby split2}
+puts $a
+puts $vec * $vec2
+```
+
+### Accessing R from Ruby
+
+One of the nice aspects of Galaaz on GraalVM, is that variables and functions defined in R, can
+be easily accessed from Ruby.  This next chunk, reads data from R and uses the 'redef_fun'
+function defined previously.  To access an R variable from Ruby the '~' function should be
+applied to the Ruby symbol representing the R variable.  Since the R variable is called 'r_vec',
+in Ruby, the symbol to acess it is ':r_vec' and thus '~:r_vec' retrieves the value of the
+variable.
+
+```{ruby access_r}
+puts ~:r_vec
+```
+
+In order to call an R function, the 'R.' module is used as follows
+
+```{ruby call_r_func}
+puts R.redef_sum($vec)
+```
+
+### Inline Ruby code
+
+Knitr allows inserting R inline by adding
+```{rb puts "&#96;r code&#96;"}
+```
+. Unfortunately, this is not possible with Ruby code as there is no provision in knitr for
+adding this kind of inline engine.  However, gKnit allows adding inline Ruby code with the
+'rb' engine.  The following text will create and inline Ruby text:
+
+````
+This is some text with inline Ruby accessing variable \$b which has value:
+```{rb puts "```{rb puts $b}\n```"}
+```
+and is followed by some other text!
+````
+
+The result of executing the above chunk is the following sentence with inline Ruby code
+
+<div style="margin-bottom:50px;">
+</div>
+
+This is some text with inline Ruby accessing variable \$b which has value:
+```{rb puts $b}
+```
+and is followed by some other text!
+
+<div style="margin-bottom:50px;">
+</div>
+
+In an inline block, it is possible to execute multiple Ruby statements by adding a semicolon
+between them:
+
+````
+Multiple statements in the 'rb' engine use semicolon:
+```{rb puts "```{rb puts $a, puts $b}\n```"}
+```
+````
+
+<div style="margin-bottom:50px;">
+</div>
+
+
+Multiple statements in the 'rb' engine use semicolon:
+```{rb puts $a; puts $b}
+```
+
+<div style="margin-bottom:50px;">
+</div>
+
+
+```{rb puts "### #{$c}"}
+```
+
+Sometimes one wants to add an inline text in a heading.  To do that in Ruby the whole heading
+needs to be returned by the inline Ruby engine.  For example the heading above, was created by
+the following chunk:
+
+````
+```{rb puts %q(```{rb puts "### #{$c}"}\n```)}
+```
+````
+
+Remember that variable '$\c' was defined in a previous Ruby chunk and is now being used to
+create the section heading for this section.
+
+
+### Plotting
+
+```{ruby diverging_bar}
+require 'ggplot'
+
+R.theme_set R.theme_bw
+
+# Data Prep
+mtcars = ~:mtcars
+mtcars.car_name = R.rownames(:mtcars)
+# compute normalized mpg 
+mtcars.mpg_z = ((mtcars.mpg - mtcars.mpg.mean)/mtcars.mpg.sd).round 2
+mtcars.mpg_type = mtcars.mpg_z < 0 ? "below" : "above"
+mtcars = mtcars[mtcars.mpg_z.order, :all]
+# convert to factor to retain sorted order in plot
+mtcars.car_name = mtcars.car_name.factor levels: mtcars.car_name
+
+# Diverging Barcharts
+# R.png
+gg = mtcars.ggplot(E.aes(x: :car_name, y: :mpg_z, label: :mpg_z)) + 
+     R.geom_bar(E.aes(fill: :mpg_type), stat: 'identity',  width: 0.5) +
+     R.scale_fill_manual(name: "Mileage", 
+                         labels: R.c("Above Average", "Below Average"), 
+                         values: R.c("above": "#00ba38", "below": "#f8766d")) + 
+     R.labs(subtitle: "Normalised mileage from 'mtcars'", 
+            title: "Diverging Bars") + 
+     R.coord_flip()
+print gg
+# R.dev__off
+# R.include_graphics("Rplot001.png")
+```
+
+### Including Ruby files
+
+R is a language that was created to be easy and fast for statisticians to use.  It was not a
+language to be used for developing large systems.  Of course, there are large systems and
+libraries in R, but the focus of the language is for developing statistical models and
+distribute that to peers.
+
+Ruby on the other hand, is a language for large software development.  Systems written in
+Ruby will have dozens or hundreds of files.  In order to document a large system with
+literate programming we cannot expect the developer to add all the files in a single '.Rmd'
+file.  gKnit provides the 'include' chunk engine to include a Ruby file as if it had being
+typed in the '.Rmd' file.
+
+To include a file the following chunk should be created, where <filename> is the name of
+the file to be include and where the extension, if it is '.rb', does not need to be added.
+If the 'relative' option is not included, then it is treated as TRUE.  When 'relative' is
+true, 'require_relative' semantics is used to load the file, when false, Ruby's $LOAD_PATH
+is searched to find the file and it is 'require'd.
+
+````
+```{include <filename>, relative = <TRUE/FALSE>}`r ''`
+```
+````
+
+Here we include file 'model.rb' which is in the same directory of this blog.  This code
+uses R 'caret' package to split a dataset in a train and test sets.
+
+````
+```{include model}`r ''`
+```
+````
+
+```{include model}
+```
+
+```{ruby model_partition}
+mtcars = ~:mtcars
+model = Model.new(mtcars, percent_train: 0.8)
+model.partition(:mpg)
+puts model.train.head
+puts model.test.head
+```
+
+### Documenting Gems
+
+gKnit also allows developers to document and load files that are not in the same directory
+of the '.Rmd' file.  When using 'relative = FALSE' in a chunk header, gKnit will look for the
+file in Ruby's \$LOAD_PATH and load it if found.
+
+Here is an example of loading the 'continuation.rb' file from TruffleRuby.
+
+````
+```{include continuation, relative = FALSE}`r ''`
+```
+````
+
+
+```{include continuation, relative = FALSE}
+```
+
+## Converting to PDF
+
+One of the beauties of knitr is that the same input can be converted to many different outputs.
+One very useful format, is, of course, PDF.  In order to converted an R markdown file to PDF
+it is necessary to have LaTeX installed on the system.  We will not explain here how to
+install LaTeX as there are plenty of documents on the web showing how to proceed.
+
+gKnit comes with a simple LaTeX style file for gknitting this blog as a PDF document.  Here is
+the Yaml header to generate this blog in PDF format instead of HTML:
+
+```
+---
+title: "gKnit - Ruby and R Knitting with Galaaz in GraalVM"
+author: "Rodrigo Botafogo"
+tags: [Galaaz, Ruby, R, TruffleRuby, FastR, GraalVM, knitr, gknit]
+date: "29 October 2018"
+output:
+  pdf_document:
+    includes:
+      in_header: ["../../sty/galaaz.sty"]
+    number_sections: yes
+---
+```
+
+# Conclusion
+
+One of the promises of GraalVM is that users/developers will be able to use the best tool
+for their task at hand, independently of the programming language the tool was written.  Galaaz
+and gKnit are not trivial implementations atop the GraalVM and Truffle interop messages;
+however, the time and effort it took to wrap Ruby over R - Galaaz - (not finished yet) or to
+wrap Knitr with gKnit is a fraction of a fraction of a fraction of the time require to
+implement the original tools.  Trying to reimplement all R packages in Ruby would require the
+same effort it is taking Python to implement NumPy, Panda and all supporting libraries and it
+is unlikely that this effort would ever be done.  GraalVM has allowed Ruby to profit "almost
+for free" from this huge set of libraries and tools that make R one of the most used
+languages for data analysis and machine learning.
+
+More interesting though than being able to wrap the R libraries with Ruby, is that Ruby adds
+value to R, by allowing developers to use powerful and modern constructs for code reuse that
+are not the strong points of R.  As shown in this blog, R and Ruby can easily communicate
+and R can be structured in classes and modules in a way that greatly expands its power and
+readability.
+
+# Installing gKnit
+
+## Prerequisites
+
+* GraalVM (>= rc8)
+* TruffleRuby
+* FastR
+
+The following R packages will be automatically installed when necessary, but could be installed prior
+to using gKnit if desired:
+
+* ggplot2
+* gridExtra
+* knitr
+
+Installation of R packages requires a development environment and can be time consuming.  In Linux,
+the gnu compiler and tools should be enough.  I am not sure what is needed on the Mac.
+
+## Preparation
+
+* gem install galaaz
+
+## Usage
+
+* gknit <filename>