galaaz 0.4.10 → 2.0.0

data/blogs/manual/manual.Rmd

@@ -1,26 +1,34 @@
 ---
 title: "Galaaz Manual"
-subtitle: "How to tightly couple Ruby and R in GraalVM"
+subtitle: "Coupling Ruby (JRuby) and GNU R for data science"
 author: "Rodrigo Botafogo"
-tags: [Galaaz, Ruby, R, TruffleRuby, FastR, GraalVM, ggplot2]
-date: "2019"
+tags: [Galaaz, Ruby, JRuby, R, "GNU R", ggplot2, knitr, dplyr, Bioconductor, Arrow]
+date: "2026"
+bibliography: "../../examples/Bibliography/stats.bib"
 output:
+  html_document:
+    self_contained: true
+    keep_md: true
+    toc: true
+    toc_depth: 3
+    number_sections: true
   pdf_document:
     includes:
       in_header: "../../sty/galaaz.sty"
     keep_tex: yes
     number_sections: yes
     toc: true
-    toc_depth: 2
-  html_document:
-    self_contained: true
-    keep_md: true
+    toc_depth: 3
   md_document:
     variant: markdown_github
 fontsize: 11pt
 ---
 
 ```{ruby setup, echo=FALSE}
+# Bridge default is 60s; some chunks (Arrow, large dplyr pipes) need more.
+ENV['GALAAZ_BRIDGE_TIMEOUT_SEC'] ||= '300'
+
+R.options(crayon__enabled: false)
 R.install_and_loads('kableExtra')
 ```
 
@@ -30,30 +38,272 @@ Galaaz is a system for tightly coupling Ruby and R. Ruby is a powerful language,
 community, a very large set of libraries and great for web development. However, it lacks 
 libraries for data science, statistics, scientific plotting and machine learning. On the 
 other hand, R is considered one of the most powerful languages for solving all of the above 
-problems. Maybe the strongest competitor to R is Python with libraries such as NumPy, 
-Panda, SciPy, SciKit-Learn and a couple more.
+problems. **Python** is a strong competitor: NumPy, pandas, SciPy, and scikit-learn are
+widely used building blocks, and **PyPI** hosts many thousands of other packages for
+numerical work, machine learning, and beyond.
+
+With Galaaz we do not intend to re-implement any of the scientific libraries in R, we allow
+for very tight coupling between the two languages to the point that the Ruby developer does
+not need to know that there is an R engine running.
+
+According to Wikipedia "Ruby is a dynamic, interpreted, reflective, object-oriented, 
+general-purpose programming language. It was designed and developed in the mid-1990s by Yukihiro 
+"Matz" Matsumoto in Japan."  It reached high popularity with the development of Ruby on Rails
+(RoR) by David Heinemeier Hansson. RoR is a web application framework first released
+around 2005. It makes extensive use of Ruby's metaprogramming features.  With RoR,
+Ruby became very popular.  According to [Ruby’s place in the TIOBE index](https://www.tiobe.com/tiobe-index/ruby/)
+it peaked in popularity around 2008, then declined until 2015 when it started picking up again.
+Ruby remains a significant language in web development and general-purpose scripting.
+
+Python, a language similar to Ruby, ranks 4th in the index.  Java, C and C++ take the
+first three positions.  Ruby is often criticized for its focus on web applications.
+But Ruby can do [much more](https://github.com/markets/awesome-ruby) than just web applications.
+Yet, for scientific computing, Ruby lags behind Python and R.  Python offers Django and
+similar frameworks for the web, plus NumPy, pandas, and a deep catalog of science and ML libraries.
+R is a free software environment for statistical computing and graphics with thousands
+of libraries for data analysis. 
+
+Until recently, there was no real perspective for Ruby to bridge this gap.
+Implementing a complete scientific computing infrastructure would take too long.
+
+**Galaaz 2.0** couples **JRuby** (Ruby on the JVM) with **GNU R**—the same R you use for
+CRAN and Bioconductor. Ruby and R run in **separate processes**; the **Galaaz bridge**
+sends requests to R and returns results to Ruby. From your point of view you still write
+Ruby: `R.c(...)`, `R.library('ggplot2')`, `~R[:mtcars]`, and dplyr-style chains on R objects.
+You do not need to learn R syntax to get a lot done, though reading R documentation for
+individual packages remains useful.
+
+Earlier experiments with Galaaz used Oracle’s **GraalVM** with TruffleRuby and FastR so that
+Ruby and R could share one runtime. That path is no longer the focus: **standard GNU R**
+gives full compatibility with the R package ecosystem (including compiled extensions and
+Bioconductor) while JRuby gives a mature Ruby with **real multithreading** for application
+and I/O code.
+
+The bridge handles **communication and typing** between the two worlds; large tables can
+also flow through **Apache Arrow** on the R side when you use the optional helpers described
+later in this manual.
+
+Library wrapping is a common way to bring features from one language into another.
+To improve performance, Python often wraps more efficient C libraries. For the
+Python developer, the existence of such C libraries is hidden.  The problem with
+library wrapping is that for any new library, there is the need to handcraft a new
+wrapper.
+
+Galaaz, instead of wrapping a single C or R library, wraps the whole R language
+in Ruby.  Doing so, all thousands of R libraries are available immediately
+to Ruby developers without any new wrapping effort.
+
+## What does Galaaz mean
+
+Galaaz is the Portuguese name for "Galahad".  From Wikipedia:
+
+    Sir Galahad (sometimes referred to as Galeas or Galath),
+    in Arthurian legend, is a knight of King Arthur's Round Table and one
+    of the three achievers of the Holy Grail. He is the illegitimate son
+    of Sir Lancelot and Elaine of Corbenic, and is renowned for his
+    gallantry and purity as the most perfect of all knights. Emerging quite
+    late in the medieval Arthurian tradition, Sir Galahad first appears in the
+    Lancelot–Grail cycle, and his story is taken up in later works such as
+    the Post-Vulgate Cycle and Sir Thomas Malory's Le Morte d'Arthur.
+    His name should not be mistaken with Galehaut, a different knight from
+    Arthurian legend. 
+
+# Command-line tools (`bin/`)
+
+The Galaaz repository ships many helpers under **`bin/`**. When working from a **clone**, call
+them as **`bin/<name>`** from the project root (or `./bin/<name>`). If you install the **gem**,
+only a subset is guaranteed on your `PATH` (see the gemspec: **`galaaz`**, **`gstudio`**, **`gknit`**, **`grun`**, **`gknit-draft`**); for development and CI, prefer the **`bin/`** copies so JVM flags and paths stay correct.
+
+Below, **current (Galaaz 2.0 + JRuby + GNU R)** means the tool is wired to **`jruby`** and
+**`bin/galaaz_jruby_env.inc.sh`** (or equivalent logic in Ruby via `lib/galaaz_jruby.rb`). **Legacy**
+means the script still targets **GraalVM** polyglot Ruby / FastR-era invocation and is **not**
+expected to work on a typical JRuby-only setup.
+
+**Table layout:** names in the first column are **`bin/`** filenames (run as `bin/<name>` from the repo root). Long options and examples sit **outside** the tables so PDF columns stay readable.
+
+```{r bin-tables-helper, echo=FALSE}
+bin_tbl <- function(df) {
+  k <- knitr::kable(df, row.names = FALSE, booktabs = TRUE, linesep = "",
+                    col.names = c("Script", "Role", "2.0?"))
+  if (knitr::is_latex_output()) {
+    k <- kableExtra::kable_styling(k, font_size = 9, latex_options = "scale_down")
+    k <- kableExtra::column_spec(k, 1, width = "2.5cm")
+    k <- kableExtra::column_spec(k, 2, width = "9.5cm")
+    k <- kableExtra::column_spec(k, 3, width = "2.8cm")
+  } else {
+    k <- kableExtra::kable_styling(k, bootstrap_options = c("striped", "condensed"), full_width = TRUE)
+  }
+  k
+}
+```
+
+```{r bin-tables-bootstrap, echo=FALSE}
+df_boot <- data.frame(
+  Script = c("galaaz-bootstrap", "galaaz-jruby", "galaaz_jruby_env.inc.sh", "install-tinytex"),
+  Role = c(
+    "WSL2 helper: Docker checks; optional TinyTeX or poppler for gKnit PDF.",
+    "JRuby with repo lib/ on LOAD_PATH and required JVM flags (e.g. Arrow).",
+    "Sourced by bash wrappers; sets GALAAZ_REQUIRED_JRUBY_J_ARGS.",
+    "Install TinyTeX for PDF output."
+  ),
+  X2 = c("Yes*", "Yes", "Yes†", "Yes"),
+  stringsAsFactors = FALSE
+)
+bin_tbl(df_boot)
+```
+
+\* Where WSL/Docker apply. **`galaaz-bootstrap` flags:** `--check`, `--apply`, `--runtime` (`docker` \| `local` \| `auto`), `--[no-]prompt-doc-tools`.
+
+† Not run directly.
+
+**`galaaz-jruby` examples** (from repo root):
+
+```text
+bin/galaaz-jruby my_script.rb
+bin/galaaz-jruby -S rspec
+```
+
+## Interactive use, examples, and Rake
+
+```{r bin-tables-interactive, echo=FALSE}
+df_ix <- data.frame(
+  Script = c("gstudio", "run_example", "galaaz"),
+  Role = c(
+    "IRB or Pry with Galaaz preloaded (JRuby + JVM flags).",
+    "Run one Ruby file using the same JRuby/JVM setup as tests.",
+    "Forward arguments to rake (needs rake; usually JRuby)."
+  ),
+  X2 = c("Yes", "Yes", "Yes"),
+  stringsAsFactors = FALSE
+)
+bin_tbl(df_ix)
+```
+
+## gKnit and document drafts
+
+```{r bin-tables-gknit, echo=FALSE}
+df_gk <- data.frame(
+  Script = c("gknit", "gknit-draft", "gknit-draft.rb", "gknit_Rscript"),
+  Role = c(
+    "Knit .Rmd via JRuby and R Markdown render.",
+    "Drafts from rticles-style templates; wrapper still uses legacy polyglot ruby.",
+    "Ruby entry: GKnit.draft (use with JRuby + LOAD_PATH).",
+    "Polyglot Rscript launcher; hard-coded LOAD_PATH sample."
+  ),
+  X2 = c("Yes", "Legacy", "JRuby", "No"),
+  stringsAsFactors = FALSE
+)
+bin_tbl(df_gk)
+```
+
+**`gknit` CLI** (see `gknit -h`): `--output_format`, `--output_file`, `--output_dir`, `--bridge_timeout_sec`, `--callback_timeout_ms`. If `--output_format` is omitted, the **first** YAML `output:` target wins.
+
+Prefer **`galaaz-jruby`** for **`gknit-draft`** workflows until that wrapper matches the **`gknit`** stack.
+
+## Tests
+
+```{r bin-tables-tests, echo=FALSE}
+df_ts <- data.frame(
+  Script = c("run_rspec", "run_all_rspec", "run_slow_rspec", "run_old_rspec", "run_rspec_subset"),
+  Role = c(
+    "Top-level specs/*_spec.rb with spec_helper (see docs/testing.md).",
+    "Compile ext/new_bridge; run specs/ and new_bridge_specs/ together.",
+    "Suites under slow-specs/ (read script header for spec_helper).",
+    "Legacy suites under old_specs/.",
+    "Numbered subset 1–18 (Documentation/Spec_Subsets.md)."
+  ),
+  X2 = c("Yes", "Yes", "Yes", "Yes", "Yes"),
+  stringsAsFactors = FALSE
+)
+bin_tbl(df_ts)
+```
+
+## Other
+
+```{r bin-tables-other, echo=FALSE}
+df_ot <- data.frame(
+  Script = c("grun", "gstudio_irb.rb / gstudio_pry.rb"),
+  Role = c(
+    "Graal-era launcher: polyglot ruby with --jvm. Use galaaz-jruby -S instead.",
+    "Loaded by gstudio; not meant to be run standalone."
+  ),
+  X2 = c("No", "Yes"),
+  stringsAsFactors = FALSE
+)
+bin_tbl(df_ot)
+```
+
+For day-to-day **2.0** use, rely on **`bin/galaaz-jruby`**, **`bin/gstudio`**, **`bin/gknit`**, **`bin/run_example`**, **`bin/run_rspec`** / **`bin/run_all_rspec`**, and **`bin/galaaz-bootstrap`** on WSL when using Dockerized R. Treat **`grun`**, **`gknit_Rscript`**, and the polyglot **`ruby`** invocation in **`gknit-draft`** as **legacy** until they are ported to the same JRuby path as **`gknit`**.
 
 # System Compatibility
 
-* Oracle Linux 7
-* Ubuntu 18.04 LTS
-* Ubuntu 16.04 LTS
-* Fedora 28
-* macOS 10.14 (Mojave)
-* macOS 10.13 (High Sierra)
+Typical development and CI targets:
 
-# Dependencies
+* **Linux** — recent Ubuntu LTS or comparable distributions (x86_64).
+* **macOS** — recent releases with JRuby and GNU R available.
+* **Windows** — use **WSL2** (same Linux stack as above); native Windows is not the primary target.
+
+The native **gatekeeper** component under `ext/new_bridge` is built with `make` and a C++ toolchain; see the project `README` if compilation fails on your platform.
 
-* TruffleRuby
-* FastR
+# Dependencies
 
+* **JRuby** — Galaaz 2.0 requires JRuby (tested with **10.1.1.0**) and a matching **JDK** (tested with **Java 21**). MRI Ruby is not supported.
+* **GNU R** — `R` and `Rscript` on your `PATH` (tested with **4.3.3**), plus a C++ toolchain (`g++`, `make`) and the **Rcpp** package to compile the gatekeeper.
+* **galaaz gem** — runtime dependency `msgpack` is pulled in by `gem install`.
+* Optional: **Docker** — if you run R in a container (common on WSL2); see bootstrap below.
+* Optional R packages for examples in this manual — e.g. `ggplot2`, `dplyr`, `knitr`, `kableExtra`, `arrow`, Bioconductor tools such as **DESeq2** (installed the usual R way).
 
 # Installation
 
-* Install GrallVM (http://www.graalvm.org/)
-* Install Ruby (gu install Ruby)
-* Install FastR (gu install R)
-* Install rake if you want to run the specs and examples (gem install rake)
+The supported install is **`gem install` + compile the gatekeeper**. You do not need a git clone.
+
+1. Install **JRuby**, a compatible **JDK**, and **GNU R** (with `Rscript` and a C++ compiler).
+2. In R, install **Rcpp**: `install.packages("Rcpp")`.
+3. Install the gem: `jruby -S gem install galaaz`
+4. Compile the native gatekeeper from the installed gem:
+
+   ```
+   gem_dir="$(jruby -e "puts Gem::Specification.find_by_name('galaaz').full_gem_path")"
+   make -C "${gem_dir}/ext/new_bridge" all
+   ```
+
+5. Ensure **`R`** starts GNU R and can install packages (network access to CRAN when you first call `R.install_and_loads`). For **Apache Arrow** on Java 9+, pass `-J--add-opens=java.base/java.nio=ALL-UNNAMED` to JRuby (from a checkout, `bin/galaaz-jruby` does this).
+
+For **gKnit**, **knitr**, **rmarkdown**, and LaTeX (PDF output), install the corresponding R packages, **Pandoc**, and a TeX distribution if you need PDF; the repository includes helpers such as **`bin/install-tinytex`** where appropriate.
+
+A **table of all `bin/` scripts** (bootstrap, JRuby wrapper, gstudio, gknit, test runners, and which ones are legacy) is in the section **Command-line tools (`bin/`)** earlier in this manual.
+
+### From a repository checkout (contributors)
+
+1. Install **bundler** if needed, then run **`jruby -S bundle install`** in the repository root.
+2. Build the bridge native code: **`make -C ext/new_bridge all`** (or **`rake compile_gatekeeper`**).
+3. Run scripts with **`bin/galaaz-jruby`** (sources **`bin/galaaz_jruby_env.inc.sh`** and adds **`-I lib`**).
+
+Maintainers can prove a built `.gem` on a throwaway Ubuntu machine (no repo inside the container) with **`./docker/cold-install/run.sh`**.
+
+## Windows + WSL2 (optional: Docker / R in a container)
+
+If you run Galaaz on Windows through WSL2 and want containerized R instances,
+Docker Desktop is the supported setup.
+
+1. Install Docker Desktop on Windows:
+   - https://www.docker.com/products/docker-desktop/
+2. Open Docker Desktop and enable WSL integration:
+   - Settings > Resources > WSL Integration
+   - Enable integration for your target distro
+   - Apply & Restart Docker Desktop
+3. In WSL, run Galaaz bootstrap:
+
+  > ruby bin/galaaz-bootstrap --apply
+  > ruby bin/galaaz-bootstrap --check
+
+Expected result:
+- docker CLI available
+- docker compose available
+- docker daemon reachable (`docker info` works)
+
+If bootstrap reports daemon is unreachable, check Docker Desktop is running and
+WSL integration is enabled for the distro where Galaaz is installed.
 
 # Usage
 
@@ -82,8 +332,8 @@ Panda, SciPy, SciKit-Learn and a couple more.
 
   > galaaz -T
   
-  Shows a list with all available executalbe tasks.  To execute a task, substitute the
-  'rake' word in the list with 'galaaz'.  For instance, the following line shows up
+  Shows a list with all available executable tasks.  To execute a task, substitute the
+   'rake' word in the list with 'galaaz'.  For instance, the following line shows up
   after 'galaaz -T'
   
   rake master_list:scatter_plot        # scatter_plot from:....
@@ -92,18 +342,948 @@ Panda, SciPy, SciKit-Learn and a couple more.
   
   > galaaz master_list:scatter_plot
 
+# JRuby, multithreading, and the R bridge
+
+Galaaz 2.0 runs Ruby on **JRuby**, so your application can use **real parallel threads** for
+I/O-bound work (HTTP clients, database connections, message consumers, and so on). R itself is
+still executed in a **single GNU R process** behind the Galaaz bridge.
+
+When several Ruby threads call into R at the same time, the bridge **serializes** those calls:
+each request is matched to a reply using an internal per-call **queue**, so you do not need to
+add your own mutex around every `R.foo` from application threads. (You should still use normal
+Ruby synchronization when **Ruby** data structures are shared between threads—for example, when
+appending rows from each thread into a shared array before sending them to R.)
+
+A practical pattern is:
+
+1. Use threads (or a connection pool) to read from **multiple databases or shards** in parallel.
+2. Merge the rows in Ruby under a `Mutex` if you collect into one structure.
+3. Hand the merged table to R **once** (for example with `R::Arrow.from_ruby_batches` and dplyr,
+   or by building a data frame) so heavy statistics run in R with fewer bridge round-trips.
+
+A runnable sketch lives in
+`examples/multithread_shards_to_r/shards_to_r.rb` (simulated shard queries; swap in your DB
+driver). For concurrency tests on the bridge itself, see `specs/bridge_concurrent_spec.rb` and
+`specs/arrow_from_ruby_batches_spec.rb`.
+
+## Long-running R calls and a completion block
+
+For R work that can take a long time, the bridge can avoid a Ruby-side **wait timeout** by
+scheduling the call and resuming in a **block** when the `RET` arrives.
+
+- **`R.eval_r_async(code, timeout: nil) { |result| ... }`** — string eval; on success, `result.value`
+  is the same formatted string as **`R.eval_r`** (use `timeout: nil` for no Ruby-side limit).
+- **`R::Async.<rname>(...) { |result| ... }`** — same dispatch as **`R.<rname>(...)`**, but async;
+  on success, `result.value` is an **`R::Object`** (or unboxed Ruby value / Symbol), like synchronous
+  **`R.<rname>`**. Optional keyword **`timeout:`** applies a Ruby-side wait limit (completion receives
+  **`NewBridge::SessionClient::TimeoutError`** if R is too slow).
+
+**Important:** **`R.foo(...) { |x| }`** is already used for dplyr-style scopes (`R::Support.new_scope`),
+so async R calls must use **`R::Async`** or **`R.eval_r_async`**, not a bare **`R.foo` with a block.**
+
+`NewBridge::EvalResult` exposes **`#ok?`**, **`#value`**, and **`#error`**. The completion block runs on a
+**background thread** (not the bridge reader thread).
+
+The example below is **plain Ruby** (no Rails). The R snippet sleeps (standing in for heavy work) and then
+returns an integer so the success branch shows a **non-nil** value. (`Sys.sleep` alone returns **NULL** in R;
+on success **`result.value`** is then **`nil`** in Ruby—that is expected, not a bridge error.)
+
+```{ruby long_r_completion_block}
+require 'thread'
+
+completion = Queue.new
+
+R.eval_r_async('({ Sys.sleep(0.3); 42L })', timeout: nil) do |result|
+  if result.ok?
+    puts "[completion] R finished; eval_r-style value: #{result.value.inspect}"
+  else
+    puts "[completion] R/bridge error: #{result.error.class}: #{result.error.message}"
+  end
+  completion.push(:done)
+end
+
+3.times do |i|
+  puts "[main] other Ruby work step #{i + 1}"
+  sleep 0.05
+end
+
+completion.pop
+puts "[main] R completion has run; exiting."
+```
+
+In a **web application**, the HTTP response usually ends before R finishes, so you would not
+`Queue#pop` in the controller; you would persist an identifier, let the completion block write
+the outcome to storage, and notify the client (poll, WebSocket, Turbo Stream, etc.). The plain
+Ruby pattern above is only to show **when** the result exists (inside the block, or after data
+written there is observed elsewhere). Runnable specs live in **`new_bridge_specs/eval_r_async_spec.rb`**.
+
+## Galaaz + Rails (JRuby) integration baseline
+
+This section documents the baseline we used to create a working Rails app with Galaaz in WSL.
+The goals were:
+
+1. Rails boots under **JRuby**.
+2. Galaaz is loaded from a local checkout (before publishing to RubyGems).
+3. A request path can execute **`R.eval(...)`** and return a result.
+
+### 1) Create the app with JRuby-friendly options
+
+Rails defaults can pull gems that are not ideal on JRuby-first setups (for example sqlite native
+extension paths and deployment extras). A minimal app avoids early friction:
+
+```bash
+cd /home/rbotafogo/desenv_linux
+jruby -S rails new hedi --skip-git --minimal --skip-kamal --skip-solid --skip-active-record
+```
+
+Then install gems:
+
+```bash
+cd /home/rbotafogo/desenv_linux/hedi
+jruby -S bundle install
+```
+
+### 2) Use Galaaz as a local path gem
+
+For local development we keep a stable path:
+
+- `~/gems/galaaz` -> symlink to your Galaaz checkout
+- optional built gem archive in `~/gems/pkg/`
+
+In Rails `Gemfile`:
+
+```ruby
+gem "galaaz", path: "/home/rbotafogo/gems/galaaz", require: false
+```
+
+And load after Rails boot in `config/application.rb`:
+
+```ruby
+config.after_initialize { require "galaaz" }
+```
+
+Why `require: false` + `after_initialize`? In this integration, loading Galaaz too early via
+`Bundler.require` triggered Rails/JRuby initialization failures.
+
+### 3) Simple request-path smoke test
+
+A direct smoke test from Rails runner:
+
+```bash
+cd /home/rbotafogo/desenv_linux/hedi
+jruby -S bundle exec rails runner "puts R.eval('sum(c(1,2,3,4,5))').inspect"
+```
+
+Expected output:
+
+```text
+15.0
+```
+
+### 4) HTTP endpoint pattern
+
+For this baseline, a small Rack endpoint was the most stable first step to prove request-time R
+evaluation. (A full ActionController stack can be enabled later as the app evolves.)
+
+Minimal pattern:
+
+1. Define a Rack app class under `lib/` that runs `R.eval(...)` and returns HTML/JSON.
+2. Point a route to that Rack app (`root to: MyRackApp`).
+3. Verify with browser/curl.
+
+### 5) Running from WSL and opening from Windows
+
+Recommended bind:
+
+```bash
+jruby -S bundle exec rails server -b 0.0.0.0 -p 3000
+```
+
+Then open from Windows:
+
+- `http://localhost:3000` (usually works with WSL localhost forwarding), or
+- `http://<wsl-ip>:3000` if needed.
+
+In development, if Host Authorization blocks requests with unexpected Host headers, use:
+
+```ruby
+# config/environments/development.rb
+config.hosts.clear
+```
+
+### 6) Troubleshooting checklist
+
+- `Could not find ... in locally installed gems`:
+  run `jruby -S bundle install` in the Rails app directory.
+- Stale PID after crash:
+  remove `tmp/pids/server.pid`.
+- Local Galaaz path changed:
+  verify `Gemfile` path target exists and rerun bundler.
+- R runtime issues:
+  confirm GNU R is installed and on `PATH` in the same shell where Rails runs.
+
+As new Rails features are added (controllers, jobs, websockets, background rendering, plot
+generation), extend this section with concrete, runnable snippets and the associated operational
+checks.
+
+# Accessing R from Ruby
+
+One of the nice aspects of Galaaz is that variables and functions defined in R can
+be easily accessed from Ruby.  For instance, to access the `mtcars` data frame from R
+in Ruby, we use the symbol `:mtcars` preceded by the `~` operator: `~R[:mtcars]` retrieves the 
+value of the `mtcars` object in R.
+
+```{ruby access_r}
+puts ~R[:mtcars]
+```
+
+## Scoped symbols and lexical scoping
+
+Galaaz 2.0 uses **scoped symbols** by default. The canonical style is `R[:name]`:
+
+- `~R[:mtcars]` fetches an R object by name.
+- `R[:a] + R[:b]` builds an expression.
+- `R[:year].up_to(R[:day])` builds range expressions.
+
+If you prefer the terse `:x` syntax, you can opt in with lexical scoping using a Ruby refinement:
+
+```ruby
+module MyScript
+  using Galaaz::SymbolDSL
+
+  def self.run
+    expr = :a + :b
+    puts expr
+    puts ~:mtcars
+  end
+end
+```
+
+`using Galaaz::SymbolDSL` is **lexically scoped**: only code in that module/file scope gets `:x` DSL behavior.
+Outside that scope, plain Ruby `Symbol` behavior is unchanged.
+
+To access an R function from Ruby, the R function needs to be preceded by `R.` scoping. 
+Below we see an example of creating a R::Vector by calling the 'c' R function
+
+```{ruby call_r_func}
+puts vec = R.c(1.0, 2.0, 3.0, 4.0)
+```
+Note that 'vec' is an object of type R::Vector:
+
+```{ruby r_object}
+puts vec.class
+```
+Every object created by a call to an R function will be of a type that inherits from
+R::Object. In R, there is also a function 'class'. In order to access that function we
+can call method 'rclass' in the R::Object:
+
+```{ruby rclass}
+puts vec.rclass
+```
+When working with R::Object(s), it is possible to use the '.' operator to pipe operations.
+When using '.', the object to which the '.' is applied becomes the first argument of the
+corresponding R function. For instance, function 'c' in R, can be used to concatenate
+two vectors or more vectors (in R, there are no scalar values, scalars are converted to
+vectors of size 1. Within Galaaz, scalar parameter is converted to a size one vector):
+
+```{ruby concat}
+puts R.c(vec, 10, 20, 30)
+```
+The call above to the 'c' function can also be done using '.' notation:
+
+```{ruby concat_with_dot}
+puts vec.c(10, 20, 30)
+```
+We will talk about vector indexing in a later section. But notice here that indexing
+an R::Vector will return another R::Vector:
+
+```{ruby indexing}
+puts vec[1]
+```
+Sometimes we want to index an R::Object and get back a Ruby object that is not wrapped
+in an R::Object, but the native Ruby object. For this, we can index the R object with
+the '>>' operator:
+
+```{ruby native_value}
+puts vec >> 0
+puts vec >> 2
+```
+
+It is also possible to call an R function with named arguments, by creating the function
+in Galaaz with named parameters. For instance, here is an example of creating a 'list'
+with named elements:
+
+```{ruby named_parameters}
+puts R.list(first_name: "Rodrigo", last_name: "Botafogo")
+```
+
+Many R functions receive another function as argument. For instance, method 'map' applies
+a function to every element of a vector. With Galaaz, it is possible to pass a Proc, 
+Method or Lambda in place of the expected R function. In this next example, we will
+add 2 to every element of our previously created vector:
+
+```{ruby proc_as_param}
+puts vec.map { |x| x + 2 }
+```
+
 # gKnitting a Document
 
-This manual has been formatted usign gKnit.  gKnit uses Knitr and R markdown to knit 
-a document in Ruby or R and output it in any of the available formats for R markdown.
-gKnit runs atop of GraalVM, and Galaaz.  In gKnit, Ruby variables are persisted between 
+This manual has been formatted using gKnit.  gKnit uses knitr and R Markdown to knit 
+a document in Ruby or R and output it in any of the available formats for R Markdown.
+gKnit runs with **JRuby**, **GNU R**, and Galaaz.  In gKnit, Ruby variables are persisted between 
 chunks, making it an ideal solution for literate programming. Also, since it is based 
-on Galaaz, Ruby chunks can have access to R variables and Polyglot Programming with 
-Ruby and R is quite natural.
+on Galaaz, Ruby chunks can have access to R variables and combining Ruby with R in one 
+document is natural.
+
+The idea of "literate programming" was first introduced by Donald Knuth in the 
+1980's [@Knuth:literate_programming].
+The main intention of this approach was to develop software interspersing macro snippets,
+traditional source code, and a natural language such as English in a document 
+that could be compiled into
+executable code and at the same time easily read by a human developer. According to Knuth
+"The practitioner of 
+literate programming can be regarded as an essayist, whose main concern is with exposition 
+and excellence of style."
+
+The idea of literate programming evolved into the idea of reproducible research, in which
+all the data, software code, documentation, graphics etc. needed to reproduce the research
+and its reports could be included in a
+single document or set of documents that when distributed to peers could be rerun generating
+the same output and reports.
+
+The R community has put a great deal of effort in reproducible research.  In 2002, Sweave was
+introduced and it allowed mixing R code with LaTeX, generating high-quality PDF documents.  A
+Sweave document could include code, the results of executing the code, graphics and text 
+such that it contained the whole narrative to reproduce the research.  In
+2012, Knitr, developed by Yihui Xie from RStudio was released to replace Sweave and to
+consolidate in one single package the many extensions and add-on packages that
+were necessary for Sweave.
+
+With Knitr, __R markdown__ was also developed, an extension to the
+Markdown format.  With __R markdown__ and Knitr it is possible to generate reports in a multitude
+of formats such as HTML, Markdown, LaTeX, PDF, DVI, etc.  __R markdown__ also allows the use of
+multiple programming languages such as R, Ruby, Python, etc. in the same document.  
+
+In __R markdown__, text is interspersed with
+code chunks that can be executed and both the code and its results can become
+part of the final report.  Although __R markdown__ allows multiple programming languages in the
+same document, only R and Python (with
+the reticulate package) can persist variables between chunks.  For other languages, such as
+Ruby, every chunk will start a new process and thus all data is lost between chunks, unless it
+is somehow stored in a data file that is read by the next chunk.
+
+Being able to persist data
+between chunks is critical for literate programming otherwise the flow of the narrative is lost
+by all the effort of having to save data and then reload it. Although this might, at first, seem like
+a small nuisance, not being able to persist data between chunks is a major issue. For example, let's
+take a look at the following simple example in which we want to show how to create a list and the
+use it.  Let's first assume that data cannot be persisted between chunks.  In the next chunk we
+create a list, then we would need to save it to file, but to save it, we need somehow to marshal the
+data into a binary format:
+
+```{ruby no_persistence}
+lst = R.list(a: 1, b: 2, c: 3)
+lst.saveRDS("lst.rds")
+```
+then, on the next chunk, where variable 'lst' is used, we need to read back it's value
+
+```{ruby load_persisted_data}
+lst = R.readRDS("lst.rds")
+puts lst
+```
+
+Now, any single code has dozens of variables that we might want to use and reuse between chunks.
+Clearly, such an approach becomes quickly unmanageable. Probably, because of 
+this problem, it is very rare to see any __R markdown__ document in the Ruby community.
+
+When variables can be used across chunks, then no overhead is needed:
+
+```{ruby persistence}
+lst = R.list(a: 1, b: 2, c: 3)
+# any other code can be added here
+```
+
+```{ruby use_var}
+puts lst
+```
+
+In the Python community, the same effort to have code and text in an integrated environment
+started around the first decade of the 2000s. In 2006 IPython 0.7.2 was released.  In 2014,
+Fernando Pérez spun off the Jupyter project from IPython, creating a web-based interactive
+computation environment.  Jupyter can now be used with many languages, including Ruby with the
+iruby gem (https://github.com/SciRuby/iruby).  In order to have multiple languages in a Jupyter
+notebook the SoS kernel was developed (https://vatlab.github.io/sos-docs/).
+
+## gKnit and __R markdown__
+
+gKnit is based on knitr and __R markdown__ and can knit a document 
+written both in Ruby and/or R and output it in any of the available formats of __R markdown__.  gKnit
+allows ruby developers to do literate programming and reproducible research by allowing them to
+have in a single document, text and code.
+
+In gKnit, Ruby variables are persisted between 
+chunks, making  it an ideal solution for literate programming in this language.  Also, 
+since it is based on Galaaz, Ruby chunks can access R variables (`~R[:name]`, `R.*`) through the
+**Galaaz bridge** while knitr drives **GNU R**—no GraalVM polyglot runtime is required.
+
+This is not a blog post on __R markdown__, and the interested user is directed to the following links
+for detailed information on its capabilities and use.
+
+* https://rmarkdown.rstudio.com/ or
+* https://bookdown.org/yihui/rmarkdown/ 
+
+In this post, we will describe just the main aspects of __R markdown__, so the user can start 
+gKnitting Ruby and R documents quickly.
+
+## The Yaml header
+
+An __R markdown__ document should start with a Yaml header and be stored in a file with 
+'.Rmd' extension. This document has the following header for gKnitting an HTML document.
+
+```
+---
+title: "How to do reproducible research in Ruby with gKnit"
+author: 
+    - "Rodrigo Botafogo"
+    - "Daniel Mossé - University of Pittsburgh"
+tags: [Tech, Data Science, Ruby, R, JRuby, Galaaz]
+date: "20/02/2019"
+output:
+  html_document:
+    self_contained: true
+    keep_md: true
+  pdf_document:
+    includes:
+      in_header: ["../../sty/galaaz.sty"]
+    number_sections: yes
+---
+```
+
+For more information on the options in the Yaml header, [check here](https://bookdown.org/yihui/rmarkdown/html-document.html).
+
+## Choosing the output format when calling gknit
+
+Yes: you can select the render target on the **command line**. **`bin/gknit`** (or **`gknit`** on your `PATH`) forwards options to **`rmarkdown::render`** via **`R::Rmarkdown.render`**.
+
+* **`--output_format FORMAT`** — name of the format, as in the YAML `output:` block. Examples:
+  * **`html_document`** — HTML (often the default you list first under `output:`).
+  * **`pdf_document`** — PDF (you need a working LaTeX setup, e.g. TinyTeX; see **`bin/install-tinytex`**).
+  * **`md_document`**, **`github_document`**, or any other format defined in your YAML.
+  * **`all`** — render **every** format declared under `output:` in the document (same idea as in R Markdown).
+
+If you **omit** **`--output_format`**, gknit passes **`NULL`** for the format argument. In that case **rmarkdown** uses the **first** format listed under **`output:`** in the YAML (and if none is specified there, behavior follows the usual rmarkdown defaults, typically HTML).
+
+Other useful flags:
+
+* **`--output_file NAME`** — output file name (optional path; see also **`--output_dir`**).
+* **`--output_dir DIR`** — directory for the rendered file (created if missing).
+* **`--bridge_timeout_sec`** / **`--callback_timeout_ms`** — longer R or install steps (see elsewhere in this manual).
+
+Examples (run from the directory where paths make sense, or use absolute paths):
+
+```text
+bin/gknit blogs/manual/manual.Rmd
+bin/gknit --output_format html_document blogs/manual/manual.Rmd
+bin/gknit --output_format pdf_document blogs/manual/manual.Rmd
+bin/gknit --output_format all blogs/manual/manual.Rmd
+```
+
+Use **`gknit -h`** for the full option list.
+
+## __R Markdown__ formatting
+
+Document formatting can be done with simple markups such as:
+
+## Headers
+
+```
+# Header 1
 
-[gknit is described in more details here](https://towardsdatascience.com/how-to-do-reproducible-research-in-ruby-with-gknit-c26d2684d64e)
+## Header 2
 
-# Vector
+### Header 3
+
+```
+
+## Lists
+
+```
+Unordered lists:
+
+* Item 1
+* Item 2
+    + Item 2a
+    + Item 2b
+```
+
+```
+Ordered Lists
+
+1. Item 1
+2. Item 2
+3. Item 3
+    + Item 3a
+    + Item 3b
+```
+
+For more R markdown formatting go to https://rmarkdown.rstudio.com/authoring_basics.html.
+
+## R chunks
+
+Running and executing Ruby and R code is actually what really interests us is this blog.  
+Inserting a code chunk is done by adding code in a block delimited by three back ticks 
+followed by an open
+curly brace ('{') followed with the engine name (r, ruby, rb, include, ...), an 
+any optional chunk_label and options, as shown below:
+
+````
+```{engine_name [chunk_label], [chunk_options]}`r ''`
+```
+````
+
+for instance, let's add an R chunk to the document labeled 'first_r_chunk'.  This is
+a very simple code just to create a variable and print it out, as follows:
+
+````
+```{r first_r_chunk}`r ''`
+vec <- c(1, 2, 3)
+print(vec)
+```
+````
+
+If this block is added to an __R markdown__ document and gKnitted the result will be:
+
+```{r first_r_chunk}
+vec <- c(1, 2, 3)
+print(vec)
+```
+
+Now let's say that we want to do some analysis in the code, but just print the result and not the
+code itself.  For this, we need to add the option 'echo = FALSE'.
+
+````
+```{r second_r_chunk, echo = FALSE}`r ''`
+vec2 <- c(10, 20, 30)
+vec3 <- vec * vec2
+print(vec3)     
+```
+````
+Here is how this block will show up in the document. Observe that the code is not shown
+and we only see the execution result in a white box
+
+```{r second_r_chunk, echo = FALSE}
+vec2 <- c(10, 20, 30)
+vec3 <- vec * vec2
+print(vec3)     
+```
+
+A description of the available chunk options can be found in https://yihui.name/knitr/.
+
+Let's add another R chunk with a function definition.  In this example, a vector 
+'r_vec' is created and
+a new function 'reduce_sum' is defined.  The chunk specification is
+
+````
+```{r data_creation}`r ''`
+r_vec <- c(1, 2, 3, 4, 5)
+
+reduce_sum <- function(...) {
+  Reduce(sum, as.list(...))
+}
+```
+````
+
+and this is how it will look like once executed.  From now on, to be concise in the
+presentation we will not show chunk definitions any longer.
+
+
+```{r data_creation}
+r_vec <- c(1, 2, 3, 4, 5)
+
+reduce_sum <- function(...) {
+  Reduce(sum, as.list(...))
+}
+```
+
+We can, possibly in another chunk, access the vector and call the function as follows:
+
+```{r using_previous}
+print(r_vec)
+print(reduce_sum(r_vec))
+```
+## R Graphics with ggplot
+
+In the following chunk, we create a bubble chart in R using ggplot and include it in 
+this document.  Note that there is no directive in the code to include the image, this
+occurs automatically.  The 'mpg' dataframe is natively available to R and to Galaaz as
+well.
+
+For the reader not knowledgeable of ggplot, ggplot is a graphics library based on "the
+grammar of graphics" [@Wilkinson:grammar_of_graphics]. The idea of the grammar of graphics
+is to build a graphics by adding layers to the plot.  More information can be found in
+https://towardsdatascience.com/a-comprehensive-guide-to-the-grammar-of-graphics-for-effective-visualization-of-multi-dimensional-1f92b4ed4149.
+
+In the plot below the 'mpg' dataset from base R is used. "The data concerns city-cycle fuel 
+consumption in miles per gallon, to be predicted in terms of 3 multivalued discrete and 5 
+continuous attributes." (Quinlan, 1993)
+
+First, the 'mpg' dataset if filtered to extract only cars from the following manumactures: Audi, Ford,
+Honda, and Hyundai and stored in the 'mpg_select' variable.  Then, the selected dataframe is passed
+to the ggplot function specifying in the aesthetic method (aes) that 'displacement' (disp) should
+be plotted in the 'x' axis and 'city mileage' should be on the 'y' axis.  In the 'labs' layer we
+pass the 'title' and 'subtitle' for the plot.  To the basic plot 'g', geom\_jitter is added, that
+plots cars from the same manufactures with the same color (col=manufactures) and the size of the
+car point equal its high way consumption (size = hwy).  Finally, a last layer is plotter containing
+a linear regression line (method = "lm") for every manufacturer.
+
+```{r bubble, dev='png'}
+# load package and data
+library(ggplot2)
+data(mpg, package="ggplot2")
+
+mpg_select <- mpg[mpg$manufacturer %in% c("audi", "ford", "honda", "hyundai"), ]
+
+# Scatterplot
+theme_set(theme_bw())  # pre-set the bw theme.
+g <- ggplot(mpg_select, aes(displ, cty)) + 
+  labs(subtitle="mpg: Displacement vs City Mileage",
+       title="Bubble chart")
+
+g + geom_jitter(aes(col=manufacturer, size=hwy)) + 
+  geom_smooth(aes(col=manufacturer), method="lm", se=F)
+```
+
+## Ruby chunks
+
+Including a Ruby chunk is just as easy as including an R chunk in the document: just 
+change the name of the engine to 'ruby'.  It is also possible to pass chunk options
+to the Ruby engine; however, this version does not accept all the options that are
+available to R chunks.  Future versions will add those options.
+
+````
+```{ruby first_ruby_chunk}`r ''`
+```
+````
+
+In this example, the ruby chunk is called 'first_ruby_chunk'.  One important 
+aspect of chunk labels is that they cannot be duplicated.  If a chunk label is 
+duplicated, gKnit will stop with an error.
+
+In the following chunk, variable 'a', 'b' and 'c' are standard Ruby variables 
+and 'vec' and 'vec2' are two vectors created  by calling the 'c' method on the 
+R module.
+
+In Galaaz, the R module allows us to access R functions transparently.  The 'c' 
+function in R, is a function that concatenates its arguments making a vector.
+
+It 
+should be clear that there is no requirement in gknit to call or use any R 
+functions.  gKnit will knit standard Ruby code, or even general text without 
+any code.
+
+```{ruby split_data}
+a = [1, 2, 3]
+b = "US$ 250.000"
+c = "The 'outputs' function"
+
+vec = R.c(1, 2, 3)
+vec2 = R.c(10, 20, 30)
+```
+
+In the next block, variables 'a', 'vec' and 'vec2' are used and printed.
+
+```{ruby split2}
+puts a
+puts vec * vec2
+```
+
+Note that 'a' is a standard Ruby Array and 'vec' and 'vec2' are vectors that behave accordingly,
+where multiplication works as expected.
+
+## Inline Ruby code
+
+When using a Ruby chunk, the code and the output are formatted in blocks as seen above.
+This formatting is not always desired.  Sometimes, we want to have the results of the
+Ruby evaluation included in the middle of a phrase. gKnit allows adding inline Ruby code 
+with the 'rb' engine.  The following chunk specification will 
+create and inline Ruby text:
+
+````
+This is some text with inline Ruby accessing variable 'b' which has value:
+```{rb puts "```{rb puts b}\n```"}
+```
+and is followed by some other text!
+````
+
+<div style="margin-bottom:30px;">
+</div>
+
+This is some text with inline Ruby accessing variable 'b' which has value:
+```{rb puts b}
+```
+and is followed by some other text!
+
+<div style="margin-bottom:30px;">
+</div>
+
+Note that it is important not to add any new line before of after the code
+block if we want everything to be in only one line, resulting in the following sentence 
+with inline Ruby code.
+
+
+```{ruby heading, echo = FALSE}
+outputs "### #{c}"
+```
+
+He have previously used the standard 'puts' method in Ruby chunks in order produce
+output.  The result of a 'puts', as seen in all previous chunks that use it,  is formatted 
+inside a white box that 
+follows the code block. Many times however, we would like to do some processing in the 
+Ruby chunk and have the result of this processing generate and output that is 
+"included" in the document as if we had typed it in __R markdown__ document.
+
+For example, suppose we want to create a new heading in our document, but the heading 
+phrase is the result of some code processing: maybe it's the first line of a file we are
+going to read.  Method 'outputs' adds its output as if typed in the __R markdown__ document. 
+
+Take now a look at variable 'c' (it was defined in a previous block above) as 
+'c = "The 'outputs' function".  "The 'outputs' function" is actually the name of this
+section and it was created using the 'outputs' function inside a Ruby chunk.
+
+The ruby chunk to generate this heading is:
+
+````
+```{ruby heading}`r ''`
+outputs "### #{c}"
+```
+````
+
+The three '###' is the way we add a Heading 3 in __R markdown__.
+
+
+### HTML Output from Ruby Chunks
+
+We've just seen the use of method 'outputs' to add text to the the __R markdown__ 
+document.  This technique can also be used to add HTML code to the document. In 
+__R markdown__, any html code typed directly in the document will be properly rendered.  
+Here, for instance, is a table definition in HTML and its output in the document:
+
+```
+<table style="width:100%">
+  <tr>
+    <th>Firstname</th>
+    <th>Lastname</th> 
+    <th>Age</th>
+  </tr>
+  <tr>
+    <td>Jill</td>
+    <td>Smith</td> 
+    <td>50</td>
+  </tr>
+  <tr>
+    <td>Eve</td>
+    <td>Jackson</td> 
+    <td>94</td>
+  </tr>
+</table>
+```
+<div style="margin-bottom:30px;">
+</div>
+
+<table style="width:100%">
+  <tr>
+    <th>Firstname</th>
+    <th>Lastname</th> 
+    <th>Age</th>
+  </tr>
+  <tr>
+    <td>Jill</td>
+    <td>Smith</td> 
+    <td>50</td>
+  </tr>
+  <tr>
+    <td>Eve</td>
+    <td>Jackson</td> 
+    <td>94</td>
+  </tr>
+</table>
+
+<div style="margin-bottom:30px;">
+</div>
+
+But manually creating HTML output is not always easy or desirable, specially
+if we intend the document to be rendered in other formats, for example, as LaTeX.
+Also, The above 
+table looks ugly.  The 'kableExtra' library is a great library for 
+creating beautiful tables. Take a look at https://cran.r-project.org/web/packages/kableExtra/vignettes/awesome_table_in_html.html
+
+In the next chunk, we output the 'mtcars' dataframe from R in a nicely formatted 
+table.  Note that we retrieve the mtcars dataframe by using '~R[:mtcars]'.
+
+```{ruby nice_table}
+R.install_and_loads('kableExtra')
+outputs (~R[:mtcars]).kable.kable_styling
+```
+
+## Including Ruby files in a chunk
+
+R is a language that was created to be easy and fast for statisticians to use.  As far
+as I know, it was not a
+language to be used for developing large systems.  Of course, there are large systems and
+libraries in R, but the focus of the language is for developing statistical models and
+distribute that to peers.
+
+Ruby on the other hand, is a language for large software development.  Systems written in
+Ruby will have dozens, hundreds or even thousands of files.  To document a 
+large system with literate programming, we cannot expect the developer to add all the 
+files in a single '.Rmd' file.  gKnit provides the 'include' chunk engine to include 
+a Ruby file as if it had being typed in the '.Rmd' file.
+
+To include a file, the following chunk should be created, where <filename> is the name of
+the file to be included and where the extension, if it is '.rb', does not need to be added.
+If the 'relative' option is not included, then it is treated as TRUE.  When 'relative' is
+true, ruby's 'require\_relative' semantics is used to load the file, when false, Ruby's 
+\$LOAD_PATH is searched to find the file and it is 'require'd.
+
+````
+```{include <filename>, relative = <TRUE/FALSE>}`r ''`
+```
+````
+
+Below we include file 'model.rb', which is in the same directory of this blog.  
+This code uses R 'caret' package to split a dataset in a train and test sets.
+The 'caret' package is a very important a useful package for doing Data Analysis,
+it has hundreds of functions for all steps of the Data Analysis workflow.  To
+use 'caret' just to split a dataset is like using the proverbial cannon to 
+kill the fly.  We use it here only to show that integrating Ruby and R and 
+using even a very complex package as 'caret' is trivial with Galaaz.
+
+A word of advice: the 'caret' package has lots of dependencies and installing
+it in a Linux system is a time consuming operation.  Method 'R.install_and_loads'
+will install the package if it is not already installed and can take a while.
+
+````
+```{include model}`r ''`
+```
+````
+
+```{include model}
+```
+
+```{ruby model_partition}
+mtcars = ~R[:mtcars]
+model = Model.new(mtcars, percent_train: 0.8)
+model.partition(:mpg)
+puts model.train.head
+puts model.test.head
+```
+
+## Documenting Gems
+
+gKnit also allows developers to document and load files that are not in the same directory
+of the '.Rmd' file.
+
+Here is an example of loading Ruby’s standard library file `find.rb`. In this example, relative
+is set to FALSE, so Ruby will look for the file in its `$LOAD_PATH`, and the user does not
+need to know its directory on disk.
+
+````
+```{include find, relative = FALSE}`r ''`
+```
+````
+
+```{include find, relative = FALSE}
+```
+
+## Converting to PDF
+
+One of the beauties of knitr is that the same input can be converted to many different outputs.
+One very useful format, is, of course, PDF.  In order to converted an __R markdown__ file to PDF
+it is necessary to have LaTeX installed on the system.  We will not explain here how to
+install LaTeX as there are plenty of documents on the web showing how to proceed.
+
+gKnit comes with a simple LaTeX style file for gknitting this blog as a PDF document.  Here is
+the Yaml header to generate this blog in PDF format instead of HTML:
+
+```
+---
+title: "gKnit - Ruby and R Knitting with Galaaz"
+author: "Rodrigo Botafogo"
+tags: [Galaaz, Ruby, R, JRuby, knitr, gknit]
+date: "29 October 2018"
+output:
+  pdf\_document:
+    includes:
+      in\_header: ["../../sty/galaaz.sty"]
+    number\_sections: yes
+---
+```
+
+## Template based documents generation
+
+When a document is converted to PDF it follows a certain conversion template. We've seen above
+the use of 'galaaz.sty' as a basic template to generate a PDF document.  Using the 
+'gknit-draft' app that comes with Galaaz, the same .Rmd file can be compiled to different 
+looking PDF documents. Galaaz automatically loads the 'rticles' R package that comes with
+templates for the following journals with the respective template name:
+
+* ACM articles: acm_article
+* ACS articles: acs_article
+* AEA journal submissions: aea_article
+* AGU journal submissions: ????
+* AMS articles: ams_article
+* American Statistical Association: asa_article
+* Biometrics articles: biometrics_article
+* Bulletin de l'AMQ journal submissions: amq_article
+* CTeX documents: ctex
+* Elsevier journal submissions: elsevier_article
+* IEEE Transaction journal submissions: ieee_article
+* JSS articles: jss_article
+* MDPI journal submissions: mdpi_article
+* Monthly Notices of the Royal Astronomical Society articles: mnras_article
+* NNRAS journal submissions: nmras_article
+* PeerJ articles: peerj_article
+* Royal Society Open Science journal submissions: rsos_article
+* Royal Statistical Society: rss_article
+* Sage journal submissions: sage_article
+* Springer journal submissions: springer_article
+* Statistics in Medicine journal submissions: sim_article
+* Copernicus Publications journal submissions: copernicus_article
+* The R Journal articles: rjournal_article
+* Frontiers articles: ???
+* Taylor & Francis articles: ???
+* Bulletin De L'AMQ: amq_article
+* PLOS journal: plos_article 
+* Proceedings of the National Academy of Sciences of the USA: pnas_article
+
+In order to create a document with one of those templates, use the following command:
+
+```
+gknit-draft --filename <my_document> --template <template> --package <package>
+            --create_dir
+```
+So, in order to create a template for writing an R Journal, use:
+
+```
+gknit-draft --filename my_r_article --template rjournal_article --package rticles
+            --create_dir
+```
+
+# Accessing R variables
+
+Galaaz allows Ruby to access variables created in R.  For example, the `mtcars` data set is 
+available in R and can be accessed from Ruby by using the tilde operator followed by the
+symbol for the variable, in this case `:mtcars`.  In the code below, method `outputs` is 
+used to output the `mtcars` data set nicely formatted in HTML by use of the `kable` and
+`kable_styling` functions. Method `outputs` is only available when used with gKnit.
+
+```{ruby view_kable}
+outputs (~R[:mtcars]).kable.kable_styling
+```
+
+# Basic Data Types
+
+## Vector
 
 Vectors can be thought of as contiguous cells containing data. Cells are accessed through
 indexing operations such as x[5]. Galaaz has six basic (‘atomic’) vector types: logical, 
@@ -116,7 +1296,7 @@ table.
 | logical   | logical   |      logical |
 | integer   | numeric   |      integer |
 | double    | numeric   |       double |
-| complex   | complex   |      comples |
+| complex   | complex   |      complex |
 | character | character |    character |
 | raw       | raw       |          raw |
 
@@ -178,7 +1358,7 @@ vec = R.c(true, true, false, false, true)
 puts vec
 ```
 
-## Combining Vectors
+### Combining Vectors
 
 The 'c' functions used to create vectors can also be used to combine two vectors:
 
@@ -193,14 +1373,14 @@ In this next example, method 'c' is chainned after 'vec1'.  This also looks like
 method of the vector, but in reallity, this is actually closer to the pipe operator.  When
 Galaaz identifies that 'c' is not a method of 'vec' it actually tries to call 'R.c' with 
 'vec1' as the first argument concatenated with all the other available arguments.  The code
-bellow is automatically converted to the code above.
+below is automatically converted to the code above.
 
 ```{ruby chainning_methods}
 vec = vec1.c(vec2)
 puts vec
 ```
 
-## Vector Arithmetic
+### Vector Arithmetic
 
 Arithmetic operations on vectors are performed element by element:
 
@@ -219,7 +1399,7 @@ vec3 = R.c(1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0)
 puts vec4 = vec1 + vec3
 ```
 
-## Vector Indexing
+### Vector Indexing
 
 Vectors can be indexed by using the '[]' operator:
 
@@ -227,7 +1407,7 @@ Vectors can be indexed by using the '[]' operator:
 puts vec4[3]
 ```
 
-We can also index a vector with another vector.  For example, in the code bellow, we take elements
+We can also index a vector with another vector.  For example, in the code below, we take elements
 1, 3, 5, and 7 from vec3:
 
 ```{ruby index_by_vector}
@@ -275,7 +1455,7 @@ full_name = R.c(First: "Rodrigo", Middle: "A", Last: "Botafogo")
 puts full_name
 ```
 
-## Extracting Native Ruby Types from a Vector
+### Extracting Native Ruby Types from a Vector
 
 Vectors created with 'R.c' are of class R::Vector.  You might have noticed that when indexing a
 vector, a new vector is returned, even if this vector has one single element. In order to use
@@ -290,19 +1470,7 @@ puts vec4 >> 4
 
 Note that indexing with '>>' starts at 0 and not at 1, also, we cannot do negative indexing.
 
-# Accessing R variables
-
-Galaaz allows Ruby to access variables created in R.  For example, the 'mtcars' data set is 
-available in R and can be accessed from Ruby by using the 'tilda' operator followed by the
-symbol for the variable, in this case ':mtcar'.  In the code bellow method 'outputs' is 
-used to output the 'mtcars' data set nicely formatted in HTML by use of the 'kable' and
-'kable_styling' functions. Method 'outputs' is only available when used with 'gknit'.
-
-```{ruby view_kable}
-outputs (~:mtcars).kable.kable_styling
-```
-
-# Matrix
+## Matrix
 
 A matrix is a collection of elements organized as a two dimensional table.  A matrix can be 
 created by the 'matrix' function:
@@ -326,7 +1494,7 @@ mat_row = R.matrix(R.c(1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0),
 puts mat_row
 ```
 
-## Indexing a Matrix
+### Indexing a Matrix
 
 A matrix can be indexed by [row, column]:
 
@@ -360,7 +1528,7 @@ and 'cbind':
 puts mat_row.cbind(mat)
 ```
 
-# List
+## List
 
 A list is a data structure that can contain sublists of different types, while vector and matrix
 can only hold one type of element.
@@ -376,7 +1544,7 @@ puts lst
 Note that 'lst' elements are named elements.
 
 
-## List Indexing
+### List Indexing
 
 List indexing, also called slicing, is done using the '[]' operator and the '[[]]' operator. Let's
 first start with the '[]' operator. The list above has three sublist indexing with '[]' will 
@@ -406,11 +1574,11 @@ then the first element of the vector was extracted (note that vectors also accep
 operator) and then the vector was indexed by its first element, extracting the native Ruby type.
 
 
-# Data Frame
+## Data Frame
 
 A data frame is a table like structure in which each column has the same number of 
 rows. Data frames are the basic structure for storing data for data analysis.  We have already
-seen a data frame previously when we accessed variable '~:mtcars'.  In order to create a
+seen a data frame previously when we accessed variable '~R[:mtcars]'.  In order to create a
 data frame, function 'data__frame' is used:
 
 ```{ruby dataframe}
@@ -421,51 +1589,51 @@ df = R.data__frame(
 puts df
 ```
 
-## Data Frame Indexing
+### Data Frame Indexing
 
 A data frame can be indexed the same way as a matrix, by using '[row, column]', where row and
 column can either be a numeric or the name of the row or column
 
 ```{ruby dataframe_index}
-puts (~:mtcars).head
-puts (~:mtcars)[1, 2]
-puts (~:mtcars)['Datsun 710', 'mpg']
+puts (~R[:mtcars]).head
+puts (~R[:mtcars])[1, 2]
+puts (~R[:mtcars])['Datsun 710', 'mpg']
 ```
 
 Extracting a column from a data frame as a vector can be done by using the double square bracket
 operator:
 
 ```{ruby dataframe_column}
-puts (~:mtcars)[['mpg']]
+puts (~R[:mtcars])[['mpg']]
 ```
 
 A data frame column can also be accessed as if it were an instance variable of the data frame:
 
 ```{ruby dataframe_instance_variable}
-puts (~:mtcars).mpg
+puts (~R[:mtcars]).mpg
 ```
 
 Slicing a data frame can be done by indexing it with a vector (we use 'head' to reduce the
 output):
 
 ```{ruby dataframe_column_slice}
-puts (~:mtcars)[R.c('mpg', 'hp')].head
+puts (~R[:mtcars])[R.c('mpg', 'hp')].head
 ```
 
 A row slice can be obtained by indexing by row and using the ':all' keyword for the column:
 
 ```{ruby dataframe_row_slice}
-puts (~:mtcars)[R.c('Datsun 710', 'Camaro Z28'), :all]
+puts (~R[:mtcars])[R.c('Datsun 710', 'Camaro Z28'), :all]
 ```
 
 Finally, a data frame can also be indexed with a logical vector.  In this next example, the
 'am' column of :mtcars is compared with 0 (with method 'eq').  When 'am' is equal to 0 the
-car is automatic.  So, by doing '(~:mtcars).am.eq 0' a logical vector is created with 
+car is automatic.  So, by doing '(~R[:mtcars]).am.eq 0' a logical vector is created with 
 'true' whenever 'am' is 0 and 'false' otherwise.
 
 ```{ruby logical_vector_filter}
 # obtain a vector with 'true' for cars with automatic transmission
-automatic = (~:mtcars).am.eq 0
+automatic = (~R[:mtcars]).am.eq 0
 puts automatic
 ```
 
@@ -474,7 +1642,7 @@ which all cars have automatic transmission.
 
 ```{ruby dataframe_logical}
 # slice the data frame by using this vector
-puts (~:mtcars)[automatic, :all]
+puts (~R[:mtcars])[automatic, :all]
 ```
 
 # Writing Expressions in Galaaz
@@ -484,24 +1652,24 @@ Galaaz extends Ruby to work with complex expressions, similar to R's expressions
 
 ## Expressions from operators
 
-The code bellow 
+The code below 
 creates an expression summing two symbols
 
 ```{ruby expressions}
-exp1 = :a + :b
+exp1 = R[:a] + R[:b]
 puts exp1
 ```
 We can build any complex mathematical expression
 
 ```{ruby expr2}
-exp2 = (:a + :b) * 2.0 + :c ** 2 / :z
+exp2 = (R[:a] + R[:b]) * 2.0 + R[:c] ** 2 / R[:z]
 puts exp2
 ```
 
 It is also possible to use inequality operators in building expressions
 
 ```{ruby expr3}
-exp3 = (:a + :b) >= :z
+exp3 = (R[:a] + R[:b]) >= :z
 puts exp3
 ```
 
@@ -510,7 +1678,7 @@ notation for those operators such as (.gt, .ge, etc.).  So the same expression w
 above can also be written as
 
 ```{ruby expr4}
-exp4 = (:a + :b).ge :z
+exp4 = (R[:a] + R[:b]).ge :z
 puts exp4
 ```
 
@@ -519,23 +1687,23 @@ those are expressions involving '==', and '='.  In order to write an expression
 need to use the method '.eq' and for '=' we need the function '.assign'
 
 ```{ruby expr5}
-exp5 = (:a + :b).eq :z
+exp5 = (R[:a] + R[:b]).eq :z
 puts exp5
 ```
 
 ```{ruby expr6}
-exp6 = :y.assign :a + :b
+exp6 = R[:y].assign R[:a] + R[:b]
 puts exp6
 ```
 In general we think that using the functional notation is preferable to using the 
 symbolic notation as otherwise, we end up writing invalid expressions such as
 
-```{ruby exp_wrong, warning=FALSE}
-exp_wrong = (:a + :b) == :z
+```{ruby exp_wrong, warning=FALSE, eval=FALSE}
+exp_wrong = (R[:a] + R[:b]) == :z
 puts exp_wrong
 ```
 and it might be difficult to understand what is going on here.  The problem lies with the fact that
-when using '==' we are comparing expression (:a + :b) to expression :z with '=='.  When the 
+when using '==' we are comparing expression (R[:a] + R[:b]) to expression :z with '=='.  When the 
 comparison is executed, the system tries to evaluate :a, :b and :z, and those symbols at 
 this time are not bound to anything and we get a "object 'a' not found" message.
 If we only use functional notation, this type of error will not occur.
@@ -550,21 +1718,21 @@ When we want the function to be part of the expression, we call the function pre
 by the letter E, such as 'E.sin(x)'
 
 ```{ruby method_expression}
-exp7 = :y.assign E.sin(:x)
+exp7 = R[:y].assign E.sin(R[:x])
 puts exp7
 ```
 
 Expressions can also be written using '.' notation:
 
 ```{ruby expression_with_dot}
-exp8 = :y.assign :x.sin
+exp8 = R[:y].assign R[:x].sin
 puts exp8
 ```
 
 When a function has multiple arguments, the first one can be used before the '.':
 
 ```{ruby expression_multiple_args}
-exp9 = :x.c(:y)
+exp9 = R[:x].c(R[:y])
 puts exp9
 ```
 
@@ -574,7 +1742,7 @@ Expressions can be evaluated by calling function 'eval' with a binding. A bindin
 with a list:
 
 ```{ruby eval_expression_list}
-exp = (:a + :b) * 2.0 + :c ** 2 / :z
+exp = (R[:a] + R[:b]) * 2.0 + R[:c] ** 2 / R[:z]
 puts exp.eval(R.list(a: 10, b: 20, c: 30, z: 40))
 ```
 
@@ -593,35 +1761,39 @@ puts exp.eval(df)
 # Manipulating Data
 
 One of the major benefits of Galaaz is to bring strong data manipulation to Ruby. The following
-examples were extracted from Hardley's "R for Data Science" (https://r4ds.had.co.nz/). This
+examples were extracted from Hadley's "R for Data Science" (https://r4ds.had.co.nz/). This
 is a highly recommended book for those not already familiar with the 'tidyverse' style of
 programming in R. In the sections to follow, we will limit ourselves to convert the R code to
 Galaaz. 
 
 For these
 examples, we will investigate the nycflights13 data set available on the package by the
-same name.  We use function 'R.install_and_loads' that checks if the library is available 
+same name.  We use function 'R.install\_and\_loads' that checks if the library is available 
 locally, and if not, installs it. This data frame contains all 336,776 flights that 
 departed from New York City in 2013. The data comes from the US Bureau of 
 Transportation Statistics.
 
+Dplyr often uses **tibbles** in place of classic data frames. In Galaaz, printing may differ from
+the R console; if you need a classic tabular printout, convert with **`as__data__frame`** (or use
+`head` / `str` in R via `R` calls).
+
 ```{ruby nycflights13}
 R.install_and_loads('nycflights13')
 R.library('dplyr')
 ```
 
 ```{ruby flights}
-flights = ~:flights
-puts flights.head.as__data__frame
+flights = ~R[:flights]
+puts flights.head
 ```
 
 ## Filtering rows with Filter
 
 In this example we filter the flights data set by giving to the filter function two expressions:
-the first :month.eq 1
+the first R[:month].eq 1
 
 ```{ruby filter_rows}
-puts flights.filter((:month.eq 1), (:day.eq 1)).head.as__data__frame
+puts flights.filter((R[:month].eq 1), (R[:day].eq 1)).head
 ```
 
 ## Logical Operators
@@ -629,7 +1801,7 @@ puts flights.filter((:month.eq 1), (:day.eq 1)).head.as__data__frame
 All flights that departed in November of December
 
 ```{ruby nov_dec}
-puts flights.filter((:month.eq 11) | (:month.eq 12)).head.as__data__frame
+puts flights.filter((R[:month].eq 11) | (R[:month].eq 12)).head
 ```
 
 The same as above, but using the 'in' operator. In R, it is possible to define many operators
@@ -638,7 +1810,7 @@ operators from Galaaz the '._' method is used, where the first argument is the o
 symbol, in this case ':in' and the second argument is the vector:
 
 ```{ruby in_op}
-puts flights.filter(:month._ :in, R.c(11, 12)).head.as__data__frame
+puts flights.filter(R[:month]._ :in, R.c(11, 12)).head
 ```
 
 ## Filtering with NA (Not Available)
@@ -650,20 +1822,20 @@ what is obtained from data frame.
 
 ```{ruby na_tibble}
 df = R.tibble(x: R.c(1, R::NA, 3))
-puts df.as__data__frame
+puts df
 ```
 
 Now filtering by :x > 1 shows all lines that satisfy this condition, where the row with R:NA does
 not.
 
 ```{ruby filter_na}
-puts df.filter(:x > 1).as__data__frame
+puts df.filter(R[:x] > 1)
 ```
 
 To match an NA use method 'is__na'
 
 ```{ruby with_na}
-puts df.filter((:x.is__na) | (:x > 1)).as__data__frame
+puts df.filter((R[:x].is__na) | (R[:x] > 1))
 ```
 
 ## Arrange Rows with arrange
@@ -671,13 +1843,13 @@ puts df.filter((:x.is__na) | (:x > 1)).as__data__frame
 Arrange reorders the rows of a data frame by the given arguments.
 
 ```{ruby arrange}
-puts flights.arrange(:year, :month, :day).head.as__data__frame
+puts flights.arrange(:year, :month, :day).head
 ```
 
 To arrange in descending order, use function 'desc'
 
 ```{ruby desc_arrange}
-puts flights.arrange(:dep_delay.desc).head.as__data__frame
+puts flights.arrange(R[:dep_delay].desc).head
 ```
 
 ## Selecting columns
@@ -685,19 +1857,19 @@ puts flights.arrange(:dep_delay.desc).head.as__data__frame
 To select specific columns from a dataset we use function 'select':
 
 ```{ruby select}
-puts flights.select(:year, :month, :day).head.as__data__frame
+puts flights.select(:year, :month, :day).head
 ```
 
 It is also possible to select column in a given range
 
 ```{ruby select_range}
-puts flights.select(:year.up_to :day).head.as__data__frame
+puts flights.select(R[:year].up_to(R[:day])).head
 ```
 
 Select all columns that start with a given name sequence
 
 ```{ruby select_starts_with}
-puts flights.select(E.starts_with('arr')).head.as__data__frame
+puts flights.select(E.starts_with('arr')).head
 ```
 
 Other functions that can be used:
@@ -714,26 +1886,26 @@ Other functions that can be used:
 A helper function that comes in handy when we just want to rearrange column order is 'Everything':
 
 ```{ruby everything}
-puts flights.select(:year, :month, :day, E.everything).head.as__data__frame
+puts flights.select(:year, :month, :day, E.everything).head
 ```
 
 ## Add variables to a dataframe with 'mutate'
 
 ```{ruby small_flights}
 flights_sm = flights.
-               select((:year.up_to :day),
+               select((R[:year].up_to(R[:day])),
                       E.ends_with('delay'),
                       :distance,
                       :air_time)
 
-puts flights_sm.head.as__data__frame
+puts flights_sm.head
 ```
 
 ```{ruby mutate}
 flights_sm = flights_sm.
-               mutate(gain: :dep_delay - :arr_delay,
-                      speed: :distance / :air_time * 60)
-puts flights_sm.head.as__data__frame
+               mutate(gain: R[:dep_delay] - R[:arr_delay],
+                      speed: R[:distance] / R[:air_time] * 60)
+puts flights_sm.head
 ```
 
 ## Summarising data
@@ -742,14 +1914,14 @@ Function 'summarise' calculates summaries for the data frame. When no 'group_by'
 a single value is obtained from the data frame:
 
 ```{ruby summarise}
-puts flights.summarise(delay: E.mean(:dep_delay, na__rm: true)).as__data__frame
+puts flights.summarise(delay: E.mean(:dep_delay, na__rm: true))
 ```
 
-When a data frame is groupe with 'group_by' summaries apply to the given group:
+When a data frame is grouped with 'group_by' summaries apply to the given group:
 
 ```{ruby summarise_group_by}
 by_day = flights.group_by(:year, :month, :day)
-puts by_day.summarise(delay: :dep_delay.mean(na__rm: true)).head.as__data__frame
+puts by_day.summarise(delay: R[:dep_delay].mean(na__rm: true)).head
 ```
 
 Next we put many operations together by pipping them one after the other:
@@ -759,23 +1931,25 @@ delays = flights.
            group_by(:dest).
            summarise(
              count: E.n,
-             dist: :distance.mean(na__rm: true),
-             delay: :arr_delay.mean(na__rm: true)).
-           filter(:count > 20, :dest != "NHL")
+             dist: R[:distance].mean(na__rm: true),
+             delay: R[:arr_delay].mean(na__rm: true)).
+           filter(R[:count] > 20, R[:dest] != "NHL")
 
-puts delays.as__data__frame.head
+puts delays.head
 ```
 
 # Using Data Table
 
+The next chunk converts the **nycflights13** `flights` tibble already loaded above into a
+**`data.table`**. That keeps the manual offline and avoids downloading a remote CSV during gknit
+(network stalls look like bridge hangs when the transfer runs inside a single R eval).
+
 ```{ruby fread}
 R.library('data.table')
-R.install_and_loads('curl')
 
-input = "https://raw.githubusercontent.com/Rdatatable/data.table/master/vignettes/flights14.csv"
-flights = R.fread(input)
-puts flights
+flights = R.as__data__table(~R[:flights])
 puts flights.dim
+puts R.head(flights, 12)
 ```
 
 ```{ruby data_table}
@@ -793,7 +1967,7 @@ puts data_table.ID
 
 ```{ruby subset_i}
 # subset rows in i
-ans = flights[(:origin.eq "JFK") & (:month.eq 6)]
+ans = flights[(R[:origin].eq "JFK") & (R[:month].eq 6)]
 puts ans.head
 
 # Get the first two rows from flights.
@@ -801,8 +1975,7 @@ puts ans.head
 ans = flights[(1..2)]
 puts ans
 
-# Sort flights first by column origin in ascending order, and then by dest in descending order:
-
+# Sort by origin asc, then dest desc (example kept commented):
 # ans = flights[E.order(:origin, -(:dest))]
 # puts ans.head
 
@@ -815,66 +1988,274 @@ puts ans
 ans = flights[:all, :arr_delay]
 puts ans.head
 
-# Select arr_delay column, but return as a data.table instead.
+# arr_delay as data.table (not plain vector).
 
-ans = flights[:all, :arr_delay.list]
+ans = flights[:all, R[:arr_delay].list]
 puts ans.head
 
-ans = flights[:all, E.list(:arr_delay, :dep_delay)]
+ans = flights[:all, E.list(R[:arr_delay], R[:dep_delay])]
+```
+
+# Apache Arrow
+
+[Apache Arrow](https://arrow.apache.org/) is a **columnar** in-memory format used heavily in R
+and Python for analytics. In Galaaz, **Ruby does not hold an Arrow C++ table itself**; instead you
+build ordinary Ruby structures (arrays of row hashes), and **`R::Arrow.from_ruby_batches`** creates
+a real **Arrow `Table` inside GNU R**. From there you use R’s **`arrow`** and **`dplyr`** packages
+as usual: **`group_by`** on the Arrow table, **`summarise`** for aggregates, then **`collect()`** to
+materialize a tibble when you need in-memory R rows.
+
+That pattern matches production use: **JRuby threads** (or sequential code) assemble many rows in
+Ruby; you pay **one** bridge-heavy handoff to R; **dplyr** runs vectorised work on the Arrow table
+in R.
+
+**Prerequisites:** install R packages **`arrow`** and **`dplyr`**. Run scripts with
+**`bin/galaaz-jruby`** (or the same JVM flags as in **`docs/testing.md`**) so the Arrow JNI stack is
+available.
+
+## Other `R::Arrow` helpers
+
+The Ruby module **`R::Arrow`** (see `lib/R_interface/r_arrow.rb`) also includes:
+
+* **`R::Arrow.table_from(df)`** — wrap an R `data.frame` / tibble as an Arrow table.
+* **`R::Arrow.read_feather` / `write_feather`**, **`read_parquet`**, **`dataset(path)`** — file and
+  dataset IO on paths visible to R.
+
+## Example: many Ruby rows → Arrow in R → grouped statistics
+
+The repository test **`slow-specs/arrow_large_pipeline_spec.rb`** builds **200k rows** in parallel
+(eight threads × 25,000 rows), pushes them through **`R::Arrow.from_ruby_batches`**, then checks that
+**dplyr** group summaries match a Ruby reference calculation. The same logic appears below at a
+**smaller scale** so this manual can knit quickly; increase `thread_count` and `rows_per_thread`
+when experimenting locally.
+
+```{ruby arrow_pipeline_example, message=FALSE, warning=FALSE}
+# Scaled-down version of slow-specs/arrow_large_pipeline_spec.rb.
+unless R::Support.eval("requireNamespace('arrow', quietly=TRUE) && requireNamespace('dplyr', quietly=TRUE)") == true
+  puts '(Skip: need arrow + dplyr in R; use bin/galaaz-jruby outside gKnit.)'
+else
+  thread_count = 4
+  rows_per_thread = 500
+  group_count = 5
+
+  batches = []
+  mutex = Mutex.new
+  threads = []
+
+  thread_count.times do |tid|
+    threads << Thread.new do
+      start = tid * rows_per_thread
+      local = (start...(start + rows_per_thread)).map do |i|
+        {
+          id: i,
+          grp: "g#{i % group_count}",
+          value: (i % 17) + 1,
+          weight: ((i % 5) + 1) * 0.5
+        }
+      end
+      mutex.synchronize { batches << local }
+    end
+  end
+  threads.each(&:join)
+
+  tbl = R::Arrow.from_ruby_batches(batches)
+  puts "R class after from_ruby_batches: #{tbl.rclass}"
+
+  grouped = R.dplyr___group_by(tbl, :grp)
+  summarised = R.dplyr___summarise(
+    grouped,
+    n: E.n(),
+    total: E.sum(:value),
+    wsum: E.sum(R[:value] * R[:weight])
+  )
+  out = R.dplyr___collect(summarised)
+
+  puts 'Per-group summary (first rows):'
+  puts R.as__data__frame(out).head(10)
+
+  total_n = 0
+  (1..(out.nrow >> 0)).each { |i| total_n += (out[['n']][i] >> 0) }
+  puts "Sum of group counts n (should equal #{thread_count * rows_per_thread}): #{total_n}"
+end
+```
+
+**What to notice:** (1) Ruby only sees **`Hash`** rows and Ruby **`Thread`** objects; (2) a single
+**`from_ruby_batches`** call creates the Arrow table in R; (3) **`dplyr___group_by`** /
+**`dplyr___summarise`** / **`dplyr___collect`** mirror **`dplyr::group_by`** /
+**`dplyr::summarise`** / **`dplyr::collect`** on an Arrow-backed table. For a lighter test, see
+**`specs/arrow_from_ruby_batches_spec.rb`**; for the full-size benchmark, run
+**`bin/run_slow_rspec slow-specs/arrow_large_pipeline_spec.rb`**.
+
+# Bioconductor and DESeq2
+
+**Bioconductor** packages are ordinary R packages installed from the Bioconductor repositories.
+Galaaz does not treat them specially: once installed in **GNU R**, you load them with
+**`R.library`** like any CRAN package.
+
+## Installing Bioconductor packages
+
+From an R session (or `R -e '...'`), use **BiocManager** (see
+[bioconductor.org](https://bioconductor.org/install/)):
+
+```r
+if (!requireNamespace("BiocManager", quietly = TRUE))
+  install.packages("BiocManager")
+BiocManager::install(c("DESeq2", "airway"))
+```
+
+The **`airway`** package ships the example **`SummarizedExperiment`** used below. **DESeq2**
+pulls in several dependencies; the first install can take several minutes.
+
+## Example: DESeq2 on the airway dataset
+
+The script **`examples/bioconductor_deseq2_airway/deseq2_airway_galaaz.rb`** is the canonical
+version in the repository. Run it from the **Galaaz repository root** with JRuby, for example:
+
+```text
+bin/galaaz-jruby examples/bioconductor_deseq2_airway/deseq2_airway_galaaz.rb
+```
+
+The workflow in Ruby mirrors a standard DESeq2 vignette:
+
+1. **`R.library('DESeq2')`** and **`R.library('airway')`**, then **`R.data('airway')`** so the
+   object exists in R’s global environment.
+2. **`airway = ~R[:airway]`** pulls the experiment into a Galaaz wrapper so you can pass it to R
+   functions as a Ruby value.
+3. **`R.DESeqDataSet(..., design: (R[:all].til R[:cell] + R[:dex]))`** builds the **`DESeqDataSet`**. The
+   **`(R[:all].til R[:cell] + R[:dex])`** form is Galaaz’s way of passing the one-sided formula
+   **`~ cell + dex`** (adjust for the design you need).
+4. Prefilter rows with almost no counts: **`keep = R.rowSums(R.counts(dds)) >= 10`** and
+   **`dds = dds[keep, :all]`**.
+5. **`dds = R.DESeq(dds)`** fits the model; **`res = R.results(dds, contrast: R.c('dex', 'trt', 'untrt'))`**
+   extracts the treatment contrast (adjust **`contrast`** for your experiment).
+6. Summaries use normal Ruby string interpolation on **`R.nrow`**, **`R.ncol`**, **`R.colnames`**, etc.
+7. **`R.pdf(...); R.plotMA(res, ...); R.dev__off`** writes DESeq2’s MA plot (path is relative to the
+   process working directory—use the repo root when running the bundled script).
+
+Related benchmarks and warm-run notes live under **`docs/deseq2_airway_benchmark.md`** and
+**`examples/bioconductor_deseq2_airway/bench_*.rb`**.
+
+Below is the full listing (same as the file in the repository). It is **not** executed while this
+manual is knitted, because **DESeq2** is heavy and may be absent on the build machine.
+
+```{ruby deseq2_airway_full_listing, eval=FALSE}
+# Canonical script: examples/bioconductor_deseq2_airway/deseq2_airway_galaaz.rb
+# Run: bin/galaaz-jruby examples/.../deseq2_airway_galaaz.rb (repo root).
+
+require 'galaaz'
+
+R.library('DESeq2')
+R.library('airway')
+R.data('airway')
+
+airway = ~R[:airway]
+
+# Build DESeq2 dataset with one-sided formula: ~ cell + dex.
+dds = R.DESeqDataSet(airway, design: (R[:all].til R[:cell] + R[:dex]))
+
+# Prefilter genes with almost no counts.
+keep = R.rowSums(R.counts(dds)) >= 10
+dds = dds[keep, :all]
+
+# Fit DE model and extract treatment effect.
+dds = R.DESeq(dds)
+res = R.results(dds, contrast: R.c('dex', 'trt', 'untrt'))
+
+# Compact sanity outputs for quick verification.
+puts "Samples: #{R.ncol(dds)}"
+puts "Genes after prefilter: #{R.nrow(dds)}"
+puts "Result rows: #{R.nrow(res)}"
+puts "Result columns: #{R.colnames(res)}"
+puts "Significant genes (padj < 0.05): #{R.sum(res.padj < 0.05, na__rm: true)}"
+
+res_ordered = res[R.order(res.padj), :all]
+puts R.head(R.as__data__frame(res_ordered), 10)
+
+# Standard DESeq2 plot call written to file.
+R.pdf('examples/bioconductor_deseq2_airway/plotMA_galaaz.pdf')
+R.plotMA(res, ylim: R.c(-5, 5))
+R.dev__off
+```
+
+If **DESeq2** and **airway** are installed, the next chunk loads the data and prints a short
+preview (it does **not** run **`DESeq`** so the manual knits quickly).
+
+```{ruby deseq2_airway_smoke, message=FALSE, warning=FALSE}
+unless R::Support.eval("requireNamespace('DESeq2', quietly=TRUE) && requireNamespace('airway', quietly=TRUE)")
+  puts '(Skip: install DESeq2 and airway via BiocManager in R to run the full example.)'
+else
+  R.library('DESeq2')
+  R.library('airway')
+  R.data('airway')
+  airway = ~R[:airway]
+  puts 'airway object (head of assay / dims via R):'
+  puts "ncol(samples): #{R.ncol(airway)}"
+  puts R.head(R.assay(airway), 3)
+end
 ```
 
+# Performance
+
+For realistic analyses, **most wall-clock time is spent inside GNU R** (model fitting, I/O inside
+R, graphics). The Galaaz **bridge** adds overhead mainly from **starting a session**, **serializing
+requests**, and **wrapping results** in Ruby objects—not from reimplementing R’s numerical work.
+
+Practical tips:
+
+* Keep **hot loops** in R or vectorized code when possible; use Ruby for orchestration, I/O, and
+  glue.
+* **Reuse one process**: running many short scripts cold-starts Ruby, the JVM, and R each time;
+  a long-lived process or repeated calls in one run amortize setup (see benchmarks below).
+* **Batch data**: merge shards in Ruby, then call **`R::Arrow.from_ruby_batches`** (or build one
+  data frame) instead of millions of tiny R calls.
+
+For measured discussion (including DESeq2-style workloads and warm comparisons), see
+**`docs/performance.md`** and **`docs/deseq2_airway_benchmark.md`** in the Galaaz repository.
+
 # Graphics in Galaaz
 
 Creating graphics in Galaaz is quite easy, as it can use all the power of ggplot2.  There are
-many resources in the web that teaches ggplot, so here we give a quick example of ggplot 
+many resources on the web that teach ggplot, so here we give a quick example of ggplot 
 integration with Ruby.  We continue to use the :mtcars dataset and we will plot a diverging
-bar plot, showing cars that have 'above' or 'below' gas consuption. Let's first prepare
+bar plot, showing cars that have 'above' or 'below' gas consumption. Let's first prepare
 the data frame with the necessary data:
 
 ```{ruby diverging_plot_pre}
-# copy the R variable :mtcars to the Ruby mtcars variable
-mtcars = ~:mtcars
-
-# create a new column 'car_name' to store the car names so that it can be
-# used for plotting. The 'rownames' of the data frame cannot be used as
-# data for plotting
-mtcars.car_name = R.rownames(:mtcars)
-
-# compute normalized mpg and add it to a new column called mpg_z
-# Note that the mean value for mpg can be obtained by calling the 'mean'
-# function on the vector 'mtcars.mpg'.  The same with the standard
-# deviation 'sd'.  The vector is then rounded to two digits with 'round 2'
+# :mtcars -> Ruby handle
+mtcars = ~R[:mtcars]
+
+# Row labels are not a plot column; copy them to car_name.
+mtcars.car_name = R.rownames(R[:mtcars])
+
+# Z-score mpg (mean/sd on mtcars.mpg); round to 2 decimals.
 mtcars.mpg_z = ((mtcars.mpg - mtcars.mpg.mean)/mtcars.mpg.sd).round 2
 
-# create a new column 'mpg_type'. Function 'ifelse' is a vectorized function
-# that looks at every element of the mpg_z vector and if the value is below
-# 0, returns 'below', otherwise returns 'above'
+# ifelse is vectorized: below / above average mpg_z.
 mtcars.mpg_type = (mtcars.mpg_z < 0).ifelse("below", "above")
 
-# order the mtcar data set by the mpg_z vector from smaler to larger values
+# Sort rows by mpg_z.
 mtcars = mtcars[mtcars.mpg_z.order, :all]
 
-# convert the car_name column to a factor to retain sorted order in plot
+# Factor car_name so plot order follows sort.
 mtcars.car_name = mtcars.car_name.factor levels: mtcars.car_name
 
-# let's look at the final data frame
 puts mtcars.head
 ```
-Now, lets plot the diverging bar plot.  When using gKnit, there is no need to call
-'R.awt' to create a plotting device, since gKnit does take care of it. Galaaz 
+Now, let's plot the diverging bar plot.  When using gKnit, you normally do **not** need to open a
+graphics device manually; gKnit arranges the figure device for chunk output. Galaaz 
 provides integration with ggplot. The interested reader should check online for more
 information on ggplot, since it is outside the scope of this manual describing 
-how ggplot works. We give here but a brief description on how this plot is generated.
+how ggplot works. Here we give only a brief description of how this plot is generated.
 
-ggplot implements the 'grammar of graphics'. In this approach, plots are build by
+ggplot implements the 'grammar of graphics'. In this approach, plots are built by
 adding layers to the plot.  On the first layer we describe what we want on the 'x'
 and 'y' axis of the plot.  In this case, we have 'car_name' on the 'x' axis and 
 'mpg\_z' on the 'y' axis. Then the type of graph is specified by adding
 'geom\_bar' (for a bar graph).  We specify that our bars should be filled using 
-'mpg\_type', which is either 'above' or 'bellow' giving then two colours for
+'mpg\_type', which is either 'above' or 'below' giving then two colours for
 filling. On the next layer we specify the labels for the graph, then we add the
 title and subtitle.  Finally, in a bar chart usually bars go on the vertical direction,
-but in this graph we want the bars to be horizontally layed so we add 'coord\_flip'.
+but in this graph we want the bars to be horizontally laid so we add 'coord\_flip'.
 
 ```{ruby diverging_bar, fig.width = 9.1, fig.height = 6.5}
 require 'ggplot'
@@ -892,7 +2273,7 @@ puts mtcars.ggplot(E.aes(x: :car_name, y: :mpg_z, label: :mpg_z)) +
 # Coding with Tidyverse
 
 In R, and when coding with 'tidyverse', arguments to a function are usually not 
-*referencially transparent*. That is, you can’t replace a value with a seemingly equivalent 
+*referentially transparent*. That is, you can’t replace a value with a seemingly equivalent 
 object that you’ve defined elsewhere. To see the problem, let's first define a data frame:
 
 ```{ruby df}
@@ -908,17 +2289,17 @@ filter(df, my_var == 1)
 ```
 It generates the following error: "object 'x' not found.
 
-However, in Galaaz, arguments are referencially transparent as can be seen by the 
-code bellow.  Note initally that 'my_var = :x' will not give the error "object 'x' not found" 
+However, in Galaaz, arguments are referentially transparent as can be seen by the 
+code below.  Note initially that 'my_var = R[:x]' will not give the error "object 'x' not found" 
 since ':x' is treated as an expression and assigned to my\_var. Then when doing (my\_var.eq 1), 
-my\_var is a variable that resolves to ':x' and it becomes equivalent to (:x.eq 1) which is
+my\_var is a variable that resolves to ':x' and it becomes equivalent to (R[:x].eq 1) which is
 what we want.
 
 ```{ruby my_var}
-my_var = :x
+my_var = R[:x]
 puts df.filter(my_var.eq 1)
 ```
-As stated by Hardley
+As stated by Hadley
 
 > dplyr code is ambiguous. Depending on what variables are defined where, 
 > filter(df, x == y) could be equivalent to any of:
@@ -930,9 +2311,9 @@ df[x == df$y, ]
 df[x == y, ]
 ```
 In galaaz this ambiguity does not exist, filter(df, x.eq y) is not a valid expression as 
-expressions are build with symbols.  In doing filter(df, :x.eq y) we are looking for elements
+expressions are build with symbols.  In doing filter(df, R[:x].eq y) we are looking for elements
 of the 'x' column that are equal to a previously defined y variable.  Finally in 
-filter(df, :x.eq :y) we are looking for elements in which the 'x' column value is equal to
+filter(df, R[:x].eq R[:y]) we are looking for elements in which the 'x' column value is equal to
 the 'y' column value. This can be seen in the following two chunks of code:
 
 ```{ruby disamb1}
@@ -940,13 +2321,13 @@ y = 1
 x = 2
 
 # looking for values where the 'x' column is equal to the 'y' column
-puts df.filter(:x.eq :y)
+puts df.filter(R[:x].eq R[:y])
 ```
 
 ```{ruby disamb2}
 # looking for values where the 'x' column is equal to the 'y' variable
 # in this case, the number 1
-puts df.filter(:x.eq y)
+puts df.filter(R[:x].eq y)
 ```
 ## Writing a function that applies to different data sets
 
@@ -973,11 +2354,12 @@ Unfortunately, in R, this function can fail silently if one of the variables isn
 in the data frame, but is present in the global environment.  We will not go through here how
 to solve this problem in R.
 
-In Galaaz the method mutate_y bellow will work fine and will never fail silently.
+In Galaaz the method mutate_y below will work fine and will never fail silently.
 
 ```{ruby mutate_y, warning=FALSE}
 def mutate_y(df)
-  df.mutate(:y.assign :a + :x)
+  # Mutate column names are Ruby kwargs (y: …). Use .assign only for R `<-` expressions.
+  df.mutate(y: R[:a] + R[:x])
 end
 ```
 Here we create a data frame that has only one column named 'x':
@@ -987,8 +2369,8 @@ df1 = R.data__frame(x: (1..3))
 puts df1
 ```
 
-Note that method mutate_y will fail independetly from the fact that variable 'a' is defined and
-in the scope of the method.  Variable 'a' has no relationship with the symbol ':a' used in the
+Note that method mutate_y will fail independently from the fact that variable 'a' is defined and
+in the scope of the method.  Variable 'a' has no relationship with the symbol `R[:a]` used in the
 definition of 'mutate\_y' above:
 
 ```{ruby call_mutate_y, warning = FALSE}
@@ -997,12 +2379,13 @@ mutate_y(df1)
 ```
 ## Different expressions
 
-Let's move to the next problem as presented by Hardley where trying to write a function in R 
+Let's move to the next problem as presented by Hadley where trying to write a function in R 
 that will receive two argumens, the first a variable and the second an expression is not trivial.
-Bellow we create a data frame and we want to write a function that groups data by a variable and
+Below we create a data frame and we want to write a function that groups data by a variable and
 summarises it by an expression:
 
 ```{r diff_expr}
+library(dplyr)
 set.seed(123)
 
 df <- data.frame(
@@ -1027,7 +2410,7 @@ d2 <- df %>%
 as.data.frame(d2)	       
 ```
 
-As shown by Hardley, one might expect this function to do the trick:
+As shown by Hadley, one might expect this function to do the trick:
 
 ```{r diff_exp_fnc}
 my_summarise <- function(df, group_var) {
@@ -1042,32 +2425,32 @@ my_summarise <- function(df, group_var) {
 
 In order to solve this problem, coding with dplyr requires the introduction of many new concepts
 and functions such as 'quo', 'quos', 'enquo', 'enquos', '!!' (bang bang), '!!!' (triple bang). 
-Again, we'll leave to Hardley the explanation on how to use all those functions.
+Again, we'll leave to Hadley the explanation on how to use all those functions.
 
 Now, let's try to implement the same function in galaaz.  The next code block first prints the
-'df' data frame defined previously in R (to access an R variable from Galaaz, we use the tilda 
-operator '~' applied to the R variable name as symbol, i.e., ':df'.
+'df' data frame defined previously in R (to access an R variable from Galaaz, we use the tilde
+operator `~` applied to the R variable name as a symbol, e.g. `:df`).
 
 ```{ruby r_dataframe}
-puts ~:df
+puts ~R[:df]
 ```
 
 We then create the 'my_summarize' method and call it passing the R data frame and 
-the group by variable ':g1':
+the group by variable 'R[:g1]':
 
 ```{ruby diff_exp_ruby_func}
 def my_summarize(df, group_var)
   df.group_by(group_var).
-    summarize(a: :a.mean)
+    summarize(a: R[:a].mean)
 end
 
-puts my_summarize(:df, :g1).as__data__frame
+puts my_summarize(~R[:df], R[:g1])
 ```
 
 It works!!! Well, let's make sure this was not just some coincidence
 
 ```{ruby group_g2}
-puts my_summarize(:df, :g2).as__data__frame
+puts my_summarize(~R[:df], R[:g2])
 ```
 
 Great, everything is fine! No magic, no new functions, no complexities, just normal, standard Ruby
@@ -1079,7 +2462,7 @@ In the previous section we've managed to get rid of all NSE formulation for a si
 does this remain true for more complex examples, or will the Galaaz way prove inpractical for
 more complex code?
 
-In the next example Hardley proposes us to write a function that given an expression such as 'a'
+In the next example Hadley proposes us to write a function that given an expression such as 'a'
 or 'a * b', calculates three summaries.  What we want a function that does the same as these R
 statements:
 
@@ -1108,9 +2491,9 @@ def my_summarise2(df, expr)
   )
 end
 
-puts my_summarise2((~:df), :a)
+puts my_summarise2((~R[:df]), :a)
 puts "\n"
-puts my_summarise2((~:df), :a * :b)
+puts my_summarise2((~R[:df]), R[:a] * R[:b])
 ```
 
 Once again, there is no need to use any special theory or functions.  The only point to be 
@@ -1118,7 +2501,7 @@ careful about is the use of 'E' to build expressions from functions 'mean', 'sum
 
 ## Different input and output variable
 
-Now the next challenge presented by Hardley is to vary the name of the output variables based on 
+Now the next challenge presented by Hadley is to vary the name of the output variables based on 
 the received expression.  So, if the input expression is 'a', we want our data frame columns to
 be named 'mean\_a' and 'sum\_a'.  Now, if the input expression is 'b', columns
 should be named 'mean\_b' and 'sum\_b'.
@@ -1144,7 +2527,7 @@ mutate(df, mean_b = mean(b), sum_b = sum(b))
 #> 4     2     2     5     4      3    15
 #> # … with 1 more row
 ```
-In order to solve this problem in R, Hardley needs to introduce some more new functions and notations:
+In order to solve this problem in R, Hadley needs to introduce some more new functions and notations:
 'quo_name' and the ':=' operator from package 'rlang'
 
 Here is our Ruby code:
@@ -1158,9 +2541,9 @@ def my_mutate(df, expr)
             sum_name => E.sum(expr))
 end
 
-puts my_mutate((~:df), :a)
+puts my_mutate((~R[:df]), :a)
 puts "\n"
-puts my_mutate((~:df), :b)
+puts my_mutate((~R[:df]), :b)
 ```
 It really seems that "Non Standard Evaluation" is actually quite standard in Galaaz! But, you 
 might have noticed a small change in the way the arguments to the mutate method were called.
@@ -1172,7 +2555,7 @@ and variable mean\_name is not followed by ':' but by '=>'.  This is standard Ru
 
 ## Capturing multiple variables
 
-Moving on with new complexities, Hardley proposes us to solve the problem in which the 
+Moving on with new complexities, Hadley proposes us to solve the problem in which the 
 summarise function will receive any number of grouping variables.
 
 This again is quite standard Ruby.  In order to receive an undefined number of paramenters
@@ -1184,7 +2567,7 @@ def my_summarise3(df, *group_vars)
     summarise(a: E.mean(:a))
 end
 
-puts my_summarise3((~:df), :g1, :g2).as__data__frame
+puts my_summarise3((~R[:df]), R[:g1], R[:g2])
 ```
 
 ## Why does R require NSE and Galaaz does not?
@@ -1202,7 +2585,7 @@ In Ruby, there is no lazy evaluation of parameters and 'a' is always a variable
 Variables assume their value as soon as they are used, so 'x = a' is immediately evaluate and 
 variable 'x' will receive the value of variable 'a' as soon as the Ruby statement is executed. 
 Ruby also provides the notion of a symbol; ':a' is a symbol and does not evaluate to anything.
-Galaaz uses Ruby symbols to build expressions that are not bound to anything: ':a.eq :b' is
+Galaaz uses Ruby symbols to build expressions that are not bound to anything: 'R[:a].eq R[:b]' is
 clearly an expression and has no relationship whatsoever with the statment 'a = b'. By using
 symbols, variables and expressions all the possible ambiguities that are found in R are 
 eliminated in Galaaz.
@@ -1212,7 +2595,7 @@ of input they are expecting, they might be expecting regular variables or they m
 expecting expressions and the R function will know how to deal with an input of the form 
 'a = b', now for the Ruby developer it might not be immediately clear if it should call the 
 function passing the value 'true' if variable 'a' is equal to variable 'b' or if it should
-call the function passing the expression ':a.eq :b'.
+call the function passing the expression 'R[:a].eq R[:b]'.
 
 
 ## Advanced dplyr features
@@ -1235,12 +2618,13 @@ In the following examples, we show the use of functions 'group\_by\_at', 'summar
 features of characters in the Starwars movies:
 
 ```{ruby starwars}
-puts (~:starwars).head.as__data__frame
+puts (~R[:starwars]).head
 ```
-The grouped_mean function bellow will receive a grouping variable and calculate summaries for
+The grouped_mean function below will receive a grouping variable and calculate summaries for
 the value\_variables given:
 
 ```{r grouped_mean}
+library(dplyr)
 grouped_mean <- function(data, grouping_variables, value_variables) {
   data %>%
     group_by_at(grouping_variables) %>%
@@ -1262,24 +2646,26 @@ def grouped_mean(data, grouping_variables, value_variables)
   data.
     group_by_at(grouping_variables).
     mutate(count: E.n).
-    summarise_at(E.c(value_variables, "count"), ~:mean, na__rm: true).
+    summarise_at(E.c(value_variables, "count"), ~R[:mean], na__rm: true).
     rename_at(value_variables, E.funs(E.paste0("mean_", value_variables)))
 end
 
-puts grouped_mean((~:starwars), "eye_color", E.c("mass", "birth_year")).as__data__frame
+puts grouped_mean((~R[:starwars]), "eye_color", E.c("mass", "birth_year"))
 ```
 
-
-[TO BE CONTINUED...]
-
+The examples above cover programmatic dplyr with string column names and `_at` helpers. The same
+Galaaz patterns (symbols, `E.*` for expression-safe functions, and Ruby methods on R-backed objects)
+extend to other tidyverse workflows; consult R package documentation for function-specific
+arguments.
 
 # Contributing
 
-
 * Fork it
-* Create your feature branch (git checkout -b my-new-feature)
-* Write Tests!
-* Commit your changes (git commit -am 'Add some feature')
-* Push to the branch (git push origin my-new-feature)
-* Create new Pull Request
-
+* Create your feature branch (`git checkout -b my-new-feature`)
+* Write tests — use **`bin/run_rspec`** or **`bin/run_all_rspec`** with **JRuby** so JVM flags and
+  the load path match **`docs/testing.md`**
+* Commit your changes (`git commit -am 'Add some feature'`)
+* Push to the branch (`git push origin my-new-feature`)
+* Open a pull request
+
+# References