galaaz 0.4.10 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (391) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +26 -0
  3. data/LICENSE +0 -0
  4. data/README.md +3123 -882
  5. data/Rakefile +62 -41
  6. data/bin/galaaz-bootstrap +137 -0
  7. data/bin/galaaz-jruby +14 -0
  8. data/bin/galaaz_jruby_env.inc.sh +6 -0
  9. data/bin/gbookdown +64 -0
  10. data/bin/gknit +223 -6
  11. data/bin/gknit-draft +105 -0
  12. data/bin/gknit-draft.rb +28 -0
  13. data/bin/gknit_Rscript +127 -0
  14. data/bin/grun +27 -1
  15. data/bin/gstudio +49 -4
  16. data/bin/{gstudio.rb → gstudio_irb.rb} +0 -0
  17. data/bin/gstudio_pry.rb +7 -0
  18. data/bin/install-tinytex +6 -0
  19. data/bin/run_all_rspec +43 -0
  20. data/bin/run_example +14 -0
  21. data/bin/run_old_rspec +19 -0
  22. data/bin/run_rspec +23 -0
  23. data/bin/run_rspec_subset +38 -0
  24. data/bin/run_slow_rspec +19 -0
  25. data/blogs/R-on-Rails-Planning-Document.md +940 -0
  26. data/blogs/README.md +100 -0
  27. data/blogs/galaaz_ggplot/galaaz_ggplot.Rmd +38 -66
  28. data/blogs/galaaz_ggplot/galaaz_ggplot.log +754 -0
  29. data/blogs/galaaz_ggplot/galaaz_ggplot.md +364 -0
  30. data/blogs/galaaz_ggplot/galaaz_ggplot.tex +607 -0
  31. data/blogs/galaaz_ggplot/galaaz_ggplot_files/figure-html/midwest_rb.png +0 -0
  32. data/blogs/galaaz_ggplot/galaaz_ggplot_files/figure-html/scatter_plot_rb.png +0 -0
  33. data/blogs/galaaz_ggplot/galaaz_ggplot_files/figure-markdown_github/midwest_rb.png +0 -0
  34. data/blogs/galaaz_ggplot/galaaz_ggplot_files/figure-markdown_github/scatter_plot_rb.png +0 -0
  35. data/blogs/galaaz_ggplot/midwest.Rmd +3 -3
  36. data/blogs/galaaz_ggplot/midwest_external_png +0 -0
  37. data/blogs/gknit/gknit.Rmd +52 -55
  38. data/blogs/gknit/gknit.md +94 -94
  39. data/blogs/gknit/gknit_files/figure-html/bubble-1.png +0 -0
  40. data/blogs/gknit/gknit_files/figure-html/diverging_bar.png +0 -0
  41. data/blogs/gknit/lst.rds +0 -0
  42. data/blogs/gknit/model.rb +1 -1
  43. data/blogs/gknit/stats.bib +0 -0
  44. data/blogs/manual/include_model_local_repro.Rmd +14 -0
  45. data/blogs/manual/include_model_local_repro.md +75 -0
  46. data/blogs/manual/lst.rds +0 -0
  47. data/blogs/manual/manual.Rmd +1582 -196
  48. data/blogs/manual/manual.log +1786 -0
  49. data/blogs/manual/manual.md +3107 -890
  50. data/blogs/manual/manual.tex +3018 -1086
  51. data/blogs/manual/manual_files/figure-html/bubble-1.png +0 -0
  52. data/blogs/manual/manual_files/figure-html/diverging_bar.png +0 -0
  53. data/blogs/manual/manual_files/figure-latex/bubble-1.png +0 -0
  54. data/blogs/manual/model.rb +41 -0
  55. data/blogs/nse_dplyr/nse_dplyr.Rmd +277 -151
  56. data/blogs/nse_dplyr/nse_dplyr.log +928 -0
  57. data/blogs/nse_dplyr/nse_dplyr.md +457 -293
  58. data/blogs/oh_my/not_so.rb +0 -0
  59. data/blogs/oh_my/oh_my.Rmd +1234 -25
  60. data/blogs/oh_my/oh_my.log +804 -0
  61. data/blogs/oh_my/oh_my.md +1808 -228
  62. data/blogs/oh_my/oh_my.tex +821 -0
  63. data/blogs/oh_my/old.Rmd +15 -14
  64. data/blogs/ruby_plot/ruby_plot.Rmd +58 -82
  65. data/blogs/ruby_plot/ruby_plot.log +885 -0
  66. data/blogs/ruby_plot/ruby_plot.md +71 -103
  67. data/blogs/ruby_plot/ruby_plot.tex +940 -0
  68. data/blogs/ruby_plot/ruby_plot_files/figure-html/dose_len.png +0 -0
  69. data/blogs/ruby_plot/ruby_plot_files/figure-html/facet_by_delivery.png +0 -0
  70. data/blogs/ruby_plot/ruby_plot_files/figure-html/facet_by_dose.png +0 -0
  71. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_by_delivery_color.png +0 -0
  72. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_by_delivery_color2.png +0 -0
  73. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_with_decorations.png +0 -0
  74. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_with_jitter.png +0 -0
  75. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_with_points.png +0 -0
  76. data/blogs/ruby_plot/ruby_plot_files/figure-html/final_box_plot.png +0 -0
  77. data/blogs/ruby_plot/ruby_plot_files/figure-html/final_violin_plot.png +0 -0
  78. data/blogs/ruby_plot/ruby_plot_files/figure-html/violin_with_jitter.png +0 -0
  79. data/blogs/ruby_plot/ruby_plot_files/figure-latex/dose_len.png +0 -0
  80. data/blogs/ruby_plot/ruby_plot_files/figure-latex/facet_by_delivery.png +0 -0
  81. data/blogs/ruby_plot/ruby_plot_files/figure-latex/facet_by_dose.png +0 -0
  82. data/blogs/ruby_plot/ruby_plot_files/figure-latex/facets_by_delivery_color.png +0 -0
  83. data/blogs/ruby_plot/ruby_plot_files/figure-latex/facets_by_delivery_color2.png +0 -0
  84. data/blogs/ruby_plot/ruby_plot_files/figure-latex/facets_with_decorations.png +0 -0
  85. data/blogs/ruby_plot/ruby_plot_files/figure-latex/facets_with_jitter.png +0 -0
  86. data/blogs/ruby_plot/ruby_plot_files/figure-latex/facets_with_points.png +0 -0
  87. data/blogs/ruby_plot/ruby_plot_files/figure-latex/final_box_plot.png +0 -0
  88. data/blogs/ruby_plot/ruby_plot_files/figure-latex/final_violin_plot.png +0 -0
  89. data/blogs/ruby_plot/ruby_plot_files/figure-latex/violin_with_jitter.png +0 -0
  90. data/blogs/test/test.Rmd +14 -0
  91. data/examples/50Plots_MasterList/Images/midwest-scatterplot.PNG +0 -0
  92. data/examples/50Plots_MasterList/ScatterPlot.rb +0 -0
  93. data/examples/50Plots_MasterList/scatter_plot.rb +0 -0
  94. data/examples/Bibliography/master.bib +50 -0
  95. data/examples/Bibliography/stats.bib +72 -0
  96. data/examples/R/calc.R +0 -0
  97. data/examples/R/java_interop.R +0 -0
  98. data/examples/bioconductor_deseq2_airway/Documentation/DESeq2-airway-walkthrough.md +56 -0
  99. data/examples/bioconductor_deseq2_airway/bench_galaaz_three_same_process.rb +53 -0
  100. data/examples/bioconductor_deseq2_airway/bench_r_three_same_process.R +34 -0
  101. data/examples/bioconductor_deseq2_airway/deseq2_airway_galaaz.rb +33 -0
  102. data/examples/bioconductor_deseq2_airway/deseq2_airway_galaaz_optimized.rb +34 -0
  103. data/examples/bioconductor_deseq2_airway/deseq2_airway_minimal.R +30 -0
  104. data/examples/bioconductor_deseq2_airway/deseq2_airway_pipeline_for_bench.R +36 -0
  105. data/examples/islr/all.rb +13 -0
  106. data/examples/islr/ch2.spec.rb +37 -7
  107. data/examples/islr/ch3.spec.rb +11 -2
  108. data/examples/islr/ch3_boston.rb +27 -0
  109. data/examples/islr/ch3_multiple_regression.rb +0 -0
  110. data/examples/islr/ch6.spec.rb +24 -1
  111. data/examples/islr/x_y_rnorm.jpg +0 -0
  112. data/examples/latex_templates/Test-acm_article/Makefile +16 -0
  113. data/examples/latex_templates/Test-acm_article/Test-acm_article.Rmd +65 -0
  114. data/examples/latex_templates/Test-acm_article/acm_proc_article-sp.cls +1670 -0
  115. data/examples/latex_templates/Test-acm_article/sensys-abstract.cls +703 -0
  116. data/examples/latex_templates/Test-acm_article/sigproc.bib +59 -0
  117. data/examples/latex_templates/Test-acs_article/Test-acs_article.Rmd +260 -0
  118. data/examples/latex_templates/Test-acs_article/acs-Test-acs_article.bib +11 -0
  119. data/examples/latex_templates/Test-acs_article/acs-my_output.bib +11 -0
  120. data/examples/latex_templates/Test-acs_article/acstest.bib +17 -0
  121. data/examples/latex_templates/Test-aea_article/AEA.cls +1414 -0
  122. data/{blogs/gknit/marshal.dump → examples/latex_templates/Test-aea_article/BibFile.bib} +0 -0
  123. data/examples/latex_templates/Test-aea_article/Test-aea_article.Rmd +108 -0
  124. data/examples/latex_templates/Test-aea_article/aea.bst +1269 -0
  125. data/examples/latex_templates/Test-aea_article/multicol.sty +853 -0
  126. data/examples/latex_templates/Test-aea_article/references.bib +0 -0
  127. data/examples/latex_templates/Test-aea_article/setspace.sty +546 -0
  128. data/examples/latex_templates/Test-amq_article/Test-amq_article.Rmd +256 -0
  129. data/examples/latex_templates/Test-amq_article/Test-amq_article.pdfsync +3397 -0
  130. data/examples/latex_templates/Test-ams_article/Test-ams_article.Rmd +215 -0
  131. data/examples/latex_templates/Test-ams_article/amstest.bib +436 -0
  132. data/examples/latex_templates/Test-asa_article/Test-asa_article.Rmd +153 -0
  133. data/examples/latex_templates/Test-asa_article/agsm.bst +1353 -0
  134. data/examples/latex_templates/Test-asa_article/bibliography.bib +233 -0
  135. data/examples/latex_templates/Test-ieee_article/IEEEtran.bst +2409 -0
  136. data/examples/latex_templates/Test-ieee_article/IEEEtran.cls +6346 -0
  137. data/examples/latex_templates/Test-ieee_article/Test-ieee_article.Rmd +175 -0
  138. data/examples/latex_templates/Test-ieee_article/mybibfile.bib +20 -0
  139. data/examples/latex_templates/Test-rjournal_article/RJournal.sty +335 -0
  140. data/examples/latex_templates/Test-rjournal_article/RJreferences.bib +18 -0
  141. data/examples/latex_templates/Test-rjournal_article/Test-rjournal_article.Rmd +52 -0
  142. data/examples/latex_templates/Test-springer_article/Test-springer_article.Rmd +65 -0
  143. data/examples/latex_templates/Test-springer_article/bibliography.bib +26 -0
  144. data/examples/latex_templates/Test-springer_article/spbasic.bst +1658 -0
  145. data/examples/latex_templates/Test-springer_article/spmpsci.bst +1512 -0
  146. data/examples/latex_templates/Test-springer_article/spphys.bst +1443 -0
  147. data/examples/latex_templates/Test-springer_article/svglov3.clo +113 -0
  148. data/examples/latex_templates/Test-springer_article/svjour3.cls +1431 -0
  149. data/examples/misc/baseball.csv +0 -0
  150. data/examples/misc/ggplot.rb +3 -2
  151. data/examples/misc/moneyball.rb +0 -0
  152. data/examples/misc/subsetting.rb +0 -0
  153. data/examples/multithread_shards_to_r/shards_to_r.rb +67 -0
  154. data/examples/rmarkdown/svm-rmarkdown-anon-ms-example/svm-rmarkdown-anon-ms-example.Rmd +73 -0
  155. data/examples/rmarkdown/svm-rmarkdown-article-example/svm-rmarkdown-article-example.Rmd +382 -0
  156. data/examples/rmarkdown/svm-rmarkdown-beamer-example/svm-rmarkdown-beamer-example.Rmd +164 -0
  157. data/examples/rmarkdown/svm-rmarkdown-cv/svm-rmarkdown-cv.Rmd +92 -0
  158. data/examples/rmarkdown/svm-rmarkdown-syllabus-example/attend-grade-relationships.csv +482 -0
  159. data/examples/rmarkdown/svm-rmarkdown-syllabus-example/svm-rmarkdown-syllabus-example.Rmd +280 -0
  160. data/examples/rmarkdown/svm-xaringan-example/svm-xaringan-example.Rmd +386 -0
  161. data/examples/sthda_ggplot/README.md +0 -0
  162. data/examples/sthda_ggplot/RUN.md +41 -0
  163. data/examples/sthda_ggplot/all.rb +0 -0
  164. data/examples/sthda_ggplot/one_variable_continuous/density_gg.rb +0 -0
  165. data/examples/sthda_ggplot/one_variable_continuous/geom_area.rb +0 -0
  166. data/examples/sthda_ggplot/one_variable_continuous/geom_density.rb +2 -0
  167. data/examples/sthda_ggplot/one_variable_continuous/geom_dotplot.rb +0 -0
  168. data/examples/sthda_ggplot/one_variable_continuous/geom_freqpoly.rb +0 -0
  169. data/examples/sthda_ggplot/one_variable_continuous/geom_histogram.rb +0 -0
  170. data/examples/sthda_ggplot/one_variable_continuous/histogram_density.rb +0 -0
  171. data/examples/sthda_ggplot/one_variable_continuous/stat.rb +0 -0
  172. data/examples/sthda_ggplot/one_variable_discrete/bar.rb +0 -0
  173. data/examples/sthda_ggplot/qplots/box_violin_dot.rb +0 -0
  174. data/examples/sthda_ggplot/qplots/scatter_plots.rb +0 -0
  175. data/examples/sthda_ggplot/scatter_gg.rb +0 -0
  176. data/examples/sthda_ggplot/two_variables_cont_bivariate/geom_bin2d.rb +0 -0
  177. data/examples/sthda_ggplot/two_variables_cont_bivariate/geom_density2d.rb +0 -0
  178. data/examples/sthda_ggplot/two_variables_cont_bivariate/geom_hex.rb +0 -0
  179. data/examples/sthda_ggplot/two_variables_cont_cont/geom_point.rb +0 -0
  180. data/examples/sthda_ggplot/two_variables_cont_cont/geom_smooth.rb +0 -0
  181. data/examples/sthda_ggplot/two_variables_cont_cont/misc.rb +0 -0
  182. data/examples/sthda_ggplot/two_variables_cont_function/geom_area.rb +4 -3
  183. data/examples/sthda_ggplot/two_variables_disc_cont/geom_bar.rb +0 -0
  184. data/examples/sthda_ggplot/two_variables_disc_cont/geom_boxplot.rb +0 -0
  185. data/examples/sthda_ggplot/two_variables_disc_cont/geom_dotplot.rb +0 -0
  186. data/examples/sthda_ggplot/two_variables_disc_cont/geom_jitter.rb +0 -0
  187. data/examples/sthda_ggplot/two_variables_disc_cont/geom_line.rb +0 -0
  188. data/examples/sthda_ggplot/two_variables_disc_cont/geom_violin.rb +0 -0
  189. data/examples/sthda_ggplot/two_variables_disc_disc/geom_jitter.rb +0 -0
  190. data/examples/sthda_ggplot/two_variables_error/geom_crossbar.rb +0 -0
  191. data/ext/new_bridge/Makefile +46 -0
  192. data/ext/new_bridge/galaaz_gatekeeper_phase0.cpp +12 -0
  193. data/ext/new_bridge/galaaz_gatekeeper_phase1.cpp +1639 -0
  194. data/lib/R_interface/galaaz_device.R +20 -0
  195. data/lib/R_interface/include_engine.R +109 -0
  196. data/lib/R_interface/new_bridge_adapter.rb +824 -0
  197. data/lib/R_interface/r.rb +177 -25
  198. data/lib/R_interface/r_arrow.rb +113 -0
  199. data/lib/R_interface/r_libs.R +4 -4
  200. data/lib/R_interface/r_methods.rb +13 -116
  201. data/lib/R_interface/r_module_s.rb +0 -0
  202. data/lib/R_interface/rbinary_operators.rb +20 -2
  203. data/lib/R_interface/rclosure.rb +5 -1
  204. data/lib/R_interface/rdata_frame.rb +34 -70
  205. data/lib/R_interface/rdevice.rb +125 -0
  206. data/lib/R_interface/rdevices.R +0 -0
  207. data/lib/R_interface/renvironment.rb +10 -4
  208. data/lib/R_interface/rexpression.rb +5 -1
  209. data/lib/R_interface/rindexed_object.rb +41 -13
  210. data/lib/R_interface/rlanguage.rb +20 -62
  211. data/lib/R_interface/rlist.rb +115 -25
  212. data/lib/R_interface/rlogical_operators.rb +0 -0
  213. data/lib/R_interface/rmatrix.rb +2 -11
  214. data/lib/R_interface/rmd_indexed_object.rb +5 -1
  215. data/lib/R_interface/robject.rb +348 -290
  216. data/lib/R_interface/rpkg.rb +1 -0
  217. data/lib/R_interface/rsupport.rb +610 -331
  218. data/lib/R_interface/rsupport_scope.rb +2 -1
  219. data/lib/R_interface/rsymbol.rb +50 -0
  220. data/lib/R_interface/ruby_callback.rb +2 -3
  221. data/lib/R_interface/ruby_extensions.rb +225 -175
  222. data/lib/R_interface/runary_operators.rb +0 -0
  223. data/lib/R_interface/rvector.rb +147 -31
  224. data/lib/galaaz.rb +0 -0
  225. data/lib/galaaz_jruby.rb +22 -0
  226. data/lib/gknit/diagnostics.rb +50 -0
  227. data/lib/gknit/draft.rb +111 -0
  228. data/lib/gknit/include_engine.rb +15 -7
  229. data/lib/gknit/knitr_engine.rb +223 -107
  230. data/lib/gknit/rb_engine.rb +3 -3
  231. data/lib/gknit/ruby_engine.rb +0 -0
  232. data/lib/gknit.rb +3 -0
  233. data/lib/new_bridge/bootstrap/windows_bootstrap.rb +285 -0
  234. data/lib/new_bridge/envelope.rb +51 -0
  235. data/lib/new_bridge/eval_result.rb +26 -0
  236. data/lib/new_bridge/framing.rb +39 -0
  237. data/lib/new_bridge/instance_pool_client.rb +38 -0
  238. data/lib/new_bridge/r_instance_manager.rb +404 -0
  239. data/lib/new_bridge/session_client.rb +530 -0
  240. data/lib/new_bridge/tcp_framed.rb +44 -0
  241. data/lib/new_bridge.rb +9 -0
  242. data/lib/util/exec_ruby.rb +95 -46
  243. data/lib/util/inline_file.rb +35 -30
  244. data/new_bridge_specs/benchmark_phase5_5_unboxing_spec.rb +96 -0
  245. data/new_bridge_specs/eval_r_async_spec.rb +113 -0
  246. data/new_bridge_specs/integration_phase5_1_concurrent_spec.rb +50 -0
  247. data/new_bridge_specs/integration_phase5_1_eval_spec.rb +16 -0
  248. data/new_bridge_specs/integration_phase5_1_r_api_spec.rb +25 -0
  249. data/new_bridge_specs/integration_phase5_1_smoke_spec.rb +31 -0
  250. data/new_bridge_specs/integration_phase5_2_dataframe_unboxing_spec.rb +19 -0
  251. data/new_bridge_specs/integration_phase5_2_handle_eval_unboxing_spec.rb +25 -0
  252. data/new_bridge_specs/integration_phase5_3_callback_args_spec.rb +28 -0
  253. data/new_bridge_specs/integration_phase5_3_callback_error_spec.rb +22 -0
  254. data/new_bridge_specs/integration_phase5_3_callback_timeout_spec.rb +28 -0
  255. data/new_bridge_specs/integration_phase5_3_callbacks_smoke_spec.rb +22 -0
  256. data/new_bridge_specs/integration_phase5_3_edge_cases_spec.rb +52 -0
  257. data/new_bridge_specs/integration_phase5_3_nested_spec.rb +30 -0
  258. data/new_bridge_specs/integration_phase5_4_concurrent_sessions_spec.rb +53 -0
  259. data/new_bridge_specs/integration_phase5_4_nested_session_callbacks_spec.rb +49 -0
  260. data/new_bridge_specs/integration_phase5_4_session_routing_spec.rb +38 -0
  261. data/new_bridge_specs/integration_phase5_5_stress_concurrency_spec.rb +52 -0
  262. data/new_bridge_specs/integration_phase5_5_unbox_walk_spec.rb +46 -0
  263. data/new_bridge_specs/phase0_protocol_spec.rb +96 -0
  264. data/new_bridge_specs/phase1_req_ret_spec.rb +66 -0
  265. data/new_bridge_specs/phase2_multi_instance_spec.rb +67 -0
  266. data/new_bridge_specs/phase3_callbacks_spec.rb +71 -0
  267. data/new_bridge_specs/phase4_2_hardening_spec.rb +252 -0
  268. data/new_bridge_specs/phase4_3_r_instance_manager_spec.rb +85 -0
  269. data/new_bridge_specs/phase4_nested_callbacks_spec.rb +123 -0
  270. data/r_requires/ggplot.rb +0 -0
  271. data/r_requires/knitr.rb +0 -0
  272. data/specs/all.rb +15 -11
  273. data/specs/arrow_from_ruby_batches_spec.rb +50 -0
  274. data/specs/arrow_semantics_spec.rb +64 -0
  275. data/specs/bridge_concurrent_spec.rb +46 -0
  276. data/specs/bridge_nested_spec.rb +25 -0
  277. data/specs/dataframe_semantics_spec.rb +122 -0
  278. data/specs/dataframe_single_index_logical_filter_spec.rb +21 -0
  279. data/specs/dispatch_probe_cache_spec.rb +38 -0
  280. data/specs/dispatch_probe_error_class_fallback_spec.rb +20 -0
  281. data/specs/dispatch_probe_fallback_spec.rb +18 -0
  282. data/specs/environment_semantics_spec.rb +89 -0
  283. data/specs/field_access_spec.rb +31 -0
  284. data/specs/figures/bg.jpeg +0 -0
  285. data/specs/figures/bg.png +0 -0
  286. data/specs/figures/bg.svg +168 -57
  287. data/specs/figures/dose_len.png +0 -0
  288. data/specs/figures/no_args.jpeg +0 -0
  289. data/specs/figures/no_args.png +0 -0
  290. data/specs/figures/no_args.svg +168 -57
  291. data/specs/figures/width_height.jpeg +0 -0
  292. data/specs/figures/width_height.png +0 -0
  293. data/specs/figures/width_height_units1.jpeg +0 -0
  294. data/specs/figures/width_height_units1.png +0 -0
  295. data/specs/figures/width_height_units2.jpeg +0 -0
  296. data/specs/figures/width_height_units2.png +0 -0
  297. data/specs/formula_semantics_spec.rb +81 -0
  298. data/specs/galaaz_util_exec_ruby_spec.rb +85 -0
  299. data/specs/galaaz_util_inline_file_spec.rb +54 -0
  300. data/specs/gknit_cli_option_permutation_spec.rb +24 -0
  301. data/specs/gknit_include_engine_spec.rb +72 -0
  302. data/specs/gknit_install_timeout_report_spec.rb +69 -0
  303. data/specs/gknit_internal_error_report_spec.rb +57 -0
  304. data/specs/gknit_vector_map_output_spec.rb +59 -0
  305. data/specs/globalenv_guardrail_spec.rb +52 -0
  306. data/specs/language_expression_semantics_spec.rb +145 -0
  307. data/specs/list_semantics_spec.rb +111 -0
  308. data/specs/new_bridge_bulk_dataframe_transfer_spec.rb +44 -0
  309. data/specs/new_bridge_bulk_vector_transfer_spec.rb +73 -0
  310. data/specs/new_bridge_callback_timeout_spec.rb +69 -0
  311. data/specs/new_bridge_eval_r_fallback_spec.rb +55 -0
  312. data/specs/nil_null_spec.rb +42 -0
  313. data/specs/object_build_phase2_spec.rb +53 -0
  314. data/specs/phase1_callback_bridge_spec.rb +84 -0
  315. data/specs/phase2_gknit_generic_rendering_guardrail_spec.rb +46 -0
  316. data/specs/phase2_gknit_no_raw_code_leakage_spec.rb +43 -0
  317. data/specs/phase3_gknit_generic_graphics_capture_spec.rb +71 -0
  318. data/specs/plot_device_semantics_spec.rb +28 -0
  319. data/specs/plot_snapshot_semantics_spec.rb +58 -0
  320. data/specs/protocol_result_spec.rb +236 -0
  321. data/specs/r_batch_fail_fast_spec.rb +47 -0
  322. data/specs/r_bridge_bootstrap_spec.rb +11 -0
  323. data/specs/r_devices.spec.rb +1 -1
  324. data/specs/r_eval.spec.rb +16 -18
  325. data/specs/r_function.spec.rb +1 -1
  326. data/specs/r_instance_manager_spec.rb +285 -0
  327. data/specs/r_list_apply.spec.rb +15 -15
  328. data/specs/r_matrix.spec.rb +0 -0
  329. data/specs/r_nse.spec.rb +5 -5
  330. data/specs/r_object_send_dispatch_spec.rb +13 -0
  331. data/specs/r_vector_comparator_spec.rb +8 -0
  332. data/specs/r_vector_creation.spec.rb +0 -0
  333. data/specs/r_vector_functions.spec.rb +0 -0
  334. data/specs/r_vector_object.spec.rb +0 -0
  335. data/specs/r_vector_operators.spec.rb +0 -0
  336. data/specs/r_vector_structured_scalar_reads_spec.rb +35 -0
  337. data/specs/r_vector_subsetting.spec.rb +0 -0
  338. data/specs/range_helper_spec.rb +21 -0
  339. data/specs/rsupport_scope_spec.rb +28 -0
  340. data/specs/rsupport_var_name_thread_safety_spec.rb +24 -0
  341. data/specs/scalar_character_spec.rb +44 -0
  342. data/specs/scoped_symbol_dsl_refinement_spec.rb +40 -0
  343. data/specs/session_env_bridge_spec.rb +25 -0
  344. data/specs/simplecov_bootstrap_spec.rb +10 -0
  345. data/specs/spec_helper.rb +10 -0
  346. data/specs/tmp.rb +41 -20
  347. data/specs/unboxing_recursion_regression_spec.rb +30 -0
  348. data/specs/unboxing_spec.rb +49 -0
  349. data/specs/verify_callbacks.rb +42 -0
  350. data/sty/galaaz.sty +0 -0
  351. data/version.rb +1 -1
  352. metadata +239 -71
  353. data/blogs/galaaz_ggplot/galaaz_ggplot.aux +0 -41
  354. data/blogs/galaaz_ggplot/galaaz_ggplot.html +0 -705
  355. data/blogs/galaaz_ggplot/galaaz_ggplot.out +0 -10
  356. data/blogs/galaaz_ggplot/galaaz_ggplot.pdf +0 -0
  357. data/blogs/galaaz_ggplot/galaaz_ggplot_files/figure-latex/midwest_rb.pdf +0 -0
  358. data/blogs/galaaz_ggplot/galaaz_ggplot_files/figure-latex/scatter_plot_rb.pdf +0 -0
  359. data/blogs/galaaz_ggplot/midwest.html +0 -188
  360. data/blogs/gknit/gknit.html +0 -2266
  361. data/blogs/gknit/gknit.pdf +0 -0
  362. data/blogs/gknit/gknit.tex +0 -1358
  363. data/blogs/manual/graph.rb +0 -29
  364. data/blogs/manual/manual.html +0 -2995
  365. data/blogs/manual/manual.pdf +0 -0
  366. data/blogs/manual/manual_files/figure-latex/diverging_bar.pdf +0 -0
  367. data/blogs/nse_dplyr/nse_dplyr.html +0 -960
  368. data/blogs/nse_dplyr/nse_dplyr.pdf +0 -0
  369. data/blogs/nse_dplyr/nse_dplyr.tex +0 -1373
  370. data/blogs/oh_my/oh_my.html +0 -680
  371. data/blogs/ruby_plot/ruby_plot.Rmd_external_figs +0 -662
  372. data/blogs/ruby_plot/ruby_plot.html +0 -729
  373. data/blogs/ruby_plot/ruby_plot.pdf +0 -0
  374. data/blogs/ruby_plot/ruby_plot_files/figure-html/dose_len.svg +0 -57
  375. data/blogs/ruby_plot/ruby_plot_files/figure-html/facet_by_delivery.svg +0 -106
  376. data/blogs/ruby_plot/ruby_plot_files/figure-html/facet_by_dose.svg +0 -110
  377. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_by_delivery_color.svg +0 -174
  378. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_by_delivery_color2.svg +0 -236
  379. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_with_jitter.svg +0 -296
  380. data/blogs/ruby_plot/ruby_plot_files/figure-html/facets_with_points.svg +0 -236
  381. data/blogs/ruby_plot/ruby_plot_files/figure-html/final_box_plot.svg +0 -218
  382. data/blogs/ruby_plot/ruby_plot_files/figure-html/final_violin_plot.svg +0 -128
  383. data/blogs/ruby_plot/ruby_plot_files/figure-html/violin_with_jitter.svg +0 -150
  384. data/examples/paper/paper.rb +0 -36
  385. data/specs/r_dataframe.spec.rb +0 -379
  386. data/specs/r_environment.spec.rb +0 -140
  387. data/specs/r_formula.spec.rb +0 -232
  388. data/specs/r_language.spec.rb +0 -112
  389. data/specs/r_list.spec.rb +0 -293
  390. data/specs/r_plots.spec.rb +0 -72
  391. data/specs/ruby_expression.spec.rb +0 -315
@@ -3,8 +3,8 @@ title: "Non Standard Evaluation in dplyr with Galaaz"
3
3
  author:
4
4
  - "Rodrigo Botafogo"
5
5
  - "Daniel Mossé - University of Pittsburgh"
6
- tags: [Tech, Data Science, Ruby, R, GraalVM]
7
- date: "10/05/2019"
6
+ tags: [Tech, Data Science, Ruby, R, JRuby, "GNU R", Galaaz, dplyr]
7
+ date: "10/05/2019 (narrative updated for Galaaz 2.0, 2026)"
8
8
  output:
9
9
  html_document:
10
10
  self_contained: true
@@ -24,74 +24,96 @@ fontsize: 11pt
24
24
 
25
25
  # Introduction
26
26
 
27
- In this post we will see how to program with _dplyr_ in Galaaz.
27
+ According to Steven Sagaert’s answer on Quora about “Is programming language R overrated?”:
28
28
 
29
- ## But first, what is Galaaz??
29
+ > R is a sophisticated language with an unusual (i.e. non-mainstream) set of features. It‘s
30
+ > an impure functional programming language with sophisticated metaprogramming and 3
31
+ > different OO systems.
32
+
33
+ > Just like common lisp you can completely customise how things work via metaprogramming.
34
+ > The biggest example is the tidyverse: by creating it’s own evaluation system (tidyeval)
35
+ > was able to create a custom syntax for dplyr.
36
+
37
+ > Mastering R (the language) and its ecosystem is not a matter of weeks or months but
38
+ > takes years. The rabbit hole goes pretty deep…
39
+
40
+ Although a highly configurable language can give programmers a great deal of power,
41
+ it can also take years to master—as noted above. Programming with _dplyr_, for instance,
42
+ means learning evaluation rules that are not always approachable for **statisticians and
43
+ analysts who are not full-time software engineers**. That is not a criticism: R was **built**
44
+ for **statisticians** who need trustworthy results on a deadline, not necessarily for building
45
+ large applications.
46
+
47
+ **Unfortunately**, when such a user moves on to more **sophisticated** programming patterns,
48
+ the learning curve can become a real hurdle.
49
+
50
+ In this post we will see how to program with _dplyr_ in Galaaz and how Ruby can simplify
51
+ the learning curve of mastering _dplyr_ coding.
52
+
53
+ # But first, what is Galaaz??
30
54
 
31
55
  Galaaz is a system for tightly coupling Ruby and R. Ruby is a powerful language, with
32
- a large community, a very large set of libraries and great for web development. However,
56
+ a large community, a very large set of libraries and great for web development. It is also
57
+ easy to learn. However,
33
58
  it lacks libraries for data science, statistics, scientific plotting and machine learning.
34
59
  On the other hand, R is considered one of the most powerful languages for solving all of the
35
- above problems. Maybe the strongest competitor to R is Python with libraries such as NumPy,
36
- Pandas, SciPy, SciKit-Learn and many more.
60
+ above problems. **Python** is a strong competitor, with NumPy, pandas, SciPy, scikit-learn,
61
+ and **many thousands** of other packages on PyPI. We will not dwell on R **versus** Python here:
62
+ both are excellent languages with different strengths.
63
+ Our interest is to bring to yet another excellent language, Ruby, the data science libraries
64
+ that it lacks.
37
65
 
38
66
  With Galaaz we do not intend to re-implement any of the scientific libraries in R. However, we
39
67
  allow for very tight coupling between the two languages to the point that the Ruby
40
68
  developer does not need to know that there is an R engine running. Also, from the point of
41
- view of the R user/developer Galaaz looks a lot like R, with just minor syntactic difference,
42
- so there is almost no learning courve for the R developer. And as we will see in this
43
- post, programming with _dplyr_ is easier in Galaaz than in R.
69
+ view of the R user/developer, Galaaz looks a lot like R, with just minor syntactic difference,
70
+ so there is almost no learning curve for the R developer. And as we will see in this
71
+ post that programming with _dplyr_ is easier in Galaaz than in R.
44
72
 
45
- R users are probably quite knowledgeable about _dplyr_, for the Ruby developer, _dplyr_ and
73
+ R users are probably quite knowledgeable about _dplyr_. For the Ruby developer, _dplyr_ and
46
74
  the _tidyverse_ libraries are a set of libraries for data manipulation in R, developed by
47
- Hardley Wickham, chief scientis at RStudio and a prolific R coder and writer.
48
-
49
- For the coupling of Ruby and R we use new technologies provided by Oracle: GraalVM,
50
- TruffleRuby and FastR:
51
-
52
- GraalVM is a universal virtual machine for running applications
53
- written in JavaScript, Python 3, Ruby, R, JVM-based languages like Java,
54
- Scala, Kotlin, and LLVM-based languages such as C and C++.
55
-
56
- GraalVM removes the isolation between programming languages and enables
57
- interoperability in a shared runtime. It can run either standalone or in
58
- the context of OpenJDK, Node.js, Oracle Database, or MySQL.
59
-
60
- GraalVM allows you to write polyglot applications with a seamless way to
61
- pass values from one language to another. With GraalVM there is no copying
62
- or marshaling necessary as it is with other polyglot systems. This lets
63
- you achieve high performance when language boundaries are crossed. Most
64
- of the time there is no additional cost for crossing a language boundary
65
- at all.
66
-
67
- Often developers have to make uncomfortable compromises that require them
68
- to rewrite their software in other languages. For example:
69
-
70
- * “That library is not available in my language. I need to rewrite it.”
71
- * “That language would be the perfect fit for my problem, but we cannot
72
- run it in our environment.”
73
- * “That problem is already solved in my language, but the language is
74
- too slow.”
75
-
76
- With GraalVM we aim to allow developers to freely choose the right language
77
- for the task at hand without making compromises.
78
-
79
- Interested readers should also check out the following sites:
80
-
81
- * [GraalVM Home](https://www.graalvm.org/)
82
- * [TruffleRuby](https://github.com/oracle/truffleruby)
83
- * [FastR](https://github.com/oracle/fastr)
84
- * [Faster R with FastR](https://medium.com/graalvm/faster-r-with-fastr-4b8db0e0dceb)
85
- * [How to make Beautiful Ruby Plots with Galaaz](https://medium.freecodecamp.org/how-to-make-beautiful-ruby-plots-with-galaaz-320848058857)
86
- * [Ruby Plotting with Galaaz: An example of tightly coupling Ruby and R in GraalVM](https://towardsdatascience.com/ruby-plotting-with-galaaz-an-example-of-tightly-coupling-ruby-and-r-in-graalvm-520b69e21021)
87
- * [How to do reproducible research in Ruby with gKnit](https://towardsdatascience.com/how-to-do-reproducible-research-in-ruby-with-gknit-c26d2684d64e)
88
- * [R for Data Science](https://r4ds.had.co.nz/)
89
- * [Advanced R](https://adv-r.hadley.nz/)
75
+ Hadley Wickham, Chief Scientist at Posit (formerly RStudio) and a prolific R coder and writer.
76
+
77
+ For the coupling of Ruby and R, **Galaaz 2.0** uses **[JRuby](https://www.jruby.org/)** (Ruby on the JVM)
78
+ together with **GNU R**. A **bridge** sends expressions and data between Ruby and an R process so that
79
+ Ruby can call **dplyr** and the rest of the tidyverse as if they were part of the same workflow.
80
+ An **earlier** Galaaz line of work used Oracle’s **GraalVM** with **TruffleRuby** and **FastR** in a single
81
+ runtime; that approach is **no longer** the supported stack—see the project **manual** for setup,
82
+ **`bin/galaaz-jruby`**, and **gKnit**.
83
+
84
+
85
+ # Tidyverse and dplyr
86
+
87
+ In [What is the tidyverse?](https://rviews.rstudio.com/2017/06/08/what-is-the-tidyverse/) the
88
+ tidyverse is explained as follows:
90
89
 
91
- ## Programming with dplyr
90
+ > The tidyverse is a coherent system of packages for data manipulation, exploration and
91
+ > visualization that share a common design philosophy. These were mostly developed by
92
+ > Hadley Wickham himself, but they are now being expanded by several contributors. Tidyverse
93
+ > packages are intended to make statisticians and data scientists more productive by
94
+ > guiding them through workflows that facilitate communication, and result in reproducible
95
+ > work products. Fundamentally, the tidyverse is about the connections between the tools
96
+ > that make the workflow possible.
92
97
 
93
- This post will follow closely the work done in https://dplyr.tidyverse.org/articles/programming.html,
94
- by Hardley Wickham. In it, Hardley states:
98
+ _dplyr_ is one of the many packages that are part of the tidyverse. It is:
99
+
100
+ > a grammar of data manipulation, providing a consistent set of verbs that help you solve
101
+ > the most common data manipulation challenges:
102
+
103
+ > 1. mutate() adds new variables that are functions of existing variables
104
+ > 2. select() picks variables based on their names.
105
+ > 3. filter() picks cases based on their values.
106
+ > 4. summarise() reduces multiple values down to a single summary.
107
+ > 5. arrange() changes the ordering of the rows.
108
+
109
+ Very often R is used interactively and users use _dplyr_ to manipulate a single dataset
110
+ without programming. When users want to replicate their work for
111
+ multiple datasets, programming becomes necessary.
112
+
113
+ # Programming with dplyr
114
+
115
+ In the vignette ["Programming with dplyr"](https://dplyr.tidyverse.org/articles/programming.html),
116
+ Hadley Wickham states:
95
117
 
96
118
  > Most dplyr functions use non-standard evaluation (NSE). This is a catch-all term that
97
119
  > means they don’t follow the usual R rules of evaluation. Instead, they capture the
@@ -106,13 +128,15 @@ by Hardley Wickham. In it, Hardley states:
106
128
  > database backends because dplyr itself doesn’t do any work, but instead generates the SQL
107
129
  > that tells the database what to do.
108
130
 
131
+ But then he goes on:
132
+
109
133
  > Unfortunately these benefits do not come for free. There are two main drawbacks:
110
134
 
111
135
  > Most dplyr arguments are not referentially transparent. That means you can’t replace a value
112
136
  > with a seemingly equivalent object that you’ve defined elsewhere. In other words, this code:
113
137
 
114
138
 
115
- ```r
139
+ ``` r
116
140
  df <- data.frame(x = 1:3, y = 3:1)
117
141
  print(filter(df, x == 1))
118
142
  #> # A tibble: 1 x 2
@@ -123,7 +147,7 @@ print(filter(df, x == 1))
123
147
  > Is not equivalent to this code:
124
148
 
125
149
 
126
- ```r
150
+ ``` r
127
151
  my_var <- x
128
152
  #> Error in eval(expr, envir, enclos): object 'x' not found
129
153
  filter(df, my_var == 1)
@@ -131,8 +155,27 @@ filter(df, my_var == 1)
131
155
  ```
132
156
  > This makes it hard to create functions with arguments that change how dplyr verbs are computed.
133
157
 
134
- In this post we will see that programming with _dplyr_ in Galaaz does not require knowledge of
135
- non-standard evaluation in R and can be accomplished by utilizing normal Ruby constructs.
158
+ As a result of this, programming with _dplyr_ requires learning a set of new ideas and concepts.
159
+ In this vignette Hadley goes on showing how to program ever more difficult problems with _dplyr_,
160
+ showing the problems it faces and the new concepts needed to solve them.
161
+
162
+ In this blog, we will look at all the problems presented by Harley on the vignette and show how
163
+ those same problems can be solved using Galaaz and the Ruby language.
164
+
165
+ This blog is organized as follows: first we show how to write expressions using Galaaz.
166
+ Expressions are a fundamental concept in _dplyr_ and are not part of basic Ruby. We extend
167
+ the Ruby language create a manipulate expressions that will be used by _dplyr_ functions.
168
+
169
+ Then we show very succintly how Ruby and R can be integrated and how R functions are
170
+ transparently called from Ruby. Galaaz [user manual](https://github.com/rbotafogo/galaaz/wiki)
171
+ (still in development) goes in much deeper detail about this integration.
172
+
173
+ Next in section "Data manipulation wiht _dplyr_" we go through all the problems on the
174
+ _dplyr_ vignette and look at how they are solved in Galaaz. We then discuss why programming
175
+ with Galaaz and _dplyr_ is easier than programming with _dplyr_ in plain R.
176
+
177
+ The following section looks at another more advanced problem and shows that Galaaz can still
178
+ handle it without any difficulty. We then provide further reading and concluding remarks.
136
179
 
137
180
  # Writing Expressions in Galaaz
138
181
 
@@ -141,40 +184,70 @@ Galaaz extends Ruby to work with expressions, similar to R's expressions build w
141
184
  formulae. For instance, in mathematics, the expression $y = sin(x)$ describes a function but cannot
142
185
  be computed unless the value of $x$ is bound to some value.
143
186
 
144
- Let's take a look at some of those expressions in Ruby:
187
+ Expressions are fundamental in _dplyr_ programming as they are the input to _dplyr_ functions,
188
+ for instance, as we will see shortly, if a data frame has a column named 'x' and we want
189
+ to add another column, y, to this dataframe that has the values of 'x' times 2, then we would
190
+ call a _dplyr_ function with the expression 'y = x * 2'.
191
+
192
+ ## A note on notation
193
+
194
+ This blog was written in Rmarkdown and automatically converted to HTML or PDF (depending on
195
+ where you are reading this blog) with gKnit (a tool provided by Galaaz). In Rmarkdown, it is
196
+ possible to write text and code blocks that are executed to generate the final report. Code
197
+ blocks appear inside a 'box' and the result of their execution appear either in another type
198
+ of 'box' with a different background (HTML) or as normal text (PDF). Every output line from
199
+ the code execution is preceded by '##'.
145
200
 
146
201
  ## Expressions from operators
147
202
 
148
- The code bellow creates an expression summing two symbols. Note that :a and :b are Ruby symbols and
149
- are not bound to any value at the time of expression definition:
203
+ The code below creates an expression summing two symbols. Note that :a and :b are Ruby symbols and
204
+ are not bound to any values at the time of expression definition:
150
205
 
151
206
 
152
- ```ruby
207
+ ``` ruby
153
208
  exp1 = :a + :b
154
209
  puts exp1
155
210
  ```
156
211
 
157
212
  ```
158
- ## a + b
213
+ ## undefined method '+' for an instance of Symbol
214
+ ```
215
+
216
+ ```
217
+ ## /home/rbotafogo/desenv_linux/galaaz/lib/util/exec_ruby.rb:170:in 'exec_ruby'
218
+ ## org/jruby/RubyKernel.java:1268:in 'eval'
219
+ ## /home/rbotafogo/desenv_linux/galaaz/lib/util/exec_ruby.rb:169:in 'exec_ruby'
220
+ ## /home/rbotafogo/desenv_linux/galaaz/lib/gknit/knitr_engine.rb:777:in 'block in initialize'
221
+ ## org/jruby/RubyBasicObject.java:2695:in 'instance_eval'
222
+ ## org/jruby/RubyBasicObject.java:2723:in 'instance_eval'
223
+ ## /home/rbotafogo/desenv_linux/galaaz/lib/gknit/knitr_engine.rb:748:in 'block in initialize'
224
+ ## /home/rbotafogo/desenv_linux/galaaz/lib/R_interface/new_bridge_adapter.rb:358:in 'block in register_callback_proc_stub'
225
+ ## /home/rbotafogo/desenv_linux/galaaz/lib/new_bridge/session_client.rb:413:in 'block in handle_call'
159
226
  ```
160
- We can build any complex mathematical expression such as:
227
+ In Galaaz, we can build any complex mathematical expression such as:
161
228
 
162
229
 
163
- ```ruby
164
- exp2 = (:a + :b) * 2.0 + :c ** 2 / :z
230
+ ``` ruby
231
+ exp2 = (R[:a] + R[:b]) * 2.0 + R[:c] ** 2 / R[:z]
165
232
  puts exp2
166
233
  ```
167
234
 
168
235
  ```
169
- ## (a + b) * 2 + c^2L/z
236
+ ## a + b * 2.0 + c ^ 2L / z
170
237
  ```
171
- The 'L' after two indicates that 2 is an integer.
238
+ Expressions are printed with the same format as the equivalent R expressions. The 'L' after
239
+ 2 indicates that 2 is an integer.
240
+
241
+ The R developer should note that in R, if she writes the
242
+ number '2', the R interpreter will convert it to float. In order to get an interger she
243
+ should write '2L'. Galaaz follows Ruby notation and '2' is an integer, while '2.0' is a
244
+ float.
172
245
 
173
246
  It is also possible to use inequality operators in building expressions:
174
247
 
175
248
 
176
- ```ruby
177
- exp3 = (:a + :b) >= :z
249
+ ``` ruby
250
+ exp3 = (R[:a] + R[:b]) >= R[:z]
178
251
  puts exp3
179
252
  ```
180
253
 
@@ -184,15 +257,15 @@ puts exp3
184
257
  Expressions' definition can also make use of normal Ruby variables without any problem:
185
258
 
186
259
 
187
- ```ruby
260
+ ``` ruby
188
261
  x = 20
189
- y = 30
190
- exp_var = (:a + :b) * x <= :z - y
262
+ y = 30.0
263
+ exp_var = (R[:a] + R[:b]) * x <= R[:z] - y
191
264
  puts exp_var
192
265
  ```
193
266
 
194
267
  ```
195
- ## (a + b) * 20L <= z - 30L
268
+ ## a + b * 20L <= z - 30.0
196
269
  ```
197
270
 
198
271
  Galaaz provides both symbolic representations for operators, such as (>, <, !=) as functional
@@ -200,8 +273,8 @@ notation for those operators such as (.gt, .ge, etc.). So the same expression w
200
273
  above can also be written as
201
274
 
202
275
 
203
- ```ruby
204
- exp4 = (:a + :b).ge :z
276
+ ``` ruby
277
+ exp4 = (R[:a] + R[:b]).ge R[:z]
205
278
  puts exp4
206
279
  ```
207
280
 
@@ -209,14 +282,16 @@ puts exp4
209
282
  ## a + b >= z
210
283
  ```
211
284
 
212
- Two type of expression, however, can only be created with the functional representation
213
- of the operators, those are expressions involving '==', and '='. In order to write an
214
- expression involving '==' we
215
- need to use the method '.eq' and for '=' we need the function '.assign'
285
+ Two types of expressions, however, can only be created with the functional representation
286
+ of the operators. Those are expressions involving '==', and '='. This is the case since
287
+ those symbols have special meaning in Ruby and should not be redefined.
288
+
289
+ In order to write an expression involving '==' we
290
+ need to use the method '.eq' and for '=' we need the function '.assign':
216
291
 
217
292
 
218
- ```ruby
219
- exp5 = (:a + :b).eq :z
293
+ ``` ruby
294
+ exp5 = (R[:a] + R[:b]).eq R[:z]
220
295
  puts exp5
221
296
  ```
222
297
 
@@ -225,96 +300,98 @@ puts exp5
225
300
  ```
226
301
 
227
302
 
228
- ```ruby
229
- exp6 = :y.assign :a + :b
303
+ ``` ruby
304
+ exp6 = R[:y].assign R[:a] + R[:b]
230
305
  puts exp6
231
306
  ```
232
307
 
233
308
  ```
234
309
  ## y <- a + b
235
310
  ```
236
- In general we think that using the functional notation is preferable to using the
237
- symbolic notation as otherwise, we end up writing invalid expressions such as
311
+ Users should be careful when writing expressions not to inadvertently use '==' or '=' as
312
+ this will generate an error, that might be a bit cryptic (in future releases of Galaza, we
313
+ plan to improve the error message).
238
314
 
239
315
 
240
- ```ruby
241
- exp_wrong = (:a + :b) == :z
316
+ ``` ruby
317
+ exp_wrong = (R[:a] + R[:b]) == R[:z]
242
318
  puts exp_wrong
243
319
  ```
244
320
 
245
321
  ```
246
- ## Message:
247
- ## Error in function (x, y, num.eq = TRUE, single.NA = TRUE, attrib.as.set = TRUE, :
248
- ## object 'a' not found (RError)
249
- ## Translated to internal error
322
+ ## false
250
323
  ```
251
- and it might be difficult to understand what is going on here. The problem lies with the fact that
252
- when using '==' we are comparing expression (:a + :b) to expression :z with '=='. When the
253
- comparison is executed, the system tries to evaluate :a, :b and :z, and those symbols at
254
- this time are not bound to anything and we get a "object 'a' not found" message.
255
- If we only use functional notation, this type of error will not occur.
324
+ The problem lies with the fact that
325
+ when using '==' we are comparing expression (R[:a] + R[:b]) to expression R[:z] with '=='. When this
326
+ comparison is executed, the system tries to evaluate :a, :b and :z, and those symbols, at
327
+ this time, are not bound to anything giving the "object 'a' not found" message.
256
328
 
257
329
  ## Expressions with R methods
258
330
 
259
331
  It is often necessary to create an expression that uses a method or function. For instance, in
260
332
  mathematics, it's quite natural to write an expressin such as $y = sin(x)$. In this case, the
261
- 'sin' function is part of the expression and should not immediately be executed. When we want
333
+ 'sin' function is part of the expression and should not be immediately executed. When we want
262
334
  the function to be part of the expression, we call the function preceeding it
263
335
  by the letter E, such as 'E.sin(x)'
264
336
 
265
337
 
266
- ```ruby
267
- exp7 = :y.assign E.sin(:x)
338
+ ``` ruby
339
+ exp7 = R[:y].assign E.sin(R[:x])
268
340
  puts exp7
269
341
  ```
270
342
 
271
343
  ```
272
344
  ## y <- sin(x)
273
345
  ```
346
+ Function expressions can also be written using '.' notation:
274
347
 
275
- Expressions can also be written using '.' notation:
276
348
 
277
-
278
- ```ruby
279
- exp8 = :y.assign :x.sin
349
+ ``` ruby
350
+ exp8 = R[:y].assign R[:x].sin
280
351
  puts exp8
281
352
  ```
282
353
 
283
354
  ```
284
355
  ## y <- sin(x)
285
356
  ```
286
-
287
- When a function has multiple arguments, the first one can be used before the '.':
357
+ When a function has multiple arguments, the first one can be used before the '.'. For instance,
358
+ the R concatenate function 'c', that concatenates two or more arguments can be part of
359
+ an expression as:
288
360
 
289
361
 
290
- ```ruby
291
- exp9 = :x.c(:y)
362
+ ``` ruby
363
+ exp9 = R[:x].c(R[:y])
292
364
  puts exp9
293
365
  ```
294
366
 
295
367
  ```
296
368
  ## c(x, y)
297
369
  ```
370
+ Note that this gives an OO feeling to the code, as if we were saying 'x' concatenates 'y'. As a
371
+ side note, '.' notation can be used as the R pipe operator '%>%', but is more general than the
372
+ pipe.
298
373
 
299
374
  ## Evaluating an Expression
300
375
 
301
- Expressions can be evaluated by calling function 'eval' with a binding. A binding can be provided
302
- with a list:
376
+ Although we are mainly focusing on expressions to pass them to _dplyr_ functions, expressions
377
+ can be evaluated by calling function 'eval' with a binding.
378
+
379
+ A binding can be provided with a list or a data frame as shown below:
303
380
 
304
381
 
305
- ```ruby
306
- exp = (:a + :b) * 2.0 + :c ** 2 / :z
382
+ ``` ruby
383
+ exp = (R[:a] + R[:b]) * 2.0 + R[:c] ** 2 / R[:z]
307
384
  puts exp.eval(R.list(a: 10, b: 20, c: 30, z: 40))
308
385
  ```
309
386
 
310
387
  ```
311
- ## [1] 82.5
388
+ ## [1] 72.5
312
389
  ```
313
390
 
314
- ... with a data frame:
391
+ with a data frame:
315
392
 
316
393
 
317
- ```ruby
394
+ ``` ruby
318
395
  df = R.data__frame(
319
396
  a: R.c(1, 2, 3),
320
397
  b: R.c(10, 20, 30),
@@ -325,7 +402,7 @@ puts exp.eval(df)
325
402
  ```
326
403
 
327
404
  ```
328
- ## [1] 32 64 96
405
+ ## [1] 31 62 93
329
406
  ```
330
407
 
331
408
  # Using Galaaz to call R functions
@@ -336,12 +413,12 @@ this post, we do not have enough space to write a complete manual on Galaaz
336
413
  (a short manual can be found at: https://www.rubydoc.info/gems/galaaz/0.4.9), so we will
337
414
  present only a few examples scripts using Galaaz.
338
415
 
339
- Basically, to call an R function from Ruby with Galaaz, one only needs to preceed the function
340
- with 'R.'. For instance, to create a vector in R, the 'c' function is used. From Galaaz, a
416
+ Basically, to call an R function from Ruby with Galaaz, one only needs to preced the function
417
+ with 'R.'. For instance, to create a vector in R, the 'c' function is used. In Galaaz, a
341
418
  vector can be created by using 'R.c':
342
419
 
343
420
 
344
- ```ruby
421
+ ``` ruby
345
422
  vec = R.c(1.0, 2, 3)
346
423
  puts vec
347
424
  ```
@@ -352,7 +429,7 @@ puts vec
352
429
  A list is created in R with the 'list' function, so in Galaaz we do:
353
430
 
354
431
 
355
- ```ruby
432
+ ``` ruby
356
433
  list = R.list(a: 1.0, b: 2, c: 3)
357
434
  puts list
358
435
  ```
@@ -370,7 +447,7 @@ puts list
370
447
  Note that we can use named arguments in our list. The same code in R would be:
371
448
 
372
449
 
373
- ```r
450
+ ``` r
374
451
  lst = list(a = 1, b = 2L, c = 3L)
375
452
  print(lst)
376
453
  ```
@@ -390,8 +467,8 @@ the expression $y = sin(45^\circ)$, which is $y = 0.850...$. In this case,
390
467
  we will use 'R.sin':
391
468
 
392
469
 
393
- ```ruby
394
- exp10 = :y.assign R.sin(45)
470
+ ``` ruby
471
+ exp10 = R[:y].assign R.sin(45)
395
472
  puts exp10
396
473
  ```
397
474
 
@@ -399,14 +476,81 @@ puts exp10
399
476
  ## y <- 0.850903524534118
400
477
  ```
401
478
 
402
- # Filtering using expressions
479
+ # Data manipulation wiht _dplyr_
480
+
481
+ In this section we will give a brief tour _dplyr_'s usage in Galaaz and how to manipulate
482
+ data in Ruby with it. This section will follow [_dplyr_'s vignette](https://dplyr.tidyverse.org/articles/dplyr.html) that explores the nycflights13 data set. This dataset contains all 336776
483
+ flights that departed from New York City in 2013. The data comes from the US Bureau of
484
+ Transportation Statistics.
485
+
486
+ Let's start by taking a look at this dataset:
487
+
488
+
489
+ ``` ruby
490
+ R.library('nycflights13')
491
+ # check it's dimension
492
+ puts ~R[:flights].dim
493
+ # and the structure
494
+ ~R[:flights].str
495
+ ```
496
+
497
+ ```
498
+ ## ~(dim(flights))
499
+ ## <environment: 0x5f0b6a5b7790>
500
+ ```
501
+
502
+ Now, let's use a first verb of _dplyr_: 'filter'. This verb, obviously, will filter the data
503
+ by the given expression. In the next block, we filter by columns 'month' and 'day'. The
504
+ first argument to the filter function is symbol ':flights'. A Ruby symbol, when given to
505
+ an R function will convert to the R variable of the same name, in this case 'flights', that
506
+ holds the nycflights13 data frame.
403
507
 
404
- Now that we know how to write expression and call R functions let's do some data manipulation in
405
- Galaaz. Let's first start by creating the same data frame that we created previously in section
406
- "Programming with dplyr":
508
+ The second and third arguments are expressions that will be used by the filter function to
509
+ filter by columns, looking for entries in which the month and day are equal to 1.
510
+
511
+
512
+ ``` ruby
513
+ puts R.filter(:flights, (R[:month].eq 1), (R[:day].eq 1))
514
+ ```
515
+
516
+ ```
517
+ ## # A tibble: 842 × 19
518
+ ## year month day dep_time sched_dep_time dep_delay arr_time sched_arr_time
519
+ ## <int> <int> <int> <int> <int> <dbl> <int> <int>
520
+ ## 1 2013 1 1 517 515 2 830 819
521
+ ## 2 2013 1 1 533 529 4 850 830
522
+ ## 3 2013 1 1 542 540 2 923 850
523
+ ## 4 2013 1 1 544 545 -1 1004 1022
524
+ ## 5 2013 1 1 554 600 -6 812 837
525
+ ## 6 2013 1 1 554 558 -4 740 728
526
+ ## 7 2013 1 1 555 600 -5 913 854
527
+ ## 8 2013 1 1 557 600 -3 709 723
528
+ ## 9 2013 1 1 557 600 -3 838 846
529
+ ## 10 2013 1 1 558 600 -2 753 745
530
+ ## # ℹ 832 more rows
531
+ ## # ℹ 11 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
532
+ ## # tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
533
+ ## # hour <dbl>, minute <dbl>, time_hour <dttm>
534
+ ```
407
535
 
408
536
 
409
- ```ruby
537
+ ## Programming with _dplyr_: problems and how to solve them in Galaaz
538
+
539
+ In this section we look at the list of problems that Hadley describes in the "Programming with dplyr"
540
+ vignette and show how those problems are solved and coded with Galaaz. Readers interested in
541
+ how those problems are treated in _dplyr_ should read the vignette and use it as a comparison with
542
+ this blog.
543
+
544
+ ## Filtering using expressions
545
+
546
+ Now that we know how to write expressions and call R functions, let's do some data manipulation in
547
+ Galaaz. Let's first start by creating a data frame. In R, the 'data.frame' function creates a
548
+ data frame. In Ruby, writing 'data.frame' will not parse as a single object. To call R
549
+ functions that have a '.' in them, we need to substitute the '.' with '__'. So, method
550
+ 'data.frame' in R, is called in Galaaz as 'R.data\_\_frame':
551
+
552
+
553
+ ``` ruby
410
554
  df = R.data__frame(x: (1..3), y: (3..1))
411
555
  puts df
412
556
  ```
@@ -417,44 +561,49 @@ puts df
417
561
  ## 2 2 2
418
562
  ## 3 3 1
419
563
  ```
420
- The 'filter' function can be called on this data frame either by using 'R.filter(df, ...)' or
421
- by using dot notation. We prefer to use dot notation as shown bellow. The argument to 'filter'
422
- in Galaaz should be an expression. Note that if we gave to filter a Ruby expression such as
564
+
565
+ _dplyr_ provides the 'filter' function, that filters data in a data brame. The 'filter'
566
+ function can be called on this data frame either by using 'R.filter(df, ...)' or
567
+ by using dot notation.
568
+
569
+ -------FIX---------
570
+
571
+ We prefer to use dot notation as shown below. The argument to 'filter' should be an
572
+ expression. Note that if we gave to filter a Ruby expression such as
423
573
  'x == 1', we would get an error, since there is no variable 'x' defined and if 'x' was a variable
424
574
  then 'x == 1' would either be 'true' or 'false'. Our goal is to filter our data frame returning
425
- all rows in which the 'x' value is equal to 1. To express this we want: ':x.eq 1', where :x will
575
+ all rows in which the 'x' value is equal to 1. To express this we want: 'R[:x].eq 1', where :x will
426
576
  be interpreted by filter as the 'x' column.
427
577
 
428
578
 
429
- ```ruby
430
- puts df.filter(:x.eq 1)
579
+ ``` ruby
580
+ puts df.filter(R[:x].eq 1)
431
581
  ```
432
582
 
433
583
  ```
434
584
  ## x y
435
585
  ## 1 1 3
436
586
  ```
437
-
438
587
  In R, and when coding with 'tidyverse', arguments to a function are usually not
439
588
  *referencially transparent*. That is, you can’t replace a value with a seemingly equivalent
440
589
  object that you’ve defined elsewhere. In other words, this code
441
590
 
442
591
 
443
- ```r
592
+ ``` r
444
593
  my_var <- x
445
594
  filter(df, my_var == 1)
446
595
  ```
447
596
  Generates the following error: "object 'x' not found.
448
597
 
449
598
  However, in Galaaz, arguments are referencially transparent as can be seen by the
450
- code bellow. Note initally that 'my_var = :x' will not give the error "object 'x' not found"
599
+ code below. Note initially that 'my_var = R[:x]' will not give the error "object 'x' not found"
451
600
  since ':x' is treated as an expression and assigned to my\_var. Then when doing (my\_var.eq 1),
452
- my\_var is a variable that resolves to ':x' and it becomes equivalent to (:x.eq 1) which is
601
+ my\_var is a variable that resolves to ':x' and it becomes equivalent to (R[:x].eq 1) which is
453
602
  what we want.
454
603
 
455
604
 
456
- ```ruby
457
- my_var = :x
605
+ ``` ruby
606
+ my_var = R[:x]
458
607
  puts df.filter(my_var.eq 1)
459
608
  ```
460
609
 
@@ -462,7 +611,7 @@ puts df.filter(my_var.eq 1)
462
611
  ## x y
463
612
  ## 1 1 3
464
613
  ```
465
- As stated by Hardley
614
+ As stated by Hadley
466
615
 
467
616
  > dplyr code is ambiguous. Depending on what variables are defined where,
468
617
  > filter(df, x == y) could be equivalent to any of:
@@ -474,18 +623,18 @@ df[x == df$y, ]
474
623
  df[x == y, ]
475
624
  ```
476
625
  In galaaz this ambiguity does not exist, filter(df, x.eq y) is not a valid expression as
477
- expressions are build with symbols. In doing filter(df, :x.eq y) we are looking for elements
626
+ expressions are build with symbols. In doing filter(df, R[:x].eq y) we are looking for elements
478
627
  of the 'x' column that are equal to a previously defined y variable. Finally in
479
- filter(df, :x.eq :y) we are looking for elements in which the 'x' column value is equal to
628
+ filter(df, R[:x].eq R[:y]) we are looking for elements in which the 'x' column value is equal to
480
629
  the 'y' column value. This can be seen in the following two chunks of code:
481
630
 
482
631
 
483
- ```ruby
632
+ ``` ruby
484
633
  y = 1
485
634
  x = 2
486
635
 
487
636
  # looking for values where the 'x' column is equal to the 'y' column
488
- puts df.filter(:x.eq :y)
637
+ puts df.filter(R[:x].eq R[:y])
489
638
  ```
490
639
 
491
640
  ```
@@ -494,17 +643,17 @@ puts df.filter(:x.eq :y)
494
643
  ```
495
644
 
496
645
 
497
- ```ruby
646
+ ``` ruby
498
647
  # looking for values where the 'x' column is equal to the 'y' variable
499
648
  # in this case, the number 1
500
- puts df.filter(:x.eq y)
649
+ puts df.filter(R[:x].eq y)
501
650
  ```
502
651
 
503
652
  ```
504
653
  ## x y
505
654
  ## 1 1 3
506
655
  ```
507
- # Writing a function that applies to different data sets
656
+ ## Writing a function that applies to different data sets
508
657
 
509
658
  Let's suppose that we want to write a function that receives as the first argument a data frame
510
659
  and as second argument an expression that adds a column to the data frame that is equal to the
@@ -529,18 +678,19 @@ Unfortunately, in R, this function can fail silently if one of the variables isn
529
678
  in the data frame, but is present in the global environment. We will not go through here how
530
679
  to solve this problem in R.
531
680
 
532
- In Galaaz the method mutate_y bellow will work fine and will never fail silently.
681
+ In Galaaz the method mutate_y below will work fine and will never fail silently.
533
682
 
534
683
 
535
- ```ruby
684
+ ``` ruby
536
685
  def mutate_y(df)
537
- df.mutate(:y.assign :a + :x)
686
+ # Mutate column names are Ruby kwargs (y: …). Use .assign only for R `<-` expressions.
687
+ df.mutate(y: R[:a] + R[:x])
538
688
  end
539
689
  ```
540
690
  Here we create a data frame that has only one column named 'x':
541
691
 
542
692
 
543
- ```ruby
693
+ ``` ruby
544
694
  df1 = R.data__frame(x: (1..3))
545
695
  puts df1
546
696
  ```
@@ -553,33 +703,29 @@ puts df1
553
703
  ```
554
704
 
555
705
  Note that method mutate_y will fail independetly from the fact that variable 'a' is defined and
556
- in the scope of the method. Variable 'a' has no relationship with the symbol ':a' used in the
706
+ in the scope of the method. Variable 'a' has no relationship with the symbol `R[:a]` used in the
557
707
  definition of 'mutate\_y' above:
558
708
 
559
709
 
560
- ```ruby
710
+ ``` ruby
561
711
  a = 10
562
712
  mutate_y(df1)
563
713
  ```
564
714
 
565
715
  ```
566
- ## Message:
567
- ## Error in mutate_impl(.data, dots) :
568
- ## Evaluation error: object 'a' not found.
569
- ## In addition: Warning message:
570
- ## In mutate_impl(.data, dots) :
571
- ## mismatched protect/unprotect (unprotect with empty protect stack) (RError)
572
- ## Translated to internal error
716
+ ## Error: ℹ In argument: `y = a + x`.
717
+ ## Caused by error:
718
+ ## ! object 'a' not found
573
719
  ```
574
- # Different expressions
720
+ ## Different expressions
575
721
 
576
- Let's move to the next problem as presented by Hardley where trying to write a function in R
722
+ Let's move to the next problem as presented by Hadley where trying to write a function in R
577
723
  that will receive two argumens, the first a variable and the second an expression is not trivial.
578
- Bellow we create a data frame and we want to write a function that groups data by a variable and
724
+ Below we create a data frame and we want to write a function that groups data by a variable and
579
725
  summarises it by an expression:
580
726
 
581
727
 
582
- ```r
728
+ ``` r
583
729
  set.seed(123)
584
730
 
585
731
  df <- data.frame(
@@ -589,33 +735,33 @@ df <- data.frame(
589
735
  b = sample(5)
590
736
  )
591
737
 
592
- as.data.frame(df)
738
+ as.data.frame(df)
593
739
  ```
594
740
 
595
741
  ```
596
742
  ## g1 g2 a b
597
- ## 1 1 1 2 1
598
- ## 2 1 2 4 3
599
- ## 3 2 1 5 4
600
- ## 4 2 2 3 2
601
- ## 5 2 1 1 5
743
+ ## 1 1 1 3 3
744
+ ## 2 1 2 2 1
745
+ ## 3 2 1 5 2
746
+ ## 4 2 2 4 5
747
+ ## 5 2 1 1 4
602
748
  ```
603
749
 
604
- ```r
750
+ ``` r
605
751
  d2 <- df %>%
606
752
  group_by(g1) %>%
607
753
  summarise(a = mean(a))
608
754
 
609
- as.data.frame(d2)
755
+ as.data.frame(d2)
610
756
  ```
611
757
 
612
758
  ```
613
- ## g1 a
614
- ## 1 1 3
615
- ## 2 2 3
759
+ ## g1 a
760
+ ## 1 1 2.500000
761
+ ## 2 2 3.333333
616
762
  ```
617
763
 
618
- ```r
764
+ ``` r
619
765
  d2 <- df %>%
620
766
  group_by(g2) %>%
621
767
  summarise(a = mean(a))
@@ -624,15 +770,15 @@ as.data.frame(d2)
624
770
  ```
625
771
 
626
772
  ```
627
- ## g2 a
628
- ## 1 1 2.666667
629
- ## 2 2 3.500000
773
+ ## g2 a
774
+ ## 1 1 3
775
+ ## 2 2 3
630
776
  ```
631
777
 
632
- As shown by Hardley, one might expect this function to do the trick:
778
+ As shown by Hadley, one might expect this function to do the trick:
633
779
 
634
780
 
635
- ```r
781
+ ``` r
636
782
  my_summarise <- function(df, group_var) {
637
783
  df %>%
638
784
  group_by(group_var) %>%
@@ -645,61 +791,66 @@ my_summarise <- function(df, group_var) {
645
791
 
646
792
  In order to solve this problem, coding with dplyr requires the introduction of many new concepts
647
793
  and functions such as 'quo', 'quos', 'enquo', 'enquos', '!!' (bang bang), '!!!' (triple bang).
648
- Again, we'll leave to Hardley the explanation on how to use all those functions.
794
+ Again, we'll leave to Hadley the explanation on how to use all those functions.
649
795
 
650
796
  Now, let's try to implement the same function in galaaz. The next code block first prints the
651
- 'df' data frame define previously in R (to access an R variable from Galaaz, we use the tilda
797
+ 'df' data frame defined previously in R (to access an R variable from Galaaz, we use the tilde
652
798
  operator '~' applied to the R variable name as symbol, i.e., ':df'. We then create the
653
799
  'my_summarize' method and call it passing the R data frame and the group by variable ':g1':
654
800
 
655
801
 
656
- ```ruby
657
- puts ~:df
658
- print "\n"
802
+ ``` ruby
803
+ puts ~R[:df]
804
+ print "
805
+ "
659
806
 
660
807
  def my_summarize(df, group_var)
661
808
  df.group_by(group_var).
662
- summarize(a: :a.mean)
809
+ summarize(a: R[:a].mean)
663
810
  end
664
811
 
665
- puts my_summarize(:df, :g1).as__data__frame
812
+ puts my_summarize(~R[:df], R[:g1])
666
813
  ```
667
814
 
668
815
  ```
669
816
  ## g1 g2 a b
670
- ## 1 1 1 2 1
671
- ## 2 1 2 4 3
672
- ## 3 2 1 5 4
673
- ## 4 2 2 3 2
674
- ## 5 2 1 1 5
817
+ ## 1 1 1 3 3
818
+ ## 2 1 2 2 1
819
+ ## 3 2 1 5 2
820
+ ## 4 2 2 4 5
821
+ ## 5 2 1 1 4
675
822
  ##
676
- ## g1 a
677
- ## 1 1 3
678
- ## 2 2 3
823
+ ## # A tibble: 2 × 2
824
+ ## g1 a
825
+ ## <dbl> <dbl>
826
+ ## 1 1 2.5
827
+ ## 2 2 3.33
679
828
  ```
680
829
  It works!!! Well, let's make sure this was not just some coincidence
681
830
 
682
831
 
683
- ```ruby
684
- puts my_summarize(:df, :g2).as__data__frame
832
+ ``` ruby
833
+ puts my_summarize(~R[:df], R[:g2])
685
834
  ```
686
835
 
687
836
  ```
688
- ## g2 a
689
- ## 1 1 2.666667
690
- ## 2 2 3.500000
837
+ ## # A tibble: 2 × 2
838
+ ## g2 a
839
+ ## <dbl> <dbl>
840
+ ## 1 1 3
841
+ ## 2 2 3
691
842
  ```
692
843
 
693
844
  Great, everything is fine! No magic, no new functions, no complexities, just normal, standard Ruby
694
845
  code. If you've ever done NSE in R, this certainly feels much safer and easy to implement.
695
846
 
696
- # Different input variables
847
+ ## Different input variables
697
848
 
698
849
  In the previous section we've managed to get rid of all NSE formulation for a simple example, but
699
850
  does this remain true for more complex examples, or will the Galaaz way prove inpractical for
700
851
  more complex code?
701
852
 
702
- In the next example Hardley proposes us to write a function that given an expression such as 'a'
853
+ In the next example Hadley proposes us to write a function that given an expression such as 'a'
703
854
  or 'a * b', calculates three summaries. What we want a function that does the same as these R
704
855
  statements:
705
856
 
@@ -720,7 +871,7 @@ summarise(df, mean = mean(a * b), sum = sum(a * b), n = n())
720
871
  Let's try it in galaaz:
721
872
 
722
873
 
723
- ```ruby
874
+ ``` ruby
724
875
  def my_summarise2(df, expr)
725
876
  df.summarize(
726
877
  mean: E.mean(expr),
@@ -729,8 +880,8 @@ def my_summarise2(df, expr)
729
880
  )
730
881
  end
731
882
 
732
- puts my_summarise2((~:df), :a)
733
- puts my_summarise2((~:df), :a * :b)
883
+ puts my_summarise2((~R[:df]), :a)
884
+ puts my_summarise2((~R[:df]), R[:a] * R[:b])
734
885
  ```
735
886
 
736
887
  ```
@@ -743,9 +894,9 @@ puts my_summarise2((~:df), :a * :b)
743
894
  Once again, there is no need to use any special theory or functions. The only point to be
744
895
  careful about is the use of 'E' to build expressions from functions 'mean', 'sum' and 'n'.
745
896
 
746
- # Different input and output variable
897
+ ## Different input and output variable
747
898
 
748
- Now the next challenge presented by Hardley is to vary the name of the output variables based on
899
+ Now the next challenge presented by Hadley is to vary the name of the output variables based on
749
900
  the received expression. So, if the input expression is 'a', we want our data frame columns to
750
901
  be named 'mean\_a' and 'sum\_a'. Now, if the input expression is 'b', columns
751
902
  should be named 'mean\_b' and 'sum\_b'.
@@ -771,13 +922,13 @@ mutate(df, mean_b = mean(b), sum_b = sum(b))
771
922
  #> 4 2 2 5 4 3 15
772
923
  #> # … with 1 more row
773
924
  ```
774
- In order to solve this problem in R, Hardley needs to introduce some more new functions and notations:
925
+ In order to solve this problem in R, Hadley needs to introduce some more new functions and notations:
775
926
  'quo_name' and the ':=' operator from package 'rlang'
776
927
 
777
928
  Here is our Ruby code:
778
929
 
779
930
 
780
- ```ruby
931
+ ``` ruby
781
932
  def my_mutate(df, expr)
782
933
  mean_name = "mean_#{expr.to_s}"
783
934
  sum_name = "sum_#{expr.to_s}"
@@ -786,23 +937,23 @@ def my_mutate(df, expr)
786
937
  sum_name => E.sum(expr))
787
938
  end
788
939
 
789
- puts my_mutate((~:df), :a)
790
- puts my_mutate((~:df), :b)
940
+ puts my_mutate((~R[:df]), :a)
941
+ puts my_mutate((~R[:df]), :b)
791
942
  ```
792
943
 
793
944
  ```
794
945
  ## g1 g2 a b mean_a sum_a
795
- ## 1 1 1 2 1 3 15
796
- ## 2 1 2 4 3 3 15
797
- ## 3 2 1 5 4 3 15
798
- ## 4 2 2 3 2 3 15
799
- ## 5 2 1 1 5 3 15
946
+ ## 1 1 1 3 3 3 15
947
+ ## 2 1 2 2 1 3 15
948
+ ## 3 2 1 5 2 3 15
949
+ ## 4 2 2 4 5 3 15
950
+ ## 5 2 1 1 4 3 15
800
951
  ## g1 g2 a b mean_b sum_b
801
- ## 1 1 1 2 1 3 15
802
- ## 2 1 2 4 3 3 15
803
- ## 3 2 1 5 4 3 15
804
- ## 4 2 2 3 2 3 15
805
- ## 5 2 1 1 5 3 15
952
+ ## 1 1 1 3 3 3 15
953
+ ## 2 1 2 2 1 3 15
954
+ ## 3 2 1 5 2 3 15
955
+ ## 4 2 2 4 5 3 15
956
+ ## 5 2 1 1 4 3 15
806
957
  ```
807
958
  It really seems that "Non Standard Evaluation" is actually quite standard in Galaaz! But, you
808
959
  might have noticed a small change in the way the arguments to the mutate method were called.
@@ -812,30 +963,33 @@ and variable mean\_name is not followed by ':' but by '=>'. This is standard Ru
812
963
 
813
964
  [explain....]
814
965
 
815
- # Capturing multiple variables
966
+ ## Capturing multiple variables
816
967
 
817
- Moving on with new complexities, Hardley proposes us to solve the problem in which the
968
+ Moving on with new complexities, Hadley proposes us to solve the problem in which the
818
969
  summarise function will receive any number of grouping variables.
819
970
 
820
971
  This again is quite standard Ruby. In order to receive an undefined number of paramenters
821
972
  the paramenter is preceded by '*':
822
973
 
823
974
 
824
- ```ruby
975
+ ``` ruby
825
976
  def my_summarise3(df, *group_vars)
826
977
  df.group_by(*group_vars).
827
978
  summarise(a: E.mean(:a))
828
979
  end
829
980
 
830
- puts my_summarise3((~:df), :g1, :g2).as__data__frame
981
+ puts my_summarise3((~R[:df]), :g1, :g2)
831
982
  ```
832
983
 
833
984
  ```
834
- ## g1 g2 a
835
- ## 1 1 1 2
836
- ## 2 1 2 4
837
- ## 3 2 1 3
838
- ## 4 2 2 3
985
+ ## # A tibble: 4 × 3
986
+ ## # Groups: g1 [2]
987
+ ## g1 g2 a
988
+ ## <dbl> <dbl> <dbl>
989
+ ## 1 1 1 3
990
+ ## 2 1 2 2
991
+ ## 3 2 1 3
992
+ ## 4 2 2 4
839
993
  ```
840
994
 
841
995
  # Why does R require NSE and Galaaz does not?
@@ -853,7 +1007,7 @@ In Ruby, there is no lazy evaluation of parameters and 'a' is always a variable
853
1007
  Variables assume their value as soon as they are used, so 'x = a' is immediately evaluate and
854
1008
  variable 'x' will receive the value of variable 'a' as soon as the Ruby statement is executed.
855
1009
  Ruby also provides the notion of a symbol; ':a' is a symbol and does not evaluate to anything.
856
- Galaaz uses Ruby symbols to build expressions that are not bound to anything: ':a.eq :b' is
1010
+ Galaaz uses Ruby symbols to build expressions that are not bound to anything: 'R[:a].eq R[:b]' is
857
1011
  clearly an expression and has no relationship whatsoever with the statment 'a = b'. By using
858
1012
  symbols, variables and expressions all the possible ambiguities that are found in R are
859
1013
  eliminated in Galaaz.
@@ -863,12 +1017,12 @@ of input they are expecting, they might be expecting regular variables or they m
863
1017
  expecting expressions and the R function will know how to deal with an input of the form
864
1018
  'a = b', now for the Ruby developer it might not be immediately clear if it should call the
865
1019
  function passing the value 'true' if variable 'a' is equal to variable 'b' or if it should
866
- call the function passing the expression ':a.eq :b'.
1020
+ call the function passing the expression 'R[:a].eq R[:b]'.
867
1021
 
868
1022
 
869
1023
  # Advanced dplyr features
870
1024
 
871
- In the blog: Programming with dplyr by using dplyr (https://www.r-bloggers.com/programming-with-dplyr-by-using-dplyr/) Iñaki Úcar shows surprise that some R users are trying to code in dplyr avoiding
1025
+ In the blog: [Programming with dplyr by using dplyr](https://www.r-bloggers.com/programming-with-dplyr-by-using-dplyr/) Iñaki Úcar shows surprise that some R users are trying to code in dplyr avoiding
872
1026
  the use of NSE. For instance he says:
873
1027
 
874
1028
  > Take the example of seplyr. It stands for standard evaluation dplyr, and enables us to
@@ -886,45 +1040,28 @@ In the following examples, we show the use of functions 'group\_by\_at', 'summar
886
1040
  features of characters in the Starwars movies:
887
1041
 
888
1042
 
889
- ```ruby
890
- puts (~:starwars).head.as__data__frame
891
- ```
892
-
893
- ```
894
- ## name height mass hair_color skin_color eye_color birth_year
895
- ## 1 Luke Skywalker 172 77 blond fair blue 19.0
896
- ## 2 C-3PO 167 75 <NA> gold yellow 112.0
897
- ## 3 R2-D2 96 32 <NA> white, blue red 33.0
898
- ## 4 Darth Vader 202 136 none white yellow 41.9
899
- ## 5 Leia Organa 150 49 brown light brown 19.0
900
- ## 6 Owen Lars 178 120 brown, grey light blue 52.0
901
- ## gender homeworld species
902
- ## 1 male Tatooine Human
903
- ## 2 <NA> Tatooine Droid
904
- ## 3 <NA> Naboo Droid
905
- ## 4 male Tatooine Human
906
- ## 5 female Alderaan Human
907
- ## 6 male Tatooine Human
908
- ## films
909
- ## 1 Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
910
- ## 2 Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope
911
- ## 3 Attack of the Clones, The Phantom Menace, Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
912
- ## 4 Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope
913
- ## 5 Revenge of the Sith, Return of the Jedi, The Empire Strikes Back, A New Hope, The Force Awakens
914
- ## 6 Attack of the Clones, Revenge of the Sith, A New Hope
915
- ## vehicles starships
916
- ## 1 Snowspeeder, Imperial Speeder Bike X-wing, Imperial shuttle
917
- ## 2
918
- ## 3
919
- ## 4 TIE Advanced x1
920
- ## 5 Imperial Speeder Bike
921
- ## 6
922
- ```
923
- The grouped_mean function bellow will receive a grouping variable and calculate summaries for
1043
+ ``` ruby
1044
+ puts (~R[:starwars]).head
1045
+ ```
1046
+
1047
+ ```
1048
+ ## # A tibble: 6 × 14
1049
+ ## name height mass hair_color skin_color eye_color birth_year sex gender
1050
+ ## <chr> <int> <dbl> <chr> <chr> <chr> <dbl> <chr> <chr>
1051
+ ## 1 Luke Sky… 172 77 blond fair blue 19 male mascu…
1052
+ ## 2 C-3PO 167 75 <NA> gold yellow 112 none mascu…
1053
+ ## 3 R2-D2 96 32 <NA> white, bl… red 33 none mascu…
1054
+ ## 4 Darth Va… 202 136 none white yellow 41.9 male mascu…
1055
+ ## 5 Leia Org… 150 49 brown light brown 19 fema… femin…
1056
+ ## 6 Owen Lars 178 120 brown, gr… light blue 52 male mascu…
1057
+ ## # ℹ 5 more variables: homeworld <chr>, species <chr>, films <list>,
1058
+ ## # vehicles <list>, starships <list>
1059
+ ```
1060
+ The grouped_mean function below will receive a grouping variable and calculate summaries for
924
1061
  the value\_variables given:
925
1062
 
926
1063
 
927
- ```r
1064
+ ``` r
928
1065
  grouped_mean <- function(data, grouping_variables, value_variables) {
929
1066
  data %>%
930
1067
  group_by_at(grouping_variables) %>%
@@ -935,7 +1072,22 @@ grouped_mean <- function(data, grouping_variables, value_variables) {
935
1072
 
936
1073
  gm = starwars %>%
937
1074
  grouped_mean("eye_color", c("mass", "birth_year"))
1075
+ ```
1076
+
1077
+ ```
1078
+ ## Warning: `funs()` was deprecated in dplyr 0.8.0.
1079
+ ## ℹ Please use a list of either functions or lambdas:
1080
+ ##
1081
+ ## # Simple named list: list(mean = mean, median = median)
1082
+ ##
1083
+ ## # Auto named with `tibble::lst()`: tibble::lst(mean, median)
1084
+ ##
1085
+ ## # Using lambdas list(~ mean(., trim = .2), ~ median(., na.rm = TRUE))
1086
+ ## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
1087
+ ## generated.
1088
+ ```
938
1089
 
1090
+ ``` r
939
1091
  as.data.frame(gm)
940
1092
  ```
941
1093
 
@@ -961,37 +1113,49 @@ as.data.frame(gm)
961
1113
  The same code with Galaaz, becomes:
962
1114
 
963
1115
 
964
- ```ruby
1116
+ ``` ruby
965
1117
  def grouped_mean(data, grouping_variables, value_variables)
966
1118
  data.
967
1119
  group_by_at(grouping_variables).
968
1120
  mutate(count: E.n).
969
- summarise_at(E.c(value_variables, "count"), ~:mean, na__rm: true).
1121
+ summarise_at(E.c(value_variables, "count"), R[:mean], na__rm: true).
970
1122
  rename_at(value_variables, E.funs(E.paste0("mean_", value_variables)))
971
1123
  end
972
1124
 
973
- puts grouped_mean((~:starwars), "eye_color", E.c("mass", "birth_year")).as__data__frame
1125
+ puts grouped_mean((~R[:starwars]), "eye_color", E.c("mass", "birth_year"))
974
1126
  ```
975
1127
 
976
1128
  ```
977
- ## eye_color mean_mass mean_birth_year count
978
- ## 1 black 76.28571 33.00000 10
979
- ## 2 blue 86.51667 67.06923 19
980
- ## 3 blue-gray 77.00000 57.00000 1
981
- ## 4 brown 66.09231 108.96429 21
982
- ## 5 dark NaN NaN 1
983
- ## 6 gold NaN NaN 1
984
- ## 7 green, yellow 159.00000 NaN 1
985
- ## 8 hazel 66.00000 34.50000 3
986
- ## 9 orange 282.33333 231.00000 8
987
- ## 10 pink NaN NaN 1
988
- ## 11 red 81.40000 33.66667 5
989
- ## 12 red, blue NaN NaN 1
990
- ## 13 unknown 31.50000 NaN 3
991
- ## 14 white 48.00000 NaN 1
992
- ## 15 yellow 81.11111 76.38000 11
1129
+ ## # A tibble: 15 × 4
1130
+ ## eye_color mean_mass mean_birth_year count
1131
+ ## <chr> <dbl> <dbl> <dbl>
1132
+ ## 1 black 76.3 33 10
1133
+ ## 2 blue 86.5 67.1 19
1134
+ ## 3 blue-gray 77 57 1
1135
+ ## 4 brown 66.1 109. 21
1136
+ ## 5 dark NaN NaN 1
1137
+ ## 6 gold NaN NaN 1
1138
+ ## 7 green, yellow 159 NaN 1
1139
+ ## 8 hazel 66 34.5 3
1140
+ ## 9 orange 282. 231 8
1141
+ ## 10 pink NaN NaN 1
1142
+ ## 11 red 81.4 33.7 5
1143
+ ## 12 red, blue NaN NaN 1
1144
+ ## 13 unknown 31.5 NaN 3
1145
+ ## 14 white 48 NaN 1
1146
+ ## 15 yellow 81.1 76.4 11
993
1147
  ```
994
1148
 
1149
+ # Further reading
1150
+
1151
+ * [JRuby](https://www.jruby.org/) — Ruby on the JVM (Galaaz 2.0)
1152
+ * [How to make Beautiful Ruby Plots with Galaaz](https://medium.freecodecamp.org/how-to-make-beautiful-ruby-plots-with-galaaz-320848058857) (plots; narrative partly pre-2.0)
1153
+ * [Ruby Plotting with Galaaz in GraalVM](https://towardsdatascience.com/ruby-plotting-with-galaaz-an-example-of-tightly-coupling-ruby-and-r-in-graalvm-520b69e21021) (older stack; ideas still useful)
1154
+ * [How to do reproducible research in Ruby with gKnit](https://towardsdatascience.com/how-to-do-reproducible-research-in-ruby-with-gknit-c26d2684d64e)
1155
+ * [R for Data Science](https://r4ds.had.co.nz/)
1156
+ * [Advanced R](https://adv-r.hadley.nz/)
1157
+ * Historical context: [GraalVM](https://www.graalvm.org/), [TruffleRuby](https://github.com/oracle/truffleruby), [FastR](https://github.com/oracle/fastr)
1158
+
995
1159
  # Conclusion
996
1160
 
997
1161
  Ruby and Galaaz provide a nice framework for developing code that uses R functions. Although R is