statsailr_procs_base 0.1.0

data/README.md

@@ -0,0 +1,195 @@
+# About statsailr_procs_base gem
+
+This 'statsailr_procs_base' gem provides a collection of fundamental PROC settings and make PROCs available for StatSailr program. The 'statsailr' gem does not have PROC settings, and this gem is essential for StatSailr to provide a useful statistics system.
+
+The reason why these basic PROC settings are separated as 'statsailr_procs_base' gem from the main 'statsailr' gem is maintainability. The StatSailr system (i.e. main system + PROCs) is still under development, and is updated frequently. PROCs and main system do not have specific release cycles yet. For those reasons, it is better to update them as separate gems.
+
+
+## Installation
+
+This gem is usually installed together with 'statsailr' gem, following the dependency setting in statsailr.gemspec. If you want to try the most up-to-date version, manually install it from Github.
+
+
+## How each StatSailr's PROC functionality is defined
+
+How each PROC block behaves or what instructions they have is defined as PROC settings. This gem provides those PROC settings. StatSailr calls PROCs providing gem's path_to_proc_setting method (StatSailr::ProcsBase::path_to_proc_setting in this gem), and uses the path information to access the gem's PROC settings. The method returns the base dicrectory path that contains PROC settings (a.k.a. lib/statsailr_procs_base/proc_setting directory). Based on the PROCs called from StatSailr, its corresponding setting files are identified and settings are loaded. Based on the settings, appropriate R functions are generated and executed.
+
+Following these settings, StatSailr converts PROC block to a series of R functions. PROC settings are written in Ruby modules. Internally StatSailr creates a Ruby object for each block, and those objects extend their functionality by mixing-in corresponding PROC setting module. Currently, those files and module names are determined from PROC's command name. For example, if the block starts with 'PROC CAT', 'ProcCat' module written in 'proc_cat.rb' is the module to be used for functionality extension.
+
+
+## Understanding PROC settings
+
+The following is typical PROC setting and its explanation.
+
+```
+# proc_print.rb
+
+module ProcPrint
+  def setting_for_head( setting )
+    setting.libname = "utils"
+    setting.func_name = "head"
+    setting.main_arg_and_how_to_treat =  [ "n", "read_as_intvec", "allow_nil"]
+    setting.runtime_args = { "x" => param("data")}
+    setting.store_result = false
+    setting.print_opt = true
+  end
+end
+```
+
+* module ProcPrint
+    + This module is setting for 'PROC PRINT' block.
+* def setting_for_head
+    + This function is setting for 'head' instruction.
+* setting.libname
+    + When libname is specified, the function is looked up from the package.
+        + In the above example, it is equivalent to utils::head()
+    + When libname is set nil, the function is looked up from environment or Global.
+* setting.envname
+    + When libname is set nil, and envname is specified, the function is looked up from this environment.
+        + This enables R functions to be called with environment name, and each PROC can implement its own R functions within their environment, and prevents function name conflicts.
+* setting.func_name
+    + R's function name to be called.
+        + package name or environment name can be specified as mentioned above.
+* setting.main_arg_and_how_to_treat
+    + This is spcified in Array of length 3.
+    + The first element is R's function argument name to be passed to.
+    + The second element of its value is how to treat the main argument.
+    + The third element of its value is whether to allow not specifying main argument. "allow_nil" or "no_nil"
+        + In the above example, value specified for main argument is interpreted as intvec, and is passed to R funcion's 'x' argument. 
+        + "allow_nil" allows users to omit main argument, and default value of 6 is used. (See head documentation https://stat.ethz.ch/R-manual/R-devel/library/utils/html/head.html ) 
+    + If the instruction does not take a main argument, nil or [nil, nil, nil] is specified.
+* setting.runtime_args
+    + This is specified in Hash or nil.
+    + This setting is used to pass objects that are generated at runtime to R function.
+        + The objects include objects spcified in PROC options and results of previous instructions within the same block.
+    + To define, param(), result() and one_from() methods can be used. Details are described later.
+* setting.store_result
+    + true/false
+        + If this setting is true, this instruction's result can be accessed via setting.runtime_args.
+        + If set false, this instruction's result cannot be accessed.
+* setting.print_opt
+    + true/false
+        + If this setting is true, the result is printed out using "print" function.
+        + If set false, the result is not printed out.
+    + String
+        + If String value is set, the result is printed out using the name specified.
+* setting.plot_opt
+    + true/false
+        + Under the situation where graphics device does not show on display and tries to output to a file, this setting is valid.
+            + If true, StatSailr conducts dev.copy() at the end of this current instruction, and saves graphics device content to file.
+        + If the current graphics device outputs to display, this setting is useless and ignored.
+
+
+## More about main_arg_and_how_to_treat
+
+The following methods can be used to parse and convert PROC instruction main argument to R object. This method name needs to be specified in String.
+
+* read_as_formula
+* read_as_strvec
+* read_as_one_str
+* read_as_numvec
+* read_as_intvec
+* read_as_realvec
+* read_as_symbol
+* read_symbols_as_strvec
+
+
+
+## More about setting.runtime_args
+
+This setting is used to access objects that are generated at runtime, and pass those objects to R function. These objects include objects that are spcified in PROC args, and results that are returned from previous instrutions within the same block.
+
+* param() method
+    + param() can access object specified in PROC option. 
+      + (The reason why 'param' is used for this access is that PROC option parameter is managed by RBridge::ParamManager.)
+    + For example, "x" => param("data") means that passing an object that is named as "data" in PROC option, and passing it to "x" argument of R function. 
+* result() method
+    + result() can access a result object of an instruction that is specified by instruction name.
+    + If result() method takes multiple argument or multiple instruction names, it returns ResultNameArray object.
+        + The last result from specified instruction results is used.
+* Pointer to R object
+    + R object that is statically generated can also be used. Usually this can be used as a default value.
+* one_from() method
+    + one_from() method can take ResultName, ResultNameAray, PramName or pointer to R object.
+    + one_from() can take multiple arguments, and their order defines priority about which object to use.
+    + (e.g.) 'setting.runtime_args = {"data" => one_from( result("factor", "numeric"), param("data")) }' means the following
+        + If there are already instructions of "factor" or "numeric", use the last result of them.
+        + If not, object that is specified by "data" in PROC options in used.
+
+
+## Enable custom R function
+
+StatSailr PROC instructions can also call custom R functions that are not defined in R's libraries. source_r_file() method does this work. ProcSettingModule is a module that provides source_r_file() method. source_r_file() method lets R function defined in the file available from R or StatSailr PROC instructions.
+
+The following example enables functions in proc_scatter.R . ( The setting file name is proc_scatter.rb and File.basename(__FILE__ , ".rb") + ".R" replaces the extension with ".R".) Note that source_r_file()'s first argument takes absolute path for the directory containing the R file. This restriction prevents from loading unintentional files.
+
+```
+# proc_scatter.rb
+module ProcScatter
+  include ProcSettingModule
+  ...
+  source_r_file( __dir__, File.basename(__FILE__ , ".rb") + ".R")
+  ...
+  ...
+end
+```
+
+## Validate PROC options
+
+Each PROC option (i.e. option that follows PROC command) can be validated. Option name, option value type and option requirement can be checked, and types are converted appropriately if necessary.
+
+For example, in many PROCs, data option is required. To guarantee the existence of data option, pass "data" with "required: true" to validate_option().
+
+* validate_option() method takes the following arguments
+    + option_name : option to validate
+    + is_a : value assigned to the option should belong to type(s) specified.
+        + "Float", "Integer", "String" or "SymbolR" can be specified.
+        + Array can be used to allow some types.
+    + as : value is finally dealt as this type specified.
+    + required : PROC command requires the option or not
+
+The following example specifies PROC PRINT requires 'data' option, SymbolR or String is accepted for its value, and the value is finally dealt as SymbolR.
+
+```
+module ProcPrint  
+  include ProcSettingModule
+  ...
+  validate_option("data", is_a: ["SymbolR", "String"], as: "SymbolR", required: true)
+```
+
+
+## Share common settings with multiple settings
+
+StatSailr PROC instruction settings can be shared by multiple setting modules. The shared setting can be included using add_setting_from() method defined in ProcSettingModule. The included file is also written in module, and the format is almost the same.
+
+In thee following example, ProcScatter or ProPlot?????? include DevCopySetting which provides functionality of dev.copy() function. 
+
+
+```
+# proc_plot.rb
+module ProcPlot
+  include ProcSettingModule
+  add_setting_from( __dir__, "proc_common/dev_copy.rb" )  # This file should have DevCopy module.
+  ...
+  ...
+end
+
+# proc_common/dev_copy.rb
+module DevCopySetting
+  def setting_for_dev_copy( setting )
+    ...
+    ...
+  end
+end
+```
+
+## Contributing
+
+Bug reports are welcome on GitHub at https://github.com/niceume/statsailr_procs_base.
+
+
+## License
+
+The gem is available as open source under the terms of the [GPL v3 License](https://www.gnu.org/licenses/gpl-3.0.en.html).
+
+

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end
+
+task default: :test

data/bin/console

@@ -0,0 +1,15 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "statsailr_procs_base"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/lib/statsailr_procs_base.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require_relative "statsailr_procs_base/version"
+require_relative "statsailr_procs_base/path"
+require_relative "statsailr_procs_base/check_statsailr_version"
+
+module StatSailr
+  module ProcsBase
+    class Error < StandardError; end
+
+    # Your code goes here...
+  end
+end
+
+StatSailr::ProcsBase::check_statsailr_version()
+

data/lib/statsailr_procs_base/check_statsailr_version.rb

@@ -0,0 +1,18 @@
+module StatSailr
+  module ProcsBase
+    def self.check_statsailr_version
+      statsailr_version = StatSailr::VERSION
+      statsailr_version_at_least = "0.7.1"
+      return version_should_be_at_least( statsailr_version , statsailr_version_at_least ) 
+    end
+
+    def self.version_should_be_at_least( gem_version, str_version )
+      if( Gem::Version.new(gem_version) >= Gem::Version.new(str_version) )
+        return true
+      else
+        puts "\e[1m" + "WARNING: statsailr gem (#{gem_version}) needs to be newer or equal to ver. #{str_version}" + "\e[22m"
+        return false
+      end
+    end
+  end
+end

data/lib/statsailr_procs_base/path.rb

@@ -0,0 +1,7 @@
+module StatSailr
+  module ProcsBase
+    def self.path_to_proc_setting
+      return __dir__ + "/" + "proc_setting"
+    end
+  end
+end

data/lib/statsailr_procs_base/proc_setting/common_utility.R

@@ -0,0 +1,16 @@
+if_exist_else = function( name, lst, default_val ){
+  if( name %in% names(lst)){
+    return( lst[[name]] )
+  }else{
+    return( default_val )
+  }
+}
+
+get_pkg_fun = function(x) {
+    if(length(grep("::", x))>0) {
+        parts<-strsplit(x, "::")[[1]]
+        getExportedValue(parts[1], parts[2])
+    } else {
+        x
+    }
+}

data/lib/statsailr_procs_base/proc_setting/proc_cat.R

@@ -0,0 +1,65 @@
+sts_cat = new.env()
+
+sts_cat$table = function( data , vars , missing = FALSE, ... ){
+  if( (! is.character(vars)) || length(vars) != 2 ){
+    stop("vars argument requires character vector with size of 2")
+  }
+
+  df_for_table = data[ , vars ]
+  if( missing ){
+    freq_ori = table(df_for_table, useNA = "ifany")
+  }else{
+    freq_ori = table(df_for_table)
+  }
+
+  freq = addmargins( freq_ori )
+  allper = format( addmargins( prop.table( freq_ori) * 100), digits = 4)
+  rowper = format( addmargins( prop.table( freq_ori, 1) * 100), digits = 4)
+  colper = format( addmargins( prop.table( freq_ori, 2) * 100), digits = 4)
+
+  nrow = nrow(freq)
+  ncol = ncol(freq)
+  nrow_ori = nrow(freq_ori)
+
+  temp = as.array( numeric( nrow * ncol * 4 ))
+  dim( temp ) = c( nrow * 4, ncol )
+  result = as.table( temp )
+
+  colnames(result) = c( colnames( freq_ori ) , "Total")
+
+  row_label1 = c( rep("%",nrow_ori ) , "%")
+  row_label2 = c( rep("Row%",nrow_ori ) , "")
+  row_label3 = c( rep("Col%",nrow_ori ) , "")
+
+  rownames(result) = as.vector( mapply( c, c( rownames( freq_ori ), "Total" ), row_label1 , row_label2 , row_label3 ))
+
+  cat( paste( "\t", names(freq), "\n", sep = "\t", collapse = "\t" ))
+  for( irow in seq(1, nrow) ){
+    for( icol in seq(1, ncol) ){
+      result[ (irow-1)*4 + 1, icol] = freq[irow, icol]
+    }
+    for( icol in seq(1, ncol) ){
+      result[ (irow-1)*4 + 2, icol] = allper[irow, icol]
+    }
+    for( icol in seq(1, ncol) ){
+      if(irow != nrow && icol != ncol){
+        result[ (irow-1)*4 + 3, icol] = rowper[irow, icol]
+      }else{
+        result[ (irow-1)*4 + 3, icol] = NA
+      }
+    }
+    for( icol in seq(1, ncol) ){
+      if(icol != ncol && irow!= nrow){
+        result[ (irow-1)*4 + 4, icol] = colper[irow, icol]
+      }else{
+        result[ (irow-1)*4 + 4, icol] = NA
+      }
+    }
+  }
+ 
+  cat( paste( vars[1] , " vs ", vars[2], "\n" ) )
+  print( result )
+
+  return( freq_ori )
+}
+

data/lib/statsailr_procs_base/proc_setting/proc_cat.rb

@@ -0,0 +1,44 @@
+module ProcCat
+  include ProcSettingModule
+
+  source_r_file( __dir__, File.basename(__FILE__ , ".rb") + ".R")
+  validate_option("data", is_a: ["SymbolR", "String"], as: "SymbolR" , required: true)
+
+  def setting_for_table( setting )
+    setting.libname = nil
+    setting.envname = "sts_cat"
+    setting.func_name = "table"
+    setting.main_arg_and_how_to_treat = ["vars", :read_as_strvec, :no_nil]
+    setting.runtime_args = {"data" => param("data") }
+    setting.store_result = true
+    setting.print_opt = false
+  end
+
+  def setting_for_xtabs( setting )
+    setting.libname = "stats"
+    setting.func_name = "xtabs"
+    setting.main_arg_and_how_to_treat = ["formula", :read_as_formula, :no_nil]
+    setting.runtime_args = {"data" => param("data") }
+    setting.store_result = true
+    setting.print_opt = true
+  end
+
+  def setting_for_fisher_test( setting )
+    setting.libname = "stats"
+    setting.func_name = "fisher.test"
+    setting.main_arg_and_how_to_treat = nil
+    setting.runtime_args = {"x" => one_from( result("table"), result("xtabs") ) }
+    setting.store_result = true
+    setting.print_opt = true
+  end
+
+  def setting_for_chisq_test( setting )
+    setting.libname = "stats"
+    setting.func_name = "chisq.test"
+    setting.main_arg_and_how_to_treat = nil
+    setting.runtime_args = {"x" => one_from( result("table"), result("xtabs") ) }
+    setting.store_result = true
+    setting.print_opt = true
+  end
+end
+

data/lib/statsailr_procs_base/proc_setting/proc_common/dev_copy.R

@@ -0,0 +1,6 @@
+sts_dev_copy = new.env()
+
+sts_dev_copy$dev_copy = function( device, ... ){
+  dev.copy( device , ...)
+  dev.off()
+}

data/lib/statsailr_procs_base/proc_setting/proc_common/dev_copy.rb

@@ -0,0 +1,15 @@
+module DevCopySetting
+  include ProcSettingModule
+  source_r_file(__dir__, File.basename(__FILE__ , ".rb") + ".R")
+
+  def setting_for_dev_copy( setting )
+    setting.libname = nil
+    setting.envname = "sts_dev_copy"
+    setting.func_name = "dev_copy"
+    setting.main_arg_and_how_to_treat = [ "device" , :read_as_symbol , :allow_nil ]
+    setting.runtime_args = {}
+    setting.store_result = false
+    setting.print_opt = false
+  end
+end
+

data/lib/statsailr_procs_base/proc_setting/proc_common/factor.R

@@ -0,0 +1,7 @@
+sts_factor = new.env()
+
+sts_factor$convert_to_factor = function( data, vars ){
+  data[vars] = lapply(data[vars], as.factor)
+  print( paste( "as.factor is applied to", paste( vars, collapse=",") , sep=" ") )
+  return( data )
+}