statsample 0.13.0 → 0.13.1

data/History.txt

@@ -1,3 +1,11 @@
+=== 0.13.1 / 2010-07-03
+
+* Rserve extensions for dataset and vector operational
+* On x86_64, variance from gsl is not exactly equal to sum of variance-covariance on Statsample::Reliability::Scale, but in delta 1e-10
+* Updated README.txt
+* Reliability::ScaleAnalysis uses covariance matrix for 'if deleted' calculations to optimize memory and speed. Test for 'if deleted' statistics
+* More string translated. Added dependency on tetrachoric on parallel analysis
+
 === 0.13.0 / 2010-06-13
 
 * Polychoric and Tetrachoric moved to gem statsample-bivariate-extension

data/Manifest.txt

@@ -84,6 +84,7 @@ lib/statsample/reliability.rb
 lib/statsample/reliability/multiscaleanalysis.rb
 lib/statsample/reliability/scaleanalysis.rb
 lib/statsample/resample.rb
+lib/statsample/rserve_extension.rb
 lib/statsample/srs.rb
 lib/statsample/test.rb
 lib/statsample/test/f.rb
@@ -121,6 +122,7 @@ test/test_permutation.rb
 test/test_regression.rb
 test/test_reliability.rb
 test/test_resample.rb
+test/test_rserve_extension.rb
 test/test_srs.rb
 test/test_statistics.rb
 test/test_stest.rb

data/README.txt

@@ -10,12 +10,12 @@ A suite for basic and advanced statistics on Ruby. Tested on Ruby 1.8.7, 1.9.1,
 Include:
 * Descriptive statistics: frequencies, median, mean, standard error, skew, kurtosis (and many others).
 * Imports and exports datasets from and to Excel, CSV and plain text files.
-* Correlations: Pearson's r, Spearman's rank correlation (rho), Tetrachoric, Polychoric.
-* Anova: generic and vector-based One-way ANOVA
+* Correlations: Pearson's r, Spearman's rank correlation (rho), point biserial, tau a, tau b, gamma,  Tetrachoric and Polychoric.
+* Anova: generic and vector-based One-way ANOVA and Two-way ANOVA
 * Tests: F, T, Levene, U-Mannwhitney.
 * Regression: Simple, Multiple (OLS), Probit  and Logit
 * Factorial Analysis: Extraction (PCA and Principal Axis), Rotation (Varimax, Equimax, Quartimax) and Parallel Analysis, for estimation of number of factors.
-* Reliability analysis for simple scale and helpers to analyze multiple scales using factor analysis and correlations
+* Reliability analysis for simple scale and a DSL to easily analyze multiple scales using factor analysis and correlations, if you want it.
 * Dominance Analysis, with multivariate dependent and bootstrap (Azen & Budescu)
 * Sample calculation related formulas
 * Creates reports on text, html and rtf, using ReportBuilder gem
@@ -42,7 +42,9 @@ Include:
     * Statsample::Factor::Equimax
     * Statsample::Factor::Quartimax
   * Statsample::Factor::ParallelAnalysis performs Horn's 'parallel analysis' to a principal components analysis to adjust for sample bias in the retention of components. 
-* Dominance Analysis. Based on Budescu and Azen papers, Statsample::DominanceAnalysis class can report dominance analysis for a sample, using uni or multivariate dependent variables and DominanceAnalysisBootstrap can execute bootstrap analysis to determine dominance stability, as recomended by  Azen & Budescu (2003) link[http://psycnet.apa.org/journals/met/8/2/129/]. 
+* Dominance Analysis. Based on Budescu and Azen papers, dominance analysis is a method to analyze the relative importance of one predictor relative to another on multiple regression
+  * Statsample::DominanceAnalysis class can report dominance analysis for a sample, using uni or multivariate dependent variables
+  * Statsample::DominanceAnalysis::Bootstrap can execute bootstrap analysis to determine dominance stability, as recomended by  Azen & Budescu (2003) link[http://psycnet.apa.org/journals/met/8/2/129/]. 
 * Module Statsample::Codification, to help to codify open questions
 * Converters to import and export data:
   * Statsample::Database : Can create sql to create tables, read and insert data
@@ -51,7 +53,7 @@ Include:
   * Statsample::Mx    : Write Mx Files
   * Statsample::GGobi : Write Ggobi files
 * Module Statsample::Crosstab provides function to create crosstab for categorical data
-* Module Statsample::Reliability provides functions to analyze scales. 
+* Module Statsample::Reliability provides functions to analyze scales with psychometric methods. 
   * Class ScaleAnalysis provides statistics like mean, standard deviation for a scale, Cronbach's alpha and standarized Cronbach's alpha, and for each item: mean, correlation with total scale, mean if deleted, Cronbach's alpha is deleted.
   * Class MultiScaleAnalysis provides a DSL to easily analyze reliability of multiple scales and retrieve correlation matrix and factor analysis of them.
 * Module Statsample::SRS (Simple Random Sampling) provides a lot of functions to estimate standard error for several type of samples
@@ -60,8 +62,8 @@ Include:
   * Statsample::Test::UMannWhitney
   * Statsample::Test::T
   * Statsample::Test::F  
-* Interfaces to gdchart, gnuplot and SVG::Graph 
-
+* Interfaces to gdchart, gnuplot and SVG::Graph (experimental)
+* Close integration with gem <tt>reportbuilder</tt>, to easily create reports on text, html and rtf formats.
 
 == Examples of use:
 
@@ -99,13 +101,13 @@ Optional:
 
 On *nix, you should install statsample-optimization to retrieve gems gsl, statistics2 and a C extension to speed some methods. 
 
-  $sudo gem install statsample-optimization
-
-To use it, on Ubuntu I recommend install build-essential and libgsl0-dev using apt-get and compile ruby 1.8 or 1.9 from source code.
+There are available precompiled version for Ruby 1.9 on x86, x86_64 and mingw32 archs.
 
-  $sudo apt-get install build-essential libgsl0-dev
+  $ sudo gem install statsample-optimization
 
+If you use Ruby 1.8, you should compile statsample-optimization, usign parameter <tt>--platform ruby</tt>
 
+  $ sudo gem install statsample-optimization --platform ruby
 
 Available setup.rb file
 

data/examples/parallel_analysis_tetrachoric.rb

@@ -2,6 +2,7 @@
 $:.unshift(File.dirname(__FILE__)+'/../lib/')
 
 require 'statsample'
+require 'statsample/bivariate/tetrachoric'
 samples=1000
 variables=10
 rng = GSL::Rng.alloc()

data/examples/reliability.rb

@@ -10,3 +10,11 @@ end
 ds.update_valid_data
 rel=Statsample::Reliability::ScaleAnalysis.new(ds)
 puts rel.summary
+
+
+ms=Statsample::Reliability::MultiScaleAnalysis.new(:name=>"Multi Scale analyss") do |m|
+  m.scale "Scale 1", ds.clone(%w{v1 v2 v3 v4 v5 v6 v7 v8 v9 v10})
+  m.scale "Scale 2", ds.clone(%w{v11 v12 v13 v14 v15 v16 v17 v18 v19})
+end
+
+puts ms.summary

data/lib/statsample.rb

@@ -112,7 +112,8 @@ module Statsample
       false
     end
   end
-  VERSION = '0.13.0'
+  
+  VERSION = '0.13.1'
   SPLIT_TOKEN = ","
   autoload(:Database, 'statsample/converters')
   autoload(:Anova, 'statsample/anova')

data/lib/statsample/factor/parallelanalysis.rb

@@ -1,120 +1,121 @@
 module Statsample
-module Factor
-  # Performs Horn's 'parallel analysis' to a principal components analysis
-  # to adjust for sample bias in the retention of components. 
-  # Can create the bootstrap samples using parameters (mean and standard
-  # deviation of each variable) or sampling for actual data.
-  # == Description
-  # "PA involves the construction of a number of correlation matrices of random variables based on the same sample size and number of variables in the real data set. The average eigenvalues from the random correlation matrices are then compared to the eigenvalues from the real data correlation matrix, such that the first observed eigenvalue is compared to the first random eigenvalue, the second observed eigenvalue is compared to the second random eigenvalue, and so on." (Hayton, Allen & Scarpello, 2004, p.194)
-  # == Usage
-  #   # ds should be any valid dataset
-  #   pa=Statsample::Factor::ParallelAnalysis.new(ds, :iterations=>100, :bootstrap_method=>:raw_data)
-  # == References:
-  # * Hayton, J., Allen, D. & Scarpello, V.(2004). Factor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis. <i>Organizational Research Methods, 7</i> (2), 191-205.
-  # * https://people.ok.ubc.ca/brioconn/nfactors/nfactors.html (for inspiration)
-class ParallelAnalysis
-  
-  include DirtyMemoize
-  include Summarizable
-  # Number of random sets to produce. 50 by default
-  attr_accessor :iterations
-  # Name of analysis
-  attr_accessor :name
-  # Dataset. You could use mock vectors when use bootstrap method
-  attr_reader :ds
-  # Bootstrap method. <tt>:raw_data</tt> used by default
-  # * <tt>:parameter</tt>: uses mean and standard deviation of each variable
-  # * <tt>:raw_data</tt> : sample with replacement from actual data. 
-  # 
-  attr_accessor :bootstrap_method
-  # Factor method.
-  # Could be Statsample::Factor::PCA or Statsample::Factor::PrincipalAxis.
-  # PCA used by default.
-  attr_accessor :factor_class
-  # Percentil over bootstrap eigenvalue should be accepted. 95 by default
-  attr_accessor :percentil
-  # Correlation matrix used with :raw_data . <tt>:correlation_matrix</tt> used by default
-  attr_accessor :matrix_method 
-  # Dataset with bootstrapped eigenvalues
-  attr_reader :ds_eigenvalues
-  # Show extra information if true
-  attr_accessor :debug
-  
-  
-  def initialize(ds, opts=Hash.new)
-    @ds=ds
-    @fields=@ds.fields
-    @n_variables=@fields.size
-    @n_cases=ds.cases
-    opts_default={
-      :name=>_("Parallel Analysis"),
-      :iterations=>50,
-      :bootstrap_method => :raw_data,
-      :factor_class => Statsample::Factor::PCA,
-      :percentil=>95, 
-      :debug=>false,
-      :matrix_method=>:correlation_matrix
-    }
-    @opts=opts_default.merge(opts)
-    @opts[:matrix_method]==:correlation_matrix if @opts[:bootstrap_method]==:parameters
-    opts_default.keys.each {|k| send("#{k}=", @opts[k]) }
-  end
-  # Number of factor to retent
-  def number_of_factors
-    total=0
-    ds_eigenvalues.fields.each_with_index do |f,i|
-      total+=1 if (@original[i]>0 and @original[i]>ds_eigenvalues[f].percentil(percentil))
-    end
-    total
-  end
-  def report_building(g) #:nodoc:
-    g.section(:name=>@name) do |s|
-      s.text _("Bootstrap Method: %s") % bootstrap_method
-      s.text _("Correlation Matrix type : %s") % matrix_method
-      s.text _("Number of variables: %d") % @n_variables
-      s.text _("Number of cases: %d") % @n_cases
-      s.text _("Number of iterations: %d") % @iterations
-      s.text _("Number or factors to preserve: %d") % number_of_factors
-      s.table(:name=>_("Eigenvalues"), :header=>[_("Eigenvalue"), _("actual"), _("mean"),"p.#{percentil}",_("preserve?")]) do |t|
+  module Factor
+    # Performs Horn's 'parallel analysis' to a principal components analysis
+    # to adjust for sample bias in the retention of components. 
+    # Can create the bootstrap samples using parameters (mean and standard
+    # deviation of each variable) or sampling for actual data.
+    # == Description
+    # "PA involves the construction of a number of correlation matrices of random variables based on the same sample size and number of variables in the real data set. The average eigenvalues from the random correlation matrices are then compared to the eigenvalues from the real data correlation matrix, such that the first observed eigenvalue is compared to the first random eigenvalue, the second observed eigenvalue is compared to the second random eigenvalue, and so on." (Hayton, Allen & Scarpello, 2004, p.194)
+    # == Usage
+    #   # ds should be any valid dataset
+    #   pa=Statsample::Factor::ParallelAnalysis.new(ds, :iterations=>100, :bootstrap_method=>:raw_data)
+    #
+    # == References:
+    # * Hayton, J., Allen, D. & Scarpello, V.(2004). Factor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis. <i>Organizational Research Methods, 7</i> (2), 191-205.
+    # * https://people.ok.ubc.ca/brioconn/nfactors/nfactors.html (for inspiration)
+    class ParallelAnalysis
+      
+      include DirtyMemoize
+      include Summarizable
+      # Number of random sets to produce. 50 by default
+      attr_accessor :iterations
+      # Name of analysis
+      attr_accessor :name
+      # Dataset. You could use mock vectors when use bootstrap method
+      attr_reader :ds
+      # Bootstrap method. <tt>:raw_data</tt> used by default
+      # * <tt>:parameter</tt>: uses mean and standard deviation of each variable
+      # * <tt>:raw_data</tt> : sample with replacement from actual data. 
+      # 
+      attr_accessor :bootstrap_method
+      # Factor method.
+      # Could be Statsample::Factor::PCA or Statsample::Factor::PrincipalAxis.
+      # PCA used by default.
+      attr_accessor :factor_class
+      # Percentil over bootstrap eigenvalue should be accepted. 95 by default
+      attr_accessor :percentil
+      # Correlation matrix used with :raw_data . <tt>:correlation_matrix</tt> used by default
+      attr_accessor :matrix_method 
+      # Dataset with bootstrapped eigenvalues
+      attr_reader :ds_eigenvalues
+      # Show extra information if true
+      attr_accessor :debug
+      
+      
+      def initialize(ds, opts=Hash.new)
+        @ds=ds
+        @fields=@ds.fields
+        @n_variables=@fields.size
+        @n_cases=ds.cases
+        opts_default={
+          :name=>_("Parallel Analysis"),
+          :iterations=>50,
+          :bootstrap_method => :raw_data,
+          :factor_class => Statsample::Factor::PCA,
+          :percentil=>95, 
+          :debug=>false,
+          :matrix_method=>:correlation_matrix
+        }
+        @opts=opts_default.merge(opts)
+        @opts[:matrix_method]==:correlation_matrix if @opts[:bootstrap_method]==:parameters
+        opts_default.keys.each {|k| send("#{k}=", @opts[k]) }
+      end
+      # Number of factor to retent
+      def number_of_factors
+        total=0
         ds_eigenvalues.fields.each_with_index do |f,i|
-          v=ds_eigenvalues[f]
-          t.row [i+1, "%0.4f" % @original[i], "%0.4f" %  v.mean, "%0.4f" %  v.percentil(percentil), (v.percentil(percentil)>0 and @original[i] > v.percentil(percentil)) ? "Yes":""]
+          total+=1 if (@original[i]>0 and @original[i]>ds_eigenvalues[f].percentil(percentil))
         end
+        total
       end
-      
-    end
-  end
-  # Perform calculation. Shouldn't be called directly for the user
-  def compute
-    @original=factor_class.new(Statsample::Bivariate.correlation_matrix(@ds), :m=>@n_variables).eigenvalues.sort.reverse
-    @ds_eigenvalues=Statsample::Dataset.new((1..@n_variables).map{|v| "ev_%05d" % v})
-    @ds_eigenvalues.fields.each {|f| @ds_eigenvalues[f].type=:scale}
-    
-    @iterations.times do |i|
-      # Create a dataset of dummy values
-      ds_bootstrap=Statsample::Dataset.new(@ds.fields)
-      if bootstrap_method==:parameter
-        rng = GSL::Rng.alloc()
+      def report_building(g) #:nodoc:
+        g.section(:name=>@name) do |s|
+          s.text _("Bootstrap Method: %s") % bootstrap_method
+          s.text _("Correlation Matrix type : %s") % matrix_method
+          s.text _("Number of variables: %d") % @n_variables
+          s.text _("Number of cases: %d") % @n_cases
+          s.text _("Number of iterations: %d") % @iterations
+          s.text _("Number or factors to preserve: %d") % number_of_factors
+          s.table(:name=>_("Eigenvalues"), :header=>[_("n"), _("data eigenvalue"), _("generated eigenvalue"),"p.#{percentil}",_("preserve?")]) do |t|
+            ds_eigenvalues.fields.each_with_index do |f,i|
+              v=ds_eigenvalues[f]
+              t.row [i+1, "%0.4f" % @original[i], "%0.4f" %  v.mean, "%0.4f" %  v.percentil(percentil), (v.percentil(percentil)>0 and @original[i] > v.percentil(percentil)) ? "Yes":""]
+            end
+          end
+          
+        end
       end
-      @fields.each do |f|
-
-        if bootstrap_method==:parameter
-          sd=@ds[f].sd
-          mean=@ds[f].mean
-          ds_bootstrap[f]=@n_cases.times.map {|c| rng.gaussian(sd)+mean}.to_scale
-        elsif bootstrap_method==:raw_data
-          ds_bootstrap[f]=ds[f].sample_with_replacement(@n_cases).to_scale
+      # Perform calculation. Shouldn't be called directly for the user
+      def compute
+        @original=factor_class.new(Statsample::Bivariate.correlation_matrix(@ds), :m=>@n_variables).eigenvalues.sort.reverse
+        @ds_eigenvalues=Statsample::Dataset.new((1..@n_variables).map{|v| "ev_%05d" % v})
+        @ds_eigenvalues.fields.each {|f| @ds_eigenvalues[f].type=:scale}
+        
+        @iterations.times do |i|
+          # Create a dataset of dummy values
+          ds_bootstrap=Statsample::Dataset.new(@ds.fields)
+          if bootstrap_method==:parameter
+            rng = GSL::Rng.alloc()
+          end
+          @fields.each do |f|
+    
+            if bootstrap_method==:parameter
+              sd=@ds[f].sd
+              mean=@ds[f].mean
+              ds_bootstrap[f]=@n_cases.times.map {|c| rng.gaussian(sd)+mean}.to_scale
+            elsif bootstrap_method==:raw_data
+              ds_bootstrap[f]=ds[f].sample_with_replacement(@n_cases).to_scale
+            end
+          end
+          fa=factor_class.new(Statsample::Bivariate.send(matrix_method, ds_bootstrap), :m=>@n_variables)
+          ev=fa.eigenvalues.sort.reverse
+          @ds_eigenvalues.add_case_array(ev)
+          puts "iteration #{i}" if $DEBUG or debug
         end
+        @ds_eigenvalues.update_valid_data
       end
-      fa=factor_class.new(Statsample::Bivariate.send(matrix_method, ds_bootstrap), :m=>@n_variables)
-      ev=fa.eigenvalues.sort.reverse
-      @ds_eigenvalues.add_case_array(ev)
-      puts "iteration #{i}" if $DEBUG or debug
+      dirty_memoize :number_of_factors, :ds_eigenvalues
+      dirty_writer :iterations, :bootstrap_method, :factor_class, :percentil
     end
-    @ds_eigenvalues.update_valid_data
   end
-  dirty_memoize :number_of_factors, :ds_eigenvalues
-  dirty_writer :iterations, :bootstrap_method, :factor_class, :percentil
-end
-end
 end

data/lib/statsample/matrix.rb

@@ -105,6 +105,11 @@ module Statsample
     def name
       @name||=get_new_name
     end
+    # Get variance for field k
+    # 
+    def variance(k)
+      submatrix([k])[0,0]
+    end
     def get_new_name
       @@covariatematrix+=1
       _("Covariate matrix %d") % @@covariatematrix

data/lib/statsample/reliability.rb

@@ -21,9 +21,18 @@ module Statsample
         }.to_dataset
         cronbach_alpha(ds)
       end
+      # Get Cronbach alpha from <tt>n</tt> cases, 
+      # <tt>s2</tt> mean variance and <tt>cov</tt>
+      # mean covariance
       def cronbach_alpha_from_n_s2_cov(n,s2,cov)
         (n.quo(n-1)) * (1-(s2.quo(s2+(n-1)*cov)))
       end
+      # Get Cronbach's alpha from a covariance matrix
+      def cronbach_alpha_from_covariance_matrix(cov)
+        n=cov.row_size
+        s2=n.times.inject(0) {|ac,i| ac+cov[i,i]}
+        (n.quo(n-1))*(1-(s2.quo(cov.total_sum)))
+      end
       # Returns n necessary to obtain specific alpha
       # given variance and covariance mean of items
       def n_for_desired_alpha(alpha,s2,cov)

data/lib/statsample/reliability/scaleanalysis.rb

@@ -12,7 +12,7 @@ module Statsample
     #  puts ia.summary
     class ScaleAnalysis
       include Summarizable
-      attr_reader :ds,:mean, :sd,:valid_n, :alpha , :alpha_standarized, :variances_mean, :covariances_mean
+      attr_reader :ds,:mean, :sd,:valid_n, :alpha , :alpha_standarized, :variances_mean, :covariances_mean, :cov_m
       attr_accessor :name
       def initialize(ds, opts=Hash.new)
         @ds=ds.dup_only_valid
@@ -29,8 +29,9 @@ module Statsample
         opts_default={:name=>"Reliability Analisis"}
         @opts=opts_default.merge(opts)
         @name=@opts[:name]
+        @cov_m=Statsample::Bivariate.covariance_matrix(@ds)
         # Mean for covariances and variances
-        @variances=@ds.fields.map {|f| @ds[f].variance}.to_scale
+        @variances=@k.times.map {|i| @cov_m[i,i]}.to_scale
         @variances_mean=@variances.mean
         @covariances_mean=(@variance-@variances.sum).quo(@k**2-@k)
         begin
@@ -125,10 +126,10 @@ module Statsample
         end
       end
       def item_statistics
-        @ds.fields.inject({}) do |a,v|
-          a[v]={:mean=>@ds[v].mean,:sds=>@ds[v].sds}
-          a
-        end
+          @is||=@ds.fields.inject({}) do |a,v|
+            a[v]={:mean=>@ds[v].mean, :sds=>Math::sqrt(@cov_m.variance(v))}
+            a
+          end
       end
       # Returns a dataset with cases ordered by score
       # and variables ordered by difficulty
@@ -152,15 +153,21 @@ module Statsample
         ds_new
       end
       def stats_if_deleted
+        @sif||=stats_if_deleted_intern
+      end
+      def stats_if_deleted_intern # :nodoc:
+        
         @ds.fields.inject({}) do |a,v|
-          ds2=@ds.clone
-          ds2.delete_vector(v)
-          total=ds2.vector_sum
+          cov_2=@cov_m.submatrix(@ds.fields-[v])
+          #ds2=@ds.clone
+          #ds2.delete_vector(v)
+          #total=ds2.vector_sum
           a[v]={}
-          a[v][:mean]=total.mean
-          a[v][:sds]=total.sds
-          a[v][:variance_sample]=total.variance_sample
-          a[v][:alpha]=Statsample::Reliability.cronbach_alpha(ds2)
+          #a[v][:mean]=total.mean
+          a[v][:mean]=@mean-item_statistics[v][:mean]
+          a[v][:variance_sample]=cov_2.total_sum
+          a[v][:sds]=Math::sqrt(a[v][:variance_sample])
+          a[v][:alpha]=Statsample::Reliability.cronbach_alpha_from_covariance_matrix(cov_2)
           a
         end
       end

data/lib/statsample/rserve_extension.rb

@@ -0,0 +1,20 @@
+# Several additions to Statsample objects, to support
+# rserve-client
+
+module Statsample
+  class Vector
+    def to_REXP
+      Rserve::REXP::Wrapper.wrap(data_with_nils)
+    end
+  end
+  class Dataset
+    def to_REXP
+      names=@fields
+      data=@fields.map {|f|
+        Rserve::REXP::Wrapper.wrap(@vectors[f].data_with_nils)
+      }
+      l=Rserve::Rlist.new(data,names)
+      Rserve::REXP.create_data_frame(l)
+    end
+  end
+end

data/lib/statsample/vector.rb

@@ -229,15 +229,14 @@ module Statsample
       set_scale_data if(@type==:scale)
       set_date_data if(@type==:date)
     end
-    
     if Statsample::STATSAMPLE__.respond_to?(:set_valid_data_intern)
-    def set_valid_data_intern #:nodoc:
-      Statsample::STATSAMPLE__.set_valid_data_intern(self)
-    end
+      def set_valid_data_intern #:nodoc:
+        Statsample::STATSAMPLE__.set_valid_data_intern(self)
+      end
     else
-    def set_valid_data_intern #:nodoc:
-      _set_valid_data_intern                  
-    end
+      def set_valid_data_intern #:nodoc:
+        _set_valid_data_intern                  
+      end
     end
     def _set_valid_data_intern #:nodoc:
       @data.each do |n|
@@ -274,22 +273,22 @@ module Statsample
     end
     # Size of total data
     def size
-    @data.size
+      @data.size
     end
     alias_method :n, :size
     
     # Retrieves i element of data
     def [](i)
-    @data[i]
+      @data[i]
     end
     # Set i element of data. 
     # Note: Use set_valid_data if you include missing values
     def []=(i,v)
-    @data[i]=v
+      @data[i]=v
     end
     # Return true if a value is valid (not nil and not included on missing values)
     def is_valid?(x)
-    !(x.nil? or @missing_values.include? x)
+      !(x.nil? or @missing_values.include? x)
     end
     # Set missing_values.
     # if update_valid = false, you should use

data/po/es/statsample.po

@@ -1,8 +1,8 @@
 msgid ""
 msgstr ""
 "Project-Id-Version: statsample 0.13.0\n"
-"POT-Creation-Date: 2010-06-21 12:23-0400\n"
-"PO-Revision-Date: 2010-06-21 12:25-0300\n"
+"POT-Creation-Date: 2010-06-21 16:08-0400\n"
+"PO-Revision-Date: 2010-06-21 16:08-0300\n"
 "Last-Translator: Claudio Bustos <clbustos@gmail.com>\n"
 "Language-Team: Desarrollador\n"
 "MIME-Version: 1.0\n"
@@ -19,10 +19,8 @@ msgstr "Prueba F"
 msgid "Mean and standard deviation"
 msgstr "Promedio y desviación estándar"
 
-#: lib/statsample/test/t.rb:209
-#: lib/statsample/factor/pca.rb:144
-#: lib/statsample/factor/pca.rb:149
-#: lib/statsample/factor/pca.rb:155
+#: lib/statsample/test/t.rb:209 lib/statsample/factor/pca.rb:144
+#: lib/statsample/factor/pca.rb:149 lib/statsample/factor/pca.rb:155
 #: lib/statsample/factor/principalaxis.rb:180
 #: lib/statsample/factor/principalaxis.rb:185
 msgid "Variable"
@@ -30,7 +28,6 @@ msgstr "Variable"
 
 #: lib/statsample/test/t.rb:209
 #: lib/statsample/dominanceanalysis/bootstrap.rb:208
-#: lib/statsample/factor/parallelanalysis.rb:78
 msgid "mean"
 msgstr "promedio"
 
@@ -38,7 +35,7 @@ msgstr "promedio"
 msgid "sd"
 msgstr "de"
 
-#: lib/statsample/test/t.rb:209
+#: lib/statsample/test/t.rb:209 lib/statsample/factor/parallelanalysis.rb:79
 msgid "n"
 msgstr "n"
 
@@ -219,17 +216,13 @@ msgstr "B"
 msgid "Within"
 msgstr "Dentro"
 
-#: lib/statsample/anova/twoway.rb:98
-#: lib/statsample/anova/oneway.rb:57
+#: lib/statsample/anova/twoway.rb:98 lib/statsample/anova/oneway.rb:57
 msgid "%s Table"
 msgstr "Tabla %s"
 
-#: lib/statsample/anova/twoway.rb:103
-#: lib/statsample/anova/oneway.rb:60
-#: lib/statsample/crosstab.rb:101
-#: lib/statsample/crosstab.rb:116
-#: lib/statsample/crosstab.rb:151
-#: lib/statsample/crosstab.rb:173
+#: lib/statsample/anova/twoway.rb:103 lib/statsample/anova/oneway.rb:60
+#: lib/statsample/crosstab.rb:101 lib/statsample/crosstab.rb:116
+#: lib/statsample/crosstab.rb:151 lib/statsample/crosstab.rb:173
 #: lib/statsample/dominanceanalysis.rb:353
 msgid "Total"
 msgstr "Total"
@@ -238,13 +231,11 @@ msgstr "Total"
 msgid "Anova Two-Way on %s"
 msgstr "Anova de dos vías en %s"
 
-#: lib/statsample/anova/twoway.rb:190
-#: lib/statsample/anova/oneway.rb:101
+#: lib/statsample/anova/twoway.rb:190 lib/statsample/anova/oneway.rb:101
 msgid "Test of Homogeneity of variances (Levene)"
 msgstr "Test de homogeneidad de varianza (Levene)"
 
-#: lib/statsample/anova/twoway.rb:195
-#: lib/statsample/anova/twoway.rb:199
+#: lib/statsample/anova/twoway.rb:195 lib/statsample/anova/twoway.rb:199
 msgid "%s Mean"
 msgstr "Promedio %s"
 
@@ -273,9 +264,8 @@ msgid "Descriptives"
 msgstr "Descriptivos"
 
 #: lib/statsample/bivariate/pearson.rb:33
-#, fuzzy
 msgid "Correlation (%s - %s)"
-msgstr "Matriz de correlaciones para %s"
+msgstr "Correlación (%s - %s)"
 
 #: lib/statsample/bivariate/pearson.rb:51
 msgid "%s : r=%0.3f (t:%0.3f, g.l.=%d, p:%0.3f / %s tails)"
@@ -285,54 +275,48 @@ msgstr "%s : r=%0.3f (t:%0.3f, g.l.=%d, p:%0.3f / %s colas)"
 msgid "Histogram %s"
 msgstr "Histograma: %s"
 
-#: lib/statsample/factor/parallelanalysis.rb:50
-#, fuzzy
+#: lib/statsample/factor/parallelanalysis.rb:51
 msgid "Parallel Analysis"
-msgstr "Análisis de Factores"
+msgstr "Análisis Paralelo"
 
-#: lib/statsample/factor/parallelanalysis.rb:72
-#, fuzzy
+#: lib/statsample/factor/parallelanalysis.rb:73
 msgid "Bootstrap Method: %s"
-msgstr "Bootstrap: %d de %d"
+msgstr "Método de Remuestreo: %s"
 
-#: lib/statsample/factor/parallelanalysis.rb:73
-#, fuzzy
+#: lib/statsample/factor/parallelanalysis.rb:74
 msgid "Correlation Matrix type : %s"
-msgstr "Matriz de correlaciones para %s"
+msgstr "Tipo de matriz de correlacion : %s"
 
-#: lib/statsample/factor/parallelanalysis.rb:74
+#: lib/statsample/factor/parallelanalysis.rb:75
 msgid "Number of variables: %d"
 msgstr "Número de variables: %d"
 
-#: lib/statsample/factor/parallelanalysis.rb:75
+#: lib/statsample/factor/parallelanalysis.rb:76
 msgid "Number of cases: %d"
 msgstr "Número de casos: %d"
 
-#: lib/statsample/factor/parallelanalysis.rb:76
-#, fuzzy
+#: lib/statsample/factor/parallelanalysis.rb:77
 msgid "Number of iterations: %d"
-msgstr "Suma de rangos %s"
+msgstr "Número de iteraciones: %d"
 
-#: lib/statsample/factor/parallelanalysis.rb:77
+#: lib/statsample/factor/parallelanalysis.rb:78
 msgid "Number or factors to preserve: %d"
 msgstr "Número de factores a preservar: %d"
 
-#: lib/statsample/factor/parallelanalysis.rb:78
-#: lib/statsample/factor/pca.rb:149
-#: lib/statsample/factor/principalaxis.rb:185
+#: lib/statsample/factor/parallelanalysis.rb:79
+#: lib/statsample/factor/pca.rb:149 lib/statsample/factor/principalaxis.rb:185
 msgid "Eigenvalues"
 msgstr "Eigenvalues"
 
-#: lib/statsample/factor/parallelanalysis.rb:78
-#, fuzzy
-msgid "Eigenvalue"
-msgstr "Eigenvalues"
+#: lib/statsample/factor/parallelanalysis.rb:79
+msgid "data eigenvalue"
+msgstr "eigenvalue de los datos"
 
-#: lib/statsample/factor/parallelanalysis.rb:78
-msgid "actual"
-msgstr "real"
+#: lib/statsample/factor/parallelanalysis.rb:79
+msgid "generated eigenvalue"
+msgstr "eigenvalue generado"
 
-#: lib/statsample/factor/parallelanalysis.rb:78
+#: lib/statsample/factor/parallelanalysis.rb:79
 msgid "preserve?"
 msgstr "¿preservar?"
 
@@ -340,36 +324,27 @@ msgstr "¿preservar?"
 msgid "Principal Component Analysis"
 msgstr "Análisis de componentes principales"
 
-#: lib/statsample/factor/pca.rb:143
-#: lib/statsample/factor/principalaxis.rb:178
+#: lib/statsample/factor/pca.rb:143 lib/statsample/factor/principalaxis.rb:178
 msgid "Number of factors: %d"
 msgstr "Número de factores: %d"
 
-#: lib/statsample/factor/pca.rb:144
-#: lib/statsample/factor/principalaxis.rb:180
+#: lib/statsample/factor/pca.rb:144 lib/statsample/factor/principalaxis.rb:180
 msgid "Communalities"
 msgstr "Comunalidades"
 
-#: lib/statsample/factor/pca.rb:144
-#: lib/statsample/factor/principalaxis.rb:180
-#, fuzzy
+#: lib/statsample/factor/pca.rb:144 lib/statsample/factor/principalaxis.rb:180
 msgid "Initial"
-msgstr "Condicional"
+msgstr "Inicial"
 
-#: lib/statsample/factor/pca.rb:144
-#: lib/statsample/factor/principalaxis.rb:180
-#, fuzzy
+#: lib/statsample/factor/pca.rb:144 lib/statsample/factor/principalaxis.rb:180
 msgid "Extraction"
-msgstr "Ecuación"
+msgstr "Extracción"
 
-#: lib/statsample/factor/pca.rb:149
-#: lib/statsample/factor/principalaxis.rb:185
-#, fuzzy
+#: lib/statsample/factor/pca.rb:149 lib/statsample/factor/principalaxis.rb:185
 msgid "Value"
 msgstr "Valor de U"
 
-#: lib/statsample/factor/pca.rb:155
-#: lib/statsample/factor/principalaxis.rb:190
+#: lib/statsample/factor/pca.rb:155 lib/statsample/factor/principalaxis.rb:190
 msgid "Component Matrix"
 msgstr "Matriz de componentes"
 
@@ -550,24 +525,20 @@ msgid "Y%d"
 msgstr "Y%d"
 
 #: lib/statsample/matrix.rb:110
-#, fuzzy
 msgid "Covariate matrix %d"
-msgstr "Matriz de correlaciones para %s"
+msgstr "Matriz de Covarianza %d"
 
 #: lib/statsample/matrix.rb:152
-#, fuzzy
 msgid "Correlation"
-msgstr "Matriz de correlaciones para %s"
+msgstr "Correlación"
 
 #: lib/statsample/matrix.rb:152
-#, fuzzy
 msgid "Covariance"
-msgstr "Promedio de las covarianzas"
+msgstr "Covarianza"
 
 #: lib/statsample/matrix.rb:152
-#, fuzzy
 msgid " Matrix"
-msgstr "Matriz de componentes"
+msgstr "Matriz"
 
 #: lib/statsample/vector.rb:614
 msgid "n :%d"
@@ -609,14 +580,21 @@ msgstr "Dataset %d"
 msgid "Cases: %d"
 msgstr "Casos: %s"
 
+#~ msgid "actual"
+#~ msgstr "real"
+
 #~ msgid "Polychoric correlation"
 #~ msgstr "Correlación policórica"
+
 #~ msgid "Minimizing using GSL Brent method\n"
 #~ msgstr "Minimizando usando método GSL Brent\n"
+
 #~ msgid "Two step minimization using %s method\n"
 #~ msgstr "Minimización en dos etapas usando método %s\n"
+
 #~ msgid "Contingence Table"
 #~ msgstr "Tabla de Contingencia"
+
 #~ msgid "Thresholds"
 #~ msgstr "Umbrales"
 
@@ -627,34 +605,48 @@ msgstr "Casos: %s"
 #, fuzzy
 #~ msgid "Threshold Y %d"
 #~ msgstr "Umbral Y:%0.3f"
+
 #~ msgid "Test of bivariate normality: X2 = %0.3f, df = %d, p= %0.5f"
 #~ msgstr "Prueba de normalidad bivariada: X2 = %0.3f, g.l. = %d, p= %0.5f"
+
 #~ msgid "SE: %0.3f"
 #~ msgstr "EE: %0.3f"
+
 #~ msgid "Threshold X: %0.3f "
 #~ msgstr "Umbral X: %0.3f"
+
 #~ msgid "Threshold Y: %0.3f "
 #~ msgstr "Umbral Y:%0.3f"
+
 #~ msgid "Tetrachoric correlation"
 #~ msgstr "Correlación tetracórica"
+
 #~ msgid "Factor Analysis: "
 #~ msgstr "Análisis de Factores:"
+
 #~ msgid "DAB: "
 #~ msgstr "RAD:"
+
 #~ msgid "Total mean"
 #~ msgstr "Promedio total"
+
 #~ msgid "Total sd"
 #~ msgstr "d.e. total"
+
 #~ msgid "Total variance"
 #~ msgstr "Varianza Total"
+
 #~ msgid "Median"
 #~ msgstr "Mediana"
+
 #~ msgid "\"Anova Two-Way on #{@ds[dep_var].name}\""
 #~ msgstr "\"Anova de dos vías en #{@ds[dep_var].name}\""
+
 #~ msgid "Crosstab"
 #~ msgstr "Tabulación cruzada"
+
 #~ msgid "Crosstab: "
 #~ msgstr "Tabulación cruzada:"
+
 #~ msgid ")}"
 #~ msgstr ")}"
-

data/po/statsample.pot

@@ -7,7 +7,7 @@
 msgid ""
 msgstr ""
 "Project-Id-Version: statsample 0.13.0\n"
-"POT-Creation-Date: 2010-06-21 12:23-0400\n"
+"POT-Creation-Date: 2010-06-21 16:08-0400\n"
 "PO-Revision-Date: 2009-08-04 15:36-0400\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language-Team: LANGUAGE <LL@li.org>\n"
@@ -33,7 +33,6 @@ msgstr ""
 
 #: lib/statsample/test/t.rb:209
 #: lib/statsample/dominanceanalysis/bootstrap.rb:208
-#: lib/statsample/factor/parallelanalysis.rb:78
 msgid "mean"
 msgstr ""
 
@@ -41,7 +40,7 @@ msgstr ""
 msgid "sd"
 msgstr ""
 
-#: lib/statsample/test/t.rb:209
+#: lib/statsample/test/t.rb:209 lib/statsample/factor/parallelanalysis.rb:79
 msgid "n"
 msgstr ""
 
@@ -281,48 +280,48 @@ msgstr ""
 msgid "Histogram %s"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:50
+#: lib/statsample/factor/parallelanalysis.rb:51
 msgid "Parallel Analysis"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:72
+#: lib/statsample/factor/parallelanalysis.rb:73
 msgid "Bootstrap Method: %s"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:73
+#: lib/statsample/factor/parallelanalysis.rb:74
 msgid "Correlation Matrix type : %s"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:74
+#: lib/statsample/factor/parallelanalysis.rb:75
 msgid "Number of variables: %d"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:75
+#: lib/statsample/factor/parallelanalysis.rb:76
 msgid "Number of cases: %d"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:76
+#: lib/statsample/factor/parallelanalysis.rb:77
 msgid "Number of iterations: %d"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:77
+#: lib/statsample/factor/parallelanalysis.rb:78
 msgid "Number or factors to preserve: %d"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:78
+#: lib/statsample/factor/parallelanalysis.rb:79
 #: lib/statsample/factor/pca.rb:149 lib/statsample/factor/principalaxis.rb:185
 msgid "Eigenvalues"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:78
-msgid "Eigenvalue"
+#: lib/statsample/factor/parallelanalysis.rb:79
+msgid "data eigenvalue"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:78
-msgid "actual"
+#: lib/statsample/factor/parallelanalysis.rb:79
+msgid "generated eigenvalue"
 msgstr ""
 
-#: lib/statsample/factor/parallelanalysis.rb:78
+#: lib/statsample/factor/parallelanalysis.rb:79
 msgid "preserve?"
 msgstr ""
 

data/test/test_reliability.rb

@@ -33,6 +33,10 @@ class StatsampleReliabilityTestCase < MiniTest::Unit::TestCase
         vm, cm = sa.variances_mean, sa.covariances_mean
         assert_in_delta(sa.alpha, Statsample::Reliability.cronbach_alpha_from_n_s2_cov(@n_variables, vm,cm), 1e-10)        
       end
+      should "method cronbach_alpha_from_covariance_matrix returns correct value" do
+        cov=Statsample::Bivariate.covariance_matrix(@ds)
+        assert_in_delta(@a, Statsample::Reliability.cronbach_alpha_from_covariance_matrix(cov),0.0000001)
+      end
       should "return correct n for desired alpha, covariance and variance" do
         sa=Statsample::Reliability::ScaleAnalysis.new(@ds)
         vm, cm = sa.variances_mean, sa.covariances_mean
@@ -102,7 +106,7 @@ class StatsampleReliabilityTestCase < MiniTest::Unit::TestCase
       
       setup do
         size=100
-        @scales=4
+        @scales=3
         @items_per_scale=10
         h={}
         @scales.times {|s|
@@ -173,13 +177,25 @@ class StatsampleReliabilityTestCase < MiniTest::Unit::TestCase
         @x4=[1,2,3,4,4,4,4,3,4,4,5,30].to_scale
         @ds={'x1'=>@x1,'x2'=>@x2,'x3'=>@x3,'x4'=>@x4}.to_dataset
         @ia=Statsample::Reliability::ScaleAnalysis.new(@ds)
-        @cov_matrix=Statsample::Bivariate.covariance_matrix(@ds)
+        @cov_matrix=@ia.cov_m
       end     
       should "return correct values for item analysis" do 
         assert_in_delta(0.980,@ia.alpha,0.001)
         assert_in_delta(0.999,@ia.alpha_standarized,0.001)
         var_mean=4.times.map{|m| @cov_matrix[m,m]}.to_scale.mean 
         assert_in_delta(var_mean, @ia.variances_mean)
+        assert_equal(@x1.mean, @ia.item_statistics['x1'][:mean])
+        assert_equal(@x4.mean, @ia.item_statistics['x4'][:mean])
+        assert_equal(@x1.sds, @ia.item_statistics['x1'][:sds])
+        assert_equal(@x4.sds, @ia.item_statistics['x4'][:sds])
+        ds2=@ds.clone
+        ds2.delete_vector('x1')
+        vector_sum=ds2.vector_sum
+        assert_equal(vector_sum.mean, @ia.stats_if_deleted['x1'][:mean])
+        assert_equal(vector_sum.sds, @ia.stats_if_deleted['x1'][:sds])
+        assert_in_delta(vector_sum.variance, @ia.stats_if_deleted['x1'][:variance_sample],1e-10)
+
+        assert_equal(Statsample::Reliability.cronbach_alpha(ds2), @ia.stats_if_deleted['x1'][:alpha])
         
         covariances=[]
         4.times.each {|i|

data/test/test_rserve_extension.rb

@@ -0,0 +1,40 @@
+require(File.dirname(__FILE__)+'/helpers_tests.rb')
+
+begin
+  require 'rserve'
+  require 'statsample/rserve_extension'
+
+class StatsampleRserveExtensionTestCase < MiniTest::Unit::TestCase
+  context "Statsample Rserve extensions" do
+    setup do
+      @r=Rserve::Connection.new
+    end
+    should "return a valid rexp for numeric vector" do
+      a=100.times.map {|i| rand()>0.9 ? nil : i+rand() }.to_scale
+      rexp=a.to_REXP
+      assert_instance_of(rexp, Rserve::REXP::Double)
+      assert_equal(rexp.to_ruby,a.data_with_nils)
+      @r.assign 'a',rexp
+      assert_equal(a.data_with_nils, @r.eval('a').to_ruby)
+    end
+    should "return a valid rserve dataframe for statsample datasets" do
+      a=100.times.map {|i| rand()>0.9 ? nil : i+rand() }.to_scale
+      b=100.times.map {|i| rand()>0.9 ? nil : i+rand() }.to_scale
+      c=100.times.map {|i| rand()>0.9 ? nil : i+rand() }.to_scale
+      ds={'a'=>a,'b'=>b,'c'=>c}.to_dataset
+      rexp=ds.to_REXP
+      assert_instance_of(rexp, Rserve::REXP::GenericVector)
+      ret=rexp.to_ruby
+      assert_equal(a.data_with_nils, ret['a']) 
+      @r.assign 'df', rexp
+      out_df=@r.eval('df').to_ruby
+      assert_equal('data.frame', out_df.attributes['class'])
+      assert_equal(['a','b','c'], out_df.attributes['names'])
+      assert_equal(a.data_with_nils, out_df['a'])
+    end
+  end
+end
+
+rescue LoadError
+  puts "Require rserve extension"
+end

data/test/test_svg_graph.rb

@@ -46,7 +46,7 @@ class StatsampleSvgGraphTestCase < MiniTest::Unit::TestCase
       File.open(file,"wb") {|fp|
         fp.write(hist.burn)
       }
-      assert(File.exists?(file))
+      #assert(File.exists?(file))
     else
       skip "Statsample::Vector#svggraph_histogram.new not tested (no ruby-gsl)"
     end

metadata

@@ -5,8 +5,8 @@ version: !ruby/object:Gem::Version
   segments: 
   - 0
   - 13
-  - 0
-  version: 0.13.0
+  - 1
+  version: 0.13.1
 platform: ruby
 authors: 
 - Claudio Bustos
@@ -35,7 +35,7 @@ cert_chain:
   rpP0jjs0
   -----END CERTIFICATE-----
 
-date: 2010-06-21 00:00:00 -04:00
+date: 2010-07-03 00:00:00 -04:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -191,12 +191,12 @@ description: |-
   Include:
   * Descriptive statistics: frequencies, median, mean, standard error, skew, kurtosis (and many others).
   * Imports and exports datasets from and to Excel, CSV and plain text files.
-  * Correlations: Pearson's r, Spearman's rank correlation (rho), Tetrachoric, Polychoric.
-  * Anova: generic and vector-based One-way ANOVA
+  * Correlations: Pearson's r, Spearman's rank correlation (rho), point biserial, tau a, tau b, gamma,  Tetrachoric and Polychoric.
+  * Anova: generic and vector-based One-way ANOVA and Two-way ANOVA
   * Tests: F, T, Levene, U-Mannwhitney.
   * Regression: Simple, Multiple (OLS), Probit  and Logit
   * Factorial Analysis: Extraction (PCA and Principal Axis), Rotation (Varimax, Equimax, Quartimax) and Parallel Analysis, for estimation of number of factors.
-  * Reliability analysis for simple scale and helpers to analyze multiple scales using factor analysis and correlations
+  * Reliability analysis for simple scale and a DSL to easily analyze multiple scales using factor analysis and correlations, if you want it.
   * Dominance Analysis, with multivariate dependent and bootstrap (Azen & Budescu)
   * Sample calculation related formulas
   * Creates reports on text, html and rtf, using ReportBuilder gem
@@ -298,6 +298,7 @@ files:
 - lib/statsample/reliability/multiscaleanalysis.rb
 - lib/statsample/reliability/scaleanalysis.rb
 - lib/statsample/resample.rb
+- lib/statsample/rserve_extension.rb
 - lib/statsample/srs.rb
 - lib/statsample/test.rb
 - lib/statsample/test/f.rb
@@ -335,6 +336,7 @@ files:
 - test/test_regression.rb
 - test/test_reliability.rb
 - test/test_resample.rb
+- test/test_rserve_extension.rb
 - test/test_srs.rb
 - test/test_statistics.rb
 - test/test_stest.rb
@@ -407,6 +409,7 @@ test_files:
 - test/test_crosstab.rb
 - test/test_distribution.rb
 - test/test_svg_graph.rb
+- test/test_rserve_extension.rb
 - test/test_csv.rb
 - test/test_matrix.rb
 - test/test_gsl.rb