rroc 0.1

data/README.markdown

@@ -0,0 +1,57 @@
+RROC: Dead-simple ROC analysis in Ruby
+==========
+
+Ported from the [ML-Mathematica](http://www.bioinf.jku.at/software/ML-Math/) set of machine learning demos.
+
+This class provides methods for Reciever Operating Characteristic (ROC) calculation; algorithm copied from the ML-Mathematica[http://www.bioinf.jku.at/software/ML-Math/] mathematica library by Steven Bedrick ([steve@bedrick.org](mailto:steve@bedrick.org)). For an excellent overview of ROC analysis, check out:
+
+Fawcett, T. "An introduction to ROC analysis" Pattern Recognition Letters 27 (2006) 861-874 [(pdf)](https://cours.etsmtl.ca/sys828/REFS/A1/Fawcett_PRL2006.pdf)
+
+Installation
+--------
+
+    gem install rroc
+
+Usage
+--------
+
+Using RROC is very simple. It expects to be given data in the form of a _n_x2 matrix (i.e., an Array of 2-element Arrays) representing the output of a binary classifier. Each row represents a "case" (document, data point, etc.); the first column represents the classifier's discriminant value and the second column represents the ground-truth class label for that case. RROC expects class labels in the form of either 1 or -1; larger discriminant values should be associated with membership in class 1. You can determine the area under the ROC curve as follows:
+
+    require 'rroc'
+    
+    my_data = open('some_data.csv').readlines.collect { |l| l.strip.split(",").map(&:to_f) }
+    auc = ROC.auc(my_data)
+    puts auc
+    
+If you want to obtain a set of points describing the ROC curve itself, you can do that by calling the `ROC.curve_points` method:
+
+    require 'rroc'
+
+    my_data = open('some_data.csv').readlines.collect { |l| l.strip.split(",").map(&:to_f) }
+    pts = ROC.curve_points(my_data) # returns something like: [[0.0, 0.0], [0.1, 0.01]... ]
+    
+The points are returned as an Array of two-element Arrays, each of which contains an X and a Y coordinate that you may then export or plot using the utility of your choice. For example, together with the `googlecharts` gem, you can obtain a Google Chart link that will display your ROC curve:
+
+
+    require 'gchart' 
+    require 'rroc'
+
+    my_data = open('some_data.csv').readlines.collect { |l| l.strip.split(",").map(&:to_f) }
+    pts = ROC.curve_points(my_data)
+    puts Gchart.scatter(:data => [pts.collect { |x| x[0] }, pts.collect { |x| x[1] }])
+
+Conclusion
+---------
+RROC is designed to be a tool for bare-bones ROC analysis in Ruby; take it for what it is and use it at your own risk. Bug reports, forks, and patches are more than welcome!
+
+License
+----------
+Copyright (c) 2011, Steven Bedrick
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+

data/lib/rroc.rb

@@ -0,0 +1,67 @@
+# This class provides methods for Reciever Operating Characteristic (ROC) curve calculation; algorithm copied from the ML-Mathematica[http://www.bioinf.jku.at/software/ML-Math/] mathematica library by Steven Bedrick (steve@bedrick.org). For an excellent overview of ROC analysis, check out:
+#
+# Fawcett, T. "An introduction to ROC analysis" Pattern Recognition Letters 27 (2006) 861-874 (pdf[https://cours.etsmtl.ca/sys828/REFS/A1/Fawcett_PRL2006.pdf])
+#
+# first col of mat is discrim. val; second col is label (+1 -> pos, -1 -> neg, higher disc. val -> more pos)
+#
+# e.g.: +[[.3, -1], [.7, 1], [.1, -1] . . . ]+
+#
+# The scale of first column is not important. The labels in the second column are.
+#
+# pts plot fpr (x-axis) against tpr (y-axis)
+
+class ROC
+ 
+  # Calculates the "area under the ROC curve" for the output of a binary classifier.
+  #
+  # @param [Array] dat Classifier output for which the AUC should be calculated, in the form of a n x 2 matrix. Each row represents a "case" (document, example, etc.); the first column is the discriminant value, and the second column _must_ be either -1 (if the ground truth class of the case is "negative") or 1 (if the ground truth is "positive").
+  # @return [Fixnum] The "area under the ROC" curve.
+  def self.auc(dat)
+    return self.calc(dat, false)
+  end
+  
+  # Returns a set of x/y coordinates describing an ROC curve for +dat+ plotting the FPR on the abscissa and the TPR on the ordinate. 
+  # @param dat (see ROC.auc) 
+  # @return [Array] x/y coordinates that, when plotted, illustrate an ROC curve for +dat+. Each element in the array is an array containing an x and y coordinate.
+  def self.curve_points(dat)
+    return self.calc(dat, true)[:points]
+  end
+  
+  private 
+  def self.calc(mat, inc_pts = false)
+  	# sort by first col, ascending, and take labels
+  	sorted_by_disc = mat.sort { |a,b| a[0] <=> b[0] }
+  	sorted_labels = sorted_by_disc.collect { |d| d[1] }
+	
+  	# now let's count the number of positive and negatives:
+  	pos = sorted_labels.count { |l| l == 1 }.to_f 
+  	neg = sorted_labels.count { |l| l == -1 }.to_f
+
+  	auc = 0.0
+	
+  	plotlist = [[0,0]]
+	
+  	if pos > 0 # are there *any* true positives? If not we can't really calculate much...
+  		c = 0.0 # how many positives we've seen thus far
+  		n = 0.0 # how many negatives we've seen thus far
+  		(sorted_labels.length - 1).downto(0) do |i| # walk backwards through the data...
+  		  if sorted_labels[i] > 0 # pos?
+  			  c += 1.0
+  		  else
+  			  n += 1.0
+  			  auc += (c / (pos * neg)) # update auc
+  		  end
+  		  plotlist << [n / neg, c / pos]
+  		end
+  #		plotlist << [1,0] # the original MMA has this, but I don't think we really need it.
+  	end
+	
+  	if inc_pts # does the caller want x/y points describing the curve?
+  	  return {:auc => auc, :points => plotlist}
+    else # if not, just return the auc
+  	  return auc
+    end
+  end
+end
+
+

metadata

@@ -0,0 +1,57 @@
+--- !ruby/object:Gem::Specification 
+name: rroc
+version: !ruby/object:Gem::Version 
+  prerelease: 
+  version: "0.1"
+platform: ruby
+authors: 
+- Steven Bedrick
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2011-06-30 00:00:00 -07:00
+default_executable: 
+dependencies: []
+
+description: 
+email: steve@bedrick.org
+executables: []
+
+extensions: []
+
+extra_rdoc_files: []
+
+files: 
+- README.markdown
+- lib/rroc.rb
+has_rdoc: true
+homepage: https://github.com/stevenbedrick/RROC
+licenses: []
+
+post_install_message: 
+rdoc_options: []
+
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.6.2
+signing_key: 
+specification_version: 3
+summary: Dead-simple ROC analysis in Ruby.
+test_files: []
+