cass 0.0.1

data/Manifest

@@ -0,0 +1,14 @@
+CHANGELOG
+LICENSE
+Manifest
+README.rdoc
+Rakefile
+cass.gemspec
+lib/cass.rb
+lib/cass/analysis.rb
+lib/cass/context.rb
+lib/cass/contrast.rb
+lib/cass/document.rb
+lib/cass/extensions.rb
+lib/cass/parser.rb
+lib/cass/stats.rb

data/README.rdoc

@@ -0,0 +1,125 @@
+= Getting started with the CASS tools
+
+CASS (Contrast Analysis of Semantic Similarity) is a set of tools for conducting contrast-based analyses of semantic similarity in text. CASS is based on the BEAGLE model described by Jones and Mewhort (2007). For a more detailed explanation, see Holtzman et al (under review).
+
+== License
+
+Copyright 2010 Tal Yarkoni and Nick Holtzman. Licensed under the GPL license. See the included LICENSE file for details.
+
+== Installation
+
+The CASS tools are packaged as a library for the Ruby programming language. You must have Ruby interpreter installed on your system, as well as the NMatrix library. To install, follow these steps:
+
+(1) <b>Install Ruby</b>--preferably 1.9 or greater. Installers for most platforms are available here[http://www.ruby-lang.org/en/downloads/].
+
+(2) <b>Install the NArray[http://narray.rubyforge.org] library</b>. On most platforms, you should be able to just type:
+
+ 	gem install narray
+
+On Windows, it's a bit more involved; follow the instructions here[http://narray.rubyforge.org].
+
+(3) <b>Install the CASS gem</b> from the command prompt, like so:
+
+	gem install cass
+
+(4) <b>Download the sample analysis files</b> (cass_sample.zip[http://casstools.org/downloads/cass_sample.zip]), which will help you get started working with CASS. Unpack the file anywhere you like, and you should be ready to roll.
+
+
+== Usage
+
+There are two general ways to use CASS:
+
+=== The easy way
+
+For users without prior programming experience, CASS is streamlined to make things as user-friendly as possible. Assuming you've installed CASS per the instructions above, you can run a full CASS analysis just by tweaking a few settings and running the analysis script (run_cass.rb) included in the sample analysis package (cass_sample.zip[http://casstools.org/downloads/cass_sample.zip]). Detailed instructions are available in [this] tutorial; in brief, here's what you need to do to get up and running:
+
+1. Download cass_sample.zip[http://casstools.org/downloads/cass_sample.zip] and unpack it somewhere. The package contains several files that tell CASS what to do, as well as some sample text you can process. These include:
+- contrasts.txt: a specification of the contrasts you'd like to run on the text, one per line. For a detailed explanation of the format, see the [tutorial].
+- default.spec: the main specification file containing all the key settings CASS needs in order to run. All settings are commented, but a more detailed explanation is provided in the [tutorial]. You can create as many .spec files as you like (no need to edit this one repeatedly!), just make sure to edit run_cass.rb to indicate which .spec file to use.
+- stopwords.txt: A sample list of stopwords to exclude from analysis (CASS will use this file by default). These are mostly high-frequency function words that carry little meaning but can strongly bias a text.
+- sample1.txt and sample2.txt: two sample documents to get you started.
+- run_cass.rb: the script you'll use to run CASS.
+
+2. Edit the settings as you please. An explanation of what everything means is in the [tutorial]; if you're just getting started, you can just leave everything as is and run the sample analysis.
+
+3. Run run_cass.rb. If you're on Windows, you may be able to double-click on the script to run it; however, if you do that, you won't see any of the output. On most platforms (and optionally on Windows), you'll have to run the script from the command prompt. You can do this by opening up a terminal window (or, in Windows, a command prompt), navigating to the directory that contains the sample analysis files, and typing:
+
+	ruby run_cass.rb
+	
+After doing that, you should get a bunch of output showing you exactly what's going on. There should also be some new files in the working directory containing the results of the analysis.
+
+Assuming the analysis ran successfully, you can now set about running your own analyses. We recommend reading the entire [tutorial] before diving in.
+
+=== As a library
+
+Advanced users familiar with Ruby or other programming languages will probably want to use CASS as a library. Assuming you've installed CASS as a gem (see above), running a basic analysis with CASS is straightforward. First, we require the gem:
+
+	require 'cass'
+	
+We don't want the inconvenience of having to call all the methods through the Cass module (e.g., Cass::Contrast.new, Cass::Document.new, etc.), so let's go ahead and include the contents of the module in the namespace:
+
+	include Cass
+
+Now we can start running analyses. Let's say we have a text file containing transcribed conversations of people discussing foods they like and dislike (e.g., cake.txt in the sample analysis package[http://casstools.org/downloads/cass_sample.zip]). Suppose we're particularly interested in two foods: cake and spinach. Our goal is to test the hypothesis that people prefer cake to spinach. Operationally, we're going to do that by examining the relative distance from 'spinach' and 'cake' to the terms 'good' and 'bad' in semantic space.
+
+The first thing to do is set up the right contrasts. In this case, we'll create a single contrast comparing the distance between cake and spinach with respect to good and bad:
+
+	contrast = Contrast.new("cake spinach good bad")
+
+CASS interprets a string of four words as two ordered pairs: 'cake' and 'spinach' form one pair, 'good' and 'bad' the other (we could, equivalently, initialize the contrast by passing the 4-element array ['cake', 'spinach', 'good', 'bad']).
+
+Next, we read the file containing the transcripts:
+
+	text = File.new("cake.txt").read
+	
+And then we can create a corresponding Document. We initialize the Document object by passing a descriptive name, the contrasts we want to run, and the full text we want to analyze:
+
+	doc = Document.new("cake_vs_spinach", contrast, text)
+
+If we want to see some information about the contents of our document, we can type:
+
+	doc.summary
+	
+And that prints something like this to our screen:
+
+	> Summary for document 'cake.txt':
+	> 4 target words (cake, spinach, good, bad)
+	> 35 words in context.
+	> Using 21 lines (containing at least one target word) for analysis.
+
+Nothing too fancy, just basic descriptive information. The summary method has some additional arguments we could use to get more detailed information (e.g., word_count, list_context, etc.), but we'll skip those for now.
+
+Now if we want to compute the interaction term for our contrast (i.e., the difference of differences, reflecting the equation (cake.good - spinach.bad) - (cake.bad - spinach.good)), all we have to do is:
+
+	contrast.apply(doc)
+
+And we get back something that looks like this:
+
+	0.5117	0.4039	0.3256	0.4511	0.2333
+
+Where the first four values represent the similarity between the 4 pairs of words used to generate the interaction term (e.g., the first value reflects the correlation between 'cake' and 'good', the second between 'spinach' and 'bad', and so on), and the fifth is the interaction term. So in this case, the result (0.23) tells us that there's a positive bias in the text, such that cake is semantically more closely related to good (relative to bad) than spinach is. Hypothesis confirmed!
+
+Well, sort of. By itself, the number 0.23 doesn't mean very much. We don't know what the standard error is, so we have no idea whether 0.23 is a very large number or a very small one that might occur pretty often just by chance. Fortunately, we can generate some bootstrapped p values quite easily. First, we generate the bootstraps:
+
+	Analysis.bootstrap_test(doc, contrasts, "speech_results.txt", 1000)
+
+Here we call the bootstrap_test method, feeding it the document we want to analyze, the Contrasts we want to apply, the filename root we want to use, and the number of iterations we want to run (generally, as many as is computationally viable). The results will be saved to a plaintext file with the specified name, and we can peruse that file at our leisure. If we open it up, the first few lines look like this:
+
+	cake.spinach.good.bad	observed	cake.txt	0.5117	0.4039	0.3256	0.4511	0.2333
+	cake.spinach.good.bad	boot_1	cake.txt	0.4118	0.2569	0.1481	0.4086	0.4154
+	cake.spinach.good.bad	boot_2	cake.txt	0.5321	0.4353	0.4349	0.5396	0.2015
+	cake.spinach.good.bad	boot_3	cake.txt	0.51	0.4216	0.3043	0.6923	0.4764
+	cake.spinach.good.bad	boot_4	cake.txt	0.6222	0.3452	0.238	0.249	0.288
+	...
+
+The columns tell us, respectively, what file the results came from, the bootstrap iteration (the first line shows us the actual, or observed value), and the observed interaction terms. Given this information, we can now compare the bootstrapped distribution to zero to test our hypothesis. We do that like this:
+
+	Analysis.p_values("speech_results.txt", 'boot')
+
+...where the first argument specifies the full path to the file containing the bootstrap results we want to summarize, and the second argument indicates the type of test that was conducted (either 'boot' or 'perm'). The results will be written to a file named speech_results_boot_p_values.txt. If we open that document up, we see this:
+
+file	contrast	N	value	p-value
+cake.txt	cake.spinach.good.bad	1000	0.2333	0.0
+cake.txt	mean	1000	0.2333	0.0
+
+As you can see, the last column (p-value) reads 0.0, which is to say, none of the 1,000 iterations we ran had a value greater than 0. So we can reject the null hypothesis of zero effect at p < .001 in this case. Put differently, it's exceedingly unlikely that we would get this result (people having a positive bias towards cake relative to spinach) just by chance. Of course, that's a contrived example that won't surprise anyone. But the point is that you can use the CASS tools in a similar way to ask other much more interesting questions about the relation between different terms in semantic space. So that's the end of this overview; to learn more about the other functionality in CASS, read the [tutorial] document, or just surf around this RDoc.

data/Rakefile

@@ -0,0 +1,12 @@
+require 'rubygems'
+require 'rake'
+require 'echoe'
+
+Echoe.new("cass", "0.0.1") { |p|
+  p.author = "Tal Yarkoni"
+  p.email = "tyarkoni@gmail.com"
+  p.summary = "A set of tools for conducting Contrast Analyses of Semantic Similarity (CASS)."
+  p.url = "http://casstools.org"
+  p.docs_host = "http://casstools.org/doc/"
+  p.runtime_dependencies = ['narray >=0.5.9.7']
+}

data/cass.gemspec

@@ -0,0 +1,33 @@
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+  s.name = %q{cass}
+  s.version = "0.0.1"
+
+  s.required_rubygems_version = Gem::Requirement.new(">= 1.2") if s.respond_to? :required_rubygems_version=
+  s.authors = ["Tal Yarkoni"]
+  s.date = %q{2010-06-15}
+  s.description = %q{A set of tools for conducting Contrast Analyses of Semantic Similarity (CASS).}
+  s.email = %q{tyarkoni@gmail.com}
+  s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README.rdoc", "lib/cass.rb", "lib/cass/analysis.rb", "lib/cass/context.rb", "lib/cass/contrast.rb", "lib/cass/document.rb", "lib/cass/extensions.rb", "lib/cass/parser.rb", "lib/cass/stats.rb"]
+  s.files = ["CHANGELOG", "LICENSE", "Manifest", "README.rdoc", "Rakefile", "cass.gemspec", "lib/cass.rb", "lib/cass/analysis.rb", "lib/cass/context.rb", "lib/cass/contrast.rb", "lib/cass/document.rb", "lib/cass/extensions.rb", "lib/cass/parser.rb", "lib/cass/stats.rb"]
+  s.homepage = %q{http://casstools.org}
+  s.rdoc_options = ["--line-numbers", "--inline-source", "--title", "Cass", "--main", "README.rdoc"]
+  s.require_paths = ["lib"]
+  s.rubyforge_project = %q{cass}
+  s.rubygems_version = %q{1.3.7}
+  s.summary = %q{A set of tools for conducting Contrast Analyses of Semantic Similarity (CASS).}
+
+  if s.respond_to? :specification_version then
+    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+    s.specification_version = 3
+
+    if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
+      s.add_runtime_dependency(%q<narray>, [">= 0.5.9.7"])
+    else
+      s.add_dependency(%q<narray>, [">= 0.5.9.7"])
+    end
+  else
+    s.add_dependency(%q<narray>, [">= 0.5.9.7"])
+  end
+end

data/lib/cass.rb

@@ -0,0 +1,14 @@
+require 'narray'
+require 'cass/stats'
+require 'cass/analysis'
+require 'cass/context'
+require 'cass/contrast'
+require 'cass/document'
+require 'cass/extensions'
+require 'cass/parser'
+
+module Cass
+  
+  VERSION = '0.1.0'
+  
+end

data/lib/cass/analysis.rb

@@ -0,0 +1,229 @@
+module Cass
+  
+  # Instantiates an analysis on one or more Documents.
+  # Currently, only the default processing stream (runSpec)
+  # is implemented. Eventually, direct methods for specific
+  # analyses (e.g., two-document permutation tests) will be
+  # supported.
+  class Analysis
+  
+    attr_accessor :docs, :contexts, :targets
+  
+    # Read and parse the specifications for an analysis, then run the analysis.
+    # Only does basic error checking for now...
+    def self.run_spec(spec_file='default.spec')
+    
+      # Basic error checking
+      abort("Error: can't find spec file (#{spec_file}).") if !File.exist?(spec_file)
+      load spec_file
+      abort("Error: can't find contrast file (#{CONTRAST_FILE}).") if !File.exist?(CONTRAST_FILE) 
+      contrasts = parse_contrasts(CONTRAST_FILE)
+    
+      # Create contrasts
+      puts "Found #{contrasts.size} contrasts." if VERBOSE
+      
+      # Set targets
+      targets = contrasts.inject([]) { |t, c| t += c.words.flatten }.uniq
+      puts "Found #{targets.size} target words." if VERBOSE
+      
+      # Create options hash
+      opts = {}
+      %w[PARSE_TEXT N_PERM N_BOOT MAX_LINES RECODE CONTEXT_SIZE MIN_PROP STOP_FILE NORMALIZE_WEIGHTS].each { |c|
+        opts[c.downcase] = Module.const_get(c) if Module.constants.include?(c)  
+      }
+      
+      # Read in files and create documents
+      docs = []
+      FILES.each { |f| 
+        abort("Error: can't find input file #{f}.") if !File.exist?(f)
+        puts "Reading in file #{f}..."
+        text = File.new(f).read
+        docs << Document.new(f.split(/\//)[-1], targets, text, opts)
+      }
+      docs
+    
+      # Load contrasts
+      contrasts = parse_contrasts(CONTRAST_FILE)
+    
+      # Make sure N_PERM is zero if we don't want stats
+      n_perm = STATS ? N_PERM : 0
+      
+      # One or two-sample test?
+      case TEST_TYPE
+      when 1
+        docs.each { |d|
+          base = File.basename(d.name, '.txt')
+          puts "\nRunning one-sample analysis on document '#{d.name}'."
+          puts "Generating #{n_perm} bootstraps..." if VERBOSE and STATS
+          bootstrap_test(d, contrasts, "#{OUTPUT_ROOT}_#{base}_results.txt", n_perm)
+          p_values("#{OUTPUT_ROOT}_#{base}_results.txt", 'boot', true) if STATS
+        }
+        
+      when 2
+        abort("Error: in order to run a permutation test, you need to pass exactly two files as input.") if FILES.size != 2
+        puts "Running two-sample comparison between '#{File.basename(FILES[0])}' and '#{File.basename(FILES[1])}'." if VERBOSE
+        puts "Generating #{n_perm} permutations..." if VERBOSE and STATS
+        permutation_test(*docs, contrasts, "#{OUTPUT_ROOT}_results.txt", n_perm)
+        p_values("#{OUTPUT_ROOT}_results.txt", 'perm', true)
+      
+      # No other test types implemented for now.
+      else
+        
+      end    
+      puts "Done!"
+    
+    end
+  
+    # Parse contrast file. Takes a filename as input and returns an array of Contrasts.
+    def self.parse_contrasts(contrast_file)
+      File.new(contrast_file).readlines.map { |l| next if l.empty?; Contrast.parse(l) }
+    end
+  
+    # Run a permutation test comparing two Documents.
+    # * doc1, doc2: The two Documents to compare
+    # * contrasts: an array of Contrasts used to compare the documents
+    # * output_file: name of output file
+    # * n_perm: number of permutations to run
+    def self.permutation_test(doc1, doc2, contrasts, output_file, n_perm)
+
+      # Merge contexts. Could change this later to allow different contexts for each
+      # document, but that would make processing substantially slower.
+      context = doc1.context
+      context.words = context.words & doc2.context.words
+      context.index_words
+      doc1.context, doc2.context = context, context
+
+      # Generate cooccurrence matrices and get observed difference.
+      doc1.cooccurrence(NORMALIZE_WEIGHTS)
+      doc2.cooccurrence(NORMALIZE_WEIGHTS)
+    
+      outf = File.new(output_file,'w')
+      outf.puts "contrast\titeration\t#{doc1.name}\t#{doc2.name}\tdifference"
+      outf.sync = true
+      # Save observed values
+      contrasts.each { |c|
+        res1, res2, diff = compare_docs(c, doc1, doc2)
+        outf.puts "#{c.words.join(".")}\tobserved\t#{res1}\t#{res2}\t#{diff}"
+      }
+      # Run permutations and save results
+      d1, d2 = doc1.clone, doc2.clone
+      n_perm.times { |i|
+        puts "\n\nRunning permutation #{i+1}..."
+        d1.clines, d2.clines = permute_labels(doc1.clines, doc2.clines)
+        d1.cooccurrence(NORMALIZE_WEIGHTS)
+        d2.cooccurrence(NORMALIZE_WEIGHTS)
+        contrasts.each { |c|
+          res1, res2, diff = compare_docs(c, d1, d2)
+          outf.puts "#{c.words.join(".")}\tperm_#{i+1}\t#{res1}\t#{res2}\t#{diff}"
+        }
+      }
+    end
+  
+    # Do a bootstrap test comparing the bootstrapped distribution to zero.
+    # * doc: The Document object to analyze
+    # * contrasts: an array of Contrast objects to apply
+    # * output_file: name of output file
+    # * n_boot: number of bootstrap iterations to run
+    def self.bootstrap_test(doc, contrasts, output_file, n_boot)
+    
+      outf = File.new(output_file,'w')
+      outf.puts(%w[contrast result_id doc_name pair_1 pair_2 pair_3 pair_4 interaction_term].join("\t"))	  
+  		outf.sync = true
+		
+  		doc.cooccurrence(NORMALIZE_WEIGHTS)
+      contrasts.each { |c|
+  			observed = c.apply(doc)
+  			outf.puts "#{c.words.join(".")}\tobserved\t#{observed}"
+  		}
+  		d1 = doc.clone
+  		n_boot.times { |i|
+  			puts "\n\nRunning bootstrap iteration #{i+1}..." if VERBOSE
+  			d1.clines = doc.resample(clines=true)
+  			# d1.context = Context.new(d1)   # Currently uses the same context; can uncomment
+  			d1.cooccurrence(NORMALIZE_WEIGHTS)
+  			contrasts.each { |c|
+  				res = c.apply(d1)
+  				outf.puts "#{c.words.join(".")}\tboot_#{i+1}\t#{res}"
+  			}
+  		}
+  	end
+  
+    # Permute labels across two documents.
+    def self.permute_labels(lines1, lines2)
+      n1 = lines1.size
+      lines = (lines1 + lines2).sort_by{rand}
+      [lines.slice!(0,n1), lines]
+    end
+  
+    # Run pairwise contrast on two docs and return difference.
+    def self.compare_docs(contrast, doc1, doc2)
+      res1, res2 = contrast.apply(doc1).split("\t")[-1].to_f, contrast.apply(doc2).split("\t")[-1].to_f
+      [res1, res2, res1 - res2]
+    end
+  
+    # Takes the results of a bootstrap or permutation test as input and saves
+    # a file summarizing the corresponding p-values.
+    # * input_file: path to the results of the bootstrapping/permutation analysis
+    # * mode: indicates the source analysis type. Must be either 'boot' or 'perm'
+    # * mean: boolean variable indicating whether or not to compute the mean across all contrasts
+    def self.p_values(input_file, mode='boot', mean=true)
+      c = File.new(input_file).readlines
+      c.shift
+      buffer = ["file\tcontrast\tN_permutations\tvalue\tp-value"]
+      tests = {}
+      c.each { |l|
+        l = l.strip.split(/\t/)
+        row = [l[0], l[1], l[-1].to_f]
+        fname =  mode == 'boot' ? l[2] : input_file
+        tests[fname] = [] if !tests.key?(fname)
+        tests[fname] << row
+      }
+    
+      tests.each { |fname, rows|
+        dists, obs, means = {}, {}, []
+        rows.each { |row|
+          test, iter, val = row
+          if iter == 'observed'
+            obs[test] = val
+          else
+            dists[test] = [] if !dists.key?(test)
+            dists[test] << val
+            if mean
+              i = iter[/\d+$/].to_i-1
+              means[i] = 0 if means[i].nil?
+              means[i] += val
+            end
+          end
+        }
+        if mean
+          means.map! { |m| m/obs.size }
+          dists['mean'] = means
+          obs['mean'] = obs.values.inject(0) {|sum, e| sum+e }/obs.size
+        end
+      
+        dists.each { |k,v|
+          v, o = v.sort, obs[k]
+          gt = v.inject(0) { |sum, e| 
+            sum + 
+            if mode == 'perm'
+              o >= e ? 1 : 0
+            else
+              e > 0 ? 1 : 0
+            end
+          }
+          p = gt.to_f / v.size
+          p = 1 - p if p > 0.5
+          line = [fname, k, v.size, o, p*2]
+          buffer << line.join("\t")
+        }
+      
+      }
+      base = File.basename(input_file, '.txt')
+      File.new("#{base}_p_values.txt",'w').puts buffer
+    end
+  
+    private_class_method :permute_labels, :compare_docs
+
+  end
+  
+end

data/lib/cass/context.rb

@@ -0,0 +1,54 @@
+module Cass
+  
+  # Represents the context of a document, i.e., a list of words to analyze, along with an index.
+  class Context
+  
+    attr_accessor :words, :index
+  
+    def initialize(doc, opts)
+      min_prop = opts['min_prop'] || 0
+      max_prop = opts['max_prop'] || 1
+      puts "Creating new context..." if VERBOSE
+      words = doc.lines.join(' ').split(/\s+/)
+      nwords = words.size
+      puts "Found #{nwords} words."
+      if min_prop > 0 or max_prop < 1
+        word_hash = Hash.new(0)
+        words.each {|w| word_hash[w] += 1 }
+        min_t, max_t = (min_prop * nwords).round, (max_prop * nwords).round
+        words = word_hash.delete_if { |w,c| c < min_t or c > max_t }.keys
+      else
+        words.uniq!
+      end
+      # words = words - doc.targets
+      words -= opts['stop_file'].read.split(/\s+/) if opts.key?('stop_file')
+      @words = opts.key?('context_size') ? words.sort_by{rand}[0, opts['context_size']] : words
+      index_words
+      puts "Using #{@words.size} words as context." if VERBOSE
+    end
+  
+    # Index the context. Necessary when words are updated manually.
+    def index_words
+      @index = {}
+      @words.each_index { |i| @index[@words[i]] = i }
+    end
+  
+    # Convenience accessor method for getting either words in the context,
+    # or their index in the array. If an integer is passed, returns a word;
+    # If a string is passed, return the index of the word in the array.
+    def [](el)
+      el.class == Integer ? @words[el] : @index[el]
+    end
+  
+    # Returns true if a word is in the context, false otherwise.
+    def key?(k)
+      @index.key?(k)
+    end
+  
+    # Number of words in the context.
+    def size
+      @words.size
+    end
+  end
+
+end