tf-idf-similarity 0.0.1

data/.gitignore

@@ -0,0 +1,6 @@
+*.gem
+.bundle
+.yardoc
+Gemfile.lock
+doc/*
+pkg/*

data/.travis.yml

@@ -0,0 +1,3 @@
+language: ruby
+rvm:
+  - 1.9.3

data/Gemfile

@@ -0,0 +1,4 @@
+source "http://rubygems.org"
+
+# Specify your gem's dependencies in scraperwiki-api.gemspec
+gemspec

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2012 Open North Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,70 @@
+# Ruby Vector Space Model (VSM) with tf*idf weights
+
+Calculates the similarity between texts using a [bag-of-words](http://en.wikipedia.org/wiki/Bag_of_words_model) [Vector Space Model](http://en.wikipedia.org/wiki/Vector_space_model) with [Term Frequency-Inverse Document Frequency](http://en.wikipedia.org/wiki/Tf*idf) weights. If your use case demands performance, use [Lucene](http://lucene.apache.org/core/) (or similar), which also implements other information retrieval functions like [BM 25](http://en.wikipedia.org/wiki/Okapi_BM25).
+
+## Usage
+
+    require 'tf-idf-similarity'
+
+    corpus = TfIdfSimilarity::Collection.new
+    corpus << TfIdfSimilarity::Document.new("Lorem ipsum dolor sit amet...")
+    corpus << TfIdfSimilarity::Document.new("Pellentesque sed ipsum dui...")
+    corpus << TfIdfSimilarity::Document.new("Nam scelerisque dui sed leo...")
+
+    p corpus.similarity_matrix
+
+This gem will use the [gsl gem](http://rb-gsl.rubyforge.org/) if available, for faster matrix multiplication.
+
+## Optimizations
+
+### [GNU Scientific Library (GSL)](http://www.gnu.org/software/gsl/)
+
+The latest `gsl` gem (`1.14.7`) is [not compatible](http://bretthard.in/2012/03/getting-related_posts-lsi-and-gsl-to-work-in-jekyll/) with the `gsl` package (`1.15`) in Homebrew:
+
+```sh
+cd /usr/local
+git checkout -b gsl-1.14 83ed49411f076e30ced04c2cbebb054b2645a431
+brew install gsl
+git checkout master
+git branch -d gsl-1.14
+gem install gsl
+```
+
+### [Automatically Tuned Linear Algebra Software (ATLAS)](http://math-atlas.sourceforge.net/)
+
+You may know this software through [Linear Algebra PACKage (LAPACK)](http://www.netlib.org/lapack/) or [Basic Linear Algebra Subprograms (BLAS)](http://www.netlib.org/blas/).
+
+The `nmatrix` gem (`0.0.1`) can't find the `cblas.h` and `clapack.h` header files. Either [set the C_INCLUDE_PATH](https://github.com/SciRuby/nmatrix#synopsis):
+
+    export C_INCLUDE_PATH=/System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/Headers/
+
+Or [create links](https://github.com/SciRuby/nmatrix/issues/21) before installing the gem:
+
+    sudo ln -s /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/Headers/cblas.h /usr/include/cblas.h
+    sudo ln -s /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/Headers/clapack.h /usr/include/clapack.h
+
+Version `0.0.2` [doesn't compile on Mac OS X Lion](https://github.com/SciRuby/nmatrix/issues/34).
+
+### Other Considerations
+
+The [narray](http://narray.rubyforge.org/) and [nmatrix](http://sciruby.com/nmatrix/) gems have no method to calculate the magnitude of a vector. [Ruby-LAPACK](http://ruby.gfd-dennou.org/products/ruby-lapack/) is a very thin wrapper around LAPACK, which has an opaque Fortran-style naming scheme. [Linalg](https://github.com/quix/linalg) and [RNum](http://rnum.rubyforge.org/) and old and not available as gems.
+
+## Extras
+
+You can access more term frequency, document frequency, and normalization formulas with:
+
+    require 'tf-idf-similarity/extras/collection'
+    require 'tf-idf-similarity/extras/document'
+
+The default tf*idf formula follows the [Lucene Conceptual Scoring Formula](http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/similarities/TFIDFSimilarity.html).
+
+## Reference
+
+* [G. Salton and C. Buckley. "Term Weighting Approaches in Automatic Text Retrieval."" Technical Report. Cornell University, Ithaca, NY, USA. 1987.](http://www.cs.odu.edu/~jbollen/IR04/readings/article1-29-03.pdf)
+* [E. Chisholm and T. G. Kolda. "New term weighting formulas for the vector space method in information retrieval." Technical Report Number ORNL-TM-13756. Oak Ridge National Laboratory, Oak Ridge, TN, USA. 1999.](http://www.sandia.gov/~tgkolda/pubs/bibtgkfiles/ornl-tm-13756.pdf)
+
+## Bugs? Questions?
+
+This gem's main repository is on GitHub: [http://github.com/opennorth/tf-idf-similarity](http://github.com/opennorth/tf-idf-similarity), where your contributions, forks, bug reports, feature requests, and feedback are greatly welcomed.
+
+Copyright (c) 2012 Open North Inc., released under the MIT license

data/Rakefile

@@ -0,0 +1,16 @@
+require 'bundler'
+Bundler::GemHelper.install_tasks
+
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec
+
+begin
+  require 'yard'
+  YARD::Rake::YardocTask.new
+rescue LoadError
+  task :yard do
+    abort 'YARD is not available. In order to run yard, you must: gem install yard'
+  end
+end

data/USAGE

@@ -0,0 +1 @@
+See README.md for full usage details.

data/lib/tf-idf-similarity.rb

@@ -0,0 +1,7 @@
+$LOAD_PATH.unshift(File.expand_path(File.dirname(__FILE__))) unless $LOAD_PATH.include?(File.expand_path(File.dirname(__FILE__)))
+
+module TfIdfSimilarity
+  autoload :Collection, 'tf-idf-similarity/collection'
+  autoload :Document, 'tf-idf-similarity/document'
+  autoload :Token, 'tf-idf-similarity/token'
+end

data/lib/tf-idf-similarity/collection.rb

@@ -0,0 +1,128 @@
+begin
+  require 'gsl'
+rescue LoadError
+  require 'matrix'
+end
+
+class TfIdfSimilarity::Collection
+  # The documents in the collection.
+  attr_reader :documents
+  # The number of times each term appears in all documents.
+  attr_reader :term_counts
+  # The number of documents each term appears in.
+  attr_reader :document_counts
+
+  def initialize
+    @documents       = []
+    @term_counts     = Hash.new 0
+    @document_counts = Hash.new 0
+  end
+
+  def <<(document)
+    document.term_counts.each do |term,count|
+      @term_counts[term]     += count
+      @document_counts[term] += 1
+    end
+    @documents << document
+  end
+
+  # @return [Array<String>] the set of the collection's terms with no duplicates
+  def terms
+    term_counts.keys
+  end
+
+  # @see http://en.wikipedia.org/wiki/Vector_space_model
+  # @see http://en.wikipedia.org/wiki/Document-term_matrix
+  # @see http://en.wikipedia.org/wiki/Cosine_similarity
+  def similarity_matrix
+    if matrix?
+      idf = []
+      term_document_matrix = Matrix.build(terms.size, documents.size) do |i,j|
+        idf[i] ||= inverse_document_frequency terms[i]
+        documents[j].term_frequency(terms[i]) * idf[i]
+      end
+    else
+      term_document_matrix = if gsl?
+        GSL::Matrix.alloc terms.size, documents.size
+      elsif narray?
+        NMatrix.float documents.size, terms.size
+      elsif nmatrix?
+        # The nmatrix gem's sparse matrices are unfortunately buggy.
+        # @see https://github.com/SciRuby/nmatrix/issues/35
+        NMatrix.new([terms.size, documents.size], :float64)
+      end
+
+      terms.each_with_index do |term,i|
+        idf = inverse_document_frequency term
+        documents.each_with_index do |document,j|
+          tfidf = document.term_frequency(term) * idf
+          if gsl? || nmatrix?
+            term_document_matrix[i, j] = tfidf
+          # NArray puts the dimensions in a different order.
+          # @see http://narray.rubyforge.org/SPEC.en
+          elsif narray?
+            term_document_matrix[j, i] = tfidf
+          end
+        end
+      end
+    end
+
+    # Columns are normalized to unit vectors, so we can calculate the cosine
+    # similarity of all document vectors.
+    matrix = normalize term_document_matrix
+
+    if nmatrix?
+      matrix.transpose.dot matrix
+    else
+      matrix.transpose * matrix
+    end
+  end
+
+  # @param [String] term a term
+  # @return [Float] the term's inverse document frequency
+  #
+  # @see http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/similarities/TFIDFSimilarity.html
+  def inverse_document_frequency(term)
+    1 + Math.log(documents.size / (document_counts[term].to_f + 1))
+  end
+  alias_method :idf, :inverse_document_frequency
+
+  # @param [Document] matrix a term-document matrix
+  # @return [Matrix] a matrix in which all document vectors are unit vectors
+  #
+  # @note Lucene normalizes document length differently.
+  def normalize(matrix)
+    if gsl?
+      matrix.each_col(&:normalize!)
+    elsif narray?
+      # @todo NArray doesn't have a method to normalize a vector.
+      # 0.upto(matrix.shape[0] - 1).each do |j|
+      #   matrix[j, true] # Normalize this column somehow.
+      # end
+      matrix
+    elsif nmatrix?
+      # @todo NMatrix doesn't have a method to normalize a vector.
+      matrix
+    else
+      Matrix.columns matrix.column_vectors.map(&:normalize)
+    end
+  end
+
+private
+
+  def gsl?
+    @gsl     ||= Object.const_defined?(:GSL)
+  end
+
+  def narray?
+    @narray  ||= Object.const_defined?(:NArray) && !gsl?
+  end
+
+  def nmatrix?
+    @nmatrix ||= Object.const_defined?(:NMatrix) && !narray?
+  end
+
+  def matrix?
+    @matrix  ||= Object.const_defined?(:Matrix)
+  end
+end

data/lib/tf-idf-similarity/document.rb

@@ -0,0 +1,62 @@
+# coding: utf-8
+require 'unicode_utils'
+
+class TfIdfSimilarity::Document
+  # An optional document identifier.
+  attr_reader :id
+  # The document's text.
+  attr_reader :text
+  # The number of times each term appears in the document.
+  attr_reader :term_counts
+
+  # @param [String] text the document's text
+  # @param [Hash] opts optional arguments
+  # @option opts [String] :id a string to identify the document
+  def initialize(text, opts = {})
+    @text        = text
+    @id          = opts[:id] || object_id
+    @term_counts = Hash.new 0
+    process
+  end
+
+  # @return [Array<String>] the set of the document's terms with no duplicates
+  def terms
+    term_counts.keys
+  end
+  
+  # @param [String] term a term
+  # @return [Float] the square root of the term count
+  #
+  # @see http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/similarities/TFIDFSimilarity.html
+  def term_frequency(term)
+    Math.sqrt term_counts[term]
+  end
+  alias_method :tf, :term_frequency
+
+private
+
+  # Tokenize the text and counts terms.
+  def process
+    tokenize(text).each do |word|
+      token = TfIdfSimilarity::Token.new word
+      if token.valid?
+        @term_counts[token.lowercase_filter.classic_filter.to_s] += 1
+      end
+    end
+  end
+
+  # Tokenizes a text, respecting the word boundary rules from Unicode’s Default
+  # Word Boundary Specification.
+  #
+  # @param [String] text a text
+  # @return [Enumerator] a token enumerator
+  #
+  # @note We should evaluate the tokenizers by {http://www.sciencemag.org/content/suppl/2010/12/16/science.1199644.DC1/Michel.SOM.revision.2.pdf Google}
+  #   or {http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.UAX29URLEmailTokenizerFactory Solr}.
+  #
+  # @see http://unicode.org/reports/tr29/#Default_Word_Boundaries
+  # @see http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.StandardTokenizerFactory
+  def tokenize(text)
+    UnicodeUtils.each_word text
+  end
+end

data/lib/tf-idf-similarity/extras/collection.rb

@@ -0,0 +1,85 @@
+require 'tf-idf-similarity/collection'
+
+class TfIdfSimilarity::Collection
+  # SMART n, Salton x, Chisholm NONE
+  def no_collection_frequency(term)
+    1.0
+  end
+
+  # SMART t, Salton f, Chisholm IDFB
+  def plain_inverse_document_frequency(term)
+    count = document_counts[term].to_f
+    Math.log documents.size / count
+  end
+  alias_method :plain_idf, :plain_inverse_document_frequency
+
+  # SMART p, Salton p, Chisholm IDFP
+  def probabilistic_inverse_document_frequency(term)
+    count = document_counts[term].to_f
+    Math.log (documents.size - count) / count
+  end
+  alias_method :probabilistic_idf, :probabilistic_inverse_document_frequency
+
+  # Chisholm IGFF
+  def global_frequency_inverse_document_frequency(term)
+    term_counts[term] / document_counts[term].to_f
+  end
+  alias_method :gfidf, :global_frequency_inverse_document_frequency
+
+  # Chisholm IGFL
+  def log_global_frequency_inverse_document_frequency(term)
+    Math.log global_frequency_inverse_document_frequency(term) + 1
+  end
+  alias_method :log_gfidf, :log_global_frequency_inverse_document_frequency
+
+  # Chisholm IGFI
+  def incremented_global_frequency_inverse_document_frequency(term)
+    global_frequency_inverse_document_frequency(term) + 1
+  end
+  alias_method :incremented_gfidf, :incremented_global_frequency_inverse_document_frequency
+
+  # Chisholm IGFS
+  def square_root_global_frequency_inverse_document_frequency(term)
+    Math.sqrt global_frequency_inverse_document_frequency(term) - 0.9
+  end
+  alias_method :square_root_gfidf, :square_root_global_frequency_inverse_document_frequency
+
+  # Chisholm ENPY
+  def entropy(term)
+    denominator = term_counts[term].to_f
+    logN = Math.log documents.size
+    1 + documents.reduce(0) do |sum,document|
+      quotient = document.term_counts[term] / denominator
+      sum += quotient * Math.log(quotient) / logN
+    end
+  end
+
+
+
+  # @param [Document] matrix a term-document matrix
+  # @return [Matrix] the same matrix
+  #
+  # SMART n, Salton x, Chisholm NONE
+  def no_normalization(matrix)
+    matrix
+  end
+
+  # @param [Document] matrix a term-document matrix
+  # @return [Matrix] a matrix in which all document vectors are unit vectors
+  #
+  # SMART c, Salton c, Chisholm COSN
+  def cosine_normalization(matrix)
+    Matrix.columns(tfidf.column_vectors.map do |column|
+      column.normalize
+    end)
+  end
+
+  # @param [Document] matrix a term-document matrix
+  # @return [Matrix] a matrix
+  #
+  # SMART u, Chisholm PUQN
+  def pivoted_unique_normalization(matrix)
+    # @todo
+    # http://nlp.stanford.edu/IR-book/html/htmledition/pivoted-normalized-document-length-1.html
+  end
+end

data/lib/tf-idf-similarity/extras/document.rb

@@ -0,0 +1,118 @@
+require 'tf-idf-similarity/document'
+
+class TfIdfSimilarity::Document
+  # @return [Float] the maximum term count of any term in the document
+  def maximum_term_count
+    @maximum_term_count ||= @term_counts.values.max.to_f
+  end
+
+  # @return [Float] the average term count of all terms in the document
+  def average_term_count
+    @average_term_count ||= @term_counts.values.reduce(:+) / @term_counts.size.to_f
+  end
+
+
+
+  # Returns the term count.
+  #
+  # SMART n, Salton t, Chisholm FREQ
+  def plain_term_frequency(term)
+    term_counts[term]
+  end
+  alias :plain_tf, :plain_term_frequency
+
+  # Returns 1 if the term is present, 0 otherwise.
+  #
+  # SMART b, Salton b, Chisholm BNRY
+  def binary_term_frequency(term)
+    count = term_counts[term]
+    if count > 0
+      1
+    else
+      0
+    end
+  end
+  alias_method :binary_tf, :binary_term_frequency
+
+  # Normalizes the term count by the maximum term count.
+  #
+  # @see http://en.wikipedia.org/wiki/Tf*idf
+  def normalized_term_frequency(term)
+    term_counts[term] / maximum_term_count
+  end
+  alias_method :normalized_tf, :normalized_term_frequency
+
+  # Further normalizes the normalized term frequency to lie between 0.5 and 1.
+  #
+  # SMART a, Salton n, Chisholm ATF1
+  def augmented_normalized_term_frequency(term)
+    0.5 + 0.5 * normalized_term_frequency(term)
+  end
+  alias_method :augmented_normalized_tf, :augmented_normalized_term_frequency
+
+  # Chisholm ATFA
+  def augmented_average_term_frequency(term)
+    count = term_counts[term]
+    if count > 0
+      0.9 + 0.1 * count / average_term_count
+    else
+      0
+    end
+  end
+  alias_method :augmented_average_tf, :augmented_average_term_frequency
+
+  # Chisholm ATFC
+  def changed_coefficient_augmented_normalized_term_frequency(term)
+    count = term_counts[term]
+    if count > 0
+      0.2 + 0.8 * count / maximum_term_count
+    else
+      0
+    end
+  end
+  alias_method :changed_coefficient_augmented_normalized_tf, :changed_coefficient_augmented_normalized_term_frequency
+
+  # SMART l, Chisholm LOGA
+  def log_term_frequency(term)
+    count = term_counts[term]
+    if count > 0
+      1 + Math.log(count)
+    else
+      0
+    end
+  end
+  alias_method :log_tf, :log_term_frequency
+
+  # SMART L, Chisholm LOGN
+  def normalized_log_term_frequency(term)
+    count = term_counts[term]
+    if count > 0
+      (1 + Math.log(count)) / (1 + Math.log(average_term_count))
+    else
+      0
+    end
+  end
+  alias_method :normalized_log_tf, :normalized_log_term_frequency
+
+  # Chisholm LOGG
+  def augmented_log_term_frequency(term)
+    count = term_counts[term]
+    if count > 0
+      0.2 + 0.8 * Math.log(count + 1)
+    else
+      0
+    end
+  end
+  alias_method :augmented_log_tf, :augmented_log_term_frequency
+
+  # Chisholm SQRT
+  def square_root_term_frequency(term)
+    count = term_counts[term]
+    if count > 0
+      Math.sqrt(count - 0.5) + 1
+    else
+      0
+    end
+  end
+  alias_method :square_root_tf, :square_root_term_frequency
+end

data/lib/tf-idf-similarity/token.rb

@@ -0,0 +1,42 @@
+# coding: utf-8
+
+# @note We can add more filters from Solr and stem using Porter's Snowball.
+#
+# @see https://github.com/aurelian/ruby-stemmer
+# @see http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.StopFilterFactory
+# @see http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.WordDelimiterFilterFactory
+# @see http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory
+class TfIdfSimilarity::Token < String
+  # Returns a falsy value if all its characters are numbers, punctuation,
+  # whitespace or control characters.
+  #
+  # @note Some implementations ignore one and two-letter words.
+  #
+  # @return [Boolean] whether the string is a token
+  def valid?
+    !self[%r{
+      \A
+        (
+         \d           | # number
+         \p{Cntrl}    | # control character
+         \p{Punct}    | # punctuation
+         [[:space:]]    # whitespace
+        )+
+      \z
+    }x]
+  end
+
+  # @return [Token] a lowercase string
+  #
+  # @see http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LowerCaseFilterFactory
+  def lowercase_filter
+    self.class.new UnicodeUtils.downcase(self, :fr)
+  end
+
+  # @return [Token] a string with no English possessive or periods in acronyms
+  #
+  # @see http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.ClassicFilterFactory
+  def classic_filter
+    self.class.new self.gsub('.', '').chomp("'s")
+  end
+end

data/lib/tf-idf-similarity/version.rb

@@ -0,0 +1,3 @@
+module TfIdfSimilarity
+  VERSION = "0.0.1"
+end

data/td-idf-similarity.gemspec

@@ -0,0 +1,22 @@
+# -*- encoding: utf-8 -*-
+$:.push File.expand_path("../lib", __FILE__)
+require "tf-idf-similarity/version"
+
+Gem::Specification.new do |s|
+  s.name        = "tf-idf-similarity"
+  s.version     = TfIdfSimilarity::VERSION
+  s.platform    = Gem::Platform::RUBY
+  s.authors     = ["Open North"]
+  s.email       = ["info@opennorth.ca"]
+  s.homepage    = "http://github.com/opennorth/tf-idf-similarity"
+  s.summary     = %q{Calculates the similarity between texts using tf*idf}
+
+  s.files         = `git ls-files`.split("\n")
+  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
+  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
+  s.require_paths = ["lib"]
+
+  s.add_runtime_dependency('unicode_utils')
+  s.add_development_dependency('rspec', '~> 2.10')
+  s.add_development_dependency('rake')
+end

metadata

@@ -0,0 +1,114 @@
+--- !ruby/object:Gem::Specification
+name: tf-idf-similarity
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+  prerelease: 
+platform: ruby
+authors:
+- Open North
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2012-09-10 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: unicode_utils
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '2.10'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '2.10'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: 
+email:
+- info@opennorth.ca
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- .gitignore
+- .travis.yml
+- Gemfile
+- LICENSE
+- README.md
+- Rakefile
+- USAGE
+- lib/tf-idf-similarity.rb
+- lib/tf-idf-similarity/collection.rb
+- lib/tf-idf-similarity/document.rb
+- lib/tf-idf-similarity/extras/collection.rb
+- lib/tf-idf-similarity/extras/document.rb
+- lib/tf-idf-similarity/token.rb
+- lib/tf-idf-similarity/version.rb
+- td-idf-similarity.gemspec
+homepage: http://github.com/opennorth/tf-idf-similarity
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+      segments:
+      - 0
+      hash: 697007281194730821
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+      segments:
+      - 0
+      hash: 697007281194730821
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.24
+signing_key: 
+specification_version: 3
+summary: Calculates the similarity between texts using tf*idf
+test_files: []