co2_filter 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: aa78d78af6f6ed5e9dc74d4104f85759dc5299d6
+  data.tar.gz: d2b184ffa6deac97b8d0afc8e9020b92fc2263cd
+SHA512:
+  metadata.gz: 231065d35af899464b9aad9c690e29591eba7ae846f58b3886b3b5a0ce963fe69b370d2a6df52e37745b8ad6debadb09d10f8162a481a182e138afb596fc7b99
+  data.tar.gz: 46ffdb54af8bec7e72366bed2772d214344c65ee8c1a7d7cee23c33953f6aaab0075c86250a28add3c4901bd809ca519fba47d359a0a62ab7d06e51c099fe902

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2016 Pivotal
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,104 @@
+= co2_filter
+
+Co2Filter is a combination Collaborative and Content-based filtering gem, with optional access to key points in the logic chain for adding your own calculations or storing them for later.
+
+Collaborative filtering recommends based on the ratings of other users and their similarity to you.
+Content-based filtering recommends based on the attributes of items and your opinions of them.
+
+Both strategies will return a predicted rating for each item still unrated.  They can each be used alone with this gem, but they can also be combined via either an averaging of the two predicted user ratings or the content-boosted collaborative filtering technique.
+
+Content-boosted collaborative filtering first applies content-based filtering to fill out a sparse user rating set, then applies collaborative filtering on the resulting dense data.
+
+Rating ranges are irrelevant to the algorithm, as rating averages are the key point of reference.  Feel free to use whatever works best for your app.  (But be warned that this means there is a chance of predicting slightly outside of the actual range.)
+
+== Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'co2_filter', git: 'https://github.com/comatose-turtle/co2_filter.git'
+
+And then execute:
+
+    $ bundle
+
+== Usage
+
+The most basic usage follows this pattern:
+    recommended = Co2Filter.filter(current_user: current_user, other_users: other_users, items: items)
+
+And the results can be used as follows:
+
+    most_recommended_item_id = recommended.ids_by_rating.first
+    top_20_recommended_items = recommended.ids_by_rating.take(20)
+    predicted_user_rating = recommended[most_recommended_item_id]
+
+The return type is a simple wrapper for a results hash.  You can extract the inner hash if necessary with +to_hash+.
+
+This gem is ORM-agnostic and expects you to select your relevant data on your own.  The data you pass in should look like:
+    current_user = {
+        # item_id => rating
+        'item1' => 5,
+        'item2' => 1,
+        'item3' => 3
+        # ...
+    }
+
+    other_users = {
+        # user_id => { item_id => rating }
+        'user1' => {
+            'item1' => 2,
+            'item2' => 5,
+            'item4' => 2
+        },
+        'user2' => {
+            'item1' => 5,
+            'item2' => 1,
+            'item4' => 5,
+            'item5' => 1
+        }
+        # ...
+    }
+
+    # A set of all items from the dataset
+    items = {
+        # item_id => { attribute_id => strength }
+    }
+
+Ids are arbitrary to the algorithm and can be strings as easily as numbers.  Ratings and strengths should be numbers of some type, and the range should be consistent across rating types (i.e. item ratings, attribute strengths), but there is no range restriction enforced by the algorithm.
+
+Attribute strength refers to a situation where attributes are applied in varying degrees rather than a simple "off" or "on" state.  If this does not apply to your app, I suggest setting all strengths to 1.
+
+=== Using Individual Filters
+
+To implement only the collaborative filter, just use:
+    Co2Filter::Collaborative.filter(current_user: current_user, other_users: other_users)
+
+To implement only the content-based filter, use:
+    Co2Filter::ContentBased.filter(user: current_user, items: items)
+
+The content-based filtering process consists of two steps:
+1. Constructing a user profile
+2. Using the user profile to determine recommendations
+
+If you are interested in doing this process piecemeal (for instance, to save the user profile to the database for later use), you can do so:
+    user_profile = Co2Filter::ContentBased.ratings_to_profile(user_ratings: current_user, items: items)
+    Co2Filter::ContentBased.filter(user: user_profile, items: items)
+
+=== Content-Boosted Collaborative Filtering
+
+Content-boosted collaborative filtering can be used as follows:
+    Co2Filter.content_boosted_collaborative_filter(current_user: current_user, other_users: other_users, items: items)
+
+This is the most processor-intensive algorithm, but it too can be split up into multiple pieces if you wish:
+    boosted_users = Co2Filter::ContentBased.boost_ratings(users: other_users, items: items)
+    Co2Filter::Collaborative.filter(current_user: current_user, other_users: boosted_users)
+Note that the second step is simply the basic collaborative filter.  If you wish to break up the +boost_ratings+ method even further, then you are actually talking about using the +Co2Filter::ContentBased.filter+ on each of the users. (See the definition for +boost_ratings+.)
+
+== Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/comatose-turtle/co2_filter. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the {Contributor Covenant}[http://contributor-covenant.org] code of conduct.
+
+== License
+
+The gem is available as open source under the terms of the {MIT License}[http://opensource.org/licenses/MIT].
+

data/Rakefile

@@ -0,0 +1,23 @@
+require "rspec/core/rake_task"
+
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+
+require 'rdoc/task'
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.main     = "README.rdoc"
+  rdoc.title    = 'Co2Filter'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.rdoc')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+Bundler::GemHelper.install_tasks
+
+RSpec::Core::RakeTask.new(:spec)
+task :default => :spec

data/lib/co2_filter/collaborative/results.rb

@@ -0,0 +1,2 @@
+class Co2Filter::Collaborative::Results < Co2Filter::Results
+end

data/lib/co2_filter/collaborative.rb

@@ -0,0 +1,133 @@
+module Co2Filter::Collaborative
+  autoload :Results, 'co2_filter/collaborative/results'
+
+  def self.filter(current_user:, other_users:, measure: :hybrid)
+    current_user = Co2Filter::RatingSet.new(current_user) unless current_user.is_a? Co2Filter::RatingSet
+    if measure == :euclidean
+      processed_users = euclidean(current_user: current_user, other_users: other_users, num_nearest: 30)
+    elsif measure == :cosine
+      processed_users = mean_centered_cosine(current_user: current_user, other_users: other_users, num_nearest: 30)
+    else
+      eu = euclidean(current_user: current_user, other_users: other_users, num_nearest: 30)
+      co = mean_centered_cosine(current_user: current_user, other_users: other_users, num_nearest: 30)
+      processed_users = {}
+      eu.each do |user_id, user|
+        processed_users[user_id] = user.merge(co[user_id]) do |k, val1, val2|
+          k == :coefficient ? (val1 + val2) / 2.0 : val1
+        end
+      end
+    end
+
+    new_items = []
+    processed_users.each do |user_id, user|
+      new_items = new_items | (user[:ratings].keys - current_user.keys)
+    end
+
+    item_ratings = {}
+    new_items.each do |item_id|
+      rating_influence_total = 0
+      weight_normal = 0
+      processed_users.reject do |user_id, user|
+        user[:ratings][item_id].nil?
+      end.each do |user_id, user|
+        rating_influence_total += user[:coefficient] * (user[:ratings][item_id] - user[:mean])
+        weight_normal += user[:coefficient].abs
+      end
+      item_ratings[item_id] = current_user.mean + rating_influence_total / weight_normal if weight_normal > 0
+    end
+
+    Results.new(item_ratings)
+  end
+
+  def self.mean_centered_cosine(current_user:, other_users:, num_nearest:)
+    processed = other_users.map do |key, user2|
+      user2 = Co2Filter::RatingSet.new(user2) unless user2.is_a? Co2Filter::RatingSet
+      [key, single_cosine(current_user, user2)]
+    end
+    processed.sort_by do |entry|
+      -(entry[1][:coefficient].abs)
+    end.take(num_nearest).inject({}) do |hash, (key, value)|
+      hash[key] = value
+      hash
+    end
+  end
+
+  def self.single_cosine(user1, user2)
+    sum1 = 0
+    sum2 = 0
+    union = user1.keys | user2.keys
+    union.each do |key|
+      if user1[key]
+        sum1 += user1[key]
+      end
+      if user2[key]
+        sum2 += user2[key]
+      end
+    end
+    mean1 = user1.length == 0 ? 0 : sum1 / user1.length
+    mean2 = user2.length == 0 ? 0 : sum2 / user2.length
+
+    numerator = 0
+    denominator1 = 0
+    denominator2 = 0
+    union.each do |key|
+      deviation1 = user1[key] ? user1[key] - mean1 : 0
+      deviation2 = user2[key] ? user2[key] - mean2 : 0
+
+      numerator += deviation1 * deviation2
+      denominator1 += deviation1**2
+      denominator2 += deviation2**2
+    end
+    {
+      ratings: user2,
+      mean: mean2,
+      coefficient: (denominator1 * denominator2 == 0 ? 0 : numerator / Math.sqrt(denominator1 * denominator2))
+    }
+  end
+
+  def self.euclidean(current_user:, other_users:, num_nearest:, range:0)
+    if range == 0
+      lowest = nil
+      highest = nil
+      current_user.each do |k, rating|
+        lowest = rating if !lowest || lowest > rating
+        highest = rating if !highest || highest < rating
+      end
+      other_users.each do |k, user|
+        user.each do |k, rating|
+          lowest = rating if !lowest || lowest > rating
+          highest = rating if !highest || highest < rating
+        end
+      end
+      range = highest - lowest
+    end
+    processed = other_users.map do |key, user2|
+      user2 = Co2Filter::RatingSet.new(user2) unless user2.is_a? Co2Filter::RatingSet
+      [key, single_euclidean(current_user, user2, range)]
+    end
+    processed.sort_by do |entry|
+      -(entry[1][:coefficient])
+    end.take(num_nearest).inject({}) do |hash, (key, value)|
+      hash[key] = value
+      hash
+    end
+  end
+
+  def self.single_euclidean(user1, user2, range)
+    numerator = 0
+    denominator = 0
+    intersect = user1.keys & user2.keys
+    intersect.each do |item_id|
+      numerator += (user1[item_id] - user2[item_id]) ** 2
+      denominator += range ** 2
+    end
+    relevancy_weight = intersect.size < 50.0 ? intersect.size / 50.0 : 1
+    coefficient = relevancy_weight * (1 - ((1.0 * numerator / denominator)**(0.5)))
+    user2 = Co2Filter::RatingSet.new(user2) unless user2.is_a? Co2Filter::RatingSet
+    {
+        ratings: user2.to_hash,
+        mean: user2.mean,
+        coefficient: coefficient
+    }
+  end
+end

data/lib/co2_filter/content_based/results.rb

@@ -0,0 +1,2 @@
+class Co2Filter::ContentBased::Results < Co2Filter::Results
+end

data/lib/co2_filter/content_based/user_profile.rb

@@ -0,0 +1,8 @@
+class Co2Filter::ContentBased::UserProfile < Co2Filter::HashWrapper
+  attr_accessor :mean
+
+  def initialize(data, mean)
+    super(data)
+    @mean = mean
+  end
+end

data/lib/co2_filter/content_based.rb

@@ -0,0 +1,56 @@
+module Co2Filter::ContentBased
+  autoload :Results, 'co2_filter/content_based/results'
+  autoload :UserProfile, 'co2_filter/content_based/user_profile'
+
+  def self.filter(user:, items:)
+    if(user.is_a?(UserProfile))
+      user_profile = user
+      new_items = items
+    elsif(user.is_a?(Hash) || user.is_a?(Co2Filter::RatingSet))
+      user = Co2Filter::RatingSet.new(user) if user.is_a?(Hash)
+      user_profile = ratings_to_profile(user_ratings: user, items: items)
+      new_items = items.reject{|item_id, v| user[item_id]}
+    end
+    results = new_items.inject({}) do |hash, (item_id, item)|
+      strength_normalizer = 0
+      hash[item_id] = 0
+      item.each do |attr_id, strength|
+        hash[item_id] += user_profile[attr_id].to_f * strength
+        strength_normalizer += strength.abs if user_profile[attr_id]
+      end
+      hash[item_id] /= strength_normalizer if strength_normalizer != 0
+      hash[item_id] += user_profile.mean
+      hash
+    end
+    Results.new(results)
+  end
+
+  def self.ratings_to_profile(user_ratings:, items:)
+    user_ratings = Co2Filter::RatingSet.new(user_ratings) unless user_ratings.is_a? Co2Filter::RatingSet
+    user_prefs = {}
+    strength_normalizers = {}
+    user_ratings.each do |item_id, score|
+      deviation = score - user_ratings.mean
+
+      items[item_id].each do |attr_id, strength|
+        user_prefs[attr_id] ||= 0
+        user_prefs[attr_id] += strength * deviation
+        strength_normalizers[attr_id] ||= 0
+        strength_normalizers[attr_id] += strength.abs
+      end
+    end
+
+    user_prefs.each do |attr_id, score|
+      user_prefs[attr_id] /= strength_normalizers[attr_id].to_f
+    end
+
+    UserProfile.new(user_prefs, user_ratings.mean)
+  end
+
+  def self.boost_ratings(users:, items:)
+    users.inject({}) do |content_boosted_users, (user_id, ratings)|
+      content_boosted_users[user_id] = ratings.merge(filter(user: ratings, items: items))
+      content_boosted_users
+    end
+  end
+end

data/lib/co2_filter/hash_wrapper.rb

@@ -0,0 +1,17 @@
+class Co2Filter::HashWrapper
+  def initialize(data)
+    @data = data.to_hash
+  end
+
+  def method_missing(method, *args, &block)
+    if [:keys, :values, :length, :size, :"[]", :"[]=", :each, :merge]
+      @data.send(method, *args, &block)
+    else
+      super(method, *args, &block)
+    end
+  end
+
+  def to_hash
+    @data
+  end
+end

data/lib/co2_filter/rating_set.rb

@@ -0,0 +1,10 @@
+class Co2Filter::RatingSet < Co2Filter::HashWrapper
+  def mean
+    @mean ||= 1.0 * @data.values.inject(:+) / @data.size
+  end
+
+  def []=(key, val)
+    super(key, val)
+    @mean = nil
+  end
+end

data/lib/co2_filter/results.rb

@@ -0,0 +1,10 @@
+class Co2Filter::Results < Co2Filter::HashWrapper
+  def ids_by_rating
+    @ids_by_rating ||=
+      @data.sort_by do |id, item_ranking|
+        -item_ranking
+      end.map do |el|
+        el[0]
+      end
+  end
+end

data/lib/co2_filter/version.rb

@@ -0,0 +1,3 @@
+module Co2Filter
+  VERSION = "0.0.1"
+end

data/lib/co2_filter.rb

@@ -0,0 +1,32 @@
+module Co2Filter
+  def self.filter(current_user: , other_users: , items: nil, user_profile: nil, content_based_results: nil)
+    raise ArgumentError.new("An 'items' or 'content_based_results' argument must be provided.") unless items || content_based_results
+    collab = Collaborative.filter(current_user: current_user, other_users: other_users)
+    
+    if content_based_results && content_based_results.is_a?(Results)
+      content = content_based_results
+    elsif user_profile.is_a? ContentBased::UserProfile
+      content = ContentBased.filter(user: user_profile, items: items)
+    else
+      content = ContentBased.filter(user: current_user, items: items)
+    end
+
+    hybrid = collab.merge(content) do |k, val1, val2|
+      (val1 + val2) / 2.0
+    end
+    Results.new(hybrid)
+  end
+
+  def self.content_boosted_collaborative_filter(current_user:, other_users:, items:)
+    content_boosted_users = ContentBased.boost_ratings(users: other_users, items: items)
+    results = Collaborative.filter(current_user: current_user, other_users: content_boosted_users)
+    Results.new(results)
+  end
+
+  autoload :VERSION, 'co2_filter/version'
+  autoload :Collaborative, 'co2_filter/collaborative'
+  autoload :ContentBased,  'co2_filter/content_based'
+  autoload :Results, 'co2_filter/results'
+  autoload :RatingSet,  'co2_filter/rating_set'
+  autoload :HashWrapper,  'co2_filter/hash_Wrapper'
+end

data/lib/tasks/co2_filter_tasks.rake

@@ -0,0 +1,4 @@
+# desc "Explaining what the task does"
+# task :co2_filter do
+#   # Task goes here
+# end

metadata

@@ -0,0 +1,101 @@
+--- !ruby/object:Gem::Specification
+name: co2_filter
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+platform: ruby
+authors:
+- Tommy Orr
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2016-02-19 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.11'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.11'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+description: 
+email:
+- torr@pivotal.io
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- MIT-LICENSE
+- README.rdoc
+- Rakefile
+- lib/co2_filter.rb
+- lib/co2_filter/collaborative.rb
+- lib/co2_filter/collaborative/results.rb
+- lib/co2_filter/content_based.rb
+- lib/co2_filter/content_based/results.rb
+- lib/co2_filter/content_based/user_profile.rb
+- lib/co2_filter/hash_wrapper.rb
+- lib/co2_filter/rating_set.rb
+- lib/co2_filter/results.rb
+- lib/co2_filter/version.rb
+- lib/tasks/co2_filter_tasks.rake
+homepage: https://github.com/comatose-turtle/co2_filter
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.4.5.1
+signing_key: 
+specification_version: 4
+summary: Uses both collaborative and content-based filtering methods to enable a complex,
+  hybrid recommendation engine.
+test_files: []