soulheart 0.0.2 → 0.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: eb5870c5f1cf50c4ebac25363602c2f3cf0c4dd8
-  data.tar.gz: ab86a41e246856d395eacc5b896f36f2cb0ed254
+  metadata.gz: b170565afcbbfc53d9ab5f0a46713af88803a3a4
+  data.tar.gz: 58445b7ecdf5713487ba1af3539ad06137a04877
 SHA512:
-  metadata.gz: 83fff39abcf059750d36f89a5aae935e2a8cc579cbfa248b0ede8abe29b2d79316ec25f486415132c8570e4a6a05f8a9300d5a8f41eea1517aeaeaede5830d31
-  data.tar.gz: 5f1ec6654a3b373ea5a4ab686005b85484e4a11d2ffd7b7e00d39932121736475d5d8176801f8f3823500780c14d4283f6c749ab663caed75ebefe0337f59ef1
+  metadata.gz: 169f0de49bcd01b4ec5b2182f0075e220ea25b0059f16bffb9347567a1012d2925c6efef9a35b212825ba6989a97e60f18970b47f8d5df2e9bed2ea6dedf7fb6
+  data.tar.gz: b5b5de55c31b6cd03be318f05c08780302b3e1eee20b124e4eda1b737ffba4eee526b051d6969643ee7709b9f90123c485b21f9efdcdf2c9677c6fe0e66201be

data/{README.markdown → README.md}

@@ -1,70 +1,76 @@
-Soulheart [![Build Status](https://travis-ci.org/sethherr/soulheart.svg)](https://travis-ci.org/sethherr/soulheart) [![Code Climate](https://codeclimate.com/github/sethherr/soulheart/badges/gpa.svg)](https://codeclimate.com/github/sethherr/soulheart) [![Test Coverage](https://codeclimate.com/github/sethherr/soulheart/badges/coverage.svg)](https://codeclimate.com/github/sethherr/soulheart/coverage)
-========
+# <img src="https://raw.githubusercontent.com/sethherr/soulheart/master/logo.png" alt="Soulheart" width="200"> Soulheart [![Build Status](https://travis-ci.org/sethherr/soulheart.svg)](https://travis-ci.org/sethherr/soulheart) [![Code Climate](https://codeclimate.com/github/sethherr/soulheart/badges/gpa.svg)](https://codeclimate.com/github/sethherr/soulheart) [![Test Coverage](https://codeclimate.com/github/sethherr/soulheart/badges/coverage.svg)](https://codeclimate.com/github/sethherr/soulheart/coverage)
 
-This is an updated fork of [Seatgeek/Soulmate](https://github.com/seatgeek/soulmate) to address a few issues - namely [CORS support](../../issues/2), [minimum entry length](../../issues/3) and [playing better with Selectize & Select2](../../issues/4) - also the future.
 
-Since [Seatgeek no longer uses Soulmate](https://news.ycombinator.com/item?id=9317891), and this is a pretty large rewrite that doesn't emphasize backward compatibility, it's a new project and gem.
+**Soulheart is a ready to use remote data source for autocomplete**. It supports:
 
-========
+- pagination
+- categories
+- sorting by priority (not just alphabetically)
+- arbitrary returns
+- loading data via gists 
+- mounting standalone or inside of a rails app
 
-**Every item has to have a unique name**
+... and is [instantly deployable to heroku](https://heroku.com/deploy) (for free).
 
-Loading data: 
+To get started, check out examples and documentation at [sethherr.github.io/soulheart](https://sethherr.github.io/soulheart).
 
-Pushing new options via rake task. Use a gist url! Do it with .csvs!
 
-Structuring data
+---
 
-The only required attribute is term, which is the search term to go by. So this would be a valid thing:
+### Adding data
 
-```javascript
-[
-  { "term": "Something sweet" },
-  { "term": "The color blue" }
-]
-```
+You can add data from json, CSV and TSV files. 
 
-Here are all the possible (non-required) values:
+Adding data is very simple - all you need is a `text` value.
 
+Soulheart uses [line delineated JSON streams](https://en.wikipedia.org/wiki/JSON_Streaming#Line_delimited_JSON), so it doesn't have to load the whole file into memory. Which just means - put each object onto a seperate line.
 
-| Key         | Type     | Default    |
-| ----------- | -----    | ---------- |
-| `priority`  | integer  | 100        |
-| `types`     | string   | 'default'  |
-| `data`      | hash     | {}         |
+For the simplest case, with just text values in JSON:
 
-e.g.
+    { "text": "Jamis" }
+    { "text": "Specialized" }
+    { "text": "Trek" }
 
-```javascript
-[
-  {
-    "term": "Something sweet",
-    "priority": 99,
-    "types": "niceness", // A set type for searching by type
-    "data": { // Any key value pair you want, this is the return value
-      "id": 99,
-      "url": "http://example.com",
-      "whatever": "The color purple"
-    }
-  },
-  { "term": "The color blue" }
-]
-```
+It accepts local files:
+
+    soulheart load my_json_file.json
+
+or remote files:
+  
+    soulheart load https://gist.githubusercontent.com/sethherr/96dbc011e508330ceec4/raw/95122b1fc9de85f241cd048f32b94568f54134e0/manufacturers.tsv
+
+
+In addition to term, there are a few optional values - 
 
-*If you set `term` in `data`, it will respond with that rather than the term it searches by. I haven't figured out a use case for this yet, but I'm sure one exists.*
+| Key          | Default     | What it does |
+| ------------ | ----------- | ------------ |
+| `priority`   | `100`       | Higher numbers come first |
+| `category`   | `'default'` | Sets the category |
+| `data`       | `{}`        | Returned object from search - the text and category will be added to this if you don't specify them. |
+
+Here is an example of what a possible hash you could pass is
+
+    { "text": "Jamis", "category": "Bike Manufacturer" }
+    { "text": "Specialized" }
+    { "text": "Trek" }
+
+*If you set `text` in `data`, it will respond with that rather than the term it searches by. I haven't figured out a use case for this yet, but I'm sure one exists.*
 
 ======
 
-I'm testing with:
+I'm testing with: `ruby >= 2.1` and `redis >= 3`. 
 
-  - ruby >= 2.1
-  - redis >= 3
+Run `bundle exec guard` to run the specs while you work, it will just test the files you change.
 
-To test, run `bundle exec guard`. It will live reload the files you change. 
+This repo includes a `config.ru` and a `Gemfile.lock` so it (and any forks of it) can be deployed to heroku. They shouldn't be in the Gem itself.
 
 
 ======
 
+This is an updated fork of [Seatgeek/Soulmate](https://github.com/seatgeek/soulmate) to address a few issues - namely [CORS support](../../issues/2), [minimum entry length](../../issues/3) and [playing better with Selectize & Select2](../../issues/4) - also the future.
+
+Since [Seatgeek no longer uses Soulmate](https://news.ycombinator.com/item?id=9317891), and this isn't backward compatible it's a new project and gem.
+
 :x::o::x::o::x::o: *Soulmate's README follows ([issue for making new documentation](../../issues/1))*
 
 Soulmate is a tool to help solve the common problem of developing a fast autocomplete feature. It uses Redis's sorted sets to build an index of partially completed words and the corresponding top matching items, and provides a simple sinatra app to query them. Soulmate finishes your sentences.

data/Rakefile

@@ -1,29 +1,29 @@
 #!/usr/bin/env rake
 
-require "bundler/gem_helper"
+require 'bundler/gem_helper'
 
 begin
   Bundler.setup(:default, :development)
 rescue Bundler::BundlerError => e
   $stderr.puts e.message
-  $stderr.puts "Run `bundle install` to install missing gems"
+  $stderr.puts 'Run `bundle install` to install missing gems'
   exit e.status_code
 end
 require 'rake'
 
-REDIS_DIR = File.expand_path(File.join("..", "test"), __FILE__)
-REDIS_CNF = File.join(REDIS_DIR, "test.conf")
-REDIS_PID = File.join(REDIS_DIR, "db", "redis.pid")
+REDIS_DIR = File.expand_path(File.join('..', 'test'), __FILE__)
+REDIS_CNF = File.join(REDIS_DIR, 'test.conf')
+REDIS_PID = File.join(REDIS_DIR, 'db', 'redis.pid')
 REDIS_LOCATION = ENV['REDIS_LOCATION']
 
-desc "Run rcov and manage server start/stop"
-task :rcoverage => [:start, :rcov, :stop]
+desc 'Run rcov and manage server start/stop'
+task rcoverage: [:start, :rcov, :stop]
 
-desc "Start the Redis server"
+desc 'Start the Redis server'
 task :start do
   redis_running = \
     begin
-      File.exists?(REDIS_PID) && Process.kill(0, File.read(REDIS_PID).to_i)
+      File.exist?(REDIS_PID) && Process.kill(0, File.read(REDIS_PID).to_i)
     rescue Errno::ESRCH
       FileUtils.rm REDIS_PID
       false
@@ -36,21 +36,20 @@ task :start do
   end
 end
 
-desc "Stop the Redis server"
+desc 'Stop the Redis server'
 task :stop do
-  if File.exists?(REDIS_PID)
-    Process.kill "INT", File.read(REDIS_PID).to_i
+  if File.exist?(REDIS_PID)
+    Process.kill 'INT', File.read(REDIS_PID).to_i
     FileUtils.rm REDIS_PID
   end
 end
 
+require 'rspec/core/rake_task'
 
-require "rspec/core/rake_task"
-
-desc "Run all specs"
+desc 'Run all specs'
 RSpec::Core::RakeTask.new(:spec) do |t|
   t.rspec_opts = %w(--color)
   t.verbose = false
 end
 
-task :default => :spec
+task default: :spec

data/bin/soulheart

@@ -12,95 +12,45 @@ require 'optparse'
 require 'tempfile'
 
 parser = OptionParser.new do |opts|
-  opts.banner = "Usage: soulheart [options] COMMAND"
+  opts.banner = 'Usage: soulheart [options] COMMAND'
 
-  opts.separator ""
-  opts.separator "Options:"
+  opts.separator ''
+  opts.separator 'Options:'
 
-  opts.on("-r", "--redis [HOST:PORT]", "Redis connection string") do |host|
+  opts.on('-r', '--redis [HOST:PORT]', 'Redis connection string') do |host|
     Soulheart.redis = host
   end
 
-  opts.on("-s", "--stop-words [FILE]", "Path to file containing a list of stop words") do |fn|
+  opts.on('-s', '--stop-words [FILE]', 'Path to file containing a list of stop words') do |fn|
     File.open(fn) do |file|
-      Soulheart.stop_words = file.readlines.map{ |l| l.strip }.reject{ |w| w.empty? }
+      Soulheart.stop_words = file.readlines.map(&:strip).reject(&:empty?)
     end
   end
 
-  opts.on("-h", "--help", "Show this message") do
+  opts.on('-h', '--help', 'Show this message') do
     puts opts
     exit
   end
 
-  opts.on("-b", "--batch-size", "Number of lines to read at a time") do |size|
+  opts.on('-b', '--batch-size', 'Number of lines to read at a time') do |size|
     BATCH_SIZE = size
   end
 
-  opts.separator ""
-  opts.separator "Commands:"
-  opts.separator "  load   TYPE FILE  Replaces collection specified by TYPE with items read from FILE in the JSON lines format."
-  opts.separator "  add    TYPE       Adds items to collection specified by TYPE read from stdin in the JSON lines format."
+  opts.separator ''
+  opts.separator 'Commands:'
+  opts.separator '  load   TYPE FILE  Replaces collection specified by TYPE with items read from FILE in the JSON lines format.'
+  opts.separator '  add    TYPE       Adds items to collection specified by TYPE read from stdin in the JSON lines format.'
   opts.separator "  remove TYPE       Removes items from collection specified by TYPE read from stdin in the JSON lines format. Items only require an 'id', all other fields are ignored."
-  opts.separator "  query  TYPE QUERY Queries for items from collection specified by TYPE."
+  opts.separator '  query  TYPE QUERY Queries for items from collection specified by TYPE.'
 end
 
-def generate(type, file)
-  include Soulheart::Helpers
-
-  begin
-    temp = Tempfile.new("soulheart")
-
-    if File.exists?(file)
-      start_time = Time.now.to_i
-      base = "soulheart-index:#{type}"
-      database = "soulheart-data:#{type}"
-      # hset = "*4\r\n$4\r\nHSET\r\n$#{database.length}\r\n#{database}\r\n$"
-      # del = "*2\r\n$3\r\nDEL\r\n$"
-      begin
-        f = File.open(file)
-        # cleanup
-        phrases = Soulheart.redis.smembers(base)
-        phrases.each do |phrase|
-          temp << gen_redis_proto("DEL", phrase)
-          # temp << del + phrase.length.to_s + "\r\n" + phrase + "\r\n"
-        end
-        temp << gen_redis_proto("DEL", base)
-        # temp << del + base.length.to_s + "\r\n" + base + "\r\n"
-        while !f.eof?
-          line = f.gets.chomp
-          line =~ /"id":(\d+)/
-          id = $1
-          line =~ /"score":(\d+)/
-          score = $1
-          json = MultiJson.decode(line)
-          temp << gen_redis_proto("HSET", database, id, line)
-          # temp << hset + $1.length.to_s + "\r\n" + $1 + "\r\n$" + line.length.to_s + "\r\n" + line + "\r\n"
-          phrase = json.key?("aliases") ? json["term"] + " " + json["aliases"] : json["term"]
-          prefixes_for_phrase(phrase).each do |p|
-            temp << gen_redis_proto("SADD", base, p)
-            temp << gen_redis_proto("ZADD", base + ":" + p, score, id)
-          end
-        end
-      ensure
-        f.close
-      end
-      puts "Converted in #{Time.now.to_i - start_time} second(s)"
-      puts "Importing into redis ..."
-      `time redis-cli --pipe < #{temp.path}`
-    else
-      puts "Couldn't open file: #{file}"
-    end
-  ensure
-    temp.close
-  end
-end
 
 def load(file)
   require 'uri'
-  if file =~ URI::regexp
+  if file =~ URI.regexp
     require 'open-uri'
     f = open(file)
-  elsif File.exists?(file)
+  elsif File.exist?(file)
     f = File.open(file)
   else
     puts "Couldn't open file: #{file}"
@@ -113,17 +63,17 @@ def load(file)
   lines = []
   begin
     if file.match(/(c|t)sv\z/i)
-      puts "Reading a CSV"
+      puts 'Reading a CSV'
       require 'csv'
-      sep = file.match(/tsv\z/i) ? "\t" : ","
+      sep = file.match(/tsv\z/i) ? "\t" : ','
       CSV.foreach(f, headers: true, col_sep: sep) do |row|
         lines << row.to_hash
         count += 1
       end
     elsif file.match(/json\z/i)
-      puts "Reading JSON"
+      puts 'Reading JSON'
       puts "Loading items in batches of #{BATCH_SIZE} ..."
-      while !f.eof?
+      until f.eof?
         lines = []
         BATCH_SIZE.times do
           break if f.eof?
@@ -132,7 +82,7 @@ def load(file)
         end
       end
     else
-      puts "unknown File type"
+      puts 'unknown File type'
     end
   ensure
     f.close
@@ -164,7 +114,7 @@ end
 def query(type, query)
   puts "> Querying '#{type}' for '#{query}'"
   matcher = Soulheart::Matcher.new(type)
-  results = matcher.matches_for_term(query, :limit => 0)
+  results = matcher.matches_for_term(query, limit: 0)
   results.each do |item|
     puts MultiJson.encode(item)
   end
@@ -172,10 +122,10 @@ def query(type, query)
 end
 
 def gen_redis_proto(*cmd)
-  proto = "*"+cmd.length.to_s+"\r\n"
+  proto = '*' + cmd.length.to_s + "\r\n"
   cmd.each{|arg|
-    proto << "$"+arg.bytesize.to_s+"\r\n"
-    proto << arg+"\r\n"
+    proto << '$' + arg.bytesize.to_s + "\r\n"
+    proto << arg + "\r\n"
   }
   proto
 end

data/bin/soulheart-web

@@ -6,23 +6,17 @@ begin
 rescue LoadError
   require 'rubygems'
   require 'vegas'
-end  
+end
 require 'soulheart/server'
 
-
-Vegas::Runner.new(Soulheart::Server, 'soulheart-web', {
-  :before_run => lambda {|v|
-    # path = (ENV['RESQUECONFIG'] || v.args.first)
-    # load path.to_s.strip if path
-  }
-}) do |runner, opts, app|
-  opts.on("-r", "--redis [HOST:PORT]", "Redis connection string") do |host|
+Vegas::Runner.new(Soulheart::Server, 'soulheart-web') do |runner, opts|
+  opts.on('-r', '--redis [HOST:PORT]', 'Redis connection string') do |host|
     runner.logger.info "Using Redis connection string '#{host}'"
     Soulheart.redis = host
   end
-  opts.on("-s", "--stop-words [FILE]", "Path to file containing a list of stop words") do |fn|
+  opts.on('-s', '--stop-words [FILE]', 'Path to file containing a list of stop words') do |fn|
     File.open(fn) do |file|
-      Soulheart.stop_words = file.readlines.map{ |l| l.strip }.reject{ |w| w.empty? }
+      Soulheart.stop_words = file.readlines.map(&:strip).reject(&:empty?)
     end
   end
 end

data/lib/soulheart/base.rb

@@ -1,9 +1,7 @@
 module Soulheart
-  
   class Base
-    
     include Helpers
-    
+
     attr_accessor :type
 
     def redis
@@ -13,15 +11,15 @@ module Soulheart
     def cache_length
       10 * 60 # Setting to 10 minutes, but making it possible to edit down the line
     end
-    
+
     def base_id
-      ENV['RACK_ENV'] != 'test' ? "soulheart:" : "soulheart_test:"
+      ENV['RACK_ENV'] != 'test' ? 'soulheart:' : 'soulheart_test:'
     end
 
     def set_category_combos_array
       redis.expire category_combos_id, 0
-      ar = redis.smembers(categories_id).map{ |c| normalize(c) }.uniq.sort
-      ar = 1.upto(ar.size).flat_map {|n| ar.combination(n).map{|el| el.join('')}}
+      ar = redis.smembers(categories_id).map { |c| normalize(c) }.uniq.sort
+      ar = 1.upto(ar.size).flat_map { |n| ar.combination(n).map { |el| el.join('') } }
       ar.last.replace('all')
       redis.sadd category_combos_id, ar
       ar
@@ -31,15 +29,19 @@ module Soulheart
       "#{base_id}category_combos:"
     end
 
+    def category_combos
+      redis.smembers(category_combos_id)
+    end
+
     def categories_id
       "#{base_id}categories:"
     end
 
-    def category_id(name='all')
+    def category_id(name = 'all')
       "#{categories_id}#{name}:"
     end
 
-    def no_query_id(category=category_id)
+    def no_query_id(category = category_id)
       "all:#{category}"
     end
 
@@ -47,8 +49,8 @@ module Soulheart
       "#{base_id}database:"
     end
 
-    def cache_id(type='all')
+    def cache_id(type = 'all')
       "#{base_id}cache:#{type}:"
     end
   end
-end
+end

data/lib/soulheart/config.rb

@@ -3,7 +3,7 @@ require 'redis'
 
 module Soulheart
   module Config
-    DEFAULT_STOP_WORDS = ["vs", "at", "the"]
+    DEFAULT_STOP_WORDS = %w(vs at the)
 
     # Accepts:
     #   1. A Redis URL String 'redis://host:port/db'
@@ -23,14 +23,12 @@ module Soulheart
     # create a new one.
     def redis
       @redis ||= (
-        url = URI(@redis_url || ENV["REDIS_URL"] || "redis://127.0.0.1:6379/0")
-        ::Redis.new({
-          # driver: :hiredis,
+        url = URI(@redis_url || ENV['REDIS_URL'] || 'redis://127.0.0.1:6379/0')
+        ::Redis.new(          # driver: :hiredis,
           host: url.host,
           port: url.port,
           db: url.path[1..-1],
-          password: url.password
-        })
+          password: url.password)
       )
     end
 

data/lib/soulheart/helpers.rb

@@ -2,21 +2,17 @@
 
 module Soulheart
   module Helpers
-    def blank?
-      respond_to?(:empty?) ? !!empty? : !self
-    end
-
     def normalize(str)
       # Letter, Mark, Number, Connector_Punctuation (Chinese, Japanese, etc.)
       str.downcase.gsub(/[^\p{Word}\ ]/i, '').strip
     end
-    
+
     def prefixes_for_phrase(phrase)
       words = normalize(phrase).split(' ').reject do |w|
         Soulheart.stop_words.include?(w)
       end
       words.map do |w|
-        (0..(w.length-1)).map{ |l| w[0..l] }
+        (0..(w.length - 1)).map { |l| w[0..l] }
       end.flatten.uniq
     end
   end

data/lib/soulheart/loader.rb

@@ -1,7 +1,5 @@
 module Soulheart
-
   class Loader < Base
-
     def default_items_hash(text, category)
       category ||= 'default'
       {
@@ -12,7 +10,7 @@ module Soulheart
         'data' => {
           'text' => text,
           'category' => category
-        },
+        }
       }
     end
 
@@ -57,31 +55,36 @@ module Soulheart
     end
 
     def clean(item)
-      raise ArgumentError, "Items must have text" unless item["text"]
-      default_items_hash(item.delete('text'), item.delete('category')).
-        tap{ |i| i['data'].merge!(item.delete('data')) if item['data'] }.
-        tap{ |i| i['priority'] = item.delete('priority').to_f if item['priority'] }.
-        merge item
+      fail ArgumentError, 'Items must have text' unless item['text']
+      default_items_hash(item.delete('text'), item.delete('category'))
+        .tap { |i| i['data'].merge!(item.delete('data')) if item['data'] }
+        .tap { |i| i['priority'] = item.delete('priority').to_f if item['priority'] }
+        .merge item
     end
 
-    def add_item(item, category_base_id=nil, cleaned: false)
+    def add_item(item, category_base_id = nil, cleaned: false)
       unless cleaned
         item = clean(item)
         category_base_id ||= category_id(item['category'])
+        item.keys.select{ |k| k[/data-/i] }.each do |key|
+          item['data'].merge!({
+              "#{key.gsub(/data-/i,'')}" => item.delete(key)
+            })
+        end
         unless redis.smembers(categories_id).include?(item['category'])
           redis.sadd categories_id, item['category']
         end
       end
       redis.pipelined do
-        redis.zadd(no_query_id(category_base_id), item["priority"], item["term"]) # Add to master set for queryless searches
+        redis.zadd(no_query_id(category_base_id), item['priority'], item['term']) # Add to master set for queryless searches
         # store the raw data in a separate key to reduce memory usage, if it's cleaned it's done
         redis.hset(results_hashes_id, item['term'], MultiJson.encode(item['data'])) unless cleaned
-        phrase = ([item["term"]] + (item["aliases"] || [])).join(' ')
+        phrase = ([item['term']] + (item['aliases'] || [])).join(' ')
         # Store all the prefixes
         prefixes_for_phrase(phrase).each do |p|
           redis.sadd(base_id, p) unless cleaned # remember prefix in a master set
           # store the normalized term in the index for each of the categories
-          redis.zadd("#{category_base_id}#{p}", item["priority"], item["term"])
+          redis.zadd("#{category_base_id}#{p}", item['priority'], item['term'])
         end
       end
       item
@@ -89,19 +92,19 @@ module Soulheart
 
     # remove only cares about an item's id, but for consistency takes an object
     def remove(item)
-      prev_item = Soulheart.redis.hget(base_id, item["term"])
+      prev_item = Soulheart.redis.hget(base_id, item['term'])
       if prev_item
         prev_item = MultiJson.decode(prev_item)
         # undo the operations done in add
         Soulheart.redis.pipelined do
-          Soulheart.redis.hdel(base_id, prev_item["term"])
-          phrase = ([prev_item["term"]] + (prev_item["aliases"] || [])).join(' ')
+          Soulheart.redis.hdel(base_id, prev_item['term'])
+          phrase = ([prev_item['term']] + (prev_item['aliases'] || [])).join(' ')
           prefixes_for_phrase(phrase).each do |p|
             Soulheart.redis.srem(base_id, p)
-            Soulheart.redis.zrem("#{base_id}:#{p}", prev_item["term"])
+            Soulheart.redis.zrem("#{base_id}:#{p}", prev_item['term'])
           end
         end
       end
     end
   end
-end
+end

data/lib/soulheart/matcher.rb

@@ -1,28 +1,29 @@
 module Soulheart
-
   class Matcher < Base
-    def initialize(params={})
+    def initialize(params = {})
       @opts = self.class.default_params_hash.merge params
       clean_opts
     end
 
+    attr_accessor :opts
+
     def self.default_params_hash
       {
         'page' => 1,
         'per_page' => 5,
         'categories' => [],
-        'query' => '',
+        'q' => '', # Query
         'cache' => true
       }
     end
 
     def clean_opts
       unless @opts['categories'] == '' || @opts['categories'] == []
-        @opts['categories'] = @opts['categories'].split(/,|\+/) unless @opts['categories'].kind_of?(Array)
-        @opts['categories'] = @opts['categories'].map{ |s| normalize(s) }.uniq.sort
+        @opts['categories'] = @opts['categories'].split(/,|\+/) unless @opts['categories'].is_a?(Array)
+        @opts['categories'] = @opts['categories'].map { |s| normalize(s) }.uniq.sort
         @opts['categories'] = [] if @opts['categories'].length == redis.scard(categories_id)
       end
-      @opts['query'] = normalize(@opts['query']).split(' ')
+      @opts['q'] = normalize(@opts['q']).split(' ') unless @opts['q'].is_a?(Array)
       # .reject{ |i| i && i.length > 0 } .split(' ').reject{  Soulmate.stop_words.include?(w) }
       @opts
     end
@@ -36,12 +37,12 @@ module Soulheart
     end
 
     def cache_id_from_opts
-      "#{cache_id(categories_string)}#{@opts['query'].join(':')}"
+      "#{cache_id(categories_string)}#{@opts['q'].join(':')}"
     end
 
     def interkeys_from_opts(cid)
       # If there isn't a query, we use a special key in redis
-      @opts['query'].empty? ? [no_query_id(cid)] : @opts['query'].map { |w| "#{cid}#{w}" }
+      @opts['q'].empty? ? [no_query_id(cid)] : @opts['q'].map { |w| "#{cid}#{w}" }
     end
 
     def matches
@@ -55,17 +56,16 @@ module Soulheart
       end
       offset = (@opts['page'].to_i - 1) * @opts['per_page'].to_i
       limit = @opts['per_page'].to_i + offset - 1
-      
+
       limit = 0 if limit < 0
       ids = redis.zrevrange(cachekey, offset, limit) # Using 'ids', even though keys are now terms
       if ids.size > 0
         results = redis.hmget(results_hashes_id, *ids)
-        results = results.reject{ |r| r.nil? } # handle cached results for ids which have since been deleted
+        results = results.reject(&:nil?) # handle cached results for ids which have since been deleted
         results.map { |r| MultiJson.decode(r) }
       else
         []
-      end      
+      end
     end
-
   end
-end
+end

data/lib/soulheart/server.rb

@@ -1,33 +1,30 @@
 require 'sinatra/base'
 require 'soulheart'
-require 'rack/contrib'
 
 module Soulheart
-
   class Server < Sinatra::Base
     include Helpers
-    
+
     before do
-      content_type 'application/json', :charset => 'utf-8'
+      content_type 'application/json', charset: 'utf-8'
       headers['Access-Control-Allow-Origin'] = '*'
       headers['Access-Control-Allow-Methods'] = 'POST, PUT, GET, OPTIONS'
       headers['Access-Control-Request-Method'] = '*'
       headers['Access-Control-Allow-Headers'] = 'Origin, X-Requested-With, Content-Type, Accept, Authorization'
     end
-        
+
     get '/' do
       matches = Matcher.new(params).matches
-      MultiJson.encode({ matches:  matches })
+      MultiJson.encode(matches:  matches)
     end
 
-    get '/status' do 
-      MultiJson.encode({ soulheart: Soulheart::VERSION, :status   => "ok" })
+    get '/status' do
+      MultiJson.encode(soulheart: Soulheart::VERSION, status: 'ok')
     end
-    
+
     not_found do
-      content_type 'application/json', :charset => 'utf-8'
-      MultiJson.encode({ :error => "not found" })
+      content_type 'application/json', charset: 'utf-8'
+      MultiJson.encode(error: 'not found')
     end
-    
   end
 end

data/lib/soulheart/version.rb

@@ -1,3 +1,3 @@
 module Soulheart
-  VERSION = "0.0.2"
-end
+  VERSION = '0.0.4'
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: soulheart
 version: !ruby/object:Gem::Version
-  version: 0.0.2
+  version: 0.0.4
 platform: ruby
 authors:
 - Seth Herr
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-06-17 00:00:00.000000000 Z
+date: 2015-06-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: hiredis
@@ -80,20 +80,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-- !ruby/object:Gem::Dependency
-  name: rack-contrib
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -145,13 +131,8 @@ executables:
 extensions: []
 extra_rdoc_files: []
 files:
-- ".gitignore"
-- ".rspec"
-- ".travis.yml"
-- Gemfile
-- Guardfile
 - LICENSE.md
-- README.markdown
+- README.md
 - Rakefile
 - bin/soulheart
 - bin/soulheart-web
@@ -163,14 +144,6 @@ files:
 - lib/soulheart/matcher.rb
 - lib/soulheart/server.rb
 - lib/soulheart/version.rb
-- logo.png
-- soulheart.gemspec
-- spec/fixtures/multiple_categories.json
-- spec/soulheart/loader_spec.rb
-- spec/soulheart/matcher_spec.rb
-- spec/soulheart/server_spec.rb
-- spec/soulheart_spec.rb
-- spec/spec_helper.rb
 homepage: https://github.com/sethherr/soulheart
 licenses:
 - MIT

data/.gitignore

@@ -1,21 +0,0 @@
-# rdoc generated
-rdoc
-
-# yard generated
-doc
-.yardoc
-
-# bundler
-.bundle
-
-gem 
-
-test/db/*.rdb
-
-dump.rdb
-tmp/
-
-Gemfile.lock
-
-.rvmrc
-.ruby-*

data/.rspec

@@ -1,2 +0,0 @@
---color
---format documentation

data/.travis.yml

@@ -1,12 +0,0 @@
-language: ruby
-rvm:
-- 2.1.0
-bundler_args: "--without=guard"
-notifications:
-  disabled: true
-script:
-- bundle exec rake spec
-# - bundle exec rubocop
-addons:
-  code_climate:
-    repo_token: e72d8fa152922cef05c311cd49d3db9016b82486b712399a8e7c7da2af5e071e

data/Gemfile

@@ -1,22 +0,0 @@
-source 'http://rubygems.org'
-
-group :development do
-  gem 'rspec',      '~> 2.14.1'
-  gem 'bundler'
-  gem 'guard'
-  gem 'guard-rspec', '~> 4.2.8'
-  gem 'rack-contrib'
-  gem 'rubocop'
-end
-
-group :test do 
-  gem 'rack-test'
-  gem "codeclimate-test-reporter", require: nil
-end
-
-gem 'rake'
-gem 'redis',      '>= 3.0.1'
-gem 'hiredis',    '~> 0.4.5'
-gem 'vegas',      '>= 0.1.0'
-gem 'sinatra'
-gem 'multi_json', '>= 1.11.0'

data/Guardfile

@@ -1,9 +0,0 @@
-rspec_opts = {
-  failed_mode: :focus
-}
-
-guard :rspec, cmd: "bundle exec rspec" do
-  watch(%r{^spec/.+_spec\.rb$})
-  watch(%r{^lib/(.+)\.rb$})     { |m| "spec/#{m[1]}_spec.rb" }
-  watch(%r{^lib/soulheart/(.+)\.rb$})     { |m| "spec/soulheart/#{m[1]}_spec.rb" }
-end

data/soulheart.gemspec

@@ -1,24 +0,0 @@
-# -*- encoding: utf-8 -*-
-require File.expand_path('../lib/soulheart/version', __FILE__)
-
-Gem::Specification.new do |gem|
-  gem.version       = Soulheart::VERSION
-  gem.authors       = ["Seth Herr"]
-  gem.email         = ["seth.william.herr@gmail.com"]
-  gem.description   = gem.summary = "Simple, fast autocomplete server for Ruby and Rails"
-  gem.homepage      = "https://github.com/sethherr/soulheart"
-  gem.license       = "MIT"
-  gem.executables   = ["soulheart", "soulheart-web"]
-  gem.files         = `git ls-files | grep -Ev '^(test)'`.split("\n")
-  gem.name          = "soulheart"
-  gem.require_paths = ["lib"]
-  gem.add_dependency                  'hiredis', '~> 0.4.5'
-  gem.add_dependency                  'redis', '>= 3.0.6'
-  gem.add_dependency                  'vegas', '>= 0.1.0'
-  gem.add_dependency                  'json'
-  gem.add_dependency                  'sinatra'
-  gem.add_development_dependency      'rack-contrib'
-  gem.add_development_dependency      'rake'
-  gem.add_development_dependency      'rspec'
-  gem.add_development_dependency      'rubocop'
-end

data/spec/fixtures/multiple_categories.json

@@ -1,15 +0,0 @@
-{"text":"Steel","category":"Frame Materials" }
-{"text":"Brompton Bicycle","priority":51,"category":"Frame Manufacturer","data":{"id":8,"url":"http://www.brompton.com"}}
-{"text":"Jamis","priority":75,"category":"Frame Manufacturer","data":{"id":2222,"url":"http://jamisbikes.com"}}
-{"text":"Surly","priority":102,"category":"Frame Manufacturer"}
-{"text":"Jagwire","priority":40,"category":"Manufacturer"}
-{"text":"Jannd","priority":41,"category":"Manufacturer"}
-{"text":"Sram","priority":50,"category":"Manufacturer","data":{"id":8,"url":"http://sram.com"}}
-{"text":"Brooks England LTD.","priority":50,"category":"Manufacturer","data":{"id":200,"url":"http://www.brooksengland.com/"}}
-{"text":"Dodger Stadium","priority":84,"data":{"id":1,"url":"\/dodger-stadium-tickets\/","subtitle":"Los Angeles, CA"},"aliases":["Chavez Ravine"]}
-{"text":"Angel Stadium","priority":90,"data":{"id":28,"url":"\/angel-stadium-tickets\/","subtitle":"Anaheim, CA"},"aliases":["Edison International Field of Anaheim"]}
-{"text":"Chase Field ","priority":80,"data":{"id":30,"url":"\/chase-field-tickets\/","subtitle":"Phoenix, AZ"},"aliases":["Bank One Ballpark", "Bank One Stadium"]}
-{"text":"Sun Life Stadium","priority":75,"data":{"id":29,"url":"\/sun-life-stadium-tickets\/","subtitle":"Miami, FL"},"aliases":["Dolphins Stadium","Land Shark Stadium"]}
-{"text":"Turner Field","priority":50,"data":{"id":2,"url":"\/turner-field-tickets\/","subtitle":"Atlanta, GA"}}
-{"text":"Citi Field","priority":92,"data":{"id":3,"url":"\/citi-field-tickets\/","subtitle":"Atlanta, GA"},"aliases":["Shea Stadium"]}
-{"text":"中国佛山 李小龙","priority":94,"data":{"id":8,"url":"\/Bruce Lee\/","subtitle":"Chinese Foshan"},"aliases":["Li XiaoLong"]}

data/spec/soulheart/loader_spec.rb

@@ -1,96 +0,0 @@
-require 'spec_helper'
-
-describe Soulheart::Loader do
-
-  describe :clean_data do 
-    it "sets the default category, priority and normalizes term" do 
-      item = { 'text' => '  FooBar' }
-      result = Soulheart::Loader.new.clean(item)
-      expect(result['priority']).to eq(100)
-      expect(result['term']).to eq('foobar')
-      expect(result['category']).to eq('default')
-      expect(result['data']['text']).to eq('  FooBar')
-    end
-    
-    it "doesn't overwrite the submitted params" do 
-      item = {
-        'text' => 'Cool ',
-        'priority' => '50',
-        'category' => 'Gooble',
-        'data' => {
-          'id' => 199
-        }
-      }
-      result = Soulheart::Loader.new.clean(item)
-      expect(result['term']).to eq('cool')
-      expect(result['priority']).to eq(50)
-      expect(result['data']['text']).to eq('Cool ')
-      expect(result['data']['id']).to eq(199)
-      expect(result['category']).to eq('gooble')
-      expect(result['data']['category']).to eq('Gooble')
-    end
-
-    it "raises argument error if text is passed" do 
-      expect{
-        Soulheart::Loader.new.clean({'name' => 'stuff'})
-      }.to raise_error(/must have/i)
-    end
-  end
-
-  describe :add_item do 
-    it "adds an item, adds prefix scopes, adds category" do 
-      item = {
-        'text' => 'Brompton Bicycle',
-        'priority' => 50,
-        'category' => 'Gooble',
-        'data' => {
-          'id' => 199
-        }
-      }
-      loader = Soulheart::Loader.new
-      redis = loader.redis
-      redis.expire loader.results_hashes_id, 0
-      loader.add_item(item)
-      redis = loader.redis 
-      target = "{\"text\":\"Brompton Bicycle\",\"category\":\"Gooble\",\"id\":199}"
-      result = redis.hget(loader.results_hashes_id, 'brompton bicycle')
-      expect(result).to eq(target)
-      prefixed = redis.zrevrange "#{loader.category_id('gooble')}:brom", 0, -1
-      expect(prefixed[0]).to eq('brompton bicycle')
-      expect(redis.smembers(loader.categories_id).include?('gooble')).to be_true
-    end
-  end
-
-  describe :store_terms do 
-    it "stores terms by priority and adds categories for each possible category combination" do
-      items = []
-      file = File.read('spec/fixtures/multiple_categories.json')
-      file.each_line { |l| items << MultiJson.decode(l) }
-      loader = Soulheart::Loader.new
-      redis = loader.redis
-      loader.delete_categories
-      loader.load(items)
-      
-      cat_prefixed = redis.zrevrange "#{loader.category_id('frame manufacturermanufacturer')}:brom", 0, -1
-      expect(cat_prefixed.count).to eq(1)
-      expect(redis.smembers(loader.categories_id).count).to be > 3
-
-      prefixed = redis.zrevrange "#{loader.category_id('all')}:bro", 0, -1
-      expect(prefixed.count).to eq(2)
-      expect(prefixed[0]).to eq('brompton bicycle')
-    end
-
-    it "stores terms by priority and doesn't add run categories if none are present" do
-      items = [
-        {'text' => 'cool thing', 'category' => 'AWESOME'},
-        {'text' => 'Sweet', 'category' => ' awesome'}
-      ]
-      loader = Soulheart::Loader.new
-      redis = loader.redis
-      loader.delete_categories
-      loader.load(items)
-      expect(redis.smembers(loader.category_combos_id).count).to eq(1)
-    end
-  end
-
-end

data/spec/soulheart/matcher_spec.rb

@@ -1,104 +0,0 @@
-require 'spec_helper'
-
-describe Soulheart::Matcher do
-
-  describe :clean_opts do 
-    it "Has the keys we need" do 
-      target_keys = ['categories', 'query', 'page', 'per_page']
-      keys = Soulheart::Matcher.default_params_hash.keys
-      expect((target_keys - keys).count).to eq(0)
-    end
-    
-    it "obeys stop words"
-  end
-
-  describe :category_id_from_opts do 
-    it 'gets the id for one' do
-      Soulheart::Loader.new.reset_categories(['cool', 'test'])
-      matcher = Soulheart::Matcher.new('categories' => ['some_category'])
-      expect(matcher.category_id_from_opts).to eq(matcher.category_id('some_category'))
-    end
-
-    it 'gets the id for all of them' do 
-      Soulheart::Loader.new.reset_categories(['cool', 'test', 'boo'])
-      matcher = Soulheart::Matcher.new('categories' => 'cool, boo, test')
-      expect(matcher.category_id_from_opts).to eq(matcher.category_id('all'))
-    end
-  end
-
-  describe :categories_string do 
-    it 'does all if none' do 
-      Soulheart::Loader.new.reset_categories(['cool', 'test'])
-      matcher = Soulheart::Matcher.new('categories' => '')
-      expect(matcher.categories_string).to eq('all')
-    end
-    it "correctly concats a string of categories" do
-      Soulheart::Loader.new.reset_categories(['cool', 'some_category', 'another cat', 'z9', 'stuff'])
-      matcher = Soulheart::Matcher.new({'categories' => 'some_category, another cat, z9'})
-      expect(matcher.categories_string).to eq('another catsome_categoryz9')
-    end
-  end
-
-
-  describe :matches do 
-    it "With no params, gets all the matches, ordered by priority and name" do
-      store_terms_fixture
-      opts = { 'per_page' => 100, 'cache' => false }
-      matches = Soulheart::Matcher.new(opts).matches
-      expect(matches.count).to be > 10
-      expect(matches[0]['text']).to eq('Surly')
-    end
-
-    it "With no query but with categories, matches categories" do
-      store_terms_fixture
-      opts = { 'per_page' => 100, 'cache' => false, 'categories' => 'manufacturer' }
-      matches = Soulheart::Matcher.new(opts).matches
-      expect(matches.count).to eq(4)
-      expect(matches[0]['text']).to eq('Sram')
-    end
-
-    it "Gets the matches matching query and priority for one item in query, all categories" do
-      store_terms_fixture
-      opts = { 'per_page' => 100, 'query' => 'j', 'cache' => false }
-      matches = Soulheart::Matcher.new(opts).matches
-      expect(matches.count).to eq(3)
-      expect(matches[0]['text']).to eq('Jamis')
-    end
-
-    it "Gets the matches matching query and priority for one item in query, one category" do
-      store_terms_fixture
-      opts = { 'per_page' => 100, 'query' => 'j', 'cache' => false, 'categories' => 'manufacturer' }
-      matches = Soulheart::Matcher.new(opts).matches
-      expect(matches.count).to eq(2)
-      expect(matches[0]['text']).to eq('Jannd')
-    end
-
-    it "Gets pages and uses them" do
-      # Pagination wrecked my mind, hence the multitude of tests
-      items = [
-        {"text" => 'First item', 'priority' => '11000' },
-        {"text" => 'Second item', 'priority' => '1999' },
-        {"text" => 'Third item', 'priority' => 1900 },
-        {"text" => 'Fourth item', 'priority' => 1800 },
-        {"text" => 'Fifth item', 'priority' => 1750 },
-        {"text" => 'Sixth item', 'priority' => 1700 },
-        {"text" => 'Seventh item', 'priority' => 1699 }
-      ]
-      loader = Soulheart::Loader.new
-      loader.delete_categories
-      loader.load(items)
-
-      page1 = Soulheart::Matcher.new({'per_page' => 1, 'cache' => false}).matches
-      expect(page1[0]['text']).to eq('First item')
-      
-      page2 = Soulheart::Matcher.new({'per_page' => 1, 'page' => 2, 'cache' => false}).matches
-      expect(page2.count).to eq(1)
-      expect(page2[0]['text']).to eq('Second item')
-
-      page3 = Soulheart::Matcher.new({'per_page' => 2, 'page' => 3, 'cache' => false}).matches
-      expect(page3[0]['text']).to eq('Fifth item')
-      expect(page3[1]['text']).to eq('Sixth item')
-    end
-  end
-
-end

data/spec/soulheart/server_spec.rb

@@ -1,26 +0,0 @@
-require 'spec_helper'
-
-describe Soulheart::Server do
-
-  describe :search do 
-    it "Has CORS headers, JSON Content-Type and it succeeds" do 
-      get '/'
-      expect(last_response.headers['Access-Control-Allow-Origin']).to eq('*')
-      expect(last_response.headers['Access-Control-Request-Method']).to eq('*')
-      expect(last_response.headers['Content-Type']).to match('json')
-      expect(last_response.status).to eq(200)
-    end
-  end
-
-  describe :status do 
-    it "Has cors headers and is valid JSON" do 
-      get '/status'
-      expect(last_response.headers['Access-Control-Allow-Origin']).to eq('*')
-      expect(last_response.headers['Access-Control-Request-Method']).to eq('*')
-      expect(last_response.headers['Content-Type']).to match('json')
-      expect(JSON.parse(last_response.body)['soulheart']).to match(/\d/)
-    end
-  end
-
-
-end

data/spec/soulheart_spec.rb

@@ -1,29 +0,0 @@
-require 'spec_helper'
-
-describe Soulheart do
-
-  it 'has a version number' do
-    expect(Soulheart::VERSION).not_to be nil
-  end
-
-  it "has a test base_id" do 
-    expect(Soulheart::Base.new.base_id).to eq('soulheart_test:')
-  end
-
-  it "collates (? probably not the word I'm looking for) all the things" do 
-    base = Soulheart::Base.new 
-    base.redis.expire base.categories_id, 0
-    base.redis.sadd base.categories_id, ['George', 'category one', 'other thing ']
-    result = base.set_category_combos_array
-    expect(result.include?("category one")).to be_true
-    expect(result.include?("george")).to be_true
-    expect(result.include?("other thing")).to be_true
-    expect(result.include?("georgeother thing")).to be_true
-    expect(result.include?("category oneother thing")).to be_true
-    expect(result.include?("category onegeorge")).to be_true
-    expect(result.include?("georgecategory one")).to be_false
-    expect(result.include?("all")).to be_true
-    expect(result.include?("category onegeorgeother thing")).to be_false
-  end
-
-end

data/spec/spec_helper.rb

@@ -1,31 +0,0 @@
-if ENV['CODECLIMATE_REPO_TOKEN']
-  require "codeclimate-test-reporter"
-  CodeClimate::TestReporter.start
-end
-require 'rack/test'
-require 'rspec'
-
-$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
-require 'soulheart'
-require 'soulheart/server'
-
-ENV['RACK_ENV'] = 'test'
-
-module RSpecMixin
-  include Rack::Test::Methods
-  def app() Soulheart::Server end
-end
-
-RSpec.configure do |config|
-  config.treat_symbols_as_metadata_keys_with_true_values = true
-  config.include RSpecMixin
-end
-
-def store_terms_fixture
-  items = []
-  file = File.read('spec/fixtures/multiple_categories.json')
-  file.each_line { |l| items << MultiJson.decode(l) }
-  loader = Soulheart::Loader.new
-  loader.delete_categories
-  loader.load(items)
-end