searchkick 0.0.1 → 0.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 93eb2e14b7cab538e0e64e16f8b20bc1082e1d0f
-  data.tar.gz: cb44a620ea08419c58a0e773de66cf4761dee785
+  metadata.gz: f37d2d366a7f03a6e2757c688385aed25678836c
+  data.tar.gz: e4648bd3692a88ccd7d9669376f8fa89af8e7431
 SHA512:
-  metadata.gz: 27d1bd7b105e33c8b72b145413441438edfcff5cfb3f1b1f08c2f3e0390ed04f8164b6da94caf38f36ea1cec7d8ac72d97c8f3b4684c2299cea37563a5f507e4
-  data.tar.gz: 75b3eadfd5b1e7dbb48ea4f94bb19bac382768f85ace0f8c367e5c926756d2c3925f4dfd7d19c27c5a65c1f0197559029e5e40d759ca32c9764ad0636de0d888
+  metadata.gz: 071e83eab034feb8784f32791728fee4cb61e3670ef9f36f97ad66961e7c12c299214e44bde1c5aec45f56b92a5d639591e2f9845a3fd2a1c8aee9e3fb14c7a9
+  data.tar.gz: e42658adc41be4c3e2232bfa510f9e2026da8c6a1303b192e5f03692d9c0d15068e3d92252955bba136bea21bfe830674ae84251e7134dc280f642d7b7f9bea6

data/.gitignore

@@ -15,3 +15,4 @@ spec/reports
 test/tmp
 test/version_tmp
 tmp
+*.log

data/README.md

@@ -1,36 +1,175 @@
 # Searchkick
 
-Search made easy
+:rocket: Search made easy
+
+Searchkick provides sensible search defaults out of the box.  It handles:
+
+- stemming - `tomatoes` matches `tomato`
+- special characters - `jalapenos` matches `jalapeños`
+- extra whitespace - `dishwasher` matches `dish washer`
+- misspellings - `zuchini` matches `zucchini`
+- custom synonyms - `qtip` matches `cotton swab`
+
+Runs on Elasticsearch
+
+:tangerine: Battle-tested at [Instacart](https://www.instacart.com)
 
 ## Usage
 
-### Reindex with Zero Downtime
+```ruby
+class Product < ActiveRecord::Base
+  searchkick
+end
+```
 
-Elasticsearch has a feature called aliases that allows you to reindex with no downtime.
+And to query, use:
 
 ```ruby
-Book.tire.reindex
+Product.search "2% Milk"
 ```
 
-This creates a new index `books_20130714181054` and points the `books` alias to the new index when complete - an atomic operation :)
+or only search specific fields:
 
-**First time:** If books is an existing index, it will be replaced by an alias.
+```ruby
+Product.search "Butter", fields: [:name, :brand]
+```
 
-Searchkick uses `find_in_batches` to import documents.  To filter documents or eagar load associations, use the `tire_import` scope.
+### Query Like SQL
 
 ```ruby
-class Book < ActiveRecord::Base
-  scope :tire_import, where(active: true).includes(:author, :chapters)
+Product.search "2% Milk", where: {in_stock: true}, limit: 10, offset: 50
+```
+
+#### Where
+
+```ruby
+where: {
+  expires_at: {gt: Time.now}, # lt, gte, lte also available
+  orders_count: 1..10,        # equivalent to {gte: 1, lte: 10}
+  aisle_id: [25, 30],         # in
+  store_id: {not: 2},         # not
+  aisle_id: {not: [25, 30]},  # not in
+  or: [
+    [{in_stock: true}, {backordered: true}]
+  ]
+}
+```
+
+#### Order
+
+```ruby
+order: {_score: :desc} # most relevant first - default
+```
+
+#### Explain
+
+```ruby
+explain: true
+```
+
+### Facets
+
+```ruby
+Product.search "2% Milk", facets: [:store_id, :aisle_id]
+```
+
+Advanced
+
+```ruby
+Product.search "2% Milk", facets: {store_id: {where: {in_stock: true}}}
+```
+
+### Synonyms
+
+```ruby
+class Product < ActiveRecord::Base
+  searchkick synonyms: [["scallion", "green onion"], ["qtip", "cotton swab"]]
+end
+```
+
+You must call `Product.reindex` after changing synonyms.
+
+### Make Searches Better Over Time
+
+Improve results with analytics on conversions and give popular documents a little boost.
+
+First, you must keep track of search conversions.  The database works well for low volume, but feel free to use redis or another datastore.
+
+```ruby
+class Search < ActiveRecord::Base
+  belongs_to :product
+  # fields: id, query, searched_at, converted_at, product_id
+end
+```
+
+Add the conversions to the index.
+
+```ruby
+class Product < ActiveRecord::Base
+  has_many :searches
+
+  searchkick conversions: true
+
+  def to_indexed_json
+    {
+      name: name,
+      conversions: searches.group("query").count.map{|query, count| {query: query, count: count} }, # TODO fix
+      _boost: Math.log(orders_count) # boost more popular products a bit
+    }
+  end
+end
+```
+
+After the reindex is complete (to prevent errors), tell the search method to use conversions.
+
+```ruby
+Product.search "Fat Free Milk", conversions: true
+```
+
+### Zero Downtime Changes
+
+```ruby
+Product.reindex
+```
+
+Behind the scenes, this creates a new index `products_20130714181054` and points the `products` alias to the new index when complete - an atomic operation :)
+
+Searchkick uses `find_in_batches` to import documents.  To filter documents or eagar load associations, use the `searchkick_import` scope.
+
+```ruby
+class Product < ActiveRecord::Base
+  scope :searchkick_import, where(active: true).includes(:searches)
 end
 ```
 
 There is also a rake task.
 
 ```sh
-rake searchkick:reindex CLASS=Book
+rake searchkick:reindex CLASS=Product
+```
+
+Thanks to Jaroslav Kalistsuk for the [original implementation](https://gist.github.com/jarosan/3124884).
+
+### Reference
+
+Reindex one item
+
+```ruby
+product = Product.find(1)
+product.update_index
 ```
 
-[Thanks to Jaroslav Kalistsuk for the original source](https://gist.github.com/jarosan/3124884)
+Partial matches (needs better name)
+
+```ruby
+Item.search "fresh honey", partial: true # matches organic honey
+```
+
+## Elasticsearch Gotchas
+
+### Inconsistent Scores
+
+Due to the distributed nature of Elasticsearch, you can get incorrect results when the number of documents in the index is low.  You can [read more about it here](http://www.elasticsearch.org/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch/).  To fix this, set the search type to `dfs_query_and_fetch`.  Alternatively, you can just use one shard with `settings: {number_of_shards: 1}`.
 
 ## Installation
 
@@ -46,6 +185,19 @@ And then execute:
 bundle
 ```
 
+## TODO
+
+- Autocomplete
+- Option to turn off fuzzy matching (should this be default?)
+- Exact phrase matches (in order)
+- Focus on results format (load: true?)
+- Test helpers - everyone should test their own search
+- Built-in synonyms from WordNet
+- Dashboard w/ real-time analytics?
+- [Suggest API](http://www.elasticsearch.org/guide/reference/api/search/suggest/) "Did you mean?"
+- Allow for "exact search" with quotes
+- Make updates to old and new index while reindexing [possibly with an another alias](http://www.kickstarter.com/backing-and-hacking)
+
 ## Contributing
 
 1. Fork it

data/Rakefile

@@ -1 +1,8 @@
 require "bundler/gem_tasks"
+require "rake/testtask"
+
+task :default => :test
+Rake::TestTask.new do |t|
+  t.libs << "test"
+  t.pattern = "test/**/*_test.rb"
+end

data/lib/searchkick.rb

@@ -1,49 +1,9 @@
 require "searchkick/version"
+require "searchkick/reindex"
+require "searchkick/search"
+require "searchkick/model"
 require "searchkick/tasks"
 require "tire"
+require "active_record" # TODO only require active_model
 
-module Searchkick
-  module ClassMethods
-
-    # https://gist.github.com/jarosan/3124884
-    def reindex
-      alias_name = klass.tire.index.name
-      new_index = alias_name + "_" + Time.now.strftime("%Y%m%d%H%M%S")
-
-      # Rake::Task["tire:import"].invoke
-      index = Tire::Index.new(new_index)
-      Tire::Tasks::Import.create_index(index, klass)
-      scope = klass.respond_to?(:tire_import) ? klass.tire_import : klass
-      scope.find_in_batches do |batch|
-        index.import batch
-      end
-
-      if a = Tire::Alias.find(alias_name)
-        puts "[IMPORT] Alias found: #{Tire::Alias.find(alias_name).indices.to_ary.join(",")}"
-        old_indices = Tire::Alias.find(alias_name).indices
-        old_indices.each do |index|
-          a.indices.delete index
-        end
-
-        a.indices.add new_index
-        a.save
-
-        old_indices.each do |index|
-          puts "[IMPORT] Deleting index: #{index}"
-          i = Tire::Index.new(index)
-          i.delete if i.exists?
-        end
-      else
-        puts "[IMPORT] No alias found. Deleting index, creating new one, and setting up alias"
-        i = Tire::Index.new(alias_name)
-        i.delete if i.exists?
-        Tire::Alias.create(name: alias_name, indices: [new_index])
-      end
-
-      puts "[IMPORT] Saved alias #{alias_name} pointing to #{new_index}"
-    end
-
-  end
-end
-
-Tire::Model::Search::ClassMethodsProxy.send :include, Searchkick::ClassMethods
+ActiveRecord::Base.send(:extend, Searchkick::Model)

data/lib/searchkick/model.rb

@@ -0,0 +1,80 @@
+module Searchkick
+  module Model
+
+    def searchkick(options = {})
+      custom_settings = {
+        analysis: {
+          analyzer: {
+            searchkick_keyword: {
+              type: "custom",
+              tokenizer: "keyword",
+              filter: ["lowercase", "snowball"]
+            },
+            default_index: {
+              type: "custom",
+              tokenizer: "standard",
+              # synonym should come last, after stemming and shingle
+              # shingle must come before snowball
+              filter: ["standard", "lowercase", "asciifolding", "stop", "snowball", "searchkick_index_shingle"]
+            },
+            searchkick_search: {
+              type: "custom",
+              tokenizer: "standard",
+              filter: ["standard", "lowercase", "asciifolding", "stop", "snowball", "searchkick_search_shingle"]
+            },
+            searchkick_search2: {
+              type: "custom",
+              tokenizer: "standard",
+              filter: ["standard", "lowercase", "asciifolding", "stop", "snowball"] #, "searchkick_search_shingle"]
+            }
+          },
+          filter: {
+            searchkick_index_shingle: {
+              type: "shingle",
+              token_separator: ""
+            },
+            # lucky find http://web.archiveorange.com/archive/v/AAfXfQ17f57FcRINsof7
+            searchkick_search_shingle: {
+              type: "shingle",
+              token_separator: "",
+              output_unigrams: false,
+              output_unigrams_if_no_shingles: true
+            }
+          }
+        }
+      }.merge(options[:settings] || {})
+      synonyms = options[:synonyms] || []
+      if synonyms.any?
+        custom_settings[:analysis][:filter][:searchkick_synonym] = {
+          type: "synonym",
+          ignore_case: true,
+          synonyms: synonyms.map{|s| s.join(" => ") } # TODO support more than 2 synonyms on a line
+        }
+        custom_settings[:analysis][:analyzer][:default_index][:filter] << "searchkick_synonym"
+        custom_settings[:analysis][:analyzer][:searchkick_search][:filter].insert(-2, "searchkick_synonym")
+        custom_settings[:analysis][:analyzer][:searchkick_search][:filter] << "searchkick_synonym"
+        custom_settings[:analysis][:analyzer][:searchkick_search2][:filter] << "searchkick_synonym"
+      end
+
+      class_eval do
+        extend Searchkick::Search
+        extend Searchkick::Reindex
+        include Tire::Model::Search
+        include Tire::Model::Callbacks
+
+        tire do
+          settings custom_settings
+          mapping do
+            # indexes field, analyzer: "searchkick"
+            if options[:conversions]
+              indexes :conversions, type: "nested" do
+                indexes :query, analyzer: "searchkick_keyword"
+                indexes :count, type: "integer"
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/searchkick/reindex.rb

@@ -0,0 +1,40 @@
+module Searchkick
+  module Reindex
+
+    # https://gist.github.com/jarosan/3124884
+    def reindex
+      alias_name = tire.index.name
+      new_index = alias_name + "_" + Time.now.strftime("%Y%m%d%H%M%S")
+
+      # Rake::Task["tire:import"].invoke
+      index = Tire::Index.new(new_index)
+      Tire::Tasks::Import.create_index(index, self) # TODO remove puts
+      scope = respond_to?(:searchkick_import) ? searchkick_import : self
+      scope.find_in_batches do |batch|
+        index.import batch
+      end
+
+      if a = Tire::Alias.find(alias_name)
+        old_indices = Tire::Alias.find(alias_name).indices
+        old_indices.each do |index|
+          a.indices.delete index
+        end
+
+        a.indices.add new_index
+        a.save
+
+        old_indices.each do |index|
+          i = Tire::Index.new(index)
+          i.delete if i.exists?
+        end
+      else
+        i = Tire::Index.new(alias_name)
+        i.delete if i.exists?
+        Tire::Alias.create(name: alias_name, indices: [new_index])
+      end
+
+      true
+    end
+
+  end
+end

data/lib/searchkick/search.rb

@@ -0,0 +1,137 @@
+module Searchkick
+  # can't check mapping for conversions since the new index may not be built
+  module Search
+    def index_types
+      Hash[ (((Product.index.mapping || {})["product"] || {})["properties"] || {}).map{|k, v| [k, v["type"]] } ].reject{|k, v| k == "conversions" || k[0] == "_" }
+    end
+
+    def search(term, options = {})
+      fields = options[:fields] || ["_all"]
+      operator = options[:partial] ? "or" : "and"
+      tire.search do
+        query do
+          boolean do
+            must do
+              dis_max do
+                query do
+                  match fields, term, boost: 10, operator: operator, analyzer: "searchkick_search"
+                end
+                query do
+                  match fields, term, boost: 10, operator: operator, analyzer: "searchkick_search2"
+                end
+                query do
+                  match fields, term, use_dis_max: false, fuzziness: 0.7, max_expansions: 1, prefix_length: 1, operator: operator, analyzer: "searchkick_search"
+                end
+                query do
+                  match fields, term, use_dis_max: false, fuzziness: 0.7, max_expansions: 1, prefix_length: 1, operator: operator, analyzer: "searchkick_search2"
+                end
+              end
+            end
+            if options[:conversions]
+              should do
+                nested path: "conversions", score_mode: "total" do
+                  query do
+                    custom_score script: "log(doc['count'].value)" do
+                      match "query", term
+                    end
+                  end
+                end
+              end
+            end
+          end
+        end
+        size options[:limit] || 100000 # return all - like sql query
+        from options[:offset] if options[:offset]
+        explain options[:explain] if options[:explain]
+
+        # order
+        if options[:order]
+          sort do
+            options[:order].each do |k, v|
+              by k, v
+            end
+          end
+        end
+
+        # where
+        # TODO expand or
+
+        where_filters =
+          proc do |where|
+            filters = []
+            (where || {}).each do |field, value|
+              if field == :or
+                value.each do |or_clause|
+                  filters << {or: or_clause.map{|or_statement| {term: or_statement} }}
+                end
+              else
+                # expand ranges
+                if value.is_a?(Range)
+                  value = {gte: value.first, (value.exclude_end? ? :lt : :lte) => value.last}
+                end
+
+                if value.is_a?(Array) # in query
+                  filters << {terms: {field => value}}
+                elsif value.is_a?(Hash)
+                  value.each do |op, op_value|
+                    if op == :not # not equal
+                      if op_value.is_a?(Array)
+                        filters << {not: {terms: {field => op_value}}}
+                      else
+                        filters << {not: {term: {field => op_value}}}
+                      end
+                    else
+                      range_query =
+                        case op
+                        when :gt
+                          {from: op_value, include_lower: false}
+                        when :gte
+                          {from: op_value, include_lower: true}
+                        when :lt
+                          {to: op_value, include_upper: false}
+                        when :lte
+                          {to: op_value, include_upper: true}
+                        else
+                          raise "Unknown where operator"
+                        end
+                      filters << {range: {field => range_query}}
+                    end
+                  end
+                else
+                  filters << {term: {field => value}}
+                end
+              end
+            end
+            filters
+          end
+
+        where_filters.call(options[:where]).each do |f|
+          type, value = f.first
+          filter type, value
+        end
+
+        # facets
+        if options[:facets]
+          facets = options[:facets] || {}
+          if facets.is_a?(Array) # convert to more advanced syntax
+            facets = Hash[ facets.map{|f| [f, {}] } ]
+          end
+
+          facets.each do |field, facet_options|
+            facet_filters = where_filters.call(facet_options[:where])
+            facet field do
+              terms field
+              if facet_filters.size == 1
+                type, value = facet_filters.first.first
+                facet_filter type, value
+              elsif facet_filters.size > 1
+                facet_filter :and, *facet_filters
+              end
+            end
+          end
+        end
+
+      end
+    end
+  end
+end

data/lib/searchkick/version.rb

@@ -1,3 +1,3 @@
 module Searchkick
-  VERSION = "0.0.1"
+  VERSION = "0.0.2"
 end

data/searchkick.gemspec

@@ -22,4 +22,7 @@ Gem::Specification.new do |spec|
 
   spec.add_development_dependency "bundler", "~> 1.3"
   spec.add_development_dependency "rake"
+  spec.add_development_dependency "minitest"
+  spec.add_development_dependency "activerecord"
+  spec.add_development_dependency "pg"
 end

data/test/searchkick_test.rb

@@ -0,0 +1,283 @@
+require "test_helper"
+
+class Product < ActiveRecord::Base
+  searchkick \
+    synonyms: [
+      ["clorox", "bleach"],
+      ["scallion", "greenonion"],
+      ["saranwrap", "plasticwrap"],
+      ["qtip", "cotton swab"],
+      ["burger", "hamburger"],
+      ["bandaid", "bandag"]
+    ],
+    settings: {
+      number_of_shards: 1
+    },
+    conversions: true
+
+  # searchkick do
+  #   string :name
+  #   boolean :visible
+  #   integer :orders_count
+  # end
+end
+
+p Product.index_types
+
+class TestSearchkick < Minitest::Unit::TestCase
+
+  def setup
+    Product.index.delete
+    Product.create_elasticsearch_index
+  end
+
+  def test_reindex
+    assert Product.reindex
+  end
+
+  # exact
+
+  def test_match
+    store_names ["Whole Milk", "Fat Free Milk", "Milk"]
+    assert_search "milk", ["Milk", "Whole Milk", "Fat Free Milk"]
+  end
+
+  def test_case
+    store_names ["Whole Milk", "Fat Free Milk", "Milk"]
+    assert_search "MILK", ["Milk", "Whole Milk", "Fat Free Milk"]
+  end
+
+  def test_cheese_space_in_index
+    store_names ["Pepper Jack Cheese Skewers"]
+    assert_search "pepperjack cheese skewers", ["Pepper Jack Cheese Skewers"]
+  end
+
+  def test_cheese_space_in_query
+    store_names ["Pepperjack Cheese Skewers"]
+    assert_search "pepper jack cheese skewers", ["Pepperjack Cheese Skewers"]
+  end
+
+  def test_middle_token
+    store_names ["Dish Washer Amazing Organic Soap"]
+    assert_search "dish soap", ["Dish Washer Amazing Organic Soap"]
+  end
+
+  def test_percent
+    store_names ["1% Milk", "2% Milk", "Whole Milk"]
+    assert_search "1%", ["1% Milk"]
+  end
+
+  # ascii
+
+  def test_jalapenos
+    store_names ["Jalapeño"]
+    assert_search "jalapeno", ["Jalapeño"]
+  end
+
+  # stemming
+
+  def test_stemming
+    store_names ["Whole Milk", "Fat Free Milk", "Milk"]
+    assert_search "milks", ["Milk", "Whole Milk", "Fat Free Milk"]
+  end
+
+  # fuzzy
+
+  def test_misspelling_sriracha
+    store_names ["Sriracha"]
+    assert_search "siracha", ["Sriracha"]
+  end
+
+  def test_misspelling_tabasco
+    store_names ["Tabasco"]
+    assert_search "tobasco", ["Tabasco"]
+  end
+
+  def test_misspelling_zucchini
+    store_names ["Zucchini"]
+    assert_search "zuchini", ["Zucchini"]
+  end
+
+  def test_misspelling_ziploc
+    store_names ["Ziploc"]
+    assert_search "zip lock", ["Ziploc"]
+  end
+
+  # conversions
+
+  def test_conversions
+    store [
+      {name: "Tomato Sauce", conversions: [{query: "tomato sauce", count: 5}, {query: "tomato", count: 200}]},
+      {name: "Tomato Paste", conversions: []},
+      {name: "Tomatoes", conversions: [{query: "tomato", count: 100}, {query: "tomato sauce", count: 2}]}
+    ]
+    assert_search "tomato", ["Tomato Sauce", "Tomatoes", "Tomato Paste"]
+  end
+
+  def test_conversions_stemmed
+    store [
+      {name: "Tomato A", conversions: [{query: "tomato", count: 2}, {query: "tomatos", count: 2}, {query: "Tomatoes", count: 2}]},
+      {name: "Tomato B", conversions: [{query: "tomato", count: 4}]}
+    ]
+    assert_search "tomato", ["Tomato A", "Tomato B"]
+  end
+
+  # spaces
+
+  def test_spaces_in_field
+    store_names ["Red Bull"]
+    assert_search "redbull", ["Red Bull"]
+  end
+
+  def test_spaces_in_query
+    store_names ["Dishwasher Soap"]
+    assert_search "dish washer", ["Dishwasher Soap"]
+  end
+
+  def test_spaces_three_words
+    store_names ["Dish Washer Soap", "Dish Washer"]
+    assert_search "dish washer soap", ["Dish Washer Soap"]
+  end
+
+  def test_spaces_stemming
+    store_names ["Almond Milk"]
+    assert_search "almondmilks", ["Almond Milk"]
+  end
+
+  # keywords
+
+  def test_keywords
+    store_names ["Clorox Bleach", "Kroger Bleach", "Saran Wrap", "Kroger Plastic Wrap", "Hamburger Buns", "Band-Aid", "Kroger 12-Pack Bandages"]
+    assert_search "clorox", ["Clorox Bleach", "Kroger Bleach"]
+    assert_search "saran wrap", ["Saran Wrap", "Kroger Plastic Wrap"]
+    assert_search "burger buns", ["Hamburger Buns"]
+    assert_search "bandaids", ["Band-Aid", "Kroger 12-Pack Bandages"]
+  end
+
+  def test_keywords_qtips
+    store_names ["Q Tips", "Kroger Cotton Swabs"]
+    assert_search "q tips", ["Q Tips", "Kroger Cotton Swabs"]
+  end
+
+  def test_keywords_reverse
+    store_names ["Scallions"]
+    assert_search "green onions", ["Scallions"]
+  end
+
+  def test_keywords_exact
+    store_names ["Green Onions", "Yellow Onions"]
+    assert_search "scallion", ["Green Onions"]
+  end
+
+  def test_keywords_stemmed
+    store_names ["Green Onions", "Yellow Onions"]
+    assert_search "scallions", ["Green Onions"]
+  end
+
+  # global boost
+
+  def test_boost
+    store [
+      {name: "Organic Tomato A", _boost: 10},
+      {name: "Tomato B"}
+    ]
+    assert_search "tomato", ["Organic Tomato A", "Tomato B"]
+  end
+
+  def test_boost_zero
+    store [
+      {name: "Zero Boost", _boost: 0}
+    ]
+    assert_search "zero", ["Zero Boost"]
+  end
+
+  # default to 1
+  def test_boost_null
+    store [
+      {name: "Zero Boost A", _boost: 1.1},
+      {name: "Zero Boost B"},
+      {name: "Zero Boost C", _boost: 0.9},
+    ]
+    assert_search "zero", ["Zero Boost A", "Zero Boost B", "Zero Boost C"]
+  end
+
+  # search method
+
+  def test_limit
+    store_names ["Product A", "Product B"]
+    assert_equal 1, Product.search("Product", limit: 1).size
+  end
+
+  def test_offset
+    store_names ["Product A", "Product B"]
+    assert_equal 1, Product.search("Product", offset: 1).size
+  end
+
+  def test_where
+    now = Time.now
+    store [
+      {name: "Product A", store_id: 1, in_stock: true, backordered: true, created_at: now, _boost: 4},
+      {name: "Product B", store_id: 2, in_stock: true, backordered: false, created_at: now - 1, _boost: 3},
+      {name: "Product C", store_id: 3, in_stock: false, backordered: true, created_at: now - 2, _boost: 2},
+      {name: "Product D", store_id: 4, in_stock: false, backordered: false, created_at: now - 3, _boost: 1},
+    ]
+    assert_search "product", ["Product A", "Product B"], where: {in_stock: true}
+    # date
+    assert_search "product", ["Product A"], where: {created_at: {gt: now - 1}}
+    assert_search "product", ["Product A", "Product B"], where: {created_at: {gte: now - 1}}
+    assert_search "product", ["Product D"], where: {created_at: {lt: now - 2}}
+    assert_search "product", ["Product C", "Product D"], where: {created_at: {lte: now - 2}}
+    # integer
+    assert_search "product", ["Product A"], where: {store_id: {lt: 2}}
+    assert_search "product", ["Product A", "Product B"], where: {store_id: {lte: 2}}
+    assert_search "product", ["Product D"], where: {store_id: {gt: 3}}
+    assert_search "product", ["Product C", "Product D"], where: {store_id: {gte: 3}}
+    # range
+    assert_search "product", ["Product A", "Product B"], where: {store_id: 1..2}
+    assert_search "product", ["Product A"], where: {store_id: 1...2}
+    assert_search "product", ["Product A", "Product B"], where: {store_id: [1, 2]}
+    assert_search "product", ["Product B", "Product C", "Product D"], where: {store_id: {not: 1}}
+    assert_search "product", ["Product C", "Product D"], where: {store_id: {not: [1, 2]}}
+    assert_search "product", ["Product A", "Product B", "Product C"], where: {or: [[{in_stock: true}, {store_id: 3}]]}
+  end
+
+  def test_order
+    store_names ["Product A", "Product B", "Product C", "Product D"]
+    assert_search "product", ["Product D", "Product C", "Product B", "Product A"], order: {name: :desc}
+  end
+
+  def test_facets
+    store [
+      {name: "Product Show", store_id: 1, in_stock: true, color: "blue"},
+      {name: "Product Hide", store_id: 2, in_stock: false, color: "green"},
+      {name: "Product B", store_id: 2, in_stock: false, color: "red"}
+    ]
+    assert_equal 2, Product.search("Product", facets: [:store_id]).facets["store_id"]["terms"].size
+    assert_equal 1, Product.search("Product", facets: {store_id: {where: {in_stock: true}}}).facets["store_id"]["terms"].size
+    assert_equal 1, Product.search("Product", facets: {store_id: {where: {in_stock: true, color: "blue"}}}).facets["store_id"]["terms"].size
+  end
+
+  def test_partial
+    store_names ["Honey"]
+    assert_search "fresh honey", []
+    assert_search "fresh honey", ["Honey"], partial: true
+  end
+
+  protected
+
+  def store(documents)
+    documents.each do |document|
+      Product.index.store ({_type: "product"}).merge(document)
+    end
+    Product.index.refresh
+  end
+
+  def store_names(names)
+    store names.map{|name| {name: name} }
+  end
+
+  def assert_search(term, expected, options = {})
+    assert_equal expected, Product.search(term, options.merge(fields: [:name], conversions: true)).map(&:name)
+  end
+
+end

data/test/test_helper.rb

@@ -0,0 +1,26 @@
+require "bundler/setup"
+Bundler.require(:default)
+require "minitest/autorun"
+require "minitest/pride"
+require "active_record"
+
+# for debugging
+# ActiveRecord::Base.logger = Logger.new(STDOUT)
+
+# rails does this in activerecord/lib/active_record/railtie.rb
+ActiveRecord::Base.default_timezone = :utc
+ActiveRecord::Base.time_zone_aware_attributes = true
+
+# migrations
+ActiveRecord::Base.establish_connection :adapter => "postgresql", :database => "searchkick_test"
+
+ActiveRecord::Migration.create_table :products, :force => true do |t|
+  t.string :name
+  t.integer :store_id
+  t.boolean :in_stock
+  t.boolean :backordered
+  t.timestamps
+end
+
+File.delete("elasticsearch.log") if File.exists?("elasticsearch.log")
+Tire.configure { logger "elasticsearch.log", :level => "debug" }

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: searchkick
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.0.2
 platform: ruby
 authors:
 - Andrew Kane
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-07-15 00:00:00.000000000 Z
+date: 2013-07-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: tire
@@ -52,6 +52,48 @@ dependencies:
     - - '>='
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: pg
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Search made easy
 email:
 - andrew@chartkick.com
@@ -65,9 +107,14 @@ files:
 - README.md
 - Rakefile
 - lib/searchkick.rb
+- lib/searchkick/model.rb
+- lib/searchkick/reindex.rb
+- lib/searchkick/search.rb
 - lib/searchkick/tasks.rb
 - lib/searchkick/version.rb
 - searchkick.gemspec
+- test/searchkick_test.rb
+- test/test_helper.rb
 homepage: https://github.com/ankane/searchkick
 licenses:
 - MIT
@@ -92,4 +139,6 @@ rubygems_version: 2.0.0
 signing_key: 
 specification_version: 4
 summary: Search made easy
-test_files: []
+test_files:
+- test/searchkick_test.rb
+- test/test_helper.rb