searchkick 0.0.1 → 0.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 93eb2e14b7cab538e0e64e16f8b20bc1082e1d0f
4
- data.tar.gz: cb44a620ea08419c58a0e773de66cf4761dee785
3
+ metadata.gz: f37d2d366a7f03a6e2757c688385aed25678836c
4
+ data.tar.gz: e4648bd3692a88ccd7d9669376f8fa89af8e7431
5
5
  SHA512:
6
- metadata.gz: 27d1bd7b105e33c8b72b145413441438edfcff5cfb3f1b1f08c2f3e0390ed04f8164b6da94caf38f36ea1cec7d8ac72d97c8f3b4684c2299cea37563a5f507e4
7
- data.tar.gz: 75b3eadfd5b1e7dbb48ea4f94bb19bac382768f85ace0f8c367e5c926756d2c3925f4dfd7d19c27c5a65c1f0197559029e5e40d759ca32c9764ad0636de0d888
6
+ metadata.gz: 071e83eab034feb8784f32791728fee4cb61e3670ef9f36f97ad66961e7c12c299214e44bde1c5aec45f56b92a5d639591e2f9845a3fd2a1c8aee9e3fb14c7a9
7
+ data.tar.gz: e42658adc41be4c3e2232bfa510f9e2026da8c6a1303b192e5f03692d9c0d15068e3d92252955bba136bea21bfe830674ae84251e7134dc280f642d7b7f9bea6
data/.gitignore CHANGED
@@ -15,3 +15,4 @@ spec/reports
15
15
  test/tmp
16
16
  test/version_tmp
17
17
  tmp
18
+ *.log
data/README.md CHANGED
@@ -1,36 +1,175 @@
1
1
  # Searchkick
2
2
 
3
- Search made easy
3
+ :rocket: Search made easy
4
+
5
+ Searchkick provides sensible search defaults out of the box. It handles:
6
+
7
+ - stemming - `tomatoes` matches `tomato`
8
+ - special characters - `jalapenos` matches `jalapeños`
9
+ - extra whitespace - `dishwasher` matches `dish washer`
10
+ - misspellings - `zuchini` matches `zucchini`
11
+ - custom synonyms - `qtip` matches `cotton swab`
12
+
13
+ Runs on Elasticsearch
14
+
15
+ :tangerine: Battle-tested at [Instacart](https://www.instacart.com)
4
16
 
5
17
  ## Usage
6
18
 
7
- ### Reindex with Zero Downtime
19
+ ```ruby
20
+ class Product < ActiveRecord::Base
21
+ searchkick
22
+ end
23
+ ```
8
24
 
9
- Elasticsearch has a feature called aliases that allows you to reindex with no downtime.
25
+ And to query, use:
10
26
 
11
27
  ```ruby
12
- Book.tire.reindex
28
+ Product.search "2% Milk"
13
29
  ```
14
30
 
15
- This creates a new index `books_20130714181054` and points the `books` alias to the new index when complete - an atomic operation :)
31
+ or only search specific fields:
16
32
 
17
- **First time:** If books is an existing index, it will be replaced by an alias.
33
+ ```ruby
34
+ Product.search "Butter", fields: [:name, :brand]
35
+ ```
18
36
 
19
- Searchkick uses `find_in_batches` to import documents. To filter documents or eagar load associations, use the `tire_import` scope.
37
+ ### Query Like SQL
20
38
 
21
39
  ```ruby
22
- class Book < ActiveRecord::Base
23
- scope :tire_import, where(active: true).includes(:author, :chapters)
40
+ Product.search "2% Milk", where: {in_stock: true}, limit: 10, offset: 50
41
+ ```
42
+
43
+ #### Where
44
+
45
+ ```ruby
46
+ where: {
47
+ expires_at: {gt: Time.now}, # lt, gte, lte also available
48
+ orders_count: 1..10, # equivalent to {gte: 1, lte: 10}
49
+ aisle_id: [25, 30], # in
50
+ store_id: {not: 2}, # not
51
+ aisle_id: {not: [25, 30]}, # not in
52
+ or: [
53
+ [{in_stock: true}, {backordered: true}]
54
+ ]
55
+ }
56
+ ```
57
+
58
+ #### Order
59
+
60
+ ```ruby
61
+ order: {_score: :desc} # most relevant first - default
62
+ ```
63
+
64
+ #### Explain
65
+
66
+ ```ruby
67
+ explain: true
68
+ ```
69
+
70
+ ### Facets
71
+
72
+ ```ruby
73
+ Product.search "2% Milk", facets: [:store_id, :aisle_id]
74
+ ```
75
+
76
+ Advanced
77
+
78
+ ```ruby
79
+ Product.search "2% Milk", facets: {store_id: {where: {in_stock: true}}}
80
+ ```
81
+
82
+ ### Synonyms
83
+
84
+ ```ruby
85
+ class Product < ActiveRecord::Base
86
+ searchkick synonyms: [["scallion", "green onion"], ["qtip", "cotton swab"]]
87
+ end
88
+ ```
89
+
90
+ You must call `Product.reindex` after changing synonyms.
91
+
92
+ ### Make Searches Better Over Time
93
+
94
+ Improve results with analytics on conversions and give popular documents a little boost.
95
+
96
+ First, you must keep track of search conversions. The database works well for low volume, but feel free to use redis or another datastore.
97
+
98
+ ```ruby
99
+ class Search < ActiveRecord::Base
100
+ belongs_to :product
101
+ # fields: id, query, searched_at, converted_at, product_id
102
+ end
103
+ ```
104
+
105
+ Add the conversions to the index.
106
+
107
+ ```ruby
108
+ class Product < ActiveRecord::Base
109
+ has_many :searches
110
+
111
+ searchkick conversions: true
112
+
113
+ def to_indexed_json
114
+ {
115
+ name: name,
116
+ conversions: searches.group("query").count.map{|query, count| {query: query, count: count} }, # TODO fix
117
+ _boost: Math.log(orders_count) # boost more popular products a bit
118
+ }
119
+ end
120
+ end
121
+ ```
122
+
123
+ After the reindex is complete (to prevent errors), tell the search method to use conversions.
124
+
125
+ ```ruby
126
+ Product.search "Fat Free Milk", conversions: true
127
+ ```
128
+
129
+ ### Zero Downtime Changes
130
+
131
+ ```ruby
132
+ Product.reindex
133
+ ```
134
+
135
+ Behind the scenes, this creates a new index `products_20130714181054` and points the `products` alias to the new index when complete - an atomic operation :)
136
+
137
+ Searchkick uses `find_in_batches` to import documents. To filter documents or eagar load associations, use the `searchkick_import` scope.
138
+
139
+ ```ruby
140
+ class Product < ActiveRecord::Base
141
+ scope :searchkick_import, where(active: true).includes(:searches)
24
142
  end
25
143
  ```
26
144
 
27
145
  There is also a rake task.
28
146
 
29
147
  ```sh
30
- rake searchkick:reindex CLASS=Book
148
+ rake searchkick:reindex CLASS=Product
149
+ ```
150
+
151
+ Thanks to Jaroslav Kalistsuk for the [original implementation](https://gist.github.com/jarosan/3124884).
152
+
153
+ ### Reference
154
+
155
+ Reindex one item
156
+
157
+ ```ruby
158
+ product = Product.find(1)
159
+ product.update_index
31
160
  ```
32
161
 
33
- [Thanks to Jaroslav Kalistsuk for the original source](https://gist.github.com/jarosan/3124884)
162
+ Partial matches (needs better name)
163
+
164
+ ```ruby
165
+ Item.search "fresh honey", partial: true # matches organic honey
166
+ ```
167
+
168
+ ## Elasticsearch Gotchas
169
+
170
+ ### Inconsistent Scores
171
+
172
+ Due to the distributed nature of Elasticsearch, you can get incorrect results when the number of documents in the index is low. You can [read more about it here](http://www.elasticsearch.org/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch/). To fix this, set the search type to `dfs_query_and_fetch`. Alternatively, you can just use one shard with `settings: {number_of_shards: 1}`.
34
173
 
35
174
  ## Installation
36
175
 
@@ -46,6 +185,19 @@ And then execute:
46
185
  bundle
47
186
  ```
48
187
 
188
+ ## TODO
189
+
190
+ - Autocomplete
191
+ - Option to turn off fuzzy matching (should this be default?)
192
+ - Exact phrase matches (in order)
193
+ - Focus on results format (load: true?)
194
+ - Test helpers - everyone should test their own search
195
+ - Built-in synonyms from WordNet
196
+ - Dashboard w/ real-time analytics?
197
+ - [Suggest API](http://www.elasticsearch.org/guide/reference/api/search/suggest/) "Did you mean?"
198
+ - Allow for "exact search" with quotes
199
+ - Make updates to old and new index while reindexing [possibly with an another alias](http://www.kickstarter.com/backing-and-hacking)
200
+
49
201
  ## Contributing
50
202
 
51
203
  1. Fork it
data/Rakefile CHANGED
@@ -1 +1,8 @@
1
1
  require "bundler/gem_tasks"
2
+ require "rake/testtask"
3
+
4
+ task :default => :test
5
+ Rake::TestTask.new do |t|
6
+ t.libs << "test"
7
+ t.pattern = "test/**/*_test.rb"
8
+ end
data/lib/searchkick.rb CHANGED
@@ -1,49 +1,9 @@
1
1
  require "searchkick/version"
2
+ require "searchkick/reindex"
3
+ require "searchkick/search"
4
+ require "searchkick/model"
2
5
  require "searchkick/tasks"
3
6
  require "tire"
7
+ require "active_record" # TODO only require active_model
4
8
 
5
- module Searchkick
6
- module ClassMethods
7
-
8
- # https://gist.github.com/jarosan/3124884
9
- def reindex
10
- alias_name = klass.tire.index.name
11
- new_index = alias_name + "_" + Time.now.strftime("%Y%m%d%H%M%S")
12
-
13
- # Rake::Task["tire:import"].invoke
14
- index = Tire::Index.new(new_index)
15
- Tire::Tasks::Import.create_index(index, klass)
16
- scope = klass.respond_to?(:tire_import) ? klass.tire_import : klass
17
- scope.find_in_batches do |batch|
18
- index.import batch
19
- end
20
-
21
- if a = Tire::Alias.find(alias_name)
22
- puts "[IMPORT] Alias found: #{Tire::Alias.find(alias_name).indices.to_ary.join(",")}"
23
- old_indices = Tire::Alias.find(alias_name).indices
24
- old_indices.each do |index|
25
- a.indices.delete index
26
- end
27
-
28
- a.indices.add new_index
29
- a.save
30
-
31
- old_indices.each do |index|
32
- puts "[IMPORT] Deleting index: #{index}"
33
- i = Tire::Index.new(index)
34
- i.delete if i.exists?
35
- end
36
- else
37
- puts "[IMPORT] No alias found. Deleting index, creating new one, and setting up alias"
38
- i = Tire::Index.new(alias_name)
39
- i.delete if i.exists?
40
- Tire::Alias.create(name: alias_name, indices: [new_index])
41
- end
42
-
43
- puts "[IMPORT] Saved alias #{alias_name} pointing to #{new_index}"
44
- end
45
-
46
- end
47
- end
48
-
49
- Tire::Model::Search::ClassMethodsProxy.send :include, Searchkick::ClassMethods
9
+ ActiveRecord::Base.send(:extend, Searchkick::Model)
@@ -0,0 +1,80 @@
1
+ module Searchkick
2
+ module Model
3
+
4
+ def searchkick(options = {})
5
+ custom_settings = {
6
+ analysis: {
7
+ analyzer: {
8
+ searchkick_keyword: {
9
+ type: "custom",
10
+ tokenizer: "keyword",
11
+ filter: ["lowercase", "snowball"]
12
+ },
13
+ default_index: {
14
+ type: "custom",
15
+ tokenizer: "standard",
16
+ # synonym should come last, after stemming and shingle
17
+ # shingle must come before snowball
18
+ filter: ["standard", "lowercase", "asciifolding", "stop", "snowball", "searchkick_index_shingle"]
19
+ },
20
+ searchkick_search: {
21
+ type: "custom",
22
+ tokenizer: "standard",
23
+ filter: ["standard", "lowercase", "asciifolding", "stop", "snowball", "searchkick_search_shingle"]
24
+ },
25
+ searchkick_search2: {
26
+ type: "custom",
27
+ tokenizer: "standard",
28
+ filter: ["standard", "lowercase", "asciifolding", "stop", "snowball"] #, "searchkick_search_shingle"]
29
+ }
30
+ },
31
+ filter: {
32
+ searchkick_index_shingle: {
33
+ type: "shingle",
34
+ token_separator: ""
35
+ },
36
+ # lucky find http://web.archiveorange.com/archive/v/AAfXfQ17f57FcRINsof7
37
+ searchkick_search_shingle: {
38
+ type: "shingle",
39
+ token_separator: "",
40
+ output_unigrams: false,
41
+ output_unigrams_if_no_shingles: true
42
+ }
43
+ }
44
+ }
45
+ }.merge(options[:settings] || {})
46
+ synonyms = options[:synonyms] || []
47
+ if synonyms.any?
48
+ custom_settings[:analysis][:filter][:searchkick_synonym] = {
49
+ type: "synonym",
50
+ ignore_case: true,
51
+ synonyms: synonyms.map{|s| s.join(" => ") } # TODO support more than 2 synonyms on a line
52
+ }
53
+ custom_settings[:analysis][:analyzer][:default_index][:filter] << "searchkick_synonym"
54
+ custom_settings[:analysis][:analyzer][:searchkick_search][:filter].insert(-2, "searchkick_synonym")
55
+ custom_settings[:analysis][:analyzer][:searchkick_search][:filter] << "searchkick_synonym"
56
+ custom_settings[:analysis][:analyzer][:searchkick_search2][:filter] << "searchkick_synonym"
57
+ end
58
+
59
+ class_eval do
60
+ extend Searchkick::Search
61
+ extend Searchkick::Reindex
62
+ include Tire::Model::Search
63
+ include Tire::Model::Callbacks
64
+
65
+ tire do
66
+ settings custom_settings
67
+ mapping do
68
+ # indexes field, analyzer: "searchkick"
69
+ if options[:conversions]
70
+ indexes :conversions, type: "nested" do
71
+ indexes :query, analyzer: "searchkick_keyword"
72
+ indexes :count, type: "integer"
73
+ end
74
+ end
75
+ end
76
+ end
77
+ end
78
+ end
79
+ end
80
+ end
@@ -0,0 +1,40 @@
1
+ module Searchkick
2
+ module Reindex
3
+
4
+ # https://gist.github.com/jarosan/3124884
5
+ def reindex
6
+ alias_name = tire.index.name
7
+ new_index = alias_name + "_" + Time.now.strftime("%Y%m%d%H%M%S")
8
+
9
+ # Rake::Task["tire:import"].invoke
10
+ index = Tire::Index.new(new_index)
11
+ Tire::Tasks::Import.create_index(index, self) # TODO remove puts
12
+ scope = respond_to?(:searchkick_import) ? searchkick_import : self
13
+ scope.find_in_batches do |batch|
14
+ index.import batch
15
+ end
16
+
17
+ if a = Tire::Alias.find(alias_name)
18
+ old_indices = Tire::Alias.find(alias_name).indices
19
+ old_indices.each do |index|
20
+ a.indices.delete index
21
+ end
22
+
23
+ a.indices.add new_index
24
+ a.save
25
+
26
+ old_indices.each do |index|
27
+ i = Tire::Index.new(index)
28
+ i.delete if i.exists?
29
+ end
30
+ else
31
+ i = Tire::Index.new(alias_name)
32
+ i.delete if i.exists?
33
+ Tire::Alias.create(name: alias_name, indices: [new_index])
34
+ end
35
+
36
+ true
37
+ end
38
+
39
+ end
40
+ end
@@ -0,0 +1,137 @@
1
+ module Searchkick
2
+ # can't check mapping for conversions since the new index may not be built
3
+ module Search
4
+ def index_types
5
+ Hash[ (((Product.index.mapping || {})["product"] || {})["properties"] || {}).map{|k, v| [k, v["type"]] } ].reject{|k, v| k == "conversions" || k[0] == "_" }
6
+ end
7
+
8
+ def search(term, options = {})
9
+ fields = options[:fields] || ["_all"]
10
+ operator = options[:partial] ? "or" : "and"
11
+ tire.search do
12
+ query do
13
+ boolean do
14
+ must do
15
+ dis_max do
16
+ query do
17
+ match fields, term, boost: 10, operator: operator, analyzer: "searchkick_search"
18
+ end
19
+ query do
20
+ match fields, term, boost: 10, operator: operator, analyzer: "searchkick_search2"
21
+ end
22
+ query do
23
+ match fields, term, use_dis_max: false, fuzziness: 0.7, max_expansions: 1, prefix_length: 1, operator: operator, analyzer: "searchkick_search"
24
+ end
25
+ query do
26
+ match fields, term, use_dis_max: false, fuzziness: 0.7, max_expansions: 1, prefix_length: 1, operator: operator, analyzer: "searchkick_search2"
27
+ end
28
+ end
29
+ end
30
+ if options[:conversions]
31
+ should do
32
+ nested path: "conversions", score_mode: "total" do
33
+ query do
34
+ custom_score script: "log(doc['count'].value)" do
35
+ match "query", term
36
+ end
37
+ end
38
+ end
39
+ end
40
+ end
41
+ end
42
+ end
43
+ size options[:limit] || 100000 # return all - like sql query
44
+ from options[:offset] if options[:offset]
45
+ explain options[:explain] if options[:explain]
46
+
47
+ # order
48
+ if options[:order]
49
+ sort do
50
+ options[:order].each do |k, v|
51
+ by k, v
52
+ end
53
+ end
54
+ end
55
+
56
+ # where
57
+ # TODO expand or
58
+
59
+ where_filters =
60
+ proc do |where|
61
+ filters = []
62
+ (where || {}).each do |field, value|
63
+ if field == :or
64
+ value.each do |or_clause|
65
+ filters << {or: or_clause.map{|or_statement| {term: or_statement} }}
66
+ end
67
+ else
68
+ # expand ranges
69
+ if value.is_a?(Range)
70
+ value = {gte: value.first, (value.exclude_end? ? :lt : :lte) => value.last}
71
+ end
72
+
73
+ if value.is_a?(Array) # in query
74
+ filters << {terms: {field => value}}
75
+ elsif value.is_a?(Hash)
76
+ value.each do |op, op_value|
77
+ if op == :not # not equal
78
+ if op_value.is_a?(Array)
79
+ filters << {not: {terms: {field => op_value}}}
80
+ else
81
+ filters << {not: {term: {field => op_value}}}
82
+ end
83
+ else
84
+ range_query =
85
+ case op
86
+ when :gt
87
+ {from: op_value, include_lower: false}
88
+ when :gte
89
+ {from: op_value, include_lower: true}
90
+ when :lt
91
+ {to: op_value, include_upper: false}
92
+ when :lte
93
+ {to: op_value, include_upper: true}
94
+ else
95
+ raise "Unknown where operator"
96
+ end
97
+ filters << {range: {field => range_query}}
98
+ end
99
+ end
100
+ else
101
+ filters << {term: {field => value}}
102
+ end
103
+ end
104
+ end
105
+ filters
106
+ end
107
+
108
+ where_filters.call(options[:where]).each do |f|
109
+ type, value = f.first
110
+ filter type, value
111
+ end
112
+
113
+ # facets
114
+ if options[:facets]
115
+ facets = options[:facets] || {}
116
+ if facets.is_a?(Array) # convert to more advanced syntax
117
+ facets = Hash[ facets.map{|f| [f, {}] } ]
118
+ end
119
+
120
+ facets.each do |field, facet_options|
121
+ facet_filters = where_filters.call(facet_options[:where])
122
+ facet field do
123
+ terms field
124
+ if facet_filters.size == 1
125
+ type, value = facet_filters.first.first
126
+ facet_filter type, value
127
+ elsif facet_filters.size > 1
128
+ facet_filter :and, *facet_filters
129
+ end
130
+ end
131
+ end
132
+ end
133
+
134
+ end
135
+ end
136
+ end
137
+ end
@@ -1,3 +1,3 @@
1
1
  module Searchkick
2
- VERSION = "0.0.1"
2
+ VERSION = "0.0.2"
3
3
  end
data/searchkick.gemspec CHANGED
@@ -22,4 +22,7 @@ Gem::Specification.new do |spec|
22
22
 
23
23
  spec.add_development_dependency "bundler", "~> 1.3"
24
24
  spec.add_development_dependency "rake"
25
+ spec.add_development_dependency "minitest"
26
+ spec.add_development_dependency "activerecord"
27
+ spec.add_development_dependency "pg"
25
28
  end
@@ -0,0 +1,283 @@
1
+ require "test_helper"
2
+
3
+ class Product < ActiveRecord::Base
4
+ searchkick \
5
+ synonyms: [
6
+ ["clorox", "bleach"],
7
+ ["scallion", "greenonion"],
8
+ ["saranwrap", "plasticwrap"],
9
+ ["qtip", "cotton swab"],
10
+ ["burger", "hamburger"],
11
+ ["bandaid", "bandag"]
12
+ ],
13
+ settings: {
14
+ number_of_shards: 1
15
+ },
16
+ conversions: true
17
+
18
+ # searchkick do
19
+ # string :name
20
+ # boolean :visible
21
+ # integer :orders_count
22
+ # end
23
+ end
24
+
25
+ p Product.index_types
26
+
27
+ class TestSearchkick < Minitest::Unit::TestCase
28
+
29
+ def setup
30
+ Product.index.delete
31
+ Product.create_elasticsearch_index
32
+ end
33
+
34
+ def test_reindex
35
+ assert Product.reindex
36
+ end
37
+
38
+ # exact
39
+
40
+ def test_match
41
+ store_names ["Whole Milk", "Fat Free Milk", "Milk"]
42
+ assert_search "milk", ["Milk", "Whole Milk", "Fat Free Milk"]
43
+ end
44
+
45
+ def test_case
46
+ store_names ["Whole Milk", "Fat Free Milk", "Milk"]
47
+ assert_search "MILK", ["Milk", "Whole Milk", "Fat Free Milk"]
48
+ end
49
+
50
+ def test_cheese_space_in_index
51
+ store_names ["Pepper Jack Cheese Skewers"]
52
+ assert_search "pepperjack cheese skewers", ["Pepper Jack Cheese Skewers"]
53
+ end
54
+
55
+ def test_cheese_space_in_query
56
+ store_names ["Pepperjack Cheese Skewers"]
57
+ assert_search "pepper jack cheese skewers", ["Pepperjack Cheese Skewers"]
58
+ end
59
+
60
+ def test_middle_token
61
+ store_names ["Dish Washer Amazing Organic Soap"]
62
+ assert_search "dish soap", ["Dish Washer Amazing Organic Soap"]
63
+ end
64
+
65
+ def test_percent
66
+ store_names ["1% Milk", "2% Milk", "Whole Milk"]
67
+ assert_search "1%", ["1% Milk"]
68
+ end
69
+
70
+ # ascii
71
+
72
+ def test_jalapenos
73
+ store_names ["Jalapeño"]
74
+ assert_search "jalapeno", ["Jalapeño"]
75
+ end
76
+
77
+ # stemming
78
+
79
+ def test_stemming
80
+ store_names ["Whole Milk", "Fat Free Milk", "Milk"]
81
+ assert_search "milks", ["Milk", "Whole Milk", "Fat Free Milk"]
82
+ end
83
+
84
+ # fuzzy
85
+
86
+ def test_misspelling_sriracha
87
+ store_names ["Sriracha"]
88
+ assert_search "siracha", ["Sriracha"]
89
+ end
90
+
91
+ def test_misspelling_tabasco
92
+ store_names ["Tabasco"]
93
+ assert_search "tobasco", ["Tabasco"]
94
+ end
95
+
96
+ def test_misspelling_zucchini
97
+ store_names ["Zucchini"]
98
+ assert_search "zuchini", ["Zucchini"]
99
+ end
100
+
101
+ def test_misspelling_ziploc
102
+ store_names ["Ziploc"]
103
+ assert_search "zip lock", ["Ziploc"]
104
+ end
105
+
106
+ # conversions
107
+
108
+ def test_conversions
109
+ store [
110
+ {name: "Tomato Sauce", conversions: [{query: "tomato sauce", count: 5}, {query: "tomato", count: 200}]},
111
+ {name: "Tomato Paste", conversions: []},
112
+ {name: "Tomatoes", conversions: [{query: "tomato", count: 100}, {query: "tomato sauce", count: 2}]}
113
+ ]
114
+ assert_search "tomato", ["Tomato Sauce", "Tomatoes", "Tomato Paste"]
115
+ end
116
+
117
+ def test_conversions_stemmed
118
+ store [
119
+ {name: "Tomato A", conversions: [{query: "tomato", count: 2}, {query: "tomatos", count: 2}, {query: "Tomatoes", count: 2}]},
120
+ {name: "Tomato B", conversions: [{query: "tomato", count: 4}]}
121
+ ]
122
+ assert_search "tomato", ["Tomato A", "Tomato B"]
123
+ end
124
+
125
+ # spaces
126
+
127
+ def test_spaces_in_field
128
+ store_names ["Red Bull"]
129
+ assert_search "redbull", ["Red Bull"]
130
+ end
131
+
132
+ def test_spaces_in_query
133
+ store_names ["Dishwasher Soap"]
134
+ assert_search "dish washer", ["Dishwasher Soap"]
135
+ end
136
+
137
+ def test_spaces_three_words
138
+ store_names ["Dish Washer Soap", "Dish Washer"]
139
+ assert_search "dish washer soap", ["Dish Washer Soap"]
140
+ end
141
+
142
+ def test_spaces_stemming
143
+ store_names ["Almond Milk"]
144
+ assert_search "almondmilks", ["Almond Milk"]
145
+ end
146
+
147
+ # keywords
148
+
149
+ def test_keywords
150
+ store_names ["Clorox Bleach", "Kroger Bleach", "Saran Wrap", "Kroger Plastic Wrap", "Hamburger Buns", "Band-Aid", "Kroger 12-Pack Bandages"]
151
+ assert_search "clorox", ["Clorox Bleach", "Kroger Bleach"]
152
+ assert_search "saran wrap", ["Saran Wrap", "Kroger Plastic Wrap"]
153
+ assert_search "burger buns", ["Hamburger Buns"]
154
+ assert_search "bandaids", ["Band-Aid", "Kroger 12-Pack Bandages"]
155
+ end
156
+
157
+ def test_keywords_qtips
158
+ store_names ["Q Tips", "Kroger Cotton Swabs"]
159
+ assert_search "q tips", ["Q Tips", "Kroger Cotton Swabs"]
160
+ end
161
+
162
+ def test_keywords_reverse
163
+ store_names ["Scallions"]
164
+ assert_search "green onions", ["Scallions"]
165
+ end
166
+
167
+ def test_keywords_exact
168
+ store_names ["Green Onions", "Yellow Onions"]
169
+ assert_search "scallion", ["Green Onions"]
170
+ end
171
+
172
+ def test_keywords_stemmed
173
+ store_names ["Green Onions", "Yellow Onions"]
174
+ assert_search "scallions", ["Green Onions"]
175
+ end
176
+
177
+ # global boost
178
+
179
+ def test_boost
180
+ store [
181
+ {name: "Organic Tomato A", _boost: 10},
182
+ {name: "Tomato B"}
183
+ ]
184
+ assert_search "tomato", ["Organic Tomato A", "Tomato B"]
185
+ end
186
+
187
+ def test_boost_zero
188
+ store [
189
+ {name: "Zero Boost", _boost: 0}
190
+ ]
191
+ assert_search "zero", ["Zero Boost"]
192
+ end
193
+
194
+ # default to 1
195
+ def test_boost_null
196
+ store [
197
+ {name: "Zero Boost A", _boost: 1.1},
198
+ {name: "Zero Boost B"},
199
+ {name: "Zero Boost C", _boost: 0.9},
200
+ ]
201
+ assert_search "zero", ["Zero Boost A", "Zero Boost B", "Zero Boost C"]
202
+ end
203
+
204
+ # search method
205
+
206
+ def test_limit
207
+ store_names ["Product A", "Product B"]
208
+ assert_equal 1, Product.search("Product", limit: 1).size
209
+ end
210
+
211
+ def test_offset
212
+ store_names ["Product A", "Product B"]
213
+ assert_equal 1, Product.search("Product", offset: 1).size
214
+ end
215
+
216
+ def test_where
217
+ now = Time.now
218
+ store [
219
+ {name: "Product A", store_id: 1, in_stock: true, backordered: true, created_at: now, _boost: 4},
220
+ {name: "Product B", store_id: 2, in_stock: true, backordered: false, created_at: now - 1, _boost: 3},
221
+ {name: "Product C", store_id: 3, in_stock: false, backordered: true, created_at: now - 2, _boost: 2},
222
+ {name: "Product D", store_id: 4, in_stock: false, backordered: false, created_at: now - 3, _boost: 1},
223
+ ]
224
+ assert_search "product", ["Product A", "Product B"], where: {in_stock: true}
225
+ # date
226
+ assert_search "product", ["Product A"], where: {created_at: {gt: now - 1}}
227
+ assert_search "product", ["Product A", "Product B"], where: {created_at: {gte: now - 1}}
228
+ assert_search "product", ["Product D"], where: {created_at: {lt: now - 2}}
229
+ assert_search "product", ["Product C", "Product D"], where: {created_at: {lte: now - 2}}
230
+ # integer
231
+ assert_search "product", ["Product A"], where: {store_id: {lt: 2}}
232
+ assert_search "product", ["Product A", "Product B"], where: {store_id: {lte: 2}}
233
+ assert_search "product", ["Product D"], where: {store_id: {gt: 3}}
234
+ assert_search "product", ["Product C", "Product D"], where: {store_id: {gte: 3}}
235
+ # range
236
+ assert_search "product", ["Product A", "Product B"], where: {store_id: 1..2}
237
+ assert_search "product", ["Product A"], where: {store_id: 1...2}
238
+ assert_search "product", ["Product A", "Product B"], where: {store_id: [1, 2]}
239
+ assert_search "product", ["Product B", "Product C", "Product D"], where: {store_id: {not: 1}}
240
+ assert_search "product", ["Product C", "Product D"], where: {store_id: {not: [1, 2]}}
241
+ assert_search "product", ["Product A", "Product B", "Product C"], where: {or: [[{in_stock: true}, {store_id: 3}]]}
242
+ end
243
+
244
+ def test_order
245
+ store_names ["Product A", "Product B", "Product C", "Product D"]
246
+ assert_search "product", ["Product D", "Product C", "Product B", "Product A"], order: {name: :desc}
247
+ end
248
+
249
+ def test_facets
250
+ store [
251
+ {name: "Product Show", store_id: 1, in_stock: true, color: "blue"},
252
+ {name: "Product Hide", store_id: 2, in_stock: false, color: "green"},
253
+ {name: "Product B", store_id: 2, in_stock: false, color: "red"}
254
+ ]
255
+ assert_equal 2, Product.search("Product", facets: [:store_id]).facets["store_id"]["terms"].size
256
+ assert_equal 1, Product.search("Product", facets: {store_id: {where: {in_stock: true}}}).facets["store_id"]["terms"].size
257
+ assert_equal 1, Product.search("Product", facets: {store_id: {where: {in_stock: true, color: "blue"}}}).facets["store_id"]["terms"].size
258
+ end
259
+
260
+ def test_partial
261
+ store_names ["Honey"]
262
+ assert_search "fresh honey", []
263
+ assert_search "fresh honey", ["Honey"], partial: true
264
+ end
265
+
266
+ protected
267
+
268
+ def store(documents)
269
+ documents.each do |document|
270
+ Product.index.store ({_type: "product"}).merge(document)
271
+ end
272
+ Product.index.refresh
273
+ end
274
+
275
+ def store_names(names)
276
+ store names.map{|name| {name: name} }
277
+ end
278
+
279
+ def assert_search(term, expected, options = {})
280
+ assert_equal expected, Product.search(term, options.merge(fields: [:name], conversions: true)).map(&:name)
281
+ end
282
+
283
+ end
@@ -0,0 +1,26 @@
1
+ require "bundler/setup"
2
+ Bundler.require(:default)
3
+ require "minitest/autorun"
4
+ require "minitest/pride"
5
+ require "active_record"
6
+
7
+ # for debugging
8
+ # ActiveRecord::Base.logger = Logger.new(STDOUT)
9
+
10
+ # rails does this in activerecord/lib/active_record/railtie.rb
11
+ ActiveRecord::Base.default_timezone = :utc
12
+ ActiveRecord::Base.time_zone_aware_attributes = true
13
+
14
+ # migrations
15
+ ActiveRecord::Base.establish_connection :adapter => "postgresql", :database => "searchkick_test"
16
+
17
+ ActiveRecord::Migration.create_table :products, :force => true do |t|
18
+ t.string :name
19
+ t.integer :store_id
20
+ t.boolean :in_stock
21
+ t.boolean :backordered
22
+ t.timestamps
23
+ end
24
+
25
+ File.delete("elasticsearch.log") if File.exists?("elasticsearch.log")
26
+ Tire.configure { logger "elasticsearch.log", :level => "debug" }
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: searchkick
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
4
+ version: 0.0.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrew Kane
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2013-07-15 00:00:00.000000000 Z
11
+ date: 2013-07-18 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: tire
@@ -52,6 +52,48 @@ dependencies:
52
52
  - - '>='
53
53
  - !ruby/object:Gem::Version
54
54
  version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: minitest
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - '>='
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - '>='
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: activerecord
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - '>='
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - '>='
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: pg
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - '>='
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - '>='
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
55
97
  description: Search made easy
56
98
  email:
57
99
  - andrew@chartkick.com
@@ -65,9 +107,14 @@ files:
65
107
  - README.md
66
108
  - Rakefile
67
109
  - lib/searchkick.rb
110
+ - lib/searchkick/model.rb
111
+ - lib/searchkick/reindex.rb
112
+ - lib/searchkick/search.rb
68
113
  - lib/searchkick/tasks.rb
69
114
  - lib/searchkick/version.rb
70
115
  - searchkick.gemspec
116
+ - test/searchkick_test.rb
117
+ - test/test_helper.rb
71
118
  homepage: https://github.com/ankane/searchkick
72
119
  licenses:
73
120
  - MIT
@@ -92,4 +139,6 @@ rubygems_version: 2.0.0
92
139
  signing_key:
93
140
  specification_version: 4
94
141
  summary: Search made easy
95
- test_files: []
142
+ test_files:
143
+ - test/searchkick_test.rb
144
+ - test/test_helper.rb