searchkick 2.3.2 → 5.2.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/CHANGELOG.md +377 -84
- data/LICENSE.txt +1 -1
- data/README.md +859 -602
- data/lib/searchkick/bulk_reindex_job.rb +13 -9
- data/lib/searchkick/controller_runtime.rb +40 -0
- data/lib/searchkick/hash_wrapper.rb +12 -0
- data/lib/searchkick/index.rb +281 -356
- data/lib/searchkick/index_cache.rb +30 -0
- data/lib/searchkick/index_options.rb +487 -281
- data/lib/searchkick/indexer.rb +15 -8
- data/lib/searchkick/log_subscriber.rb +57 -0
- data/lib/searchkick/middleware.rb +9 -2
- data/lib/searchkick/model.rb +72 -118
- data/lib/searchkick/multi_search.rb +9 -10
- data/lib/searchkick/process_batch_job.rb +12 -15
- data/lib/searchkick/process_queue_job.rb +22 -13
- data/lib/searchkick/query.rb +458 -217
- data/lib/searchkick/railtie.rb +7 -0
- data/lib/searchkick/record_data.rb +128 -0
- data/lib/searchkick/record_indexer.rb +164 -0
- data/lib/searchkick/reindex_queue.rb +51 -9
- data/lib/searchkick/reindex_v2_job.rb +10 -32
- data/lib/searchkick/relation.rb +247 -0
- data/lib/searchkick/relation_indexer.rb +155 -0
- data/lib/searchkick/results.rb +201 -82
- data/lib/searchkick/version.rb +1 -1
- data/lib/searchkick/where.rb +11 -0
- data/lib/searchkick.rb +269 -97
- data/lib/tasks/searchkick.rake +37 -0
- metadata +24 -178
- data/.gitignore +0 -22
- data/.travis.yml +0 -39
- data/Gemfile +0 -16
- data/Rakefile +0 -20
- data/benchmark/Gemfile +0 -23
- data/benchmark/benchmark.rb +0 -97
- data/lib/searchkick/logging.rb +0 -242
- data/lib/searchkick/tasks.rb +0 -33
- data/searchkick.gemspec +0 -28
- data/test/aggs_test.rb +0 -197
- data/test/autocomplete_test.rb +0 -75
- data/test/boost_test.rb +0 -202
- data/test/callbacks_test.rb +0 -59
- data/test/ci/before_install.sh +0 -17
- data/test/errors_test.rb +0 -19
- data/test/gemfiles/activerecord31.gemfile +0 -7
- data/test/gemfiles/activerecord32.gemfile +0 -7
- data/test/gemfiles/activerecord40.gemfile +0 -8
- data/test/gemfiles/activerecord41.gemfile +0 -8
- data/test/gemfiles/activerecord42.gemfile +0 -7
- data/test/gemfiles/activerecord50.gemfile +0 -7
- data/test/gemfiles/apartment.gemfile +0 -8
- data/test/gemfiles/cequel.gemfile +0 -8
- data/test/gemfiles/mongoid2.gemfile +0 -7
- data/test/gemfiles/mongoid3.gemfile +0 -6
- data/test/gemfiles/mongoid4.gemfile +0 -7
- data/test/gemfiles/mongoid5.gemfile +0 -7
- data/test/gemfiles/mongoid6.gemfile +0 -12
- data/test/gemfiles/nobrainer.gemfile +0 -8
- data/test/gemfiles/parallel_tests.gemfile +0 -8
- data/test/geo_shape_test.rb +0 -175
- data/test/highlight_test.rb +0 -78
- data/test/index_test.rb +0 -166
- data/test/inheritance_test.rb +0 -83
- data/test/marshal_test.rb +0 -8
- data/test/match_test.rb +0 -276
- data/test/misspellings_test.rb +0 -56
- data/test/model_test.rb +0 -42
- data/test/multi_search_test.rb +0 -36
- data/test/multi_tenancy_test.rb +0 -22
- data/test/order_test.rb +0 -46
- data/test/pagination_test.rb +0 -70
- data/test/partial_reindex_test.rb +0 -58
- data/test/query_test.rb +0 -35
- data/test/records_test.rb +0 -10
- data/test/reindex_test.rb +0 -64
- data/test/reindex_v2_job_test.rb +0 -32
- data/test/routing_test.rb +0 -23
- data/test/should_index_test.rb +0 -32
- data/test/similar_test.rb +0 -28
- data/test/sql_test.rb +0 -214
- data/test/suggest_test.rb +0 -95
- data/test/support/kaminari.yml +0 -21
- data/test/synonyms_test.rb +0 -67
- data/test/test_helper.rb +0 -567
- data/test/where_test.rb +0 -223
data/README.md
CHANGED
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
:rocket: Intelligent search made easy
|
4
4
|
|
5
|
-
Searchkick learns what
|
5
|
+
**Searchkick learns what your users are looking for.** As more people search, it gets smarter and the results get better. It’s friendly for developers - and magical for your users.
|
6
6
|
|
7
7
|
Searchkick handles:
|
8
8
|
|
@@ -10,7 +10,7 @@ Searchkick handles:
|
|
10
10
|
- special characters - `jalapeno` matches `jalapeño`
|
11
11
|
- extra whitespace - `dishwasher` matches `dish washer`
|
12
12
|
- misspellings - `zuchini` matches `zucchini`
|
13
|
-
- custom synonyms - `
|
13
|
+
- custom synonyms - `pop` matches `soda`
|
14
14
|
|
15
15
|
Plus:
|
16
16
|
|
@@ -19,47 +19,57 @@ Plus:
|
|
19
19
|
- easily personalize results for each user
|
20
20
|
- autocomplete
|
21
21
|
- “Did you mean” suggestions
|
22
|
-
-
|
22
|
+
- supports many languages
|
23
|
+
- works with Active Record and Mongoid
|
23
24
|
|
24
|
-
|
25
|
+
Check out [Searchjoy](https://github.com/ankane/searchjoy) for analytics and [Autosuggest](https://github.com/ankane/autosuggest) for query suggestions
|
25
26
|
|
26
27
|
:tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
|
27
28
|
|
28
|
-
[![Build Status](https://
|
29
|
+
[![Build Status](https://github.com/ankane/searchkick/workflows/build/badge.svg?branch=master)](https://github.com/ankane/searchkick/actions)
|
29
30
|
|
30
31
|
## Contents
|
31
32
|
|
32
33
|
- [Getting Started](#getting-started)
|
33
34
|
- [Querying](#querying)
|
34
35
|
- [Indexing](#indexing)
|
36
|
+
- [Intelligent Search](#intelligent-search)
|
35
37
|
- [Instant Search / Autocomplete](#instant-search--autocomplete)
|
36
38
|
- [Aggregations](#aggregations)
|
39
|
+
- [Testing](#testing)
|
37
40
|
- [Deployment](#deployment)
|
38
41
|
- [Performance](#performance)
|
39
|
-
- [
|
42
|
+
- [Advanced Search](#advanced)
|
40
43
|
- [Reference](#reference)
|
44
|
+
- [Contributing](#contributing)
|
41
45
|
|
42
46
|
## Getting Started
|
43
47
|
|
44
|
-
|
48
|
+
Install [Elasticsearch](https://www.elastic.co/downloads/elasticsearch) or [OpenSearch](https://opensearch.org/downloads.html). For Homebrew, use:
|
45
49
|
|
46
50
|
```sh
|
47
|
-
brew install elasticsearch
|
48
|
-
brew services start elasticsearch
|
51
|
+
brew install elastic/tap/elasticsearch-full
|
52
|
+
brew services start elasticsearch-full
|
53
|
+
# or
|
54
|
+
brew install opensearch
|
55
|
+
brew services start opensearch
|
49
56
|
```
|
50
57
|
|
51
|
-
Add
|
58
|
+
Add these lines to your application’s Gemfile:
|
52
59
|
|
53
60
|
```ruby
|
54
|
-
gem
|
61
|
+
gem "searchkick"
|
62
|
+
|
63
|
+
gem "elasticsearch" # select one
|
64
|
+
gem "opensearch-ruby" # select one
|
55
65
|
```
|
56
66
|
|
57
|
-
The latest version works with Elasticsearch
|
67
|
+
The latest version works with Elasticsearch 7 and 8 and OpenSearch 1 and 2. For Elasticsearch 6, use version 4.6.3 and [this readme](https://github.com/ankane/searchkick/blob/v4.6.3/README.md).
|
58
68
|
|
59
69
|
Add searchkick to models you want to search.
|
60
70
|
|
61
71
|
```ruby
|
62
|
-
class Product <
|
72
|
+
class Product < ApplicationRecord
|
63
73
|
searchkick
|
64
74
|
end
|
65
75
|
```
|
@@ -73,20 +83,20 @@ Product.reindex
|
|
73
83
|
And to query, use:
|
74
84
|
|
75
85
|
```ruby
|
76
|
-
products = Product.search
|
86
|
+
products = Product.search("apples")
|
77
87
|
products.each do |product|
|
78
88
|
puts product.name
|
79
89
|
end
|
80
90
|
```
|
81
91
|
|
82
|
-
Searchkick supports the complete [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html). As your search becomes more advanced, we recommend you use the [
|
92
|
+
Searchkick supports the complete [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) and [OpenSearch Search API](https://opensearch.org/docs/latest/opensearch/rest-api/search/). As your search becomes more advanced, we recommend you use the [search server DSL](#advanced) for maximum flexibility.
|
83
93
|
|
84
94
|
## Querying
|
85
95
|
|
86
96
|
Query like SQL
|
87
97
|
|
88
98
|
```ruby
|
89
|
-
Product.search
|
99
|
+
Product.search("apples", where: {in_stock: true}, limit: 10, offset: 50)
|
90
100
|
```
|
91
101
|
|
92
102
|
Search specific fields
|
@@ -99,14 +109,20 @@ Where
|
|
99
109
|
|
100
110
|
```ruby
|
101
111
|
where: {
|
102
|
-
expires_at: {gt: Time.now},
|
103
|
-
orders_count: 1..10,
|
104
|
-
aisle_id: [25, 30],
|
105
|
-
store_id: {not: 2},
|
106
|
-
aisle_id: {not: [25, 30]},
|
107
|
-
user_ids: {all: [1, 3]},
|
108
|
-
category:
|
109
|
-
|
112
|
+
expires_at: {gt: Time.now}, # lt, gte, lte also available
|
113
|
+
orders_count: 1..10, # equivalent to {gte: 1, lte: 10}
|
114
|
+
aisle_id: [25, 30], # in
|
115
|
+
store_id: {not: 2}, # not
|
116
|
+
aisle_id: {not: [25, 30]}, # not in
|
117
|
+
user_ids: {all: [1, 3]}, # all elements in array
|
118
|
+
category: {like: "%frozen%"}, # like
|
119
|
+
category: {ilike: "%frozen%"}, # ilike
|
120
|
+
category: /frozen .+/, # regexp
|
121
|
+
category: {prefix: "frozen"}, # prefix
|
122
|
+
store_id: {exists: true}, # exists
|
123
|
+
_or: [{in_stock: true}, {backordered: true}],
|
124
|
+
_and: [{in_stock: true}, {backordered: true}],
|
125
|
+
_not: {store_id: 1} # negate a condition
|
110
126
|
}
|
111
127
|
```
|
112
128
|
|
@@ -116,7 +132,7 @@ Order
|
|
116
132
|
order: {_score: :desc} # most relevant first - default
|
117
133
|
```
|
118
134
|
|
119
|
-
[All of these sort options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-
|
135
|
+
[All of these sort options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/sort-search-results.html)
|
120
136
|
|
121
137
|
Limit / offset
|
122
138
|
|
@@ -130,9 +146,11 @@ Select
|
|
130
146
|
select: [:name]
|
131
147
|
```
|
132
148
|
|
149
|
+
[These source filtering options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-fields.html#source-filtering)
|
150
|
+
|
133
151
|
### Results
|
134
152
|
|
135
|
-
Searches return a `Searchkick::
|
153
|
+
Searches return a `Searchkick::Relation` object. This responds like an array to most methods.
|
136
154
|
|
137
155
|
```ruby
|
138
156
|
results = Product.search("milk")
|
@@ -141,7 +159,7 @@ results.any?
|
|
141
159
|
results.each { |result| ... }
|
142
160
|
```
|
143
161
|
|
144
|
-
By default, ids are fetched from
|
162
|
+
By default, ids are fetched from the search server and records are fetched from your database. To fetch everything from the search server, use:
|
145
163
|
|
146
164
|
```ruby
|
147
165
|
Product.search("apples", load: false)
|
@@ -159,12 +177,14 @@ Get the time the search took (in milliseconds)
|
|
159
177
|
results.took
|
160
178
|
```
|
161
179
|
|
162
|
-
Get the full response from
|
180
|
+
Get the full response from the search server
|
163
181
|
|
164
182
|
```ruby
|
165
183
|
results.response
|
166
184
|
```
|
167
185
|
|
186
|
+
**Note:** By default, Elasticsearch and OpenSearch [limit paging](#deep-paging) to the first 10,000 results for performance. This applies to the total count as well.
|
187
|
+
|
168
188
|
### Boosting
|
169
189
|
|
170
190
|
Boost important fields
|
@@ -188,14 +208,23 @@ boost_where: {user_id: {value: 1, factor: 100}} # default factor is 1000
|
|
188
208
|
boost_where: {user_id: [{value: 1, factor: 100}, {value: 2, factor: 200}]}
|
189
209
|
```
|
190
210
|
|
191
|
-
|
211
|
+
Boost by recency
|
212
|
+
|
213
|
+
```ruby
|
214
|
+
boost_by_recency: {created_at: {scale: "7d", decay: 0.5}}
|
215
|
+
```
|
216
|
+
|
217
|
+
You can also boost by:
|
218
|
+
|
219
|
+
- [Conversions](#intelligent-search)
|
220
|
+
- [Distance](#boost-by-distance)
|
192
221
|
|
193
222
|
### Get Everything
|
194
223
|
|
195
224
|
Use a `*` for the query.
|
196
225
|
|
197
226
|
```ruby
|
198
|
-
Product.search
|
227
|
+
Product.search("*")
|
199
228
|
```
|
200
229
|
|
201
230
|
### Pagination
|
@@ -204,7 +233,7 @@ Plays nicely with kaminari and will_paginate.
|
|
204
233
|
|
205
234
|
```ruby
|
206
235
|
# controller
|
207
|
-
@products = Product.search
|
236
|
+
@products = Product.search("milk", page: params[:page], per_page: 20)
|
208
237
|
```
|
209
238
|
|
210
239
|
View with kaminari
|
@@ -224,19 +253,19 @@ View with will_paginate
|
|
224
253
|
By default, results must match all words in the query.
|
225
254
|
|
226
255
|
```ruby
|
227
|
-
Product.search
|
256
|
+
Product.search("fresh honey") # fresh AND honey
|
228
257
|
```
|
229
258
|
|
230
259
|
To change this, use:
|
231
260
|
|
232
261
|
```ruby
|
233
|
-
Product.search
|
262
|
+
Product.search("fresh honey", operator: "or") # fresh OR honey
|
234
263
|
```
|
235
264
|
|
236
265
|
By default, results must match the entire word - `back` will not match `backpack`. You can change this behavior with:
|
237
266
|
|
238
267
|
```ruby
|
239
|
-
class Product <
|
268
|
+
class Product < ApplicationRecord
|
240
269
|
searchkick word_start: [:name]
|
241
270
|
end
|
242
271
|
```
|
@@ -244,27 +273,35 @@ end
|
|
244
273
|
And to search (after you reindex):
|
245
274
|
|
246
275
|
```ruby
|
247
|
-
Product.search
|
276
|
+
Product.search("back", fields: [:name], match: :word_start)
|
248
277
|
```
|
249
278
|
|
250
279
|
Available options are:
|
251
280
|
|
281
|
+
Option | Matches | Example
|
282
|
+
--- | --- | ---
|
283
|
+
`:word` | entire word | `apple` matches `apple`
|
284
|
+
`:word_start` | start of word | `app` matches `apple`
|
285
|
+
`:word_middle` | any part of word | `ppl` matches `apple`
|
286
|
+
`:word_end` | end of word | `ple` matches `apple`
|
287
|
+
`:text_start` | start of text | `gre` matches `green apple`, `app` does not match
|
288
|
+
`:text_middle` | any part of text | `een app` matches `green apple`
|
289
|
+
`:text_end` | end of text | `ple` matches `green apple`, `een` does not match
|
290
|
+
|
291
|
+
The default is `:word`. The most matches will happen with `:word_middle`.
|
292
|
+
|
293
|
+
To specify different matching for different fields, use:
|
294
|
+
|
252
295
|
```ruby
|
253
|
-
:
|
254
|
-
:word_start
|
255
|
-
:word_middle
|
256
|
-
:word_end
|
257
|
-
:text_start
|
258
|
-
:text_middle
|
259
|
-
:text_end
|
296
|
+
Product.search(query, fields: [{name: :word_start}, {brand: :word_middle}])
|
260
297
|
```
|
261
298
|
|
262
299
|
### Exact Matches
|
263
300
|
|
264
|
-
To match a field exactly (case-
|
301
|
+
To match a field exactly (case-sensitive), use:
|
265
302
|
|
266
303
|
```ruby
|
267
|
-
|
304
|
+
Product.search(query, fields: [{name: :exact}])
|
268
305
|
```
|
269
306
|
|
270
307
|
### Phrase Matches
|
@@ -272,85 +309,128 @@ User.search query, fields: [{email: :exact}, :name]
|
|
272
309
|
To only match the exact order, use:
|
273
310
|
|
274
311
|
```ruby
|
275
|
-
|
312
|
+
Product.search("fresh honey", match: :phrase)
|
276
313
|
```
|
277
314
|
|
278
|
-
### Language
|
315
|
+
### Stemming and Language
|
316
|
+
|
317
|
+
Searchkick stems words by default for better matching. `apple` and `apples` both stem to `appl`, so searches for either term will have the same matches.
|
279
318
|
|
280
319
|
Searchkick defaults to English for stemming. To change this, use:
|
281
320
|
|
282
321
|
```ruby
|
283
|
-
class Product <
|
322
|
+
class Product < ApplicationRecord
|
284
323
|
searchkick language: "german"
|
285
324
|
end
|
286
325
|
```
|
287
326
|
|
288
|
-
|
327
|
+
See the [list of languages](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-stemmer-tokenfilter.html#analysis-stemmer-tokenfilter-configure-parms). A few languages require plugins:
|
289
328
|
|
290
|
-
|
329
|
+
- `chinese` - [analysis-ik plugin](https://github.com/medcl/elasticsearch-analysis-ik)
|
330
|
+
- `chinese2` - [analysis-smartcn plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html)
|
331
|
+
- `japanese` - [analysis-kuromoji plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html)
|
332
|
+
- `korean` - [analysis-openkoreantext plugin](https://github.com/open-korean-text/elasticsearch-analysis-openkoreantext)
|
333
|
+
- `korean2` - [analysis-nori plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-nori.html)
|
334
|
+
- `polish` - [analysis-stempel plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-stempel.html)
|
335
|
+
- `ukrainian` - [analysis-ukrainian plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/7.4/analysis-ukrainian.html)
|
336
|
+
- `vietnamese` - [analysis-vietnamese plugin](https://github.com/duydo/elasticsearch-analysis-vietnamese)
|
337
|
+
|
338
|
+
You can also use a Hunspell dictionary for stemming.
|
339
|
+
|
340
|
+
```ruby
|
341
|
+
class Product < ApplicationRecord
|
342
|
+
searchkick stemmer: {type: "hunspell", locale: "en_US"}
|
343
|
+
end
|
344
|
+
```
|
345
|
+
|
346
|
+
Disable stemming with:
|
291
347
|
|
292
348
|
```ruby
|
293
|
-
class
|
294
|
-
searchkick
|
349
|
+
class Image < ApplicationRecord
|
350
|
+
searchkick stem: false
|
295
351
|
end
|
296
352
|
```
|
297
353
|
|
298
|
-
|
354
|
+
Exclude certain words from stemming with:
|
355
|
+
|
356
|
+
```ruby
|
357
|
+
class Image < ApplicationRecord
|
358
|
+
searchkick stem_exclusion: ["apples"]
|
359
|
+
end
|
360
|
+
```
|
299
361
|
|
300
|
-
|
362
|
+
Or change how words are stemmed:
|
301
363
|
|
302
364
|
```ruby
|
303
|
-
|
365
|
+
class Image < ApplicationRecord
|
366
|
+
searchkick stemmer_override: ["apples => other"]
|
367
|
+
end
|
304
368
|
```
|
305
369
|
|
306
|
-
|
370
|
+
### Synonyms
|
307
371
|
|
308
372
|
```ruby
|
309
|
-
|
373
|
+
class Product < ApplicationRecord
|
374
|
+
searchkick search_synonyms: [["pop", "soda"], ["burger", "hamburger"]]
|
375
|
+
end
|
310
376
|
```
|
311
377
|
|
312
|
-
|
378
|
+
Call `Product.reindex` after changing synonyms. Synonyms are applied at search time before stemming, and can be a single word or multiple words.
|
313
379
|
|
314
|
-
|
380
|
+
For directional synonyms, use:
|
315
381
|
|
316
382
|
```ruby
|
317
|
-
|
318
|
-
|
319
|
-
scope :search_import, -> { includes(:tags) }
|
383
|
+
search_synonyms: ["lightbulb => halogenlamp"]
|
384
|
+
```
|
320
385
|
|
321
|
-
|
322
|
-
|
323
|
-
|
324
|
-
|
325
|
-
|
386
|
+
### Dynamic Synonyms
|
387
|
+
|
388
|
+
The above approach works well when your synonym list is static, but in practice, this is often not the case. When you analyze search conversions, you often want to add new synonyms without a full reindex.
|
389
|
+
|
390
|
+
#### Elasticsearch 7.3+ and OpenSearch
|
391
|
+
|
392
|
+
For Elasticsearch 7.3+ and OpenSearch, we recommend placing synonyms in a file on the search server (in the `config` directory). This allows you to reload synonyms without reindexing.
|
393
|
+
|
394
|
+
```txt
|
395
|
+
pop, soda
|
396
|
+
burger, hamburger
|
397
|
+
```
|
398
|
+
|
399
|
+
Then use:
|
400
|
+
|
401
|
+
```ruby
|
402
|
+
class Product < ApplicationRecord
|
403
|
+
searchkick search_synonyms: "synonyms.txt"
|
326
404
|
end
|
327
405
|
```
|
328
406
|
|
329
|
-
|
407
|
+
And reload with:
|
330
408
|
|
331
409
|
```ruby
|
332
|
-
Product.
|
410
|
+
Product.search_index.reload_synonyms
|
333
411
|
```
|
334
412
|
|
335
|
-
|
413
|
+
#### Elasticsearch < 7.3
|
336
414
|
|
337
|
-
|
415
|
+
You can use a library like [ActsAsTaggableOn](https://github.com/mbleigh/acts-as-taggable-on) and do:
|
338
416
|
|
339
|
-
|
417
|
+
```ruby
|
418
|
+
class Product < ApplicationRecord
|
419
|
+
acts_as_taggable
|
420
|
+
scope :search_import, -> { includes(:tags) }
|
340
421
|
|
341
|
-
|
342
|
-
|
343
|
-
|
344
|
-
|
345
|
-
|
422
|
+
def search_data
|
423
|
+
{
|
424
|
+
name_tagged: "#{name} #{tags.map(&:name).join(" ")}"
|
425
|
+
}
|
426
|
+
end
|
427
|
+
end
|
346
428
|
```
|
347
429
|
|
348
|
-
|
430
|
+
Search with:
|
349
431
|
|
350
432
|
```ruby
|
351
|
-
|
352
|
-
searchkick wordnet: true
|
353
|
-
end
|
433
|
+
Product.search(query, fields: [:name_tagged])
|
354
434
|
```
|
355
435
|
|
356
436
|
### Misspellings
|
@@ -360,13 +440,13 @@ By default, Searchkick handles misspelled queries by returning results with an [
|
|
360
440
|
You can change this with:
|
361
441
|
|
362
442
|
```ruby
|
363
|
-
Product.search
|
443
|
+
Product.search("zucini", misspellings: {edit_distance: 2}) # zucchini
|
364
444
|
```
|
365
445
|
|
366
446
|
To prevent poor precision and improve performance for correctly spelled queries (which should be a majority for most applications), Searchkick can first perform a search without misspellings, and if there are too few results, perform another with them.
|
367
447
|
|
368
448
|
```ruby
|
369
|
-
Product.search
|
449
|
+
Product.search("zuchini", misspellings: {below: 5})
|
370
450
|
```
|
371
451
|
|
372
452
|
If there are fewer than 5 results, a 2nd search is performed with misspellings enabled. The result of this query is returned.
|
@@ -374,15 +454,23 @@ If there are fewer than 5 results, a 2nd search is performed with misspellings e
|
|
374
454
|
Turn off misspellings with:
|
375
455
|
|
376
456
|
```ruby
|
377
|
-
Product.search
|
457
|
+
Product.search("zuchini", misspellings: false) # no zucchini
|
458
|
+
```
|
459
|
+
|
460
|
+
Specify which fields can include misspellings with:
|
461
|
+
|
462
|
+
```ruby
|
463
|
+
Product.search("zucini", fields: [:name, :color], misspellings: {fields: [:name]})
|
378
464
|
```
|
379
465
|
|
466
|
+
> When doing this, you must also specify fields to search
|
467
|
+
|
380
468
|
### Bad Matches
|
381
469
|
|
382
470
|
If a user searches `butter`, they may also get results for `peanut butter`. To prevent this, use:
|
383
471
|
|
384
472
|
```ruby
|
385
|
-
Product.search
|
473
|
+
Product.search("butter", exclude: ["peanut butter"])
|
386
474
|
```
|
387
475
|
|
388
476
|
You can map queries and terms to exclude with:
|
@@ -393,7 +481,13 @@ exclude_queries = {
|
|
393
481
|
"cream" => ["ice cream", "whipped cream"]
|
394
482
|
}
|
395
483
|
|
396
|
-
Product.search
|
484
|
+
Product.search(query, exclude: exclude_queries[query])
|
485
|
+
```
|
486
|
+
|
487
|
+
You can demote results by boosting by a factor less than one:
|
488
|
+
|
489
|
+
```ruby
|
490
|
+
Product.search("butter", boost_where: {category: {value: "pantry", factor: 0.5}})
|
397
491
|
```
|
398
492
|
|
399
493
|
### Emoji
|
@@ -403,13 +497,13 @@ Search :ice_cream::cake: and get `ice cream cake`!
|
|
403
497
|
Add this line to your application’s Gemfile:
|
404
498
|
|
405
499
|
```ruby
|
406
|
-
gem
|
500
|
+
gem "gemoji-parser"
|
407
501
|
```
|
408
502
|
|
409
503
|
And use:
|
410
504
|
|
411
505
|
```ruby
|
412
|
-
Product.search
|
506
|
+
Product.search("🍨🍰", emoji: true)
|
413
507
|
```
|
414
508
|
|
415
509
|
## Indexing
|
@@ -417,7 +511,7 @@ Product.search "🍨🍰", emoji: true
|
|
417
511
|
Control what data is indexed with the `search_data` method. Call `Product.reindex` after changing this method.
|
418
512
|
|
419
513
|
```ruby
|
420
|
-
class Product <
|
514
|
+
class Product < ApplicationRecord
|
421
515
|
belongs_to :department
|
422
516
|
|
423
517
|
def search_data
|
@@ -433,17 +527,15 @@ end
|
|
433
527
|
Searchkick uses `find_in_batches` to import documents. To eager load associations, use the `search_import` scope.
|
434
528
|
|
435
529
|
```ruby
|
436
|
-
class Product <
|
530
|
+
class Product < ApplicationRecord
|
437
531
|
scope :search_import, -> { includes(:department) }
|
438
532
|
end
|
439
533
|
```
|
440
534
|
|
441
|
-
By default, all records are indexed. To control which records are indexed, use the `should_index?` method
|
535
|
+
By default, all records are indexed. To control which records are indexed, use the `should_index?` method.
|
442
536
|
|
443
537
|
```ruby
|
444
|
-
class Product <
|
445
|
-
scope :search_import, -> { where(active: true) }
|
446
|
-
|
538
|
+
class Product < ApplicationRecord
|
447
539
|
def should_index?
|
448
540
|
active # only index active records
|
449
541
|
end
|
@@ -470,11 +562,11 @@ For large data sets, try [parallel reindexing](#parallel-reindexing).
|
|
470
562
|
|
471
563
|
- app starts
|
472
564
|
|
473
|
-
###
|
565
|
+
### Strategies
|
474
566
|
|
475
567
|
There are four strategies for keeping the index synced with your database.
|
476
568
|
|
477
|
-
1.
|
569
|
+
1. Inline (default)
|
478
570
|
|
479
571
|
Anytime a record is inserted, updated, or deleted
|
480
572
|
|
@@ -483,12 +575,12 @@ There are four strategies for keeping the index synced with your database.
|
|
483
575
|
Use background jobs for better performance
|
484
576
|
|
485
577
|
```ruby
|
486
|
-
class Product <
|
578
|
+
class Product < ApplicationRecord
|
487
579
|
searchkick callbacks: :async
|
488
580
|
end
|
489
581
|
```
|
490
582
|
|
491
|
-
|
583
|
+
Jobs are added to a queue named `searchkick`.
|
492
584
|
|
493
585
|
3. Queuing
|
494
586
|
|
@@ -499,16 +591,24 @@ There are four strategies for keeping the index synced with your database.
|
|
499
591
|
Turn off automatic syncing
|
500
592
|
|
501
593
|
```ruby
|
502
|
-
class Product <
|
594
|
+
class Product < ApplicationRecord
|
503
595
|
searchkick callbacks: false
|
504
596
|
end
|
505
597
|
```
|
506
598
|
|
599
|
+
And reindex a record or relation manually.
|
600
|
+
|
601
|
+
```ruby
|
602
|
+
product.reindex
|
603
|
+
# or
|
604
|
+
store.products.reindex(mode: :async)
|
605
|
+
```
|
606
|
+
|
507
607
|
You can also do bulk updates.
|
508
608
|
|
509
609
|
```ruby
|
510
610
|
Searchkick.callbacks(:bulk) do
|
511
|
-
|
611
|
+
Product.find_each(&:update_fields)
|
512
612
|
end
|
513
613
|
```
|
514
614
|
|
@@ -516,83 +616,186 @@ Or temporarily skip updates.
|
|
516
616
|
|
517
617
|
```ruby
|
518
618
|
Searchkick.callbacks(false) do
|
519
|
-
|
619
|
+
Product.find_each(&:update_fields)
|
520
620
|
end
|
521
621
|
```
|
522
622
|
|
523
|
-
|
623
|
+
Or override the model’s strategy.
|
624
|
+
|
625
|
+
```ruby
|
626
|
+
product.reindex(mode: :async) # :inline or :queue
|
627
|
+
```
|
628
|
+
|
629
|
+
### Associations
|
524
630
|
|
525
631
|
Data is **not** automatically synced when an association is updated. If this is desired, add a callback to reindex:
|
526
632
|
|
527
633
|
```ruby
|
528
|
-
class Image <
|
634
|
+
class Image < ApplicationRecord
|
529
635
|
belongs_to :product
|
530
636
|
|
531
637
|
after_commit :reindex_product
|
532
638
|
|
533
639
|
def reindex_product
|
534
|
-
product.reindex
|
640
|
+
product.reindex
|
535
641
|
end
|
536
642
|
end
|
537
643
|
```
|
538
644
|
|
539
|
-
###
|
645
|
+
### Default Scopes
|
540
646
|
|
541
|
-
|
542
|
-
|
543
|
-
[Searchjoy](https://github.com/ankane/searchjoy) makes it easy.
|
647
|
+
If you have a default scope that filters records, use the `should_index?` method to exclude them from indexing:
|
544
648
|
|
545
649
|
```ruby
|
546
|
-
Product
|
650
|
+
class Product < ApplicationRecord
|
651
|
+
default_scope { where(deleted_at: nil) }
|
652
|
+
|
653
|
+
def should_index?
|
654
|
+
deleted_at.nil?
|
655
|
+
end
|
656
|
+
end
|
547
657
|
```
|
548
658
|
|
549
|
-
|
659
|
+
If you want to index and search filtered records, set:
|
550
660
|
|
551
|
-
|
661
|
+
```ruby
|
662
|
+
class Product < ApplicationRecord
|
663
|
+
searchkick unscope: true
|
664
|
+
end
|
665
|
+
```
|
552
666
|
|
553
|
-
|
554
|
-
- top searches with no results
|
667
|
+
## Intelligent Search
|
555
668
|
|
556
|
-
|
669
|
+
The best starting point to improve your search **by far** is to track searches and conversions. [Searchjoy](https://github.com/ankane/searchjoy) makes it easy.
|
557
670
|
|
558
|
-
|
671
|
+
```ruby
|
672
|
+
Product.search("apple", track: {user_id: current_user.id})
|
673
|
+
```
|
559
674
|
|
560
|
-
|
675
|
+
[See the docs](https://github.com/ankane/searchjoy) for how to install and use. Focus on top searches with a low conversion rate.
|
561
676
|
|
562
|
-
|
677
|
+
Searchkick can then use the conversion data to learn what users are looking for. If a user searches for “ice cream” and adds Ben & Jerry’s Chunky Monkey to the cart (our conversion metric at Instacart), that item gets a little more weight for similar searches. This can make a huge difference on the quality of your search.
|
563
678
|
|
564
|
-
|
679
|
+
Add conversion data with:
|
565
680
|
|
566
681
|
```ruby
|
567
|
-
class Product <
|
568
|
-
has_many :
|
682
|
+
class Product < ApplicationRecord
|
683
|
+
has_many :conversions, class_name: "Searchjoy::Conversion", as: :convertable
|
684
|
+
has_many :searches, class_name: "Searchjoy::Search", through: :conversions
|
569
685
|
|
570
|
-
searchkick conversions: [
|
686
|
+
searchkick conversions: [:conversions] # name of field
|
571
687
|
|
572
688
|
def search_data
|
573
689
|
{
|
574
690
|
name: name,
|
575
|
-
conversions: searches.group(:query).
|
691
|
+
conversions: searches.group(:query).distinct.count(:user_id)
|
576
692
|
# {"ice cream" => 234, "chocolate" => 67, "cream" => 2}
|
577
693
|
}
|
578
694
|
end
|
579
695
|
end
|
580
696
|
```
|
581
697
|
|
582
|
-
Reindex and set up a cron job to add new conversions daily.
|
698
|
+
Reindex and set up a cron job to add new conversions daily. For zero downtime deployment, temporarily set `conversions: false` in your search calls until the data is reindexed.
|
699
|
+
|
700
|
+
### Performant Conversions
|
701
|
+
|
702
|
+
A performant way to do conversions is to cache them to prevent N+1 queries. For Postgres, create a migration with:
|
703
|
+
|
704
|
+
```ruby
|
705
|
+
add_column :products, :search_conversions, :jsonb
|
706
|
+
```
|
707
|
+
|
708
|
+
For MySQL, use `:json`, and for others, use `:text` with a [JSON serializer](https://api.rubyonrails.org/classes/ActiveRecord/AttributeMethods/Serialization/ClassMethods.html).
|
709
|
+
|
710
|
+
Next, update your model. Create a separate method for conversion data so you can use [partial reindexing](#partial-reindexing).
|
711
|
+
|
712
|
+
```ruby
|
713
|
+
class Product < ApplicationRecord
|
714
|
+
searchkick conversions: [:conversions]
|
715
|
+
|
716
|
+
def search_data
|
717
|
+
{
|
718
|
+
name: name,
|
719
|
+
category: category
|
720
|
+
}.merge(conversions_data)
|
721
|
+
end
|
722
|
+
|
723
|
+
def conversions_data
|
724
|
+
{
|
725
|
+
conversions: search_conversions || {}
|
726
|
+
}
|
727
|
+
end
|
728
|
+
end
|
729
|
+
```
|
730
|
+
|
731
|
+
Deploy and reindex your data. For zero downtime deployment, temporarily set `conversions: false` in your search calls until the data is reindexed.
|
732
|
+
|
733
|
+
```ruby
|
734
|
+
Product.reindex
|
735
|
+
```
|
736
|
+
|
737
|
+
Then, create a job to update the conversions column and reindex records with new conversions. Here’s one you can use for Searchjoy:
|
738
|
+
|
739
|
+
```ruby
|
740
|
+
class UpdateConversionsJob < ApplicationJob
|
741
|
+
def perform(class_name, since: nil, update: true, reindex: true)
|
742
|
+
model = Searchkick.load_model(class_name)
|
743
|
+
|
744
|
+
# get records that have a recent conversion
|
745
|
+
recently_converted_ids =
|
746
|
+
Searchjoy::Conversion.where(convertable_type: class_name).where(created_at: since..)
|
747
|
+
.order(:convertable_id).distinct.pluck(:convertable_id)
|
748
|
+
|
749
|
+
# split into batches
|
750
|
+
recently_converted_ids.in_groups_of(1000, false) do |ids|
|
751
|
+
if update
|
752
|
+
# fetch conversions
|
753
|
+
conversions =
|
754
|
+
Searchjoy::Conversion.where(convertable_id: ids, convertable_type: class_name)
|
755
|
+
.joins(:search).where.not(searchjoy_searches: {user_id: nil})
|
756
|
+
.group(:convertable_id, :query).distinct.count(:user_id)
|
757
|
+
|
758
|
+
# group by record
|
759
|
+
conversions_by_record = {}
|
760
|
+
conversions.each do |(id, query), count|
|
761
|
+
(conversions_by_record[id] ||= {})[query] = count
|
762
|
+
end
|
763
|
+
|
764
|
+
# update conversions column
|
765
|
+
model.transaction do
|
766
|
+
conversions_by_record.each do |id, conversions|
|
767
|
+
model.where(id: id).update_all(search_conversions: conversions)
|
768
|
+
end
|
769
|
+
end
|
770
|
+
end
|
771
|
+
|
772
|
+
if reindex
|
773
|
+
# reindex conversions data
|
774
|
+
model.where(id: ids).reindex(:conversions_data)
|
775
|
+
end
|
776
|
+
end
|
777
|
+
end
|
778
|
+
end
|
779
|
+
```
|
780
|
+
|
781
|
+
Run the job:
|
583
782
|
|
584
783
|
```ruby
|
585
|
-
|
784
|
+
UpdateConversionsJob.perform_now("Product")
|
586
785
|
```
|
587
786
|
|
588
|
-
|
787
|
+
And set it up to run daily.
|
589
788
|
|
590
|
-
|
789
|
+
```ruby
|
790
|
+
UpdateConversionsJob.perform_later("Product", since: 1.day.ago)
|
791
|
+
```
|
792
|
+
|
793
|
+
## Personalized Results
|
591
794
|
|
592
795
|
Order results differently for each user. For example, show a user’s previously purchased products before other results.
|
593
796
|
|
594
797
|
```ruby
|
595
|
-
class Product <
|
798
|
+
class Product < ApplicationRecord
|
596
799
|
def search_data
|
597
800
|
{
|
598
801
|
name: name,
|
@@ -605,23 +808,23 @@ end
|
|
605
808
|
Reindex and search with:
|
606
809
|
|
607
810
|
```ruby
|
608
|
-
Product.search
|
811
|
+
Product.search("milk", boost_where: {orderer_ids: current_user.id})
|
609
812
|
```
|
610
813
|
|
611
|
-
|
814
|
+
## Instant Search / Autocomplete
|
612
815
|
|
613
816
|
Autocomplete predicts what a user will type, making the search experience faster and easier.
|
614
817
|
|
615
|
-
![Autocomplete](https://
|
818
|
+
![Autocomplete](https://gist.github.com/ankane/b6988db2802aca68a589b31e41b44195/raw/40febe948427e5bc53ec4e5dc248822855fef76f/autocomplete.png)
|
616
819
|
|
617
|
-
**Note:** To autocomplete on
|
820
|
+
**Note:** To autocomplete on search terms rather than results, check out [Autosuggest](https://github.com/ankane/autosuggest).
|
618
821
|
|
619
822
|
**Note 2:** If you only have a few thousand records, don’t use Searchkick for autocomplete. It’s *much* faster to load all records into JavaScript and autocomplete there (eliminates network requests).
|
620
823
|
|
621
824
|
First, specify which fields use this feature. This is necessary since autocomplete can increase the index size significantly, but don’t worry - this gives you blazing faster queries.
|
622
825
|
|
623
826
|
```ruby
|
624
|
-
class Movie <
|
827
|
+
class Movie < ApplicationRecord
|
625
828
|
searchkick word_start: [:title, :director]
|
626
829
|
end
|
627
830
|
```
|
@@ -629,10 +832,10 @@ end
|
|
629
832
|
Reindex and search with:
|
630
833
|
|
631
834
|
```ruby
|
632
|
-
Movie.search
|
835
|
+
Movie.search("jurassic pa", fields: [:title], match: :word_start)
|
633
836
|
```
|
634
837
|
|
635
|
-
Typically, you want to use a JavaScript library like [typeahead.js](
|
838
|
+
Typically, you want to use a JavaScript library like [typeahead.js](https://twitter.github.io/typeahead.js/) or [jQuery UI](https://jqueryui.com/autocomplete/).
|
636
839
|
|
637
840
|
#### Here’s how to make it work with Rails
|
638
841
|
|
@@ -676,12 +879,12 @@ Then add the search box and JavaScript code to a view.
|
|
676
879
|
</script>
|
677
880
|
```
|
678
881
|
|
679
|
-
|
882
|
+
## Suggestions
|
680
883
|
|
681
|
-
![Suggest](https://
|
884
|
+
![Suggest](https://gist.github.com/ankane/b6988db2802aca68a589b31e41b44195/raw/40febe948427e5bc53ec4e5dc248822855fef76f/recursion.png)
|
682
885
|
|
683
886
|
```ruby
|
684
|
-
class Product <
|
887
|
+
class Product < ApplicationRecord
|
685
888
|
searchkick suggest: [:name] # fields to generate suggestions
|
686
889
|
end
|
687
890
|
```
|
@@ -689,51 +892,51 @@ end
|
|
689
892
|
Reindex and search with:
|
690
893
|
|
691
894
|
```ruby
|
692
|
-
products = Product.search
|
895
|
+
products = Product.search("peantu butta", suggest: true)
|
693
896
|
products.suggestions # ["peanut butter"]
|
694
897
|
```
|
695
898
|
|
696
|
-
|
899
|
+
## Aggregations
|
697
900
|
|
698
901
|
[Aggregations](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html) provide aggregated search data.
|
699
902
|
|
700
|
-
![Aggregations](https://
|
903
|
+
![Aggregations](https://gist.github.com/ankane/b6988db2802aca68a589b31e41b44195/raw/40febe948427e5bc53ec4e5dc248822855fef76f/facets.png)
|
701
904
|
|
702
905
|
```ruby
|
703
|
-
products = Product.search
|
906
|
+
products = Product.search("chuck taylor", aggs: [:product_type, :gender, :brand])
|
704
907
|
products.aggs
|
705
908
|
```
|
706
909
|
|
707
910
|
By default, `where` conditions apply to aggregations.
|
708
911
|
|
709
912
|
```ruby
|
710
|
-
Product.search
|
913
|
+
Product.search("wingtips", where: {color: "brandy"}, aggs: [:size])
|
711
914
|
# aggregations for brandy wingtips are returned
|
712
915
|
```
|
713
916
|
|
714
917
|
Change this with:
|
715
918
|
|
716
919
|
```ruby
|
717
|
-
Product.search
|
920
|
+
Product.search("wingtips", where: {color: "brandy"}, aggs: [:size], smart_aggs: false)
|
718
921
|
# aggregations for all wingtips are returned
|
719
922
|
```
|
720
923
|
|
721
924
|
Set `where` conditions for each aggregation separately with:
|
722
925
|
|
723
926
|
```ruby
|
724
|
-
Product.search
|
927
|
+
Product.search("wingtips", aggs: {size: {where: {color: "brandy"}}})
|
725
928
|
```
|
726
929
|
|
727
930
|
Limit
|
728
931
|
|
729
932
|
```ruby
|
730
|
-
Product.search
|
933
|
+
Product.search("apples", aggs: {store_id: {limit: 10}})
|
731
934
|
```
|
732
935
|
|
733
936
|
Order
|
734
937
|
|
735
938
|
```ruby
|
736
|
-
Product.search
|
939
|
+
Product.search("wingtips", aggs: {color: {order: {"_key" => "asc"}}}) # alphabetically
|
737
940
|
```
|
738
941
|
|
739
942
|
[All of these options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-order)
|
@@ -742,84 +945,39 @@ Ranges
|
|
742
945
|
|
743
946
|
```ruby
|
744
947
|
price_ranges = [{to: 20}, {from: 20, to: 50}, {from: 50}]
|
745
|
-
Product.search
|
948
|
+
Product.search("*", aggs: {price: {ranges: price_ranges}})
|
746
949
|
```
|
747
950
|
|
748
951
|
Minimum document count
|
749
952
|
|
750
953
|
```ruby
|
751
|
-
Product.search
|
954
|
+
Product.search("apples", aggs: {store_id: {min_doc_count: 2}})
|
752
955
|
```
|
753
956
|
|
754
|
-
|
957
|
+
Script support
|
755
958
|
|
756
959
|
```ruby
|
757
|
-
Product.search
|
960
|
+
Product.search("*", aggs: {color: {script: {source: "'Color: ' + _value"}}})
|
758
961
|
```
|
759
962
|
|
760
|
-
|
761
|
-
|
762
|
-
1. Replace `facets` with `aggs` in searches. **Note:** Stats facets are not supported at this time.
|
763
|
-
|
764
|
-
```ruby
|
765
|
-
products = Product.search "chuck taylor", facets: [:brand]
|
766
|
-
# to
|
767
|
-
products = Product.search "chuck taylor", aggs: [:brand]
|
768
|
-
```
|
769
|
-
|
770
|
-
2. Replace the `facets` method with `aggs` for results.
|
771
|
-
|
772
|
-
```ruby
|
773
|
-
products.facets
|
774
|
-
# to
|
775
|
-
products.aggs
|
776
|
-
```
|
777
|
-
|
778
|
-
The keys in results differ slightly. Instead of:
|
779
|
-
|
780
|
-
```json
|
781
|
-
{
|
782
|
-
"_type":"terms",
|
783
|
-
"missing":0,
|
784
|
-
"total":45,
|
785
|
-
"other":34,
|
786
|
-
"terms":[
|
787
|
-
{"term":14.0,"count":11}
|
788
|
-
]
|
789
|
-
}
|
790
|
-
```
|
791
|
-
|
792
|
-
You get:
|
793
|
-
|
794
|
-
```json
|
795
|
-
{
|
796
|
-
"doc_count":45,
|
797
|
-
"doc_count_error_upper_bound":0,
|
798
|
-
"sum_other_doc_count":34,
|
799
|
-
"buckets":[
|
800
|
-
{"key":14.0,"doc_count":11}
|
801
|
-
]
|
802
|
-
}
|
803
|
-
```
|
804
|
-
|
805
|
-
Update your application to handle this.
|
963
|
+
Date histogram
|
806
964
|
|
807
|
-
|
965
|
+
```ruby
|
966
|
+
Product.search("pear", aggs: {products_per_year: {date_histogram: {field: :created_at, interval: :year}}})
|
967
|
+
```
|
808
968
|
|
809
|
-
|
969
|
+
For other aggregation types, including sub-aggregations, use `body_options`:
|
810
970
|
|
811
|
-
|
812
|
-
|
813
|
-
|
814
|
-
aggs: {date_field: {date_ranges: date_ranges}}
|
815
|
-
```
|
971
|
+
```ruby
|
972
|
+
Product.search("orange", body_options: {aggs: {price: {histogram: {field: :price, interval: 10}}}})
|
973
|
+
```
|
816
974
|
|
817
|
-
|
975
|
+
## Highlight
|
818
976
|
|
819
977
|
Specify which fields to index with highlighting.
|
820
978
|
|
821
979
|
```ruby
|
822
|
-
class
|
980
|
+
class Band < ApplicationRecord
|
823
981
|
searchkick highlight: [:name]
|
824
982
|
end
|
825
983
|
```
|
@@ -827,40 +985,47 @@ end
|
|
827
985
|
Highlight the search query in the results.
|
828
986
|
|
829
987
|
```ruby
|
830
|
-
bands = Band.search
|
988
|
+
bands = Band.search("cinema", highlight: true)
|
831
989
|
```
|
832
990
|
|
833
|
-
**Note:** The `fields` option is required, unless highlight options are given - see below.
|
834
|
-
|
835
991
|
View the highlighted fields with:
|
836
992
|
|
837
993
|
```ruby
|
838
|
-
bands.each do |band|
|
839
|
-
|
994
|
+
bands.with_highlights.each do |band, highlights|
|
995
|
+
highlights[:name] # "Two Door <em>Cinema</em> Club"
|
840
996
|
end
|
841
997
|
```
|
842
998
|
|
843
999
|
To change the tag, use:
|
844
1000
|
|
845
1001
|
```ruby
|
846
|
-
Band.search
|
1002
|
+
Band.search("cinema", highlight: {tag: "<strong>"})
|
847
1003
|
```
|
848
1004
|
|
849
1005
|
To highlight and search different fields, use:
|
850
1006
|
|
851
1007
|
```ruby
|
852
|
-
Band.search
|
1008
|
+
Band.search("cinema", fields: [:name], highlight: {fields: [:description]})
|
1009
|
+
```
|
1010
|
+
|
1011
|
+
By default, the entire field is highlighted. To get small snippets instead, use:
|
1012
|
+
|
1013
|
+
```ruby
|
1014
|
+
bands = Band.search("cinema", highlight: {fragment_size: 20})
|
1015
|
+
bands.with_highlights(multiple: true).each do |band, highlights|
|
1016
|
+
highlights[:name].join(" and ")
|
1017
|
+
end
|
853
1018
|
```
|
854
1019
|
|
855
|
-
Additional options
|
1020
|
+
Additional options can be specified for each field:
|
856
1021
|
|
857
1022
|
```ruby
|
858
|
-
Band.search
|
1023
|
+
Band.search("cinema", fields: [:name], highlight: {fields: {name: {fragment_size: 200}}})
|
859
1024
|
```
|
860
1025
|
|
861
|
-
You can find available highlight options in the [Elasticsearch reference](https://www.elastic.co/guide/en/elasticsearch/reference/current/
|
1026
|
+
You can find available highlight options in the [Elasticsearch reference](https://www.elastic.co/guide/en/elasticsearch/reference/current/highlighting.html).
|
862
1027
|
|
863
|
-
|
1028
|
+
## Similar Items
|
864
1029
|
|
865
1030
|
Find similar items.
|
866
1031
|
|
@@ -869,14 +1034,14 @@ product = Product.first
|
|
869
1034
|
product.similar(fields: [:name], where: {size: "12 oz"})
|
870
1035
|
```
|
871
1036
|
|
872
|
-
|
1037
|
+
## Geospatial Searches
|
873
1038
|
|
874
1039
|
```ruby
|
875
|
-
class Restaurant <
|
1040
|
+
class Restaurant < ApplicationRecord
|
876
1041
|
searchkick locations: [:location]
|
877
1042
|
|
878
1043
|
def search_data
|
879
|
-
attributes.merge
|
1044
|
+
attributes.merge(location: {lat: latitude, lon: longitude})
|
880
1045
|
end
|
881
1046
|
end
|
882
1047
|
```
|
@@ -884,19 +1049,21 @@ end
|
|
884
1049
|
Reindex and search with:
|
885
1050
|
|
886
1051
|
```ruby
|
887
|
-
Restaurant.search
|
1052
|
+
Restaurant.search("pizza", where: {location: {near: {lat: 37, lon: -114}, within: "100mi"}}) # or 160km
|
888
1053
|
```
|
889
1054
|
|
890
1055
|
Bounded by a box
|
891
1056
|
|
892
1057
|
```ruby
|
893
|
-
Restaurant.search
|
1058
|
+
Restaurant.search("sushi", where: {location: {top_left: {lat: 38, lon: -123}, bottom_right: {lat: 37, lon: -122}}})
|
894
1059
|
```
|
895
1060
|
|
1061
|
+
**Note:** `top_right` and `bottom_left` also work
|
1062
|
+
|
896
1063
|
Bounded by a polygon
|
897
1064
|
|
898
1065
|
```ruby
|
899
|
-
Restaurant.search
|
1066
|
+
Restaurant.search("dessert", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123}, {lat: 39, lon: -123}, {lat: 37, lon: 122}]}}})
|
900
1067
|
```
|
901
1068
|
|
902
1069
|
### Boost By Distance
|
@@ -904,13 +1071,13 @@ Restaurant.search "dessert", where: {location: {geo_polygon: {points: [{lat: 38,
|
|
904
1071
|
Boost results by distance - closer results are boosted more
|
905
1072
|
|
906
1073
|
```ruby
|
907
|
-
Restaurant.search
|
1074
|
+
Restaurant.search("noodles", boost_by_distance: {location: {origin: {lat: 37, lon: -122}}})
|
908
1075
|
```
|
909
1076
|
|
910
|
-
Also supports [additional options](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#
|
1077
|
+
Also supports [additional options](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#function-decay)
|
911
1078
|
|
912
1079
|
```ruby
|
913
|
-
Restaurant.search
|
1080
|
+
Restaurant.search("wings", boost_by_distance: {location: {origin: {lat: 37, lon: -122}, function: "linear", scale: "30mi", decay: 0.5}})
|
914
1081
|
```
|
915
1082
|
|
916
1083
|
### Geo Shapes
|
@@ -918,10 +1085,8 @@ Restaurant.search "wings", boost_by_distance: {location: {origin: {lat: 37, lon:
|
|
918
1085
|
You can also index and search geo shapes.
|
919
1086
|
|
920
1087
|
```ruby
|
921
|
-
class Restaurant <
|
922
|
-
searchkick geo_shape:
|
923
|
-
bounds: {tree: "geohash", precision: "1km"}
|
924
|
-
}
|
1088
|
+
class Restaurant < ApplicationRecord
|
1089
|
+
searchkick geo_shape: [:bounds]
|
925
1090
|
|
926
1091
|
def search_data
|
927
1092
|
attributes.merge(
|
@@ -939,25 +1104,19 @@ See the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsea
|
|
939
1104
|
Find shapes intersecting with the query shape
|
940
1105
|
|
941
1106
|
```ruby
|
942
|
-
Restaurant.search
|
1107
|
+
Restaurant.search("soup", where: {bounds: {geo_shape: {type: "polygon", coordinates: [[{lat: 38, lon: -123}, ...]]}}})
|
943
1108
|
```
|
944
1109
|
|
945
1110
|
Falling entirely within the query shape
|
946
1111
|
|
947
1112
|
```ruby
|
948
|
-
Restaurant.search
|
1113
|
+
Restaurant.search("salad", where: {bounds: {geo_shape: {type: "circle", relation: "within", coordinates: {lat: 38, lon: -123}, radius: "1km"}}})
|
949
1114
|
```
|
950
1115
|
|
951
1116
|
Not touching the query shape
|
952
1117
|
|
953
1118
|
```ruby
|
954
|
-
Restaurant.search
|
955
|
-
```
|
956
|
-
|
957
|
-
Containing the query shape (Elasticsearch 2.2+)
|
958
|
-
|
959
|
-
```ruby
|
960
|
-
Restaurant.search "fries", where: {bounds: {geo_shape: {type: "envelope", relation: "contains", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
|
1119
|
+
Restaurant.search("burger", where: {bounds: {geo_shape: {type: "envelope", relation: "disjoint", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}})
|
961
1120
|
```
|
962
1121
|
|
963
1122
|
## Inheritance
|
@@ -969,26 +1128,37 @@ class Dog < Animal
|
|
969
1128
|
end
|
970
1129
|
```
|
971
1130
|
|
1131
|
+
In your parent model, set:
|
1132
|
+
|
1133
|
+
```ruby
|
1134
|
+
class Animal < ApplicationRecord
|
1135
|
+
searchkick inheritance: true
|
1136
|
+
end
|
1137
|
+
```
|
1138
|
+
|
972
1139
|
The parent and child model can both reindex.
|
973
1140
|
|
974
1141
|
```ruby
|
975
1142
|
Animal.reindex
|
976
|
-
Dog.reindex # equivalent
|
1143
|
+
Dog.reindex # equivalent, all animals reindexed
|
977
1144
|
```
|
978
1145
|
|
979
1146
|
And to search, use:
|
980
1147
|
|
981
1148
|
```ruby
|
982
|
-
Animal.search
|
983
|
-
Dog.search
|
984
|
-
Animal.search
|
1149
|
+
Animal.search("*") # all animals
|
1150
|
+
Dog.search("*") # just dogs
|
1151
|
+
Animal.search("*", type: [Dog, Cat]) # just cats and dogs
|
985
1152
|
```
|
986
1153
|
|
987
|
-
**
|
1154
|
+
**Notes:**
|
988
1155
|
|
989
|
-
|
990
|
-
|
991
|
-
```
|
1156
|
+
1. The `suggest` option retrieves suggestions from the parent at the moment.
|
1157
|
+
|
1158
|
+
```ruby
|
1159
|
+
Dog.search("airbudd", suggest: true) # suggestions for all animals
|
1160
|
+
```
|
1161
|
+
2. This relies on a `type` field that is automatically added to the indexed document. Be wary of defining your own `type` field in `search_data`, as it will take precedence.
|
992
1162
|
|
993
1163
|
## Debugging Queries
|
994
1164
|
|
@@ -1000,13 +1170,13 @@ Product.search("soap", debug: true)
|
|
1000
1170
|
|
1001
1171
|
This prints useful info to `stdout`.
|
1002
1172
|
|
1003
|
-
See how
|
1173
|
+
See how the search server scores your queries with:
|
1004
1174
|
|
1005
1175
|
```ruby
|
1006
1176
|
Product.search("soap", explain: true).response
|
1007
1177
|
```
|
1008
1178
|
|
1009
|
-
See how
|
1179
|
+
See how the search server tokenizes your queries with:
|
1010
1180
|
|
1011
1181
|
```ruby
|
1012
1182
|
Product.search_index.tokens("Dish Washer Soap", analyzer: "searchkick_index")
|
@@ -1029,99 +1199,291 @@ Product.search_index.tokens("dieg", analyzer: "searchkick_word_search")
|
|
1029
1199
|
# ["dieg"] - match!!
|
1030
1200
|
```
|
1031
1201
|
|
1032
|
-
See the [complete list of analyzers](
|
1033
|
-
|
1034
|
-
## Deployment
|
1202
|
+
See the [complete list of analyzers](lib/searchkick/index_options.rb#L36).
|
1035
1203
|
|
1036
|
-
|
1204
|
+
## Testing
|
1037
1205
|
|
1038
|
-
|
1206
|
+
As you iterate on your search, it’s a good idea to add tests.
|
1039
1207
|
|
1040
|
-
|
1208
|
+
For performance, only enable Searchkick callbacks for the tests that need it.
|
1041
1209
|
|
1042
|
-
|
1043
|
-
# SearchBox
|
1044
|
-
heroku addons:create searchbox:starter
|
1045
|
-
heroku config:set ELASTICSEARCH_URL=`heroku config:get SEARCHBOX_URL`
|
1210
|
+
### Parallel Tests
|
1046
1211
|
|
1047
|
-
|
1048
|
-
heroku addons:create bonsai
|
1049
|
-
heroku config:set ELASTICSEARCH_URL=`heroku config:get BONSAI_URL`
|
1212
|
+
Rails 6 enables parallel tests by default. Add to your `test/test_helper.rb`:
|
1050
1213
|
|
1051
|
-
|
1052
|
-
|
1053
|
-
|
1054
|
-
|
1214
|
+
```ruby
|
1215
|
+
class ActiveSupport::TestCase
|
1216
|
+
parallelize_setup do |worker|
|
1217
|
+
Searchkick.index_suffix = worker
|
1055
1218
|
|
1056
|
-
|
1219
|
+
# reindex models
|
1220
|
+
Product.reindex
|
1057
1221
|
|
1058
|
-
|
1059
|
-
|
1222
|
+
# and disable callbacks
|
1223
|
+
Searchkick.disable_callbacks
|
1224
|
+
end
|
1225
|
+
end
|
1060
1226
|
```
|
1061
1227
|
|
1062
|
-
|
1063
|
-
|
1064
|
-
Include `elasticsearch 1.0.15` or greater in your Gemfile.
|
1228
|
+
And use:
|
1065
1229
|
|
1066
1230
|
```ruby
|
1067
|
-
|
1068
|
-
|
1231
|
+
class ProductTest < ActiveSupport::TestCase
|
1232
|
+
def setup
|
1233
|
+
Searchkick.enable_callbacks
|
1234
|
+
end
|
1069
1235
|
|
1070
|
-
|
1236
|
+
def teardown
|
1237
|
+
Searchkick.disable_callbacks
|
1238
|
+
end
|
1071
1239
|
|
1072
|
-
|
1073
|
-
|
1240
|
+
def test_search
|
1241
|
+
Product.create!(name: "Apple")
|
1242
|
+
Product.search_index.refresh
|
1243
|
+
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1244
|
+
end
|
1245
|
+
end
|
1074
1246
|
```
|
1075
1247
|
|
1076
|
-
|
1077
|
-
|
1078
|
-
```ruby
|
1079
|
-
gem 'faraday_middleware-aws-signers-v4'
|
1080
|
-
```
|
1248
|
+
### Minitest
|
1081
1249
|
|
1082
|
-
|
1250
|
+
Add to your `test/test_helper.rb`:
|
1083
1251
|
|
1084
1252
|
```ruby
|
1085
|
-
|
1086
|
-
|
1087
|
-
secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
|
1088
|
-
region: "us-east-1"
|
1089
|
-
}
|
1090
|
-
```
|
1091
|
-
|
1092
|
-
Then deploy and reindex:
|
1253
|
+
# reindex models
|
1254
|
+
Product.reindex
|
1093
1255
|
|
1094
|
-
|
1095
|
-
|
1256
|
+
# and disable callbacks
|
1257
|
+
Searchkick.disable_callbacks
|
1096
1258
|
```
|
1097
1259
|
|
1098
|
-
|
1099
|
-
|
1100
|
-
Create an initializer `config/initializers/elasticsearch.rb` with:
|
1260
|
+
And use:
|
1101
1261
|
|
1102
1262
|
```ruby
|
1103
|
-
|
1104
|
-
|
1263
|
+
class ProductTest < Minitest::Test
|
1264
|
+
def setup
|
1265
|
+
Searchkick.enable_callbacks
|
1266
|
+
end
|
1105
1267
|
|
1106
|
-
|
1268
|
+
def teardown
|
1269
|
+
Searchkick.disable_callbacks
|
1270
|
+
end
|
1107
1271
|
|
1108
|
-
|
1109
|
-
|
1272
|
+
def test_search
|
1273
|
+
Product.create!(name: "Apple")
|
1274
|
+
Product.search_index.refresh
|
1275
|
+
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1276
|
+
end
|
1277
|
+
end
|
1110
1278
|
```
|
1111
1279
|
|
1112
|
-
###
|
1280
|
+
### RSpec
|
1113
1281
|
|
1114
|
-
|
1282
|
+
Add to your `spec/spec_helper.rb`:
|
1115
1283
|
|
1116
1284
|
```ruby
|
1117
|
-
|
1118
|
-
|
1119
|
-
|
1120
|
-
|
1285
|
+
RSpec.configure do |config|
|
1286
|
+
config.before(:suite) do
|
1287
|
+
# reindex models
|
1288
|
+
Product.reindex
|
1289
|
+
|
1290
|
+
# and disable callbacks
|
1291
|
+
Searchkick.disable_callbacks
|
1292
|
+
end
|
1293
|
+
|
1294
|
+
config.around(:each, search: true) do |example|
|
1295
|
+
Searchkick.callbacks(nil) do
|
1296
|
+
example.run
|
1297
|
+
end
|
1298
|
+
end
|
1299
|
+
end
|
1300
|
+
```
|
1301
|
+
|
1302
|
+
And use:
|
1303
|
+
|
1304
|
+
```ruby
|
1305
|
+
describe Product, search: true do
|
1306
|
+
it "searches" do
|
1307
|
+
Product.create!(name: "Apple")
|
1308
|
+
Product.search_index.refresh
|
1309
|
+
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1310
|
+
end
|
1311
|
+
end
|
1312
|
+
```
|
1313
|
+
|
1314
|
+
### Factory Bot
|
1315
|
+
|
1316
|
+
Use a trait and an after `create` hook for each indexed model:
|
1317
|
+
|
1318
|
+
```ruby
|
1319
|
+
FactoryBot.define do
|
1320
|
+
factory :product do
|
1321
|
+
# ...
|
1322
|
+
|
1323
|
+
# Note: This should be the last trait in the list so `reindex` is called
|
1324
|
+
# after all the other callbacks complete.
|
1325
|
+
trait :reindex do
|
1326
|
+
after(:create) do |product, _evaluator|
|
1327
|
+
product.reindex(refresh: true)
|
1328
|
+
end
|
1329
|
+
end
|
1330
|
+
end
|
1331
|
+
end
|
1332
|
+
|
1333
|
+
# use it
|
1334
|
+
FactoryBot.create(:product, :some_trait, :reindex, some_attribute: "foo")
|
1335
|
+
```
|
1336
|
+
|
1337
|
+
### GitHub Actions
|
1338
|
+
|
1339
|
+
Check out [setup-elasticsearch](https://github.com/ankane/setup-elasticsearch) for an easy way to install Elasticsearch:
|
1340
|
+
|
1341
|
+
```yml
|
1342
|
+
- uses: ankane/setup-elasticsearch@v1
|
1343
|
+
```
|
1344
|
+
|
1345
|
+
And [setup-opensearch](https://github.com/ankane/setup-opensearch) for an easy way to install OpenSearch:
|
1346
|
+
|
1347
|
+
```yml
|
1348
|
+
- uses: ankane/setup-opensearch@v1
|
1349
|
+
```
|
1350
|
+
|
1351
|
+
## Deployment
|
1352
|
+
|
1353
|
+
For the search server, Searchkick uses `ENV["ELASTICSEARCH_URL"]` for Elasticsearch and `ENV["OPENSEARCH_URL"]` for OpenSearch. This defaults to `http://localhost:9200`.
|
1354
|
+
|
1355
|
+
- [Elastic Cloud](#elastic-cloud)
|
1356
|
+
- [Heroku](#heroku)
|
1357
|
+
- [Amazon OpenSearch Service](#amazon-opensearch-service)
|
1358
|
+
- [Self-Hosted and Other](#self-hosted-and-other)
|
1359
|
+
|
1360
|
+
### Elastic Cloud
|
1361
|
+
|
1362
|
+
Create an initializer `config/initializers/elasticsearch.rb` with:
|
1363
|
+
|
1364
|
+
```ruby
|
1365
|
+
ENV["ELASTICSEARCH_URL"] = "https://user:password@host:port"
|
1366
|
+
```
|
1367
|
+
|
1368
|
+
Then deploy and reindex:
|
1369
|
+
|
1370
|
+
```sh
|
1371
|
+
rake searchkick:reindex:all
|
1372
|
+
```
|
1373
|
+
|
1374
|
+
### Heroku
|
1375
|
+
|
1376
|
+
Choose an add-on: [Bonsai](https://elements.heroku.com/addons/bonsai), [SearchBox](https://elements.heroku.com/addons/searchbox), or [Elastic Cloud](https://elements.heroku.com/addons/foundelasticsearch).
|
1377
|
+
|
1378
|
+
For Elasticsearch on Bonsai:
|
1379
|
+
|
1380
|
+
```sh
|
1381
|
+
heroku addons:create bonsai
|
1382
|
+
heroku config:set ELASTICSEARCH_URL=`heroku config:get BONSAI_URL`
|
1383
|
+
```
|
1384
|
+
|
1385
|
+
For OpenSearch on Bonsai:
|
1386
|
+
|
1387
|
+
```sh
|
1388
|
+
heroku addons:create bonsai --engine=opensearch
|
1389
|
+
heroku config:set OPENSEARCH_URL=`heroku config:get BONSAI_URL`
|
1390
|
+
```
|
1391
|
+
|
1392
|
+
For SearchBox:
|
1393
|
+
|
1394
|
+
```sh
|
1395
|
+
heroku addons:create searchbox:starter
|
1396
|
+
heroku config:set ELASTICSEARCH_URL=`heroku config:get SEARCHBOX_URL`
|
1397
|
+
```
|
1398
|
+
|
1399
|
+
For Elastic Cloud (previously Found):
|
1400
|
+
|
1401
|
+
```sh
|
1402
|
+
heroku addons:create foundelasticsearch
|
1403
|
+
heroku addons:open foundelasticsearch
|
1404
|
+
```
|
1405
|
+
|
1406
|
+
Visit the Shield page and reset your password. You’ll need to add the username and password to your url. Get the existing url with:
|
1407
|
+
|
1408
|
+
```sh
|
1409
|
+
heroku config:get FOUNDELASTICSEARCH_URL
|
1410
|
+
```
|
1411
|
+
|
1412
|
+
And add `elastic:password@` right after `https://` and add port `9243` at the end:
|
1413
|
+
|
1414
|
+
```sh
|
1415
|
+
heroku config:set ELASTICSEARCH_URL=https://elastic:password@12345.us-east-1.aws.found.io:9243
|
1416
|
+
```
|
1417
|
+
|
1418
|
+
Then deploy and reindex:
|
1419
|
+
|
1420
|
+
```sh
|
1421
|
+
heroku run rake searchkick:reindex:all
|
1422
|
+
```
|
1423
|
+
|
1424
|
+
### Amazon OpenSearch Service
|
1425
|
+
|
1426
|
+
Create an initializer `config/initializers/opensearch.rb` with:
|
1427
|
+
|
1428
|
+
```ruby
|
1429
|
+
ENV["OPENSEARCH_URL"] = "https://es-domain-1234.us-east-1.es.amazonaws.com:443"
|
1430
|
+
```
|
1431
|
+
|
1432
|
+
To use signed requests, include in your Gemfile:
|
1433
|
+
|
1434
|
+
```ruby
|
1435
|
+
gem "faraday_middleware-aws-sigv4"
|
1436
|
+
```
|
1437
|
+
|
1438
|
+
and add to your initializer:
|
1439
|
+
|
1440
|
+
```ruby
|
1441
|
+
Searchkick.aws_credentials = {
|
1442
|
+
access_key_id: ENV["AWS_ACCESS_KEY_ID"],
|
1443
|
+
secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
|
1444
|
+
region: "us-east-1"
|
1121
1445
|
}
|
1122
1446
|
```
|
1123
1447
|
|
1124
|
-
|
1448
|
+
Then deploy and reindex:
|
1449
|
+
|
1450
|
+
```sh
|
1451
|
+
rake searchkick:reindex:all
|
1452
|
+
```
|
1453
|
+
|
1454
|
+
### Self-Hosted and Other
|
1455
|
+
|
1456
|
+
Create an initializer with:
|
1457
|
+
|
1458
|
+
```ruby
|
1459
|
+
ENV["ELASTICSEARCH_URL"] = "https://user:password@host:port"
|
1460
|
+
# or
|
1461
|
+
ENV["OPENSEARCH_URL"] = "https://user:password@host:port"
|
1462
|
+
```
|
1463
|
+
|
1464
|
+
Then deploy and reindex:
|
1465
|
+
|
1466
|
+
```sh
|
1467
|
+
rake searchkick:reindex:all
|
1468
|
+
```
|
1469
|
+
|
1470
|
+
### Data Protection
|
1471
|
+
|
1472
|
+
We recommend encrypting data at rest and in transit (even inside your own network). This is especially important if you send [personal data](https://en.wikipedia.org/wiki/Personally_identifiable_information) of your users to the search server.
|
1473
|
+
|
1474
|
+
Bonsai, Elastic Cloud, and Amazon OpenSearch Service all support encryption at rest and HTTPS.
|
1475
|
+
|
1476
|
+
### Automatic Failover
|
1477
|
+
|
1478
|
+
Create an initializer with multiple hosts:
|
1479
|
+
|
1480
|
+
```ruby
|
1481
|
+
ENV["ELASTICSEARCH_URL"] = "https://user:password@host1,https://user:password@host2"
|
1482
|
+
# or
|
1483
|
+
ENV["OPENSEARCH_URL"] = "https://user:password@host1,https://user:password@host2"
|
1484
|
+
```
|
1485
|
+
|
1486
|
+
See [elastic-transport](https://github.com/elastic/elastic-transport-ruby) or [opensearch-transport](https://github.com/opensearch-project/opensearch-ruby/tree/main/opensearch-transport) for a complete list of options.
|
1125
1487
|
|
1126
1488
|
### Lograge
|
1127
1489
|
|
@@ -1144,7 +1506,7 @@ See [Production Rails](https://github.com/ankane/production_rails) for other goo
|
|
1144
1506
|
Significantly increase performance with faster JSON generation. Add [Oj](https://github.com/ohler55/oj) to your Gemfile.
|
1145
1507
|
|
1146
1508
|
```ruby
|
1147
|
-
gem
|
1509
|
+
gem "oj"
|
1148
1510
|
```
|
1149
1511
|
|
1150
1512
|
This speeds up all JSON generation and parsing in your application (automatically!)
|
@@ -1154,23 +1516,23 @@ This speeds up all JSON generation and parsing in your application (automaticall
|
|
1154
1516
|
Significantly increase performance with persistent HTTP connections. Add [Typhoeus](https://github.com/typhoeus/typhoeus) to your Gemfile and it’ll automatically be used.
|
1155
1517
|
|
1156
1518
|
```ruby
|
1157
|
-
gem
|
1519
|
+
gem "typhoeus"
|
1158
1520
|
```
|
1159
1521
|
|
1160
1522
|
To reduce log noise, create an initializer with:
|
1161
1523
|
|
1162
1524
|
```ruby
|
1163
|
-
Ethon.logger = Logger.new(
|
1525
|
+
Ethon.logger = Logger.new(nil)
|
1164
1526
|
```
|
1165
1527
|
|
1166
1528
|
If you run into issues on Windows, check out [this post](https://www.rastating.com/fixing-issues-in-typhoeus-and-httparty-on-windows/).
|
1167
1529
|
|
1168
1530
|
### Searchable Fields
|
1169
1531
|
|
1170
|
-
By default, all string fields are searchable (can be used in `fields` option). Speed up indexing and reduce index size by only making some fields searchable.
|
1532
|
+
By default, all string fields are searchable (can be used in `fields` option). Speed up indexing and reduce index size by only making some fields searchable.
|
1171
1533
|
|
1172
1534
|
```ruby
|
1173
|
-
class Product <
|
1535
|
+
class Product < ApplicationRecord
|
1174
1536
|
searchkick searchable: [:name]
|
1175
1537
|
end
|
1176
1538
|
```
|
@@ -1180,19 +1542,19 @@ end
|
|
1180
1542
|
By default, all string fields are filterable (can be used in `where` option). Speed up indexing and reduce index size by only making some fields filterable.
|
1181
1543
|
|
1182
1544
|
```ruby
|
1183
|
-
class Product <
|
1545
|
+
class Product < ApplicationRecord
|
1184
1546
|
searchkick filterable: [:brand]
|
1185
1547
|
end
|
1186
1548
|
```
|
1187
1549
|
|
1188
|
-
**Note:** Non-string fields
|
1550
|
+
**Note:** Non-string fields are always filterable and should not be passed to this option.
|
1189
1551
|
|
1190
1552
|
### Parallel Reindexing
|
1191
1553
|
|
1192
1554
|
For large data sets, you can use background jobs to parallelize reindexing.
|
1193
1555
|
|
1194
1556
|
```ruby
|
1195
|
-
Product.reindex(
|
1557
|
+
Product.reindex(mode: :async)
|
1196
1558
|
# {index_name: "products_production_20170111210018065"}
|
1197
1559
|
```
|
1198
1560
|
|
@@ -1214,16 +1576,16 @@ And use:
|
|
1214
1576
|
Searchkick.reindex_status(index_name)
|
1215
1577
|
```
|
1216
1578
|
|
1217
|
-
You can also have Searchkick wait for reindexing to complete
|
1579
|
+
You can also have Searchkick wait for reindexing to complete
|
1218
1580
|
|
1219
1581
|
```ruby
|
1220
|
-
|
1582
|
+
Product.reindex(mode: :async, wait: true)
|
1221
1583
|
```
|
1222
1584
|
|
1223
1585
|
You can use [ActiveJob::TrafficControl](https://github.com/nickelser/activejob-traffic_control) to control concurrency. Install the gem:
|
1224
1586
|
|
1225
1587
|
```ruby
|
1226
|
-
gem
|
1588
|
+
gem "activejob-traffic_control", ">= 0.1.3"
|
1227
1589
|
```
|
1228
1590
|
|
1229
1591
|
And create an initializer with:
|
@@ -1243,7 +1605,7 @@ This will allow only 3 jobs to run at once.
|
|
1243
1605
|
You can specify a longer refresh interval while reindexing to increase performance.
|
1244
1606
|
|
1245
1607
|
```ruby
|
1246
|
-
Product.reindex(
|
1608
|
+
Product.reindex(mode: :async, refresh_interval: "30s")
|
1247
1609
|
```
|
1248
1610
|
|
1249
1611
|
**Note:** This only makes a noticable difference with parallel reindexing.
|
@@ -1256,7 +1618,7 @@ Product.search_index.promote(index_name, update_refresh_interval: true)
|
|
1256
1618
|
|
1257
1619
|
### Queuing
|
1258
1620
|
|
1259
|
-
Push ids of records needing
|
1621
|
+
Push ids of records needing reindexing to a queue and reindex in bulk for better performance. First, set up Redis in an initializer. We recommend using [connection_pool](https://github.com/mperham/connection_pool).
|
1260
1622
|
|
1261
1623
|
```ruby
|
1262
1624
|
Searchkick.redis = ConnectionPool.new { Redis.new }
|
@@ -1265,7 +1627,7 @@ Searchkick.redis = ConnectionPool.new { Redis.new }
|
|
1265
1627
|
And ask your models to queue updates.
|
1266
1628
|
|
1267
1629
|
```ruby
|
1268
|
-
class Product <
|
1630
|
+
class Product < ApplicationRecord
|
1269
1631
|
searchkick callbacks: :queue
|
1270
1632
|
end
|
1271
1633
|
```
|
@@ -1286,10 +1648,10 @@ For more tips, check out [Keeping Elasticsearch in Sync](https://www.elastic.co/
|
|
1286
1648
|
|
1287
1649
|
### Routing
|
1288
1650
|
|
1289
|
-
Searchkick supports [
|
1651
|
+
Searchkick supports [routing](https://www.elastic.co/blog/customizing-your-document-routing), which can significantly speed up searches.
|
1290
1652
|
|
1291
1653
|
```ruby
|
1292
|
-
class Business <
|
1654
|
+
class Business < ApplicationRecord
|
1293
1655
|
searchkick routing: true
|
1294
1656
|
|
1295
1657
|
def search_routing
|
@@ -1301,7 +1663,7 @@ end
|
|
1301
1663
|
Reindex and search with:
|
1302
1664
|
|
1303
1665
|
```ruby
|
1304
|
-
Business.search
|
1666
|
+
Business.search("ice cream", routing: params[:city_id])
|
1305
1667
|
```
|
1306
1668
|
|
1307
1669
|
### Partial Reindexing
|
@@ -1309,14 +1671,15 @@ Business.search "ice cream", routing: params[:city_id]
|
|
1309
1671
|
Reindex a subset of attributes to reduce time spent generating search data and cut down on network traffic.
|
1310
1672
|
|
1311
1673
|
```ruby
|
1312
|
-
class Product <
|
1674
|
+
class Product < ApplicationRecord
|
1313
1675
|
def search_data
|
1314
1676
|
{
|
1315
|
-
name: name
|
1316
|
-
|
1677
|
+
name: name,
|
1678
|
+
category: category
|
1679
|
+
}.merge(prices_data)
|
1317
1680
|
end
|
1318
1681
|
|
1319
|
-
def
|
1682
|
+
def prices_data
|
1320
1683
|
{
|
1321
1684
|
price: price,
|
1322
1685
|
sale_price: sale_price
|
@@ -1328,85 +1691,22 @@ end
|
|
1328
1691
|
And use:
|
1329
1692
|
|
1330
1693
|
```ruby
|
1331
|
-
Product.reindex(:
|
1332
|
-
```
|
1333
|
-
|
1334
|
-
### Performant Conversions
|
1335
|
-
|
1336
|
-
Split out conversions into a separate method so you can use partial reindexing, and cache conversions to prevent N+1 queries. Be sure to use a centralized cache store like Memcached or Redis.
|
1337
|
-
|
1338
|
-
```ruby
|
1339
|
-
class Product < ActiveRecord::Base
|
1340
|
-
def search_data
|
1341
|
-
{
|
1342
|
-
name: name
|
1343
|
-
}.merge(search_conversions)
|
1344
|
-
end
|
1345
|
-
|
1346
|
-
def search_conversions
|
1347
|
-
{
|
1348
|
-
conversions: Rails.cache.read("search_conversions:#{self.class.name}:#{id}") || {}
|
1349
|
-
}
|
1350
|
-
end
|
1351
|
-
end
|
1352
|
-
```
|
1353
|
-
|
1354
|
-
Create a job to update the cache and reindex records with new conversions.
|
1355
|
-
|
1356
|
-
```ruby
|
1357
|
-
class ReindexConversionsJob < ActiveJob::Base
|
1358
|
-
def perform(class_name)
|
1359
|
-
# get records that have a recent conversion
|
1360
|
-
recently_converted_ids =
|
1361
|
-
Searchjoy::Search.where("convertable_type = ? AND converted_at > ?", class_name, 1.day.ago)
|
1362
|
-
.order(:convertable_id).uniq.pluck(:convertable_id)
|
1363
|
-
|
1364
|
-
# split into groups
|
1365
|
-
recently_converted_ids.in_groups_of(1000, false) do |ids|
|
1366
|
-
# fetch conversions
|
1367
|
-
conversions =
|
1368
|
-
Searchjoy::Search.where(convertable_id: ids, convertable_type: class_name)
|
1369
|
-
.group(:convertable_id, :query).uniq.count(:user_id)
|
1370
|
-
|
1371
|
-
# group conversions by record
|
1372
|
-
conversions_by_record = {}
|
1373
|
-
conversions.each do |(id, query), count|
|
1374
|
-
(conversions_by_record[id] ||= {})[query] = count
|
1375
|
-
end
|
1376
|
-
|
1377
|
-
# write to cache
|
1378
|
-
conversions_by_record.each do |id, conversions|
|
1379
|
-
Rails.cache.write("search_conversions:#{class_name}:#{id}", conversions)
|
1380
|
-
end
|
1381
|
-
|
1382
|
-
# partial reindex
|
1383
|
-
class_name.constantize.where(id: ids).reindex(:search_conversions)
|
1384
|
-
end
|
1385
|
-
end
|
1386
|
-
end
|
1387
|
-
```
|
1388
|
-
|
1389
|
-
Run the job with:
|
1390
|
-
|
1391
|
-
```ruby
|
1392
|
-
ReindexConversionsJob.perform_later("Product")
|
1694
|
+
Product.reindex(:prices_data)
|
1393
1695
|
```
|
1394
1696
|
|
1395
1697
|
## Advanced
|
1396
1698
|
|
1397
|
-
Searchkick makes it easy to use the Elasticsearch DSL on its own.
|
1699
|
+
Searchkick makes it easy to use the Elasticsearch or OpenSearch DSL on its own.
|
1398
1700
|
|
1399
1701
|
### Advanced Mapping
|
1400
1702
|
|
1401
1703
|
Create a custom mapping:
|
1402
1704
|
|
1403
1705
|
```ruby
|
1404
|
-
class Product <
|
1706
|
+
class Product < ApplicationRecord
|
1405
1707
|
searchkick mappings: {
|
1406
|
-
|
1407
|
-
|
1408
|
-
name: {type: "string", analyzer: "keyword"}
|
1409
|
-
}
|
1708
|
+
properties: {
|
1709
|
+
name: {type: "keyword"}
|
1410
1710
|
}
|
1411
1711
|
}
|
1412
1712
|
end
|
@@ -1416,7 +1716,7 @@ end
|
|
1416
1716
|
To keep the mappings and settings generated by Searchkick, use:
|
1417
1717
|
|
1418
1718
|
```ruby
|
1419
|
-
class Product <
|
1719
|
+
class Product < ApplicationRecord
|
1420
1720
|
searchkick merge_mappings: true, mappings: {...}
|
1421
1721
|
end
|
1422
1722
|
```
|
@@ -1426,11 +1726,9 @@ end
|
|
1426
1726
|
And use the `body` option to search:
|
1427
1727
|
|
1428
1728
|
```ruby
|
1429
|
-
products = Product.search
|
1729
|
+
products = Product.search(body: {query: {match: {name: "milk"}}})
|
1430
1730
|
```
|
1431
1731
|
|
1432
|
-
**Note:** This replaces the entire body, so other options are ignored.
|
1433
|
-
|
1434
1732
|
View the response with:
|
1435
1733
|
|
1436
1734
|
```ruby
|
@@ -1440,21 +1738,21 @@ products.response
|
|
1440
1738
|
To modify the query generated by Searchkick, use:
|
1441
1739
|
|
1442
1740
|
```ruby
|
1443
|
-
products = Product.search
|
1741
|
+
products = Product.search("milk", body_options: {min_score: 1})
|
1444
1742
|
```
|
1445
1743
|
|
1446
1744
|
or
|
1447
1745
|
|
1448
1746
|
```ruby
|
1449
1747
|
products =
|
1450
|
-
Product.search
|
1748
|
+
Product.search("apples") do |body|
|
1451
1749
|
body[:min_score] = 1
|
1452
1750
|
end
|
1453
1751
|
```
|
1454
1752
|
|
1455
|
-
###
|
1753
|
+
### Client
|
1456
1754
|
|
1457
|
-
|
1755
|
+
To access the `Elasticsearch::Client` or `OpenSearch::Client` directly, use:
|
1458
1756
|
|
1459
1757
|
```ruby
|
1460
1758
|
Searchkick.client
|
@@ -1465,50 +1763,83 @@ Searchkick.client
|
|
1465
1763
|
To batch search requests for performance, use:
|
1466
1764
|
|
1467
1765
|
```ruby
|
1468
|
-
|
1469
|
-
|
1470
|
-
Searchkick.multi_search([
|
1766
|
+
products = Product.search("snacks")
|
1767
|
+
coupons = Coupon.search("snacks")
|
1768
|
+
Searchkick.multi_search([products, coupons])
|
1471
1769
|
```
|
1472
1770
|
|
1473
|
-
Then use `
|
1771
|
+
Then use `products` and `coupons` as typical results.
|
1474
1772
|
|
1475
|
-
**Note:** Errors are not raised as with single requests. Use the `error` method on each query to check for errors.
|
1773
|
+
**Note:** Errors are not raised as with single requests. Use the `error` method on each query to check for errors.
|
1476
1774
|
|
1477
|
-
## Multiple
|
1775
|
+
## Multiple Models
|
1478
1776
|
|
1479
|
-
Search across multiple
|
1777
|
+
Search across multiple models with:
|
1480
1778
|
|
1481
1779
|
```ruby
|
1482
|
-
Searchkick.search
|
1780
|
+
Searchkick.search("milk", models: [Product, Category])
|
1483
1781
|
```
|
1484
1782
|
|
1485
|
-
Boost specific
|
1783
|
+
Boost specific models with:
|
1486
1784
|
|
1487
1785
|
```ruby
|
1488
1786
|
indices_boost: {Category => 2, Product => 1}
|
1489
1787
|
```
|
1490
1788
|
|
1491
|
-
##
|
1789
|
+
## Multi-Tenancy
|
1492
1790
|
|
1493
|
-
|
1791
|
+
Check out [this great post](https://www.tiagoamaro.com.br/2014/12/11/multi-tenancy-with-searchkick/) on the [Apartment](https://github.com/influitive/apartment) gem. Follow a similar pattern if you use another gem.
|
1792
|
+
|
1793
|
+
## Scroll API
|
1794
|
+
|
1795
|
+
Searchkick also supports the [scroll API](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results). Scrolling is not intended for real time user requests, but rather for processing large amounts of data.
|
1796
|
+
|
1797
|
+
```ruby
|
1798
|
+
Product.search("*", scroll: "1m").scroll do |batch|
|
1799
|
+
# process batch ...
|
1800
|
+
end
|
1801
|
+
```
|
1802
|
+
|
1803
|
+
You can also scroll batches manually.
|
1804
|
+
|
1805
|
+
```ruby
|
1806
|
+
products = Product.search("*", scroll: "1m")
|
1807
|
+
while products.any?
|
1808
|
+
# process batch ...
|
1809
|
+
|
1810
|
+
products = products.scroll
|
1811
|
+
end
|
1812
|
+
|
1813
|
+
products.clear_scroll
|
1814
|
+
```
|
1815
|
+
|
1816
|
+
## Deep Paging
|
1817
|
+
|
1818
|
+
By default, Elasticsearch and OpenSearch limit paging to the first 10,000 results. [Here’s why](https://www.elastic.co/guide/en/elasticsearch/guide/current/pagination.html). We don’t recommend changing this, but if you really need all results, you can use:
|
1494
1819
|
|
1495
1820
|
```ruby
|
1496
|
-
|
1821
|
+
class Product < ApplicationRecord
|
1822
|
+
searchkick deep_paging: true
|
1823
|
+
end
|
1497
1824
|
```
|
1498
1825
|
|
1499
|
-
|
1826
|
+
If you just need an accurate total count, you can instead use:
|
1500
1827
|
|
1501
|
-
|
1828
|
+
```ruby
|
1829
|
+
Product.search("pears", body_options: {track_total_hits: true})
|
1830
|
+
```
|
1502
1831
|
|
1503
|
-
|
1832
|
+
## Nested Data
|
1504
1833
|
|
1505
|
-
|
1834
|
+
To query nested data, use dot notation.
|
1506
1835
|
|
1507
|
-
|
1836
|
+
```ruby
|
1837
|
+
Product.search("san", fields: ["store.city"], where: {"store.zip_code" => 12345})
|
1838
|
+
```
|
1508
1839
|
|
1509
|
-
|
1840
|
+
## Nearest Neighbors
|
1510
1841
|
|
1511
|
-
|
1842
|
+
You can use custom mapping and searching to index vectors and perform k-nearest neighbor search. See the examples for [Elasticsearch](examples/elasticsearch_knn.rb) and [OpenSearch](examples/opensearch_knn.rb).
|
1512
1843
|
|
1513
1844
|
## Reference
|
1514
1845
|
|
@@ -1517,8 +1848,6 @@ Reindex one record
|
|
1517
1848
|
```ruby
|
1518
1849
|
product = Product.find(1)
|
1519
1850
|
product.reindex
|
1520
|
-
# or to reindex in the background
|
1521
|
-
product.reindex_async
|
1522
1851
|
```
|
1523
1852
|
|
1524
1853
|
Reindex multiple records
|
@@ -1542,7 +1871,7 @@ Product.search_index.clean_indices
|
|
1542
1871
|
Use custom settings
|
1543
1872
|
|
1544
1873
|
```ruby
|
1545
|
-
class Product <
|
1874
|
+
class Product < ApplicationRecord
|
1546
1875
|
searchkick settings: {number_of_shards: 3}
|
1547
1876
|
end
|
1548
1877
|
```
|
@@ -1550,7 +1879,7 @@ end
|
|
1550
1879
|
Use a different index name
|
1551
1880
|
|
1552
1881
|
```ruby
|
1553
|
-
class Product <
|
1882
|
+
class Product < ApplicationRecord
|
1554
1883
|
searchkick index_name: "products_v2"
|
1555
1884
|
end
|
1556
1885
|
```
|
@@ -1558,7 +1887,7 @@ end
|
|
1558
1887
|
Use a dynamic index name
|
1559
1888
|
|
1560
1889
|
```ruby
|
1561
|
-
class Product <
|
1890
|
+
class Product < ApplicationRecord
|
1562
1891
|
searchkick index_name: -> { "#{name.tableize}-#{I18n.locale}" }
|
1563
1892
|
end
|
1564
1893
|
```
|
@@ -1566,11 +1895,17 @@ end
|
|
1566
1895
|
Prefix the index name
|
1567
1896
|
|
1568
1897
|
```ruby
|
1569
|
-
class Product <
|
1898
|
+
class Product < ApplicationRecord
|
1570
1899
|
searchkick index_prefix: "datakick"
|
1571
1900
|
end
|
1572
1901
|
```
|
1573
1902
|
|
1903
|
+
For all models
|
1904
|
+
|
1905
|
+
```ruby
|
1906
|
+
Searchkick.index_prefix = "datakick"
|
1907
|
+
```
|
1908
|
+
|
1574
1909
|
Use a different term for boosting by conversions
|
1575
1910
|
|
1576
1911
|
```ruby
|
@@ -1580,7 +1915,7 @@ Product.search("banana", conversions_term: "organic banana")
|
|
1580
1915
|
Multiple conversion fields
|
1581
1916
|
|
1582
1917
|
```ruby
|
1583
|
-
class Product <
|
1918
|
+
class Product < ApplicationRecord
|
1584
1919
|
has_many :searches, class_name: "Searchjoy::Search"
|
1585
1920
|
|
1586
1921
|
# searchkick also supports multiple "conversions" fields
|
@@ -1589,7 +1924,7 @@ class Product < ActiveRecord::Base
|
|
1589
1924
|
def search_data
|
1590
1925
|
{
|
1591
1926
|
name: name,
|
1592
|
-
unique_user_conversions: searches.group(:query).
|
1927
|
+
unique_user_conversions: searches.group(:query).distinct.count(:user_id),
|
1593
1928
|
# {"ice cream" => 234, "chocolate" => 67, "cream" => 2}
|
1594
1929
|
total_conversions: searches.group(:query).count
|
1595
1930
|
# {"ice cream" => 412, "chocolate" => 117, "cream" => 6}
|
@@ -1633,36 +1968,60 @@ Searchkick.queue_name = :search_reindex
|
|
1633
1968
|
Eager load associations
|
1634
1969
|
|
1635
1970
|
```ruby
|
1636
|
-
Product.search
|
1971
|
+
Product.search("milk", includes: [:brand, :stores])
|
1637
1972
|
```
|
1638
1973
|
|
1639
|
-
Eager load different associations by model
|
1974
|
+
Eager load different associations by model
|
1640
1975
|
|
1641
1976
|
```ruby
|
1642
|
-
Searchkick.search("*",
|
1977
|
+
Searchkick.search("*", models: [Product, Store], model_includes: {Product => [:store], Store => [:product]})
|
1978
|
+
```
|
1979
|
+
|
1980
|
+
Run additional scopes on results
|
1981
|
+
|
1982
|
+
```ruby
|
1983
|
+
Product.search("milk", scope_results: ->(r) { r.with_attached_images })
|
1984
|
+
```
|
1985
|
+
|
1986
|
+
Specify default fields to search
|
1987
|
+
|
1988
|
+
```ruby
|
1989
|
+
class Product < ApplicationRecord
|
1990
|
+
searchkick default_fields: [:name]
|
1991
|
+
end
|
1643
1992
|
```
|
1644
1993
|
|
1645
1994
|
Turn off special characters
|
1646
1995
|
|
1647
1996
|
```ruby
|
1648
|
-
class Product <
|
1997
|
+
class Product < ApplicationRecord
|
1649
1998
|
# A will not match Ä
|
1650
1999
|
searchkick special_characters: false
|
1651
2000
|
end
|
1652
2001
|
```
|
1653
2002
|
|
1654
|
-
|
2003
|
+
Turn on stemming for conversions
|
1655
2004
|
|
1656
2005
|
```ruby
|
1657
|
-
class Product <
|
1658
|
-
searchkick
|
2006
|
+
class Product < ApplicationRecord
|
2007
|
+
searchkick stem_conversions: true
|
1659
2008
|
end
|
1660
2009
|
```
|
1661
2010
|
|
2011
|
+
Make search case-sensitive
|
2012
|
+
|
2013
|
+
```ruby
|
2014
|
+
class Product < ApplicationRecord
|
2015
|
+
searchkick case_sensitive: true
|
2016
|
+
end
|
2017
|
+
```
|
2018
|
+
|
2019
|
+
**Note:** If misspellings are enabled (default), results with a single character case difference will match. Turn off misspellings if this is not desired.
|
2020
|
+
|
1662
2021
|
Change import batch size
|
1663
2022
|
|
1664
2023
|
```ruby
|
1665
|
-
class Product <
|
2024
|
+
class Product < ApplicationRecord
|
1666
2025
|
searchkick batch_size: 200 # defaults to 1000
|
1667
2026
|
end
|
1668
2027
|
```
|
@@ -1673,23 +2032,34 @@ Create index without importing
|
|
1673
2032
|
Product.reindex(import: false)
|
1674
2033
|
```
|
1675
2034
|
|
1676
|
-
|
2035
|
+
Use a different id
|
1677
2036
|
|
1678
2037
|
```ruby
|
1679
|
-
|
1680
|
-
|
2038
|
+
class Product < ApplicationRecord
|
2039
|
+
def search_document_id
|
2040
|
+
custom_id
|
2041
|
+
end
|
2042
|
+
end
|
1681
2043
|
```
|
1682
2044
|
|
1683
|
-
Add [request parameters](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-
|
2045
|
+
Add [request parameters](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-search-api-query-params) like `search_type`
|
1684
2046
|
|
1685
2047
|
```ruby
|
1686
2048
|
Product.search("carrots", request_params: {search_type: "dfs_query_then_fetch"})
|
1687
2049
|
```
|
1688
2050
|
|
2051
|
+
Set options across all models
|
2052
|
+
|
2053
|
+
```ruby
|
2054
|
+
Searchkick.model_options = {
|
2055
|
+
batch_size: 200
|
2056
|
+
}
|
2057
|
+
```
|
2058
|
+
|
1689
2059
|
Reindex conditionally
|
1690
2060
|
|
1691
2061
|
```ruby
|
1692
|
-
class Product <
|
2062
|
+
class Product < ApplicationRecord
|
1693
2063
|
searchkick callbacks: false
|
1694
2064
|
|
1695
2065
|
# add the callbacks manually
|
@@ -1706,207 +2076,94 @@ rake searchkick:reindex:all
|
|
1706
2076
|
Turn on misspellings after a certain number of characters
|
1707
2077
|
|
1708
2078
|
```ruby
|
1709
|
-
Product.search
|
2079
|
+
Product.search("api", misspellings: {prefix_length: 2}) # api, apt, no ahi
|
1710
2080
|
```
|
1711
2081
|
|
1712
|
-
**Note:** With this option, if the query length is the same as `prefix_length`, misspellings are turned off
|
2082
|
+
**Note:** With this option, if the query length is the same as `prefix_length`, misspellings are turned off with Elasticsearch 7 and OpenSearch 1
|
1713
2083
|
|
1714
2084
|
```ruby
|
1715
|
-
Product.search
|
2085
|
+
Product.search("ah", misspellings: {prefix_length: 2}) # ah, no aha
|
1716
2086
|
```
|
1717
2087
|
|
1718
|
-
|
1719
|
-
|
1720
|
-
For performance, only enable Searchkick callbacks for the tests that need it.
|
1721
|
-
|
1722
|
-
### Minitest
|
1723
|
-
|
1724
|
-
Add to your `test/test_helper.rb`:
|
2088
|
+
BigDecimal values are indexed as floats by default so they can be used for boosting. Convert them to strings to keep full precision.
|
1725
2089
|
|
1726
2090
|
```ruby
|
1727
|
-
|
1728
|
-
|
1729
|
-
|
1730
|
-
|
1731
|
-
|
1732
|
-
```
|
1733
|
-
|
1734
|
-
And use:
|
1735
|
-
|
1736
|
-
```ruby
|
1737
|
-
class ProductTest < Minitest::Test
|
1738
|
-
def setup
|
1739
|
-
Searchkick.enable_callbacks
|
1740
|
-
end
|
1741
|
-
|
1742
|
-
def teardown
|
1743
|
-
Searchkick.disable_callbacks
|
1744
|
-
end
|
1745
|
-
|
1746
|
-
def test_search
|
1747
|
-
Product.create!(name: "Apple")
|
1748
|
-
Product.search_index.refresh
|
1749
|
-
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
2091
|
+
class Product < ApplicationRecord
|
2092
|
+
def search_data
|
2093
|
+
{
|
2094
|
+
units: units.to_s("F")
|
2095
|
+
}
|
1750
2096
|
end
|
1751
2097
|
end
|
1752
2098
|
```
|
1753
2099
|
|
1754
|
-
|
2100
|
+
## Gotchas
|
1755
2101
|
|
1756
|
-
|
1757
|
-
|
1758
|
-
```ruby
|
1759
|
-
RSpec.configure do |config|
|
1760
|
-
config.before(:suite) do
|
1761
|
-
# reindex models
|
1762
|
-
Product.reindex
|
1763
|
-
|
1764
|
-
# and disable callbacks
|
1765
|
-
Searchkick.disable_callbacks
|
1766
|
-
end
|
1767
|
-
|
1768
|
-
config.around(:each, search: true) do |example|
|
1769
|
-
Searchkick.enable_callbacks
|
1770
|
-
example.run
|
1771
|
-
Searchkick.disable_callbacks
|
1772
|
-
end
|
1773
|
-
end
|
1774
|
-
```
|
2102
|
+
### Consistency
|
1775
2103
|
|
1776
|
-
|
2104
|
+
Elasticsearch and OpenSearch are eventually consistent, meaning it can take up to a second for a change to reflect in search. You can use the `refresh` method to have it show up immediately.
|
1777
2105
|
|
1778
2106
|
```ruby
|
1779
|
-
|
1780
|
-
|
1781
|
-
Product.create!(name: "Apple")
|
1782
|
-
Product.search_index.refresh
|
1783
|
-
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1784
|
-
end
|
1785
|
-
end
|
2107
|
+
product.save!
|
2108
|
+
Product.search_index.refresh
|
1786
2109
|
```
|
1787
2110
|
|
1788
|
-
###
|
2111
|
+
### Inconsistent Scores
|
1789
2112
|
|
1790
|
-
|
2113
|
+
Due to the distributed nature of Elasticsearch and OpenSearch, you can get incorrect results when the number of documents in the index is low. You can [read more about it here](https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch). To fix this, do:
|
1791
2114
|
|
1792
2115
|
```ruby
|
1793
|
-
|
1794
|
-
|
1795
|
-
# ...
|
1796
|
-
|
1797
|
-
# Note: This should be the last trait in the list so `reindex` is called
|
1798
|
-
# after all the other callbacks complete.
|
1799
|
-
trait :reindex do
|
1800
|
-
after(:create) do |product, _evaluator|
|
1801
|
-
product.reindex(refresh: true)
|
1802
|
-
end
|
1803
|
-
end
|
1804
|
-
end
|
2116
|
+
class Product < ApplicationRecord
|
2117
|
+
searchkick settings: {number_of_shards: 1}
|
1805
2118
|
end
|
1806
|
-
|
1807
|
-
# use it
|
1808
|
-
FactoryGirl.create(:product, :some_trait, :reindex, some_attribute: "foo")
|
1809
|
-
```
|
1810
|
-
|
1811
|
-
### Parallel Tests
|
1812
|
-
|
1813
|
-
Set:
|
1814
|
-
|
1815
|
-
```ruby
|
1816
|
-
Searchkick.index_suffix = ENV["TEST_ENV_NUMBER"]
|
1817
2119
|
```
|
1818
2120
|
|
1819
|
-
|
1820
|
-
|
1821
|
-
Check out [this great post](https://www.tiagoamaro.com.br/2014/12/11/multi-tenancy-with-searchkick/) on the [Apartment](https://github.com/influitive/apartment) gem. Follow a similar pattern if you use another gem.
|
2121
|
+
For convenience, this is set by default in the test environment.
|
1822
2122
|
|
1823
2123
|
## Upgrading
|
1824
2124
|
|
1825
|
-
|
1826
|
-
|
1827
|
-
Important notes are listed below.
|
1828
|
-
|
1829
|
-
### 2.0.0
|
2125
|
+
### 5.0
|
1830
2126
|
|
1831
|
-
|
2127
|
+
Searchkick 5 supports both the `elasticsearch` and `opensearch-ruby` gems. Add the one you want to use to your Gemfile:
|
1832
2128
|
|
1833
|
-
|
1834
|
-
|
1835
|
-
|
1836
|
-
-
|
1837
|
-
|
1838
|
-
- The `query` and `json` options have been removed in favor of `body`
|
1839
|
-
- The `include` option has been removed in favor of `includes`
|
1840
|
-
- The `personalize` option has been removed in favor of `boost_where`
|
1841
|
-
- The `partial` option has been removed in favor of `operator`
|
1842
|
-
- Renamed `select_v2` to `select` (legacy `select` no longer available)
|
1843
|
-
- The `_all` field is disabled if `searchable` option is used (for performance)
|
1844
|
-
- The `partial_reindex(:method_name)` method has been replaced with `reindex(:method_name)`
|
1845
|
-
- The `unsearchable` and `only_analyzed` options have been removed in favor of `searchable` and `filterable`
|
1846
|
-
- `load: false` no longer returns an array in Elasticsearch 2
|
1847
|
-
|
1848
|
-
### 1.0.0
|
1849
|
-
|
1850
|
-
- Added support for Elasticsearch 2.0
|
1851
|
-
- Facets are deprecated in favor of [aggregations](#aggregations) - see [how to upgrade](#moving-from-facets)
|
1852
|
-
|
1853
|
-
#### Breaking Changes
|
1854
|
-
|
1855
|
-
- **ActiveRecord 4.1+ and Mongoid 3+:** Attempting to reindex with a scope now throws a `Searchkick::DangerousOperation` error to keep your from accidentally recreating your index with only a few records.
|
1856
|
-
|
1857
|
-
```ruby
|
1858
|
-
Product.where(color: "brandy").reindex # error!
|
1859
|
-
```
|
1860
|
-
|
1861
|
-
If this is what you intend to do, use:
|
1862
|
-
|
1863
|
-
```ruby
|
1864
|
-
Product.where(color: "brandy").reindex(accept_danger: true)
|
1865
|
-
```
|
1866
|
-
|
1867
|
-
- Misspellings are enabled by default for [partial matches](#partial-matches). Use `misspellings: false` to disable.
|
1868
|
-
- [Transpositions](https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance) are enabled by default for misspellings. Use `misspellings: {transpositions: false}` to disable.
|
1869
|
-
|
1870
|
-
### 0.6.0 and 0.7.0
|
1871
|
-
|
1872
|
-
If running Searchkick `0.6.0` or `0.7.0` and Elasticsearch `0.90`, we recommend upgrading to Searchkick `0.6.1` or `0.7.1` to fix an issue that causes downtime when reindexing.
|
2129
|
+
```ruby
|
2130
|
+
gem "elasticsearch"
|
2131
|
+
# or
|
2132
|
+
gem "opensearch-ruby"
|
2133
|
+
```
|
1873
2134
|
|
1874
|
-
|
2135
|
+
If using the deprecated `faraday_middleware-aws-signers-v4` gem, switch to `faraday_middleware-aws-sigv4`.
|
1875
2136
|
|
1876
|
-
|
2137
|
+
Also, searches now use lazy loading:
|
1877
2138
|
|
1878
|
-
|
2139
|
+
```ruby
|
2140
|
+
# search not executed
|
2141
|
+
Product.search("milk")
|
1879
2142
|
|
1880
|
-
|
2143
|
+
# search executed
|
2144
|
+
Product.search("milk").to_a
|
2145
|
+
```
|
1881
2146
|
|
1882
|
-
|
2147
|
+
You can reindex relations in the background:
|
1883
2148
|
|
1884
2149
|
```ruby
|
1885
|
-
|
1886
|
-
|
2150
|
+
store.products.reindex(mode: :async)
|
2151
|
+
# or
|
2152
|
+
store.products.reindex(mode: :queue)
|
1887
2153
|
```
|
1888
2154
|
|
1889
|
-
|
2155
|
+
And there’s a [new option](#default-scopes) for models with default scopes.
|
1890
2156
|
|
1891
|
-
|
2157
|
+
Check out the [changelog](https://github.com/ankane/searchkick/blob/master/CHANGELOG.md#500-2022-02-21) for the full list of changes.
|
1892
2158
|
|
1893
|
-
|
1894
|
-
class Product < ActiveRecord::Base
|
1895
|
-
searchkick settings: {number_of_shards: 1}
|
1896
|
-
end
|
1897
|
-
```
|
2159
|
+
## History
|
1898
2160
|
|
1899
|
-
|
2161
|
+
View the [changelog](https://github.com/ankane/searchkick/blob/master/CHANGELOG.md).
|
1900
2162
|
|
1901
2163
|
## Thanks
|
1902
2164
|
|
1903
2165
|
Thanks to Karel Minarik for [Elasticsearch Ruby](https://github.com/elasticsearch/elasticsearch-ruby) and [Tire](https://github.com/karmi/retire), Jaroslav Kalistsuk for [zero downtime reindexing](https://gist.github.com/jarosan/3124884), and Alex Leschenko for [Elasticsearch autocomplete](https://github.com/leschenko/elasticsearch_autocomplete).
|
1904
2166
|
|
1905
|
-
## Roadmap
|
1906
|
-
|
1907
|
-
- Reindex API
|
1908
|
-
- Incorporate human eval
|
1909
|
-
|
1910
2167
|
## Contributing
|
1911
2168
|
|
1912
2169
|
Everyone is encouraged to help improve this project. Here are a few ways you can help:
|
@@ -1916,13 +2173,13 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
|
|
1916
2173
|
- Write, clarify, or fix documentation
|
1917
2174
|
- Suggest or add new features
|
1918
2175
|
|
1919
|
-
|
1920
|
-
|
1921
|
-
To get started with development and testing:
|
2176
|
+
To get started with development:
|
1922
2177
|
|
1923
2178
|
```sh
|
1924
2179
|
git clone https://github.com/ankane/searchkick.git
|
1925
2180
|
cd searchkick
|
1926
2181
|
bundle install
|
1927
|
-
rake test
|
2182
|
+
bundle exec rake test
|
1928
2183
|
```
|
2184
|
+
|
2185
|
+
Feel free to open an issue to get feedback on your idea before spending too much time on it.
|