searchkick 4.0.0 → 4.4.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +156 -96
- data/LICENSE.txt +1 -1
- data/README.md +324 -218
- data/lib/searchkick.rb +37 -14
- data/lib/searchkick/bulk_indexer.rb +5 -3
- data/lib/searchkick/index.rb +51 -5
- data/lib/searchkick/index_options.rb +479 -350
- data/lib/searchkick/logging.rb +11 -8
- data/lib/searchkick/model.rb +15 -9
- data/lib/searchkick/process_batch_job.rb +2 -2
- data/lib/searchkick/process_queue_job.rb +19 -11
- data/lib/searchkick/query.rb +129 -12
- data/lib/searchkick/results.rb +66 -7
- data/lib/searchkick/version.rb +1 -1
- data/lib/tasks/searchkick.rake +19 -12
- metadata +3 -46
- data/CONTRIBUTING.md +0 -53
data/LICENSE.txt
CHANGED
data/README.md
CHANGED
@@ -10,7 +10,7 @@ Searchkick handles:
|
|
10
10
|
- special characters - `jalapeno` matches `jalapeño`
|
11
11
|
- extra whitespace - `dishwasher` matches `dish washer`
|
12
12
|
- misspellings - `zuchini` matches `zucchini`
|
13
|
-
- custom synonyms - `
|
13
|
+
- custom synonyms - `pop` matches `soda`
|
14
14
|
|
15
15
|
Plus:
|
16
16
|
|
@@ -24,6 +24,8 @@ Plus:
|
|
24
24
|
|
25
25
|
:tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
|
26
26
|
|
27
|
+
:speech_balloon: Get [handcrafted updates](https://chartkick.us7.list-manage.com/subscribe?u=952c861f99eb43084e0a49f98&id=6ea6541e8e&group[0][4]=true) for new features
|
28
|
+
|
27
29
|
[![Build Status](https://travis-ci.org/ankane/searchkick.svg?branch=master)](https://travis-ci.org/ankane/searchkick)
|
28
30
|
|
29
31
|
## Contents
|
@@ -31,16 +33,19 @@ Plus:
|
|
31
33
|
- [Getting Started](#getting-started)
|
32
34
|
- [Querying](#querying)
|
33
35
|
- [Indexing](#indexing)
|
36
|
+
- [Intelligent Search](#intelligent-search)
|
34
37
|
- [Instant Search / Autocomplete](#instant-search--autocomplete)
|
35
38
|
- [Aggregations](#aggregations)
|
39
|
+
- [Testing](#testing)
|
36
40
|
- [Deployment](#deployment)
|
37
41
|
- [Performance](#performance)
|
38
42
|
- [Elasticsearch DSL](#advanced)
|
39
43
|
- [Reference](#reference)
|
44
|
+
- [Contributing](#contributing)
|
40
45
|
|
41
46
|
## Getting Started
|
42
47
|
|
43
|
-
[Install Elasticsearch](https://www.elastic.co/
|
48
|
+
[Install Elasticsearch](https://www.elastic.co/downloads/elasticsearch). For Homebrew, use:
|
44
49
|
|
45
50
|
```sh
|
46
51
|
brew install elasticsearch
|
@@ -98,14 +103,19 @@ Where
|
|
98
103
|
|
99
104
|
```ruby
|
100
105
|
where: {
|
101
|
-
expires_at: {gt: Time.now},
|
102
|
-
orders_count: 1..10,
|
103
|
-
aisle_id: [25, 30],
|
104
|
-
store_id: {not: 2},
|
105
|
-
aisle_id: {not: [25, 30]},
|
106
|
-
user_ids: {all: [1, 3]},
|
107
|
-
category:
|
108
|
-
|
106
|
+
expires_at: {gt: Time.now}, # lt, gte, lte also available
|
107
|
+
orders_count: 1..10, # equivalent to {gte: 1, lte: 10}
|
108
|
+
aisle_id: [25, 30], # in
|
109
|
+
store_id: {not: 2}, # not
|
110
|
+
aisle_id: {not: [25, 30]}, # not in
|
111
|
+
user_ids: {all: [1, 3]}, # all elements in array
|
112
|
+
category: {like: "%frozen%"}, # like
|
113
|
+
category: /frozen .+/, # regexp
|
114
|
+
category: {prefix: "frozen"}, # prefix
|
115
|
+
store_id: {exists: true}, # exists
|
116
|
+
_or: [{in_stock: true}, {backordered: true}],
|
117
|
+
_and: [{in_stock: true}, {backordered: true}],
|
118
|
+
_not: {store_id: 1} # negate a condition
|
109
119
|
}
|
110
120
|
```
|
111
121
|
|
@@ -129,7 +139,7 @@ Select
|
|
129
139
|
select: [:name]
|
130
140
|
```
|
131
141
|
|
132
|
-
[These source filtering options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-source-filtering
|
142
|
+
[These source filtering options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-source-filtering)
|
133
143
|
|
134
144
|
### Results
|
135
145
|
|
@@ -166,6 +176,8 @@ Get the full response from Elasticsearch
|
|
166
176
|
results.response
|
167
177
|
```
|
168
178
|
|
179
|
+
**Note:** By default, Elasticsearch [limits paging](#deep-paging-master) to the first 10,000 results for performance. With Elasticsearch 7, this applies to the total count as well.
|
180
|
+
|
169
181
|
### Boosting
|
170
182
|
|
171
183
|
Boost important fields
|
@@ -287,7 +299,9 @@ To only match the exact order, use:
|
|
287
299
|
User.search "fresh honey", match: :phrase
|
288
300
|
```
|
289
301
|
|
290
|
-
### Language
|
302
|
+
### Stemming and Language
|
303
|
+
|
304
|
+
Searchkick stems words by default for better matching. `apple` and `apples` both stem to `appl`, so searches for either term will have the same matches.
|
291
305
|
|
292
306
|
Searchkick defaults to English for stemming. To change this, use:
|
293
307
|
|
@@ -297,44 +311,91 @@ class Product < ApplicationRecord
|
|
297
311
|
end
|
298
312
|
```
|
299
313
|
|
300
|
-
|
301
|
-
|
302
|
-
A few languages require plugins:
|
314
|
+
See the [list of stemmers](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-stemmer-tokenfilter.html). A few languages require plugins:
|
303
315
|
|
304
316
|
- `chinese` - [analysis-ik plugin](https://github.com/medcl/elasticsearch-analysis-ik)
|
305
|
-
- `
|
317
|
+
- `chinese2` - [analysis-smartcn plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/7.4/analysis-smartcn.html)
|
318
|
+
- `japanese` - [analysis-kuromoji plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/7.4/analysis-kuromoji.html)
|
306
319
|
- `korean` - [analysis-openkoreantext plugin](https://github.com/open-korean-text/elasticsearch-analysis-openkoreantext)
|
307
|
-
- `
|
308
|
-
- `
|
320
|
+
- `korean2` - [analysis-nori plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/7.4/analysis-nori.html)
|
321
|
+
- `polish` - [analysis-stempel plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/7.4/analysis-stempel.html)
|
322
|
+
- `ukrainian` - [analysis-ukrainian plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/7.4/analysis-ukrainian.html)
|
309
323
|
- `vietnamese` - [analysis-vietnamese plugin](https://github.com/duydo/elasticsearch-analysis-vietnamese)
|
310
324
|
|
325
|
+
Disable stemming with:
|
326
|
+
|
327
|
+
```ruby
|
328
|
+
class Image < ApplicationRecord
|
329
|
+
searchkick stem: false
|
330
|
+
end
|
331
|
+
```
|
332
|
+
|
333
|
+
Exclude certain words from stemming with:
|
334
|
+
|
335
|
+
```ruby
|
336
|
+
class Image < ApplicationRecord
|
337
|
+
searchkick stem_exclusion: ["apples"]
|
338
|
+
end
|
339
|
+
```
|
340
|
+
|
341
|
+
Or change how words are stemmed:
|
342
|
+
|
343
|
+
```ruby
|
344
|
+
class Image < ApplicationRecord
|
345
|
+
searchkick stemmer_override: ["apples => other"]
|
346
|
+
end
|
347
|
+
```
|
348
|
+
|
311
349
|
### Synonyms
|
312
350
|
|
313
351
|
```ruby
|
314
352
|
class Product < ApplicationRecord
|
315
|
-
searchkick
|
353
|
+
searchkick search_synonyms: [["pop", "soda"], ["burger", "hamburger"]]
|
316
354
|
end
|
317
355
|
```
|
318
356
|
|
319
|
-
Call `Product.reindex` after changing synonyms.
|
357
|
+
Call `Product.reindex` after changing synonyms. Synonyms are applied at search time before stemming, and can be a single word or multiple words.
|
320
358
|
|
321
|
-
|
359
|
+
For directional synonyms, use:
|
360
|
+
|
361
|
+
```ruby
|
362
|
+
search_synonyms: ["lightbulb => halogenlamp"]
|
363
|
+
```
|
364
|
+
|
365
|
+
### Dynamic Synonyms
|
366
|
+
|
367
|
+
The above approach works well when your synonym list is static, but in practice, this is often not the case. When you analyze search conversions, you often want to add new synonyms without a full reindex.
|
322
368
|
|
323
|
-
|
369
|
+
#### Elasticsearch 7.3+
|
370
|
+
|
371
|
+
For Elasticsearch 7.3+, we recommend placing synonyms in a file on the Elasticsearch server (in the `config` directory). This allows you to reload synonyms without reindexing.
|
372
|
+
|
373
|
+
```txt
|
374
|
+
pop, soda
|
375
|
+
burger, hamburger
|
376
|
+
```
|
377
|
+
|
378
|
+
Then use:
|
324
379
|
|
325
380
|
```ruby
|
326
|
-
|
381
|
+
search_synonyms: "synonyms.txt"
|
327
382
|
```
|
328
383
|
|
329
|
-
|
384
|
+
Add [elasticsearch-xpack](https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch-xpack) to your Gemfile:
|
330
385
|
|
331
386
|
```ruby
|
332
|
-
|
387
|
+
gem 'elasticsearch-xpack', '>= 7.8.0'
|
333
388
|
```
|
334
389
|
|
335
|
-
|
390
|
+
And use:
|
391
|
+
|
392
|
+
```ruby
|
393
|
+
Product.search_index.reload_synonyms
|
394
|
+
```
|
395
|
+
|
396
|
+
#### Elasticsearch < 7.3
|
336
397
|
|
337
|
-
|
398
|
+
You can use a library like [ActsAsTaggableOn](https://github.com/mbleigh/acts-as-taggable-on) and do:
|
338
399
|
|
339
400
|
```ruby
|
340
401
|
class Product < ApplicationRecord
|
@@ -486,7 +547,7 @@ For large data sets, try [parallel reindexing](#parallel-reindexing).
|
|
486
547
|
|
487
548
|
- app starts
|
488
549
|
|
489
|
-
###
|
550
|
+
### Strategies
|
490
551
|
|
491
552
|
There are four strategies for keeping the index synced with your database.
|
492
553
|
|
@@ -536,7 +597,7 @@ Searchkick.callbacks(false) do
|
|
536
597
|
end
|
537
598
|
```
|
538
599
|
|
539
|
-
|
600
|
+
### Associations
|
540
601
|
|
541
602
|
Data is **not** automatically synced when an association is updated. If this is desired, add a callback to reindex:
|
542
603
|
|
@@ -552,11 +613,9 @@ class Image < ApplicationRecord
|
|
552
613
|
end
|
553
614
|
```
|
554
615
|
|
555
|
-
|
556
|
-
|
557
|
-
The best starting point to improve your search **by far** is to track searches and conversions.
|
616
|
+
## Intelligent Search
|
558
617
|
|
559
|
-
[Searchjoy](https://github.com/ankane/searchjoy) makes it easy.
|
618
|
+
The best starting point to improve your search **by far** is to track searches and conversions. [Searchjoy](https://github.com/ankane/searchjoy) makes it easy.
|
560
619
|
|
561
620
|
```ruby
|
562
621
|
Product.search "apple", track: {user_id: current_user.id}
|
@@ -569,15 +628,9 @@ Focus on:
|
|
569
628
|
- top searches with low conversions
|
570
629
|
- top searches with no results
|
571
630
|
|
572
|
-
|
631
|
+
Searchkick can then use the conversion data to learn what users are looking for. If a user searches for “ice cream” and adds Ben & Jerry’s Chunky Monkey to the cart (our conversion metric at Instacart), that item gets a little more weight for similar searches.
|
573
632
|
|
574
|
-
|
575
|
-
|
576
|
-
The first step is to define your conversion metric and start tracking conversions. The database works well for low volume, but feel free to use Redis or another datastore.
|
577
|
-
|
578
|
-
Searchkick automatically treats `apple` and `APPLE` the same.
|
579
|
-
|
580
|
-
Next, add conversions to the index.
|
633
|
+
Add conversion data with:
|
581
634
|
|
582
635
|
```ruby
|
583
636
|
class Product < ApplicationRecord
|
@@ -601,9 +654,11 @@ Reindex and set up a cron job to add new conversions daily.
|
|
601
654
|
rake searchkick:reindex CLASS=Product
|
602
655
|
```
|
603
656
|
|
604
|
-
|
657
|
+
This can make a huge difference on the quality of your search.
|
605
658
|
|
606
|
-
|
659
|
+
For a more performant way to reindex conversion data, check out [performant conversions](#performant-conversions).
|
660
|
+
|
661
|
+
## Personalized Results
|
607
662
|
|
608
663
|
Order results differently for each user. For example, show a user’s previously purchased products before other results.
|
609
664
|
|
@@ -624,13 +679,13 @@ Reindex and search with:
|
|
624
679
|
Product.search "milk", boost_where: {orderer_ids: current_user.id}
|
625
680
|
```
|
626
681
|
|
627
|
-
|
682
|
+
## Instant Search / Autocomplete
|
628
683
|
|
629
684
|
Autocomplete predicts what a user will type, making the search experience faster and easier.
|
630
685
|
|
631
686
|
![Autocomplete](https://gist.github.com/ankane/b6988db2802aca68a589b31e41b44195/raw/40febe948427e5bc53ec4e5dc248822855fef76f/autocomplete.png)
|
632
687
|
|
633
|
-
**Note:** To autocomplete on
|
688
|
+
**Note:** To autocomplete on search terms rather than results, check out [Autosuggest](https://github.com/ankane/autosuggest).
|
634
689
|
|
635
690
|
**Note 2:** If you only have a few thousand records, don’t use Searchkick for autocomplete. It’s *much* faster to load all records into JavaScript and autocomplete there (eliminates network requests).
|
636
691
|
|
@@ -692,7 +747,7 @@ Then add the search box and JavaScript code to a view.
|
|
692
747
|
</script>
|
693
748
|
```
|
694
749
|
|
695
|
-
|
750
|
+
## Suggestions
|
696
751
|
|
697
752
|
![Suggest](https://gist.github.com/ankane/b6988db2802aca68a589b31e41b44195/raw/40febe948427e5bc53ec4e5dc248822855fef76f/recursion.png)
|
698
753
|
|
@@ -709,7 +764,7 @@ products = Product.search "peantu butta", suggest: true
|
|
709
764
|
products.suggestions # ["peanut butter"]
|
710
765
|
```
|
711
766
|
|
712
|
-
|
767
|
+
## Aggregations
|
713
768
|
|
714
769
|
[Aggregations](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html) provide aggregated search data.
|
715
770
|
|
@@ -752,8 +807,6 @@ Order
|
|
752
807
|
Product.search "wingtips", aggs: {color: {order: {"_key" => "asc"}}} # alphabetically
|
753
808
|
```
|
754
809
|
|
755
|
-
**Note:** Use `_term` instead of `_key` in Elasticsearch 5
|
756
|
-
|
757
810
|
[All of these options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-order)
|
758
811
|
|
759
812
|
Ranges
|
@@ -784,10 +837,10 @@ Product.search "pear", aggs: {products_per_year: {date_histogram: {field: :creat
|
|
784
837
|
For other aggregation types, including sub-aggregations, use `body_options`:
|
785
838
|
|
786
839
|
```ruby
|
787
|
-
Product.search "orange", body_options: {aggs: {price: {histogram: {field: :price, interval: 10}}}
|
840
|
+
Product.search "orange", body_options: {aggs: {price: {histogram: {field: :price, interval: 10}}}}
|
788
841
|
```
|
789
842
|
|
790
|
-
|
843
|
+
## Highlight
|
791
844
|
|
792
845
|
Specify which fields to index with highlighting.
|
793
846
|
|
@@ -840,7 +893,7 @@ Band.search "cinema", fields: [:name], highlight: {fields: {name: {fragment_size
|
|
840
893
|
|
841
894
|
You can find available highlight options in the [Elasticsearch reference](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-highlighting.html#_highlighted_fragments).
|
842
895
|
|
843
|
-
|
896
|
+
## Similar Items
|
844
897
|
|
845
898
|
Find similar items.
|
846
899
|
|
@@ -849,7 +902,7 @@ product = Product.first
|
|
849
902
|
product.similar(fields: [:name], where: {size: "12 oz"})
|
850
903
|
```
|
851
904
|
|
852
|
-
|
905
|
+
## Geospatial Searches
|
853
906
|
|
854
907
|
```ruby
|
855
908
|
class Restaurant < ApplicationRecord
|
@@ -1016,13 +1069,165 @@ Product.search_index.tokens("dieg", analyzer: "searchkick_word_search")
|
|
1016
1069
|
|
1017
1070
|
See the [complete list of analyzers](https://github.com/ankane/searchkick/blob/31780ddac7a89eab1e0552a32b403f2040a37931/lib/searchkick/index_options.rb#L32).
|
1018
1071
|
|
1072
|
+
## Testing
|
1073
|
+
|
1074
|
+
As you iterate on your search, it’s a good idea to add tests.
|
1075
|
+
|
1076
|
+
For performance, only enable Searchkick callbacks for the tests that need it.
|
1077
|
+
|
1078
|
+
### Parallel Tests
|
1079
|
+
|
1080
|
+
Rails 6 enables parallel tests by default. Add to your `test/test_helper.rb`:
|
1081
|
+
|
1082
|
+
```ruby
|
1083
|
+
class ActiveSupport::TestCase
|
1084
|
+
parallelize_setup do |worker|
|
1085
|
+
Searchkick.index_suffix = worker
|
1086
|
+
|
1087
|
+
# reindex models
|
1088
|
+
Product.reindex
|
1089
|
+
|
1090
|
+
# and disable callbacks
|
1091
|
+
Searchkick.disable_callbacks
|
1092
|
+
end
|
1093
|
+
end
|
1094
|
+
```
|
1095
|
+
|
1096
|
+
And use:
|
1097
|
+
|
1098
|
+
```ruby
|
1099
|
+
class ProductTest < ActiveSupport::TestCase
|
1100
|
+
def setup
|
1101
|
+
Searchkick.enable_callbacks
|
1102
|
+
end
|
1103
|
+
|
1104
|
+
def teardown
|
1105
|
+
Searchkick.disable_callbacks
|
1106
|
+
end
|
1107
|
+
|
1108
|
+
def test_search
|
1109
|
+
Product.create!(name: "Apple")
|
1110
|
+
Product.search_index.refresh
|
1111
|
+
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1112
|
+
end
|
1113
|
+
end
|
1114
|
+
```
|
1115
|
+
|
1116
|
+
### Minitest
|
1117
|
+
|
1118
|
+
Add to your `test/test_helper.rb`:
|
1119
|
+
|
1120
|
+
```ruby
|
1121
|
+
# reindex models
|
1122
|
+
Product.reindex
|
1123
|
+
|
1124
|
+
# and disable callbacks
|
1125
|
+
Searchkick.disable_callbacks
|
1126
|
+
```
|
1127
|
+
|
1128
|
+
And use:
|
1129
|
+
|
1130
|
+
```ruby
|
1131
|
+
class ProductTest < Minitest::Test
|
1132
|
+
def setup
|
1133
|
+
Searchkick.enable_callbacks
|
1134
|
+
end
|
1135
|
+
|
1136
|
+
def teardown
|
1137
|
+
Searchkick.disable_callbacks
|
1138
|
+
end
|
1139
|
+
|
1140
|
+
def test_search
|
1141
|
+
Product.create!(name: "Apple")
|
1142
|
+
Product.search_index.refresh
|
1143
|
+
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1144
|
+
end
|
1145
|
+
end
|
1146
|
+
```
|
1147
|
+
|
1148
|
+
### RSpec
|
1149
|
+
|
1150
|
+
Add to your `spec/spec_helper.rb`:
|
1151
|
+
|
1152
|
+
```ruby
|
1153
|
+
RSpec.configure do |config|
|
1154
|
+
config.before(:suite) do
|
1155
|
+
# reindex models
|
1156
|
+
Product.reindex
|
1157
|
+
|
1158
|
+
# and disable callbacks
|
1159
|
+
Searchkick.disable_callbacks
|
1160
|
+
end
|
1161
|
+
|
1162
|
+
config.around(:each, search: true) do |example|
|
1163
|
+
Searchkick.callbacks(nil) do
|
1164
|
+
example.run
|
1165
|
+
end
|
1166
|
+
end
|
1167
|
+
end
|
1168
|
+
```
|
1169
|
+
|
1170
|
+
And use:
|
1171
|
+
|
1172
|
+
```ruby
|
1173
|
+
describe Product, search: true do
|
1174
|
+
it "searches" do
|
1175
|
+
Product.create!(name: "Apple")
|
1176
|
+
Product.search_index.refresh
|
1177
|
+
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1178
|
+
end
|
1179
|
+
end
|
1180
|
+
```
|
1181
|
+
|
1182
|
+
### Factory Bot
|
1183
|
+
|
1184
|
+
Use a trait and an after `create` hook for each indexed model:
|
1185
|
+
|
1186
|
+
```ruby
|
1187
|
+
FactoryBot.define do
|
1188
|
+
factory :product do
|
1189
|
+
# ...
|
1190
|
+
|
1191
|
+
# Note: This should be the last trait in the list so `reindex` is called
|
1192
|
+
# after all the other callbacks complete.
|
1193
|
+
trait :reindex do
|
1194
|
+
after(:create) do |product, _evaluator|
|
1195
|
+
product.reindex(refresh: true)
|
1196
|
+
end
|
1197
|
+
end
|
1198
|
+
end
|
1199
|
+
end
|
1200
|
+
|
1201
|
+
# use it
|
1202
|
+
FactoryBot.create(:product, :some_trait, :reindex, some_attribute: "foo")
|
1203
|
+
```
|
1204
|
+
|
1019
1205
|
## Deployment
|
1020
1206
|
|
1021
1207
|
Searchkick uses `ENV["ELASTICSEARCH_URL"]` for the Elasticsearch server. This defaults to `http://localhost:9200`.
|
1022
1208
|
|
1209
|
+
- [Elastic Cloud](#elastic-cloud)
|
1210
|
+
- [Heroku](#heroku)
|
1211
|
+
- [Amazon Elasticsearch Service](#amazon-elasticsearch-service)
|
1212
|
+
- [Self-Hosted and Other](#other)
|
1213
|
+
|
1214
|
+
### Elastic Cloud
|
1215
|
+
|
1216
|
+
Create an initializer `config/initializers/elasticsearch.rb` with:
|
1217
|
+
|
1218
|
+
```ruby
|
1219
|
+
ENV["ELASTICSEARCH_URL"] = "https://user:password@host:port"
|
1220
|
+
```
|
1221
|
+
|
1222
|
+
Then deploy and reindex:
|
1223
|
+
|
1224
|
+
```sh
|
1225
|
+
rake searchkick:reindex:all
|
1226
|
+
```
|
1227
|
+
|
1023
1228
|
### Heroku
|
1024
1229
|
|
1025
|
-
Choose an add-on: [Bonsai](https://elements.heroku.com/addons/bonsai)
|
1230
|
+
Choose an add-on: [Bonsai](https://elements.heroku.com/addons/bonsai), [SearchBox](https://elements.heroku.com/addons/searchbox), or [Elastic Cloud](https://elements.heroku.com/addons/foundelasticsearch).
|
1026
1231
|
|
1027
1232
|
For Bonsai:
|
1028
1233
|
|
@@ -1031,6 +1236,13 @@ heroku addons:create bonsai
|
|
1031
1236
|
heroku config:set ELASTICSEARCH_URL=`heroku config:get BONSAI_URL`
|
1032
1237
|
```
|
1033
1238
|
|
1239
|
+
For SearchBox:
|
1240
|
+
|
1241
|
+
```sh
|
1242
|
+
heroku addons:create searchbox:starter
|
1243
|
+
heroku config:set ELASTICSEARCH_URL=`heroku config:get SEARCHBOX_URL`
|
1244
|
+
```
|
1245
|
+
|
1034
1246
|
For Elastic Cloud (previously Found):
|
1035
1247
|
|
1036
1248
|
```sh
|
@@ -1044,16 +1256,16 @@ Visit the Shield page and reset your password. You’ll need to add the username
|
|
1044
1256
|
heroku config:get FOUNDELASTICSEARCH_URL
|
1045
1257
|
```
|
1046
1258
|
|
1047
|
-
And add `elastic:password@` right after `https
|
1259
|
+
And add `elastic:password@` right after `https://` and add port `9243` at the end:
|
1048
1260
|
|
1049
1261
|
```sh
|
1050
|
-
heroku config:set ELASTICSEARCH_URL=https://elastic:password@12345.us-east-1.aws.found.io
|
1262
|
+
heroku config:set ELASTICSEARCH_URL=https://elastic:password@12345.us-east-1.aws.found.io:9243
|
1051
1263
|
```
|
1052
1264
|
|
1053
1265
|
Then deploy and reindex:
|
1054
1266
|
|
1055
1267
|
```sh
|
1056
|
-
heroku run rake searchkick:reindex
|
1268
|
+
heroku run rake searchkick:reindex:all
|
1057
1269
|
```
|
1058
1270
|
|
1059
1271
|
### Amazon Elasticsearch Service
|
@@ -1061,7 +1273,7 @@ heroku run rake searchkick:reindex CLASS=Product
|
|
1061
1273
|
Create an initializer `config/initializers/elasticsearch.rb` with:
|
1062
1274
|
|
1063
1275
|
```ruby
|
1064
|
-
ENV["ELASTICSEARCH_URL"] = "https://es-domain-1234.us-east-1.es.amazonaws.com"
|
1276
|
+
ENV["ELASTICSEARCH_URL"] = "https://es-domain-1234.us-east-1.es.amazonaws.com:443"
|
1065
1277
|
```
|
1066
1278
|
|
1067
1279
|
To use signed requests, include in your Gemfile:
|
@@ -1083,21 +1295,21 @@ Searchkick.aws_credentials = {
|
|
1083
1295
|
Then deploy and reindex:
|
1084
1296
|
|
1085
1297
|
```sh
|
1086
|
-
rake searchkick:reindex
|
1298
|
+
rake searchkick:reindex:all
|
1087
1299
|
```
|
1088
1300
|
|
1089
|
-
### Other
|
1301
|
+
### Self-Hosted and Other
|
1090
1302
|
|
1091
1303
|
Create an initializer `config/initializers/elasticsearch.rb` with:
|
1092
1304
|
|
1093
1305
|
```ruby
|
1094
|
-
ENV["ELASTICSEARCH_URL"] = "https://user:password@host"
|
1306
|
+
ENV["ELASTICSEARCH_URL"] = "https://user:password@host:port"
|
1095
1307
|
```
|
1096
1308
|
|
1097
1309
|
Then deploy and reindex:
|
1098
1310
|
|
1099
1311
|
```sh
|
1100
|
-
rake searchkick:reindex
|
1312
|
+
rake searchkick:reindex:all
|
1101
1313
|
```
|
1102
1314
|
|
1103
1315
|
### Data Protection
|
@@ -1160,7 +1372,7 @@ If you run into issues on Windows, check out [this post](https://www.rastating.c
|
|
1160
1372
|
|
1161
1373
|
### Searchable Fields
|
1162
1374
|
|
1163
|
-
By default, all string fields are searchable (can be used in `fields` option). Speed up indexing and reduce index size by only making some fields searchable.
|
1375
|
+
By default, all string fields are searchable (can be used in `fields` option). Speed up indexing and reduce index size by only making some fields searchable.
|
1164
1376
|
|
1165
1377
|
```ruby
|
1166
1378
|
class Product < ApplicationRecord
|
@@ -1396,10 +1608,8 @@ Create a custom mapping:
|
|
1396
1608
|
```ruby
|
1397
1609
|
class Product < ApplicationRecord
|
1398
1610
|
searchkick mappings: {
|
1399
|
-
|
1400
|
-
|
1401
|
-
name: {type: "keyword"}
|
1402
|
-
}
|
1611
|
+
properties: {
|
1612
|
+
name: {type: "keyword"}
|
1403
1613
|
}
|
1404
1614
|
}
|
1405
1615
|
end
|
@@ -1479,6 +1689,49 @@ Boost specific models with:
|
|
1479
1689
|
indices_boost: {Category => 2, Product => 1}
|
1480
1690
|
```
|
1481
1691
|
|
1692
|
+
## Multi-Tenancy
|
1693
|
+
|
1694
|
+
Check out [this great post](https://www.tiagoamaro.com.br/2014/12/11/multi-tenancy-with-searchkick/) on the [Apartment](https://github.com/influitive/apartment) gem. Follow a similar pattern if you use another gem.
|
1695
|
+
|
1696
|
+
## Scroll API
|
1697
|
+
|
1698
|
+
Searchkick also supports the [scroll API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html). Scrolling is not intended for real time user requests, but rather for processing large amounts of data.
|
1699
|
+
|
1700
|
+
```ruby
|
1701
|
+
Product.search("*", scroll: "1m").scroll do |batch|
|
1702
|
+
# process batch ...
|
1703
|
+
end
|
1704
|
+
```
|
1705
|
+
|
1706
|
+
You can also scroll batches manually.
|
1707
|
+
|
1708
|
+
```ruby
|
1709
|
+
products = Product.search "*", scroll: "1m"
|
1710
|
+
while products.any?
|
1711
|
+
# process batch ...
|
1712
|
+
|
1713
|
+
products = products.scroll
|
1714
|
+
end
|
1715
|
+
|
1716
|
+
products.clear_scroll
|
1717
|
+
```
|
1718
|
+
|
1719
|
+
## Deep Paging
|
1720
|
+
|
1721
|
+
By default, Elasticsearch limits paging to the first 10,000 results. [Here’s why](https://www.elastic.co/guide/en/elasticsearch/guide/current/pagination.html). We don’t recommend changing this, but if you really need all results, you can use:
|
1722
|
+
|
1723
|
+
```ruby
|
1724
|
+
class Product < ApplicationRecord
|
1725
|
+
searchkick deep_paging: true
|
1726
|
+
end
|
1727
|
+
```
|
1728
|
+
|
1729
|
+
If you just need an accurate total count with Elasticsearch 7, you can instead use:
|
1730
|
+
|
1731
|
+
```ruby
|
1732
|
+
Product.search("pears", body_options: {track_total_hits: true})
|
1733
|
+
```
|
1734
|
+
|
1482
1735
|
## Nested Data
|
1483
1736
|
|
1484
1737
|
To query nested data, use dot notation.
|
@@ -1646,14 +1899,6 @@ class Product < ApplicationRecord
|
|
1646
1899
|
end
|
1647
1900
|
```
|
1648
1901
|
|
1649
|
-
Turn off stemming
|
1650
|
-
|
1651
|
-
```ruby
|
1652
|
-
class Product < ApplicationRecord
|
1653
|
-
searchkick stem: false
|
1654
|
-
end
|
1655
|
-
```
|
1656
|
-
|
1657
1902
|
Turn on stemming for conversions
|
1658
1903
|
|
1659
1904
|
```ruby
|
@@ -1662,14 +1907,6 @@ class Product < ApplicationRecord
|
|
1662
1907
|
end
|
1663
1908
|
```
|
1664
1909
|
|
1665
|
-
Use a different [similarity algorithm](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html) for scoring
|
1666
|
-
|
1667
|
-
```ruby
|
1668
|
-
class Product < ApplicationRecord
|
1669
|
-
searchkick similarity: "classic"
|
1670
|
-
end
|
1671
|
-
```
|
1672
|
-
|
1673
1910
|
Make search case-sensitive
|
1674
1911
|
|
1675
1912
|
```ruby
|
@@ -1754,142 +1991,11 @@ Product.search "api", misspellings: {prefix_length: 2} # api, apt, no ahi
|
|
1754
1991
|
Product.search "ah", misspellings: {prefix_length: 2} # ah, no aha
|
1755
1992
|
```
|
1756
1993
|
|
1757
|
-
## Testing
|
1758
|
-
|
1759
|
-
For performance, only enable Searchkick callbacks for the tests that need it.
|
1760
|
-
|
1761
|
-
### Minitest
|
1762
|
-
|
1763
|
-
Add to your `test/test_helper.rb`:
|
1764
|
-
|
1765
|
-
```ruby
|
1766
|
-
# reindex models
|
1767
|
-
Product.reindex
|
1768
|
-
|
1769
|
-
# and disable callbacks
|
1770
|
-
Searchkick.disable_callbacks
|
1771
|
-
```
|
1772
|
-
|
1773
|
-
And use:
|
1774
|
-
|
1775
|
-
```ruby
|
1776
|
-
class ProductTest < Minitest::Test
|
1777
|
-
def setup
|
1778
|
-
Searchkick.enable_callbacks
|
1779
|
-
end
|
1780
|
-
|
1781
|
-
def teardown
|
1782
|
-
Searchkick.disable_callbacks
|
1783
|
-
end
|
1784
|
-
|
1785
|
-
def test_search
|
1786
|
-
Product.create!(name: "Apple")
|
1787
|
-
Product.search_index.refresh
|
1788
|
-
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1789
|
-
end
|
1790
|
-
end
|
1791
|
-
```
|
1792
|
-
|
1793
|
-
### RSpec
|
1794
|
-
|
1795
|
-
Add to your `spec/spec_helper.rb`:
|
1796
|
-
|
1797
|
-
```ruby
|
1798
|
-
RSpec.configure do |config|
|
1799
|
-
config.before(:suite) do
|
1800
|
-
# reindex models
|
1801
|
-
Product.reindex
|
1802
|
-
|
1803
|
-
# and disable callbacks
|
1804
|
-
Searchkick.disable_callbacks
|
1805
|
-
end
|
1806
|
-
|
1807
|
-
config.around(:each, search: true) do |example|
|
1808
|
-
Searchkick.callbacks(true) do
|
1809
|
-
example.run
|
1810
|
-
end
|
1811
|
-
end
|
1812
|
-
end
|
1813
|
-
```
|
1814
|
-
|
1815
|
-
And use:
|
1816
|
-
|
1817
|
-
```ruby
|
1818
|
-
describe Product, search: true do
|
1819
|
-
it "searches" do
|
1820
|
-
Product.create!(name: "Apple")
|
1821
|
-
Product.search_index.refresh
|
1822
|
-
assert_equal ["Apple"], Product.search("apple").map(&:name)
|
1823
|
-
end
|
1824
|
-
end
|
1825
|
-
```
|
1826
|
-
|
1827
|
-
### Factory Bot
|
1828
|
-
|
1829
|
-
Use a trait and an after `create` hook for each indexed model:
|
1830
|
-
|
1831
|
-
```ruby
|
1832
|
-
FactoryBot.define do
|
1833
|
-
factory :product do
|
1834
|
-
# ...
|
1835
|
-
|
1836
|
-
# Note: This should be the last trait in the list so `reindex` is called
|
1837
|
-
# after all the other callbacks complete.
|
1838
|
-
trait :reindex do
|
1839
|
-
after(:create) do |product, _evaluator|
|
1840
|
-
product.reindex(refresh: true)
|
1841
|
-
end
|
1842
|
-
end
|
1843
|
-
end
|
1844
|
-
end
|
1845
|
-
|
1846
|
-
# use it
|
1847
|
-
FactoryBot.create(:product, :some_trait, :reindex, some_attribute: "foo")
|
1848
|
-
```
|
1849
|
-
|
1850
|
-
### Parallel Tests
|
1851
|
-
|
1852
|
-
Set:
|
1853
|
-
|
1854
|
-
```ruby
|
1855
|
-
Searchkick.index_suffix = ENV["TEST_ENV_NUMBER"]
|
1856
|
-
```
|
1857
|
-
|
1858
|
-
## Multi-Tenancy
|
1859
|
-
|
1860
|
-
Check out [this great post](https://www.tiagoamaro.com.br/2014/12/11/multi-tenancy-with-searchkick/) on the [Apartment](https://github.com/influitive/apartment) gem. Follow a similar pattern if you use another gem.
|
1861
|
-
|
1862
|
-
## Upgrading
|
1863
|
-
|
1864
|
-
See [how to upgrade to Searchkick 3](docs/Searchkick-3-Upgrade.md)
|
1865
|
-
|
1866
1994
|
## Elasticsearch 6 to 7 Upgrade
|
1867
1995
|
|
1868
1996
|
1. Install Searchkick 4
|
1869
1997
|
2. Upgrade your Elasticsearch cluster
|
1870
1998
|
|
1871
|
-
## Elasticsearch 5 to 6 Upgrade
|
1872
|
-
|
1873
|
-
Elasticsearch 6 removes the ability to reindex with the `_all` field. Before you upgrade, we recommend disabling this field manually and specifying default fields on your models.
|
1874
|
-
|
1875
|
-
```ruby
|
1876
|
-
class Product < ApplicationRecord
|
1877
|
-
searchkick _all: false, default_fields: [:name]
|
1878
|
-
end
|
1879
|
-
```
|
1880
|
-
|
1881
|
-
If you need search across multiple fields, we recommend creating a similar field in your search data.
|
1882
|
-
|
1883
|
-
```ruby
|
1884
|
-
class Product < ApplicationRecord
|
1885
|
-
def search_data
|
1886
|
-
{
|
1887
|
-
all: [name, size, quantity].join(" ")
|
1888
|
-
}
|
1889
|
-
end
|
1890
|
-
end
|
1891
|
-
```
|
1892
|
-
|
1893
1999
|
## Elasticsearch Gotchas
|
1894
2000
|
|
1895
2001
|
### Consistency
|
@@ -1930,13 +2036,13 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
|
|
1930
2036
|
- Write, clarify, or fix documentation
|
1931
2037
|
- Suggest or add new features
|
1932
2038
|
|
1933
|
-
|
1934
|
-
|
1935
|
-
To get started with development and testing:
|
2039
|
+
To get started with development:
|
1936
2040
|
|
1937
2041
|
```sh
|
1938
2042
|
git clone https://github.com/ankane/searchkick.git
|
1939
2043
|
cd searchkick
|
1940
2044
|
bundle install
|
1941
|
-
rake test
|
2045
|
+
bundle exec rake test
|
1942
2046
|
```
|
2047
|
+
|
2048
|
+
Feel free to open an issue to get feedback on your idea before spending too much time on it.
|