searchkick 4.6.0 → 5.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +42 -0
- data/README.md +127 -77
- data/lib/searchkick/bulk_reindex_job.rb +12 -8
- data/lib/searchkick/controller_runtime.rb +40 -0
- data/lib/searchkick/index.rb +134 -63
- data/lib/searchkick/index_cache.rb +30 -0
- data/lib/searchkick/index_options.rb +16 -66
- data/lib/searchkick/indexer.rb +15 -8
- data/lib/searchkick/log_subscriber.rb +57 -0
- data/lib/searchkick/middleware.rb +1 -1
- data/lib/searchkick/model.rb +44 -47
- data/lib/searchkick/process_batch_job.rb +9 -25
- data/lib/searchkick/process_queue_job.rb +3 -2
- data/lib/searchkick/query.rb +67 -65
- data/lib/searchkick/record_data.rb +0 -1
- data/lib/searchkick/record_indexer.rb +135 -50
- data/lib/searchkick/reindex_queue.rb +25 -2
- data/lib/searchkick/reindex_v2_job.rb +10 -34
- data/lib/searchkick/relation.rb +36 -0
- data/lib/searchkick/relation_indexer.rb +150 -0
- data/lib/searchkick/results.rb +26 -27
- data/lib/searchkick/version.rb +1 -1
- data/lib/searchkick.rb +167 -79
- data/lib/tasks/searchkick.rake +6 -3
- metadata +11 -28
- data/lib/searchkick/bulk_indexer.rb +0 -173
- data/lib/searchkick/logging.rb +0 -246
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: bc223f746f73c44e5c6f56b5e351cb31ae5a655069d92f5bfd2a0b2e531afae8
|
4
|
+
data.tar.gz: 1969964f34a5adaf3ed746230fbef57673454f7304d890466e2814d27eb2cb47
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 17259c3e071d69c3852584aba480b523211b167f1249951f32d1bd2f761fc07322144283f0ccabac4878c0a2ff2f4c9dd8def2db2beac004c449839052e503ea
|
7
|
+
data.tar.gz: 1109a53d99eb30f7e76cab8df00505c82af5e898c4e3f1821773932b8f6917e68f6bfedfa952b8e4acb25387607edf10856cc994021943671086af4d751cc728
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,45 @@
|
|
1
|
+
## 5.0.0 (2022-02-21)
|
2
|
+
|
3
|
+
- Searches now use lazy loading (similar to Active Record)
|
4
|
+
- Added `unscope` option to better support working with default scopes
|
5
|
+
- Added support for `:async` and `:queue` modes for `reindex` on relation
|
6
|
+
- Added basic protection from unfiltered parameters to `where` option
|
7
|
+
- Added `models` option to `similar` method
|
8
|
+
- Changed async full reindex to fetch ids instead of using ranges for numeric primary keys with Active Record
|
9
|
+
- Changed `searchkick_index_options` to return symbol keys (instead of mix of strings and symbols)
|
10
|
+
- Changed non-anchored regular expressions to match expected results (previously warned)
|
11
|
+
- Changed record reindex to return `true` to match model and relation reindex
|
12
|
+
- Updated async reindex job to call `search_import` for nested associations
|
13
|
+
- Fixed removing records when `should_index?` is `false` when `reindex` called on relation
|
14
|
+
- Fixed issue with `merge_mappings` for fields that use `searchkick` options
|
15
|
+
- Raise error when `search` called on relations
|
16
|
+
- Raise `ArgumentError` (instead of warning) for invalid regular expression modifiers
|
17
|
+
- Raise `ArgumentError` instead of `RuntimeError` for unknown operators
|
18
|
+
- Removed mapping of `id` to `_id` with `order` option
|
19
|
+
- Removed `wordnet` option (no longer worked)
|
20
|
+
- Removed dependency on `elasticsearch` gem (can use `elasticsearch` or `opensearch-ruby`)
|
21
|
+
- Dropped support for Elasticsearch 6
|
22
|
+
- Dropped support for Ruby < 2.6 and Active Record < 5.2
|
23
|
+
- Dropped support for NoBrainer and Cequel
|
24
|
+
- Dropped support for `faraday_middleware-aws-signers-v4` (use `faraday_middleware-aws-sigv4` instead)
|
25
|
+
|
26
|
+
## 4.6.3 (2021-11-19)
|
27
|
+
|
28
|
+
- Added support for reloadable synonyms for OpenSearch
|
29
|
+
- Added experimental support for `opensearch-ruby` gem
|
30
|
+
- Removed `elasticsearch-xpack` dependency for reloadable synonyms
|
31
|
+
|
32
|
+
## 4.6.2 (2021-11-15)
|
33
|
+
|
34
|
+
- Added support for beginless ranges to `where` option
|
35
|
+
- Fixed `like` and `ilike` with `+` character
|
36
|
+
- Fixed warning about accessing system indices when no model or index specified
|
37
|
+
|
38
|
+
## 4.6.1 (2021-09-25)
|
39
|
+
|
40
|
+
- Added `ilike` operator for Elasticsearch 7.10+
|
41
|
+
- Fixed missing methods with `multi_search`
|
42
|
+
|
1
43
|
## 4.6.0 (2021-08-22)
|
2
44
|
|
3
45
|
- Added support for case-insensitive regular expressions with Elasticsearch 7.10+
|
data/README.md
CHANGED
@@ -20,7 +20,7 @@ Plus:
|
|
20
20
|
- autocomplete
|
21
21
|
- “Did you mean” suggestions
|
22
22
|
- supports many languages
|
23
|
-
- works with Active Record
|
23
|
+
- works with Active Record and Mongoid
|
24
24
|
|
25
25
|
Check out [Searchjoy](https://github.com/ankane/searchjoy) for analytics and [Autosuggest](https://github.com/ankane/autosuggest) for query suggestions
|
26
26
|
|
@@ -39,26 +39,34 @@ Check out [Searchjoy](https://github.com/ankane/searchjoy) for analytics and [Au
|
|
39
39
|
- [Testing](#testing)
|
40
40
|
- [Deployment](#deployment)
|
41
41
|
- [Performance](#performance)
|
42
|
-
- [
|
42
|
+
- [Advanced Search](#advanced)
|
43
43
|
- [Reference](#reference)
|
44
44
|
- [Contributing](#contributing)
|
45
45
|
|
46
|
+
Searchkick 5.0 was recently released! See [how to upgrade](#upgrading)
|
47
|
+
|
46
48
|
## Getting Started
|
47
49
|
|
48
50
|
Install [Elasticsearch](https://www.elastic.co/downloads/elasticsearch) or [OpenSearch](https://opensearch.org/downloads.html). For Homebrew, use:
|
49
51
|
|
50
52
|
```sh
|
51
|
-
brew install elasticsearch
|
52
|
-
brew services start elasticsearch
|
53
|
+
brew install elasticsearch
|
54
|
+
brew services start elasticsearch
|
55
|
+
# or
|
56
|
+
brew install opensearch
|
57
|
+
brew services start opensearch
|
53
58
|
```
|
54
59
|
|
55
|
-
Add
|
60
|
+
Add these lines to your application’s Gemfile:
|
56
61
|
|
57
62
|
```ruby
|
58
|
-
gem
|
63
|
+
gem "searchkick"
|
64
|
+
|
65
|
+
gem "elasticsearch" # select one
|
66
|
+
gem "opensearch-ruby" # select one
|
59
67
|
```
|
60
68
|
|
61
|
-
The latest version works with Elasticsearch
|
69
|
+
The latest version works with Elasticsearch 7 and 8 and OpenSearch 1. For Elasticsearch 6, use version 4.6.3 and [this readme](https://github.com/ankane/searchkick/blob/v4.6.3/README.md).
|
62
70
|
|
63
71
|
Add searchkick to models you want to search.
|
64
72
|
|
@@ -83,7 +91,7 @@ products.each do |product|
|
|
83
91
|
end
|
84
92
|
```
|
85
93
|
|
86
|
-
Searchkick supports the complete [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html). As your search becomes more advanced, we recommend you use the [
|
94
|
+
Searchkick supports the complete [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) and [OpenSearch Search API](https://opensearch.org/docs/latest/opensearch/rest-api/search/). As your search becomes more advanced, we recommend you use the [search server DSL](#advanced) for maximum flexibility.
|
87
95
|
|
88
96
|
## Querying
|
89
97
|
|
@@ -103,19 +111,20 @@ Where
|
|
103
111
|
|
104
112
|
```ruby
|
105
113
|
where: {
|
106
|
-
expires_at: {gt: Time.now},
|
107
|
-
orders_count: 1..10,
|
108
|
-
aisle_id: [25, 30],
|
109
|
-
store_id: {not: 2},
|
110
|
-
aisle_id: {not: [25, 30]},
|
111
|
-
user_ids: {all: [1, 3]},
|
112
|
-
category: {like: "%frozen%"},
|
113
|
-
category:
|
114
|
-
category:
|
115
|
-
|
114
|
+
expires_at: {gt: Time.now}, # lt, gte, lte also available
|
115
|
+
orders_count: 1..10, # equivalent to {gte: 1, lte: 10}
|
116
|
+
aisle_id: [25, 30], # in
|
117
|
+
store_id: {not: 2}, # not
|
118
|
+
aisle_id: {not: [25, 30]}, # not in
|
119
|
+
user_ids: {all: [1, 3]}, # all elements in array
|
120
|
+
category: {like: "%frozen%"}, # like
|
121
|
+
category: {ilike: "%frozen%"}, # ilike
|
122
|
+
category: /frozen .+/, # regexp
|
123
|
+
category: {prefix: "frozen"}, # prefix
|
124
|
+
store_id: {exists: true}, # exists
|
116
125
|
_or: [{in_stock: true}, {backordered: true}],
|
117
126
|
_and: [{in_stock: true}, {backordered: true}],
|
118
|
-
_not: {store_id: 1}
|
127
|
+
_not: {store_id: 1} # negate a condition
|
119
128
|
}
|
120
129
|
```
|
121
130
|
|
@@ -143,7 +152,7 @@ select: [:name]
|
|
143
152
|
|
144
153
|
### Results
|
145
154
|
|
146
|
-
Searches return a `Searchkick::
|
155
|
+
Searches return a `Searchkick::Relation` object. This responds like an array to most methods.
|
147
156
|
|
148
157
|
```ruby
|
149
158
|
results = Product.search("milk")
|
@@ -152,7 +161,7 @@ results.any?
|
|
152
161
|
results.each { |result| ... }
|
153
162
|
```
|
154
163
|
|
155
|
-
By default, ids are fetched from
|
164
|
+
By default, ids are fetched from the search server and records are fetched from your database. To fetch everything from the search server, use:
|
156
165
|
|
157
166
|
```ruby
|
158
167
|
Product.search("apples", load: false)
|
@@ -170,13 +179,13 @@ Get the time the search took (in milliseconds)
|
|
170
179
|
results.took
|
171
180
|
```
|
172
181
|
|
173
|
-
Get the full response from
|
182
|
+
Get the full response from the search server
|
174
183
|
|
175
184
|
```ruby
|
176
185
|
results.response
|
177
186
|
```
|
178
187
|
|
179
|
-
**Note:** By default, Elasticsearch [
|
188
|
+
**Note:** By default, Elasticsearch and OpenSearch [limit paging](#deep-paging) to the first 10,000 results for performance. This applies to the total count as well.
|
180
189
|
|
181
190
|
### Boosting
|
182
191
|
|
@@ -374,9 +383,9 @@ search_synonyms: ["lightbulb => halogenlamp"]
|
|
374
383
|
|
375
384
|
The above approach works well when your synonym list is static, but in practice, this is often not the case. When you analyze search conversions, you often want to add new synonyms without a full reindex.
|
376
385
|
|
377
|
-
#### Elasticsearch 7.3+
|
386
|
+
#### Elasticsearch 7.3+ and OpenSearch
|
378
387
|
|
379
|
-
For Elasticsearch 7.3
|
388
|
+
For Elasticsearch 7.3+ and OpenSearch, we recommend placing synonyms in a file on the search server (in the `config` directory). This allows you to reload synonyms without reindexing.
|
380
389
|
|
381
390
|
```txt
|
382
391
|
pop, soda
|
@@ -386,22 +395,18 @@ burger, hamburger
|
|
386
395
|
Then use:
|
387
396
|
|
388
397
|
```ruby
|
389
|
-
|
390
|
-
|
391
|
-
|
392
|
-
Add [elasticsearch-xpack](https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch-xpack) to your Gemfile:
|
393
|
-
|
394
|
-
```ruby
|
395
|
-
gem 'elasticsearch-xpack', '>= 7.8', '< 7.14'
|
398
|
+
class Product < ApplicationRecord
|
399
|
+
searchkick search_synonyms: "synonyms.txt"
|
400
|
+
end
|
396
401
|
```
|
397
402
|
|
398
|
-
And
|
403
|
+
And reload with:
|
399
404
|
|
400
405
|
```ruby
|
401
406
|
Product.search_index.reload_synonyms
|
402
407
|
```
|
403
408
|
|
404
|
-
#### Elasticsearch < 7.3
|
409
|
+
#### Elasticsearch < 7.3
|
405
410
|
|
406
411
|
You can use a library like [ActsAsTaggableOn](https://github.com/mbleigh/acts-as-taggable-on) and do:
|
407
412
|
|
@@ -488,7 +493,7 @@ Search :ice_cream::cake: and get `ice cream cake`!
|
|
488
493
|
Add this line to your application’s Gemfile:
|
489
494
|
|
490
495
|
```ruby
|
491
|
-
gem
|
496
|
+
gem "gemoji-parser"
|
492
497
|
```
|
493
498
|
|
494
499
|
And use:
|
@@ -523,12 +528,10 @@ class Product < ApplicationRecord
|
|
523
528
|
end
|
524
529
|
```
|
525
530
|
|
526
|
-
By default, all records are indexed. To control which records are indexed, use the `should_index?` method
|
531
|
+
By default, all records are indexed. To control which records are indexed, use the `should_index?` method.
|
527
532
|
|
528
533
|
```ruby
|
529
534
|
class Product < ApplicationRecord
|
530
|
-
scope :search_import, -> { where(active: true) }
|
531
|
-
|
532
535
|
def should_index?
|
533
536
|
active # only index active records
|
534
537
|
end
|
@@ -621,6 +624,28 @@ class Image < ApplicationRecord
|
|
621
624
|
end
|
622
625
|
```
|
623
626
|
|
627
|
+
### Default Scopes
|
628
|
+
|
629
|
+
If you have a default scope that filters records, use the `should_index?` method to exclude them from indexing:
|
630
|
+
|
631
|
+
```ruby
|
632
|
+
class Product < ApplicationRecord
|
633
|
+
default_scope { where(deleted_at: nil) }
|
634
|
+
|
635
|
+
def should_index?
|
636
|
+
deleted_at.nil?
|
637
|
+
end
|
638
|
+
end
|
639
|
+
```
|
640
|
+
|
641
|
+
If you want to index and search filtered records, set:
|
642
|
+
|
643
|
+
```ruby
|
644
|
+
class Product < ApplicationRecord
|
645
|
+
searchkick unscope: true
|
646
|
+
end
|
647
|
+
```
|
648
|
+
|
624
649
|
## Intelligent Search
|
625
650
|
|
626
651
|
The best starting point to improve your search **by far** is to track searches and conversions. [Searchjoy](https://github.com/ankane/searchjoy) makes it easy.
|
@@ -1046,13 +1071,13 @@ Product.search("soap", debug: true)
|
|
1046
1071
|
|
1047
1072
|
This prints useful info to `stdout`.
|
1048
1073
|
|
1049
|
-
See how
|
1074
|
+
See how the search server scores your queries with:
|
1050
1075
|
|
1051
1076
|
```ruby
|
1052
1077
|
Product.search("soap", explain: true).response
|
1053
1078
|
```
|
1054
1079
|
|
1055
|
-
See how
|
1080
|
+
See how the search server tokenizes your queries with:
|
1056
1081
|
|
1057
1082
|
```ruby
|
1058
1083
|
Product.search_index.tokens("Dish Washer Soap", analyzer: "searchkick_index")
|
@@ -1226,11 +1251,11 @@ And [setup-opensearch](https://github.com/ankane/setup-opensearch) for an easy w
|
|
1226
1251
|
|
1227
1252
|
## Deployment
|
1228
1253
|
|
1229
|
-
Searchkick uses `ENV["ELASTICSEARCH_URL"]` for
|
1254
|
+
For the search server, Searchkick uses `ENV["ELASTICSEARCH_URL"]` for Elasticsearch and `ENV["OPENSEARCH_URL"]` for OpenSearch. This defaults to `http://localhost:9200`.
|
1230
1255
|
|
1231
1256
|
- [Elastic Cloud](#elastic-cloud)
|
1232
1257
|
- [Heroku](#heroku)
|
1233
|
-
- [Amazon
|
1258
|
+
- [Amazon OpenSearch Service](#amazon-opensearch-service)
|
1234
1259
|
- [Self-Hosted and Other](#self-hosted-and-other)
|
1235
1260
|
|
1236
1261
|
### Elastic Cloud
|
@@ -1251,13 +1276,20 @@ rake searchkick:reindex:all
|
|
1251
1276
|
|
1252
1277
|
Choose an add-on: [Bonsai](https://elements.heroku.com/addons/bonsai), [SearchBox](https://elements.heroku.com/addons/searchbox), or [Elastic Cloud](https://elements.heroku.com/addons/foundelasticsearch).
|
1253
1278
|
|
1254
|
-
For Bonsai:
|
1279
|
+
For Elasticsearch on Bonsai:
|
1255
1280
|
|
1256
1281
|
```sh
|
1257
|
-
heroku addons:create bonsai
|
1282
|
+
heroku addons:create bonsai
|
1258
1283
|
heroku config:set ELASTICSEARCH_URL=`heroku config:get BONSAI_URL`
|
1259
1284
|
```
|
1260
1285
|
|
1286
|
+
For OpenSearch on Bonsai:
|
1287
|
+
|
1288
|
+
```sh
|
1289
|
+
heroku addons:create bonsai --engine=opensearch
|
1290
|
+
heroku config:set OPENSEARCH_URL=`heroku config:get BONSAI_URL`
|
1291
|
+
```
|
1292
|
+
|
1261
1293
|
For SearchBox:
|
1262
1294
|
|
1263
1295
|
```sh
|
@@ -1290,18 +1322,18 @@ Then deploy and reindex:
|
|
1290
1322
|
heroku run rake searchkick:reindex:all
|
1291
1323
|
```
|
1292
1324
|
|
1293
|
-
### Amazon
|
1325
|
+
### Amazon OpenSearch Service
|
1294
1326
|
|
1295
|
-
Create an initializer `config/initializers/
|
1327
|
+
Create an initializer `config/initializers/opensearch.rb` with:
|
1296
1328
|
|
1297
1329
|
```ruby
|
1298
|
-
ENV["
|
1330
|
+
ENV["OPENSEARCH_URL"] = "https://es-domain-1234.us-east-1.es.amazonaws.com:443"
|
1299
1331
|
```
|
1300
1332
|
|
1301
1333
|
To use signed requests, include in your Gemfile:
|
1302
1334
|
|
1303
1335
|
```ruby
|
1304
|
-
gem
|
1336
|
+
gem "faraday_middleware-aws-sigv4"
|
1305
1337
|
```
|
1306
1338
|
|
1307
1339
|
and add to your initializer:
|
@@ -1322,10 +1354,12 @@ rake searchkick:reindex:all
|
|
1322
1354
|
|
1323
1355
|
### Self-Hosted and Other
|
1324
1356
|
|
1325
|
-
Create an initializer
|
1357
|
+
Create an initializer with:
|
1326
1358
|
|
1327
1359
|
```ruby
|
1328
1360
|
ENV["ELASTICSEARCH_URL"] = "https://user:password@host:port"
|
1361
|
+
# or
|
1362
|
+
ENV["OPENSEARCH_URL"] = "https://user:password@host:port"
|
1329
1363
|
```
|
1330
1364
|
|
1331
1365
|
Then deploy and reindex:
|
@@ -1336,9 +1370,9 @@ rake searchkick:reindex:all
|
|
1336
1370
|
|
1337
1371
|
### Data Protection
|
1338
1372
|
|
1339
|
-
We recommend encrypting data at rest and in transit (even inside your own network). This is especially important if you send [personal data](https://en.wikipedia.org/wiki/Personally_identifiable_information) of your users to
|
1373
|
+
We recommend encrypting data at rest and in transit (even inside your own network). This is especially important if you send [personal data](https://en.wikipedia.org/wiki/Personally_identifiable_information) of your users to the search server.
|
1340
1374
|
|
1341
|
-
Bonsai, Elastic Cloud, and Amazon
|
1375
|
+
Bonsai, Elastic Cloud, and Amazon OpenSearch Service all support encryption at rest and HTTPS.
|
1342
1376
|
|
1343
1377
|
### Automatic Failover
|
1344
1378
|
|
@@ -1371,7 +1405,7 @@ See [Production Rails](https://github.com/ankane/production_rails) for other goo
|
|
1371
1405
|
Significantly increase performance with faster JSON generation. Add [Oj](https://github.com/ohler55/oj) to your Gemfile.
|
1372
1406
|
|
1373
1407
|
```ruby
|
1374
|
-
gem
|
1408
|
+
gem "oj"
|
1375
1409
|
```
|
1376
1410
|
|
1377
1411
|
This speeds up all JSON generation and parsing in your application (automatically!)
|
@@ -1381,7 +1415,7 @@ This speeds up all JSON generation and parsing in your application (automaticall
|
|
1381
1415
|
Significantly increase performance with persistent HTTP connections. Add [Typhoeus](https://github.com/typhoeus/typhoeus) to your Gemfile and it’ll automatically be used.
|
1382
1416
|
|
1383
1417
|
```ruby
|
1384
|
-
gem
|
1418
|
+
gem "typhoeus"
|
1385
1419
|
```
|
1386
1420
|
|
1387
1421
|
To reduce log noise, create an initializer with:
|
@@ -1450,7 +1484,7 @@ Product.reindex(async: {wait: true})
|
|
1450
1484
|
You can use [ActiveJob::TrafficControl](https://github.com/nickelser/activejob-traffic_control) to control concurrency. Install the gem:
|
1451
1485
|
|
1452
1486
|
```ruby
|
1453
|
-
gem
|
1487
|
+
gem "activejob-traffic_control", ">= 0.1.3"
|
1454
1488
|
```
|
1455
1489
|
|
1456
1490
|
And create an initializer with:
|
@@ -1513,7 +1547,7 @@ For more tips, check out [Keeping Elasticsearch in Sync](https://www.elastic.co/
|
|
1513
1547
|
|
1514
1548
|
### Routing
|
1515
1549
|
|
1516
|
-
Searchkick supports [
|
1550
|
+
Searchkick supports [routing](https://www.elastic.co/blog/customizing-your-document-routing), which can significantly speed up searches.
|
1517
1551
|
|
1518
1552
|
```ruby
|
1519
1553
|
class Business < ApplicationRecord
|
@@ -1621,7 +1655,7 @@ ReindexConversionsJob.perform_later("Product")
|
|
1621
1655
|
|
1622
1656
|
## Advanced
|
1623
1657
|
|
1624
|
-
Searchkick makes it easy to use the Elasticsearch DSL on its own.
|
1658
|
+
Searchkick makes it easy to use the Elasticsearch or OpenSearch DSL on its own.
|
1625
1659
|
|
1626
1660
|
### Advanced Mapping
|
1627
1661
|
|
@@ -1675,7 +1709,7 @@ products =
|
|
1675
1709
|
end
|
1676
1710
|
```
|
1677
1711
|
|
1678
|
-
###
|
1712
|
+
### Client
|
1679
1713
|
|
1680
1714
|
Searchkick is built on top of the [elasticsearch](https://github.com/elastic/elasticsearch-ruby) gem. To access the client directly, use:
|
1681
1715
|
|
@@ -1688,8 +1722,8 @@ Searchkick.client
|
|
1688
1722
|
To batch search requests for performance, use:
|
1689
1723
|
|
1690
1724
|
```ruby
|
1691
|
-
products = Product.search("snacks"
|
1692
|
-
coupons = Coupon.search("snacks"
|
1725
|
+
products = Product.search("snacks")
|
1726
|
+
coupons = Coupon.search("snacks")
|
1693
1727
|
Searchkick.multi_search([products, coupons])
|
1694
1728
|
```
|
1695
1729
|
|
@@ -1740,7 +1774,7 @@ products.clear_scroll
|
|
1740
1774
|
|
1741
1775
|
## Deep Paging
|
1742
1776
|
|
1743
|
-
By default, Elasticsearch
|
1777
|
+
By default, Elasticsearch and OpenSearch limit paging to the first 10,000 results. [Here’s why](https://www.elastic.co/guide/en/elasticsearch/guide/current/pagination.html). We don’t recommend changing this, but if you really need all results, you can use:
|
1744
1778
|
|
1745
1779
|
```ruby
|
1746
1780
|
class Product < ApplicationRecord
|
@@ -1748,7 +1782,7 @@ class Product < ApplicationRecord
|
|
1748
1782
|
end
|
1749
1783
|
```
|
1750
1784
|
|
1751
|
-
If you just need an accurate total count
|
1785
|
+
If you just need an accurate total count, you can instead use:
|
1752
1786
|
|
1753
1787
|
```ruby
|
1754
1788
|
Product.search("pears", body_options: {track_total_hits: true})
|
@@ -1963,13 +1997,6 @@ class Product < ApplicationRecord
|
|
1963
1997
|
end
|
1964
1998
|
```
|
1965
1999
|
|
1966
|
-
Lazy searching
|
1967
|
-
|
1968
|
-
```ruby
|
1969
|
-
products = Product.search("carrots", execute: false)
|
1970
|
-
products.each { ... } # search not executed until here
|
1971
|
-
```
|
1972
|
-
|
1973
2000
|
Add [request parameters](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-search-api-query-params) like `search_type`
|
1974
2001
|
|
1975
2002
|
```ruby
|
@@ -2007,22 +2034,17 @@ Turn on misspellings after a certain number of characters
|
|
2007
2034
|
Product.search "api", misspellings: {prefix_length: 2} # api, apt, no ahi
|
2008
2035
|
```
|
2009
2036
|
|
2010
|
-
**Note:** With this option, if the query length is the same as `prefix_length`, misspellings are turned off
|
2037
|
+
**Note:** With this option, if the query length is the same as `prefix_length`, misspellings are turned off with Elasticsearch 7 and OpenSearch
|
2011
2038
|
|
2012
2039
|
```ruby
|
2013
2040
|
Product.search "ah", misspellings: {prefix_length: 2} # ah, no aha
|
2014
2041
|
```
|
2015
2042
|
|
2016
|
-
##
|
2017
|
-
|
2018
|
-
1. Install Searchkick 4
|
2019
|
-
2. Upgrade your Elasticsearch cluster
|
2020
|
-
|
2021
|
-
## Elasticsearch Gotchas
|
2043
|
+
## Gotchas
|
2022
2044
|
|
2023
2045
|
### Consistency
|
2024
2046
|
|
2025
|
-
Elasticsearch
|
2047
|
+
Elasticsearch and OpenSearch are eventually consistent, meaning it can take up to a second for a change to reflect in search. You can use the `refresh` method to have it show up immediately.
|
2026
2048
|
|
2027
2049
|
```ruby
|
2028
2050
|
product.save!
|
@@ -2031,7 +2053,7 @@ Product.search_index.refresh
|
|
2031
2053
|
|
2032
2054
|
### Inconsistent Scores
|
2033
2055
|
|
2034
|
-
Due to the distributed nature of Elasticsearch, you can get incorrect results when the number of documents in the index is low. You can [read more about it here](https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch). To fix this, do:
|
2056
|
+
Due to the distributed nature of Elasticsearch and OpenSearch, you can get incorrect results when the number of documents in the index is low. You can [read more about it here](https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch). To fix this, do:
|
2035
2057
|
|
2036
2058
|
```ruby
|
2037
2059
|
class Product < ApplicationRecord
|
@@ -2041,6 +2063,34 @@ end
|
|
2041
2063
|
|
2042
2064
|
For convenience, this is set by default in the test environment.
|
2043
2065
|
|
2066
|
+
## Upgrading
|
2067
|
+
|
2068
|
+
### 5.0
|
2069
|
+
|
2070
|
+
Searchkick 5 supports both the `elasticsearch` and `opensearch-ruby` gems. Add the one you want to use to your Gemfile:
|
2071
|
+
|
2072
|
+
```ruby
|
2073
|
+
gem "elasticsearch"
|
2074
|
+
# or
|
2075
|
+
gem "opensearch-ruby"
|
2076
|
+
```
|
2077
|
+
|
2078
|
+
If using the deprecated `faraday_middleware-aws-signers-v4` gem, switch to `faraday_middleware-aws-sigv4`.
|
2079
|
+
|
2080
|
+
Also, searches now use lazy loading:
|
2081
|
+
|
2082
|
+
```ruby
|
2083
|
+
# search not executed
|
2084
|
+
Product.search("milk")
|
2085
|
+
|
2086
|
+
# search executed
|
2087
|
+
Product.search("milk").to_a
|
2088
|
+
```
|
2089
|
+
|
2090
|
+
And there’s a [new option](#default-scopes) for models with default scopes.
|
2091
|
+
|
2092
|
+
Check out the [changelog](https://github.com/ankane/searchkick/blob/master/CHANGELOG.md) for the full list of changes.
|
2093
|
+
|
2044
2094
|
## History
|
2045
2095
|
|
2046
2096
|
View the [changelog](https://github.com/ankane/searchkick/blob/master/CHANGELOG.md).
|
@@ -2,16 +2,20 @@ module Searchkick
|
|
2
2
|
class BulkReindexJob < ActiveJob::Base
|
3
3
|
queue_as { Searchkick.queue_name }
|
4
4
|
|
5
|
+
# TODO remove min_id and max_id in Searchkick 6
|
5
6
|
def perform(class_name:, record_ids: nil, index_name: nil, method_name: nil, batch_id: nil, min_id: nil, max_id: nil)
|
6
|
-
|
7
|
-
index =
|
7
|
+
model = Searchkick.load_model(class_name)
|
8
|
+
index = model.searchkick_index(name: index_name)
|
9
|
+
|
10
|
+
# legacy
|
8
11
|
record_ids ||= min_id..max_id
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
)
|
12
|
+
|
13
|
+
relation = Searchkick.scope(model)
|
14
|
+
relation = Searchkick.load_records(relation, record_ids)
|
15
|
+
relation = relation.search_import if relation.respond_to?(:search_import)
|
16
|
+
|
17
|
+
RecordIndexer.new(index).reindex(relation, mode: :inline, method_name: method_name, full: false)
|
18
|
+
RelationIndexer.new(index).batch_completed(batch_id) if batch_id
|
15
19
|
end
|
16
20
|
end
|
17
21
|
end
|
@@ -0,0 +1,40 @@
|
|
1
|
+
# based on https://gist.github.com/mnutt/566725
|
2
|
+
module Searchkick
|
3
|
+
module ControllerRuntime
|
4
|
+
extend ActiveSupport::Concern
|
5
|
+
|
6
|
+
protected
|
7
|
+
|
8
|
+
attr_internal :searchkick_runtime
|
9
|
+
|
10
|
+
def process_action(action, *args)
|
11
|
+
# We also need to reset the runtime before each action
|
12
|
+
# because of queries in middleware or in cases we are streaming
|
13
|
+
# and it won't be cleaned up by the method below.
|
14
|
+
Searchkick::LogSubscriber.reset_runtime
|
15
|
+
super
|
16
|
+
end
|
17
|
+
|
18
|
+
def cleanup_view_runtime
|
19
|
+
searchkick_rt_before_render = Searchkick::LogSubscriber.reset_runtime
|
20
|
+
runtime = super
|
21
|
+
searchkick_rt_after_render = Searchkick::LogSubscriber.reset_runtime
|
22
|
+
self.searchkick_runtime = searchkick_rt_before_render + searchkick_rt_after_render
|
23
|
+
runtime - searchkick_rt_after_render
|
24
|
+
end
|
25
|
+
|
26
|
+
def append_info_to_payload(payload)
|
27
|
+
super
|
28
|
+
payload[:searchkick_runtime] = (searchkick_runtime || 0) + Searchkick::LogSubscriber.reset_runtime
|
29
|
+
end
|
30
|
+
|
31
|
+
module ClassMethods
|
32
|
+
def log_process_action(payload)
|
33
|
+
messages = super
|
34
|
+
runtime = payload[:searchkick_runtime]
|
35
|
+
messages << ("Searchkick: %.1fms" % runtime.to_f) if runtime.to_f > 0
|
36
|
+
messages
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|