searchkick 4.6.2 → 5.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +36 -0
- data/README.md +130 -73
- data/lib/searchkick/bulk_reindex_job.rb +12 -8
- data/lib/searchkick/controller_runtime.rb +40 -0
- data/lib/searchkick/index.rb +159 -74
- data/lib/searchkick/index_cache.rb +30 -0
- data/lib/searchkick/index_options.rb +17 -67
- data/lib/searchkick/indexer.rb +15 -8
- data/lib/searchkick/log_subscriber.rb +57 -0
- data/lib/searchkick/middleware.rb +1 -1
- data/lib/searchkick/model.rb +48 -49
- data/lib/searchkick/process_batch_job.rb +9 -25
- data/lib/searchkick/process_queue_job.rb +3 -2
- data/lib/searchkick/query.rb +39 -55
- data/lib/searchkick/record_data.rb +1 -1
- data/lib/searchkick/record_indexer.rb +136 -51
- data/lib/searchkick/reindex_queue.rb +26 -3
- data/lib/searchkick/reindex_v2_job.rb +10 -34
- data/lib/searchkick/relation.rb +36 -0
- data/lib/searchkick/relation_indexer.rb +150 -0
- data/lib/searchkick/results.rb +31 -32
- data/lib/searchkick/version.rb +1 -1
- data/lib/searchkick.rb +169 -81
- data/lib/tasks/searchkick.rake +6 -3
- metadata +11 -28
- data/lib/searchkick/bulk_indexer.rb +0 -173
- data/lib/searchkick/logging.rb +0 -246
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ab19b7696cc775bda54eef31aaf9fea04a13a3027335440023f58f0c3b761fa0
|
4
|
+
data.tar.gz: 461f230a62f440e84684e9eb42cc85d307b726830eba80999c5648aef13de92e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5382e2de584271fc9f80802f858329c613f3b9aff7877c9bb9653dab550deb6ed1ac6971cbb2bc306f4d6e67056f16ee98eea04d69f2fd9ff15fd899e37f8962
|
7
|
+
data.tar.gz: dd8121e4991c9aaad5cd331cdc3adef3c703ee47abf8238ea3cc6feb1b1e5dcd85cb59c9f7ce91c086644d5946c5ccba936f48d0bc27988a59cca81adf161bb3
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,39 @@
|
|
1
|
+
## 5.0.1 (2022-02-27)
|
2
|
+
|
3
|
+
- Prefer `mode: :async` over `async: true` for full reindex
|
4
|
+
- Fixed instance method overriding with concerns
|
5
|
+
|
6
|
+
## 5.0.0 (2022-02-21)
|
7
|
+
|
8
|
+
- Searches now use lazy loading (similar to Active Record)
|
9
|
+
- Added `unscope` option to better support working with default scopes
|
10
|
+
- Added support for `:async` and `:queue` modes for `reindex` on relation
|
11
|
+
- Added basic protection from unfiltered parameters to `where` option
|
12
|
+
- Added `models` option to `similar` method
|
13
|
+
- Changed async full reindex to fetch ids instead of using ranges for numeric primary keys with Active Record
|
14
|
+
- Changed `searchkick_index_options` to return symbol keys (instead of mix of strings and symbols)
|
15
|
+
- Changed non-anchored regular expressions to match expected results (previously warned)
|
16
|
+
- Changed record reindex to return `true` to match model and relation reindex
|
17
|
+
- Updated async reindex job to call `search_import` for nested associations
|
18
|
+
- Fixed removing records when `should_index?` is `false` when `reindex` called on relation
|
19
|
+
- Fixed issue with `merge_mappings` for fields that use `searchkick` options
|
20
|
+
- Raise error when `search` called on relations
|
21
|
+
- Raise `ArgumentError` (instead of warning) for invalid regular expression modifiers
|
22
|
+
- Raise `ArgumentError` instead of `RuntimeError` for unknown operators
|
23
|
+
- Removed mapping of `id` to `_id` with `order` option (not supported in Elasticsearch 8)
|
24
|
+
- Removed `wordnet` option (no longer worked)
|
25
|
+
- Removed dependency on `elasticsearch` gem (can use `elasticsearch` or `opensearch-ruby`)
|
26
|
+
- Dropped support for Elasticsearch 6
|
27
|
+
- Dropped support for Ruby < 2.6 and Active Record < 5.2
|
28
|
+
- Dropped support for NoBrainer and Cequel
|
29
|
+
- Dropped support for `faraday_middleware-aws-signers-v4` (use `faraday_middleware-aws-sigv4` instead)
|
30
|
+
|
31
|
+
## 4.6.3 (2021-11-19)
|
32
|
+
|
33
|
+
- Added support for reloadable synonyms for OpenSearch
|
34
|
+
- Added experimental support for `opensearch-ruby` gem
|
35
|
+
- Removed `elasticsearch-xpack` dependency for reloadable synonyms
|
36
|
+
|
1
37
|
## 4.6.2 (2021-11-15)
|
2
38
|
|
3
39
|
- Added support for beginless ranges to `where` option
|
data/README.md
CHANGED
@@ -20,7 +20,7 @@ Plus:
|
|
20
20
|
- autocomplete
|
21
21
|
- “Did you mean” suggestions
|
22
22
|
- supports many languages
|
23
|
-
- works with Active Record
|
23
|
+
- works with Active Record and Mongoid
|
24
24
|
|
25
25
|
Check out [Searchjoy](https://github.com/ankane/searchjoy) for analytics and [Autosuggest](https://github.com/ankane/autosuggest) for query suggestions
|
26
26
|
|
@@ -39,26 +39,34 @@ Check out [Searchjoy](https://github.com/ankane/searchjoy) for analytics and [Au
|
|
39
39
|
- [Testing](#testing)
|
40
40
|
- [Deployment](#deployment)
|
41
41
|
- [Performance](#performance)
|
42
|
-
- [
|
42
|
+
- [Advanced Search](#advanced)
|
43
43
|
- [Reference](#reference)
|
44
44
|
- [Contributing](#contributing)
|
45
45
|
|
46
|
+
Searchkick 5.0 was recently released! See [how to upgrade](#upgrading)
|
47
|
+
|
46
48
|
## Getting Started
|
47
49
|
|
48
50
|
Install [Elasticsearch](https://www.elastic.co/downloads/elasticsearch) or [OpenSearch](https://opensearch.org/downloads.html). For Homebrew, use:
|
49
51
|
|
50
52
|
```sh
|
51
|
-
brew install elasticsearch
|
52
|
-
brew services start elasticsearch
|
53
|
+
brew install elasticsearch
|
54
|
+
brew services start elasticsearch
|
55
|
+
# or
|
56
|
+
brew install opensearch
|
57
|
+
brew services start opensearch
|
53
58
|
```
|
54
59
|
|
55
|
-
Add
|
60
|
+
Add these lines to your application’s Gemfile:
|
56
61
|
|
57
62
|
```ruby
|
58
|
-
gem
|
63
|
+
gem "searchkick"
|
64
|
+
|
65
|
+
gem "elasticsearch" # select one
|
66
|
+
gem "opensearch-ruby" # select one
|
59
67
|
```
|
60
68
|
|
61
|
-
The latest version works with Elasticsearch
|
69
|
+
The latest version works with Elasticsearch 7 and 8 and OpenSearch 1. For Elasticsearch 6, use version 4.6.3 and [this readme](https://github.com/ankane/searchkick/blob/v4.6.3/README.md).
|
62
70
|
|
63
71
|
Add searchkick to models you want to search.
|
64
72
|
|
@@ -83,7 +91,7 @@ products.each do |product|
|
|
83
91
|
end
|
84
92
|
```
|
85
93
|
|
86
|
-
Searchkick supports the complete [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html). As your search becomes more advanced, we recommend you use the [
|
94
|
+
Searchkick supports the complete [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) and [OpenSearch Search API](https://opensearch.org/docs/latest/opensearch/rest-api/search/). As your search becomes more advanced, we recommend you use the [search server DSL](#advanced) for maximum flexibility.
|
87
95
|
|
88
96
|
## Querying
|
89
97
|
|
@@ -144,7 +152,7 @@ select: [:name]
|
|
144
152
|
|
145
153
|
### Results
|
146
154
|
|
147
|
-
Searches return a `Searchkick::
|
155
|
+
Searches return a `Searchkick::Relation` object. This responds like an array to most methods.
|
148
156
|
|
149
157
|
```ruby
|
150
158
|
results = Product.search("milk")
|
@@ -153,7 +161,7 @@ results.any?
|
|
153
161
|
results.each { |result| ... }
|
154
162
|
```
|
155
163
|
|
156
|
-
By default, ids are fetched from
|
164
|
+
By default, ids are fetched from the search server and records are fetched from your database. To fetch everything from the search server, use:
|
157
165
|
|
158
166
|
```ruby
|
159
167
|
Product.search("apples", load: false)
|
@@ -171,13 +179,13 @@ Get the time the search took (in milliseconds)
|
|
171
179
|
results.took
|
172
180
|
```
|
173
181
|
|
174
|
-
Get the full response from
|
182
|
+
Get the full response from the search server
|
175
183
|
|
176
184
|
```ruby
|
177
185
|
results.response
|
178
186
|
```
|
179
187
|
|
180
|
-
**Note:** By default, Elasticsearch [
|
188
|
+
**Note:** By default, Elasticsearch and OpenSearch [limit paging](#deep-paging) to the first 10,000 results for performance. This applies to the total count as well.
|
181
189
|
|
182
190
|
### Boosting
|
183
191
|
|
@@ -375,9 +383,9 @@ search_synonyms: ["lightbulb => halogenlamp"]
|
|
375
383
|
|
376
384
|
The above approach works well when your synonym list is static, but in practice, this is often not the case. When you analyze search conversions, you often want to add new synonyms without a full reindex.
|
377
385
|
|
378
|
-
#### Elasticsearch 7.3+
|
386
|
+
#### Elasticsearch 7.3+ and OpenSearch
|
379
387
|
|
380
|
-
For Elasticsearch 7.3
|
388
|
+
For Elasticsearch 7.3+ and OpenSearch, we recommend placing synonyms in a file on the search server (in the `config` directory). This allows you to reload synonyms without reindexing.
|
381
389
|
|
382
390
|
```txt
|
383
391
|
pop, soda
|
@@ -387,22 +395,18 @@ burger, hamburger
|
|
387
395
|
Then use:
|
388
396
|
|
389
397
|
```ruby
|
390
|
-
|
391
|
-
|
392
|
-
|
393
|
-
Add [elasticsearch-xpack](https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch-xpack) to your Gemfile:
|
394
|
-
|
395
|
-
```ruby
|
396
|
-
gem 'elasticsearch-xpack', '>= 7.8', '< 7.14'
|
398
|
+
class Product < ApplicationRecord
|
399
|
+
searchkick search_synonyms: "synonyms.txt"
|
400
|
+
end
|
397
401
|
```
|
398
402
|
|
399
|
-
And
|
403
|
+
And reload with:
|
400
404
|
|
401
405
|
```ruby
|
402
406
|
Product.search_index.reload_synonyms
|
403
407
|
```
|
404
408
|
|
405
|
-
#### Elasticsearch < 7.3
|
409
|
+
#### Elasticsearch < 7.3
|
406
410
|
|
407
411
|
You can use a library like [ActsAsTaggableOn](https://github.com/mbleigh/acts-as-taggable-on) and do:
|
408
412
|
|
@@ -489,7 +493,7 @@ Search :ice_cream::cake: and get `ice cream cake`!
|
|
489
493
|
Add this line to your application’s Gemfile:
|
490
494
|
|
491
495
|
```ruby
|
492
|
-
gem
|
496
|
+
gem "gemoji-parser"
|
493
497
|
```
|
494
498
|
|
495
499
|
And use:
|
@@ -524,12 +528,10 @@ class Product < ApplicationRecord
|
|
524
528
|
end
|
525
529
|
```
|
526
530
|
|
527
|
-
By default, all records are indexed. To control which records are indexed, use the `should_index?` method
|
531
|
+
By default, all records are indexed. To control which records are indexed, use the `should_index?` method.
|
528
532
|
|
529
533
|
```ruby
|
530
534
|
class Product < ApplicationRecord
|
531
|
-
scope :search_import, -> { where(active: true) }
|
532
|
-
|
533
535
|
def should_index?
|
534
536
|
active # only index active records
|
535
537
|
end
|
@@ -594,7 +596,7 @@ You can also do bulk updates.
|
|
594
596
|
|
595
597
|
```ruby
|
596
598
|
Searchkick.callbacks(:bulk) do
|
597
|
-
|
599
|
+
Product.find_each(&:update_fields)
|
598
600
|
end
|
599
601
|
```
|
600
602
|
|
@@ -602,7 +604,7 @@ Or temporarily skip updates.
|
|
602
604
|
|
603
605
|
```ruby
|
604
606
|
Searchkick.callbacks(false) do
|
605
|
-
|
607
|
+
Product.find_each(&:update_fields)
|
606
608
|
end
|
607
609
|
```
|
608
610
|
|
@@ -622,6 +624,28 @@ class Image < ApplicationRecord
|
|
622
624
|
end
|
623
625
|
```
|
624
626
|
|
627
|
+
### Default Scopes
|
628
|
+
|
629
|
+
If you have a default scope that filters records, use the `should_index?` method to exclude them from indexing:
|
630
|
+
|
631
|
+
```ruby
|
632
|
+
class Product < ApplicationRecord
|
633
|
+
default_scope { where(deleted_at: nil) }
|
634
|
+
|
635
|
+
def should_index?
|
636
|
+
deleted_at.nil?
|
637
|
+
end
|
638
|
+
end
|
639
|
+
```
|
640
|
+
|
641
|
+
If you want to index and search filtered records, set:
|
642
|
+
|
643
|
+
```ruby
|
644
|
+
class Product < ApplicationRecord
|
645
|
+
searchkick unscope: true
|
646
|
+
end
|
647
|
+
```
|
648
|
+
|
625
649
|
## Intelligent Search
|
626
650
|
|
627
651
|
The best starting point to improve your search **by far** is to track searches and conversions. [Searchjoy](https://github.com/ankane/searchjoy) makes it easy.
|
@@ -1047,13 +1071,13 @@ Product.search("soap", debug: true)
|
|
1047
1071
|
|
1048
1072
|
This prints useful info to `stdout`.
|
1049
1073
|
|
1050
|
-
See how
|
1074
|
+
See how the search server scores your queries with:
|
1051
1075
|
|
1052
1076
|
```ruby
|
1053
1077
|
Product.search("soap", explain: true).response
|
1054
1078
|
```
|
1055
1079
|
|
1056
|
-
See how
|
1080
|
+
See how the search server tokenizes your queries with:
|
1057
1081
|
|
1058
1082
|
```ruby
|
1059
1083
|
Product.search_index.tokens("Dish Washer Soap", analyzer: "searchkick_index")
|
@@ -1076,7 +1100,7 @@ Product.search_index.tokens("dieg", analyzer: "searchkick_word_search")
|
|
1076
1100
|
# ["dieg"] - match!!
|
1077
1101
|
```
|
1078
1102
|
|
1079
|
-
See the [complete list of analyzers](
|
1103
|
+
See the [complete list of analyzers](lib/searchkick/index_options.rb#L36).
|
1080
1104
|
|
1081
1105
|
## Testing
|
1082
1106
|
|
@@ -1227,7 +1251,7 @@ And [setup-opensearch](https://github.com/ankane/setup-opensearch) for an easy w
|
|
1227
1251
|
|
1228
1252
|
## Deployment
|
1229
1253
|
|
1230
|
-
Searchkick uses `ENV["ELASTICSEARCH_URL"]` for
|
1254
|
+
For the search server, Searchkick uses `ENV["ELASTICSEARCH_URL"]` for Elasticsearch and `ENV["OPENSEARCH_URL"]` for OpenSearch. This defaults to `http://localhost:9200`.
|
1231
1255
|
|
1232
1256
|
- [Elastic Cloud](#elastic-cloud)
|
1233
1257
|
- [Heroku](#heroku)
|
@@ -1252,13 +1276,20 @@ rake searchkick:reindex:all
|
|
1252
1276
|
|
1253
1277
|
Choose an add-on: [Bonsai](https://elements.heroku.com/addons/bonsai), [SearchBox](https://elements.heroku.com/addons/searchbox), or [Elastic Cloud](https://elements.heroku.com/addons/foundelasticsearch).
|
1254
1278
|
|
1255
|
-
For Bonsai:
|
1279
|
+
For Elasticsearch on Bonsai:
|
1256
1280
|
|
1257
1281
|
```sh
|
1258
|
-
heroku addons:create bonsai
|
1282
|
+
heroku addons:create bonsai
|
1259
1283
|
heroku config:set ELASTICSEARCH_URL=`heroku config:get BONSAI_URL`
|
1260
1284
|
```
|
1261
1285
|
|
1286
|
+
For OpenSearch on Bonsai:
|
1287
|
+
|
1288
|
+
```sh
|
1289
|
+
heroku addons:create bonsai --engine=opensearch
|
1290
|
+
heroku config:set OPENSEARCH_URL=`heroku config:get BONSAI_URL`
|
1291
|
+
```
|
1292
|
+
|
1262
1293
|
For SearchBox:
|
1263
1294
|
|
1264
1295
|
```sh
|
@@ -1293,16 +1324,16 @@ heroku run rake searchkick:reindex:all
|
|
1293
1324
|
|
1294
1325
|
### Amazon OpenSearch Service
|
1295
1326
|
|
1296
|
-
Create an initializer `config/initializers/
|
1327
|
+
Create an initializer `config/initializers/opensearch.rb` with:
|
1297
1328
|
|
1298
1329
|
```ruby
|
1299
|
-
ENV["
|
1330
|
+
ENV["OPENSEARCH_URL"] = "https://es-domain-1234.us-east-1.es.amazonaws.com:443"
|
1300
1331
|
```
|
1301
1332
|
|
1302
1333
|
To use signed requests, include in your Gemfile:
|
1303
1334
|
|
1304
1335
|
```ruby
|
1305
|
-
gem
|
1336
|
+
gem "faraday_middleware-aws-sigv4"
|
1306
1337
|
```
|
1307
1338
|
|
1308
1339
|
and add to your initializer:
|
@@ -1323,10 +1354,12 @@ rake searchkick:reindex:all
|
|
1323
1354
|
|
1324
1355
|
### Self-Hosted and Other
|
1325
1356
|
|
1326
|
-
Create an initializer
|
1357
|
+
Create an initializer with:
|
1327
1358
|
|
1328
1359
|
```ruby
|
1329
1360
|
ENV["ELASTICSEARCH_URL"] = "https://user:password@host:port"
|
1361
|
+
# or
|
1362
|
+
ENV["OPENSEARCH_URL"] = "https://user:password@host:port"
|
1330
1363
|
```
|
1331
1364
|
|
1332
1365
|
Then deploy and reindex:
|
@@ -1337,19 +1370,19 @@ rake searchkick:reindex:all
|
|
1337
1370
|
|
1338
1371
|
### Data Protection
|
1339
1372
|
|
1340
|
-
We recommend encrypting data at rest and in transit (even inside your own network). This is especially important if you send [personal data](https://en.wikipedia.org/wiki/Personally_identifiable_information) of your users to
|
1373
|
+
We recommend encrypting data at rest and in transit (even inside your own network). This is especially important if you send [personal data](https://en.wikipedia.org/wiki/Personally_identifiable_information) of your users to the search server.
|
1341
1374
|
|
1342
|
-
Bonsai, Elastic Cloud, and Amazon
|
1375
|
+
Bonsai, Elastic Cloud, and Amazon OpenSearch Service all support encryption at rest and HTTPS.
|
1343
1376
|
|
1344
1377
|
### Automatic Failover
|
1345
1378
|
|
1346
|
-
Create an initializer
|
1379
|
+
Create an initializer with multiple hosts:
|
1347
1380
|
|
1348
1381
|
```ruby
|
1349
1382
|
ENV["ELASTICSEARCH_URL"] = "https://user:password@host1,https://user:password@host2"
|
1350
1383
|
```
|
1351
1384
|
|
1352
|
-
See [
|
1385
|
+
See [elastic-transport](https://github.com/elastic/elastic-transport-ruby) or [opensearch-transport](https://github.com/opensearch-project/opensearch-ruby/tree/main/opensearch-transport) for a complete list of options.
|
1353
1386
|
|
1354
1387
|
### Lograge
|
1355
1388
|
|
@@ -1372,7 +1405,7 @@ See [Production Rails](https://github.com/ankane/production_rails) for other goo
|
|
1372
1405
|
Significantly increase performance with faster JSON generation. Add [Oj](https://github.com/ohler55/oj) to your Gemfile.
|
1373
1406
|
|
1374
1407
|
```ruby
|
1375
|
-
gem
|
1408
|
+
gem "oj"
|
1376
1409
|
```
|
1377
1410
|
|
1378
1411
|
This speeds up all JSON generation and parsing in your application (automatically!)
|
@@ -1382,7 +1415,7 @@ This speeds up all JSON generation and parsing in your application (automaticall
|
|
1382
1415
|
Significantly increase performance with persistent HTTP connections. Add [Typhoeus](https://github.com/typhoeus/typhoeus) to your Gemfile and it’ll automatically be used.
|
1383
1416
|
|
1384
1417
|
```ruby
|
1385
|
-
gem
|
1418
|
+
gem "typhoeus"
|
1386
1419
|
```
|
1387
1420
|
|
1388
1421
|
To reduce log noise, create an initializer with:
|
@@ -1420,7 +1453,7 @@ end
|
|
1420
1453
|
For large data sets, you can use background jobs to parallelize reindexing.
|
1421
1454
|
|
1422
1455
|
```ruby
|
1423
|
-
Product.reindex(
|
1456
|
+
Product.reindex(mode: :async)
|
1424
1457
|
# {index_name: "products_production_20170111210018065"}
|
1425
1458
|
```
|
1426
1459
|
|
@@ -1445,13 +1478,13 @@ Searchkick.reindex_status(index_name)
|
|
1445
1478
|
You can also have Searchkick wait for reindexing to complete
|
1446
1479
|
|
1447
1480
|
```ruby
|
1448
|
-
Product.reindex(
|
1481
|
+
Product.reindex(mode: :async, wait: true)
|
1449
1482
|
```
|
1450
1483
|
|
1451
1484
|
You can use [ActiveJob::TrafficControl](https://github.com/nickelser/activejob-traffic_control) to control concurrency. Install the gem:
|
1452
1485
|
|
1453
1486
|
```ruby
|
1454
|
-
gem
|
1487
|
+
gem "activejob-traffic_control", ">= 0.1.3"
|
1455
1488
|
```
|
1456
1489
|
|
1457
1490
|
And create an initializer with:
|
@@ -1471,7 +1504,7 @@ This will allow only 3 jobs to run at once.
|
|
1471
1504
|
You can specify a longer refresh interval while reindexing to increase performance.
|
1472
1505
|
|
1473
1506
|
```ruby
|
1474
|
-
Product.reindex(
|
1507
|
+
Product.reindex(mode: :async, refresh_interval: "30s")
|
1475
1508
|
```
|
1476
1509
|
|
1477
1510
|
**Note:** This only makes a noticable difference with parallel reindexing.
|
@@ -1514,7 +1547,7 @@ For more tips, check out [Keeping Elasticsearch in Sync](https://www.elastic.co/
|
|
1514
1547
|
|
1515
1548
|
### Routing
|
1516
1549
|
|
1517
|
-
Searchkick supports [
|
1550
|
+
Searchkick supports [routing](https://www.elastic.co/blog/customizing-your-document-routing), which can significantly speed up searches.
|
1518
1551
|
|
1519
1552
|
```ruby
|
1520
1553
|
class Business < ApplicationRecord
|
@@ -1622,7 +1655,7 @@ ReindexConversionsJob.perform_later("Product")
|
|
1622
1655
|
|
1623
1656
|
## Advanced
|
1624
1657
|
|
1625
|
-
Searchkick makes it easy to use the Elasticsearch DSL on its own.
|
1658
|
+
Searchkick makes it easy to use the Elasticsearch or OpenSearch DSL on its own.
|
1626
1659
|
|
1627
1660
|
### Advanced Mapping
|
1628
1661
|
|
@@ -1676,9 +1709,9 @@ products =
|
|
1676
1709
|
end
|
1677
1710
|
```
|
1678
1711
|
|
1679
|
-
###
|
1712
|
+
### Client
|
1680
1713
|
|
1681
|
-
|
1714
|
+
To access the `Elasticsearch::Client` or `OpenSearch::Client` directly, use:
|
1682
1715
|
|
1683
1716
|
```ruby
|
1684
1717
|
Searchkick.client
|
@@ -1689,8 +1722,8 @@ Searchkick.client
|
|
1689
1722
|
To batch search requests for performance, use:
|
1690
1723
|
|
1691
1724
|
```ruby
|
1692
|
-
products = Product.search("snacks"
|
1693
|
-
coupons = Coupon.search("snacks"
|
1725
|
+
products = Product.search("snacks")
|
1726
|
+
coupons = Coupon.search("snacks")
|
1694
1727
|
Searchkick.multi_search([products, coupons])
|
1695
1728
|
```
|
1696
1729
|
|
@@ -1741,7 +1774,7 @@ products.clear_scroll
|
|
1741
1774
|
|
1742
1775
|
## Deep Paging
|
1743
1776
|
|
1744
|
-
By default, Elasticsearch
|
1777
|
+
By default, Elasticsearch and OpenSearch limit paging to the first 10,000 results. [Here’s why](https://www.elastic.co/guide/en/elasticsearch/guide/current/pagination.html). We don’t recommend changing this, but if you really need all results, you can use:
|
1745
1778
|
|
1746
1779
|
```ruby
|
1747
1780
|
class Product < ApplicationRecord
|
@@ -1749,7 +1782,7 @@ class Product < ApplicationRecord
|
|
1749
1782
|
end
|
1750
1783
|
```
|
1751
1784
|
|
1752
|
-
If you just need an accurate total count
|
1785
|
+
If you just need an accurate total count, you can instead use:
|
1753
1786
|
|
1754
1787
|
```ruby
|
1755
1788
|
Product.search("pears", body_options: {track_total_hits: true})
|
@@ -1964,13 +1997,6 @@ class Product < ApplicationRecord
|
|
1964
1997
|
end
|
1965
1998
|
```
|
1966
1999
|
|
1967
|
-
Lazy searching
|
1968
|
-
|
1969
|
-
```ruby
|
1970
|
-
products = Product.search("carrots", execute: false)
|
1971
|
-
products.each { ... } # search not executed until here
|
1972
|
-
```
|
1973
|
-
|
1974
2000
|
Add [request parameters](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-search-api-query-params) like `search_type`
|
1975
2001
|
|
1976
2002
|
```ruby
|
@@ -2008,22 +2034,17 @@ Turn on misspellings after a certain number of characters
|
|
2008
2034
|
Product.search "api", misspellings: {prefix_length: 2} # api, apt, no ahi
|
2009
2035
|
```
|
2010
2036
|
|
2011
|
-
**Note:** With this option, if the query length is the same as `prefix_length`, misspellings are turned off
|
2037
|
+
**Note:** With this option, if the query length is the same as `prefix_length`, misspellings are turned off with Elasticsearch 7 and OpenSearch
|
2012
2038
|
|
2013
2039
|
```ruby
|
2014
2040
|
Product.search "ah", misspellings: {prefix_length: 2} # ah, no aha
|
2015
2041
|
```
|
2016
2042
|
|
2017
|
-
##
|
2018
|
-
|
2019
|
-
1. Install Searchkick 4
|
2020
|
-
2. Upgrade your Elasticsearch cluster
|
2021
|
-
|
2022
|
-
## Elasticsearch Gotchas
|
2043
|
+
## Gotchas
|
2023
2044
|
|
2024
2045
|
### Consistency
|
2025
2046
|
|
2026
|
-
Elasticsearch
|
2047
|
+
Elasticsearch and OpenSearch are eventually consistent, meaning it can take up to a second for a change to reflect in search. You can use the `refresh` method to have it show up immediately.
|
2027
2048
|
|
2028
2049
|
```ruby
|
2029
2050
|
product.save!
|
@@ -2032,7 +2053,7 @@ Product.search_index.refresh
|
|
2032
2053
|
|
2033
2054
|
### Inconsistent Scores
|
2034
2055
|
|
2035
|
-
Due to the distributed nature of Elasticsearch, you can get incorrect results when the number of documents in the index is low. You can [read more about it here](https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch). To fix this, do:
|
2056
|
+
Due to the distributed nature of Elasticsearch and OpenSearch, you can get incorrect results when the number of documents in the index is low. You can [read more about it here](https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch). To fix this, do:
|
2036
2057
|
|
2037
2058
|
```ruby
|
2038
2059
|
class Product < ApplicationRecord
|
@@ -2042,6 +2063,42 @@ end
|
|
2042
2063
|
|
2043
2064
|
For convenience, this is set by default in the test environment.
|
2044
2065
|
|
2066
|
+
## Upgrading
|
2067
|
+
|
2068
|
+
### 5.0
|
2069
|
+
|
2070
|
+
Searchkick 5 supports both the `elasticsearch` and `opensearch-ruby` gems. Add the one you want to use to your Gemfile:
|
2071
|
+
|
2072
|
+
```ruby
|
2073
|
+
gem "elasticsearch"
|
2074
|
+
# or
|
2075
|
+
gem "opensearch-ruby"
|
2076
|
+
```
|
2077
|
+
|
2078
|
+
If using the deprecated `faraday_middleware-aws-signers-v4` gem, switch to `faraday_middleware-aws-sigv4`.
|
2079
|
+
|
2080
|
+
Also, searches now use lazy loading:
|
2081
|
+
|
2082
|
+
```ruby
|
2083
|
+
# search not executed
|
2084
|
+
Product.search("milk")
|
2085
|
+
|
2086
|
+
# search executed
|
2087
|
+
Product.search("milk").to_a
|
2088
|
+
```
|
2089
|
+
|
2090
|
+
You can reindex relations in the background:
|
2091
|
+
|
2092
|
+
```ruby
|
2093
|
+
store.products.reindex(mode: :async)
|
2094
|
+
# or
|
2095
|
+
store.products.reindex(mode: :queue)
|
2096
|
+
```
|
2097
|
+
|
2098
|
+
And there’s a [new option](#default-scopes) for models with default scopes.
|
2099
|
+
|
2100
|
+
Check out the [changelog](https://github.com/ankane/searchkick/blob/master/CHANGELOG.md#500-2022-02-21) for the full list of changes.
|
2101
|
+
|
2045
2102
|
## History
|
2046
2103
|
|
2047
2104
|
View the [changelog](https://github.com/ankane/searchkick/blob/master/CHANGELOG.md).
|
@@ -2,16 +2,20 @@ module Searchkick
|
|
2
2
|
class BulkReindexJob < ActiveJob::Base
|
3
3
|
queue_as { Searchkick.queue_name }
|
4
4
|
|
5
|
+
# TODO remove min_id and max_id in Searchkick 6
|
5
6
|
def perform(class_name:, record_ids: nil, index_name: nil, method_name: nil, batch_id: nil, min_id: nil, max_id: nil)
|
6
|
-
|
7
|
-
index =
|
7
|
+
model = Searchkick.load_model(class_name)
|
8
|
+
index = model.searchkick_index(name: index_name)
|
9
|
+
|
10
|
+
# legacy
|
8
11
|
record_ids ||= min_id..max_id
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
)
|
12
|
+
|
13
|
+
relation = Searchkick.scope(model)
|
14
|
+
relation = Searchkick.load_records(relation, record_ids)
|
15
|
+
relation = relation.search_import if relation.respond_to?(:search_import)
|
16
|
+
|
17
|
+
RecordIndexer.new(index).reindex(relation, mode: :inline, method_name: method_name, full: false)
|
18
|
+
RelationIndexer.new(index).batch_completed(batch_id) if batch_id
|
15
19
|
end
|
16
20
|
end
|
17
21
|
end
|
@@ -0,0 +1,40 @@
|
|
1
|
+
# based on https://gist.github.com/mnutt/566725
|
2
|
+
module Searchkick
|
3
|
+
module ControllerRuntime
|
4
|
+
extend ActiveSupport::Concern
|
5
|
+
|
6
|
+
protected
|
7
|
+
|
8
|
+
attr_internal :searchkick_runtime
|
9
|
+
|
10
|
+
def process_action(action, *args)
|
11
|
+
# We also need to reset the runtime before each action
|
12
|
+
# because of queries in middleware or in cases we are streaming
|
13
|
+
# and it won't be cleaned up by the method below.
|
14
|
+
Searchkick::LogSubscriber.reset_runtime
|
15
|
+
super
|
16
|
+
end
|
17
|
+
|
18
|
+
def cleanup_view_runtime
|
19
|
+
searchkick_rt_before_render = Searchkick::LogSubscriber.reset_runtime
|
20
|
+
runtime = super
|
21
|
+
searchkick_rt_after_render = Searchkick::LogSubscriber.reset_runtime
|
22
|
+
self.searchkick_runtime = searchkick_rt_before_render + searchkick_rt_after_render
|
23
|
+
runtime - searchkick_rt_after_render
|
24
|
+
end
|
25
|
+
|
26
|
+
def append_info_to_payload(payload)
|
27
|
+
super
|
28
|
+
payload[:searchkick_runtime] = (searchkick_runtime || 0) + Searchkick::LogSubscriber.reset_runtime
|
29
|
+
end
|
30
|
+
|
31
|
+
module ClassMethods
|
32
|
+
def log_process_action(payload)
|
33
|
+
messages = super
|
34
|
+
runtime = payload[:searchkick_runtime]
|
35
|
+
messages << ("Searchkick: %.1fms" % runtime.to_f) if runtime.to_f > 0
|
36
|
+
messages
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|