searchkick 2.1.0 → 2.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/Gemfile +1 -0
- data/README.md +103 -38
- data/lib/searchkick/index.rb +23 -10
- data/lib/searchkick/logging.rb +19 -1
- data/lib/searchkick/model.rb +1 -1
- data/lib/searchkick/query.rb +28 -5
- data/lib/searchkick/reindex_queue.rb +5 -9
- data/lib/searchkick/version.rb +1 -1
- data/lib/searchkick.rb +12 -0
- data/test/match_test.rb +19 -0
- data/test/order_test.rb +2 -2
- data/test/reindex_test.rb +0 -2
- data/test/test_helper.rb +7 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5fcfaef2d99cf6def7035bc882eecf161f687dca
|
4
|
+
data.tar.gz: a50200b8381e063e3c355586c898d1d8dc3b360a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6ddbce0ef6d29c51b2884ffc1791acedb886a3a4915889b059941fdcf0c9a5f9cbe68e50df7a82934d9cbc8b278b032bd789a2ec57d354f7b2fc09ebe9a91f8b
|
7
|
+
data.tar.gz: ab3dbbb296596682c5b9caaf6ba47f5d5299e60fd66bc53f31d14ecd9ac99f3f2f96404ea5f71e36f4cf02e310ffabb8c0d5c1b9a155d05e474e1e03bae67764
|
data/CHANGELOG.md
CHANGED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -34,6 +34,7 @@ Plus:
|
|
34
34
|
- [Getting Started](#getting-started)
|
35
35
|
- [Querying](#querying)
|
36
36
|
- [Indexing](#indexing)
|
37
|
+
- [Instant Search / Autocomplete](#instant-search--autocomplete)
|
37
38
|
- [Aggregations](#aggregations)
|
38
39
|
- [Deployment](#deployment)
|
39
40
|
- [Performance](#performance)
|
@@ -376,6 +377,14 @@ Turn off misspellings with:
|
|
376
377
|
Product.search "zuchini", misspellings: false # no zucchini
|
377
378
|
```
|
378
379
|
|
380
|
+
### Bad Matches
|
381
|
+
|
382
|
+
If a user searches `butter`, they may also get results for `peanut butter`. To prevent this, use:
|
383
|
+
|
384
|
+
```ruby
|
385
|
+
Product.search "butter", exclude: ["peanut butter"]
|
386
|
+
```
|
387
|
+
|
379
388
|
### Emoji
|
380
389
|
|
381
390
|
Search :ice_cream::cake: and get `ice cream cake`!
|
@@ -528,6 +537,11 @@ Product.search "apple", track: {user_id: current_user.id}
|
|
528
537
|
|
529
538
|
[See the docs](https://github.com/ankane/searchjoy) for how to install and use.
|
530
539
|
|
540
|
+
Focus on:
|
541
|
+
|
542
|
+
- top searches with low conversions
|
543
|
+
- top searches with no results
|
544
|
+
|
531
545
|
### Keep Getting Better
|
532
546
|
|
533
547
|
Searchkick can use conversion data to learn what users are looking for. If a user searches for “ice cream” and adds Ben & Jerry’s Chunky Monkey to the cart (our conversion metric at Instacart), that item gets a little more weight for similar searches.
|
@@ -589,20 +603,22 @@ Autocomplete predicts what a user will type, making the search experience faster
|
|
589
603
|
|
590
604
|

|
591
605
|
|
592
|
-
**Note:**
|
606
|
+
**Note:** To autocomplete on general categories (like `cereal` rather than product names), check out [Autosuggest](https://github.com/ankane/autosuggest).
|
607
|
+
|
608
|
+
**Note 2:** If you only have a few thousand records, don’t use Searchkick for autocomplete. It’s *much* faster to load all records into JavaScript and autocomplete there (eliminates network requests).
|
593
609
|
|
594
610
|
First, specify which fields use this feature. This is necessary since autocomplete can increase the index size significantly, but don’t worry - this gives you blazing faster queries.
|
595
611
|
|
596
612
|
```ruby
|
597
|
-
class
|
598
|
-
searchkick word_start: [:title, :
|
613
|
+
class Movie < ActiveRecord::Base
|
614
|
+
searchkick word_start: [:title, :director]
|
599
615
|
end
|
600
616
|
```
|
601
617
|
|
602
618
|
Reindex and search with:
|
603
619
|
|
604
620
|
```ruby
|
605
|
-
|
621
|
+
Movie.search "jurassic pa", match: :word_start
|
606
622
|
```
|
607
623
|
|
608
624
|
Typically, you want to use a JavaScript library like [typeahead.js](http://twitter.github.io/typeahead.js/) or [jQuery UI](http://jqueryui.com/autocomplete/).
|
@@ -612,11 +628,10 @@ Typically, you want to use a JavaScript library like [typeahead.js](http://twitt
|
|
612
628
|
First, add a route and controller action.
|
613
629
|
|
614
630
|
```ruby
|
615
|
-
|
616
|
-
class BooksController < ApplicationController
|
631
|
+
class MoviesController < ApplicationController
|
617
632
|
def autocomplete
|
618
|
-
render json:
|
619
|
-
fields: ["title^5", "
|
633
|
+
render json: Movie.search(params[:query], {
|
634
|
+
fields: ["title^5", "director"],
|
620
635
|
match: :word_start,
|
621
636
|
limit: 10,
|
622
637
|
load: false,
|
@@ -626,6 +641,8 @@ class BooksController < ApplicationController
|
|
626
641
|
end
|
627
642
|
```
|
628
643
|
|
644
|
+
**Note:** Use `load: false` and `misspellings: {below: n}` (or `misspellings: false`) for best performance.
|
645
|
+
|
629
646
|
Then add the search box and JavaScript code to a view.
|
630
647
|
|
631
648
|
```html
|
@@ -634,16 +651,16 @@ Then add the search box and JavaScript code to a view.
|
|
634
651
|
<script src="jquery.js"></script>
|
635
652
|
<script src="typeahead.bundle.js"></script>
|
636
653
|
<script>
|
637
|
-
var
|
654
|
+
var movies = new Bloodhound({
|
638
655
|
datumTokenizer: Bloodhound.tokenizers.whitespace,
|
639
656
|
queryTokenizer: Bloodhound.tokenizers.whitespace,
|
640
657
|
remote: {
|
641
|
-
url: '/
|
658
|
+
url: '/movies/autocomplete?query=%QUERY',
|
642
659
|
wildcard: '%QUERY'
|
643
660
|
}
|
644
661
|
});
|
645
662
|
$('#query').typeahead(null, {
|
646
|
-
source:
|
663
|
+
source: movies
|
647
664
|
});
|
648
665
|
</script>
|
649
666
|
```
|
@@ -844,7 +861,7 @@ product.similar(fields: [:name], where: {size: "12 oz"})
|
|
844
861
|
### Geospatial Searches
|
845
862
|
|
846
863
|
```ruby
|
847
|
-
class
|
864
|
+
class Restaurant < ActiveRecord::Base
|
848
865
|
searchkick locations: [:location]
|
849
866
|
|
850
867
|
def search_data
|
@@ -856,19 +873,19 @@ end
|
|
856
873
|
Reindex and search with:
|
857
874
|
|
858
875
|
```ruby
|
859
|
-
|
876
|
+
Restaurant.search "pizza", where: {location: {near: {lat: 37, lon: -114}, within: "100mi"}} # or 160km
|
860
877
|
```
|
861
878
|
|
862
879
|
Bounded by a box
|
863
880
|
|
864
881
|
```ruby
|
865
|
-
|
882
|
+
Restaurant.search "sushi", where: {location: {top_left: {lat: 38, lon: -123}, bottom_right: {lat: 37, lon: -122}}}
|
866
883
|
```
|
867
884
|
|
868
885
|
Bounded by a polygon
|
869
886
|
|
870
887
|
```ruby
|
871
|
-
|
888
|
+
Restaurant.search "dessert", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123}, {lat: 39, lon: -123}, {lat: 37, lon: 122}]}}}
|
872
889
|
```
|
873
890
|
|
874
891
|
### Boost By Distance
|
@@ -876,13 +893,13 @@ City.search "san", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123
|
|
876
893
|
Boost results by distance - closer results are boosted more
|
877
894
|
|
878
895
|
```ruby
|
879
|
-
|
896
|
+
Restaurant.search "noodles", boost_by_distance: {location: {origin: {lat: 37, lon: -122}}}
|
880
897
|
```
|
881
898
|
|
882
899
|
Also supports [additional options](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_decay_functions)
|
883
900
|
|
884
901
|
```ruby
|
885
|
-
|
902
|
+
Restaurant.search "wings", boost_by_distance: {location: {origin: {lat: 37, lon: -122}, function: "linear", scale: "30mi", decay: 0.5}}
|
886
903
|
```
|
887
904
|
|
888
905
|
### Geo Shapes
|
@@ -890,7 +907,7 @@ City.search "san", boost_by_distance: {location: {origin: {lat: 37, lon: -122},
|
|
890
907
|
You can also index and search geo shapes.
|
891
908
|
|
892
909
|
```ruby
|
893
|
-
class
|
910
|
+
class Restaurant < ActiveRecord::Base
|
894
911
|
searchkick geo_shape: {
|
895
912
|
bounds: {tree: "geohash", precision: "1km"}
|
896
913
|
}
|
@@ -911,25 +928,25 @@ See the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsea
|
|
911
928
|
Find shapes intersecting with the query shape
|
912
929
|
|
913
930
|
```ruby
|
914
|
-
|
931
|
+
Restaurant.search "soup", where: {bounds: {geo_shape: {type: "polygon", coordinates: [[{lat: 38, lon: -123}, ...]]}}}
|
915
932
|
```
|
916
933
|
|
917
934
|
Falling entirely within the query shape
|
918
935
|
|
919
936
|
```ruby
|
920
|
-
|
937
|
+
Restaurant.search "salad", where: {bounds: {geo_shape: {type: "circle", relation: "within", coordinates: [{lat: 38, lon: -123}], radius: "1km"}}}
|
921
938
|
```
|
922
939
|
|
923
940
|
Not touching the query shape
|
924
941
|
|
925
942
|
```ruby
|
926
|
-
|
943
|
+
Restaurant.search "burger", where: {bounds: {geo_shape: {type: "envelope", relation: "disjoint", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
|
927
944
|
```
|
928
945
|
|
929
946
|
Containing the query shape (Elasticsearch 2.2+)
|
930
947
|
|
931
948
|
```ruby
|
932
|
-
|
949
|
+
Restaurant.search "fries", where: {bounds: {geo_shape: {type: "envelope", relation: "contains", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
|
933
950
|
```
|
934
951
|
|
935
952
|
## Inheritance
|
@@ -1137,7 +1154,7 @@ If you run into issues on Windows, check out [this post](https://www.rastating.c
|
|
1137
1154
|
|
1138
1155
|
### Searchable Fields
|
1139
1156
|
|
1140
|
-
By default, all string fields are searchable. Speed up indexing and reduce index size by only making some fields searchable.
|
1157
|
+
By default, all string fields are searchable (can be used in `fields` option). Speed up indexing and reduce index size by only making some fields searchable. This disables the `_all` field unless it’s listed.
|
1141
1158
|
|
1142
1159
|
```ruby
|
1143
1160
|
class Product < ActiveRecord::Base
|
@@ -1182,12 +1199,45 @@ And use:
|
|
1182
1199
|
Searchkick.reindex_status(index_name)
|
1183
1200
|
```
|
1184
1201
|
|
1202
|
+
You can use [ActiveJob::TrafficControl](https://github.com/nickelser/activejob-traffic_control) to control concurrency. Install the gem and create an initializer with:
|
1203
|
+
|
1204
|
+
```ruby
|
1205
|
+
require "active_job/traffic_control"
|
1206
|
+
|
1207
|
+
Searchkick.redis = Redis.new
|
1208
|
+
ActiveJob::TrafficControl.client = Searchkick.redis
|
1209
|
+
|
1210
|
+
class Searchkick::BulkReindexJob
|
1211
|
+
include ActiveJob::TrafficControl::Concurrency
|
1212
|
+
|
1213
|
+
concurrency 3, drop: false
|
1214
|
+
end
|
1215
|
+
```
|
1216
|
+
|
1217
|
+
This will allow only 3 jobs to run at once.
|
1218
|
+
|
1219
|
+
### Refresh Interval
|
1220
|
+
|
1221
|
+
You can specify a longer refresh interval while reindexing to increase performance.
|
1222
|
+
|
1223
|
+
```ruby
|
1224
|
+
Product.reindex(async: true, refresh_interval: "30s")
|
1225
|
+
```
|
1226
|
+
|
1227
|
+
**Note:** This only makes a noticable difference with parallel reindexing.
|
1228
|
+
|
1229
|
+
When promoting, have it restored to the value in your mapping (defaults to `1s`).
|
1230
|
+
|
1231
|
+
```ruby
|
1232
|
+
Product.search_index.promote(index_name, update_refresh_interval: true)
|
1233
|
+
```
|
1234
|
+
|
1185
1235
|
### Queuing
|
1186
1236
|
|
1187
|
-
Push ids of records needing reindexed to a queue and reindex in bulk for better performance. First, set up Redis in an initializer.
|
1237
|
+
Push ids of records needing reindexed to a queue and reindex in bulk for better performance. First, set up Redis in an initializer. We recommend using [connection_pool](https://github.com/mperham/connection_pool).
|
1188
1238
|
|
1189
1239
|
```ruby
|
1190
|
-
Searchkick.redis = Redis.new
|
1240
|
+
Searchkick.redis = ConnectionPool.new { Redis.new }
|
1191
1241
|
```
|
1192
1242
|
|
1193
1243
|
And ask your models to queue updates.
|
@@ -1291,12 +1341,13 @@ class ReindexConversionsJob < ActiveJob::Base
|
|
1291
1341
|
|
1292
1342
|
# split into groups
|
1293
1343
|
recently_converted_ids.in_groups_of(1000, false) do |ids|
|
1294
|
-
# fetch conversions
|
1295
|
-
conversions_by_record = {}
|
1344
|
+
# fetch conversions
|
1296
1345
|
conversions =
|
1297
1346
|
Searchjoy::Search.where(convertable_id: ids, convertable_type: class_name)
|
1298
1347
|
.group(:convertable_id, :query).uniq.count(:user_id)
|
1299
1348
|
|
1349
|
+
# group conversions by record
|
1350
|
+
conversions_by_record = {}
|
1300
1351
|
conversions.each do |(id, query), count|
|
1301
1352
|
(conversions_by_record[id] ||= {})[query] = count
|
1302
1353
|
end
|
@@ -1379,6 +1430,14 @@ products =
|
|
1379
1430
|
end
|
1380
1431
|
```
|
1381
1432
|
|
1433
|
+
### Elasticsearch Gem
|
1434
|
+
|
1435
|
+
Searchkick is built on top of the [elasticsearch](https://github.com/elastic/elasticsearch-ruby) gem. To access the client directly, use:
|
1436
|
+
|
1437
|
+
```ruby
|
1438
|
+
Searchkick.client
|
1439
|
+
```
|
1440
|
+
|
1382
1441
|
## Multi Search
|
1383
1442
|
|
1384
1443
|
To batch search requests for performance, use:
|
@@ -1415,6 +1474,20 @@ To query nested data, use dot notation.
|
|
1415
1474
|
User.search "san", fields: ["address.city"], where: {"address.zip_code" => 12345}
|
1416
1475
|
```
|
1417
1476
|
|
1477
|
+
## Search Concepts
|
1478
|
+
|
1479
|
+
### Precision and Recall
|
1480
|
+
|
1481
|
+
[Precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall) are two key concepts in search (also known as *information retrieval*). To help illustrate, let’s walk through an example.
|
1482
|
+
|
1483
|
+
You have a store with 16 types of apples. A user searches for `apples` gets 10 results. 8 of the results are for apples, and 2 are for apple juice.
|
1484
|
+
|
1485
|
+
**Precision** is the fraction of documents in the results that are relevant. There are 10 results and 8 are relevant, so precision is 80%.
|
1486
|
+
|
1487
|
+
**Recall** is the fraction of relevant documents in the results out of all relevant documents. There are 16 apples and only 8 in the results, so recall is 50%.
|
1488
|
+
|
1489
|
+
There’s typically a trade-off between the two. As you tweak your search to increase precision (not return irrelevant documents), there’s are greater chance a relevant document also isn’t returned, which decreases recall. The opposite also applies. As you try to increase recall (return a higher number of relevent documents), there’s a greater chance you also return an irrelevant document, decreasing precision.
|
1490
|
+
|
1418
1491
|
## Reference
|
1419
1492
|
|
1420
1493
|
Reindex one record
|
@@ -1575,15 +1648,12 @@ Product.search("carrots", request_params: {search_type: "dfs_query_then_fetch"})
|
|
1575
1648
|
|
1576
1649
|
Reindex conditionally
|
1577
1650
|
|
1578
|
-
**Note:** With ActiveRecord, use this feature with caution - [transaction rollbacks can cause data inconsistencies](https://github.com/elasticsearch/elasticsearch-rails/blob/master/elasticsearch-model/README.md#custom-callbacks)
|
1579
|
-
|
1580
1651
|
```ruby
|
1581
1652
|
class Product < ActiveRecord::Base
|
1582
1653
|
searchkick callbacks: false
|
1583
1654
|
|
1584
1655
|
# add the callbacks manually
|
1585
|
-
|
1586
|
-
after_destroy :reindex
|
1656
|
+
after_commit :reindex, if: -> (model) { model.previous_changes.key?("name") } # use your own condition
|
1587
1657
|
end
|
1588
1658
|
```
|
1589
1659
|
|
@@ -1702,13 +1772,8 @@ Thanks to Karel Minarik for [Elasticsearch Ruby](https://github.com/elasticsearc
|
|
1702
1772
|
|
1703
1773
|
## Roadmap
|
1704
1774
|
|
1705
|
-
-
|
1706
|
-
-
|
1707
|
-
- Semantic search features
|
1708
|
-
- Search multiple fields for different terms
|
1709
|
-
- Search across models
|
1710
|
-
- Search nested objects
|
1711
|
-
- Much finer customization
|
1775
|
+
- Reindex API
|
1776
|
+
- Incorporate human eval
|
1712
1777
|
|
1713
1778
|
## Contributing
|
1714
1779
|
|
data/lib/searchkick/index.rb
CHANGED
@@ -66,26 +66,31 @@ module Searchkick
|
|
66
66
|
alias_method :swap, :promote
|
67
67
|
|
68
68
|
# record based
|
69
|
+
# use helpers for notifications
|
69
70
|
|
70
71
|
def store(record)
|
71
|
-
|
72
|
+
bulk_index_helper([record])
|
72
73
|
end
|
73
74
|
|
74
75
|
def remove(record)
|
75
|
-
|
76
|
+
bulk_delete_helper([record])
|
77
|
+
end
|
78
|
+
|
79
|
+
def update_record(record, method_name)
|
80
|
+
bulk_update_helper([record], method_name)
|
76
81
|
end
|
77
82
|
|
78
83
|
def bulk_delete(records)
|
79
|
-
|
84
|
+
bulk_delete_helper(records)
|
80
85
|
end
|
81
86
|
|
82
87
|
def bulk_index(records)
|
83
|
-
|
88
|
+
bulk_index_helper(records)
|
84
89
|
end
|
85
90
|
alias_method :import, :bulk_index
|
86
91
|
|
87
92
|
def bulk_update(records, method_name)
|
88
|
-
|
93
|
+
bulk_update_helper(records, method_name)
|
89
94
|
end
|
90
95
|
|
91
96
|
def record_data(r)
|
@@ -253,7 +258,7 @@ module Searchkick
|
|
253
258
|
|
254
259
|
if batch
|
255
260
|
import_or_update scope.to_a, method_name, async
|
256
|
-
|
261
|
+
Searchkick.with_redis { |r| r.srem(batches_key, batch_id) } if batch_id
|
257
262
|
elsif full && async
|
258
263
|
full_reindex_async(scope)
|
259
264
|
elsif scope.respond_to?(:find_in_batches)
|
@@ -278,7 +283,7 @@ module Searchkick
|
|
278
283
|
end
|
279
284
|
|
280
285
|
def batches_left
|
281
|
-
|
286
|
+
Searchkick.with_redis { |r| r.scard(batches_key) }
|
282
287
|
end
|
283
288
|
|
284
289
|
# other
|
@@ -454,7 +459,7 @@ module Searchkick
|
|
454
459
|
index_name: name,
|
455
460
|
batch_id: batch_id
|
456
461
|
}.merge(options))
|
457
|
-
|
462
|
+
Searchkick.with_redis { |r| r.sadd(batches_key, batch_id) }
|
458
463
|
end
|
459
464
|
|
460
465
|
def batch_size
|
@@ -475,8 +480,16 @@ module Searchkick
|
|
475
480
|
end
|
476
481
|
end
|
477
482
|
|
478
|
-
def
|
479
|
-
Searchkick.
|
483
|
+
def bulk_index_helper(records)
|
484
|
+
Searchkick.indexer.queue(records.map { |r| {index: record_data(r).merge(data: search_data(r))} })
|
485
|
+
end
|
486
|
+
|
487
|
+
def bulk_delete_helper(records)
|
488
|
+
Searchkick.indexer.queue(records.reject { |r| r.id.blank? }.map { |r| {delete: record_data(r)} })
|
489
|
+
end
|
490
|
+
|
491
|
+
def bulk_update_helper(records, method_name)
|
492
|
+
Searchkick.indexer.queue(records.map { |r| {update: record_data(r).merge(data: {doc: search_data(r, method_name)})} })
|
480
493
|
end
|
481
494
|
|
482
495
|
def batches_key
|
data/lib/searchkick/logging.rb
CHANGED
@@ -45,12 +45,27 @@ module Searchkick
|
|
45
45
|
end
|
46
46
|
end
|
47
47
|
|
48
|
-
def
|
48
|
+
def update_record(record, method_name)
|
49
|
+
event = {
|
50
|
+
name: "#{record.searchkick_klass.name} Update",
|
51
|
+
id: search_id(record)
|
52
|
+
}
|
53
|
+
if Searchkick.callbacks_value == :bulk
|
54
|
+
super
|
55
|
+
else
|
56
|
+
ActiveSupport::Notifications.instrument("request.searchkick", event) do
|
57
|
+
super
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
def bulk_index(records)
|
49
63
|
if records.any?
|
50
64
|
event = {
|
51
65
|
name: "#{records.first.searchkick_klass.name} Import",
|
52
66
|
count: records.size
|
53
67
|
}
|
68
|
+
event[:id] = search_id(records.first) if records.size == 1
|
54
69
|
if Searchkick.callbacks_value == :bulk
|
55
70
|
super
|
56
71
|
else
|
@@ -60,6 +75,7 @@ module Searchkick
|
|
60
75
|
end
|
61
76
|
end
|
62
77
|
end
|
78
|
+
alias_method :import, :bulk_index
|
63
79
|
|
64
80
|
def bulk_update(records, *args)
|
65
81
|
if records.any?
|
@@ -67,6 +83,7 @@ module Searchkick
|
|
67
83
|
name: "#{records.first.searchkick_klass.name} Update",
|
68
84
|
count: records.size
|
69
85
|
}
|
86
|
+
event[:id] = search_id(records.first) if records.size == 1
|
70
87
|
if Searchkick.callbacks_value == :bulk
|
71
88
|
super
|
72
89
|
else
|
@@ -83,6 +100,7 @@ module Searchkick
|
|
83
100
|
name: "#{records.first.searchkick_klass.name} Delete",
|
84
101
|
count: records.size
|
85
102
|
}
|
103
|
+
event[:id] = search_id(records.first) if records.size == 1
|
86
104
|
if Searchkick.callbacks_value == :bulk
|
87
105
|
super
|
88
106
|
else
|
data/lib/searchkick/model.rb
CHANGED
@@ -120,7 +120,7 @@ module Searchkick
|
|
120
120
|
end
|
121
121
|
else
|
122
122
|
if method_name
|
123
|
-
self.class.searchkick_index.
|
123
|
+
self.class.searchkick_index.update_record(self, method_name)
|
124
124
|
else
|
125
125
|
self.class.searchkick_index.reindex_record(self)
|
126
126
|
end
|
data/lib/searchkick/query.rb
CHANGED
@@ -14,7 +14,7 @@ module Searchkick
|
|
14
14
|
|
15
15
|
def initialize(klass, term = "*", **options)
|
16
16
|
unknown_keywords = options.keys - [:aggs, :body, :body_options, :boost,
|
17
|
-
:boost_by, :boost_by_distance, :boost_where, :conversions, :debug, :emoji, :execute, :explain,
|
17
|
+
:boost_by, :boost_by_distance, :boost_where, :conversions, :debug, :emoji, :exclude, :execute, :explain,
|
18
18
|
:fields, :highlight, :includes, :index_name, :indices_boost, :limit, :load,
|
19
19
|
:match, :misspellings, :offset, :operator, :order, :padding, :page, :per_page, :profile,
|
20
20
|
:request_params, :routing, :select, :similar, :smart_aggs, :suggest, :track, :type, :where]
|
@@ -264,6 +264,7 @@ module Searchkick
|
|
264
264
|
end
|
265
265
|
|
266
266
|
fields.each do |field|
|
267
|
+
queries_to_add = []
|
267
268
|
qs = []
|
268
269
|
|
269
270
|
factor = boost_fields[field] || 1
|
@@ -290,7 +291,7 @@ module Searchkick
|
|
290
291
|
]
|
291
292
|
elsif field.end_with?(".exact")
|
292
293
|
f = field.split(".")[0..-2].join(".")
|
293
|
-
|
294
|
+
queries_to_add << {match: {f => shared_options.merge(analyzer: "keyword")}}
|
294
295
|
else
|
295
296
|
analyzer = field =~ /\.word_(start|middle|end)\z/ ? "searchkick_word_search" : "searchkick_autocomplete_search"
|
296
297
|
qs << shared_options.merge(analyzer: analyzer)
|
@@ -300,21 +301,43 @@ module Searchkick
|
|
300
301
|
qs.concat qs.map { |q| q.except(:cutoff_frequency).merge(fuzziness: edit_distance, prefix_length: prefix_length, max_expansions: max_expansions, boost: factor).merge(transpositions) }
|
301
302
|
end
|
302
303
|
|
304
|
+
q2 = qs.map { |q| {match_type => {field => q}} }
|
305
|
+
|
303
306
|
# boost exact matches more
|
304
307
|
if field =~ /\.word_(start|middle|end)\z/ && searchkick_options[:word] != false
|
305
|
-
|
308
|
+
queries_to_add << {
|
306
309
|
bool: {
|
307
310
|
must: {
|
308
311
|
bool: {
|
309
|
-
should:
|
312
|
+
should: q2
|
310
313
|
}
|
311
314
|
},
|
312
315
|
should: {match_type => {field.sub(/\.word_(start|middle|end)\z/, ".analyzed") => qs.first}}
|
313
316
|
}
|
314
317
|
}
|
315
318
|
else
|
316
|
-
|
319
|
+
queries_to_add.concat(q2)
|
320
|
+
end
|
321
|
+
|
322
|
+
if options[:exclude]
|
323
|
+
must_not =
|
324
|
+
options[:exclude].map do |phrase|
|
325
|
+
{
|
326
|
+
match_phrase: {
|
327
|
+
field => phrase
|
328
|
+
}
|
329
|
+
}
|
330
|
+
end
|
331
|
+
|
332
|
+
queries_to_add = [{
|
333
|
+
bool: {
|
334
|
+
should: queries_to_add,
|
335
|
+
must_not: must_not
|
336
|
+
}
|
337
|
+
}]
|
317
338
|
end
|
339
|
+
|
340
|
+
queries.concat(queries_to_add)
|
318
341
|
end
|
319
342
|
|
320
343
|
payload = {
|
@@ -5,36 +5,32 @@ module Searchkick
|
|
5
5
|
def initialize(name)
|
6
6
|
@name = name
|
7
7
|
|
8
|
-
raise Searchkick::Error, "Searchkick.redis not set" unless redis
|
8
|
+
raise Searchkick::Error, "Searchkick.redis not set" unless Searchkick.redis
|
9
9
|
end
|
10
10
|
|
11
11
|
def push(record_id)
|
12
|
-
|
12
|
+
Searchkick.with_redis { |r| r.lpush(redis_key, record_id) }
|
13
13
|
end
|
14
14
|
|
15
15
|
# TODO use reliable queuing
|
16
16
|
def reserve(limit: 1000)
|
17
17
|
record_ids = Set.new
|
18
|
-
while record_ids.size < limit && record_id =
|
18
|
+
while record_ids.size < limit && record_id = Searchkick.with_redis { |r| r.rpop(redis_key) }
|
19
19
|
record_ids << record_id
|
20
20
|
end
|
21
21
|
record_ids.to_a
|
22
22
|
end
|
23
23
|
|
24
24
|
def clear
|
25
|
-
|
25
|
+
Searchkick.with_redis { |r| r.del(redis_key) }
|
26
26
|
end
|
27
27
|
|
28
28
|
def length
|
29
|
-
|
29
|
+
Searchkick.with_redis { |r| r.llen(redis_key) }
|
30
30
|
end
|
31
31
|
|
32
32
|
private
|
33
33
|
|
34
|
-
def redis
|
35
|
-
Searchkick.redis
|
36
|
-
end
|
37
|
-
|
38
34
|
def redis_key
|
39
35
|
"searchkick:reindex_queue:#{name}"
|
40
36
|
end
|
data/lib/searchkick/version.rb
CHANGED
data/lib/searchkick.rb
CHANGED
@@ -145,6 +145,18 @@ module Searchkick
|
|
145
145
|
end
|
146
146
|
end
|
147
147
|
|
148
|
+
def self.with_redis
|
149
|
+
if redis
|
150
|
+
if redis.respond_to?(:with)
|
151
|
+
redis.with do |r|
|
152
|
+
yield r
|
153
|
+
end
|
154
|
+
else
|
155
|
+
yield redis
|
156
|
+
end
|
157
|
+
end
|
158
|
+
end
|
159
|
+
|
148
160
|
# private
|
149
161
|
def self.load_records(records, ids)
|
150
162
|
records =
|
data/test/match_test.rb
CHANGED
@@ -159,6 +159,25 @@ class MatchTest < Minitest::Test
|
|
159
159
|
assert_search "almondmilks", ["Almond Milk"]
|
160
160
|
end
|
161
161
|
|
162
|
+
# butter
|
163
|
+
|
164
|
+
def test_butter
|
165
|
+
store_names ["Butter Tub", "Peanut Butter Tub"]
|
166
|
+
assert_search "butter", ["Butter Tub"], exclude: ["peanut butter"]
|
167
|
+
end
|
168
|
+
|
169
|
+
def test_butter_word_start
|
170
|
+
store_names ["Butter Tub", "Peanut Butter Tub"]
|
171
|
+
assert_search "butter", ["Butter Tub"], exclude: ["peanut butter"], match: :word_start
|
172
|
+
end
|
173
|
+
|
174
|
+
def test_butter_exact
|
175
|
+
store_names ["Butter Tub", "Peanut Butter Tub"]
|
176
|
+
assert_search "butter", [], exclude: ["peanut butter"], match: :exact
|
177
|
+
end
|
178
|
+
|
179
|
+
# other
|
180
|
+
|
162
181
|
def test_all
|
163
182
|
store_names ["Product A", "Product B"]
|
164
183
|
assert_search "*", ["Product A", "Product B"]
|
data/test/order_test.rb
CHANGED
@@ -29,12 +29,12 @@ class OrderTest < Minitest::Test
|
|
29
29
|
|
30
30
|
def test_order_ignore_unmapped
|
31
31
|
skip unless elasticsearch_below50?
|
32
|
-
assert_order "product", [], order: {not_mapped: {ignore_unmapped: true}}
|
32
|
+
assert_order "product", [], order: {not_mapped: {ignore_unmapped: true}}, conversions: false
|
33
33
|
end
|
34
34
|
|
35
35
|
def test_order_unmapped_type
|
36
36
|
skip if elasticsearch_below50?
|
37
|
-
assert_order "product", [], order: {not_mapped: {unmapped_type: "long"}}
|
37
|
+
assert_order "product", [], order: {not_mapped: {unmapped_type: "long"}}, conversions: false
|
38
38
|
end
|
39
39
|
|
40
40
|
def test_order_array
|
data/test/reindex_test.rb
CHANGED
data/test/test_helper.rb
CHANGED
@@ -14,7 +14,13 @@ File.delete("elasticsearch.log") if File.exist?("elasticsearch.log")
|
|
14
14
|
Searchkick.client.transport.logger = Logger.new("elasticsearch.log")
|
15
15
|
Searchkick.search_timeout = 5
|
16
16
|
|
17
|
-
|
17
|
+
if defined?(Redis)
|
18
|
+
if defined?(ConnectionPool)
|
19
|
+
Searchkick.redis = ConnectionPool.new { Redis.new }
|
20
|
+
else
|
21
|
+
Searchkick.redis = Redis.new
|
22
|
+
end
|
23
|
+
end
|
18
24
|
|
19
25
|
puts "Running against Elasticsearch #{Searchkick.server_version}"
|
20
26
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: searchkick
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.1.
|
4
|
+
version: 2.1.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-01-
|
11
|
+
date: 2017-01-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activemodel
|
@@ -189,7 +189,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
189
189
|
version: '0'
|
190
190
|
requirements: []
|
191
191
|
rubyforge_project:
|
192
|
-
rubygems_version: 2.
|
192
|
+
rubygems_version: 2.6.8
|
193
193
|
signing_key:
|
194
194
|
specification_version: 4
|
195
195
|
summary: Searchkick learns what your users are looking for. As more people search,
|