searchkick 2.1.0 → 2.1.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/Gemfile +1 -0
- data/README.md +103 -38
- data/lib/searchkick/index.rb +23 -10
- data/lib/searchkick/logging.rb +19 -1
- data/lib/searchkick/model.rb +1 -1
- data/lib/searchkick/query.rb +28 -5
- data/lib/searchkick/reindex_queue.rb +5 -9
- data/lib/searchkick/version.rb +1 -1
- data/lib/searchkick.rb +12 -0
- data/test/match_test.rb +19 -0
- data/test/order_test.rb +2 -2
- data/test/reindex_test.rb +0 -2
- data/test/test_helper.rb +7 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5fcfaef2d99cf6def7035bc882eecf161f687dca
|
4
|
+
data.tar.gz: a50200b8381e063e3c355586c898d1d8dc3b360a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6ddbce0ef6d29c51b2884ffc1791acedb886a3a4915889b059941fdcf0c9a5f9cbe68e50df7a82934d9cbc8b278b032bd789a2ec57d354f7b2fc09ebe9a91f8b
|
7
|
+
data.tar.gz: ab3dbbb296596682c5b9caaf6ba47f5d5299e60fd66bc53f31d14ecd9ac99f3f2f96404ea5f71e36f4cf02e310ffabb8c0d5c1b9a155d05e474e1e03bae67764
|
data/CHANGELOG.md
CHANGED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -34,6 +34,7 @@ Plus:
|
|
34
34
|
- [Getting Started](#getting-started)
|
35
35
|
- [Querying](#querying)
|
36
36
|
- [Indexing](#indexing)
|
37
|
+
- [Instant Search / Autocomplete](#instant-search--autocomplete)
|
37
38
|
- [Aggregations](#aggregations)
|
38
39
|
- [Deployment](#deployment)
|
39
40
|
- [Performance](#performance)
|
@@ -376,6 +377,14 @@ Turn off misspellings with:
|
|
376
377
|
Product.search "zuchini", misspellings: false # no zucchini
|
377
378
|
```
|
378
379
|
|
380
|
+
### Bad Matches
|
381
|
+
|
382
|
+
If a user searches `butter`, they may also get results for `peanut butter`. To prevent this, use:
|
383
|
+
|
384
|
+
```ruby
|
385
|
+
Product.search "butter", exclude: ["peanut butter"]
|
386
|
+
```
|
387
|
+
|
379
388
|
### Emoji
|
380
389
|
|
381
390
|
Search :ice_cream::cake: and get `ice cream cake`!
|
@@ -528,6 +537,11 @@ Product.search "apple", track: {user_id: current_user.id}
|
|
528
537
|
|
529
538
|
[See the docs](https://github.com/ankane/searchjoy) for how to install and use.
|
530
539
|
|
540
|
+
Focus on:
|
541
|
+
|
542
|
+
- top searches with low conversions
|
543
|
+
- top searches with no results
|
544
|
+
|
531
545
|
### Keep Getting Better
|
532
546
|
|
533
547
|
Searchkick can use conversion data to learn what users are looking for. If a user searches for “ice cream” and adds Ben & Jerry’s Chunky Monkey to the cart (our conversion metric at Instacart), that item gets a little more weight for similar searches.
|
@@ -589,20 +603,22 @@ Autocomplete predicts what a user will type, making the search experience faster
|
|
589
603
|
|
590
604
|
![Autocomplete](https://raw.githubusercontent.com/ankane/searchkick/gh-pages/autocomplete.png)
|
591
605
|
|
592
|
-
**Note:**
|
606
|
+
**Note:** To autocomplete on general categories (like `cereal` rather than product names), check out [Autosuggest](https://github.com/ankane/autosuggest).
|
607
|
+
|
608
|
+
**Note 2:** If you only have a few thousand records, don’t use Searchkick for autocomplete. It’s *much* faster to load all records into JavaScript and autocomplete there (eliminates network requests).
|
593
609
|
|
594
610
|
First, specify which fields use this feature. This is necessary since autocomplete can increase the index size significantly, but don’t worry - this gives you blazing faster queries.
|
595
611
|
|
596
612
|
```ruby
|
597
|
-
class
|
598
|
-
searchkick word_start: [:title, :
|
613
|
+
class Movie < ActiveRecord::Base
|
614
|
+
searchkick word_start: [:title, :director]
|
599
615
|
end
|
600
616
|
```
|
601
617
|
|
602
618
|
Reindex and search with:
|
603
619
|
|
604
620
|
```ruby
|
605
|
-
|
621
|
+
Movie.search "jurassic pa", match: :word_start
|
606
622
|
```
|
607
623
|
|
608
624
|
Typically, you want to use a JavaScript library like [typeahead.js](http://twitter.github.io/typeahead.js/) or [jQuery UI](http://jqueryui.com/autocomplete/).
|
@@ -612,11 +628,10 @@ Typically, you want to use a JavaScript library like [typeahead.js](http://twitt
|
|
612
628
|
First, add a route and controller action.
|
613
629
|
|
614
630
|
```ruby
|
615
|
-
|
616
|
-
class BooksController < ApplicationController
|
631
|
+
class MoviesController < ApplicationController
|
617
632
|
def autocomplete
|
618
|
-
render json:
|
619
|
-
fields: ["title^5", "
|
633
|
+
render json: Movie.search(params[:query], {
|
634
|
+
fields: ["title^5", "director"],
|
620
635
|
match: :word_start,
|
621
636
|
limit: 10,
|
622
637
|
load: false,
|
@@ -626,6 +641,8 @@ class BooksController < ApplicationController
|
|
626
641
|
end
|
627
642
|
```
|
628
643
|
|
644
|
+
**Note:** Use `load: false` and `misspellings: {below: n}` (or `misspellings: false`) for best performance.
|
645
|
+
|
629
646
|
Then add the search box and JavaScript code to a view.
|
630
647
|
|
631
648
|
```html
|
@@ -634,16 +651,16 @@ Then add the search box and JavaScript code to a view.
|
|
634
651
|
<script src="jquery.js"></script>
|
635
652
|
<script src="typeahead.bundle.js"></script>
|
636
653
|
<script>
|
637
|
-
var
|
654
|
+
var movies = new Bloodhound({
|
638
655
|
datumTokenizer: Bloodhound.tokenizers.whitespace,
|
639
656
|
queryTokenizer: Bloodhound.tokenizers.whitespace,
|
640
657
|
remote: {
|
641
|
-
url: '/
|
658
|
+
url: '/movies/autocomplete?query=%QUERY',
|
642
659
|
wildcard: '%QUERY'
|
643
660
|
}
|
644
661
|
});
|
645
662
|
$('#query').typeahead(null, {
|
646
|
-
source:
|
663
|
+
source: movies
|
647
664
|
});
|
648
665
|
</script>
|
649
666
|
```
|
@@ -844,7 +861,7 @@ product.similar(fields: [:name], where: {size: "12 oz"})
|
|
844
861
|
### Geospatial Searches
|
845
862
|
|
846
863
|
```ruby
|
847
|
-
class
|
864
|
+
class Restaurant < ActiveRecord::Base
|
848
865
|
searchkick locations: [:location]
|
849
866
|
|
850
867
|
def search_data
|
@@ -856,19 +873,19 @@ end
|
|
856
873
|
Reindex and search with:
|
857
874
|
|
858
875
|
```ruby
|
859
|
-
|
876
|
+
Restaurant.search "pizza", where: {location: {near: {lat: 37, lon: -114}, within: "100mi"}} # or 160km
|
860
877
|
```
|
861
878
|
|
862
879
|
Bounded by a box
|
863
880
|
|
864
881
|
```ruby
|
865
|
-
|
882
|
+
Restaurant.search "sushi", where: {location: {top_left: {lat: 38, lon: -123}, bottom_right: {lat: 37, lon: -122}}}
|
866
883
|
```
|
867
884
|
|
868
885
|
Bounded by a polygon
|
869
886
|
|
870
887
|
```ruby
|
871
|
-
|
888
|
+
Restaurant.search "dessert", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123}, {lat: 39, lon: -123}, {lat: 37, lon: 122}]}}}
|
872
889
|
```
|
873
890
|
|
874
891
|
### Boost By Distance
|
@@ -876,13 +893,13 @@ City.search "san", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123
|
|
876
893
|
Boost results by distance - closer results are boosted more
|
877
894
|
|
878
895
|
```ruby
|
879
|
-
|
896
|
+
Restaurant.search "noodles", boost_by_distance: {location: {origin: {lat: 37, lon: -122}}}
|
880
897
|
```
|
881
898
|
|
882
899
|
Also supports [additional options](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_decay_functions)
|
883
900
|
|
884
901
|
```ruby
|
885
|
-
|
902
|
+
Restaurant.search "wings", boost_by_distance: {location: {origin: {lat: 37, lon: -122}, function: "linear", scale: "30mi", decay: 0.5}}
|
886
903
|
```
|
887
904
|
|
888
905
|
### Geo Shapes
|
@@ -890,7 +907,7 @@ City.search "san", boost_by_distance: {location: {origin: {lat: 37, lon: -122},
|
|
890
907
|
You can also index and search geo shapes.
|
891
908
|
|
892
909
|
```ruby
|
893
|
-
class
|
910
|
+
class Restaurant < ActiveRecord::Base
|
894
911
|
searchkick geo_shape: {
|
895
912
|
bounds: {tree: "geohash", precision: "1km"}
|
896
913
|
}
|
@@ -911,25 +928,25 @@ See the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsea
|
|
911
928
|
Find shapes intersecting with the query shape
|
912
929
|
|
913
930
|
```ruby
|
914
|
-
|
931
|
+
Restaurant.search "soup", where: {bounds: {geo_shape: {type: "polygon", coordinates: [[{lat: 38, lon: -123}, ...]]}}}
|
915
932
|
```
|
916
933
|
|
917
934
|
Falling entirely within the query shape
|
918
935
|
|
919
936
|
```ruby
|
920
|
-
|
937
|
+
Restaurant.search "salad", where: {bounds: {geo_shape: {type: "circle", relation: "within", coordinates: [{lat: 38, lon: -123}], radius: "1km"}}}
|
921
938
|
```
|
922
939
|
|
923
940
|
Not touching the query shape
|
924
941
|
|
925
942
|
```ruby
|
926
|
-
|
943
|
+
Restaurant.search "burger", where: {bounds: {geo_shape: {type: "envelope", relation: "disjoint", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
|
927
944
|
```
|
928
945
|
|
929
946
|
Containing the query shape (Elasticsearch 2.2+)
|
930
947
|
|
931
948
|
```ruby
|
932
|
-
|
949
|
+
Restaurant.search "fries", where: {bounds: {geo_shape: {type: "envelope", relation: "contains", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
|
933
950
|
```
|
934
951
|
|
935
952
|
## Inheritance
|
@@ -1137,7 +1154,7 @@ If you run into issues on Windows, check out [this post](https://www.rastating.c
|
|
1137
1154
|
|
1138
1155
|
### Searchable Fields
|
1139
1156
|
|
1140
|
-
By default, all string fields are searchable. Speed up indexing and reduce index size by only making some fields searchable.
|
1157
|
+
By default, all string fields are searchable (can be used in `fields` option). Speed up indexing and reduce index size by only making some fields searchable. This disables the `_all` field unless it’s listed.
|
1141
1158
|
|
1142
1159
|
```ruby
|
1143
1160
|
class Product < ActiveRecord::Base
|
@@ -1182,12 +1199,45 @@ And use:
|
|
1182
1199
|
Searchkick.reindex_status(index_name)
|
1183
1200
|
```
|
1184
1201
|
|
1202
|
+
You can use [ActiveJob::TrafficControl](https://github.com/nickelser/activejob-traffic_control) to control concurrency. Install the gem and create an initializer with:
|
1203
|
+
|
1204
|
+
```ruby
|
1205
|
+
require "active_job/traffic_control"
|
1206
|
+
|
1207
|
+
Searchkick.redis = Redis.new
|
1208
|
+
ActiveJob::TrafficControl.client = Searchkick.redis
|
1209
|
+
|
1210
|
+
class Searchkick::BulkReindexJob
|
1211
|
+
include ActiveJob::TrafficControl::Concurrency
|
1212
|
+
|
1213
|
+
concurrency 3, drop: false
|
1214
|
+
end
|
1215
|
+
```
|
1216
|
+
|
1217
|
+
This will allow only 3 jobs to run at once.
|
1218
|
+
|
1219
|
+
### Refresh Interval
|
1220
|
+
|
1221
|
+
You can specify a longer refresh interval while reindexing to increase performance.
|
1222
|
+
|
1223
|
+
```ruby
|
1224
|
+
Product.reindex(async: true, refresh_interval: "30s")
|
1225
|
+
```
|
1226
|
+
|
1227
|
+
**Note:** This only makes a noticable difference with parallel reindexing.
|
1228
|
+
|
1229
|
+
When promoting, have it restored to the value in your mapping (defaults to `1s`).
|
1230
|
+
|
1231
|
+
```ruby
|
1232
|
+
Product.search_index.promote(index_name, update_refresh_interval: true)
|
1233
|
+
```
|
1234
|
+
|
1185
1235
|
### Queuing
|
1186
1236
|
|
1187
|
-
Push ids of records needing reindexed to a queue and reindex in bulk for better performance. First, set up Redis in an initializer.
|
1237
|
+
Push ids of records needing reindexed to a queue and reindex in bulk for better performance. First, set up Redis in an initializer. We recommend using [connection_pool](https://github.com/mperham/connection_pool).
|
1188
1238
|
|
1189
1239
|
```ruby
|
1190
|
-
Searchkick.redis = Redis.new
|
1240
|
+
Searchkick.redis = ConnectionPool.new { Redis.new }
|
1191
1241
|
```
|
1192
1242
|
|
1193
1243
|
And ask your models to queue updates.
|
@@ -1291,12 +1341,13 @@ class ReindexConversionsJob < ActiveJob::Base
|
|
1291
1341
|
|
1292
1342
|
# split into groups
|
1293
1343
|
recently_converted_ids.in_groups_of(1000, false) do |ids|
|
1294
|
-
# fetch conversions
|
1295
|
-
conversions_by_record = {}
|
1344
|
+
# fetch conversions
|
1296
1345
|
conversions =
|
1297
1346
|
Searchjoy::Search.where(convertable_id: ids, convertable_type: class_name)
|
1298
1347
|
.group(:convertable_id, :query).uniq.count(:user_id)
|
1299
1348
|
|
1349
|
+
# group conversions by record
|
1350
|
+
conversions_by_record = {}
|
1300
1351
|
conversions.each do |(id, query), count|
|
1301
1352
|
(conversions_by_record[id] ||= {})[query] = count
|
1302
1353
|
end
|
@@ -1379,6 +1430,14 @@ products =
|
|
1379
1430
|
end
|
1380
1431
|
```
|
1381
1432
|
|
1433
|
+
### Elasticsearch Gem
|
1434
|
+
|
1435
|
+
Searchkick is built on top of the [elasticsearch](https://github.com/elastic/elasticsearch-ruby) gem. To access the client directly, use:
|
1436
|
+
|
1437
|
+
```ruby
|
1438
|
+
Searchkick.client
|
1439
|
+
```
|
1440
|
+
|
1382
1441
|
## Multi Search
|
1383
1442
|
|
1384
1443
|
To batch search requests for performance, use:
|
@@ -1415,6 +1474,20 @@ To query nested data, use dot notation.
|
|
1415
1474
|
User.search "san", fields: ["address.city"], where: {"address.zip_code" => 12345}
|
1416
1475
|
```
|
1417
1476
|
|
1477
|
+
## Search Concepts
|
1478
|
+
|
1479
|
+
### Precision and Recall
|
1480
|
+
|
1481
|
+
[Precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall) are two key concepts in search (also known as *information retrieval*). To help illustrate, let’s walk through an example.
|
1482
|
+
|
1483
|
+
You have a store with 16 types of apples. A user searches for `apples` gets 10 results. 8 of the results are for apples, and 2 are for apple juice.
|
1484
|
+
|
1485
|
+
**Precision** is the fraction of documents in the results that are relevant. There are 10 results and 8 are relevant, so precision is 80%.
|
1486
|
+
|
1487
|
+
**Recall** is the fraction of relevant documents in the results out of all relevant documents. There are 16 apples and only 8 in the results, so recall is 50%.
|
1488
|
+
|
1489
|
+
There’s typically a trade-off between the two. As you tweak your search to increase precision (not return irrelevant documents), there’s are greater chance a relevant document also isn’t returned, which decreases recall. The opposite also applies. As you try to increase recall (return a higher number of relevent documents), there’s a greater chance you also return an irrelevant document, decreasing precision.
|
1490
|
+
|
1418
1491
|
## Reference
|
1419
1492
|
|
1420
1493
|
Reindex one record
|
@@ -1575,15 +1648,12 @@ Product.search("carrots", request_params: {search_type: "dfs_query_then_fetch"})
|
|
1575
1648
|
|
1576
1649
|
Reindex conditionally
|
1577
1650
|
|
1578
|
-
**Note:** With ActiveRecord, use this feature with caution - [transaction rollbacks can cause data inconsistencies](https://github.com/elasticsearch/elasticsearch-rails/blob/master/elasticsearch-model/README.md#custom-callbacks)
|
1579
|
-
|
1580
1651
|
```ruby
|
1581
1652
|
class Product < ActiveRecord::Base
|
1582
1653
|
searchkick callbacks: false
|
1583
1654
|
|
1584
1655
|
# add the callbacks manually
|
1585
|
-
|
1586
|
-
after_destroy :reindex
|
1656
|
+
after_commit :reindex, if: -> (model) { model.previous_changes.key?("name") } # use your own condition
|
1587
1657
|
end
|
1588
1658
|
```
|
1589
1659
|
|
@@ -1702,13 +1772,8 @@ Thanks to Karel Minarik for [Elasticsearch Ruby](https://github.com/elasticsearc
|
|
1702
1772
|
|
1703
1773
|
## Roadmap
|
1704
1774
|
|
1705
|
-
-
|
1706
|
-
-
|
1707
|
-
- Semantic search features
|
1708
|
-
- Search multiple fields for different terms
|
1709
|
-
- Search across models
|
1710
|
-
- Search nested objects
|
1711
|
-
- Much finer customization
|
1775
|
+
- Reindex API
|
1776
|
+
- Incorporate human eval
|
1712
1777
|
|
1713
1778
|
## Contributing
|
1714
1779
|
|
data/lib/searchkick/index.rb
CHANGED
@@ -66,26 +66,31 @@ module Searchkick
|
|
66
66
|
alias_method :swap, :promote
|
67
67
|
|
68
68
|
# record based
|
69
|
+
# use helpers for notifications
|
69
70
|
|
70
71
|
def store(record)
|
71
|
-
|
72
|
+
bulk_index_helper([record])
|
72
73
|
end
|
73
74
|
|
74
75
|
def remove(record)
|
75
|
-
|
76
|
+
bulk_delete_helper([record])
|
77
|
+
end
|
78
|
+
|
79
|
+
def update_record(record, method_name)
|
80
|
+
bulk_update_helper([record], method_name)
|
76
81
|
end
|
77
82
|
|
78
83
|
def bulk_delete(records)
|
79
|
-
|
84
|
+
bulk_delete_helper(records)
|
80
85
|
end
|
81
86
|
|
82
87
|
def bulk_index(records)
|
83
|
-
|
88
|
+
bulk_index_helper(records)
|
84
89
|
end
|
85
90
|
alias_method :import, :bulk_index
|
86
91
|
|
87
92
|
def bulk_update(records, method_name)
|
88
|
-
|
93
|
+
bulk_update_helper(records, method_name)
|
89
94
|
end
|
90
95
|
|
91
96
|
def record_data(r)
|
@@ -253,7 +258,7 @@ module Searchkick
|
|
253
258
|
|
254
259
|
if batch
|
255
260
|
import_or_update scope.to_a, method_name, async
|
256
|
-
|
261
|
+
Searchkick.with_redis { |r| r.srem(batches_key, batch_id) } if batch_id
|
257
262
|
elsif full && async
|
258
263
|
full_reindex_async(scope)
|
259
264
|
elsif scope.respond_to?(:find_in_batches)
|
@@ -278,7 +283,7 @@ module Searchkick
|
|
278
283
|
end
|
279
284
|
|
280
285
|
def batches_left
|
281
|
-
|
286
|
+
Searchkick.with_redis { |r| r.scard(batches_key) }
|
282
287
|
end
|
283
288
|
|
284
289
|
# other
|
@@ -454,7 +459,7 @@ module Searchkick
|
|
454
459
|
index_name: name,
|
455
460
|
batch_id: batch_id
|
456
461
|
}.merge(options))
|
457
|
-
|
462
|
+
Searchkick.with_redis { |r| r.sadd(batches_key, batch_id) }
|
458
463
|
end
|
459
464
|
|
460
465
|
def batch_size
|
@@ -475,8 +480,16 @@ module Searchkick
|
|
475
480
|
end
|
476
481
|
end
|
477
482
|
|
478
|
-
def
|
479
|
-
Searchkick.
|
483
|
+
def bulk_index_helper(records)
|
484
|
+
Searchkick.indexer.queue(records.map { |r| {index: record_data(r).merge(data: search_data(r))} })
|
485
|
+
end
|
486
|
+
|
487
|
+
def bulk_delete_helper(records)
|
488
|
+
Searchkick.indexer.queue(records.reject { |r| r.id.blank? }.map { |r| {delete: record_data(r)} })
|
489
|
+
end
|
490
|
+
|
491
|
+
def bulk_update_helper(records, method_name)
|
492
|
+
Searchkick.indexer.queue(records.map { |r| {update: record_data(r).merge(data: {doc: search_data(r, method_name)})} })
|
480
493
|
end
|
481
494
|
|
482
495
|
def batches_key
|
data/lib/searchkick/logging.rb
CHANGED
@@ -45,12 +45,27 @@ module Searchkick
|
|
45
45
|
end
|
46
46
|
end
|
47
47
|
|
48
|
-
def
|
48
|
+
def update_record(record, method_name)
|
49
|
+
event = {
|
50
|
+
name: "#{record.searchkick_klass.name} Update",
|
51
|
+
id: search_id(record)
|
52
|
+
}
|
53
|
+
if Searchkick.callbacks_value == :bulk
|
54
|
+
super
|
55
|
+
else
|
56
|
+
ActiveSupport::Notifications.instrument("request.searchkick", event) do
|
57
|
+
super
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
def bulk_index(records)
|
49
63
|
if records.any?
|
50
64
|
event = {
|
51
65
|
name: "#{records.first.searchkick_klass.name} Import",
|
52
66
|
count: records.size
|
53
67
|
}
|
68
|
+
event[:id] = search_id(records.first) if records.size == 1
|
54
69
|
if Searchkick.callbacks_value == :bulk
|
55
70
|
super
|
56
71
|
else
|
@@ -60,6 +75,7 @@ module Searchkick
|
|
60
75
|
end
|
61
76
|
end
|
62
77
|
end
|
78
|
+
alias_method :import, :bulk_index
|
63
79
|
|
64
80
|
def bulk_update(records, *args)
|
65
81
|
if records.any?
|
@@ -67,6 +83,7 @@ module Searchkick
|
|
67
83
|
name: "#{records.first.searchkick_klass.name} Update",
|
68
84
|
count: records.size
|
69
85
|
}
|
86
|
+
event[:id] = search_id(records.first) if records.size == 1
|
70
87
|
if Searchkick.callbacks_value == :bulk
|
71
88
|
super
|
72
89
|
else
|
@@ -83,6 +100,7 @@ module Searchkick
|
|
83
100
|
name: "#{records.first.searchkick_klass.name} Delete",
|
84
101
|
count: records.size
|
85
102
|
}
|
103
|
+
event[:id] = search_id(records.first) if records.size == 1
|
86
104
|
if Searchkick.callbacks_value == :bulk
|
87
105
|
super
|
88
106
|
else
|
data/lib/searchkick/model.rb
CHANGED
@@ -120,7 +120,7 @@ module Searchkick
|
|
120
120
|
end
|
121
121
|
else
|
122
122
|
if method_name
|
123
|
-
self.class.searchkick_index.
|
123
|
+
self.class.searchkick_index.update_record(self, method_name)
|
124
124
|
else
|
125
125
|
self.class.searchkick_index.reindex_record(self)
|
126
126
|
end
|
data/lib/searchkick/query.rb
CHANGED
@@ -14,7 +14,7 @@ module Searchkick
|
|
14
14
|
|
15
15
|
def initialize(klass, term = "*", **options)
|
16
16
|
unknown_keywords = options.keys - [:aggs, :body, :body_options, :boost,
|
17
|
-
:boost_by, :boost_by_distance, :boost_where, :conversions, :debug, :emoji, :execute, :explain,
|
17
|
+
:boost_by, :boost_by_distance, :boost_where, :conversions, :debug, :emoji, :exclude, :execute, :explain,
|
18
18
|
:fields, :highlight, :includes, :index_name, :indices_boost, :limit, :load,
|
19
19
|
:match, :misspellings, :offset, :operator, :order, :padding, :page, :per_page, :profile,
|
20
20
|
:request_params, :routing, :select, :similar, :smart_aggs, :suggest, :track, :type, :where]
|
@@ -264,6 +264,7 @@ module Searchkick
|
|
264
264
|
end
|
265
265
|
|
266
266
|
fields.each do |field|
|
267
|
+
queries_to_add = []
|
267
268
|
qs = []
|
268
269
|
|
269
270
|
factor = boost_fields[field] || 1
|
@@ -290,7 +291,7 @@ module Searchkick
|
|
290
291
|
]
|
291
292
|
elsif field.end_with?(".exact")
|
292
293
|
f = field.split(".")[0..-2].join(".")
|
293
|
-
|
294
|
+
queries_to_add << {match: {f => shared_options.merge(analyzer: "keyword")}}
|
294
295
|
else
|
295
296
|
analyzer = field =~ /\.word_(start|middle|end)\z/ ? "searchkick_word_search" : "searchkick_autocomplete_search"
|
296
297
|
qs << shared_options.merge(analyzer: analyzer)
|
@@ -300,21 +301,43 @@ module Searchkick
|
|
300
301
|
qs.concat qs.map { |q| q.except(:cutoff_frequency).merge(fuzziness: edit_distance, prefix_length: prefix_length, max_expansions: max_expansions, boost: factor).merge(transpositions) }
|
301
302
|
end
|
302
303
|
|
304
|
+
q2 = qs.map { |q| {match_type => {field => q}} }
|
305
|
+
|
303
306
|
# boost exact matches more
|
304
307
|
if field =~ /\.word_(start|middle|end)\z/ && searchkick_options[:word] != false
|
305
|
-
|
308
|
+
queries_to_add << {
|
306
309
|
bool: {
|
307
310
|
must: {
|
308
311
|
bool: {
|
309
|
-
should:
|
312
|
+
should: q2
|
310
313
|
}
|
311
314
|
},
|
312
315
|
should: {match_type => {field.sub(/\.word_(start|middle|end)\z/, ".analyzed") => qs.first}}
|
313
316
|
}
|
314
317
|
}
|
315
318
|
else
|
316
|
-
|
319
|
+
queries_to_add.concat(q2)
|
320
|
+
end
|
321
|
+
|
322
|
+
if options[:exclude]
|
323
|
+
must_not =
|
324
|
+
options[:exclude].map do |phrase|
|
325
|
+
{
|
326
|
+
match_phrase: {
|
327
|
+
field => phrase
|
328
|
+
}
|
329
|
+
}
|
330
|
+
end
|
331
|
+
|
332
|
+
queries_to_add = [{
|
333
|
+
bool: {
|
334
|
+
should: queries_to_add,
|
335
|
+
must_not: must_not
|
336
|
+
}
|
337
|
+
}]
|
317
338
|
end
|
339
|
+
|
340
|
+
queries.concat(queries_to_add)
|
318
341
|
end
|
319
342
|
|
320
343
|
payload = {
|
@@ -5,36 +5,32 @@ module Searchkick
|
|
5
5
|
def initialize(name)
|
6
6
|
@name = name
|
7
7
|
|
8
|
-
raise Searchkick::Error, "Searchkick.redis not set" unless redis
|
8
|
+
raise Searchkick::Error, "Searchkick.redis not set" unless Searchkick.redis
|
9
9
|
end
|
10
10
|
|
11
11
|
def push(record_id)
|
12
|
-
|
12
|
+
Searchkick.with_redis { |r| r.lpush(redis_key, record_id) }
|
13
13
|
end
|
14
14
|
|
15
15
|
# TODO use reliable queuing
|
16
16
|
def reserve(limit: 1000)
|
17
17
|
record_ids = Set.new
|
18
|
-
while record_ids.size < limit && record_id =
|
18
|
+
while record_ids.size < limit && record_id = Searchkick.with_redis { |r| r.rpop(redis_key) }
|
19
19
|
record_ids << record_id
|
20
20
|
end
|
21
21
|
record_ids.to_a
|
22
22
|
end
|
23
23
|
|
24
24
|
def clear
|
25
|
-
|
25
|
+
Searchkick.with_redis { |r| r.del(redis_key) }
|
26
26
|
end
|
27
27
|
|
28
28
|
def length
|
29
|
-
|
29
|
+
Searchkick.with_redis { |r| r.llen(redis_key) }
|
30
30
|
end
|
31
31
|
|
32
32
|
private
|
33
33
|
|
34
|
-
def redis
|
35
|
-
Searchkick.redis
|
36
|
-
end
|
37
|
-
|
38
34
|
def redis_key
|
39
35
|
"searchkick:reindex_queue:#{name}"
|
40
36
|
end
|
data/lib/searchkick/version.rb
CHANGED
data/lib/searchkick.rb
CHANGED
@@ -145,6 +145,18 @@ module Searchkick
|
|
145
145
|
end
|
146
146
|
end
|
147
147
|
|
148
|
+
def self.with_redis
|
149
|
+
if redis
|
150
|
+
if redis.respond_to?(:with)
|
151
|
+
redis.with do |r|
|
152
|
+
yield r
|
153
|
+
end
|
154
|
+
else
|
155
|
+
yield redis
|
156
|
+
end
|
157
|
+
end
|
158
|
+
end
|
159
|
+
|
148
160
|
# private
|
149
161
|
def self.load_records(records, ids)
|
150
162
|
records =
|
data/test/match_test.rb
CHANGED
@@ -159,6 +159,25 @@ class MatchTest < Minitest::Test
|
|
159
159
|
assert_search "almondmilks", ["Almond Milk"]
|
160
160
|
end
|
161
161
|
|
162
|
+
# butter
|
163
|
+
|
164
|
+
def test_butter
|
165
|
+
store_names ["Butter Tub", "Peanut Butter Tub"]
|
166
|
+
assert_search "butter", ["Butter Tub"], exclude: ["peanut butter"]
|
167
|
+
end
|
168
|
+
|
169
|
+
def test_butter_word_start
|
170
|
+
store_names ["Butter Tub", "Peanut Butter Tub"]
|
171
|
+
assert_search "butter", ["Butter Tub"], exclude: ["peanut butter"], match: :word_start
|
172
|
+
end
|
173
|
+
|
174
|
+
def test_butter_exact
|
175
|
+
store_names ["Butter Tub", "Peanut Butter Tub"]
|
176
|
+
assert_search "butter", [], exclude: ["peanut butter"], match: :exact
|
177
|
+
end
|
178
|
+
|
179
|
+
# other
|
180
|
+
|
162
181
|
def test_all
|
163
182
|
store_names ["Product A", "Product B"]
|
164
183
|
assert_search "*", ["Product A", "Product B"]
|
data/test/order_test.rb
CHANGED
@@ -29,12 +29,12 @@ class OrderTest < Minitest::Test
|
|
29
29
|
|
30
30
|
def test_order_ignore_unmapped
|
31
31
|
skip unless elasticsearch_below50?
|
32
|
-
assert_order "product", [], order: {not_mapped: {ignore_unmapped: true}}
|
32
|
+
assert_order "product", [], order: {not_mapped: {ignore_unmapped: true}}, conversions: false
|
33
33
|
end
|
34
34
|
|
35
35
|
def test_order_unmapped_type
|
36
36
|
skip if elasticsearch_below50?
|
37
|
-
assert_order "product", [], order: {not_mapped: {unmapped_type: "long"}}
|
37
|
+
assert_order "product", [], order: {not_mapped: {unmapped_type: "long"}}, conversions: false
|
38
38
|
end
|
39
39
|
|
40
40
|
def test_order_array
|
data/test/reindex_test.rb
CHANGED
data/test/test_helper.rb
CHANGED
@@ -14,7 +14,13 @@ File.delete("elasticsearch.log") if File.exist?("elasticsearch.log")
|
|
14
14
|
Searchkick.client.transport.logger = Logger.new("elasticsearch.log")
|
15
15
|
Searchkick.search_timeout = 5
|
16
16
|
|
17
|
-
|
17
|
+
if defined?(Redis)
|
18
|
+
if defined?(ConnectionPool)
|
19
|
+
Searchkick.redis = ConnectionPool.new { Redis.new }
|
20
|
+
else
|
21
|
+
Searchkick.redis = Redis.new
|
22
|
+
end
|
23
|
+
end
|
18
24
|
|
19
25
|
puts "Running against Elasticsearch #{Searchkick.server_version}"
|
20
26
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: searchkick
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.1.
|
4
|
+
version: 2.1.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-01-
|
11
|
+
date: 2017-01-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activemodel
|
@@ -189,7 +189,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
189
189
|
version: '0'
|
190
190
|
requirements: []
|
191
191
|
rubyforge_project:
|
192
|
-
rubygems_version: 2.
|
192
|
+
rubygems_version: 2.6.8
|
193
193
|
signing_key:
|
194
194
|
specification_version: 4
|
195
195
|
summary: Searchkick learns what your users are looking for. As more people search,
|