searchkick 2.1.0 → 2.1.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 95d822316595ca2806bc9f2e3e44644ee2ab5fd8
4
- data.tar.gz: 27f86dc98a28378ee489b5adf7bfd11be054baf1
3
+ metadata.gz: 5fcfaef2d99cf6def7035bc882eecf161f687dca
4
+ data.tar.gz: a50200b8381e063e3c355586c898d1d8dc3b360a
5
5
  SHA512:
6
- metadata.gz: fb18990e391306eb6d3f9a6193b8c14acf8cb22691f0a470d7c47b40c6146a58bc7babc6250378dfadf86b85f5f6846bd79f5302969b54de632df77aa7a55809
7
- data.tar.gz: 0983c46e8eda2d402cc632a9849e97fca61f3036cfa96378ff81921dc6901416671b6192a184a23dc04615534422f79a7f00c2694c3b9cb46d7b38f7a2410345
6
+ metadata.gz: 6ddbce0ef6d29c51b2884ffc1791acedb886a3a4915889b059941fdcf0c9a5f9cbe68e50df7a82934d9cbc8b278b032bd789a2ec57d354f7b2fc09ebe9a91f8b
7
+ data.tar.gz: ab3dbbb296596682c5b9caaf6ba47f5d5299e60fd66bc53f31d14ecd9ac99f3f2f96404ea5f71e36f4cf02e310ffabb8c0d5c1b9a155d05e474e1e03bae67764
data/CHANGELOG.md CHANGED
@@ -1,3 +1,9 @@
1
+ ## 2.1.1
2
+
3
+ - Fixed duplicate notifications
4
+ - Added support for `connection_pool`
5
+ - Added `exclude` option
6
+
1
7
  ## 2.1.0
2
8
 
3
9
  - Background reindexing and queues are officially supported
data/Gemfile CHANGED
@@ -9,3 +9,4 @@ gem "gemoji-parser"
9
9
  gem "typhoeus"
10
10
  gem "activejob"
11
11
  gem "redis"
12
+ gem "connection_pool"
data/README.md CHANGED
@@ -34,6 +34,7 @@ Plus:
34
34
  - [Getting Started](#getting-started)
35
35
  - [Querying](#querying)
36
36
  - [Indexing](#indexing)
37
+ - [Instant Search / Autocomplete](#instant-search--autocomplete)
37
38
  - [Aggregations](#aggregations)
38
39
  - [Deployment](#deployment)
39
40
  - [Performance](#performance)
@@ -376,6 +377,14 @@ Turn off misspellings with:
376
377
  Product.search "zuchini", misspellings: false # no zucchini
377
378
  ```
378
379
 
380
+ ### Bad Matches
381
+
382
+ If a user searches `butter`, they may also get results for `peanut butter`. To prevent this, use:
383
+
384
+ ```ruby
385
+ Product.search "butter", exclude: ["peanut butter"]
386
+ ```
387
+
379
388
  ### Emoji
380
389
 
381
390
  Search :ice_cream::cake: and get `ice cream cake`!
@@ -528,6 +537,11 @@ Product.search "apple", track: {user_id: current_user.id}
528
537
 
529
538
  [See the docs](https://github.com/ankane/searchjoy) for how to install and use.
530
539
 
540
+ Focus on:
541
+
542
+ - top searches with low conversions
543
+ - top searches with no results
544
+
531
545
  ### Keep Getting Better
532
546
 
533
547
  Searchkick can use conversion data to learn what users are looking for. If a user searches for “ice cream” and adds Ben & Jerry’s Chunky Monkey to the cart (our conversion metric at Instacart), that item gets a little more weight for similar searches.
@@ -589,20 +603,22 @@ Autocomplete predicts what a user will type, making the search experience faster
589
603
 
590
604
  ![Autocomplete](https://raw.githubusercontent.com/ankane/searchkick/gh-pages/autocomplete.png)
591
605
 
592
- **Note:** If you only have a few thousand records, don’t use Searchkick for autocomplete. It’s *much* faster to load all records into JavaScript and autocomplete there (eliminates network requests).
606
+ **Note:** To autocomplete on general categories (like `cereal` rather than product names), check out [Autosuggest](https://github.com/ankane/autosuggest).
607
+
608
+ **Note 2:** If you only have a few thousand records, don’t use Searchkick for autocomplete. It’s *much* faster to load all records into JavaScript and autocomplete there (eliminates network requests).
593
609
 
594
610
  First, specify which fields use this feature. This is necessary since autocomplete can increase the index size significantly, but don’t worry - this gives you blazing faster queries.
595
611
 
596
612
  ```ruby
597
- class Book < ActiveRecord::Base
598
- searchkick word_start: [:title, :author]
613
+ class Movie < ActiveRecord::Base
614
+ searchkick word_start: [:title, :director]
599
615
  end
600
616
  ```
601
617
 
602
618
  Reindex and search with:
603
619
 
604
620
  ```ruby
605
- Book.search "tipping poi", match: :word_start
621
+ Movie.search "jurassic pa", match: :word_start
606
622
  ```
607
623
 
608
624
  Typically, you want to use a JavaScript library like [typeahead.js](http://twitter.github.io/typeahead.js/) or [jQuery UI](http://jqueryui.com/autocomplete/).
@@ -612,11 +628,10 @@ Typically, you want to use a JavaScript library like [typeahead.js](http://twitt
612
628
  First, add a route and controller action.
613
629
 
614
630
  ```ruby
615
- # app/controllers/books_controller.rb
616
- class BooksController < ApplicationController
631
+ class MoviesController < ApplicationController
617
632
  def autocomplete
618
- render json: Book.search(params[:query], {
619
- fields: ["title^5", "author"],
633
+ render json: Movie.search(params[:query], {
634
+ fields: ["title^5", "director"],
620
635
  match: :word_start,
621
636
  limit: 10,
622
637
  load: false,
@@ -626,6 +641,8 @@ class BooksController < ApplicationController
626
641
  end
627
642
  ```
628
643
 
644
+ **Note:** Use `load: false` and `misspellings: {below: n}` (or `misspellings: false`) for best performance.
645
+
629
646
  Then add the search box and JavaScript code to a view.
630
647
 
631
648
  ```html
@@ -634,16 +651,16 @@ Then add the search box and JavaScript code to a view.
634
651
  <script src="jquery.js"></script>
635
652
  <script src="typeahead.bundle.js"></script>
636
653
  <script>
637
- var books = new Bloodhound({
654
+ var movies = new Bloodhound({
638
655
  datumTokenizer: Bloodhound.tokenizers.whitespace,
639
656
  queryTokenizer: Bloodhound.tokenizers.whitespace,
640
657
  remote: {
641
- url: '/books/autocomplete?query=%QUERY',
658
+ url: '/movies/autocomplete?query=%QUERY',
642
659
  wildcard: '%QUERY'
643
660
  }
644
661
  });
645
662
  $('#query').typeahead(null, {
646
- source: books
663
+ source: movies
647
664
  });
648
665
  </script>
649
666
  ```
@@ -844,7 +861,7 @@ product.similar(fields: [:name], where: {size: "12 oz"})
844
861
  ### Geospatial Searches
845
862
 
846
863
  ```ruby
847
- class City < ActiveRecord::Base
864
+ class Restaurant < ActiveRecord::Base
848
865
  searchkick locations: [:location]
849
866
 
850
867
  def search_data
@@ -856,19 +873,19 @@ end
856
873
  Reindex and search with:
857
874
 
858
875
  ```ruby
859
- City.search "san", where: {location: {near: {lat: 37, lon: -114}, within: "100mi"}} # or 160km
876
+ Restaurant.search "pizza", where: {location: {near: {lat: 37, lon: -114}, within: "100mi"}} # or 160km
860
877
  ```
861
878
 
862
879
  Bounded by a box
863
880
 
864
881
  ```ruby
865
- City.search "san", where: {location: {top_left: {lat: 38, lon: -123}, bottom_right: {lat: 37, lon: -122}}}
882
+ Restaurant.search "sushi", where: {location: {top_left: {lat: 38, lon: -123}, bottom_right: {lat: 37, lon: -122}}}
866
883
  ```
867
884
 
868
885
  Bounded by a polygon
869
886
 
870
887
  ```ruby
871
- City.search "san", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123}, {lat: 39, lon: -123}, {lat: 37, lon: 122}]}}}
888
+ Restaurant.search "dessert", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123}, {lat: 39, lon: -123}, {lat: 37, lon: 122}]}}}
872
889
  ```
873
890
 
874
891
  ### Boost By Distance
@@ -876,13 +893,13 @@ City.search "san", where: {location: {geo_polygon: {points: [{lat: 38, lon: -123
876
893
  Boost results by distance - closer results are boosted more
877
894
 
878
895
  ```ruby
879
- City.search "san", boost_by_distance: {location: {origin: {lat: 37, lon: -122}}}
896
+ Restaurant.search "noodles", boost_by_distance: {location: {origin: {lat: 37, lon: -122}}}
880
897
  ```
881
898
 
882
899
  Also supports [additional options](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_decay_functions)
883
900
 
884
901
  ```ruby
885
- City.search "san", boost_by_distance: {location: {origin: {lat: 37, lon: -122}, function: "linear", scale: "30mi", decay: 0.5}}
902
+ Restaurant.search "wings", boost_by_distance: {location: {origin: {lat: 37, lon: -122}, function: "linear", scale: "30mi", decay: 0.5}}
886
903
  ```
887
904
 
888
905
  ### Geo Shapes
@@ -890,7 +907,7 @@ City.search "san", boost_by_distance: {location: {origin: {lat: 37, lon: -122},
890
907
  You can also index and search geo shapes.
891
908
 
892
909
  ```ruby
893
- class City < ActiveRecord::Base
910
+ class Restaurant < ActiveRecord::Base
894
911
  searchkick geo_shape: {
895
912
  bounds: {tree: "geohash", precision: "1km"}
896
913
  }
@@ -911,25 +928,25 @@ See the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsea
911
928
  Find shapes intersecting with the query shape
912
929
 
913
930
  ```ruby
914
- City.search "san", where: {bounds: {geo_shape: {type: "polygon", coordinates: [[{lat: 38, lon: -123}, ...]]}}}
931
+ Restaurant.search "soup", where: {bounds: {geo_shape: {type: "polygon", coordinates: [[{lat: 38, lon: -123}, ...]]}}}
915
932
  ```
916
933
 
917
934
  Falling entirely within the query shape
918
935
 
919
936
  ```ruby
920
- City.search "san", where: {bounds: {geo_shape: {type: "circle", relation: "within", coordinates: [{lat: 38, lon: -123}], radius: "1km"}}}
937
+ Restaurant.search "salad", where: {bounds: {geo_shape: {type: "circle", relation: "within", coordinates: [{lat: 38, lon: -123}], radius: "1km"}}}
921
938
  ```
922
939
 
923
940
  Not touching the query shape
924
941
 
925
942
  ```ruby
926
- City.search "san", where: {bounds: {geo_shape: {type: "envelope", relation: "disjoint", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
943
+ Restaurant.search "burger", where: {bounds: {geo_shape: {type: "envelope", relation: "disjoint", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
927
944
  ```
928
945
 
929
946
  Containing the query shape (Elasticsearch 2.2+)
930
947
 
931
948
  ```ruby
932
- City.search "san", where: {bounds: {geo_shape: {type: "envelope", relation: "contains", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
949
+ Restaurant.search "fries", where: {bounds: {geo_shape: {type: "envelope", relation: "contains", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
933
950
  ```
934
951
 
935
952
  ## Inheritance
@@ -1137,7 +1154,7 @@ If you run into issues on Windows, check out [this post](https://www.rastating.c
1137
1154
 
1138
1155
  ### Searchable Fields
1139
1156
 
1140
- By default, all string fields are searchable. Speed up indexing and reduce index size by only making some fields searchable.
1157
+ By default, all string fields are searchable (can be used in `fields` option). Speed up indexing and reduce index size by only making some fields searchable. This disables the `_all` field unless it’s listed.
1141
1158
 
1142
1159
  ```ruby
1143
1160
  class Product < ActiveRecord::Base
@@ -1182,12 +1199,45 @@ And use:
1182
1199
  Searchkick.reindex_status(index_name)
1183
1200
  ```
1184
1201
 
1202
+ You can use [ActiveJob::TrafficControl](https://github.com/nickelser/activejob-traffic_control) to control concurrency. Install the gem and create an initializer with:
1203
+
1204
+ ```ruby
1205
+ require "active_job/traffic_control"
1206
+
1207
+ Searchkick.redis = Redis.new
1208
+ ActiveJob::TrafficControl.client = Searchkick.redis
1209
+
1210
+ class Searchkick::BulkReindexJob
1211
+ include ActiveJob::TrafficControl::Concurrency
1212
+
1213
+ concurrency 3, drop: false
1214
+ end
1215
+ ```
1216
+
1217
+ This will allow only 3 jobs to run at once.
1218
+
1219
+ ### Refresh Interval
1220
+
1221
+ You can specify a longer refresh interval while reindexing to increase performance.
1222
+
1223
+ ```ruby
1224
+ Product.reindex(async: true, refresh_interval: "30s")
1225
+ ```
1226
+
1227
+ **Note:** This only makes a noticable difference with parallel reindexing.
1228
+
1229
+ When promoting, have it restored to the value in your mapping (defaults to `1s`).
1230
+
1231
+ ```ruby
1232
+ Product.search_index.promote(index_name, update_refresh_interval: true)
1233
+ ```
1234
+
1185
1235
  ### Queuing
1186
1236
 
1187
- Push ids of records needing reindexed to a queue and reindex in bulk for better performance. First, set up Redis in an initializer.
1237
+ Push ids of records needing reindexed to a queue and reindex in bulk for better performance. First, set up Redis in an initializer. We recommend using [connection_pool](https://github.com/mperham/connection_pool).
1188
1238
 
1189
1239
  ```ruby
1190
- Searchkick.redis = Redis.new
1240
+ Searchkick.redis = ConnectionPool.new { Redis.new }
1191
1241
  ```
1192
1242
 
1193
1243
  And ask your models to queue updates.
@@ -1291,12 +1341,13 @@ class ReindexConversionsJob < ActiveJob::Base
1291
1341
 
1292
1342
  # split into groups
1293
1343
  recently_converted_ids.in_groups_of(1000, false) do |ids|
1294
- # fetch conversions and group by record
1295
- conversions_by_record = {}
1344
+ # fetch conversions
1296
1345
  conversions =
1297
1346
  Searchjoy::Search.where(convertable_id: ids, convertable_type: class_name)
1298
1347
  .group(:convertable_id, :query).uniq.count(:user_id)
1299
1348
 
1349
+ # group conversions by record
1350
+ conversions_by_record = {}
1300
1351
  conversions.each do |(id, query), count|
1301
1352
  (conversions_by_record[id] ||= {})[query] = count
1302
1353
  end
@@ -1379,6 +1430,14 @@ products =
1379
1430
  end
1380
1431
  ```
1381
1432
 
1433
+ ### Elasticsearch Gem
1434
+
1435
+ Searchkick is built on top of the [elasticsearch](https://github.com/elastic/elasticsearch-ruby) gem. To access the client directly, use:
1436
+
1437
+ ```ruby
1438
+ Searchkick.client
1439
+ ```
1440
+
1382
1441
  ## Multi Search
1383
1442
 
1384
1443
  To batch search requests for performance, use:
@@ -1415,6 +1474,20 @@ To query nested data, use dot notation.
1415
1474
  User.search "san", fields: ["address.city"], where: {"address.zip_code" => 12345}
1416
1475
  ```
1417
1476
 
1477
+ ## Search Concepts
1478
+
1479
+ ### Precision and Recall
1480
+
1481
+ [Precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall) are two key concepts in search (also known as *information retrieval*). To help illustrate, let’s walk through an example.
1482
+
1483
+ You have a store with 16 types of apples. A user searches for `apples` gets 10 results. 8 of the results are for apples, and 2 are for apple juice.
1484
+
1485
+ **Precision** is the fraction of documents in the results that are relevant. There are 10 results and 8 are relevant, so precision is 80%.
1486
+
1487
+ **Recall** is the fraction of relevant documents in the results out of all relevant documents. There are 16 apples and only 8 in the results, so recall is 50%.
1488
+
1489
+ There’s typically a trade-off between the two. As you tweak your search to increase precision (not return irrelevant documents), there’s are greater chance a relevant document also isn’t returned, which decreases recall. The opposite also applies. As you try to increase recall (return a higher number of relevent documents), there’s a greater chance you also return an irrelevant document, decreasing precision.
1490
+
1418
1491
  ## Reference
1419
1492
 
1420
1493
  Reindex one record
@@ -1575,15 +1648,12 @@ Product.search("carrots", request_params: {search_type: "dfs_query_then_fetch"})
1575
1648
 
1576
1649
  Reindex conditionally
1577
1650
 
1578
- **Note:** With ActiveRecord, use this feature with caution - [transaction rollbacks can cause data inconsistencies](https://github.com/elasticsearch/elasticsearch-rails/blob/master/elasticsearch-model/README.md#custom-callbacks)
1579
-
1580
1651
  ```ruby
1581
1652
  class Product < ActiveRecord::Base
1582
1653
  searchkick callbacks: false
1583
1654
 
1584
1655
  # add the callbacks manually
1585
- after_save :reindex, if: -> (model) { model.name_changed? } # use your own condition
1586
- after_destroy :reindex
1656
+ after_commit :reindex, if: -> (model) { model.previous_changes.key?("name") } # use your own condition
1587
1657
  end
1588
1658
  ```
1589
1659
 
@@ -1702,13 +1772,8 @@ Thanks to Karel Minarik for [Elasticsearch Ruby](https://github.com/elasticsearc
1702
1772
 
1703
1773
  ## Roadmap
1704
1774
 
1705
- - More features for large data sets
1706
- - Improve section on testing
1707
- - Semantic search features
1708
- - Search multiple fields for different terms
1709
- - Search across models
1710
- - Search nested objects
1711
- - Much finer customization
1775
+ - Reindex API
1776
+ - Incorporate human eval
1712
1777
 
1713
1778
  ## Contributing
1714
1779
 
@@ -66,26 +66,31 @@ module Searchkick
66
66
  alias_method :swap, :promote
67
67
 
68
68
  # record based
69
+ # use helpers for notifications
69
70
 
70
71
  def store(record)
71
- bulk_index([record])
72
+ bulk_index_helper([record])
72
73
  end
73
74
 
74
75
  def remove(record)
75
- bulk_delete([record])
76
+ bulk_delete_helper([record])
77
+ end
78
+
79
+ def update_record(record, method_name)
80
+ bulk_update_helper([record], method_name)
76
81
  end
77
82
 
78
83
  def bulk_delete(records)
79
- Searchkick.indexer.queue(records.reject { |r| r.id.blank? }.map { |r| {delete: record_data(r)} })
84
+ bulk_delete_helper(records)
80
85
  end
81
86
 
82
87
  def bulk_index(records)
83
- Searchkick.indexer.queue(records.map { |r| {index: record_data(r).merge(data: search_data(r))} })
88
+ bulk_index_helper(records)
84
89
  end
85
90
  alias_method :import, :bulk_index
86
91
 
87
92
  def bulk_update(records, method_name)
88
- Searchkick.indexer.queue(records.map { |r| {update: record_data(r).merge(data: {doc: search_data(r, method_name)})} })
93
+ bulk_update_helper(records, method_name)
89
94
  end
90
95
 
91
96
  def record_data(r)
@@ -253,7 +258,7 @@ module Searchkick
253
258
 
254
259
  if batch
255
260
  import_or_update scope.to_a, method_name, async
256
- redis.srem(batches_key, batch_id) if batch_id && redis
261
+ Searchkick.with_redis { |r| r.srem(batches_key, batch_id) } if batch_id
257
262
  elsif full && async
258
263
  full_reindex_async(scope)
259
264
  elsif scope.respond_to?(:find_in_batches)
@@ -278,7 +283,7 @@ module Searchkick
278
283
  end
279
284
 
280
285
  def batches_left
281
- redis.scard(batches_key) if redis
286
+ Searchkick.with_redis { |r| r.scard(batches_key) }
282
287
  end
283
288
 
284
289
  # other
@@ -454,7 +459,7 @@ module Searchkick
454
459
  index_name: name,
455
460
  batch_id: batch_id
456
461
  }.merge(options))
457
- redis.sadd(batches_key, batch_id) if redis
462
+ Searchkick.with_redis { |r| r.sadd(batches_key, batch_id) }
458
463
  end
459
464
 
460
465
  def batch_size
@@ -475,8 +480,16 @@ module Searchkick
475
480
  end
476
481
  end
477
482
 
478
- def redis
479
- Searchkick.redis
483
+ def bulk_index_helper(records)
484
+ Searchkick.indexer.queue(records.map { |r| {index: record_data(r).merge(data: search_data(r))} })
485
+ end
486
+
487
+ def bulk_delete_helper(records)
488
+ Searchkick.indexer.queue(records.reject { |r| r.id.blank? }.map { |r| {delete: record_data(r)} })
489
+ end
490
+
491
+ def bulk_update_helper(records, method_name)
492
+ Searchkick.indexer.queue(records.map { |r| {update: record_data(r).merge(data: {doc: search_data(r, method_name)})} })
480
493
  end
481
494
 
482
495
  def batches_key
@@ -45,12 +45,27 @@ module Searchkick
45
45
  end
46
46
  end
47
47
 
48
- def import(records)
48
+ def update_record(record, method_name)
49
+ event = {
50
+ name: "#{record.searchkick_klass.name} Update",
51
+ id: search_id(record)
52
+ }
53
+ if Searchkick.callbacks_value == :bulk
54
+ super
55
+ else
56
+ ActiveSupport::Notifications.instrument("request.searchkick", event) do
57
+ super
58
+ end
59
+ end
60
+ end
61
+
62
+ def bulk_index(records)
49
63
  if records.any?
50
64
  event = {
51
65
  name: "#{records.first.searchkick_klass.name} Import",
52
66
  count: records.size
53
67
  }
68
+ event[:id] = search_id(records.first) if records.size == 1
54
69
  if Searchkick.callbacks_value == :bulk
55
70
  super
56
71
  else
@@ -60,6 +75,7 @@ module Searchkick
60
75
  end
61
76
  end
62
77
  end
78
+ alias_method :import, :bulk_index
63
79
 
64
80
  def bulk_update(records, *args)
65
81
  if records.any?
@@ -67,6 +83,7 @@ module Searchkick
67
83
  name: "#{records.first.searchkick_klass.name} Update",
68
84
  count: records.size
69
85
  }
86
+ event[:id] = search_id(records.first) if records.size == 1
70
87
  if Searchkick.callbacks_value == :bulk
71
88
  super
72
89
  else
@@ -83,6 +100,7 @@ module Searchkick
83
100
  name: "#{records.first.searchkick_klass.name} Delete",
84
101
  count: records.size
85
102
  }
103
+ event[:id] = search_id(records.first) if records.size == 1
86
104
  if Searchkick.callbacks_value == :bulk
87
105
  super
88
106
  else
@@ -120,7 +120,7 @@ module Searchkick
120
120
  end
121
121
  else
122
122
  if method_name
123
- self.class.searchkick_index.bulk_update([self], method_name)
123
+ self.class.searchkick_index.update_record(self, method_name)
124
124
  else
125
125
  self.class.searchkick_index.reindex_record(self)
126
126
  end
@@ -14,7 +14,7 @@ module Searchkick
14
14
 
15
15
  def initialize(klass, term = "*", **options)
16
16
  unknown_keywords = options.keys - [:aggs, :body, :body_options, :boost,
17
- :boost_by, :boost_by_distance, :boost_where, :conversions, :debug, :emoji, :execute, :explain,
17
+ :boost_by, :boost_by_distance, :boost_where, :conversions, :debug, :emoji, :exclude, :execute, :explain,
18
18
  :fields, :highlight, :includes, :index_name, :indices_boost, :limit, :load,
19
19
  :match, :misspellings, :offset, :operator, :order, :padding, :page, :per_page, :profile,
20
20
  :request_params, :routing, :select, :similar, :smart_aggs, :suggest, :track, :type, :where]
@@ -264,6 +264,7 @@ module Searchkick
264
264
  end
265
265
 
266
266
  fields.each do |field|
267
+ queries_to_add = []
267
268
  qs = []
268
269
 
269
270
  factor = boost_fields[field] || 1
@@ -290,7 +291,7 @@ module Searchkick
290
291
  ]
291
292
  elsif field.end_with?(".exact")
292
293
  f = field.split(".")[0..-2].join(".")
293
- queries << {match: {f => shared_options.merge(analyzer: "keyword")}}
294
+ queries_to_add << {match: {f => shared_options.merge(analyzer: "keyword")}}
294
295
  else
295
296
  analyzer = field =~ /\.word_(start|middle|end)\z/ ? "searchkick_word_search" : "searchkick_autocomplete_search"
296
297
  qs << shared_options.merge(analyzer: analyzer)
@@ -300,21 +301,43 @@ module Searchkick
300
301
  qs.concat qs.map { |q| q.except(:cutoff_frequency).merge(fuzziness: edit_distance, prefix_length: prefix_length, max_expansions: max_expansions, boost: factor).merge(transpositions) }
301
302
  end
302
303
 
304
+ q2 = qs.map { |q| {match_type => {field => q}} }
305
+
303
306
  # boost exact matches more
304
307
  if field =~ /\.word_(start|middle|end)\z/ && searchkick_options[:word] != false
305
- queries << {
308
+ queries_to_add << {
306
309
  bool: {
307
310
  must: {
308
311
  bool: {
309
- should: qs.map { |q| {match_type => {field => q}} }
312
+ should: q2
310
313
  }
311
314
  },
312
315
  should: {match_type => {field.sub(/\.word_(start|middle|end)\z/, ".analyzed") => qs.first}}
313
316
  }
314
317
  }
315
318
  else
316
- queries.concat(qs.map { |q| {match_type => {field => q}} })
319
+ queries_to_add.concat(q2)
320
+ end
321
+
322
+ if options[:exclude]
323
+ must_not =
324
+ options[:exclude].map do |phrase|
325
+ {
326
+ match_phrase: {
327
+ field => phrase
328
+ }
329
+ }
330
+ end
331
+
332
+ queries_to_add = [{
333
+ bool: {
334
+ should: queries_to_add,
335
+ must_not: must_not
336
+ }
337
+ }]
317
338
  end
339
+
340
+ queries.concat(queries_to_add)
318
341
  end
319
342
 
320
343
  payload = {
@@ -5,36 +5,32 @@ module Searchkick
5
5
  def initialize(name)
6
6
  @name = name
7
7
 
8
- raise Searchkick::Error, "Searchkick.redis not set" unless redis
8
+ raise Searchkick::Error, "Searchkick.redis not set" unless Searchkick.redis
9
9
  end
10
10
 
11
11
  def push(record_id)
12
- redis.lpush(redis_key, record_id)
12
+ Searchkick.with_redis { |r| r.lpush(redis_key, record_id) }
13
13
  end
14
14
 
15
15
  # TODO use reliable queuing
16
16
  def reserve(limit: 1000)
17
17
  record_ids = Set.new
18
- while record_ids.size < limit && record_id = redis.rpop(redis_key)
18
+ while record_ids.size < limit && record_id = Searchkick.with_redis { |r| r.rpop(redis_key) }
19
19
  record_ids << record_id
20
20
  end
21
21
  record_ids.to_a
22
22
  end
23
23
 
24
24
  def clear
25
- redis.del(redis_key)
25
+ Searchkick.with_redis { |r| r.del(redis_key) }
26
26
  end
27
27
 
28
28
  def length
29
- redis.llen(redis_key)
29
+ Searchkick.with_redis { |r| r.llen(redis_key) }
30
30
  end
31
31
 
32
32
  private
33
33
 
34
- def redis
35
- Searchkick.redis
36
- end
37
-
38
34
  def redis_key
39
35
  "searchkick:reindex_queue:#{name}"
40
36
  end
@@ -1,3 +1,3 @@
1
1
  module Searchkick
2
- VERSION = "2.1.0"
2
+ VERSION = "2.1.1"
3
3
  end
data/lib/searchkick.rb CHANGED
@@ -145,6 +145,18 @@ module Searchkick
145
145
  end
146
146
  end
147
147
 
148
+ def self.with_redis
149
+ if redis
150
+ if redis.respond_to?(:with)
151
+ redis.with do |r|
152
+ yield r
153
+ end
154
+ else
155
+ yield redis
156
+ end
157
+ end
158
+ end
159
+
148
160
  # private
149
161
  def self.load_records(records, ids)
150
162
  records =
data/test/match_test.rb CHANGED
@@ -159,6 +159,25 @@ class MatchTest < Minitest::Test
159
159
  assert_search "almondmilks", ["Almond Milk"]
160
160
  end
161
161
 
162
+ # butter
163
+
164
+ def test_butter
165
+ store_names ["Butter Tub", "Peanut Butter Tub"]
166
+ assert_search "butter", ["Butter Tub"], exclude: ["peanut butter"]
167
+ end
168
+
169
+ def test_butter_word_start
170
+ store_names ["Butter Tub", "Peanut Butter Tub"]
171
+ assert_search "butter", ["Butter Tub"], exclude: ["peanut butter"], match: :word_start
172
+ end
173
+
174
+ def test_butter_exact
175
+ store_names ["Butter Tub", "Peanut Butter Tub"]
176
+ assert_search "butter", [], exclude: ["peanut butter"], match: :exact
177
+ end
178
+
179
+ # other
180
+
162
181
  def test_all
163
182
  store_names ["Product A", "Product B"]
164
183
  assert_search "*", ["Product A", "Product B"]
data/test/order_test.rb CHANGED
@@ -29,12 +29,12 @@ class OrderTest < Minitest::Test
29
29
 
30
30
  def test_order_ignore_unmapped
31
31
  skip unless elasticsearch_below50?
32
- assert_order "product", [], order: {not_mapped: {ignore_unmapped: true}}
32
+ assert_order "product", [], order: {not_mapped: {ignore_unmapped: true}}, conversions: false
33
33
  end
34
34
 
35
35
  def test_order_unmapped_type
36
36
  skip if elasticsearch_below50?
37
- assert_order "product", [], order: {not_mapped: {unmapped_type: "long"}}
37
+ assert_order "product", [], order: {not_mapped: {unmapped_type: "long"}}, conversions: false
38
38
  end
39
39
 
40
40
  def test_order_array
data/test/reindex_test.rb CHANGED
@@ -48,7 +48,5 @@ class ReindexTest < Minitest::Test
48
48
  Product.search_index.promote(index.name, update_refresh_interval: true)
49
49
  assert_equal "1s", index.refresh_interval
50
50
  assert_equal "1s", Product.search_index.refresh_interval
51
- ensure
52
- Product.reindex
53
51
  end
54
52
  end
data/test/test_helper.rb CHANGED
@@ -14,7 +14,13 @@ File.delete("elasticsearch.log") if File.exist?("elasticsearch.log")
14
14
  Searchkick.client.transport.logger = Logger.new("elasticsearch.log")
15
15
  Searchkick.search_timeout = 5
16
16
 
17
- Searchkick.redis = Redis.new if defined?(Redis)
17
+ if defined?(Redis)
18
+ if defined?(ConnectionPool)
19
+ Searchkick.redis = ConnectionPool.new { Redis.new }
20
+ else
21
+ Searchkick.redis = Redis.new
22
+ end
23
+ end
18
24
 
19
25
  puts "Running against Elasticsearch #{Searchkick.server_version}"
20
26
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: searchkick
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.1.0
4
+ version: 2.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrew Kane
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2017-01-16 00:00:00.000000000 Z
11
+ date: 2017-01-18 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activemodel
@@ -189,7 +189,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
189
189
  version: '0'
190
190
  requirements: []
191
191
  rubyforge_project:
192
- rubygems_version: 2.5.1
192
+ rubygems_version: 2.6.8
193
193
  signing_key:
194
194
  specification_version: 4
195
195
  summary: Searchkick learns what your users are looking for. As more people search,