searchkick 1.0.3 → 1.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +10 -0
- data/README.md +24 -16
- data/lib/searchkick/index.rb +47 -19
- data/lib/searchkick/logging.rb +3 -3
- data/lib/searchkick/query.rb +136 -111
- data/lib/searchkick/results.rb +13 -1
- data/lib/searchkick/version.rb +1 -1
- data/test/ci/before_install.sh +2 -0
- data/test/highlight_test.rb +3 -1
- data/test/match_test.rb +7 -0
- data/test/misspellings_test.rb +10 -0
- data/test/synonyms_test.rb +5 -0
- data/test/test_helper.rb +3 -1
- metadata +4 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 35f9202e63447e7c182bd2a34737e7a519888e91
|
4
|
+
data.tar.gz: 19a7a8d24dd5ff687687e235c64971cb979f42f2
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 762a4b1e6ce4a31e2a3d75ac235954222f26c3d5361d0bb8fd1504aeaf3f294c0f569a15208d1e2e38428715bdaa1bdc879356b2040bff5bb313f763e3091ccc
|
7
|
+
data.tar.gz: 1f7c6ea8338b6fe7479a22a55d523a753903fd703f142522806b83926be3cf3b20d10b1b2c39eb507657af97f48e84372d1c6cb102e10fe6a05ea40031eefe16
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,13 @@
|
|
1
|
+
## 1.1.0
|
2
|
+
|
3
|
+
- Added `below` option to misspellings to improve performance
|
4
|
+
- Fixed synonyms for `word_*` partial matches
|
5
|
+
- Added `searchable` option
|
6
|
+
- Added `similarity` option
|
7
|
+
- Added `match` option
|
8
|
+
- Added `word` option
|
9
|
+
- Added highlighted fields to `load: false`
|
10
|
+
|
1
11
|
## 1.0.3
|
2
12
|
|
3
13
|
- Added support for Elasticsearch 2.1
|
data/README.md
CHANGED
@@ -127,7 +127,7 @@ Searches return a `Searchkick::Results` object. This responds like an array to m
|
|
127
127
|
results = Product.search("milk")
|
128
128
|
results.size
|
129
129
|
results.any?
|
130
|
-
results.each { ... }
|
130
|
+
results.each { |result| ... }
|
131
131
|
```
|
132
132
|
|
133
133
|
Get total results
|
@@ -227,7 +227,7 @@ end
|
|
227
227
|
And to search (after you reindex):
|
228
228
|
|
229
229
|
```ruby
|
230
|
-
Product.search "back", fields: [
|
230
|
+
Product.search "back", fields: [:name], match: :word_start
|
231
231
|
```
|
232
232
|
|
233
233
|
Available options are:
|
@@ -242,16 +242,10 @@ Available options are:
|
|
242
242
|
:text_end
|
243
243
|
```
|
244
244
|
|
245
|
-
To boost fields, use:
|
246
|
-
|
247
|
-
```ruby
|
248
|
-
fields: [{"name^2" => :word_start}] # better interface on the way
|
249
|
-
```
|
250
|
-
|
251
245
|
### Exact Matches
|
252
246
|
|
253
247
|
```ruby
|
254
|
-
User.search
|
248
|
+
User.search params[:q], fields: [{email: :exact}, :name]
|
255
249
|
```
|
256
250
|
|
257
251
|
### Language
|
@@ -309,7 +303,15 @@ You can change this with:
|
|
309
303
|
Product.search "zucini", misspellings: {edit_distance: 2} # zucchini
|
310
304
|
```
|
311
305
|
|
312
|
-
|
306
|
+
To improve performance for correctly spelled queries (which should be a majority for most applications), Searchkick can first perform a search without misspellings, and if there are few results, perform another with them. [master]
|
307
|
+
|
308
|
+
```ruby
|
309
|
+
Product.search "zuchini", misspellings: {below: 5}
|
310
|
+
```
|
311
|
+
|
312
|
+
If there are fewer than 5 results, a 2nd search is performed for misspellings.
|
313
|
+
|
314
|
+
Turn off misspellings with:
|
313
315
|
|
314
316
|
```ruby
|
315
317
|
Product.search "zuchini", misspellings: false # no zucchini
|
@@ -498,14 +500,14 @@ First, specify which fields use this feature. This is necessary since autocompl
|
|
498
500
|
|
499
501
|
```ruby
|
500
502
|
class City < ActiveRecord::Base
|
501
|
-
searchkick
|
503
|
+
searchkick match: :word_start
|
502
504
|
end
|
503
505
|
```
|
504
506
|
|
505
507
|
Reindex and search with:
|
506
508
|
|
507
509
|
```ruby
|
508
|
-
City.search "san fr", fields: [
|
510
|
+
City.search "san fr", fields: [:name]
|
509
511
|
```
|
510
512
|
|
511
513
|
Typically, you want to use a JavaScript library like [typeahead.js](http://twitter.github.io/typeahead.js/) or [jQuery UI](http://jqueryui.com/autocomplete/).
|
@@ -517,11 +519,9 @@ First, add a route and controller action.
|
|
517
519
|
```ruby
|
518
520
|
# app/controllers/cities_controller.rb
|
519
521
|
class CitiesController < ApplicationController
|
520
|
-
|
521
522
|
def autocomplete
|
522
|
-
render json: City.search(params[:query], fields: [
|
523
|
+
render json: City.search(params[:query], fields: [:name], limit: 10).map(&:name)
|
523
524
|
end
|
524
|
-
|
525
525
|
end
|
526
526
|
```
|
527
527
|
|
@@ -546,7 +546,7 @@ Then add the search box and JavaScript code to a view.
|
|
546
546
|
|
547
547
|
```ruby
|
548
548
|
class Product < ActiveRecord::Base
|
549
|
-
searchkick suggest: [
|
549
|
+
searchkick suggest: [:name] # fields to generate suggestions
|
550
550
|
end
|
551
551
|
```
|
552
552
|
|
@@ -1147,6 +1147,14 @@ class Product < ActiveRecord::Base
|
|
1147
1147
|
end
|
1148
1148
|
```
|
1149
1149
|
|
1150
|
+
Use [Okapi BM25](https://www.elastic.co/guide/en/elasticsearch/guide/current/pluggable-similarites.html) for ranking [master]
|
1151
|
+
|
1152
|
+
```ruby
|
1153
|
+
class Product < ActiveRecord::Base
|
1154
|
+
searchkick similarity: "BM25"
|
1155
|
+
end
|
1156
|
+
```
|
1157
|
+
|
1150
1158
|
Change import batch size
|
1151
1159
|
|
1152
1160
|
```ruby
|
data/lib/searchkick/index.rb
CHANGED
@@ -353,6 +353,10 @@ module Searchkick
|
|
353
353
|
settings.merge!(number_of_shards: 1, number_of_replicas: 0)
|
354
354
|
end
|
355
355
|
|
356
|
+
if options[:similarity]
|
357
|
+
settings[:similarity] = {default: {type: options[:similarity]}}
|
358
|
+
end
|
359
|
+
|
356
360
|
settings.deep_merge!(options[:settings] || {})
|
357
361
|
|
358
362
|
# synonyms
|
@@ -376,6 +380,10 @@ module Searchkick
|
|
376
380
|
# - Use directional synonyms where appropriate. You want to make sure that you're not injecting terms that are too general.
|
377
381
|
settings[:analysis][:analyzer][:default_index][:filter].insert(4, "searchkick_synonym")
|
378
382
|
settings[:analysis][:analyzer][:default_index][:filter] << "searchkick_synonym"
|
383
|
+
|
384
|
+
%w(word_start word_middle word_end).each do |type|
|
385
|
+
settings[:analysis][:analyzer]["searchkick_#{type}_index".to_sym][:filter].insert(2, "searchkick_synonym")
|
386
|
+
end
|
379
387
|
end
|
380
388
|
|
381
389
|
if options[:wordnet]
|
@@ -387,6 +395,10 @@ module Searchkick
|
|
387
395
|
|
388
396
|
settings[:analysis][:analyzer][:default_index][:filter].insert(4, "searchkick_wordnet")
|
389
397
|
settings[:analysis][:analyzer][:default_index][:filter] << "searchkick_wordnet"
|
398
|
+
|
399
|
+
%w(word_start word_middle word_end).each do |type|
|
400
|
+
settings[:analysis][:analyzer]["searchkick_#{type}_index".to_sym][:filter].insert(2, "searchkick_wordnet")
|
401
|
+
end
|
390
402
|
end
|
391
403
|
|
392
404
|
if options[:special_characters] == false
|
@@ -409,29 +421,34 @@ module Searchkick
|
|
409
421
|
end
|
410
422
|
|
411
423
|
mapping_options = Hash[
|
412
|
-
[:autocomplete, :suggest, :text_start, :text_middle, :text_end, :word_start, :word_middle, :word_end, :highlight]
|
424
|
+
[:autocomplete, :suggest, :word, :text_start, :text_middle, :text_end, :word_start, :word_middle, :word_end, :highlight, :searchable]
|
413
425
|
.map { |type| [type, (options[type] || []).map(&:to_s)] }
|
414
426
|
]
|
415
427
|
|
428
|
+
word = options[:word] != false && (!options[:match] || options[:match] == :word)
|
429
|
+
|
416
430
|
mapping_options.values.flatten.uniq.each do |field|
|
417
431
|
field_mapping = {
|
418
432
|
type: "multi_field",
|
419
433
|
fields: {
|
420
|
-
field => {type: "string", index: "not_analyzed"}
|
421
|
-
"analyzed" => {type: "string", index: "analyzed"}
|
422
|
-
# term_vector: "with_positions_offsets" for fast / correct highlighting
|
423
|
-
# http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-highlighting.html#_fast_vector_highlighter
|
434
|
+
field => {type: "string", index: "not_analyzed"}
|
424
435
|
}
|
425
436
|
}
|
426
437
|
|
427
|
-
mapping_options.
|
428
|
-
if
|
429
|
-
field_mapping[:fields][
|
438
|
+
if !options[:searchable] || mapping_options[:searchable].include?(field)
|
439
|
+
if word
|
440
|
+
field_mapping[:fields]["analyzed"] = {type: "string", index: "analyzed"}
|
441
|
+
|
442
|
+
if mapping_options[:highlight].include?(field)
|
443
|
+
field_mapping[:fields]["analyzed"][:term_vector] = "with_positions_offsets"
|
444
|
+
end
|
430
445
|
end
|
431
|
-
end
|
432
446
|
|
433
|
-
|
434
|
-
|
447
|
+
mapping_options.except(:highlight, :searchable).each do |type, fields|
|
448
|
+
if options[:match] == type || fields.include?(field)
|
449
|
+
field_mapping[:fields][type] = {type: "string", index: "analyzed", analyzer: "searchkick_#{type}_index"}
|
450
|
+
end
|
451
|
+
end
|
435
452
|
end
|
436
453
|
|
437
454
|
mapping[field] = field_mapping
|
@@ -455,6 +472,24 @@ module Searchkick
|
|
455
472
|
routing = {required: true, path: options[:routing].to_s}
|
456
473
|
end
|
457
474
|
|
475
|
+
dynamic_fields = {
|
476
|
+
# analyzed field must be the default field for include_in_all
|
477
|
+
# http://www.elasticsearch.org/guide/reference/mapping/multi-field-type/
|
478
|
+
# however, we can include the not_analyzed field in _all
|
479
|
+
# and the _all index analyzer will take care of it
|
480
|
+
"{name}" => {type: "string", index: "not_analyzed", include_in_all: !options[:searchable]}
|
481
|
+
}
|
482
|
+
|
483
|
+
unless options[:searchable]
|
484
|
+
if options[:match] && options[:match] != :word
|
485
|
+
dynamic_fields[options[:match]] = {type: "string", index: "analyzed", analyzer: "searchkick_#{options[:match]}_index"}
|
486
|
+
end
|
487
|
+
|
488
|
+
if word
|
489
|
+
dynamic_fields["analyzed"] = {type: "string", index: "analyzed"}
|
490
|
+
end
|
491
|
+
end
|
492
|
+
|
458
493
|
mappings = {
|
459
494
|
_default_: {
|
460
495
|
properties: mapping,
|
@@ -468,14 +503,7 @@ module Searchkick
|
|
468
503
|
mapping: {
|
469
504
|
# http://www.elasticsearch.org/guide/reference/mapping/multi-field-type/
|
470
505
|
type: "multi_field",
|
471
|
-
fields:
|
472
|
-
# analyzed field must be the default field for include_in_all
|
473
|
-
# http://www.elasticsearch.org/guide/reference/mapping/multi-field-type/
|
474
|
-
# however, we can include the not_analyzed field in _all
|
475
|
-
# and the _all index analyzer will take care of it
|
476
|
-
"{name}" => {type: "string", index: "not_analyzed"},
|
477
|
-
"analyzed" => {type: "string", index: "analyzed"}
|
478
|
-
}
|
506
|
+
fields: dynamic_fields
|
479
507
|
}
|
480
508
|
}
|
481
509
|
}
|
data/lib/searchkick/logging.rb
CHANGED
@@ -2,16 +2,16 @@
|
|
2
2
|
|
3
3
|
module Searchkick
|
4
4
|
class Query
|
5
|
-
def
|
5
|
+
def execute_search_with_instrumentation
|
6
6
|
event = {
|
7
7
|
name: "#{searchkick_klass.name} Search",
|
8
8
|
query: params
|
9
9
|
}
|
10
10
|
ActiveSupport::Notifications.instrument("search.searchkick", event) do
|
11
|
-
|
11
|
+
execute_search_without_instrumentation
|
12
12
|
end
|
13
13
|
end
|
14
|
-
alias_method_chain :
|
14
|
+
alias_method_chain :execute_search, :instrumentation
|
15
15
|
end
|
16
16
|
|
17
17
|
class Index
|
data/lib/searchkick/query.rb
CHANGED
@@ -22,15 +22,114 @@ module Searchkick
|
|
22
22
|
@klass = klass
|
23
23
|
@term = term
|
24
24
|
@options = options
|
25
|
+
@match_suffix = options[:match] || searchkick_options[:match] || "analyzed"
|
25
26
|
|
27
|
+
prepare
|
28
|
+
end
|
29
|
+
|
30
|
+
def searchkick_index
|
31
|
+
klass.searchkick_index
|
32
|
+
end
|
33
|
+
|
34
|
+
def searchkick_options
|
35
|
+
klass.searchkick_options
|
36
|
+
end
|
37
|
+
|
38
|
+
def searchkick_klass
|
39
|
+
klass.searchkick_klass
|
40
|
+
end
|
41
|
+
|
42
|
+
def params
|
43
|
+
params = {
|
44
|
+
index: options[:index_name] || searchkick_index.name,
|
45
|
+
body: body
|
46
|
+
}
|
47
|
+
params.merge!(type: @type) if @type
|
48
|
+
params.merge!(routing: @routing) if @routing
|
49
|
+
params
|
50
|
+
end
|
51
|
+
|
52
|
+
def execute
|
53
|
+
@execute ||= begin
|
54
|
+
begin
|
55
|
+
response = execute_search
|
56
|
+
if @misspellings_below && response["hits"]["total"] < @misspellings_below
|
57
|
+
prepare
|
58
|
+
response = execute_search
|
59
|
+
end
|
60
|
+
rescue => e # TODO rescue type
|
61
|
+
status_code = e.message[1..3].to_i
|
62
|
+
if status_code == 404
|
63
|
+
raise MissingIndexError, "Index missing - run #{searchkick_klass.name}.reindex"
|
64
|
+
elsif status_code == 500 && (
|
65
|
+
e.message.include?("IllegalArgumentException[minimumSimilarity >= 1]") ||
|
66
|
+
e.message.include?("No query registered for [multi_match]") ||
|
67
|
+
e.message.include?("[match] query does not support [cutoff_frequency]]") ||
|
68
|
+
e.message.include?("No query registered for [function_score]]")
|
69
|
+
)
|
70
|
+
|
71
|
+
raise UnsupportedVersionError, "This version of Searchkick requires Elasticsearch 1.0 or greater"
|
72
|
+
elsif status_code == 400
|
73
|
+
if e.message.include?("[multi_match] analyzer [searchkick_search] not found")
|
74
|
+
raise InvalidQueryError, "Bad mapping - run #{searchkick_klass.name}.reindex"
|
75
|
+
else
|
76
|
+
raise InvalidQueryError, e.message
|
77
|
+
end
|
78
|
+
else
|
79
|
+
raise e
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
# apply facet limit in client due to
|
84
|
+
# https://github.com/elasticsearch/elasticsearch/issues/1305
|
85
|
+
@facet_limits.each do |field, limit|
|
86
|
+
field = field.to_s
|
87
|
+
facet = response["facets"][field]
|
88
|
+
response["facets"][field]["terms"] = facet["terms"].first(limit)
|
89
|
+
response["facets"][field]["other"] = facet["total"] - facet["terms"].sum { |term| term["count"] }
|
90
|
+
end
|
91
|
+
|
92
|
+
opts = {
|
93
|
+
page: @page,
|
94
|
+
per_page: @per_page,
|
95
|
+
padding: @padding,
|
96
|
+
load: @load,
|
97
|
+
includes: options[:include] || options[:includes],
|
98
|
+
json: !options[:json].nil?,
|
99
|
+
match_suffix: @match_suffix,
|
100
|
+
highlighted_fields: @highlighted_fields || []
|
101
|
+
}
|
102
|
+
Searchkick::Results.new(searchkick_klass, response, opts)
|
103
|
+
end
|
104
|
+
end
|
105
|
+
|
106
|
+
def to_curl
|
107
|
+
query = params
|
108
|
+
type = query[:type]
|
109
|
+
index = query[:index].is_a?(Array) ? query[:index].join(",") : query[:index]
|
110
|
+
|
111
|
+
# no easy way to tell which host the client will use
|
112
|
+
host = Searchkick.client.transport.hosts.first
|
113
|
+
credentials = (host[:user] || host[:password]) ? "#{host[:user]}:#{host[:password]}@" : nil
|
114
|
+
"curl #{host[:protocol]}://#{credentials}#{host[:host]}:#{host[:port]}/#{CGI.escape(index)}#{type ? "/#{type.map { |t| CGI.escape(t) }.join(',')}" : ''}/_search?pretty -d '#{query[:body].to_json}'"
|
115
|
+
end
|
116
|
+
|
117
|
+
private
|
118
|
+
|
119
|
+
def execute_search
|
120
|
+
Searchkick.client.search(params)
|
121
|
+
end
|
122
|
+
|
123
|
+
def prepare
|
26
124
|
boost_fields = {}
|
125
|
+
fields = options[:fields] || searchkick_options[:searchable]
|
27
126
|
fields =
|
28
|
-
if
|
127
|
+
if fields
|
29
128
|
if options[:autocomplete]
|
30
|
-
|
129
|
+
fields.map { |f| "#{f}.autocomplete" }
|
31
130
|
else
|
32
|
-
|
33
|
-
k, v = value.is_a?(Hash) ? value.to_a.first : [value, :word]
|
131
|
+
fields.map do |value|
|
132
|
+
k, v = value.is_a?(Hash) ? value.to_a.first : [value, options[:match] || searchkick_options[:match] || :word]
|
34
133
|
k2, boost = k.to_s.split("^", 2)
|
35
134
|
field = "#{k2}.#{v == :word ? 'analyzed' : v}"
|
36
135
|
boost_fields[field] = boost.to_f if boost
|
@@ -93,6 +192,36 @@ module Searchkick
|
|
93
192
|
}
|
94
193
|
else
|
95
194
|
queries = []
|
195
|
+
|
196
|
+
misspellings =
|
197
|
+
if options.key?(:misspellings)
|
198
|
+
options[:misspellings]
|
199
|
+
elsif options.key?(:mispellings)
|
200
|
+
options[:mispellings] # why not?
|
201
|
+
else
|
202
|
+
true
|
203
|
+
end
|
204
|
+
|
205
|
+
if misspellings.is_a?(Hash) && misspellings[:below] && !@misspellings_below
|
206
|
+
@misspellings_below = misspellings[:below].to_i
|
207
|
+
misspellings = false
|
208
|
+
end
|
209
|
+
|
210
|
+
if misspellings != false
|
211
|
+
edit_distance = (misspellings.is_a?(Hash) && (misspellings[:edit_distance] || misspellings[:distance])) || 1
|
212
|
+
transpositions =
|
213
|
+
if misspellings.is_a?(Hash) && misspellings.key?(:transpositions)
|
214
|
+
{fuzzy_transpositions: misspellings[:transpositions]}
|
215
|
+
elsif below14?
|
216
|
+
{}
|
217
|
+
else
|
218
|
+
{fuzzy_transpositions: true}
|
219
|
+
end
|
220
|
+
prefix_length = (misspellings.is_a?(Hash) && misspellings[:prefix_length]) || 0
|
221
|
+
default_max_expansions = @misspellings_below ? 20 : 3
|
222
|
+
max_expansions = (misspellings.is_a?(Hash) && misspellings[:max_expansions]) || default_max_expansions
|
223
|
+
end
|
224
|
+
|
96
225
|
fields.each do |field|
|
97
226
|
qs = []
|
98
227
|
|
@@ -103,29 +232,6 @@ module Searchkick
|
|
103
232
|
boost: 10 * factor
|
104
233
|
}
|
105
234
|
|
106
|
-
misspellings =
|
107
|
-
if options.key?(:misspellings)
|
108
|
-
options[:misspellings]
|
109
|
-
elsif options.key?(:mispellings)
|
110
|
-
options[:mispellings] # why not?
|
111
|
-
else
|
112
|
-
true
|
113
|
-
end
|
114
|
-
|
115
|
-
if misspellings != false
|
116
|
-
edit_distance = (misspellings.is_a?(Hash) && (misspellings[:edit_distance] || misspellings[:distance])) || 1
|
117
|
-
transpositions =
|
118
|
-
if misspellings.is_a?(Hash) && misspellings.key?(:transpositions)
|
119
|
-
{fuzzy_transpositions: misspellings[:transpositions]}
|
120
|
-
elsif below14?
|
121
|
-
{}
|
122
|
-
else
|
123
|
-
{fuzzy_transpositions: true}
|
124
|
-
end
|
125
|
-
prefix_length = (misspellings.is_a?(Hash) && misspellings[:prefix_length]) || 0
|
126
|
-
max_expansions = (misspellings.is_a?(Hash) && misspellings[:max_expansions]) || 3
|
127
|
-
end
|
128
|
-
|
129
235
|
if field == "_all" || field.end_with?(".analyzed")
|
130
236
|
shared_options[:cutoff_frequency] = 0.001 unless operator == "and" || misspellings == false
|
131
237
|
qs.concat [
|
@@ -436,10 +542,12 @@ module Searchkick
|
|
436
542
|
payload[:highlight][:fields] = {}
|
437
543
|
|
438
544
|
highlight_fields.each do |name, opts|
|
439
|
-
payload[:highlight][:fields]["#{name}
|
545
|
+
payload[:highlight][:fields]["#{name}.#{@match_suffix}"] = opts || {}
|
440
546
|
end
|
441
547
|
end
|
442
548
|
end
|
549
|
+
|
550
|
+
@highlighted_fields = payload[:highlight][:fields].keys
|
443
551
|
end
|
444
552
|
|
445
553
|
# An empty array will cause only the _id and _type for each hit to be returned
|
@@ -466,89 +574,6 @@ module Searchkick
|
|
466
574
|
@load = load
|
467
575
|
end
|
468
576
|
|
469
|
-
def searchkick_index
|
470
|
-
klass.searchkick_index
|
471
|
-
end
|
472
|
-
|
473
|
-
def searchkick_options
|
474
|
-
klass.searchkick_options
|
475
|
-
end
|
476
|
-
|
477
|
-
def searchkick_klass
|
478
|
-
klass.searchkick_klass
|
479
|
-
end
|
480
|
-
|
481
|
-
def params
|
482
|
-
params = {
|
483
|
-
index: options[:index_name] || searchkick_index.name,
|
484
|
-
body: body
|
485
|
-
}
|
486
|
-
params.merge!(type: @type) if @type
|
487
|
-
params.merge!(routing: @routing) if @routing
|
488
|
-
params
|
489
|
-
end
|
490
|
-
|
491
|
-
def execute
|
492
|
-
@execute ||= begin
|
493
|
-
begin
|
494
|
-
response = Searchkick.client.search(params)
|
495
|
-
rescue => e # TODO rescue type
|
496
|
-
status_code = e.message[1..3].to_i
|
497
|
-
if status_code == 404
|
498
|
-
raise MissingIndexError, "Index missing - run #{searchkick_klass.name}.reindex"
|
499
|
-
elsif status_code == 500 && (
|
500
|
-
e.message.include?("IllegalArgumentException[minimumSimilarity >= 1]") ||
|
501
|
-
e.message.include?("No query registered for [multi_match]") ||
|
502
|
-
e.message.include?("[match] query does not support [cutoff_frequency]]") ||
|
503
|
-
e.message.include?("No query registered for [function_score]]")
|
504
|
-
)
|
505
|
-
|
506
|
-
raise UnsupportedVersionError, "This version of Searchkick requires Elasticsearch 1.0 or greater"
|
507
|
-
elsif status_code == 400
|
508
|
-
if e.message.include?("[multi_match] analyzer [searchkick_search] not found")
|
509
|
-
raise InvalidQueryError, "Bad mapping - run #{searchkick_klass.name}.reindex"
|
510
|
-
else
|
511
|
-
raise InvalidQueryError, e.message
|
512
|
-
end
|
513
|
-
else
|
514
|
-
raise e
|
515
|
-
end
|
516
|
-
end
|
517
|
-
|
518
|
-
# apply facet limit in client due to
|
519
|
-
# https://github.com/elasticsearch/elasticsearch/issues/1305
|
520
|
-
@facet_limits.each do |field, limit|
|
521
|
-
field = field.to_s
|
522
|
-
facet = response["facets"][field]
|
523
|
-
response["facets"][field]["terms"] = facet["terms"].first(limit)
|
524
|
-
response["facets"][field]["other"] = facet["total"] - facet["terms"].sum { |term| term["count"] }
|
525
|
-
end
|
526
|
-
|
527
|
-
opts = {
|
528
|
-
page: @page,
|
529
|
-
per_page: @per_page,
|
530
|
-
padding: @padding,
|
531
|
-
load: @load,
|
532
|
-
includes: options[:include] || options[:includes],
|
533
|
-
json: !options[:json].nil?
|
534
|
-
}
|
535
|
-
Searchkick::Results.new(searchkick_klass, response, opts)
|
536
|
-
end
|
537
|
-
end
|
538
|
-
|
539
|
-
def to_curl
|
540
|
-
query = params
|
541
|
-
type = query[:type]
|
542
|
-
index = query[:index].is_a?(Array) ? query[:index].join(",") : query[:index]
|
543
|
-
|
544
|
-
# no easy way to tell which host the client will use
|
545
|
-
host = Searchkick.client.transport.hosts.first
|
546
|
-
credentials = (host[:user] || host[:password]) ? "#{host[:user]}:#{host[:password]}@" : nil
|
547
|
-
"curl #{host[:protocol]}://#{credentials}#{host[:host]}:#{host[:port]}/#{CGI.escape(index)}#{type ? "/#{type.map { |t| CGI.escape(t) }.join(',')}" : ''}/_search?pretty -d '#{query[:body].to_json}'"
|
548
|
-
end
|
549
|
-
|
550
|
-
private
|
551
|
-
|
552
577
|
def where_filters(where)
|
553
578
|
filters = []
|
554
579
|
(where || {}).each do |field, value|
|
data/lib/searchkick/results.rb
CHANGED
@@ -42,6 +42,14 @@ module Searchkick
|
|
42
42
|
else
|
43
43
|
hit.except("fields").merge(hit["fields"])
|
44
44
|
end
|
45
|
+
|
46
|
+
if hit["highlight"]
|
47
|
+
highlight = Hash[hit["highlight"].map { |k, v| [base_field(k), v.first] }]
|
48
|
+
options[:highlighted_fields].map{ |k| base_field(k) }.each do |k|
|
49
|
+
result["highlighted_#{k}"] ||= (highlight[k] || result[k])
|
50
|
+
end
|
51
|
+
end
|
52
|
+
|
45
53
|
result["id"] ||= result["_id"] # needed for legacy reasons
|
46
54
|
Hashie::Mash.new(result)
|
47
55
|
end
|
@@ -65,7 +73,7 @@ module Searchkick
|
|
65
73
|
each_with_hit.map do |model, hit|
|
66
74
|
details = {}
|
67
75
|
if hit["highlight"]
|
68
|
-
details[:highlight] = Hash[hit["highlight"].map { |k, v| [(options[:json] ? k : k.sub(
|
76
|
+
details[:highlight] = Hash[hit["highlight"].map { |k, v| [(options[:json] ? k : k.sub(/\.#{@options[:match_suffix]}\z/, "")).to_sym, v.first] }]
|
69
77
|
end
|
70
78
|
[model, details]
|
71
79
|
end
|
@@ -189,5 +197,9 @@ module Searchkick
|
|
189
197
|
raise "Not sure how to load records"
|
190
198
|
end
|
191
199
|
end
|
200
|
+
|
201
|
+
def base_field(k)
|
202
|
+
k.sub(/\.(analyzed|word_start|word_middle|word_end|text_start|text_middle|text_end|exact)\z/, "")
|
203
|
+
end
|
192
204
|
end
|
193
205
|
end
|
data/lib/searchkick/version.rb
CHANGED
data/test/ci/before_install.sh
CHANGED
data/test/highlight_test.rb
CHANGED
@@ -27,7 +27,8 @@ class HighlightTest < Minitest::Test
|
|
27
27
|
|
28
28
|
def test_field_options
|
29
29
|
store_names ["Two Door Cinema Club are a Northern Irish indie rock band"]
|
30
|
-
|
30
|
+
fragment_size = ENV["MATCH"] == "word_start" ? 26 : 20
|
31
|
+
assert_equal "Two Door <em>Cinema</em> Club are", Product.search("cinema", fields: [:name], highlight: {fields: {name: {fragment_size: fragment_size}}}).with_details.first[1][:highlight][:name]
|
31
32
|
end
|
32
33
|
|
33
34
|
def test_multiple_words
|
@@ -36,6 +37,7 @@ class HighlightTest < Minitest::Test
|
|
36
37
|
end
|
37
38
|
|
38
39
|
def test_json
|
40
|
+
skip if ENV["MATCH"] == "word_start"
|
39
41
|
store_names ["Two Door Cinema Club"]
|
40
42
|
json = {
|
41
43
|
query: {
|
data/test/match_test.rb
CHANGED
@@ -198,6 +198,13 @@ class MatchTest < Minitest::Test
|
|
198
198
|
assert_search "almond", []
|
199
199
|
end
|
200
200
|
|
201
|
+
def test_unsearchable_where
|
202
|
+
store [
|
203
|
+
{name: "Unsearchable", description: "Almond"}
|
204
|
+
]
|
205
|
+
assert_search "*", ["Unsearchable"], where: {description: "Almond"}
|
206
|
+
end
|
207
|
+
|
201
208
|
def test_emoji
|
202
209
|
skip unless defined?(EmojiParser)
|
203
210
|
store_names ["Banana"]
|
data/test/misspellings_test.rb
CHANGED
@@ -33,4 +33,14 @@ class MisspellingsTest < Minitest::Test
|
|
33
33
|
]
|
34
34
|
assert_search "red blue", ["red", "blue", "cyan", "magenta"], operator: "or", fields: ["color"], misspellings: false
|
35
35
|
end
|
36
|
+
|
37
|
+
def test_misspellings_below_unmet
|
38
|
+
store_names ["abc", "abd", "aee"]
|
39
|
+
assert_search "abc", ["abc", "abd"], misspellings: {below: 2}
|
40
|
+
end
|
41
|
+
|
42
|
+
def test_misspellings_below_met
|
43
|
+
store_names ["abc", "abd", "aee"]
|
44
|
+
assert_search "abc", ["abc"], misspellings: {below: 1}
|
45
|
+
end
|
36
46
|
end
|
data/test/synonyms_test.rb
CHANGED
@@ -41,6 +41,11 @@ class SynonymsTest < Minitest::Test
|
|
41
41
|
assert_search "scallions", ["Green Onions"]
|
42
42
|
end
|
43
43
|
|
44
|
+
def test_word_start
|
45
|
+
store_names ["Clorox Bleach", "Kroger Bleach"]
|
46
|
+
assert_search "clorox", ["Clorox Bleach", "Kroger Bleach"], fields: [{name: :word_start}]
|
47
|
+
end
|
48
|
+
|
44
49
|
def test_wordnet
|
45
50
|
skip unless ENV["TEST_WORDNET"]
|
46
51
|
store_names ["Creature", "Beast", "Dragon"], Animal
|
data/test/test_helper.rb
CHANGED
@@ -216,7 +216,9 @@ class Product
|
|
216
216
|
word_middle: [:name],
|
217
217
|
word_end: [:name],
|
218
218
|
highlight: [:name],
|
219
|
-
unsearchable: [:description]
|
219
|
+
# unsearchable: [:description],
|
220
|
+
searchable: [:name, :color],
|
221
|
+
match: ENV["MATCH"] ? ENV["MATCH"].to_sym : nil
|
220
222
|
|
221
223
|
attr_accessor :conversions, :user_ids, :aisle
|
222
224
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: searchkick
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-
|
11
|
+
date: 2015-12-08 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activemodel
|
@@ -174,7 +174,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
174
174
|
version: '0'
|
175
175
|
requirements: []
|
176
176
|
rubyforge_project:
|
177
|
-
rubygems_version: 2.4.5
|
177
|
+
rubygems_version: 2.4.5
|
178
178
|
signing_key:
|
179
179
|
specification_version: 4
|
180
180
|
summary: Searchkick learns what your users are looking for. As more people search,
|
@@ -216,3 +216,4 @@ test_files:
|
|
216
216
|
- test/synonyms_test.rb
|
217
217
|
- test/test_helper.rb
|
218
218
|
- test/where_test.rb
|
219
|
+
has_rdoc:
|