search_flip 3.2.0 → 3.5.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/workflows/test.yml +3 -2
- data/.rubocop.yml +2 -1
- data/CHANGELOG.md +19 -0
- data/README.md +39 -35
- data/lib/search_flip.rb +9 -3
- data/lib/search_flip/bulk.rb +1 -1
- data/lib/search_flip/config.rb +7 -1
- data/lib/search_flip/connection.rb +28 -15
- data/lib/search_flip/criteria.rb +29 -8
- data/lib/search_flip/http_client.rb +30 -5
- data/lib/search_flip/index.rb +10 -4
- data/lib/search_flip/json.rb +4 -9
- data/lib/search_flip/to_json.rb +1 -29
- data/lib/search_flip/version.rb +1 -1
- data/search_flip.gemspec +1 -0
- data/spec/search_flip/criteria_spec.rb +29 -6
- data/spec/search_flip/http_client_spec.rb +7 -3
- data/spec/search_flip/index_spec.rb +1 -1
- data/spec/search_flip/json_spec.rb +45 -0
- metadata +18 -4
- data/spec/search_flip/to_json_spec.rb +0 -28
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b534f2309a3433637b7e8c661d098bcbb04aa6c047655ad3c4ab944cbb860bf7
|
4
|
+
data.tar.gz: ec32a8f3ef761bb9d1f4dd99e1e31448b5eceb88a40643c63cada7bc4bedeaa7
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ad28e9a932ad4a572bb401593e1d0c9551d2aeb7e128ffad6d1c237f8e78f7a5385a2e94e4a113ae47bcc4576c07bfc74ab24bcbc2d65e8e7f2751951b5d09da
|
7
|
+
data.tar.gz: 73c88dc39c4709dc716f3a45428771c08ce83ec51b278bcbd1285663e5cdafc2c66181b61ad78ee741feb2963bde607916ddaab8ffe8ec35124e695d37f287b7
|
data/.github/workflows/test.yml
CHANGED
@@ -11,11 +11,11 @@ jobs:
|
|
11
11
|
- elasticsearch:5.4
|
12
12
|
- docker.elastic.co/elasticsearch/elasticsearch:6.7.0
|
13
13
|
- docker.elastic.co/elasticsearch/elasticsearch:7.0.0
|
14
|
-
- docker.elastic.co/elasticsearch/elasticsearch:7.
|
14
|
+
- docker.elastic.co/elasticsearch/elasticsearch:7.11.2
|
15
15
|
ruby:
|
16
|
-
- 2.5
|
17
16
|
- 2.6
|
18
17
|
- 2.7
|
18
|
+
- 3.0
|
19
19
|
services:
|
20
20
|
elasticsearch:
|
21
21
|
image: ${{ matrix.elasticsearch }}
|
@@ -28,6 +28,7 @@ jobs:
|
|
28
28
|
- uses: actions/setup-ruby@v1
|
29
29
|
with:
|
30
30
|
ruby-version: ${{ matrix.ruby }}
|
31
|
+
- run: gem install bundler
|
31
32
|
- run: bundle
|
32
33
|
- run: sleep 10
|
33
34
|
- run: bundle exec rspec
|
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,6 +1,25 @@
|
|
1
1
|
|
2
2
|
# CHANGELOG
|
3
3
|
|
4
|
+
## v3.5.0
|
5
|
+
|
6
|
+
* Add `SearchFlip::Criteria#http_timeout` to allow specifying timeouts on
|
7
|
+
a query level
|
8
|
+
|
9
|
+
## v3.4.0
|
10
|
+
|
11
|
+
* Expose `Http#timeout` via `SearchFlip::HTTPClient`
|
12
|
+
|
13
|
+
## v3.3.0
|
14
|
+
|
15
|
+
* Update httprb
|
16
|
+
* Changed oj default options
|
17
|
+
* Allow to set oj json options
|
18
|
+
|
19
|
+
## v3.2.1
|
20
|
+
|
21
|
+
* Fix `refresh` having a empty body breaking in elasticsearch 7.11
|
22
|
+
|
4
23
|
## v3.2.0
|
5
24
|
|
6
25
|
* Fix `index_scope` not being passed in `each_record` without block
|
data/README.md
CHANGED
@@ -3,7 +3,7 @@
|
|
3
3
|
|
4
4
|
**Full-Featured Elasticsearch Ruby Client with a Chainable DSL**
|
5
5
|
|
6
|
-
[![Build
|
6
|
+
[![Build](https://github.com/mrkamel/search_flip/workflows/test/badge.svg)](https://github.com/mrkamel/search_flip/actions?query=workflow%3Atest+branch%3Amaster)
|
7
7
|
[![Gem Version](https://badge.fury.io/rb/search_flip.svg)](http://badge.fury.io/rb/search_flip)
|
8
8
|
|
9
9
|
Using SearchFlip it is dead-simple to create index classes that correspond to
|
@@ -51,8 +51,7 @@ CommentIndex.search("hello world").where(available: true).sort(id: "desc").aggre
|
|
51
51
|
|
52
52
|
```
|
53
53
|
|
54
|
-
Finally, SearchFlip comes with a minimal set of dependencies
|
55
|
-
and oj only).
|
54
|
+
Finally, SearchFlip comes with a minimal set of dependencies.
|
56
55
|
|
57
56
|
## Reference Docs
|
58
57
|
|
@@ -699,6 +698,14 @@ Specify a timeout to limit query processing time:
|
|
699
698
|
CommentIndex.timeout("3s").execute
|
700
699
|
```
|
701
700
|
|
701
|
+
* `http_timeout`
|
702
|
+
|
703
|
+
Specify a http timeout for the request which will be send to Elasticsearch:
|
704
|
+
|
705
|
+
```ruby
|
706
|
+
CommentIndex.http_timeout(3).execute
|
707
|
+
```
|
708
|
+
|
702
709
|
* `terminate_after`
|
703
710
|
|
704
711
|
Activate early query termination to stop query processing after the specified
|
@@ -757,7 +764,7 @@ end
|
|
757
764
|
This allows to use different clusters per index e.g. when migrating indices to
|
758
765
|
new versions of Elasticsearch.
|
759
766
|
|
760
|
-
You can specify basic auth, additional headers, etc via:
|
767
|
+
You can specify basic auth, additional headers, request timeouts, etc via:
|
761
768
|
|
762
769
|
```ruby
|
763
770
|
http_client = SearchFlip::HTTPClient.new
|
@@ -774,6 +781,9 @@ http_client = http_client.via("proxy.host", 8080)
|
|
774
781
|
# Custom headers
|
775
782
|
http_client = http_client.headers(key: "value")
|
776
783
|
|
784
|
+
# Timeouts
|
785
|
+
http_client = http_client.timeout(20)
|
786
|
+
|
777
787
|
SearchFlip::Connection.new(base_url: "...", http_client: http_client)
|
778
788
|
```
|
779
789
|
|
@@ -883,52 +893,46 @@ Thus, if your ORM supports `.find_each`, `#id` and `#where` you are already
|
|
883
893
|
good to go. Otherwise, simply add your custom implementation of those methods
|
884
894
|
that work with whatever ORM you use.
|
885
895
|
|
886
|
-
##
|
896
|
+
## JSON
|
887
897
|
|
888
|
-
|
889
|
-
|
890
|
-
|
891
|
-
However, `JSON.generate` in ruby by default outputs something like:
|
898
|
+
SearchFlip is using the [Oj gem](https://github.com/ohler55/oj) to generate
|
899
|
+
JSON. More concretely, SearchFlip is using:
|
892
900
|
|
893
901
|
```ruby
|
894
|
-
|
895
|
-
# => "{\"time\":\"2018-02-22 18:19:33 UTC\"}"
|
902
|
+
Oj.dump({ key: "value" }, mode: :custom, use_to_json: true, time_format: :xmlschema, bigdecimal_as_decimal: false)
|
896
903
|
```
|
897
904
|
|
898
|
-
|
899
|
-
|
900
|
-
However,
|
901
|
-
|
905
|
+
The `use_to_json` option is used for maximum compatibility, most importantly
|
906
|
+
when using rails `ActiveSupport::TimeWithZone` timestamps, which `oj` can not
|
907
|
+
serialize natively. However, `use_to_json` adds performance overhead. You can
|
908
|
+
change the json options via:
|
902
909
|
|
903
910
|
```ruby
|
904
|
-
|
905
|
-
|
906
|
-
|
907
|
-
|
908
|
-
|
911
|
+
SearchFlip::Config[:json_options] = {
|
912
|
+
mode: :custom,
|
913
|
+
use_to_json: false,
|
914
|
+
time_format: :xmlschema,
|
915
|
+
bigdecimal_as_decimal: false
|
916
|
+
}
|
909
917
|
```
|
910
918
|
|
911
|
-
|
912
|
-
JSON. More concretely, SearchFlip is using:
|
919
|
+
However, you then have to convert timestamps manually for indexation via e.g.:
|
913
920
|
|
914
921
|
```ruby
|
915
|
-
|
916
|
-
|
922
|
+
class MyIndex
|
923
|
+
# ...
|
917
924
|
|
918
|
-
|
925
|
+
def self.serialize(model)
|
926
|
+
{
|
927
|
+
# ...
|
919
928
|
|
920
|
-
|
921
|
-
|
922
|
-
|
929
|
+
created_at: model.created_at.to_time
|
930
|
+
}
|
931
|
+
end
|
932
|
+
end
|
923
933
|
```
|
924
934
|
|
925
|
-
|
926
|
-
`Date` and `DateTime` to get proper serialization. You can either add them on
|
927
|
-
your own, via other libraries or by simply using:
|
928
|
-
|
929
|
-
```ruby
|
930
|
-
require "search_flip/to_json"
|
931
|
-
```
|
935
|
+
Please check out the oj docs for more details.
|
932
936
|
|
933
937
|
## Feature Support
|
934
938
|
|
data/lib/search_flip.rb
CHANGED
@@ -3,6 +3,7 @@ require "forwardable"
|
|
3
3
|
require "http"
|
4
4
|
require "hashie"
|
5
5
|
require "thread"
|
6
|
+
require "json"
|
6
7
|
require "oj"
|
7
8
|
require "set"
|
8
9
|
|
@@ -32,10 +33,15 @@ require "search_flip/index"
|
|
32
33
|
require "search_flip/model"
|
33
34
|
|
34
35
|
module SearchFlip
|
35
|
-
class
|
36
|
-
class ConnectionError < StandardError; end
|
36
|
+
class Error < StandardError; end
|
37
37
|
|
38
|
-
class
|
38
|
+
class NotSupportedError < Error; end
|
39
|
+
|
40
|
+
class HttpError < Error; end
|
41
|
+
class ConnectionError < HttpError; end
|
42
|
+
class TimeoutError < HttpError; end
|
43
|
+
|
44
|
+
class ResponseError < Error
|
39
45
|
attr_reader :code, :body
|
40
46
|
|
41
47
|
def initialize(code:, body:)
|
data/lib/search_flip/bulk.rb
CHANGED
data/lib/search_flip/config.rb
CHANGED
@@ -5,6 +5,12 @@ module SearchFlip
|
|
5
5
|
bulk_limit: 1_000,
|
6
6
|
bulk_max_mb: 100,
|
7
7
|
auto_refresh: false,
|
8
|
-
instrumenter: NullInstrumenter.new
|
8
|
+
instrumenter: NullInstrumenter.new,
|
9
|
+
json_options: {
|
10
|
+
mode: :custom,
|
11
|
+
use_to_json: true,
|
12
|
+
time_format: :xmlschema,
|
13
|
+
bigdecimal_as_decimal: false
|
14
|
+
}
|
9
15
|
}
|
10
16
|
end
|
@@ -28,7 +28,11 @@ module SearchFlip
|
|
28
28
|
|
29
29
|
def version
|
30
30
|
@version_mutex.synchronize do
|
31
|
-
@version ||=
|
31
|
+
@version ||= begin
|
32
|
+
response = http_client.headers(accept: "application/json").get("#{base_url}/")
|
33
|
+
|
34
|
+
SearchFlip::JSON.parse(response.to_s)["version"]["number"]
|
35
|
+
end
|
32
36
|
end
|
33
37
|
end
|
34
38
|
|
@@ -40,7 +44,9 @@ module SearchFlip
|
|
40
44
|
# @return [Hash] The raw response
|
41
45
|
|
42
46
|
def cluster_health
|
43
|
-
http_client.headers(accept: "application/json").get("#{base_url}/_cluster/health")
|
47
|
+
response = http_client.headers(accept: "application/json").get("#{base_url}/_cluster/health")
|
48
|
+
|
49
|
+
SearchFlip::JSON.parse(response.to_s)
|
44
50
|
end
|
45
51
|
|
46
52
|
# Uses the Elasticsearch Multi Search API to execute multiple search requests
|
@@ -71,7 +77,7 @@ module SearchFlip
|
|
71
77
|
.headers(accept: "application/json", content_type: "application/x-ndjson")
|
72
78
|
.post("#{base_url}/_msearch", body: payload)
|
73
79
|
|
74
|
-
raw_response.
|
80
|
+
SearchFlip::JSON.parse(raw_response.to_s)["responses"].map.with_index do |response, index|
|
75
81
|
SearchFlip::Response.new(criterias[index], response)
|
76
82
|
end
|
77
83
|
end
|
@@ -90,10 +96,11 @@ module SearchFlip
|
|
90
96
|
# @return [Hash] The raw response
|
91
97
|
|
92
98
|
def update_aliases(payload)
|
93
|
-
http_client
|
99
|
+
response = http_client
|
94
100
|
.headers(accept: "application/json", content_type: "application/json")
|
95
101
|
.post("#{base_url}/_aliases", body: SearchFlip::JSON.generate(payload))
|
96
|
-
|
102
|
+
|
103
|
+
SearchFlip::JSON.parse(response.to_s)
|
97
104
|
end
|
98
105
|
|
99
106
|
# Sends an analyze request to Elasticsearch. Raises
|
@@ -105,10 +112,11 @@ module SearchFlip
|
|
105
112
|
# @return [Hash] The raw response
|
106
113
|
|
107
114
|
def analyze(request, params = {})
|
108
|
-
http_client
|
115
|
+
response = http_client
|
109
116
|
.headers(accept: "application/json")
|
110
117
|
.post("#{base_url}/_analyze", json: request, params: params)
|
111
|
-
|
118
|
+
|
119
|
+
SearchFlip::JSON.parse(response.to_s)
|
112
120
|
end
|
113
121
|
|
114
122
|
# Fetches information about the specified index aliases. Raises
|
@@ -124,10 +132,11 @@ module SearchFlip
|
|
124
132
|
# @return [Hash] The raw response
|
125
133
|
|
126
134
|
def get_aliases(index_name: "*", alias_name: "*")
|
127
|
-
http_client
|
135
|
+
response = http_client
|
128
136
|
.headers(accept: "application/json", content_type: "application/json")
|
129
137
|
.get("#{base_url}/#{index_name}/_alias/#{alias_name}")
|
130
|
-
|
138
|
+
|
139
|
+
SearchFlip::JSON.parse(response.to_s)
|
131
140
|
end
|
132
141
|
|
133
142
|
# Returns whether or not the associated Elasticsearch alias already
|
@@ -159,10 +168,11 @@ module SearchFlip
|
|
159
168
|
# @return [Array] The raw response
|
160
169
|
|
161
170
|
def get_indices(name = "*", params: {})
|
162
|
-
http_client
|
171
|
+
response = http_client
|
163
172
|
.headers(accept: "application/json", content_type: "application/json")
|
164
173
|
.get("#{base_url}/_cat/indices/#{name}", params: params)
|
165
|
-
|
174
|
+
|
175
|
+
SearchFlip::JSON.parse(response.to_s)
|
166
176
|
end
|
167
177
|
|
168
178
|
alias_method :cat_indices, :get_indices
|
@@ -259,10 +269,11 @@ module SearchFlip
|
|
259
269
|
# @return [Hash] The index settings
|
260
270
|
|
261
271
|
def get_index_settings(index_name)
|
262
|
-
http_client
|
272
|
+
response = http_client
|
263
273
|
.headers(accept: "application/json")
|
264
274
|
.get("#{index_url(index_name)}/_settings")
|
265
|
-
|
275
|
+
|
276
|
+
SearchFlip::JSON.parse(response.to_s)
|
266
277
|
end
|
267
278
|
|
268
279
|
# Sends a refresh request to Elasticsearch. Raises
|
@@ -272,7 +283,7 @@ module SearchFlip
|
|
272
283
|
# @return [Boolean] Returns true or raises SearchFlip::ResponseError
|
273
284
|
|
274
285
|
def refresh(index_names = nil)
|
275
|
-
http_client.post("#{index_names ? index_url(Array(index_names).join(",")) : base_url}/_refresh"
|
286
|
+
http_client.post("#{index_names ? index_url(Array(index_names).join(",")) : base_url}/_refresh")
|
276
287
|
|
277
288
|
true
|
278
289
|
end
|
@@ -310,7 +321,9 @@ module SearchFlip
|
|
310
321
|
url = type_name ? type_url(index_name, type_name) : index_url(index_name)
|
311
322
|
params = type_name && version.to_f >= 6.7 ? { include_type_name: true } : {}
|
312
323
|
|
313
|
-
http_client.headers(accept: "application/json").get("#{url}/_mapping", params: params)
|
324
|
+
response = http_client.headers(accept: "application/json").get("#{url}/_mapping", params: params)
|
325
|
+
|
326
|
+
SearchFlip::JSON.parse(response.to_s)
|
314
327
|
end
|
315
328
|
|
316
329
|
# Deletes the specified index from Elasticsearch. Raises
|
data/lib/search_flip/criteria.rb
CHANGED
@@ -26,7 +26,8 @@ module SearchFlip
|
|
26
26
|
|
27
27
|
attr_accessor :target, :profile_value, :source_value, :suggest_values, :includes_values,
|
28
28
|
:eager_load_values, :preload_values, :failsafe_value, :scroll_args, :terminate_after_value,
|
29
|
-
:timeout_value, :preference_value, :search_type_value, :routing_value, :track_total_hits_value
|
29
|
+
:timeout_value, :preference_value, :search_type_value, :routing_value, :track_total_hits_value,
|
30
|
+
:http_timeout_value
|
30
31
|
|
31
32
|
# Creates a new criteria while merging the attributes (constraints,
|
32
33
|
# settings, etc) of the current criteria with the attributes of another one
|
@@ -47,7 +48,7 @@ module SearchFlip
|
|
47
48
|
[
|
48
49
|
:profile_value, :failsafe_value, :terminate_after_value, :timeout_value, :offset_value,
|
49
50
|
:limit_value, :scroll_args, :source_value, :preference_value, :search_type_value,
|
50
|
-
:routing_value, :track_total_hits_value, :explain_value
|
51
|
+
:routing_value, :track_total_hits_value, :explain_value, :http_timeout_value
|
51
52
|
].each do |name|
|
52
53
|
criteria.send(:"#{name}=", other.send(name)) unless other.send(name).nil?
|
53
54
|
end
|
@@ -148,6 +149,22 @@ module SearchFlip
|
|
148
149
|
end
|
149
150
|
end
|
150
151
|
|
152
|
+
# Specifies a http timeout, such that a SearchFlip::TimeoutError will be
|
153
|
+
# thrown when the request times out.
|
154
|
+
#
|
155
|
+
# @example
|
156
|
+
# ProductIndex.http_timeout(3).search("hello world")
|
157
|
+
#
|
158
|
+
# @param value [Fixnum] The timeout value
|
159
|
+
#
|
160
|
+
# @return [SearchFlip::Criteria] A newly created extended criteria
|
161
|
+
|
162
|
+
def http_timeout(value)
|
163
|
+
fresh.tap do |criteria|
|
164
|
+
criteria.http_timeout_value = value
|
165
|
+
end
|
166
|
+
end
|
167
|
+
|
151
168
|
# Specifies early query termination, such that the processing will be
|
152
169
|
# stopped after the specified number of results has been accumulated.
|
153
170
|
#
|
@@ -330,10 +347,13 @@ module SearchFlip
|
|
330
347
|
dupped_request.delete(:from)
|
331
348
|
dupped_request.delete(:size)
|
332
349
|
|
350
|
+
http_request = connection.http_client
|
351
|
+
http_request = http_request.timeout(http_timeout_value) if http_timeout_value
|
352
|
+
|
333
353
|
if connection.version.to_i >= 5
|
334
|
-
|
354
|
+
http_request.post("#{target.type_url}/_delete_by_query", params: request_params.merge(params), json: dupped_request)
|
335
355
|
else
|
336
|
-
|
356
|
+
http_request.delete("#{target.type_url}/_query", params: request_params.merge(params), json: dupped_request)
|
337
357
|
end
|
338
358
|
|
339
359
|
target.refresh if SearchFlip::Config[:auto_refresh]
|
@@ -501,8 +521,8 @@ module SearchFlip
|
|
501
521
|
end
|
502
522
|
|
503
523
|
# Executes the search request for the current criteria, ie sends the
|
504
|
-
# request to Elasticsearch and returns the response. Connection
|
505
|
-
# response errors will be rescued if you specify the criteria to be
|
524
|
+
# request to Elasticsearch and returns the response. Connection, timeout
|
525
|
+
# and response errors will be rescued if you specify the criteria to be
|
506
526
|
# #failsafe, such that an empty response is returned instead.
|
507
527
|
#
|
508
528
|
# @example
|
@@ -590,6 +610,7 @@ module SearchFlip
|
|
590
610
|
|
591
611
|
def execute!
|
592
612
|
http_request = connection.http_client.headers(accept: "application/json")
|
613
|
+
http_request = http_request.timeout(http_timeout_value) if http_timeout_value
|
593
614
|
|
594
615
|
http_response =
|
595
616
|
if scroll_args && scroll_args[:id]
|
@@ -608,8 +629,8 @@ module SearchFlip
|
|
608
629
|
http_request.post("#{target.type_url}/_search", params: request_params, json: request)
|
609
630
|
end
|
610
631
|
|
611
|
-
SearchFlip::Response.new(self, http_response.
|
612
|
-
rescue SearchFlip::ConnectionError, SearchFlip::ResponseError => e
|
632
|
+
SearchFlip::Response.new(self, SearchFlip::JSON.parse(http_response.to_s))
|
633
|
+
rescue SearchFlip::ConnectionError, SearchFlip::TimeoutError, SearchFlip::ResponseError => e
|
613
634
|
raise e unless failsafe_value
|
614
635
|
|
615
636
|
SearchFlip::Response.new(self, "took" => 0, "hits" => { "total" => 0, "hits" => [] })
|
@@ -1,7 +1,28 @@
|
|
1
1
|
module SearchFlip
|
2
|
-
# The SearchFlip::HTTPClient class wraps the http gem
|
3
|
-
#
|
4
|
-
# with
|
2
|
+
# The SearchFlip::HTTPClient class wraps the http gem and responsible for the
|
3
|
+
# http request/response handling, ie communicating with Elasticsearch. You
|
4
|
+
# only need to use it directly if you need authentication to communicate with
|
5
|
+
# Elasticsearch or if you want to set some custom http settings.
|
6
|
+
#
|
7
|
+
# @example
|
8
|
+
# http_client = SearchFlip::HTTPClient.new
|
9
|
+
#
|
10
|
+
# # Basic Auth
|
11
|
+
# http_client = http_client.basic_auth(user: "username", pass: "password")
|
12
|
+
#
|
13
|
+
# # Raw Auth Header
|
14
|
+
# http_client = http_client.auth("Bearer VGhlIEhUVFAgR2VtLCBST0NLUw")
|
15
|
+
#
|
16
|
+
# # Proxy Settings
|
17
|
+
# http_client = http_client.via("proxy.host", 8080)
|
18
|
+
#
|
19
|
+
# # Custom headers
|
20
|
+
# http_client = http_client.headers(key: "value")
|
21
|
+
#
|
22
|
+
# # Timeouts
|
23
|
+
# http_client = http_client.timeout(20)
|
24
|
+
#
|
25
|
+
# SearchFlip::Connection.new(base_url: "...", http_client: http_client)
|
5
26
|
|
6
27
|
class HTTPClient
|
7
28
|
attr_accessor :request, :plugins
|
@@ -14,11 +35,11 @@ module SearchFlip
|
|
14
35
|
class << self
|
15
36
|
extend Forwardable
|
16
37
|
|
17
|
-
def_delegators :new, :headers, :via, :basic_auth, :auth
|
38
|
+
def_delegators :new, :headers, :via, :basic_auth, :auth, :timeout
|
18
39
|
def_delegators :new, :get, :post, :put, :delete, :head
|
19
40
|
end
|
20
41
|
|
21
|
-
[:headers, :via, :basic_auth, :auth].each do |method|
|
42
|
+
[:headers, :via, :basic_auth, :auth, :timeout].each do |method|
|
22
43
|
define_method method do |*args|
|
23
44
|
dup.tap do |client|
|
24
45
|
client.request = request.send(method, *args)
|
@@ -45,6 +66,10 @@ module SearchFlip
|
|
45
66
|
response
|
46
67
|
rescue HTTP::ConnectionError => e
|
47
68
|
raise SearchFlip::ConnectionError, e.message
|
69
|
+
rescue HTTP::TimeoutError => e
|
70
|
+
raise SearchFlip::TimeoutError, e.message
|
71
|
+
rescue HTTP::Error => e
|
72
|
+
raise SearchFlip::HttpError, e.message
|
48
73
|
end
|
49
74
|
end
|
50
75
|
end
|
data/lib/search_flip/index.rb
CHANGED
@@ -254,7 +254,7 @@ module SearchFlip
|
|
254
254
|
:page, :per, :search, :highlight, :suggest, :custom, :find_in_batches, :find_results_in_batches,
|
255
255
|
:find_each, :find_each_result, :failsafe, :total_entries, :total_count, :timeout, :terminate_after,
|
256
256
|
:records, :results, :must, :must_not, :should, :preference, :search_type, :routing,
|
257
|
-
:track_total_hits, :explain
|
257
|
+
:track_total_hits, :explain, :http_timeout
|
258
258
|
|
259
259
|
# Override to specify the type name used within Elasticsearch. Recap,
|
260
260
|
# this gem uses an individual index for each index class, because
|
@@ -455,7 +455,9 @@ module SearchFlip
|
|
455
455
|
# @return [Hash] The specified document
|
456
456
|
|
457
457
|
def get(id, params = {})
|
458
|
-
connection.http_client.headers(accept: "application/json").get("#{type_url}/#{id}", params: params)
|
458
|
+
response = connection.http_client.headers(accept: "application/json").get("#{type_url}/#{id}", params: params)
|
459
|
+
|
460
|
+
SearchFlip::JSON.parse(response.to_s)
|
459
461
|
end
|
460
462
|
|
461
463
|
# Retrieves the documents specified by ids from elasticsearch.
|
@@ -471,7 +473,9 @@ module SearchFlip
|
|
471
473
|
# @return [Hash] The raw response
|
472
474
|
|
473
475
|
def mget(request, params = {})
|
474
|
-
connection.http_client.headers(accept: "application/json").post("#{type_url}/_mget", json: request, params: params)
|
476
|
+
response = connection.http_client.headers(accept: "application/json").post("#{type_url}/_mget", json: request, params: params)
|
477
|
+
|
478
|
+
SearchFlip::JSON.parse(response.to_s)
|
475
479
|
end
|
476
480
|
|
477
481
|
# Sends an analyze request to Elasticsearch. Raises
|
@@ -483,7 +487,9 @@ module SearchFlip
|
|
483
487
|
# @return [Hash] The raw response
|
484
488
|
|
485
489
|
def analyze(request, params = {})
|
486
|
-
connection.http_client.headers(accept: "application/json").post("#{index_url}/_analyze", json: request, params: params)
|
490
|
+
response = connection.http_client.headers(accept: "application/json").post("#{index_url}/_analyze", json: request, params: params)
|
491
|
+
|
492
|
+
SearchFlip::JSON.parse(response.to_s)
|
487
493
|
end
|
488
494
|
|
489
495
|
# Sends a index refresh request to Elasticsearch. Raises
|
data/lib/search_flip/json.rb
CHANGED
@@ -1,16 +1,11 @@
|
|
1
1
|
module SearchFlip
|
2
2
|
class JSON
|
3
|
-
|
4
|
-
|
5
|
-
use_to_json: true
|
6
|
-
}
|
7
|
-
|
8
|
-
def self.default_options
|
9
|
-
@default_options
|
3
|
+
def self.generate(obj)
|
4
|
+
Oj.dump(obj, SearchFlip::Config[:json_options])
|
10
5
|
end
|
11
6
|
|
12
|
-
def self.
|
13
|
-
|
7
|
+
def self.parse(json)
|
8
|
+
::JSON.parse(json)
|
14
9
|
end
|
15
10
|
end
|
16
11
|
end
|
data/lib/search_flip/to_json.rb
CHANGED
@@ -1,29 +1 @@
|
|
1
|
-
|
2
|
-
require "date"
|
3
|
-
require "json"
|
4
|
-
|
5
|
-
class Time
|
6
|
-
def to_json(*args)
|
7
|
-
iso8601(6).to_json
|
8
|
-
end
|
9
|
-
end
|
10
|
-
|
11
|
-
class Date
|
12
|
-
def to_json(*args)
|
13
|
-
iso8601.to_json
|
14
|
-
end
|
15
|
-
end
|
16
|
-
|
17
|
-
class DateTime
|
18
|
-
def to_json(*args)
|
19
|
-
iso8601(6).to_json
|
20
|
-
end
|
21
|
-
end
|
22
|
-
|
23
|
-
if defined?(ActiveSupport)
|
24
|
-
class ActiveSupport::TimeWithZone
|
25
|
-
def to_json(*args)
|
26
|
-
iso8601(6).to_json
|
27
|
-
end
|
28
|
-
end
|
29
|
-
end
|
1
|
+
warn "[DEPRECATION] Using search_flip/to_json is not neccessary anymore"
|
data/lib/search_flip/version.rb
CHANGED
data/search_flip.gemspec
CHANGED
@@ -97,7 +97,8 @@ RSpec.describe SearchFlip::Criteria do
|
|
97
97
|
methods = [
|
98
98
|
:profile_value, :failsafe_value, :terminate_after_value, :timeout_value,
|
99
99
|
:offset_value, :limit_value, :scroll_args, :source_value, :preference_value,
|
100
|
-
:search_type_value, :routing_value, :track_total_hits_value, :explain_value
|
100
|
+
:search_type_value, :routing_value, :track_total_hits_value, :explain_value,
|
101
|
+
:http_timeout_value
|
101
102
|
]
|
102
103
|
|
103
104
|
methods.each do |method|
|
@@ -191,6 +192,22 @@ RSpec.describe SearchFlip::Criteria do
|
|
191
192
|
end
|
192
193
|
end
|
193
194
|
|
195
|
+
describe "#http_timeout" do
|
196
|
+
it "sets the query timeout" do
|
197
|
+
http_client = double("client").as_null_object
|
198
|
+
allow(http_client).to receive(:timeout).and_return(http_client)
|
199
|
+
allow(http_client).to receive(:post).and_raise(SearchFlip::TimeoutError)
|
200
|
+
allow(ProductIndex.connection).to receive(:http_client).and_return(http_client)
|
201
|
+
|
202
|
+
expect { ProductIndex.http_timeout(1).execute }.to raise_error(SearchFlip::TimeoutError)
|
203
|
+
expect(http_client).to have_received(:timeout).with(1)
|
204
|
+
end
|
205
|
+
|
206
|
+
it "executes without errors" do
|
207
|
+
expect { ProductIndex.http_timeout(1).execute }.not_to raise_error
|
208
|
+
end
|
209
|
+
end
|
210
|
+
|
194
211
|
describe "#terminate_after" do
|
195
212
|
it "sets the terminate after value" do
|
196
213
|
query = ProductIndex.terminate_after(1)
|
@@ -1204,13 +1221,19 @@ RSpec.describe SearchFlip::Criteria do
|
|
1204
1221
|
end
|
1205
1222
|
|
1206
1223
|
describe "#failsafe" do
|
1207
|
-
|
1208
|
-
|
1224
|
+
[SearchFlip::ConnectionError, SearchFlip::TimeoutError, SearchFlip::ResponseError.new(code: "code", body: "body")].each do |error|
|
1225
|
+
it "prevents #{error}" do
|
1226
|
+
http_client = double("client").as_null_object
|
1227
|
+
allow(http_client).to receive(:post).and_raise(error)
|
1228
|
+
allow(ProductIndex.connection).to receive(:http_client).and_return(http_client)
|
1229
|
+
|
1230
|
+
expect { ProductIndex.all.execute }.to raise_error(error)
|
1209
1231
|
|
1210
|
-
|
1232
|
+
query = ProductIndex.failsafe(true)
|
1211
1233
|
|
1212
|
-
|
1213
|
-
|
1234
|
+
expect(query.records).to eq([])
|
1235
|
+
expect(query.total_entries).to eq(0)
|
1236
|
+
end
|
1214
1237
|
end
|
1215
1238
|
end
|
1216
1239
|
|
@@ -7,7 +7,7 @@ class HttpTestRequest
|
|
7
7
|
self.calls = []
|
8
8
|
end
|
9
9
|
|
10
|
-
[:via, :basic_auth, :auth].each do |method|
|
10
|
+
[:headers, :via, :basic_auth, :auth, :timeout].each do |method|
|
11
11
|
define_method method do |*args|
|
12
12
|
dup.tap do |request|
|
13
13
|
request.calls = calls + [[method, args]]
|
@@ -20,7 +20,7 @@ RSpec.describe SearchFlip::HTTPClient do
|
|
20
20
|
describe "delegation" do
|
21
21
|
subject { SearchFlip::HTTPClient }
|
22
22
|
|
23
|
-
[:headers, :via, :basic_auth, :auth].each do |method|
|
23
|
+
[:headers, :via, :basic_auth, :auth, :timeout].each do |method|
|
24
24
|
it { should delegate(method).to(:new) }
|
25
25
|
end
|
26
26
|
|
@@ -56,8 +56,12 @@ RSpec.describe SearchFlip::HTTPClient do
|
|
56
56
|
end
|
57
57
|
end
|
58
58
|
|
59
|
-
[:via, :basic_auth, :auth].each do |method|
|
59
|
+
[:headers, :via, :basic_auth, :auth, :timeout].each do |method|
|
60
60
|
describe "##{method}" do
|
61
|
+
it "is understood by HTTP" do
|
62
|
+
expect(HTTP.respond_to?(method)).to eq(true)
|
63
|
+
end
|
64
|
+
|
61
65
|
it "creates a dupped instance" do
|
62
66
|
client = SearchFlip::HTTPClient.new
|
63
67
|
client.request = HttpTestRequest.new
|
@@ -14,7 +14,7 @@ RSpec.describe SearchFlip::Index do
|
|
14
14
|
:total_entries, :total_count, :terminate_after, :timeout, :records, :results,
|
15
15
|
:must, :must_not, :should, :find_each_result,
|
16
16
|
:find_results_in_batches, :preference, :search_type, :routing,
|
17
|
-
:track_total_hits, :explain
|
17
|
+
:track_total_hits, :explain, :http_timeout
|
18
18
|
]
|
19
19
|
|
20
20
|
methods.each do |method|
|
@@ -0,0 +1,45 @@
|
|
1
|
+
require File.expand_path("../spec_helper", __dir__)
|
2
|
+
|
3
|
+
RSpec.describe SearchFlip::JSON do
|
4
|
+
describe ".generate" do
|
5
|
+
it "encodes timestamps correctly" do
|
6
|
+
Timecop.freeze "2020-06-01 12:00:00 UTC" do
|
7
|
+
expect(described_class.generate(timestamp: Time.now.utc)).to eq('{"timestamp":"2020-06-01T12:00:00.000Z"}')
|
8
|
+
end
|
9
|
+
end
|
10
|
+
|
11
|
+
it "encodes bigdecimals as string" do
|
12
|
+
expect(described_class.generate(value: BigDecimal(1))).to eq('{"value":"1.0"}')
|
13
|
+
end
|
14
|
+
|
15
|
+
it "delegates to Oj" do
|
16
|
+
allow(Oj).to receive(:dump)
|
17
|
+
|
18
|
+
payload = { key: "value" }
|
19
|
+
|
20
|
+
described_class.generate(payload)
|
21
|
+
|
22
|
+
expect(Oj).to have_received(:dump).with(payload, mode: :custom, use_to_json: true, time_format: :xmlschema, bigdecimal_as_decimal: false)
|
23
|
+
end
|
24
|
+
|
25
|
+
it "generates json" do
|
26
|
+
expect(described_class.generate(key: "value")).to eq('{"key":"value"}')
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
describe ".parse" do
|
31
|
+
it "returns the parsed json payload" do
|
32
|
+
expect(described_class.parse('{"key":"value"}')).to eq("key" => "value")
|
33
|
+
end
|
34
|
+
|
35
|
+
it "delegates to JSON" do
|
36
|
+
allow(JSON).to receive(:parse)
|
37
|
+
|
38
|
+
payload = '{"key":"value"}'
|
39
|
+
|
40
|
+
described_class.parse(payload)
|
41
|
+
|
42
|
+
expect(JSON).to have_received(:parse).with(payload)
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: search_flip
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 3.
|
4
|
+
version: 3.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Benjamin Vetter
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-07-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -178,6 +178,20 @@ dependencies:
|
|
178
178
|
- - ">="
|
179
179
|
- !ruby/object:Gem::Version
|
180
180
|
version: '0'
|
181
|
+
- !ruby/object:Gem::Dependency
|
182
|
+
name: json
|
183
|
+
requirement: !ruby/object:Gem::Requirement
|
184
|
+
requirements:
|
185
|
+
- - ">="
|
186
|
+
- !ruby/object:Gem::Version
|
187
|
+
version: '0'
|
188
|
+
type: :runtime
|
189
|
+
prerelease: false
|
190
|
+
version_requirements: !ruby/object:Gem::Requirement
|
191
|
+
requirements:
|
192
|
+
- - ">="
|
193
|
+
- !ruby/object:Gem::Version
|
194
|
+
version: '0'
|
181
195
|
- !ruby/object:Gem::Dependency
|
182
196
|
name: oj
|
183
197
|
requirement: !ruby/object:Gem::Requirement
|
@@ -259,11 +273,11 @@ files:
|
|
259
273
|
- spec/search_flip/criteria_spec.rb
|
260
274
|
- spec/search_flip/http_client_spec.rb
|
261
275
|
- spec/search_flip/index_spec.rb
|
276
|
+
- spec/search_flip/json_spec.rb
|
262
277
|
- spec/search_flip/model_spec.rb
|
263
278
|
- spec/search_flip/null_instrumenter_spec.rb
|
264
279
|
- spec/search_flip/response_spec.rb
|
265
280
|
- spec/search_flip/result_spec.rb
|
266
|
-
- spec/search_flip/to_json_spec.rb
|
267
281
|
- spec/spec_helper.rb
|
268
282
|
homepage: https://github.com/mrkamel/search_flip
|
269
283
|
licenses:
|
@@ -300,9 +314,9 @@ test_files:
|
|
300
314
|
- spec/search_flip/criteria_spec.rb
|
301
315
|
- spec/search_flip/http_client_spec.rb
|
302
316
|
- spec/search_flip/index_spec.rb
|
317
|
+
- spec/search_flip/json_spec.rb
|
303
318
|
- spec/search_flip/model_spec.rb
|
304
319
|
- spec/search_flip/null_instrumenter_spec.rb
|
305
320
|
- spec/search_flip/response_spec.rb
|
306
321
|
- spec/search_flip/result_spec.rb
|
307
|
-
- spec/search_flip/to_json_spec.rb
|
308
322
|
- spec/spec_helper.rb
|
@@ -1,28 +0,0 @@
|
|
1
|
-
require File.expand_path("../spec_helper", __dir__)
|
2
|
-
require "search_flip/to_json"
|
3
|
-
|
4
|
-
RSpec.describe "to_json" do
|
5
|
-
it "uses the correct format for Time" do
|
6
|
-
Timecop.freeze Time.parse("2018-01-01 12:00:00 UTC") do
|
7
|
-
expect(Time.now.utc.to_json).to eq("\"2018-01-01T12:00:00.000000Z\"")
|
8
|
-
end
|
9
|
-
end
|
10
|
-
|
11
|
-
it "uses the correct format for Date" do
|
12
|
-
Timecop.freeze Time.parse("2018-01-01 12:00:00 UTC") do
|
13
|
-
expect(Date.today.to_json).to eq("\"2018-01-01\"")
|
14
|
-
end
|
15
|
-
end
|
16
|
-
|
17
|
-
it "uses the correct format for DateTime" do
|
18
|
-
Timecop.freeze Time.parse("2018-01-01 12:00:00 UTC") do
|
19
|
-
expect(Time.now.utc.to_json).to eq("\"2018-01-01T12:00:00.000000Z\"")
|
20
|
-
end
|
21
|
-
end
|
22
|
-
|
23
|
-
it "uses the correct format for TimeWithZone" do
|
24
|
-
Timecop.freeze Time.parse("2018-01-01 12:00:00 UTC") do
|
25
|
-
expect(Time.find_zone("UTC").now.to_json).to eq("\"2018-01-01T12:00:00.000000Z\"")
|
26
|
-
end
|
27
|
-
end
|
28
|
-
end
|