search_flip 3.0.0.beta3 → 3.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +3 -0
- data/.travis.yml +3 -1
- data/CHANGELOG.md +11 -0
- data/README.md +62 -3
- data/lib/search_flip.rb +2 -0
- data/lib/search_flip/aggregatable.rb +4 -3
- data/lib/search_flip/aggregation.rb +4 -1
- data/lib/search_flip/aws_sigv4_plugin.rb +47 -0
- data/lib/search_flip/bulk.rb +5 -5
- data/lib/search_flip/config.rb +2 -1
- data/lib/search_flip/criteria.rb +75 -60
- data/lib/search_flip/http_client.rb +10 -8
- data/lib/search_flip/index.rb +1 -1
- data/lib/search_flip/null_instrumenter.rb +21 -0
- data/lib/search_flip/response.rb +1 -1
- data/lib/search_flip/version.rb +1 -1
- data/search_flip.gemspec +2 -0
- data/spec/search_flip/aws_sigv4_plugin_spec.rb +41 -0
- data/spec/search_flip/criteria_spec.rb +42 -0
- data/spec/search_flip/http_client_spec.rb +17 -0
- data/spec/search_flip/null_instrumenter_spec.rb +43 -0
- metadata +38 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f4e3ca349bd68555b2826d84c17b192335752947babe02ee51c5501cb04f79ca
|
4
|
+
data.tar.gz: e95021068918513669da2730d613157ad7151995c4a25f532f4274c3a848710f
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: dcb46f99778d2243e6fb4187f85eb9a1db9aedc25a5f2b8287c3cdb8dcd259253bc3430e1e36521ea5794a2a9188f91473ae8d8572edc8c775ac0d234ac2e284
|
7
|
+
data.tar.gz: 1dbb3ba32c9c2a16f1c2b30aba7257dffc39b708f3e68ec8552c6d6d8ac6a751027a2899a3865051f7d1fdd7a6acd7a1f456eba552c7b50f4c2956292f41d413
|
data/.rubocop.yml
CHANGED
data/.travis.yml
CHANGED
@@ -5,9 +5,11 @@ env:
|
|
5
5
|
- ES_IMAGE=elasticsearch:5.4
|
6
6
|
- ES_IMAGE=docker.elastic.co/elasticsearch/elasticsearch:6.7.0
|
7
7
|
- ES_IMAGE=docker.elastic.co/elasticsearch/elasticsearch:7.0.0
|
8
|
-
- ES_IMAGE=docker.elastic.co/elasticsearch/elasticsearch:7.
|
8
|
+
- ES_IMAGE=docker.elastic.co/elasticsearch/elasticsearch:7.6.0
|
9
9
|
rvm:
|
10
|
+
- ruby-2.5.3
|
10
11
|
- ruby-2.6.2
|
12
|
+
- ruby-2.7.1
|
11
13
|
before_install:
|
12
14
|
- docker-compose up -d
|
13
15
|
- sleep 10
|
data/CHANGELOG.md
CHANGED
@@ -1,6 +1,12 @@
|
|
1
1
|
|
2
2
|
# CHANGELOG
|
3
3
|
|
4
|
+
## v3.1.0
|
5
|
+
|
6
|
+
* Added plugin support in `SearchFlip::HTTPClient`
|
7
|
+
* Added `SearchFlip::AwsSigv4Plugin` to be able to use AWS Elasticsearch with
|
8
|
+
signed requests
|
9
|
+
|
4
10
|
## v3.0.0
|
5
11
|
|
6
12
|
* Added `Criteria#to_query`, which returns a raw query including all queries
|
@@ -23,6 +29,11 @@
|
|
23
29
|
* Added `SearchFlip::Result.from_hit`
|
24
30
|
* Added support for `source`, `sort`, `page`, `per`, `paginate`, `explain`, and
|
25
31
|
`highlight` to aggregations
|
32
|
+
* Added support for instrumentation
|
33
|
+
|
34
|
+
## v2.3.2
|
35
|
+
|
36
|
+
* Remove ruby 2.7 warnings
|
26
37
|
|
27
38
|
## v2.3.1
|
28
39
|
|
data/README.md
CHANGED
@@ -274,7 +274,7 @@ CommentIndex.create(Comment.first, { bulk_max_mb: 100 }, routing: "routing_key")
|
|
274
274
|
CommentIndex.update(Comment.first, ...)
|
275
275
|
```
|
276
276
|
|
277
|
-
Checkout the
|
277
|
+
Checkout the Elasticsearch [Bulk API] docs for more info as well as
|
278
278
|
[SearchFlip::Bulk](http://www.rubydoc.info/github/mrkamel/search_flip/SearchFlip/Bulk)
|
279
279
|
for a complete list of available options to control the bulk indexing of
|
280
280
|
SearchFlip.
|
@@ -475,8 +475,8 @@ end
|
|
475
475
|
```
|
476
476
|
|
477
477
|
Generally, aggregation results returned by Elasticsearch are returned as a
|
478
|
-
`SearchFlip::Result`, which basically is `Hashie::Mash
|
479
|
-
them via:
|
478
|
+
`SearchFlip::Result`, which basically is a `Hashie::Mash`, such that you can
|
479
|
+
access them via:
|
480
480
|
|
481
481
|
```ruby
|
482
482
|
query.aggregations(:username)["mrkamel"].revenue.value
|
@@ -769,6 +769,41 @@ http_client = http_client.headers(key: "value")
|
|
769
769
|
SearchFlip::Connection.new(base_url: "...", http_client: http_client)
|
770
770
|
```
|
771
771
|
|
772
|
+
## AWS Elasticsearch / Signed Requests
|
773
|
+
|
774
|
+
To use SearchFlip with AWS Elasticsearch and signed requests, you have to add
|
775
|
+
`aws-sdk-core` to your Gemfile and tell SearchFlip to use the
|
776
|
+
`SearchFlip::AwsSigv4Plugin`:
|
777
|
+
|
778
|
+
```ruby
|
779
|
+
require "search_flip/aws_sigv4_plugin"
|
780
|
+
|
781
|
+
MyConnection = SearchFlip::Connection.new(
|
782
|
+
base_url: "https://your-elasticsearch-cluster.es.amazonaws.com",
|
783
|
+
http_client: SearchFlip::HTTPClient.new(
|
784
|
+
plugins: [
|
785
|
+
SearchFlip::AwsSigv4Plugin.new(
|
786
|
+
region: "...",
|
787
|
+
access_key_id: "...",
|
788
|
+
secret_access_key: "..."
|
789
|
+
)
|
790
|
+
]
|
791
|
+
)
|
792
|
+
)
|
793
|
+
```
|
794
|
+
|
795
|
+
Again, in your index you need to specify this connection:
|
796
|
+
|
797
|
+
```ruby
|
798
|
+
class MyIndex
|
799
|
+
include SearchFlip::Index
|
800
|
+
|
801
|
+
def self.connection
|
802
|
+
MyConnection
|
803
|
+
end
|
804
|
+
end
|
805
|
+
```
|
806
|
+
|
772
807
|
## Routing and other index-time options
|
773
808
|
|
774
809
|
Override `index_options` in case you want to use routing or pass other
|
@@ -790,6 +825,30 @@ end
|
|
790
825
|
|
791
826
|
These options will be passed whenever records get indexed, deleted, etc.
|
792
827
|
|
828
|
+
## Instrumentation
|
829
|
+
|
830
|
+
SearchFlip supports instrumentation for request execution via
|
831
|
+
`ActiveSupport::Notifications` compatible instrumenters to e.g. allow global
|
832
|
+
performance tracing, etc.
|
833
|
+
|
834
|
+
To use instrumentation, configure the instrumenter:
|
835
|
+
|
836
|
+
```ruby
|
837
|
+
SearchFlip::Config[:instrumenter] = ActiveSupport::Notifications.notifier
|
838
|
+
```
|
839
|
+
|
840
|
+
Subsequently, you can subscribe to notifcations for `request.search_flip`:
|
841
|
+
|
842
|
+
```ruby
|
843
|
+
ActiveSupport::Notifications.subscribe("request.search_flip") do |name, start, finish, id, payload|
|
844
|
+
payload[:index] # the index class
|
845
|
+
payload[:request] # the request hash sent to Elasticsearch
|
846
|
+
payload[:response] # the SearchFlip::Response object or nil in case of errors
|
847
|
+
end
|
848
|
+
```
|
849
|
+
|
850
|
+
A notification will be send for every request that is sent to Elasticsearch.
|
851
|
+
|
793
852
|
## Non-ActiveRecord models
|
794
853
|
|
795
854
|
SearchFlip ships with built-in support for ActiveRecord models, but using
|
data/lib/search_flip.rb
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
require "ruby2_keywords"
|
1
2
|
require "forwardable"
|
2
3
|
require "http"
|
3
4
|
require "hashie"
|
@@ -6,6 +7,7 @@ require "oj"
|
|
6
7
|
require "set"
|
7
8
|
|
8
9
|
require "search_flip/version"
|
10
|
+
require "search_flip/null_instrumenter"
|
9
11
|
require "search_flip/helper"
|
10
12
|
require "search_flip/exceptions"
|
11
13
|
require "search_flip/json"
|
@@ -58,10 +58,11 @@ module SearchFlip
|
|
58
58
|
aggregation = yield(SearchFlip::Aggregation.new(target: target))
|
59
59
|
|
60
60
|
if field_or_hash.is_a?(Hash)
|
61
|
-
|
62
|
-
|
61
|
+
aggregation_hash = field_or_hash.values.first
|
62
|
+
aggregation_hash = aggregation_hash[:top_hits] if aggregation_hash.is_a?(Hash) && aggregation_hash.key?(:top_hits)
|
63
|
+
aggregation_hash = aggregation_hash["top_hits"] if aggregation_hash.is_a?(Hash) && aggregation_hash.key?("top_hits")
|
63
64
|
|
64
|
-
|
65
|
+
aggregation_hash.merge!(aggregation.to_hash)
|
65
66
|
else
|
66
67
|
hash[field_or_hash].merge!(aggregation.to_hash)
|
67
68
|
end
|
@@ -73,7 +73,8 @@ module SearchFlip
|
|
73
73
|
unsupported_methods = [
|
74
74
|
:profile_value, :failsafe_value, :terminate_after_value, :timeout_value, :scroll_args,
|
75
75
|
:suggest_values, :includes_values, :preload_values, :eager_load_values, :post_must_values,
|
76
|
-
:post_must_not_values, :post_filter_values, :preference_value, :search_type_value,
|
76
|
+
:post_must_not_values, :post_filter_values, :preference_value, :search_type_value,
|
77
|
+
:routing_value
|
77
78
|
]
|
78
79
|
|
79
80
|
unsupported_methods.each do |unsupported_method|
|
@@ -111,6 +112,8 @@ module SearchFlip
|
|
111
112
|
end
|
112
113
|
end
|
113
114
|
|
115
|
+
ruby2_keywords :method_missing
|
116
|
+
|
114
117
|
# @api private
|
115
118
|
#
|
116
119
|
# Simply dups the object for api compatability.
|
@@ -0,0 +1,47 @@
|
|
1
|
+
require "aws-sdk-core"
|
2
|
+
require "uri"
|
3
|
+
|
4
|
+
module SearchFlip
|
5
|
+
# The SearchFlip::AwsSigV4Plugin is a plugin for the SearchFlip::HTTPClient
|
6
|
+
# to be used with AWS Elasticsearch to sign requests, i.e. add signed
|
7
|
+
# headers, before sending the request to Elasticsearch.
|
8
|
+
#
|
9
|
+
# @example
|
10
|
+
# MyConnection = SearchFlip::Connection.new(
|
11
|
+
# base_url: "https://your-elasticsearch-cluster.es.amazonaws.com",
|
12
|
+
# http_client: SearchFlip::HTTPClient.new(
|
13
|
+
# plugins: [
|
14
|
+
# SearchFlip::AwsSigv4Plugin.new(
|
15
|
+
# region: "...",
|
16
|
+
# access_key_id: "...",
|
17
|
+
# secret_access_key: "..."
|
18
|
+
# )
|
19
|
+
# ]
|
20
|
+
# )
|
21
|
+
# )
|
22
|
+
|
23
|
+
class AwsSigv4Plugin
|
24
|
+
attr_accessor :signer
|
25
|
+
|
26
|
+
def initialize(options = {})
|
27
|
+
self.signer = Aws::Sigv4::Signer.new({ service: "es" }.merge(options))
|
28
|
+
end
|
29
|
+
|
30
|
+
def call(request, method, uri, options = {})
|
31
|
+
full_uri = URI.parse(uri)
|
32
|
+
full_uri.query = URI.encode_www_form(options[:params]) if options[:params]
|
33
|
+
|
34
|
+
signature_request = {
|
35
|
+
http_method: method.to_s.upcase,
|
36
|
+
url: full_uri.to_s
|
37
|
+
}
|
38
|
+
|
39
|
+
signature_request[:body] = options[:body] if options.key?(:body)
|
40
|
+
signature_request[:body] = options[:json].respond_to?(:to_str) ? options[:json] : JSON.generate(options[:json]) if options[:json]
|
41
|
+
|
42
|
+
signature = signer.sign_request(signature_request)
|
43
|
+
|
44
|
+
request.headers(signature.headers)
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
data/lib/search_flip/bulk.rb
CHANGED
@@ -77,7 +77,7 @@ module SearchFlip
|
|
77
77
|
# and versioning
|
78
78
|
|
79
79
|
def index(id, object, options = {})
|
80
|
-
perform
|
80
|
+
perform(:index, id, SearchFlip::JSON.generate(object), options)
|
81
81
|
end
|
82
82
|
|
83
83
|
# @api private
|
@@ -86,7 +86,7 @@ module SearchFlip
|
|
86
86
|
#
|
87
87
|
# @see #index
|
88
88
|
|
89
|
-
def import(*args)
|
89
|
+
ruby2_keywords def import(*args)
|
90
90
|
index(*args)
|
91
91
|
end
|
92
92
|
|
@@ -100,7 +100,7 @@ module SearchFlip
|
|
100
100
|
# and versioning
|
101
101
|
|
102
102
|
def create(id, object, options = {})
|
103
|
-
perform
|
103
|
+
perform(:create, id, SearchFlip::JSON.generate(object), options)
|
104
104
|
end
|
105
105
|
|
106
106
|
# @api private
|
@@ -113,7 +113,7 @@ module SearchFlip
|
|
113
113
|
# and versioning
|
114
114
|
|
115
115
|
def update(id, object, options = {})
|
116
|
-
perform
|
116
|
+
perform(:update, id, SearchFlip::JSON.generate(object), options)
|
117
117
|
end
|
118
118
|
|
119
119
|
# @api private
|
@@ -125,7 +125,7 @@ module SearchFlip
|
|
125
125
|
# and versioning
|
126
126
|
|
127
127
|
def delete(id, options = {})
|
128
|
-
perform
|
128
|
+
perform(:delete, id, nil, options)
|
129
129
|
end
|
130
130
|
|
131
131
|
private
|
data/lib/search_flip/config.rb
CHANGED
data/lib/search_flip/criteria.rb
CHANGED
@@ -206,7 +206,7 @@ module SearchFlip
|
|
206
206
|
#
|
207
207
|
# @return [SearchFlip::Criteria] Simply returns self
|
208
208
|
|
209
|
-
def with_settings(*args)
|
209
|
+
ruby2_keywords def with_settings(*args)
|
210
210
|
fresh.tap do |criteria|
|
211
211
|
criteria.target = target.with_settings(*args)
|
212
212
|
end
|
@@ -219,45 +219,47 @@ module SearchFlip
|
|
219
219
|
# @return [Hash] The generated request object
|
220
220
|
|
221
221
|
def request
|
222
|
-
|
223
|
-
|
224
|
-
|
225
|
-
|
226
|
-
|
227
|
-
|
228
|
-
|
229
|
-
|
230
|
-
|
231
|
-
|
232
|
-
|
222
|
+
@request ||= begin
|
223
|
+
res = {}
|
224
|
+
|
225
|
+
if must_values || must_not_values || filter_values
|
226
|
+
res[:query] = {
|
227
|
+
bool: {
|
228
|
+
must: must_values.to_a,
|
229
|
+
must_not: must_not_values.to_a,
|
230
|
+
filter: filter_values.to_a
|
231
|
+
}.reject { |_, value| value.empty? }
|
232
|
+
}
|
233
|
+
end
|
233
234
|
|
234
|
-
|
235
|
-
|
236
|
-
|
237
|
-
|
238
|
-
|
239
|
-
|
240
|
-
|
241
|
-
|
242
|
-
|
243
|
-
|
244
|
-
|
245
|
-
|
246
|
-
|
247
|
-
|
248
|
-
|
249
|
-
|
250
|
-
|
251
|
-
|
252
|
-
|
253
|
-
|
235
|
+
res.update(from: offset_value_with_default, size: limit_value_with_default)
|
236
|
+
|
237
|
+
res[:track_total_hits] = track_total_hits_value unless track_total_hits_value.nil?
|
238
|
+
res[:explain] = explain_value unless explain_value.nil?
|
239
|
+
res[:timeout] = timeout_value if timeout_value
|
240
|
+
res[:terminate_after] = terminate_after_value if terminate_after_value
|
241
|
+
res[:highlight] = highlight_values if highlight_values
|
242
|
+
res[:suggest] = suggest_values if suggest_values
|
243
|
+
res[:sort] = sort_values if sort_values
|
244
|
+
res[:aggregations] = aggregation_values if aggregation_values
|
245
|
+
|
246
|
+
if post_must_values || post_must_not_values || post_filter_values
|
247
|
+
res[:post_filter] = {
|
248
|
+
bool: {
|
249
|
+
must: post_must_values.to_a,
|
250
|
+
must_not: post_must_not_values.to_a,
|
251
|
+
filter: post_filter_values.to_a
|
252
|
+
}.reject { |_, value| value.empty? }
|
253
|
+
}
|
254
|
+
end
|
254
255
|
|
255
|
-
|
256
|
-
|
256
|
+
res[:_source] = source_value unless source_value.nil?
|
257
|
+
res[:profile] = true if profile_value
|
257
258
|
|
258
|
-
|
259
|
+
res.update(custom_value) if custom_value
|
259
260
|
|
260
|
-
|
261
|
+
res
|
262
|
+
end
|
261
263
|
end
|
262
264
|
|
263
265
|
# Adds a suggestion section with the given name to the request.
|
@@ -520,30 +522,13 @@ module SearchFlip
|
|
520
522
|
|
521
523
|
def execute
|
522
524
|
@response ||= begin
|
523
|
-
|
524
|
-
|
525
|
-
|
526
|
-
|
527
|
-
|
528
|
-
|
529
|
-
|
530
|
-
json: { scroll: scroll_args[:timeout], scroll_id: scroll_args[:id] }
|
531
|
-
)
|
532
|
-
elsif scroll_args
|
533
|
-
http_request.post(
|
534
|
-
"#{target.type_url}/_search",
|
535
|
-
params: request_params.merge(scroll: scroll_args[:timeout]),
|
536
|
-
json: request
|
537
|
-
)
|
538
|
-
else
|
539
|
-
http_request.post("#{target.type_url}/_search", params: request_params, json: request)
|
540
|
-
end
|
541
|
-
|
542
|
-
SearchFlip::Response.new(self, http_response.parse)
|
543
|
-
rescue SearchFlip::ConnectionError, SearchFlip::ResponseError => e
|
544
|
-
raise e unless failsafe_value
|
545
|
-
|
546
|
-
SearchFlip::Response.new(self, "took" => 0, "hits" => { "total" => 0, "hits" => [] })
|
525
|
+
Config[:instrumenter].instrument("request.search_flip", index: target, request: request) do |payload|
|
526
|
+
response = execute!
|
527
|
+
|
528
|
+
payload[:response] = response
|
529
|
+
|
530
|
+
response
|
531
|
+
end
|
547
532
|
end
|
548
533
|
end
|
549
534
|
|
@@ -585,6 +570,7 @@ module SearchFlip
|
|
585
570
|
|
586
571
|
def fresh
|
587
572
|
dup.tap do |criteria|
|
573
|
+
criteria.instance_variable_set(:@request, nil)
|
588
574
|
criteria.instance_variable_set(:@response, nil)
|
589
575
|
end
|
590
576
|
end
|
@@ -601,6 +587,8 @@ module SearchFlip
|
|
601
587
|
end
|
602
588
|
end
|
603
589
|
|
590
|
+
ruby2_keywords :method_missing
|
591
|
+
|
604
592
|
def_delegators :response, :total_entries, :total_count, :current_page, :previous_page,
|
605
593
|
:prev_page, :next_page, :first_page?, :last_page?, :out_of_range?, :total_pages,
|
606
594
|
:hits, :ids, :count, :size, :length, :took, :aggregations, :suggestions,
|
@@ -610,6 +598,33 @@ module SearchFlip
|
|
610
598
|
|
611
599
|
private
|
612
600
|
|
601
|
+
def execute!
|
602
|
+
http_request = connection.http_client.headers(accept: "application/json")
|
603
|
+
|
604
|
+
http_response =
|
605
|
+
if scroll_args && scroll_args[:id]
|
606
|
+
http_request.post(
|
607
|
+
"#{connection.base_url}/_search/scroll",
|
608
|
+
params: request_params,
|
609
|
+
json: { scroll: scroll_args[:timeout], scroll_id: scroll_args[:id] }
|
610
|
+
)
|
611
|
+
elsif scroll_args
|
612
|
+
http_request.post(
|
613
|
+
"#{target.type_url}/_search",
|
614
|
+
params: request_params.merge(scroll: scroll_args[:timeout]),
|
615
|
+
json: request
|
616
|
+
)
|
617
|
+
else
|
618
|
+
http_request.post("#{target.type_url}/_search", params: request_params, json: request)
|
619
|
+
end
|
620
|
+
|
621
|
+
SearchFlip::Response.new(self, http_response.parse)
|
622
|
+
rescue SearchFlip::ConnectionError, SearchFlip::ResponseError => e
|
623
|
+
raise e unless failsafe_value
|
624
|
+
|
625
|
+
SearchFlip::Response.new(self, "took" => 0, "hits" => { "total" => 0, "hits" => [] })
|
626
|
+
end
|
627
|
+
|
613
628
|
def yield_in_batches(options = {})
|
614
629
|
return enum_for(:yield_in_batches, options) unless block_given?
|
615
630
|
|
@@ -1,15 +1,14 @@
|
|
1
1
|
module SearchFlip
|
2
|
-
# @api private
|
3
|
-
#
|
4
2
|
# The SearchFlip::HTTPClient class wraps the http gem, is for internal use
|
5
3
|
# and responsible for the http request/response handling, ie communicating
|
6
4
|
# with Elasticsearch.
|
7
5
|
|
8
6
|
class HTTPClient
|
9
|
-
attr_accessor :request
|
7
|
+
attr_accessor :request, :plugins
|
10
8
|
|
11
|
-
def initialize
|
9
|
+
def initialize(plugins: [])
|
12
10
|
self.request = HTTP
|
11
|
+
self.plugins = plugins
|
13
12
|
end
|
14
13
|
|
15
14
|
class << self
|
@@ -25,18 +24,21 @@ module SearchFlip
|
|
25
24
|
client.request = request.send(method, *args)
|
26
25
|
end
|
27
26
|
end
|
27
|
+
|
28
|
+
ruby2_keywords method
|
28
29
|
end
|
29
30
|
|
30
31
|
[:get, :post, :put, :delete, :head].each do |method|
|
31
|
-
define_method
|
32
|
-
execute(method,
|
32
|
+
define_method(method) do |uri, options = {}|
|
33
|
+
execute(method, uri, options)
|
33
34
|
end
|
34
35
|
end
|
35
36
|
|
36
37
|
private
|
37
38
|
|
38
|
-
def execute(method,
|
39
|
-
|
39
|
+
def execute(method, uri, options = {})
|
40
|
+
final_request = plugins.inject(self) { |res, cur| cur.call(res, method, uri, options) }
|
41
|
+
response = final_request.request.send(method, uri, options)
|
40
42
|
|
41
43
|
raise SearchFlip::ResponseError.new(code: response.code, body: response.body.to_s) unless response.status.success?
|
42
44
|
|
data/lib/search_flip/index.rb
CHANGED
@@ -0,0 +1,21 @@
|
|
1
|
+
module SearchFlip
|
2
|
+
class NullInstrumenter
|
3
|
+
def instrument(name, payload = {})
|
4
|
+
start(name, payload)
|
5
|
+
|
6
|
+
begin
|
7
|
+
yield(payload) if block_given?
|
8
|
+
ensure
|
9
|
+
finish(name, payload)
|
10
|
+
end
|
11
|
+
end
|
12
|
+
|
13
|
+
def start(_name, _payload)
|
14
|
+
true
|
15
|
+
end
|
16
|
+
|
17
|
+
def finish(_name, _payload)
|
18
|
+
true
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
data/lib/search_flip/response.rb
CHANGED
@@ -222,7 +222,7 @@ module SearchFlip
|
|
222
222
|
#
|
223
223
|
# @return [Array] An array of database records
|
224
224
|
|
225
|
-
def records
|
225
|
+
def records
|
226
226
|
@records ||= begin
|
227
227
|
sort_map = ids.each_with_index.each_with_object({}) { |(id, index), hash| hash[id.to_s] = index }
|
228
228
|
|
data/lib/search_flip/version.rb
CHANGED
data/search_flip.gemspec
CHANGED
@@ -25,6 +25,7 @@ Gem::Specification.new do |spec|
|
|
25
25
|
MESSAGE
|
26
26
|
|
27
27
|
spec.add_development_dependency "activerecord", ">= 3.0"
|
28
|
+
spec.add_development_dependency "aws-sdk-core"
|
28
29
|
spec.add_development_dependency "bundler"
|
29
30
|
spec.add_development_dependency "factory_bot"
|
30
31
|
spec.add_development_dependency "rake"
|
@@ -37,4 +38,5 @@ Gem::Specification.new do |spec|
|
|
37
38
|
spec.add_dependency "hashie"
|
38
39
|
spec.add_dependency "http"
|
39
40
|
spec.add_dependency "oj"
|
41
|
+
spec.add_dependency "ruby2_keywords"
|
40
42
|
end
|
@@ -0,0 +1,41 @@
|
|
1
|
+
require File.expand_path("../spec_helper", __dir__)
|
2
|
+
require "search_flip/aws_sigv4_plugin"
|
3
|
+
|
4
|
+
RSpec.describe SearchFlip::AwsSigv4Plugin do
|
5
|
+
describe "#call" do
|
6
|
+
subject(:plugin) do
|
7
|
+
SearchFlip::AwsSigv4Plugin.new(
|
8
|
+
region: "us-east-1",
|
9
|
+
access_key_id: "access key",
|
10
|
+
secret_access_key: "secret access key"
|
11
|
+
)
|
12
|
+
end
|
13
|
+
|
14
|
+
let(:client) { SearchFlip::HTTPClient.new }
|
15
|
+
|
16
|
+
it "adds the signed headers to the request" do
|
17
|
+
Timecop.freeze Time.parse("2020-01-01 12:00:00 UTC") do
|
18
|
+
expect(client).to receive(:headers).with(
|
19
|
+
"host" => "localhost",
|
20
|
+
"authorization" => /.*/,
|
21
|
+
"x-amz-content-sha256" => /.*/,
|
22
|
+
"x-amz-date" => /20200101T120000Z/
|
23
|
+
)
|
24
|
+
|
25
|
+
plugin.call(client, :get, "http://localhost/index")
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
it "feeds the http method, full url and body to the signer" do
|
30
|
+
signing_request = {
|
31
|
+
http_method: "GET",
|
32
|
+
url: "http://localhost/index?param=value",
|
33
|
+
body: JSON.generate(key: "value")
|
34
|
+
}
|
35
|
+
|
36
|
+
expect(plugin.signer).to receive(:sign_request).with(signing_request).and_call_original
|
37
|
+
|
38
|
+
plugin.call(client, :get, "http://localhost/index", params: { param: "value" }, json: { key: "value" })
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
@@ -1198,9 +1198,12 @@ RSpec.describe SearchFlip::Criteria do
|
|
1198
1198
|
|
1199
1199
|
query = ProductIndex.criteria.tap(&:records)
|
1200
1200
|
|
1201
|
+
expect(query.instance_variable_get(:@request)).not_to be_nil
|
1201
1202
|
expect(query.instance_variable_get(:@response)).not_to be_nil
|
1202
1203
|
|
1203
1204
|
expect(query.object_id).not_to eq(query.fresh.object_id)
|
1205
|
+
|
1206
|
+
expect(query.fresh.instance_variable_get(:@request)).to be_nil
|
1204
1207
|
expect(query.fresh.instance_variable_get(:@response)).to be_nil
|
1205
1208
|
end
|
1206
1209
|
end
|
@@ -1234,10 +1237,49 @@ RSpec.describe SearchFlip::Criteria do
|
|
1234
1237
|
end
|
1235
1238
|
end
|
1236
1239
|
|
1240
|
+
describe "#execute" do
|
1241
|
+
around do |example|
|
1242
|
+
default_instrumenter = SearchFlip::Config[:instrumenter]
|
1243
|
+
|
1244
|
+
SearchFlip::Config[:instrumenter] = ActiveSupport::Notifications.instrumenter
|
1245
|
+
|
1246
|
+
begin
|
1247
|
+
example.run
|
1248
|
+
ensure
|
1249
|
+
SearchFlip::Config[:instrumenter] = default_instrumenter
|
1250
|
+
end
|
1251
|
+
end
|
1252
|
+
|
1253
|
+
let(:notifications) { [] }
|
1254
|
+
|
1255
|
+
let!(:subscriber) do
|
1256
|
+
ActiveSupport::Notifications.subscribe("request.search_flip") do |*args|
|
1257
|
+
notifications << args
|
1258
|
+
end
|
1259
|
+
end
|
1260
|
+
|
1261
|
+
after { ActiveSupport::Notifications.unsubscribe(subscriber) }
|
1262
|
+
|
1263
|
+
it "instruments the request" do
|
1264
|
+
ProductIndex.match_all.execute
|
1265
|
+
|
1266
|
+
expect(notifications).to be_present
|
1267
|
+
end
|
1268
|
+
|
1269
|
+
it "passes the index, request and response" do
|
1270
|
+
ProductIndex.match_all.execute
|
1271
|
+
|
1272
|
+
expect(notifications.first[4][:index]).to eq(ProductIndex)
|
1273
|
+
expect(notifications.first[4][:request]).to be_present
|
1274
|
+
expect(notifications.first[4][:response]).to be_present
|
1275
|
+
end
|
1276
|
+
end
|
1277
|
+
|
1237
1278
|
describe "#track_total_hits" do
|
1238
1279
|
it "is added to the request" do
|
1239
1280
|
if ProductIndex.connection.version.to_i >= 7
|
1240
1281
|
query = ProductIndex.track_total_hits(false)
|
1282
|
+
|
1241
1283
|
expect(query.request[:track_total_hits]).to eq(false)
|
1242
1284
|
expect { query.execute }.not_to raise_error
|
1243
1285
|
end
|
@@ -39,6 +39,23 @@ RSpec.describe SearchFlip::HTTPClient do
|
|
39
39
|
end
|
40
40
|
end
|
41
41
|
|
42
|
+
describe "plugins" do
|
43
|
+
subject do
|
44
|
+
SearchFlip::HTTPClient.new(
|
45
|
+
plugins: [
|
46
|
+
->(request, _method, _uri, _options = {}) { request.headers("First-Header" => "Value") },
|
47
|
+
->(request, _method, _uri, _options = {}) { request.headers("Second-Header" => "Value") }
|
48
|
+
]
|
49
|
+
)
|
50
|
+
end
|
51
|
+
|
52
|
+
it "injects the plugins and uses their result in the request" do
|
53
|
+
stub_request(:get, "http://localhost/path").with(query: { key: "value" }, headers: { "First-Header" => "Value", "Second-Header" => "Value" }).and_return(body: "success")
|
54
|
+
|
55
|
+
expect(subject.get("http://localhost/path", params: { key: "value" }).body.to_s).to eq("success")
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
42
59
|
[:via, :basic_auth, :auth].each do |method|
|
43
60
|
describe "##{method}" do
|
44
61
|
it "creates a dupped instance" do
|
@@ -0,0 +1,43 @@
|
|
1
|
+
require File.expand_path("../spec_helper", __dir__)
|
2
|
+
|
3
|
+
RSpec.describe SearchFlip::NullInstrumenter do
|
4
|
+
subject { described_class.new }
|
5
|
+
|
6
|
+
describe "#instrument" do
|
7
|
+
it "calls start" do
|
8
|
+
allow(subject).to receive(:start)
|
9
|
+
|
10
|
+
subject.instrument("name", { key: "value" }) {}
|
11
|
+
|
12
|
+
expect(subject).to have_received(:start)
|
13
|
+
end
|
14
|
+
|
15
|
+
it "calls finish" do
|
16
|
+
allow(subject).to receive(:finish)
|
17
|
+
|
18
|
+
subject.instrument("name", { key: "value" }) {}
|
19
|
+
|
20
|
+
expect(subject).to have_received(:finish)
|
21
|
+
end
|
22
|
+
|
23
|
+
it "yields and passes the payload" do
|
24
|
+
yielded_payload = nil
|
25
|
+
|
26
|
+
subject.instrument("name", { key: "value" }) { |payload| yielded_payload = payload }
|
27
|
+
|
28
|
+
expect(yielded_payload).to eq(key: "value")
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
describe "#start" do
|
33
|
+
it "returns true" do
|
34
|
+
expect(subject.start("name", { key: "value" })).to eq(true)
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
describe "#finish" do
|
39
|
+
it "returns true" do
|
40
|
+
expect(subject.finish("name", { key: "value" })).to eq(true)
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: search_flip
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 3.
|
4
|
+
version: 3.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Benjamin Vetter
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-
|
11
|
+
date: 2020-06-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -24,6 +24,20 @@ dependencies:
|
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: '3.0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: aws-sdk-core
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - ">="
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
34
|
+
type: :development
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - ">="
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
27
41
|
- !ruby/object:Gem::Dependency
|
28
42
|
name: bundler
|
29
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -178,6 +192,20 @@ dependencies:
|
|
178
192
|
- - ">="
|
179
193
|
- !ruby/object:Gem::Version
|
180
194
|
version: '0'
|
195
|
+
- !ruby/object:Gem::Dependency
|
196
|
+
name: ruby2_keywords
|
197
|
+
requirement: !ruby/object:Gem::Requirement
|
198
|
+
requirements:
|
199
|
+
- - ">="
|
200
|
+
- !ruby/object:Gem::Version
|
201
|
+
version: '0'
|
202
|
+
type: :runtime
|
203
|
+
prerelease: false
|
204
|
+
version_requirements: !ruby/object:Gem::Requirement
|
205
|
+
requirements:
|
206
|
+
- - ">="
|
207
|
+
- !ruby/object:Gem::Version
|
208
|
+
version: '0'
|
181
209
|
description: Full-Featured Elasticsearch Ruby Client with a Chainable DSL
|
182
210
|
email:
|
183
211
|
- vetter@flakks.com
|
@@ -198,6 +226,7 @@ files:
|
|
198
226
|
- lib/search_flip.rb
|
199
227
|
- lib/search_flip/aggregatable.rb
|
200
228
|
- lib/search_flip/aggregation.rb
|
229
|
+
- lib/search_flip/aws_sigv4_plugin.rb
|
201
230
|
- lib/search_flip/bulk.rb
|
202
231
|
- lib/search_flip/config.rb
|
203
232
|
- lib/search_flip/connection.rb
|
@@ -212,6 +241,7 @@ files:
|
|
212
241
|
- lib/search_flip/index.rb
|
213
242
|
- lib/search_flip/json.rb
|
214
243
|
- lib/search_flip/model.rb
|
244
|
+
- lib/search_flip/null_instrumenter.rb
|
215
245
|
- lib/search_flip/paginatable.rb
|
216
246
|
- lib/search_flip/post_filterable.rb
|
217
247
|
- lib/search_flip/response.rb
|
@@ -223,12 +253,14 @@ files:
|
|
223
253
|
- search_flip.gemspec
|
224
254
|
- spec/delegate_matcher.rb
|
225
255
|
- spec/search_flip/aggregation_spec.rb
|
256
|
+
- spec/search_flip/aws_sigv4_plugin_spec.rb
|
226
257
|
- spec/search_flip/bulk_spec.rb
|
227
258
|
- spec/search_flip/connection_spec.rb
|
228
259
|
- spec/search_flip/criteria_spec.rb
|
229
260
|
- spec/search_flip/http_client_spec.rb
|
230
261
|
- spec/search_flip/index_spec.rb
|
231
262
|
- spec/search_flip/model_spec.rb
|
263
|
+
- spec/search_flip/null_instrumenter_spec.rb
|
232
264
|
- spec/search_flip/response_spec.rb
|
233
265
|
- spec/search_flip/result_spec.rb
|
234
266
|
- spec/search_flip/to_json_spec.rb
|
@@ -251,9 +283,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
251
283
|
version: '0'
|
252
284
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
253
285
|
requirements:
|
254
|
-
- - "
|
286
|
+
- - ">="
|
255
287
|
- !ruby/object:Gem::Version
|
256
|
-
version:
|
288
|
+
version: '0'
|
257
289
|
requirements: []
|
258
290
|
rubygems_version: 3.0.3
|
259
291
|
signing_key:
|
@@ -262,12 +294,14 @@ summary: Full-Featured Elasticsearch Ruby Client with a Chainable DSL
|
|
262
294
|
test_files:
|
263
295
|
- spec/delegate_matcher.rb
|
264
296
|
- spec/search_flip/aggregation_spec.rb
|
297
|
+
- spec/search_flip/aws_sigv4_plugin_spec.rb
|
265
298
|
- spec/search_flip/bulk_spec.rb
|
266
299
|
- spec/search_flip/connection_spec.rb
|
267
300
|
- spec/search_flip/criteria_spec.rb
|
268
301
|
- spec/search_flip/http_client_spec.rb
|
269
302
|
- spec/search_flip/index_spec.rb
|
270
303
|
- spec/search_flip/model_spec.rb
|
304
|
+
- spec/search_flip/null_instrumenter_spec.rb
|
271
305
|
- spec/search_flip/response_spec.rb
|
272
306
|
- spec/search_flip/result_spec.rb
|
273
307
|
- spec/search_flip/to_json_spec.rb
|