chewy 7.2.1 → 7.2.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +20 -0
- data/README.md +19 -8
- data/chewy.gemspec +1 -2
- data/lib/chewy/index/actions.rb +8 -13
- data/lib/chewy/index/adapter/object.rb +0 -10
- data/lib/chewy/index/adapter/orm.rb +0 -12
- data/lib/chewy/index/import.rb +15 -27
- data/lib/chewy/index/import/routine.rb +2 -2
- data/lib/chewy/minitest/helpers.rb +77 -0
- data/lib/chewy/rake_helper.rb +1 -1
- data/lib/chewy/rspec.rb +2 -0
- data/lib/chewy/rspec/build_query.rb +12 -0
- data/lib/chewy/rspec/helpers.rb +55 -0
- data/lib/chewy/search/parameters/order.rb +6 -19
- data/lib/chewy/version.rb +1 -1
- data/migration_guide.md +1 -1
- data/spec/chewy/index/actions_spec.rb +6 -6
- data/spec/chewy/index/import/routine_spec.rb +2 -2
- data/spec/chewy/index/import_spec.rb +1 -32
- data/spec/chewy/minitest/helpers_spec.rb +110 -0
- data/spec/chewy/rspec/build_query_spec.rb +34 -0
- data/spec/chewy/rspec/helpers_spec.rb +61 -0
- data/spec/chewy/search/parameters/order_spec.rb +18 -11
- data/spec/chewy/search/parameters_spec.rb +1 -1
- data/spec/chewy/search/request_spec.rb +1 -1
- data/spec/spec_helper.rb +0 -1
- data/spec/support/active_record.rb +1 -1
- metadata +10 -19
- data/lib/chewy/index/import/thread_safe_progress_bar.rb +0 -40
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 229c412b1e3464060b367c9c378d3662bccf2329b4e4eaa7617541243dbd6e38
|
4
|
+
data.tar.gz: 9bc674550cd6152e8a18853fa46c91873c7a626c426cd50aa2cb8c67b1c9455e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e3e8409f81bc2504e9edbf5093fc9329bf443862cb96e113c9c00fc586c1e0e3a4273b3d6e82466f9b3d301e3d9eafcc72892cce718aff7dcfbbce85aba1698d
|
7
|
+
data.tar.gz: 33b5b0e21ef7c9896f1ed381130f225a1d51e073112213f631fb1c57c6d69c49af54817b1de3afc9be325c79ac24f9c173275b4448ffb7b43e82403bee03bd70
|
data/CHANGELOG.md
CHANGED
@@ -8,6 +8,25 @@
|
|
8
8
|
|
9
9
|
### Bugs Fixed
|
10
10
|
|
11
|
+
## 7.2.2 (2021-05-24)
|
12
|
+
|
13
|
+
### Changes
|
14
|
+
|
15
|
+
* [#800](https://github.com/toptal/chewy/pull/800): Revert [#787](https://github.com/toptal/chewy/pull/787) progressbar feature to avoid performance degradation in parallel import ([@rabotyaga][])
|
16
|
+
|
17
|
+
* [#795](https://github.com/toptal/chewy/issues/795): **(Breaking)** Change the Chewy::Search::Parameters::Order implementation to use Array ([@jiajiawang][]):
|
18
|
+
* To allow multiple sorting options that may have the same key name. For example script based sorting whose key will always be `_script`.
|
19
|
+
* Behaviour change of chained `order` calls.
|
20
|
+
* e.g. `.order(_script: {a: 1}).order(_script: {b: 2})`
|
21
|
+
* Before `{:sort=>[{"_script"=>{:b=>2}}]}`
|
22
|
+
* After `{:sort=>[{"_script"=>{:a=>1}},{"_script"=>{:b=>2}}]}`
|
23
|
+
|
24
|
+
* [#654](https://github.com/toptal/chewy/issues/654): Add helpers and matchers for testing ([@Vitalina-Vakulchyk][]):
|
25
|
+
* `mock_elasticsearch_response` helpers both Rspec and Minitest - to mock elasticsearch response
|
26
|
+
* `mock_elasticsearch_response_sources` helpers both Rspec and Minitest - to mock elasticsearch response sources
|
27
|
+
* `assert_elasticsearch_query` helper for Minitest - to compare request and expected query (returns `true`/`false`)
|
28
|
+
* `build_query` matcher for Rspec - to compare request and expected query (returns `true`/`false`)
|
29
|
+
|
11
30
|
## 7.2.1 (2021-05-11)
|
12
31
|
|
13
32
|
### New Features
|
@@ -631,6 +650,7 @@
|
|
631
650
|
[@inbeom]: https://github.com/inbeom
|
632
651
|
[@jesjos]: https://github.com/jesjos
|
633
652
|
[@JF-Lalonde]: https://github.com/JF-Lalonde
|
653
|
+
[@jiajiawang]: https://github.com/jiajiawang
|
634
654
|
[@jimmybaker]: https://github.com/jimmybaker
|
635
655
|
[@jirikolarik]: https://github.com/jirikolarik
|
636
656
|
[@jirutka]: https://github.com/jirutka
|
data/README.md
CHANGED
@@ -53,7 +53,7 @@ Chewy is compatible with MRI 2.6-3.0¹.
|
|
53
53
|
| ------------- | ---------------------------------- |
|
54
54
|
| 7.2.x | 7.x |
|
55
55
|
| 7.1.x | 7.x |
|
56
|
-
| 7.0.
|
56
|
+
| 7.0.x | 6.8, 7.x |
|
57
57
|
| 6.0.0 | 5.x, 6.x |
|
58
58
|
| 5.x | 5.x, limited support for 1.x & 2.x |
|
59
59
|
|
@@ -1032,12 +1032,6 @@ rake chewy:reset[users,cities] # resets UsersIndex and CitiesIndex
|
|
1032
1032
|
rake chewy:reset[-users,cities] # resets every index in the application except specified ones
|
1033
1033
|
```
|
1034
1034
|
|
1035
|
-
#### Progressbar for `chewy:reset` tasks
|
1036
|
-
|
1037
|
-
You can optionally output the `progressbar` for `chewy:reset` and `chewy:parallel:reset` during import.
|
1038
|
-
|
1039
|
-
Progressbar is hidden by default. Set `ENV['PROGRESS']` to `true` to display it.
|
1040
|
-
|
1041
1035
|
#### `chewy:upgrade`
|
1042
1036
|
|
1043
1037
|
Performs reset exactly the same way as `chewy:reset` does, but only when the index specification (setting or mapping) was changed.
|
@@ -1117,7 +1111,16 @@ rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)",users] # apply journaled cha
|
|
1117
1111
|
|
1118
1112
|
### RSpec integration
|
1119
1113
|
|
1120
|
-
Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features:
|
1114
|
+
Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features:
|
1115
|
+
|
1116
|
+
[update_index](lib/chewy/rspec/update_index.rb) helper
|
1117
|
+
`mock_elasticsearch_response` helper to mock elasticsearch response
|
1118
|
+
`mock_elasticsearch_response_sources` helper to mock elasticsearch response sources
|
1119
|
+
`build_query` matcher to compare request and expected query (returns `true`/`false`)
|
1120
|
+
|
1121
|
+
To use `mock_elasticsearch_response` and `mock_elasticsearch_response_sources` helpers add `include Chewy::Rspec::Helpers` to your tests.
|
1122
|
+
|
1123
|
+
See [chewy/rspec/](lib/chewy/rspec/) for more details.
|
1121
1124
|
|
1122
1125
|
### Minitest integration
|
1123
1126
|
|
@@ -1127,6 +1130,14 @@ Since you can set `:bypass` strategy for test suites and manually handle import
|
|
1127
1130
|
|
1128
1131
|
But if you require chewy to index/update model regularly in your test suite then you can specify `:urgent` strategy for documents indexing. Add `Chewy.strategy(:urgent)` to test_helper.rb.
|
1129
1132
|
|
1133
|
+
Also, you can use additional helpers:
|
1134
|
+
|
1135
|
+
`mock_elasticsearch_response` to mock elasticsearch response
|
1136
|
+
`mock_elasticsearch_response_sources` to mock elasticsearch response sources
|
1137
|
+
`assert_elasticsearch_query` to compare request and expected query (returns `true`/`false`)
|
1138
|
+
|
1139
|
+
See [chewy/minitest/](lib/chewy/minitest/) for more details.
|
1140
|
+
|
1130
1141
|
### DatabaseCleaner
|
1131
1142
|
|
1132
1143
|
If you use `DatabaseCleaner` in your tests with [the `transaction` strategy](https://github.com/DatabaseCleaner/database_cleaner#how-to-use), you may run into the problem that `ActiveRecord`'s models are not indexed automatically on save despite the fact that you set the callbacks to do this with the `update_index` method. The issue arises because `chewy` indices data on `after_commit` run as default, but all `after_commit` callbacks are not run with the `DatabaseCleaner`'s' `transaction` strategy. You can solve this issue by changing the `Chewy.use_after_commit_callbacks` option. Just add the following initializer in your Rails application:
|
data/chewy.gemspec
CHANGED
@@ -31,7 +31,6 @@ Gem::Specification.new do |spec| # rubocop:disable Metrics/BlockLength
|
|
31
31
|
spec.add_development_dependency 'unparser'
|
32
32
|
|
33
33
|
spec.add_dependency 'activesupport', '>= 5.2'
|
34
|
-
spec.add_dependency 'elasticsearch', '>=
|
34
|
+
spec.add_dependency 'elasticsearch', '>= 7.12.0'
|
35
35
|
spec.add_dependency 'elasticsearch-dsl'
|
36
|
-
spec.add_dependency 'ruby-progressbar'
|
37
36
|
end
|
data/lib/chewy/index/actions.rb
CHANGED
@@ -146,7 +146,7 @@ module Chewy
|
|
146
146
|
# @param journal [true, false] journaling is switched off for import during reset by default
|
147
147
|
# @param import_options [Hash] options, passed to the import call
|
148
148
|
# @return [true, false] false in case of errors
|
149
|
-
def reset!(suffix = nil, apply_journal: true, journal: false,
|
149
|
+
def reset!(suffix = nil, apply_journal: true, journal: false, **import_options)
|
150
150
|
result = if suffix.present?
|
151
151
|
start_time = Time.now
|
152
152
|
indexes = self.indexes - [index_name]
|
@@ -159,13 +159,17 @@ module Chewy
|
|
159
159
|
result = import import_options.merge(
|
160
160
|
suffix: suffix,
|
161
161
|
journal: journal,
|
162
|
-
refresh: !Chewy.reset_disable_refresh_interval
|
163
|
-
progressbar: progressbar
|
162
|
+
refresh: !Chewy.reset_disable_refresh_interval
|
164
163
|
)
|
165
164
|
original_index_settings suffixed_name
|
166
165
|
|
167
166
|
delete if indexes.blank?
|
168
|
-
update_aliases
|
167
|
+
client.indices.update_aliases body: {actions: [
|
168
|
+
*indexes.map do |index|
|
169
|
+
{remove: {index: index, alias: general_name}}
|
170
|
+
end,
|
171
|
+
{add: {index: suffixed_name, alias: general_name}}
|
172
|
+
]}
|
169
173
|
client.indices.delete index: indexes if indexes.present?
|
170
174
|
|
171
175
|
self.journal.apply(start_time, **import_options) if apply_journal
|
@@ -233,15 +237,6 @@ module Chewy
|
|
233
237
|
|
234
238
|
private
|
235
239
|
|
236
|
-
def update_aliases(indexes, general_name, suffixed_name)
|
237
|
-
client.indices.update_aliases body: {actions: [
|
238
|
-
*indexes.map do |index|
|
239
|
-
{remove: {index: index, alias: general_name}}
|
240
|
-
end,
|
241
|
-
{add: {index: suffixed_name, alias: general_name}}
|
242
|
-
]}
|
243
|
-
end
|
244
|
-
|
245
240
|
def optimize_index_settings(index_name)
|
246
241
|
settings = {}
|
247
242
|
settings[:refresh_interval] = -1 if Chewy.reset_disable_refresh_interval
|
@@ -192,16 +192,6 @@ module Chewy
|
|
192
192
|
end
|
193
193
|
end
|
194
194
|
|
195
|
-
def import_count(*args)
|
196
|
-
collection = if args.first.empty? && @target.respond_to?(import_all_method)
|
197
|
-
@target.send(import_all_method)
|
198
|
-
else
|
199
|
-
args.flatten(1).compact
|
200
|
-
end
|
201
|
-
|
202
|
-
collection.count
|
203
|
-
end
|
204
|
-
|
205
195
|
private
|
206
196
|
|
207
197
|
def import_objects(objects, options)
|
@@ -108,18 +108,6 @@ module Chewy
|
|
108
108
|
ids.map { |id| loaded_objects[id.to_s] }
|
109
109
|
end
|
110
110
|
|
111
|
-
def import_count(*args)
|
112
|
-
collection = if args.first.empty?
|
113
|
-
default_scope
|
114
|
-
elsif args.first.is_a?(relation_class)
|
115
|
-
args.first
|
116
|
-
else
|
117
|
-
args.flatten.compact
|
118
|
-
end
|
119
|
-
|
120
|
-
collection.count
|
121
|
-
end
|
122
|
-
|
123
111
|
private
|
124
112
|
|
125
113
|
def import_objects(collection, options)
|
data/lib/chewy/index/import.rb
CHANGED
@@ -2,19 +2,17 @@ require 'chewy/index/import/journal_builder'
|
|
2
2
|
require 'chewy/index/import/bulk_builder'
|
3
3
|
require 'chewy/index/import/bulk_request'
|
4
4
|
require 'chewy/index/import/routine'
|
5
|
-
require 'chewy/index/import/thread_safe_progress_bar'
|
6
5
|
|
7
6
|
module Chewy
|
8
7
|
class Index
|
9
8
|
module Import
|
10
9
|
extend ActiveSupport::Concern
|
11
10
|
|
12
|
-
IMPORT_WORKER = lambda do |index, options, total,
|
11
|
+
IMPORT_WORKER = lambda do |index, options, total, ids, iteration|
|
13
12
|
::Process.setproctitle("chewy [#{index}]: import data (#{iteration + 1}/#{total})")
|
14
13
|
routine = Routine.new(index, **options)
|
15
14
|
index.adapter.import(*ids, routine.options) do |action_objects|
|
16
15
|
routine.process(**action_objects)
|
17
|
-
progress_bar.increment(action_objects.map { |_, v| v.size }.sum) if routine.options[:progressbar]
|
18
16
|
end
|
19
17
|
{errors: routine.errors, import: routine.stats, leftovers: routine.leftovers}
|
20
18
|
end
|
@@ -152,10 +150,8 @@ module Chewy
|
|
152
150
|
|
153
151
|
def import_linear(objects, routine)
|
154
152
|
ActiveSupport::Notifications.instrument 'import_objects.chewy', index: self do |payload|
|
155
|
-
progress_bar = ThreadSafeProgressBar.new(routine.options[:progressbar]) { adapter.import_count(objects) }
|
156
153
|
adapter.import(*objects, routine.options) do |action_objects|
|
157
154
|
routine.process(**action_objects)
|
158
|
-
progress_bar.increment(action_objects.map { |_, v| v.size }.sum) if routine.options[:progressbar]
|
159
155
|
end
|
160
156
|
routine.perform_bulk(routine.leftovers)
|
161
157
|
payload[:import] = routine.stats
|
@@ -170,20 +166,24 @@ module Chewy
|
|
170
166
|
ActiveSupport::Notifications.instrument 'import_objects.chewy', index: self do |payload|
|
171
167
|
batches = adapter.import_references(*objects, routine.options.slice(:batch_size)).to_a
|
172
168
|
|
173
|
-
progress_bar = ThreadSafeProgressBar.new(routine.options[:progressbar]) { adapter.import_count(objects) }
|
174
169
|
::ActiveRecord::Base.connection.close if defined?(::ActiveRecord::Base)
|
175
|
-
|
176
|
-
|
177
|
-
|
178
|
-
|
179
|
-
|
180
|
-
end
|
181
|
-
end
|
182
|
-
|
170
|
+
results = ::Parallel.map_with_index(
|
171
|
+
batches,
|
172
|
+
routine.parallel_options,
|
173
|
+
&IMPORT_WORKER.curry[self, routine.options, batches.size]
|
174
|
+
)
|
183
175
|
::ActiveRecord::Base.connection.reconnect! if defined?(::ActiveRecord::Base)
|
184
176
|
errors, import, leftovers = process_parallel_import_results(results)
|
185
177
|
|
186
|
-
|
178
|
+
if leftovers.present?
|
179
|
+
batches = leftovers.each_slice(routine.options[:batch_size])
|
180
|
+
results = ::Parallel.map_with_index(
|
181
|
+
batches,
|
182
|
+
routine.parallel_options,
|
183
|
+
&LEFTOVERS_WORKER.curry[self, routine.options, batches.size]
|
184
|
+
)
|
185
|
+
errors.concat(results.flatten(1))
|
186
|
+
end
|
187
187
|
|
188
188
|
payload[:import] = import
|
189
189
|
payload[:errors] = payload_errors(errors) if errors.present?
|
@@ -191,18 +191,6 @@ module Chewy
|
|
191
191
|
end
|
192
192
|
end
|
193
193
|
|
194
|
-
def execute_leftovers(leftovers, routine, self_object, errors)
|
195
|
-
return unless leftovers.present?
|
196
|
-
|
197
|
-
batches = leftovers.each_slice(routine.options[:batch_size])
|
198
|
-
results = ::Parallel.map_with_index(
|
199
|
-
batches,
|
200
|
-
routine.parallel_options,
|
201
|
-
&LEFTOVERS_WORKER.curry[self_object, routine.options, batches.size]
|
202
|
-
)
|
203
|
-
errors.concat(results.flatten(1))
|
204
|
-
end
|
205
|
-
|
206
194
|
def process_parallel_import_results(results)
|
207
195
|
results.each_with_object([[], {}, []]) do |r, (e, i, l)|
|
208
196
|
e.concat(r[:errors])
|
@@ -51,9 +51,9 @@ module Chewy
|
|
51
51
|
@parallel_options = @options.delete(:parallel)
|
52
52
|
if @parallel_options && !@parallel_options.is_a?(Hash)
|
53
53
|
@parallel_options = if @parallel_options.is_a?(Integer)
|
54
|
-
{
|
54
|
+
{in_processes: @parallel_options}
|
55
55
|
else
|
56
|
-
{
|
56
|
+
{}
|
57
57
|
end
|
58
58
|
end
|
59
59
|
@errors = []
|
@@ -42,10 +42,87 @@ module Chewy
|
|
42
42
|
# Run indexing for the database changes during the block provided.
|
43
43
|
# By default, indexing is run at the end of the block.
|
44
44
|
# @param strategy [Symbol] the Chewy index update strategy see Chewy docs.
|
45
|
+
#
|
45
46
|
def run_indexing(strategy: :atomic, &block)
|
46
47
|
Chewy.strategy strategy, &block
|
47
48
|
end
|
48
49
|
|
50
|
+
# Mock Elasticsearch response
|
51
|
+
# Simple usage - just pass index, expected raw response
|
52
|
+
# and block with the query.
|
53
|
+
#
|
54
|
+
# @param index [Chewy::Index] the index to watch, eg EntitiesIndex.
|
55
|
+
# @param raw_response [Hash] hash with response.
|
56
|
+
#
|
57
|
+
def mock_elasticsearch_response(index, raw_response)
|
58
|
+
mocked_request = Chewy::Search::Request.new(index)
|
59
|
+
|
60
|
+
original_new = Chewy::Search::Request.method(:new)
|
61
|
+
|
62
|
+
Chewy::Search::Request.define_singleton_method(:new) { |*_args| mocked_request }
|
63
|
+
|
64
|
+
original_perform = mocked_request.method(:perform)
|
65
|
+
mocked_request.define_singleton_method(:perform) { raw_response }
|
66
|
+
|
67
|
+
yield
|
68
|
+
ensure
|
69
|
+
mocked_request.define_singleton_method(:perform, original_perform)
|
70
|
+
Chewy::Search::Request.define_singleton_method(:new, original_new)
|
71
|
+
end
|
72
|
+
|
73
|
+
# Mock Elasticsearch response with defined sources
|
74
|
+
# Simple usage - just pass index, expected sources
|
75
|
+
# and block with the query.
|
76
|
+
#
|
77
|
+
# @param index [Chewy::Index] the index to watch, eg EntitiesIndex.
|
78
|
+
# @param hits [Hash] hash with sources.
|
79
|
+
#
|
80
|
+
def mock_elasticsearch_response_sources(index, hits, &block)
|
81
|
+
raw_response = {
|
82
|
+
'took' => 4,
|
83
|
+
'timed_out' => false,
|
84
|
+
'_shards' => {
|
85
|
+
'total' => 1,
|
86
|
+
'successful' => 1,
|
87
|
+
'skipped' => 0,
|
88
|
+
'failed' => 0
|
89
|
+
},
|
90
|
+
'hits' => {
|
91
|
+
'total' => {
|
92
|
+
'value' => hits.count,
|
93
|
+
'relation' => 'eq'
|
94
|
+
},
|
95
|
+
'max_score' => 1.0,
|
96
|
+
'hits' => hits.each_with_index.map do |hit, i|
|
97
|
+
{
|
98
|
+
'_index' => index.index_name,
|
99
|
+
'_type' => '_doc',
|
100
|
+
'_id' => (i + 1).to_s,
|
101
|
+
'_score' => 3.14,
|
102
|
+
'_source' => hit
|
103
|
+
}
|
104
|
+
end
|
105
|
+
}
|
106
|
+
}
|
107
|
+
|
108
|
+
mock_elasticsearch_response(index, raw_response, &block)
|
109
|
+
end
|
110
|
+
|
111
|
+
# Check the assertion that actual Elasticsearch query is rendered
|
112
|
+
# to the expected query
|
113
|
+
#
|
114
|
+
# @param query [::Query] the actual Elasticsearch query.
|
115
|
+
# @param expected_query [Hash] expected query.
|
116
|
+
#
|
117
|
+
# @return [Boolean]
|
118
|
+
# True - in the case when actual Elasticsearch query is rendered to the expected query.
|
119
|
+
# False - in the opposite case.
|
120
|
+
#
|
121
|
+
def assert_elasticsearch_query(query, expected_query)
|
122
|
+
actual_query = query.render
|
123
|
+
assert_equal expected_query, actual_query, "got #{actual_query.inspect} instead of expected query."
|
124
|
+
end
|
125
|
+
|
49
126
|
module ClassMethods
|
50
127
|
# Declare that all tests in this file require real indexing, always.
|
51
128
|
# In my completely unscientific experiments, this roughly doubled test runtime.
|
data/lib/chewy/rake_helper.rb
CHANGED
@@ -271,7 +271,7 @@ module Chewy
|
|
271
271
|
|
272
272
|
def reset_one(index, output, parallel: false)
|
273
273
|
output.puts "Resetting #{index}"
|
274
|
-
index.reset!((Time.now.to_f * 1000).round, parallel: parallel
|
274
|
+
index.reset!((Time.now.to_f * 1000).round, parallel: parallel)
|
275
275
|
end
|
276
276
|
end
|
277
277
|
end
|
data/lib/chewy/rspec.rb
CHANGED
@@ -0,0 +1,12 @@
|
|
1
|
+
# Rspec helper to compare request and expected query
|
2
|
+
# To use it - add `require 'chewy/rspec/build_query'` to the `spec_helper.rb`
|
3
|
+
# Simple usage - just pass expected response as argument
|
4
|
+
# and then call needed query.
|
5
|
+
#
|
6
|
+
# expect { method1.method2...methodN }.to build_query(expected_query)
|
7
|
+
#
|
8
|
+
RSpec::Matchers.define :build_query do |expected_query = {}|
|
9
|
+
match do |request|
|
10
|
+
request.render == expected_query
|
11
|
+
end
|
12
|
+
end
|
@@ -0,0 +1,55 @@
|
|
1
|
+
module Chewy
|
2
|
+
module Rspec
|
3
|
+
module Helpers
|
4
|
+
extend ActiveSupport::Concern
|
5
|
+
# Rspec helper to mock elasticsearch response
|
6
|
+
# To use it - add `require 'chewy/rspec'` to the `spec_helper.rb`
|
7
|
+
#
|
8
|
+
# mock_elasticsearch_response(CitiesIndex, raw_response)
|
9
|
+
# expect(CitiesIndex.query({}).hits).to eq(hits)
|
10
|
+
#
|
11
|
+
def mock_elasticsearch_response(index, raw_response)
|
12
|
+
mocked_request = Chewy::Search::Request.new(index)
|
13
|
+
allow(Chewy::Search::Request).to receive(:new).and_return(mocked_request)
|
14
|
+
allow(mocked_request).to receive(:perform).and_return(raw_response)
|
15
|
+
end
|
16
|
+
|
17
|
+
# Rspec helper to mock Elasticsearch response source
|
18
|
+
# To use it - add `require 'chewy/rspec'` to the `spec_helper.rb`
|
19
|
+
#
|
20
|
+
# mock_elasticsearch_response_sources(CitiesIndex, sources)
|
21
|
+
# expect(CitiesIndex.query({}).hits).to eq(hits)
|
22
|
+
#
|
23
|
+
def mock_elasticsearch_response_sources(index, hits)
|
24
|
+
raw_response = {
|
25
|
+
'took' => 4,
|
26
|
+
'timed_out' => false,
|
27
|
+
'_shards' => {
|
28
|
+
'total' => 1,
|
29
|
+
'successful' => 1,
|
30
|
+
'skipped' => 0,
|
31
|
+
'failed' => 0
|
32
|
+
},
|
33
|
+
'hits' => {
|
34
|
+
'total' => {
|
35
|
+
'value' => hits.count,
|
36
|
+
'relation' => 'eq'
|
37
|
+
},
|
38
|
+
'max_score' => 1.0,
|
39
|
+
'hits' => hits.each_with_index.map do |hit, i|
|
40
|
+
{
|
41
|
+
'_index' => index.index_name,
|
42
|
+
'_type' => '_doc',
|
43
|
+
'_id' => (i + 1).to_s,
|
44
|
+
'_score' => 3.14,
|
45
|
+
'_source' => hit
|
46
|
+
}
|
47
|
+
end
|
48
|
+
}
|
49
|
+
}
|
50
|
+
|
51
|
+
mock_elasticsearch_response(index, raw_response)
|
52
|
+
end
|
53
|
+
end
|
54
|
+
end
|
55
|
+
end
|
@@ -17,7 +17,7 @@ module Chewy
|
|
17
17
|
# @param other_value [Object] any acceptable storage value
|
18
18
|
# @return [Object] updated value
|
19
19
|
def update!(other_value)
|
20
|
-
value.
|
20
|
+
value.concat(normalize(other_value))
|
21
21
|
end
|
22
22
|
|
23
23
|
# Size requires specialized rendering logic, it should return
|
@@ -28,20 +28,7 @@ module Chewy
|
|
28
28
|
def render
|
29
29
|
return if value.blank?
|
30
30
|
|
31
|
-
sort
|
32
|
-
options ? {field => options} : field
|
33
|
-
end
|
34
|
-
{sort: sort}
|
35
|
-
end
|
36
|
-
|
37
|
-
# Comparison also reqires additional logic. Hashes are compared
|
38
|
-
# orderlessly, but for `sort` parameter oder is important, so we
|
39
|
-
# compare hash key collections additionally.
|
40
|
-
#
|
41
|
-
# @see Chewy::Search::Parameters::Storage#==
|
42
|
-
# @return [true, false]
|
43
|
-
def ==(other)
|
44
|
-
super && value.keys == other.value.keys
|
31
|
+
{sort: value}
|
45
32
|
end
|
46
33
|
|
47
34
|
private
|
@@ -49,13 +36,13 @@ module Chewy
|
|
49
36
|
def normalize(value)
|
50
37
|
case value
|
51
38
|
when Array
|
52
|
-
value.each_with_object(
|
53
|
-
res.
|
39
|
+
value.each_with_object([]) do |sv, res|
|
40
|
+
res.concat(normalize(sv))
|
54
41
|
end
|
55
42
|
when Hash
|
56
|
-
value.stringify_keys
|
43
|
+
[value.stringify_keys]
|
57
44
|
else
|
58
|
-
value.present? ?
|
45
|
+
value.present? ? [value.to_s] : []
|
59
46
|
end
|
60
47
|
end
|
61
48
|
end
|
data/lib/chewy/version.rb
CHANGED
data/migration_guide.md
CHANGED
@@ -9,7 +9,7 @@ Chewy alongside a matching Elasticsearch version.
|
|
9
9
|
In order to upgrade Chewy 6/Elasticsearch 6 to Chewy 7/Elasticsearch 7 in the most seamless manner you have to:
|
10
10
|
|
11
11
|
* Upgrade to the latest 6.x stable releases, namely Chewy 6.0, Elasticsearch 6.8
|
12
|
-
* Study carefully [Breaking changes in 7.0](https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.
|
12
|
+
* Study carefully [Breaking changes in 7.0](https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html), make sure your application conforms.
|
13
13
|
* Run your test suite on Chewy 7.0 / Elasticsearch 7
|
14
14
|
* Run manual tests on Chewy 7.0 / Elasticsearch 7
|
15
15
|
* Upgrade to Chewy 7.0
|
@@ -502,7 +502,7 @@ describe Chewy::Index::Actions do
|
|
502
502
|
specify do
|
503
503
|
expect(CitiesIndex.client.indices).to receive(:put_settings).with(index: name, body: before_import_body).once
|
504
504
|
expect(CitiesIndex.client.indices).to receive(:put_settings).with(index: name, body: after_import_body).once
|
505
|
-
expect(CitiesIndex).to receive(:import).with(suffix: suffix,
|
505
|
+
expect(CitiesIndex).to receive(:import).with(suffix: suffix, journal: false, refresh: false).and_call_original
|
506
506
|
expect(CitiesIndex.reset!(suffix)).to eq(true)
|
507
507
|
end
|
508
508
|
|
@@ -525,7 +525,7 @@ describe Chewy::Index::Actions do
|
|
525
525
|
.to receive(:put_settings).with(index: name, body: before_import_body).once
|
526
526
|
expect(CitiesIndex.client.indices).to receive(:put_settings).with(index: name, body: after_import_body).once
|
527
527
|
expect(CitiesIndex)
|
528
|
-
.to receive(:import).with(suffix: suffix,
|
528
|
+
.to receive(:import).with(suffix: suffix, journal: false, refresh: false).and_call_original
|
529
529
|
expect(CitiesIndex.reset!(suffix)).to eq(true)
|
530
530
|
end
|
531
531
|
|
@@ -541,7 +541,7 @@ describe Chewy::Index::Actions do
|
|
541
541
|
let(:reset_disable_refresh_interval) { false }
|
542
542
|
specify do
|
543
543
|
expect(CitiesIndex.client.indices).not_to receive(:put_settings)
|
544
|
-
expect(CitiesIndex).to receive(:import).with(suffix: suffix,
|
544
|
+
expect(CitiesIndex).to receive(:import).with(suffix: suffix, journal: false, refresh: true).and_call_original
|
545
545
|
expect(CitiesIndex.reset!(suffix)).to eq(true)
|
546
546
|
end
|
547
547
|
end
|
@@ -568,7 +568,7 @@ describe Chewy::Index::Actions do
|
|
568
568
|
specify do
|
569
569
|
expect(CitiesIndex.client.indices).to receive(:put_settings).with(index: name, body: before_import_body).once
|
570
570
|
expect(CitiesIndex.client.indices).to receive(:put_settings).with(index: name, body: after_import_body).once
|
571
|
-
expect(CitiesIndex).to receive(:import).with(suffix: suffix,
|
571
|
+
expect(CitiesIndex).to receive(:import).with(suffix: suffix, journal: false, refresh: true).and_call_original
|
572
572
|
expect(CitiesIndex.reset!(suffix)).to eq(true)
|
573
573
|
end
|
574
574
|
end
|
@@ -577,7 +577,7 @@ describe Chewy::Index::Actions do
|
|
577
577
|
let(:reset_no_replicas) { false }
|
578
578
|
specify do
|
579
579
|
expect(CitiesIndex.client.indices).not_to receive(:put_settings)
|
580
|
-
expect(CitiesIndex).to receive(:import).with(suffix: suffix,
|
580
|
+
expect(CitiesIndex).to receive(:import).with(suffix: suffix, journal: false, refresh: true).and_call_original
|
581
581
|
expect(CitiesIndex.reset!(suffix)).to eq(true)
|
582
582
|
end
|
583
583
|
end
|
@@ -667,7 +667,7 @@ describe Chewy::Index::Actions do
|
|
667
667
|
specify do
|
668
668
|
expect(CitiesIndex)
|
669
669
|
.to receive(:import)
|
670
|
-
.with(suffix: 'suffix',
|
670
|
+
.with(suffix: 'suffix', parallel: true, journal: false, refresh: true)
|
671
671
|
.once.and_return(true)
|
672
672
|
expect(CitiesIndex.reset!('suffix', parallel: true)).to eq(true)
|
673
673
|
end
|
@@ -64,8 +64,8 @@ describe Chewy::Index::Import::Routine do
|
|
64
64
|
|
65
65
|
describe '#parallel_options' do
|
66
66
|
specify { expect(described_class.new(CitiesIndex).parallel_options).to be_nil }
|
67
|
-
specify { expect(described_class.new(CitiesIndex, parallel: true).parallel_options).to eq({
|
68
|
-
specify { expect(described_class.new(CitiesIndex, parallel: 3).parallel_options).to eq(
|
67
|
+
specify { expect(described_class.new(CitiesIndex, parallel: true).parallel_options).to eq({}) }
|
68
|
+
specify { expect(described_class.new(CitiesIndex, parallel: 3).parallel_options).to eq(in_processes: 3) }
|
69
69
|
specify do
|
70
70
|
expect(described_class.new(CitiesIndex, parallel: {in_threads: 2}).parallel_options).to eq(in_threads: 2)
|
71
71
|
end
|
@@ -493,35 +493,6 @@ describe Chewy::Index::Import do
|
|
493
493
|
|
494
494
|
it_behaves_like 'importing'
|
495
495
|
end
|
496
|
-
|
497
|
-
context 'with progressbar output' do
|
498
|
-
let(:mocked_progressbar) { Struct.new(:progress, :total).new(0, 100) }
|
499
|
-
|
500
|
-
it 'imports tracks progress in a single batch' do
|
501
|
-
expect(ProgressBar).to receive(:create).and_return(mocked_progressbar)
|
502
|
-
expect(mocked_progressbar).to receive(:progress).at_least(:once).and_call_original
|
503
|
-
expect(CitiesIndex).to receive(:import_parallel).and_call_original
|
504
|
-
|
505
|
-
CitiesIndex.import(parallel: 1, progressbar: true)
|
506
|
-
|
507
|
-
expect(mocked_progressbar.progress).to eq(3)
|
508
|
-
expect(mocked_progressbar.total).to eq(3)
|
509
|
-
end
|
510
|
-
|
511
|
-
it 'imports tracks progress in many batches' do
|
512
|
-
expect(ProgressBar).to receive(:create).and_return(mocked_progressbar)
|
513
|
-
expect(mocked_progressbar).to receive(:progress).at_least(:once).and_call_original
|
514
|
-
expect(CitiesIndex).to receive(:import_parallel).and_call_original
|
515
|
-
|
516
|
-
batches = City.pluck(:id).map { |id| [id] }
|
517
|
-
expect(CitiesIndex.adapter).to receive(:import_references).and_return(batches)
|
518
|
-
|
519
|
-
CitiesIndex.import(parallel: 3, progressbar: true)
|
520
|
-
|
521
|
-
expect(mocked_progressbar.progress).to eq(3)
|
522
|
-
expect(mocked_progressbar.total).to eq(3)
|
523
|
-
end
|
524
|
-
end
|
525
496
|
end
|
526
497
|
|
527
498
|
describe '.import!', :orm do
|
@@ -535,9 +506,7 @@ describe Chewy::Index::Import do
|
|
535
506
|
end
|
536
507
|
end
|
537
508
|
|
538
|
-
specify
|
539
|
-
expect { CitiesIndex.import!(dummy_cities) }.to raise_error Chewy::ImportFailed
|
540
|
-
end
|
509
|
+
specify { expect { CitiesIndex.import!(dummy_cities) }.to raise_error Chewy::ImportFailed }
|
541
510
|
end
|
542
511
|
end
|
543
512
|
|
@@ -12,6 +12,10 @@ describe :minitest_helper do
|
|
12
12
|
|
13
13
|
include ::Chewy::Minitest::Helpers
|
14
14
|
|
15
|
+
def assert_equal(expected, actual, message)
|
16
|
+
raise message unless expected == actual
|
17
|
+
end
|
18
|
+
|
15
19
|
before do
|
16
20
|
Chewy.massacre
|
17
21
|
end
|
@@ -22,6 +26,112 @@ describe :minitest_helper do
|
|
22
26
|
end
|
23
27
|
end
|
24
28
|
|
29
|
+
describe 'mock_elasticsearch_response' do
|
30
|
+
let(:hits) do
|
31
|
+
[
|
32
|
+
{
|
33
|
+
'_index' => 'dummies',
|
34
|
+
'_type' => '_doc',
|
35
|
+
'_id' => '1',
|
36
|
+
'_score' => 3.14,
|
37
|
+
'_source' => source
|
38
|
+
}
|
39
|
+
]
|
40
|
+
end
|
41
|
+
|
42
|
+
let(:source) { {'name' => 'some_name'} }
|
43
|
+
let(:sources) { [source] }
|
44
|
+
|
45
|
+
context 'mocks by raw response' do
|
46
|
+
let(:raw_response) do
|
47
|
+
{
|
48
|
+
'took' => 4,
|
49
|
+
'timed_out' => false,
|
50
|
+
'_shards' => {
|
51
|
+
'total' => 1,
|
52
|
+
'successful' => 1,
|
53
|
+
'skipped' => 0,
|
54
|
+
'failed' => 0
|
55
|
+
},
|
56
|
+
'hits' => {
|
57
|
+
'total' => {
|
58
|
+
'value' => 1,
|
59
|
+
'relation' => 'eq'
|
60
|
+
},
|
61
|
+
'max_score' => 1.0,
|
62
|
+
'hits' => hits
|
63
|
+
}
|
64
|
+
}
|
65
|
+
end
|
66
|
+
|
67
|
+
specify do
|
68
|
+
mock_elasticsearch_response(DummiesIndex, raw_response) do
|
69
|
+
expect(DummiesIndex.query({}).hits).to eq(hits)
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
context 'mocks by response sources' do
|
75
|
+
specify do
|
76
|
+
mock_elasticsearch_response_sources(DummiesIndex, sources) do
|
77
|
+
expect(DummiesIndex.query({}).hits).to eq(hits)
|
78
|
+
end
|
79
|
+
end
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
describe 'assert correct elasticsearch query' do
|
84
|
+
let(:query) do
|
85
|
+
DummiesIndex.filter.should { multi_match foo: 'bar' }.filter { match foo: 'bar' }
|
86
|
+
end
|
87
|
+
|
88
|
+
let(:expected_query) do
|
89
|
+
{
|
90
|
+
index: ['dummies'],
|
91
|
+
body: {
|
92
|
+
query: {
|
93
|
+
bool: {
|
94
|
+
filter: {
|
95
|
+
bool: {
|
96
|
+
must: {
|
97
|
+
match: {foo: 'bar'}
|
98
|
+
},
|
99
|
+
should: {
|
100
|
+
multi_match: {foo: 'bar'}
|
101
|
+
}
|
102
|
+
}
|
103
|
+
}
|
104
|
+
}
|
105
|
+
}
|
106
|
+
}
|
107
|
+
}
|
108
|
+
end
|
109
|
+
|
110
|
+
context 'will be built' do
|
111
|
+
specify do
|
112
|
+
expect { assert_elasticsearch_query(query, expected_query) }.not_to raise_error
|
113
|
+
end
|
114
|
+
end
|
115
|
+
|
116
|
+
context 'will not be built' do
|
117
|
+
let(:unexpected_query) do
|
118
|
+
{
|
119
|
+
index: ['what?'],
|
120
|
+
body: {}
|
121
|
+
}
|
122
|
+
end
|
123
|
+
|
124
|
+
let(:unexpected_query_error_message) do
|
125
|
+
'got {:index=>["dummies"], :body=>{:query=>{:bool=>{:filter=>{:bool=>{:must=>{:match=>{:foo=>"bar"}}, :should=>{:multi_match=>{:foo=>"bar"}}}}}}}} instead of expected query.'
|
126
|
+
end
|
127
|
+
|
128
|
+
specify do
|
129
|
+
expect { assert_elasticsearch_query(query, unexpected_query) }
|
130
|
+
.to raise_error(RuntimeError, unexpected_query_error_message)
|
131
|
+
end
|
132
|
+
end
|
133
|
+
end
|
134
|
+
|
25
135
|
context 'assert_indexes' do
|
26
136
|
specify 'doesn\'t fail when index updates correctly' do
|
27
137
|
expect do
|
@@ -0,0 +1,34 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe :build_query do
|
4
|
+
before do
|
5
|
+
stub_model(:city)
|
6
|
+
stub_index(:cities) { index_scope City }
|
7
|
+
CitiesIndex.create
|
8
|
+
end
|
9
|
+
|
10
|
+
let(:expected_query) do
|
11
|
+
{
|
12
|
+
index: ['cities'],
|
13
|
+
body: {
|
14
|
+
query: {
|
15
|
+
match: {name: 'name'}
|
16
|
+
}
|
17
|
+
}
|
18
|
+
}
|
19
|
+
end
|
20
|
+
let(:dummy_query) { {match: {name: 'name'}} }
|
21
|
+
let(:unexpected_query) { {match: {name: 'name'}} }
|
22
|
+
|
23
|
+
context 'build expected query' do
|
24
|
+
specify do
|
25
|
+
expect(CitiesIndex.query(dummy_query)).to build_query(expected_query)
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
context 'not to build unexpected query' do
|
30
|
+
specify do
|
31
|
+
expect(CitiesIndex.query(dummy_query)).not_to build_query(unexpected_query)
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
@@ -0,0 +1,61 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe :rspec_helper do
|
4
|
+
include ::Chewy::Rspec::Helpers
|
5
|
+
|
6
|
+
before do
|
7
|
+
stub_model(:city)
|
8
|
+
stub_index(:cities) { index_scope City }
|
9
|
+
CitiesIndex.create
|
10
|
+
end
|
11
|
+
|
12
|
+
let(:hits) do
|
13
|
+
[
|
14
|
+
{
|
15
|
+
'_index' => 'cities',
|
16
|
+
'_type' => '_doc',
|
17
|
+
'_id' => '1',
|
18
|
+
'_score' => 3.14,
|
19
|
+
'_source' => source
|
20
|
+
}
|
21
|
+
]
|
22
|
+
end
|
23
|
+
|
24
|
+
let(:source) { {'name' => 'some_name'} }
|
25
|
+
let(:sources) { [source] }
|
26
|
+
|
27
|
+
context :mock_elasticsearch_response do
|
28
|
+
let(:raw_response) do
|
29
|
+
{
|
30
|
+
'took' => 4,
|
31
|
+
'timed_out' => false,
|
32
|
+
'_shards' => {
|
33
|
+
'total' => 1,
|
34
|
+
'successful' => 1,
|
35
|
+
'skipped' => 0,
|
36
|
+
'failed' => 0
|
37
|
+
},
|
38
|
+
'hits' => {
|
39
|
+
'total' => {
|
40
|
+
'value' => 1,
|
41
|
+
'relation' => 'eq'
|
42
|
+
},
|
43
|
+
'max_score' => 1.0,
|
44
|
+
'hits' => hits
|
45
|
+
}
|
46
|
+
}
|
47
|
+
end
|
48
|
+
|
49
|
+
specify do
|
50
|
+
mock_elasticsearch_response(CitiesIndex, raw_response)
|
51
|
+
expect(CitiesIndex.query({}).hits).to eq(hits)
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
context :mock_elasticsearch_response_sources do
|
56
|
+
specify do
|
57
|
+
mock_elasticsearch_response_sources(CitiesIndex, sources)
|
58
|
+
expect(CitiesIndex.query({}).hits).to eq(hits)
|
59
|
+
end
|
60
|
+
end
|
61
|
+
end
|
@@ -4,26 +4,29 @@ describe Chewy::Search::Parameters::Order do
|
|
4
4
|
subject { described_class.new(%i[foo bar]) }
|
5
5
|
|
6
6
|
describe '#initialize' do
|
7
|
-
specify { expect(described_class.new.value).to eq(
|
8
|
-
specify { expect(described_class.new(nil).value).to eq(
|
9
|
-
specify { expect(described_class.new('').value).to eq(
|
10
|
-
specify { expect(described_class.new(42).value).to eq('42'
|
11
|
-
specify { expect(described_class.new([42, 43]).value).to eq(
|
12
|
-
specify { expect(described_class.new(
|
13
|
-
specify { expect(described_class.new([
|
7
|
+
specify { expect(described_class.new.value).to eq([]) }
|
8
|
+
specify { expect(described_class.new(nil).value).to eq([]) }
|
9
|
+
specify { expect(described_class.new('').value).to eq([]) }
|
10
|
+
specify { expect(described_class.new(42).value).to eq(['42']) }
|
11
|
+
specify { expect(described_class.new([42, 43]).value).to eq(%w[42 43]) }
|
12
|
+
specify { expect(described_class.new([42, 42]).value).to eq(%w[42 42]) }
|
13
|
+
specify { expect(described_class.new([42, [43, 44]]).value).to eq(%w[42 43 44]) }
|
14
|
+
specify { expect(described_class.new(a: 1).value).to eq([{'a' => 1}]) }
|
15
|
+
specify { expect(described_class.new(['a', {a: 1}, {a: 2}]).value).to eq(['a', {'a' => 1}, {'a' => 2}]) }
|
16
|
+
specify { expect(described_class.new(['', 43, {a: 1}]).value).to eq(['43', {'a' => 1}]) }
|
14
17
|
end
|
15
18
|
|
16
19
|
describe '#replace!' do
|
17
20
|
specify do
|
18
21
|
expect { subject.replace!(foo: {}) }
|
19
22
|
.to change { subject.value }
|
20
|
-
.from(
|
23
|
+
.from(%w[foo bar]).to([{'foo' => {}}])
|
21
24
|
end
|
22
25
|
|
23
26
|
specify do
|
24
27
|
expect { subject.replace!(nil) }
|
25
28
|
.to change { subject.value }
|
26
|
-
.from(
|
29
|
+
.from(%w[foo bar]).to([])
|
27
30
|
end
|
28
31
|
end
|
29
32
|
|
@@ -31,7 +34,7 @@ describe Chewy::Search::Parameters::Order do
|
|
31
34
|
specify do
|
32
35
|
expect { subject.update!(foo: {}) }
|
33
36
|
.to change { subject.value }
|
34
|
-
.from(
|
37
|
+
.from(%w[foo bar]).to(['foo', 'bar', {'foo' => {}}])
|
35
38
|
end
|
36
39
|
|
37
40
|
specify { expect { subject.update!(nil) }.not_to change { subject.value } }
|
@@ -41,7 +44,7 @@ describe Chewy::Search::Parameters::Order do
|
|
41
44
|
specify do
|
42
45
|
expect { subject.merge!(described_class.new(foo: {})) }
|
43
46
|
.to change { subject.value }
|
44
|
-
.from(
|
47
|
+
.from(%w[foo bar]).to(['foo', 'bar', {'foo' => {}}])
|
45
48
|
end
|
46
49
|
|
47
50
|
specify { expect { subject.merge!(described_class.new) }.not_to change { subject.value } }
|
@@ -51,6 +54,7 @@ describe Chewy::Search::Parameters::Order do
|
|
51
54
|
specify { expect(described_class.new.render).to be_nil }
|
52
55
|
specify { expect(described_class.new(:foo).render).to eq(sort: ['foo']) }
|
53
56
|
specify { expect(described_class.new([:foo, {bar: 42}, :baz]).render).to eq(sort: ['foo', {'bar' => 42}, 'baz']) }
|
57
|
+
specify { expect(described_class.new([:foo, {bar: 42}, {bar: 43}, :baz]).render).to eq(sort: ['foo', {'bar' => 42}, {'bar' => 43}, 'baz']) }
|
54
58
|
end
|
55
59
|
|
56
60
|
describe '#==' do
|
@@ -59,7 +63,10 @@ describe Chewy::Search::Parameters::Order do
|
|
59
63
|
specify { expect(described_class.new(:foo)).not_to eq(described_class.new(:bar)) }
|
60
64
|
specify { expect(described_class.new(%i[foo bar])).to eq(described_class.new(%i[foo bar])) }
|
61
65
|
specify { expect(described_class.new(%i[foo bar])).not_to eq(described_class.new(%i[bar foo])) }
|
66
|
+
specify { expect(described_class.new(%i[foo foo])).not_to eq(described_class.new(%i[foo])) }
|
62
67
|
specify { expect(described_class.new(foo: {a: 42})).to eq(described_class.new(foo: {a: 42})) }
|
63
68
|
specify { expect(described_class.new(foo: {a: 42})).not_to eq(described_class.new(foo: {b: 42})) }
|
69
|
+
specify { expect(described_class.new(['foo', {'foo' => 42}])).not_to eq(described_class.new([{'foo' => 42}, 'foo'])) }
|
70
|
+
specify { expect(described_class.new([{'foo' => 42}, {'foo' => 43}])).not_to eq(described_class.new([{'foo' => 43}, {'foo' => 42}])) }
|
64
71
|
end
|
65
72
|
end
|
@@ -13,7 +13,7 @@ describe Chewy::Search::Parameters do
|
|
13
13
|
|
14
14
|
specify { expect(subject.storages[:limit]).to equal(limit) }
|
15
15
|
specify { expect(subject.storages[:limit].value).to eq(3) }
|
16
|
-
specify { expect(subject.storages[:order].value).to eq('foo'
|
16
|
+
specify { expect(subject.storages[:order].value).to eq(['foo']) }
|
17
17
|
|
18
18
|
specify { expect { described_class.new(offset: limit) }.to raise_error(TypeError) }
|
19
19
|
end
|
@@ -177,7 +177,7 @@ describe Chewy::Search::Request do
|
|
177
177
|
describe '#order' do
|
178
178
|
specify { expect(subject.order(:foo).render[:body]).to include(sort: ['foo']) }
|
179
179
|
specify { expect(subject.order(foo: 42).order(nil).render[:body]).to include(sort: ['foo' => 42]) }
|
180
|
-
specify { expect(subject.order(foo: 42).order(foo: 43).render[:body]).to include(sort: ['foo' => 43]) }
|
180
|
+
specify { expect(subject.order(foo: 42).order(foo: 43).render[:body]).to include(sort: [{'foo' => 42}, {'foo' => 43}]) }
|
181
181
|
specify { expect(subject.order(:foo).order(:bar, :baz).render[:body]).to include(sort: %w[foo bar baz]) }
|
182
182
|
specify { expect(subject.order(nil).render[:body]).to be_blank }
|
183
183
|
specify { expect { subject.order(:foo) }.not_to change { subject.render } }
|
data/spec/spec_helper.rb
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
require 'database_cleaner'
|
2
2
|
|
3
|
-
ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: 'file::memory:?cache=shared', pool:
|
3
|
+
ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: 'file::memory:?cache=shared', pool: 10)
|
4
4
|
ActiveRecord::Base.logger = Logger.new('/dev/null')
|
5
5
|
if ActiveRecord::Base.respond_to?(:raise_in_transactional_callbacks)
|
6
6
|
ActiveRecord::Base.raise_in_transactional_callbacks = true
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: chewy
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 7.2.
|
4
|
+
version: 7.2.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Toptal, LLC
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2021-05-
|
12
|
+
date: 2021-05-24 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: database_cleaner
|
@@ -185,14 +185,14 @@ dependencies:
|
|
185
185
|
requirements:
|
186
186
|
- - ">="
|
187
187
|
- !ruby/object:Gem::Version
|
188
|
-
version:
|
188
|
+
version: 7.12.0
|
189
189
|
type: :runtime
|
190
190
|
prerelease: false
|
191
191
|
version_requirements: !ruby/object:Gem::Requirement
|
192
192
|
requirements:
|
193
193
|
- - ">="
|
194
194
|
- !ruby/object:Gem::Version
|
195
|
-
version:
|
195
|
+
version: 7.12.0
|
196
196
|
- !ruby/object:Gem::Dependency
|
197
197
|
name: elasticsearch-dsl
|
198
198
|
requirement: !ruby/object:Gem::Requirement
|
@@ -207,20 +207,6 @@ dependencies:
|
|
207
207
|
- - ">="
|
208
208
|
- !ruby/object:Gem::Version
|
209
209
|
version: '0'
|
210
|
-
- !ruby/object:Gem::Dependency
|
211
|
-
name: ruby-progressbar
|
212
|
-
requirement: !ruby/object:Gem::Requirement
|
213
|
-
requirements:
|
214
|
-
- - ">="
|
215
|
-
- !ruby/object:Gem::Version
|
216
|
-
version: '0'
|
217
|
-
type: :runtime
|
218
|
-
prerelease: false
|
219
|
-
version_requirements: !ruby/object:Gem::Requirement
|
220
|
-
requirements:
|
221
|
-
- - ">="
|
222
|
-
- !ruby/object:Gem::Version
|
223
|
-
version: '0'
|
224
210
|
description: Chewy provides functionality for Elasticsearch index handling, documents
|
225
211
|
import mappings and chainable query DSL
|
226
212
|
email:
|
@@ -272,7 +258,6 @@ files:
|
|
272
258
|
- lib/chewy/index/import/bulk_request.rb
|
273
259
|
- lib/chewy/index/import/journal_builder.rb
|
274
260
|
- lib/chewy/index/import/routine.rb
|
275
|
-
- lib/chewy/index/import/thread_safe_progress_bar.rb
|
276
261
|
- lib/chewy/index/mapping.rb
|
277
262
|
- lib/chewy/index/observe.rb
|
278
263
|
- lib/chewy/index/settings.rb
|
@@ -290,6 +275,8 @@ files:
|
|
290
275
|
- lib/chewy/rake_helper.rb
|
291
276
|
- lib/chewy/repository.rb
|
292
277
|
- lib/chewy/rspec.rb
|
278
|
+
- lib/chewy/rspec/build_query.rb
|
279
|
+
- lib/chewy/rspec/helpers.rb
|
293
280
|
- lib/chewy/rspec/update_index.rb
|
294
281
|
- lib/chewy/runtime.rb
|
295
282
|
- lib/chewy/runtime/version.rb
|
@@ -379,6 +366,8 @@ files:
|
|
379
366
|
- spec/chewy/multi_search_spec.rb
|
380
367
|
- spec/chewy/rake_helper_spec.rb
|
381
368
|
- spec/chewy/repository_spec.rb
|
369
|
+
- spec/chewy/rspec/build_query_spec.rb
|
370
|
+
- spec/chewy/rspec/helpers_spec.rb
|
382
371
|
- spec/chewy/rspec/update_index_spec.rb
|
383
372
|
- spec/chewy/runtime/version_spec.rb
|
384
373
|
- spec/chewy/runtime_spec.rb
|
@@ -487,6 +476,8 @@ test_files:
|
|
487
476
|
- spec/chewy/multi_search_spec.rb
|
488
477
|
- spec/chewy/rake_helper_spec.rb
|
489
478
|
- spec/chewy/repository_spec.rb
|
479
|
+
- spec/chewy/rspec/build_query_spec.rb
|
480
|
+
- spec/chewy/rspec/helpers_spec.rb
|
490
481
|
- spec/chewy/rspec/update_index_spec.rb
|
491
482
|
- spec/chewy/runtime/version_spec.rb
|
492
483
|
- spec/chewy/runtime_spec.rb
|
@@ -1,40 +0,0 @@
|
|
1
|
-
module Chewy
|
2
|
-
class Index
|
3
|
-
module Import
|
4
|
-
# This class performs the threading for parallel import to avoid concurrency during progressbar output.
|
5
|
-
#
|
6
|
-
# @see Chewy::Type::Import::ClassMethods#import with `parallel: true` option
|
7
|
-
class ThreadSafeProgressBar
|
8
|
-
def initialize(enabled)
|
9
|
-
@enabled = enabled
|
10
|
-
|
11
|
-
return unless @enabled
|
12
|
-
|
13
|
-
@mutex = Mutex.new
|
14
|
-
@released = false
|
15
|
-
@progressbar = ProgressBar.create total: nil
|
16
|
-
Thread.new do
|
17
|
-
ActiveRecord::Base.connection_pool.with_connection do
|
18
|
-
@mutex.synchronize { @released = true }
|
19
|
-
@progressbar.total = yield
|
20
|
-
end
|
21
|
-
end
|
22
|
-
end
|
23
|
-
|
24
|
-
def increment(value)
|
25
|
-
return unless @enabled
|
26
|
-
|
27
|
-
@mutex.synchronize do
|
28
|
-
@progressbar.progress += value
|
29
|
-
end
|
30
|
-
end
|
31
|
-
|
32
|
-
def wait_until_ready
|
33
|
-
return true unless @enabled
|
34
|
-
|
35
|
-
@mutex.synchronize { @released } until @released
|
36
|
-
end
|
37
|
-
end
|
38
|
-
end
|
39
|
-
end
|
40
|
-
end
|