es-elasticity 0.14.1 → 1.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -0
- data/Gemfile.lock +1 -1
- data/README.md +42 -38
- data/es-elasticity.gemspec +2 -2
- data/lib/elasticity/base_document.rb +1 -1
- data/lib/elasticity/bulk.rb +18 -18
- data/lib/elasticity/index_config.rb +17 -5
- data/lib/elasticity/index_mapper.rb +8 -9
- data/lib/elasticity/railtie.rb +1 -1
- data/lib/elasticity/search.rb +4 -4
- data/lib/elasticity/strategies/alias_index.rb +26 -32
- data/lib/elasticity/strategies/single_index.rb +14 -20
- data/lib/elasticity/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 8c13e2b279bf48ef4d133c348954b1b8b19c07a9e3f8af7e539d3bf0e359f485
|
4
|
+
data.tar.gz: d9ca4629b1c2c091fb77cac05213b944ef4d877a845f3d6c8e606a37cfbc6fec
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 01d086f62d915982aeae1fa1135836f8442c6eb578ff8c6daa1bfcb8bd4d70b0e54647a38e3b8d22fac0a477afc6ff11cfcc832215acca5d93f8db4d70344065
|
7
|
+
data.tar.gz: 8df1713fd97df4ce332d48a72b6ae9e364a591cc17ca19f6e6a3c621157f595936c84122b8d515ccc18044e2f272da891e2ab22bce4445cc24c00cd112cb74c0
|
data/CHANGELOG.md
CHANGED
@@ -6,6 +6,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
|
7
7
|
## [Unreleased]
|
8
8
|
|
9
|
+
## [1.0.1] - 2022-08-16
|
10
|
+
### Changed
|
11
|
+
- Revert the additional `frozen_string_literal` updates in 1.0.0 due to some issues with some classes. We'll update these in a separate release to isolate the problem.
|
12
|
+
|
13
|
+
## [1.0.0] - 2022-08-09
|
14
|
+
### Changed
|
15
|
+
- Consistently use quotations
|
16
|
+
- Update API to remove deprecation warnings for Elasticsearch 7.X.X in preparation for upgrade to 8.
|
17
|
+
- Freeze string literals
|
18
|
+
- Remove `_type` from query/create/update/delete methods
|
19
|
+
- Remove `use_new_timestamp_format` config
|
20
|
+
- Remove `include_type_name_on_create` config
|
21
|
+
|
9
22
|
## [0.14.1] - 2022-08-09
|
10
23
|
### Changed
|
11
24
|
- Don't publish pre-release builds to RubyGems
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -29,9 +29,10 @@ Or install it yourself as:
|
|
29
29
|
## Usage
|
30
30
|
|
31
31
|
### Version Support
|
32
|
-
This gem has [elasticsearch-ruby](https://github.com/elastic/elasticsearch-ruby) as a dependency. In order to use different versions of elasticsearch you will need to match your version of elasticsearch-ruby to the version of elasticsearch you want to use ([see here](https://github.com/elastic/elasticsearch-ruby#compatibility). Elasticity should work across all versions of elastisearch-ruby, although they have not all been tested so there are likely edge cases.
|
33
32
|
|
34
|
-
|
33
|
+
This gem has [elasticsearch-ruby](https://github.com/elastic/elasticsearch-ruby) as a dependency. In order to use different versions of elasticsearch you will need to match your version of elasticsearch-ruby to the version of elasticsearch you want to use ([see here](https://github.com/elastic/elasticsearch-ruby#compatibility). Elasticity should work across all versions of elastisearch-ruby, although they have not all been tested so there are likely edge cases.
|
34
|
+
|
35
|
+
Currently tests are run on CirlceCI against elasticsearch 7.17.3 with elasticsearch-ruby 7.17.1.
|
35
36
|
|
36
37
|
### Configuration
|
37
38
|
|
@@ -149,7 +150,6 @@ documents = [
|
|
149
150
|
Search::User.bulk_update(documents)
|
150
151
|
```
|
151
152
|
|
152
|
-
|
153
153
|
### Searching
|
154
154
|
|
155
155
|
Class methods have access to the `search` method, which returns a lazy evaluated search. That means that the search will only be performed when the data is necessary, not when the `search` method is called.
|
@@ -194,7 +194,8 @@ adults = adults.active_records(User)
|
|
194
194
|
#### Search Args
|
195
195
|
|
196
196
|
##### explain: true
|
197
|
-
|
197
|
+
|
198
|
+
For `search` definitions we support passing `{ explain: true }` to the search as a second argument in order to surface the reason a search result was returned.
|
198
199
|
|
199
200
|
```ruby
|
200
201
|
# example in single search
|
@@ -211,6 +212,7 @@ multi = Elasticity::MultiSearch.new do |m|
|
|
211
212
|
m.add(:c, search_c, documents: ::SearchDoc::C)
|
212
213
|
end
|
213
214
|
```
|
215
|
+
|
214
216
|
For more information about the `active_records` method, read [ActiveRecord integration](#activerecord-integration).
|
215
217
|
|
216
218
|
### Segmented Documents
|
@@ -324,15 +326,15 @@ When the mapping needs to change, a hot remapping can be performed by doing the
|
|
324
326
|
1. Create a new index with the new mapping;
|
325
327
|
2. change the update alias to point to the new index, and change main alias to point to both indexes; at this point it will look something like this:
|
326
328
|
|
327
|
-
|
328
|
-
|
329
|
-
|
330
|
-
|
331
|
-
|
332
|
-
|
333
|
-
|
334
|
-
|
335
|
-
|
329
|
+
```
|
330
|
+
|¯¯¯¯¯¯¯¯¯¯¯¯¯|----------------------> |¯¯¯¯¯¯¯¯¯¯¯¯¯|
|
331
|
+
| MainAlias | | Old Index |
|
332
|
+
|_____________|----------| |_____________|
|
333
|
+
|
|
334
|
+
|¯¯¯¯¯¯¯¯¯¯¯¯¯| |-----------> |¯¯¯¯¯¯¯¯¯¯¯¯¯|
|
335
|
+
| UpdateAlias |----------------------> | New Index |
|
336
|
+
|_____________| |_____________|
|
337
|
+
```
|
336
338
|
|
337
339
|
3. iterate over all documents on the old index, copying them to the new index;
|
338
340
|
4. change aliases to point only to the new index;
|
@@ -344,43 +346,43 @@ This is a simplified version, there are other things that happen to ensure consi
|
|
344
346
|
|
345
347
|
ActiveRecord integration is mainly a set of conventions rather than implementation, with the exception of one method that allows mapping documents back to a relation. Here is the list of conventions:
|
346
348
|
|
347
|
-
|
348
|
-
|
349
|
-
|
349
|
+
- have a class method on the document called `from_active_record` that creates a document object from the active record object;
|
350
|
+
- have a class method on the Document for rebuilding the index from the records;
|
351
|
+
- have an `after_save` and an `after_destroy` callbacks on the ActiveRecord model;
|
350
352
|
|
351
353
|
For example:
|
352
354
|
|
353
|
-
|
354
|
-
|
355
|
-
|
356
|
-
|
355
|
+
```ruby
|
356
|
+
class User < ActiveRecord::Base
|
357
|
+
after_save :update_index_document
|
358
|
+
after_destroy :delete_index_document
|
357
359
|
|
358
|
-
|
359
|
-
|
360
|
-
|
360
|
+
def update_index_document
|
361
|
+
Search::User.from_active_record(self).update
|
362
|
+
end
|
361
363
|
|
362
|
-
|
363
|
-
|
364
|
-
end
|
364
|
+
def delete_index_document
|
365
|
+
Search::User.delete(self.id)
|
365
366
|
end
|
367
|
+
end
|
366
368
|
|
367
|
-
|
368
|
-
|
369
|
+
class Search::User < Elasticity::Document
|
370
|
+
# ... configuration
|
369
371
|
|
370
|
-
|
371
|
-
|
372
|
-
|
372
|
+
def self.from_active_record(ar)
|
373
|
+
new(name: ar.name, birthdate: ar.birthdate)
|
374
|
+
end
|
373
375
|
|
374
|
-
|
375
|
-
|
376
|
+
def self.rebuild_index
|
377
|
+
self.recreate_index
|
376
378
|
|
377
|
-
|
378
|
-
|
379
|
-
|
380
|
-
end
|
379
|
+
User.find_in_batches do |batch|
|
380
|
+
documents = batch.map { |record| from_active_record(record) }
|
381
|
+
self.bulk_index(documents)
|
381
382
|
end
|
382
383
|
end
|
383
|
-
|
384
|
+
end
|
385
|
+
```
|
384
386
|
|
385
387
|
This makes the code very clear in intent, easier to see when and how things happen and under the developer control, keeping both parts very decoupled.
|
386
388
|
|
@@ -417,6 +419,7 @@ Search::User.adults.active_records(User.where(active: true))
|
|
417
419
|
```
|
418
420
|
|
419
421
|
## Upgrading from 0.7.0 to 0.8.0
|
422
|
+
|
420
423
|
The default persistence strategy changed from SingleIndex to AliasIndex in version 0.8.0 Add the following to your Document configuration to maintain the legacy behaviour.
|
421
424
|
|
422
425
|
```ruby
|
@@ -424,6 +427,7 @@ The default persistence strategy changed from SingleIndex to AliasIndex in versi
|
|
424
427
|
```
|
425
428
|
|
426
429
|
## Roadmap
|
430
|
+
|
427
431
|
- [ ] Define from_active_record interface
|
428
432
|
- [ ] Write more detailed documentation section for:
|
429
433
|
- [ ] Model definition
|
data/es-elasticity.gemspec
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
# coding: utf-8
|
2
|
-
lib = File.expand_path(
|
2
|
+
lib = File.expand_path("../lib", __FILE__)
|
3
3
|
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
-
require
|
4
|
+
require "elasticity/version"
|
5
5
|
|
6
6
|
Gem::Specification.new do |spec|
|
7
7
|
spec.name = "es-elasticity"
|
data/lib/elasticity/bulk.rb
CHANGED
@@ -5,16 +5,16 @@ module Elasticity
|
|
5
5
|
@operations = []
|
6
6
|
end
|
7
7
|
|
8
|
-
def index(index_name,
|
9
|
-
@operations << { index: { _index: index_name,
|
8
|
+
def index(index_name, id, attributes)
|
9
|
+
@operations << { index: { _index: index_name, _id: id, data: attributes }}
|
10
10
|
end
|
11
11
|
|
12
|
-
def update(index_name,
|
13
|
-
@operations << { update: { _index: index_name,
|
12
|
+
def update(index_name, id, attributes)
|
13
|
+
@operations << { update: { _index: index_name, _id: id, data: attributes }}
|
14
14
|
end
|
15
15
|
|
16
|
-
def delete(index_name,
|
17
|
-
@operations << { delete: { _index: index_name,
|
16
|
+
def delete(index_name, id)
|
17
|
+
@operations << { delete: { _index: index_name, _id: id }}
|
18
18
|
end
|
19
19
|
|
20
20
|
def execute
|
@@ -27,16 +27,16 @@ module Elasticity
|
|
27
27
|
@index_name = index_name
|
28
28
|
end
|
29
29
|
|
30
|
-
def index(
|
31
|
-
super(@index_name,
|
30
|
+
def index(id, attributes)
|
31
|
+
super(@index_name, id, attributes)
|
32
32
|
end
|
33
33
|
|
34
|
-
def update(
|
35
|
-
super(@index_name,
|
34
|
+
def update(id, attributes)
|
35
|
+
super(@index_name, id, attributes)
|
36
36
|
end
|
37
37
|
|
38
|
-
def delete(
|
39
|
-
super(@index_name,
|
38
|
+
def delete(id)
|
39
|
+
super(@index_name, id)
|
40
40
|
end
|
41
41
|
end
|
42
42
|
|
@@ -47,17 +47,17 @@ module Elasticity
|
|
47
47
|
@delete_indexes = delete_indexes
|
48
48
|
end
|
49
49
|
|
50
|
-
def index(
|
51
|
-
super(@update_alias,
|
50
|
+
def index(id, attributes)
|
51
|
+
super(@update_alias, id, attributes)
|
52
52
|
end
|
53
53
|
|
54
|
-
def update(
|
55
|
-
super(@update_alias,
|
54
|
+
def update(id, attributes)
|
55
|
+
super(@update_alias, id, attributes)
|
56
56
|
end
|
57
57
|
|
58
|
-
def delete(
|
58
|
+
def delete(id)
|
59
59
|
@delete_indexes.each do |index|
|
60
|
-
super(index,
|
60
|
+
super(index, id)
|
61
61
|
end
|
62
62
|
end
|
63
63
|
end
|
@@ -2,17 +2,18 @@ module Elasticity
|
|
2
2
|
class IndexConfig
|
3
3
|
class SubclassError < StandardError; end
|
4
4
|
|
5
|
-
SUBCLASSES_WARNING = "Indices created in Elasticsearch 6.0.0 or later may only contain a single mapping type. "
|
5
|
+
SUBCLASSES_WARNING = "Indices created in Elasticsearch 6.0.0 or later may only contain a single mapping type. "\
|
6
6
|
"Therefore, doument-type based inheritance has been disabled by Elasticity"
|
7
|
-
SUBCLASSES_ERROR = "Mapping types have been completely removed in Elasticsearch 7.0.0. "
|
7
|
+
SUBCLASSES_ERROR = "Mapping types have been completely removed in Elasticsearch 7.0.0. "\
|
8
8
|
"Therefore, doument-type based inheritance has been disabled by Elasticity"
|
9
|
-
VERSION_FOR_SUBCLASS_WARNING = "6.0.0"
|
10
|
-
VERSION_FOR_SUBCLASS_ERROR = "7.0.0"
|
9
|
+
VERSION_FOR_SUBCLASS_WARNING = "6.0.0"
|
10
|
+
VERSION_FOR_SUBCLASS_ERROR = "7.0.0"
|
11
11
|
ATTRS = [
|
12
12
|
:index_base_name, :document_type, :mapping, :strategy, :subclasses,
|
13
13
|
:settings, :use_new_timestamp_format, :include_type_name_on_create
|
14
14
|
].freeze
|
15
15
|
VALIDATABLE_ATTRS = [:index_base_name, :document_type, :strategy].freeze
|
16
|
+
DEPRECATED_ATTRS = [:use_new_timestamp_format, :include_type_name_on_create].freeze
|
16
17
|
|
17
18
|
attr_accessor(*ATTRS)
|
18
19
|
|
@@ -23,6 +24,7 @@ module Elasticity
|
|
23
24
|
@elasticity_config = elasticity_config
|
24
25
|
yield(self)
|
25
26
|
subclasses_warning_or_exception
|
27
|
+
warn_deprecated_config
|
26
28
|
validate!
|
27
29
|
end
|
28
30
|
|
@@ -38,9 +40,10 @@ module Elasticity
|
|
38
40
|
|
39
41
|
def definition
|
40
42
|
return @definition if defined?(@definition)
|
43
|
+
|
41
44
|
@definition = {
|
42
45
|
settings: merge_settings,
|
43
|
-
mappings:
|
46
|
+
mappings: @mapping.nil? ? {} : @mapping.deep_stringify_keys
|
44
47
|
}
|
45
48
|
subclasses.each do |doc_type, subclass|
|
46
49
|
@definition[:mappings][doc_type] = subclass.constantize.mapping
|
@@ -80,6 +83,15 @@ module Elasticity
|
|
80
83
|
@elasticity_config.settings.merge(settings || {})
|
81
84
|
end
|
82
85
|
|
86
|
+
def warn_deprecated_config
|
87
|
+
DEPRECATED_ATTRS.each do |attr|
|
88
|
+
ActiveSupport::Deprecation.warn(
|
89
|
+
"#{attr} is deprecated and will be "\
|
90
|
+
"removed in the next major release."
|
91
|
+
) if public_send(attr).present?
|
92
|
+
end
|
93
|
+
end
|
94
|
+
|
83
95
|
def subclasses_warning_or_exception
|
84
96
|
return if subclasses.nil? || subclasses.empty?
|
85
97
|
raise(SubclassError.new(SUBCLASSES_ERROR)) if es_version_meets_or_exceeds?(VERSION_FOR_SUBCLASS_ERROR)
|
@@ -84,7 +84,7 @@ module Elasticity
|
|
84
84
|
|
85
85
|
# Index the given document
|
86
86
|
def index_document(id, document_hash)
|
87
|
-
@strategy.index_document(
|
87
|
+
@strategy.index_document(id, document_hash)
|
88
88
|
end
|
89
89
|
|
90
90
|
# Searches the index using the parameters provided in the body hash, following the same
|
@@ -99,25 +99,25 @@ module Elasticity
|
|
99
99
|
|
100
100
|
# Fetches one specific document from the index by ID.
|
101
101
|
def get(id)
|
102
|
-
doc = @strategy.get_document(
|
103
|
-
@document_klass.new(doc["_source"].merge(_id: doc[
|
102
|
+
doc = @strategy.get_document(id)
|
103
|
+
@document_klass.new(doc["_source"].merge(_id: doc["_id"])) if doc.present?
|
104
104
|
end
|
105
105
|
|
106
106
|
# Removes one specific document from the index.
|
107
107
|
def delete(id)
|
108
|
-
@strategy.delete_document(
|
108
|
+
@strategy.delete_document(id)
|
109
109
|
end
|
110
110
|
|
111
111
|
# Removes entries based on a search
|
112
112
|
def delete_by_search(search)
|
113
|
-
@strategy.delete_by_query(
|
113
|
+
@strategy.delete_by_query(search.body)
|
114
114
|
end
|
115
115
|
|
116
116
|
# Bulk index the provided documents
|
117
117
|
def bulk_index(documents)
|
118
118
|
@strategy.bulk do |b|
|
119
119
|
documents.each do |doc|
|
120
|
-
b.index(
|
120
|
+
b.index(doc._id, doc.to_document)
|
121
121
|
end
|
122
122
|
end
|
123
123
|
end
|
@@ -127,7 +127,6 @@ module Elasticity
|
|
127
127
|
@strategy.bulk do |b|
|
128
128
|
documents.each do |doc|
|
129
129
|
b.update(
|
130
|
-
document_type,
|
131
130
|
doc[:_id],
|
132
131
|
{ doc: { doc[:attr_name] => doc[:attr_value] } }
|
133
132
|
)
|
@@ -139,7 +138,7 @@ module Elasticity
|
|
139
138
|
def bulk_delete(ids)
|
140
139
|
@strategy.bulk do |b|
|
141
140
|
ids.each do |id|
|
142
|
-
b.delete(
|
141
|
+
b.delete(id)
|
143
142
|
end
|
144
143
|
end
|
145
144
|
end
|
@@ -156,7 +155,7 @@ module Elasticity
|
|
156
155
|
|
157
156
|
if hit["highlight"]
|
158
157
|
highlighted_attrs = hit["highlight"].each_with_object({}) do |(name, v), attrs|
|
159
|
-
name = name.gsub(/\..*\z/,
|
158
|
+
name = name.gsub(/\..*\z/, "")
|
160
159
|
|
161
160
|
attrs[name] ||= v
|
162
161
|
end
|
data/lib/elasticity/railtie.rb
CHANGED
data/lib/elasticity/search.rb
CHANGED
@@ -6,7 +6,7 @@ module Elasticity
|
|
6
6
|
end
|
7
7
|
|
8
8
|
# Elasticity::Search::Definition is a struct that encapsulates all the data specific to one
|
9
|
-
#
|
9
|
+
# Elasticsearch search.
|
10
10
|
class Definition
|
11
11
|
attr_accessor :index_name, :document_types, :body
|
12
12
|
|
@@ -22,20 +22,20 @@ module Elasticity
|
|
22
22
|
end
|
23
23
|
|
24
24
|
def to_count_args
|
25
|
-
{ index: @index_name
|
25
|
+
{ index: @index_name }.tap do |args|
|
26
26
|
body = @body.slice(:query)
|
27
27
|
args[:body] = body if body.present?
|
28
28
|
end
|
29
29
|
end
|
30
30
|
|
31
31
|
def to_search_args
|
32
|
-
@search_args.merge({ index: @index_name,
|
32
|
+
@search_args.merge({ index: @index_name, body: @body })
|
33
33
|
end
|
34
34
|
|
35
35
|
def to_msearch_args
|
36
36
|
search_body = @search_args.merge(@body)
|
37
37
|
|
38
|
-
{ index: @index_name,
|
38
|
+
{ index: @index_name, search: search_body }
|
39
39
|
end
|
40
40
|
end
|
41
41
|
|
@@ -4,24 +4,18 @@ module Elasticity
|
|
4
4
|
# runtime changes by simply atomically updating the aliases. For example, look at the remap method
|
5
5
|
# implementation.
|
6
6
|
class AliasIndex
|
7
|
-
SNAPSHOT_ERROR_SNIPPET = "Cannot delete indices that are being snapshotted"
|
7
|
+
SNAPSHOT_ERROR_SNIPPET = "Cannot delete indices that are being snapshotted"
|
8
8
|
RETRYABLE_ERROR_SNIPPETS = [
|
9
9
|
SNAPSHOT_ERROR_SNIPPET
|
10
10
|
].freeze
|
11
11
|
|
12
12
|
STATUSES = [:missing, :ok]
|
13
13
|
|
14
|
-
def initialize(client, index_base_name, document_type, use_new_timestamp_format =
|
14
|
+
def initialize(client, index_base_name, document_type, use_new_timestamp_format = nil, include_type_name_on_create = nil)
|
15
15
|
@client = client
|
16
16
|
@main_alias = index_base_name
|
17
17
|
@update_alias = "#{index_base_name}_update"
|
18
18
|
@document_type = document_type
|
19
|
-
|
20
|
-
# Deprecated: The use_new_timestamp_format option is no longer used and will be removed in the next version.
|
21
|
-
@use_new_timestamp_format = use_new_timestamp_format
|
22
|
-
|
23
|
-
# included for compatibility with v7
|
24
|
-
@include_type_name_on_create = include_type_name_on_create
|
25
19
|
end
|
26
20
|
|
27
21
|
def ref_index_name
|
@@ -63,22 +57,22 @@ module Elasticity
|
|
63
57
|
})
|
64
58
|
|
65
59
|
@client.index_refresh(index: original_index)
|
66
|
-
cursor = @client.search index: original_index, search_type: :query_then_fetch, scroll:
|
60
|
+
cursor = @client.search index: original_index, search_type: :query_then_fetch, scroll: "10m", size: 100
|
67
61
|
loop do
|
68
|
-
hits = cursor[
|
62
|
+
hits = cursor["hits"]["hits"]
|
69
63
|
break if hits.empty?
|
70
64
|
|
71
65
|
# Fetch documents based on the ids that existed when the migration started, to make sure we only migrate
|
72
66
|
# documents that haven't been deleted.
|
73
67
|
id_docs = hits.map do |hit|
|
74
|
-
{ _index: original_index,
|
68
|
+
{ _index: original_index, _id: hit["_id"] }
|
75
69
|
end
|
76
70
|
|
77
71
|
docs = @client.mget(body: { docs: id_docs }, refresh: true)["docs"]
|
78
72
|
break if docs.empty?
|
79
73
|
|
80
74
|
# Modify document hashes to match the mapping definition so that legacy fields aren't added
|
81
|
-
defined_mapping_fields = index_def[:mappings][
|
75
|
+
defined_mapping_fields = index_def[:mappings]["properties"].keys
|
82
76
|
|
83
77
|
# Move only documents that still exists on the old index, into the new index.
|
84
78
|
ops = []
|
@@ -86,7 +80,7 @@ module Elasticity
|
|
86
80
|
if doc["found"]
|
87
81
|
legacy_fields = doc["_source"].keys - defined_mapping_fields
|
88
82
|
legacy_fields.each { |field| doc["_source"].delete(field) }
|
89
|
-
ops << { index: { _index: new_index,
|
83
|
+
ops << { index: { _index: new_index, _id: doc["_id"], data: doc["_source"] } }
|
90
84
|
end
|
91
85
|
end
|
92
86
|
|
@@ -96,12 +90,12 @@ module Elasticity
|
|
96
90
|
ops = []
|
97
91
|
@client.mget(body: { docs: id_docs }, refresh: true)["docs"].each_with_index do |new_doc, idx|
|
98
92
|
if docs[idx]["found"] && !new_doc["found"]
|
99
|
-
ops << { delete: { _index: new_index,
|
93
|
+
ops << { delete: { _index: new_index, _id: new_doc["_id"] } }
|
100
94
|
end
|
101
95
|
end
|
102
96
|
|
103
97
|
@client.bulk(body: ops) unless ops.empty?
|
104
|
-
cursor = @client.scroll(scroll_id: cursor[
|
98
|
+
cursor = @client.scroll(scroll_id: cursor["_scroll_id"], scroll: "1m", body: { scroll_id: cursor["_scroll_id"] })
|
105
99
|
end
|
106
100
|
|
107
101
|
# Update aliases to only point to the new index.
|
@@ -133,19 +127,19 @@ module Elasticity
|
|
133
127
|
})
|
134
128
|
|
135
129
|
@client.index_refresh(index: new_index)
|
136
|
-
cursor = @client.search index: new_index, search_type: :query_then_fetch, scroll:
|
130
|
+
cursor = @client.search index: new_index, search_type: :query_then_fetch, scroll: "1m", size: 100
|
137
131
|
loop do
|
138
|
-
hits = cursor[
|
132
|
+
hits = cursor["hits"]["hits"]
|
139
133
|
break if hits.empty?
|
140
134
|
|
141
135
|
# Move all the documents that exists on the new index back to the old index
|
142
136
|
ops = []
|
143
137
|
hits.each do |doc|
|
144
|
-
ops << { index: { _index: original_index,
|
138
|
+
ops << { index: { _index: original_index, _id: doc["_id"], data: doc["_source"] } }
|
145
139
|
end
|
146
140
|
|
147
141
|
@client.bulk(body: ops)
|
148
|
-
cursor = @client.scroll(scroll_id: cursor[
|
142
|
+
cursor = @client.scroll(scroll_id: cursor["_scroll_id"], scroll: "1m")
|
149
143
|
end
|
150
144
|
|
151
145
|
@client.index_refresh(index: original_index)
|
@@ -224,8 +218,8 @@ module Elasticity
|
|
224
218
|
create(index_def)
|
225
219
|
end
|
226
220
|
|
227
|
-
def index_document(
|
228
|
-
res = @client.index(index: @update_alias,
|
221
|
+
def index_document(id, attributes)
|
222
|
+
res = @client.index(index: @update_alias, id: id, body: attributes)
|
229
223
|
|
230
224
|
if id = res["_id"]
|
231
225
|
[id, res["_shards"] && res["_shards"]["successful"].to_i > 0]
|
@@ -234,24 +228,24 @@ module Elasticity
|
|
234
228
|
end
|
235
229
|
end
|
236
230
|
|
237
|
-
def delete_document(
|
231
|
+
def delete_document(id)
|
238
232
|
ops = (main_indexes | update_indexes).map do |index|
|
239
|
-
{ delete: { _index: index,
|
233
|
+
{ delete: { _index: index, _id: id } }
|
240
234
|
end
|
241
235
|
|
242
236
|
@client.bulk(body: ops)
|
243
237
|
end
|
244
238
|
|
245
|
-
def get_document(
|
246
|
-
@client.get(index: @main_alias,
|
239
|
+
def get_document(id)
|
240
|
+
@client.get(index: @main_alias, id: id)
|
247
241
|
end
|
248
242
|
|
249
243
|
def search_index
|
250
244
|
@main_alias
|
251
245
|
end
|
252
246
|
|
253
|
-
def delete_by_query(
|
254
|
-
@client.delete_by_query(index: @main_alias,
|
247
|
+
def delete_by_query(body)
|
248
|
+
@client.delete_by_query(index: @main_alias, body: body)
|
255
249
|
end
|
256
250
|
|
257
251
|
def bulk
|
@@ -269,21 +263,21 @@ module Elasticity
|
|
269
263
|
end
|
270
264
|
|
271
265
|
def settings
|
272
|
-
@client.index_get_settings(index: @main_alias
|
266
|
+
@client.index_get_settings(index: @main_alias).values.first
|
273
267
|
rescue Elasticsearch::Transport::Transport::Errors::NotFound
|
274
268
|
nil
|
275
269
|
end
|
276
270
|
|
277
271
|
def mappings
|
278
272
|
ActiveSupport::Deprecation.warn(
|
279
|
-
|
280
|
-
|
273
|
+
"Elasticity::Strategies::AliasIndex#mappings is deprecated, "\
|
274
|
+
"use mapping instead"
|
281
275
|
)
|
282
276
|
mapping
|
283
277
|
end
|
284
278
|
|
285
279
|
def mapping
|
286
|
-
@client.index_get_mapping(index: @main_alias
|
280
|
+
@client.index_get_mapping(index: @main_alias).values.first
|
287
281
|
rescue Elasticsearch::Transport::Transport::Errors::NotFound
|
288
282
|
nil
|
289
283
|
end
|
@@ -297,7 +291,7 @@ module Elasticity
|
|
297
291
|
|
298
292
|
def create_index(index_def)
|
299
293
|
name = build_index_name
|
300
|
-
@client.index_create(index: name, body: index_def
|
294
|
+
@client.index_create(index: name, body: index_def)
|
301
295
|
name
|
302
296
|
end
|
303
297
|
|
@@ -3,16 +3,10 @@ module Elasticity
|
|
3
3
|
class SingleIndex
|
4
4
|
STATUSES = [:missing, :ok]
|
5
5
|
|
6
|
-
def initialize(client, index_name, document_type, use_new_timestamp_format =
|
6
|
+
def initialize(client, index_name, document_type, use_new_timestamp_format = nil, include_type_name_on_create = nil)
|
7
7
|
@client = client
|
8
8
|
@index_name = index_name
|
9
9
|
@document_type = document_type
|
10
|
-
|
11
|
-
# included for compatibility with v7
|
12
|
-
@include_type_name_on_create = include_type_name_on_create
|
13
|
-
|
14
|
-
# Deprecated: The use_new_timestamp_format option is no longer used and will be removed in the next version.
|
15
|
-
@use_new_timestamp_format = use_new_timestamp_format
|
16
10
|
end
|
17
11
|
|
18
12
|
def ref_index_name
|
@@ -29,7 +23,7 @@ module Elasticity
|
|
29
23
|
|
30
24
|
def create(index_def)
|
31
25
|
if missing?
|
32
|
-
@client.index_create(index: @index_name, body: index_def
|
26
|
+
@client.index_create(index: @index_name, body: index_def)
|
33
27
|
else
|
34
28
|
raise IndexError.new(@index_name, "index already exist")
|
35
29
|
end
|
@@ -52,8 +46,8 @@ module Elasticity
|
|
52
46
|
create(index_def)
|
53
47
|
end
|
54
48
|
|
55
|
-
def index_document(
|
56
|
-
res = @client.index(index: @index_name,
|
49
|
+
def index_document(id, attributes)
|
50
|
+
res = @client.index(index: @index_name, id: id, body: attributes)
|
57
51
|
|
58
52
|
if id = res["_id"]
|
59
53
|
[id, res["created"]]
|
@@ -62,20 +56,20 @@ module Elasticity
|
|
62
56
|
end
|
63
57
|
end
|
64
58
|
|
65
|
-
def delete_document(
|
66
|
-
@client.delete(index: @index_name,
|
59
|
+
def delete_document(id)
|
60
|
+
@client.delete(index: @index_name, id: id)
|
67
61
|
end
|
68
62
|
|
69
|
-
def get_document(
|
70
|
-
@client.get(index: @index_name,
|
63
|
+
def get_document(id)
|
64
|
+
@client.get(index: @index_name, id: id)
|
71
65
|
end
|
72
66
|
|
73
67
|
def search_index
|
74
68
|
@index_name
|
75
69
|
end
|
76
70
|
|
77
|
-
def delete_by_query(
|
78
|
-
@client.delete_by_query(index: @index_name,
|
71
|
+
def delete_by_query(body)
|
72
|
+
@client.delete_by_query(index: @index_name, body: body)
|
79
73
|
end
|
80
74
|
|
81
75
|
def bulk
|
@@ -85,21 +79,21 @@ module Elasticity
|
|
85
79
|
end
|
86
80
|
|
87
81
|
def settings
|
88
|
-
@client.index_get_settings(index: @index_name
|
82
|
+
@client.index_get_settings(index: @index_name).values.first
|
89
83
|
rescue Elasticsearch::Transport::Transport::Errors::NotFound
|
90
84
|
nil
|
91
85
|
end
|
92
86
|
|
93
87
|
def mappings
|
94
88
|
ActiveSupport::Deprecation.warn(
|
95
|
-
|
96
|
-
|
89
|
+
"Elasticity::Strategies::SingleIndex#mappings is deprecated, "\
|
90
|
+
"use mapping instead"
|
97
91
|
)
|
98
92
|
mapping
|
99
93
|
end
|
100
94
|
|
101
95
|
def mapping
|
102
|
-
@client.index_get_mapping(index: @index_name
|
96
|
+
@client.index_get_mapping(index: @index_name).values.first
|
103
97
|
rescue Elasticsearch::Transport::Transport::Errors::NotFound
|
104
98
|
nil
|
105
99
|
end
|
data/lib/elasticity/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: es-elasticity
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 1.0.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Rodrigo Kochenburger
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-08-
|
11
|
+
date: 2022-08-16 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|