elasticsearch 7.13.3 → 8.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1dd904fe3389952a5434f9bdd54e43b085d686146620423f1f55e5e1cfcea9cc
4
- data.tar.gz: 8456c08f65bde635e6238f01f6227d4688003c1083ac69c6b45029ba031d7679
3
+ metadata.gz: 829c510c4fb0664f58d4be87a7d0d8eed2906b68e659f63b33a7e082cba6c1ee
4
+ data.tar.gz: bafb973a40c4eda2a5de4fb0abe5a7f4308efbe8105f531545953a405d2cb83f
5
5
  SHA512:
6
- metadata.gz: cc9635e7f85dddfec58ea02f18f31a586f0e671abbbfb19038564c976097d982bad0f1777b94798a1f4baca88213d7ac03ca27dc162792903eed1724f3f62289
7
- data.tar.gz: c4aeeeb547a5e678ff1421b0fbcdc521662d6606afc359f6e813cf38278b274be320e2b912d09b7e722f2a46f8518fbe0a741748f2f9ff10c8a407b703e11d2b
6
+ metadata.gz: 4c98e9e0c6cc219b281da17019021ce7bd7e28120c9390b05dd71550bf2527c99570002f779f504f10820effe1fc2cd0b2824213bb445184fb7b77eabc69532b
7
+ data.tar.gz: 6178b702bd1077554debcf12783ffd1cda149c507c87284cbe60028a0f06fe0db8f2c140f3c29fd19fca23ec456012087ff25f6dcec34553e63d550146248ede
data/Gemfile CHANGED
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  # Licensed to Elasticsearch B.V. under one or more contributor
2
4
  # license agreements. See the NOTICE file distributed with
3
5
  # this work for additional information regarding copyright
@@ -20,14 +22,14 @@ source 'https://rubygems.org'
20
22
  # Specify your gem's dependencies in elasticsearch.gemspec
21
23
  gemspec
22
24
 
23
- if File.exist? File.expand_path("../../elasticsearch-api/elasticsearch-api.gemspec", __FILE__)
24
- gem 'elasticsearch-api', :path => File.expand_path("../../elasticsearch-api", __FILE__), :require => false
25
+ if File.exist? File.expand_path('../elasticsearch-api/elasticsearch-api.gemspec', __dir__)
26
+ gem 'elasticsearch-api', path: File.expand_path('../elasticsearch-api', __dir__), require: false
25
27
  end
26
28
 
27
- if File.exist? File.expand_path("../../elasticsearch-transport/elasticsearch-transport.gemspec", __FILE__)
28
- gem 'elasticsearch-transport', :path => File.expand_path("../../elasticsearch-transport", __FILE__), :require => false
29
+ if ENV['TRANSPORT_VERSION'] == 'main'
30
+ gem 'elastic-transport', git: 'https://github.com/elastic/elastic-transport-ruby.git', branch: 'main'
29
31
  end
30
32
 
31
- if File.exist? File.expand_path("../../elasticsearch-extensions", __FILE__)
32
- gem 'elasticsearch-extensions', :path => File.expand_path("../../elasticsearch-extensions", __FILE__), :require => true
33
+ if RUBY_VERSION >= '3.0'
34
+ gem 'opentelemetry-sdk', require: false
33
35
  end
data/README.md CHANGED
@@ -2,6 +2,29 @@
2
2
 
3
3
  The `elasticsearch` library provides a Ruby client and API for [Elasticsearch](http://elasticsearch.com).
4
4
 
5
+ ## Usage
6
+
7
+ This gem is a wrapper for two separate libraries:
8
+
9
+ * [`elastic-transport`](https://github.com/elastic/elastic-transport-ruby/), which provides a low-level Ruby client for connecting to [Elastic](http://elasticsearch.com) services.
10
+ * [`elasticsearch-api`](https://github.com/elasticsearch/elasticsearch-ruby/tree/main/elasticsearch-api), which provides a Ruby API for the Elasticsearch RESTful API.
11
+
12
+ Install the `elasticsearch` package and use the API directly:
13
+
14
+ ```ruby
15
+ require 'elasticsearch'
16
+
17
+ client = Elasticsearch::Client.new(log: true)
18
+
19
+ client.cluster.health
20
+
21
+ client.transport.reload_connections!
22
+
23
+ client.search(q: 'test')
24
+
25
+ # etc.
26
+ ```
27
+
5
28
  Features overview:
6
29
 
7
30
  * Pluggable logging and tracing
@@ -14,25 +37,13 @@ Features overview:
14
37
  * Extensive documentation and examples
15
38
  * Emphasis on modularity and extendability of both the client and API libraries
16
39
 
17
- (For integration with Ruby models and Rails applications,
18
- see the <https://github.com/elasticsearch/elasticsearch-rails> project.)
40
+ (For integration with Ruby models and Rails applications, see the <https://github.com/elasticsearch/elasticsearch-rails> project.)
19
41
 
20
42
  ## Compatibility
21
43
 
22
- The Elasticsearch client for Ruby is compatible with Ruby 1.9 and higher.
44
+ We follow Ruby’s own maintenance policy and officially support all currently maintained versions per [Ruby Maintenance Branches](https://www.ruby-lang.org/en/downloads/branches/).
23
45
 
24
- The client's API is compatible with Elasticsearch's API versions from 0.90 till current,
25
- just use a release matching major version of Elasticsearch.
26
-
27
- | Ruby | | Elasticsearch |
28
- |:-------------:|:-:| :-----------: |
29
- | 0.90 | → | 0.90 |
30
- | 1.x | → | 1.x |
31
- | 2.x | → | 2.x |
32
- | 5.x | → | 5.x |
33
- | 6.x | → | 6.x |
34
- | 7.x | → | 7.x |
35
- | master | → | master |
46
+ Language clients are forward compatible; meaning that clients support communicating with greater minor versions of Elasticsearch. Elastic language clients are also backwards compatible with lesser supported minor Elasticsearch versions.
36
47
 
37
48
  ## Installation
38
49
 
@@ -51,41 +62,68 @@ or install it from a source code checkout:
51
62
  bundle install
52
63
  rake install
53
64
 
54
- ## Usage
65
+ ## Configuration
55
66
 
56
- This library is a wrapper for two separate libraries:
67
+ * [Identifying running tasks with X-Opaque-Id](#identifying-running-tasks-with-x-opaque-id)
68
+ * [Api Key Authentication](#api-key-authentication)
57
69
 
58
- * [`elasticsearch-transport`](https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch-transport),
59
- which provides a low-level Ruby client for connecting to an [Elasticsearch](http://elasticsearch.com) cluster
60
- * [`elasticsearch-api`](https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch-api),
61
- which provides a Ruby API for the Elasticsearch RESTful API
70
+ ### Identifying running tasks with X-Opaque-Id
62
71
 
63
- Install the `elasticsearch` package and use the API directly:
72
+ The X-Opaque-Id header allows to track certain calls, or associate certain tasks with the client that started them ([more on the Elasticsearch docs](https://www.elastic.co/guide/en/elasticsearch/reference/master/tasks.html#_identifying_running_tasks)). To use this feature, you need to set an id for `opaque_id` on the client on each request. Example:
64
73
 
65
74
  ```ruby
66
- require 'elasticsearch'
75
+ client = Elasticsearch::Client.new
76
+ client.search(index: 'myindex', q: 'title:test', opaque_id: '123456')
77
+ ```
78
+ The search request will include the following HTTP Header:
79
+ ```
80
+ X-Opaque-Id: 123456
81
+ ```
67
82
 
68
- client = Elasticsearch::Client.new log: true
83
+ You can also set a prefix for X-Opaque-Id when initializing the client. This will be prepended to the id you set before each request if you're using X-Opaque-Id. Example:
84
+ ```ruby
85
+ client = Elastic::Transport::Client.new(opaque_id_prefix: 'eu-west1_')
86
+ client.search(index: 'myindex', q: 'title:test', opaque_id: '123456')
87
+ ```
88
+ The request will include the following HTTP Header:
89
+ ```
90
+ X-Opaque-Id: eu-west1_123456
91
+ ```
69
92
 
70
- client.cluster.health
93
+ ### Api Key Authentication
71
94
 
72
- client.transport.reload_connections!
95
+ You can use [**API Key authentication**](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html):
73
96
 
74
- client.search q: 'test'
97
+ ``` ruby
98
+ Elasticsearch::Client.new(
99
+ host: host,
100
+ transport_options: transport_options,
101
+ api_key: credentials
102
+ )
103
+ ```
75
104
 
76
- # etc.
105
+ Where credentials is either the base64 encoding of `id` and `api_key` joined by a colon or a hash with the `id` and `api_key`:
106
+
107
+ ``` ruby
108
+ Elasticsearch::Client.new(
109
+ host: host,
110
+ transport_options: transport_options,
111
+ api_key: {id: 'my_id', api_key: 'my_api_key'}
112
+ )
77
113
  ```
78
114
 
115
+ ## API and Transport
116
+
79
117
  Please refer to the specific library documentation for details:
80
118
 
81
119
  * **Transport**:
82
- [[README]](https://github.com/elasticsearch/elasticsearch-ruby/blob/master/elasticsearch-transport/README.md)
83
- [[Documentation]](http://rubydoc.info/gems/elasticsearch-transport/file/README.markdown)
120
+ [[README]](https://github.com/elastic/elastic-transport-ruby#elastic-transport)
121
+ [[Documentation]](https://rubydoc.info/github/elastic/elastic-transport-ruby/)
84
122
 
85
123
  * **API**:
86
- [[README]](https://github.com/elasticsearch/elasticsearch-ruby/blob/master/elasticsearch-api/README.md)
87
- [[Documentation]](http://rubydoc.info/gems/elasticsearch-api/file/README.markdown)
124
+ [[README]](https://github.com/elastic/elasticsearch-ruby/tree/main/elasticsearch-api#elasticsearchapi)
125
+ [[Documentation]](https://rubydoc.info/gems/elasticsearch-api)
88
126
 
89
127
  ## License
90
128
 
91
- This software is licensed under the [Apache 2 license](./LICENSE).
129
+ This software is licensed under the [Apache 2 license](./LICENSE).
data/Rakefile CHANGED
@@ -15,40 +15,35 @@
15
15
  # specific language governing permissions and limitations
16
16
  # under the License.
17
17
 
18
- require "bundler/gem_tasks"
18
+ require 'bundler/gem_tasks'
19
19
 
20
- desc "Run unit tests"
21
- task :test => 'test:unit'
20
+ task(:default) { system 'rake --tasks' }
21
+
22
+ desc 'Run unit tests'
23
+ task test: 'test:spec'
22
24
 
23
25
  # ----- Test tasks ------------------------------------------------------------
26
+ require 'rspec/core/rake_task'
24
27
 
25
- require 'rake/testtask'
26
28
  namespace :test do
27
- desc "Wait for Elasticsearch to be in a green state"
29
+ desc 'Wait for Elasticsearch to be in a green state'
28
30
  task :wait_for_green do
29
31
  sh '../scripts/wait-cluster.sh'
30
32
  end
31
33
 
32
- Rake::TestTask.new(:unit) do |test|
33
- test.libs << 'lib' << 'test'
34
- test.test_files = FileList["test/unit/**/*_test.rb"]
35
- test.verbose = false
36
- test.warning = false
34
+ RSpec::Core::RakeTask.new(:integration) do |t|
35
+ t.pattern = 'spec/integration/**{,/*/**}/*_spec.rb'
37
36
  end
38
37
 
39
- Rake::TestTask.new(:integration) do |test|
40
- test.deps = [ :wait_for_green ]
41
- test.libs << 'lib' << 'test'
42
- test.test_files = FileList["test/integration/**/*_test.rb"]
43
- test.verbose = false
44
- test.warning = false
38
+ RSpec::Core::RakeTask.new(:unit) do |t|
39
+ t.pattern = 'spec/unit/**{,/*/**}/*_spec.rb'
45
40
  end
46
41
 
47
- Rake::TestTask.new(:all) do |test|
48
- test.libs << 'lib' << 'test'
49
- test.test_files = FileList["test/unit/**/*_test.rb", "test/integration/**/*_test.rb"]
42
+ desc 'Run unit and integration tests'
43
+ task :all do
44
+ Rake::Task['test:unit'].invoke
45
+ Rake::Task['test:integration'].invoke
50
46
  end
51
-
52
47
  end
53
48
 
54
49
  # ----- Documentation tasks ---------------------------------------------------
@@ -1,30 +1,14 @@
1
1
  #!/usr/bin/env ruby
2
2
 
3
3
  $LOAD_PATH.unshift(File.expand_path('../../elasticsearch/lib', __dir__))
4
- $LOAD_PATH.unshift(File.expand_path('../../elasticsearch-transport/lib', __dir__))
5
- $LOAD_PATH.unshift(File.expand_path('../../elasticsearch-dsl/lib', __dir__))
6
4
  $LOAD_PATH.unshift(File.expand_path('../../elasticsearch-api/lib', __dir__))
7
- $LOAD_PATH.unshift(File.expand_path('../../elasticsearch-xpack/lib', __dir__))
8
- $LOAD_PATH.unshift(File.expand_path('../../elasticsearch-extensions/lib', __dir__))
5
+ $LOAD_PATH.unshift(File.expand_path('../../elasticsearch/lib/elasticsearch/helpers', __dir__))
9
6
 
10
7
  require 'elasticsearch'
11
- require 'elasticsearch-transport'
12
8
  require 'elasticsearch-api'
13
-
14
- gems_not_loaded = ['elasticsearch-dsl', 'elasticsearch/xpack', 'elasticsearch-extensions'].reject do |gem|
15
- begin
16
- (require gem) || true
17
- rescue LoadError
18
- false
19
- end
20
- end
21
-
22
- unless gems_not_loaded.empty?
23
- warn "The following gems were not loaded: [#{gems_not_loaded.join(', ')}]. Please install and require them explicitly."
24
- end
9
+ require 'elasticsearch/helpers/bulk_helper'
25
10
 
26
11
  include Elasticsearch
27
- include Elasticsearch::DSL if defined?(Elasticsearch::DSL)
28
12
 
29
13
  begin
30
14
  require 'pry'
@@ -22,54 +22,43 @@ require 'elasticsearch/version'
22
22
  Gem::Specification.new do |s|
23
23
  s.name = 'elasticsearch'
24
24
  s.version = Elasticsearch::VERSION
25
- s.authors = ['Karel Minarik']
26
- s.email = ['karel.minarik@elasticsearch.org']
25
+ s.authors = ['Elastic Client Library Maintainers']
26
+ s.email = ['client-libs@elastic.co']
27
27
  s.summary = 'Ruby integrations for Elasticsearch'
28
- s.homepage = 'https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/7.x/index.html'
28
+ s.homepage = 'https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current/index.html'
29
29
  s.license = 'Apache-2.0'
30
30
  s.metadata = {
31
- 'homepage_uri' => 'https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/7.x/index.html',
32
- 'changelog_uri' => 'https://github.com/elastic/elasticsearch-ruby/blob/7.x/CHANGELOG.md',
33
- 'source_code_uri' => 'https://github.com/elastic/elasticsearch-ruby/tree/7.x',
31
+ 'homepage_uri' => 'https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current/index.html',
32
+ 'changelog_uri' => 'https://github.com/elastic/elasticsearch-ruby/blob/main/CHANGELOG.md',
33
+ 'source_code_uri' => 'https://github.com/elastic/elasticsearch-ruby/tree/main',
34
34
  'bug_tracker_uri' => 'https://github.com/elastic/elasticsearch-ruby/issues'
35
35
  }
36
- s.files = `git ls-files`.split($/)
37
- s.executables = s.files.grep(%r{^bin/}) { |f| File.basename(f) }
38
- s.executables << 'elastic_ruby_console'
39
- s.test_files = s.files.grep(%r{^(test|spec|features)/})
36
+ s.files = `git ls-files`.split($/)
37
+ s.executables = s.files.grep(%r{^bin/}) { |f| File.basename(f) }
38
+ s.executables << 'elastic_ruby_console'
39
+ s.test_files = s.files.grep(%r{^(test|spec|features)/})
40
40
  s.require_paths = ['lib']
41
41
  s.bindir = 'bin'
42
42
 
43
- s.extra_rdoc_files = [ 'README.md', 'LICENSE' ]
44
- s.rdoc_options = [ '--charset=UTF-8' ]
43
+ s.extra_rdoc_files = ['README.md', 'LICENSE.txt']
44
+ s.rdoc_options = ['--charset=UTF-8']
45
45
 
46
- s.required_ruby_version = '>= 2.4'
47
-
48
- s.add_dependency 'elasticsearch-transport', '7.13.3'
49
- s.add_dependency 'elasticsearch-api', '7.13.3'
46
+ s.required_ruby_version = '>= 2.5'
50
47
 
48
+ s.add_dependency 'elastic-transport', '~> 8.3'
49
+ s.add_dependency 'elasticsearch-api', '8.14.0'
50
+
51
+ s.add_development_dependency 'base64'
51
52
  s.add_development_dependency 'bundler'
52
-
53
- s.add_development_dependency 'rake', '~> 13'
54
-
55
- s.add_development_dependency 'elasticsearch-extensions'
56
-
57
- s.add_development_dependency 'ansi'
58
- s.add_development_dependency 'shoulda-context'
59
- s.add_development_dependency 'mocha'
60
- s.add_development_dependency 'yard'
53
+ s.add_development_dependency 'debug' unless defined?(JRUBY_VERSION)
61
54
  s.add_development_dependency 'pry'
62
-
63
-
64
- s.add_development_dependency 'minitest'
65
- s.add_development_dependency 'minitest-reporters'
66
- s.add_development_dependency 'ruby-prof' unless defined?(JRUBY_VERSION) || defined?(Rubinius)
55
+ s.add_development_dependency 'rake'
67
56
  s.add_development_dependency 'require-prof' unless defined?(JRUBY_VERSION) || defined?(Rubinius)
68
- s.add_development_dependency 'simplecov', '~> 0.17', '< 0.18'
69
- s.add_development_dependency 'simplecov-rcov'
70
- s.add_development_dependency 'cane'
71
-
72
- s.add_development_dependency 'test-unit', '~> 2'
57
+ s.add_development_dependency 'rspec'
58
+ s.add_development_dependency 'ruby-prof' unless defined?(JRUBY_VERSION) || defined?(Rubinius)
59
+ s.add_development_dependency 'simplecov'
60
+ s.add_development_dependency 'webmock'
61
+ s.add_development_dependency 'yard'
73
62
 
74
63
  s.description = <<-DESC.gsub(/^ /, '')
75
64
  Ruby integrations for Elasticsearch (client, API, etc.)
@@ -0,0 +1,129 @@
1
+ # Licensed to Elasticsearch B.V. under one or more contributor
2
+ # license agreements. See the NOTICE file distributed with
3
+ # this work for additional information regarding copyright
4
+ # ownership. Elasticsearch B.V. licenses this file to you under
5
+ # the Apache License, Version 2.0 (the "License"); you may
6
+ # not use this file except in compliance with the License.
7
+ # You may obtain a copy of the License at
8
+ #
9
+ # http://www.apache.org/licenses/LICENSE-2.0
10
+ #
11
+ # Unless required by applicable law or agreed to in writing,
12
+ # software distributed under the License is distributed on an
13
+ # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14
+ # KIND, either express or implied. See the License for the
15
+ # specific language governing permissions and limitations
16
+ # under the License.
17
+
18
+ module Elasticsearch
19
+ module Helpers
20
+ # Elasticsearch Client Helper for the Bulk API
21
+ #
22
+ # @see https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-bulk.html
23
+ #
24
+ class BulkHelper
25
+ attr_accessor :index
26
+
27
+ # Create a BulkHelper
28
+ #
29
+ # @param [Elasticsearch::Client] client Instance of Elasticsearch client to use.
30
+ # @param [String] index Index on which to perform the Bulk actions.
31
+ # @param [Hash] params Parameters to re-use in every bulk call
32
+ #
33
+ def initialize(client, index, params = {})
34
+ @client = client
35
+ @index = index
36
+ @params = params
37
+ end
38
+
39
+ # Index documents using the Bulk API.
40
+ #
41
+ # @param [Array<Hash>] docs The documents to be indexed.
42
+ # @param [Hash] params Parameters to use in the bulk ingestion. See the official Elastic documentation for Bulk API for parameters to send to the Bulk API.
43
+ # @option params [Integer] slice number of documents to send to the Bulk API for eatch batch of ingestion.
44
+ # @param block [Block] Optional block to run after ingesting a batch of documents.
45
+ # @yieldparam response [Elasticsearch::Transport::Response] The response object from calling the Bulk API.
46
+ # @yieldparam ingest_docs [Array<Hash>] The collection of documents sent in the bulk request.
47
+ #
48
+ def ingest(docs, params = {}, body = {}, &block)
49
+ ingest_docs = docs.map { |doc| { index: { _index: @index, data: doc} } }
50
+ if (slice = params.delete(:slice))
51
+ ingest_docs.each_slice(slice) do |items|
52
+ ingest(items.map { |item| item[:index][:data] }, params, &block)
53
+ end
54
+ else
55
+ bulk_request(ingest_docs, params, &block)
56
+ end
57
+ end
58
+
59
+ # Delete documents using the Bulk API
60
+ #
61
+ # @param [Array] ids Array of id's for documents to delete.
62
+ # @param [Hash] params Parameters to send to bulk delete.
63
+ #
64
+ def delete(ids, params = {}, body = {})
65
+ delete_docs = ids.map { |id| { delete: { _index: @index, _id: id} } }
66
+ @client.bulk({ body: delete_docs }.merge(params.merge(@params)))
67
+ end
68
+
69
+ # Update documents using the Bulk API
70
+ #
71
+ # @param [Array<Hash>] docs (Required) The documents to be updated.
72
+ # @option params [Integer] slice number of documents to send to the Bulk API for eatch batch of updates.
73
+ # @param block [Block] Optional block to run after ingesting a batch of documents.
74
+ #
75
+ # @yieldparam response [Elasticsearch::Transport::Response] The response object from calling the Bulk API.
76
+ # @yieldparam ingest_docs [Array<Hash>] The collection of documents sent in the bulk request.
77
+ #
78
+ def update(docs, params = {}, body = {}, &block)
79
+ ingest_docs = docs.map do |doc|
80
+ { update: { _index: @index, _id: doc.delete('id'), data: { doc: doc } } }
81
+ end
82
+ if (slice = params.delete(:slice))
83
+ ingest_docs.each_slice(slice) { |items| update(items, params, &block) }
84
+ else
85
+ bulk_request(ingest_docs, params, &block)
86
+ end
87
+ end
88
+
89
+ # Ingest data directly from a JSON file
90
+ #
91
+ # @param [String] file (Required) The file path.
92
+ # @param [Hash] params Parameters to use in the bulk ingestion.
93
+ # @option params [Integer] slice number of documents to send to the Bulk API for eatch batch of updates.
94
+ # @option params [Array|String] keys If the data needs to be digged from the JSON file, the
95
+ # keys can be passed in with this parameter to find it.
96
+ #
97
+ # E.g.: If the data in the parsed JSON Hash is found in
98
+ # +json_parsed['data']['items']+, keys would be passed
99
+ # like this (as an Array):
100
+ #
101
+ # +bulk_helper.ingest_json(file, { keys: ['data', 'items'] })+
102
+ #
103
+ # or as a String:
104
+ #
105
+ # +bulk_helper.ingest_json(file, { keys: 'data, items' })+
106
+ #
107
+ # @yieldparam response [Elasticsearch::Transport::Response] The response object from calling the Bulk API.
108
+ # @yieldparam ingest_docs [Array<Hash>] The collection of documents sent in the bulk request.
109
+ #
110
+ def ingest_json(file, params = {}, &block)
111
+ data = JSON.parse(File.read(file))
112
+ if (keys = params.delete(:keys))
113
+ keys = keys.split(',') if keys.is_a?(String)
114
+ data = data.dig(*keys)
115
+ end
116
+
117
+ ingest(data, params, &block)
118
+ end
119
+
120
+ private
121
+
122
+ def bulk_request(ingest_docs, params, &block)
123
+ response = @client.bulk({ body: ingest_docs }.merge(params.merge(@params)))
124
+ yield response, ingest_docs if block_given?
125
+ response
126
+ end
127
+ end
128
+ end
129
+ end
@@ -0,0 +1,72 @@
1
+ # Licensed to Elasticsearch B.V. under one or more contributor
2
+ # license agreements. See the NOTICE file distributed with
3
+ # this work for additional information regarding copyright
4
+ # ownership. Elasticsearch B.V. licenses this file to you under
5
+ # the Apache License, Version 2.0 (the "License"); you may
6
+ # not use this file except in compliance with the License.
7
+ # You may obtain a copy of the License at
8
+ #
9
+ # http://www.apache.org/licenses/LICENSE-2.0
10
+ #
11
+ # Unless required by applicable law or agreed to in writing,
12
+ # software distributed under the License is distributed on an
13
+ # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14
+ # KIND, either express or implied. See the License for the
15
+ # specific language governing permissions and limitations
16
+ # under the License.
17
+
18
+ module Elasticsearch
19
+ module Helpers
20
+ # Elasticsearch Client Helper for the ES|QL API
21
+ #
22
+ # @see https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-query-api.html
23
+ #
24
+ module ESQLHelper
25
+ # Query helper for ES|QL
26
+ #
27
+ # By default, the `esql.query` API returns a Hash response with the following keys:
28
+ #
29
+ # * `columns` with the value being an Array of `{ name: type }` Hashes for each column.
30
+ #
31
+ # * `values` with the value being an Array of Arrays with the values for each row.
32
+ #
33
+ # This helper function returns an Array of hashes with the columns as keys and the respective
34
+ # values: `{ column['name'] => value }`.
35
+ #
36
+ # @param client [Elasticsearch::Client] an instance of the Client to use for the query.
37
+ # @param query [Hash, String] The query to be passed to the ES|QL query API.
38
+ # @param params [Hash] options to pass to the ES|QL query API.
39
+ # @param parser [Hash] Hash of column name keys and Proc values to transform the value of
40
+ # a given column.
41
+ # @example Using the ES|QL helper
42
+ # require 'elasticsearch/helpers/esql_helper'
43
+ # query = <<~ESQL
44
+ # FROM sample_data
45
+ # | EVAL duration_ms = ROUND(event.duration / 1000000.0, 1)
46
+ # ESQL
47
+ # response = Elasticsearch::Helpers::ESQLHelper.query(client, query)
48
+ #
49
+ # @example Using the ES|QL helper with a parser
50
+ # response = Elasticsearch::Helpers::ESQLHelper.query(
51
+ # client,
52
+ # query,
53
+ # parser: { '@timestamp' => Proc.new { |t| DateTime.parse(t) } }
54
+ # )
55
+ #
56
+ # @see https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current/Helpers.html#_esql_helper
57
+ #
58
+ def self.query(client, query, params = {}, parser: {})
59
+ response = client.esql.query({ body: { query: query }, format: 'json' }.merge(params))
60
+
61
+ columns = response['columns']
62
+ response['values'].map do |value|
63
+ (value.length - 1).downto(0).map do |index|
64
+ key = columns[index]['name']
65
+ value[index] = parser[key].call(value[index]) if value[index] && parser[key]
66
+ { key => value[index] }
67
+ end.reduce({}, :merge)
68
+ end
69
+ end
70
+ end
71
+ end
72
+ end
@@ -0,0 +1,95 @@
1
+ # Licensed to Elasticsearch B.V. under one or more contributor
2
+ # license agreements. See the NOTICE file distributed with
3
+ # this work for additional information regarding copyright
4
+ # ownership. Elasticsearch B.V. licenses this file to you under
5
+ # the Apache License, Version 2.0 (the "License"); you may
6
+ # not use this file except in compliance with the License.
7
+ # You may obtain a copy of the License at
8
+ #
9
+ # http://www.apache.org/licenses/LICENSE-2.0
10
+ #
11
+ # Unless required by applicable law or agreed to in writing,
12
+ # software distributed under the License is distributed on an
13
+ # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14
+ # KIND, either express or implied. See the License for the
15
+ # specific language governing permissions and limitations
16
+ # under the License.
17
+
18
+ module Elasticsearch
19
+ module Helpers
20
+ # Elasticsearch Client Helper for the Scroll API
21
+ #
22
+ # @see https://www.elastic.co/guide/en/elasticsearch/reference/current/scroll-api.html
23
+ #
24
+ class ScrollHelper
25
+ include Enumerable
26
+
27
+ # Create a ScrollHelper
28
+ #
29
+ # @param [Elasticsearch::Client] client (Required) Instance of Elasticsearch client to use.
30
+ # @param [String] index (Required) Index on which to perform the Bulk actions.
31
+ # @param [Hash] body Body parameters to re-use in every scroll request
32
+ # @param [Time] scroll Specify how long a consistent view of the index should be maintained for scrolled search
33
+ #
34
+ def initialize(client, index, body, scroll = '1m')
35
+ @index = index
36
+ @client = client
37
+ @scroll = scroll
38
+ @body = body
39
+ end
40
+
41
+ # Implementation of +each+ for Enumerable module inclusion
42
+ #
43
+ # @yieldparam document [Hash] yields a document found in the search hits.
44
+ #
45
+ def each(&block)
46
+ @docs = []
47
+ @scroll_id = nil
48
+ refresh_docs
49
+ for doc in @docs do
50
+ refresh_docs
51
+ yield doc
52
+ end
53
+ clear
54
+ end
55
+
56
+ # Results from a scroll.
57
+ # Can be called repeatedly (e.g. in a loop) to get the scroll pages.
58
+ #
59
+ def results
60
+ if @scroll_id
61
+ scroll_request
62
+ else
63
+ initial_search
64
+ end
65
+ rescue StandardError => e
66
+ raise e
67
+ end
68
+
69
+ # Clear Scroll and resets inner documents collection
70
+ #
71
+ def clear
72
+ @client.clear_scroll(body: { scroll_id: @scroll_id }) if @scroll_id
73
+ @docs = []
74
+ end
75
+
76
+ private
77
+
78
+ def refresh_docs
79
+ @docs ||= []
80
+ @docs << results
81
+ @docs.flatten!
82
+ end
83
+
84
+ def initial_search
85
+ response = @client.search(index: @index, scroll: @scroll, body: @body)
86
+ @scroll_id = response['_scroll_id']
87
+ response['hits']['hits']
88
+ end
89
+
90
+ def scroll_request
91
+ @client.scroll(body: {scroll: @scroll, scroll_id: @scroll_id})['hits']['hits']
92
+ end
93
+ end
94
+ end
95
+ end
@@ -16,5 +16,5 @@
16
16
  # under the License.
17
17
 
18
18
  module Elasticsearch
19
- VERSION = '7.13.3'.freeze
19
+ VERSION = '8.14.0'.freeze
20
20
  end