elasticsearch-transport 6.3.0 → 6.8.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7ac9d2e54aee696db7620c3e1b8d9144c28adc52abb6807c35eea61bb01103cb
-  data.tar.gz: 801ee3f07fdd965739ef4d28efe3b658e21d1f3bad2c63aca323f8dc877327bd
+  metadata.gz: 84ce4d26d5dd03b5bc17d89e4ce0a873b5d57083f07cf54cf78552d7e88ff61f
+  data.tar.gz: 69b164ec36cd3cb2ee3355c0806a0109e21a69ca605a9f87b020f9d8476e5096
 SHA512:
-  metadata.gz: 33162c0aafa4702e15a01e86fd5b34962cd28af4146d97f3e69de15b1271cd2ea4021facb12928fd69440fd64a31fef4aaf306a78327da2336958f55a3c36313
-  data.tar.gz: dd0440bd172c86867ada057bd5d1f53b765714b2eb594dee643da5beec9a74da6a0447a07ed813aeb83f21cdcd823429a6f987c610553554f9acd1654e4f077d
+  metadata.gz: e6a9fb8c521ffedd0faf79006e477e4360a2af6df876cee41ddd87278ddf06de34a69e9ccbdad0d83629a0cf1c52d0eefbf4facdb38a591a219c156a5d7d3fdf
+  data.tar.gz: 2d729907c495a2946b68a193262b14261534d6811b2ea5517248977758ed869c3c2803284ff69bc9b9f5a863a9440a9f675f53287f136e5813677b95b1d106c8

data/README.md

@@ -28,12 +28,16 @@ Features overview:
 * Node reloading (based on cluster state) on errors or on demand
 
 For optimal performance, use a HTTP library which supports persistent ("keep-alive") connections,
-such as [Typhoeus](https://github.com/typhoeus/typhoeus).
-Just require the library (`require 'typhoeus'; require 'typhoeus/adapters/faraday'`) in your code,
-and it will be automatically used; currently these libraries will be automatically detected and used:
-[Patron](https://github.com/toland/patron),
-[HTTPClient](https://rubygems.org/gems/httpclient) and
-[Net::HTTP::Persistent](https://rubygems.org/gems/net-http-persistent).
+such as [patron](https://github.com/toland/patron) or [Typhoeus](https://github.com/typhoeus/typhoeus).
+Just require the library (`require 'patron'`) in your code, and it will be automatically used.
+
+Currently these libraries will be automatically detected and used:
+- [Patron](https://github.com/toland/patron)
+- [Typhoeus](https://github.com/typhoeus/typhoeus)
+- [HTTPClient](https://rubygems.org/gems/httpclient)
+- [Net::HTTP::Persistent](https://rubygems.org/gems/net-http-persistent)
+
+**Note on [Typhoeus](https://github.com/typhoeus/typhoeus)**: You need to use v1.4.0 or up since older versions are not compatible with Faraday 1.0.
 
 For detailed information, see example configurations [below](#transport-implementations).
 
@@ -69,6 +73,22 @@ Full documentation is available at <http://rubydoc.info/gems/elasticsearch-trans
 
 ## Configuration
 
+* [Setting Hosts](#setting-hosts)
+* [Default port](#default-port)
+* [Connect using an Elastic Cloud ID](#connect-using-an-elastic-cloud-id)
+* [Authentication](#authentication)
+* [Logging](#logging)
+* [Identifying running tasks with X-Opaque-Id](#identifying-running-tasks-with-x-opaque-id)
+* [Setting Timeouts](#setting-timeouts)
+* [Randomizing Hosts](#randomizing-hosts)
+* [Retrying on Failures](#retrying-on-failures)
+* [Reloading Hosts](#reloading-hosts)
+* [Connection Selector](#connection-selector)
+* [Transport Implementations](#transport-implementations)
+* [Serializer implementations](#serializer-implementations)
+* [Exception Handling](#exception-handling)
+* [Development and Community](#development-and-community)
+
 The client supports many configurations options for setting up and managing connections,
 configuring logging, customizing the transport library, etc.
 
@@ -128,6 +148,26 @@ use the `transport_options` option:
     Elasticsearch::Client.new url: 'https://username:password@example.com:9200',
                               transport_options: { ssl: { ca_file: '/path/to/cacert.pem' } }
 
+You can also use [**API Key authentication**](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html):
+
+``` ruby
+Elasticsearch::Client.new(
+  host: host,
+  transport_options: transport_options,
+  api_key: credentials
+)
+```
+
+Where credentials is either the base64 encoding of `id` and `api_key` joined by a colon or a hash with the `id` and `api_key`:
+
+``` ruby
+Elasticsearch::Client.new(
+  host: host,
+  transport_options: transport_options,
+  api_key: {id: 'my_id', api_key: 'my_api_key'}
+)
+```
+
 ### Logging
 
 To log requests and responses to standard output with the default logger (an instance of Ruby's {::Logger} class),
@@ -158,6 +198,29 @@ You can pass the client any conforming logger implementation:
 
     client = Elasticsearch::Client.new logger: log
 
+### Identifying running tasks with X-Opaque-Id
+
+The X-Opaque-Id header allows to track certain calls, or associate certain tasks with the client that started them ([more on the Elasticsearch docs](https://www.elastic.co/guide/en/elasticsearch/reference/master/tasks.html#_identifying_running_tasks)). To use this feature, you need to set an id for `opaque_id` on the client on each request. Example:
+
+```ruby
+client = Elasticsearch::Client.new
+client.search(index: 'myindex', q: 'title:test', opaque_id: '123456')
+```
+The search request will include the following HTTP Header:
+```
+X-Opaque-Id: 123456
+```
+
+You can also set a prefix for X-Opaque-Id when initializing the client. This will be prepended to the id you set before each request if you're using X-Opaque-Id. Example:
+```ruby
+client = Elasticsearch::Client.new(opaque_id_prefix: 'eu-west1')
+client.search(index: 'myindex', q: 'title:test', opaque_id: '123456')
+```
+The request will include the following HTTP Header:
+```
+X-Opaque-Id: eu-west1_123456
+```
+
 ### Setting Timeouts
 
 For many operations in Elasticsearch, the default timeouts of HTTP libraries are too low.
@@ -292,44 +355,35 @@ constructor, use the `transport_options` key:
 
 To configure the _Faraday_ instance directly, use a block:
 
-    require 'typhoeus'
-    require 'typhoeus/adapters/faraday'
+    require 'patron'
 
     client = Elasticsearch::Client.new(host: 'localhost', port: '9200') do |f|
       f.response :logger
-      f.adapter  :typhoeus
+      f.adapter  :patron
     end
 
-You can use any standard Faraday middleware and plugins in the configuration block,
-for example sign the requests for the [AWS Elasticsearch service](https://aws.amazon.com/elasticsearch-service/):
-
-    require 'faraday_middleware/aws_signers_v4'
-
-    client = Elasticsearch::Client.new url: 'https://search-my-cluster-abc123....es.amazonaws.com' do |f|
-      f.request :aws_signers_v4,
-                credentials: Aws::Credentials.new(ENV['AWS_ACCESS_KEY'], ENV['AWS_SECRET_ACCESS_KEY']),
-                service_name: 'es',
-                region: 'us-east-1'
-    end
+You can use any standard Faraday middleware and plugins in the configuration block, for example sign the requests for the [AWS Elasticsearch service](https://aws.amazon.com/elasticsearch-service/). See [the AWS documentation](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-request-signing.html#es-request-signing-ruby) for an example.
 
 You can also initialize the transport class yourself, and pass it to the client constructor
 as the `transport` argument:
 
-    require 'typhoeus'
-    require 'typhoeus/adapters/faraday'
+```ruby
+require 'patron'
 
-    transport_configuration = lambda do |f|
-      f.response :logger
-      f.adapter  :typhoeus
-    end
+transport_configuration = lambda do |f|
+  f.response :logger
+  f.adapter  :patron
+end
 
-    transport = Elasticsearch::Transport::Transport::HTTP::Faraday.new \
-      hosts: [ { host: 'localhost', port: '9200' } ],
-      &transport_configuration
+transport = Elasticsearch::Transport::Transport::HTTP::Faraday.new \
+  hosts: [ { host: 'localhost', port: '9200' } ],
+  &transport_configuration
+
+# Pass the transport to the client
+#
+client = Elasticsearch::Client.new transport: transport
+```
 
-    # Pass the transport to the client
-    #
-    client = Elasticsearch::Client.new transport: transport
 
 Instead of passing the transport to the constructor, you can inject it at run time:
 

data/elasticsearch-transport.gemspec

@@ -4,75 +4,56 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'elasticsearch/transport/version'
 
 Gem::Specification.new do |s|
-  s.name          = "elasticsearch-transport"
+  s.name          = 'elasticsearch-transport'
   s.version       = Elasticsearch::Transport::VERSION
-  s.authors       = ["Karel Minarik"]
-  s.email         = ["karel.minarik@elasticsearch.org"]
-  s.summary       = "Ruby client for Elasticsearch."
-  s.homepage      = "https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch-transport"
-  s.license       = "Apache-2.0"
-
+  s.authors       = ['Karel Minarik']
+  s.email         = ['karel.minarik@elasticsearch.org']
+  s.summary       = 'Ruby client for Elasticsearch.'
+  s.homepage      = 'https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current/index.html'
+  s.license       = 'Apache-2.0'
+  s.metadata = {
+    'homepage_uri' => 'https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current/index.html',
+    'changelog_uri' => 'https://github.com/elastic/elasticsearch-ruby/blob/6.x/CHANGELOG.md',
+    'source_code_uri' => 'https://github.com/elastic/elasticsearch-ruby/tree/6.x/elasticsearch-transport',
+    'bug_tracker_uri' => 'https://github.com/elastic/elasticsearch-ruby/issues'
+  }
   s.files         = `git ls-files`.split($/)
   s.executables   = s.files.grep(%r{^bin/}) { |f| File.basename(f) }
   s.test_files    = s.files.grep(%r{^(test|spec|features)/})
-  s.require_paths = ["lib"]
-
-  s.extra_rdoc_files  = [ "README.md", "LICENSE.txt" ]
-  s.rdoc_options      = [ "--charset=UTF-8" ]
-
-  s.required_ruby_version = '>= 1.9'
-
-  s.add_dependency "multi_json"
-  s.add_dependency "faraday"
-
-  if defined?(RUBY_VERSION) && RUBY_VERSION < '1.9'
-    s.add_dependency "system_timer"
-  end
-
-  s.add_development_dependency "bundler"
-
-  if defined?(RUBY_VERSION) && RUBY_VERSION > '1.9'
-    s.add_development_dependency "rake", "~> 11.1"
-  else
-    s.add_development_dependency "rake", "< 11.0"
-  end
-
-  if defined?(RUBY_VERSION) && RUBY_VERSION > '1.9'
-    s.add_development_dependency "elasticsearch-extensions"
-  end
-
-  s.add_development_dependency "ansi"
-  s.add_development_dependency "shoulda-context"
-  s.add_development_dependency "mocha"
-  s.add_development_dependency "turn"
-  s.add_development_dependency "yard"
-  s.add_development_dependency "pry"
-
+  s.require_paths = ['lib']
+
+  s.extra_rdoc_files  = ['README.md', 'LICENSE.txt']
+  s.rdoc_options      = ['--charset=UTF-8']
+
+  s.required_ruby_version = '>= 2.4'
+
+  s.add_dependency 'multi_json'
+  s.add_dependency 'faraday', '~> 1'
+  s.add_development_dependency 'ansi'
+  s.add_development_dependency 'bundler'
+  s.add_development_dependency 'elasticsearch-extensions'
+  s.add_development_dependency 'mocha'
+  s.add_development_dependency 'pry'
+  s.add_development_dependency 'rake', '~> 13'
+  s.add_development_dependency 'shoulda-context'
+  s.add_development_dependency 'turn'
+  s.add_development_dependency 'yard'
   # Gems for testing integrations
-  s.add_development_dependency "curb"   unless defined? JRUBY_VERSION
-  s.add_development_dependency "patron" unless defined? JRUBY_VERSION
-  s.add_development_dependency "typhoeus", '~> 0.6'
-  s.add_development_dependency "net-http-persistent"
-  s.add_development_dependency "manticore", '~> 0.5.2' if defined? JRUBY_VERSION
-  s.add_development_dependency "hashie"
-
-  # Prevent unit test failures on Ruby 1.8
-  if defined?(RUBY_VERSION) && RUBY_VERSION < '1.9'
-    s.add_development_dependency "test-unit", '~> 2'
-    s.add_development_dependency "json", '~> 1.8'
-  end
-
-  if defined?(RUBY_VERSION) && RUBY_VERSION > '1.9'
-    s.add_development_dependency "minitest", "~> 4.0"
-    s.add_development_dependency "ruby-prof"    unless defined?(JRUBY_VERSION) || defined?(Rubinius)
-    s.add_development_dependency "require-prof" unless defined?(JRUBY_VERSION) || defined?(Rubinius)
-    s.add_development_dependency "simplecov"
-    s.add_development_dependency "simplecov-rcov"
-    s.add_development_dependency "cane"
-  end
-
-  if defined?(RUBY_VERSION) && RUBY_VERSION > '2.2'
-    s.add_development_dependency "test-unit", '~> 2'
+  s.add_development_dependency 'cane'
+  s.add_development_dependency 'hashie'
+  s.add_development_dependency 'httpclient'
+  s.add_development_dependency 'manticore', '~> 0.5.2' if defined? JRUBY_VERSION
+  s.add_development_dependency 'minitest', '~> 4.0'
+  s.add_development_dependency 'net-http-persistent'
+  s.add_development_dependency 'simplecov', '~> 0.17', '< 0.18'
+  s.add_development_dependency 'simplecov-rcov'
+  s.add_development_dependency 'test-unit', '~> 2'
+  s.add_development_dependency 'typhoeus', '~> 1.4'
+  unless defined?(JRUBY_VERSION) || defined?(Rubinius)
+    s.add_development_dependency 'curb'
+    s.add_development_dependency 'patron'
+    s.add_development_dependency 'require-prof'
+    s.add_development_dependency 'ruby-prof'
   end
 
   s.description = <<-DESC.gsub(/^    /, '')

data/lib/elasticsearch-transport.rb

@@ -1 +1,5 @@
+# Licensed to Elasticsearch B.V under one or more agreements.
+# Elasticsearch B.V licenses this file to you under the Apache 2.0 License.
+# See the LICENSE file in the project root for more information
+
 require 'elasticsearch/transport'

data/lib/elasticsearch/transport.rb

@@ -1,3 +1,7 @@
+# Licensed to Elasticsearch B.V under one or more agreements.
+# Elasticsearch B.V licenses this file to you under the Apache 2.0 License.
+# See the LICENSE file in the project root for more information
+
 require "uri"
 require "time"
 require "timeout"

data/lib/elasticsearch/transport/client.rb

@@ -1,3 +1,7 @@
+# Licensed to Elasticsearch B.V under one or more agreements.
+# Elasticsearch B.V licenses this file to you under the Apache 2.0 License.
+# See the LICENSE file in the project root for more information
+
 module Elasticsearch
   module Transport
 
@@ -85,11 +89,19 @@ module Elasticsearch
       # @option arguments [String] :send_get_body_as Specify the HTTP method to use for GET requests with a body.
       #                                              (Default: GET)
       #
+      # @option api_key [String, Hash] :api_key Use API Key Authentication, either the base64 encoding of `id` and `api_key`
+      #                                         joined by a colon as a String, or a hash with the `id` and `api_key` values.
+      # @option opaque_id_prefix [String] :opaque_id_prefix set a prefix for X-Opaque-Id when initializing the client. This
+      #                                                     will be prepended to the id you set before each request if
+      #                                                     you're using X-Opaque-Id
+      # @option enable_meta_header [Boolean] :enable_meta_header Enable sending the meta data header to Cloud.
+      #                                                          (Default: true)
+      #
       # @yield [faraday] Access and configure the `Faraday::Connection` instance directly with a block
       #
-      def initialize(arguments={}, &block)
-        @options = arguments
-        @arguments = arguments
+      def initialize(arguments = {}, &block)
+        @options = arguments.each_with_object({}){ |(k,v), args| args[k.to_sym] = v }
+        @arguments = @options
         @arguments[:logger] ||= @arguments[:log]   ? DEFAULT_LOGGER.call() : nil
         @arguments[:tracer] ||= @arguments[:trace] ? DEFAULT_TRACER.call() : nil
         @arguments[:reload_connections] ||= false
@@ -98,9 +110,13 @@ module Elasticsearch
         @arguments[:randomize_hosts]    ||= false
         @arguments[:transport_options]  ||= {}
         @arguments[:http]               ||= {}
-        @options[:http]               ||= {}
+        @arguments[:enable_meta_header] = arguments.fetch(:enable_meta_header) { true }
+        @options[:http]                 ||= {}
+
+        set_api_key if (@api_key = @arguments[:api_key])
 
-        @seeds = __extract_hosts(@arguments[:hosts] ||
+
+        @seeds ||= __extract_hosts(@arguments[:hosts] ||
                                      @arguments[:host] ||
                                      @arguments[:url] ||
                                      @arguments[:urls] ||
@@ -108,9 +124,10 @@ module Elasticsearch
                                      DEFAULT_HOST)
 
         @send_get_body_as = @arguments[:send_get_body_as] || 'GET'
+        @opaque_id_prefix = @arguments[:opaque_id_prefix] || nil
 
         if @arguments[:request_timeout]
-          @arguments[:transport_options][:request] = { :timeout => @arguments[:request_timeout] }
+          @arguments[:transport_options][:request] = { timeout: @arguments[:request_timeout] }
         end
 
         @arguments[:transport_options][:headers] ||= {}
@@ -122,29 +139,120 @@ module Elasticsearch
         if @arguments[:transport]
           @transport = @arguments[:transport]
         else
-          transport_class  = @arguments[:transport_class] || DEFAULT_TRANSPORT_CLASS
-          if transport_class == Transport::HTTP::Faraday
-            @transport = transport_class.new(:hosts => @seeds, :options => @arguments) do |faraday|
-              block.call faraday if block
-              unless (h = faraday.builder.handlers.last) && h.name.start_with?("Faraday::Adapter")
-                faraday.adapter(@arguments[:adapter] || __auto_detect_adapter)
-              end
-            end
-          else
-            @transport = transport_class.new(:hosts => @seeds, :options => @arguments)
-          end
+          @transport_class = @arguments[:transport_class] || DEFAULT_TRANSPORT_CLASS
+          @transport = if @transport_class == Transport::HTTP::Faraday
+                         @arguments[:adapter] ||= __auto_detect_adapter
+                         set_meta_header
+                         @transport_class.new(hosts: @seeds, options: @arguments) do |faraday|
+                           faraday.adapter(@arguments[:adapter])
+                           block&.call faraday
+                         end
+                       else
+                         set_meta_header
+                         @transport_class.new(hosts: @seeds, options: @arguments)
+                       end
         end
       end
 
       # Performs a request through delegation to {#transport}.
       #
-      def perform_request(method, path, params={}, body=nil, headers=nil)
+      def perform_request(method, path, params = {}, body = nil, headers = nil)
         method = @send_get_body_as if 'GET' == method && body
+        if (opaque_id = params.delete(:opaque_id))
+          headers = {} if headers.nil?
+          opaque_id = @opaque_id_prefix ? "#{@opaque_id_prefix}#{opaque_id}" : opaque_id
+          headers.merge!('X-Opaque-Id' => opaque_id)
+        end
         transport.perform_request(method, path, params, body, headers)
       end
 
       private
 
+      def set_api_key
+        @api_key = __encode(@api_key) if @api_key.is_a? Hash
+        add_header('Authorization' => "ApiKey #{@api_key}")
+        @arguments.delete(:user)
+        @arguments.delete(:password)
+      end
+
+      def add_header(header)
+        headers = @arguments[:transport_options]&.[](:headers) || {}
+        headers.merge!(header)
+        @arguments[:transport_options].merge!(
+          headers: headers
+        )
+      end
+
+      def set_meta_header
+        return if @arguments[:enable_meta_header] == false
+
+        service, version = meta_header_service_version
+
+        meta_headers = {
+          service.to_sym => version,
+          rb: RUBY_VERSION,
+          t: Elasticsearch::Transport::VERSION
+        }
+        meta_headers.merge!(meta_header_engine) if meta_header_engine
+        meta_headers.merge!(meta_header_adapter) if meta_header_adapter
+
+        add_header({ 'x-elastic-client-meta' => meta_headers.map { |k, v| "#{k}=#{v}" }.join(',') })
+      end
+
+      def meta_header_service_version
+        if defined?(Elastic::META_HEADER_SERVICE_VERSION)
+          Elastic::META_HEADER_SERVICE_VERSION
+        elsif defined?(Elasticsearch::VERSION)
+          [:es, client_meta_version(Elasticsearch::VERSION)]
+        else
+          [:es, client_meta_version(Elasticsearch::Transport::VERSION)]
+        end
+      end
+
+      def client_meta_version(version)
+        regexp = /^([0-9]+\.[0-9]+\.[0-9]+)(\.?[a-z0-9.-]+)?$/
+        match = version.match(regexp)
+        return "#{match[1]}p" if (match[2])
+
+        version
+      end
+
+      def meta_header_engine
+        case RUBY_ENGINE
+        when 'ruby'
+          {}
+        when 'jruby'
+          { jv: ENV_JAVA['java.version'], jr: JRUBY_VERSION }
+        when 'rbx'
+          { rbx: RUBY_VERSION }
+        else
+          { RUBY_ENGINE.to_sym => RUBY_VERSION }
+        end
+      end
+
+      def meta_header_adapter
+        if @transport_class == Transport::HTTP::Faraday
+          {fd: Faraday::VERSION}.merge(
+            case @arguments[:adapter]
+            when :patron
+              {pt: Patron::VERSION}
+            when :net_http
+              {nh: defined?(Net::HTTP::VERSION) ? Net::HTTP::VERSION : Net::HTTP::HTTPVersion}
+            when :typhoeus
+              {ty: Typhoeus::VERSION}
+            when :httpclient
+              {hc: HTTPClient::VERSION}
+            when :net_http_persistent
+              {np: Net::HTTP::Persistent::VERSION}
+            end
+          )
+        elsif defined?(Transport::HTTP::Curb) && @transport_class == Transport::HTTP::Curb
+          {cl: Curl::CURB_VERSION}
+        elsif defined?(Transport::HTTP::Manticore) && @transport_class == Transport::HTTP::Manticore
+          {mc: Manticore::VERSION}
+        end
+      end
+
       # Normalizes and returns hosts configuration.
       #
       # Arrayifies the `hosts_config` argument and extracts `host` and `port` info from strings.
@@ -202,8 +310,14 @@ module Elasticsearch
                        raise ArgumentError, "Please pass host as a String, URI or Hash -- #{host.class} given."
                      end
 
-        @options[:http][:user] ||= host_parts[:user]
-        @options[:http][:password] ||= host_parts[:password]
+        if @api_key
+          # Remove Basic Auth if using API KEY
+          host_parts.delete(:user)
+          host_parts.delete(:password)
+        else
+          @options[:http][:user] ||= host_parts[:user]
+          @options[:http][:password] ||= host_parts[:password]
+        end
 
         host_parts[:port] = host_parts[:port].to_i if host_parts[:port]
         host_parts[:path].chomp!('/') if host_parts[:path]
@@ -232,6 +346,13 @@ module Elasticsearch
           ::Faraday.default_adapter
         end
       end
+
+      # Encode credentials for the Authorization Header
+      # Credentials is the base64 encoding of id and api_key joined by a colon
+      # @see https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html
+      def __encode(api_key)
+        Base64.strict_encode64([api_key[:id], api_key[:api_key]].join(':'))
+      end
     end
   end
 end