fluent-plugin-elasticsearch 4.0.11 → 4.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c90e0b347a5c90ec23231df927384dadcc4b91b813a4e2c11df23539dfe05628
-  data.tar.gz: d6e296c8006870a5f85bd0b97294c69d797fc3f40b574ede7645fcd808ae6356
+  metadata.gz: c7923225803a37c0076c7cb4c4d6e94c1ec9ccf120ce4067b45db8c0fc19f17f
+  data.tar.gz: f1ce4098fb1e4e563e00aae9fe8d30b2bc9cfcb08100616f0ff00b3dfa210667
 SHA512:
-  metadata.gz: 8466c4e3a13ef5c5ca1bda536a0baf7044fdaa0bc3ae1e1c7e6d0e6cf3dc7596d2b68e3f61ed2fb788f8ef2d8392939b8f63fd6aedc53e5818725bc6e0910aa6
-  data.tar.gz: b0572e50952ce49e44b280e94764d0d204d482e8e89cc2124e9a3025c1bd4cf7debe5625ad141337e5f81e6b6816286ee81f931f6b782f0a8cffcbfa0ed12dc3
+  metadata.gz: '0579350a7eba583c7c4784f5500d9faa6df4e398b4b0d0e4aacf230502751a39fb9591d189217c869a82f6da689ffb435445aea7bd5920332787e787170b37d7'
+  data.tar.gz: 8d150716862d82b313a31b104842b368aac3f945bece0a11074bf43a25830c91e6b5ee0556a23e93fdc2159ab20185e08847b4c94bb10033fe7b259741b332b6

data/History.md

@@ -2,6 +2,24 @@
 
 ### [Unreleased]
 
+### 4.1.4
+- ElasticsearchGenID update docs for hash_type (#809)
+- Handle api key header (#808)
+- index_template: Handle error object entirely on index creation failure (#807)
+
+### 4.1.3
+- Load multiple templates even if template_name and template_file given (#799)
+- Handle elasticsearch-ruby 7.9.0 using HTTP method changes (#795)
+
+### 4.1.2
+- Use Hash#dig instead of Hash#[] on retrieving version information from Elasticsearch info API (#793)
+
+### 4.1.1
+- Correct ILM explain on logstash_format case (#786)
+
+### 4.1.0
+- Implement Fallback selector and configurable selector class (#782)
+
 ### 4.0.11
 - Add custom and time placeholders combination testcase for ILM (#781)
 - Really support ILM on `logstash_format` enabled environment (#779)

data/README.ElasticsearchGenID.md

@@ -26,7 +26,7 @@ In your Fluentd configuration, use `@type elasticsearch_genid`. Additional confi
   use_entire_record false
   record_keys []
   separator _
-  hash_type md5
+  hash_type sha1
 </match>
 ```
 
@@ -95,7 +95,7 @@ You can specify separator charactor to creating seed for hash generation.
 hash_type sha1
 ```
 
-You can specify hash algorithm.
+You can specify hash algorithm. Support algorithms `md5`, `sha1`, `sha256`, `sha512`. Default: `sha1`
 
 ## Advanced Usage
 

data/README.md

@@ -78,12 +78,14 @@ Current maintainers: @cosmo0920
   + [Hash flattening](#hash-flattening)
   + [Generate Hash ID](#generate-hash-id)
   + [sniffer_class_name](#sniffer-class-name)
+  + [selector_class_name](#selector-class-name)
   + [reload_after](#reload-after)
   + [validate_client_version](#validate-client-version)
   + [unrecoverable_error_types](#unrecoverable-error-types)
   + [verify_es version at startup](#verify_es_version_at_startup)
   + [default_elasticsearch_version](#default_elasticsearch_version)
   + [custom_headers](#custom_headers)
+  + [api_key](#api_key)
   + [Not seeing a config you need?](#not-seeing-a-config-you-need)
   + [Dynamic configuration](#dynamic-configuration)
   + [Placeholders](#placeholders)
@@ -120,10 +122,12 @@ Current maintainers: @cosmo0920
 
 ## Requirements
 
-| fluent-plugin-elasticsearch  | fluentd | ruby |
-|-------------------|---------|------|
-| >= 2.0.0 | >= v0.14.20 | >= 2.1 |
-|  < 2.0.0 | >= v0.12.0 | >= 1.9 |
+| fluent-plugin-elasticsearch  | fluentd     | ruby   |
+|:----------------------------:|:-----------:|:------:|
+| >= 4.0.1                     | >= v0.14.22 | >= 2.3 |
+| >= 3.2.4 && < 4.0.1          | >= v0.14.22 | >= 2.1 |
+| >= 2.0.0 && < 3.2.3          | >= v0.14.20 | >= 2.1 |
+|  < 2.0.0                     | >= v0.12.0  | >= 1.9 |
 
 NOTE: For v0.12 version, you should use 1.x.y version. Please send patch into v0.12 branch if you encountered 1.x version's bug.
 
@@ -428,7 +432,7 @@ Specify index templates in form of hash. Can contain multiple templates.
 templates { "template_name_1": "path_to_template_1_file", "template_name_2": "path_to_template_2_file"}
 ```
 
-If `template_file` and `template_name` are set, then this parameter will be ignored.
+**Note:** Before ES plugin v4.1.2, if `template_file` and `template_name` are set, then this parameter will be ignored. In 4.1.3 or later, `template_file` and `template_name` can work with `templates`.
 
 ### customize_template
 
@@ -478,6 +482,8 @@ deflector_alias test-current
 
 If [rollover_index](#rollover_index) is set, then this parameter will be in effect otherwise ignored.
 
+**NOTE:** Since 4.1.1, `deflector_alias` is prohibited to use with `enable_ilm`.
+
 ### index_prefix
 
 This parameter is marked as obsoleted.
@@ -989,7 +995,7 @@ reload_after 100
 
 #### Tips
 
-The included sniffer class does not required `out_elasticsearch`.
+The included sniffer class is not required `out_elasticsearch`.
 You should tell Fluentd where the sniffer class exists.
 
 If you use td-agent, you must put the following lines into `TD_AGENT_DEFAULT` file:
@@ -1006,6 +1012,38 @@ sniffer=$(td-agent-gem contents fluent-plugin-elasticsearch|grep elasticsearch_s
 $ fluentd -r $sniffer [AND YOUR OTHER OPTIONS]
 ```
 
+### Selector Class Name
+
+The default selector used by the `Elasticsearch::Transport` class works well when Fluentd should behave round robin and random selector cases. This doesn't work well when Fluentd should behave fallbacking from exhausted ES cluster to normal ES cluster.
+The parameter `selector_class_name` gives you the ability to provide your own Selector class to implement whatever selection nodes logic you require.
+
+The below configuration is using plugin built-in `ElasticseatchFallbackSelector`:
+
+```
+hosts exhausted-host:9201,normal-host:9200
+selector_class_name "Fluent::Plugin::ElasticseatchFallbackSelector"
+```
+
+#### Tips
+
+The included selector class is required in `out_elasticsearch` by default.
+But, your custom selector class is not required in `out_elasticsearch`.
+You should tell Fluentd where the selector class exists.
+
+If you use td-agent, you must put the following lines into `TD_AGENT_DEFAULT` file:
+
+```
+selector=/path/to/your_awesome_selector.rb
+TD_AGENT_OPTIONS="--use-v1-config -r $selector"
+```
+
+If you use Fluentd directly, you must pass the following lines as Fluentd command line option:
+
+```
+selector=/path/to/your_awesome_selector.rb
+$ fluentd -r $selector [AND YOUR OTHER OPTIONS]
+```
+
 ### Reload After
 
 When `reload_connections true`, this is the integer number of operations after which the plugin will
@@ -1069,6 +1107,14 @@ This parameter adds additional headers to request. The default value is `{}`.
 custom_headers {"token":"secret"}
 ```
 
+### api_key
+
+This parameter adds authentication header. The default value is `nil`.
+
+```
+api_key "ElasticsearchAPIKEY"
+```
+
 ### Not seeing a config you need?
 
 We try to keep the scope of this plugin small and not add too many configuration options. If you think an option would be useful to others, feel free to open an issue or contribute a Pull Request.
@@ -1720,7 +1766,7 @@ ILM target index alias is created with `index_name` or an index which is calcula
 
 From Elasticsearch plugin v4.0.0, ILM target index will be calculated from `index_name` (normal mode) or `logstash_prefix` (using with `logstash_format`as true).
 
-When using `deflector_alias` parameter, Elasticsearch plugin will create ILM target indices alias with `deflector_alias` instead of `index_name` or an index which is calculated from `logstash_prefix`. This behavior should be kept due to backward ILM feature compatibility.
+**NOTE:** Before Elasticsearch plugin v4.1.0, using `deflector_alias` parameter when ILM is enabled is permitted and handled, but, in the later releases such that 4.1.1 or later, it cannot use with when ILM is enabled.
 
 And also, ILM feature users should specify their Elasticsearch template for ILM enabled indices.
 Because ILM settings are injected into their Elasticsearch templates.
@@ -1733,7 +1779,13 @@ It usually should be used with default value which is `default`.
 
 Then, ILM parameters are used in alias index like as:
 
-`<index_name/logstash_prefix><index_separator><application_name>-000001`.
+##### Simple `index_name` case:
+
+`<index_name><index_separator><application_name>-000001`.
+
+##### `logstash_format` as `true` case:
+
+`<logstash_prefix><logstash_prefix_separator><application_name><logstash_prefix_separator><logstash_dateformat>-000001`.
 
 #### Example ILM settings
 

data/fluent-plugin-elasticsearch.gemspec

@@ -3,7 +3,7 @@ $:.push File.expand_path('../lib', __FILE__)
 
 Gem::Specification.new do |s|
   s.name          = 'fluent-plugin-elasticsearch'
-  s.version       = '4.0.11'
+  s.version       = '4.1.4'
   s.authors       = ['diogo', 'pitr', 'Hiroshi Hatake']
   s.email         = ['pitr.vern@gmail.com', 'me@diogoterror.com', 'cosmo0920.wp@gmail.com']
   s.description   = %q{Elasticsearch output plugin for Fluent event collector}

data/lib/fluent/plugin/elasticsearch_fallback_selector.rb

@@ -0,0 +1,9 @@
+require 'elasticsearch/transport/transport/connections/selector'
+
+class Fluent::Plugin::ElasticseatchFallbackSelector
+  include Elasticsearch::Transport::Transport::Connections::Selector::Base
+
+  def select(options={})
+    connections.first
+  end
+end

data/lib/fluent/plugin/elasticsearch_index_template.rb

@@ -61,15 +61,16 @@ module Fluent::ElasticsearchIndexTemplate
     if e.message =~ /"already exists"/ || e.message =~ /resource_already_exists_exception/
       log.debug("Index #{index_name} already exists")
     else
-      log.error("Error while index creation - #{index_name}: #{e.inspect}")
+      log.error("Error while index creation - #{index_name}", error: e)
     end
   end
 
-  def template_install(name, template_file, overwrite, enable_ilm = false, deflector_alias_name = nil, ilm_policy_id = nil, host = nil)
+  def template_install(name, template_file, overwrite, enable_ilm = false, deflector_alias_name = nil, ilm_policy_id = nil, host = nil, target_index = nil)
     inject_template_name = get_template_name(enable_ilm, name, deflector_alias_name)
     if overwrite
       template_put(inject_template_name,
                    enable_ilm ? inject_ilm_settings_to_template(deflector_alias_name,
+                                                                target_index,
                                                                 ilm_policy_id,
                                                                 get_template(template_file)) :
                      get_template(template_file), host)
@@ -80,6 +81,7 @@ module Fluent::ElasticsearchIndexTemplate
     if !template_exists?(inject_template_name, host)
       template_put(inject_template_name,
                    enable_ilm ? inject_ilm_settings_to_template(deflector_alias_name,
+                                                                target_index,
                                                                 ilm_policy_id,
                                                                 get_template(template_file)) :
                      get_template(template_file), host)
@@ -89,10 +91,12 @@ module Fluent::ElasticsearchIndexTemplate
     end
   end
 
-  def template_custom_install(template_name, template_file, overwrite, customize_template, enable_ilm, deflector_alias_name, ilm_policy_id, host)
+  def template_custom_install(template_name, template_file, overwrite, customize_template, enable_ilm, deflector_alias_name, ilm_policy_id, host, target_index)
     template_custom_name = get_template_name(enable_ilm, template_name, deflector_alias_name)
     custom_template = if enable_ilm
-                        inject_ilm_settings_to_template(deflector_alias_name, ilm_policy_id,
+                        inject_ilm_settings_to_template(deflector_alias_name,
+                                                        target_index,
+                                                        ilm_policy_id,
                                                         get_custom_template(template_file,
                                                                             customize_template))
                       else
@@ -115,26 +119,30 @@ module Fluent::ElasticsearchIndexTemplate
     enable_ilm ? deflector_alias_name : template_name
   end
 
-  def inject_ilm_settings_to_template(deflector_alias_name, ilm_policy_id, template)
+  def inject_ilm_settings_to_template(deflector_alias, target_index, ilm_policy_id, template)
     log.debug("Overwriting index patterns when Index Lifecycle Management is enabled.")
     template.delete('template') if template.include?('template')
-    template['index_patterns'] = "#{deflector_alias_name}-*"
-    template['order'] = template['order'] ? template['order'] + deflector_alias_name.split('-').length : 50 + deflector_alias_name.split('-').length
+    template['index_patterns'] = "#{target_index}-*"
+    template['order'] = template['order'] ? template['order'] + target_index.split('-').length : 50 + target_index.split('-').length
     if template['settings'] && (template['settings']['index.lifecycle.name'] || template['settings']['index.lifecycle.rollover_alias'])
       log.debug("Overwriting index lifecycle name and rollover alias when Index Lifecycle Management is enabled.")
     end
-    template['settings'].update({ 'index.lifecycle.name' => ilm_policy_id, 'index.lifecycle.rollover_alias' => deflector_alias_name})
+    template['settings'].update({ 'index.lifecycle.name' => ilm_policy_id, 'index.lifecycle.rollover_alias' => deflector_alias})
     template
   end
 
-  def create_rollover_alias(index_prefix, rollover_index, deflector_alias_name, app_name, index_date_pattern, index_separator, enable_ilm, ilm_policy_id, ilm_policy, ilm_policy_overwrite, host)
+  def create_rollover_alias(target_index, rollover_index, deflector_alias_name, app_name, index_date_pattern, index_separator, enable_ilm, ilm_policy_id, ilm_policy, ilm_policy_overwrite, host)
      # ILM request to create alias.
     if rollover_index || enable_ilm
       if !client.indices.exists_alias(:name => deflector_alias_name)
-        if index_date_pattern.empty?
-          index_name_temp='<'+index_prefix.downcase+index_separator+app_name.downcase+'-000001>'
+        if @logstash_format
+          index_name_temp = '<'+target_index+'-000001>'
         else
-          index_name_temp='<'+index_prefix.downcase+index_separator+app_name.downcase+'-{'+index_date_pattern+'}-000001>'
+          if index_date_pattern.empty?
+            index_name_temp = '<'+target_index.downcase+index_separator+app_name.downcase+'-000001>'
+          else
+            index_name_temp = '<'+target_index.downcase+index_separator+app_name.downcase+'-{'+index_date_pattern+'}-000001>'
+          end
         end
         indexcreation(index_name_temp, host)
         body = rollover_alias_payload(deflector_alias_name)

data/lib/fluent/plugin/out_elasticsearch.rb

@@ -8,6 +8,7 @@ rescue LoadError
 end
 require 'json'
 require 'uri'
+require 'base64'
 begin
   require 'strptime'
 rescue LoadError
@@ -25,6 +26,7 @@ require_relative 'elasticsearch_error_handler'
 require_relative 'elasticsearch_index_template'
 require_relative 'elasticsearch_index_lifecycle_management'
 require_relative 'elasticsearch_tls'
+require_relative 'elasticsearch_fallback_selector'
 begin
   require_relative 'oj_serializer'
 rescue LoadError
@@ -50,12 +52,13 @@ module Fluent::Plugin
       end
     end
 
-    RequestInfo = Struct.new(:host, :index, :ilm_index)
+    RequestInfo = Struct.new(:host, :index, :ilm_index, :ilm_alias)
 
     attr_reader :alias_indexes
     attr_reader :template_names
     attr_reader :ssl_version_options
     attr_reader :compressable_connection
+    attr_reader :api_key_header
 
     helpers :event_emitter, :compat_parameters, :record_accessor, :timer
 
@@ -137,6 +140,7 @@ EOC
     config_param :with_transporter_log, :bool, :default => false
     config_param :emit_error_for_missing_id, :bool, :default => false
     config_param :sniffer_class_name, :string, :default => nil
+    config_param :selector_class_name, :string, :default => nil
     config_param :reload_after, :integer, :default => DEFAULT_RELOAD_AFTER
     config_param :content_type, :enum, list: [:"application/json", :"application/x-ndjson"], :default => :"application/json",
                  :deprecated => <<EOC
@@ -153,6 +157,7 @@ EOC
     config_param :default_elasticsearch_version, :integer, :default => DEFAULT_ELASTICSEARCH_VERSION
     config_param :log_es_400_reason, :bool, :default => false
     config_param :custom_headers, :hash, :default => {}
+    config_param :api_key, :string, :default => nil, :secret => true
     config_param :suppress_doc_wrap, :bool, :default => false
     config_param :ignore_exceptions, :array, :default => [], value_type: :string, :desc => "Ignorable exception list"
     config_param :exception_backup, :bool, :default => true, :desc => "Chunk backup flag when ignore exception occured"
@@ -210,6 +215,8 @@ EOC
         @remove_keys_on_update = @remove_keys_on_update.split ','
       end
 
+      @api_key_header = setup_api_key
+
       raise Fluent::ConfigError, "'max_retry_putting_template' must be greater than or equal to zero." if @max_retry_putting_template < 0
       raise Fluent::ConfigError, "'max_retry_get_es_version' must be greater than or equal to zero." if @max_retry_get_es_version < 0
 
@@ -232,7 +239,7 @@ EOC
       if !dry_run?
         if @template_name && @template_file
           if @enable_ilm
-            raise Fluent::ConfigError, "deflector_alias is prohibited to use with 'logstash_format at same time." if @logstash_format and @deflector_alias
+            raise Fluent::ConfigError, "deflector_alias is prohibited to use with enable_ilm at same time." if @deflector_alias
           end
           if @ilm_policy.empty? && @ilm_policy_overwrite
             raise Fluent::ConfigError, "ilm_policy_overwrite requires a non empty ilm_policy."
@@ -245,7 +252,8 @@ EOC
             template_installation_actual(@deflector_alias ? @deflector_alias : @index_name, @template_name, @customize_template, @application_name, @index_name, @ilm_policy_id)
           end
           verify_ilm_working if @enable_ilm
-        elsif @templates
+        end 
+        if @templates
           retry_operate(@max_retry_putting_template, @fail_on_putting_template_retry_exceed) do
             templates_hash_install(@templates, @template_overwrite)
           end
@@ -300,6 +308,13 @@ EOC
         raise Fluent::ConfigError, "Could not load sniffer class #{@sniffer_class_name}: #{ex}"
       end
 
+      @selector_class = nil
+      begin
+        @selector_class = Object.const_get(@selector_class_name) if @selector_class_name
+      rescue Exception => ex
+        raise Fluent::ConfigError, "Could not load selector class #{@selector_class_name}: #{ex}"
+      end
+
       @last_seen_major_version = if major_version = handle_last_seen_es_major_version
                                    major_version
                                  else
@@ -388,6 +403,12 @@ EOC
       end
     end
 
+    def setup_api_key
+      return {} unless @api_key
+
+      { "Authorization" => "ApiKey " + Base64.strict_encode64(@api_key) }
+    end
+
     def dry_run?
       if Fluent::Engine.respond_to?(:dry_run_mode)
         Fluent::Engine.dry_run_mode
@@ -452,7 +473,10 @@ EOC
 
     def detect_es_major_version
       @_es_info ||= client.info
-      @_es_info["version"]["number"].to_i
+      unless version = @_es_info.dig("version", "number")
+        version = @default_elasticsearch_version
+      end
+      version.to_i
     end
 
     def client_library_version
@@ -545,7 +569,10 @@ EOC
                        else
                          {}
                        end
-        headers = { 'Content-Type' => @content_type.to_s }.merge(@custom_headers).merge(gzip_headers)
+        headers = { 'Content-Type' => @content_type.to_s }
+                    .merge(@custom_headers)
+                    .merge(@api_key_header)
+                    .merge(gzip_headers)
         ssl_options = { verify: @ssl_verify, ca_file: @ca_file}.merge(@ssl_version_options)
 
         transport = Elasticsearch::Transport::Transport::HTTP::Faraday.new(connection_options.merge(
@@ -566,6 +593,7 @@ EOC
                                                                               },
                                                                               sniffer_class: @sniffer_class,
                                                                               serializer_class: @serializer_class,
+                                                                              selector_class: @selector_class,
                                                                               compression: compress_connection,
                                                                             }), &adapter_conf)
         Elasticsearch::Client.new transport: transport
@@ -761,9 +789,9 @@ EOC
         begin
           meta, header, record = process_message(tag, meta, header, time, record, extracted_values)
           info = if @include_index_in_url
-                   RequestInfo.new(host, meta.delete("_index".freeze), meta.delete("_alias".freeze))
+                   RequestInfo.new(host, meta.delete("_index".freeze), meta["_index".freeze], meta.delete("_alias".freeze))
                  else
-                   RequestInfo.new(host, nil, meta.delete("_alias".freeze))
+                   RequestInfo.new(host, nil, meta["_index".freeze], meta.delete("_alias".freeze))
                  end
 
           if split_request?(bulk_message, info)
@@ -934,16 +962,16 @@ EOC
 
     def template_installation_actual(deflector_alias, template_name, customize_template, application_name, target_index, ilm_policy_id, host=nil)
       if template_name && @template_file
-        if @alias_indexes.include? deflector_alias
+        if !@logstash_format && @alias_indexes.include?(deflector_alias)
           log.debug("Index alias #{deflector_alias} already exists (cached)")
-        elsif @template_names.include? template_name
+        elsif !@logstash_format && @template_names.include?(template_name)
           log.debug("Template name #{template_name} already exists (cached)")
         else
           retry_operate(@max_retry_putting_template, @fail_on_putting_template_retry_exceed) do
             if customize_template
-              template_custom_install(template_name, @template_file, @template_overwrite, customize_template, @enable_ilm, deflector_alias, ilm_policy_id, host)
+              template_custom_install(template_name, @template_file, @template_overwrite, customize_template, @enable_ilm, deflector_alias, ilm_policy_id, host, target_index)
             else
-              template_install(template_name, @template_file, @template_overwrite, @enable_ilm, deflector_alias, ilm_policy_id, host)
+              template_install(template_name, @template_file, @template_overwrite, @enable_ilm, deflector_alias, ilm_policy_id, host, target_index)
             end
             ilm_policy = @ilm_policies[ilm_policy_id] || {}
             create_rollover_alias(target_index, @rollover_index, deflector_alias, application_name, @index_date_pattern, @index_separator, @enable_ilm, ilm_policy_id, ilm_policy, @ilm_policy_overwrite, host)
@@ -957,11 +985,11 @@ EOC
     # send_bulk given a specific bulk request, the original tag,
     # chunk, and bulk_message_count
     def send_bulk(data, tag, chunk, bulk_message_count, extracted_values, info)
-      logstash_prefix, _logstash_dateformat, index_name, _type_name, template_name, customize_template, deflector_alias, application_name, _pipeline, ilm_policy_id = extracted_values
+      _logstash_prefix, _logstash_dateformat, index_name, _type_name, template_name, customize_template, deflector_alias, application_name, _pipeline, ilm_policy_id = extracted_values
       if deflector_alias
         template_installation(deflector_alias, template_name, customize_template, application_name, index_name, ilm_policy_id, info.host)
       else
-        template_installation(info.ilm_index, template_name, customize_template, application_name, @logstash_format ? logstash_prefix : index_name, ilm_policy_id, info.host)
+        template_installation(info.ilm_index, template_name, customize_template, application_name, @logstash_format ? info.ilm_alias : index_name, ilm_policy_id, info.host)
       end
 
       begin

data/test/plugin/test_elasticsearch_fallback_selector.rb

@@ -0,0 +1,73 @@
+require_relative '../helper'
+require 'fluent/test/driver/output'
+require 'fluent/plugin/out_elasticsearch'
+
+class ElasticsearchFallbackSelectorTest < Test::Unit::TestCase
+  attr_accessor :index_cmds
+
+  def setup
+    Fluent::Test.setup
+    @driver = nil
+    log = Fluent::Engine.log
+    log.out.logs.slice!(0, log.out.logs.length)
+  end
+
+  def stub_elastic(url="http://localhost:9200/_bulk")
+    stub_request(:post, url).with do |req|
+      @index_cmds = req.body.split("\n").map {|r| JSON.parse(r) }
+    end
+  end
+
+  def stub_elastic_info(url="http://localhost:9200/", version="6.4.2")
+    body ="{\"version\":{\"number\":\"#{version}\"}}"
+    stub_request(:get, url).to_return({:status => 200, :body => body, :headers => { 'Content-Type' => 'json' } })
+  end
+
+  def stub_elastic_info_not_found(url="http://localhost:9200/", version="6.4.2")
+    stub_request(:get, url).to_return(:status => [404, "Not Found"])
+  end
+
+  def stub_elastic_info_unavailable(url="http://localhost:9200/", version="6.4.2")
+    stub_request(:get, url).to_return(:status => [503, "Service Unavailable"])
+  end
+
+  def sample_record(content={})
+    {'age' => 26, 'request_id' => '42', 'parent_id' => 'parent', 'routing_id' => 'routing'}.merge(content)
+  end
+
+  def driver(conf='')
+    @driver ||= Fluent::Test::Driver::Output.new(Fluent::Plugin::ElasticsearchOutput) {
+      # v0.12's test driver assume format definition. This simulates ObjectBufferedOutput format
+      if !defined?(Fluent::Plugin::Output)
+        def format(tag, time, record)
+          [time, record].to_msgpack
+        end
+      end
+    }.configure(conf)
+  end
+
+  def test_fallback_on_info
+    stub_elastic_info_not_found("http://localhost:9202/")
+    stub_elastic_info_unavailable("http://localhost:9201/")
+    stub_elastic_info
+    stub_elastic
+    config = %[
+      hosts localhost:9202,localhost:9201,localhost:9200
+      selector_class_name Fluent::Plugin::ElasticseatchFallbackSelector
+      @log_level debug
+      with_transporter_log true
+      reload_connections true
+      reload_after 10
+    ]
+    assert_raise(Elasticsearch::Transport::Transport::Errors::NotFound) do
+      driver(config)
+    end
+    driver.run(default_tag: 'test') do
+      driver.feed(sample_record)
+    end
+    assert_equal(2, index_cmds.length)
+    assert_equal("fluentd", index_cmds.first['index']['_index'])
+  end
+
+  # TODO: on feed phase test case
+end

data/test/plugin/test_in_elasticsearch.rb

@@ -20,6 +20,11 @@ class ElasticsearchInputTest < Test::Unit::TestCase
     @driver = nil
     log = Fluent::Engine.log
     log.out.logs.slice!(0, log.out.logs.length)
+    @http_method = if Gem::Version.new(Elasticsearch::VERSION) >= Gem::Version.new("7.9.0")
+                     :post
+                   else
+                     :get
+                   end
   end
 
   def driver(conf='')
@@ -313,7 +318,7 @@ class ElasticsearchInputTest < Test::Unit::TestCase
   end
 
   def test_emit
-    stub_request(:get, "http://localhost:9200/fluentd/_search?scroll=1m&size=1000").
+    stub_request(@http_method, "http://localhost:9200/fluentd/_search?scroll=1m&size=1000").
       with(body: "{\"sort\":[\"_doc\"]}").
       to_return(status: 200, body: sample_response.to_s,
                 headers: {'Content-Type' => 'application/json'})
@@ -328,7 +333,7 @@ class ElasticsearchInputTest < Test::Unit::TestCase
 
   def test_emit_with_custom_index_name
     index_name = "logstash"
-    stub_request(:get, "http://localhost:9200/#{index_name}/_search?scroll=1m&size=1000").
+    stub_request(@http_method, "http://localhost:9200/#{index_name}/_search?scroll=1m&size=1000").
       with(body: "{\"sort\":[\"_doc\"]}").
       to_return(status: 200, body: sample_response(index_name).to_s,
                 headers: {'Content-Type' => 'application/json'})
@@ -343,7 +348,7 @@ class ElasticsearchInputTest < Test::Unit::TestCase
 
   def test_emit_with_parse_timestamp
     index_name = "fluentd"
-    stub_request(:get, "http://localhost:9200/#{index_name}/_search?scroll=1m&size=1000").
+    stub_request(@http_method, "http://localhost:9200/#{index_name}/_search?scroll=1m&size=1000").
       with(body: "{\"sort\":[\"_doc\"]}").
       to_return(status: 200, body: sample_response(index_name).to_s,
                 headers: {'Content-Type' => 'application/json'})
@@ -361,7 +366,7 @@ class ElasticsearchInputTest < Test::Unit::TestCase
 
   def test_emit_with_parse_timestamp_and_timstamp_format
     index_name = "fluentd"
-    stub_request(:get, "http://localhost:9200/#{index_name}/_search?scroll=1m&size=1000").
+    stub_request(@http_method, "http://localhost:9200/#{index_name}/_search?scroll=1m&size=1000").
       with(body: "{\"sort\":[\"_doc\"]}").
       to_return(status: 200, body: sample_response(index_name).to_s,
                 headers: {'Content-Type' => 'application/json'})
@@ -380,7 +385,7 @@ class ElasticsearchInputTest < Test::Unit::TestCase
   end
 
   def test_emit_with_docinfo
-    stub_request(:get, "http://localhost:9200/fluentd/_search?scroll=1m&size=1000").
+    stub_request(@http_method, "http://localhost:9200/fluentd/_search?scroll=1m&size=1000").
       with(body: "{\"sort\":[\"_doc\"]}").
       to_return(status: 200, body: sample_response.to_s,
                 headers: {'Content-Type' => 'application/json'})
@@ -399,11 +404,11 @@ class ElasticsearchInputTest < Test::Unit::TestCase
   end
 
   def test_emit_with_slices
-    stub_request(:get, "http://localhost:9200/fluentd/_search?scroll=1m&size=1000").
+    stub_request(@http_method, "http://localhost:9200/fluentd/_search?scroll=1m&size=1000").
       with(body: "{\"sort\":[\"_doc\"],\"slice\":{\"id\":0,\"max\":2}}").
       to_return(status: 200, body: sample_response.to_s,
                 headers: {'Content-Type' => 'application/json'})
-    stub_request(:get, "http://localhost:9200/fluentd/_search?scroll=1m&size=1000").
+    stub_request(@http_method, "http://localhost:9200/fluentd/_search?scroll=1m&size=1000").
       with(body: "{\"sort\":[\"_doc\"],\"slice\":{\"id\":1,\"max\":2}}").
       to_return(status: 200, body: sample_response.to_s,
                 headers: {'Content-Type' => 'application/json'})
@@ -419,12 +424,12 @@ class ElasticsearchInputTest < Test::Unit::TestCase
   end
 
   def test_emit_with_size
-    stub_request(:get, "http://localhost:9200/fluentd/_search?scroll=1m&size=1").
+    stub_request(@http_method, "http://localhost:9200/fluentd/_search?scroll=1m&size=1").
       with(body: "{\"sort\":[\"_doc\"]}").
       to_return(status: 200, body: sample_scroll_response.to_s,
                 headers: {'Content-Type' => 'application/json'})
     connection = 0
-    scroll_request = stub_request(:get, "http://localhost:9200/_search/scroll?scroll=1m").
+    scroll_request = stub_request(@http_method, "http://localhost:9200/_search/scroll?scroll=1m").
       with(
         body: "{\"scroll_id\":\"WomkoUKG0QPB679Ulo6TqQgh3pIGRUmrl9qXXGK3EeiQh9rbYNasTkspZQcJ01uz\"}") do
       connection += 1

data/test/plugin/test_out_elasticsearch.rb

@@ -252,6 +252,7 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
     assert_false instance.compression
     assert_equal :no_compression, instance.compression_level
     assert_true instance.http_backend_excon_nonblock
+    assert_equal({}, instance.api_key_header)
   end
 
   test 'configure compression' do
@@ -707,7 +708,10 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
       Fluent::Plugin::ElasticsearchOutput.module_eval(<<-CODE)
         def detect_es_major_version
           @_es_info ||= client.info
-          @_es_info["version"]["number"].to_i
+          unless version = @_es_info.dig("version", "number")
+            version = @default_elasticsearch_version
+          end
+          version.to_i
         end
       CODE
       Fluent::Plugin::ElasticsearchOutput.module_eval(<<-CODE)
@@ -753,7 +757,10 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
       Fluent::Plugin::ElasticsearchOutput.module_eval(<<-CODE)
         def detect_es_major_version
           @_es_info ||= client.info
-          @_es_info["version"]["number"].to_i
+          unless version = @_es_info.dig("version", "number")
+            version = @default_elasticsearch_version
+          end
+          version.to_i
         end
       CODE
       Fluent::Plugin::ElasticsearchOutput.module_eval(<<-CODE)
@@ -1078,32 +1085,32 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
         with(basic_auth: ['john', 'doe']).
         to_return(:status => 200, :body => "", :headers => {})
       # check if template exists
-      stub_request(:get, "https://logs.google.com:777/es//_template/logstash-log-#{date_str}").
+      stub_request(:get, "https://logs.google.com:777/es//_template/logstash-#{date_str}").
         with(basic_auth: ['john', 'doe']).
         to_return(:status => 404, :body => "", :headers => {})
       # creation
-      stub_request(:put, "https://logs.google.com:777/es//_template/logstash-log-#{date_str}").
+      stub_request(:put, "https://logs.google.com:777/es//_template/logstash-#{date_str}").
         with(basic_auth: ['john', 'doe']).
         to_return(:status => 200, :body => "", :headers => {})
       # check if alias exists
-      stub_request(:head, "https://logs.google.com:777/es//_alias/logstash-log-#{date_str}").
+      stub_request(:head, "https://logs.google.com:777/es//_alias/logstash-#{date_str}").
         with(basic_auth: ['john', 'doe']).
         to_return(:status => 404, :body => "", :headers => {})
-      stub_request(:get, "https://logs.google.com:777/es//_template/logstash-log-#{date_str}").
+      stub_request(:get, "https://logs.google.com:777/es//_template/logstash-#{date_str}").
         with(basic_auth: ['john', 'doe']).
         to_return(status: 404, body: "", headers: {})
-      stub_request(:put, "https://logs.google.com:777/es//_template/logstash-log-#{date_str}").
+      stub_request(:put, "https://logs.google.com:777/es//_template/logstash-#{date_str}").
         with(basic_auth: ['john', 'doe'],
              body: "{\"settings\":{\"number_of_shards\":1,\"index.lifecycle.name\":\"logstash-policy\",\"index.lifecycle.rollover_alias\":\"logstash-log-#{date_str}\"},\"mappings\":{\"type1\":{\"_source\":{\"enabled\":false},\"properties\":{\"host_name\":{\"type\":\"string\",\"index\":\"not_analyzed\"},\"created_at\":{\"type\":\"date\",\"format\":\"EEE MMM dd HH:mm:ss Z YYYY\"}}}},\"index_patterns\":\"logstash-log-#{date_str}-*\",\"order\":53}").
         to_return(status: 200, body: "", headers: {})
       # put the alias for the index
-      stub_request(:put, "https://logs.google.com:777/es//%3Clogstash-log-%7Bnow%2Fw%7Bxxxx.ww%7D%7D-000001%3E").
+      stub_request(:put, "https://logs.google.com:777/es//%3Clogstash-log-#{date_str}-000001%3E").
         with(basic_auth: ['john', 'doe']).
         to_return(:status => 200, :body => "", :headers => {})
 
-      stub_request(:put, "https://logs.google.com:777/es//%3Clogstash-log-%7Bnow%2Fw%7Bxxxx.ww%7D%7D-000001%3E/#{alias_endpoint}/logstash-log-#{date_str}").
+      stub_request(:put, "https://logs.google.com:777/es//%3Clogstash-log-#{date_str}-000001%3E/#{alias_endpoint}/logstash-#{date_str}").
         with(basic_auth: ['john', 'doe'],
-             body: "{\"aliases\":{\"logstash-log-#{date_str}\":{\"is_write_index\":true}}}").
+             body: "{\"aliases\":{\"logstash-#{date_str}\":{\"is_write_index\":true}}}").
         to_return(status: 200, body: "", headers: {})
       stub_request(:get, "https://logs.google.com:777/es//_xpack").
         with(basic_auth: ['john', 'doe']).
@@ -1122,7 +1129,7 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
       driver.run(default_tag: 'test') do
         driver.feed(sample_record)
       end
-      assert_requested(:put, "https://logs.google.com:777/es//_template/logstash-log-#{date_str}", times: 1)
+      assert_requested(:put, "https://logs.google.com:777/es//_template/logstash-#{date_str}", times: 1)
 
       assert_requested(elastic_request)
     end
@@ -1213,51 +1220,10 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
         enable_ilm      true
       }
 
-      # connection start
-      stub_request(:head, "https://logs.google.com:777/es//").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 200, :body => "", :headers => {})
-      # check if template exists
-      stub_request(:get, "https://logs.google.com:777/es//_template/logstash").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 404, :body => "", :headers => {})
-      # creation
-      stub_request(:put, "https://logs.google.com:777/es//_template/logstash").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 200, :body => "", :headers => {})
-      # check if alias exists
-      stub_request(:head, "https://logs.google.com:777/es//_alias/myapp_deflector").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 404, :body => "", :headers => {})
-      stub_request(:get, "https://logs.google.com:777/es//_template/myapp_deflector").
-        with(basic_auth: ['john', 'doe']).
-        to_return(status: 404, body: "", headers: {})
-      stub_request(:put, "https://logs.google.com:777/es//_template/myapp_deflector").
-        with(basic_auth: ['john', 'doe'],
-             body: "{\"settings\":{\"number_of_shards\":1,\"index.lifecycle.name\":\"logstash-policy\",\"index.lifecycle.rollover_alias\":\"myapp_deflector\"},\"mappings\":{\"type1\":{\"_source\":{\"enabled\":false},\"properties\":{\"host_name\":{\"type\":\"string\",\"index\":\"not_analyzed\"},\"created_at\":{\"type\":\"date\",\"format\":\"EEE MMM dd HH:mm:ss Z YYYY\"}}}},\"index_patterns\":\"myapp_deflector-*\",\"order\":51}").
-        to_return(status: 200, body: "", headers: {})
-      # put the alias for the index
-      stub_request(:put, "https://logs.google.com:777/es//%3Clogstash-default-%7Bnow%2Fw%7Bxxxx.ww%7D%7D-000001%3E").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 200, :body => "", :headers => {})
-      stub_request(:put, "https://logs.google.com:777/es//%3Clogstash-default-%7Bnow%2Fw%7Bxxxx.ww%7D%7D-000001%3E/#{alias_endpoint}/myapp_deflector").
-        with(basic_auth: ['john', 'doe'],
-             :body => "{\"aliases\":{\"myapp_deflector\":{\"is_write_index\":true}}}").
-        to_return(:status => 200, :body => "", :headers => {})
-      stub_request(:get, "https://logs.google.com:777/es//_xpack").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 200, :body => '{"features":{"ilm":{"available":true,"enabled":true}}}', :headers => {"Content-Type"=> "application/json"})
-      stub_request(:get, "https://logs.google.com:777/es//_ilm/policy/logstash-policy").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 404, :body => "", :headers => {})
-      stub_request(:put, "https://logs.google.com:777/es//_ilm/policy/logstash-policy").
-        with(basic_auth: ['john', 'doe'],
-             :body => "{\"policy\":{\"phases\":{\"hot\":{\"actions\":{\"rollover\":{\"max_size\":\"50gb\",\"max_age\":\"30d\"}}}}}}").
-        to_return(:status => 200, :body => "", :headers => {})
-
-      driver(config)
-
-      assert_requested(:put, "https://logs.google.com:777/es//_template/myapp_deflector", times: 1)
+      # Should raise error because multiple alias indices IllegalArgument Error on executing ILM feature
+      assert_raise(Fluent::ConfigError) do
+        driver(config)
+      end
     end
 
     def test_template_create_with_rollover_index_and_default_ilm_with_empty_index_date_pattern
@@ -1996,6 +1962,7 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
       application_name myapp
     }
 
+    timestr = Time.now.strftime("%Y.%m.%d")
     # connection start
     stub_request(:head, "https://logs.google.com:777/es//").
       with(basic_auth: ['john', 'doe']).
@@ -2009,17 +1976,17 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
       with(basic_auth: ['john', 'doe']).
       to_return(:status => 200, :body => "", :headers => {})
     # creation of index which can rollover
-    stub_request(:put, "https://logs.google.com:777/es//%3Cmylogs-myapp-%7Bnow%2Fw%7Bxxxx.ww%7D%7D-000001%3E").
+    stub_request(:put, "https://logs.google.com:777/es//%3Cmylogs-myapp-#{timestr}-000001%3E").
       with(basic_auth: ['john', 'doe']).
       to_return(:status => 200, :body => "", :headers => {})
     # check if alias exists
-    timestr = Time.now.strftime("%Y.%m.%d")
-    stub_request(:head, "https://logs.google.com:777/es//_alias/mylogs-myapp-#{timestr}").
+    stub_request(:head, "https://logs.google.com:777/es//_alias/mylogs-#{timestr}").
       with(basic_auth: ['john', 'doe']).
       to_return(:status => 404, :body => "", :headers => {})
     # put the alias for the index
-    stub_request(:put, "https://logs.google.com:777/es//%3Cmylogs-myapp-%7Bnow%2Fw%7Bxxxx.ww%7D%7D-000001%3E/#{alias_endpoint}/mylogs-myapp-#{timestr}").
-      with(basic_auth: ['john', 'doe']).
+    stub_request(:put, "https://logs.google.com:777/es//%3Cmylogs-myapp-#{timestr}-000001%3E/#{alias_endpoint}/mylogs-#{timestr}").
+      with(basic_auth: ['john', 'doe'],
+           body: "{\"aliases\":{\"mylogs-#{timestr}\":{\"is_write_index\":true}}}").
       to_return(:status => 200, :body => "", :headers => {})
 
     driver(config)
@@ -2202,52 +2169,10 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
         enable_ilm      true
       }
 
-      # connection start
-      stub_request(:head, "https://logs.google.com:777/es//").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 200, :body => "", :headers => {})
-      # check if template exists
-      stub_request(:get, "https://logs.google.com:777/es//_template/myapp_alias_template").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 404, :body => "", :headers => {})
-      # creation
-      stub_request(:put, "https://logs.google.com:777/es//_template/myapp_alias_template").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 200, :body => "", :headers => {})
-      # creation of index which can rollover
-      stub_request(:put, "https://logs.google.com:777/es//%3Cmylogs-myapp-%7Bnow%2Fw%7Bxxxx.ww%7D%7D-000001%3E").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 200, :body => "", :headers => {})
-      # check if alias exists
-      stub_request(:head, "https://logs.google.com:777/es//_alias/myapp_deflector").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 404, :body => "", :headers => {})
-      stub_request(:get, "https://logs.google.com:777/es//_template/myapp_deflector").
-        with(basic_auth: ['john', 'doe']).
-        to_return(status: 404, body: "", headers: {})
-      stub_request(:put, "https://logs.google.com:777/es//_template/myapp_deflector").
-        with(basic_auth: ['john', 'doe'],
-             body: "{\"order\":6,\"settings\":{\"index.lifecycle.name\":\"fluentd-policy\",\"index.lifecycle.rollover_alias\":\"myapp_deflector\"},\"mappings\":{},\"aliases\":{\"myapp-logs-alias\":{}},\"index_patterns\":\"myapp_deflector-*\"}").
-        to_return(status: 200, body: "", headers: {})
-      # put the alias for the index
-      stub_request(:put, "https://logs.google.com:777/es//%3Cmylogs-myapp-%7Bnow%2Fw%7Bxxxx.ww%7D%7D-000001%3E/#{alias_endpoint}/myapp_deflector").
-        with(basic_auth: ['john', 'doe'],
-             :body => "{\"aliases\":{\"myapp_deflector\":{\"is_write_index\":true}}}").
-        to_return(:status => 200, :body => "", :headers => {})
-      stub_request(:get, "https://logs.google.com:777/es//_xpack").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 200, :body => '{"features":{"ilm":{"available":true,"enabled":true}}}', :headers => {"Content-Type"=> "application/json"})
-      stub_request(:get, "https://logs.google.com:777/es//_ilm/policy/fluentd-policy").
-        with(basic_auth: ['john', 'doe']).
-        to_return(:status => 404, :body => "", :headers => {})
-      stub_request(:put, "https://logs.google.com:777/es//_ilm/policy/fluentd-policy").
-        with(basic_auth: ['john', 'doe'],
-             :body => "{\"policy\":{\"phases\":{\"hot\":{\"actions\":{\"rollover\":{\"max_size\":\"50gb\",\"max_age\":\"30d\"}}}}}}").
-        to_return(:status => 200, :body => "", :headers => {})
-
-      driver(config)
-
-      assert_requested(:put, "https://logs.google.com:777/es//_template/myapp_deflector", times: 1)
+      # Should raise error because multiple alias indices IllegalArgument Error on executing ILM feature
+      assert_raise(Fluent::ConfigError) do
+        driver(config)
+      end
     end
 
     def test_custom_template_with_rollover_index_create_and_default_ilm_and_placeholders
@@ -2735,7 +2660,7 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
     assert_requested(:put, "https://logs.google.com:777/es//_template/logstash3", times: 1)
   end
 
-  def test_templates_not_used
+  def test_templates_are_also_used
     cwd = File.dirname(__FILE__)
     template_file = File.join(cwd, 'test_template.json')
 
@@ -2779,8 +2704,8 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
 
     assert_requested(:put, "https://logs.google.com:777/es//_template/logstash", times: 1)
 
-    assert_not_requested(:put, "https://logs.google.com:777/es//_template/logstash1")
-    assert_not_requested(:put, "https://logs.google.com:777/es//_template/logstash2")
+    assert_requested(:put, "https://logs.google.com:777/es//_template/logstash1")
+    assert_requested(:put, "https://logs.google.com:777/es//_template/logstash2")
   end
 
   def test_templates_can_be_partially_created_if_error_occurs
@@ -2986,6 +2911,18 @@ class ElasticsearchOutputTest < Test::Unit::TestCase
     assert_requested(elastic_request)
   end
 
+  def test_api_key_header
+    stub_request(:head, "http://localhost:9200/").
+      to_return(:status => 200, :body => "", :headers => {})
+    elastic_request = stub_request(:post, "http://localhost:9200/_bulk").
+                        with(headers: {'Authorization'=>'ApiKey dGVzdGF1dGhoZWFkZXI='})
+    driver.configure(%[api_key testauthheader])
+    driver.run(default_tag: 'test') do
+      driver.feed(sample_record)
+    end
+    assert_requested(elastic_request)
+  end
+
   def test_write_message_with_bad_chunk
     driver.configure("target_index_key bad_value\n@log_level debug\n")
     stub_elastic

metadata

@@ -1,16 +1,16 @@
 --- !ruby/object:Gem::Specification
 name: fluent-plugin-elasticsearch
 version: !ruby/object:Gem::Version
-  version: 4.0.11
+  version: 4.1.4
 platform: ruby
 authors:
 - diogo
 - pitr
 - Hiroshi Hatake
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-07-09 00:00:00.000000000 Z
+date: 2020-09-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: fluentd
@@ -160,6 +160,7 @@ files:
 - lib/fluent/plugin/elasticsearch_constants.rb
 - lib/fluent/plugin/elasticsearch_error.rb
 - lib/fluent/plugin/elasticsearch_error_handler.rb
+- lib/fluent/plugin/elasticsearch_fallback_selector.rb
 - lib/fluent/plugin/elasticsearch_index_lifecycle_management.rb
 - lib/fluent/plugin/elasticsearch_index_template.rb
 - lib/fluent/plugin/elasticsearch_simple_sniffer.rb
@@ -172,6 +173,7 @@ files:
 - test/helper.rb
 - test/plugin/test_alias_template.json
 - test/plugin/test_elasticsearch_error_handler.rb
+- test/plugin/test_elasticsearch_fallback_selector.rb
 - test/plugin/test_elasticsearch_index_lifecycle_management.rb
 - test/plugin/test_elasticsearch_tls.rb
 - test/plugin/test_filter_elasticsearch_genid.rb
@@ -186,7 +188,7 @@ licenses:
 - Apache-2.0
 metadata:
   changelog_uri: https://github.com/uken/fluent-plugin-elasticsearch/blob/master/History.md
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -202,13 +204,14 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubygems_version: 3.1.2
-signing_key: 
+signing_key:
 specification_version: 4
 summary: Elasticsearch output plugin for Fluent event collector
 test_files:
 - test/helper.rb
 - test/plugin/test_alias_template.json
 - test/plugin/test_elasticsearch_error_handler.rb
+- test/plugin/test_elasticsearch_fallback_selector.rb
 - test/plugin/test_elasticsearch_index_lifecycle_management.rb
 - test/plugin/test_elasticsearch_tls.rb
 - test/plugin/test_filter_elasticsearch_genid.rb