fluent-plugin-elasticsearch 2.4.1 → 2.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 7818d400af943a625cb1b193e512885f38118b83
-  data.tar.gz: aa62c55e4d724d7bd5b913ebd3dd33a9ac49f503
+SHA256:
+  metadata.gz: 9d722819bc4644442e84bd13acbd6d1fe7845d6fc01119d3a4c8475321d1195d
+  data.tar.gz: a4956fa749f46343aa3609cc319cff6c5e28797b50384ece1284566ee773045f
 SHA512:
-  metadata.gz: 045b1b49e09d1499e15c7bf8b260f4a1316780bfabfed18bb3eef0ffcb13d3eadee21d2307c45810bda4b9df4caf2bbafce155948c8f86d6b96f8710f1e858bf
-  data.tar.gz: 69b16b63ec8616b3db021a61b4778c3e5ea59c66a70958e29eea239906c82d3299b161f301b7d4221ccc7ac35dd458a12f77855379dd20ddf3af69a46d855e8e
+  metadata.gz: 55cb3916853e0a3ad24bc450aecd577f13cbcb20f8d9a4b446ffa4beb7dce6df5fcd8ca4583846f53921b69404421f3bd8a714dedaac596745600a4b7084744e
+  data.tar.gz: e46f86ba7109bce5f51628f3209497c3a5ec706641f25cb267391a173384acc26c626bce2f1280d1d72f1cebb7e9188f76afa6baf1246e2b01f6f5e3c913b3f5

data/History.md

@@ -4,6 +4,10 @@
 - Log ES response errors (#230)
 - Use latest elasticsearch-ruby (#240)
 
+### 2.5.0
+- Using nested record in `id_key`, `parent_key`, and `routing_key` (#351
+- Fix inverted case of a proper noun "Elasticsearch" (#349)
+
 ### 2.4.1
 - Add config parameter to enable elasticsearch-ruby's transporter logging (#342)
 

data/ISSUE_TEMPLATE.md

@@ -16,5 +16,9 @@ Provide example config and message
 
 #### Using Fluentd and ES plugin versions
 
-* Fluentd v0.12 or v0.14
+* OS version
+* Fluentd v0.12 or v0.14/v1.0
+  * paste result of ``fluentd --version`` or ``td-agent --version``
 * ES plugin 2.x.y or 1.x.y
+  * paste boot log of fluentd or td-agent
+  * paste result of ``fluent-gem list``, ``td-agent-gem list`` or your Gemfile.lock

data/README.md

@@ -7,7 +7,7 @@
 [![Issue Stats](http://issuestats.com/github/uken/fluent-plugin-elasticsearch/badge/pr)](http://issuestats.com/github/uken/fluent-plugin-elasticsearch)
 [![Issue Stats](http://issuestats.com/github/uken/fluent-plugin-elasticsearch/badge/issue)](http://issuestats.com/github/uken/fluent-plugin-elasticsearch)
 
-Send your logs to ElasticSearch (and search them with Kibana maybe?)
+Send your logs to Elasticsearch (and search them with Kibana maybe?)
 
 Note: For Amazon Elasticsearch Service please consider using [fluent-plugin-aws-elasticsearch-service](https://github.com/atomita/fluent-plugin-aws-elasticsearch-service)
 
@@ -68,7 +68,7 @@ Current maintainers: @cosmo0920
 
 | fluent-plugin-elasticsearch  | fluentd | ruby |
 |-------------------|---------|------|
-| >= 2.0.0 | >= v0.14.0 | >= 2.1 |
+| >= 2.0.0 | >= v0.14.20 | >= 2.1 |
 |  < 2.0.0 | >= v0.12.0 | >= 1.9 |
 
 NOTE: For v0.12 version, you should use 1.x.y version. Please send patch into v0.12 branch if you encountered 1.x version's bug.
@@ -95,7 +95,7 @@ In your Fluentd configuration, use `@type elasticsearch`. Additional configurati
 
 ### Index templates
 
-This plugin creates ElasticSearch indices by merely writing to them. Consider using [Index Templates](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html) to gain control of what get indexed and how. See [this example](https://github.com/uken/fluent-plugin-elasticsearch/issues/33#issuecomment-38693282) for a good starting point.
+This plugin creates Elasticsearch indices by merely writing to them. Consider using [Index Templates](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html) to gain control of what get indexed and how. See [this example](https://github.com/uken/fluent-plugin-elasticsearch/issues/33#issuecomment-38693282) for a good starting point.
 
 ## Configuration
 
@@ -107,9 +107,9 @@ hosts host1:port1,host2:port2,host3:port3
 hosts https://customhost.com:443/path,https://username:password@host-failover.com:443
 ```
 
-You can specify multiple ElasticSearch hosts with separator ",".
+You can specify multiple Elasticsearch hosts with separator ",".
 
-If you specify multiple hosts, this plugin will load balance updates to ElasticSearch. This is an [elasticsearch-ruby](https://github.com/elasticsearch/elasticsearch-ruby) feature, the default strategy is round-robin.
+If you specify multiple hosts, this plugin will load balance updates to Elasticsearch. This is an [elasticsearch-ruby](https://github.com/elasticsearch/elasticsearch-ruby) feature, the default strategy is round-robin.
 
 And this plugin will escape required URL encoded characters within `%{}` placeholders.
 
@@ -145,7 +145,7 @@ Specify `ssl_verify false` to skip ssl verification (defaults to true)
 logstash_format true # defaults to false
 ```
 
-This is meant to make writing data into ElasticSearch indices compatible to what [Logstash](https://www.elastic.co/products/logstash) calls them. By doing this, one could take advantage of [Kibana](https://www.elastic.co/products/kibana). See logstash_prefix and logstash_dateformat to customize this index name pattern. The index name will be `#{logstash_prefix}-#{formated_date}`
+This is meant to make writing data into Elasticsearch indices compatible to what [Logstash](https://www.elastic.co/products/logstash) calls them. By doing this, one could take advantage of [Kibana](https://www.elastic.co/products/kibana). See logstash_prefix and logstash_dateformat to customize this index name pattern. The index name will be `#{logstash_prefix}-#{formated_date}`
 
 :warning: Setting this option to `true` will ignore the `index_name` setting. The default index name prefix is `logstash-`.
 
@@ -332,7 +332,7 @@ One of [template_file](#template_file) or [templates](#templates) must also be s
 
 You can specify HTTP request timeout.
 
-This is useful when ElasticSearch cannot return response for bulk request within the default of 5 seconds.
+This is useful when Elasticsearch cannot return response for bulk request within the default of 5 seconds.
 
 ```
 request_timeout 15s # defaults to 5s
@@ -340,7 +340,7 @@ request_timeout 15s # defaults to 5s
 
 ### reload_connections
 
-You can tune how the elasticsearch-transport host reloading feature works. By default it will reload the host list from the server every 10,000th request to spread the load. This can be an issue if your ElasticSearch cluster is behind a Reverse Proxy, as Fluentd process may not have direct network access to the ElasticSearch nodes.
+You can tune how the elasticsearch-transport host reloading feature works. By default it will reload the host list from the server every 10,000th request to spread the load. This can be an issue if your Elasticsearch cluster is behind a Reverse Proxy, as Fluentd process may not have direct network access to the Elasticsearch nodes.
 
 ```
 reload_connections false # defaults to true
@@ -380,7 +380,7 @@ This will add the Fluentd tag in the JSON record. For instance, if you have a co
 </match>
 ```
 
-The record inserted into ElasticSearch would be
+The record inserted into Elasticsearch would be
 
 ```
 {"_key": "my.logs", "name": "Johnny Doeie"}
@@ -392,9 +392,9 @@ The record inserted into ElasticSearch would be
 id_key request_id # use "request_id" field as a record id in ES
 ```
 
-By default, all records inserted into ElasticSearch get a random _id. This option allows to use a field in the record as an identifier.
+By default, all records inserted into Elasticsearch get a random _id. This option allows to use a field in the record as an identifier.
 
-This following record `{"name": "Johnny", "request_id": "87d89af7daffad6"}` will trigger the following ElasticSearch command
+This following record `{"name": "Johnny", "request_id": "87d89af7daffad6"}` will trigger the following Elasticsearch command
 
 ```
 { "index" : { "_index": "logstash-2013.01.01", "_type": "fluentd", "_id": "87d89af7daffad6" } }
@@ -422,7 +422,32 @@ Example configuration for [fluent-plugin-genhashvalue](https://github.com/mtakem
 </filter>
 ```
 
-:warning: In order to avoid hash-collisions and loosing data careful consideration is required when choosing the keys in the event record that should be used to calculate the hash 
+:warning: In order to avoid hash-collisions and loosing data careful consideration is required when choosing the keys in the event record that should be used to calculate the hash
+
+#### Using nested key
+
+Nested key specifying syntax is also supported.
+
+With the following configuration
+
+```aconf
+id_key $.nested.request_id
+```
+
+and the following nested record
+
+```json
+{"nested":{"name": "Johnny", "request_id": "87d89af7daffad6"}}
+```
+
+will trigger the following Elasticsearch command
+
+```
+{"index":{"_index":"fluentd","_type":"fluentd","_id":"87d89af7daffad6"}}
+{"nested":{"name":"Johnny","request_id":"87d89af7daffad6"}}
+```
+
+:warning: Note that [Hash flattening](#hash-flattening) may be conflict nested record feature.
 
 ### parent_key
 
@@ -435,7 +460,7 @@ If your input is
 { "name": "Johnny", "a_parent": "my_parent" }
 ```
 
-ElasticSearch command would be
+Elasticsearch command would be
 
 ```
 { "index" : { "_index": "****", "_type": "****", "_id": "****", "_parent": "my_parent" } }
@@ -444,6 +469,31 @@ ElasticSearch command would be
 
 if `parent_key` is not configed or the `parent_key` is absent in input record, nothing will happen.
 
+#### Using nested key
+
+Nested key specifying syntax is also supported.
+
+With the following configuration
+
+```aconf
+parent_key $.nested.a_parent
+```
+
+and the following nested record
+
+```json
+{"nested":{ "name": "Johnny", "a_parent": "my_parent" }}
+```
+
+will trigger the following Elasticsearch command
+
+```
+{"index":{"_index":"fluentd","_type":"fluentd","_parent":"my_parent"}}
+{"nested":{"name":"Johnny","a_parent":"my_parent"}}
+```
+
+:warning: Note that [Hash flattening](#hash-flattening) may be conflict nested record feature.
+
 ### routing_key
 
 Similar to `parent_key` config, will add `_routing` into elasticsearch command if `routing_key` is set and the field does exist in input event.
@@ -520,12 +570,12 @@ with_transporter_log true
 
 ### Client/host certificate options
 
-Need to verify ElasticSearch's certificate?  You can use the following parameter to specify a CA instead of using an environment variable.
+Need to verify Elasticsearch's certificate?  You can use the following parameter to specify a CA instead of using an environment variable.
 ```
 ca_file /path/to/your/ca/cert
 ```
 
-Does your ElasticSearch cluster want to verify client connections?  You can specify the following parameters to use your client certificate, key, and key password for your connection.
+Does your Elasticsearch cluster want to verify client connections?  You can specify the following parameters to use your client certificate, key, and key password for your connection.
 ```
 client_cert /path/to/your/client/cert
 client_key /path/to/your/private/key
@@ -600,7 +650,7 @@ Here is a sample config:
 
 We try to keep the scope of this plugin small and not add too many configuration options. If you think an option would be useful to others, feel free to open an issue or contribute a Pull Request.
 
-Alternatively, consider using [fluent-plugin-forest](https://github.com/tagomoris/fluent-plugin-forest). For example, to configure multiple tags to be sent to different ElasticSearch indices:
+Alternatively, consider using [fluent-plugin-forest](https://github.com/tagomoris/fluent-plugin-forest). For example, to configure multiple tags to be sent to different Elasticsearch indices:
 
 ```
 <match my.logs.*>
@@ -620,7 +670,7 @@ And yet another option is described in Dynamic Configuration section.
 
 ### Dynamic configuration
 
-If you want configurations to depend on information in messages, you can use `elasticsearch_dynamic`. This is an experimental variation of the ElasticSearch plugin allows configuration values to be specified in ways such as the below:
+If you want configurations to depend on information in messages, you can use `elasticsearch_dynamic`. This is an experimental variation of the Elasticsearch plugin allows configuration values to be specified in ways such as the below:
 
 ```
 <match my.logs.*>

data/fluent-plugin-elasticsearch.gemspec

@@ -3,10 +3,10 @@ $:.push File.expand_path('../lib', __FILE__)
 
 Gem::Specification.new do |s|
   s.name          = 'fluent-plugin-elasticsearch'
-  s.version       = '2.4.1'
+  s.version       = '2.5.0'
   s.authors       = ['diogo', 'pitr']
   s.email         = ['pitr.vern@gmail.com', 'me@diogoterror.com']
-  s.description   = %q{ElasticSearch output plugin for Fluent event collector}
+  s.description   = %q{Elasticsearch output plugin for Fluent event collector}
   s.summary       = s.description
   s.homepage      = 'https://github.com/uken/fluent-plugin-elasticsearch'
   s.license       = 'Apache-2.0'
@@ -18,7 +18,7 @@ Gem::Specification.new do |s|
 
   s.required_ruby_version = Gem::Requirement.new(">= 2.0".freeze)
 
-  s.add_runtime_dependency 'fluentd', '>= 0.14.8'
+  s.add_runtime_dependency 'fluentd', '>= 0.14.20'
   s.add_runtime_dependency 'excon', '>= 0'
   s.add_runtime_dependency 'elasticsearch'
 

data/lib/fluent/plugin/out_elasticsearch.rb

@@ -19,7 +19,7 @@ module Fluent::Plugin
   class ElasticsearchOutput < Output
     class ConnectionFailure < StandardError; end
 
-    helpers :event_emitter, :compat_parameters
+    helpers :event_emitter, :compat_parameters, :record_accessor
 
     Fluent::Plugin.register_output('elasticsearch', self)
 
@@ -120,6 +120,11 @@ module Fluent::Plugin
         templates_hash_install(@templates, @template_overwrite)
       end
 
+      # Consider missing the prefix of "$." in nested key specifiers.
+      @id_key = convert_compat_id_key(@id_key) if @id_key
+      @parent_key = convert_compat_id_key(@parent_key) if @parent_key
+      @routing_key = convert_compat_id_key(@routing_key) if @routing_key
+
       @meta_config_map = create_meta_config_map
 
       begin
@@ -147,11 +152,18 @@ module Fluent::Plugin
       end
     end
 
+    def convert_compat_id_key(key)
+      if key.include?('.') && !key.start_with?('$[')
+        key = "$.#{key}" unless key.start_with?('$.')
+      end
+      key
+    end
+
     def create_meta_config_map
       result = []
-      result << [@id_key, '_id'] if @id_key
-      result << [@parent_key, '_parent'] if @parent_key
-      result << [@routing_key, '_routing'] if @routing_key
+      result << [record_accessor_create(@id_key), '_id'] if @id_key
+      result << [record_accessor_create(@parent_key), '_parent'] if @parent_key
+      result << [record_accessor_create(@routing_key), '_routing'] if @routing_key
       result
     end
 
@@ -403,8 +415,10 @@ module Fluent::Plugin
           meta["pipeline".freeze] = @pipeline
         end
 
-        @meta_config_map.each do |record_key, meta_key|
-          meta[meta_key] = record[record_key] if record[record_key]
+        @meta_config_map.each do |record_accessor, meta_key|
+          if raw_value = record_accessor.call(record)
+            meta[meta_key] = raw_value
+          end
         end
 
         if @remove_keys

data/lib/fluent/plugin/out_elasticsearch_dynamic.rb

@@ -187,8 +187,10 @@ module Fluent::Plugin
         meta = {"_index" => target_index, "_type" => dynamic_conf['type_name']}
 
         @meta_config_map.each_pair do |config_name, meta_key|
-          if dynamic_conf[config_name] && record[dynamic_conf[config_name]]
-            meta[meta_key] = record[dynamic_conf[config_name]]
+          if dynamic_conf[config_name] && accessor = record_accessor_create(dynamic_conf[config_name])
+            if raw_value = accessor.call(record)
+              meta[meta_key] = raw_value
+            end
           end
         end
 

data/test/plugin/test_out_elasticsearch.rb

@@ -34,6 +34,12 @@ class ElasticsearchOutput < Test::Unit::TestCase
     {'age' => 26, 'request_id' => '42', 'parent_id' => 'parent', 'routing_id' => 'routing'}
   end
 
+  def nested_sample_record
+    {'nested' =>
+     {'age' => 26, 'parent_id' => 'parent', 'routing_id' => 'routing', 'request_id' => '42'}
+    }
+  end
+
   def stub_elastic_ping(url="http://localhost:9200")
     stub_request(:head, url).to_return(:status => 200, :body => "", :headers => {})
   end
@@ -1429,6 +1435,38 @@ class ElasticsearchOutput < Test::Unit::TestCase
     assert_equal(index_cmds[0]['index']['_id'], '42')
   end
 
+  class NestedIdKeyTest < self
+    def test_adds_nested_id_key_with_dot
+      driver.configure("id_key nested.request_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_id'], '42')
+    end
+
+    def test_adds_nested_id_key_with_dollar_dot
+      driver.configure("id_key $.nested.request_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_id'], '42')
+    end
+
+    def test_adds_nested_id_key_with_bracket
+      driver.configure("id_key $['nested']['request_id']\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_id'], '42')
+    end
+  end
+
   def test_doesnt_add_id_key_if_missing_when_configured
     driver.configure("id_key another_request_id\n")
     stub_elastic_ping
@@ -1458,6 +1496,38 @@ class ElasticsearchOutput < Test::Unit::TestCase
     assert_equal(index_cmds[0]['index']['_parent'], 'parent')
   end
 
+  class NestedParentKeyTest < self
+    def test_adds_nested_parent_key_with_dot
+      driver.configure("parent_key nested.parent_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_parent'], 'parent')
+    end
+
+    def test_adds_nested_parent_key_with_dollar_dot
+      driver.configure("parent_key $.nested.parent_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_parent'], 'parent')
+    end
+
+    def test_adds_nested_parent_key_with_bracket
+      driver.configure("parent_key $['nested']['parent_id']\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_parent'], 'parent')
+    end
+  end
+
   def test_doesnt_add_parent_key_if_missing_when_configured
     driver.configure("parent_key another_parent_id\n")
     stub_elastic_ping
@@ -1487,6 +1557,38 @@ class ElasticsearchOutput < Test::Unit::TestCase
     assert_equal(index_cmds[0]['index']['_routing'], 'routing')
   end
 
+  class NestedRoutingKeyTest < self
+    def test_adds_nested_routing_key_with_dot
+      driver.configure("routing_key nested.routing_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_routing'], 'routing')
+    end
+
+    def test_adds_nested_routing_key_with_dollar_dot
+      driver.configure("routing_key $.nested.routing_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_routing'], 'routing')
+    end
+
+    def test_adds_nested_routing_key_with_bracket
+      driver.configure("routing_key $['nested']['routing_id']\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_routing'], 'routing')
+    end
+  end
+
   def test_doesnt_add_routing_key_if_missing_when_configured
     driver.configure("routing_key another_routing_id\n")
     stub_elastic_ping

data/test/plugin/test_out_elasticsearch_dynamic.rb

@@ -31,6 +31,12 @@ class ElasticsearchOutputDynamic < Test::Unit::TestCase
     {'age' => 26, 'request_id' => '42', 'parent_id' => 'parent', 'routing_id' => 'routing'}
   end
 
+  def nested_sample_record
+    {'nested' =>
+     {'age' => 26, 'parent_id' => 'parent', 'routing_id' => 'routing', 'request_id' => '42'}
+    }
+  end
+
   def stub_elastic_ping(url="http://localhost:9200")
     stub_request(:head, url).to_return(:status => 200, :body => "", :headers => {})
   end
@@ -582,6 +588,38 @@ class ElasticsearchOutputDynamic < Test::Unit::TestCase
     assert_equal(index_cmds[0]['index']['_id'], '42')
   end
 
+  class NestedIdKeyTest < self
+    def test_adds_nested_id_key_with_dot
+      driver.configure("id_key nested.request_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_id'], '42')
+    end
+
+    def test_adds_nested_id_key_with_dollar_dot
+      driver.configure("id_key $.nested.request_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_id'], '42')
+    end
+
+    def test_adds_nested_id_key_with_bracket
+      driver.configure("id_key $['nested']['request_id']\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_id'], '42')
+    end
+  end
+
   def test_doesnt_add_id_key_if_missing_when_configured
     driver.configure("id_key another_request_id\n")
     stub_elastic_ping
@@ -611,6 +649,38 @@ class ElasticsearchOutputDynamic < Test::Unit::TestCase
     assert_equal(index_cmds[0]['index']['_parent'], 'parent')
   end
 
+  class NestedParentKeyTest < self
+    def test_adds_nested_parent_key_with_dot
+      driver.configure("parent_key nested.parent_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_parent'], 'parent')
+    end
+
+    def test_adds_nested_parent_key_with_dollar_dot
+      driver.configure("parent_key $.nested.parent_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_parent'], 'parent')
+    end
+
+    def test_adds_nested_parent_key_with_bracket
+      driver.configure("parent_key $['nested']['parent_id']\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_parent'], 'parent')
+    end
+  end
+
   def test_doesnt_add_parent_key_if_missing_when_configured
     driver.configure("parent_key another_parent_id\n")
     stub_elastic_ping
@@ -640,6 +710,38 @@ class ElasticsearchOutputDynamic < Test::Unit::TestCase
     assert_equal(index_cmds[0]['index']['_routing'], 'routing')
   end
 
+  class NestedRoutingKeyTest < self
+    def test_adds_nested_routing_key_with_dot
+      driver.configure("routing_key nested.routing_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_routing'], 'routing')
+    end
+
+    def test_adds_nested_routing_key_with_dollar_dot
+      driver.configure("routing_key $.nested.routing_id\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_routing'], 'routing')
+    end
+
+    def test_adds_nested_routing_key_with_bracket
+      driver.configure("routing_key $['nested']['routing_id']\n")
+      stub_elastic_ping
+      stub_elastic
+      driver.run(default_tag: 'test') do
+        driver.feed(nested_sample_record)
+      end
+      assert_equal(index_cmds[0]['index']['_routing'], 'routing')
+    end
+  end
+
   def test_doesnt_add_routing_key_if_missing_when_configured
     driver.configure("routing_key another_routing_id\n")
     stub_elastic_ping

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fluent-plugin-elasticsearch
 version: !ruby/object:Gem::Version
-  version: 2.4.1
+  version: 2.5.0
 platform: ruby
 authors:
 - diogo
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-12-20 00:00:00.000000000 Z
+date: 2018-01-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: fluentd
@@ -17,14 +17,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.14.8
+        version: 0.14.20
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.14.8
+        version: 0.14.20
 - !ruby/object:Gem::Dependency
   name: excon
   requirement: !ruby/object:Gem::Requirement
@@ -123,7 +123,7 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '2.0'
-description: ElasticSearch output plugin for Fluent event collector
+description: Elasticsearch output plugin for Fluent event collector
 email:
 - pitr.vern@gmail.com
 - me@diogoterror.com
@@ -175,10 +175,10 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.13
+rubygems_version: 2.7.3
 signing_key: 
 specification_version: 4
-summary: ElasticSearch output plugin for Fluent event collector
+summary: Elasticsearch output plugin for Fluent event collector
 test_files:
 - test/helper.rb
 - test/plugin/test_filter_elasticsearch_genid.rb