fluent-plugin-elasticsearch 0.8.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c59dd5df9cd9a3ae7cb7e32b79fce58855888dbe
-  data.tar.gz: 3bb7808c913b37088eb610bb62a59394da068f0a
+  metadata.gz: d8a7cd259caf8132773810062c7823f800a0a6cd
+  data.tar.gz: d8f23990fb581c5b965f353afaf245d63334f25f
 SHA512:
-  metadata.gz: dcd423ab475f32322b88467b74456c290196d5dc9ffb9afddbaa845a91852875f64eedaf84edd77f68c56a56db99102d96077cd31e8ce3fc691a370976ed905e
-  data.tar.gz: f7bd63fe77c03c7d9e889869c1cc06d1f4a266be4eb503bfdd708b6cee133c1dde5bb03e0415ff85606a8d94f377837ed2962b5d912d90eb45fbfd36635a2542
+  metadata.gz: 26d71e33553eee2abf69ee9b41b30b5cd0e7c59b5a5f0241ec16edca9f65f1ea49695add3e69f14e2ec37680ea2d042ec4fe918fbadc9a4ba8ce700efde939e1
+  data.tar.gz: 5af60188c0ddc9097c573f36acfe9b3eab207ba556b5fa1e7e2f668122ecc86b07f7ad8561bb9c073afa6e3f097c26d3ef3d8180c3fcf8a49226d616fe1a81f4

data/History.md

@@ -2,6 +2,9 @@
 
 ### Future
 
+### 0.9.0
+- Add `ssl_verify` option (#108)
+
 ### 0.8.0
 - Replace Patron with Excon HTTP client
 

data/README.md

@@ -31,6 +31,8 @@ This plugin creates ElasticSearch indices by merely writing to them. Consider us
 
 **More options:**
 
+**hosts**
+
 ```
 hosts host1:port1,host2:port2,host3:port3
 ```
@@ -45,6 +47,8 @@ You can specify multiple elasticsearch hosts with separator ",".
 
 If you specify multiple hosts, this plugin will load balance updates to elasticsearch. This is an [elasticsearch-ruby](https://github.com/elasticsearch/elasticsearch-ruby) feature, the default strategy is round-robin.
 
+**user, password, path, scheme, ssl_verify**
+
 If you specify this option, host and port options are ignored.
 
 ```
@@ -56,6 +60,9 @@ scheme https
 
 You can specify user and password for HTTP basic auth. If used in conjunction with a hosts list, then these options will be used by default i.e. if you do not provide any of these options within the hosts listed.
 
+Specify `ssl_verify false` to skip ssl verification (defaults to true)
+
+**logstash_format**
 
 ```
 logstash_format true # defaults to false
@@ -63,16 +70,22 @@ logstash_format true # defaults to false
 
 This is meant to make writing data into elasticsearch compatible to what logstash writes. By doing this, one could take advantade of [kibana](http://kibana.org/).
 
+**logstash_prefix**
+
 ```
 logstash_prefix mylogs # defaults to "logstash"
 ```
 
-By default, the records inserted into index `logstash-YYMMDD`. This option allows to insert into specified index like `mylogs-YYMMDD`.
+**logstash_dateformat**
+
+By default, the records inserted into index `logstash-YYMMDD`. This option allows to insert into specified index like `mylogs-YYYYMM` for a monthly index.
 
 ```
 logstash_dateformat %Y.%m. # defaults to "%Y.%m.%d"
 ```
 
+**time_key**
+
 By default, when inserting records in logstash format, @timestamp is dynamically created with the time at log ingestion. If you'd like to use a custom time. Include an @timestamp with your record.
 
 ```
@@ -105,7 +118,7 @@ The output will be
 }
 ```
 
-By default, the records inserted into index `logstash-YYMMDD`. This option allows to insert into specified index like `logstash-YYYYMM` for a monthly index.
+**utc_index**
 
 ```
 utc_index true
@@ -113,6 +126,8 @@ utc_index true
 
 By default, the records inserted into index `logstash-YYMMDD` with utc (Coordinated Universal Time). This option allows to use local time if you describe utc_index to false.
 
+**request_timeout**
+
 ```
 request_timeout 15s # defaults to 5s
 ```
@@ -121,13 +136,15 @@ You can specify HTTP request timeout.
 
 This is useful when Elasticsearch cannot return response for bulk request within the default of 5 seconds.
 
+**reload_connections**
+
 ```
 reload_connections false # defaults to true
 ```
 
-You can tune how the elasticsearch-transport host reloading feature works. By default it will reload the host list from the server
-every 10,000th request to spread the load. This can be an issue if your ElasticSearch cluster is behind a Reverse Proxy,
-as fluentd process may not have direct network access to the ElasticSearch nodes.
+**reload_on_failure**
+
+You can tune how the elasticsearch-transport host reloading feature works. By default it will reload the host list from the server every 10,000th request to spread the load. This can be an issue if your ElasticSearch cluster is behind a Reverse Proxy, as fluentd process may not have direct network access to the ElasticSearch nodes.
 
 ```
 reload_on_failure true # defaults to false
@@ -136,7 +153,7 @@ reload_on_failure true # defaults to false
 Indicates that the elasticsearch-transport will try to reload the nodes addresses if there is a failure while making the
 request, this can be useful to quickly remove a dead node from the list of addresses.
 
----
+**include_tag_key, tag_key**
 
 ```
 include_tag_key true # defaults to false
@@ -159,7 +176,7 @@ The record inserted into elasticsearch would be
 {"_key":"my.logs", "name":"Johnny Doeie"}
 ```
 
----
+**id_key**
 
 ```
 id_key request_id # use "request_id" field as a record id in ES
@@ -174,7 +191,7 @@ This following record `{"name":"Johnny","request_id":"87d89af7daffad6"}` will tr
 { "name": "Johnny", "request_id": "87d89af7daffad6" }
 ```
 
----
+**Buffered output options**
 
 fluentd-plugin-elasticsearch is a buffered output that uses elasticseach's bulk API. So additional buffer configuration would be (with default values):
 
@@ -186,9 +203,9 @@ retry_wait 1.0
 num_threads 1
 ```
 
----
+**Not seeing a config you need?**
 
-Please consider using [fluent-plugin-forest](https://github.com/tagomoris/fluent-plugin-forest) to send multiple logs to multiple ElasticSearch indices:
+We try to keep the scope of this plugin small. If you need more configuration options, please consider using [fluent-plugin-forest](https://github.com/tagomoris/fluent-plugin-forest). For example, to configure multiple tags to be sent to different ElasticSearch indices:
 
 ```
 <match my.logs.*>

data/fluent-plugin-elasticsearch.gemspec

@@ -3,9 +3,9 @@ $:.push File.expand_path('../lib', __FILE__)
 
 Gem::Specification.new do |s|
   s.name          = 'fluent-plugin-elasticsearch'
-  s.version       = '0.8.0'
+  s.version       = '0.9.0'
   s.authors       = ['diogo', 'pitr']
-  s.email         = ['pitr@uken.com', 'diogo@uken.com']
+  s.email         = ['pitr.vern@gmail.com', 'diogo@uken.com']
   s.description   = %q{ElasticSearch output plugin for Fluent event collector}
   s.summary       = s.description
   s.homepage      = 'https://github.com/uken/fluent-plugin-elasticsearch'

data/lib/fluent/plugin/out_elasticsearch.rb

@@ -28,6 +28,7 @@ class Fluent::ElasticsearchOutput < Fluent::BufferedOutput
   config_param :reload_connections, :bool, :default => true
   config_param :reload_on_failure, :bool, :default => false
   config_param :time_key, :string, :default => nil
+  config_param :ssl_verify , :bool, :default => true
 
   include Fluent::SetTagKeyMixin
   config_set_default :include_tag_key, false
@@ -53,7 +54,8 @@ class Fluent::ElasticsearchOutput < Fluent::BufferedOutput
                                                                             reload_on_failure: @reload_on_failure,
                                                                             retry_on_failure: 5,
                                                                             transport_options: {
-                                                                              request: { timeout: @request_timeout }
+                                                                              request: { timeout: @request_timeout },
+                                                                              ssl: { verify: @ssl_verify }
                                                                             }
                                                                           }), &adapter_conf)
       es = Elasticsearch::Client.new transport: transport

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fluent-plugin-elasticsearch
 version: !ruby/object:Gem::Version
-  version: 0.8.0
+  version: 0.9.0
 platform: ruby
 authors:
 - diogo
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-04-16 00:00:00.000000000 Z
+date: 2015-06-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: fluentd
@@ -83,7 +83,7 @@ dependencies:
         version: '1'
 description: ElasticSearch output plugin for Fluent event collector
 email:
-- pitr@uken.com
+- pitr.vern@gmail.com
 - diogo@uken.com
 executables: []
 extensions: []
@@ -128,4 +128,3 @@ summary: ElasticSearch output plugin for Fluent event collector
 test_files:
 - test/helper.rb
 - test/plugin/test_out_elasticsearch.rb
-has_rdoc: