logstash-output-influx 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 7011a8bd6191ba93a69f4a36ca7c03f38a272b7a
+  data.tar.gz: 63e1372b1432081b0f4e0be68c9cb9827c66b609
+SHA512:
+  metadata.gz: d95c601b3b085e9ffb075e13e035247848ab2f08b1f46b1537f25f61a78577a90bfa19056d3152cee2e775b26e3a0c4708349e8c07b70c9f24c4bbcda7c97ad9
+  data.tar.gz: d034b572f2cb4042259128c1deef1d9e61f80374a6ca4dfccdcb3f0b2f32569e53981e095fd0ce8acba2a02d7d7871f78069ff577e8bfd97c439635c141ddecc

data/CHANGELOG.md

@@ -0,0 +1,98 @@
+## 5.2.4
+  - Relax dependency on http_client mixin since current major works on both
+
+## 5.2.3
+  - Fixed handling of empty `retryable_codes` [#99](https://github.com/logstash-plugins/logstash-output-http/pull/99)
+
+## 5.2.2
+  - Fix high CPU usage on retries in json_batch mode. [#89](https://github.com/logstash-plugins/logstash-output-http/pull/89)
+  - Enable compression in json_batch mode. [#89](https://github.com/logstash-plugins/logstash-output-http/pull/89)
+
+## 5.2.1
+  - Docs: Set the default_codec doc attribute.
+
+## 5.2.0
+  - Added json_batch format
+  - Make 429 responses log at debug, not error level. They are really just flow control
+
+## 5.1.2
+  - Add check to avoid hanging pipeline if an empty event array is passed in. #80
+
+## 5.1.1
+  - Update gemspec summary
+
+## 5.1.0
+  - Adding a new option `http_compression` for sending compressed payload with the `Content-Encoding: gzip` header to the configured http endpoint #66
+
+## 5.0.1
+  - Fix some documentation issues
+
+## 5.0.0
+ - Breaking: bump dependency in breaking version of logstash-mixin-http_client
+
+## 4.3.3
+ - Docs: Add plugin description.
+
+## 4.3.2
+ - Docs: Update plugin title.
+
+## 4.3.1
+ - Fix deadlock that could occur in certain situations. All users should upgrade to the latest version.
+   This deadlock was caused by certain async HTTP APIs being called out of order thus creating a race.
+
+## 4.3.0
+ - Add `user` and `password` options to support HTTP basic auth
+ 
+## 4.2.0
+ - Allow nested array in field definition, See https://github.com/logstash-plugins/logstash-output-http/pull/53
+
+## 4.1.0
+ - Allow nested field definitions in `mappings`
+
+## 4.0.0
+ - Major overhaul of internals, adds new retry options
+ - Allow users to specify non-standard response codes as ignorable
+ - Set concurrency level to shared allowing for greater efficiency across threads
+ 
+## 3.1.1
+  - Relax constraint on logstash-core-plugin-api to >= 1.60 <= 2.99
+
+## 3.1.0
+ - breaking,config: Remove deprecated config 'verify_ssl'. Please use 'ssl_certificate_validation'.
+
+## 3.0.1
+ - Republish all the gems under jruby.
+
+## 3.0.0
+ - Update the plugin to the version 2.0 of the plugin api, this change is required for Logstash 5.0 compatibility. See https://github.com/elastic/logstash/issues/5141
+
+## 2.1.3
+ - Depend on logstash-core-plugin-api instead of logstash-core, removing the need to mass update plugins on major releases of logstash
+
+## 2.1.2
+ - New dependency requirements for logstash-core for the 5.0 release
+
+## 2.1.1
+ - Require http_client mixin with better keepalive handling
+
+
+## 2.1.0
+ - Properly close the client on #close
+ - Optimized execution for Logstash 2.2 ng pipeline
+
+## 2.0.5
+ - fixed memory leak
+
+## 2.0.3
+ - fixed potential race condition on async callbacks
+ - silenced specs equest logs and other spec robustness fixes
+
+## 2.0.0
+ - Plugins were updated to follow the new shutdown semantic, this mainly allows Logstash to instruct input plugins to terminate gracefully,
+   instead of using Thread.raise on the plugins' threads. Ref: https://github.com/elastic/logstash/pull/3895
+ - Dependency on logstash-core update to 2.0
+
+## 1.1.0
+ - Concurrent execution
+ - Add many HTTP options via the http_client mixin
+ - Switch to manticore as HTTP Client

data/CONTRIBUTORS

@@ -0,0 +1,16 @@
+The following is a list of people who have contributed ideas, code, bug
+reports, or in general have helped logstash along its way.
+
+Contributors:
+* Christian S. (squiddle)
+* Colin Surprenant (colinsurprenant)
+* John E. Vincent (lusis)
+* Jordan Sissel (jordansissel)
+* Kurt Hurtado (kurtado)
+* Pier-Hugues Pellerin (ph)
+* Richard Pijnenburg (electrical)
+
+Note: If you've sent us patches, bug reports, or otherwise contributed to
+Logstash, and you aren't on the list above and want to be, please let us know
+and we'll make sure you're here. Contributions from folks like you are what make
+open source awesome.

data/Gemfile

@@ -0,0 +1,11 @@
+source 'https://rubygems.org'
+
+gemspec
+
+logstash_path = ENV["LOGSTASH_PATH"] || "../../logstash"
+use_logstash_source = ENV["LOGSTASH_SOURCE"] && ENV["LOGSTASH_SOURCE"].to_s == "1"
+
+if Dir.exist?(logstash_path) && use_logstash_source
+  gem 'logstash-core', :path => "#{logstash_path}/logstash-core"
+  gem 'logstash-core-plugin-api', :path => "#{logstash_path}/logstash-core-plugin-api"
+end

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2012-2018 Elasticsearch <http://www.elastic.co>
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/NOTICE.TXT

@@ -0,0 +1,5 @@
+Elasticsearch
+Copyright 2012-2015 Elasticsearch
+
+This product includes software developed by The Apache Software
+Foundation (http://www.apache.org/).

data/README.md

@@ -0,0 +1,89 @@
+# Logstash Plugin
+
+[![Travis Build Status](https://travis-ci.org/logstash-plugins/logstash-output-http.svg)](https://travis-ci.org/logstash-plugins/logstash-output-influx)
+
+This is a plugin for [Logstash](https://github.com/elastic/logstash).
+
+It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
+
+## Usage
+
+<pre><code>
+output {
+  influx {
+    url => "http://localhost:8086/"
+    db => "mydb"
+    is_batch => true
+  }
+}
+</code></pre>
+
+## Developing
+
+### 1. Plugin Developement and Testing
+
+#### Code
+- To get started, you'll need JRuby with the Bundler gem installed.
+
+- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
+
+- Install dependencies
+```sh
+bundle install
+```
+
+#### Test
+
+- Update your dependencies
+
+```sh
+bundle install
+```
+
+- Run tests
+
+```sh
+bundle exec rspec
+```
+
+### 2. Running your unpublished Plugin in Logstash
+
+#### 2.1 Run in a local Logstash clone
+
+- Edit Logstash `Gemfile` and add the local plugin path, for example:
+```ruby
+gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
+```
+- Install plugin
+```sh
+# Logstash 2.3 and higher
+bin/logstash-plugin install --no-verify
+
+# Prior to Logstash 2.3
+bin/plugin install --no-verify
+
+```
+- Run Logstash with your plugin
+```sh
+bin/logstash -e 'filter {awesome {}}'
+```
+At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
+
+#### 2.2 Run in an installed Logstash
+
+You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
+
+- Build your plugin gem
+```sh
+gem build logstash-output-influx.gemspec
+```
+- Install the plugin from the Logstash home
+```sh
+# Logstash 2.3 and higher
+bin/logstash-plugin install --no-verify
+
+# Prior to Logstash 2.3
+bin/plugin install --no-verify
+
+```
+- Start Logstash and proceed to test the plugin

data/docs/index.asciidoc

@@ -0,0 +1,391 @@
+:plugin: influx
+:type: output
+:default_codec: plain
+
+///////////////////////////////////////////
+START - GENERATED VARIABLES, DO NOT EDIT!
+///////////////////////////////////////////
+:version: %VERSION%
+:release_date: %RELEASE_DATE%
+:changelog_url: %CHANGELOG_URL%
+:include_path: ../../../../logstash/docs/include
+///////////////////////////////////////////
+END - GENERATED VARIABLES, DO NOT EDIT!
+///////////////////////////////////////////
+
+[id="plugins-{type}s-{plugin}"]
+
+=== Influx output plugin
+
+include::{include_path}/plugin_header.asciidoc[]
+
+==== Description
+
+This output lets you send events to a generic HTTP(S) endpoint.
+
+This output will execute up to 'pool_max' requests in parallel for performance.
+Consider this when tuning this plugin for performance.
+
+Additionally, note that when parallel execution is used strict ordering of events is not
+guaranteed!
+
+Beware, this gem does not yet support codecs. Please use the 'format' option for now.
+
+[id="plugins-{type}s-{plugin}-options"]
+==== Http Output Configuration Options
+
+This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> described later.
+
+[cols="<,<,<",options="header",]
+|=======================================================================
+|Setting |Input type|Required
+| <<plugins-{type}s-{plugin}-automatic_retries>> |<<number,number>>|No
+| <<plugins-{type}s-{plugin}-cacert>> |a valid filesystem path|No
+| <<plugins-{type}s-{plugin}-client_cert>> |a valid filesystem path|No
+| <<plugins-{type}s-{plugin}-client_key>> |a valid filesystem path|No
+| <<plugins-{type}s-{plugin}-connect_timeout>> |<<number,number>>|No
+| <<plugins-{type}s-{plugin}-content_type>> |<<string,string>>|No
+| <<plugins-{type}s-{plugin}-cookies>> |<<boolean,boolean>>|No
+| <<plugins-{type}s-{plugin}-follow_redirects>> |<<boolean,boolean>>|No
+| <<plugins-{type}s-{plugin}-format>> |<<string,string>>, one of `["json", "json_batch", "form", "message"]`|No
+| <<plugins-{type}s-{plugin}-headers>> |<<hash,hash>>|No
+| <<plugins-{type}s-{plugin}-http_compression>> |<<boolean,boolean>>|No
+| <<plugins-{type}s-{plugin}-http_method>> |<<string,string>>, one of `["put", "post", "patch", "delete", "get", "head"]`|Yes
+| <<plugins-{type}s-{plugin}-ignorable_codes>> |<<number,number>>|No
+| <<plugins-{type}s-{plugin}-keepalive>> |<<boolean,boolean>>|No
+| <<plugins-{type}s-{plugin}-keystore>> |a valid filesystem path|No
+| <<plugins-{type}s-{plugin}-keystore_password>> |<<password,password>>|No
+| <<plugins-{type}s-{plugin}-keystore_type>> |<<string,string>>|No
+| <<plugins-{type}s-{plugin}-mapping>> |<<hash,hash>>|No
+| <<plugins-{type}s-{plugin}-message>> |<<string,string>>|No
+| <<plugins-{type}s-{plugin}-pool_max>> |<<number,number>>|No
+| <<plugins-{type}s-{plugin}-pool_max_per_route>> |<<number,number>>|No
+| <<plugins-{type}s-{plugin}-proxy>> |<<,>>|No
+| <<plugins-{type}s-{plugin}-request_timeout>> |<<number,number>>|No
+| <<plugins-{type}s-{plugin}-retry_failed>> |<<boolean,boolean>>|No
+| <<plugins-{type}s-{plugin}-retry_non_idempotent>> |<<boolean,boolean>>|No
+| <<plugins-{type}s-{plugin}-retryable_codes>> |<<number,number>>|No
+| <<plugins-{type}s-{plugin}-socket_timeout>> |<<number,number>>|No
+| <<plugins-{type}s-{plugin}-truststore>> |a valid filesystem path|No
+| <<plugins-{type}s-{plugin}-truststore_password>> |<<password,password>>|No
+| <<plugins-{type}s-{plugin}-truststore_type>> |<<string,string>>|No
+| <<plugins-{type}s-{plugin}-url>> |<<string,string>>|Yes
+| <<plugins-{type}s-{plugin}-validate_after_inactivity>> |<<number,number>>|No
+|=======================================================================
+
+Also see <<plugins-{type}s-{plugin}-common-options>> for a list of options supported by all
+output plugins.
+
+&nbsp;
+
+[id="plugins-{type}s-{plugin}-automatic_retries"]
+===== `automatic_retries` 
+
+  * Value type is <<number,number>>
+  * Default value is `1`
+
+How many times should the client retry a failing URL. We highly recommend NOT setting this value
+to zero if keepalive is enabled. Some servers incorrectly end keepalives early requiring a retry!
+Only IO related failures will be retried, such as connection timeouts and unreachable hosts.
+Valid but non 2xx HTTP responses will always be retried, regardless of the value of this setting,
+unless `retry_failed` is set.
+Note: if `retry_non_idempotent` is NOT set only GET, HEAD, PUT, DELETE, OPTIONS, and TRACE requests will be retried.
+
+[id="plugins-{type}s-{plugin}-cacert"]
+===== `cacert` 
+
+  * Value type is <<path,path>>
+  * There is no default value for this setting.
+
+If you need to use a custom X.509 CA (.pem certs) specify the path to that here
+
+[id="plugins-{type}s-{plugin}-client_cert"]
+===== `client_cert` 
+
+  * Value type is <<path,path>>
+  * There is no default value for this setting.
+
+If you'd like to use a client certificate (note, most people don't want this) set the path to the x509 cert here
+
+[id="plugins-{type}s-{plugin}-client_key"]
+===== `client_key` 
+
+  * Value type is <<path,path>>
+  * There is no default value for this setting.
+
+If you're using a client certificate specify the path to the encryption key here
+
+[id="plugins-{type}s-{plugin}-connect_timeout"]
+===== `connect_timeout` 
+
+  * Value type is <<number,number>>
+  * Default value is `10`
+
+Timeout (in seconds) to wait for a connection to be established. Default is `10s`
+
+[id="plugins-{type}s-{plugin}-content_type"]
+===== `content_type` 
+
+  * Value type is <<string,string>>
+  * There is no default value for this setting.
+
+Content type
+
+If not specified, this defaults to the following:
+
+* if format is "json", "application/json"
+* if format is "json_batch", "application/json". Each Logstash batch of events will be concatenated into a single array and sent in one request.
+* if format is "form", "application/x-www-form-urlencoded"
+
+[id="plugins-{type}s-{plugin}-cookies"]
+===== `cookies` 
+
+  * Value type is <<boolean,boolean>>
+  * Default value is `true`
+
+Enable cookie support. With this enabled the client will persist cookies
+across requests as a normal web browser would. Enabled by default
+
+[id="plugins-{type}s-{plugin}-follow_redirects"]
+===== `follow_redirects` 
+
+  * Value type is <<boolean,boolean>>
+  * Default value is `true`
+
+Should redirects be followed? Defaults to `true`
+
+[id="plugins-{type}s-{plugin}-format"]
+===== `format` 
+
+  * Value can be any of: `json`, `json_batch`, `form`, `message`
+  * Default value is `"json"`
+
+Set the format of the http body.
+
+If json_batch, each batch of events received by this output will be placed
+into a single JSON array and sent in one request. This is particularly useful
+for high throughput scenarios such as sending data between Logstash instaces.
+
+If form, then the body will be the mapping (or whole event) converted
+into a query parameter string, e.g. `foo=bar&baz=fizz...`
+
+If message, then the body will be the result of formatting the event according to message
+
+Otherwise, the event is sent as json.
+
+[id="plugins-{type}s-{plugin}-headers"]
+===== `headers` 
+
+  * Value type is <<hash,hash>>
+  * There is no default value for this setting.
+
+Custom headers to use
+format is `headers => ["X-My-Header", "%{host}"]`
+
+[id="plugins-{type}s-{plugin}-http_compression"]
+===== `http_compression`
+
+  * Value type is <<boolean,boolean>>
+  * Default value is `false`
+
+Enable request compression support. With this enabled the plugin will compress
+http requests using gzip.
+
+[id="plugins-{type}s-{plugin}-http_method"]
+===== `http_method` 
+
+  * This is a required setting.
+  * Value can be any of: `put`, `post`, `patch`, `delete`, `get`, `head`
+  * There is no default value for this setting.
+
+The HTTP Verb. One of "put", "post", "patch", "delete", "get", "head"
+
+[id="plugins-{type}s-{plugin}-ignorable_codes"]
+===== `ignorable_codes` 
+
+  * Value type is <<number,number>>
+  * There is no default value for this setting.
+
+If you would like to consider some non-2xx codes to be successes 
+enumerate them here. Responses returning these codes will be considered successes
+
+[id="plugins-{type}s-{plugin}-keepalive"]
+===== `keepalive` 
+
+  * Value type is <<boolean,boolean>>
+  * Default value is `true`
+
+Turn this on to enable HTTP keepalive support. We highly recommend setting `automatic_retries` to at least
+one with this to fix interactions with broken keepalive implementations.
+
+[id="plugins-{type}s-{plugin}-keystore"]
+===== `keystore` 
+
+  * Value type is <<path,path>>
+  * There is no default value for this setting.
+
+If you need to use a custom keystore (`.jks`) specify that here. This does not work with .pem keys!
+
+[id="plugins-{type}s-{plugin}-keystore_password"]
+===== `keystore_password` 
+
+  * Value type is <<password,password>>
+  * There is no default value for this setting.
+
+Specify the keystore password here.
+Note, most .jks files created with keytool require a password!
+
+[id="plugins-{type}s-{plugin}-keystore_type"]
+===== `keystore_type` 
+
+  * Value type is <<string,string>>
+  * Default value is `"JKS"`
+
+Specify the keystore type here. One of `JKS` or `PKCS12`. Default is `JKS`
+
+[id="plugins-{type}s-{plugin}-mapping"]
+===== `mapping` 
+
+  * Value type is <<hash,hash>>
+  * There is no default value for this setting.
+
+This lets you choose the structure and parts of the event that are sent.
+
+
+For example:
+[source,ruby]
+   mapping => {"foo" => "%{host}"
+              "bar" => "%{type}"}
+
+[id="plugins-{type}s-{plugin}-message"]
+===== `message` 
+
+  * Value type is <<string,string>>
+  * There is no default value for this setting.
+
+
+
+[id="plugins-{type}s-{plugin}-pool_max"]
+===== `pool_max` 
+
+  * Value type is <<number,number>>
+  * Default value is `50`
+
+Max number of concurrent connections. Defaults to `50`
+
+[id="plugins-{type}s-{plugin}-pool_max_per_route"]
+===== `pool_max_per_route` 
+
+  * Value type is <<number,number>>
+  * Default value is `25`
+
+Max number of concurrent connections to a single host. Defaults to `25`
+
+[id="plugins-{type}s-{plugin}-proxy"]
+===== `proxy` 
+
+  * Value type is <<string,string>>
+  * There is no default value for this setting.
+
+If you'd like to use an HTTP proxy . This supports multiple configuration syntaxes:
+
+1. Proxy host in form: `http://proxy.org:1234`
+2. Proxy host in form: `{host => "proxy.org", port => 80, scheme => 'http', user => 'username@host', password => 'password'}`
+3. Proxy host in form: `{url =>  'http://proxy.org:1234', user => 'username@host', password => 'password'}`
+
+[id="plugins-{type}s-{plugin}-request_timeout"]
+===== `request_timeout` 
+
+  * Value type is <<number,number>>
+  * Default value is `60`
+
+This module makes it easy to add a very fully configured HTTP client to logstash
+based on [Manticore](https://github.com/cheald/manticore).
+For an example of its usage see https://github.com/logstash-plugins/logstash-input-http_poller
+Timeout (in seconds) for the entire request
+
+[id="plugins-{type}s-{plugin}-retry_failed"]
+===== `retry_failed` 
+
+  * Value type is <<boolean,boolean>>
+  * Default value is `true`
+
+Set this to false if you don't want this output to retry failed requests
+
+[id="plugins-{type}s-{plugin}-retry_non_idempotent"]
+===== `retry_non_idempotent` 
+
+  * Value type is <<boolean,boolean>>
+  * Default value is `false`
+
+If `automatic_retries` is enabled this will cause non-idempotent HTTP verbs (such as POST) to be retried.
+This only affects connectivity related errors (see related `automatic_retries` setting).
+
+[id="plugins-{type}s-{plugin}-retryable_codes"]
+===== `retryable_codes` 
+
+  * Value type is <<number,number>>
+  * Default value is `[429, 500, 502, 503, 504]`
+
+If encountered as response codes this plugin will retry these requests
+
+[id="plugins-{type}s-{plugin}-socket_timeout"]
+===== `socket_timeout` 
+
+  * Value type is <<number,number>>
+  * Default value is `10`
+
+Timeout (in seconds) to wait for data on the socket. Default is `10s`
+
+[id="plugins-{type}s-{plugin}-truststore"]
+===== `truststore` 
+
+  * Value type is <<path,path>>
+  * There is no default value for this setting.
+
+If you need to use a custom truststore (`.jks`) specify that here. This does not work with .pem certs!
+
+[id="plugins-{type}s-{plugin}-truststore_password"]
+===== `truststore_password` 
+
+  * Value type is <<password,password>>
+  * There is no default value for this setting.
+
+Specify the truststore password here.
+Note, most .jks files created with keytool require a password!
+
+[id="plugins-{type}s-{plugin}-truststore_type"]
+===== `truststore_type` 
+
+  * Value type is <<string,string>>
+  * Default value is `"JKS"`
+
+Specify the truststore type here. One of `JKS` or `PKCS12`. Default is `JKS`
+
+[id="plugins-{type}s-{plugin}-url"]
+===== `url` 
+
+  * This is a required setting.
+  * Value type is <<string,string>>
+  * There is no default value for this setting.
+
+URL to use
+
+[id="plugins-{type}s-{plugin}-validate_after_inactivity"]
+===== `validate_after_inactivity` 
+
+  * Value type is <<number,number>>
+  * Default value is `200`
+
+How long to wait before checking if the connection is stale before executing a request on a connection using keepalive.
+You may want to set this lower, possibly to 0 if you get connection errors regularly
+Quoting the Apache commons docs (this client is based Apache Commmons):
+'Defines period of inactivity in milliseconds after which persistent connections must be re-validated prior to being leased to the consumer. Non-positive value passed to this method disables connection validation. This check helps detect connections that have become stale (half-closed) while kept inactive in the pool.'
+See https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html#setValidateAfterInactivity(int)[these docs for more info]
+
+
+
+[id="plugins-{type}s-{plugin}-common-options"]
+include::{include_path}/{type}.asciidoc[]
+
+:default_codec!: