logstash-output-dynatrace 0.2.0 → 0.2.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 74790f2358ebc0f61dbc88da2c1af0beed1ac35a3a0f021a18e1d365e7751e3a
4
- data.tar.gz: 4d8cbb3c59a4bd6224457f49c25f01932686662551c7642f6fc65047663413e7
3
+ metadata.gz: 0a8f5b5423d993317f8f73d5416c7c17b2961c833ca95da455f2f4742bcbe81b
4
+ data.tar.gz: 249eaf43d553a14a30ee84328514d6771acba98c8ef4da3bee738169b73008ec
5
5
  SHA512:
6
- metadata.gz: 2f970e3aa82cb27219224b241c3b30d3e0c5e3a6a70cec220ce3d7952507ab9a7a1d5c207f22602385871baee310ec3fe3b291c1f8e486462656d5c75b60cbf1
7
- data.tar.gz: dd03de995d2038dfcf533fa050b6ceb16dd627a5e3258c95528e96eaf4b345efacb92f3cc81e922c3af604732f4ed532342e245bc0e1d74d4a3592394746ec50
6
+ metadata.gz: bdb3df6d43d9e2361958b925830af0ec1f686fa02d2d9adbd6488e063e890885aae42589cf5d60e4a2b53c89b6d4a121e9dff1b3500fe5773c1a1cf8203d415a
7
+ data.tar.gz: 20525e9f0cea6acb310656de596404e086ac755508dfa6d603a0d7b6f1dfb030c36ec32db60cf89fbe7942732c14dd2c0fbea375df5874c981d5805c16ce67a3
data/CONTRIBUTING.md ADDED
@@ -0,0 +1,93 @@
1
+ # Logstash Dynatrace output plugin
2
+
3
+ ## Developing
4
+
5
+ ### 1. Plugin Development and Testing
6
+
7
+ #### Code
8
+
9
+ - To get started, you'll need JRuby with the Bundler gem installed.
10
+
11
+ - Clone the plugin from github
12
+
13
+ - Install dependencies
14
+
15
+ ```sh
16
+ bundle install
17
+ ```
18
+
19
+ #### Test
20
+
21
+ - Update your dependencies
22
+
23
+ ```sh
24
+ bundle install
25
+ ```
26
+
27
+ - Run tests
28
+
29
+ ```sh
30
+ bundle exec rspec
31
+ ```
32
+
33
+ ### 2. Running your unpublished Plugin in Logstash
34
+
35
+ #### 2.1 Run in a local Logstash clone
36
+
37
+ - Edit Logstash `Gemfile` and add the local plugin path, for example:
38
+
39
+ ```ruby
40
+ gem "logstash-output-dynatrace", :path => "/your/local/logstash-output-dynatrace"
41
+ ```
42
+
43
+ - Install plugin
44
+
45
+ ```sh
46
+ # Logstash 2.3 and higher
47
+ bin/logstash-plugin install --no-verify
48
+
49
+ # Prior to Logstash 2.3
50
+ bin/plugin install --no-verify
51
+ ```
52
+
53
+ - Run Logstash with your plugin
54
+
55
+ ```sh
56
+ bin/logstash -e \
57
+ 'input { generator { count => 100 } } output { dynatrace { api_key => "your_api_key_here" ingest_endpoint_url => "https://{your-environment-id}.live.dynatrace.com/api/v2/logs/ingest" } }'
58
+ ```
59
+
60
+ At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
61
+
62
+ #### 2.2 Run in an installed Logstash
63
+
64
+ You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
65
+
66
+ - Build your plugin gem
67
+
68
+ ```sh
69
+ gem build logstash-output-dynatrace.gemspec
70
+
71
+ ```
72
+
73
+ - Install the plugin from the Logstash home
74
+
75
+ ```sh
76
+ # Logstash 2.3 and higher
77
+ bin/logstash-plugin install --no-verify
78
+
79
+ # Prior to Logstash 2.3
80
+ bin/plugin install --no-verify
81
+ ```
82
+
83
+ - Start Logstash and proceed to test the plugin
84
+
85
+ ## Contributing
86
+
87
+ All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
88
+
89
+ Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
90
+
91
+ It is more important to the community that you are able to contribute.
92
+
93
+ For more information about contributing, see the [Logstash CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
data/README.md CHANGED
@@ -1,103 +1,150 @@
1
- # Logstash Plugin
1
+ # Logstash Dynatrace output plugin
2
2
 
3
- [![Travis Build Status](https://travis-ci.com/logstash-plugins/logstash-output-example.svg)](https://travis-ci.com/logstash-plugins/logstash-output-example)
3
+ [![Travis Build Status](https://app.travis-ci.com/dynatrace-oss/logstash-output-dynatrace.svg)](https://app.travis-ci.com/dynatrace-oss/logstash-output-dynatrace)
4
4
 
5
5
  > This project is developed and maintained by Dynatrace R&D.
6
6
 
7
- A [Logstash](https://github.com/elastic/logstash) output plugin for sending logs to the Dynatrace [Generic log ingest API v2](https://www.dynatrace.com/support/help/how-to-use-dynatrace/log-monitoring/log-monitoring-v2/post-log-ingest/).
7
+ - [Installation Prerequisites](#installation-prerequisites)
8
+ - [Installation Steps](#installation-steps)
9
+ - [Example Configuration](#example-configuration)
10
+ - [Configuration Overview](#configuration-overview)
11
+ - [Dynatrace-Specific Options](#dynatrace-specific-options)
12
+ - [Common Options](#common-options)
13
+ - [Configuration Detail](#configuration-detail)
14
+ - [`ingest_endpoint_url`](#ingest_endpoint_url)
15
+ - [`api_key`](#api_key)
16
+ - [`ssl_verify_none`](#ssl_verify_none)
17
+ - [`codec`](#codec)
18
+ - [`enable_metric`](#enable_metric)
19
+ - [`id`](#id)
20
+ - [Troubleshooting issues](#troubleshooting-issues)
21
+ - [Enable Debug Logs](#enable-debug-logs)
8
22
 
9
- ## Documentation
23
+ A [Logstash](https://github.com/elastic/logstash) output plugin for sending logs to the Dynatrace [Generic log ingest API v2](https://www.dynatrace.com/support/help/how-to-use-dynatrace/log-monitoring/log-monitoring-v2/post-log-ingest/).
24
+ Please review the documentation for this API before using the plugin.
10
25
 
11
- Logstash provides plugin documentation in a [central location](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-dynatrace.html).
26
+ ## Installation Prerequisites
12
27
 
13
- ## Developing
28
+ - Logstash 7.6+
14
29
 
15
- ### 1. Plugin Developement and Testing
30
+ ## Installation Steps
16
31
 
17
- #### Code
32
+ Logstash is typically installed in the `/usr/share/logstash` directory, and plugins are installed using the `/usr/share/logstash/bin/logstash-plugin` command.
33
+ If your logstash installation directory is different than this, your `logstash-plugin` command may be in a different location.
18
34
 
19
- - To get started, you'll need JRuby with the Bundler gem installed.
35
+ ```sh
36
+ /usr/share/logstash/bin/logstash-plugin install logstash-output-dynatrace
37
+ ```
20
38
 
21
- - Clone the plugin from github
39
+ ## Example Configuration
22
40
 
23
- - Install dependencies
41
+ See below for a detailed explanation of the options used in this example configuration.
24
42
 
25
- ```sh
26
- bundle install
43
+ ```ruby
44
+ output {
45
+ dynatrace {
46
+ id => "dynatrace_output"
47
+ ingest_endpoint_url => "${ACTIVE_GATE_URL}/api/v2/logs/ingest"
48
+ api_key => "${API_KEY}"
49
+ }
50
+ }
27
51
  ```
28
52
 
29
- #### Test
53
+ ## Configuration Overview
30
54
 
31
- - Update your dependencies
55
+ The following configuration options are supported by the Dynatrace output plugin as well as the common options supported by all output plugins described below.
32
56
 
33
- ```sh
34
- bundle install
35
- ```
57
+ ### Dynatrace-Specific Options
36
58
 
37
- - Run tests
38
59
 
39
- ```sh
40
- bundle exec rspec
41
- ```
60
+ | Setting | Input Type | Required |
61
+ | --------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -------- |
62
+ | [`ingest_endpoint_url`](#ingest_endpoint_url) | [String](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string) | Yes |
63
+ | [`api_key`](#api_key) | [String](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string) | Yes |
64
+ | [`ssl_verify_none`](#ssl_verify_none) | [Boolean](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#boolean) | No |
42
65
 
43
- ### 2. Running your unpublished Plugin in Logstash
44
66
 
45
- #### 2.1 Run in a local Logstash clone
67
+ ### Common Options
46
68
 
47
- - Edit Logstash `Gemfile` and add the local plugin path, for example:
69
+ The following configuration options are supported by all output plugins:
48
70
 
49
- ```ruby
50
- gem "logstash-output-dynatrace", :path => "/your/local/logstash-output-dynatrace"
51
- ```
71
+ | Setting | Input type | Required |
72
+ | --------------------------------- | ----------------------------------------------------------------------------------------------------- | -------- |
73
+ | [`codec`](#codec) | [Codec](https://www.elastic.co/guide/en/logstash/7.16/configuration-file-structure.html#codec) | No |
74
+ | [`enable_metric`](#enable_metric) | [Boolean](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#boolean) | No |
75
+ | [`id`](#id) | [String](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string) | No |
52
76
 
53
- - Install plugin
77
+ ## Configuration Detail
54
78
 
55
- ```sh
56
- # Logstash 2.3 and higher
57
- bin/logstash-plugin install --no-verify
79
+ ### `ingest_endpoint_url`
58
80
 
59
- # Prior to Logstash 2.3
60
- bin/plugin install --no-verify
61
- ```
81
+ * Value type is [string](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string)
82
+ * Required
62
83
 
63
- - Run Logstash with your plugin
84
+ This is the full URL of the [Generic log ingest API v2](https://www.dynatrace.com/support/help/how-to-use-dynatrace/log-monitoring/log-monitoring-v2/post-log-ingest/) endpoint on your ActiveGate.
85
+ Example: `"ingest_endpoint_url" => "https://abc123456.live.dynatrace.com/api/v2/logs/ingest"`
64
86
 
65
- ```sh
66
- bin/logstash -e \
67
- 'input { generator { count => 100 } } output { dynatrace { api_key => "your_api_key_here" ingest_endpoint_url => "https://{your-environment-id}.live.dynatrace.com/api/v2/logs/ingest" } }'
68
- ```
87
+ ### `api_key`
69
88
 
70
- At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
89
+ * Value type is [string](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string)
90
+ * Required
71
91
 
72
- #### 2.2 Run in an installed Logstash
92
+ This is the [Dynatrace API token](https://www.dynatrace.com/support/help/dynatrace-api/basics/dynatrace-api-authentication/) which will be used to authenticate log ingest requests.
93
+ It requires the `logs.ingest` (Ingest Logs) scope to be set and it is recommended to limit scope to only this one.
94
+ Example: `"api_key" => "dt0c01.4XLO3..."`
73
95
 
74
- You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
96
+ ### `ssl_verify_none`
75
97
 
76
- - Build your plugin gem
98
+ * Value type is [boolean](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#boolean)
99
+ * Optional
100
+ * Default value is `false`
77
101
 
78
- ```sh
79
- gem build logstash-output-dynatrace.gemspec
102
+ It is recommended to leave this optional configuration set to `false` unless absolutely required.
103
+ Setting `ssl_verify_none` to `true` causes the output plugin to skip certificate verification when sending log ingest requests to SSL and TLS protected HTTPS endpoints.
104
+ This option may be required if you are using a self-signed certificate, an expired certificate, or a certificate which was generated for a different domain than the one in use.
80
105
 
81
- ```
106
+ ### `codec`
82
107
 
83
- - Install the plugin from the Logstash home
108
+ * Value type is codec
109
+ * Default value is "plain"
84
110
 
85
- ```sh
86
- # Logstash 2.3 and higher
87
- bin/logstash-plugin install --no-verify
111
+ The codec used for output data. Output codecs are a convenient method for encoding your data before it leaves the output without needing a separate filter in your Logstash pipeline.
112
+
113
+ ### `enable_metric`
88
114
 
89
- # Prior to Logstash 2.3
90
- bin/plugin install --no-verify
115
+ * Value type is [boolean](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#boolean)
116
+ * Default value is true
117
+
118
+ Disable or enable metric logging for this specific plugin instance. By default we record all the metrics we can, but you can disable metrics collection for a specific plugin.
119
+
120
+ ### `id`
121
+
122
+ * Value type is string
123
+ * There is no default value for this setting.
124
+
125
+ Add a unique ID to the plugin configuration. If no ID is specified, Logstash will generate one. It is strongly recommended to set this ID in your configuration. This is particularly useful when you have two or more plugins of the same type. For example, if you have 2 dynatrace outputs. Adding a named ID in this case will help in monitoring Logstash when using the monitoring APIs.
126
+
127
+ ```ruby
128
+ output {
129
+ dynatrace {
130
+ id => "my_plugin_id"
131
+ }
132
+ }
91
133
  ```
92
134
 
93
- - Start Logstash and proceed to test the plugin
135
+ ## Troubleshooting issues
94
136
 
95
- ## Contributing
137
+ When troubleshooting, always try to reduce the configuration as much as possible.
138
+ It is recommended to disable all plugins except the Dynatrace output plugin and
139
+ a simple input plugin like the [http input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html)
140
+ in order to isolate problems caused by only the Dynatrace output plugin.
96
141
 
97
- All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
142
+ ### Enable Debug Logs
98
143
 
99
- Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
144
+ See <https://www.elastic.co/guide/en/logstash/current/logging.html#logging>.
100
145
 
101
- It is more important to the community that you are able to contribute.
146
+ You can enable debug logging in one of several ways:
102
147
 
103
- For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
148
+ - Use the `--log.level debug` command line flag
149
+ - Configure the `log4j2.properties` file (usually in `/etc/logstash`) - More info [here](https://www.elastic.co/guide/en/logstash/current/logging.html#log4j2)
150
+ - Use the logging API - details [here](https://www.elastic.co/guide/en/logstash/current/logging.html#_update_logging_levels)
@@ -19,6 +19,7 @@ require 'logstash/outputs/base'
19
19
  require 'logstash/json'
20
20
 
21
21
  MAX_RETRIES = 5
22
+ PLUGIN_VERSION = '0.2.1'
22
23
 
23
24
  module LogStash
24
25
  module Outputs
@@ -27,8 +28,6 @@ module LogStash
27
28
 
28
29
  # An output which sends logs to the Dynatrace log ingest v2 endpoint formatted as JSON
29
30
  class Dynatrace < LogStash::Outputs::Base
30
- @plugin_version = ::File.read(::File.expand_path('../../../VERSION', __dir__)).strip
31
-
32
31
  config_name 'dynatrace'
33
32
 
34
33
  # The full URL of the Dynatrace log ingestion endpoint:
@@ -47,6 +46,7 @@ module LogStash
47
46
  attr_accessor :uri, :plugin_version
48
47
 
49
48
  def register
49
+ @logger.debug("Registering plugin")
50
50
  require 'net/https'
51
51
  require 'uri'
52
52
  @uri = URI.parse(@ingest_endpoint_url.uri.to_s)
@@ -61,7 +61,7 @@ module LogStash
61
61
 
62
62
  def headers
63
63
  {
64
- 'User-Agent' => "logstash-output-dynatrace v#{@plugin_version}",
64
+ 'User-Agent' => "logstash-output-dynatrace v#{PLUGIN_VERSION}",
65
65
  'Content-Type' => 'application/json; charset=utf-8',
66
66
  'Authorization' => "Api-Token #{@api_key}"
67
67
  }
@@ -69,6 +69,7 @@ module LogStash
69
69
 
70
70
  # Takes an array of events
71
71
  def multi_receive(events)
72
+ @logger.debug("Received #{events.length} events")
72
73
  return if events.length.zero?
73
74
 
74
75
  retries = 0
@@ -93,6 +94,8 @@ module LogStash
93
94
  @logger.error(failure_message)
94
95
  return
95
96
  end
97
+
98
+ @logger.debug("successfully sent #{events.length} events")
96
99
  rescue Net::HTTPBadResponse, RetryableError => e
97
100
  # indicates a protocol error
98
101
  if retries < MAX_RETRIES
@@ -103,8 +106,11 @@ module LogStash
103
106
  retry
104
107
  else
105
108
  @logger.error("Failed to export logs to Dynatrace.")
109
+ return
106
110
  end
107
111
  end
112
+
113
+ @logger.debug("Successfully exported #{events.length} events with #{retries} retries")
108
114
  end
109
115
 
110
116
  def send(request)
@@ -14,11 +14,11 @@
14
14
  # See the License for the specific language governing permissions and
15
15
  # limitations under the License.
16
16
 
17
- version = File.read(File.expand_path('VERSION', __dir__)).strip
17
+ require_relative './version'
18
18
 
19
19
  Gem::Specification.new do |s|
20
20
  s.name = 'logstash-output-dynatrace'
21
- s.version = version
21
+ s.version = DynatraceConstants::VERSION
22
22
  s.summary = 'A logstash output plugin for sending logs to the Dynatrace Generic log ingest API v2'
23
23
  s.description = <<-EOF
24
24
  This gem is a Logstash plugin required to be installed on top of the Logstash
@@ -32,8 +32,7 @@ Gem::Specification.new do |s|
32
32
  s.require_paths = ['lib']
33
33
 
34
34
  # Files
35
- s.files = Dir['lib/**/*', 'spec/**/*', 'vendor/**/*', '*.gemspec', '*.md', 'CONTRIBUTORS', 'Gemfile', 'LICENSE',
36
- 'NOTICE.TXT', 'VERSION']
35
+ s.files = Dir['lib/**/*', 'spec/**/*', '*.gemspec', '*.md', 'Gemfile', 'LICENSE','version.rb']
37
36
  # Tests
38
37
  s.test_files = s.files.grep(%r{^(test|spec|features)/})
39
38
 
@@ -15,6 +15,7 @@
15
15
  # limitations under the License.
16
16
 
17
17
  require_relative '../spec_helper'
18
+ require_relative '../../version'
18
19
  require 'logstash/codecs/plain'
19
20
  require 'logstash/event'
20
21
  require 'sinatra'
@@ -35,7 +36,6 @@ describe LogStash::Outputs::Dynatrace do
35
36
 
36
37
  before do
37
38
  subject.register
38
- subject.plugin_version = "1.2.3"
39
39
  end
40
40
 
41
41
  it 'does not send empty events' do
@@ -79,7 +79,7 @@ describe LogStash::Outputs::Dynatrace do
79
79
 
80
80
  it 'includes user agent' do
81
81
  allow(subject).to receive(:send) do |req|
82
- expect(req['User-Agent']).to eql('logstash-output-dynatrace v1.2.3')
82
+ expect(req['User-Agent']).to eql("logstash-output-dynatrace v#{::DynatraceConstants::VERSION}")
83
83
  Net::HTTPOK.new "1.1", "200", "OK"
84
84
  end
85
85
  subject.multi_receive(events)
@@ -87,7 +87,7 @@ describe LogStash::Outputs::Dynatrace do
87
87
  end
88
88
 
89
89
  it 'does not log on success' do
90
- allow(subject.logger).to receive(:debug) { raise "should not log" }
90
+ allow(subject.logger).to receive(:debug)
91
91
  allow(subject.logger).to receive(:info) { raise "should not log" }
92
92
  allow(subject.logger).to receive(:error) { raise "should not log" }
93
93
  allow(subject.logger).to receive(:warn) { raise "should not log" }
data/version.rb ADDED
@@ -0,0 +1,20 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2021 Dynatrace LLC
4
+ #
5
+ # Licensed under the Apache License, Version 2.0 (the "License");
6
+ # you may not use this file except in compliance with the License.
7
+ # You may obtain a copy of the License at
8
+ #
9
+ # http://www.apache.org/licenses/LICENSE-2.0
10
+ #
11
+ # Unless required by applicable law or agreed to in writing, software
12
+ # distributed under the License is distributed on an "AS IS" BASIS,
13
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14
+ # See the License for the specific language governing permissions and
15
+ # limitations under the License.
16
+
17
+ module DynatraceConstants
18
+ # Also required to change the version in lib/logstash/outputs/dynatrace.rb
19
+ VERSION = '0.2.1'
20
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-output-dynatrace
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.0
4
+ version: 0.2.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dynatrace Open Source Engineering
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-07-14 00:00:00.000000000 Z
11
+ date: 2022-01-14 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: logstash-codec-json
@@ -153,14 +153,15 @@ extensions: []
153
153
  extra_rdoc_files: []
154
154
  files:
155
155
  - CHANGELOG.md
156
+ - CONTRIBUTING.md
156
157
  - Gemfile
157
158
  - LICENSE
158
159
  - README.md
159
- - VERSION
160
160
  - lib/logstash/outputs/dynatrace.rb
161
161
  - logstash-output-dynatrace.gemspec
162
162
  - spec/outputs/dynatrace_spec.rb
163
163
  - spec/spec_helper.rb
164
+ - version.rb
164
165
  homepage: https://github.com/dynatrace-oss/logstash-output-dynatrace
165
166
  licenses:
166
167
  - Apache-2.0
data/VERSION DELETED
@@ -1 +0,0 @@
1
- 0.2.0