logstash-output-dynatrace 0.2.0 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +3 -0
- data/CONTRIBUTING.md +93 -0
- data/README.md +106 -59
- data/lib/logstash/outputs/dynatrace.rb +22 -21
- data/logstash-output-dynatrace.gemspec +3 -7
- data/spec/outputs/dynatrace_spec.rb +53 -42
- data/version.rb +20 -0
- metadata +4 -45
- data/VERSION +0 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: aefa283dfb7437921963fafa7f068d82363dd421506ebab8b146026f5ca7e30c
|
|
4
|
+
data.tar.gz: d8843b75b161044bbc20cd6ee4b6f28777ff98bd2cab487e669240d662ce3e4a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4ef069a726a4765d9d4395156ab4bfa13ddaf5aa83fe6934ffd8e610af6330f6ecca44a16de538d85551b7c416b8fd97340907a5b6ad6cdf1c8a0abf0b5fb2a2
|
|
7
|
+
data.tar.gz: 244956ca7a924de908e05987eb5d13392c007c864f4462b51914ff766db8784aefc7e1b24eb52ff68a747e530f22f5233bb1af155a4ca461f8adce61dee605e9
|
data/CHANGELOG.md
CHANGED
data/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# Logstash Dynatrace output plugin
|
|
2
|
+
|
|
3
|
+
## Developing
|
|
4
|
+
|
|
5
|
+
### 1. Plugin Development and Testing
|
|
6
|
+
|
|
7
|
+
#### Code
|
|
8
|
+
|
|
9
|
+
- To get started, you'll need JRuby with the Bundler gem installed.
|
|
10
|
+
|
|
11
|
+
- Clone the plugin from github
|
|
12
|
+
|
|
13
|
+
- Install dependencies
|
|
14
|
+
|
|
15
|
+
```sh
|
|
16
|
+
bundle install
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
#### Test
|
|
20
|
+
|
|
21
|
+
- Update your dependencies
|
|
22
|
+
|
|
23
|
+
```sh
|
|
24
|
+
bundle install
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
- Run tests
|
|
28
|
+
|
|
29
|
+
```sh
|
|
30
|
+
bundle exec rspec
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### 2. Running your unpublished Plugin in Logstash
|
|
34
|
+
|
|
35
|
+
#### 2.1 Run in a local Logstash clone
|
|
36
|
+
|
|
37
|
+
- Edit Logstash `Gemfile` and add the local plugin path, for example:
|
|
38
|
+
|
|
39
|
+
```ruby
|
|
40
|
+
gem "logstash-output-dynatrace", :path => "/your/local/logstash-output-dynatrace"
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
- Install plugin
|
|
44
|
+
|
|
45
|
+
```sh
|
|
46
|
+
# Logstash 2.3 and higher
|
|
47
|
+
bin/logstash-plugin install --no-verify
|
|
48
|
+
|
|
49
|
+
# Prior to Logstash 2.3
|
|
50
|
+
bin/plugin install --no-verify
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
- Run Logstash with your plugin
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
bin/logstash -e \
|
|
57
|
+
'input { generator { count => 100 } } output { dynatrace { api_key => "your_api_key_here" ingest_endpoint_url => "https://{your-environment-id}.live.dynatrace.com/api/v2/logs/ingest" } }'
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
|
|
61
|
+
|
|
62
|
+
#### 2.2 Run in an installed Logstash
|
|
63
|
+
|
|
64
|
+
You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
|
|
65
|
+
|
|
66
|
+
- Build your plugin gem
|
|
67
|
+
|
|
68
|
+
```sh
|
|
69
|
+
gem build logstash-output-dynatrace.gemspec
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
- Install the plugin from the Logstash home
|
|
74
|
+
|
|
75
|
+
```sh
|
|
76
|
+
# Logstash 2.3 and higher
|
|
77
|
+
bin/logstash-plugin install --no-verify
|
|
78
|
+
|
|
79
|
+
# Prior to Logstash 2.3
|
|
80
|
+
bin/plugin install --no-verify
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
- Start Logstash and proceed to test the plugin
|
|
84
|
+
|
|
85
|
+
## Contributing
|
|
86
|
+
|
|
87
|
+
All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
|
|
88
|
+
|
|
89
|
+
Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
|
|
90
|
+
|
|
91
|
+
It is more important to the community that you are able to contribute.
|
|
92
|
+
|
|
93
|
+
For more information about contributing, see the [Logstash CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
|
data/README.md
CHANGED
|
@@ -1,103 +1,150 @@
|
|
|
1
|
-
# Logstash
|
|
1
|
+
# Logstash Dynatrace output plugin
|
|
2
2
|
|
|
3
|
-
[](https://app.travis-ci.com/dynatrace-oss/logstash-output-dynatrace)
|
|
4
4
|
|
|
5
5
|
> This project is developed and maintained by Dynatrace R&D.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
- [Installation Prerequisites](#installation-prerequisites)
|
|
8
|
+
- [Installation Steps](#installation-steps)
|
|
9
|
+
- [Example Configuration](#example-configuration)
|
|
10
|
+
- [Configuration Overview](#configuration-overview)
|
|
11
|
+
- [Dynatrace-Specific Options](#dynatrace-specific-options)
|
|
12
|
+
- [Common Options](#common-options)
|
|
13
|
+
- [Configuration Detail](#configuration-detail)
|
|
14
|
+
- [`ingest_endpoint_url`](#ingest_endpoint_url)
|
|
15
|
+
- [`api_key`](#api_key)
|
|
16
|
+
- [`ssl_verify_none`](#ssl_verify_none)
|
|
17
|
+
- [`codec`](#codec)
|
|
18
|
+
- [`enable_metric`](#enable_metric)
|
|
19
|
+
- [`id`](#id)
|
|
20
|
+
- [Troubleshooting issues](#troubleshooting-issues)
|
|
21
|
+
- [Enable Debug Logs](#enable-debug-logs)
|
|
8
22
|
|
|
9
|
-
|
|
23
|
+
A [Logstash](https://github.com/elastic/logstash) output plugin for sending logs to the Dynatrace [Generic log ingest API v2](https://www.dynatrace.com/support/help/how-to-use-dynatrace/log-monitoring/log-monitoring-v2/post-log-ingest/).
|
|
24
|
+
Please review the documentation for this API before using the plugin.
|
|
10
25
|
|
|
11
|
-
|
|
26
|
+
## Installation Prerequisites
|
|
12
27
|
|
|
13
|
-
|
|
28
|
+
- Logstash 7.6+
|
|
14
29
|
|
|
15
|
-
|
|
30
|
+
## Installation Steps
|
|
16
31
|
|
|
17
|
-
|
|
32
|
+
Logstash is typically installed in the `/usr/share/logstash` directory, and plugins are installed using the `/usr/share/logstash/bin/logstash-plugin` command.
|
|
33
|
+
If your logstash installation directory is different than this, your `logstash-plugin` command may be in a different location.
|
|
18
34
|
|
|
19
|
-
|
|
35
|
+
```sh
|
|
36
|
+
/usr/share/logstash/bin/logstash-plugin install logstash-output-dynatrace
|
|
37
|
+
```
|
|
20
38
|
|
|
21
|
-
|
|
39
|
+
## Example Configuration
|
|
22
40
|
|
|
23
|
-
|
|
41
|
+
See below for a detailed explanation of the options used in this example configuration.
|
|
24
42
|
|
|
25
|
-
```
|
|
26
|
-
|
|
43
|
+
```ruby
|
|
44
|
+
output {
|
|
45
|
+
dynatrace {
|
|
46
|
+
id => "dynatrace_output"
|
|
47
|
+
ingest_endpoint_url => "${ACTIVE_GATE_URL}/api/v2/logs/ingest"
|
|
48
|
+
api_key => "${API_KEY}"
|
|
49
|
+
}
|
|
50
|
+
}
|
|
27
51
|
```
|
|
28
52
|
|
|
29
|
-
|
|
53
|
+
## Configuration Overview
|
|
30
54
|
|
|
31
|
-
|
|
55
|
+
The following configuration options are supported by the Dynatrace output plugin as well as the common options supported by all output plugins described below.
|
|
32
56
|
|
|
33
|
-
|
|
34
|
-
bundle install
|
|
35
|
-
```
|
|
57
|
+
### Dynatrace-Specific Options
|
|
36
58
|
|
|
37
|
-
- Run tests
|
|
38
59
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
60
|
+
| Setting | Input Type | Required |
|
|
61
|
+
| --------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -------- |
|
|
62
|
+
| [`ingest_endpoint_url`](#ingest_endpoint_url) | [String](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string) | Yes |
|
|
63
|
+
| [`api_key`](#api_key) | [String](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string) | Yes |
|
|
64
|
+
| [`ssl_verify_none`](#ssl_verify_none) | [Boolean](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#boolean) | No |
|
|
42
65
|
|
|
43
|
-
### 2. Running your unpublished Plugin in Logstash
|
|
44
66
|
|
|
45
|
-
|
|
67
|
+
### Common Options
|
|
46
68
|
|
|
47
|
-
|
|
69
|
+
The following configuration options are supported by all output plugins:
|
|
48
70
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
71
|
+
| Setting | Input type | Required |
|
|
72
|
+
| --------------------------------- | ----------------------------------------------------------------------------------------------------- | -------- |
|
|
73
|
+
| [`codec`](#codec) | [Codec](https://www.elastic.co/guide/en/logstash/7.16/configuration-file-structure.html#codec) | No |
|
|
74
|
+
| [`enable_metric`](#enable_metric) | [Boolean](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#boolean) | No |
|
|
75
|
+
| [`id`](#id) | [String](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string) | No |
|
|
52
76
|
|
|
53
|
-
|
|
77
|
+
## Configuration Detail
|
|
54
78
|
|
|
55
|
-
|
|
56
|
-
# Logstash 2.3 and higher
|
|
57
|
-
bin/logstash-plugin install --no-verify
|
|
79
|
+
### `ingest_endpoint_url`
|
|
58
80
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
```
|
|
81
|
+
* Value type is [string](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string)
|
|
82
|
+
* Required
|
|
62
83
|
|
|
63
|
-
-
|
|
84
|
+
This is the full URL of the [Generic log ingest API v2](https://www.dynatrace.com/support/help/how-to-use-dynatrace/log-monitoring/log-monitoring-v2/post-log-ingest/) endpoint on your ActiveGate.
|
|
85
|
+
Example: `"ingest_endpoint_url" => "https://abc123456.live.dynatrace.com/api/v2/logs/ingest"`
|
|
64
86
|
|
|
65
|
-
|
|
66
|
-
bin/logstash -e \
|
|
67
|
-
'input { generator { count => 100 } } output { dynatrace { api_key => "your_api_key_here" ingest_endpoint_url => "https://{your-environment-id}.live.dynatrace.com/api/v2/logs/ingest" } }'
|
|
68
|
-
```
|
|
87
|
+
### `api_key`
|
|
69
88
|
|
|
70
|
-
|
|
89
|
+
* Value type is [string](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string)
|
|
90
|
+
* Required
|
|
71
91
|
|
|
72
|
-
|
|
92
|
+
This is the [Dynatrace API token](https://www.dynatrace.com/support/help/dynatrace-api/basics/dynatrace-api-authentication/) which will be used to authenticate log ingest requests.
|
|
93
|
+
It requires the `logs.ingest` (Ingest Logs) scope to be set and it is recommended to limit scope to only this one.
|
|
94
|
+
Example: `"api_key" => "dt0c01.4XLO3..."`
|
|
73
95
|
|
|
74
|
-
|
|
96
|
+
### `ssl_verify_none`
|
|
75
97
|
|
|
76
|
-
|
|
98
|
+
* Value type is [boolean](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#boolean)
|
|
99
|
+
* Optional
|
|
100
|
+
* Default value is `false`
|
|
77
101
|
|
|
78
|
-
|
|
79
|
-
|
|
102
|
+
It is recommended to leave this optional configuration set to `false` unless absolutely required.
|
|
103
|
+
Setting `ssl_verify_none` to `true` causes the output plugin to skip certificate verification when sending log ingest requests to SSL and TLS protected HTTPS endpoints.
|
|
104
|
+
This option may be required if you are using a self-signed certificate, an expired certificate, or a certificate which was generated for a different domain than the one in use.
|
|
80
105
|
|
|
81
|
-
|
|
106
|
+
### `codec`
|
|
82
107
|
|
|
83
|
-
|
|
108
|
+
* Value type is codec
|
|
109
|
+
* Default value is "plain"
|
|
84
110
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
111
|
+
The codec used for output data. Output codecs are a convenient method for encoding your data before it leaves the output without needing a separate filter in your Logstash pipeline.
|
|
112
|
+
|
|
113
|
+
### `enable_metric`
|
|
88
114
|
|
|
89
|
-
|
|
90
|
-
|
|
115
|
+
* Value type is [boolean](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#boolean)
|
|
116
|
+
* Default value is true
|
|
117
|
+
|
|
118
|
+
Disable or enable metric logging for this specific plugin instance. By default we record all the metrics we can, but you can disable metrics collection for a specific plugin.
|
|
119
|
+
|
|
120
|
+
### `id`
|
|
121
|
+
|
|
122
|
+
* Value type is string
|
|
123
|
+
* There is no default value for this setting.
|
|
124
|
+
|
|
125
|
+
Add a unique ID to the plugin configuration. If no ID is specified, Logstash will generate one. It is strongly recommended to set this ID in your configuration. This is particularly useful when you have two or more plugins of the same type. For example, if you have 2 dynatrace outputs. Adding a named ID in this case will help in monitoring Logstash when using the monitoring APIs.
|
|
126
|
+
|
|
127
|
+
```ruby
|
|
128
|
+
output {
|
|
129
|
+
dynatrace {
|
|
130
|
+
id => "my_plugin_id"
|
|
131
|
+
}
|
|
132
|
+
}
|
|
91
133
|
```
|
|
92
134
|
|
|
93
|
-
|
|
135
|
+
## Troubleshooting issues
|
|
94
136
|
|
|
95
|
-
|
|
137
|
+
When troubleshooting, always try to reduce the configuration as much as possible.
|
|
138
|
+
It is recommended to disable all plugins except the Dynatrace output plugin and
|
|
139
|
+
a simple input plugin like the [http input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html)
|
|
140
|
+
in order to isolate problems caused by only the Dynatrace output plugin.
|
|
96
141
|
|
|
97
|
-
|
|
142
|
+
### Enable Debug Logs
|
|
98
143
|
|
|
99
|
-
|
|
144
|
+
See <https://www.elastic.co/guide/en/logstash/current/logging.html#logging>.
|
|
100
145
|
|
|
101
|
-
|
|
146
|
+
You can enable debug logging in one of several ways:
|
|
102
147
|
|
|
103
|
-
|
|
148
|
+
- Use the `--log.level debug` command line flag
|
|
149
|
+
- Configure the `log4j2.properties` file (usually in `/etc/logstash`) - More info [here](https://www.elastic.co/guide/en/logstash/current/logging.html#log4j2)
|
|
150
|
+
- Use the logging API - details [here](https://www.elastic.co/guide/en/logstash/current/logging.html#_update_logging_levels)
|
|
@@ -19,6 +19,7 @@ require 'logstash/outputs/base'
|
|
|
19
19
|
require 'logstash/json'
|
|
20
20
|
|
|
21
21
|
MAX_RETRIES = 5
|
|
22
|
+
PLUGIN_VERSION = '0.3.0'
|
|
22
23
|
|
|
23
24
|
module LogStash
|
|
24
25
|
module Outputs
|
|
@@ -27,8 +28,6 @@ module LogStash
|
|
|
27
28
|
|
|
28
29
|
# An output which sends logs to the Dynatrace log ingest v2 endpoint formatted as JSON
|
|
29
30
|
class Dynatrace < LogStash::Outputs::Base
|
|
30
|
-
@plugin_version = ::File.read(::File.expand_path('../../../VERSION', __dir__)).strip
|
|
31
|
-
|
|
32
31
|
config_name 'dynatrace'
|
|
33
32
|
|
|
34
33
|
# The full URL of the Dynatrace log ingestion endpoint:
|
|
@@ -47,6 +46,7 @@ module LogStash
|
|
|
47
46
|
attr_accessor :uri, :plugin_version
|
|
48
47
|
|
|
49
48
|
def register
|
|
49
|
+
@logger.debug("Registering plugin")
|
|
50
50
|
require 'net/https'
|
|
51
51
|
require 'uri'
|
|
52
52
|
@uri = URI.parse(@ingest_endpoint_url.uri.to_s)
|
|
@@ -61,7 +61,7 @@ module LogStash
|
|
|
61
61
|
|
|
62
62
|
def headers
|
|
63
63
|
{
|
|
64
|
-
'User-Agent' => "logstash-output-dynatrace
|
|
64
|
+
'User-Agent' => "logstash-output-dynatrace/#{PLUGIN_VERSION}",
|
|
65
65
|
'Content-Type' => 'application/json; charset=utf-8',
|
|
66
66
|
'Authorization' => "Api-Token #{@api_key}"
|
|
67
67
|
}
|
|
@@ -69,33 +69,33 @@ module LogStash
|
|
|
69
69
|
|
|
70
70
|
# Takes an array of events
|
|
71
71
|
def multi_receive(events)
|
|
72
|
+
@logger.debug("Received #{events.length} events")
|
|
72
73
|
return if events.length.zero?
|
|
73
74
|
|
|
74
75
|
retries = 0
|
|
75
76
|
begin
|
|
76
77
|
request = Net::HTTP::Post.new(uri, headers)
|
|
77
78
|
request.body = "#{LogStash::Json.dump(events.map(&:to_hash)).chomp}\n"
|
|
78
|
-
response =
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
79
|
+
response = @client.request(request)
|
|
80
|
+
|
|
81
|
+
case response
|
|
82
|
+
when Net::HTTPSuccess
|
|
83
|
+
@logger.debug("successfully sent #{events.length} events#{" with #{retries} retries" if retries > 0}")
|
|
84
|
+
when Net::HTTPServerError
|
|
85
|
+
@logger.error("Encountered an HTTP server error", :message => response.message, :code => response.code, :body => response.body) if retries == 0
|
|
86
|
+
when Net::HTTPNotFound
|
|
87
|
+
@logger.error("Encountered a 404 Not Found error. Please check that log ingest is enabled and your API token has the `logs.ingest` (Ingest Logs) scope.", :message => response.message, :code => response.code)
|
|
88
|
+
when Net::HTTPClientError
|
|
89
|
+
@logger.error("Encountered an HTTP client error", :message => response.message, :code => response.code, :body => response.body)
|
|
90
|
+
else
|
|
91
|
+
@logger.error("Encountered an unexpected response code", :message => response.message, :code => response.code)
|
|
85
92
|
end
|
|
86
93
|
|
|
87
|
-
|
|
88
|
-
@logger.error("#{failure_message} Please check that log ingest is enabled and your API token has the `logs.ingest` (Ingest Logs) scope.")
|
|
89
|
-
return
|
|
90
|
-
end
|
|
94
|
+
raise RetryableError.new "code #{response.code}" if retryable(response)
|
|
91
95
|
|
|
92
|
-
if response.is_a? Net::HTTPClientError
|
|
93
|
-
@logger.error(failure_message)
|
|
94
|
-
return
|
|
95
|
-
end
|
|
96
96
|
rescue Net::HTTPBadResponse, RetryableError => e
|
|
97
97
|
# indicates a protocol error
|
|
98
|
-
if retries < MAX_RETRIES
|
|
98
|
+
if retries < MAX_RETRIES
|
|
99
99
|
sleep_seconds = 2 ** retries
|
|
100
100
|
@logger.warn("Failed to contact dynatrace: #{e.message}. Trying again after #{sleep_seconds} seconds.")
|
|
101
101
|
sleep sleep_seconds
|
|
@@ -103,12 +103,13 @@ module LogStash
|
|
|
103
103
|
retry
|
|
104
104
|
else
|
|
105
105
|
@logger.error("Failed to export logs to Dynatrace.")
|
|
106
|
+
return
|
|
106
107
|
end
|
|
107
108
|
end
|
|
108
109
|
end
|
|
109
110
|
|
|
110
|
-
def
|
|
111
|
-
|
|
111
|
+
def retryable(response)
|
|
112
|
+
return response.is_a? Net::HTTPServerError
|
|
112
113
|
end
|
|
113
114
|
end
|
|
114
115
|
end
|
|
@@ -14,11 +14,11 @@
|
|
|
14
14
|
# See the License for the specific language governing permissions and
|
|
15
15
|
# limitations under the License.
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
require_relative './version'
|
|
18
18
|
|
|
19
19
|
Gem::Specification.new do |s|
|
|
20
20
|
s.name = 'logstash-output-dynatrace'
|
|
21
|
-
s.version =
|
|
21
|
+
s.version = DynatraceConstants::VERSION
|
|
22
22
|
s.summary = 'A logstash output plugin for sending logs to the Dynatrace Generic log ingest API v2'
|
|
23
23
|
s.description = <<-EOF
|
|
24
24
|
This gem is a Logstash plugin required to be installed on top of the Logstash
|
|
@@ -32,8 +32,7 @@ Gem::Specification.new do |s|
|
|
|
32
32
|
s.require_paths = ['lib']
|
|
33
33
|
|
|
34
34
|
# Files
|
|
35
|
-
s.files = Dir['lib/**/*', 'spec/**/*', '
|
|
36
|
-
'NOTICE.TXT', 'VERSION']
|
|
35
|
+
s.files = Dir['lib/**/*', 'spec/**/*', '*.gemspec', '*.md', 'Gemfile', 'LICENSE','version.rb']
|
|
37
36
|
# Tests
|
|
38
37
|
s.test_files = s.files.grep(%r{^(test|spec|features)/})
|
|
39
38
|
|
|
@@ -44,11 +43,8 @@ Gem::Specification.new do |s|
|
|
|
44
43
|
s.add_runtime_dependency 'logstash-codec-json'
|
|
45
44
|
s.add_runtime_dependency 'logstash-core-plugin-api', '>= 2.0.0', '< 3'
|
|
46
45
|
|
|
47
|
-
s.add_development_dependency 'insist'
|
|
48
46
|
s.add_development_dependency 'logstash-devutils'
|
|
49
47
|
s.add_development_dependency 'logstash-input-generator'
|
|
50
|
-
s.add_development_dependency 'sinatra'
|
|
51
|
-
s.add_development_dependency 'webrick'
|
|
52
48
|
|
|
53
49
|
s.add_development_dependency 'rubocop', '1.9.1'
|
|
54
50
|
s.add_development_dependency 'rubocop-rake', '0.5.1'
|
|
@@ -15,10 +15,9 @@
|
|
|
15
15
|
# limitations under the License.
|
|
16
16
|
|
|
17
17
|
require_relative '../spec_helper'
|
|
18
|
+
require_relative '../../version'
|
|
18
19
|
require 'logstash/codecs/plain'
|
|
19
20
|
require 'logstash/event'
|
|
20
|
-
require 'sinatra'
|
|
21
|
-
require 'insist'
|
|
22
21
|
require 'net/http'
|
|
23
22
|
require 'json'
|
|
24
23
|
|
|
@@ -31,97 +30,109 @@ describe LogStash::Outputs::Dynatrace do
|
|
|
31
30
|
end
|
|
32
31
|
let(:url) { "http://localhost/good" }
|
|
33
32
|
let(:key) { 'api.key' }
|
|
33
|
+
|
|
34
34
|
let(:subject) { LogStash::Outputs::Dynatrace.new({ 'api_key' => key, 'ingest_endpoint_url' => url }) }
|
|
35
|
+
let(:client) { subject.instance_variable_get(:@client) }
|
|
36
|
+
|
|
37
|
+
let(:ok) { Net::HTTPOK.new "1.1", "200", "OK" }
|
|
38
|
+
let(:server_error) { Net::HTTPServerError.new "1.1", "500", "Internal Server Error" }
|
|
39
|
+
let(:client_error) { Net::HTTPClientError.new("1.1", '400', 'Client error') }
|
|
40
|
+
let(:not_found) { Net::HTTPNotFound.new "1.1", "404", "Not Found" }
|
|
41
|
+
|
|
42
|
+
let(:body) { "this is a failure" }
|
|
35
43
|
|
|
36
44
|
before do
|
|
37
45
|
subject.register
|
|
38
|
-
subject.plugin_version = "1.2.3"
|
|
39
46
|
end
|
|
40
47
|
|
|
41
48
|
it 'does not send empty events' do
|
|
42
|
-
|
|
49
|
+
expect(client).to_not receive(:request)
|
|
43
50
|
subject.multi_receive([])
|
|
44
|
-
expect(subject).to_not have_received(:send)
|
|
45
51
|
end
|
|
46
52
|
|
|
47
53
|
context 'server response success' do
|
|
48
54
|
it 'sends events' do
|
|
49
|
-
|
|
55
|
+
expect(client).to receive(:request) do |req|
|
|
50
56
|
body = JSON.parse(req.body)
|
|
51
57
|
expect(body.length).to eql(2)
|
|
52
58
|
expect(body[0]['message']).to eql('message 1')
|
|
53
59
|
expect(body[0]['@timestamp']).to eql('2021-06-25T15:46:45.693Z')
|
|
54
60
|
expect(body[1]['message']).to eql('message 2')
|
|
55
61
|
expect(body[1]['@timestamp']).to eql('2021-06-25T15:46:46.693Z')
|
|
56
|
-
|
|
62
|
+
ok
|
|
57
63
|
end
|
|
58
64
|
subject.multi_receive(events)
|
|
59
|
-
expect(subject).to have_received(:send)
|
|
60
65
|
end
|
|
61
66
|
|
|
62
67
|
it 'includes authorization header' do
|
|
63
|
-
|
|
68
|
+
expect(client).to receive(:request) do |req|
|
|
64
69
|
expect(req['Authorization']).to eql("Api-Token #{key}")
|
|
65
|
-
|
|
70
|
+
ok
|
|
66
71
|
end
|
|
67
72
|
subject.multi_receive(events)
|
|
68
|
-
expect(subject).to have_received(:send)
|
|
69
73
|
end
|
|
70
74
|
|
|
71
75
|
it 'includes content type header' do
|
|
72
|
-
|
|
76
|
+
expect(client).to receive(:request) do |req|
|
|
73
77
|
expect(req['Content-Type']).to eql('application/json; charset=utf-8')
|
|
74
|
-
|
|
78
|
+
ok
|
|
75
79
|
end
|
|
76
80
|
subject.multi_receive(events)
|
|
77
|
-
expect(subject).to have_received(:send)
|
|
78
81
|
end
|
|
79
82
|
|
|
80
83
|
it 'includes user agent' do
|
|
81
|
-
|
|
82
|
-
expect(req['User-Agent']).to eql(
|
|
83
|
-
|
|
84
|
+
expect(client).to receive(:request) do |req|
|
|
85
|
+
expect(req['User-Agent']).to eql("logstash-output-dynatrace/#{::DynatraceConstants::VERSION}")
|
|
86
|
+
ok
|
|
84
87
|
end
|
|
85
88
|
subject.multi_receive(events)
|
|
86
|
-
expect(subject).to have_received(:send)
|
|
87
89
|
end
|
|
88
90
|
|
|
89
91
|
it 'does not log on success' do
|
|
90
|
-
allow(subject.logger).to receive(:debug)
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
Net::HTTPOK.new "1.1", "200", "OK"
|
|
96
|
-
end
|
|
92
|
+
allow(subject.logger).to receive(:debug)
|
|
93
|
+
expect(subject.logger).to_not receive(:info)
|
|
94
|
+
expect(subject.logger).to_not receive(:error)
|
|
95
|
+
expect(subject.logger).to_not receive(:warn)
|
|
96
|
+
expect(client).to receive(:request) { ok }
|
|
97
97
|
subject.multi_receive(events)
|
|
98
|
-
expect(subject).to have_received(:send)
|
|
99
98
|
end
|
|
100
99
|
end
|
|
101
100
|
|
|
102
|
-
context 'with
|
|
103
|
-
it '
|
|
104
|
-
|
|
101
|
+
context 'with server error' do
|
|
102
|
+
it 'retries 5 times with exponential backoff' do
|
|
103
|
+
# This prevents the elusive "undefined method `close' for nil:NilClass" error.
|
|
104
|
+
expect(server_error).to receive(:body) { body }.once
|
|
105
|
+
expect(subject.logger).to receive(:error).with("Encountered an HTTP server error", {:body=>body, :code=>"500", :message=> "Internal Server Error"}).once
|
|
106
|
+
expect(client).to receive(:request) { server_error }.exactly(6).times
|
|
107
|
+
|
|
108
|
+
|
|
109
|
+
expect(subject).to receive(:sleep).with(1).ordered
|
|
110
|
+
expect(subject).to receive(:sleep).with(2).ordered
|
|
111
|
+
expect(subject).to receive(:sleep).with(4).ordered
|
|
112
|
+
expect(subject).to receive(:sleep).with(8).ordered
|
|
113
|
+
expect(subject).to receive(:sleep).with(16).ordered
|
|
114
|
+
|
|
115
|
+
expect(subject.logger).to receive(:error).with("Failed to export logs to Dynatrace.")
|
|
105
116
|
subject.multi_receive(events)
|
|
106
|
-
expect(subject).to have_received(:send).once
|
|
107
117
|
end
|
|
108
118
|
end
|
|
109
119
|
|
|
110
|
-
context 'with
|
|
111
|
-
it '
|
|
112
|
-
allow(subject).to receive(:
|
|
113
|
-
|
|
114
|
-
|
|
120
|
+
context 'with client error' do
|
|
121
|
+
it 'does not retry on 404' do
|
|
122
|
+
allow(subject.logger).to receive(:error)
|
|
123
|
+
expect(client).to receive(:request) { not_found }.once
|
|
115
124
|
subject.multi_receive(events)
|
|
125
|
+
end
|
|
116
126
|
|
|
117
|
-
|
|
118
|
-
expect(
|
|
119
|
-
|
|
120
|
-
expect(
|
|
121
|
-
expect(subject).to have_received(:sleep).with(16).ordered
|
|
127
|
+
it 'logs the response body' do
|
|
128
|
+
expect(client).to receive(:request) { client_error }
|
|
129
|
+
# This prevents the elusive "undefined method `close' for nil:NilClass" error.
|
|
130
|
+
expect(client_error).to receive(:body) { body }
|
|
122
131
|
|
|
123
|
-
expect(subject).to
|
|
124
|
-
|
|
132
|
+
expect(subject.logger).to receive(:error).with("Encountered an HTTP client error",
|
|
133
|
+
{:body=>body, :code=>"400", :message=> "Client error"})
|
|
134
|
+
|
|
135
|
+
subject.multi_receive(events)
|
|
125
136
|
end
|
|
126
137
|
end
|
|
127
138
|
end
|
data/version.rb
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Copyright 2021 Dynatrace LLC
|
|
4
|
+
#
|
|
5
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
6
|
+
# you may not use this file except in compliance with the License.
|
|
7
|
+
# You may obtain a copy of the License at
|
|
8
|
+
#
|
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
10
|
+
#
|
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
14
|
+
# See the License for the specific language governing permissions and
|
|
15
|
+
# limitations under the License.
|
|
16
|
+
|
|
17
|
+
module DynatraceConstants
|
|
18
|
+
# Also required to change the version in lib/logstash/outputs/dynatrace.rb
|
|
19
|
+
VERSION = '0.3.0'
|
|
20
|
+
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: logstash-output-dynatrace
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Dynatrace Open Source Engineering
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 2022-10-06 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: logstash-codec-json
|
|
@@ -44,20 +44,6 @@ dependencies:
|
|
|
44
44
|
- - "<"
|
|
45
45
|
- !ruby/object:Gem::Version
|
|
46
46
|
version: '3'
|
|
47
|
-
- !ruby/object:Gem::Dependency
|
|
48
|
-
name: insist
|
|
49
|
-
requirement: !ruby/object:Gem::Requirement
|
|
50
|
-
requirements:
|
|
51
|
-
- - ">="
|
|
52
|
-
- !ruby/object:Gem::Version
|
|
53
|
-
version: '0'
|
|
54
|
-
type: :development
|
|
55
|
-
prerelease: false
|
|
56
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
57
|
-
requirements:
|
|
58
|
-
- - ">="
|
|
59
|
-
- !ruby/object:Gem::Version
|
|
60
|
-
version: '0'
|
|
61
47
|
- !ruby/object:Gem::Dependency
|
|
62
48
|
name: logstash-devutils
|
|
63
49
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -86,34 +72,6 @@ dependencies:
|
|
|
86
72
|
- - ">="
|
|
87
73
|
- !ruby/object:Gem::Version
|
|
88
74
|
version: '0'
|
|
89
|
-
- !ruby/object:Gem::Dependency
|
|
90
|
-
name: sinatra
|
|
91
|
-
requirement: !ruby/object:Gem::Requirement
|
|
92
|
-
requirements:
|
|
93
|
-
- - ">="
|
|
94
|
-
- !ruby/object:Gem::Version
|
|
95
|
-
version: '0'
|
|
96
|
-
type: :development
|
|
97
|
-
prerelease: false
|
|
98
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
99
|
-
requirements:
|
|
100
|
-
- - ">="
|
|
101
|
-
- !ruby/object:Gem::Version
|
|
102
|
-
version: '0'
|
|
103
|
-
- !ruby/object:Gem::Dependency
|
|
104
|
-
name: webrick
|
|
105
|
-
requirement: !ruby/object:Gem::Requirement
|
|
106
|
-
requirements:
|
|
107
|
-
- - ">="
|
|
108
|
-
- !ruby/object:Gem::Version
|
|
109
|
-
version: '0'
|
|
110
|
-
type: :development
|
|
111
|
-
prerelease: false
|
|
112
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
113
|
-
requirements:
|
|
114
|
-
- - ">="
|
|
115
|
-
- !ruby/object:Gem::Version
|
|
116
|
-
version: '0'
|
|
117
75
|
- !ruby/object:Gem::Dependency
|
|
118
76
|
name: rubocop
|
|
119
77
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -153,14 +111,15 @@ extensions: []
|
|
|
153
111
|
extra_rdoc_files: []
|
|
154
112
|
files:
|
|
155
113
|
- CHANGELOG.md
|
|
114
|
+
- CONTRIBUTING.md
|
|
156
115
|
- Gemfile
|
|
157
116
|
- LICENSE
|
|
158
117
|
- README.md
|
|
159
|
-
- VERSION
|
|
160
118
|
- lib/logstash/outputs/dynatrace.rb
|
|
161
119
|
- logstash-output-dynatrace.gemspec
|
|
162
120
|
- spec/outputs/dynatrace_spec.rb
|
|
163
121
|
- spec/spec_helper.rb
|
|
122
|
+
- version.rb
|
|
164
123
|
homepage: https://github.com/dynatrace-oss/logstash-output-dynatrace
|
|
165
124
|
licenses:
|
|
166
125
|
- Apache-2.0
|
data/VERSION
DELETED
|
@@ -1 +0,0 @@
|
|
|
1
|
-
0.2.0
|