logstash-output-newrelic 0.9.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.github/CONTRIBUTING.md +65 -0
- data/.github/ISSUE_TEMPLATE.md +9 -0
- data/.github/PULL_REQUEST_TEMPLATE.md +1 -0
- data/.gitignore +2 -0
- data/.travis.yml +11 -0
- data/CHANGELOG.md +5 -0
- data/Gemfile +2 -0
- data/LICENSE +22 -0
- data/NOTICE.TXT +5 -0
- data/README.md +99 -0
- data/Rakefile +7 -0
- data/bin/logstash-newrelic +2 -0
- data/bin/logstash-newrelic-debug +2 -0
- data/bin/logstash-newrelic.conf.template +10 -0
- data/lib/logstash/outputs/newrelic.rb +220 -0
- data/logstash-output-newrelic.gemspec +27 -0
- data/spec/outputs/newrelic_spec.rb +66 -0
- metadata +111 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 0dab3fd3983521835dc40275981c1ba236c575a9
|
4
|
+
data.tar.gz: 96f288b237871caace9d9b14509416fe71a359e6
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 82fd7991698df057f2118ec98144803b95837db16656ff73d6515b9edef3ec2f83ca40cc82fdbdad5a6a5dd6239229dda6a9fc63ba592234beec8af75e3e8f4b
|
7
|
+
data.tar.gz: 73cb2eb32c2020f83f72f505ee09ab1dd719cb6d6c7c2913c6702406ea25c97dcbcec26f94dda4e946c31d704d4b949fa22af3ec63caa4111dbcee8ff367b3f8
|
@@ -0,0 +1,65 @@
|
|
1
|
+
# Contributing to Logstash
|
2
|
+
|
3
|
+
All contributions are welcome: ideas, patches, documentation, bug reports,
|
4
|
+
complaints, etc!
|
5
|
+
|
6
|
+
Programming is not a required skill, and there are many ways to help out!
|
7
|
+
It is more important to us that you are able to contribute.
|
8
|
+
|
9
|
+
That said, some basic guidelines, which you are free to ignore :)
|
10
|
+
|
11
|
+
## Want to learn?
|
12
|
+
|
13
|
+
Want to lurk about and see what others are doing with Logstash?
|
14
|
+
|
15
|
+
* The irc channel (#logstash on irc.freenode.org) is a good place for this
|
16
|
+
* The [forum](https://discuss.elastic.co/c/logstash) is also
|
17
|
+
great for learning from others.
|
18
|
+
|
19
|
+
## Got Questions?
|
20
|
+
|
21
|
+
Have a problem you want Logstash to solve for you?
|
22
|
+
|
23
|
+
* You can ask a question in the [forum](https://discuss.elastic.co/c/logstash)
|
24
|
+
* Alternately, you are welcome to join the IRC channel #logstash on
|
25
|
+
irc.freenode.org and ask for help there!
|
26
|
+
|
27
|
+
## Have an Idea or Feature Request?
|
28
|
+
|
29
|
+
* File a ticket on [GitHub](https://github.com/elastic/logstash/issues). Please remember that GitHub is used only for issues and feature requests. If you have a general question, the [forum](https://discuss.elastic.co/c/logstash) or IRC would be the best place to ask.
|
30
|
+
|
31
|
+
## Something Not Working? Found a Bug?
|
32
|
+
|
33
|
+
If you think you found a bug, it probably is a bug.
|
34
|
+
|
35
|
+
* If it is a general Logstash or a pipeline issue, file it in [Logstash GitHub](https://github.com/elasticsearch/logstash/issues)
|
36
|
+
* If it is specific to a plugin, please file it in the respective repository under [logstash-plugins](https://github.com/logstash-plugins)
|
37
|
+
* or ask the [forum](https://discuss.elastic.co/c/logstash).
|
38
|
+
|
39
|
+
# Contributing Documentation and Code Changes
|
40
|
+
|
41
|
+
If you have a bugfix or new feature that you would like to contribute to
|
42
|
+
logstash, and you think it will take more than a few minutes to produce the fix
|
43
|
+
(ie; write code), it is worth discussing the change with the Logstash users and developers first! You can reach us via [GitHub](https://github.com/elastic/logstash/issues), the [forum](https://discuss.elastic.co/c/logstash), or via IRC (#logstash on freenode irc)
|
44
|
+
Please note that Pull Requests without tests will not be merged. If you would like to contribute but do not have experience with writing tests, please ping us on IRC/forum or create a PR and ask our help.
|
45
|
+
|
46
|
+
## Contributing to plugins
|
47
|
+
|
48
|
+
Check our [documentation](https://www.elastic.co/guide/en/logstash/current/contributing-to-logstash.html) on how to contribute to plugins or write your own! It is super easy!
|
49
|
+
|
50
|
+
## Contribution Steps
|
51
|
+
|
52
|
+
1. Test your changes! [Run](https://github.com/elastic/logstash#testing) the test suite
|
53
|
+
2. Please make sure you have signed our [Contributor License
|
54
|
+
Agreement](https://www.elastic.co/contributor-agreement/). We are not
|
55
|
+
asking you to assign copyright to us, but to give us the right to distribute
|
56
|
+
your code without restriction. We ask this of all contributors in order to
|
57
|
+
assure our users of the origin and continuing existence of the code. You
|
58
|
+
only need to sign the CLA once.
|
59
|
+
3. Send a pull request! Push your changes to your fork of the repository and
|
60
|
+
[submit a pull
|
61
|
+
request](https://help.github.com/articles/using-pull-requests). In the pull
|
62
|
+
request, describe what your changes do and mention any bugs/issues related
|
63
|
+
to the pull request.
|
64
|
+
|
65
|
+
|
@@ -0,0 +1,9 @@
|
|
1
|
+
Please post all product and debugging questions on our [forum](https://discuss.elastic.co/c/logstash). Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.
|
2
|
+
|
3
|
+
For all general issues, please provide the following details for fast resolution:
|
4
|
+
|
5
|
+
- Version:
|
6
|
+
- Operating System:
|
7
|
+
- Config File (if you have sensitive info, please remove it):
|
8
|
+
- Sample Data:
|
9
|
+
- Steps to Reproduce:
|
@@ -0,0 +1 @@
|
|
1
|
+
Thanks for contributing to Logstash! If you haven't already signed our CLA, here's a handy link: https://www.elastic.co/contributor-agreement/
|
data/.gitignore
ADDED
data/.travis.yml
ADDED
data/CHANGELOG.md
ADDED
@@ -0,0 +1,5 @@
|
|
1
|
+
## 2.0.0
|
2
|
+
- Plugins were updated to follow the new shutdown semantic, this mainly allows Logstash to instruct input plugins to terminate gracefully,
|
3
|
+
instead of using Thread.raise on the plugins' threads. Ref: https://github.com/elastic/logstash/pull/3895
|
4
|
+
- Dependency on logstash-core update to 2.0
|
5
|
+
|
data/Gemfile
ADDED
data/LICENSE
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2015 Seth Schwartzman
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
13
|
+
copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
21
|
+
SOFTWARE.
|
22
|
+
|
data/NOTICE.TXT
ADDED
data/README.md
ADDED
@@ -0,0 +1,99 @@
|
|
1
|
+
# Logstash Plugin
|
2
|
+
|
3
|
+
[![Build
|
4
|
+
Status](http://build-eu-00.elastic.co/view/LS%20Plugins/view/LS%20Outputs/job/logstash-plugin-output-newrelic-unit/badge/icon)](http://build-eu-00.elastic.co/view/LS%20Plugins/view/LS%20Outputs/job/logstash-plugin-output-newrelic-unit/)
|
5
|
+
|
6
|
+
This is a plugin for [Logstash](https://github.com/elastic/logstash).
|
7
|
+
|
8
|
+
It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
|
9
|
+
|
10
|
+
## Documentation
|
11
|
+
|
12
|
+
Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one [central location](http://www.elastic.co/guide/en/logstash/current/).
|
13
|
+
|
14
|
+
- For formatting code or config example, you can use the asciidoc `[source,ruby]` directive
|
15
|
+
- For more asciidoc formatting tips, see the excellent reference here https://github.com/elastic/docs#asciidoc-guide
|
16
|
+
|
17
|
+
## Need Help?
|
18
|
+
|
19
|
+
Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
|
20
|
+
|
21
|
+
## Developing
|
22
|
+
|
23
|
+
### 1. Plugin Developement and Testing
|
24
|
+
|
25
|
+
#### Code
|
26
|
+
- To get started, you'll need JRuby with the Bundler gem installed.
|
27
|
+
|
28
|
+
- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
|
29
|
+
|
30
|
+
- Install dependencies
|
31
|
+
```sh
|
32
|
+
bundle install
|
33
|
+
```
|
34
|
+
|
35
|
+
#### Test
|
36
|
+
|
37
|
+
- Update your dependencies
|
38
|
+
|
39
|
+
```sh
|
40
|
+
bundle install
|
41
|
+
```
|
42
|
+
|
43
|
+
- Run tests
|
44
|
+
|
45
|
+
```sh
|
46
|
+
bundle exec rspec
|
47
|
+
```
|
48
|
+
|
49
|
+
### 2. Running your unpublished Plugin in Logstash
|
50
|
+
|
51
|
+
#### 2.1 Run in a local Logstash clone
|
52
|
+
|
53
|
+
- Edit Logstash `Gemfile` and add the local plugin path, for example:
|
54
|
+
```ruby
|
55
|
+
gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
|
56
|
+
```
|
57
|
+
- Install plugin
|
58
|
+
```sh
|
59
|
+
# Logstash 2.3 and higher
|
60
|
+
bin/logstash-plugin install --no-verify
|
61
|
+
|
62
|
+
# Prior to Logstash 2.3
|
63
|
+
bin/plugin install --no-verify
|
64
|
+
|
65
|
+
```
|
66
|
+
- Run Logstash with your plugin
|
67
|
+
```sh
|
68
|
+
bin/logstash -e 'filter {awesome {}}'
|
69
|
+
```
|
70
|
+
At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
|
71
|
+
|
72
|
+
#### 2.2 Run in an installed Logstash
|
73
|
+
|
74
|
+
You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
|
75
|
+
|
76
|
+
- Build your plugin gem
|
77
|
+
```sh
|
78
|
+
gem build logstash-filter-awesome.gemspec
|
79
|
+
```
|
80
|
+
- Install the plugin from the Logstash home
|
81
|
+
```sh
|
82
|
+
# Logstash 2.3 and higher
|
83
|
+
bin/logstash-plugin install --no-verify
|
84
|
+
|
85
|
+
# Prior to Logstash 2.3
|
86
|
+
bin/plugin install --no-verify
|
87
|
+
|
88
|
+
```
|
89
|
+
- Start Logstash and proceed to test the plugin
|
90
|
+
|
91
|
+
## Contributing
|
92
|
+
|
93
|
+
All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
|
94
|
+
|
95
|
+
Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
|
96
|
+
|
97
|
+
It is more important to the community that you are able to contribute.
|
98
|
+
|
99
|
+
For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
|
data/Rakefile
ADDED
@@ -0,0 +1,220 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
require "json"
|
3
|
+
require "logstash/outputs/base"
|
4
|
+
require "logstash/namespace"
|
5
|
+
require "net/http" # Connectivity back to New Relic Insights
|
6
|
+
require "net/https" # Connectivity back to New Relic Insights
|
7
|
+
require "stud/buffer" # For buffering events being sent
|
8
|
+
require "time"
|
9
|
+
require "uri"
|
10
|
+
|
11
|
+
# This output sends logstash events to New Relic Insights as custom events.
|
12
|
+
#
|
13
|
+
# You can learn more about New Relic Insights here:
|
14
|
+
# https://docs.newrelic.com/docs/insights/new-relic-insights/understanding-insights/new-relic-insights
|
15
|
+
class LogStash::Outputs::NewRelic < LogStash::Outputs::Base
|
16
|
+
include Stud::Buffer
|
17
|
+
|
18
|
+
config_name "newrelic"
|
19
|
+
milestone 1
|
20
|
+
|
21
|
+
# Your New Relic account ID. This is the 5 or 6-digit number found in the URL when you are logged into New Relic:
|
22
|
+
# https://rpm.newrelic.com/accounts/[account_id]/...
|
23
|
+
config :account_id, :validate => :string, :required => true
|
24
|
+
|
25
|
+
# Your Insights Insert Key. You will need to generate one if you haven't already, as described here:
|
26
|
+
# https://docs.newrelic.com/docs/insights/new-relic-insights/adding-querying-data/inserting-custom-events-insights-api#register
|
27
|
+
config :insert_key, :validate => :string, :required => true
|
28
|
+
|
29
|
+
# The name for your event type. Use alphanumeric characters only.
|
30
|
+
# If left out, your events will be stored under "logstashEvent".
|
31
|
+
config :event_type, :validate => :string, :default => "LogstashEvent"
|
32
|
+
|
33
|
+
# Should the log events be sent to Insights over https instead of plain http (typically yes).
|
34
|
+
config :proto, :validate => :string, :default => "https"
|
35
|
+
|
36
|
+
# Proxy info - all optional
|
37
|
+
# If using a proxy, only proxy_host is required.
|
38
|
+
config :proxy_host, :validate => :string
|
39
|
+
# Proxy_port will default to port 80 if left out.
|
40
|
+
config :proxy_port, :validate => :number, :default => 80
|
41
|
+
# Proxy_user should be left out if connecting to your proxy unauthenticated.
|
42
|
+
config :proxy_user, :validate => :string
|
43
|
+
# Proxy_password should be left out if connecting to your proxy unauthenticated.
|
44
|
+
config :proxy_password, :validate => :password, :default => ""
|
45
|
+
|
46
|
+
# Batch Processing - all optional
|
47
|
+
# This plugin uses the New Relic Insights REST API to send data.
|
48
|
+
# To make efficient REST API calls, we will buffer a certain number of events before flushing that out to Insights.
|
49
|
+
config :batch, :validate => :boolean, :default => true
|
50
|
+
# This setting controls how many events will be buffered before sending a batch of events.
|
51
|
+
config :batch_events, :validate => :number, :default => 10
|
52
|
+
# This setting controls how long the output will wait before sending a batch of a events,
|
53
|
+
# should the minimum specified in batch_events not be met yet.
|
54
|
+
config :batch_timeout, :validate => :number, :default => 5
|
55
|
+
|
56
|
+
# New Relic Insights Reserved Words
|
57
|
+
# https://docs.newrelic.com/docs/insights/new-relic-insights/adding-querying-data/inserting-custom-events#keywords
|
58
|
+
# backticks = change "word" to "`word`"
|
59
|
+
# moved = change "word" to "word_moved"
|
60
|
+
# skipped = skip this attribute altogether (not used so far)
|
61
|
+
# If you enter anything else, the "word" will change to the "anything else"
|
62
|
+
RESWORDS = {
|
63
|
+
"accountId" => "moved",
|
64
|
+
"appId" => "moved",
|
65
|
+
"timestamp" => "logstash timestamp",
|
66
|
+
"@timestamp" => "logstash timestamp",
|
67
|
+
"type" => "moved",
|
68
|
+
"ago" => "backticks",
|
69
|
+
"and" => "backticks",
|
70
|
+
"as" => "backticks",
|
71
|
+
"auto" => "backticks",
|
72
|
+
"begin" => "backticks",
|
73
|
+
"begintime" => "backticks",
|
74
|
+
"compare" => "backticks",
|
75
|
+
"day" => "backticks",
|
76
|
+
"days" => "backticks",
|
77
|
+
"end" => "backticks",
|
78
|
+
"endtime" => "backticks",
|
79
|
+
"explain" => "backticks",
|
80
|
+
"facet" => "backticks",
|
81
|
+
"from" => "backticks",
|
82
|
+
"hour" => "backticks",
|
83
|
+
"hours" => "backticks",
|
84
|
+
"in" => "backticks",
|
85
|
+
"is" => "backticks",
|
86
|
+
"like" => "backticks",
|
87
|
+
"limit" => "backticks",
|
88
|
+
"minute" => "backticks",
|
89
|
+
"minutes" => "backticks",
|
90
|
+
"month" => "backticks",
|
91
|
+
"months" => "backticks",
|
92
|
+
"not" => "backticks",
|
93
|
+
"null" => "backticks",
|
94
|
+
"offset" => "backticks",
|
95
|
+
"or" => "backticks",
|
96
|
+
"second" => "backticks",
|
97
|
+
"seconds" => "backticks",
|
98
|
+
"select" => "backticks",
|
99
|
+
"since" => "backticks",
|
100
|
+
"timeseries" => "backticks",
|
101
|
+
"until" => "backticks",
|
102
|
+
"week" => "backticks",
|
103
|
+
"weeks" => "backticks",
|
104
|
+
"where" => "backticks",
|
105
|
+
"with" => "backticks",
|
106
|
+
}
|
107
|
+
|
108
|
+
public
|
109
|
+
def register
|
110
|
+
# URL to send event over http(s) to the New Relic Insights REST API
|
111
|
+
@url = URI.parse("#{@proto}://insights-collector.newrelic.com/v1/accounts/#{@account_id}/events")
|
112
|
+
@logger.info("New Relic Insights output initialized.")
|
113
|
+
@logger.info("New Relic URL: #{@url}")
|
114
|
+
if @batch
|
115
|
+
@logger.info("Batch processing of events enabled.")
|
116
|
+
if @batch_events > 1000
|
117
|
+
raise RuntimeError.new("New Relic Insights only allows a batch_events parameter of 1000 or less")
|
118
|
+
end
|
119
|
+
buffer_initialize(
|
120
|
+
:max_items => @batch_events,
|
121
|
+
:max_interval => @batch_timeout,
|
122
|
+
:logger => @logger
|
123
|
+
)
|
124
|
+
end
|
125
|
+
end # def register
|
126
|
+
|
127
|
+
public
|
128
|
+
def receive(event)
|
129
|
+
return unless output?(event)
|
130
|
+
if event == LogStash::SHUTDOWN
|
131
|
+
finished
|
132
|
+
return
|
133
|
+
end
|
134
|
+
parsed_event = parse_event(event)
|
135
|
+
if @batch
|
136
|
+
buffer_receive(parsed_event)
|
137
|
+
else
|
138
|
+
send_to_insights(parsed_event)
|
139
|
+
end
|
140
|
+
end # def receive
|
141
|
+
|
142
|
+
public
|
143
|
+
# Insights REST API handles multiple events the same way as a single event. Score!
|
144
|
+
# All we need to do on 'flush' is send the contents of the events buffer.
|
145
|
+
def flush(events, teardown=false)
|
146
|
+
@logger.debug("Sending batch of #{events.size} events to insights")
|
147
|
+
send_to_insights(events)
|
148
|
+
end # def flush
|
149
|
+
|
150
|
+
public
|
151
|
+
def teardown
|
152
|
+
buffer_flush(:final => true)
|
153
|
+
finished
|
154
|
+
end # def teardown
|
155
|
+
|
156
|
+
# Turn event into an Insights-compliant event
|
157
|
+
public
|
158
|
+
def parse_event(event)
|
159
|
+
this_event = event.to_hash
|
160
|
+
output_event = Hash.new
|
161
|
+
|
162
|
+
# Setting eventType to what's in the config
|
163
|
+
output_event['eventType'] = @event_type
|
164
|
+
|
165
|
+
# Setting timestamp to what logstash reports
|
166
|
+
if this_event.key?('@timestamp')
|
167
|
+
timestamp_parsed = Time.parse(this_event['@timestamp'].to_s)
|
168
|
+
output_event['timestamp'] = timestamp_parsed.to_i
|
169
|
+
elsif this_event.key?('timestamp')
|
170
|
+
timestamp_parsed = Time.parse(this_event['timestamp'].to_s)
|
171
|
+
output_event['timestamp'] = timestamp_parsed.to_i
|
172
|
+
end
|
173
|
+
|
174
|
+
# Search event's attribute names for reserved words, replace with 'compliant' versions
|
175
|
+
# Storing 'compliant' key names in "EVENT_KEYS" to minimize time spent doing this
|
176
|
+
this_event.each_key do |event_key|
|
177
|
+
if RESWORDS.has_key?(event_key)
|
178
|
+
@logger.debug("Reserved word found", :reserved_word => event_key)
|
179
|
+
if RESWORDS[event_key] == "skipped"
|
180
|
+
next
|
181
|
+
elsif RESWORDS[event_key] == "moved"
|
182
|
+
proper_name = event_key + "_moved"
|
183
|
+
elsif RESWORDS[event_key] == "backticks"
|
184
|
+
proper_name = "`" + event_key + "`"
|
185
|
+
else
|
186
|
+
proper_name = RESWORDS[event_key]
|
187
|
+
end
|
188
|
+
else
|
189
|
+
proper_name = event_key
|
190
|
+
end
|
191
|
+
output_event[proper_name] = this_event[event_key]
|
192
|
+
end
|
193
|
+
return output_event
|
194
|
+
end # def parse_event
|
195
|
+
|
196
|
+
# Send event(s) to the NR Insights REST API.
|
197
|
+
# Can handle a single event or batched events.
|
198
|
+
def send_to_insights(event)
|
199
|
+
http = Net::HTTP.new(@url.host, @url.port, @proxy_host, @proxy_port, @proxy_user, @proxy_password.value)
|
200
|
+
if @url.scheme == 'https'
|
201
|
+
http.use_ssl = true
|
202
|
+
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
|
203
|
+
end
|
204
|
+
|
205
|
+
# Insights uses a POST, requires Content-Type and X-Insert-Key.
|
206
|
+
request = Net::HTTP::Post.new(@url.path)
|
207
|
+
request['Content-Type'] = "application/json"
|
208
|
+
request['X-Insert-Key'] = @insert_key
|
209
|
+
request.body = event.to_json
|
210
|
+
@logger.debug("Request Body:", :request_body => request.body)
|
211
|
+
|
212
|
+
response = http.request(request)
|
213
|
+
if response.is_a?(Net::HTTPSuccess)
|
214
|
+
@logger.debug("Event sent to New Relic SUCCEEDED! Response Code:", :response_code => response.code)
|
215
|
+
else
|
216
|
+
@logger.warn("Event sent to New Relic FAILED. Error:", :error => response.error!)
|
217
|
+
end
|
218
|
+
end # def send_to_insights
|
219
|
+
|
220
|
+
end # class LogStash::Outputs::NewRelic
|
@@ -0,0 +1,27 @@
|
|
1
|
+
Gem::Specification.new do |s|
|
2
|
+
|
3
|
+
s.name = "logstash-output-newrelic"
|
4
|
+
s.version = "0.9.1"
|
5
|
+
s.summary = "New Relic Insights output plugin for Logstash"
|
6
|
+
s.description = "Use logstash to ship log events to New Relic Insights. This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program."
|
7
|
+
s.authors = ["The Chocolate Factory"]
|
8
|
+
s.email = "info@chocolatefirm.com"
|
9
|
+
s.homepage = "http://www.chocolatefirm.com"
|
10
|
+
s.licenses = ["MIT"]
|
11
|
+
s.require_paths = ["lib"]
|
12
|
+
|
13
|
+
# Files
|
14
|
+
s.files = `git ls-files`.split($\)+::Dir.glob('vendor/*')
|
15
|
+
|
16
|
+
# Tests
|
17
|
+
s.test_files = s.files.grep(%r{^(test|spec|features)/})
|
18
|
+
|
19
|
+
# Special flag to let us know this is actually a logstash plugin
|
20
|
+
s.metadata = { "logstash_plugin" => "true", "logstash_group" => "output" }
|
21
|
+
|
22
|
+
# Gem dependencies
|
23
|
+
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
|
24
|
+
s.add_runtime_dependency 'logstash-codec-plain'
|
25
|
+
s.add_development_dependency 'logstash-devutils'
|
26
|
+
|
27
|
+
end
|
@@ -0,0 +1,66 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
require "logstash/devutils/rspec/spec_helper"
|
3
|
+
require "logstash/outputs/newrelic"
|
4
|
+
require "logstash/event"
|
5
|
+
require "logstash/timestamp"
|
6
|
+
require "time"
|
7
|
+
|
8
|
+
# TO DO (in order of easiest to hardest):
|
9
|
+
# + Test batch_timeout
|
10
|
+
# + Test proxy (oy gevalt)
|
11
|
+
|
12
|
+
describe LogStash::Outputs::NewRelic do
|
13
|
+
|
14
|
+
let (:simple_event_contents) { { 'message' => 'hello', 'topic_name' => 'my_topic', 'host' => '172.0.0.1' } }
|
15
|
+
let(:options) { { 'account_id' => "284929",
|
16
|
+
'insert_key' => "BYh7sByiVrkfqcDa2eqVMhjxafkdyuX0" } }
|
17
|
+
let(:simple_output) { LogStash::Plugin.lookup("output", "newrelic").new(options) }
|
18
|
+
|
19
|
+
describe "#register" do
|
20
|
+
it "should register" do
|
21
|
+
output = LogStash::Plugin.lookup("output", "newrelic").new(options)
|
22
|
+
expect { output.register }.to_not raise_error
|
23
|
+
end
|
24
|
+
|
25
|
+
it "should NOT register when batch_events > 1000" do
|
26
|
+
options['batch_events'] = 1001
|
27
|
+
output = LogStash::Plugin.lookup("output", "newrelic").new(options)
|
28
|
+
expect { output.register }.to raise_error(RuntimeError)
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
describe "#parse_event" do
|
33
|
+
it "should convert attribute names" do
|
34
|
+
simple_event_contents['accountId'] = '123456'
|
35
|
+
simple_event_contents['compare'] = 'backtick that'
|
36
|
+
simple_event_contents['test_of_anything_else'] = 'leave this'
|
37
|
+
simple_event = LogStash::Event.new(simple_event_contents)
|
38
|
+
test_event = simple_output.parse_event(simple_event)
|
39
|
+
expect(test_event['accountId_moved']).to eq(simple_event_contents['accountId'])
|
40
|
+
expect(test_event['`compare`']).to eq(simple_event_contents['compare'])
|
41
|
+
expect(test_event['test_of_anything_else']).to eq(simple_event_contents['test_of_anything_else'])
|
42
|
+
end
|
43
|
+
end
|
44
|
+
|
45
|
+
describe "#receive" do
|
46
|
+
it "should send a single event" do
|
47
|
+
options['batch'] = false
|
48
|
+
output = LogStash::Plugin.lookup("output", "newrelic").new(options)
|
49
|
+
output.register
|
50
|
+
simple_event = LogStash::Event.new(simple_event_contents)
|
51
|
+
expect { output.receive(simple_event) }.to_not raise_error
|
52
|
+
end
|
53
|
+
|
54
|
+
it "should send multiple events" do
|
55
|
+
batch_event_count = 5
|
56
|
+
options["batch_events"] = batch_event_count
|
57
|
+
output = LogStash::Plugin.lookup("output", "newrelic").new(options)
|
58
|
+
output.register
|
59
|
+
for i in 0..batch_event_count
|
60
|
+
simple_event_contents['iteration'] = i
|
61
|
+
simple_event = LogStash::Event.new(simple_event_contents)
|
62
|
+
expect { output.receive(simple_event) }.to_not raise_error
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
66
|
+
end
|
metadata
ADDED
@@ -0,0 +1,111 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: logstash-output-newrelic
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.9.1
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- The Chocolate Factory
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2016-09-22 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
15
|
+
requirements:
|
16
|
+
- - '>='
|
17
|
+
- !ruby/object:Gem::Version
|
18
|
+
version: '1.60'
|
19
|
+
- - <=
|
20
|
+
- !ruby/object:Gem::Version
|
21
|
+
version: '2.99'
|
22
|
+
name: logstash-core-plugin-api
|
23
|
+
prerelease: false
|
24
|
+
type: :runtime
|
25
|
+
version_requirements: !ruby/object:Gem::Requirement
|
26
|
+
requirements:
|
27
|
+
- - '>='
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: '1.60'
|
30
|
+
- - <=
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '2.99'
|
33
|
+
- !ruby/object:Gem::Dependency
|
34
|
+
requirement: !ruby/object:Gem::Requirement
|
35
|
+
requirements:
|
36
|
+
- - '>='
|
37
|
+
- !ruby/object:Gem::Version
|
38
|
+
version: '0'
|
39
|
+
name: logstash-codec-plain
|
40
|
+
prerelease: false
|
41
|
+
type: :runtime
|
42
|
+
version_requirements: !ruby/object:Gem::Requirement
|
43
|
+
requirements:
|
44
|
+
- - '>='
|
45
|
+
- !ruby/object:Gem::Version
|
46
|
+
version: '0'
|
47
|
+
- !ruby/object:Gem::Dependency
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
49
|
+
requirements:
|
50
|
+
- - '>='
|
51
|
+
- !ruby/object:Gem::Version
|
52
|
+
version: '0'
|
53
|
+
name: logstash-devutils
|
54
|
+
prerelease: false
|
55
|
+
type: :development
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
57
|
+
requirements:
|
58
|
+
- - '>='
|
59
|
+
- !ruby/object:Gem::Version
|
60
|
+
version: '0'
|
61
|
+
description: Use logstash to ship log events to New Relic Insights. This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program.
|
62
|
+
email: info@chocolatefirm.com
|
63
|
+
executables: []
|
64
|
+
extensions: []
|
65
|
+
extra_rdoc_files: []
|
66
|
+
files:
|
67
|
+
- .github/CONTRIBUTING.md
|
68
|
+
- .github/ISSUE_TEMPLATE.md
|
69
|
+
- .github/PULL_REQUEST_TEMPLATE.md
|
70
|
+
- .gitignore
|
71
|
+
- .travis.yml
|
72
|
+
- CHANGELOG.md
|
73
|
+
- Gemfile
|
74
|
+
- LICENSE
|
75
|
+
- NOTICE.TXT
|
76
|
+
- README.md
|
77
|
+
- Rakefile
|
78
|
+
- bin/logstash-newrelic
|
79
|
+
- bin/logstash-newrelic-debug
|
80
|
+
- bin/logstash-newrelic.conf.template
|
81
|
+
- lib/logstash/outputs/newrelic.rb
|
82
|
+
- logstash-output-newrelic.gemspec
|
83
|
+
- spec/outputs/newrelic_spec.rb
|
84
|
+
homepage: http://www.chocolatefirm.com
|
85
|
+
licenses:
|
86
|
+
- MIT
|
87
|
+
metadata:
|
88
|
+
logstash_plugin: 'true'
|
89
|
+
logstash_group: output
|
90
|
+
post_install_message:
|
91
|
+
rdoc_options: []
|
92
|
+
require_paths:
|
93
|
+
- lib
|
94
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
95
|
+
requirements:
|
96
|
+
- - '>='
|
97
|
+
- !ruby/object:Gem::Version
|
98
|
+
version: '0'
|
99
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - '>='
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '0'
|
104
|
+
requirements: []
|
105
|
+
rubyforge_project:
|
106
|
+
rubygems_version: 2.4.5
|
107
|
+
signing_key:
|
108
|
+
specification_version: 4
|
109
|
+
summary: New Relic Insights output plugin for Logstash
|
110
|
+
test_files:
|
111
|
+
- spec/outputs/newrelic_spec.rb
|