logstash-filter-forwarded 1.0.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/CHANGELOG.md +12 -0
- data/CONTRIBUTORS +1 -0
- data/Gemfile +4 -0
- data/LICENSE +13 -0
- data/NOTICE.TXT +4 -0
- data/README.md +107 -0
- data/lib/logstash/filters/forwarded.rb +124 -0
- data/logstash-filter-forwarded.gemspec +25 -0
- data/spec/filters/forwarded_spec.rb +215 -0
- metadata +103 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 9836c06be1410c903946dd50caa1db25579a8b9b
|
4
|
+
data.tar.gz: 8d1113c740982044469e4b3ca019d4ef5de091fc
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: b3564a029f707a9e3985572c9dadc97c3cbce86db446d6e552217119fad0262ae7993bd22f2f746d264f70cdb471d6538321cc665b366080bbd2ebce318dfdd4
|
7
|
+
data.tar.gz: 12f5ccc4062b569e666bf57f7dd8553c73eac3aaa6d4b7d9f705f0ab5506140433296b4a4fab3d9555c40c4c4aa5e5a1a1024a70cc6465e16247ac6cf318075f
|
data/CHANGELOG.md
ADDED
data/CONTRIBUTORS
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
|
data/Gemfile
ADDED
data/LICENSE
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
Copyright (c) 2012–2016 Elasticsearch <http://www.elastic.co>
|
2
|
+
|
3
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
+
you may not use this file except in compliance with the License.
|
5
|
+
You may obtain a copy of the License at
|
6
|
+
|
7
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
8
|
+
|
9
|
+
Unless required by applicable law or agreed to in writing, software
|
10
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
+
See the License for the specific language governing permissions and
|
13
|
+
limitations under the License.
|
data/NOTICE.TXT
ADDED
data/README.md
ADDED
@@ -0,0 +1,107 @@
|
|
1
|
+
# Logstash Plugin
|
2
|
+
|
3
|
+
[![Travis Build Status](https://travis-ci.org/IngaFeick/logstash-filter-forwarded.svg)](https://travis-ci.org/IngaFeick/logstash-filter-forwarded)
|
4
|
+
|
5
|
+
This is a plugin for [Logstash](https://github.com/elastic/logstash).
|
6
|
+
|
7
|
+
It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
|
8
|
+
|
9
|
+
The forwarded plugin allows you to dissect a list of ip addresses from the x-forwarded-for header and to identify the client ip in that list, regardless of whether or not the header complies to the x-forwarded-for format. Unless specified otherwise, it will add a field "forwarded_client_ip" to your event, along with a "forwarded_proxy_list" containing all the other ips in the x-forwarded-for string.
|
10
|
+
|
11
|
+
## Documentation
|
12
|
+
|
13
|
+
Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one [central location](http://www.elastic.co/guide/en/logstash/current/).
|
14
|
+
|
15
|
+
- For formatting code or config example, you can use the asciidoc `[source,ruby]` directive
|
16
|
+
- For more asciidoc formatting tips, see the excellent reference here https://github.com/elastic/docs#asciidoc-guide
|
17
|
+
|
18
|
+
## Need Help?
|
19
|
+
|
20
|
+
Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
|
21
|
+
|
22
|
+
## Developing
|
23
|
+
|
24
|
+
### 1. Plugin Developement and Testing
|
25
|
+
|
26
|
+
#### Code
|
27
|
+
- To get started, you'll need JRuby with the Bundler gem installed.
|
28
|
+
|
29
|
+
- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
|
30
|
+
|
31
|
+
- Install dependencies
|
32
|
+
```sh
|
33
|
+
bundle install
|
34
|
+
```
|
35
|
+
|
36
|
+
#### Test
|
37
|
+
|
38
|
+
- Update your dependencies
|
39
|
+
|
40
|
+
```sh
|
41
|
+
bundle install
|
42
|
+
```
|
43
|
+
|
44
|
+
- Pull down GeoIP database files
|
45
|
+
|
46
|
+
```sh
|
47
|
+
bundle exec rake vendor
|
48
|
+
```
|
49
|
+
|
50
|
+
|
51
|
+
- Run tests
|
52
|
+
|
53
|
+
```sh
|
54
|
+
bundle exec rspec
|
55
|
+
```
|
56
|
+
|
57
|
+
### 2. Running your unpublished Plugin in Logstash
|
58
|
+
|
59
|
+
#### 2.1 Run in a local Logstash clone
|
60
|
+
|
61
|
+
- Edit Logstash `Gemfile` and add the local plugin path, for example:
|
62
|
+
```ruby
|
63
|
+
gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
|
64
|
+
```
|
65
|
+
- Install plugin
|
66
|
+
```sh
|
67
|
+
# Logstash 2.3 and higher
|
68
|
+
bin/logstash-plugin install --no-verify
|
69
|
+
|
70
|
+
# Prior to Logstash 2.3
|
71
|
+
bin/plugin install --no-verify
|
72
|
+
|
73
|
+
```
|
74
|
+
- Run Logstash with your plugin
|
75
|
+
```sh
|
76
|
+
bin/logstash -e 'filter {awesome {}}'
|
77
|
+
```
|
78
|
+
At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
|
79
|
+
|
80
|
+
#### 2.2 Run in an installed Logstash
|
81
|
+
|
82
|
+
You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
|
83
|
+
|
84
|
+
- Build your plugin gem
|
85
|
+
```sh
|
86
|
+
gem build logstash-filter-awesome.gemspec
|
87
|
+
```
|
88
|
+
- Install the plugin from the Logstash home
|
89
|
+
```sh
|
90
|
+
# Logstash 2.3 and higher
|
91
|
+
bin/logstash-plugin install --no-verify
|
92
|
+
|
93
|
+
# Prior to Logstash 2.3
|
94
|
+
bin/plugin install --no-verify
|
95
|
+
|
96
|
+
```
|
97
|
+
- Start Logstash and proceed to test the plugin
|
98
|
+
|
99
|
+
## Contributing
|
100
|
+
|
101
|
+
All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
|
102
|
+
|
103
|
+
Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
|
104
|
+
|
105
|
+
It is more important to the community that you are able to contribute.
|
106
|
+
|
107
|
+
For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
|
@@ -0,0 +1,124 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
require "logstash/filters/base"
|
3
|
+
require "logstash/namespace"
|
4
|
+
require "ipaddr" # needed for network range check
|
5
|
+
require "ipaddress" # needed for validity check
|
6
|
+
|
7
|
+
# The forwarded filter extracts the client ip from a list of ip adresses. The client ip might be at any position in the list, since not every x-forwarded-for header comes in the correct ordering.
|
8
|
+
# It adds two new fields to the event:
|
9
|
+
# - forwarded_client_ip : string
|
10
|
+
# - forwarded_proxy_list : string[]
|
11
|
+
|
12
|
+
class LogStash::Filters::Forwarded < LogStash::Filters::Base
|
13
|
+
config_name "forwarded"
|
14
|
+
|
15
|
+
# The field containing the x-forwarded-for string
|
16
|
+
config :source, :validate => :string, :required => true
|
17
|
+
|
18
|
+
# list of ip patterns that private ips start with.
|
19
|
+
config :private_ipv4_prefixes, :validate => :array, :required => false, :default => ["10.0.0.0/8", "192.168.0.0/16" ,"172.16.0.0/12"]
|
20
|
+
|
21
|
+
# Private IP Addresses have the following ranges:
|
22
|
+
# 10.0.0.0 - 10.255.255.255
|
23
|
+
# 172.16.0.0 - 172.31.255.255
|
24
|
+
# 192.168.0.0 - 192.168.255.255
|
25
|
+
|
26
|
+
# The name of the new field containing client ip (optional)
|
27
|
+
config :target_client_ip, :validate => :string, :required => false, :default => "forwarded_client_ip"
|
28
|
+
|
29
|
+
# The name of the new field containing proxy list (optional)
|
30
|
+
config :target_proxy_list, :validate => :string, :required => false, :default => "forwarded_proxy_list"
|
31
|
+
|
32
|
+
public
|
33
|
+
def register
|
34
|
+
@private_ipv4_ranges = @private_ipv4_prefixes.collect do | adress |
|
35
|
+
begin
|
36
|
+
IPAddr.new(adress)
|
37
|
+
rescue ArgumentError => e
|
38
|
+
@logger.error("Register: invalid IP network, skipping", :adress => adress, :exception => e)
|
39
|
+
raise e
|
40
|
+
end
|
41
|
+
end
|
42
|
+
@private_ipv4_ranges.compact!
|
43
|
+
end # def register
|
44
|
+
|
45
|
+
public
|
46
|
+
def filter(event)
|
47
|
+
return unless filter?(event)
|
48
|
+
|
49
|
+
begin
|
50
|
+
forwarded = event.get(@source)
|
51
|
+
|
52
|
+
return unless forwarded and !forwarded.empty?
|
53
|
+
|
54
|
+
client_ip, proxies = analyse(forwarded)
|
55
|
+
|
56
|
+
event.set(@target_client_ip, client_ip) if client_ip
|
57
|
+
event.set(@target_proxy_list, proxies) if proxies
|
58
|
+
filter_matched(event)
|
59
|
+
|
60
|
+
rescue Exception => e
|
61
|
+
@logger.debug("Unknown error while looking up GeoIP data", :exception => e, :field => @source, :event => event)
|
62
|
+
# raise e
|
63
|
+
end # begin
|
64
|
+
end # def filter
|
65
|
+
|
66
|
+
def analyse(ip)
|
67
|
+
return nil, nil if ip.nil?
|
68
|
+
# convert the x-forwarded-for string into an array of its comma separated value, if it isn't already.
|
69
|
+
ip_list = ip.is_a?(Array) ? ip : ip.downcase.split(",")
|
70
|
+
|
71
|
+
# remove some well-known invalid values
|
72
|
+
ip_list = ip_list.map { |x| x.strip }.reject { |x| ["-", "unknown"].include? x}
|
73
|
+
|
74
|
+
# the IpAddr library cannot handle ips with port numbers
|
75
|
+
ip_list = ip_list.map { |x| remove_port_number(x) }
|
76
|
+
|
77
|
+
# get the first public ip in the list
|
78
|
+
client_ip = get_client_ip(ip_list)
|
79
|
+
|
80
|
+
# remove the public / client ip from the list and use the remainder as the list of proxies involved.
|
81
|
+
proxies = ip_list.nil? ? [] : ip_list - [client_ip]
|
82
|
+
|
83
|
+
return client_ip, proxies
|
84
|
+
end # def analyse
|
85
|
+
|
86
|
+
def get_client_ip(ip_array)
|
87
|
+
ip_array.each do | ip |
|
88
|
+
begin
|
89
|
+
next if !IPAddress.valid? ip
|
90
|
+
|
91
|
+
ipo = IPAddr.new(ip)
|
92
|
+
is_private = ipo.ipv6? ? is_private_ipv6(ip) : is_private_ipv4(ipo)
|
93
|
+
return ip if !is_private
|
94
|
+
rescue => e
|
95
|
+
# not a valid ip, moving on.
|
96
|
+
# @logger.debug("get_client_ip() failed", :exception => e, :field => @source, :ip_array => ip_array)
|
97
|
+
next
|
98
|
+
end
|
99
|
+
end # each
|
100
|
+
nil
|
101
|
+
end # get_client_ip
|
102
|
+
|
103
|
+
def remove_port_number(ip)
|
104
|
+
tokens = ip.split(":")
|
105
|
+
if tokens.size <=2 then tokens[0] else ip end
|
106
|
+
end
|
107
|
+
|
108
|
+
def is_private_ipv6(ip)
|
109
|
+
ip.start_with?("fd") || ip.start_with?("fc")
|
110
|
+
end # is_private_ipv6
|
111
|
+
|
112
|
+
|
113
|
+
def is_private_ipv4(ipo)
|
114
|
+
begin
|
115
|
+
@private_ipv4_ranges.each do | ip_range |
|
116
|
+
return true if ip_range.include?(ipo)
|
117
|
+
end # each
|
118
|
+
false
|
119
|
+
rescue => e
|
120
|
+
@logger.debug("Couldn't check if ip is private.", :input_data => ip, :exception => e)
|
121
|
+
end # begin
|
122
|
+
end # is_private
|
123
|
+
|
124
|
+
end # class LogStash::Filters::Forwarded
|
@@ -0,0 +1,25 @@
|
|
1
|
+
Gem::Specification.new do |s|
|
2
|
+
|
3
|
+
s.name = 'logstash-filter-forwarded'
|
4
|
+
s.version = '1.0.4'
|
5
|
+
s.licenses = ['Apache License (2.0)']
|
6
|
+
s.summary = "$summary"
|
7
|
+
s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"
|
8
|
+
s.authors = ["Inga Feick"]
|
9
|
+
s.email = 'inga.feick@trivago.com'
|
10
|
+
s.require_paths = ["lib"]
|
11
|
+
|
12
|
+
# Files
|
13
|
+
s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
|
14
|
+
|
15
|
+
# Tests
|
16
|
+
s.test_files = s.files.grep(%r{^(test|spec|features)/})
|
17
|
+
|
18
|
+
# Special flag to let us know this is actually a logstash plugin
|
19
|
+
s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
|
20
|
+
|
21
|
+
# Gem dependencies
|
22
|
+
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
|
23
|
+
s.add_runtime_dependency 'ipaddress'
|
24
|
+
s.add_development_dependency 'logstash-devutils'
|
25
|
+
end
|
@@ -0,0 +1,215 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
require "logstash/devutils/rspec/spec_helper"
|
3
|
+
require "logstash/filters/forwarded"
|
4
|
+
require "logstash/timestamp"
|
5
|
+
|
6
|
+
describe LogStash::Filters::Forwarded do
|
7
|
+
|
8
|
+
let(:plugin) { LogStash::Filters::Forwarded.new("source" => "message") }
|
9
|
+
|
10
|
+
before do
|
11
|
+
plugin.register
|
12
|
+
plugin.filter(event)
|
13
|
+
end
|
14
|
+
|
15
|
+
# Private IP Addresses have the following ranges:
|
16
|
+
#10.0.0.0 - 10.255.255.255
|
17
|
+
#172.16.0.0 - 172.31.255.255
|
18
|
+
#192.168.0.0 - 192.168.255.255
|
19
|
+
|
20
|
+
|
21
|
+
context "1) multiple client ips" do
|
22
|
+
let(:event) { LogStash::Event.new(:message => "123.45.67.89,61.160.232.222") }
|
23
|
+
it "should take the first client ip" do
|
24
|
+
expect(event.get("forwarded_client_ip")).to eq("123.45.67.89")
|
25
|
+
expect(event.get("forwarded_proxy_list")).to eq(["61.160.232.222"])
|
26
|
+
|
27
|
+
end # it
|
28
|
+
end # context
|
29
|
+
|
30
|
+
context "2) proper x-forwarded-for" do
|
31
|
+
let(:event) { LogStash::Event.new(:message => "84.30.67.207, 10.1.2.162") }
|
32
|
+
it "should take the first public ip" do
|
33
|
+
expect(event.get("forwarded_client_ip")).to eq("84.30.67.207")
|
34
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.162"])
|
35
|
+
end # it
|
36
|
+
end # context
|
37
|
+
|
38
|
+
context "3) proper x-forwarded-for with multiple proxies" do
|
39
|
+
let(:event) { LogStash::Event.new(:message => "94.254.183.48, 10.1.2.161, 10.1.2.162") }
|
40
|
+
it "should take the first ip" do
|
41
|
+
expect(event.get("forwarded_client_ip")).to eq("94.254.183.48")
|
42
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.161","10.1.2.162"])
|
43
|
+
end # it
|
44
|
+
end # context
|
45
|
+
|
46
|
+
context "4) proper x-forwarded-for with multiple proxies, no whitespace" do
|
47
|
+
let(:event) { LogStash::Event.new(:message => "51.174.213.194,10.1.2.100,10.1.2.83") }
|
48
|
+
it "should take the first client ip" do
|
49
|
+
expect(event.get("forwarded_client_ip")).to eq("51.174.213.194")
|
50
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.100","10.1.2.83"])
|
51
|
+
end # it
|
52
|
+
end # context
|
53
|
+
|
54
|
+
context "5) single client ip" do
|
55
|
+
let(:event) { LogStash::Event.new(:message => "185.22.141.112") }
|
56
|
+
it "should set the client ip and leave the proxy list empty" do
|
57
|
+
expect(event.get("forwarded_client_ip")).to eq("185.22.141.112")
|
58
|
+
expect(event.get("forwarded_proxy_list")).to eq([])
|
59
|
+
end # it
|
60
|
+
end # context
|
61
|
+
|
62
|
+
context "6) single proxy ip" do
|
63
|
+
let(:event) { LogStash::Event.new(:message => "10.1.2.162") }
|
64
|
+
it "should set the proxy list and leave the client ip empty" do
|
65
|
+
expect(event.get("forwarded_client_ip")).to eq(nil)
|
66
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.162"])
|
67
|
+
end # it
|
68
|
+
end # context
|
69
|
+
|
70
|
+
context "7) multiple proxy ips" do
|
71
|
+
let(:event) { LogStash::Event.new(:message => "10.1.2.162, 10.1.3.255") }
|
72
|
+
it "should set the proxy list and leave the client ip empty" do
|
73
|
+
expect(event.get("forwarded_client_ip")).to eq(nil)
|
74
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.162","10.1.3.255"])
|
75
|
+
end # it
|
76
|
+
end # context
|
77
|
+
|
78
|
+
context "8) empty message" do
|
79
|
+
let(:event) { LogStash::Event.new(:message => "") }
|
80
|
+
it "should return empty or nil values" do
|
81
|
+
expect(event.get("forwarded_client_ip")).to eq(nil)
|
82
|
+
expect(event.get("forwarded_proxy_list")).to eq(nil)
|
83
|
+
end # it
|
84
|
+
end # context
|
85
|
+
|
86
|
+
context "9) multiple ips in wrong order" do
|
87
|
+
let(:event) { LogStash::Event.new(:message => "10.144.80.56, 82.132.186.219") }
|
88
|
+
it "should take the client ip from the right end of the list" do
|
89
|
+
expect(event.get("forwarded_client_ip")).to eq("82.132.186.219")
|
90
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.144.80.56"])
|
91
|
+
end # it
|
92
|
+
end # context
|
93
|
+
|
94
|
+
context "10) edge case test for 192.x range" do
|
95
|
+
let(:event) { LogStash::Event.new(:message => "192.168.255.255, 192.169.0.13") }
|
96
|
+
it "should take the client ip from the right end of the list" do
|
97
|
+
expect(event.get("forwarded_client_ip")).to eq("192.169.0.13")
|
98
|
+
expect(event.get("forwarded_proxy_list")).to eq(["192.168.255.255"])
|
99
|
+
end # it
|
100
|
+
|
101
|
+
end # context
|
102
|
+
|
103
|
+
context "11) edge case test for 172.x range" do
|
104
|
+
let(:event) { LogStash::Event.new(:message => "172.10.0.0,172.16.0.0") }
|
105
|
+
it "should take the client ip from the right end of the list" do
|
106
|
+
expect(event.get("forwarded_client_ip")).to eq("172.10.0.0")
|
107
|
+
expect(event.get("forwarded_proxy_list")).to eq(["172.16.0.0"])
|
108
|
+
end # it
|
109
|
+
|
110
|
+
end # context
|
111
|
+
|
112
|
+
context "12) invalid ips in string" do
|
113
|
+
let(:event) { LogStash::Event.new(:message => "unknown, 207.248.75.2") }
|
114
|
+
it "should ignore the 'unknown' ip" do
|
115
|
+
expect(event.get("forwarded_client_ip")).to eq("207.248.75.2")
|
116
|
+
expect(event.get("forwarded_proxy_list")).to eq([])
|
117
|
+
end # it
|
118
|
+
end # context
|
119
|
+
|
120
|
+
context "13) invalid ips in string pt. 2" do
|
121
|
+
let(:event) { LogStash::Event.new(:message => "10.122.18.79, unknown, 200.152.43.203") }
|
122
|
+
it "should ignore the 'unknown' ip" do
|
123
|
+
expect(event.get("forwarded_client_ip")).to eq("200.152.43.203")
|
124
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.122.18.79"])
|
125
|
+
end # it
|
126
|
+
end # context
|
127
|
+
|
128
|
+
context "14) ipv4 and v6 mixed ips" do
|
129
|
+
let(:event) { LogStash::Event.new(:message => "2405:204:828e:fa5a::e64:38a5, 64.233.173.148") }
|
130
|
+
it "should take the client ip from the right end of the list" do
|
131
|
+
expect(event.get("forwarded_client_ip")).to eq("2405:204:828e:fa5a::e64:38a5")
|
132
|
+
expect(event.get("forwarded_proxy_list")).to eq(["64.233.173.148"])
|
133
|
+
end # it
|
134
|
+
end # context
|
135
|
+
|
136
|
+
context "15) ipv6 private ip only" do
|
137
|
+
let(:event) { LogStash::Event.new(:message => "fd8e:3ea6:dd4b:e20b:xxxx:xxxx:xxxx:xxxx") }
|
138
|
+
it "should have an empty client ip" do
|
139
|
+
expect(event.get("forwarded_client_ip")).to eq(nil)
|
140
|
+
expect(event.get("forwarded_proxy_list")).to eq(["fd8e:3ea6:dd4b:e20b:xxxx:xxxx:xxxx:xxxx"])
|
141
|
+
end # it
|
142
|
+
end # context
|
143
|
+
|
144
|
+
context "16) ipv6" do
|
145
|
+
let(:event) { LogStash::Event.new(:message => "fc8e:3ea6:dd4b:e20b:xxxx:xxxx:xxxx:xxxx,2405:204:828e:fa5a::e64:38a5") }
|
146
|
+
it "should be able to handle ipv6 addresses" do
|
147
|
+
expect(event.get("forwarded_client_ip")).to eq("2405:204:828e:fa5a::e64:38a5")
|
148
|
+
expect(event.get("forwarded_proxy_list")).to eq(["fc8e:3ea6:dd4b:e20b:xxxx:xxxx:xxxx:xxxx"])
|
149
|
+
end # it
|
150
|
+
end # context
|
151
|
+
|
152
|
+
context "17) override existing fields in event" do
|
153
|
+
let(:event) { LogStash::Event.new(:message => "2405:204:828e:fa5a::e64:38a5, 127.0.0.2", :forwarded_client_ip => "127.0.0.1") }
|
154
|
+
it "should ignore the old value for the forwarded_client_ip" do
|
155
|
+
expect(event.get("forwarded_client_ip")).to eq("2405:204:828e:fa5a::e64:38a5")
|
156
|
+
expect(event.get("forwarded_proxy_list")).to eq(["127.0.0.2"])
|
157
|
+
end # it
|
158
|
+
end # context
|
159
|
+
|
160
|
+
context "18) input field doesn't exist" do
|
161
|
+
let(:event) { LogStash::Event.new() }
|
162
|
+
it "should not raise an exception" do
|
163
|
+
expect {plugin.filter(event)}.not_to raise_error
|
164
|
+
end # it
|
165
|
+
end # context
|
166
|
+
|
167
|
+
context "19) unresolved host names" do
|
168
|
+
let(:event) { LogStash::Event.new(:message => "192.168.4.61,wmt00091.kan,wmt00091.kan, 64.134.227.116") }
|
169
|
+
it "should drop the unresolved hosts" do
|
170
|
+
expect(event.get("forwarded_client_ip")).to eq("64.134.227.116")
|
171
|
+
expect(event.get("forwarded_proxy_list")).to eq(["192.168.4.61","wmt00091.kan","wmt00091.kan"])
|
172
|
+
end # it
|
173
|
+
end # context
|
174
|
+
|
175
|
+
context "20) ip field is an array" do
|
176
|
+
let(:event) { LogStash::Event.new(:message => ["51.174.213.194","10.1.2.100","10.1.2.83"]) }
|
177
|
+
it "should take the first client ip" do
|
178
|
+
expect(event.get("forwarded_client_ip")).to eq("51.174.213.194")
|
179
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.100","10.1.2.83"])
|
180
|
+
end # it
|
181
|
+
end # context
|
182
|
+
|
183
|
+
context "21) ip array with port numbers" do
|
184
|
+
let(:event) { LogStash::Event.new(:message => ["51.174.213.198:8080","10.1.2.104","10.1.2.83:80"]) }
|
185
|
+
it "should remove the port numbers" do
|
186
|
+
expect(event.get("forwarded_client_ip")).to eq("51.174.213.198")
|
187
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.104","10.1.2.83"])
|
188
|
+
end # it
|
189
|
+
end # context
|
190
|
+
|
191
|
+
context "22) multiple client ips with port numbers" do
|
192
|
+
let(:event) { LogStash::Event.new(:message => "123.45.67.99:8080,61.160.232.227:123") }
|
193
|
+
it "should remove the port numbers" do
|
194
|
+
expect(event.get("forwarded_client_ip")).to eq("123.45.67.99")
|
195
|
+
expect(event.get("forwarded_proxy_list")).to eq(["61.160.232.227"])
|
196
|
+
end # it
|
197
|
+
end # context
|
198
|
+
|
199
|
+
context "23) proper x-forwarded-for with port numbers" do
|
200
|
+
let(:event) { LogStash::Event.new(:message => "84.30.67.208:123, 10.1.2.163") }
|
201
|
+
it "should remove the port number" do
|
202
|
+
expect(event.get("forwarded_client_ip")).to eq("84.30.67.208")
|
203
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.163"])
|
204
|
+
end # it
|
205
|
+
end # context
|
206
|
+
|
207
|
+
context "24) proper x-forwarded-for with port numbers" do
|
208
|
+
let(:event) { LogStash::Event.new(:message => "84.30.67.209, 10.1.2.164:8080") }
|
209
|
+
it "should remove the port number" do
|
210
|
+
expect(event.get("forwarded_client_ip")).to eq("84.30.67.209")
|
211
|
+
expect(event.get("forwarded_proxy_list")).to eq(["10.1.2.164"])
|
212
|
+
end # it
|
213
|
+
end # context
|
214
|
+
|
215
|
+
end
|
metadata
ADDED
@@ -0,0 +1,103 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: logstash-filter-forwarded
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 1.0.4
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Inga Feick
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2017-10-04 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
15
|
+
requirements:
|
16
|
+
- - '>='
|
17
|
+
- !ruby/object:Gem::Version
|
18
|
+
version: '1.60'
|
19
|
+
- - <=
|
20
|
+
- !ruby/object:Gem::Version
|
21
|
+
version: '2.99'
|
22
|
+
name: logstash-core-plugin-api
|
23
|
+
prerelease: false
|
24
|
+
type: :runtime
|
25
|
+
version_requirements: !ruby/object:Gem::Requirement
|
26
|
+
requirements:
|
27
|
+
- - '>='
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: '1.60'
|
30
|
+
- - <=
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '2.99'
|
33
|
+
- !ruby/object:Gem::Dependency
|
34
|
+
requirement: !ruby/object:Gem::Requirement
|
35
|
+
requirements:
|
36
|
+
- - '>='
|
37
|
+
- !ruby/object:Gem::Version
|
38
|
+
version: '0'
|
39
|
+
name: ipaddress
|
40
|
+
prerelease: false
|
41
|
+
type: :runtime
|
42
|
+
version_requirements: !ruby/object:Gem::Requirement
|
43
|
+
requirements:
|
44
|
+
- - '>='
|
45
|
+
- !ruby/object:Gem::Version
|
46
|
+
version: '0'
|
47
|
+
- !ruby/object:Gem::Dependency
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
49
|
+
requirements:
|
50
|
+
- - '>='
|
51
|
+
- !ruby/object:Gem::Version
|
52
|
+
version: '0'
|
53
|
+
name: logstash-devutils
|
54
|
+
prerelease: false
|
55
|
+
type: :development
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
57
|
+
requirements:
|
58
|
+
- - '>='
|
59
|
+
- !ruby/object:Gem::Version
|
60
|
+
version: '0'
|
61
|
+
description: This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program
|
62
|
+
email: inga.feick@trivago.com
|
63
|
+
executables: []
|
64
|
+
extensions: []
|
65
|
+
extra_rdoc_files: []
|
66
|
+
files:
|
67
|
+
- CHANGELOG.md
|
68
|
+
- CONTRIBUTORS
|
69
|
+
- Gemfile
|
70
|
+
- LICENSE
|
71
|
+
- NOTICE.TXT
|
72
|
+
- README.md
|
73
|
+
- lib/logstash/filters/forwarded.rb
|
74
|
+
- logstash-filter-forwarded.gemspec
|
75
|
+
- spec/filters/forwarded_spec.rb
|
76
|
+
homepage:
|
77
|
+
licenses:
|
78
|
+
- Apache License (2.0)
|
79
|
+
metadata:
|
80
|
+
logstash_plugin: 'true'
|
81
|
+
logstash_group: filter
|
82
|
+
post_install_message:
|
83
|
+
rdoc_options: []
|
84
|
+
require_paths:
|
85
|
+
- lib
|
86
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
87
|
+
requirements:
|
88
|
+
- - '>='
|
89
|
+
- !ruby/object:Gem::Version
|
90
|
+
version: '0'
|
91
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
92
|
+
requirements:
|
93
|
+
- - '>='
|
94
|
+
- !ruby/object:Gem::Version
|
95
|
+
version: '0'
|
96
|
+
requirements: []
|
97
|
+
rubyforge_project:
|
98
|
+
rubygems_version: 2.4.5
|
99
|
+
signing_key:
|
100
|
+
specification_version: 4
|
101
|
+
summary: $summary
|
102
|
+
test_files:
|
103
|
+
- spec/filters/forwarded_spec.rb
|