logstash-filter-real_ip 0.1.0 → 0.1.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (7) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +5 -1
  3. data/README.md +94 -64
  4. data/lib/logstash/filters/real_ip.rb +38 -8
  5. data/logstash-filter-real_ip.gemspec +2 -2
  6. data/spec/filters/real_ip_spec.rb +12 -0
  7. metadata +3 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 88e7483965ddd5c17fa615a7d996119471c37c3fe962c960e4100fec3072af04
4
- data.tar.gz: d0eea1f5ecdf8d83188df244d9c01ca2f521affee30c56c3e6d694085ac1683c
3
+ metadata.gz: 03a6d392653c84b6e9979c4242b33addc2bef2b9edbc06a02ab12bf4b0e817eb
4
+ data.tar.gz: b86af8b9cb8beeb0018548058bda54666cc96a8c9a79dfa808e537ea24910ef9
5
5
  SHA512:
6
- metadata.gz: b7fbcf884b25d0aaf0801ab27ec7288d5864c007947f2834a6c3026178cc72a333c2d0da4fc49b5fed8d43e21b189aed2c9ab5be3175e33cfdc4eba02393bdb3
7
- data.tar.gz: f2d85853339251efb7eb5a52eb1fde05d0d3ee0c5164e2384f29e5484e970903fbfc400de50ea53d5de140cda5c15baad2136639f57fdd159b8c6709a323748c
6
+ metadata.gz: 80d539a35dc6a9ae7957e2051e75a3440aed0a885ea52bc565d757d5166a6623f17e54a933d4c1440922d89a44d1b0d19ab98b4e9613d298a8a71ba979e0ba9c
7
+ data.tar.gz: c8a5626f432719c803a2abd375f31ae4d23390702e124cd9ed90233dd9bbd32c9f2ab77569d5ba7155c18e16999123783c120360d6fd88686a33eed975677d73
data/CHANGELOG.md CHANGED
@@ -1,2 +1,6 @@
1
+
2
+ ## 0.1.1
3
+ - fix: Fix edge case where x-forwarded-for is empty or only contains trusted IPs
4
+
1
5
  ## 0.1.0
2
- - Plugin created with the logstash plugin generator
6
+ - Plugin created with the logstash plugin generator
data/README.md CHANGED
@@ -1,86 +1,116 @@
1
- # Logstash Plugin
1
+ Evaluate an HTTP request's client address like Apache httpd's mod_remoteip or
2
+ Nginx's realip module.
2
3
 
3
- This is a plugin for [Logstash](https://github.com/elastic/logstash).
4
-
5
- It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
6
-
7
- ## Documentation
8
-
9
- Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one [central location](http://www.elastic.co/guide/en/logstash/current/).
10
-
11
- - For formatting code or config example, you can use the asciidoc `[source,ruby]` directive
12
- - For more asciidoc formatting tips, see the excellent reference here https://github.com/elastic/docs#asciidoc-guide
13
-
14
- ## Need Help?
15
-
16
- Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
17
-
18
- ## Developing
4
+ For an event like this...
5
+ ```ruby
6
+ {
7
+ "remote_addr" => "10.1.1.1"
8
+ "x_fwd_for" => ["1.2.3.4", "10.2.2.2"]
9
+ }
10
+ ```
19
11
 
20
- ### 1. Plugin Developement and Testing
12
+ ...an example configuration looks like so:
13
+ ```ruby
14
+ filter {
15
+ real_ip {
16
+ remote_address_field => "remote_addr"
17
+ x_forwarded_for_field => "x_fwd_for"
18
+ trusted_networks => [
19
+ "10.0.0.0/8",
20
+ "192.168.0.0/16"
21
+ ]
22
+ }
23
+ }
24
+ ```
25
+ This will evaluate the real client IP, writing it to a new field "realip".
26
+ For above example event that would be "1.2.3.4"
21
27
 
22
- #### Code
23
- - To get started, you'll need JRuby with the Bundler gem installed.
28
+ Often web servers don't provide the value of the X-Forwarded-For header as
29
+ an array. For ease of use the real_ip plugin provides capabilities to parse
30
+ such a comma-separated string. To enable this feature, use the
31
+ `x_forwarded_for_is_string` option.
24
32
 
25
- - Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
33
+ For an event like this...
34
+ ```ruby
35
+ {
36
+ "remote_addr" => "10.1.1.1"
37
+ "x_fwd_for" => "1.2.3.4, 10.2.2.2"
38
+ }
39
+ ```
26
40
 
27
- - Install dependencies
28
- ```sh
29
- bundle install
41
+ ...an example configuration looks like so:
42
+ ```ruby
43
+ filter {
44
+ real_ip {
45
+ remote_address_field => "remote_addr"
46
+ x_forwarded_for_field => "x_fwd_for"
47
+ x_forwarded_for_is_string => true
48
+ trusted_networks => [
49
+ "10.0.0.0/8",
50
+ "192.168.0.0/16"
51
+ ]
52
+ }
53
+ }
30
54
  ```
31
55
 
32
- #### Test
56
+ In case the plugin fails to evaluate the real client IP, it will add a tag to
57
+ the event, by default `_real_ip_lookup_failure`.
58
+ The plugin will fail if one of below it true:
59
+ * The `remote_address` field is absent.
60
+ * The `remote_address` field doesn't contain an IP address.
61
+ * The filter is configured using `x_forwarded_for_is_string = true`, but the
62
+ `x_forwarded_for` field isn't a string.
63
+ * The `x_forwarded_for` field contains anything other that IP addresses.
33
64
 
34
- - Update your dependencies
65
+ #### Evaluation behavior
66
+ The plugin checks whether the `remote_address_field` is trusted, if not, it
67
+ will be written to `target_field`, and evaluation ends.
35
68
 
36
- ```sh
37
- bundle install
38
- ```
69
+ Otherwise each IP in the `x_forwarded_for_field` is checked, from right to
70
+ left until an untrusted IP is encountered, which will be written to
71
+ `target_field` and evaluation ends at that point.
39
72
 
40
- - Run tests
73
+ In case `remote_address_field` and all IPs in `x_forwarded_for_field` are
74
+ trusted, the left-most IP of the `x_forwarded_for_field` is written to
75
+ `target_field`.
41
76
 
42
- ```sh
43
- bundle exec rspec
44
- ```
77
+ #### Configuration
78
+ ##### remote_address_field
79
+ * type: string
80
+ * default: `""`
45
81
 
46
- ### 2. Running your unpublished Plugin in Logstash
82
+ Name of the field that contains the layer 3 remote IP address
47
83
 
48
- #### 2.1 Run in a local Logstash clone
84
+ ##### x_forwarded_for_field
85
+ * type: string
86
+ * default: `""`
49
87
 
50
- - Edit Logstash `Gemfile` and add the local plugin path, for example:
51
- ```ruby
52
- gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
53
- ```
54
- - Install plugin
55
- ```sh
56
- bin/logstash-plugin install --no-verify
57
- ```
58
- - Run Logstash with your plugin
59
- ```sh
60
- bin/logstash -e 'filter {awesome {}}'
61
- ```
62
- At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
88
+ Name of the field that contains the X-Forwarded-For header value
63
89
 
64
- #### 2.2 Run in an installed Logstash
90
+ ##### x_forwarded_for_is_string
91
+ * type: boolean
92
+ * default: `false`
65
93
 
66
- You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
94
+ Specifies whether the `x_forwarded_for_field` contains a comman-separated
95
+ string instead of an array.
67
96
 
68
- - Build your plugin gem
69
- ```sh
70
- gem build logstash-filter-awesome.gemspec
71
- ```
72
- - Install the plugin from the Logstash home
73
- ```sh
74
- bin/logstash-plugin install /your/local/plugin/logstash-filter-awesome.gem
75
- ```
76
- - Start Logstash and proceed to test the plugin
97
+ ##### trusted_networks
98
+ * type: array
99
+ * default: `[]`
77
100
 
78
- ## Contributing
101
+ A list of trusted networks addresses. Be sure that you only specify
102
+ addresses that you trust will correctly manipulate the X-Forwarded-For
103
+ header.
79
104
 
80
- All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
105
+ ##### target_field
106
+ * type: string
107
+ * default: `"real_ip"`
81
108
 
82
- Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
109
+ Name of the field that this plugin will write the evaluated real client IP
110
+ address to.
83
111
 
84
- It is more important to the community that you are able to contribute.
112
+ ##### tags_on_failure
113
+ * type: array
114
+ * default: `["_real_ip_lookup_failure"]`
85
115
 
86
- For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
116
+ In case of error during evaluation, these tags will be set.
data/lib/logstash/filters/real_ip.rb CHANGED
@@ -6,13 +6,15 @@ require "ipaddr"
6
6
  # Evaluate an HTTP request's client address like Apache httpd's mod_remoteip or
7
7
  # Nginx's realip module.
8
8
  #
9
- # For an event like this:
9
+ #
10
+ # For an event like this...
10
11
  # [source,ruby]
11
12
  # {
12
- # "remote_addr" => "10.1.1.1"
13
- # "x_fwd_for" => ["1.2.3.4", "10.2.2.2"]
13
+ # "remote_addr" => "10.1.1.1"
14
+ # "x_fwd_for" => ["1.2.3.4", "10.2.2.2"]
14
15
  # }
15
- # An example configuration looks like so:
16
+ #
17
+ # ...an example configuration looks like so:
16
18
  # [source,ruby]
17
19
  # filter {
18
20
  # real_ip {
@@ -31,12 +33,15 @@ require "ipaddr"
31
33
  # an array. For ease of use the real_ip plugin provides capabilities to parse
32
34
  # such a comma-separated string. To enable this feature, use the
33
35
  # `x_forwarded_for_is_string` option.
36
+ #
37
+ # For an event like this...
34
38
  # [source,ruby]
35
39
  # {
36
- # "remote_addr" => "10.1.1.1"
37
- # "x_fwd_for" => "1.2.3.4, 10.2.2.2"
40
+ # "remote_addr" => "10.1.1.1"
41
+ # "x_fwd_for" => "1.2.3.4, 10.2.2.2"
38
42
  # }
39
- # An example configuration looks like so:
43
+ #
44
+ # ...an example configuration looks like so:
40
45
  # [source,ruby]
41
46
  # filter {
42
47
  # real_ip {
@@ -59,6 +64,18 @@ require "ipaddr"
59
64
  # `x_forwarded_for` field isn't a string.
60
65
  # * The `x_forwarded_for` field contains anything other that IP addresses.
61
66
  #
67
+ # ==== Evaluation behavior ====
68
+ # The plugin checks whether the `remote_address_field` is trusted, if not, it
69
+ # will be written to `target_field`, and evaluation ends.
70
+ #
71
+ # Otherwise each IP in the `x_forwarded_for_field` is checked, from right to
72
+ # left until an untrusted IP is encountered, which will be written to
73
+ # `target_field` and evaluation ends at that point.
74
+ #
75
+ # In case `remote_address_field` and all IPs in `x_forwarded_for_field` are
76
+ # trusted, the left-most IP of the `x_forwarded_for_field` is written to
77
+ # `target_field`.
78
+ #
62
79
  class LogStash::Filters::RealIp < LogStash::Filters::Base
63
80
  config_name "real_ip"
64
81
 
@@ -174,7 +191,14 @@ class LogStash::Filters::RealIp < LogStash::Filters::Base
174
191
  end
175
192
  end
176
193
 
177
- # In case X-Forwarded-For header is set, but empty
194
+ # in case x_forwarded_for is empty
195
+ if fwdfor.length == 0
196
+ event.set(@target_field, remote_addr)
197
+ filter_matched(event)
198
+ return
199
+ end
200
+
201
+ # In case X-Forwarded-For header is set, but zero-length string
178
202
  if fwdfor.length == 1 and fwdfor[0].length < 1
179
203
  @logger.debug? and @logger.debug("xfwdfor header was present but empty, evaluate to remote_addr", :address => remote_addr)
180
204
  event.set(@target_field, remote_addr)
@@ -200,5 +224,11 @@ class LogStash::Filters::RealIp < LogStash::Filters::Base
200
224
  end
201
225
  end
202
226
 
227
+ # in case remote_addr and all x_forwarded_for IPs are trusted, use the
228
+ # left-most IP from x_forwarded_for
229
+ event.set(@target_field, fwdfor[0])
230
+ filter_matched(event)
231
+ return
232
+
203
233
  end # def filter
204
234
  end # class LogStash::Filters::RealIp
data/logstash-filter-real_ip.gemspec CHANGED
@@ -1,9 +1,9 @@
1
1
  Gem::Specification.new do |s|
2
2
  s.name = 'logstash-filter-real_ip'
3
- s.version = '0.1.0'
3
+ s.version = '0.1.1'
4
4
  s.licenses = ['Apache-2.0']
5
5
  s.summary = 'Evaluates real client IP from X-Forwarded-For header'
6
- s.description = ''
6
+ s.description = 'See https://github.com/fholzer/logstash-filter-real_ip/blob/master/README.md for details.'
7
7
  s.homepage = 'https://github.com/fholzer/logstash-filter-real_ip'
8
8
  s.authors = ['Ferdinand Holzer']
9
9
  s.email = 'ferdinand.holzer@gmail.com'
data/spec/filters/real_ip_spec.rb CHANGED
@@ -49,6 +49,18 @@ describe LogStash::Filters::RealIp do
49
49
  expect(subject).to include("real_ip")
50
50
  expect(subject.get('real_ip')).to eq('1.2.3.4')
51
51
  end
52
+
53
+ sample("remote_addr" => "10.2.3.4", "xfwdfor" => ["192.168.5.6", "192.168.3.4"]) do
54
+ expect(subject).not_to include("tags")
55
+ expect(subject).to include("real_ip")
56
+ expect(subject.get('real_ip')).to eq('192.168.5.6')
57
+ end
58
+
59
+ sample("remote_addr" => "10.2.3.4", "xfwdfor" => []) do
60
+ expect(subject).not_to include("tags")
61
+ expect(subject).to include("real_ip")
62
+ expect(subject.get('real_ip')).to eq('10.2.3.4')
63
+ end
52
64
  end
53
65
 
54
66
  describe "Evaluates real IP correctly with flat string xfwfor" do
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-filter-real_ip
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ferdinand Holzer
@@ -38,7 +38,8 @@ dependencies:
38
38
  - - ">="
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0'
41
- description: ''
41
+ description: See https://github.com/fholzer/logstash-filter-real_ip/blob/master/README.md
42
+ for details.
42
43
  email: ferdinand.holzer@gmail.com
43
44
  executables: []
44
45
  extensions: []