logstash-filter-useragent 3.1.0-java

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 2882cc7695727f80d8010a52e600c24e88c44ad4
4
+ data.tar.gz: 8a174676a2edf46ce48adfc9540c28403aa2c918
5
+ SHA512:
6
+ metadata.gz: 0d2bdcc43b9fb5433a6b41c94544c3f4fe8979e79772e2973c5225d7fcf1edda14692d3cec151ea7db4647ec6c59c226f4b0c09a3ec7dde9af3a3dd7d2dfa1e1
7
+ data.tar.gz: 340fb536b59287c08e69365a4c759d76666c2c254d837df867305861d31f8c6626b7858252de844bbd236a4cb5389aa859b01f0c660fa587bd3d347c654ef7d2
data/CHANGELOG.md ADDED
@@ -0,0 +1,36 @@
1
+ ## 3.1.0
2
+ - Parser performance increase of a factor of about 2.5 by moving core parser logic from Ruby to Java
3
+ - Lower memory footprint of Java implementation allowed for increasing the default parser cache size from 1k to 100k
4
+ without an increase in memory consumption
5
+
6
+ ## 3.0.3
7
+ - Move one log message from info to debug to avoid noise
8
+
9
+ ## 3.0.2
10
+ - Relax constraint on logstash-core-plugin-api to >= 1.60 <= 2.99
11
+
12
+ ## 3.0.1
13
+ - Republish all the gems under jruby.
14
+ ## 3.0.0
15
+ - Update the plugin to the version 2.0 of the plugin api, this change is required for Logstash 5.0 compatibility. See https://github.com/elastic/logstash/issues/5141
16
+ # 2.0.8
17
+ - Revert addition of Mutex. This plugin now depends on jruby having threadsafe regexps
18
+ # 2.0.7
19
+ - Add Mutex to help on non-threadsafe JRuby versions
20
+ # 2.0.6
21
+ - Depend on logstash-core-plugin-api instead of logstash-core, removing the need to mass update plugins on major releases of logstash
22
+ # 2.0.5
23
+ - New dependency requirements for logstash-core for the 5.0 release
24
+ ## 2.0.4
25
+ - Refactored field references, fixed specs and some cleanups
26
+
27
+ ## 2.0.0
28
+ - Plugins were updated to follow the new shutdown semantic, this mainly allows Logstash to instruct input plugins to terminate gracefully,
29
+ instead of using Thread.raise on the plugins' threads. Ref: https://github.com/elastic/logstash/pull/3895
30
+ - Dependency on logstash-core update to 2.0
31
+
32
+ ## 2.0.1
33
+ - Add ability to replace source with target
34
+
35
+ ## 1.1.0
36
+ - Add LRU cache
data/CONTRIBUTORS ADDED
@@ -0,0 +1,18 @@
1
+ The following is a list of people who have contributed ideas, code, bug
2
+ reports, or in general have helped logstash along its way.
3
+
4
+ Contributors:
5
+ * Alex Tambellini (atambo)
6
+ * Colin Surprenant (colinsurprenant)
7
+ * Dan Everton (deverton)
8
+ * Jay Luker (lbjay)
9
+ * Jordan Sissel (jordansissel)
10
+ * Kubes (pkubat)
11
+ * Pier-Hugues Pellerin (ph)
12
+ * Richard Pijnenburg (electrical)
13
+ * Suyog Rao (suyograo)
14
+
15
+ Note: If you've sent us patches, bug reports, or otherwise contributed to
16
+ Logstash, and you aren't on the list above and want to be, please let us know
17
+ and we'll make sure you're here. Contributions from folks like you are what make
18
+ open source awesome.
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in logstash-mass_effect.gemspec
4
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,13 @@
1
+ Copyright (c) 2012–2016 Elasticsearch <http://www.elastic.co>
2
+
3
+ Licensed under the Apache License, Version 2.0 (the "License");
4
+ you may not use this file except in compliance with the License.
5
+ You may obtain a copy of the License at
6
+
7
+ http://www.apache.org/licenses/LICENSE-2.0
8
+
9
+ Unless required by applicable law or agreed to in writing, software
10
+ distributed under the License is distributed on an "AS IS" BASIS,
11
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
+ See the License for the specific language governing permissions and
13
+ limitations under the License.
data/NOTICE.TXT ADDED
@@ -0,0 +1,5 @@
1
+ Elasticsearch
2
+ Copyright 2012-2017 Elasticsearch
3
+
4
+ This product includes software developed by The Apache Software
5
+ Foundation (http://www.apache.org/).
data/README.md ADDED
@@ -0,0 +1,99 @@
1
+ # Logstash Plugin
2
+
3
+ [![Travis Build Status](https://travis-ci.org/logstash-plugins/logstash-filter-useragent.svg)](https://travis-ci.org/logstash-plugins/logstash-filter-useragent)
4
+
5
+ This is a plugin for [Logstash](https://github.com/elastic/logstash).
6
+
7
+ It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
8
+
9
+ ## Documentation
10
+
11
+ Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one [central location](http://www.elastic.co/guide/en/logstash/current/).
12
+
13
+ - For formatting code or config example, you can use the asciidoc `[source,ruby]` directive
14
+ - For more asciidoc formatting tips, see the excellent reference here https://github.com/elastic/docs#asciidoc-guide
15
+
16
+ ## Need Help?
17
+
18
+ Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
19
+
20
+ ## Developing
21
+
22
+ ### 1. Plugin Developement and Testing
23
+
24
+ #### Code
25
+ - To get started, you'll need JRuby with the Bundler gem installed.
26
+
27
+ - Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
28
+
29
+ - Install dependencies
30
+ ```sh
31
+ bundle install
32
+ rake install_jars
33
+ ```
34
+
35
+ #### Test
36
+
37
+ - Update your dependencies
38
+
39
+ ```sh
40
+ bundle install
41
+ ```
42
+
43
+ - Run tests
44
+
45
+ ```sh
46
+ bundle exec rake test
47
+ ```
48
+
49
+ ### 2. Running your unpublished Plugin in Logstash
50
+
51
+ #### 2.1 Run in a local Logstash clone
52
+
53
+ - Edit Logstash `Gemfile` and add the local plugin path, for example:
54
+ ```ruby
55
+ gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
56
+ ```
57
+ - Install plugin
58
+ ```sh
59
+ # Logstash 2.3 and higher
60
+ bin/logstash-plugin install --no-verify
61
+
62
+ # Prior to Logstash 2.3
63
+ bin/plugin install --no-verify
64
+
65
+ ```
66
+ - Run Logstash with your plugin
67
+ ```sh
68
+ bin/logstash -e 'filter {awesome {}}'
69
+ ```
70
+ At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
71
+
72
+ #### 2.2 Run in an installed Logstash
73
+
74
+ You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
75
+
76
+ - Build your plugin gem
77
+ ```sh
78
+ gem build logstash-filter-awesome.gemspec
79
+ ```
80
+ - Install the plugin from the Logstash home
81
+ ```sh
82
+ # Logstash 2.3 and higher
83
+ bin/logstash-plugin install --no-verify
84
+
85
+ # Prior to Logstash 2.3
86
+ bin/plugin install --no-verify
87
+
88
+ ```
89
+ - Start Logstash and proceed to test the plugin
90
+
91
+ ## Contributing
92
+
93
+ All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
94
+
95
+ Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
96
+
97
+ It is more important to the community that you are able to contribute.
98
+
99
+ For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
@@ -0,0 +1,117 @@
1
+ :plugin: useragent
2
+ :type: filter
3
+
4
+ ///////////////////////////////////////////
5
+ START - GENERATED VARIABLES, DO NOT EDIT!
6
+ ///////////////////////////////////////////
7
+ :version: %VERSION%
8
+ :release_date: %RELEASE_DATE%
9
+ :changelog_url: %CHANGELOG_URL%
10
+ :include_path: ../../../logstash/docs/include
11
+ ///////////////////////////////////////////
12
+ END - GENERATED VARIABLES, DO NOT EDIT!
13
+ ///////////////////////////////////////////
14
+
15
+ [id="plugins-{type}-{plugin}"]
16
+
17
+ === Useragent
18
+
19
+ include::{include_path}/plugin_header.asciidoc[]
20
+
21
+ ==== Description
22
+
23
+ Parse user agent strings into structured data based on BrowserScope data
24
+
25
+ UserAgent filter, adds information about user agent like family, operating
26
+ system, version, and device
27
+
28
+ Logstash releases ship with the regexes.yaml database made available from
29
+ ua-parser with an Apache 2.0 license. For more details on ua-parser, see
30
+ <https://github.com/tobie/ua-parser/>.
31
+
32
+ [id="plugins-{type}s-{plugin}-options"]
33
+ ==== Useragent Filter Configuration Options
34
+
35
+ This plugin supports the following configuration options plus the <<plugins-{type}s-common-options>> described later.
36
+
37
+ [cols="<,<,<",options="header",]
38
+ |=======================================================================
39
+ |Setting |Input type|Required
40
+ | <<plugins-{type}s-{plugin}-lru_cache_size>> |<<number,number>>|No
41
+ | <<plugins-{type}s-{plugin}-prefix>> |<<string,string>>|No
42
+ | <<plugins-{type}s-{plugin}-regexes>> |<<string,string>>|No
43
+ | <<plugins-{type}s-{plugin}-source>> |<<string,string>>|Yes
44
+ | <<plugins-{type}s-{plugin}-target>> |<<string,string>>|No
45
+ |=======================================================================
46
+
47
+ Also see <<plugins-{type}s-common-options>> for a list of options supported by all
48
+ filter plugins.
49
+
50
+ &nbsp;
51
+
52
+ [id="plugins-{type}s-{plugin}-lru_cache_size"]
53
+ ===== `lru_cache_size`
54
+
55
+ * Value type is <<number,number>>
56
+ * Default value is `1000`
57
+
58
+ UA parsing is surprisingly expensive. This filter uses an LRU cache to take advantage of the fact that
59
+ user agents are often found adjacent to one another in log files and rarely have a random distribution.
60
+ The higher you set this the more likely an item is to be in the cache and the faster this filter will run.
61
+ However, if you set this too high you can use more memory than desired.
62
+
63
+ Experiment with different values for this option to find the best performance for your dataset.
64
+
65
+ This MUST be set to a value > 0. There is really no reason to not want this behavior, the overhead is minimal
66
+ and the speed gains are large.
67
+
68
+ It is important to note that this config value is global. That is to say all instances of the user agent filter
69
+ share the same cache. The last declared cache size will 'win'. The reason for this is that there would be no benefit
70
+ to having multiple caches for different instances at different points in the pipeline, that would just increase the
71
+ number of cache misses and waste memory.
72
+
73
+ [id="plugins-{type}s-{plugin}-prefix"]
74
+ ===== `prefix`
75
+
76
+ * Value type is <<string,string>>
77
+ * Default value is `""`
78
+
79
+ A string to prepend to all of the extracted keys
80
+
81
+ [id="plugins-{type}s-{plugin}-regexes"]
82
+ ===== `regexes`
83
+
84
+ * Value type is <<string,string>>
85
+ * There is no default value for this setting.
86
+
87
+ `regexes.yaml` file to use
88
+
89
+ If not specified, this will default to the `regexes.yaml` that ships
90
+ with logstash.
91
+
92
+ You can find the latest version of this here:
93
+ <https://github.com/ua-parser/uap-core/blob/master/regexes.yaml>
94
+
95
+ [id="plugins-{type}s-{plugin}-source"]
96
+ ===== `source`
97
+
98
+ * This is a required setting.
99
+ * Value type is <<string,string>>
100
+ * There is no default value for this setting.
101
+
102
+ The field containing the user agent string. If this field is an
103
+ array, only the first value will be used.
104
+
105
+ [id="plugins-{type}s-{plugin}-target"]
106
+ ===== `target`
107
+
108
+ * Value type is <<string,string>>
109
+ * There is no default value for this setting.
110
+
111
+ The name of the field to assign user agent data into.
112
+
113
+ If not specified user agent data will be stored in the root of the event.
114
+
115
+
116
+
117
+ include::{include_path}/{type}.asciidoc[]
@@ -0,0 +1,4 @@
1
+ # encoding: utf-8
2
+
3
+ require 'jar_dependencies'
4
+ require_jar('org.logstash.filters', 'logstash-filter-useragent', '3.1.0')
@@ -0,0 +1,149 @@
1
+ # encoding: utf-8
2
+ require "java"
3
+ require "logstash-filter-useragent_jars"
4
+ require "logstash/filters/base"
5
+ require "logstash/namespace"
6
+ require "tempfile"
7
+ require "thread"
8
+
9
+ # Parse user agent strings into structured data based on BrowserScope data
10
+ #
11
+ # UserAgent filter, adds information about user agent like family, operating
12
+ # system, version, and device
13
+ #
14
+ # Logstash releases ship with the regexes.yaml database made available from
15
+ # ua-parser with an Apache 2.0 license. For more details on ua-parser, see
16
+ # <https://github.com/tobie/ua-parser/>.
17
+ class LogStash::Filters::UserAgent < LogStash::Filters::Base
18
+
19
+ config_name "useragent"
20
+
21
+ # The field containing the user agent string. If this field is an
22
+ # array, only the first value will be used.
23
+ config :source, :validate => :string, :required => true
24
+
25
+ # The name of the field to assign user agent data into.
26
+ #
27
+ # If not specified user agent data will be stored in the root of the event.
28
+ config :target, :validate => :string
29
+
30
+ # `regexes.yaml` file to use
31
+ #
32
+ # If not specified, this will default to the `regexes.yaml` that ships
33
+ # with logstash.
34
+ #
35
+ # You can find the latest version of this here:
36
+ # <https://github.com/ua-parser/uap-core/blob/master/regexes.yaml>
37
+ config :regexes, :validate => :string
38
+
39
+ # A string to prepend to all of the extracted keys
40
+ config :prefix, :validate => :string, :default => ''
41
+
42
+ # UA parsing is surprisingly expensive. This filter uses an LRU cache to take advantage of the fact that
43
+ # user agents are often found adjacent to one another in log files and rarely have a random distribution.
44
+ # The higher you set this the more likely an item is to be in the cache and the faster this filter will run.
45
+ # However, if you set this too high you can use more memory than desired.
46
+ #
47
+ # Experiment with different values for this option to find the best performance for your dataset.
48
+ #
49
+ # This MUST be set to a value > 0. There is really no reason to not want this behavior, the overhead is minimal
50
+ # and the speed gains are large.
51
+ #
52
+ # It is important to note that this config value is global. That is to say all instances of the user agent filter
53
+ # share the same cache. The last declared cache size will 'win'. The reason for this is that there would be no benefit
54
+ # to having multiple caches for different instances at different points in the pipeline, that would just increase the
55
+ # number of cache misses and waste memory.
56
+ config :lru_cache_size, :validate => :number, :default => 100_000
57
+
58
+ def register
59
+
60
+ if @regexes.nil?
61
+ @parser = org.logstash.uaparser.CachingParser.new(lru_cache_size)
62
+ else
63
+ @logger.debug("Using user agent regexes", :regexes => @regexes)
64
+ yaml = ::File.open(regexes, "rb")
65
+ @parser = org.logstash.uaparser.CachingParser.new(yaml.read, lru_cache_size)
66
+ yaml.close
67
+ end
68
+
69
+ # make @target in the format [field name] if defined, i.e. surrounded by brakets
70
+ normalized_target = (@target && @target !~ /^\[[^\[\]]+\]$/) ? "[#{@target}]" : ""
71
+
72
+ # predefine prefixed field names
73
+ @prefixed_name = "#{normalized_target}[#{@prefix}name]"
74
+ @prefixed_os = "#{normalized_target}[#{@prefix}os]"
75
+ @prefixed_os_name = "#{normalized_target}[#{@prefix}os_name]"
76
+ @prefixed_os_major = "#{normalized_target}[#{@prefix}os_major]"
77
+ @prefixed_os_minor = "#{normalized_target}[#{@prefix}os_minor]"
78
+ @prefixed_device = "#{normalized_target}[#{@prefix}device]"
79
+ @prefixed_major = "#{normalized_target}[#{@prefix}major]"
80
+ @prefixed_minor = "#{normalized_target}[#{@prefix}minor]"
81
+ @prefixed_patch = "#{normalized_target}[#{@prefix}patch]"
82
+ @prefixed_build = "#{normalized_target}[#{@prefix}build]"
83
+ end
84
+
85
+ def filter(event)
86
+ useragent = event.get(@source)
87
+ useragent = useragent.first if useragent.is_a?(Array)
88
+
89
+ return if useragent.nil? || useragent.empty?
90
+
91
+ begin
92
+ ua_data = lookup_useragent(useragent)
93
+ rescue StandardError => e
94
+ @logger.error("Uknown error while parsing user agent data", :exception => e, :field => @source, :event => event)
95
+ return
96
+ end
97
+
98
+ return unless ua_data
99
+
100
+ event.remove(@source) if @target == @source
101
+ set_fields(event, ua_data)
102
+
103
+ filter_matched(event)
104
+ end
105
+
106
+ # should be private but need to stay public for specs
107
+ # TODO: (colin) the related specs should be refactored to not rely on private methods.
108
+ def lookup_useragent(useragent)
109
+ return unless useragent
110
+
111
+ # the UserAgentParser::Parser class is not thread safe, indications are that it is probably
112
+ # caused by the underlying JRuby regex code that is not thread safe.
113
+ # see https://github.com/logstash-plugins/logstash-filter-useragent/issues/25
114
+ @parser.parse(useragent)
115
+ end
116
+
117
+ private
118
+
119
+ def set_fields(event, ua_data)
120
+ # UserAgentParser outputs as US-ASCII.
121
+
122
+ event.set(@prefixed_name, ua_data.userAgent.family.dup.force_encoding(Encoding::UTF_8))
123
+
124
+ #OSX, Android and maybe iOS parse correctly, ua-agent parsing for Windows does not provide this level of detail
125
+
126
+ # Calls in here use #dup because there's potential for later filters to modify these values
127
+ # and corrupt the cache. See uap source here for details https://github.com/ua-parser/uap-ruby/tree/master/lib/user_agent_parser
128
+ if (os = ua_data.os)
129
+ # The OS is a rich object
130
+ event.set(@prefixed_os, ua_data.os.family.dup.force_encoding(Encoding::UTF_8))
131
+ event.set(@prefixed_os_name, os.family.dup.force_encoding(Encoding::UTF_8)) if os.family
132
+
133
+ # These are all strings
134
+ if os.minor && os.major
135
+ event.set(@prefixed_os_major, os.major.dup.force_encoding(Encoding::UTF_8)) if os.major
136
+ event.set(@prefixed_os_minor, os.minor.dup.force_encoding(Encoding::UTF_8)) if os.minor
137
+ end
138
+ end
139
+
140
+ event.set(@prefixed_device, ua_data.device.to_s.dup.force_encoding(Encoding::UTF_8)) if ua_data.device
141
+
142
+ if (ua_version = ua_data.userAgent)
143
+ event.set(@prefixed_major, ua_version.major.dup.force_encoding(Encoding::UTF_8)) if ua_version.major
144
+ event.set(@prefixed_minor, ua_version.minor.dup.force_encoding(Encoding::UTF_8)) if ua_version.minor
145
+ event.set(@prefixed_patch, ua_version.patch.dup.force_encoding(Encoding::UTF_8)) if ua_version.patch
146
+ event.set(@prefixed_build, ua_version.patchMinor.dup.force_encoding(Encoding::UTF_8)) if ua_version.patchMinor
147
+ end
148
+ end
149
+ end
@@ -0,0 +1,27 @@
1
+ Gem::Specification.new do |s|
2
+
3
+ s.name = 'logstash-filter-useragent'
4
+ s.version = '3.1.0'
5
+ s.licenses = ['Apache License (2.0)']
6
+ s.summary = "Parse user agent strings into structured data based on BrowserScope data"
7
+ s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"
8
+ s.authors = ["Elastic"]
9
+ s.email = 'info@elastic.co'
10
+ s.homepage = "http://www.elastic.co/guide/en/logstash/current/index.html"
11
+ s.platform = "java"
12
+ s.require_paths = ["lib", "vendor/jar-dependencies"]
13
+
14
+ # Files
15
+ s.files = Dir["lib/**/*","spec/**/*","*.gemspec","*.md","CONTRIBUTORS","Gemfile","LICENSE","NOTICE.TXT","vendor/jar-dependencies/**/*.jar","vendor/jar-dependencies/**/*.rb", "VERSION", "docs/**/*"]
16
+
17
+ # Tests
18
+ s.test_files = s.files.grep(%r{^(test|spec|features)/})
19
+
20
+ # Special flag to let us know this is actually a logstash plugin
21
+ s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
22
+
23
+ # Gem dependencies
24
+ s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
25
+ s.add_development_dependency 'logstash-devutils'
26
+ end
27
+
@@ -0,0 +1,129 @@
1
+ # encoding: utf-8
2
+
3
+ require "logstash/devutils/rspec/spec_helper"
4
+ require "logstash/filters/useragent"
5
+
6
+ describe LogStash::Filters::UserAgent do
7
+
8
+ describe "defaults" do
9
+ config <<-CONFIG
10
+ filter {
11
+ useragent {
12
+ source => "message"
13
+ target => "ua"
14
+ }
15
+ }
16
+ CONFIG
17
+
18
+ sample "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.63 Safari/537.31" do
19
+ insist { subject }.include?("ua")
20
+ insist { subject.get("[ua][name]") } == "Chrome"
21
+ insist { subject.get("[ua][os]") } == "Linux"
22
+ insist { subject.get("[ua][major]") } == "26"
23
+ insist { subject.get("[ua][minor]") } == "0"
24
+ end
25
+ end
26
+
27
+ describe "Without target field" do
28
+ config <<-CONFIG
29
+ filter {
30
+ useragent {
31
+ source => "message"
32
+ }
33
+ }
34
+ CONFIG
35
+
36
+ sample "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.63 Safari/537.31" do
37
+ insist { subject.get("name") } == "Chrome"
38
+ insist { subject.get("os") } == "Linux"
39
+ insist { subject.get("major") } == "26"
40
+ insist { subject.get("minor") } == "0"
41
+ end
42
+ end
43
+
44
+ describe "Without user agent" do
45
+ config <<-CONFIG
46
+ filter {
47
+ useragent {
48
+ source => "message"
49
+ target => "ua"
50
+ }
51
+ }
52
+ CONFIG
53
+
54
+ sample "foo" => "bar" do
55
+ reject { subject }.include?("ua")
56
+ end
57
+
58
+ sample "" do
59
+ reject { subject }.include?("ua")
60
+ end
61
+ end
62
+
63
+ describe "LRU object identity" do
64
+ let(:ua_string) { "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36" }
65
+ let(:uafilter) { LogStash::Filters::UserAgent.new("source" => "foo") }
66
+ let(:ua_data) { uafilter.lookup_useragent(ua_string) }
67
+
68
+ subject(:target) { LogStash::Event.new("foo" => ua_string) }
69
+
70
+ before do
71
+ uafilter.register
72
+
73
+ # Stub this out because this UA doesn't have this field
74
+ allow(ua_data.userAgent).to receive(:patchMinor).and_return("foo")
75
+
76
+ # expect(event).receive(:lookup_useragent)
77
+ uafilter.filter(target)
78
+ end
79
+
80
+ {
81
+ "name" => lambda {|uad| uad.userAgent.family},
82
+ "os" => lambda {|uad| uad.os.family},
83
+ "os_name" => lambda {|uad| uad.os.family},
84
+ "os_major" => lambda {|uad| uad.os.major},
85
+ "os_minor" => lambda {|uad| uad.os.minor},
86
+ "device" => lambda {|uad| uad.device.to_s},
87
+ "major" => lambda {|uad| uad.userAgent.major},
88
+ "minor" => lambda {|uad| uad.userAgent.minor},
89
+ "patch" => lambda {|uad| uad.userAgent.patch},
90
+ "build" => lambda {|uad| uad.userAgent.patchMinor}
91
+ }.each do |field, uad_getter|
92
+ context "for the #{field} field" do
93
+ let(:value) {uad_getter.call(ua_data)}
94
+ let(:target_field) { target.get(field)}
95
+
96
+ it "should not have a nil value" do
97
+ expect(target_field).to be_truthy
98
+ end
99
+
100
+ it "should have equivalent values" do
101
+ expect(target_field).to eql(value)
102
+ end
103
+
104
+ it "should dup/clone the field to prevent cache corruption" do
105
+ expect(target_field.object_id).not_to eql(value.object_id)
106
+ end
107
+ end
108
+ end
109
+ end
110
+
111
+ describe "Replace source with target" do
112
+ config <<-CONFIG
113
+ filter {
114
+ useragent {
115
+ source => "message"
116
+ target => "message"
117
+ }
118
+ }
119
+ CONFIG
120
+
121
+ sample "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.63 Safari/537.31" do
122
+ insist { subject.to_hash }.include?("message")
123
+ insist { subject.get("[message][name]") } == "Chrome"
124
+ insist { subject.get("[message][os]") } == "Linux"
125
+ insist { subject.get("[message][major]") } == "26"
126
+ insist { subject.get("[message][minor]") } == "0"
127
+ end
128
+ end
129
+ end
metadata ADDED
@@ -0,0 +1,93 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: logstash-filter-useragent
3
+ version: !ruby/object:Gem::Version
4
+ version: 3.1.0
5
+ platform: java
6
+ authors:
7
+ - Elastic
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2017-05-10 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - ">="
17
+ - !ruby/object:Gem::Version
18
+ version: '1.60'
19
+ - - "<="
20
+ - !ruby/object:Gem::Version
21
+ version: '2.99'
22
+ name: logstash-core-plugin-api
23
+ prerelease: false
24
+ type: :runtime
25
+ version_requirements: !ruby/object:Gem::Requirement
26
+ requirements:
27
+ - - ">="
28
+ - !ruby/object:Gem::Version
29
+ version: '1.60'
30
+ - - "<="
31
+ - !ruby/object:Gem::Version
32
+ version: '2.99'
33
+ - !ruby/object:Gem::Dependency
34
+ requirement: !ruby/object:Gem::Requirement
35
+ requirements:
36
+ - - ">="
37
+ - !ruby/object:Gem::Version
38
+ version: '0'
39
+ name: logstash-devutils
40
+ prerelease: false
41
+ type: :development
42
+ version_requirements: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - ">="
45
+ - !ruby/object:Gem::Version
46
+ version: '0'
47
+ description: This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program
48
+ email: info@elastic.co
49
+ executables: []
50
+ extensions: []
51
+ extra_rdoc_files: []
52
+ files:
53
+ - CHANGELOG.md
54
+ - CONTRIBUTORS
55
+ - Gemfile
56
+ - LICENSE
57
+ - NOTICE.TXT
58
+ - README.md
59
+ - docs/index.asciidoc
60
+ - lib/logstash-filter-useragent_jars.rb
61
+ - lib/logstash/filters/useragent.rb
62
+ - logstash-filter-useragent.gemspec
63
+ - spec/filters/useragent_spec.rb
64
+ - vendor/jar-dependencies/org/logstash/filters/logstash-filter-useragent/3.1.0/logstash-filter-useragent-3.1.0.jar
65
+ homepage: http://www.elastic.co/guide/en/logstash/current/index.html
66
+ licenses:
67
+ - Apache License (2.0)
68
+ metadata:
69
+ logstash_plugin: 'true'
70
+ logstash_group: filter
71
+ post_install_message:
72
+ rdoc_options: []
73
+ require_paths:
74
+ - lib
75
+ - vendor/jar-dependencies
76
+ required_ruby_version: !ruby/object:Gem::Requirement
77
+ requirements:
78
+ - - ">="
79
+ - !ruby/object:Gem::Version
80
+ version: '0'
81
+ required_rubygems_version: !ruby/object:Gem::Requirement
82
+ requirements:
83
+ - - ">="
84
+ - !ruby/object:Gem::Version
85
+ version: '0'
86
+ requirements: []
87
+ rubyforge_project:
88
+ rubygems_version: 2.4.8
89
+ signing_key:
90
+ specification_version: 4
91
+ summary: Parse user agent strings into structured data based on BrowserScope data
92
+ test_files:
93
+ - spec/filters/useragent_spec.rb