logstash-filter-device_detector 0.1.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: b4654c9ab2e2eb6e23f7166c5ad18eeb9af885cc33a9556a2541bb82578e9ba6
4
+ data.tar.gz: 26199a2e87a7557932ecc837d13a6c063ca770cd4b2e6e0ea28be38701383ecd
5
+ SHA512:
6
+ metadata.gz: f2301841fd07ee8c4b49d229d00ce49fdd66e505d6f2a0d9ed1125ae1410d50b885478da5e5a528b08fed655a1bdee0d8eb9dad984f7a73a202d4c292f8002c4
7
+ data.tar.gz: 53c9c88a4adfee4cd0cb24fa8a3a82b37977074bfb84dd186ba74aa40f0b7dc3e9d6715bc8ea555ef5d8b7ffb6b0b2ed191d9df675d59b2268087afd6f4da6dd
data/CHANGELOG.md ADDED
@@ -0,0 +1,2 @@
1
+ ## 0.1.0
2
+ - Plugin created with the logstash plugin generator
data/CONTRIBUTORS ADDED
@@ -0,0 +1,10 @@
1
+ The following is a list of people who have contributed ideas, code, bug
2
+ reports, or in general have helped logstash along its way.
3
+
4
+ Contributors:
5
+ * Dukang - dukanghub@gmail.com
6
+
7
+ Note: If you've sent us patches, bug reports, or otherwise contributed to
8
+ Logstash, and you aren't on the list above and want to be, please let us know
9
+ and we'll make sure you're here. Contributions from folks like you are what make
10
+ open source awesome.
data/DEVELOPER.md ADDED
@@ -0,0 +1,2 @@
1
+ # logstash-filter-device_detector
2
+ Example filter plugin. This should help bootstrap your effort to write your own filter plugin!
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source 'https://rubygems.org'
2
+ gemspec
3
+
data/LICENSE ADDED
@@ -0,0 +1,11 @@
1
+ Licensed under the Apache License, Version 2.0 (the "License");
2
+ you may not use this file except in compliance with the License.
3
+ You may obtain a copy of the License at
4
+
5
+ http://www.apache.org/licenses/LICENSE-2.0
6
+
7
+ Unless required by applicable law or agreed to in writing, software
8
+ distributed under the License is distributed on an "AS IS" BASIS,
9
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
10
+ See the License for the specific language governing permissions and
11
+ limitations under the License.
data/README.md ADDED
@@ -0,0 +1,91 @@
1
+ # logstash-filter-device_detector
2
+
3
+ ## 说明
4
+
5
+ 这是一个基于ruby语言的`logstash filter`插件,用来解析`http_user_agent`,调用了外部库。
6
+
7
+ ## 开发环境测试
8
+
9
+ ### 1. 插件开发环境和测试
10
+
11
+ #### Code
12
+ - 首先,您需要安装了Bundler gem的JRuby。
13
+ - 执行下面的命令安装依赖
14
+ ```sh
15
+ bundle install
16
+ ```
17
+
18
+ #### Test
19
+
20
+ - Update your dependencies
21
+
22
+ ```sh
23
+ bundle install
24
+ ```
25
+
26
+ - Run tests
27
+
28
+ ```sh
29
+ bundle exec rspec
30
+ ```
31
+
32
+ 这里在windows可能执行失败,暂时没研究什么原因。
33
+
34
+ ### 2. 在logstash中运行未发布的插件
35
+
36
+ #### 2.1 直接在logstash使用克隆下来的插件代码
37
+
38
+ 进入logstash安装目录,一般是`/usr/share/logstash`
39
+
40
+ - 编辑 `Gemfile` ,添加本地插件路径,如下所示:
41
+ ```ruby
42
+ gem "logstash-filter-device_detector", :path => "/path/to/logstash-filter-device-detector"
43
+ ```
44
+ - 安装插件
45
+ ```sh
46
+ bin/logstash-plugin install --no-verify
47
+ ```
48
+ - 运行logstash测试此插件
49
+ ```sh
50
+ cd /etc/logstash/conf.d
51
+ vim test.conf
52
+ # 添加如下内容
53
+ input {
54
+ stdin {}
55
+ }
56
+ filter {
57
+ device_detector {
58
+ source => "message"
59
+ }
60
+ }
61
+ output {
62
+ stdout { codec => rubydebug }
63
+ }
64
+ # 运行logstash
65
+ /usr/share/logstash/bin/logstash -f test.conf
66
+ ```
67
+ 然后在屏幕输入useragent就可以看到效果了。
68
+
69
+ #### 2.2 在logstash使用gem安装插件
70
+
71
+ You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
72
+
73
+ - 构建gem
74
+ ```sh
75
+ gem build logstash-filter-device-detector.gemspec
76
+ ```
77
+ - 进入logstash安装目录,安装gem插件
78
+ ```sh
79
+ bin/logstash-plugin install /your/local/plugin/logstash-filter-awesome.gem
80
+ ```
81
+ - Start Logstash and proceed to test the plugin
82
+
83
+ ## Contributing
84
+
85
+ All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
86
+
87
+ Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
88
+
89
+ It is more important to the community that you are able to contribute.
90
+
91
+ For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
@@ -0,0 +1,118 @@
1
+ # encoding: utf-8
2
+ require "logstash/filters/base"
3
+ require "device_detector"
4
+ require 'json'
5
+
6
+ class LogStash::Filters::DeviceDetector < LogStash::Filters::Base
7
+
8
+ config_name "device_detector"
9
+
10
+ config :source, :validate => :string, :default => "http_user_agent"
11
+
12
+ config :target, :validate => :string, :default => "ua"
13
+
14
+ config :tag_on_unknown, :validate => :array, :default => [ ]
15
+
16
+ config :tag_on_bot, :validate => :array, :default => [ ]
17
+
18
+ public
19
+ def register
20
+
21
+ end
22
+
23
+ public
24
+ def filter(event)
25
+
26
+ # Receive source
27
+ useragent = event.get(@source)
28
+ return if useragent.nil? || !useragent.is_a?(String) || useragent.strip == ""
29
+
30
+ # Parse user-agent via device-detector
31
+ begin
32
+ data = DeviceDetector.new(useragent)
33
+ rescue StandardError => e
34
+ @logger.error("Uknown error while parsing device data", :exception => e, :field => @source, :event => event)
35
+ return
36
+ end
37
+ return unless data
38
+
39
+ # Remove original source (if its also the target)
40
+ event.remove(@source) if @target == @source
41
+
42
+ # Set all fields
43
+ begin
44
+ unless data.known?
45
+ @tag_on_unknown.each { |tag| event.tag(tag) }
46
+ end
47
+ if data.bot?
48
+ @tag_on_bot.each { |tag| event.tag(tag) }
49
+ end
50
+ is_mobile = false
51
+ is_bot = false
52
+ spider = ""
53
+ mozilla = false
54
+ model = true
55
+ platform = "Other"
56
+ os = "Other"
57
+ engine = ""
58
+ engine_version = ""
59
+ browser = ""
60
+ browser_version = ""
61
+
62
+ if data.device_type =~ /phone/
63
+ is_mobile = true
64
+ end
65
+
66
+ if data.bot?
67
+ is_bot = true
68
+ spider = data.bot_name
69
+ end
70
+ if data.device_type
71
+ platform = data.device_type
72
+ end
73
+ if data.os_full_version
74
+ os = "#{data.os_name} #{data.os_full_version}"
75
+ end
76
+ if data.name
77
+ browser = data.name
78
+ if data.name =~ /irefox/
79
+ mozilla = true
80
+ end
81
+ end
82
+ if data.full_version
83
+ browser_version = data.full_version
84
+ end
85
+ # 构造输出哈希表
86
+ output = {
87
+ "isMobile" => is_mobile,
88
+ "isBot" => is_bot,
89
+ "mozilla" => mozilla,
90
+ "model" => model,
91
+ "platform" => platform,
92
+ "os" => os,
93
+ "engine" => engine,
94
+ "engineVersion" => engine_version,
95
+ "browser" => browser,
96
+ "browserVersion" => browser_version
97
+ }
98
+ event.set("httpUserAgentJson", output.to_json)
99
+ event.set("os", os)
100
+ event.set("browser", "#{browser},#{browser_version}")
101
+ event.set("spider", spider)
102
+ event.set("#{@target}[browser][name]", data.name) if data.name
103
+ event.set("#{@target}[browser][version]", data.full_version) if data.full_version
104
+ event.set("#{@target}[os][name]", data.os_name) if data.os_name
105
+ event.set("#{@target}[os][version]", data.os_full_version) if data.os_full_version
106
+ event.set("#{@target}[device][name]", data.device_name) if data.device_name
107
+ event.set("#{@target}[device][brand]", data.device_brand) if data.device_brand
108
+ event.set("#{@target}[device][type]", data.device_type) if data.device_type
109
+ event.set("#{@target}[bot][name]", data.bot_name) if data.bot_name
110
+ event.set("#{@target}[bot][name]", data.bot_name) if data.bot_name
111
+ rescue StandardError => e
112
+ @logger.error("Uknown error while setting device data", :exception => e, :field => @source, :event => event)
113
+ return
114
+ end
115
+
116
+ filter_matched(event)
117
+ end
118
+ end
@@ -0,0 +1,24 @@
1
+ Gem::Specification.new do |s|
2
+ s.name = 'logstash-filter-device_detector'
3
+ s.version = '0.1.1'
4
+ s.licenses = ['Apache-2.0']
5
+ s.summary = '使用device_detector解析useragent的logstash-filter插件.'
6
+ s.description = 'Detects a vast amount of different devices automaticly based on regex rules.'
7
+ s.homepage = 'https://github.com/Dukanghub/logstash-filter-device_detector'
8
+ s.authors = ['Dukang']
9
+ s.email = 'dukanghub@gmail.com'
10
+ s.require_paths = ['lib']
11
+
12
+ # Files
13
+ s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
14
+ # Tests
15
+ s.test_files = s.files.grep(%r{^(test|spec|features)/})
16
+
17
+ # Special flag to let us know this is actually a logstash plugin
18
+ s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
19
+
20
+ # Gem dependencies
21
+ s.add_runtime_dependency "logstash-core-plugin-api", "~> 2.0"
22
+ s.add_runtime_dependency "device_detector", "~> 1.0"
23
+ s.add_development_dependency "logstash-devutils", "~> 0"
24
+ end
@@ -0,0 +1,22 @@
1
+ # encoding: utf-8
2
+ require_relative '../spec_helper'
3
+ require "logstash/filters/device_detector"
4
+
5
+ describe LogStash::Filters::DeviceDetector do
6
+ describe "Set to Hello World" do
7
+ let(:config) do <<-CONFIG
8
+ filter {
9
+ device_detector {
10
+ source => "useragent"
11
+ target => "device_detector"
12
+ }
13
+ }
14
+ CONFIG
15
+ end
16
+
17
+ sample("message" => "some text") do
18
+ expect(subject).to include("message")
19
+ expect(subject.get('message')).to eq('Hello World')
20
+ end
21
+ end
22
+ end
@@ -0,0 +1,2 @@
1
+ # encoding: utf-8
2
+ require "logstash/devutils/rspec/spec_helper"
metadata ADDED
@@ -0,0 +1,99 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: logstash-filter-device_detector
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.1
5
+ platform: ruby
6
+ authors:
7
+ - Dukang
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2023-04-15 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - "~>"
17
+ - !ruby/object:Gem::Version
18
+ version: '2.0'
19
+ name: logstash-core-plugin-api
20
+ prerelease: false
21
+ type: :runtime
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '2.0'
27
+ - !ruby/object:Gem::Dependency
28
+ requirement: !ruby/object:Gem::Requirement
29
+ requirements:
30
+ - - "~>"
31
+ - !ruby/object:Gem::Version
32
+ version: '1.0'
33
+ name: device_detector
34
+ prerelease: false
35
+ type: :runtime
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '1.0'
41
+ - !ruby/object:Gem::Dependency
42
+ requirement: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - "~>"
45
+ - !ruby/object:Gem::Version
46
+ version: '0'
47
+ name: logstash-devutils
48
+ prerelease: false
49
+ type: :development
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ description: Detects a vast amount of different devices automaticly based on regex
56
+ rules.
57
+ email: dukanghub@gmail.com
58
+ executables: []
59
+ extensions: []
60
+ extra_rdoc_files: []
61
+ files:
62
+ - CHANGELOG.md
63
+ - CONTRIBUTORS
64
+ - DEVELOPER.md
65
+ - Gemfile
66
+ - LICENSE
67
+ - README.md
68
+ - lib/logstash/filters/device_detector.rb
69
+ - logstash-filter-device-detector.gemspec
70
+ - spec/filters/device_detector_spec.rb
71
+ - spec/spec_helper.rb
72
+ homepage: https://github.com/Dukanghub/logstash-filter-device_detector
73
+ licenses:
74
+ - Apache-2.0
75
+ metadata:
76
+ logstash_plugin: 'true'
77
+ logstash_group: filter
78
+ post_install_message:
79
+ rdoc_options: []
80
+ require_paths:
81
+ - lib
82
+ required_ruby_version: !ruby/object:Gem::Requirement
83
+ requirements:
84
+ - - ">="
85
+ - !ruby/object:Gem::Version
86
+ version: '0'
87
+ required_rubygems_version: !ruby/object:Gem::Requirement
88
+ requirements:
89
+ - - ">="
90
+ - !ruby/object:Gem::Version
91
+ version: '0'
92
+ requirements: []
93
+ rubygems_version: 3.3.25
94
+ signing_key:
95
+ specification_version: 4
96
+ summary: 使用device_detector解析useragent的logstash-filter插件.
97
+ test_files:
98
+ - spec/filters/device_detector_spec.rb
99
+ - spec/spec_helper.rb