logstash-filter-device_detector 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: b4654c9ab2e2eb6e23f7166c5ad18eeb9af885cc33a9556a2541bb82578e9ba6
4
+ data.tar.gz: 26199a2e87a7557932ecc837d13a6c063ca770cd4b2e6e0ea28be38701383ecd
5
+ SHA512:
6
+ metadata.gz: f2301841fd07ee8c4b49d229d00ce49fdd66e505d6f2a0d9ed1125ae1410d50b885478da5e5a528b08fed655a1bdee0d8eb9dad984f7a73a202d4c292f8002c4
7
+ data.tar.gz: 53c9c88a4adfee4cd0cb24fa8a3a82b37977074bfb84dd186ba74aa40f0b7dc3e9d6715bc8ea555ef5d8b7ffb6b0b2ed191d9df675d59b2268087afd6f4da6dd
data/CHANGELOG.md ADDED
@@ -0,0 +1,2 @@
1
+ ## 0.1.0
2
+ - Plugin created with the logstash plugin generator
data/CONTRIBUTORS ADDED
@@ -0,0 +1,10 @@
1
+ The following is a list of people who have contributed ideas, code, bug
2
+ reports, or in general have helped logstash along its way.
3
+
4
+ Contributors:
5
+ * Dukang - dukanghub@gmail.com
6
+
7
+ Note: If you've sent us patches, bug reports, or otherwise contributed to
8
+ Logstash, and you aren't on the list above and want to be, please let us know
9
+ and we'll make sure you're here. Contributions from folks like you are what make
10
+ open source awesome.
data/DEVELOPER.md ADDED
@@ -0,0 +1,2 @@
1
+ # logstash-filter-device_detector
2
+ Example filter plugin. This should help bootstrap your effort to write your own filter plugin!
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source 'https://rubygems.org'
2
+ gemspec
3
+
data/LICENSE ADDED
@@ -0,0 +1,11 @@
1
+ Licensed under the Apache License, Version 2.0 (the "License");
2
+ you may not use this file except in compliance with the License.
3
+ You may obtain a copy of the License at
4
+
5
+ http://www.apache.org/licenses/LICENSE-2.0
6
+
7
+ Unless required by applicable law or agreed to in writing, software
8
+ distributed under the License is distributed on an "AS IS" BASIS,
9
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
10
+ See the License for the specific language governing permissions and
11
+ limitations under the License.
data/README.md ADDED
@@ -0,0 +1,91 @@
1
+ # logstash-filter-device_detector
2
+
3
+ ## 说明
4
+
5
+ 这是一个基于ruby语言的`logstash filter`插件,用来解析`http_user_agent`,调用了外部库。
6
+
7
+ ## 开发环境测试
8
+
9
+ ### 1. 插件开发环境和测试
10
+
11
+ #### Code
12
+ - 首先,您需要安装了Bundler gem的JRuby。
13
+ - 执行下面的命令安装依赖
14
+ ```sh
15
+ bundle install
16
+ ```
17
+
18
+ #### Test
19
+
20
+ - Update your dependencies
21
+
22
+ ```sh
23
+ bundle install
24
+ ```
25
+
26
+ - Run tests
27
+
28
+ ```sh
29
+ bundle exec rspec
30
+ ```
31
+
32
+ 这里在windows可能执行失败,暂时没研究什么原因。
33
+
34
+ ### 2. 在logstash中运行未发布的插件
35
+
36
+ #### 2.1 直接在logstash使用克隆下来的插件代码
37
+
38
+ 进入logstash安装目录,一般是`/usr/share/logstash`
39
+
40
+ - 编辑 `Gemfile` ,添加本地插件路径,如下所示:
41
+ ```ruby
42
+ gem "logstash-filter-device_detector", :path => "/path/to/logstash-filter-device-detector"
43
+ ```
44
+ - 安装插件
45
+ ```sh
46
+ bin/logstash-plugin install --no-verify
47
+ ```
48
+ - 运行logstash测试此插件
49
+ ```sh
50
+ cd /etc/logstash/conf.d
51
+ vim test.conf
52
+ # 添加如下内容
53
+ input {
54
+ stdin {}
55
+ }
56
+ filter {
57
+ device_detector {
58
+ source => "message"
59
+ }
60
+ }
61
+ output {
62
+ stdout { codec => rubydebug }
63
+ }
64
+ # 运行logstash
65
+ /usr/share/logstash/bin/logstash -f test.conf
66
+ ```
67
+ 然后在屏幕输入useragent就可以看到效果了。
68
+
69
+ #### 2.2 在logstash使用gem安装插件
70
+
71
+ You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
72
+
73
+ - 构建gem
74
+ ```sh
75
+ gem build logstash-filter-device-detector.gemspec
76
+ ```
77
+ - 进入logstash安装目录,安装gem插件
78
+ ```sh
79
+ bin/logstash-plugin install /your/local/plugin/logstash-filter-awesome.gem
80
+ ```
81
+ - Start Logstash and proceed to test the plugin
82
+
83
+ ## Contributing
84
+
85
+ All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
86
+
87
+ Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
88
+
89
+ It is more important to the community that you are able to contribute.
90
+
91
+ For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
@@ -0,0 +1,118 @@
1
+ # encoding: utf-8
2
+ require "logstash/filters/base"
3
+ require "device_detector"
4
+ require 'json'
5
+
6
+ class LogStash::Filters::DeviceDetector < LogStash::Filters::Base
7
+
8
+ config_name "device_detector"
9
+
10
+ config :source, :validate => :string, :default => "http_user_agent"
11
+
12
+ config :target, :validate => :string, :default => "ua"
13
+
14
+ config :tag_on_unknown, :validate => :array, :default => [ ]
15
+
16
+ config :tag_on_bot, :validate => :array, :default => [ ]
17
+
18
+ public
19
+ def register
20
+
21
+ end
22
+
23
+ public
24
+ def filter(event)
25
+
26
+ # Receive source
27
+ useragent = event.get(@source)
28
+ return if useragent.nil? || !useragent.is_a?(String) || useragent.strip == ""
29
+
30
+ # Parse user-agent via device-detector
31
+ begin
32
+ data = DeviceDetector.new(useragent)
33
+ rescue StandardError => e
34
+ @logger.error("Uknown error while parsing device data", :exception => e, :field => @source, :event => event)
35
+ return
36
+ end
37
+ return unless data
38
+
39
+ # Remove original source (if its also the target)
40
+ event.remove(@source) if @target == @source
41
+
42
+ # Set all fields
43
+ begin
44
+ unless data.known?
45
+ @tag_on_unknown.each { |tag| event.tag(tag) }
46
+ end
47
+ if data.bot?
48
+ @tag_on_bot.each { |tag| event.tag(tag) }
49
+ end
50
+ is_mobile = false
51
+ is_bot = false
52
+ spider = ""
53
+ mozilla = false
54
+ model = true
55
+ platform = "Other"
56
+ os = "Other"
57
+ engine = ""
58
+ engine_version = ""
59
+ browser = ""
60
+ browser_version = ""
61
+
62
+ if data.device_type =~ /phone/
63
+ is_mobile = true
64
+ end
65
+
66
+ if data.bot?
67
+ is_bot = true
68
+ spider = data.bot_name
69
+ end
70
+ if data.device_type
71
+ platform = data.device_type
72
+ end
73
+ if data.os_full_version
74
+ os = "#{data.os_name} #{data.os_full_version}"
75
+ end
76
+ if data.name
77
+ browser = data.name
78
+ if data.name =~ /irefox/
79
+ mozilla = true
80
+ end
81
+ end
82
+ if data.full_version
83
+ browser_version = data.full_version
84
+ end
85
+ # 构造输出哈希表
86
+ output = {
87
+ "isMobile" => is_mobile,
88
+ "isBot" => is_bot,
89
+ "mozilla" => mozilla,
90
+ "model" => model,
91
+ "platform" => platform,
92
+ "os" => os,
93
+ "engine" => engine,
94
+ "engineVersion" => engine_version,
95
+ "browser" => browser,
96
+ "browserVersion" => browser_version
97
+ }
98
+ event.set("httpUserAgentJson", output.to_json)
99
+ event.set("os", os)
100
+ event.set("browser", "#{browser},#{browser_version}")
101
+ event.set("spider", spider)
102
+ event.set("#{@target}[browser][name]", data.name) if data.name
103
+ event.set("#{@target}[browser][version]", data.full_version) if data.full_version
104
+ event.set("#{@target}[os][name]", data.os_name) if data.os_name
105
+ event.set("#{@target}[os][version]", data.os_full_version) if data.os_full_version
106
+ event.set("#{@target}[device][name]", data.device_name) if data.device_name
107
+ event.set("#{@target}[device][brand]", data.device_brand) if data.device_brand
108
+ event.set("#{@target}[device][type]", data.device_type) if data.device_type
109
+ event.set("#{@target}[bot][name]", data.bot_name) if data.bot_name
110
+ event.set("#{@target}[bot][name]", data.bot_name) if data.bot_name
111
+ rescue StandardError => e
112
+ @logger.error("Uknown error while setting device data", :exception => e, :field => @source, :event => event)
113
+ return
114
+ end
115
+
116
+ filter_matched(event)
117
+ end
118
+ end
@@ -0,0 +1,24 @@
1
+ Gem::Specification.new do |s|
2
+ s.name = 'logstash-filter-device_detector'
3
+ s.version = '0.1.1'
4
+ s.licenses = ['Apache-2.0']
5
+ s.summary = '使用device_detector解析useragent的logstash-filter插件.'
6
+ s.description = 'Detects a vast amount of different devices automaticly based on regex rules.'
7
+ s.homepage = 'https://github.com/Dukanghub/logstash-filter-device_detector'
8
+ s.authors = ['Dukang']
9
+ s.email = 'dukanghub@gmail.com'
10
+ s.require_paths = ['lib']
11
+
12
+ # Files
13
+ s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
14
+ # Tests
15
+ s.test_files = s.files.grep(%r{^(test|spec|features)/})
16
+
17
+ # Special flag to let us know this is actually a logstash plugin
18
+ s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
19
+
20
+ # Gem dependencies
21
+ s.add_runtime_dependency "logstash-core-plugin-api", "~> 2.0"
22
+ s.add_runtime_dependency "device_detector", "~> 1.0"
23
+ s.add_development_dependency "logstash-devutils", "~> 0"
24
+ end
@@ -0,0 +1,22 @@
1
+ # encoding: utf-8
2
+ require_relative '../spec_helper'
3
+ require "logstash/filters/device_detector"
4
+
5
+ describe LogStash::Filters::DeviceDetector do
6
+ describe "Set to Hello World" do
7
+ let(:config) do <<-CONFIG
8
+ filter {
9
+ device_detector {
10
+ source => "useragent"
11
+ target => "device_detector"
12
+ }
13
+ }
14
+ CONFIG
15
+ end
16
+
17
+ sample("message" => "some text") do
18
+ expect(subject).to include("message")
19
+ expect(subject.get('message')).to eq('Hello World')
20
+ end
21
+ end
22
+ end
@@ -0,0 +1,2 @@
1
+ # encoding: utf-8
2
+ require "logstash/devutils/rspec/spec_helper"
metadata ADDED
@@ -0,0 +1,99 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: logstash-filter-device_detector
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.1
5
+ platform: ruby
6
+ authors:
7
+ - Dukang
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2023-04-15 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - "~>"
17
+ - !ruby/object:Gem::Version
18
+ version: '2.0'
19
+ name: logstash-core-plugin-api
20
+ prerelease: false
21
+ type: :runtime
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '2.0'
27
+ - !ruby/object:Gem::Dependency
28
+ requirement: !ruby/object:Gem::Requirement
29
+ requirements:
30
+ - - "~>"
31
+ - !ruby/object:Gem::Version
32
+ version: '1.0'
33
+ name: device_detector
34
+ prerelease: false
35
+ type: :runtime
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '1.0'
41
+ - !ruby/object:Gem::Dependency
42
+ requirement: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - "~>"
45
+ - !ruby/object:Gem::Version
46
+ version: '0'
47
+ name: logstash-devutils
48
+ prerelease: false
49
+ type: :development
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ description: Detects a vast amount of different devices automaticly based on regex
56
+ rules.
57
+ email: dukanghub@gmail.com
58
+ executables: []
59
+ extensions: []
60
+ extra_rdoc_files: []
61
+ files:
62
+ - CHANGELOG.md
63
+ - CONTRIBUTORS
64
+ - DEVELOPER.md
65
+ - Gemfile
66
+ - LICENSE
67
+ - README.md
68
+ - lib/logstash/filters/device_detector.rb
69
+ - logstash-filter-device-detector.gemspec
70
+ - spec/filters/device_detector_spec.rb
71
+ - spec/spec_helper.rb
72
+ homepage: https://github.com/Dukanghub/logstash-filter-device_detector
73
+ licenses:
74
+ - Apache-2.0
75
+ metadata:
76
+ logstash_plugin: 'true'
77
+ logstash_group: filter
78
+ post_install_message:
79
+ rdoc_options: []
80
+ require_paths:
81
+ - lib
82
+ required_ruby_version: !ruby/object:Gem::Requirement
83
+ requirements:
84
+ - - ">="
85
+ - !ruby/object:Gem::Version
86
+ version: '0'
87
+ required_rubygems_version: !ruby/object:Gem::Requirement
88
+ requirements:
89
+ - - ">="
90
+ - !ruby/object:Gem::Version
91
+ version: '0'
92
+ requirements: []
93
+ rubygems_version: 3.3.25
94
+ signing_key:
95
+ specification_version: 4
96
+ summary: 使用device_detector解析useragent的logstash-filter插件.
97
+ test_files:
98
+ - spec/filters/device_detector_spec.rb
99
+ - spec/spec_helper.rb