fluent-plugin-windows-eventlog 0.1.0 → 0.4.0

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -1,66 +1,285 @@
1
- # fluent-plugin-windows-eventlog
2
-
3
- ## Component
4
-
5
- #### fluentd Input plugin for Windows Event Log
6
-
7
- [Fluentd](http://fluentd.org) plugin to read Windows Event Log.
8
- You must use fluentd 'Windows' brach to use me, and it doesn't work on Linux of course.
9
-
10
- ## Installation
11
- gem install fluent-plugin-windows-eventlog
12
-
13
- ## Configuration
14
- #### fluentd Input plugin for Windows Event Log
15
-
16
- <source>
17
- type windows_eventlog
18
- channels application,system
19
- pos_file c:\temp\mypos
20
- read_interval 2
21
- tag winevt.raw
22
- </source>
23
-
24
-
25
- #### parameters
26
-
27
- |name | description |
28
- |:----- |:----- |
29
- |channels | (option) 'applicaion' as default. one or combination of {application, system, setup, security}. If you want to read setup or security, administrator priv is required to launch fluentd. |
30
- |pos_file | (option, but higly recommended) a path of position file to save record numbers. |
31
- |read_interval | (option) a read interval in second. 2 seconds as default.|
32
- |from_encoding | (option) an input characters encoding. nil as default.|
33
- |encoding | (option) an output characters encoding. nil as default.|
34
-
35
-
36
- #### read keys
37
- This plugin reads follows from Windws Event Log. No customization is allowed currently.
38
-
39
- |key|
40
- |:----- |
41
- |record_number |
42
- |time_generated|
43
- |time_written |
44
- |event_id |
45
- |event_type |
46
- |event_category |
47
- |source_name |
48
- |computer_name |
49
- |user |
50
- |description |
51
-
52
-
53
-
54
- ## Etc.
55
- 'read_from_head' is not supporeted currently.You can read newer records after you start first.
56
- No customize to read information keys.
57
-
58
-
59
-
60
-
61
- ## Copyright
62
- ####Copyright
63
- Copyright(C) 2014- @okahashi117
64
- ####License
65
- Apache License, Version 2.0
66
-
1
+ # fluent-plugin-windows-eventlog
2
+
3
+ ## Component
4
+
5
+ ### fluentd Input plugin for the Windows Event Log
6
+
7
+ [Fluentd](https://www.fluentd.org/) plugin to read the Windows Event Log.
8
+
9
+ ## Installation
10
+ ridk exec gem install fluent-plugin-windows-eventlog
11
+
12
+ ## Configuration
13
+
14
+ ### in_windows_eventlog
15
+
16
+ Check [in_windows_eventlog2](https://github.com/fluent/fluent-plugin-windows-eventlog#in_windows_eventlog2) first. `in_windows_eventlog` will be replaced with `in_windows_eventlog2`.
17
+
18
+ fluentd Input plugin for the Windows Event Log using old Windows Event Logging API
19
+
20
+ <source>
21
+ @type windows_eventlog
22
+ @id windows_eventlog
23
+ channels application,system
24
+ read_interval 2
25
+ tag winevt.raw
26
+ <storage>
27
+ @type local # @type local is the default.
28
+ persistent true # default is true. Set to false to use in-memory storage.
29
+ path ./tmp/storage.json # This is required when persistent is true.
30
+ # Or, please consider using <system> section's `root_dir` parameter.
31
+ </storage>
32
+ </source>
33
+
34
+ #### parameters
35
+
36
+ |name | description |
37
+ |:----- |:----- |
38
+ |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
39
+ |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
40
+ |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
41
+ |`from_encoding` | (option) Input character encoding. `nil` as default.|
42
+ |`encoding` | (option) Output character encoding. `nil` as default.|
43
+ |`read_from_head` | (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
44
+ |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
45
+ |`parse_description`| (option) parse `description` field and set parsed result into the record. `parse` and `string_inserts` fields are removed|
46
+
47
+ ##### Available keys
48
+
49
+ This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
50
+
51
+ |key|
52
+ |:----- |
53
+ |`record_number` |
54
+ |`time_generated`|
55
+ |`time_written` |
56
+ |`event_id` |
57
+ |`event_type` |
58
+ |`event_category`|
59
+ |`source_name` |
60
+ |`computer_name` |
61
+ |`user` |
62
+ |`description` |
63
+ |`string_inserts`|
64
+
65
+ ##### `parse_description` details
66
+
67
+ Here is an example with `parse_description true`.
68
+
69
+ ```
70
+ {
71
+ "channel": "security",
72
+ "record_number": "91698",
73
+ "time_generated": "2017-08-29 20:12:29 +0000",
74
+ "time_written": "2017-08-29 20:12:29 +0000",
75
+ "event_id": "4798",
76
+ "event_type": "audit_success",
77
+ "event_category": "13824",
78
+ "source_name": "Microsoft-Windows-Security-Auditing",
79
+ "computer_name": "TEST",
80
+ "user": "",
81
+ "description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-XXX\r\n\tAccount Name:\t\tTEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x7dc\r\n\tProcess Name:\t\tC:\\Windows\\System32\\LogonUI.exe\r\n",
82
+ "string_inserts": [
83
+ "Administrator",
84
+ "TEST",
85
+ "S-XXX-YYY-ZZZ",
86
+ "S-XXX",
87
+ "TEST$",
88
+ "WORKGROUP",
89
+ "0x3e7",
90
+ "0x7dc",
91
+ "C:\\Windows\\System32\\LogonUI.exe"
92
+ ]
93
+ }
94
+ ```
95
+
96
+ This record is transformed to
97
+
98
+ ```
99
+ {
100
+ "channel": "security",
101
+ "record_number": "91698",
102
+ "time_generated": "2017-08-29 20:12:29 +0000",
103
+ "time_written": "2017-08-29 20:12:29 +0000",
104
+ "event_id": "4798",
105
+ "event_type": "audit_success",
106
+ "event_category": "13824",
107
+ "source_name": "Microsoft-Windows-Security-Auditing",
108
+ "computer_name": "TEST",
109
+ "user": "",
110
+ "description_title": "A user's local group membership was enumerated.",
111
+ "subject.security_id": "S-XXX",
112
+ "subject.account_name": "TEST$",
113
+ "subject.account_domain": "WORKGROUP",
114
+ "subject.logon_id": "0x3e7",
115
+ "user.security_id": "S-XXX-YYY-ZZZ",
116
+ "user.account_name": "Administrator",
117
+ "user.account_domain": "TEST",
118
+ "process_information.process_id": "0x7dc",
119
+ "process_information.process_name": "C:\\Windows\\System32\\LogonUI.exe\r\n"
120
+ }
121
+ ```
122
+
123
+ NOTE: This feature assumes `description` field has following formats:
124
+
125
+ - group delimiter: `\r\n\r\n`
126
+ - record delimiter: `\r\n\t`
127
+ - field delimiter: `\t\t`
128
+
129
+ If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
130
+
131
+ ### in_windows_eventlog2
132
+
133
+ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging API. This is successor to `in_windows_eventlog`. See also [this slide](https://www.slideshare.net/cosmo0920/fluentd-meetup-2019) for the details of `in_windows_eventlog2` plugin.
134
+
135
+ <source>
136
+ @type windows_eventlog2
137
+ @id windows_eventlog2
138
+ channels application,system
139
+ read_interval 2
140
+ tag winevt.raw
141
+ render_as_xml false # default is true.
142
+ rate_limit 200 # default is -1(Winevt::EventLog::Subscribe::RATE_INFINITE).
143
+ <storage>
144
+ @type local # @type local is the default.
145
+ persistent true # default is true. Set to false to use in-memory storage.
146
+ path ./tmp/storage.json # This is required when persistent is true.
147
+ # Or, please consider using <system> section's `root_dir` parameter.
148
+ </storage>
149
+ <parse>
150
+ @type winevt_xml # @type winevt_xml is the default. winevt_xml and none parsers are supported for now.
151
+ </parse>
152
+ </source>
153
+
154
+ **NOTE:** in_windows_eventlog2 always handles EventLog records as UTF-8 characters. Users don't have to specify encoding related parameters and they are not provided.
155
+
156
+ **NOTE:** When `Description` contains error message such as `The message resource is present but the message was not found in the message table.`, eventlog's resource file (.mui) related to error generating event is something wrong. This issue is also occurred in built-in Windows Event Viewer which is the part of Windows management tool.
157
+
158
+ **NOTE:** When `render_as_xml` as `false`, the dependent winevt_c gem renders Windows EventLog as Ruby Hash object directly. This reduces bottleneck to consume EventLog. Specifying `render_as_xml` as `false` should be faster consuming than `render_as_xml` as `true` case.
159
+
160
+ **NOTE:** If you encountered CPU spike due to massively huge EventLog channel, `rate_limit` parameter may help you. Currently, this paramter can handle the multiples of 10 or -1(`Winevt::EventLog::Subscribe::RATE_INFINITE`).
161
+
162
+ #### parameters
163
+
164
+ |name | description |
165
+ |:----- |:----- |
166
+ |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
167
+ |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
168
+ |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
169
+ |`from_encoding` | (option) Input character encoding. `nil` as default.|
170
+ |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
171
+ |`<parse>` | Setting for `parser` plugin for parsing raw XML EventLog records. |
172
+ |`parse_description`| (option) parse `description` field and set parsed result into the record. `Description` and `EventData` fields are removed|
173
+
174
+ ##### Available keys
175
+
176
+ This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
177
+
178
+ |key|
179
+ |:----- |
180
+ |`ProviderName` |
181
+ |`ProviderGuid` |
182
+ |`EventID` |
183
+ |`Qualifiers` |
184
+ |`Level` |
185
+ |`Task` |
186
+ |`Opcode` |
187
+ |`Keywords` |
188
+ |`TimeCreated` |
189
+ |`EventRecordId` |
190
+ |`ActivityID` |
191
+ |`RelatedActivityID`|
192
+ |`ProcessID` |
193
+ |`ThreadID` |
194
+ |`Channel` |
195
+ |`Computer` |
196
+ |`UserID` |
197
+ |`Version` |
198
+ |`Description` |
199
+ |`EventData` |
200
+
201
+ ##### `parse_description` details
202
+
203
+ Here is an example with `parse_description true`.
204
+
205
+ ```
206
+ {
207
+ "ProviderName": "Microsoft-Windows-Security-Auditing",
208
+ "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
209
+ "EventID": "4798",
210
+ "Qualifiers": "",
211
+ "Level": "0",
212
+ "Task": "13824",
213
+ "Opcode": "0",
214
+ "Keywords": "0x8020000000000000",
215
+ "TimeCreated": "2019-06-19T03:10:01.982940200Z",
216
+ "EventRecordID": "87028",
217
+ "ActivityID": "",
218
+ "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
219
+ "ThreadID": "16888",
220
+ "Channel": "Security",
221
+ "Computer": "DESKTOP-TEST",
222
+ "UserID": "",
223
+ "Version": "0",
224
+ "Description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-X-Y-Z\r\n\tAccount Name:\t\tDESKTOP-TEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ0\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tDESKTOP-TEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0xbac\r\n\tProcess Name:\t\tC:\\Windows\\System32\\svchost.exe\r\n",
225
+ "EventData": [
226
+ "Administrator",
227
+ "DESKTOP-TEST",
228
+ "S-XXX-YYY-ZZZ",
229
+ "S-X-Y-Z",
230
+ "DESKTOP-TEST$",
231
+ "WORKGROUP",
232
+ "0x3e7",
233
+ "0xbac",
234
+ "C:\\Windows\\System32\\svchost.exe"
235
+ ]
236
+ }
237
+ ```
238
+
239
+ This record is transformed to
240
+
241
+ ```
242
+ {
243
+ "ProviderName": "Microsoft-Windows-Security-Auditing",
244
+ "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
245
+ "EventID": "4798",
246
+ "Qualifiers": "",
247
+ "Level": "0",
248
+ "Task": "13824",
249
+ "Opcode": "0",
250
+ "Keywords": "0x8020000000000000",
251
+ "TimeCreated": "2019-06-19T03:10:01.982940200Z",
252
+ "EventRecordID": "87028",
253
+ "ActivityID": "",
254
+ "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
255
+ "ThreadID": "16888",
256
+ "Channel": "Security",
257
+ "Computer": "DESKTOP-TEST",
258
+ "UserID": "",
259
+ "Version": "0",
260
+ "DescriptionTitle": "A user's local group membership was enumerated.",
261
+ "subject.security_id": "S-X-Y-Z",
262
+ "subject.account_name": "DESKTOP-TEST$",
263
+ "subject.account_domain": "WORKGROUP",
264
+ "subject.logon_id": "0x3e7",
265
+ "user.security_id": "S-XXX-YYY-ZZZ",
266
+ "user.account_name": "Administrator",
267
+ "user.account_domain": "DESKTOP-TEST",
268
+ "process_information.process_id": "0xbac",
269
+ "process_information.process_name": "C:\\Windows\\System32\\svchost.exe"
270
+ }
271
+ ```
272
+
273
+ NOTE: This feature assumes `description` field has following formats:
274
+
275
+ - group delimiter: `\r\n\r\n`
276
+ - record delimiter: `\r\n\t`
277
+ - field delimiter: `\t\t`
278
+
279
+ If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
280
+
281
+ ## Copyright
282
+ ### Copyright
283
+ Copyright(C) 2014- @okahashi117
284
+ ### License
285
+ Apache License, Version 2.0
data/Rakefile CHANGED
@@ -1,10 +1,10 @@
1
- require "bundler/gem_tasks"
2
- require "rake/testtask"
3
-
4
- Rake::TestTask.new(:test) do |test|
5
- test.libs << 'lib' << 'test'
6
- test.pattern = 'test/**/test_*.rb'
7
- test.verbose = true
8
- end
9
-
10
- task default: :test
1
+ require "bundler/gem_tasks"
2
+ require "rake/testtask"
3
+
4
+ Rake::TestTask.new(:test) do |test|
5
+ test.libs << 'lib' << 'test'
6
+ test.pattern = 'test/**/test_*.rb'
7
+ test.verbose = true
8
+ end
9
+
10
+ task default: :test
data/appveyor.yml ADDED
@@ -0,0 +1,24 @@
1
+ version: '{build}'
2
+
3
+ # init:
4
+ # - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))
5
+
6
+ install:
7
+ - SET PATH=C:\Ruby%ruby_version%\bin;%PATH%
8
+ - ruby --version
9
+ - gem --version
10
+ - ridk.cmd exec bundle install
11
+ build: off
12
+ test_script:
13
+ - bundle exec rake test
14
+ # - bundle exec rake test TESTOPTS=-v
15
+
16
+ branches:
17
+ only:
18
+ - master
19
+
20
+ # https://www.appveyor.com/docs/installed-software/#ruby
21
+ environment:
22
+ matrix:
23
+ - ruby_version: "24-x64"
24
+ - ruby_version: "24"
@@ -1,25 +1,28 @@
1
- # coding: utf-8
2
- lib = File.expand_path('../lib', __FILE__)
3
- $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
-
5
- Gem::Specification.new do |spec|
6
- spec.name = "fluent-plugin-windows-eventlog"
7
- spec.version = "0.1.0"
8
- spec.authors = ["okahashi117", "Hiroshi Hatake", "Masahiro Nakagawa"]
9
- spec.email = ["naruki_okahashi@jbat.co.jp", "cosmo0920.wp@gmail.com", "repeatedly@gmail.com"]
10
- spec.summary = %q{Fluentd Input plugin to read windows event log.}
11
- spec.description = %q{Fluentd Input plugin to read windwos event log.}
12
- spec.homepage = "https://github.com/fluent/fluent-plugin-windows-eventlog"
13
- spec.license = "Apache-2.0"
14
-
15
- spec.files = `git ls-files -z`.split("\x0")
16
- spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
17
- spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
18
- spec.require_paths = ["lib"]
19
-
20
- spec.add_development_dependency "bundler"
21
- spec.add_development_dependency "rake"
22
- spec.add_development_dependency "test-unit", "~> 3.2.0"
23
- spec.add_runtime_dependency "fluentd", [">= 0.14.11", "< 2"]
24
- spec.add_runtime_dependency "win32-eventlog"
25
- end
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+
5
+ Gem::Specification.new do |spec|
6
+ spec.name = "fluent-plugin-windows-eventlog"
7
+ spec.version = "0.4.0"
8
+ spec.authors = ["okahashi117", "Hiroshi Hatake", "Masahiro Nakagawa"]
9
+ spec.email = ["naruki_okahashi@jbat.co.jp", "cosmo0920.oucc@gmail.com", "repeatedly@gmail.com"]
10
+ spec.summary = %q{Fluentd Input plugin to read windows event log.}
11
+ spec.description = %q{Fluentd Input plugin to read windows event log.}
12
+ spec.homepage = "https://github.com/fluent/fluent-plugin-windows-eventlog"
13
+ spec.license = "Apache-2.0"
14
+
15
+ spec.files = `git ls-files -z`.split("\x0")
16
+ spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
17
+ spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
18
+ spec.require_paths = ["lib"]
19
+
20
+ spec.add_development_dependency "bundler"
21
+ spec.add_development_dependency "rake"
22
+ spec.add_development_dependency "test-unit", "~> 3.2.0"
23
+ spec.add_runtime_dependency "fluentd", [">= 0.14.12", "< 2"]
24
+ spec.add_runtime_dependency "win32-eventlog"
25
+ spec.add_runtime_dependency "winevt_c", ">= 0.6.0"
26
+ spec.add_runtime_dependency "nokogiri", "~> 1.10"
27
+ spec.add_runtime_dependency "fluent-plugin-parser-winevt_xml", ">= 0.1.2"
28
+ end