fluent-plugin-windows-eventlog 0.1.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,66 +1,285 @@
1
- # fluent-plugin-windows-eventlog
2
-
3
- ## Component
4
-
5
- #### fluentd Input plugin for Windows Event Log
6
-
7
- [Fluentd](http://fluentd.org) plugin to read Windows Event Log.
8
- You must use fluentd 'Windows' brach to use me, and it doesn't work on Linux of course.
9
-
10
- ## Installation
11
- gem install fluent-plugin-windows-eventlog
12
-
13
- ## Configuration
14
- #### fluentd Input plugin for Windows Event Log
15
-
16
- <source>
17
- type windows_eventlog
18
- channels application,system
19
- pos_file c:\temp\mypos
20
- read_interval 2
21
- tag winevt.raw
22
- </source>
23
-
24
-
25
- #### parameters
26
-
27
- |name | description |
28
- |:----- |:----- |
29
- |channels | (option) 'applicaion' as default. one or combination of {application, system, setup, security}. If you want to read setup or security, administrator priv is required to launch fluentd. |
30
- |pos_file | (option, but higly recommended) a path of position file to save record numbers. |
31
- |read_interval | (option) a read interval in second. 2 seconds as default.|
32
- |from_encoding | (option) an input characters encoding. nil as default.|
33
- |encoding | (option) an output characters encoding. nil as default.|
34
-
35
-
36
- #### read keys
37
- This plugin reads follows from Windws Event Log. No customization is allowed currently.
38
-
39
- |key|
40
- |:----- |
41
- |record_number |
42
- |time_generated|
43
- |time_written |
44
- |event_id |
45
- |event_type |
46
- |event_category |
47
- |source_name |
48
- |computer_name |
49
- |user |
50
- |description |
51
-
52
-
53
-
54
- ## Etc.
55
- 'read_from_head' is not supporeted currently.You can read newer records after you start first.
56
- No customize to read information keys.
57
-
58
-
59
-
60
-
61
- ## Copyright
62
- ####Copyright
63
- Copyright(C) 2014- @okahashi117
64
- ####License
65
- Apache License, Version 2.0
66
-
1
+ # fluent-plugin-windows-eventlog
2
+
3
+ ## Component
4
+
5
+ ### fluentd Input plugin for the Windows Event Log
6
+
7
+ [Fluentd](https://www.fluentd.org/) plugin to read the Windows Event Log.
8
+
9
+ ## Installation
10
+ ridk exec gem install fluent-plugin-windows-eventlog
11
+
12
+ ## Configuration
13
+
14
+ ### in_windows_eventlog
15
+
16
+ Check [in_windows_eventlog2](https://github.com/fluent/fluent-plugin-windows-eventlog#in_windows_eventlog2) first. `in_windows_eventlog` will be replaced with `in_windows_eventlog2`.
17
+
18
+ fluentd Input plugin for the Windows Event Log using old Windows Event Logging API
19
+
20
+ <source>
21
+ @type windows_eventlog
22
+ @id windows_eventlog
23
+ channels application,system
24
+ read_interval 2
25
+ tag winevt.raw
26
+ <storage>
27
+ @type local # @type local is the default.
28
+ persistent true # default is true. Set to false to use in-memory storage.
29
+ path ./tmp/storage.json # This is required when persistent is true.
30
+ # Or, please consider using <system> section's `root_dir` parameter.
31
+ </storage>
32
+ </source>
33
+
34
+ #### parameters
35
+
36
+ |name | description |
37
+ |:----- |:----- |
38
+ |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
39
+ |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
40
+ |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
41
+ |`from_encoding` | (option) Input character encoding. `nil` as default.|
42
+ |`encoding` | (option) Output character encoding. `nil` as default.|
43
+ |`read_from_head` | (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
44
+ |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
45
+ |`parse_description`| (option) parse `description` field and set parsed result into the record. `parse` and `string_inserts` fields are removed|
46
+
47
+ ##### Available keys
48
+
49
+ This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
50
+
51
+ |key|
52
+ |:----- |
53
+ |`record_number` |
54
+ |`time_generated`|
55
+ |`time_written` |
56
+ |`event_id` |
57
+ |`event_type` |
58
+ |`event_category`|
59
+ |`source_name` |
60
+ |`computer_name` |
61
+ |`user` |
62
+ |`description` |
63
+ |`string_inserts`|
64
+
65
+ ##### `parse_description` details
66
+
67
+ Here is an example with `parse_description true`.
68
+
69
+ ```
70
+ {
71
+ "channel": "security",
72
+ "record_number": "91698",
73
+ "time_generated": "2017-08-29 20:12:29 +0000",
74
+ "time_written": "2017-08-29 20:12:29 +0000",
75
+ "event_id": "4798",
76
+ "event_type": "audit_success",
77
+ "event_category": "13824",
78
+ "source_name": "Microsoft-Windows-Security-Auditing",
79
+ "computer_name": "TEST",
80
+ "user": "",
81
+ "description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-XXX\r\n\tAccount Name:\t\tTEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x7dc\r\n\tProcess Name:\t\tC:\\Windows\\System32\\LogonUI.exe\r\n",
82
+ "string_inserts": [
83
+ "Administrator",
84
+ "TEST",
85
+ "S-XXX-YYY-ZZZ",
86
+ "S-XXX",
87
+ "TEST$",
88
+ "WORKGROUP",
89
+ "0x3e7",
90
+ "0x7dc",
91
+ "C:\\Windows\\System32\\LogonUI.exe"
92
+ ]
93
+ }
94
+ ```
95
+
96
+ This record is transformed to
97
+
98
+ ```
99
+ {
100
+ "channel": "security",
101
+ "record_number": "91698",
102
+ "time_generated": "2017-08-29 20:12:29 +0000",
103
+ "time_written": "2017-08-29 20:12:29 +0000",
104
+ "event_id": "4798",
105
+ "event_type": "audit_success",
106
+ "event_category": "13824",
107
+ "source_name": "Microsoft-Windows-Security-Auditing",
108
+ "computer_name": "TEST",
109
+ "user": "",
110
+ "description_title": "A user's local group membership was enumerated.",
111
+ "subject.security_id": "S-XXX",
112
+ "subject.account_name": "TEST$",
113
+ "subject.account_domain": "WORKGROUP",
114
+ "subject.logon_id": "0x3e7",
115
+ "user.security_id": "S-XXX-YYY-ZZZ",
116
+ "user.account_name": "Administrator",
117
+ "user.account_domain": "TEST",
118
+ "process_information.process_id": "0x7dc",
119
+ "process_information.process_name": "C:\\Windows\\System32\\LogonUI.exe\r\n"
120
+ }
121
+ ```
122
+
123
+ NOTE: This feature assumes `description` field has following formats:
124
+
125
+ - group delimiter: `\r\n\r\n`
126
+ - record delimiter: `\r\n\t`
127
+ - field delimiter: `\t\t`
128
+
129
+ If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
130
+
131
+ ### in_windows_eventlog2
132
+
133
+ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging API. This is successor to `in_windows_eventlog`. See also [this slide](https://www.slideshare.net/cosmo0920/fluentd-meetup-2019) for the details of `in_windows_eventlog2` plugin.
134
+
135
+ <source>
136
+ @type windows_eventlog2
137
+ @id windows_eventlog2
138
+ channels application,system
139
+ read_interval 2
140
+ tag winevt.raw
141
+ render_as_xml false # default is true.
142
+ rate_limit 200 # default is -1(Winevt::EventLog::Subscribe::RATE_INFINITE).
143
+ <storage>
144
+ @type local # @type local is the default.
145
+ persistent true # default is true. Set to false to use in-memory storage.
146
+ path ./tmp/storage.json # This is required when persistent is true.
147
+ # Or, please consider using <system> section's `root_dir` parameter.
148
+ </storage>
149
+ <parse>
150
+ @type winevt_xml # @type winevt_xml is the default. winevt_xml and none parsers are supported for now.
151
+ </parse>
152
+ </source>
153
+
154
+ **NOTE:** in_windows_eventlog2 always handles EventLog records as UTF-8 characters. Users don't have to specify encoding related parameters and they are not provided.
155
+
156
+ **NOTE:** When `Description` contains error message such as `The message resource is present but the message was not found in the message table.`, eventlog's resource file (.mui) related to error generating event is something wrong. This issue is also occurred in built-in Windows Event Viewer which is the part of Windows management tool.
157
+
158
+ **NOTE:** When `render_as_xml` as `false`, the dependent winevt_c gem renders Windows EventLog as Ruby Hash object directly. This reduces bottleneck to consume EventLog. Specifying `render_as_xml` as `false` should be faster consuming than `render_as_xml` as `true` case.
159
+
160
+ **NOTE:** If you encountered CPU spike due to massively huge EventLog channel, `rate_limit` parameter may help you. Currently, this paramter can handle the multiples of 10 or -1(`Winevt::EventLog::Subscribe::RATE_INFINITE`).
161
+
162
+ #### parameters
163
+
164
+ |name | description |
165
+ |:----- |:----- |
166
+ |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
167
+ |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
168
+ |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
169
+ |`from_encoding` | (option) Input character encoding. `nil` as default.|
170
+ |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
171
+ |`<parse>` | Setting for `parser` plugin for parsing raw XML EventLog records. |
172
+ |`parse_description`| (option) parse `description` field and set parsed result into the record. `Description` and `EventData` fields are removed|
173
+
174
+ ##### Available keys
175
+
176
+ This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
177
+
178
+ |key|
179
+ |:----- |
180
+ |`ProviderName` |
181
+ |`ProviderGuid` |
182
+ |`EventID` |
183
+ |`Qualifiers` |
184
+ |`Level` |
185
+ |`Task` |
186
+ |`Opcode` |
187
+ |`Keywords` |
188
+ |`TimeCreated` |
189
+ |`EventRecordId` |
190
+ |`ActivityID` |
191
+ |`RelatedActivityID`|
192
+ |`ProcessID` |
193
+ |`ThreadID` |
194
+ |`Channel` |
195
+ |`Computer` |
196
+ |`UserID` |
197
+ |`Version` |
198
+ |`Description` |
199
+ |`EventData` |
200
+
201
+ ##### `parse_description` details
202
+
203
+ Here is an example with `parse_description true`.
204
+
205
+ ```
206
+ {
207
+ "ProviderName": "Microsoft-Windows-Security-Auditing",
208
+ "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
209
+ "EventID": "4798",
210
+ "Qualifiers": "",
211
+ "Level": "0",
212
+ "Task": "13824",
213
+ "Opcode": "0",
214
+ "Keywords": "0x8020000000000000",
215
+ "TimeCreated": "2019-06-19T03:10:01.982940200Z",
216
+ "EventRecordID": "87028",
217
+ "ActivityID": "",
218
+ "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
219
+ "ThreadID": "16888",
220
+ "Channel": "Security",
221
+ "Computer": "DESKTOP-TEST",
222
+ "UserID": "",
223
+ "Version": "0",
224
+ "Description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-X-Y-Z\r\n\tAccount Name:\t\tDESKTOP-TEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ0\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tDESKTOP-TEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0xbac\r\n\tProcess Name:\t\tC:\\Windows\\System32\\svchost.exe\r\n",
225
+ "EventData": [
226
+ "Administrator",
227
+ "DESKTOP-TEST",
228
+ "S-XXX-YYY-ZZZ",
229
+ "S-X-Y-Z",
230
+ "DESKTOP-TEST$",
231
+ "WORKGROUP",
232
+ "0x3e7",
233
+ "0xbac",
234
+ "C:\\Windows\\System32\\svchost.exe"
235
+ ]
236
+ }
237
+ ```
238
+
239
+ This record is transformed to
240
+
241
+ ```
242
+ {
243
+ "ProviderName": "Microsoft-Windows-Security-Auditing",
244
+ "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
245
+ "EventID": "4798",
246
+ "Qualifiers": "",
247
+ "Level": "0",
248
+ "Task": "13824",
249
+ "Opcode": "0",
250
+ "Keywords": "0x8020000000000000",
251
+ "TimeCreated": "2019-06-19T03:10:01.982940200Z",
252
+ "EventRecordID": "87028",
253
+ "ActivityID": "",
254
+ "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
255
+ "ThreadID": "16888",
256
+ "Channel": "Security",
257
+ "Computer": "DESKTOP-TEST",
258
+ "UserID": "",
259
+ "Version": "0",
260
+ "DescriptionTitle": "A user's local group membership was enumerated.",
261
+ "subject.security_id": "S-X-Y-Z",
262
+ "subject.account_name": "DESKTOP-TEST$",
263
+ "subject.account_domain": "WORKGROUP",
264
+ "subject.logon_id": "0x3e7",
265
+ "user.security_id": "S-XXX-YYY-ZZZ",
266
+ "user.account_name": "Administrator",
267
+ "user.account_domain": "DESKTOP-TEST",
268
+ "process_information.process_id": "0xbac",
269
+ "process_information.process_name": "C:\\Windows\\System32\\svchost.exe"
270
+ }
271
+ ```
272
+
273
+ NOTE: This feature assumes `description` field has following formats:
274
+
275
+ - group delimiter: `\r\n\r\n`
276
+ - record delimiter: `\r\n\t`
277
+ - field delimiter: `\t\t`
278
+
279
+ If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
280
+
281
+ ## Copyright
282
+ ### Copyright
283
+ Copyright(C) 2014- @okahashi117
284
+ ### License
285
+ Apache License, Version 2.0
data/Rakefile CHANGED
@@ -1,10 +1,10 @@
1
- require "bundler/gem_tasks"
2
- require "rake/testtask"
3
-
4
- Rake::TestTask.new(:test) do |test|
5
- test.libs << 'lib' << 'test'
6
- test.pattern = 'test/**/test_*.rb'
7
- test.verbose = true
8
- end
9
-
10
- task default: :test
1
+ require "bundler/gem_tasks"
2
+ require "rake/testtask"
3
+
4
+ Rake::TestTask.new(:test) do |test|
5
+ test.libs << 'lib' << 'test'
6
+ test.pattern = 'test/**/test_*.rb'
7
+ test.verbose = true
8
+ end
9
+
10
+ task default: :test
data/appveyor.yml ADDED
@@ -0,0 +1,24 @@
1
+ version: '{build}'
2
+
3
+ # init:
4
+ # - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))
5
+
6
+ install:
7
+ - SET PATH=C:\Ruby%ruby_version%\bin;%PATH%
8
+ - ruby --version
9
+ - gem --version
10
+ - ridk.cmd exec bundle install
11
+ build: off
12
+ test_script:
13
+ - bundle exec rake test
14
+ # - bundle exec rake test TESTOPTS=-v
15
+
16
+ branches:
17
+ only:
18
+ - master
19
+
20
+ # https://www.appveyor.com/docs/installed-software/#ruby
21
+ environment:
22
+ matrix:
23
+ - ruby_version: "24-x64"
24
+ - ruby_version: "24"
@@ -1,25 +1,28 @@
1
- # coding: utf-8
2
- lib = File.expand_path('../lib', __FILE__)
3
- $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
-
5
- Gem::Specification.new do |spec|
6
- spec.name = "fluent-plugin-windows-eventlog"
7
- spec.version = "0.1.0"
8
- spec.authors = ["okahashi117", "Hiroshi Hatake", "Masahiro Nakagawa"]
9
- spec.email = ["naruki_okahashi@jbat.co.jp", "cosmo0920.wp@gmail.com", "repeatedly@gmail.com"]
10
- spec.summary = %q{Fluentd Input plugin to read windows event log.}
11
- spec.description = %q{Fluentd Input plugin to read windwos event log.}
12
- spec.homepage = "https://github.com/fluent/fluent-plugin-windows-eventlog"
13
- spec.license = "Apache-2.0"
14
-
15
- spec.files = `git ls-files -z`.split("\x0")
16
- spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
17
- spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
18
- spec.require_paths = ["lib"]
19
-
20
- spec.add_development_dependency "bundler"
21
- spec.add_development_dependency "rake"
22
- spec.add_development_dependency "test-unit", "~> 3.2.0"
23
- spec.add_runtime_dependency "fluentd", [">= 0.14.11", "< 2"]
24
- spec.add_runtime_dependency "win32-eventlog"
25
- end
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+
5
+ Gem::Specification.new do |spec|
6
+ spec.name = "fluent-plugin-windows-eventlog"
7
+ spec.version = "0.4.0"
8
+ spec.authors = ["okahashi117", "Hiroshi Hatake", "Masahiro Nakagawa"]
9
+ spec.email = ["naruki_okahashi@jbat.co.jp", "cosmo0920.oucc@gmail.com", "repeatedly@gmail.com"]
10
+ spec.summary = %q{Fluentd Input plugin to read windows event log.}
11
+ spec.description = %q{Fluentd Input plugin to read windows event log.}
12
+ spec.homepage = "https://github.com/fluent/fluent-plugin-windows-eventlog"
13
+ spec.license = "Apache-2.0"
14
+
15
+ spec.files = `git ls-files -z`.split("\x0")
16
+ spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
17
+ spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
18
+ spec.require_paths = ["lib"]
19
+
20
+ spec.add_development_dependency "bundler"
21
+ spec.add_development_dependency "rake"
22
+ spec.add_development_dependency "test-unit", "~> 3.2.0"
23
+ spec.add_runtime_dependency "fluentd", [">= 0.14.12", "< 2"]
24
+ spec.add_runtime_dependency "win32-eventlog"
25
+ spec.add_runtime_dependency "winevt_c", ">= 0.6.0"
26
+ spec.add_runtime_dependency "nokogiri", "~> 1.10"
27
+ spec.add_runtime_dependency "fluent-plugin-parser-winevt_xml", ">= 0.1.2"
28
+ end