fluent-plugin-windows-eventlog 0.8.0 → 0.8.2

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -1,385 +1,387 @@
1
- # fluent-plugin-windows-eventlog
2
-
3
- ## Component
4
-
5
- ### fluentd Input plugin for the Windows Event Log
6
-
7
- [Fluentd](https://www.fluentd.org/) plugin to read the Windows Event Log.
8
-
9
- ## Installation
10
- ridk exec gem install fluent-plugin-windows-eventlog
11
-
12
- ## Configuration
13
-
14
- ### in_windows_eventlog
15
-
16
- Check [in_windows_eventlog2](https://github.com/fluent/fluent-plugin-windows-eventlog#in_windows_eventlog2) first. `in_windows_eventlog` will be replaced with `in_windows_eventlog2`.
17
-
18
- fluentd Input plugin for the Windows Event Log using old Windows Event Logging API
19
-
20
- <source>
21
- @type windows_eventlog
22
- @id windows_eventlog
23
- channels application,system
24
- read_interval 2
25
- tag winevt.raw
26
- <storage>
27
- @type local # @type local is the default.
28
- persistent true # default is true. Set to false to use in-memory storage.
29
- path ./tmp/storage.json # This is required when persistent is true.
30
- # Or, please consider using <system> section's `root_dir` parameter.
31
- </storage>
32
- </source>
33
-
34
- #### parameters
35
-
36
- |name | description |
37
- |:----- |:----- |
38
- |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
39
- |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
40
- |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
41
- |`from_encoding` | (option) Input character encoding. `nil` as default.|
42
- |`encoding` | (option) Output character encoding. `nil` as default.|
43
- |`read_from_head` | (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
44
- |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
45
- |`parse_description`| (option) parse `description` field and set parsed result into the record. `parse` and `string_inserts` fields are removed|
46
-
47
- ##### Available keys
48
-
49
- This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
50
-
51
- |key|
52
- |:----- |
53
- |`record_number` |
54
- |`time_generated`|
55
- |`time_written` |
56
- |`event_id` |
57
- |`event_type` |
58
- |`event_category`|
59
- |`source_name` |
60
- |`computer_name` |
61
- |`user` |
62
- |`description` |
63
- |`string_inserts`|
64
-
65
- ##### `parse_description` details
66
-
67
- Here is an example with `parse_description true`.
68
-
69
- ```
70
- {
71
- "channel": "security",
72
- "record_number": "91698",
73
- "time_generated": "2017-08-29 20:12:29 +0000",
74
- "time_written": "2017-08-29 20:12:29 +0000",
75
- "event_id": "4798",
76
- "event_type": "audit_success",
77
- "event_category": "13824",
78
- "source_name": "Microsoft-Windows-Security-Auditing",
79
- "computer_name": "TEST",
80
- "user": "",
81
- "description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-XXX\r\n\tAccount Name:\t\tTEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x7dc\r\n\tProcess Name:\t\tC:\\Windows\\System32\\LogonUI.exe\r\n",
82
- "string_inserts": [
83
- "Administrator",
84
- "TEST",
85
- "S-XXX-YYY-ZZZ",
86
- "S-XXX",
87
- "TEST$",
88
- "WORKGROUP",
89
- "0x3e7",
90
- "0x7dc",
91
- "C:\\Windows\\System32\\LogonUI.exe"
92
- ]
93
- }
94
- ```
95
-
96
- This record is transformed to
97
-
98
- ```
99
- {
100
- "channel": "security",
101
- "record_number": "91698",
102
- "time_generated": "2017-08-29 20:12:29 +0000",
103
- "time_written": "2017-08-29 20:12:29 +0000",
104
- "event_id": "4798",
105
- "event_type": "audit_success",
106
- "event_category": "13824",
107
- "source_name": "Microsoft-Windows-Security-Auditing",
108
- "computer_name": "TEST",
109
- "user": "",
110
- "description_title": "A user's local group membership was enumerated.",
111
- "subject.security_id": "S-XXX",
112
- "subject.account_name": "TEST$",
113
- "subject.account_domain": "WORKGROUP",
114
- "subject.logon_id": "0x3e7",
115
- "user.security_id": "S-XXX-YYY-ZZZ",
116
- "user.account_name": "Administrator",
117
- "user.account_domain": "TEST",
118
- "process_information.process_id": "0x7dc",
119
- "process_information.process_name": "C:\\Windows\\System32\\LogonUI.exe\r\n"
120
- }
121
- ```
122
-
123
- NOTE: This feature assumes `description` field has following formats:
124
-
125
- - group delimiter: `\r\n\r\n`
126
- - record delimiter: `\r\n\t`
127
- - field delimiter: `\t\t`
128
-
129
- If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
130
-
131
- ### in_windows_eventlog2
132
-
133
- fluentd Input plugin for the Windows Event Log using newer Windows Event Logging API. This is successor to `in_windows_eventlog`. See also [this slide](https://www.slideshare.net/cosmo0920/fluentd-meetup-2019) for the details of `in_windows_eventlog2` plugin.
134
-
135
- <source>
136
- @type windows_eventlog2
137
- @id windows_eventlog2
138
- channels application,system # Also be able to use `<subscribe>` directive.
139
- read_existing_events false
140
- read_interval 2
141
- tag winevt.raw
142
- render_as_xml false # default is false.
143
- rate_limit 200 # default is -1(Winevt::EventLog::Subscribe::RATE_INFINITE).
144
- # preserve_qualifiers_on_hash true # default is false.
145
- # read_all_channels false # default is false.
146
- # description_locale en_US # default is nil. It means that system locale is used for obtaining description.
147
- # refresh_subscription_interval 10m # default is nil. It specifies refresh interval for channel subscriptions.
148
- <storage>
149
- @type local # @type local is the default.
150
- persistent true # default is true. Set to false to use in-memory storage.
151
- path ./tmp/storage.json # This is required when persistent is true.
152
- # Or, please consider using <system> section's `root_dir` parameter.
153
- </storage>
154
- <parse>
155
- @type winevt_xml # @type winevt_xml is the default. winevt_xml and none parsers are supported for now.
156
- # When set up it as true, this plugin preserves "Qualifiers" and "EventID" keys.
157
- # When set up it as false, this plugin calculates actual "EventID" from "Qualifiers" and removing "Qualifiers".
158
- # With the following equation:
159
- # (EventID & 0xffff) | (Qualifiers & 0xffff) << 16
160
- preserve_qualifiers true
161
- </parse>
162
- # <subscribe>
163
- # channles application, system
164
- # read_existing_events false # read_existing_events should be applied each of subscribe directive(s)
165
- # remote_server 127.0.0.1 # Remote server ip/fqdn
166
- # remote_domain WORKGROUP # Domain name
167
- # remote_username fluentd # Remoting access account name
168
- # remote_password changeme! # Remoting access account password
169
- # </subscribe>
170
- </source>
171
-
172
- **NOTE:** in_windows_eventlog2 always handles EventLog records as UTF-8 characters. Users don't have to specify encoding related parameters and they are not provided.
173
-
174
- **NOTE:** When `Description` contains error message such as `The message resource is present but the message was not found in the message table.`, eventlog's resource file (.mui) related to error generating event is something wrong. This issue is also occurred in built-in Windows Event Viewer which is the part of Windows management tool.
175
-
176
- **NOTE:** When `render_as_xml` as `true`, `fluent-plugin-parser-winevt_xml` plugin should be needed to parse XML rendered Windows EventLog string.
177
-
178
- **NOTE:** If you encountered CPU spike due to massively huge EventLog channel, `rate_limit` parameter may help you. Currently, this paramter can handle the multiples of 10 or -1(`Winevt::EventLog::Subscribe::RATE_INFINITE`).
179
-
180
- #### parameters
181
-
182
- |name | description |
183
- |:----- |:----- |
184
- |`channels` | (option) No default value just empty array, but 'application' is used as default due to backward compatibility. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
185
- |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
186
- |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
187
- |`from_encoding` | (option) Input character encoding. `nil` as default.|
188
- |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
189
- |`<parse>` | Setting for `parser` plugin for parsing raw XML EventLog records. |
190
- |`parse_description`| (option) parse `description` field and set parsed result into the record. `Description` and `EventData` fields are removed|
191
- |`read_from_head` | **Deprecated** (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
192
- |`read_existing_events` | (option) Read the entries which already exist before fluentd is started. Defaults to `false`.|
193
- |`render_as_xml` | (option) Render Windows EventLog as XML or Ruby Hash object directly. Defaults to `false`.|
194
- |`rate_limit` | (option) Specify rate limit to consume EventLog. Default is `Winevt::EventLog::Subscribe::RATE_INFINITE`.|
195
- |`preserve_qualifiers_on_hash` | (option) When set up it as true, this plugin preserves "Qualifiers" and "EventID" keys. When set up it as false, this plugin calculates actual "EventID" from "Qualifiers" and removing "Qualifiers". Default is `false`.|
196
- |`read_all_channels`| (option) Read from all channels. Default is `false`|
197
- |`description_locale`| (option) Specify description locale. Default is `nil`. See also: [Supported locales](https://github.com/fluent-plugins-nursery/winevt_c#multilingual-description) |
198
- |`refresh_subscription_interval`|(option) It specifies refresh interval for channel subscriptions. Default is `nil`.|
199
- |`<subscribe>` | Setting for subscribe channels. |
200
-
201
- ##### subscribe section
202
-
203
- |name | description |
204
- |:----- |:----- |
205
- |`channels` | One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges. |
206
- |`read_existing_events` | (option) Read the entries which already exist before fluentd is started. Defaults to `false`. |
207
- |`remote_server` | (option) Remoting access server ip address/fqdn. Defaults to `nil`. |
208
- |`remote_domain` | (option) Remoting access server joining domain name. Defaults to `nil`. |
209
- |`remote_username` | (option) Remoting access access account's username. Defaults to `nil`. |
210
- |`remote_password` | (option) Remoting access access account's password. Defaults to `nil`. |
211
-
212
-
213
- **Motivation:** subscribe directive is designed for applying `read_existing_events` each of channels which is specified in subscribe section(s).
214
-
215
- e.g) The previous configuration can handle `read_existing_events` but this parameter only specifies `read_existing_events` or not for channels which are specified in `channels`.
216
-
217
- ```aconf
218
- channels ["Application", "Security", "HardwareEvents"]
219
- read_existing_events true
220
- ```
221
-
222
- is interpreted as "Application", "Security", and "HardwareEvents" should be read existing events.
223
-
224
- But some users want to configure to:
225
-
226
- * "Application" and "Security" channels just tailing
227
- * "HardwareEvent" channel read existing events before launching Fluentd
228
-
229
- With `<subscribe>` directive, this requirements can be represendted as:
230
-
231
- ```aconf
232
- <subscribe>
233
- channles ["Application", "Security"]
234
- # read_existing_events false
235
- </subscribe>
236
- <subscribe>
237
- channles ["HardwareEvent"]
238
- read_existing_events true
239
- </subscribe>
240
- ```
241
-
242
- This configuration can be handled as:
243
-
244
- * "Application" and "Security" channels just tailing
245
- * "HardwareEvent" channel read existing events before launching Fluentd
246
-
247
- ###### Remoting access
248
-
249
- `<subscribe>` section supports remoting access parameters:
250
-
251
- * `remote_server`
252
- * `remote_domain`
253
- * `remote_username`
254
- * `remote_password`
255
-
256
- These parameters are only in `<subscribe>` directive.
257
-
258
- Note that before using this feature, remoting access users should belong to "Event Log Readers" group:
259
-
260
- ```console
261
- > net localgroup "Event Log Readers" <domain\username> /add
262
- ```
263
-
264
- And then, users also should set up their remote box's Firewall configuration:
265
-
266
- ```console
267
- > netsh advfirewall firewall set rule group="Remote Event Log Management" new enable=yes
268
- ```
269
-
270
- As a security best practices, remoting access account _should not be administrator account_.
271
-
272
- For graphical instructions, please refer to [Preconfigure a Machine to Collect Remote Windows Events | Sumo Logic](https://help.sumologic.com/03Send-Data/Sources/01Sources-for-Installed-Collectors/Remote-Windows-Event-Log-Source/Preconfigure-a-Machine-to-Collect-Remote-Windows-Events) document for example.
273
-
274
- ##### Available keys
275
-
276
- This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
277
-
278
- |key|
279
- |:----- |
280
- |`ProviderName` |
281
- |`ProviderGuid` |
282
- |`EventID` |
283
- |`Qualifiers` |
284
- |`Level` |
285
- |`Task` |
286
- |`Opcode` |
287
- |`Keywords` |
288
- |`TimeCreated` |
289
- |`EventRecordId` |
290
- |`ActivityID` |
291
- |`RelatedActivityID`|
292
- |`ProcessID` |
293
- |`ThreadID` |
294
- |`Channel` |
295
- |`Computer` |
296
- |`UserID` |
297
- |`Version` |
298
- |`Description` |
299
- |`EventData` |
300
-
301
- ##### `parse_description` details
302
-
303
- Here is an example with `parse_description true`.
304
-
305
- ```
306
- {
307
- "ProviderName": "Microsoft-Windows-Security-Auditing",
308
- "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
309
- "EventID": "4798",
310
- "Qualifiers": "",
311
- "Level": "0",
312
- "Task": "13824",
313
- "Opcode": "0",
314
- "Keywords": "0x8020000000000000",
315
- "TimeCreated": "2019-06-19T03:10:01.982940200Z",
316
- "EventRecordID": "87028",
317
- "ActivityID": "",
318
- "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
319
- "ThreadID": "16888",
320
- "Channel": "Security",
321
- "Computer": "DESKTOP-TEST",
322
- "UserID": "",
323
- "Version": "0",
324
- "Description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-X-Y-Z\r\n\tAccount Name:\t\tDESKTOP-TEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ0\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tDESKTOP-TEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0xbac\r\n\tProcess Name:\t\tC:\\Windows\\System32\\svchost.exe\r\n",
325
- "EventData": [
326
- "Administrator",
327
- "DESKTOP-TEST",
328
- "S-XXX-YYY-ZZZ",
329
- "S-X-Y-Z",
330
- "DESKTOP-TEST$",
331
- "WORKGROUP",
332
- "0x3e7",
333
- "0xbac",
334
- "C:\\Windows\\System32\\svchost.exe"
335
- ]
336
- }
337
- ```
338
-
339
- This record is transformed to
340
-
341
- ```
342
- {
343
- "ProviderName": "Microsoft-Windows-Security-Auditing",
344
- "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
345
- "EventID": "4798",
346
- "Qualifiers": "",
347
- "Level": "0",
348
- "Task": "13824",
349
- "Opcode": "0",
350
- "Keywords": "0x8020000000000000",
351
- "TimeCreated": "2019-06-19T03:10:01.982940200Z",
352
- "EventRecordID": "87028",
353
- "ActivityID": "",
354
- "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
355
- "ThreadID": "16888",
356
- "Channel": "Security",
357
- "Computer": "DESKTOP-TEST",
358
- "UserID": "",
359
- "Version": "0",
360
- "DescriptionTitle": "A user's local group membership was enumerated.",
361
- "subject.security_id": "S-X-Y-Z",
362
- "subject.account_name": "DESKTOP-TEST$",
363
- "subject.account_domain": "WORKGROUP",
364
- "subject.logon_id": "0x3e7",
365
- "user.security_id": "S-XXX-YYY-ZZZ",
366
- "user.account_name": "Administrator",
367
- "user.account_domain": "DESKTOP-TEST",
368
- "process_information.process_id": "0xbac",
369
- "process_information.process_name": "C:\\Windows\\System32\\svchost.exe"
370
- }
371
- ```
372
-
373
- NOTE: This feature assumes `description` field has following formats:
374
-
375
- - group delimiter: `\r\n\r\n`
376
- - record delimiter: `\r\n\t`
377
- - field delimiter: `\t\t`
378
-
379
- If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
380
-
381
- ## Copyright
382
- ### Copyright
383
- Copyright(C) 2014- @okahashi117
384
- ### License
385
- Apache License, Version 2.0
1
+ # fluent-plugin-windows-eventlog
2
+
3
+ ## Component
4
+
5
+ ### fluentd Input plugin for the Windows Event Log
6
+
7
+ [Fluentd](https://www.fluentd.org/) plugin to read the Windows Event Log.
8
+
9
+ ## Installation
10
+ ridk exec gem install fluent-plugin-windows-eventlog
11
+
12
+ ## Configuration
13
+
14
+ ### in_windows_eventlog
15
+
16
+ Check [in_windows_eventlog2](https://github.com/fluent/fluent-plugin-windows-eventlog#in_windows_eventlog2) first. `in_windows_eventlog` will be replaced with `in_windows_eventlog2`.
17
+
18
+ fluentd Input plugin for the Windows Event Log using old Windows Event Logging API
19
+
20
+ <source>
21
+ @type windows_eventlog
22
+ @id windows_eventlog
23
+ channels application,system
24
+ read_interval 2
25
+ tag winevt.raw
26
+ <storage>
27
+ @type local # @type local is the default.
28
+ persistent true # default is true. Set to false to use in-memory storage.
29
+ path ./tmp/storage.json # This is required when persistent is true.
30
+ # Or, please consider using <system> section's `root_dir` parameter.
31
+ </storage>
32
+ </source>
33
+
34
+ #### parameters
35
+
36
+ |name | description |
37
+ |:----- |:----- |
38
+ |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
39
+ |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
40
+ |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
41
+ |`from_encoding` | (option) Input character encoding. `nil` as default.|
42
+ |`encoding` | (option) Output character encoding. `nil` as default.|
43
+ |`read_from_head` | (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
44
+ |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
45
+ |`parse_description`| (option) parse `description` field and set parsed result into the record. `parse` and `string_inserts` fields are removed|
46
+
47
+ ##### Available keys
48
+
49
+ This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
50
+
51
+ |key|
52
+ |:----- |
53
+ |`record_number` |
54
+ |`time_generated`|
55
+ |`time_written` |
56
+ |`event_id` |
57
+ |`event_type` |
58
+ |`event_category`|
59
+ |`source_name` |
60
+ |`computer_name` |
61
+ |`user` |
62
+ |`description` |
63
+ |`string_inserts`|
64
+
65
+ ##### `parse_description` details
66
+
67
+ Here is an example with `parse_description true`.
68
+
69
+ ```
70
+ {
71
+ "channel": "security",
72
+ "record_number": "91698",
73
+ "time_generated": "2017-08-29 20:12:29 +0000",
74
+ "time_written": "2017-08-29 20:12:29 +0000",
75
+ "event_id": "4798",
76
+ "event_type": "audit_success",
77
+ "event_category": "13824",
78
+ "source_name": "Microsoft-Windows-Security-Auditing",
79
+ "computer_name": "TEST",
80
+ "user": "",
81
+ "description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-XXX\r\n\tAccount Name:\t\tTEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x7dc\r\n\tProcess Name:\t\tC:\\Windows\\System32\\LogonUI.exe\r\n",
82
+ "string_inserts": [
83
+ "Administrator",
84
+ "TEST",
85
+ "S-XXX-YYY-ZZZ",
86
+ "S-XXX",
87
+ "TEST$",
88
+ "WORKGROUP",
89
+ "0x3e7",
90
+ "0x7dc",
91
+ "C:\\Windows\\System32\\LogonUI.exe"
92
+ ]
93
+ }
94
+ ```
95
+
96
+ This record is transformed to
97
+
98
+ ```
99
+ {
100
+ "channel": "security",
101
+ "record_number": "91698",
102
+ "time_generated": "2017-08-29 20:12:29 +0000",
103
+ "time_written": "2017-08-29 20:12:29 +0000",
104
+ "event_id": "4798",
105
+ "event_type": "audit_success",
106
+ "event_category": "13824",
107
+ "source_name": "Microsoft-Windows-Security-Auditing",
108
+ "computer_name": "TEST",
109
+ "user": "",
110
+ "description_title": "A user's local group membership was enumerated.",
111
+ "subject.security_id": "S-XXX",
112
+ "subject.account_name": "TEST$",
113
+ "subject.account_domain": "WORKGROUP",
114
+ "subject.logon_id": "0x3e7",
115
+ "user.security_id": "S-XXX-YYY-ZZZ",
116
+ "user.account_name": "Administrator",
117
+ "user.account_domain": "TEST",
118
+ "process_information.process_id": "0x7dc",
119
+ "process_information.process_name": "C:\\Windows\\System32\\LogonUI.exe\r\n"
120
+ }
121
+ ```
122
+
123
+ NOTE: This feature assumes `description` field has following formats:
124
+
125
+ - group delimiter: `\r\n\r\n`
126
+ - record delimiter: `\r\n\t`
127
+ - field delimiter: `\t\t`
128
+
129
+ If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
130
+
131
+ ### in_windows_eventlog2
132
+
133
+ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging API. This is successor to `in_windows_eventlog`. See also [this slide](https://www.slideshare.net/cosmo0920/fluentd-meetup-2019) for the details of `in_windows_eventlog2` plugin.
134
+
135
+ <source>
136
+ @type windows_eventlog2
137
+ @id windows_eventlog2
138
+ channels application,system # Also be able to use `<subscribe>` directive.
139
+ read_existing_events false
140
+ read_interval 2
141
+ tag winevt.raw
142
+ render_as_xml false # default is false.
143
+ rate_limit 200 # default is -1(Winevt::EventLog::Subscribe::RATE_INFINITE).
144
+ # preserve_qualifiers_on_hash true # default is false.
145
+ # read_all_channels false # default is false.
146
+ # description_locale en_US # default is nil. It means that system locale is used for obtaining description.
147
+ # refresh_subscription_interval 10m # default is nil. It specifies refresh interval for channel subscriptions.
148
+ # event_query "Event/System[EventID!=1001]" # default is "*".
149
+ <storage>
150
+ @type local # @type local is the default.
151
+ persistent true # default is true. Set to false to use in-memory storage.
152
+ path ./tmp/storage.json # This is required when persistent is true. If migrating from eventlog v1 please ensure that you remove the old .pos folder
153
+ # Or, please consider using <system> section's `root_dir` parameter.
154
+ </storage>
155
+ # <parse> # Note: parsing is only available when render_as_xml true
156
+ # @type winevt_xml # @type winevt_xml is the default. winevt_xml and none parsers are supported for now.
157
+ # When set up it as true, this plugin preserves "Qualifiers" and "EventID" keys.
158
+ # When set up it as false, this plugin calculates actual "EventID" from "Qualifiers" and removing "Qualifiers".
159
+ # With the following equation:
160
+ # (EventID & 0xffff) | (Qualifiers & 0xffff) << 16
161
+ # preserve_qualifiers true # preserve_qualifiers_on_hash can be used as a setting outside <parse> if render_as_xml is false
162
+ # </parse>
163
+ # <subscribe>
164
+ # channles application, system
165
+ # read_existing_events false # read_existing_events should be applied each of subscribe directive(s)
166
+ # remote_server 127.0.0.1 # Remote server ip/fqdn
167
+ # remote_domain WORKGROUP # Domain name
168
+ # remote_username fluentd # Remoting access account name
169
+ # remote_password changeme! # Remoting access account password
170
+ # </subscribe>
171
+ </source>
172
+
173
+ **NOTE:** in_windows_eventlog2 always handles EventLog records as UTF-8 characters. Users don't have to specify encoding related parameters and they are not provided.
174
+
175
+ **NOTE:** When `Description` contains error message such as `The message resource is present but the message was not found in the message table.`, eventlog's resource file (.mui) related to error generating event is something wrong. This issue is also occurred in built-in Windows Event Viewer which is the part of Windows management tool.
176
+
177
+ **NOTE:** When `render_as_xml` as `true`, `fluent-plugin-parser-winevt_xml` plugin should be needed to parse XML rendered Windows EventLog string.
178
+
179
+ **NOTE:** If you encountered CPU spike due to massively huge EventLog channel, `rate_limit` parameter may help you. Currently, this paramter can handle the multiples of 10 or -1(`Winevt::EventLog::Subscribe::RATE_INFINITE`).
180
+
181
+ #### parameters
182
+
183
+ |name | description |
184
+ |:----- |:----- |
185
+ |`channels` | (option) No default value just empty array, but 'application' is used as default due to backward compatibility. One or more of {'application', 'system', 'setup', 'security'} and other evtx, which is the brand new Windows XML Event Log (EVTX) format since Windows Vista, formatted channels. Theoritically, `in_windows_ventlog2` may read all of channels except for debug and analytical typed channels. If you want to read 'setup' or 'security' logs or some privileged channels, you must launch fluentd with administrator privileges.|
186
+ |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
187
+ |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
188
+ |`from_encoding` | (option) Input character encoding. `nil` as default.|
189
+ |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
190
+ |`<parse>` | Setting for `parser` plugin for parsing raw XML EventLog records. |
191
+ |`parse_description`| (option) parse `description` field and set parsed result into the record. `Description` and `EventData` fields are removed|
192
+ |`read_from_head` | **Deprecated** (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
193
+ |`read_existing_events` | (option) Read the entries which already exist before fluentd is started. Defaults to `false`.|
194
+ |`render_as_xml` | (option) Render Windows EventLog as XML or Ruby Hash object directly. Defaults to `false`.|
195
+ |`rate_limit` | (option) Specify rate limit to consume EventLog. Default is `Winevt::EventLog::Subscribe::RATE_INFINITE`.|
196
+ |`preserve_qualifiers_on_hash` | (option) When set up it as true, this plugin preserves "Qualifiers" and "EventID" keys. When set up it as false, this plugin calculates actual "EventID" from "Qualifiers" and removing "Qualifiers". Default is `false`.|
197
+ |`read_all_channels`| (option) Read from all channels. Default is `false`|
198
+ |`description_locale`| (option) Specify description locale. Default is `nil`. See also: [Supported locales](https://github.com/fluent-plugins-nursery/winevt_c#multilingual-description) |
199
+ |`refresh_subscription_interval`|(option) It specifies refresh interval for channel subscriptions. Default is `nil`.|
200
+ |`event_query`|(option) It specifies query for deny/allow/filter events with XPath 1.0 or structured XML query. Default is `"*"` (retrieving all events).|
201
+ |`<subscribe>` | Setting for subscribe channels. |
202
+
203
+ ##### subscribe section
204
+
205
+ |name | description |
206
+ |:----- |:----- |
207
+ |`channels` | One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges. |
208
+ |`read_existing_events` | (option) Read the entries which already exist before fluentd is started. Defaults to `false`. |
209
+ |`remote_server` | (option) Remoting access server ip address/fqdn. Defaults to `nil`. |
210
+ |`remote_domain` | (option) Remoting access server joining domain name. Defaults to `nil`. |
211
+ |`remote_username` | (option) Remoting access access account's username. Defaults to `nil`. |
212
+ |`remote_password` | (option) Remoting access access account's password. Defaults to `nil`. |
213
+
214
+
215
+ **Motivation:** subscribe directive is designed for applying `read_existing_events` each of channels which is specified in subscribe section(s).
216
+
217
+ e.g) The previous configuration can handle `read_existing_events` but this parameter only specifies `read_existing_events` or not for channels which are specified in `channels`.
218
+
219
+ ```aconf
220
+ channels ["Application", "Security", "HardwareEvents"]
221
+ read_existing_events true
222
+ ```
223
+
224
+ is interpreted as "Application", "Security", and "HardwareEvents" should be read existing events.
225
+
226
+ But some users want to configure to:
227
+
228
+ * "Application" and "Security" channels just tailing
229
+ * "HardwareEvent" channel read existing events before launching Fluentd
230
+
231
+ With `<subscribe>` directive, this requirements can be represendted as:
232
+
233
+ ```aconf
234
+ <subscribe>
235
+ channels ["Application", "Security"]
236
+ # read_existing_events false
237
+ </subscribe>
238
+ <subscribe>
239
+ channels ["HardwareEvent"]
240
+ read_existing_events true
241
+ </subscribe>
242
+ ```
243
+
244
+ This configuration can be handled as:
245
+
246
+ * "Application" and "Security" channels just tailing
247
+ * "HardwareEvent" channel read existing events before launching Fluentd
248
+
249
+ ###### Remoting access
250
+
251
+ `<subscribe>` section supports remoting access parameters:
252
+
253
+ * `remote_server`
254
+ * `remote_domain`
255
+ * `remote_username`
256
+ * `remote_password`
257
+
258
+ These parameters are only in `<subscribe>` directive.
259
+
260
+ Note that before using this feature, remoting access users should belong to "Event Log Readers" group:
261
+
262
+ ```console
263
+ > net localgroup "Event Log Readers" <domain\username> /add
264
+ ```
265
+
266
+ And then, users also should set up their remote box's Firewall configuration:
267
+
268
+ ```console
269
+ > netsh advfirewall firewall set rule group="Remote Event Log Management" new enable=yes
270
+ ```
271
+
272
+ As a security best practices, remoting access account _should not be administrator account_.
273
+
274
+ For graphical instructions, please refer to [Preconfigure a Machine to Collect Remote Windows Events | Sumo Logic](https://help.sumologic.com/03Send-Data/Sources/01Sources-for-Installed-Collectors/Remote-Windows-Event-Log-Source/Preconfigure-a-Machine-to-Collect-Remote-Windows-Events) document for example.
275
+
276
+ ##### Available keys
277
+
278
+ This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
279
+
280
+ |key|
281
+ |:----- |
282
+ |`ProviderName` |
283
+ |`ProviderGuid` |
284
+ |`EventID` |
285
+ |`Qualifiers` |
286
+ |`Level` |
287
+ |`Task` |
288
+ |`Opcode` |
289
+ |`Keywords` |
290
+ |`TimeCreated` |
291
+ |`EventRecordId` |
292
+ |`ActivityID` |
293
+ |`RelatedActivityID`|
294
+ |`ProcessID` |
295
+ |`ThreadID` |
296
+ |`Channel` |
297
+ |`Computer` |
298
+ |`UserID` |
299
+ |`Version` |
300
+ |`Description` |
301
+ |`EventData` |
302
+
303
+ ##### `parse_description` details
304
+
305
+ Here is an example with `parse_description true`.
306
+
307
+ ```
308
+ {
309
+ "ProviderName": "Microsoft-Windows-Security-Auditing",
310
+ "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
311
+ "EventID": "4798",
312
+ "Qualifiers": "",
313
+ "Level": "0",
314
+ "Task": "13824",
315
+ "Opcode": "0",
316
+ "Keywords": "0x8020000000000000",
317
+ "TimeCreated": "2019-06-19T03:10:01.982940200Z",
318
+ "EventRecordID": "87028",
319
+ "ActivityID": "",
320
+ "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
321
+ "ThreadID": "16888",
322
+ "Channel": "Security",
323
+ "Computer": "DESKTOP-TEST",
324
+ "UserID": "",
325
+ "Version": "0",
326
+ "Description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-X-Y-Z\r\n\tAccount Name:\t\tDESKTOP-TEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ0\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tDESKTOP-TEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0xbac\r\n\tProcess Name:\t\tC:\\Windows\\System32\\svchost.exe\r\n",
327
+ "EventData": [
328
+ "Administrator",
329
+ "DESKTOP-TEST",
330
+ "S-XXX-YYY-ZZZ",
331
+ "S-X-Y-Z",
332
+ "DESKTOP-TEST$",
333
+ "WORKGROUP",
334
+ "0x3e7",
335
+ "0xbac",
336
+ "C:\\Windows\\System32\\svchost.exe"
337
+ ]
338
+ }
339
+ ```
340
+
341
+ This record is transformed to
342
+
343
+ ```
344
+ {
345
+ "ProviderName": "Microsoft-Windows-Security-Auditing",
346
+ "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
347
+ "EventID": "4798",
348
+ "Qualifiers": "",
349
+ "Level": "0",
350
+ "Task": "13824",
351
+ "Opcode": "0",
352
+ "Keywords": "0x8020000000000000",
353
+ "TimeCreated": "2019-06-19T03:10:01.982940200Z",
354
+ "EventRecordID": "87028",
355
+ "ActivityID": "",
356
+ "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
357
+ "ThreadID": "16888",
358
+ "Channel": "Security",
359
+ "Computer": "DESKTOP-TEST",
360
+ "UserID": "",
361
+ "Version": "0",
362
+ "DescriptionTitle": "A user's local group membership was enumerated.",
363
+ "subject.security_id": "S-X-Y-Z",
364
+ "subject.account_name": "DESKTOP-TEST$",
365
+ "subject.account_domain": "WORKGROUP",
366
+ "subject.logon_id": "0x3e7",
367
+ "user.security_id": "S-XXX-YYY-ZZZ",
368
+ "user.account_name": "Administrator",
369
+ "user.account_domain": "DESKTOP-TEST",
370
+ "process_information.process_id": "0xbac",
371
+ "process_information.process_name": "C:\\Windows\\System32\\svchost.exe"
372
+ }
373
+ ```
374
+
375
+ NOTE: This feature assumes `description` field has following formats:
376
+
377
+ - group delimiter: `\r\n\r\n`
378
+ - record delimiter: `\r\n\t`
379
+ - field delimiter: `\t\t`
380
+
381
+ If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
382
+
383
+ ## Copyright
384
+ ### Copyright
385
+ Copyright(C) 2014- @okahashi117
386
+ ### License
387
+ Apache License, Version 2.0