fluent-plugin-windows-eventlog 0.8.3 → 0.9.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/workflows/unit-test.yml +1 -1
- data/CHANGELOG.md +6 -0
- data/README.md +23 -126
- data/fluent-plugin-winevtlog.gemspec +2 -2
- data/in_windows_eventlog(old).md +120 -0
- data/lib/fluent/plugin/in_windows_eventlog.rb +1 -1
- data/lib/fluent/plugin/in_windows_eventlog2.rb +69 -17
- data/test/plugin/test_in_windows_eventlog2.rb +111 -2
- metadata +9 -8
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b003ba2ba48568f0aef6dbc9cb8181990d6c0c7aaf519c46716575f647bcc1f2
|
4
|
+
data.tar.gz: 6031954f7b32595d6c2ab9be37eb464c9bf5b0797ca5cd5eb4838b78adb29d36
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: fc3ae080dd4fb87945a4ec803204e3d2a2753e4daeaa3ef316df6788514799c07d47b80b561755371d657cd151b60404a298a7ed8fa6887e422f1dd083421cb9
|
7
|
+
data.tar.gz: 941626f0a4e9656682eaf44b570590080f8aa9437307dbfd02c22efcd1355dbf8523d15e8531745b1a9a52c962847e874dae49addf818ca2b775011c2316ab36
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,9 @@
|
|
1
|
+
# Release v0.9.0 - 2024/08/02
|
2
|
+
* in_windows_eventlog2: Enable expanding user names from SID and add `preserve_sid_on_hash` option
|
3
|
+
* in_windows_eventlog2: Add Delimiter and Casing options for parsing
|
4
|
+
* in_windows_eventlog2: Not to load WinevtXMLparser by default
|
5
|
+
* in_windows_eventlog2: Make it possible to work without Nokogiri
|
6
|
+
|
1
7
|
# Release v0.8.3 - 2023/01/19
|
2
8
|
* Permit using nokogiri 1.14.0
|
3
9
|
|
data/README.md
CHANGED
@@ -6,131 +6,24 @@
|
|
6
6
|
|
7
7
|
[Fluentd](https://www.fluentd.org/) plugin to read the Windows Event Log.
|
8
8
|
|
9
|
-
|
10
|
-
ridk exec gem install fluent-plugin-windows-eventlog
|
11
|
-
|
12
|
-
## Configuration
|
13
|
-
|
14
|
-
### in_windows_eventlog
|
15
|
-
|
16
|
-
Check [in_windows_eventlog2](https://github.com/fluent/fluent-plugin-windows-eventlog#in_windows_eventlog2) first. `in_windows_eventlog` will be replaced with `in_windows_eventlog2`.
|
17
|
-
|
18
|
-
fluentd Input plugin for the Windows Event Log using old Windows Event Logging API
|
19
|
-
|
20
|
-
<source>
|
21
|
-
@type windows_eventlog
|
22
|
-
@id windows_eventlog
|
23
|
-
channels application,system
|
24
|
-
read_interval 2
|
25
|
-
tag winevt.raw
|
26
|
-
<storage>
|
27
|
-
@type local # @type local is the default.
|
28
|
-
persistent true # default is true. Set to false to use in-memory storage.
|
29
|
-
path ./tmp/storage.json # This is required when persistent is true.
|
30
|
-
# Or, please consider using <system> section's `root_dir` parameter.
|
31
|
-
</storage>
|
32
|
-
</source>
|
33
|
-
|
34
|
-
#### parameters
|
35
|
-
|
36
|
-
|name | description |
|
37
|
-
|:----- |:----- |
|
38
|
-
|`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
|
39
|
-
|`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
|
40
|
-
|`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
|
41
|
-
|`from_encoding` | (option) Input character encoding. `nil` as default.|
|
42
|
-
|`encoding` | (option) Output character encoding. `nil` as default.|
|
43
|
-
|`read_from_head` | (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
|
44
|
-
|`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
|
45
|
-
|`parse_description`| (option) parse `description` field and set parsed result into the record. `parse` and `string_inserts` fields are removed|
|
46
|
-
|
47
|
-
##### Available keys
|
48
|
-
|
49
|
-
This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
|
50
|
-
|
51
|
-
|key|
|
52
|
-
|:----- |
|
53
|
-
|`record_number` |
|
54
|
-
|`time_generated`|
|
55
|
-
|`time_written` |
|
56
|
-
|`event_id` |
|
57
|
-
|`event_type` |
|
58
|
-
|`event_category`|
|
59
|
-
|`source_name` |
|
60
|
-
|`computer_name` |
|
61
|
-
|`user` |
|
62
|
-
|`description` |
|
63
|
-
|`string_inserts`|
|
64
|
-
|
65
|
-
##### `parse_description` details
|
9
|
+
This repository contains 2 Fluentd plugins:
|
66
10
|
|
67
|
-
|
11
|
+
* in_windows_eventlog
|
12
|
+
* in_windows_eventlog2
|
68
13
|
|
69
|
-
|
70
|
-
{
|
71
|
-
"channel": "security",
|
72
|
-
"record_number": "91698",
|
73
|
-
"time_generated": "2017-08-29 20:12:29 +0000",
|
74
|
-
"time_written": "2017-08-29 20:12:29 +0000",
|
75
|
-
"event_id": "4798",
|
76
|
-
"event_type": "audit_success",
|
77
|
-
"event_category": "13824",
|
78
|
-
"source_name": "Microsoft-Windows-Security-Auditing",
|
79
|
-
"computer_name": "TEST",
|
80
|
-
"user": "",
|
81
|
-
"description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-XXX\r\n\tAccount Name:\t\tTEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x7dc\r\n\tProcess Name:\t\tC:\\Windows\\System32\\LogonUI.exe\r\n",
|
82
|
-
"string_inserts": [
|
83
|
-
"Administrator",
|
84
|
-
"TEST",
|
85
|
-
"S-XXX-YYY-ZZZ",
|
86
|
-
"S-XXX",
|
87
|
-
"TEST$",
|
88
|
-
"WORKGROUP",
|
89
|
-
"0x3e7",
|
90
|
-
"0x7dc",
|
91
|
-
"C:\\Windows\\System32\\LogonUI.exe"
|
92
|
-
]
|
93
|
-
}
|
94
|
-
```
|
14
|
+
The former one is obsolete, please don't use in newly deployment.
|
95
15
|
|
96
|
-
This
|
16
|
+
This document describes about the later one.
|
17
|
+
If you want to know about the obsolete one, please see [in_windows_eventlog(old).md](in_windows_eventlog(old).md)
|
97
18
|
|
98
|
-
|
99
|
-
|
100
|
-
"channel": "security",
|
101
|
-
"record_number": "91698",
|
102
|
-
"time_generated": "2017-08-29 20:12:29 +0000",
|
103
|
-
"time_written": "2017-08-29 20:12:29 +0000",
|
104
|
-
"event_id": "4798",
|
105
|
-
"event_type": "audit_success",
|
106
|
-
"event_category": "13824",
|
107
|
-
"source_name": "Microsoft-Windows-Security-Auditing",
|
108
|
-
"computer_name": "TEST",
|
109
|
-
"user": "",
|
110
|
-
"description_title": "A user's local group membership was enumerated.",
|
111
|
-
"subject.security_id": "S-XXX",
|
112
|
-
"subject.account_name": "TEST$",
|
113
|
-
"subject.account_domain": "WORKGROUP",
|
114
|
-
"subject.logon_id": "0x3e7",
|
115
|
-
"user.security_id": "S-XXX-YYY-ZZZ",
|
116
|
-
"user.account_name": "Administrator",
|
117
|
-
"user.account_domain": "TEST",
|
118
|
-
"process_information.process_id": "0x7dc",
|
119
|
-
"process_information.process_name": "C:\\Windows\\System32\\LogonUI.exe\r\n"
|
120
|
-
}
|
121
|
-
```
|
122
|
-
|
123
|
-
NOTE: This feature assumes `description` field has following formats:
|
124
|
-
|
125
|
-
- group delimiter: `\r\n\r\n`
|
126
|
-
- record delimiter: `\r\n\t`
|
127
|
-
- field delimiter: `\t\t`
|
19
|
+
## Installation
|
20
|
+
ridk exec gem install fluent-plugin-windows-eventlog
|
128
21
|
|
129
|
-
|
22
|
+
## in_windows_eventlog2
|
130
23
|
|
131
|
-
|
24
|
+
Fluentd Input plugin for the Windows Event Log using newer Windows Event Logging API. This is successor to [in_windows_eventlog](in_windows_eventlog(old).md). See also [this slide](https://www.slideshare.net/cosmo0920/fluentd-meetup-2019) for the details of `in_windows_eventlog2` plugin.
|
132
25
|
|
133
|
-
|
26
|
+
## Configuration
|
134
27
|
|
135
28
|
<source>
|
136
29
|
@type windows_eventlog2
|
@@ -142,6 +35,7 @@ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging
|
|
142
35
|
render_as_xml false # default is false.
|
143
36
|
rate_limit 200 # default is -1(Winevt::EventLog::Subscribe::RATE_INFINITE).
|
144
37
|
# preserve_qualifiers_on_hash true # default is false.
|
38
|
+
# preserve_sid_on_hash false # default is true.
|
145
39
|
# read_all_channels false # default is false.
|
146
40
|
# description_locale en_US # default is nil. It means that system locale is used for obtaining description.
|
147
41
|
# refresh_subscription_interval 10m # default is nil. It specifies refresh interval for channel subscriptions.
|
@@ -176,31 +70,34 @@ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging
|
|
176
70
|
|
177
71
|
**NOTE:** When `render_as_xml` as `true`, `fluent-plugin-parser-winevt_xml` plugin should be needed to parse XML rendered Windows EventLog string.
|
178
72
|
|
179
|
-
**NOTE:** If you encountered CPU spike due to massively huge EventLog channel, `rate_limit` parameter may help you.
|
73
|
+
**NOTE:** If you encountered CPU or memory spike due to massively huge EventLog channel, `rate_limit` parameter may help you. This paramter can handle the multiples of 10 or -1(`Winevt::EventLog::Subscribe::RATE_INFINITE`).
|
180
74
|
|
181
|
-
|
75
|
+
### parameters
|
182
76
|
|
183
77
|
|name | description |
|
184
78
|
|:----- |:----- |
|
185
79
|
|`channels` | (option) No default value just empty array, but 'application' is used as default due to backward compatibility. One or more of {'application', 'system', 'setup', 'security'} and other evtx, which is the brand new Windows XML Event Log (EVTX) format since Windows Vista, formatted channels. Theoritically, `in_windows_ventlog2` may read all of channels except for debug and analytical typed channels. If you want to read 'setup' or 'security' logs or some privileged channels, you must launch fluentd with administrator privileges.|
|
186
80
|
|`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
|
187
81
|
|`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
|
188
|
-
|`from_encoding` | (option) Input character encoding. `nil` as default.|
|
189
82
|
|`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
|
190
83
|
|`<parse>` | Setting for `parser` plugin for parsing raw XML EventLog records. |
|
191
84
|
|`parse_description`| (option) parse `description` field and set parsed result into the record. `Description` and `EventData` fields are removed|
|
85
|
+
|`description_key_delimiter`| (option) (Only applicable if parse_description is true) Change the character placed between the parent_key and key. Set the value to "" for no delimiter. Defaults to `.` .|
|
86
|
+
|`description_word_delimiter`| (option) (Only applicable if parse_description is true) Change the character placed between each word of the key. Set the value to "" for no delimiter. Defaults to `_` .|
|
87
|
+
|`downcase_description_keys`| (option) (Only applicable if parse_description is true) Specify whether to downcase the keys that are parsed from the Description. Defaults to `true`.|
|
192
88
|
|`read_from_head` | **Deprecated** (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
|
193
89
|
|`read_existing_events` | (option) Read the entries which already exist before fluentd is started. Defaults to `false`.|
|
194
90
|
|`render_as_xml` | (option) Render Windows EventLog as XML or Ruby Hash object directly. Defaults to `false`.|
|
195
|
-
|`rate_limit` | (option) Specify rate limit to consume EventLog. Default is `Winevt::EventLog::Subscribe::RATE_INFINITE
|
91
|
+
|`rate_limit` | (option) Specify rate limit to consume EventLog. This is the approximate maximum number of records read per second. If more than this value is read in a second, this stops reading and waits until the next `read_interval`. This value must be a multiple of 10. Default is `-1`(`Winevt::EventLog::Subscribe::RATE_INFINITE`) and this means there is no upper limit. The log flow rate for setting this is approximately as follows: `rate_limit / read_interval [logs/second]` |
|
196
92
|
|`preserve_qualifiers_on_hash` | (option) When set up it as true, this plugin preserves "Qualifiers" and "EventID" keys. When set up it as false, this plugin calculates actual "EventID" from "Qualifiers" and removing "Qualifiers". Default is `false`.|
|
93
|
+
|`preserve_sid_on_hash` | (option) When set up it as true, this plugin preserves "UserID" key which includes SID of users. When set up it as false, this plugin just eliminates "UserID". This option is only effective for hash format (render_as_xml false) . Default is `true`.|
|
197
94
|
|`read_all_channels`| (option) Read from all channels. Default is `false`|
|
198
95
|
|`description_locale`| (option) Specify description locale. Default is `nil`. See also: [Supported locales](https://github.com/fluent-plugins-nursery/winevt_c#multilingual-description) |
|
199
96
|
|`refresh_subscription_interval`|(option) It specifies refresh interval for channel subscriptions. Default is `nil`.|
|
200
97
|
|`event_query`|(option) It specifies query for deny/allow/filter events with XPath 1.0 or structured XML query. Default is `"*"` (retrieving all events).|
|
201
98
|
|`<subscribe>` | Setting for subscribe channels. |
|
202
99
|
|
203
|
-
|
100
|
+
#### subscribe section
|
204
101
|
|
205
102
|
|name | description |
|
206
103
|
|:----- |:----- |
|
@@ -246,7 +143,7 @@ This configuration can be handled as:
|
|
246
143
|
* "Application" and "Security" channels just tailing
|
247
144
|
* "HardwareEvent" channel read existing events before launching Fluentd
|
248
145
|
|
249
|
-
|
146
|
+
##### Remoting access
|
250
147
|
|
251
148
|
`<subscribe>` section supports remoting access parameters:
|
252
149
|
|
@@ -273,7 +170,7 @@ As a security best practices, remoting access account _should not be administrat
|
|
273
170
|
|
274
171
|
For graphical instructions, please refer to [Preconfigure a Machine to Collect Remote Windows Events | Sumo Logic](https://help.sumologic.com/03Send-Data/Sources/01Sources-for-Installed-Collectors/Remote-Windows-Event-Log-Source/Preconfigure-a-Machine-to-Collect-Remote-Windows-Events) document for example.
|
275
172
|
|
276
|
-
|
173
|
+
#### Available keys
|
277
174
|
|
278
175
|
This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
|
279
176
|
|
@@ -300,7 +197,7 @@ This plugin reads the following fields from Windows Event Log entries. Use the `
|
|
300
197
|
|`Description` |
|
301
198
|
|`EventData` |
|
302
199
|
|
303
|
-
|
200
|
+
#### `parse_description` details
|
304
201
|
|
305
202
|
Here is an example with `parse_description true`.
|
306
203
|
|
@@ -4,7 +4,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
4
4
|
|
5
5
|
Gem::Specification.new do |spec|
|
6
6
|
spec.name = "fluent-plugin-windows-eventlog"
|
7
|
-
spec.version = "0.
|
7
|
+
spec.version = "0.9.0"
|
8
8
|
spec.authors = ["okahashi117", "Hiroshi Hatake", "Masahiro Nakagawa"]
|
9
9
|
spec.email = ["naruki_okahashi@jbat.co.jp", "cosmo0920.oucc@gmail.com", "repeatedly@gmail.com"]
|
10
10
|
spec.summary = %q{Fluentd Input plugin to read windows event log.}
|
@@ -24,5 +24,5 @@ Gem::Specification.new do |spec|
|
|
24
24
|
spec.add_development_dependency "fluent-plugin-parser-winevt_xml", ">= 0.1.2"
|
25
25
|
spec.add_runtime_dependency "fluentd", [">= 0.14.12", "< 2"]
|
26
26
|
spec.add_runtime_dependency "win32-eventlog"
|
27
|
-
spec.add_runtime_dependency "winevt_c", ">= 0.
|
27
|
+
spec.add_runtime_dependency "winevt_c", ">= 0.11.0"
|
28
28
|
end
|
@@ -0,0 +1,120 @@
|
|
1
|
+
# in_windows_eventlog (old)
|
2
|
+
|
3
|
+
This is a document about `in_windows_eventlog`, which is the old version of `in_windows_eventlog2`.
|
4
|
+
|
5
|
+
Please use `in_windows_eventlog2` since this will be replaced with it.
|
6
|
+
|
7
|
+
## Configuration
|
8
|
+
|
9
|
+
fluentd Input plugin for the Windows Event Log using old Windows Event Logging API
|
10
|
+
|
11
|
+
<source>
|
12
|
+
@type windows_eventlog
|
13
|
+
@id windows_eventlog
|
14
|
+
channels application,system
|
15
|
+
read_interval 2
|
16
|
+
tag winevt.raw
|
17
|
+
<storage>
|
18
|
+
@type local # @type local is the default.
|
19
|
+
persistent true # default is true. Set to false to use in-memory storage.
|
20
|
+
path ./tmp/storage.json # This is required when persistent is true.
|
21
|
+
# Or, please consider using <system> section's `root_dir` parameter.
|
22
|
+
</storage>
|
23
|
+
</source>
|
24
|
+
|
25
|
+
### parameters
|
26
|
+
|
27
|
+
|name | description |
|
28
|
+
|:----- |:----- |
|
29
|
+
|`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
|
30
|
+
|`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
|
31
|
+
|`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
|
32
|
+
|`from_encoding` | (option) Input character encoding. `nil` as default.|
|
33
|
+
|`encoding` | (option) Output character encoding. `nil` as default.|
|
34
|
+
|`read_from_head` | (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
|
35
|
+
|`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
|
36
|
+
|`parse_description`| (option) parse `description` field and set parsed result into the record. `parse` and `string_inserts` fields are removed|
|
37
|
+
|
38
|
+
### Available keys
|
39
|
+
|
40
|
+
This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
|
41
|
+
|
42
|
+
|key|
|
43
|
+
|:----- |
|
44
|
+
|`record_number` |
|
45
|
+
|`time_generated`|
|
46
|
+
|`time_written` |
|
47
|
+
|`event_id` |
|
48
|
+
|`event_type` |
|
49
|
+
|`event_category`|
|
50
|
+
|`source_name` |
|
51
|
+
|`computer_name` |
|
52
|
+
|`user` |
|
53
|
+
|`description` |
|
54
|
+
|`string_inserts`|
|
55
|
+
|
56
|
+
### `parse_description` details
|
57
|
+
|
58
|
+
Here is an example with `parse_description true`.
|
59
|
+
|
60
|
+
```
|
61
|
+
{
|
62
|
+
"channel": "security",
|
63
|
+
"record_number": "91698",
|
64
|
+
"time_generated": "2017-08-29 20:12:29 +0000",
|
65
|
+
"time_written": "2017-08-29 20:12:29 +0000",
|
66
|
+
"event_id": "4798",
|
67
|
+
"event_type": "audit_success",
|
68
|
+
"event_category": "13824",
|
69
|
+
"source_name": "Microsoft-Windows-Security-Auditing",
|
70
|
+
"computer_name": "TEST",
|
71
|
+
"user": "",
|
72
|
+
"description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-XXX\r\n\tAccount Name:\t\tTEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x7dc\r\n\tProcess Name:\t\tC:\\Windows\\System32\\LogonUI.exe\r\n",
|
73
|
+
"string_inserts": [
|
74
|
+
"Administrator",
|
75
|
+
"TEST",
|
76
|
+
"S-XXX-YYY-ZZZ",
|
77
|
+
"S-XXX",
|
78
|
+
"TEST$",
|
79
|
+
"WORKGROUP",
|
80
|
+
"0x3e7",
|
81
|
+
"0x7dc",
|
82
|
+
"C:\\Windows\\System32\\LogonUI.exe"
|
83
|
+
]
|
84
|
+
}
|
85
|
+
```
|
86
|
+
|
87
|
+
This record is transformed to
|
88
|
+
|
89
|
+
```
|
90
|
+
{
|
91
|
+
"channel": "security",
|
92
|
+
"record_number": "91698",
|
93
|
+
"time_generated": "2017-08-29 20:12:29 +0000",
|
94
|
+
"time_written": "2017-08-29 20:12:29 +0000",
|
95
|
+
"event_id": "4798",
|
96
|
+
"event_type": "audit_success",
|
97
|
+
"event_category": "13824",
|
98
|
+
"source_name": "Microsoft-Windows-Security-Auditing",
|
99
|
+
"computer_name": "TEST",
|
100
|
+
"user": "",
|
101
|
+
"description_title": "A user's local group membership was enumerated.",
|
102
|
+
"subject.security_id": "S-XXX",
|
103
|
+
"subject.account_name": "TEST$",
|
104
|
+
"subject.account_domain": "WORKGROUP",
|
105
|
+
"subject.logon_id": "0x3e7",
|
106
|
+
"user.security_id": "S-XXX-YYY-ZZZ",
|
107
|
+
"user.account_name": "Administrator",
|
108
|
+
"user.account_domain": "TEST",
|
109
|
+
"process_information.process_id": "0x7dc",
|
110
|
+
"process_information.process_name": "C:\\Windows\\System32\\LogonUI.exe\r\n"
|
111
|
+
}
|
112
|
+
```
|
113
|
+
|
114
|
+
NOTE: This feature assumes `description` field has following formats:
|
115
|
+
|
116
|
+
- group delimiter: `\r\n\r\n`
|
117
|
+
- record delimiter: `\r\n\t`
|
118
|
+
- field delimiter: `\t\t`
|
119
|
+
|
120
|
+
If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
|
@@ -49,7 +49,7 @@ module Fluent::Plugin
|
|
49
49
|
end
|
50
50
|
|
51
51
|
def configure(conf)
|
52
|
-
log.warn "in_windows_eventlog is deprecated. It will be removed in the future version."
|
52
|
+
log.warn "in_windows_eventlog is deprecated. It will be removed in the future version. Please use in_windows_eventlog2 bundled with this plugin instead."
|
53
53
|
super
|
54
54
|
@chs = @channels.map {|ch| ch.strip.downcase }.uniq
|
55
55
|
if @chs.empty?
|
@@ -1,10 +1,16 @@
|
|
1
1
|
require 'winevt'
|
2
2
|
require 'fluent/plugin/input'
|
3
3
|
require 'fluent/plugin'
|
4
|
-
require_relative 'bookmark_sax_parser'
|
5
4
|
|
6
5
|
module Fluent::Plugin
|
7
6
|
class WindowsEventLog2Input < Input
|
7
|
+
begin
|
8
|
+
require_relative 'bookmark_sax_parser'
|
9
|
+
@@bookmark_parser_avaiable = true
|
10
|
+
rescue LoadError
|
11
|
+
@@bookmark_parser_avaiable = false
|
12
|
+
end
|
13
|
+
|
8
14
|
Fluent::Plugin.register_input('windows_eventlog2', self)
|
9
15
|
|
10
16
|
class ReconnectError < Fluent::UnrecoverableError; end
|
@@ -29,6 +35,7 @@ module Fluent::Plugin
|
|
29
35
|
"Channel" => ["Channel", :string],
|
30
36
|
"Computer" => ["Computer", :string],
|
31
37
|
"UserID" => ["UserID", :string],
|
38
|
+
"User" => ["User", :string],
|
32
39
|
"Version" => ["Version", :string],
|
33
40
|
"Description" => ["Description", :string],
|
34
41
|
"EventData" => ["EventData", :array]}
|
@@ -40,9 +47,13 @@ module Fluent::Plugin
|
|
40
47
|
config_param :read_from_head, :bool, default: false, deprecated: "Use `read_existing_events' instead."
|
41
48
|
config_param :read_existing_events, :bool, default: false
|
42
49
|
config_param :parse_description, :bool, default: false
|
50
|
+
config_param :description_key_delimiter, :string, default: "."
|
51
|
+
config_param :description_word_delimiter, :string, default: "_"
|
52
|
+
config_param :downcase_description_keys, :bool, default: true
|
43
53
|
config_param :render_as_xml, :bool, default: false
|
44
54
|
config_param :rate_limit, :integer, default: Winevt::EventLog::Subscribe::RATE_INFINITE
|
45
55
|
config_param :preserve_qualifiers_on_hash, :bool, default: false
|
56
|
+
config_param :preserve_sid_on_hash, :bool, default: true
|
46
57
|
config_param :read_all_channels, :bool, default: false
|
47
58
|
config_param :description_locale, :string, default: nil
|
48
59
|
config_param :refresh_subscription_interval, :time, default: nil
|
@@ -64,7 +75,7 @@ module Fluent::Plugin
|
|
64
75
|
end
|
65
76
|
|
66
77
|
config_section :parse do
|
67
|
-
config_set_default :@type, '
|
78
|
+
config_set_default :@type, 'windows_eventlog2_dummy'
|
68
79
|
config_set_default :estimate_current_event, false
|
69
80
|
end
|
70
81
|
|
@@ -122,7 +133,12 @@ module Fluent::Plugin
|
|
122
133
|
@winevt_xml = false
|
123
134
|
@parser = nil
|
124
135
|
if @render_as_xml
|
125
|
-
|
136
|
+
parser_config = @parser_configs.first
|
137
|
+
if parser_config["@type"] == "windows_eventlog2_dummy"
|
138
|
+
@parser = parser_create(usage: "parse_xml", type: "winevt_xml", conf: conf.elements("parse").first)
|
139
|
+
else
|
140
|
+
@parser = parser_create
|
141
|
+
end
|
126
142
|
@winevt_xml = @parser.respond_to?(:winevt_xml?) && @parser.winevt_xml?
|
127
143
|
class << self
|
128
144
|
alias_method :on_notify, :on_notify_xml
|
@@ -142,6 +158,15 @@ module Fluent::Plugin
|
|
142
158
|
@keynames.delete('Qualifiers')
|
143
159
|
end
|
144
160
|
@keynames.delete('EventData') if @parse_description
|
161
|
+
if @render_as_xml && !@preserve_sid_on_hash
|
162
|
+
raise Fluent::ConfigError, "preserve_sid_on_hash is effective with Hash object rendering(render_as_xml as false)."
|
163
|
+
end
|
164
|
+
if @render_as_xml
|
165
|
+
@keynames.delete('User')
|
166
|
+
end
|
167
|
+
if !@render_as_xml && !@preserve_sid_on_hash
|
168
|
+
@keynames.delete('UserID')
|
169
|
+
end
|
145
170
|
|
146
171
|
locale = Winevt::EventLog::Locale.new
|
147
172
|
if @description_locale && unsupported_locale?(locale, @description_locale)
|
@@ -227,11 +252,16 @@ module Fluent::Plugin
|
|
227
252
|
end
|
228
253
|
|
229
254
|
def subscription(ch, read_existing_events, remote_session)
|
230
|
-
bookmarkXml = @bookmarks_storage.get(ch) || ""
|
231
255
|
bookmark = nil
|
232
|
-
|
233
|
-
|
256
|
+
bookmarkXml = @bookmarks_storage.get(ch) || ""
|
257
|
+
unless bookmarkXml.empty?
|
258
|
+
if bookmark_valid?(bookmarkXml, ch)
|
259
|
+
bookmark = Winevt::EventLog::Bookmark.new(bookmarkXml)
|
260
|
+
else
|
261
|
+
log.warn "This stored bookmark is incomplete for using. Referring `read_existing_events` parameter to subscribe: #{bookmarkXml}, channel: #{ch}"
|
262
|
+
end
|
234
263
|
end
|
264
|
+
|
235
265
|
subscribe = Winevt::EventLog::Subscribe.new
|
236
266
|
subscribe.read_existing_events = read_existing_events
|
237
267
|
begin
|
@@ -239,6 +269,9 @@ module Fluent::Plugin
|
|
239
269
|
if !@render_as_xml && @preserve_qualifiers_on_hash
|
240
270
|
subscribe.preserve_qualifiers = @preserve_qualifiers_on_hash
|
241
271
|
end
|
272
|
+
if !@render_as_xml && !@preserve_sid_on_hash
|
273
|
+
subscribe.preserve_sid = @preserve_sid_on_hash
|
274
|
+
end
|
242
275
|
rescue Winevt::EventLog::Query::Error => e
|
243
276
|
raise Fluent::ConfigError, "Invalid Bookmark XML is loaded. #{e}"
|
244
277
|
end
|
@@ -258,19 +291,26 @@ module Fluent::Plugin
|
|
258
291
|
end
|
259
292
|
end
|
260
293
|
|
261
|
-
def
|
262
|
-
|
294
|
+
def bookmark_valid?(bookmarkXml, channel)
|
295
|
+
if @@bookmark_parser_avaiable
|
296
|
+
bookmark_valid_strictly?(bookmarkXml, channel)
|
297
|
+
else
|
298
|
+
bookmarklist_is_not_empty?(bookmarkXml, channel)
|
299
|
+
end
|
300
|
+
end
|
263
301
|
|
302
|
+
def bookmark_valid_strictly?(bookmarkXml, channel)
|
264
303
|
evtxml = WinevtBookmarkDocument.new
|
265
304
|
parser = Nokogiri::XML::SAX::Parser.new(evtxml)
|
266
305
|
parser.parse(bookmarkXml)
|
267
306
|
result = evtxml.result
|
268
|
-
|
269
|
-
|
270
|
-
|
271
|
-
|
272
|
-
|
273
|
-
|
307
|
+
!result.empty? && (result[:channel].downcase == channel.downcase) && result[:is_current]
|
308
|
+
end
|
309
|
+
|
310
|
+
def bookmarklist_is_not_empty?(bookmarkXml, channel)
|
311
|
+
# Empty example: "<BookmarkList>\r\n</BookmarkList>"
|
312
|
+
# Not empty example: "<BookmarkList>\r\n <Bookmark Channel='Setup' RecordId='777' IsCurrent='true'/>\r\n</BookmarkList>"
|
313
|
+
bookmarkXml.include?("Channel")
|
274
314
|
end
|
275
315
|
|
276
316
|
def escape_channel(ch)
|
@@ -389,7 +429,7 @@ module Fluent::Plugin
|
|
389
429
|
elsif parent_key.nil?
|
390
430
|
record[to_key(key)] = value
|
391
431
|
else
|
392
|
-
k = "#{parent_key}
|
432
|
+
k = "#{parent_key}#{@description_key_delimiter}#{to_key(key)}"
|
393
433
|
record[k] = value
|
394
434
|
end
|
395
435
|
end
|
@@ -401,10 +441,22 @@ module Fluent::Plugin
|
|
401
441
|
end
|
402
442
|
|
403
443
|
def to_key(key)
|
404
|
-
key.downcase!
|
405
|
-
key.gsub!(' '.freeze,
|
444
|
+
key.downcase! if @downcase_description_keys
|
445
|
+
key.gsub!(' '.freeze, @description_word_delimiter)
|
406
446
|
key
|
407
447
|
end
|
408
448
|
####
|
409
449
|
end
|
450
|
+
|
451
|
+
class WindowsEventLog2DummyParser < Parser
|
452
|
+
Fluent::Plugin.register_parser('windows_eventlog2_dummy', self)
|
453
|
+
|
454
|
+
def configure(conf)
|
455
|
+
super
|
456
|
+
end
|
457
|
+
|
458
|
+
def parse(text)
|
459
|
+
raise NotImplementedError, "This is a dummy parser for the default setting and can not be used actually."
|
460
|
+
end
|
461
|
+
end
|
410
462
|
end
|
@@ -176,6 +176,18 @@ class WindowsEventLog2InputTest < Test::Unit::TestCase
|
|
176
176
|
])
|
177
177
|
end
|
178
178
|
end
|
179
|
+
|
180
|
+
test "default parser should not be WinevtXMLparser" do
|
181
|
+
# Because it is not a mandatory dependency.
|
182
|
+
d = create_driver CONFIG
|
183
|
+
assert_equal 1, d.instance._parsers.size
|
184
|
+
assert_not_equal "Fluent::Plugin::WinevtXMLparser", d.instance._parsers.values.first.class.name
|
185
|
+
end
|
186
|
+
|
187
|
+
test "parser should be WinevtXMLparser if render_as_xml is enabled" do
|
188
|
+
d = create_driver XML_RENDERING_CONFIG
|
189
|
+
assert_equal Fluent::Plugin::WinevtXMLparser, d.instance.instance_variable_get(:@parser).class
|
190
|
+
end
|
179
191
|
end
|
180
192
|
|
181
193
|
data("Japanese" => ["ja_JP", false],
|
@@ -226,6 +238,36 @@ DESC
|
|
226
238
|
assert_equal(expected, h)
|
227
239
|
end
|
228
240
|
|
241
|
+
def test_parse_desc_camelcase
|
242
|
+
d = create_driver(config_element("ROOT", "", {"tag" => "fluent.eventlog",
|
243
|
+
"parse_description" => true,
|
244
|
+
"description_key_delimiter" => "",
|
245
|
+
"description_word_delimiter" => "",
|
246
|
+
"downcase_description_keys" => false
|
247
|
+
}, [
|
248
|
+
config_element("storage", "", {
|
249
|
+
'@type' => 'local',
|
250
|
+
'persistent' => false
|
251
|
+
}),
|
252
|
+
]))
|
253
|
+
desc =<<-DESC
|
254
|
+
A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-X-Y-XX-WWWWWW-VVVV\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tDESKTOP-FLUENTTEST\r\n\tLogon ID:\t\t0x3185B1\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-X-Y-XX-WWWWWW-VVVV\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tDESKTOP-FLUENTTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x50b8\r\n\tProcess Name:\t\tC:\\msys64\\usr\\bin\\make.exe
|
255
|
+
DESC
|
256
|
+
h = {"Description" => desc}
|
257
|
+
expected = {"DescriptionTitle" => "A user's local group membership was enumerated.",
|
258
|
+
"SubjectSecurityID" => "S-X-Y-XX-WWWWWW-VVVV",
|
259
|
+
"SubjectAccountName" => "Administrator",
|
260
|
+
"SubjectAccountDomain" => "DESKTOP-FLUENTTEST",
|
261
|
+
"SubjectLogonID" => "0x3185B1",
|
262
|
+
"UserSecurityID" => "S-X-Y-XX-WWWWWW-VVVV",
|
263
|
+
"UserAccountName" => "Administrator",
|
264
|
+
"UserAccountDomain" => "DESKTOP-FLUENTTEST",
|
265
|
+
"ProcessInformationProcessID" => "0x50b8",
|
266
|
+
"ProcessInformationProcessName" => "C:\\msys64\\usr\\bin\\make.exe"}
|
267
|
+
d.instance.parse_desc(h)
|
268
|
+
assert_equal(expected, h)
|
269
|
+
end
|
270
|
+
|
229
271
|
def test_parse_privileges_description
|
230
272
|
d = create_driver
|
231
273
|
desc = ["Special privileges assigned to new logon.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-X-Y-ZZ\r\n\t",
|
@@ -284,7 +326,7 @@ DESC
|
|
284
326
|
end
|
285
327
|
|
286
328
|
def test_write
|
287
|
-
d = create_driver
|
329
|
+
d = create_driver XML_RENDERING_CONFIG
|
288
330
|
|
289
331
|
service = Fluent::Plugin::EventService.new
|
290
332
|
|
@@ -300,6 +342,7 @@ DESC
|
|
300
342
|
assert_equal("65500", record["EventID"])
|
301
343
|
assert_equal("4", record["Level"])
|
302
344
|
assert_equal("fluent-plugins", record["ProviderName"])
|
345
|
+
assert_false(record.has_key?("User"))
|
303
346
|
end
|
304
347
|
|
305
348
|
CONFIG_WITH_NON_EXISTENT_CHANNEL = config_element("ROOT", "", {
|
@@ -345,6 +388,7 @@ DESC
|
|
345
388
|
assert_equal("65500", record["EventID"])
|
346
389
|
assert_equal("4", record["Level"])
|
347
390
|
assert_equal("fluent-plugins", record["ProviderName"])
|
391
|
+
assert_true(record.has_key?("User"))
|
348
392
|
end
|
349
393
|
|
350
394
|
|
@@ -406,6 +450,7 @@ DESC
|
|
406
450
|
assert_equal("65500", record["EventID"])
|
407
451
|
assert_equal("4", record["Level"])
|
408
452
|
assert_equal("fluent-plugins", record["ProviderName"])
|
453
|
+
assert_true(record.has_key?("User"))
|
409
454
|
end
|
410
455
|
|
411
456
|
class HashRendered < self
|
@@ -451,7 +496,7 @@ DESC
|
|
451
496
|
service = Fluent::Plugin::EventService.new
|
452
497
|
subscribe = Winevt::EventLog::Subscribe.new
|
453
498
|
|
454
|
-
omit "
|
499
|
+
omit "subscribe.preserve_qualifiers? does not respond" unless subscribe.respond_to?(:preserve_qualifiers?)
|
455
500
|
|
456
501
|
d.run(expect_emits: 1) do
|
457
502
|
service.run
|
@@ -465,6 +510,70 @@ DESC
|
|
465
510
|
assert_true(record.has_key?("EventData"))
|
466
511
|
assert_true(record.has_key?("Qualifiers"))
|
467
512
|
end
|
513
|
+
|
514
|
+
def test_write_with_preserving_sid
|
515
|
+
require 'winevt'
|
516
|
+
|
517
|
+
d = create_driver(config_element("ROOT", "", {"tag" => "fluent.eventlog",
|
518
|
+
"render_as_xml" => false,
|
519
|
+
'preserve_sid_on_hash' => true
|
520
|
+
}, [
|
521
|
+
config_element("storage", "", {
|
522
|
+
'@type' => 'local',
|
523
|
+
'persistent' => false
|
524
|
+
}),
|
525
|
+
]))
|
526
|
+
|
527
|
+
service = Fluent::Plugin::EventService.new
|
528
|
+
subscribe = Winevt::EventLog::Subscribe.new
|
529
|
+
|
530
|
+
omit "subscribe.preserve_sid? does not respond" unless subscribe.respond_to?(:preserve_sid?)
|
531
|
+
|
532
|
+
d.run(expect_emits: 1) do
|
533
|
+
service.run
|
534
|
+
end
|
535
|
+
|
536
|
+
assert(d.events.length >= 1)
|
537
|
+
event = d.events.last
|
538
|
+
record = event.last
|
539
|
+
|
540
|
+
assert_true(record.has_key?("Description"))
|
541
|
+
assert_true(record.has_key?("EventData"))
|
542
|
+
assert_true(record.has_key?("UserID"))
|
543
|
+
assert_true(record.has_key?("User"))
|
544
|
+
end
|
545
|
+
|
546
|
+
def test_write_with_not_preserving_sid
|
547
|
+
require 'winevt'
|
548
|
+
|
549
|
+
d = create_driver(config_element("ROOT", "", {"tag" => "fluent.eventlog",
|
550
|
+
"render_as_xml" => false,
|
551
|
+
'preserve_sid_on_hash' => false
|
552
|
+
}, [
|
553
|
+
config_element("storage", "", {
|
554
|
+
'@type' => 'local',
|
555
|
+
'persistent' => false
|
556
|
+
}),
|
557
|
+
]))
|
558
|
+
|
559
|
+
service = Fluent::Plugin::EventService.new
|
560
|
+
subscribe = Winevt::EventLog::Subscribe.new
|
561
|
+
|
562
|
+
omit "subscribe.preserve_sid? does not respond" unless subscribe.respond_to?(:preserve_sid?)
|
563
|
+
|
564
|
+
d.run(expect_emits: 1) do
|
565
|
+
service.run
|
566
|
+
end
|
567
|
+
|
568
|
+
assert(d.events.length >= 1)
|
569
|
+
event = d.events.last
|
570
|
+
record = event.last
|
571
|
+
|
572
|
+
assert_true(record.has_key?("Description"))
|
573
|
+
assert_true(record.has_key?("EventData"))
|
574
|
+
assert_false(record.has_key?("UserID"))
|
575
|
+
assert_true(record.has_key?("User"))
|
576
|
+
end
|
468
577
|
end
|
469
578
|
|
470
579
|
class PersistBookMark < self
|
metadata
CHANGED
@@ -1,16 +1,16 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: fluent-plugin-windows-eventlog
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.9.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- okahashi117
|
8
8
|
- Hiroshi Hatake
|
9
9
|
- Masahiro Nakagawa
|
10
|
-
autorequire:
|
10
|
+
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2024-08-02 00:00:00.000000000 Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
16
|
name: bundler
|
@@ -128,14 +128,14 @@ dependencies:
|
|
128
128
|
requirements:
|
129
129
|
- - ">="
|
130
130
|
- !ruby/object:Gem::Version
|
131
|
-
version: 0.
|
131
|
+
version: 0.11.0
|
132
132
|
type: :runtime
|
133
133
|
prerelease: false
|
134
134
|
version_requirements: !ruby/object:Gem::Requirement
|
135
135
|
requirements:
|
136
136
|
- - ">="
|
137
137
|
- !ruby/object:Gem::Version
|
138
|
-
version: 0.
|
138
|
+
version: 0.11.0
|
139
139
|
description: Fluentd Input plugin to read windows event log.
|
140
140
|
email:
|
141
141
|
- naruki_okahashi@jbat.co.jp
|
@@ -154,6 +154,7 @@ files:
|
|
154
154
|
- README.md
|
155
155
|
- Rakefile
|
156
156
|
- fluent-plugin-winevtlog.gemspec
|
157
|
+
- in_windows_eventlog(old).md
|
157
158
|
- lib/fluent/plugin/bookmark_sax_parser.rb
|
158
159
|
- lib/fluent/plugin/in_windows_eventlog.rb
|
159
160
|
- lib/fluent/plugin/in_windows_eventlog2.rb
|
@@ -167,7 +168,7 @@ homepage: https://github.com/fluent/fluent-plugin-windows-eventlog
|
|
167
168
|
licenses:
|
168
169
|
- Apache-2.0
|
169
170
|
metadata: {}
|
170
|
-
post_install_message:
|
171
|
+
post_install_message:
|
171
172
|
rdoc_options: []
|
172
173
|
require_paths:
|
173
174
|
- lib
|
@@ -182,8 +183,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
182
183
|
- !ruby/object:Gem::Version
|
183
184
|
version: '0'
|
184
185
|
requirements: []
|
185
|
-
rubygems_version: 3.
|
186
|
-
signing_key:
|
186
|
+
rubygems_version: 3.4.1
|
187
|
+
signing_key:
|
187
188
|
specification_version: 4
|
188
189
|
summary: Fluentd Input plugin to read windows event log.
|
189
190
|
test_files:
|