fluent-plugin-windows-eventlog 0.2.2 → 0.3.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (19) hide show
  1. checksums.yaml +5 -5
  2. data/.gitignore +14 -14
  3. data/CHANGELOG.md +16 -0
  4. data/Gemfile +4 -4
  5. data/LICENSE.txt +203 -203
  6. data/README.md +279 -132
  7. data/Rakefile +10 -10
  8. data/appveyor.yml +24 -34
  9. data/fluent-plugin-winevtlog.gemspec +27 -25
  10. data/lib/fluent/plugin/in_windows_eventlog.rb +234 -234
  11. data/lib/fluent/plugin/in_windows_eventlog2.rb +169 -0
  12. data/lib/fluent/plugin/parser_winevt_xml.rb +34 -0
  13. data/test/data/eventlog.xml +1 -0
  14. data/test/generate-windows-event.rb +47 -47
  15. data/test/helper.rb +35 -32
  16. data/test/plugin/test_in_windows_eventlog2.rb +182 -0
  17. data/test/plugin/test_in_winevtlog.rb +48 -48
  18. data/test/plugin/test_parser_winevt_xml.rb +42 -0
  19. metadata +41 -4
data/README.md CHANGED
@@ -1,132 +1,279 @@
1
- # fluent-plugin-windows-eventlog
2
-
3
- ## Component
4
-
5
- ### fluentd Input plugin for the Windows Event Log
6
-
7
- [Fluentd](http://fluentd.org) plugin to read the Windows Event Log.
8
-
9
- ## Installation
10
- gem install fluent-plugin-windows-eventlog
11
-
12
- ## Configuration
13
-
14
- ### fluentd Input plugin for the Windows Event Log
15
-
16
- <source>
17
- @type windows_eventlog
18
- @id windows_eventlog
19
- channels application,system
20
- read_interval 2
21
- tag winevt.raw
22
- <storage>
23
- @type local # @type local is the default.
24
- persistent true # default is true. Set to false to use in-memory storage.
25
- path ./tmp/storage.json # This is required when persistent is true.
26
- # Or, please consider using <system> section's `root_dir` parameter.
27
- </storage>
28
- </source>
29
-
30
- ### parameters
31
-
32
- |name | description |
33
- |:----- |:----- |
34
- |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
35
- |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
36
- |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
37
- |`from_encoding` | (option) Input character encoding. `nil` as default.|
38
- |`encoding` | (option) Output character encoding. `nil` as default.|
39
- |`read_from_head` | (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
40
- |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
41
- |`parse_description`| (option) parse `description` field and set parsed result into the record. `parse` and `string_inserts` fields are removed|
42
-
43
- #### Available keys
44
-
45
- This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
46
-
47
- |key|
48
- |:----- |
49
- |`record_number` |
50
- |`time_generated`|
51
- |`time_written` |
52
- |`event_id` |
53
- |`event_type` |
54
- |`event_category`|
55
- |`source_name` |
56
- |`computer_name` |
57
- |`user` |
58
- |`description` |
59
- |`string_inserts`|
60
-
61
- #### `parse_description` details
62
-
63
- Here is an example with `parse_description true`.
64
-
65
- ```
66
- {
67
- "channel": "security",
68
- "record_number": "91698",
69
- "time_generated": "2017-08-29 20:12:29 +0000",
70
- "time_written": "2017-08-29 20:12:29 +0000",
71
- "event_id": "4798",
72
- "event_type": "audit_success",
73
- "event_category": "13824",
74
- "source_name": "Microsoft-Windows-Security-Auditing",
75
- "computer_name": "TEST",
76
- "user": "",
77
- "description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-XXX\r\n\tAccount Name:\t\tTEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x7dc\r\n\tProcess Name:\t\tC:\\Windows\\System32\\LogonUI.exe\r\n",
78
- "string_inserts": [
79
- "Administrator",
80
- "TEST",
81
- "S-XXX-YYY-ZZZ",
82
- "S-XXX",
83
- "TEST$",
84
- "WORKGROUP",
85
- "0x3e7",
86
- "0x7dc",
87
- "C:\\Windows\\System32\\LogonUI.exe"
88
- ]
89
- }
90
- ```
91
-
92
- This record is transformed to
93
-
94
- ```
95
- {
96
- "channel": "security",
97
- "record_number": "91698",
98
- "time_generated": "2017-08-29 20:12:29 +0000",
99
- "time_written": "2017-08-29 20:12:29 +0000",
100
- "event_id": "4798",
101
- "event_type": "audit_success",
102
- "event_category": "13824",
103
- "source_name": "Microsoft-Windows-Security-Auditing",
104
- "computer_name": "TEST",
105
- "user": "",
106
- "description_title": "A user's local group membership was enumerated.",
107
- "subject.security_id": "S-XXX",
108
- "subject.account_name": "TEST$",
109
- "subject.account_domain": "WORKGROUP",
110
- "subject.logon_id": "0x3e7",
111
- "user.security_id": "S-XXX-YYY-ZZZ",
112
- "user.account_name": "Administrator",
113
- "user.account_domain": "TEST",
114
- "process_information.process_id": "0x7dc",
115
- "process_information.process_name": "C:\\Windows\\System32\\LogonUI.exe\r\n"
116
- }
117
- ```
118
-
119
- NOTE: This feature assumes `description` field has following formats:
120
-
121
- - group delimiter: `\r\n\r\n`
122
- - record delimiter: `\r\n\t`
123
- - field delimiter: `\t\t`
124
-
125
- If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
126
-
127
- ## Copyright
128
- ### Copyright
129
- Copyright(C) 2014- @okahashi117
130
- ### License
131
- Apache License, Version 2.0
132
-
1
+ # fluent-plugin-windows-eventlog
2
+
3
+ ## Component
4
+
5
+ ### fluentd Input plugin for the Windows Event Log
6
+
7
+ [Fluentd](http://fluentd.org) plugin to read the Windows Event Log.
8
+
9
+ ## Installation
10
+ gem install fluent-plugin-windows-eventlog
11
+
12
+ ## Configuration
13
+
14
+ ### in_windows_eventlog
15
+
16
+ Check [in_windows_eventlog2](https://github.com/fluent/fluent-plugin-windows-eventlog#in_windows_eventlog2) first. `in_windows_eventlog` will be replaced with `in_windows_eventlog2`.
17
+
18
+ #### fluentd Input plugin for the Windows Event Log using old Windows Event Logging API
19
+
20
+ <source>
21
+ @type windows_eventlog
22
+ @id windows_eventlog
23
+ channels application,system
24
+ read_interval 2
25
+ tag winevt.raw
26
+ <storage>
27
+ @type local # @type local is the default.
28
+ persistent true # default is true. Set to false to use in-memory storage.
29
+ path ./tmp/storage.json # This is required when persistent is true.
30
+ # Or, please consider using <system> section's `root_dir` parameter.
31
+ </storage>
32
+ </source>
33
+
34
+ #### parameters
35
+
36
+ |name | description |
37
+ |:----- |:----- |
38
+ |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
39
+ |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
40
+ |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
41
+ |`from_encoding` | (option) Input character encoding. `nil` as default.|
42
+ |`encoding` | (option) Output character encoding. `nil` as default.|
43
+ |`read_from_head` | (option) Start to read the entries from the oldest, not from when fluentd is started. Defaults to `false`.|
44
+ |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
45
+ |`parse_description`| (option) parse `description` field and set parsed result into the record. `parse` and `string_inserts` fields are removed|
46
+
47
+ ##### Available keys
48
+
49
+ This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
50
+
51
+ |key|
52
+ |:----- |
53
+ |`record_number` |
54
+ |`time_generated`|
55
+ |`time_written` |
56
+ |`event_id` |
57
+ |`event_type` |
58
+ |`event_category`|
59
+ |`source_name` |
60
+ |`computer_name` |
61
+ |`user` |
62
+ |`description` |
63
+ |`string_inserts`|
64
+
65
+ ##### `parse_description` details
66
+
67
+ Here is an example with `parse_description true`.
68
+
69
+ ```
70
+ {
71
+ "channel": "security",
72
+ "record_number": "91698",
73
+ "time_generated": "2017-08-29 20:12:29 +0000",
74
+ "time_written": "2017-08-29 20:12:29 +0000",
75
+ "event_id": "4798",
76
+ "event_type": "audit_success",
77
+ "event_category": "13824",
78
+ "source_name": "Microsoft-Windows-Security-Auditing",
79
+ "computer_name": "TEST",
80
+ "user": "",
81
+ "description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-XXX\r\n\tAccount Name:\t\tTEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tTEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x7dc\r\n\tProcess Name:\t\tC:\\Windows\\System32\\LogonUI.exe\r\n",
82
+ "string_inserts": [
83
+ "Administrator",
84
+ "TEST",
85
+ "S-XXX-YYY-ZZZ",
86
+ "S-XXX",
87
+ "TEST$",
88
+ "WORKGROUP",
89
+ "0x3e7",
90
+ "0x7dc",
91
+ "C:\\Windows\\System32\\LogonUI.exe"
92
+ ]
93
+ }
94
+ ```
95
+
96
+ This record is transformed to
97
+
98
+ ```
99
+ {
100
+ "channel": "security",
101
+ "record_number": "91698",
102
+ "time_generated": "2017-08-29 20:12:29 +0000",
103
+ "time_written": "2017-08-29 20:12:29 +0000",
104
+ "event_id": "4798",
105
+ "event_type": "audit_success",
106
+ "event_category": "13824",
107
+ "source_name": "Microsoft-Windows-Security-Auditing",
108
+ "computer_name": "TEST",
109
+ "user": "",
110
+ "description_title": "A user's local group membership was enumerated.",
111
+ "subject.security_id": "S-XXX",
112
+ "subject.account_name": "TEST$",
113
+ "subject.account_domain": "WORKGROUP",
114
+ "subject.logon_id": "0x3e7",
115
+ "user.security_id": "S-XXX-YYY-ZZZ",
116
+ "user.account_name": "Administrator",
117
+ "user.account_domain": "TEST",
118
+ "process_information.process_id": "0x7dc",
119
+ "process_information.process_name": "C:\\Windows\\System32\\LogonUI.exe\r\n"
120
+ }
121
+ ```
122
+
123
+ NOTE: This feature assumes `description` field has following formats:
124
+
125
+ - group delimiter: `\r\n\r\n`
126
+ - record delimiter: `\r\n\t`
127
+ - field delimiter: `\t\t`
128
+
129
+ If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
130
+
131
+ ### in_windows_eventlog2
132
+
133
+ #### fluentd Input plugin for the Windows Event Log using newer Windows Event Logging API
134
+
135
+ <source>
136
+ @type windows_eventlog2
137
+ @id windows_eventlog2
138
+ channels application,system
139
+ read_interval 2
140
+ tag winevt.raw
141
+ <storage>
142
+ @type local # @type local is the default.
143
+ persistent true # default is true. Set to false to use in-memory storage.
144
+ path ./tmp/storage.json # This is required when persistent is true.
145
+ # Or, please consider using <system> section's `root_dir` parameter.
146
+ </storage>
147
+ <parse>
148
+ @type winevt_xml # @type winevt_xml is the default. winevt_xml and none parsers are supported for now.
149
+ </parse>
150
+ </source>
151
+
152
+ **NOTE:** in_windows_eventlog2 always handles EventLog records as UTF-8 characters. Users don't have to specify encoding related parameters and they are not provided.
153
+
154
+ **NOTE:** When `Description` contains error message such as `The message resource is present but the message was not found in the message table.`, eventlog's resource file (.mui) related to error generating event is something wrong. This issue is also occurred in built-in Windows Event Viewer which is the part of Windows management tool.
155
+
156
+ #### parameters
157
+
158
+ |name | description |
159
+ |:----- |:----- |
160
+ |`channels` | (option) 'application' as default. One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges.|
161
+ |`keys` | (option) A subset of [keys](#read-keys) to read. Defaults to all keys.|
162
+ |`read_interval` | (option) Read interval in seconds. 2 seconds as default.|
163
+ |`from_encoding` | (option) Input character encoding. `nil` as default.|
164
+ |`<storage>` | Setting for `storage` plugin for recording read position like `in_tail`'s `pos_file`.|
165
+ |`<parse>` | Setting for `parser` plugin for parsing raw XML EventLog records. |
166
+ |`parse_description`| (option) parse `description` field and set parsed result into the record. `Description` and `EventData` fields are removed|
167
+
168
+ ##### Available keys
169
+
170
+ This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
171
+
172
+ |key|
173
+ |:----- |
174
+ |`ProviderName` |
175
+ |`ProviderGuid` |
176
+ |`EventID` |
177
+ |`Qualifiers` |
178
+ |`Level` |
179
+ |`Task` |
180
+ |`Opcode` |
181
+ |`Keywords` |
182
+ |`TimeCreated` |
183
+ |`EventRecordId` |
184
+ |`ActivityID` |
185
+ |`RelatedActivityID`|
186
+ |`ProcessID` |
187
+ |`ThreadID` |
188
+ |`Channel` |
189
+ |`Computer` |
190
+ |`UserID` |
191
+ |`Version` |
192
+ |`Description` |
193
+ |`EventData` |
194
+
195
+ ##### `parse_description` details
196
+
197
+ Here is an example with `parse_description true`.
198
+
199
+ ```
200
+ {
201
+ "ProviderName": "Microsoft-Windows-Security-Auditing",
202
+ "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
203
+ "EventID": "4798",
204
+ "Qualifiers": "",
205
+ "Level": "0",
206
+ "Task": "13824",
207
+ "Opcode": "0",
208
+ "Keywords": "0x8020000000000000",
209
+ "TimeCreated": "2019-06-19T03:10:01.982940200Z",
210
+ "EventRecordID": "87028",
211
+ "ActivityID": "",
212
+ "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
213
+ "ThreadID": "16888",
214
+ "Channel": "Security",
215
+ "Computer": "DESKTOP-TEST",
216
+ "UserID": "",
217
+ "Version": "0",
218
+ "Description": "A user's local group membership was enumerated.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-X-Y-Z\r\n\tAccount Name:\t\tDESKTOP-TEST$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nUser:\r\n\tSecurity ID:\t\tS-XXX-YYY-ZZZ0\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tDESKTOP-TEST\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0xbac\r\n\tProcess Name:\t\tC:\\Windows\\System32\\svchost.exe\r\n",
219
+ "EventData": [
220
+ "Administrator",
221
+ "DESKTOP-TEST",
222
+ "S-XXX-YYY-ZZZ",
223
+ "S-X-Y-Z",
224
+ "DESKTOP-TEST$",
225
+ "WORKGROUP",
226
+ "0x3e7",
227
+ "0xbac",
228
+ "C:\\Windows\\System32\\svchost.exe"
229
+ ]
230
+ }
231
+ ```
232
+
233
+ This record is transformed to
234
+
235
+ ```
236
+ {
237
+ "ProviderName": "Microsoft-Windows-Security-Auditing",
238
+ "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
239
+ "EventID": "4798",
240
+ "Qualifiers": "",
241
+ "Level": "0",
242
+ "Task": "13824",
243
+ "Opcode": "0",
244
+ "Keywords": "0x8020000000000000",
245
+ "TimeCreated": "2019-06-19T03:10:01.982940200Z",
246
+ "EventRecordID": "87028",
247
+ "ActivityID": "",
248
+ "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
249
+ "ThreadID": "16888",
250
+ "Channel": "Security",
251
+ "Computer": "DESKTOP-TEST",
252
+ "UserID": "",
253
+ "Version": "0",
254
+ "DescriptionTitle": "A user's local group membership was enumerated.",
255
+ "subject.security_id": "S-X-Y-Z",
256
+ "subject.account_name": "DESKTOP-TEST$",
257
+ "subject.account_domain": "WORKGROUP",
258
+ "subject.logon_id": "0x3e7",
259
+ "user.security_id": "S-XXX-YYY-ZZZ",
260
+ "user.account_name": "Administrator",
261
+ "user.account_domain": "DESKTOP-TEST",
262
+ "process_information.process_id": "0xbac",
263
+ "process_information.process_name": "C:\\Windows\\System32\\svchost.exe"
264
+ }
265
+ ```
266
+
267
+ NOTE: This feature assumes `description` field has following formats:
268
+
269
+ - group delimiter: `\r\n\r\n`
270
+ - record delimiter: `\r\n\t`
271
+ - field delimiter: `\t\t`
272
+
273
+ If your `description` doesn't follow this format, the parsed result is only `description_title` field with same `description` content.
274
+
275
+ ## Copyright
276
+ ### Copyright
277
+ Copyright(C) 2014- @okahashi117
278
+ ### License
279
+ Apache License, Version 2.0
data/Rakefile CHANGED
@@ -1,10 +1,10 @@
1
- require "bundler/gem_tasks"
2
- require "rake/testtask"
3
-
4
- Rake::TestTask.new(:test) do |test|
5
- test.libs << 'lib' << 'test'
6
- test.pattern = 'test/**/test_*.rb'
7
- test.verbose = true
8
- end
9
-
10
- task default: :test
1
+ require "bundler/gem_tasks"
2
+ require "rake/testtask"
3
+
4
+ Rake::TestTask.new(:test) do |test|
5
+ test.libs << 'lib' << 'test'
6
+ test.pattern = 'test/**/test_*.rb'
7
+ test.verbose = true
8
+ end
9
+
10
+ task default: :test
data/appveyor.yml CHANGED
@@ -1,34 +1,24 @@
1
- version: '{build}'
2
-
3
- # init:
4
- # - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))
5
-
6
- install:
7
- - SET PATH=C:\Ruby%ruby_version%\bin;%PATH%
8
- - IF %ridk%==0 "%devkit%\\devkitvars.bat"
9
- - ruby --version
10
- - gem --version
11
- - IF %ridk%==1 ridk.cmd exec bundle install
12
- - IF %ridk%==0 bundle install
13
- build: off
14
- test_script:
15
- - bundle exec rake test
16
- # - bundle exec rake test TESTOPTS=-v
17
-
18
- branches:
19
- only:
20
- - master
21
-
22
- # https://www.appveyor.com/docs/installed-software/#ruby
23
- environment:
24
- matrix:
25
- - ruby_version: "23-x64"
26
- devkit: C:\Ruby23-x64\DevKit
27
- ridk: 0
28
- - ruby_version: "23"
29
- devkit: C:\Ruby23\DevKit
30
- ridk: 0
31
- - ruby_version: "24-x64"
32
- ridk: 1
33
- - ruby_version: "24"
34
- ridk: 1
1
+ version: '{build}'
2
+
3
+ # init:
4
+ # - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))
5
+
6
+ install:
7
+ - SET PATH=C:\Ruby%ruby_version%\bin;%PATH%
8
+ - ruby --version
9
+ - gem --version
10
+ - ridk.cmd exec bundle install
11
+ build: off
12
+ test_script:
13
+ - bundle exec rake test
14
+ # - bundle exec rake test TESTOPTS=-v
15
+
16
+ branches:
17
+ only:
18
+ - master
19
+
20
+ # https://www.appveyor.com/docs/installed-software/#ruby
21
+ environment:
22
+ matrix:
23
+ - ruby_version: "24-x64"
24
+ - ruby_version: "24"