logstash-codec-cef 6.1.2-java → 6.2.3-java
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +15 -1
- data/docs/index.asciidoc +452 -9
- data/lib/logstash/codecs/cef.rb +272 -111
- data/lib/logstash/codecs/cef/timestamp_normalizer.rb +115 -0
- data/logstash-codec-cef.gemspec +3 -1
- data/spec/codecs/cef/timestamp_normalizer_spec.rb +274 -0
- data/spec/codecs/cef_spec.rb +500 -306
- metadata +37 -7
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d45c024f0bdb71f6056b553e138fff8d57479d021ab3ba8b98ba69384bf9898f
|
|
4
|
+
data.tar.gz: 7ddb2bd1427fcf5c2ca91e762a326d0a9e73e17d911dbadaf4dffc4fdcfc50e8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4e1afac1d4c0c05fa8bc4db6f61063d591e47ba85623b8e2530cddbf599e4e2cc6e3b968f1effbe23bb196a1864111d70eee6b354d7404cabb6f27b1fa431be2
|
|
7
|
+
data.tar.gz: 872d25b0b8f8b2aa3f2e884794df1afd614cb26408ce7bdcdc4fc9e7a90cd9b6750659b2da37b4072229cec8f83ebbbd5eca691af32926404e967e1a00c4c628
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
|
+
## 6.2.3
|
|
2
|
+
- Feat: event_factory support [#94](https://github.com/logstash-plugins/logstash-codec-cef/pull/94)
|
|
3
|
+
|
|
4
|
+
## 6.2.2
|
|
5
|
+
- Fixed invalid Field Reference that could occur when ECS mode was enabled and the CEF field `fileHash` was parsed.
|
|
6
|
+
- Added expanded mapping for numbered `deviceCustom*` and `deviceCustom*Label` fields so that all now include numbers 1 through 15. [#89](https://github.com/logstash-plugins/logstash-codec-cef/pull/89).
|
|
7
|
+
|
|
8
|
+
## 6.2.1
|
|
9
|
+
- Added field mapping to docs.
|
|
10
|
+
- Fixed ECS mapping of `deviceMacAddress` field. [#88](https://github.com/logstash-plugins/logstash-codec-cef/pull/88).
|
|
11
|
+
|
|
12
|
+
## 6.2.0
|
|
13
|
+
- Introduce ECS Compatibility mode [#83](https://github.com/logstash-plugins/logstash-codec-cef/pull/83).
|
|
14
|
+
|
|
1
15
|
## 6.1.2
|
|
2
|
-
- Added error log with full payload when something bad happens in decoding a message[#84](https://github.com/logstash-plugins/logstash-codec-cef/pull/84)
|
|
16
|
+
- Added error log with full payload when something bad happens in decoding a message [#84](https://github.com/logstash-plugins/logstash-codec-cef/pull/84)
|
|
3
17
|
|
|
4
18
|
## 6.1.1
|
|
5
19
|
- Improved encoding performance, especially when encoding many extension fields [#81](https://github.com/logstash-plugins/logstash-codec-cef/pull/81)
|
data/docs/index.asciidoc
CHANGED
|
@@ -27,14 +27,385 @@ https://community.saas.hpe.com/dcvta86296/attachments/dcvta86296/connector-docum
|
|
|
27
27
|
If this codec receives a payload from an input that is not a valid CEF message, then it will
|
|
28
28
|
produce an event with the payload as the 'message' field and a '_cefparsefailure' tag.
|
|
29
29
|
|
|
30
|
+
==== Compatibility with the Elastic Common Schema (ECS)
|
|
31
|
+
|
|
32
|
+
This plugin can be used to decode CEF events _into_ the Elastic Common Schema, or to encode ECS-compatible events into CEF.
|
|
33
|
+
It can also be used _without_ ECS, encoding and decoding events using only CEF-defined field names and keys.
|
|
34
|
+
|
|
35
|
+
The ECS Compatibility mode for a specific plugin instance can be controlled by setting <<plugins-{type}s-{plugin}-ecs_compatibility>> when defining the codec:
|
|
36
|
+
|
|
37
|
+
[source,sh]
|
|
38
|
+
-----
|
|
39
|
+
input {
|
|
40
|
+
tcp {
|
|
41
|
+
# ...
|
|
42
|
+
codec => cef {
|
|
43
|
+
ecs_compatibility => v1
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
}
|
|
47
|
+
-----
|
|
48
|
+
|
|
49
|
+
If left unspecified, the value of the `pipeline.ecs_compatibility` setting is used.
|
|
50
|
+
|
|
51
|
+
===== Timestamps and ECS compatiblity
|
|
52
|
+
|
|
53
|
+
When decoding in ECS Compatibility Mode, timestamp-type fields are parsed and normalized
|
|
54
|
+
to specific points on the timeline.
|
|
55
|
+
|
|
56
|
+
Because the CEF format allows ambiguous timestamp formats, some reasonable assumptions are made:
|
|
57
|
+
|
|
58
|
+
- When the timestamp does not include a year, we assume it happened in the recent past
|
|
59
|
+
(or _very_ near future to accommodate out-of-sync clocks and timezone offsets).
|
|
60
|
+
- When the timestamp does not include UTC-offset information, we use the event's
|
|
61
|
+
timezone (`dtz` or `deviceTimeZone` field), or fall through to this plugin's
|
|
62
|
+
<<plugins-{type}s-{plugin}-default_timezone>>.
|
|
63
|
+
- Localized timestamps are parsed using the provided <<plugins-{type}s-{plugin}-locale>>.
|
|
64
|
+
|
|
65
|
+
[id="plugins-{type}s-{plugin}-field-mapping"]
|
|
66
|
+
===== Field mapping
|
|
67
|
+
|
|
68
|
+
The header fields from each CEF payload is expanded to the following fields, depending on whether ECS is enabled.
|
|
69
|
+
|
|
70
|
+
[id="plugins-{type}s-{plugin}-header-field"]
|
|
71
|
+
====== Header field mapping
|
|
72
|
+
|=====
|
|
73
|
+
|ECS Disabled | ECS Field
|
|
74
|
+
|
|
75
|
+
|`cefVersion` |`[cef][version]`
|
|
76
|
+
|`deviceVendor` |`[observer][vendor]`
|
|
77
|
+
|`deviceProduct` |`[observer][product]`
|
|
78
|
+
|`deviceVersion` |`[observer][version]`
|
|
79
|
+
|`deviceEventClassId`|`[event][code]`
|
|
80
|
+
|`name` |`[cef][name]`
|
|
81
|
+
|`severity` |`[event][severity]`
|
|
82
|
+
|=====
|
|
83
|
+
|
|
84
|
+
When decoding CEF payloads with `ecs_compatibility => disabled`, the abbreviated CEF Keys found in extensions are expanded, and CEF Field Names are inserted at the root level of the event.
|
|
85
|
+
|
|
86
|
+
When decoding in an ECS Compatibility mode, the ECS Fields are populated from the corresponding CEF Field Names _or_ CEF Keys found in the payload's extensions.
|
|
87
|
+
|
|
88
|
+
The following is a mapping between these fields.
|
|
89
|
+
|
|
90
|
+
// Templates for short-hand notes in the table below
|
|
91
|
+
:cef-ambiguous-higher: pass:quotes[Multiple possible CEF fields map to this ECS Field. When decoding, the last entry encountered wins. When encoding, this field has _higher_ priority.]
|
|
92
|
+
:cef-ambiguous-lower: pass:quotes[Multiple possible CEF fields map to this ECS Field. When decoding, the last entry encountered wins. When encoding, this field has _lower_ priority.]
|
|
93
|
+
:cef-normalize-timestamp: pass:quotes[This field contains a timestamp. In ECS Compatibility Mode, it is parsed to a specific point in time.]
|
|
94
|
+
:cef-plugin-config-condition: pass:quotes[When plugin configured with]
|
|
95
|
+
|
|
96
|
+
|
|
97
|
+
[id="plugins-{type}s-{plugin}-ext-field"]
|
|
98
|
+
====== Extension field mapping
|
|
99
|
+
|=======================================================================================================================
|
|
100
|
+
|CEF Field Name (optional CEF Key) |ECS Field
|
|
101
|
+
|
|
102
|
+
|`agentAddress` (`agt`) |`[agent][ip]`
|
|
103
|
+
|`agentDnsDomain` |`[cef][agent][registered_domain]`
|
|
104
|
+
|
|
105
|
+
{cef-ambiguous-higher}
|
|
106
|
+
|`agentHostName` (`ahost`) |`[agent][name]`
|
|
107
|
+
|`agentId` (`aid`) |`[agent][id]`
|
|
108
|
+
|`agentMacAddress` (`amac`) |`[agent][mac]`
|
|
109
|
+
|`agentNtDomain` |`[cef][agent][registered_domain]`
|
|
110
|
+
|
|
111
|
+
{cef-ambiguous-lower}
|
|
112
|
+
|`agentReceiptTime` (`art`) |`[event][created]`
|
|
113
|
+
|
|
114
|
+
{cef-normalize-timestamp}
|
|
115
|
+
|`agentTimeZone` (`atz`) |`[cef][agent][timezone]`
|
|
116
|
+
|`agentTranslatedAddress` |`[cef][agent][nat][ip]`
|
|
117
|
+
|`agentTranslatedZoneExternalID` |`[cef][agent][translated_zone][external_id]`
|
|
118
|
+
|`agentTranslatedZoneURI` |`[cef][agent][translated_zone][uri]`
|
|
119
|
+
|`agentType` (`at`) |`[agent][type]`
|
|
120
|
+
|`agentVersion` (`av`) |`[agent][version]`
|
|
121
|
+
|`agentZoneExternalID` |`[cef][agent][zone][external_id]`
|
|
122
|
+
|`agentZoneURI` |`[cef][agent][zone][uri]`
|
|
123
|
+
|`applicationProtocol` (`app`) |`[network][protocol]`
|
|
124
|
+
|`baseEventCount` (`cnt`) |`[cef][base_event_count]`
|
|
125
|
+
|`bytesIn` (`in`) |`[source][bytes]`
|
|
126
|
+
|`bytesOut` (`out`) |`[destination][bytes]`
|
|
127
|
+
|`categoryDeviceType` (`catdt`) |`[cef][device_type]`
|
|
128
|
+
|`customerExternalID` |`[organization][id]`
|
|
129
|
+
|`customerURI` |`[organization][name]`
|
|
130
|
+
|`destinationAddress` (`dst`) |`[destination][ip]`
|
|
131
|
+
|`destinationDnsDomain` |`[destination][registered_domain]`
|
|
132
|
+
|
|
133
|
+
{cef-ambiguous-higher}
|
|
134
|
+
|`destinationGeoLatitude` (`dlat`) |`[destination][geo][location][lat]`
|
|
135
|
+
|`destinationGeoLongitude` (`dlong`) |`[destination][geo][location][lon]`
|
|
136
|
+
|`destinationHostName` (`dhost`) |`[destination][domain]`
|
|
137
|
+
|`destinationMacAddress` (`dmac`) |`[destination][mac]`
|
|
138
|
+
|`destinationNtDomain` (`dntdom`) |`[destination][registered_domain]`
|
|
139
|
+
|
|
140
|
+
{cef-ambiguous-lower}
|
|
141
|
+
|`destinationPort` (`dpt`) |`[destination][port]`
|
|
142
|
+
|`destinationProcessId` (`dpid`) |`[destination][process][pid]`
|
|
143
|
+
|`destinationProcessName` (`dproc`) |`[destination][process][name]`
|
|
144
|
+
|`destinationServiceName` |`[destination][service][name]`
|
|
145
|
+
|`destinationTranslatedAddress` |`[destination][nat][ip]`
|
|
146
|
+
|`destinationTranslatedPort` |`[destination][nat][port]`
|
|
147
|
+
|`destinationTranslatedZoneExternalID` |`[cef][destination][translated_zone][external_id]`
|
|
148
|
+
|`destinationTranslatedZoneURI` |`[cef][destination][translated_zone][uri]`
|
|
149
|
+
|`destinationUserId` (`duid`) |`[destination][user][id]`
|
|
150
|
+
|`destinationUserName` (`duser`) |`[destination][user][name]`
|
|
151
|
+
|`destinationUserPrivileges` (`dpriv`) |`[destination][user][group][name]`
|
|
152
|
+
|`destinationZoneExternalID` |`[cef][destination][zone][external_id]`
|
|
153
|
+
|`destinationZoneURI` |`[cef][destination][zone][uri]`
|
|
154
|
+
|`deviceAction` (`act`) |`[event][action]`
|
|
155
|
+
.2+|`deviceAddress` (`dvc`) |`[observer][ip]`
|
|
156
|
+
|
|
157
|
+
{cef-plugin-config-condition} `device => observer`
|
|
158
|
+
|`[host][ip]`
|
|
159
|
+
|
|
160
|
+
{cef-plugin-config-condition} `device => host`
|
|
161
|
+
|`deviceCustomFloatingPoint1` (`cfp1`) |`[cef][device_custom_floating_point_1][value]`
|
|
162
|
+
|`deviceCustomFloatingPoint1Label` (`cfp1Label`)|`[cef][device_custom_floating_point_1][label]`
|
|
163
|
+
|`deviceCustomFloatingPoint2` (`cfp2`) |`[cef][device_custom_floating_point_2][value]`
|
|
164
|
+
|`deviceCustomFloatingPoint2Label` (`cfp2Label`)|`[cef][device_custom_floating_point_2][label]`
|
|
165
|
+
|`deviceCustomFloatingPoint3` (`cfp3`) |`[cef][device_custom_floating_point_3][value]`
|
|
166
|
+
|`deviceCustomFloatingPoint3Label` (`cfp3Label`)|`[cef][device_custom_floating_point_3][label]`
|
|
167
|
+
|`deviceCustomFloatingPoint4` (`cfp4`) |`[cef][device_custom_floating_point_4][value]`
|
|
168
|
+
|`deviceCustomFloatingPoint4Label` (`cfp4Label`)|`[cef][device_custom_floating_point_4][label]`
|
|
169
|
+
|`deviceCustomFloatingPoint5` (`cfp5`) |`[cef][device_custom_floating_point_5][value]`
|
|
170
|
+
|`deviceCustomFloatingPoint5Label` (`cfp5Label`)|`[cef][device_custom_floating_point_5][label]`
|
|
171
|
+
|`deviceCustomFloatingPoint6` (`cfp6`) |`[cef][device_custom_floating_point_6][value]`
|
|
172
|
+
|`deviceCustomFloatingPoint6Label` (`cfp6Label`)|`[cef][device_custom_floating_point_6][label]`
|
|
173
|
+
|`deviceCustomFloatingPoint7` (`cfp7`) |`[cef][device_custom_floating_point_7][value]`
|
|
174
|
+
|`deviceCustomFloatingPoint7Label` (`cfp7Label`)|`[cef][device_custom_floating_point_7][label]`
|
|
175
|
+
|`deviceCustomFloatingPoint8` (`cfp8`) |`[cef][device_custom_floating_point_8][value]`
|
|
176
|
+
|`deviceCustomFloatingPoint8Label` (`cfp8Label`)|`[cef][device_custom_floating_point_8][label]`
|
|
177
|
+
|`deviceCustomFloatingPoint9` (`cfp9`) |`[cef][device_custom_floating_point_9][value]`
|
|
178
|
+
|`deviceCustomFloatingPoint9Label` (`cfp9Label`)|`[cef][device_custom_floating_point_9][label]`
|
|
179
|
+
|`deviceCustomFloatingPoint10` (`cfp10`) |`[cef][device_custom_floating_point_10][value]`
|
|
180
|
+
|`deviceCustomFloatingPoint10Label` (`cfp10Label`)|`[cef][device_custom_floating_point_10][label]`
|
|
181
|
+
|`deviceCustomFloatingPoint11` (`cfp11`) |`[cef][device_custom_floating_point_11][value]`
|
|
182
|
+
|`deviceCustomFloatingPoint11Label` (`cfp11Label`)|`[cef][device_custom_floating_point_11][label]`
|
|
183
|
+
|`deviceCustomFloatingPoint12` (`cfp12`) |`[cef][device_custom_floating_point_12][value]`
|
|
184
|
+
|`deviceCustomFloatingPoint12Label` (`cfp12Label`)|`[cef][device_custom_floating_point_12][label]`
|
|
185
|
+
|`deviceCustomFloatingPoint13` (`cfp13`) |`[cef][device_custom_floating_point_13][value]`
|
|
186
|
+
|`deviceCustomFloatingPoint13Label` (`cfp13Label`)|`[cef][device_custom_floating_point_13][label]`
|
|
187
|
+
|`deviceCustomFloatingPoint14` (`cfp14`) |`[cef][device_custom_floating_point_14][value]`
|
|
188
|
+
|`deviceCustomFloatingPoint14Label` (`cfp14Label`)|`[cef][device_custom_floating_point_14][label]`
|
|
189
|
+
|`deviceCustomFloatingPoint15` (`cfp15`) |`[cef][device_custom_floating_point_15][value]`
|
|
190
|
+
|`deviceCustomFloatingPoint15Label` (`cfp15Label`)|`[cef][device_custom_floating_point_15][label]`
|
|
191
|
+
|`deviceCustomIPv6Address1` (`c6a1`) |`[cef][device_custom_ipv6_address_1][value]`
|
|
192
|
+
|`deviceCustomIPv6Address1Label` (`c6a1Label`) |`[cef][device_custom_ipv6_address_1][label]`
|
|
193
|
+
|`deviceCustomIPv6Address2` (`c6a2`) |`[cef][device_custom_ipv6_address_2][value]`
|
|
194
|
+
|`deviceCustomIPv6Address2Label` (`c6a2Label`) |`[cef][device_custom_ipv6_address_2][label]`
|
|
195
|
+
|`deviceCustomIPv6Address3` (`c6a3`) |`[cef][device_custom_ipv6_address_3][value]`
|
|
196
|
+
|`deviceCustomIPv6Address3Label` (`c6a3Label`) |`[cef][device_custom_ipv6_address_3][label]`
|
|
197
|
+
|`deviceCustomIPv6Address4` (`c6a4`) |`[cef][device_custom_ipv6_address_4][value]`
|
|
198
|
+
|`deviceCustomIPv6Address4Label` (`c6a4Label`) |`[cef][device_custom_ipv6_address_4][label]`
|
|
199
|
+
|`deviceCustomIPv6Address5` (`c6a5`) |`[cef][device_custom_ipv6_address_5][value]`
|
|
200
|
+
|`deviceCustomIPv6Address5Label` (`c6a5Label`) |`[cef][device_custom_ipv6_address_5][label]`
|
|
201
|
+
|`deviceCustomIPv6Address6` (`c6a6`) |`[cef][device_custom_ipv6_address_6][value]`
|
|
202
|
+
|`deviceCustomIPv6Address6Label` (`c6a6Label`) |`[cef][device_custom_ipv6_address_6][label]`
|
|
203
|
+
|`deviceCustomIPv6Address7` (`c6a7`) |`[cef][device_custom_ipv6_address_7][value]`
|
|
204
|
+
|`deviceCustomIPv6Address7Label` (`c6a7Label`) |`[cef][device_custom_ipv6_address_7][label]`
|
|
205
|
+
|`deviceCustomIPv6Address8` (`c6a8`) |`[cef][device_custom_ipv6_address_8][value]`
|
|
206
|
+
|`deviceCustomIPv6Address8Label` (`c6a8Label`) |`[cef][device_custom_ipv6_address_8][label]`
|
|
207
|
+
|`deviceCustomIPv6Address9` (`c6a9`) |`[cef][device_custom_ipv6_address_9][value]`
|
|
208
|
+
|`deviceCustomIPv6Address9Label` (`c6a9Label`) |`[cef][device_custom_ipv6_address_9][label]`
|
|
209
|
+
|`deviceCustomIPv6Address10` (`c6a10`) |`[cef][device_custom_ipv6_address_10][value]`
|
|
210
|
+
|`deviceCustomIPv6Address10Label` (`c6a10Label`)|`[cef][device_custom_ipv6_address_10][label]`
|
|
211
|
+
|`deviceCustomIPv6Address11` (`c6a11`) |`[cef][device_custom_ipv6_address_11][value]`
|
|
212
|
+
|`deviceCustomIPv6Address11Label` (`c6a11Label`)|`[cef][device_custom_ipv6_address_11][label]`
|
|
213
|
+
|`deviceCustomIPv6Address12` (`c6a12`) |`[cef][device_custom_ipv6_address_12][value]`
|
|
214
|
+
|`deviceCustomIPv6Address12Label` (`c6a12Label`)|`[cef][device_custom_ipv6_address_12][label]`
|
|
215
|
+
|`deviceCustomIPv6Address13` (`c6a13`) |`[cef][device_custom_ipv6_address_13][value]`
|
|
216
|
+
|`deviceCustomIPv6Address13Label` (`c6a13Label`)|`[cef][device_custom_ipv6_address_13][label]`
|
|
217
|
+
|`deviceCustomIPv6Address14` (`c6a14`) |`[cef][device_custom_ipv6_address_14][value]`
|
|
218
|
+
|`deviceCustomIPv6Address14Label` (`c6a14Label`)|`[cef][device_custom_ipv6_address_14][label]`
|
|
219
|
+
|`deviceCustomIPv6Address15` (`c6a15`) |`[cef][device_custom_ipv6_address_15][value]`
|
|
220
|
+
|`deviceCustomIPv6Address15Label` (`c6a15Label`)|`[cef][device_custom_ipv6_address_15][label]`
|
|
221
|
+
|`deviceCustomNumber1` (`cn1`) |`[cef][device_custom_number_1][value]`
|
|
222
|
+
|`deviceCustomNumber1Label` (`cn1Label`) |`[cef][device_custom_number_1][label]`
|
|
223
|
+
|`deviceCustomNumber2` (`cn2`) |`[cef][device_custom_number_2][value]`
|
|
224
|
+
|`deviceCustomNumber2Label` (`cn2Label`) |`[cef][device_custom_number_2][label]`
|
|
225
|
+
|`deviceCustomNumber3` (`cn3`) |`[cef][device_custom_number_3][value]`
|
|
226
|
+
|`deviceCustomNumber3Label` (`cn3Label`) |`[cef][device_custom_number_3][label]`
|
|
227
|
+
|`deviceCustomNumber4` (`cn4`) |`[cef][device_custom_number_4][value]`
|
|
228
|
+
|`deviceCustomNumber4Label` (`cn4Label`) |`[cef][device_custom_number_4][label]`
|
|
229
|
+
|`deviceCustomNumber5` (`cn5`) |`[cef][device_custom_number_5][value]`
|
|
230
|
+
|`deviceCustomNumber5Label` (`cn5Label`) |`[cef][device_custom_number_5][label]`
|
|
231
|
+
|`deviceCustomNumber6` (`cn6`) |`[cef][device_custom_number_6][value]`
|
|
232
|
+
|`deviceCustomNumber6Label` (`cn6Label`) |`[cef][device_custom_number_6][label]`
|
|
233
|
+
|`deviceCustomNumber7` (`cn7`) |`[cef][device_custom_number_7][value]`
|
|
234
|
+
|`deviceCustomNumber7Label` (`cn7Label`) |`[cef][device_custom_number_7][label]`
|
|
235
|
+
|`deviceCustomNumber8` (`cn8`) |`[cef][device_custom_number_8][value]`
|
|
236
|
+
|`deviceCustomNumber8Label` (`cn8Label`) |`[cef][device_custom_number_8][label]`
|
|
237
|
+
|`deviceCustomNumber9` (`cn9`) |`[cef][device_custom_number_9][value]`
|
|
238
|
+
|`deviceCustomNumber9Label` (`cn9Label`) |`[cef][device_custom_number_9][label]`
|
|
239
|
+
|`deviceCustomNumber10` (`cn10`) |`[cef][device_custom_number_10][value]`
|
|
240
|
+
|`deviceCustomNumber10Label` (`cn10Label`) |`[cef][device_custom_number_10][label]`
|
|
241
|
+
|`deviceCustomNumber11` (`cn11`) |`[cef][device_custom_number_11][value]`
|
|
242
|
+
|`deviceCustomNumber11Label` (`cn11Label`) |`[cef][device_custom_number_11][label]`
|
|
243
|
+
|`deviceCustomNumber12` (`cn12`) |`[cef][device_custom_number_12][value]`
|
|
244
|
+
|`deviceCustomNumber12Label` (`cn12Label`) |`[cef][device_custom_number_12][label]`
|
|
245
|
+
|`deviceCustomNumber13` (`cn13`) |`[cef][device_custom_number_13][value]`
|
|
246
|
+
|`deviceCustomNumber13Label` (`cn13Label`) |`[cef][device_custom_number_13][label]`
|
|
247
|
+
|`deviceCustomNumber14` (`cn14`) |`[cef][device_custom_number_14][value]`
|
|
248
|
+
|`deviceCustomNumber14Label` (`cn14Label`) |`[cef][device_custom_number_14][label]`
|
|
249
|
+
|`deviceCustomNumber15` (`cn15`) |`[cef][device_custom_number_15][value]`
|
|
250
|
+
|`deviceCustomNumber15Label` (`cn15Label`) |`[cef][device_custom_number_15][label]`
|
|
251
|
+
|`deviceCustomString1` (`cs1`) |`[cef][device_custom_string_1][value]`
|
|
252
|
+
|`deviceCustomString1Label` (`cs1Label`) |`[cef][device_custom_string_1][label]`
|
|
253
|
+
|`deviceCustomString2` (`cs2`) |`[cef][device_custom_string_2][value]`
|
|
254
|
+
|`deviceCustomString2Label` (`cs2Label`) |`[cef][device_custom_string_2][label]`
|
|
255
|
+
|`deviceCustomString3` (`cs3`) |`[cef][device_custom_string_3][value]`
|
|
256
|
+
|`deviceCustomString3Label` (`cs3Label`) |`[cef][device_custom_string_3][label]`
|
|
257
|
+
|`deviceCustomString4` (`cs4`) |`[cef][device_custom_string_4][value]`
|
|
258
|
+
|`deviceCustomString4Label` (`cs4Label`) |`[cef][device_custom_string_4][label]`
|
|
259
|
+
|`deviceCustomString5` (`cs5`) |`[cef][device_custom_string_5][value]`
|
|
260
|
+
|`deviceCustomString5Label` (`cs5Label`) |`[cef][device_custom_string_5][label]`
|
|
261
|
+
|`deviceCustomString6` (`cs6`) |`[cef][device_custom_string_6][value]`
|
|
262
|
+
|`deviceCustomString6Label` (`cs6Label`) |`[cef][device_custom_string_6][label]`
|
|
263
|
+
|`deviceCustomString7` (`cs7`) |`[cef][device_custom_string_7][value]`
|
|
264
|
+
|`deviceCustomString7Label` (`cs7Label`) |`[cef][device_custom_string_7][label]`
|
|
265
|
+
|`deviceCustomString8` (`cs8`) |`[cef][device_custom_string_8][value]`
|
|
266
|
+
|`deviceCustomString8Label` (`cs8Label`) |`[cef][device_custom_string_8][label]`
|
|
267
|
+
|`deviceCustomString9` (`cs9`) |`[cef][device_custom_string_9][value]`
|
|
268
|
+
|`deviceCustomString9Label` (`cs9Label`) |`[cef][device_custom_string_9][label]`
|
|
269
|
+
|`deviceCustomString10` (`cs10`) |`[cef][device_custom_string_10][value]`
|
|
270
|
+
|`deviceCustomString10Label` (`cs10Label`) |`[cef][device_custom_string_10][label]`
|
|
271
|
+
|`deviceCustomString11` (`cs11`) |`[cef][device_custom_string_11][value]`
|
|
272
|
+
|`deviceCustomString11Label` (`cs11Label`) |`[cef][device_custom_string_11][label]`
|
|
273
|
+
|`deviceCustomString12` (`cs12`) |`[cef][device_custom_string_12][value]`
|
|
274
|
+
|`deviceCustomString12Label` (`cs12Label`) |`[cef][device_custom_string_12][label]`
|
|
275
|
+
|`deviceCustomString13` (`cs13`) |`[cef][device_custom_string_13][value]`
|
|
276
|
+
|`deviceCustomString13Label` (`cs13Label`) |`[cef][device_custom_string_13][label]`
|
|
277
|
+
|`deviceCustomString14` (`cs14`) |`[cef][device_custom_string_14][value]`
|
|
278
|
+
|`deviceCustomString14Label` (`cs14Label`) |`[cef][device_custom_string_14][label]`
|
|
279
|
+
|`deviceCustomString15` (`cs15`) |`[cef][device_custom_string_15][value]`
|
|
280
|
+
|`deviceCustomString15Label` (`cs15Label`) |`[cef][device_custom_string_15][label]`
|
|
281
|
+
|`deviceDirection` |`[network][direction]`
|
|
282
|
+
.2+|`deviceDnsDomain` |`[observer][registered_domain]`
|
|
283
|
+
|
|
284
|
+
{cef-plugin-config-condition} `device => observer`.
|
|
285
|
+
|`[host][registered_domain]`
|
|
286
|
+
|
|
287
|
+
{cef-plugin-config-condition} `device => host`.
|
|
288
|
+
|`deviceEventCategory` (`cat`) |`[cef][category]`
|
|
289
|
+
.2+|`deviceExternalId` |`[observer][name]`
|
|
290
|
+
|
|
291
|
+
{cef-plugin-config-condition} `device => observer`.
|
|
292
|
+
|`[host][id]`
|
|
293
|
+
|
|
294
|
+
{cef-plugin-config-condition} `device => host`.
|
|
295
|
+
|`deviceFacility` |`[log][syslog][facility][code]`
|
|
296
|
+
.2+|`deviceHostName` (`dvchost`) |`[observer][hostname]`
|
|
297
|
+
|
|
298
|
+
{cef-plugin-config-condition} `device => observer`.
|
|
299
|
+
|`[host][name]`
|
|
300
|
+
|
|
301
|
+
{cef-plugin-config-condition} `device => host`.
|
|
302
|
+
|`deviceInboundInterface` |`[observer][ingress][interface][name]`
|
|
303
|
+
.2+|`deviceMacAddress` (`dvcmac`) |`[observer][mac]`
|
|
304
|
+
|
|
305
|
+
{cef-plugin-config-condition} `device => observer`.
|
|
306
|
+
|`[host][mac]`
|
|
307
|
+
|
|
308
|
+
{cef-plugin-config-condition} `device => host`.
|
|
309
|
+
|`deviceNtDomain` |`[cef][nt_domain]`
|
|
310
|
+
|`deviceOutboundInterface` |`[observer][egress][interface][name]`
|
|
311
|
+
|`devicePayloadId` |`[cef][payload_id]`
|
|
312
|
+
|`deviceProcessId` (`dvcpid`) |`[process][pid]`
|
|
313
|
+
|`deviceProcessName` |`[process][name]`
|
|
314
|
+
|`deviceReceiptTime` (`rt`) |`@timestamp`
|
|
315
|
+
|
|
316
|
+
{cef-normalize-timestamp}
|
|
317
|
+
|`deviceTimeZone` (`dtz`) |`[event][timezone]`
|
|
318
|
+
|`deviceTranslatedAddress` |`[host][nat][ip]`
|
|
319
|
+
|`deviceTranslatedZoneExternalID` |`[cef][translated_zone][external_id]`
|
|
320
|
+
|`deviceTranslatedZoneURI` |`[cef][translated_zone][uri]`
|
|
321
|
+
|`deviceVersion` |`[observer][version]`
|
|
322
|
+
|`deviceZoneExternalID` |`[cef][zone][external_id]`
|
|
323
|
+
|`deviceZoneURI` |`[cef][zone][uri]`
|
|
324
|
+
|`endTime` (`end`) |`[event][end]`
|
|
325
|
+
|
|
326
|
+
{cef-normalize-timestamp}
|
|
327
|
+
|`eventId` |`[event][id]`
|
|
328
|
+
|`eventOutcome` (`outcome`) |`[event][outcome]`
|
|
329
|
+
|`externalId` |`[cef][external_id]`
|
|
330
|
+
|`fileCreateTime` |`[file][created]`
|
|
331
|
+
|`fileHash` |`[file][hash]`
|
|
332
|
+
|`fileId` |`[file][inode]`
|
|
333
|
+
|`fileModificationTime` |`[file][mtime]`
|
|
334
|
+
|
|
335
|
+
{cef-normalize-timestamp}
|
|
336
|
+
|`fileName` (`fname`) |`[file][name]`
|
|
337
|
+
|`filePath` |`[file][path]`
|
|
338
|
+
|`filePermission` |`[file][group]`
|
|
339
|
+
|`fileSize` (`fsize`) |`[file][size]`
|
|
340
|
+
|`fileType` |`[file][extension]`
|
|
341
|
+
|`managerReceiptTime` (`mrt`) |`[event][ingested]`
|
|
342
|
+
|
|
343
|
+
{cef-normalize-timestamp}
|
|
344
|
+
|`message` (`msg`) |`[message]`
|
|
345
|
+
|`oldFileCreateTime` |`[cef][old_file][created]`
|
|
346
|
+
|
|
347
|
+
{cef-normalize-timestamp}
|
|
348
|
+
|`oldFileHash` |`[cef][old_file][hash]`
|
|
349
|
+
|`oldFileId` |`[cef][old_file][inode]`
|
|
350
|
+
|`oldFileModificationTime` |`[cef][old_file][mtime]`
|
|
351
|
+
|
|
352
|
+
{cef-normalize-timestamp}
|
|
353
|
+
|`oldFileName` |`[cef][old_file][name]`
|
|
354
|
+
|`oldFilePath` |`[cef][old_file][path]`
|
|
355
|
+
|`oldFilePermission` |`[cef][old_file][group]`
|
|
356
|
+
|`oldFileSize` |`[cef][old_file][size]`
|
|
357
|
+
|`oldFileType` |`[cef][old_file][extension]`
|
|
358
|
+
|`rawEvent` |`[event][original]`
|
|
359
|
+
|`Reason` (`reason`) |`[event][reason]`
|
|
360
|
+
|`requestClientApplication` |`[user_agent][original]`
|
|
361
|
+
|`requestContext` |`[http][request][referrer]`
|
|
362
|
+
|`requestCookies` |`[cef][request][cookies]`
|
|
363
|
+
|`requestMethod` |`[http][request][method]`
|
|
364
|
+
|`requestUrl` (`request`) |`[url][original]`
|
|
365
|
+
|`sourceAddress` (`src`) |`[source][ip]`
|
|
366
|
+
|`sourceDnsDomain` |`[source][registered_domain]`
|
|
367
|
+
|
|
368
|
+
{cef-ambiguous-higher}
|
|
369
|
+
|`sourceGeoLatitude` (`slat`) |`[source][geo][location][lat]`
|
|
370
|
+
|`sourceGeoLongitude` (`slong`) |`[source][geo][location][lon]`
|
|
371
|
+
|`sourceHostName` (`shost`) |`[source][domain]`
|
|
372
|
+
|`sourceMacAddress` (`smac`) |`[source][mac]`
|
|
373
|
+
|`sourceNtDomain` (`sntdom`) |`[source][registered_domain]`
|
|
374
|
+
|
|
375
|
+
{cef-ambiguous-lower}
|
|
376
|
+
|`sourcePort` (`spt`) |`[source][port]`
|
|
377
|
+
|`sourceProcessId` (`spid`) |`[source][process][pid]`
|
|
378
|
+
|`sourceProcessName` (`sproc`) |`[source][process][name]`
|
|
379
|
+
|`sourceServiceName` |`[source][service][name]`
|
|
380
|
+
|`sourceTranslatedAddress` |`[source][nat][ip]`
|
|
381
|
+
|`sourceTranslatedPort` |`[source][nat][port]`
|
|
382
|
+
|`sourceTranslatedZoneExternalID` |`[cef][source][translated_zone][external_id]`
|
|
383
|
+
|`sourceTranslatedZoneURI` |`[cef][source][translated_zone][uri]`
|
|
384
|
+
|`sourceUserId` (`suid`) |`[source][user][id]`
|
|
385
|
+
|`sourceUserName` (`suser`) |`[source][user][name]`
|
|
386
|
+
|`sourceUserPrivileges` (`spriv`) |`[source][user][group][name]`
|
|
387
|
+
|`sourceZoneExternalID` |`[cef][source][zone][external_id]`
|
|
388
|
+
|`sourceZoneURI` |`[cef][source][zone][uri]`
|
|
389
|
+
|`startTime` (`start`) |`[event][start]`
|
|
390
|
+
|
|
391
|
+
{cef-normalize-timestamp}
|
|
392
|
+
|`transportProtocol` (`proto`) |`[network][transport]`
|
|
393
|
+
|`type` |`[cef][type]`
|
|
394
|
+
|=======================================================================================================================
|
|
395
|
+
|
|
396
|
+
|
|
30
397
|
[id="plugins-{type}s-{plugin}-options"]
|
|
31
398
|
==== Cef Codec Configuration Options
|
|
32
399
|
|
|
33
400
|
[cols="<,<,<",options="header",]
|
|
34
401
|
|=======================================================================
|
|
35
402
|
|Setting |Input type|Required
|
|
403
|
+
| <<plugins-{type}s-{plugin}-default_timezone>> |<<string,string>>|No
|
|
36
404
|
| <<plugins-{type}s-{plugin}-delimiter>> |<<string,string>>|No
|
|
405
|
+
| <<plugins-{type}s-{plugin}-device>> |<<string,string>>|No
|
|
406
|
+
| <<plugins-{type}s-{plugin}-ecs_compatibility>> |<<string,string>>|No
|
|
37
407
|
| <<plugins-{type}s-{plugin}-fields>> |<<array,array>>|No
|
|
408
|
+
| <<plugins-{type}s-{plugin}-locale>> |<<string,string>>|No
|
|
38
409
|
| <<plugins-{type}s-{plugin}-name>> |<<string,string>>|No
|
|
39
410
|
| <<plugins-{type}s-{plugin}-product>> |<<string,string>>|No
|
|
40
411
|
| <<plugins-{type}s-{plugin}-reverse_mapping>> |<<boolean,boolean>>|No
|
|
@@ -46,6 +417,21 @@ produce an event with the payload as the 'message' field and a '_cefparsefailure
|
|
|
46
417
|
|
|
47
418
|
|
|
48
419
|
|
|
420
|
+
[id="plugins-{type}s-{plugin}-default_timezone"]
|
|
421
|
+
===== `default_timezone`
|
|
422
|
+
|
|
423
|
+
* Value type is <<string,string>>
|
|
424
|
+
* Supported values are:
|
|
425
|
+
** https://en.wikipedia.org/wiki/List_of_tz_database_time_zones[Timezone names] (such as `Europe/Moscow`, `America/Argentina/Buenos_Aires`)
|
|
426
|
+
** UTC Offsets (such as `-08:00`, `+03:00`)
|
|
427
|
+
* The default value is your system time zone
|
|
428
|
+
* This option has no effect when _encoding_.
|
|
429
|
+
|
|
430
|
+
When parsing timestamp fields in ECS mode and encountering timestamps that
|
|
431
|
+
do not contain UTC-offset information, the `deviceTimeZone` (`dtz`) field
|
|
432
|
+
from the CEF payload is used to interpret the given time. If the event does
|
|
433
|
+
not include timezone information, this `default_timezone` is used instead.
|
|
434
|
+
|
|
49
435
|
[id="plugins-{type}s-{plugin}-delimiter"]
|
|
50
436
|
===== `delimiter`
|
|
51
437
|
|
|
@@ -69,21 +455,69 @@ This setting allows the following character sequences to have special meaning:
|
|
|
69
455
|
* `\\r` (backslash "r") - means carriage return (ASCII 0x0D)
|
|
70
456
|
* `\\n` (backslash "n") - means newline (ASCII 0x0A)
|
|
71
457
|
|
|
458
|
+
[id="plugins-{type}s-{plugin}-device"]
|
|
459
|
+
===== `device`
|
|
460
|
+
|
|
461
|
+
* Value type is <<string,string>>
|
|
462
|
+
* Supported values are:
|
|
463
|
+
** `observer`: indicates that device-specific fields represent the device used to _observe_ the event.
|
|
464
|
+
** `host`: indicates that device-specific fields represent the device on which the event _occurred_.
|
|
465
|
+
* The default value for this setting is `observer`.
|
|
466
|
+
* Option has no effect when <<plugins-{type}s-{plugin}-ecs_compatibility,`ecs_compatibility => disabled`>>.
|
|
467
|
+
* Option has no effect when _encoding_
|
|
468
|
+
|
|
469
|
+
Defines a set of device-specific CEF fields as either representing the device on which an
|
|
470
|
+
event _occurred_, or merely the device from which the event was _observed_.
|
|
471
|
+
This causes the relevant fields to be routed to either the `host` or the `observer`
|
|
472
|
+
top-level groupings.
|
|
473
|
+
|
|
474
|
+
If the codec handles data from a variety of sources, the ECS recommendation is to use `observer`.
|
|
475
|
+
|
|
476
|
+
[id="plugins-{type}s-{plugin}-ecs_compatibility"]
|
|
477
|
+
===== `ecs_compatibility`
|
|
478
|
+
|
|
479
|
+
* Value type is <<string,string>>
|
|
480
|
+
* Supported values are:
|
|
481
|
+
** `disabled`: uses CEF-defined field names in the event (e.g., `bytesIn`, `sourceAddress`)
|
|
482
|
+
** `v1`: supports ECS-compatible event fields (e.g., `[source][bytes]`, `[source][ip]`)
|
|
483
|
+
* Default value depends on which version of Logstash is running:
|
|
484
|
+
** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
|
|
485
|
+
** Otherwise, the default value is `disabled`.
|
|
486
|
+
|
|
487
|
+
Controls this plugin's compatibility with the {ecs-ref}[Elastic Common Schema (ECS)].
|
|
488
|
+
|
|
72
489
|
[id="plugins-{type}s-{plugin}-fields"]
|
|
73
490
|
===== `fields`
|
|
74
491
|
|
|
75
492
|
* Value type is <<array,array>>
|
|
76
493
|
* Default value is `[]`
|
|
494
|
+
* Option has no effect when _decoding_
|
|
495
|
+
|
|
496
|
+
When this codec is used in an Output Plugin, a list of fields can be provided to be included in CEF extensions part as key/value pairs.
|
|
497
|
+
|
|
498
|
+
[id="plugins-{type}s-{plugin}-locale"]
|
|
499
|
+
===== `locale`
|
|
77
500
|
|
|
78
|
-
|
|
501
|
+
* Value type is <<string,string>>
|
|
502
|
+
* Supported values are:
|
|
503
|
+
** Abbreviated language_COUNTRY format (e.g., `en_GB`, `pt_BR`)
|
|
504
|
+
** Valid https://tools.ietf.org/html/bcp47[IETF BCP 47] language tag (e.g., `zh-cmn-Hans-CN`)
|
|
505
|
+
* The default value is your system locale
|
|
506
|
+
* Option has no effect when _encoding_
|
|
507
|
+
|
|
508
|
+
When parsing timestamp fields in ECS mode and encountering timestamps in
|
|
509
|
+
a localized format, this `locale` is used to interpret locale-specific strings
|
|
510
|
+
such as month abbreviations.
|
|
79
511
|
|
|
80
512
|
[id="plugins-{type}s-{plugin}-name"]
|
|
81
513
|
===== `name`
|
|
82
514
|
|
|
83
515
|
* Value type is <<string,string>>
|
|
84
516
|
* Default value is `"Logstash"`
|
|
517
|
+
* Option has no effect when _decoding_
|
|
85
518
|
|
|
86
|
-
|
|
519
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
520
|
+
value of the name field in the CEF header. The new value can include `%{foo}` strings
|
|
87
521
|
to help you build a new value from other parts of the event.
|
|
88
522
|
|
|
89
523
|
[id="plugins-{type}s-{plugin}-product"]
|
|
@@ -91,8 +525,10 @@ to help you build a new value from other parts of the event.
|
|
|
91
525
|
|
|
92
526
|
* Value type is <<string,string>>
|
|
93
527
|
* Default value is `"Logstash"`
|
|
528
|
+
* Option has no effect when _decoding_
|
|
94
529
|
|
|
95
|
-
|
|
530
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
531
|
+
value of the device product field in CEF header. The new value can include `%{foo}` strings
|
|
96
532
|
to help you build a new value from other parts of the event.
|
|
97
533
|
|
|
98
534
|
|
|
@@ -101,6 +537,7 @@ to help you build a new value from other parts of the event.
|
|
|
101
537
|
|
|
102
538
|
* Value type is <<boolean,boolean>>
|
|
103
539
|
* Default value is `false`
|
|
540
|
+
* Option has no effect when _decoding_
|
|
104
541
|
|
|
105
542
|
Set to true to adhere to the specifications and encode using the CEF key name (short name) for the CEF field names.
|
|
106
543
|
|
|
@@ -109,8 +546,10 @@ Set to true to adhere to the specifications and encode using the CEF key name (s
|
|
|
109
546
|
|
|
110
547
|
* Value type is <<string,string>>
|
|
111
548
|
* Default value is `"6"`
|
|
549
|
+
* Option has no effect when _decoding_
|
|
112
550
|
|
|
113
|
-
|
|
551
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
552
|
+
value of the severity field in CEF header. The new value can include `%{foo}` strings
|
|
114
553
|
to help you build a new value from other parts of the event.
|
|
115
554
|
|
|
116
555
|
Defined as field of type string to allow sprintf. The value will be validated
|
|
@@ -122,8 +561,10 @@ All invalid values will be mapped to the default of 6.
|
|
|
122
561
|
|
|
123
562
|
* Value type is <<string,string>>
|
|
124
563
|
* Default value is `"Logstash"`
|
|
564
|
+
* Option has no effect when _decoding_
|
|
125
565
|
|
|
126
|
-
|
|
566
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
567
|
+
value of the signature ID field in CEF header. The new value can include `%{foo}` strings
|
|
127
568
|
to help you build a new value from other parts of the event.
|
|
128
569
|
|
|
129
570
|
[id="plugins-{type}s-{plugin}-vendor"]
|
|
@@ -131,8 +572,10 @@ to help you build a new value from other parts of the event.
|
|
|
131
572
|
|
|
132
573
|
* Value type is <<string,string>>
|
|
133
574
|
* Default value is `"Elasticsearch"`
|
|
575
|
+
* Option has no effect when _decoding_
|
|
134
576
|
|
|
135
|
-
|
|
577
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
578
|
+
value of the device vendor field in CEF header. The new value can include `%{foo}` strings
|
|
136
579
|
to help you build a new value from other parts of the event.
|
|
137
580
|
|
|
138
581
|
[id="plugins-{type}s-{plugin}-version"]
|
|
@@ -140,8 +583,8 @@ to help you build a new value from other parts of the event.
|
|
|
140
583
|
|
|
141
584
|
* Value type is <<string,string>>
|
|
142
585
|
* Default value is `"1.0"`
|
|
586
|
+
* Option has no effect when _decoding_
|
|
143
587
|
|
|
144
|
-
|
|
588
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
589
|
+
value of the device version field in CEF header. The new value can include `%{foo}` strings
|
|
145
590
|
to help you build a new value from other parts of the event.
|
|
146
|
-
|
|
147
|
-
|